{"id":22966,"date":"2026-08-01T06:22:16","date_gmt":"2026-08-01T06:22:16","guid":{"rendered":"https:\/\/scannn.com\/the-economic-benefit-of-refactoring\/"},"modified":"2026-08-01T06:22:16","modified_gmt":"2026-08-01T06:22:16","slug":"the-economic-benefit-of-refactoring","status":"publish","type":"post","link":"https:\/\/scannn.com\/lv\/the-economic-benefit-of-refactoring\/","title":{"rendered":"The Economic Benefit of Refactoring"},"content":{"rendered":"\n
\n

As part of getting to grips with the new world of agentic engineering,
\nI built an application to support my work. It\u2019s a sophisticated app:
\nhigh-quality web UI with dynamic refresh and look-up, modals and
\nauto-save, integrations to external systems, machine learning and text
\nanalysis, background jobs, and a proper environment setup with fully
\nautomated deployment. It\u2019s approximately 150,000 lines of code,
\nprimarily in Rust (~120 kLoC) with the remainder in TypeScript and
\nTerraform.<\/p>\n

This was entirely written by agents. Mostly Claude Code, and some use
\nof Cursor. I didn\u2019t read or review any of the code, except
\noccasionally, out of interest.<\/p>\n

While building the application, I could see some things going
\nawry. After watching an edit to line 4,000 of a file scroll by in the
\nterminal, I had a closer look. The data access layer had grown to over
\n6,000 lines. As more features landed, this continued to
\ngrow. Every query, read or write, repeated the same HTTP request
\nsetup, the same JSON encoding and decoding. Eventually, it reached 17,155
\nlines. In a single Rust file.<\/p>\n

An experiment in refactoring<\/h2>\n

The 17,155 line file was the entire data access layer. A single,
\nself-contained module. Reviewing the code, there was no
\nde-duplication, no internal language, limited extraction of functions,
\nand very little extraction of classes. It did have a clear boundary
\nwith an interface to preserve. It was a great target for refactoring.<\/p>\n

The goal of refactoring an agentic code base is to spend tokens now in
\nrefactoring to make token consumption for future work lower. An
\nexperiment should be able to show that as this file was refactored the
\ntoken cost of making separate feature implementations in this code
\nbase would decrease.<\/p>\n

Precisely because agents never learn this was now possible to run as
\nan experiment. I could prompt a fresh agent to make exactly the same
\nchange after every refactoring stage. Unlike a human engineer, the
\nexperiment would not be tainted by learning from previous steps.<\/p>\n

    \n
  1. Create an overall refactoring plan, following strict refactoring
    \ndiscipline.<\/li>\n
  2. Craft a representative change, described in a single prompt.<\/li>\n
  3. Establish a baseline cost of change: in a sub-agent, execute that
    \nprompt, including asking the sub-agent to report token consumption.<\/li>\n
  4. Throw away the change.<\/li>\n
  5. In a loop:\n
      \n
    1. Apply a single step of the overall refactoring.<\/li>\n
    2. In a sub-agent, execute exactly<\/em> the same change receiving the
      \ntoken cost of the change.<\/li>\n
    3. Throw away the change.<\/li>\n<\/ol>\n<\/li>\n
    4. Record all token costs, time to execute the change, and lines of
      \ncode after each step of the refactoring, including the baseline.<\/li>\n<\/ol>\n

      The prompt used for the representative change and the refactoring
      \nsteps applied are shown in the appendices, below.<\/p>\n

      One caveat: Claude doesn\u2019t provide reliable methods for counting
      \ntokens live despite showing token counts, reporting tokens consumed
      \nper session, and billing<\/strong> for tokens. I\u2019m assuming this is a
      \ntemporary issue that will improve over time. Instead, the sub-agent
      \nreported the number of characters received and sent and used
      \ntiktoken<\/a> to approximate tokens, by dividing character count by
      \nfour.<\/p>\n

      Results<\/h2>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
      Step<\/th>\nData Access Layer LoC<\/th>\nLargest file LoC<\/th>\nTotal Rust LoC<\/th>\nInput tokens per change<\/th>\nOutput tokens per change<\/th>\nTime per change (s)<\/th>\n<\/tr>\n<\/thead>\n
      Baseline<\/td>\n17,155<\/td>\n17,155<\/td>\n50,359<\/td>\n159,564<\/td>\n1,705<\/td>\n342<\/td>\n<\/tr>\n
      Step 1 (FirestoreClient)<\/td>\n16,706<\/td>\n16,706<\/td>\n49,910<\/td>\n155,205<\/td>\n1,723<\/td>\n530<\/td>\n<\/tr>\n
      Step 2 (extract_doc_id, new_link)<\/td>\n16,562<\/td>\n16,562<\/td>\n49,766<\/td>\n159,227<\/td>\n2,105<\/td>\n574<\/td>\n<\/tr>\n
      Step 3 (link-query helpers)<\/td>\n16,567<\/td>\n16,567<\/td>\n49,771<\/td>\n154,054<\/td>\n2,105<\/td>\n524<\/td>\n<\/tr>\n
      Step 4 (FakeStore predicates)<\/td>\n16,577<\/td>\n16,577<\/td>\n49,781<\/td>\n154,146<\/td>\n2,060<\/td>\n654<\/td>\n<\/tr>\n
      Step 5 (value ctors)<\/td>\n16,469<\/td>\n16,469<\/td>\n49,673<\/td>\n171,251<\/td>\n2,036<\/td>\n1,353<\/td>\n<\/tr>\n
      Step 6 (FieldsBuilder)<\/td>\n16,469<\/td>\n16,469<\/td>\n49,673<\/td>\n171,251<\/td>\n2,036<\/td>\n1,353<\/td>\n<\/tr>\n
      Step 7 (queries.rs)<\/td>\n16,474<\/td>\n15,670<\/td>\n49,678<\/td>\n151,850<\/td>\n1,800<\/td>\n587<\/td>\n<\/tr>\n
      Step 8 (traits.rs)<\/td>\n16,508<\/td>\n13,845<\/td>\n49,712<\/td>\n132,558<\/td>\n1,723<\/td>\n446<\/td>\n<\/tr>\n
      Step 9 (traits\/ split)<\/td>\n16,508<\/td>\n13,845<\/td>\n49,712<\/td>\n132,558<\/td>\n1,723<\/td>\n446<\/td>\n<\/tr>\n
      Step 10 (codec.rs)<\/td>\n16,521<\/td>\n12,846<\/td>\n49,725<\/td>\n131,871<\/td>\n1,750<\/td>\n540<\/td>\n<\/tr>\n
      Step 11 (fake_store.rs)<\/td>\n16,535<\/td>\n11,122<\/td>\n49,739<\/td>\n133,016<\/td>\n2,460<\/td>\n600<\/td>\n<\/tr>\n
      Step 12 (store\/ split)<\/td>\n16,550<\/td>\n9,269<\/td>\n49,754<\/td>\n104,080<\/td>\n2,050<\/td>\n490<\/td>\n<\/tr>\n
      Step 13 (co-locate tests)<\/td>\n16,550<\/td>\n9,269<\/td>\n49,754<\/td>\n104,080<\/td>\n2,050<\/td>\n490<\/td>\n<\/tr>\n
      Step 14 (complete fake_store.rs)<\/td>\n16,553<\/td>\n7,225<\/td>\n49,757<\/td>\n107,205<\/td>\n2,453<\/td>\n523<\/td>\n<\/tr>\n
      Step 15 (store\/ split)<\/td>\n16,608<\/td>\n3,695<\/td>\n49,812<\/td>\n27,360<\/td>\n2,113<\/td>\n454<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

      The interesting metrics here are the total lines of code in the data
      \naccess layer, the total lines of code in the largest single file<\/em> in
      \nthe data access layer and the input tokens consumed while producing
      \nthe change.<\/p>\n

      <\/p>\n

      This chart shows four things. The first point is the baseline, step 0,
      \nand then the same metrics are repeated after<\/em> each refactoring step
      \nhas been applied.<\/p>\n

        \n
      1. The total lines of code in the data access layer as a
        \nwhole. Initially, this is just the single file I started with. This
        \nbecomes many files as refactorings are applied. By the end there
        \nare 19 Rust files.<\/li>\n
      2. The lines of code in the single largest file in the data access
        \nlayer. This started as the entirety of the data layer in the single
        \ninitial file. By the end, the single largest file is a test
        \nlibrary. Further refactoring passes could apply the same approach
        \nto this.<\/li>\n
      3. The total input tokens consumed by the sub-agent while applying the
        \nrepresentative change.<\/li>\n
      4. The total output tokens produced by the sub-agent while applying
        \nthe representative change.<\/li>\n<\/ol>\n

        Refactoring reduces token consumption<\/h2>\n

        The results are clear. Input tokens stay fairly flat until the largest
        \nfile starts to fall, and then they drop before, in the words of
        \nClaude, falling off a cliff.<\/p>\n

        Between the base line and the final refactoring, input tokens for the
        \nsame task reduced from 159,564<\/strong> to 27,360<\/strong>. A saving of
        \n132,204<\/strong> tokens, or 83%<\/strong>. And that saving is not a one-off. Every single
        \nchange that touches the data access layer from this point forward now
        \ncosts significantly less.<\/p>\n

        How much of a saving? Assuming Sonnet 5 pricing at the time of writing
        \nof $3\/MTok, 39.7 cents. Not a lot. Does it multiply? How will this
        \nplay out across debugging? More complicated features? This is
        \nrefactoring only one portion of the code base, can the whole code base
        \nbe aggressively refactored to find savings everywhere? How much would
        \nthose refactorings cost?<\/em><\/p>\n

        This saving is because the agent has to read less code. But it is not
        \nbecause there is less code to read. The overall code in the data
        \naccess layer as a whole has stayed fairly constant. Therefore to be
        \nable to bank this saving, the agent must be able to successfully
        \nidentify the smallest subset of files necessary to read. The results
        \nmake it appear this was happening. Reading the Claude Code thinking
        \noutput and file read summaries as the change was being applied also
        \nindicates the sub-agent was successfully reading smaller and smaller
        \nsections of code each time.<\/p>\n

        In other words, randomly cutting the file into smaller files is
        \nunlikely to help as much: even if each file were smaller, the agent
        \nwould be forced to read through many files looking for the relevant
        \ncode. While the step with the biggest effect happens at the end, the
        \nprevious steps were refactorings to set up this saving. This was not
        \nplanned. It was simply a result of how refactoring typically proceeds:
        \nlocal file changes to extract duplication, before breaking down into
        \nsmaller files once a repeating core emerges.<\/p>\n

        The refactoring did not make the representative change smaller. The
        \nnumber of tokens produced when writing code was largely unaffected:
        \nthe output tokens do not move very much. Those tokens are five times
        \nthe price of the input tokens. But, there are a lot less of them. Are
        \nthere refactorings that could be applied to reduce output token
        \nproduction? I need a more complex sample change to explore these
        \nquestions. The noise of the non-deterministic code generation process
        \nis hiding any variance caused by changes in the factoring of the code.<\/p>\n

        Notes on the process<\/h2>\n

        Claude was not good at refactoring. If you read the prompt and the
        \nrefactoring steps below, it\u2019s clear that the refactorings produced
        \nwere directly in response to the prompt. Claude is unable to look at
        \ncode, look at refactorings in general and work out which are suitable
        \nto apply: a human needs to actively guide it. This marries with wider
        \nexperience in this app. The development harness includes an explicit
        \nrefactoring step. That refactoring step did not prompt Claude into
        \nimproving this file. More anecdotally, Claude.ai was better than
        \nClaude Code. I used both interfaces to create the refactoring
        \nplan. Claude Code spotted extract function as the first
        \nstep. Claude.ai went further and saw an entire client class to be
        \nextracted.<\/p>\n

        It was also bad at applying them. The mechanical act of refactoring
        \nwas performed by writing Python scripts using grep and sed. These
        \nscripts frequently got confused by indentation. Oh, the irony. In
        \naddition, the single most valuable refactoring was missed in the first
        \npass, and had to be re-applied as a follow-up step. This is why the
        \nnumber of steps in the figure don\u2019t match the refactoring steps in the
        \nappendix.<\/p>\n

        It took about eight hours to complete the entire experiment. This was
        \nmostly unattended. The only intervention was after six hours 40
        \nminutes when it appeared to have finished, but had skipped that step
        \nand needed to be redirected. This experiment was running on slow
        \nhotel WiFi. I wondered if that contributed to time taken. But on
        \ndeeper analysis of the code base, the cargo temporary build cache had
        \nbecome very large. Test execution was suffering, significantly.<\/p>\n

        Further work and broader implications<\/h2>\n

        Unfortunately, it didn\u2019t occur to me to perform a count of the tokens
        \nrequired to create and execute the refactoring plan until it was
        \nalready complete. I\u2019ve looked at my aggregate consumption across the
        \ntime window where I was doing this work, including designing and
        \nrunning the experiment. I can\u2019t say how many tokens were required to
        \nperform the refactoring. The upper bound is five million,
        \nhowever. This includes creating the refactoring plan twice, the work
        \nto design the experiment including the representative change, and
        \nvarious other tasks. Future work should include a more accurate count
        \nof tokens consumed to refactor.<\/p>\n

        This is just one experiment, on a significant application that is
        \nstill greenfield and built and maintained by a single developer. But,
        \nI believe this is a potentially interesting first step. This effort
        \nshows the value, in time and money of refactoring. As well as
        \nmeasuring how expensive refactoring is. It would be interesting to
        \nlook at more complex changes, at wider refactoring, refactoring
        \ncontinuously, and even the relative value of different refactoring
        \napproaches.<\/p>\n

        This is just the beginning.<\/p>\n

        Appendices<\/h2>\n

        Note: These appendices include the prompts that I used, and the
        \noutput that was returned. The only editing applied has been to remove
        \nthe specific code changes to be made. These are included without
        \nediting to show how the agents were directed. There are no hidden
        \ntricks. As such, there is some language in here that might be
        \nconfusing. The error is in the original.<\/em><\/p>\n

        \nThe representative change<\/summary>\n

        This is the recorded prompt that was fed to each sub-agent, there was
        \nno further context supplied other than the code base and accompanying
        \narchitecture documentation. Every sub-agent was starting with exactly
        \nthe same information.<\/p>\n

        \n

        You are working in the Rust project at ~\/dev\/your-project-name<\/code>.<\/p>\n

        Add a new ItemWatchStore<\/code> public async trait to the Firestore layer, following existing patterns exactly. The trait must have three methods:<\/p>\n

          \n
        • async fn watch_item(&self, item_id: &str, user_id: &str) -> Result<()><\/code><\/li>\n
        • async fn unwatch_item(&self, item_id: &str, user_id: &str) -> Result<()><\/code><\/li>\n
        • async fn watched_items_for_user(&self, user_id: &str) -> Result<Vec<String>><\/code><\/li>\n<\/ul>\n

          Watches are stored in a item_watches<\/code> Firestore collection. Each document has fields: itemId<\/code> (string), userId<\/code> (string), createdAt<\/code> (timestamp). There is no Rust struct for a watch record \u2014 the methods return Vec<String><\/code> (item ids).<\/p>\n

          Implement the trait for both FakeStore<\/code> (using an in-memory Vec<(String, String)><\/code> field added to FakeStoreInner<\/code>) and FirestoreStore<\/code> (using the same HTTP patterns used for other store impls in this file).<\/p>\n

          At the very end of your response<\/strong>, output exactly this JSON block (fill in real values):<\/p>\n

          {\n  \"files_read\": [\n    {\"path\": \"src\/firestore.rs\", \"chars\": 123456},\n    ...\n  ],\n  \"response_chars\": 7890\n}\n<\/code><\/pre>\n
          Do NOT commit the change. Stop after writing the code.<\/p>\n<\/blockquote>\n<\/details>\n
          \nRefactoring steps<\/summary>\n
          This is the prompt that was used to create the refactoring plan.<\/p>\n
          \n
          Following the strict definition that a refactoring is a provably
          \ncorrectness preserving series of code edits, and using Martin
          \nFowler\u2019s 2nd edition of Refactoring as the source, examine
          \n@src\/firestore.rs. This is a 17K LoC Rust file. No file should be
          \nthat long. It is almost certainly not using an internal language to
          \nbuild and manage queries. Produce and describe, but don\u2019t execute, a
          \nsequence of refactorings that would massively reduce the line count
          \nof that file, without changing the interface at all.<\/p>\n<\/blockquote>\n
          Following is the description of the refactorings applied, extracted
          \nfrom the plan built and followed by Claude. The actual plan includes
          \npredicted code changes. For each refactoring, the individual steps to
          \nfollow were listed. Each of those steps was individually testable, and
          \nwas individually tested. This is a stricter refactoring than most
          \nhuman engineers would follow.<\/p>\n
          The steps listed here don\u2019t line up directly with the measured changes
          \nabove as Claude skipped the most valuable single refactoring
          \n(splitting out the store into sub-files) on the first pass and had to
          \ncomplete that afterwards as two additional steps.<\/p>\n
          Fowler ref:<\/strong> Extract Class<\/em> (7.5); Extract Function<\/em> (6.1) for
          \neach primitive<\/p>\n
          FirestoreStore<\/code> currently conflates two responsibilities:<\/p>\n
            \n
          • Domain query orchestration<\/strong> \u2014 which query to run, which documents
            \nto write, how to parse results into domain types<\/li>\n
          • Firestore HTTP transport<\/strong> \u2014 auth headers, URL construction, JSON
            \nencoding\/decoding of Firestore wire types,
            \nretry-on-PRECONDITION_FAILED<\/li>\n<\/ul>\n
            Fowler \u00a77.5 calls for extracting a new class when you can identify a
            \ncoherent subset of a class\u2019s data and behaviour. The transport
            \nresponsibility owns: client: reqwest::Client<\/code>, project_id: String<\/code>,
            \nMetadataAuth<\/code>, and documents_url()<\/code> \/ auth_header()<\/code>. Extract these
            \ninto a new FirestoreClient<\/code> struct.<\/p>\n
            Estimated savings: ~1,200 lines in FirestoreStore<\/code> impls;
            \nFirestoreClient<\/code> adds ~120 lines net.<\/strong><\/p>\n
            Step 2 \u2014 Extract Function: extract_doc_id<\/code> and new_link<\/code> (Fowler \u00a76.1)<\/h4>\n
            Fowler ref:<\/strong> Extract Function<\/em> (6.1)<\/p>\n
              \n
            • \n
              extract_doc_id<\/code><\/strong> \u2014 The expression
              \ndoc.name.rsplit('\/').next()?.to_string()<\/code> appears verbatim at the
              \nstart of all 20 parse_*_document<\/code> functions. Extract it.<\/p>\n<\/li>\n
            • \n
              new_link<\/code><\/strong> \u2014 Building a Link<\/code> struct with metadata:
              \nHashMap::new()<\/code> and provenance: None<\/code> and a fresh UUID appears 62
              \ntimes. Extract a factory function.<\/p>\n<\/li>\n<\/ul>\n
              Estimated savings: ~500 lines<\/strong> (62 \u00d7 ~10-line structs \u2192 62 \u00d7
              \n~2-line calls; 20 parse functions each lose 1 line of boilerplate).<\/p>\n
              Fowler ref:<\/strong> Extract Function<\/em> (6.1)<\/p>\n
              Two sub-patterns recur inside the FirestoreStore<\/code> trait impls after running a link query:<\/p>\n