ROI Case File No.472 'Why It Takes Ten Years'
Why It Takes Ten Years
Chapter 1: They Can't Move Without Asking Someone
"When junior engineers receive a spec sheet, the first thing they do is walk over to a senior. They act after asking. Or more accurately—they can't act without asking."
Takuya Yoshimura, Head of Engineering at TechNova, said this while looking at notes he'd been keeping on the department's past three years. The technical division has roughly fifty members nationwide.
"We bring in one to three new graduates every year," Yoshimura continued. "It takes about ten years to reach full competency in design work. During those ten years, juniors move by asking seniors. Seniors answer questions while managing their own workloads. Both sides carry the cost."
"What kinds of information do juniors look up most often?" Claude asked.
"Three categories," Yoshimura answered. "First: past precedents of similar projects—what construction methods were chosen, what problems came up. Second: government standards and regulations, including MLIT guidelines, which require checking for recent amendments. Third: design judgment criteria—the implicit rules of 'under these conditions, go with this spec.'"
"The third category is the real problem," Gemini said. "Precedents and regulations exist in documents. Implicit rules exist only inside the senior's head."
Yoshimura nodded. "That's where the time goes. Time searching for past data, and time waiting to catch a senior. Add those together, and a significant portion of a junior's day disappears just in the act of searching."
"How much time do mid-level staff spend responding to junior questions?" I asked.
Yoshimura flipped through his notes. "When I asked them directly, the most common answer was four to six hours per week. Some said as many as eight. Several said it's affecting their own core work."
"If six hours per week is occurring across twenty of fifty staff members," Gemini calculated, "that's 480 hours per month consumed solely by knowledge transfer."
Yoshimura closed his eyes for a moment. "I'd never put a number on it."
Chapter 2: The Four Rotations of PDCA
"This case calls for PDCA."
Claude wrote four letters on the whiteboard: P, D, C, A.
"PDCA stands for Plan, Do, Check, Act—a framework of four stages repeated continuously to improve precision over time," I explained. "Deploying an AI chatbot isn't a one-time event. The more it's used, the more data it accumulates; the more data it has, the better its answers become—this positive cycle is what a chatbot implementation is really about. PDCA is the blueprint for deliberately driving that cycle."
"Let's start by measuring the current cost," Gemini said, opening ROI Polygraph. Work logs and senior staff interview data from Yoshimura were entered.
The numbers came back.
"Monthly knowledge transfer costs are in," Gemini read aloud. "480 hours per month from twenty mid-level staff responding to junior questions, at ¥3,200/hour: ¥1,536,000. 300 hours per month from twenty junior staff spent on information gathering and research, at ¥2,400/hour: ¥720,000. Combined monthly total: ¥2,256,000. Annualized: ¥27,072,000."
Yoshimura stared at the figure in silence. After a moment: "I'd never seen it on an annual basis."
"The reason it takes ten years is inside this number," I continued. "Now let's design with PDCA."
[P — Plan: Design What to Train the System On]
"The planning phase has three decisions," Claude said. "First: what types of data to feed the chatbot, and in what priority order. Second: an initial set of anticipated Q&A pairs based on what kinds of questions are actually being asked. Third: evaluation criteria to judge answer quality."
"How do we determine priority?" Yoshimura asked.
"By frequency—what juniors ask seniors most often," Claude answered. "Of the three categories you identified—precedents, regulations, implicit rules—start with the most frequent. Trying to load everything at once means the system goes live with low precision. It's far better to build high precision across the top one hundred questions. That's how you earn trust from the field."
"So starting narrow is the key," Yoshimura confirmed.
"You can always expand later," I responded. "Start too broad and users won't know what to ask."
[D — Do: Start Running on Minimal Data]
"In the execution phase, we limit the initial rollout to one location and one team," Gemini continued. "Company-wide deployment comes after precision is confirmed. For the first run, we trial with a small group—five junior and two mid-level staff—for one month."
"What do we look at during that month?" Yoshimura asked.
"Two things," Claude answered. "Number of questions submitted to the chatbot, and satisfaction ratings on the responses. Rising question volume means it's being used. Low-rated answers become the targets for the next data improvement cycle."
[C — Check: Measure Response Quality]
"In the check phase, we evaluate the month's data across three metrics," I continued. "First: did questions to seniors decrease? Second: what proportion of queries were resolved fully by the chatbot? Third: what's the satisfaction score for the responses? If all three improve, we proceed to broader rollout."
"How do we measure the decrease in questions to seniors?" Yoshimura asked.
"During the trial month, we ask the two mid-level staff to log every question they receive from juniors," Gemini answered. "If it's down from the prior month, that's evidence the chatbot is handling those queries. If it's up, there are topics the chatbot can't yet cover—and those become the next Plan's additions."
[A — Act: Design a System That Gets Better With Use]
"The improvement phase is the core of this project," Claude said. "Any response rated poorly by users automatically gets flagged. Those flags compile into a monthly list, a designated person adds the correct answer, and the system relearns from the updated data. Once this cycle runs continuously, the system improves with use."
"Who handles the additions?" Yoshimura asked.
"Initially, just one monthly review from you is sufficient," I answered. "If fewer than twenty responses are flagged per month, it takes under an hour to review. That's the task that replaces the six hours per week mid-level staff are currently spending."
"Let's run the investment plan through ROI Proposal Generator," Gemini proposed.
Implementation costs and projected savings for the AI chatbot were laid out.
- Initial cost: Data preparation + chatbot build + initial training — ¥2,000,000
- Monthly cost: System maintenance + cloud hosting — ¥120,000/month
- Year 1 savings: 50% reduction in senior-response hours = ¥768,000/month; 40% reduction in junior research time = ¥288,000/month; total = ¥1,056,000/month
- Net monthly savings: ¥1,056,000 − ¥120,000 = ¥936,000/month
- Payback period: ¥2,000,000 ÷ ¥936,000 = approx. 2.1 months
"Payback in two months," Gemini summarized. "In year two and beyond, annual savings are projected at ¥11,232,000. If the ten-year learning curve shortens, that figure grows further."
Yoshimura reviewed the numbers. "I'd been treating the ten years as inevitable. But today I saw what that decade actually costs."
Chapter 3: An Organization That Can Carry Its Knowledge
"Let me lay out the implementation plan," I said, standing at the whiteboard.
"Month one—data preparation. Extract the top one hundred frequent questions and write out the answers. Recruit mid-level staff for one hour per week to put their implicit rules into words. Month two—small-group pilot. Trial with five junior and two mid-level staff. Month three—evaluate and improve. Review the three metrics and retrain on low-precision responses. Month four onward—phased expansion, rolling out location by location while continuing to run PDCA."
"The step of putting implicit rules into words sounds like it'll take the longest," Yoshimura said.
"That step has the most value," Claude answered. "Knowledge that isn't verbalized disappears from the organization the moment the person who holds it retires. The process of training the chatbot turns that knowledge into a document. The chatbot is the means—the verbalization is one of the ends."
Yoshimura's expression shifted slightly. "That's—actually urgent, now that you say it. Several mid-level staff are within a few years of retirement."
"It's also an opportunity to turn what's inside senior minds into organizational assets," I said quietly.
Chapter 4: The Day Knowledge Became Portable
Eight months later, a report arrived from Yoshimura.
During the data preparation phase, seven mid-level staff members contributed a combined 312 implicit rules in written form. "When we put it into words, we realized how much of what we considered common sense had never reached the juniors at all," one of the participating seniors said, as Yoshimura recorded in his report.
In the one-month trial, 340 questions were submitted to the chatbot. 78% of those were queries that, previously, the junior staff said they would have asked a senior. The question volume directed at the two mid-level trial participants dropped 61% versus the prior month.
Three months later, full rollout to five locations nationwide was complete. Changes to the ten-year learning curve were still being measured. "We don't know yet whether ten years becomes five or seven," Yoshimura wrote. "But the junior staff say the path to becoming competent feels a little more visible now."
One mid-level employee—a veteran with fewer than two years until retirement—participated most actively in the verbalization work. "I wanted to leave behind what I know," he said, as Yoshimura noted in his report.
Knowledge had moved from inside people's heads to a shelf the whole organization could reach.
"The reason it takes ten years is that experience hasn't been turned into language. Knowledge that hasn't been verbalized can't be carried. It disappears from the organization the moment the person who holds it retires. What PDCA designs isn't a chatbot—it's a structure where knowledge can be carried. Plan by narrowing, Do by testing, Check by measuring, Act by accumulating—those four rotations build a system that improves with use. On the day 312 rules from inside senior minds became organizational assets, the ten-year figure shrank, just a little."
Related Files
Tools Used
- ROI Polygraph — Visualizing knowledge transfer costs and junior research hours
- ROI Proposal Generator — Simulating ROI on AI chatbot implementation
