ROI Case File No.585: They Tried to Finish It in One Shot and Had No Way to Loop Back and Fix It
They Tried to Finish It in One Shot and Had No Way to Loop Back and Fix It
Chapter 1: Accuracy Won't Rise, and the Moves Have Run Out
"We're looking for a partner to work with us on AI image recognition for range hoods."
Soku Mizuki, head of the development planning office at TechGenius, said this as he laid out the situation. "We hold fifty to sixty percent share of the domestic range hood market. But since we're mainly ODM and OEM, consumers don't know our name. We haven't captured share in web sales either. We want to change that, so we're building a mechanism that identifies products from images."
"What state is the current prototype in?" Claude asked.
"We had an existing vendor build it, but the accuracy isn't there," Mizuki answered. "They're short on resources too, so labeling and prompt creation are beyond them. I've been thinking we have no choice but to lock the specification and rebuild it at scale."
"Under what conditions does the accuracy fail?" I confirmed.
"...We haven't looked that far," Mizuki answered. "We've been treating it as one lump: low accuracy. Which conditions produce which errors, and deciding the next move after looking at the errors—we've never worked that way."
"If you try to finish it in one pass, no clues are left behind to fix it with," I replied. "Let's break this down with OODA."
Chapter 2: OODA Asks—Loop Short and Fix as You Go
"This case calls for OODA."
Claude wrote "Observe / Orient / Decide / Act" on the whiteboard.
"OODA—observe, orient, decide, act—is a framework for cycling those four in short periods and raising accuracy as you go," I explained. "The essence is not trying to build the finished product in one pass. Image recognition accuracy isn't determined by a specification on paper. Observe what it got wrong and how, in the field; assemble the cause; decide the next move; and try it. The number of loops you turn becomes the accuracy."
"First, let's measure the current cost," Gemini said, opening ROI Polygraph. The data Mizuki provided went in.
"The monthly costs are out," Gemini read off. "Visual checking of misrecognitions and the resulting rework averages 170 hours per month; at ¥3,900 per hour, that's ¥663,000 per month. Development stagnation from the existing vendor's resource shortage averages ¥400,000 per month. The in-house burden of labeling work averages ¥340,000 per month. Opportunity loss from failing to capture web sales share averages ¥380,000 per month. Accuracy plateauing from the absence of an improvement cycle averages ¥320,000 per month. The total is ¥2,103,000 per month. Annualized, roughly ¥25,240,000."
Mizuki stared at the figures. "I was only looking at development not progressing. Once you add the misrecognition checks and the lost sales, it comes to this much."
"Then let's design with OODA," I continued.
[Observe—Look at the substance of the misrecognitions, in the field]
"First, observe," Claude said. "'Low accuracy' gives you nothing to act on. Was it a unit with oil grime, backlighting, or two models with similar part numbers? Under which condition, and mistaken for what? Classify and record the shape of the errors. This is the starting point for everything."
[Orient—Assemble the cause of the errors into a hypothesis]
"Next, orient," Gemini continued. "From the observed error patterns, assemble the cause. Are there too few training images under a specific condition, is the labeling inconsistent, or is it something even a human couldn't distinguish? A different type of cause calls for a different move."
[Decide—Fix the next move and the deadline for judging it]
"After orient, decide," I continued. "Choose one hypothesis to test next and set when you'll judge it. The partner selection criteria belong here too. Beyond experience in image recognition development, can they put hands on labeling and prompt creation? Looping requires hands."
[Act—Test in short cycles, then return to observe]
"Last, act," Claude continued. "One lap in two weeks. Test, then observe the error patterns again. Because you're not completing it in one pass, a miss costs little. The more laps you turn, the more accuracy accumulates."
[Calculating the payback]
"Let's run the numbers with ROI Proposal Generator," Gemini proposed.
- Initial cost: Misrecognition classification design, partner selection, training data preparation and labeling capacity, systematizing the improvement cycle, and prototype rebuild — ¥5,200,000 total
- Monthly cost: AI platform usage fees plus ongoing model maintenance — ¥230,000 per month
- Monthly savings: Reduced visual checking and rework from misrecognition = ¥500,000 per month (assuming a 70% reduction), elimination of development stagnation = ¥340,000 per month, externalization of the labeling burden = ¥280,000 per month, recovery of web sales opportunity through improved accuracy = ¥300,000 per month — ¥1,420,000 per month in total
- Net monthly savings: ¥1,420,000 − ¥230,000 = ¥1,190,000 per month
- Payback period: ¥5,200,000 ÷ ¥1,190,000 = approximately 4.4 months
"A payback of just over four months," Gemini summarized. "What makes it work is not rebuilding at scale, but looping short and fixing as you go. Lock the specification and build it once, and when it misses you can't tell what to fix, so you rebuild the whole thing again. Loop, and the misses turn into clues. The investment doesn't swing at air."
Mizuki looked over the figures. "I thought having a skilled vendor rebuild it would settle it. Without a loop, whoever builds it stalls in the same place."
"OODA is a tool for looping short and fixing as you go," I replied.
Chapter 3: An Implementation Plan That Loops Short
"Let me lay out how to proceed," I said, standing at the whiteboard.
"Month one—systematize misrecognition classification and recording, and inventory the error patterns. Month two—select the partner and secure labeling capacity. Months three and four—run the improvement cycle on a two-week rhythm. Month five—confirm the accuracy target is met and integrate into web sales. Month six—verify results under live operation. Month seven onward—embed the looping habit and expand to more models."
"Wouldn't it be better to set the accuracy target first and build until we meet it?" Mizuki asked.
"You need the target, but the building method is backwards," Claude replied. "Building until you meet it means the misses along the way go unrecorded, and time passes without anyone knowing what worked. Loop short, and each lap leaves behind what worked, which makes the next lap faster. Building the loop first comes before building it out."
Mizuki took notes. "Before rebuilding, build the loop that fixes it. I can see the order now."
Chapter 4: The Day They Looped, Fixed, and the Accuracy Held
Ten months later, a report arrived from Mizuki.
Image recognition accuracy rose as the laps accumulated. "The first two laps told us we were failing on oil grime and backlighting. We added images under those conditions and it stabilized all at once," Mizuki wrote.
The labeling burden vanished as well. Choosing a partner who would put hands on it kept the lap speed up. "A side-job request wouldn't have turned the loop. Whether there's someone to attach the labels made all the difference in how it moved," the report said.
The largest change showed up in how misses were handled. From trying to finish in one shot, they moved to looping and fixing. "We used to stop at 'accuracy is low.' Once we wrote out which condition produced which error, the place to fix was clear every single time," Mizuki wrote.
Web sales started moving too. Better identification made the consumer-facing entry point usable. "A customer who doesn't know the part number can now reach our product from a single photo. That covered for our weak name recognition," the report said.
As a secondary effect, their way of developing changed. Testing before the specification is locked took root. "We stopped deciding the finished form before building. Now we decide while looping," Mizuki wrote.
At the end of Mizuki's report, he had written: "I thought the image recognition problem was the vendor's lack of capability. But the real problem was that we tried to finish it in one shot and had no way to loop back and fix it. The moment OODA started us looping from observation, the misses turned into clues. Before rebuilding, building the loop came first."
The day a company that had tried to finish in one shot became a company that accumulates accuracy in short cycles, AI image recognition development had changed from a single locked-specification build into a design that keeps cycling observe, orient, decide, and act, the report noted.
"AI development consultations almost always arrive in the form of 'accuracy is poor, so we want to rebuild.' But before you rebuild, there is something to ask. Under which condition, and how, is it failing? What OODA asks about is observation, orientation, decision, action—and the laps between them. Loop, and a miss becomes a clue rather than a loss. The day a company that had tried to finish in one shot managed to loop and fix, what changed was not the model's performance but the very perspective of looping short and accumulating accuracy."
Related Files
Tools Used
- ROI Polygraph — Visualizing rework hours from misrecognition, development stagnation, and the in-house labeling burden
- ROI Proposal Generator — Investment-recovery simulation for an AI image recognition system driven by building a short-cycle improvement loop
