ROI Case File No.575 'The Defects Were Visible, but We Weren't Measuring the Users' Experience'

The defects were visible, but we weren't measuring the users' experience

---

Chapter 1: The integration is bad—but where do we fix it?

"I'm looking for a company we can consult with about our core-system integration."

Mitsuru Kaida, in charge of accounting systems at TechWave, described the situation with a cornered air. "There's a problem in the integration between Kanjo Bugyo and OBIC7. The monthly close takes a whole week. Double-checking crops up, and real-time information sharing isn't possible either. Operational efficiency has dropped markedly."

"How are you trying to fix that integration?" Claude asked.

"By trying to knock down the defects one after another," Kaida answered. "Where does the data fail to mesh, where are the errors. I chase the technical defects. But no matter how much I fix, I get no sense that the burden on the working frontline has eased."

"Have you measured where, and how much, the users are in trouble?" I confirmed.

"…I haven't," Kaida answered. "I've only been looking at defects. User satisfaction, which task is a burden, which feature is used. I've never measured the experience."

"The defects are visible, but unless you measure the users' experience, you can't play an effective hand," I replied. "Let's break it down with HEART."

Chapter 2: What HEART asks—measure the users' experience across five

"This case calls for HEART."

Claude wrote on the whiteboard: "Happiness, Engagement, Adoption, Retention, Task success."

"HEART—the five metrics of Happiness, Engagement, Adoption, Retention, and Task success rate—is a framework that measures the users' experience in numbers and pins down where to fix," I explained. "The crux is not looking only at defects. Knock down technical errors and you still can't tell where the users' burden lies. Happiness, engagement, adoption, retention, task success rate. Because you measure the experience across five, the effective single move comes into view. It's a tool for measuring the experience before fixing."

"First, let's measure the current cost," Gemini said, opening ROI Polygraph. He entered the data Kaida had provided.

"The monthly cost is out," Gemini read off. "Integration and double-check labor for the week-long monthly close averages 190 hours a month; at ¥3,700 an hour, that's ¥703,000 a month. Information delay from the inability to share in real time averages ¥400,000 a month. Rework from integration errors averages ¥360,000 a month. The risk of improvements swinging at empty air because the users' experience isn't measured averages ¥340,000 a month. Processing stalls from person-dependence average ¥320,000 a month. The total is ¥2,123,000 a month—about ¥25.48 million a year."

Kaida stared at the figures. "I'd been watching only the integration defects. Once you add the cost of improvements swinging at empty air because the experience isn't measured, it comes to this much."

"Then let's design it with HEART," I continued.

---

[Happiness & Engagement—measure where the burden lies]

"First, happiness and engagement," Claude said. "Survey the employees' satisfaction and surface the complaints about the system. Measure each department's engagement and pin down which business process is a particular burden. Not defects—measure the weight of the experience."

---

[Adoption & Retention—measure which features are actually used]

"Next, adoption and retention," Gemini continued. "Analyze usage of the current system and evaluate which features are used and which aren't. Survey the resistance to change and see whether a smooth migration is possible. Measure how it's actually used."

---

[Task success rate—measure where things get stuck]

"After adoption, task success rate," I continued. "Measure each task's success rate and pin down where problems particularly occur. Where in the monthly close things get stuck becomes visible in numbers. Measure the clogging of the experience precisely."

---

[Judgment—choose between remediation and replacement on cost-effectiveness]

"Last, judgment," Claude continued. "On the experience measured across five metrics, compare a remediation plan and a replacement plan. This time the data showed it. On cost-effectiveness, handling it through remediation works best. Because you measured, you can choose without hesitation."

---

[Estimating the payback]

"Let's run the numbers with the ROI Proposal Generator," Gemini proposed.

• Initial cost: experience survey, integration-remediation design, real-time integration, process automation, and embedding support—¥5.3 million total
• Monthly cost: system operation plus ongoing maintenance—¥230,000
• Monthly savings: efficiency of the monthly close = ¥520,000 (assuming a 70% reduction); resolution of information delay through real-time sharing = ¥360,000; reduction of rework from integration errors = ¥280,000; resolution of person-dependence = ¥280,000; ¥1,440,000 a month total
• Net monthly savings: ¥1,440,000 − ¥230,000 = ¥1,210,000 a month
• Payback period: ¥5.3 million ÷ ¥1,210,000 = about 4.4 months

"A little over four months to recover," Gemini summarized. "What works is not knocking down defects alone but measuring the users' experience before fixing. Chase errors and you still can't find where the burden lives. Because you measure across five metrics, you can even choose between remediation and replacement on cost-effectiveness. The investment doesn't swing at empty air."

Kaida said as he checked the numbers. "I thought knocking down defects would settle it. Unless you measure the experience, you can't tell where a fix would work."

"HEART is a tool for measuring the users' experience before fixing," I replied.

Chapter 3: An implementation plan that measures the experience before fixing

"Let me lay out the approach," I said, standing before the whiteboard.

"Month one—an experience survey across five metrics, pinning down satisfaction and the burdensome processes. Month two—analyze usage and measure task success rates. Month three—compare the remediation and replacement plans on cost-effectiveness and decide. Months four and five—implement the integration remediation and real-time integration and process automation. Month six—trial run and effect verification. Month seven onward—embedding support and continuous measurement of the experience metrics."

"Wouldn't it be faster to knock down the defects first?" Kaida confirmed.

"That's the pitfall," Claude replied. "Chase defects and you can't see where the users' burden lies, so even after fixing you feel no result. Measure the experience first with HEART, and the effective spot becomes clear, and you can choose remediation or replacement with grounds. Measuring is the shortcut to avoiding wasted remediation."

Kaida said as he took notes. "Before knocking down defects, measure the users' experience. I can see the order now."

Chapter 4: The day, measuring the experience, the effective hand came into view

Ten months later, a report arrived from Kaida.

The monthly close shrank greatly after the remediation. "The monthly close that had taken a week became dramatically shorter. The double-check labor vanished," Kaida wrote.

Information sharing changed too. Real-time integration untangled the information delay. "Kanjo Bugyo and OBIC7 meshed, and we could share information in real time," the report said.

The biggest change showed in the starting point of improvement. From knocking down defects, they shifted to measuring the experience before fixing. "I'd been chasing errors one after another. Once I measured the users' experience across five, where a fix would work came into view, and I could judge that remediation would do," Kaida wrote.

The grounds for the judgment were clear too. The cost-effectiveness comparison backed the choice of remediation. "I didn't have to rush into a replacement-first stance. The numbers I measured showed that remediation was enough," the report said.

As a side effect, the way the system was viewed changed. Rather than defects, the habit of measuring by experience took root. "I stopped ending at 'knock down the error.' I began to think in terms of where, and how much, the users are in trouble," Kaida wrote.

At the end of Kaida's report was written: "I thought the integration trouble was technical defects. But the real problem was that the defects were visible while I wasn't measuring the users' experience. The moment I measured the five metrics with HEART, the effective hand came into view. Before knocking down defects, measuring the experience came first."

On the day a company that could see the defects but wasn't measuring the users' experience became a company that could measure the experience before fixing, the system integration had changed from defect-crushing into a design that measures the users' experience across five metrics before fixing.

"A consultation about system integration usually arrives in the form of 'we want to fix the defects.' But before knocking down defects, there's something to ask. Are you measuring the users' experience? What HEART asks is the five of happiness, engagement, adoption, retention, and task success. Measure the experience and both the effective spot and the remediation-or-replacement choice come into view. On the day a company that could see the defects but wasn't measuring the experience could measure across five, what changed was not the precision of the technology but the very perspective of measuring the users' experience before fixing."

---

Related Files

Tools Used

• ROI Polygraph — visualizing monthly-close labor, information delay, and the risk of improvements swinging at empty air
• ROI Proposal Generator — a payback simulation for the core-system integration, starting from measurement of the users' experience