Sample ROI report

The measurement report a NorthCape pilot produces in week six. It is the document the keep-or-kill decision gets made from, so it leads with how the numbers were captured and only then with what they came to.

1. Engagement summary

2. Scope of what was measured

One workflow, measured end to end, on the firm's own volumes. Anything the pilot touched but did not measure is listed here, so the saving cannot be read as covering more ground than it does.

In scope

• Time spent by the intake team on each new matter, from the enquiry landing to the matter being open and complete in the practice management system.
• Conflict checking, identity document collection and the opening file note.
• Drafting and issuing the client care letter and the costs disclosure.
• Exceptions raised by the agent, and the time a person spent resolving them.

Not in scope

• Everything after the matter is opened: searches, settlement and file closure.
• Enquiries that did not convert to a matter. Volume was recorded, handling time was not.
• Fee earner time. No fee earner task was automated and none was measured.
• The firm's other two offices, which run a different intake process.

3. How the baseline was captured

The baseline is the part of a pilot report most likely to be wrong, so it is described here in enough detail that the firm can dispute it. It was captured in week 1, before any agent existed, and signed off by the client sponsor before the build started.

• Volume. 240 matters a month, the twelve month average of matters opened, exported from the practice management system by the firm's practice manager.
• Handling time. 40 matters timed end to end during week 1 by the two intake officers, using a start and stop stamp added to the intake form. Mean 38.0 minutes, median 36 minutes.
• Loaded hourly cost. $68, supplied by the firm's finance manager as salary, on-costs and overhead recovery for the intake team. This is the firm's own figure and was not adjusted.
• Working month. 4.33 weeks, used for every conversion between weekly and monthly figures.

4. Baseline figures

5. What the agent did

The agent was built in weeks 2 and 3 and ran supervised from week 4. It does the assembly work and stops at every point where a judgement is required.

• Reads the enquiry, whether it arrived by web form, email or referral portal.
• Extracts the parties, the property, the transaction type and the target settlement date.
• Runs the conflict check against the practice management system and reports the result.
• Opens the matter, populates the fields and attaches the source correspondence.
• Drafts the client care letter and the costs disclosure from the firm's own templates.
• Requests the identity documents and chases them on the firm's existing schedule.
• Raises an exception, with the reason, wherever a rule does not resolve cleanly.

What stayed with a person

• Every conflict check result with a possible match.
• Sending the client care letter. The agent drafts it, a person sends it.
• Any fee outside the firm's standard scale.
• Every exception, all of which route to the supervising conveyancer.

6. Post-deployment figures

Measured in week 5, with the same start and stop stamps and the same sample size as the baseline. The timed sample deliberately includes matters that raised an exception, so the mean below is inclusive of exception handling rather than measuring only the clean path.

7. The delta

The 96 hours are hours of intake officer capacity, not a headcount reduction. The firm's stated intent is to absorb growth without a third intake hire, which is a different decision from a redundancy and would produce a different number if the firm ever chose otherwise.

8. What was not counted

Every figure above is intake handling time and nothing else. The following were all observed during the pilot and none of them are in the saving.

• Ongoing running cost. Model inference and hosting for this workflow ran under $200 a month during the pilot, roughly 3% of the monthly saving. It is excluded because two months is not long enough to establish a steady rate, not because it is nil.
• The supervising conveyancer's time. Week 4 supervision took about 4 hours. That is a one-off cost of standing the agent up. It is not deducted above, and it is also not a permanent overhead.
• Downstream fee earner time. Fee earners reported cleaner opening files. Nothing was measured, so nothing is claimed.
• Re-keying into the searches platform. Still manual, still out of scope, and the obvious candidate for the next workflow.
• Time to instruction. The median fell from 2 days to under 1. Whether that converts more enquiries is unknown on a five week window, so it is not counted as revenue.
• Error and rework rates. No baseline for intake errors existed before the pilot, so no comparison is possible.
• Staff experience. Both intake officers said the work was better. That is not a number and it is not in the saving.

9. Exception rate

An exception is any matter the agent stopped on and handed to a person with a reason. It is a health measure rather than a failure measure. A rate near zero would suggest the rules were too permissive, not that the agent was performing well.

Across weeks 4 and 5, 10 of 111 matters raised an exception, a rate of 9%. Every one was resolved by the supervising conveyancer within the same working day.

10. Cost and payback

• Payback on the pilot fee: 3.3 weeks. The saving is $1,506 a week, so $5,000 is recovered part way through the fourth week of running.
• Payback on the whole first year investment: week 22. $33,000 at $1,506 a week.
• The annualised figure assumes volume and handling time hold. If volume falls by a quarter, so does the saving. Nothing in this report is a guarantee of either.

11. Recommendation

Proceed to deployment for this workflow. The measured saving covers the deployment cost inside the first year with a wide margin, the exception rate sits in the range that indicates the rules are set sensibly, and no exception during the pilot produced a client-facing error.

Conditions on that recommendation

• The client care letter stays approval-gated. That is not a temporary supervision measure and it should not be revisited at the three month review.
• Conflict check matches stay with the supervising conveyancer permanently.
• Re-measure at the end of month three using the same method. If the mean has drifted above 20 minutes, the rules need revisiting rather than the agent needing more autonomy.
• Do not extend to the other two offices until their intake process has its own baseline. Their process is different and this baseline does not describe it.

What would change this recommendation

• An exception rate above 20% sustained over a month, which would mean the rules do not fit the work.
• Any exception producing a client-facing error, which would move the whole workflow back to supervised.
• Monthly volume falling below 150 matters, at which point the deployment cost stops being recovered inside a year.

Appendix A: raw counts

Weeks 2 and 3 were build weeks with no agent in the workflow and are not counted. Week 4 is reported separately because a supervised run is not a steady state: every action waited for a person, which is why its mean sits between the baseline and the measured figure.

Appendix B: constants used

All figures are Australian dollars and exclude GST. Percentages are rounded to the nearest whole number and times to one decimal place, so a column may not sum exactly.