Written by Kiwi Dynamics Team 1What exact workflow will this fix?
Ask the agency to name the specific workflow they intend to automate, receipt entry, debtor chasing, client intake, in plain language, not in the abstract language of transformation or efficiency. If they cannot describe the exact task, the exact input, and the exact output in one sentence, they likely have not scoped a real solution yet, they have scoped a pitch.
2Can I see it running in production?
Ask to see a live example of the agency's work running in an actual business, not a demo built for the sales call. A system that has only ever been shown in a controlled walkthrough has not yet met the messiness of real client data, real edge cases, and a real team using it under time pressure, and that gap matters enormously in accounting.
3How is success actually measured?
Ask exactly how the agency will measure whether the engagement worked, hours saved per week, debtor days reduced, onboarding time cut from weeks to days. An agency confident in its own work will have a specific number in mind before the project even starts, not a vague promise to reassess later.
4What happens to our client data?
Ask precisely where client financial data goes, which systems it touches, who can access it, and whether it is used to train any model beyond your own firm's use. Accounting data is sensitive by nature and any agency worth hiring should have a clear, specific answer ready without you having to press for it.
5Who reviews the AI's output?
Ask what stays in a human's hands and what does not, specifically who reviews AI-generated figures, coded transactions, or drafted returns before anything reaches a client. A serious agency will insist on a human review step for anything with real financial or compliance consequences, not just offer one as an optional extra.
6How does this fit our software stack?
Ask how the proposed system will actually connect to Xero, or whatever practice management and accounting software the firm already runs, and what breaks if that software updates its interface. A tool that requires the firm to change how it works around the AI, rather than the AI fitting into how the firm already works, is a red flag.
7What is the realistic timeline to live?
Ask for a realistic date the first workflow will be live and in use, not a roadmap of everything that could eventually be built. Firms should be wary of agencies that lead with an ambitious multi-phase vision before committing to a specific, working piece of software on a specific date.
8What happens when it gets something wrong?
Ask what happens when the AI gets something wrong, a miscoded transaction, a receipt misread, an email answered incorrectly. The answer should describe a specific fallback and correction process, not a general assurance that the system is accurate, because every system makes mistakes and the plan for handling them is what actually matters.
9Do you understand accounting, or just AI?
Ask whether the people building the system have worked with accounting or bookkeeping workflows before, or whether this is a generic AI build being adapted on the fly. GST treatment, debtor sensitivities, and BAS deadlines are specific enough that an agency without any accounting context will spend your project's early weeks learning things a specialised partner already knows.
10What does this cost against the time saved?
Ask for the cost of the engagement stated next to the hours or dollars it is expected to save, in the same conversation, not in separate documents. If an agency cannot or will not put those two numbers side by side, that is itself useful information about how confident they are in the outcome.
The right agency answers these questions specifically, with examples, not with reassurance. Kiwi Dynamics builds production AI, not slideware, for accounting and bookkeeping firms across New Zealand and Australia, and is happy to be asked every one of these questions before any work begins.