How to assess an office workflow before adding AI

Observe a complete task, including the work after it

Follow one item from arrival to its usable outcome. For meeting notes, the outcome might be accepted actions in the team queue, rather than a polished summary. For service operations, it might be a request with enough context for an owner to respond. Record copying, searching, waiting, checking and correcting as separate activities.

Review a representative sample: routine inputs, ambiguous wording, missing attachments and unusual cases. Note active minutes and elapsed time separately. A delay caused by waiting for a manager needs a different remedy from slow transcription. NIST’s AI RMF Core treats context, measurement and oversight as continuing responsibilities; it supports this approach without prescribing a particular tool.

Place AI where interpretation is actually needed

A rule can check whether an email address exists, assign a department from a chosen form option or calculate an overdue date. Those steps should produce the same result for the same input. AI may help turn a long conversation into candidate actions or recognise the topic of an unstructured request. Keep its interpretation separate from the rule that decides what happens next.

Agree which source material may enter the pilot, who can access the results and how long they are retained. Test with anonymised material where possible. Do not upload a complete mailbox simply because the target task concerns a few messages. If the necessary source is unavailable or the team cannot approve its use, change the pilot scope.

Worked example: draft actions from a service meeting

Hypothetical example, not a client project: an operations meeting includes “Alex could check the replacement stock” and later “Alex will check it after the supplier confirms delivery.” The draft should preserve the condition and point to the relevant source passage. It must not turn the first suggestion into an unconditional assignment or invent a Friday deadline.

Ask for an action, proposed owner, stated deadline, dependency and source location. An unstated deadline remains blank. The meeting lead confirms whether the action was agreed and clarifies the missing date. Only accepted actions enter the service queue. A second colleague should be able to open the record, find the supporting passage and understand why the item is still waiting.

Write a pilot brief someone else can operate

Start in shadow mode: the existing process continues while the pilot prepares drafts for comparison. Give the reviewer a precise question, such as “Does every action preserve its owner, condition and deadline from the source?” Microsoft’s Power Automate approval documentation illustrates an explicit workflow that waits for an approver’s response. A notification alone does not establish that approval occurred.

• Input: authorised source, trigger, document version and excluded material.
• Output: required fields, allowed empty values and evidence location.
• Review: named reviewer, backup reviewer and correction reasons.
• Handoff: destination, accepted status and next owner.
• Exception: unreadable input, conflicting statements and overdue review.
• Recovery: pause control, manual route and log of attempted actions.

Measure the whole cost, including corrections

Use the same type of work for the baseline and pilot. Effective minutes per completed item = preparation + review + correction + recovery minutes, divided by accepted items. Net time benefit per item = baseline minutes per item minus pilot minutes per item. Report the sample size and input mix; a small test of clean notes cannot establish performance on noisy recordings.

Track invented actions, omitted actions and changed conditions separately. Check accepted output against the source, because approval rates can rise when reviewers become less attentive. Record unresolved items as well as completed ones. Compare recurring software charges in USD with support, review and maintenance effort; faster generation is only one part of the cost.

Agree the release and stop conditions before the pilot

Set the tolerated error types and the required evidence with the task owner before testing. Expand only if the agreed cases pass, reviewers can keep up and the manual fallback still works. One successful meeting is a reason to continue testing, not evidence that the workflow can safely run unattended.

Pause if the pilot introduces commitments absent from the source, sends information to the wrong destination, accumulates unreviewed work or takes more total effort without a compensating benefit. Return affected items to the manual queue and preserve the input and correction history. After changing the model, prompt or source format, replay the agreed examples before releasing that version.