Expert knowledge needs context
A maintenance recommendation can depend on equipment type, operating state and document version. Retrieving relevant text does not establish that all conditions for application are known.
Energy-domain knowledge assistants.
A knowledge assistant should do more than sound like an expert.
Progress on computer-use benchmarks reminds us that AI capability has task boundaries. One aggregate score cannot establish reliability inside an enterprise. We believe an energy knowledge assistant should first establish whether its evidence applies, then optimise fluency and breadth.
A SIGNAL WORTH EXAMINING~66%
task success reported for agents on the OSWorld computer-use benchmark2
2026 · OSWorld benchmarkTHE ZHICE INTERPRETATION
Agents create value within a controllable scope of delegation.
Task completion has improved substantially, yet failure remains. The enterprise question extends beyond intelligence: how long a chain can be delegated, what consequences can it trigger, and who takes over when it fails? In energy businesses, authority, knowledge grounding and recovery will jointly determine usable capability alongside the model.
THE ARGUMENT BEHIND THE VIEW
A judgement needs a complete argument.
A knowledge assistant should do more than sound like an expert.
EXAMINING AI THROUGH THE TASK
THE REASONINGThe business conditions behind the view
Answers change human attention
Confident presentation can reduce the user’s inclination to verify. The experience should expose decisive evidence, unresolved conditions and useful questions instead of only presenting a polished conclusion.
Expert corrections belong in a learning loop
Field corrections reveal hidden assumptions. Using feedback only as a satisfaction score misses the opportunity to turn individual experience into sustained organisational knowledge.
IMPLICATIONS FOR LEADERS
What should leaders reconsider?
Define scope through specific questions
Separate retrieval, solution assistance and consequential judgement. Establish citation and review requirements for each rather than assigning every responsibility to one assistant label.
Make evidence part of the experience
Let users return to original passages, versions and conditions. Explain conflicts rather than smoothing them into a fluent account.
Test reliability through business questions
Continuously assess common, boundary and easily misused questions. Observe whether users can take the right next action, not simply whether they like the answer.
WHEN TO RECONSIDER
Heavy review can undermine usefulness for low-consequence general retrieval. Requirements should follow consequences rather than imposing one threshold on every response.
Can the user see what supports the answer, when it applies and what still requires judgement?
FROM PERSPECTIVE TO PRACTICEPut this perspective to work.
ExpandCollapse
Put this perspective to work.
Manuals, maintenance records, procedures and project learning are dispersed across systems. General chat tools may miss equipment, version and operating-condition differences. A domain assistant needs task-specific retrieval, citation and abstention behaviour.
Define tasks and sources
Choose tasks such as equipment-document retrieval and procedure comparison. Curate authoritative, current sources and separately constrain high-impact judgements such as incident response and work approval.
Design evidence-backed answers
Attach source passages, versions and applicability to key conclusions. Explain gaps and escalate when evidence conflicts, is insufficient or falls outside scope.
Evaluate with domain samples
Have domain experts build common, difficult and unanswerable questions. Test retrieval and answers separately, using feedback to correct sources and rules as well as prompts.
Work products
Task boundaries and trusted-source register
Citation, conflict-resolution and escalation rules
Expert evaluation set and knowledge-operating model
What to measure
- Correct, evidenced answers
- Experts assess conclusions, citations and applicability together, reporting severe errors separately.
- Appropriate abstention and escalation
- Check whether the system recognises insufficient evidence or out-of-scope requests and provides a useful next step.
The assistant supplies supporting information and does not replace authorised safety decisions or work approvals under valid operating procedures.
Data notes and research foundationsExpandCollapse
- 1Stanford HAI · AI Index 2025
Comparison of inference cost at GPT-3.5-level MMLU performance; not a reduction in total enterprise AI or project costs.
Stanford 2025 · Nov 2022–Oct 2024 · GPT-3.5-level performance - 2Stanford HAI · AI Index 2026
OSWorld is a structured computer-use benchmark; the result is not an energy-production success rate or a deployment threshold.
Stanford AI Index 2026 · OSWorld benchmark - 3IBM Institute for Business Value · 2025 CDO Study
Self-reported status in a cross-industry, cross-geography survey, not a measured maturity score for energy enterprises.
IBM 2025 · 1,700 senior data and analytics leaders globally
THE NEXT CONVERSATIONThe question worth discussing:
Contact ZHICE