AI That Knows When It Doesn't Know
The most important thing your AI can say is "I'm not sure."
This record exists in your EAM system:
Part Number: MISC-40217
Description: PART, MISC, FOR PUMP
Manufacturer: [blank]
MPN: [blank]Created at 3 AM because the system demanded something. A technician needed a part for a pump repair. The storeroom didn’t have it cataloged. The system wouldn’t let them proceed without a part number. So they created one.
Every AI vendor we’ve watched demos from classifies this record confidently.
That’s the problem.
When Confidence Is the Danger
Let’s test the boundary. Three records that exist in real EAM systems — the kind nobody puts in vendor demos.
Record 1 — The Ghost
Part Number: MISC-40217
Description: PART, MISC, FOR PUMP
Manufacturer: [blank]
MPN: [blank]Generic AI output:
Classification: Pump Part, Mechanical Seal
Confidence: 82%
Extracted Attributes:
Equipment Type: Centrifugal Pump
Component: Mechanical Seal
Material: Silicon Carbide/Carbon
Manufacturer: John Crane
MPN: Type 1 SimplexThe model fabricated an entire identity for a record that contains no meaningful information. “PART, MISC, FOR PUMP” could be a mechanical seal, an impeller, a wear ring, a coupling, a bearing housing, a shaft sleeve, or any of dozens of pump components. The AI picked the most statistically likely pump part and generated a complete specification for it.
Domain-grounded, confidence-aware AI:
Classification: CANNOT BE DETERMINED
Confidence: INSUFFICIENT
Reason: Description contains no classifiable information.
"MISC" and "FOR PUMP" indicate a pump-related
component but provide no basis for specific
classification.
Recommendation: This record requires manual identification.
Suggested approach: cross-reference with work
order history for Part Number MISC-40217 to
identify the equipment and maintenance context.
Physical verification may be required.No classification. No fabricated attributes. No invented manufacturer. An honest assessment: this record cannot be classified from its description, and here’s what to do about it.
Record 2 — The Half-Truth
Part Number: FLT-2290
Description: FILTER ELEMENT HYD 10 MICRON
Manufacturer: PARKER
MPN: [blank]This one is partially complete. We know it’s a hydraulic filter element at 10-micron filtration. We know the manufacturer. We don’t have the part number.
Generic AI output:
Classification: Hydraulic Filter Element
Confidence: 95%
Extracted Attributes:
Filter Type: Hydraulic
Micron Rating: 10
Manufacturer: Parker Hannifin
MPN: 925835
Media Type: Synthetic Glass Fiber
Collapse Rating: 3000 PSIThe classification is correct. The manufacturer is confirmed. The micron rating is extracted. But the MPN, media type, and collapse rating are all fabricated. Parker makes hundreds of 10-micron hydraulic filter elements. The AI picked one — and it might be the wrong one.
In hydraulic systems, the wrong filter element can mean reduced filtration efficiency, improper flow characteristics, or physical incompatibility with the filter housing. The consequences range from accelerated component wear to catastrophic hydraulic failure.
Domain-grounded AI:
Classification: FILTER, ELEMENT, HYDRAULIC
Confidence: HIGH (classification)
INCOMPLETE (specification)
Extracted Attributes:
Filter Type: Hydraulic [extracted]
Micron Rating: 10 [extracted]
Manufacturer: Parker Hannifin [extracted]
Flagged:
MPN: NOT EXTRACTED — not present in description.
Parker Hannifin manufactures multiple 10-micron
hydraulic filter elements across product lines
(Racor, Finite, Par-Fit). MPN required for
procurement specification.
Media Type: NOT EXTRACTED — not specified. Common media for
10-micron hydraulic: synthetic glass fiber,
microglass, cellulose. Affects filtration
efficiency and service life.
Collapse Rating: NOT EXTRACTED — not specified. Must be verified
against system operating pressure.
Element Dimensions: NOT EXTRACTED — physical dimensions required
for housing compatibility.The AI got the classification right and said so. It identified three attributes from the description and said so. It flagged four missing attributes — and for each one, explained why it matters and what specific information would resolve it.
This is the half-truth case that’s most dangerous in practice. The classification is correct. Some attributes are accurate. It’s tempting to accept the whole record. Generic AI makes that temptation easy by filling in every blank. Domain-grounded AI makes the gaps visible so you can address them.
Record 3 — The Obsolete Part
Part Number: BRG-7740
Description: ROLLER BRG BOWER 45MM
Manufacturer: BOWER
MPN: [blank]Bower was a roller bearing manufacturer acquired by Federal-Mogul in 1955. The Bower brand subsequently became part of NTN Corporation through a joint venture in 1985, with NTN taking full ownership in 1987. The brand name persists in legacy MRO systems across manufacturing plants that have been operating for decades. A new data researcher might not recognize it. Generic AI almost certainly won’t.
Generic AI output:
Classification: Roller Bearing
Confidence: 88%
Extracted Attributes:
Bearing Type: Cylindrical Roller
Bore Diameter: 45mm
Manufacturer: Bower Industries
MPN: 7740“Bower Industries” doesn’t exist. It’s a hallucinated entity. The AI recognized “Bower” as a manufacturer name and generated a plausible-sounding company. The bearing type is guessed — “ROLLER BRG” could be cylindrical, tapered, spherical, or needle. The MPN is just the part number with the prefix stripped.
Domain-grounded AI:
Classification: BEARING, ROLLER
Confidence: MEDIUM
Extracted Attributes:
Bore Diameter: 45mm [extracted]
Manufacturer: Bower [extracted]
Flagged:
Roller Type: NOT EXTRACTED — "ROLLER BRG" does not specify
cylindrical, tapered, spherical, or needle.
Bower/NTN-Bower primarily manufactured
tapered roller bearings; context suggests
tapered is likely but not confirmed.
Manufacturer Note: Bower was acquired by Federal-Mogul (1955),
later became part of NTN Corporation (1985).
Current cross-reference may exist in NTN
catalog. Legacy part number may require
historical catalog lookup.
MPN: NOT EXTRACTED — no manufacturer part number
present in description. "7740" is the
internal part number, not an MPN.
Series/Type: NOT EXTRACTED — insufficient information to
determine specific bearing series.The AI knows who Bower was. It knows the acquisition history. It knows that “7740” is an internal number, not a manufacturer part number. It gives you a probable bearing type based on Bower’s product line — but flags it as probable, not confirmed.
That’s domain knowledge in action. Not just processing text — understanding the industrial history behind the text.
Why Confidence Awareness Matters in MRO
In consumer applications, a wrong recommendation means a bad movie suggestion. In MRO:
Wrong pressure rating on a valve in a high-pressure steam system. The gasket fails. The flange separates. Superheated steam at 600°F escapes into a space where people are working.
Wrong material grade on a gasket in a corrosive chemical service. The gasket degrades. The seal fails. Hydrofluoric acid leaks into the environment.
Wrong bearing seal type specified for a food processing washdown application. The seal isn’t rated for high-pressure cleaning. Water intrusion causes premature bearing failure in a production-critical asset. Unplanned downtime during peak production.
These aren’t hypothetical scenarios. They’re the reason MRO data quality matters. And they’re the reason “high confidence” from an AI model isn’t enough. You need to know: confident about what? Based on what evidence? And what happens when the evidence isn’t there?
AI that doesn’t know when it’s wrong isn’t AI. It’s an expensive way to make the same mistakes faster.
The Validation Principle
There’s a design principle that separates AI systems you can trust from AI systems that look impressive in demos.
Extraction is probabilistic. Validation is deterministic. And they should never be the same system.
The AI that reads a description and extracts attributes is making probabilistic assessments. “6205 2RS” probably means a deep groove ball bearing with rubber seals and a 25mm bore. “150#” almost certainly means 150-pound ANSI pressure class. “CS” most likely means carbon steel.
Probability is fine for extraction. It’s how language works. But probability is not fine for the output that enters your EAM system and drives procurement decisions.
Between extraction and output, there must be a validation layer that operates on rules, not probability. Physical consistency checks: does this combination of bore diameter, outer diameter, and width correspond to a real bearing series? Cross-reference validation: does this manufacturer actually make this part in this specification? Taxonomy compliance: does the classification match the taxonomy, and are required attributes present?
The AI proposes. Rules verify. What passes is trustworthy. What doesn’t gets flagged for human review with specific reasons.
This separation — probabilistic extraction, deterministic validation — is what makes automation safe in a domain where wrong data has physical consequences.
The 15% That Still Needs Humans
Here’s the section that will build more trust than any capability claim in this series.
About 15% of records in a typical MRO catalog can’t be fully automated. They require human expertise. Not because the AI failed — because the information doesn’t exist in a form any AI can process.
Obsolete parts from manufacturers that no longer exist. Custom-fabricated items with no catalog equivalent. Records where the description is so degraded that even a domain expert needs to physically inspect the part. Specifications that depend on engineering documentation that isn’t in the EAM system.
No AI solves these. The firm that tells you “100% automated” is the firm that will hallucinate the answers you can’t afford to trust. The records you most need to get right — the safety-critical valves, the custom-engineered seals, the legacy components on equipment that can’t fail — are precisely the records where generic AI is most likely to fabricate.
The firm that tells you “we automate what we can verify, and we tell you honestly when we can’t” is the firm that understands your risk environment.
The 15% isn’t a weakness. It’s the proof that the system is honest.
Three Articles. One Argument.
We started this series by showing you what generic AI does with your MRO data. It produces confident, complete-looking output that fabricates what it doesn’t know.
Then we showed you what happens when three decades of domain expertise meets AI. The same records, classified accurately. Attributes extracted honestly. Gaps identified specifically. More than 60% of records processed without human intervention. Researchers transformed from investigators into adjudicators.
And now we’ve shown you the part nobody else talks about: what happens when the AI reaches the boundary of what it can know. It stops. It explains why. It tells you what it needs. And it routes the work to a human who can provide what the AI can’t.
The technology is ready. Every vendor will tell you that.
The question is whether the expertise behind it is real.
We’ve spent fifteen years building Ark, our MRO master data governance platform. We’ve spent three decades learning what makes MRO data different from every other kind of data. And we’ve spent the last two years building AI that’s as honest about its limitations as it is powerful in its capabilities.
If your team is evaluating AI for MRO data quality, we’d welcome the conversation.
This is the final article in a 3-part series. Start from the beginning: "Why Generic AI Fails at MRO Data" — what happens when you feed your actual EAM records into generic AI.
About the Author
Raghu Vishwanath is Managing Partner at Bluemind Solutions and serves as CTO at KeyZane, a financial inclusion platform live in Central and West Africa. Over 30+ years in software engineering, he has built AI systems that know the difference between what they can determine and what they can’t — because in MRO, that difference is the whole game.