Why Generic AI Fails at MRO Data
Your AI vendor's demo worked perfectly. Your data won't.
Your AI vendor’s demo was flawless.
Classifications appeared instantly. Part descriptions cleaned themselves up. Attributes materialized from unstructured text. Someone on your evaluation team said, “This could save us millions.”
The demo used 40 records. They were clean. Well-structured. Complete descriptions with manufacturer part numbers, clear specifications, and standard naming conventions.
Your EAM system has 200,000 records. They look nothing like that demo.
Someone on the team — usually the maintenance planner who’s lived inside the system for fifteen years — asks the question that changes the conversation: “What happens when we feed it our actual data?”
That question is worth more than the demo.
What Generic AI Actually Does With Your Data
Let’s find out. Take three records from the EAM system of a large industrial manufacturer. Not curated records. Not demo-ready examples. The kind that actually live in your system right now.
Record 1 — The Bearing
Part Number: 6205-BEARING
Description: 6205 2RS BEARING
Manufacturer: [blank]
MPN: [blank]You know this record. It was created during a 2 AM emergency. The technician needed the part, couldn’t find it under its original entry, and created a new record to get the purchase order moving. No time for manufacturer details. No time for proper classification.
Feed this to a generic AI classification model. Here’s what comes back:
AI Classification: Ball Bearing
AI Confidence: 97%
Extracted Attributes:
Bore Diameter: 25mm
Outer Diameter: 52mm
Width: 15mm
Seal Type: Contact Rubber Seal (2RS)
Bearing Type: Deep Groove
Manufacturer: SKF
MPN: 6205-2RSLooks impressive. The confidence score is 97%. The dimensions appear correct. A procurement manager looking at this output would approve it without hesitation.
Three problems.
The bore diameter, outer diameter, and width were not in the description. The AI inferred them from the “6205” designation — which is correct for this specific bearing, but the model has no way to verify this against the actual part. It presented inferred data as extracted fact. No distinction. No caveat.
The manufacturer was listed as SKF. The original record had no manufacturer. The AI hallucinated it. SKF is the most common manufacturer of 6205 bearings, so it’s a reasonable guess — but it’s a guess presented as a fact. The actual part could be NSK, NTN, FAG, or a dozen other manufacturers. For inventory matching and procurement, this matters.
The MPN was returned as “6205-2RS.” The description said “6205 2RS.” The AI didn’t know that “2RS” is a widely used generic seal designation, that “2RS1” is SKF’s legacy designation, that “2RSH” is SKF’s current designation, or that each manufacturer uses its own suffix — DDU for NSK, LLU for NTN, 2RSR for FAG — all indicating the same functional seal type. It just cleaned up the spacing and called it a part number.
Ninety-seven percent confident. At least three errors that matter.
Record 2 — The Valve
Part Number: VLV-4812
Description: GATE VALVE 4IN 150# CS FLGD
Manufacturer: [blank]
MPN: [blank]This is a safety-critical record. A gate valve in a process system. The description uses standard MRO shorthand that any valve specialist reads fluently: 4-inch, 150-pound pressure class, carbon steel, flanged ends.
Generic AI output:
AI Classification: Gate Valve
AI Confidence: 94%
Extracted Attributes:
Size: 4 inches
Pressure Class: 150
Body Material: Carbon Steel
End Connection: Flanged
Bore Type: Full Bore
Manufacturer: Crane
MPN: F4-150-CSThe classification and basic attributes look correct. But two things were fabricated.
“Full Bore” appears nowhere in the description. A gate valve can be full bore or reduced bore — the distinction affects flow capacity and is specified in engineering documentation. The AI defaulted to the most common configuration. In a high-pressure system, wrong bore specification means wrong valve, which means a potential safety incident.
The manufacturer and MPN are completely invented. No “F4-150-CS” part number exists in Crane’s catalog. The AI generated a plausible-looking part number because the pattern resembled catalog formats it had seen in training data. A procurement team using this output would issue a purchase order for a part that doesn’t exist.
Record 3 — The Gasket
Part Number: GSK-0091
Description: GASKET FLANGE 4IN RF 150#
Manufacturer: [blank]
MPN: [blank]Now it gets harder. This is a flange gasket — 4-inch, raised face, 150-pound class. But the description says nothing about material, thickness, or gasket type (spiral wound, ring joint, sheet). These attributes determine whether the gasket can handle the temperature, pressure, and chemical environment of the application.
Generic AI output:
AI Classification: Flange Gasket
AI Confidence: 91%
Extracted Attributes:
Size: 4 inches
Flange Face: Raised Face
Pressure Class: 150
Material: Graphite/Stainless Steel
Gasket Type: Spiral Wound
Thickness: 1/8"
Manufacturer: Flexitallic
MPN: CG-0150-0400Every attribute below “Pressure Class” is fabricated. The AI generated the most statistically likely gasket configuration for a 4-inch 150# application. It might even be right. But “might be right” and “verified” are not the same thing in a process environment where the wrong gasket material in a corrosive service can cause a catastrophic failure.
The confidence score: 91%.
Why This Happens
The AI model doesn’t know what a bearing does.
It doesn’t know that pressure class on a valve is a safety-critical attribute. It doesn’t know that “2RS” and “2RS1” and “2RSH” and “DDU” all represent equivalent sealing systems from different manufacturers. It doesn’t know that a gasket’s material compatibility depends on the service environment, which isn’t in the part description and can’t be inferred from text alone.
It processes text. It doesn’t understand parts.
The model is confident because confidence is what it’s designed to produce. Large language models generate the statistically most likely completion of a sequence. When the sequence is “4-inch gate valve, 150#, carbon steel,” the most likely next token for bore type is “full bore.” The most likely manufacturer is the market leader. The most likely part number follows a pattern the model has seen before.
Confidence is not accuracy. In MRO, the difference can shut down a production line.
This isn’t a deficiency of any specific AI model. It’s structural. Generic AI treats MRO descriptions the way it treats restaurant reviews or product listings — as text to be processed. It has no model of physical reality. No understanding of why specifications exist. No concept of safety criticality.
The Knowledge Problem
The vendors selling you AI classification trained their models on general data. Your MRO catalog isn’t general data.
It’s decades of human inconsistency layered on domain-specific complexity. Emergency entries created at 2 AM. Vendor catalog imports with different formatting conventions. Legacy migrations that transferred chaos from one system to another. Abbreviations that mean different things across industries. Specifications that require physical knowledge to interpret.
The bearing that became five records in your system didn’t happen because people were careless. It happened because MRO data is genuinely hard. The descriptions are compressed. The context is missing. The abbreviations are inconsistent. The manufacturer references are incomplete.
Any AI that doesn’t understand why MRO data is hard will produce confidently wrong output. It will fill in blanks with plausible fictions. It will present inference as fact. It will assign 95% confidence to hallucinated manufacturer part numbers.
And here’s the part that should concern your evaluation team: the output looks right. It’s formatted correctly. The attributes are reasonable. The confidence scores are high. Without deep domain knowledge, there’s no way to tell the difference between an AI that extracted real data and an AI that fabricated plausible data.
That’s not a technology problem. It’s a knowledge problem.
What Your Evaluation Should Actually Test
Before you sign that AI contract, take fifty records from your worst data. Not the demo set. The records your maintenance planners complain about. The emergency entries with blank manufacturer fields. The legacy records with descriptions that are basically sentence fragments. The imports from acquisitions that nobody ever standardized.
Feed them to the AI.
Then have your most experienced MRO cataloger — the person who’s been managing your parts data for a decade — review the output. Not for formatting. For accuracy. For fabricated attributes. For hallucinated part numbers. For the difference between what the AI extracted from the text and what the AI invented to fill the gaps.
We’ve spent three decades learning what a bearing is. What a gasket does. Why a valve’s pressure class matters. That knowledge doesn’t live in a model. It lives in the people who build the model.
The question isn’t whether AI can improve MRO data quality. It can.
The question is whether the AI knows what it doesn’t know.
Most don’t.
Next in the series: what happens when AI is built by people who understand what they're classifying.
About the Author
Raghu Vishwanath is Managing Partner at Bluemind Solutions and serves as CTO at KeyZane, a financial inclusion platform live in Central and West Africa. Over 30+ years in software engineering, he has learned that the hardest problems in data aren’t technical — they’re knowledge problems disguised as technical ones.