How to Find Duplicates in the Material Master, One Pass at a Time

The buyer in Ohio orders a pressure transmitter. A week later the storeroom in Bremen ships the same transmitter to a sister plant, from a shelf that holds six of them. Nobody made a mistake. The two plants know the part under two article numbers, and nothing in the system says they are one part.

Most master data teams can tell a story like that. Duplicates are part of the job: something to catch and clean up, rarely something anyone tracks as a number. What the team usually lacks is a method that finds them all, in an extract of tens of thousands of lines, without starting a project.

How one part becomes four records

Duplicates are not created by careless people. They are created by time. The same part enters the master years apart, through different plants, by people who each describe it well in their own words. Here is one pressure transmitter as it sits in a master today.

At the Bremen plant it is "DRUCKMESSUMFORMER 0-10BAR 4-20MA", with the maker's full name and the part number PT4410B2. Three years later someone in Bremen creates "TRANSMITTER PRESSURE 10 BAR", with the maker's name abbreviated and the part number written PT-4410-B2. At the plant in Valencia it is "TRANSMISOR PRESION 0-10 BAR", with no maker at all, only the distributor's order number. In Ohio it is "PRESS XMTR 0-145PSI 4-20MA", with no part number.

Four records, three plants, one part. Each record was correct when it was made. Together they hide the part from anyone searching for it, which is the cost that matters most: a 2026 study from SUPSI in Lugano found that duplicates and partially specified records make parts that exist "effectively undiscoverable" (Bulloni et al., SUPSI, 2026, on arXiv). What that does to stock and spend is the subject of the hidden costs of poor spare parts master data.

Find duplicates in the material master in three passes

You can start today, with an extract of the material master and a spreadsheet. Work through it in three passes, cheapest first:

• Exact manufacturer part number. Sort by the part number field and flag every number that appears more than once. This takes minutes and every hit is solid. It misses the transmitter completely: PT4410B2 and PT-4410-B2 are different strings.
• Normalized part number and maker. Strip spaces, dashes, dots and slashes from the part number, and write each maker's name one way. Now the two Bremen records match. This pass takes an afternoon, mostly spent agreeing on makers' names. It still misses Valencia, which carries a distributor's number, and Ohio, which carries none.
• Short texts. For the records the first two passes cannot reach, read the short texts and compare. This is where the transmitter's last two records turn up: a Spanish text, a range in psi instead of bar, an abbreviation nobody else uses. It is also where the method stops scaling. Pass three is reading, line by line, and in a large master it never finishes.

Run the three passes on one material group before you plan anything bigger. Pumps, valves or sensors are good candidates, because they are bought often and described in many ways. The result tells you two things: how many duplicates the easy passes find, and how many records are left over that only reading can settle.

Why the manual method runs out

The first two passes depend on a field the duplicates often do not have. A record with a distributor's number, or with no part number at all, never reaches them. Those are often the records created in a hurry, during a breakdown or for a one-off order, which is exactly when duplicates are made.

The third pass depends on people, and people read short texts the way they write them. The engineer who knows that 145 psi is 10 bar may not read Spanish. The colleague who reads Spanish may not know the maker behind a distributor's number. In one measured case at a multi-site manufacturer, duplicate items made up about 9% of inventory (a case study published by Reliable Plant). That is one case, not a benchmark, but it gives a sense of how much can sit in the records nobody has read.

Two more problems come up in every team that tries this. First, a match found by reading is a judgement, and the next person may judge differently. A group needs a fact behind it before anyone merges records or moves stock. Second, the extract changes while you work. New records keep arriving, and some of them are the next duplicates. A missed search that ends in a new record is how most of them start, as why keyword search misses spare parts in the material master shows.

Groups you can check, on the master as it is

For the rest of the master, the check has to do the reading for you and still give you something to verify. Partium Cockpit does that with Duplicate Monitor. Duplicate Monitor finds the same part filed under different article numbers. For the transmitter, that brings the Valencia and Ohio records into the group the spreadsheet could only half build.

A group is only useful if you can trust it before you act on it. Duplicate Monitor confirms each group by the manufacturer part number. The result reads like the spreadsheet you were trying to build. Every duplicate group lists the article numbers that are one part.

None of this needs a clean master to start from. Your data, as it is. No cleanup project before you start. Short texts, duplicates, missing manufacturers and catalogs in several languages are the input we build for.

What to do once a group is confirmed, and who signs off before stock moves, is a separate step and a separate post. For now, run the three passes on one material group, then compare what you found with what Duplicate Monitor finds on the same extract. Book a demo, or ask for the demo license and run it on your own material master. And if a buyer keeps ordering parts that turn out to be on the shelf, read why MRO buyers reorder parts they already have.