Data Matching vs Merging: Guide to Record Linkage

“Just merge the duplicates” is one of those instructions that sounds simple until someone has to actually write the code. Merging what, exactly? Merging based on what signal? And how do you know two records are duplicates in the first place, rather than two different customers who happen to share a last name?

Credit bureaus like Experian answer this at massive scale — matching and linking records across billions of rows without a human ever eyeballing a comparison. The technique is called data matching (or record linkage), and it’s a distinct step from data merging, even though the two get used interchangeably.

Matching and Merging Are Two Different Jobs

Data matching answers one question: do these two records refer to the same real-world entity? It’s a comparison problem — take two records, decide “same” or “different,” and it has to run across every candidate pair in a dataset without a human checking each one.

Data merging is what happens after you’ve decided two records match: combining them into a single surviving record, deciding which fields win when they conflict (newer timestamp? more complete value? higher-trust source system?), and retiring the duplicate.

Skip straight to merging without a solid matching step, and you get one of two failure modes: real duplicates that never get flagged (an under-matching problem), or unrelated records fused together because a fuzzy rule was too aggressive (an over-matching problem — genuinely dangerous when the records belong to different people).

Two Ways to Match: Fuzzy vs. Deterministic

Most homegrown matching logic reaches for fuzzy string similarity — Levenshtein distance, Jaro-Winkler, soundex — scoring how “close” two strings look and merging anything above a threshold. It works, sort of, until you tune the threshold too loose and merge “John Smith, 12 Oak St” with “John Smith, 120 Oak St” — a different house, a very different problem.

The alternative is deterministic matching: transform both records into the same canonical form first, then compare for exact equality. No thresholds, no tuning, no probabilistic judgment call — either the canonical keys match or they don’t. For addresses specifically, that canonical form comes from standardization: parsing “12 Oak St” and “12 Oak Street” into the same structured components and the same expanded string.

curl -X POST "https://api.goodvat.com/v1/address/normalize" \
  -H "Authorization: Bearer $GOODVAT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "query": "12 Oak St, Denver CO 80202" }'

{
  "houseNumber": "12",
  "road": "oak street",
  "city": "denver",
  "state": "co",
  "postcode": "80202",
  "country": "USA",
  "expansions": [
    "12 oak street denver co 80202 usa"
  ]
}

Feed “12 Oak Street, Denver, CO 80202” through the same call and you get the identical expansions string. That’s your join key — deterministic matching, no similarity scoring required.

Where Address Standardization Fits

GoodVat’s Address Standardization API produces exactly that canonical key: structured fields plus fully expanded expansions strings, generated from free-form address text. It’s the input a matching engine needs — it isn’t the matching engine itself, and it doesn’t merge records or decide survivorship for you.

A realistic pipeline looks like this:

1. Standardize every address field on ingestion, so downstream systems store the same canonical form regardless of how the address was originally typed.
2. Match by comparing canonical keys — a straight equality join for exact-match candidates, with fuzzy matching reserved for genuinely ambiguous cases (name variants, missing unit numbers) rather than the address itself.
3. Merge matched pairs according to your own survivorship rules — that part is business logic your MDM platform or dedup job owns.

One thing standardization deliberately doesn’t do: confirm the address is real or deliverable. That’s address validation or verification territory — useful before you mail something to the golden record you just built, but a separate step from the matching problem this post is about.

Fix the input to your matching logic, and a surprising amount of “our dedup is unreliable” pain just disappears.

Read the docs for the full field reference.