Prepare paper records for a system migration

A migration is the moment when everything wrong with your paper becomes everything wrong with your database, permanently. The restoration is the easy part. The part that determines whether the migration succeeds is what happens to values the pipeline is not sure about.

Decide the withheld policy first

Before any processing, decide what your target system does with a withheld value. There are only three sane answers: reject the record, load it with the field empty and a review flag, or hold it in a queue. Choosing 'load it with a best guess' is how a migration poisons a system for a decade.

Run a representative pilot

1. 1Pick 200-400 records spanning your worst decades, not your best ones.
2. 2Run the full pipeline including capture and verification.
3. 3Measure the withheld rate per field, not overall. One catastrophic field can hide behind a good average.
4. 4Hand-check a sample of accepted values, not just withheld ones - that is how you find out whether the checks are actually catching things.

Use every check the documents allow

Which verification is available depends on the document family: check digits on identifiers, column totals on ledgers, two independent readers on grades, medication safety on clinical fields. Turn all of them on. A check that fires once in a thousand records is still worth more than the time it costs.

Keep the provenance

Every treatment is a revertible version and every job retains the untouched original. Keep the jobs after the migration completes. When somebody queries a field in three years, being able to show the original capture beside the restored page beside the captured value is the difference between an answer and an argument.