What does cleaning the data mean?
Definition
Data cleaning is the process of finding and fixing problems in a dataset, such as duplicate records, missing details and wrong values, so the data is as complete and accurate as possible before anyone analyzes it.
- Category
- Data
- Also called
- cleaning the data, data cleansing, data scrubbing
- Last updated
Reviewed by Anthon Wansland
Head of Product & Co-Founder
Serus is a privacy company with paid tools to find and remove your personal data. This page is free and based on official sources.
Sources checked: registries.ncats.nih.gov, centre.humdata.org, data.hawaii.gov
Last verified against the official page:
When people say they are "cleaning the data", they mean tidying a dataset before using it. The NIH's rare disease registry program describes it as removing duplicate records and records missing vital information. It also removes values that can't be right, such as a person listed as 25 feet tall. The goal, in its words, is to make the dataset as complete and accurate as possible before analysis.
Other public bodies use different words for the same job. The UN's Centre for Humanitarian Data calls it correcting and/or standardizing data from a record set, table or database. Hawaii's State Data Office, using the term data cleansing, says it means identifying and correcting data that are inaccurate, missing or incomplete.
What cleaning usually involves
- Removing duplicates: the same person entered twice, once as "Robert Lee" and once as "Bob Lee".
- Handling missing values: fill the blank from another record, ask the source, or drop the row.
- Fixing errors: typos and impossible numbers.
- Standardizing: "St." and "Street" get stored one way.
Why it matters to you
Most datasets about people hold names and addresses. Customer lists and patient registries do. So do marketing files. Cleaning is how that data gets merged, matched and corrected. A wrong detail about you can spread. A corrected one can too. Tidy data is also easier to link across sources, which can make you easier to find.
Cleaning isn't a way to get your data deleted. Duplicates get merged, and the NIH's definition also drops records that are missing vital information, but a complete record stays in. It also differs from data masking or de-identification, which hide who the data is about.
Example
A county clinic exports 4,000 vaccination records. Two rows are the same patient, and one of them gives a birth year of 1895. A staff member merges the pair, checks the chart (it should be 1985) and fixes it. That is data cleaning: the clinic's age reports are right now, and the patient still sits in the dataset.
How to protect yourself
- 1
Keep an untouched copy first
If you clean your own spreadsheet or contact list, save the original before you change anything. Merging duplicates or deleting rows can't be undone once you overwrite the file.
- 2
Check the fixes against a real record
Don't assume an automatic fix is right. Compare a sample of changed rows with the source, as the clinic did with the paper chart, so a correction doesn't swap one error for another.
- 3
Limit who can open raw files
Raw files with names and addresses are the ones being cleaned. Share them only with people who need them. Delete working copies when the job is done.
- 4
Reduce what is out there to be matched
The fewer places list your address, the fewer files can match and merge it. Serus finds where your address and phone number are listed and removes them.
See where your personal data is exposed
Serus finds it on data brokers and people-search sites and removes it.
Frequently asked
Is data cleaning the same as data cleansing?
Yes. Data cleansing and data scrubbing are other names for the same work. Hawaii's State Data Office, for example, defines data cleansing as identifying and correcting data that are inaccurate, missing or incomplete.
Does cleaning the data delete anything?
Sometimes. The NIH describes removing duplicate records and records missing vital information. Other fixes only correct or standardize values, so what is removed depends on the rules of the project.
Why is data cleaned before analysis?
Because analysis on wrong or incomplete data gives wrong results. The NIH says the goal is to make the dataset as complete and accurate as possible before it is analyzed.
Can I ask a company to correct wrong data about me?
In many places you can ask. Which rights apply depends on where you live and the company, so check the privacy notice of the company holding the data for how to request a correction.
Sources
Something wrong or out of date? Tell us and we'll re-check it.
What changed
- Corrected the claim that cleaning never removes people: the NIH definition also drops records missing vital information.