Data cleaning refers to the process of identifying, detecting, and resolving unwanted, duplicate, or irrelevant records within a dataset. When multiple data sources are integrated, the risk of duplication increases significantly, leading to unnecessary clutter in business systems. Such redundancies can make it difficult for organizations to generate accurate results and insights.
Manually detecting and removing duplicate or irrelevant records is often complex and time-consuming. To address this, Odoo ERP provides a powerful Data Cleaning module that automates much of the process. In Odoo 18, this module has been enhanced with new tools and advanced features, making data cleaning even more efficient. By using it, businesses can systematically eliminate redundant and unnecessary data, ensuring cleaner records and more reliable outcomes.
The Field Cleaning and Deduplication features in the Odoo 18 Data Cleaning module ensure that your data remains consistent and accurate by applying effective tools and strategies.
Deduplication refers to the process of identifying and removing duplicate records, particularly in computer systems. With a robust and well-defined deduplication tool, unnecessary copies of similar data can be detected and eliminated. This helps maintain a database that contains only relevant and essential information, preventing redundancy and improving overall data quality.
This blog provides a clear overview of how to set up deduplication rules in the Odoo 18 Data Cleaning module.
Data redundancy is a common long-term challenge for many organizations. Professionals often spend considerable time and effort dealing with duplicate or unnecessary records. However, the Odoo 18 Data Cleaning module simplifies this task, making the entire process more efficient and manageable.
The Deduplication tool in Odoo 18 makes it easy to simplify and speed up your data cleaning activities. In this section, we’ll see how to configure Deduplication rules within the Data Cleaning module.
Deduplication Rules
Businesses often collect information from multiple channels. For example, when a company gathers leads, the data may come from social media, websites, and other platforms. Bringing all this information together in one place frequently results in duplicate entries. This is where the Odoo Data Cleaning module proves valuable. With its Deduplication Rules feature, the system helps you efficiently manage and eliminate duplicate records.
Using the Deduplication Rules feature, you can quickly detect and handle duplicate records. In addition, Odoo provides a dedicated menu where all Deduplication rules can be organized and managed efficiently.
To get started, you’ll need to install the Data Cleaning module from the Odoo Apps before accessing its features.
After installing the Data Cleaning module, you can open it to access its features. In the interface, you’ll find the menu option for managing Deduplication. Under the Configuration tab, the Rules section provides access to the Deduplication menu.
Select the Deduplication option to open the corresponding window. The illustration of this window is shown below.
In the displayed window, you’ll find all the Deduplication rules that have been created earlier. Each rule is listed with details such as its name and the associated model. You can also clearly identify whether a rule is active or inactive through the available toggle buttons.
Adding a New Deduplication Rule
To set up a new Deduplication Rule, click on the Create button. This will open a form where you can define the details of the rule, as illustrated below.
With the creation form, you can define a new Deduplication Rule for the data-cleaning process. Start by entering a Name for the rule. Next, select the appropriate Model from the dropdown list of available models. Once the model is chosen, you can narrow down the data by applying Filters with specific conditions. The New Rule option allows you to set these conditions more precisely, helping you target the exact records you want.
By setting these conditions, you can refine and identify the records relevant to the deduplication process directly from the Domain field.
After this, You’ll find the Duplicate Removal setting, which provides two choices: Archive or Delete. Select Delete to permanently remove duplicates, or choose Archive to keep them stored without deleting.
Next, choose the Merge Mode, which can be set to either Manual or Automatic. If you select the Manual option, a Notify Users field will appear. In this field, you can specify the users who should be alerted whenever new records need merging. Additionally, you can schedule reminders for these users—daily, weekly, or monthly—so they can review and merge the records manually.
If you choose the Automatic Merge mode, an extra field called Similarity Threshold will appear. In this field, you can enter a percentage value, which determines the minimum similarity required for records to be merged automatically. Any records exceeding this threshold will be combined without manual intervention.
Likewise, an active/inactive toggle is available to enable or disable the deduplication rule. Finally, under the Deduplication Rules section, you can define multiple rules that guide the system in suggesting records for merging if they meet at least one condition. Here, you can specify the unique ID along with the matching criteria, which can be set as either an Exact Match or a Case/Accent Insensitive Match.
You can add this deduplication rule using the Add a line option. By clicking this button, you can specify multiple fields and set conditions as needed.
Once all the required details are entered, click the Save button to store the new Deduplication Rule. After saving, you can use the Duplicate button located at the top left to proceed further.
The system will then take you to a new window displaying all the records from the selected model that are identified as similar.
The Data Cleaning module in Odoo 18 is highly beneficial for setting up different deduplication rules and efficiently managing duplicate records.
To read more about How to Recycle and Optimize Records Using Odoo 18 Data Cleaning Module, refer to our blog How to Recycle and Optimize Records Using Odoo 18 Data Cleaning Module.