Using the Duplicate Check Job Filter

By applying a filter before starting a job, it's possible to run the job on a subset of data. For example, create a duplicate check job for duplicate Leads from the United States only. Or, only compare the records that were created today with all other records. Configure the filter in the 'Add New Job' modal, or work with predefined filters.

Configuring the filter

Click the [ + Filter ] button to add a filter to the modal. At "Type", you can select what kind of filter you want to apply. In setting up the filter it is possible to compare:

1. A subset of records with the same subset of records (default).
2. A subset of records with all other records.
3. A subset of records with a different subset of records.
4.  A subset of a source object with a subset of a match object. 

1. Compare a subset of records with the same subset of records

An example of this is if you only want to compare records from the United States. To achieve that, at "Type", choose "Set filter". Make sure you compare the set filter with "Records that meet above filter criteria" at "compare with". 

Compare records where (Country = United States) with records where (Country = United States). 

2. Compare a subset of records with all records 

In some cases, it makes perfect sense to compare a subset of records with all records. A great use case for this is, after you deduplicated your legacy data, to run (scheduled) jobs that only take into account records that were created in the last few days with the entire dataset. That way, you don't have to run large batch jobs, and you still get the desired results. 

At Type, set a filter. For example, choose the "Created Date" field, with a Date Literal set to "LAST_N_DAYS: 3". At "Compare with", choose "All records". 

Compare records where (Created Date = LAST_N_DAYS: 3) with All records.

3. Compare a subset of records with a different subset of records

A third option allows you to, for example, compare records from the company "Mac Donalds" with the records from the company "Burger King" - a subset of records with a different subset of records. 

At "Type", set a filter. For example, choose "Company" with a value set to "Mac Donalds". At "Compare with", also set a filter. Choose "Company" with a value set to "Burger King". 

Compare records where (Company = Mac Donalds) with records where (Company = Burger King).

4. A subset of a source object with a subset of a match object.

When defining two seperate objects at Source Object and Match Object, you will run a cross object job. After clicking the [+ Filter ] button, you are able to define a filter per object. This allows you to, for example, only compare Lead records from the United States with Contact records from the United States. 

At "Type", choose "Set a filter". For example, choose the "Country" field at Leads with "United States" as value. Do the same thing for contacts. 

Compare Lead records where (Country = United States) with Contact record where (Country = United States). 

Using filter logic (optional)

When defining multiple filter lines, it's possible to use filter logic. Enter each filter line number, separated by a filter logic operator. For example, (1 AND 2) OR 3 finds records that match both Filter 1 and Filter 2, or Filter 3.

If you don't set filter logic, all filters lines will be handled as if the operator is AND. 

Here’s a complete table of filter logic operators:

Predefined Filters

When you have predefined filters defined, see that they are available for selection at "Type" as well. If you often use the same filter, save time by defining them in a predefined filter at the DC Setup.