Purpose
Finds rows with the same key fields and keeps the first, the last, or only keys that occur once. It does not sort the source before selection: “first” and “last” refer to the existing row order.
Inputs and outputs
One in input and one out output with the same structure; the order of the selected occurrences is preserved.
Parameters
| Parameter | Settings and values | Options and limits |
|---|---|---|
keys | Key fields; an empty list means compare the entire row. | Count: 0–256 |
keep | first — the first row, last — the last row, unique — only keys occurring exactly once. | Options: first, last, uniqueDefault: first |
comparison | typed considers types; text compares textual representations of values. | Options: typed, textDefault: typed |
case_sensitive | Make text keys case-sensitive. | Default: enabled |
trim | Ignore leading and trailing spaces in a text key. | Default: disabled |
Step-by-step configuration
- Read the fields and choose the record’s natural key.
- Decide which occurrence to retain; establish the required order in the source if needed.
- Choose the comparison type, case sensitivity and whitespace handling.
- Check which rows remain and how many duplicates were removed.
Rules and diagnostics
- NULL is not treated as empty text.
- typed distinguishes the text “1” from the number 1; text mode converts values to textual representations.
- last returns rows in the order of their last appearances.
- unique removes every occurrence of a repeated key rather than retaining one of them.
Configuration example
For repeated customer records, choose keys=[CustomerID], keep=last. If the source is ordered from older to newer records, the output contains the latest version of each customer.