User Tools

Site Tools


admin_functionality:settings:set_attribute_group_rules

Match Group Transforms


This section will allow you to modify and standardise your data via predefined transforms. This data will then be utilised for matching purpose. After clicking on ‘Match Group Transforms’ under 'Settings', you will navigate to the screen below:

Rules: On the left hand side, a pre-defined list of 'Match Rules' will appear in alphabetical order (ascending-by default). You can drag and drop transform(s) to configure against any Match Group(s). After a drag and drop, the pop-up will configure the parameters used within the transform.

You can enter/select the parameters and then click the ‘Add’ button to add the transform to the selected Match Group.


Match Group Rules

Match Groups: By default, groups will appear in alphabetical order (ascending). You can rename a Match Group by a double click. It is mandatory to create group(s) as prerequisite and then only transforms can be configured against group(s). You can drag and drop the transform(s) in upward and downward direction to change the order.

Parameters: Displays the parameter description for the transform selected. On double click; a pop-up will appear to edit the transform parameters.

Test Input Phrase: You can also see the effect of a transform. You can enter the 'Test Input Phrase' at Match Group level. The results will be returned in the ‘Test Result’ column as each individual transform is applied. The ‘Test Input Phrase’ and ‘Test Result’ will not be saved in database.

Test Result: For the submitted phrase, the final result will appear at group level and a step by step result will appear at transform level as shown below:

Create Group: You can only create new groups on the Match Group section. Without the Match group, there is no group available to apply data transformation. Please see Match Groups for more infromation on creating a new match group.


Transforms

During the matching process – Data Transformations can alter the way that DQ for Dynamics/Workbooks looks at your data (it does not change the actual data). This is very effective for transforming specific data element purely for the purpose of matching. E.g. in the company name field you may have a record of TrueData Ltd and another record entered as TrueData Plc. In this case the “business element” (Ltd or Plc) may be considered irrelevant so you only wish to match on the core of the word “TrueData”. You would use transformations to “exclude” these elements in this case. See individual transformation categories for a more detailed explanation.

NOTE: Users cannot apply transform rules on the multi select option set.

Custom Exclude

This allows you to work with delimiters stored within the database. Simply insert the left and right delimiter and select your mode. The screen below allows you to configure transform parameters for a ‘Custom Exclude’:

Custom Exclude Modes (options):

  1. Remove between Delimiters
  2. Remove Delimiters Only
  3. Remove Delimiters and between
  4. Remove Outside the Delimiters

Custom Transform

This will search for a customised word or phrase and replace it with a custom string. The screen below allows you to configure the transform parameters for ‘Custom Transform’:

Custom Transform Library

This is a flexible feature which allows you to bespoke/tailor a match to cater for the unique nature of each user's data. A list of required transforms can be configured using ‘Category’ & ‘Custom Transform Library Configuration’ screens in 'Custom Settings' option under 'Settings' menu. The screen below allows you to configure transform parameters for ‘Custom Transform Library’:

Extract Letter

This is used to eliminate one or more characters from left or right end of the data. The screen below allows you to configure transform parameters for ‘Extract Letter(s)’:

Extract Name

This is used to eliminate a particular part of the name information from the data. The screen below allows you to configure transform parameters for ‘Extract Name’:

Mode:

  1. Prefix/Title
  2. First Name
  3. Middle Name(s)
  4. Last Name
  5. Suffix/Qualification

Extract Word

This is used to eliminate whole word(s) from left or right end of the data. The screen below allows you to configure transform parameters for ‘Extract Word’:

Remove Characters

This is used to eliminate all Vowels, Consonants, Numbers, Punctuation and Other Characters from the data. The screen below allows you to configure transform parameters for ‘Remove Characters’:

Mode:

  1. Vowels: A E I O U
  2. Consonants: B C D F G H J K L M N P Q R S T V W X Y Z
  3. Numbers: 0 1 2 3 4 5 6 7 8 9
  4. Punctuation: , . : ; - '
  5. Other Characters: ` ! £ $ % ^ & * ( ) _ + = [ { ] } @ # ~ < > ? / | \

Split String

This transform will allow you to split a string based on a custom delimiter, you have to option to return the result with or without the specified delimiter.

Transform Words

This is used to Normalise, Exclude, Elaborate or Abbreviate standard elements from your data. The screen below allows you to configure transform parameters for ‘Transform Words’:

Abbreviate:

Category Example
Addressing 'Road to 'Rd', 'Avenue' to 'Ave'
Business 'Limited' to 'Ltd', 'Company' to 'Co'
Countries 'United Kingdom' to 'UK'
DateEvents 'January' to 'Jan'
JobTitles 'Manager' to 'Mgr', 'Colonel' to 'Col'
Numbers 'Twenty' to '20', 'Nine' to '9'
Qualifications 'Bachelor of Science' to 'BSc'
Salutations 'Doctor' to 'Dr', 'Mister' to 'Mr'
WeightsMeasures 'Ounces' to 'Oz'
Miscellaneous 'Object' to 'Obj'
Forenames 'Robert' to 'Bob', 'Antony' to 'Tony'

Elaborate:

Category Example
Addressing 'Rd to 'Road', 'Ave' to 'Avenue'
Business 'Ltd' to 'Limited', 'Co' to 'Company'
Countries 'UK' to 'United Kingdom'
DateEvents 'Jan' to 'January'
JobTitles 'Mgr' to 'Manager', 'Col' to 'Colonel'
Numbers '20' to 'Twenty', '9' to 'Nine'
Qualifications 'BSc' to 'Bachelor of Science'
Salutations 'Dr' to 'Doctor', 'Mr' to 'Mister'
WeightsMeasures 'Ounces' to 'Oz'
Miscellaneous 'Obj' to 'Object'
Forenames 'Bob' to 'Robert', 'Tony' to 'Antony'

Exclude:

Category Example
Addressing Exclude text such as 'Road“ and 'Rd'
Business Exclude text such as 'Ltd' and 'Limited'
Countries Exclude text such as 'UK' and 'USA'
DateEvents Exclude text such as 'Mon' and 'January'
JobTitles Exclude text such as 'Mgr' and 'Manager'
Numbers Exclude text such as '100' and 'Hundred'
Qualifications Exclude text such as 'BA' and 'BSc'
Salutations Exclude text such as 'Mr' and 'Dr'
WeightsMeasures Exclude text such as 'Oz' and 'Ounces'
Miscellaneous Exclude text such as 'Obj' and 'Object'
Forenames Exclude text such as 'Andi' and 'Robert'

Normalise:

Category Example
Addressing 'Garden' to 'Gardens', 'Gdns' to GND'
Business 'Company', 'Comp' to 'CO'
Countries 'United Kingdom', 'Great Britain', 'GBR' to 'GB'
DateEvents 'January' to 'Jan', 'Monday' to 'Mon'
JobTitles 'Engineer', 'Engr' to 'ENG'
Numbers 'Nought', 'Null', 'Nil' to '0'
Qualifications 'Dr of Philosophy', 'DPhil' to 'PhD'
Salutations 'Mrs', 'Ms', 'Madam' to 'MRS'
WeightsMeasures 'Inches', 'Inch', 'Ins' to 'IN'
Miscellaneous 'Cheque', 'Check' to 'Chq'
Forenames 'Andrew', 'Andrea', 'Andres' to 'Andi'

For a detailed overview of our data transformations, please see our Transform Guide

Trim String

This will eliminate any spaces at the beginning and/or at the end of an attribute. There is no screen to configure the transform parameters for ‘Trim String’. You can directly drag and drop the ‘Trim String’ rule for any group and it can be viewed as shown below:


Match Key

Once the data has been transformed, purely for the purpose of matching, it can then be tokenised by a fuzzy matching alorithm. This can be applied by selecting the ‘Match Key’ drop-down. The 'Match Key' is used to select an algorithm for phonetic match token generation.

The 'Match Key' drop-down will have six choices:

Soundex

Soundex retains the first letter of the input string to formulate its match token. Soundex removes vowels (a, e, i, o, u) and h and w from the input string. The remaining letters are assigned numbers using a lookup table to produce a token of 4 characters.

This means ‘Cathy’ and ‘Kathy’ will not match as their match tokens begin with a ‘C’ from Cathy and a ‘K’ from Kathy. As such, Soundex does not match well where the start of a word sounds the same but is not the same. Also, due to the numeric substitution it is possible to be shown non-matches (false positive) matches.

DQSoundex

DQSoundex overloads Soundex with the advanced capabilities of DQFonetix™. This improves the start of word logic and modifies the first letter(s) of an input string. DQSoundex will de-pluralise and pre-process the start of words to manage variances like ‘C’ to ‘K’ as in 'Cathy' and 'Kathy', as well as ‘Ph’ as in Phonetix to ‘F’ in Fonetix.

Metaphone

Metaphone improves the Soundex algorithm by using information about variations and inconsistencies in English spelling and pronunciation, to produce a more accurate encoding.

This allows you to find more precise matches than the simple Soundex algorithm. Metaphone considers a larger set of character transformations than Soundex and therefore analyses a string phonetically with far more accuracy.

DQMetaphone

DQMetaphone like DQSoundex is an enhanced Metaphone technology with the advanced capabilities of DQFonetix™. This improves the start of word logic and modifies the first letter(s) of an input string. DQSoundex will de-pluralise and pre-process the start of words to manage variances and improve matching.

In the case shown below (Christopher), Metaphone would have generated three of five names matches. However, after running DQ’s advanced algorithms and advanced logic, DQMetaphone allows ‘Kh’ from 'Khristopher' to match with the ‘Ch’ from 'Christopher'. Thus generating the same match key token.

DQFonetix™

DQFonetix™ contains our advanced phonetic algorithms developed over the last 25 years by DQ Global. The algorithm is property DQ Global and hence we do not share the specification of the process. However, DQPhonetix™ has four key features:

  • Five spoken languages – English, Spanish, French, Italian and German
  • Avoids false matches
  • Overcomes character variances
  • Deals with diacritics

DQPhonetix™ provides your CRM system with the most varied matching window to highlight duplicate matches that may not be picked up – or falsely matches - in Soundex and Metaphone.

No Match Key

Selecting no match key will not generate a phonetic token, hence no match token will be generated. However, this allows you to match identical strings.


Miscellaneous Functions

These functions will allow you to navigate the DQ for Dynamics/Workbooks application. They are as follows:

  1. Expand All - Expand all groups to see attributes, transforms, display attributes etc.
  2. Add Group - Add a Match Group, Transform Group, Display Group.
  3. Delete - Delete a rule set or multiple rule sets
  4. Refresh - Refresh the rules displayed on the screen displayed

admin_functionality/settings/set_attribute_group_rules.txt · Last modified: 2021/07/01 11:22 by conor.doyle