Duplicate records in databases do not only cause high costs, they can also lead to many other problems. Not least when consolidating different data inventories, e.g. during fusion or adding value from acquired data, the duplicate search is an indispensable and critical business process. This is, why data quality more and more becomes a significant value for every business.

Fuzzy Duplicate Search

When searching for such duplicate records, you must distinguish between sharp (with exact accordance) and similar duplicates.

Grouping for exact matches is something, what any DBMS can do in seconds.

On the other hand is the detection of similar records something, that can be hardly achieved and what is computationally intensive. This only succeeds with specialized tools. A problem is, that without the capable methods you cannot even estimate, in which dimension such duplicates exist in your data. You just can't see it.

On the Search for similar Records

Over years, phonetic algorithms (e.g. SoundEx) are used to find what sounds similar. This approach already brings up some results, which go beyond any sharp comparison. Thereby permutations of the strings (like twisting and mirroring) stay unconsidered and a too strong emphasis is layed upon the first letter.

Much better is the use of pattern-matching algorithms (e.g. Levenshtein Distance). Such algorithms can consider permutations, but are highly calculation cost intensive.

So another problem is the total running time of the calculation: When using pattern-matching algorithms, in principle each record must be compared with each other. This means for n records the total number of (n - 1) * n / 2 comparisons. That are ½ trillion (1012) complex calculations for a datatable with 1 million records. The calculation could last for years.

The FuzzyDupes Method

The FuzzyDupes method was developed within 9 years by now, completely and originally done by Kroll-Software. The calculation kernel contains more than 7.000 lines of code.

FuzzyDupes makes use of a Trigram-Hashidex for building clusters. This is a mathematical exact and reliable way to preselect good candidates for the deeper search, which does not depend on phonetical algorithms. The deep search pattern-matching algorithm was also developed by Kroll-Software and it can better consider all permutations than any other known algorithm.

All used algorithms are based completely on pattern-matching and are therefore language- and culture independent. Unicode is fully supported and so this works not only with latin characters. It is mathematically verifiable, that all similarities are detected consistently and reliably.

FuzzyDupes 2012 uses parallel Execution and 64-bit

The two critical resources for the duplicate search application are memory usage and calculation time.

With large data tables, the process has a high demand on RAM, which can only be allocated on 64-bit systems. 32-bit systems can only address a maximum of 2,4 gigabyte of memory.

We put in much development effort to parallelize the algorithms and unleash the power of modern multi-core cpu's. FuzzyDupes scales very good with the number of cores.

So the current version offers the search in bigger data tables on todays standard computers.

Why is FuzzyDupes so affordable compared with other duplicate search software ?

Duplicate search programs used to be specialized solutions and the preserve of a very limited clientele. In addition, these applications could only be run on mainframe computers due to the high computing power required. As a result, the applications were very expensive.

We believe that the ability to perform fuzzy duplicate searches is crucial for every company maintaining a customer database. We want to make our application accessible to small and medium sized enterprises and recognize that the price must stand in direct relation to the benefit gained. These considerations form the basis for the price of our product. However, the benefit for your company can far exceed the cost of a FuzzyDupes license.

New in FuzzyDupes Version 2012

Notebly higher power and speed through parallel execution and 64-bit
Thereby practically unlimited size of data searchable
Full usage of modern Core-iX cpu's and 64-bit systems
Uses DotNet 4.0 Framework
Display of the match-factor in the results
Support of MS-Access and MS-Excel data sources even on 64-bit systems
The new 32-bit launcher offers access to 32-bit data sources on 64-bit systems

Supported Data Sources / DBMS:

MS-Access, MS-Access 2007* and 2010* on 32-bit and 64-bit systems
MS-Excel, MS-Excel 2007* and 2010* on 32-bit and 64-bit systems
MS SQL-Server
Text/CSV Files
Other Datasources with ODBC-Driver or OLEdb Provider, e.g. Oracle, IBM DB2, MySQL, dBase, Foxpro, Paradox, FileMaker, Cache, PostgreSQL, etc.
Search and deletion from MS-Outlook contact folders.
This makes FuzzyDupes the solution for cleansing Outlook contacts
Windows Addressbook
MS-Sharepoint Server
BulkMailer Address Database
32-bit data sources can be accessed on 64-bit systems using the FuzzyDupes 32-bit launcher
New - FuzzyDupes 2012 - fuzzy duplicate search - dedupe and data cleansing software 64-bit ODBC data sources
New - FuzzyDupes 2012 - fuzzy duplicate search - dedupe and data cleansing software Windows Contacts / Windows Mail

Home:

http://www.kroll-software.com/products/fuzzydupes_e.asp

Only for V.I.P