Comments on: Efficient auto-complete with a ternary search tree

By: John Haugeland

John Haugeland — Wed, 03 Feb 2010 02:20:11 +0000

Igor: If you want to support nearest match, the easiest thing to do is to store and search against lexemes (say, metaphone or soundex or whatever). If you want, you can take the Levenstein difference between the potential string and the source string as a quick sort criterion for small matches; it wouldn’t scale, though, since you’d have to apply it to every source string available.

For larger matching that scales on machines with a fair amount of memory available, the easiest way to deal with it is to knock out one lexeme from the word at a time, and match the gap with a GADDAG or something.

Also, a suffix trie is a rather simpler way to resolve this quandry you’re approaching.

By: Gallery of Processor Cache Effects

Gallery of Processor Cache Effects — Tue, 02 Feb 2010 06:42:47 +0000

[...] Efficient auto-complete with a ternary search tree [...]

By: Igor Ostrovsky

Igor Ostrovsky — Thu, 14 Jan 2010 17:11:49 +0000

alex, it may look like “AB” is in the diagram, but really it is just “A”. A letter only counts if you follow a down-arrow that starts from that letter.

By: alex yang

alex yang — Thu, 14 Jan 2010 16:56:34 +0000

If ternary tree structure like the above pictrue,words seq should be “AB”,”ABCD”,”ABBA” & “BCD”.

By: Igor Ostrovsky

Igor Ostrovsky — Fri, 04 Sep 2009 06:52:16 +0000

Barry: You could save space by storing the casing as a bitmask rather than a string. Still, even that is some extra space.

Seide: I am not familiar with lucene. Either way, this article is more about explaining a cool data structure than about giving the most practical solution for auto-complete.

By: seide

seide — Thu, 03 Sep 2009 10:08:22 +0000

Why not just use a full-text search index like lucene?

By: Barry Kelly

Barry Kelly — Thu, 03 Sep 2009 09:15:29 +0000

Igor: one reason for using a trie (especially a Patricia trie) for lookup of strings with lots of common prefixes is that it can be far more space-efficient than e.g. a sorted array used with binary search. Storing the full, properly-cased string in the terminal nodes negates this advantage.

By: links for 2009-09-02 « Blarney Fellow

links for 2009-09-02 « Blarney Fellow — Thu, 03 Sep 2009 01:10:34 +0000

[...] Efficient auto-complete with a ternary search tree | Igor Ostrovsky Blogging (tags: tree search strings ui) [...]

By: Igor Ostrovsky

Igor Ostrovsky — Wed, 02 Sep 2009 23:21:56 +0000

Somejan & Barry: Both very good points. As is usual, the list of caveats and alternate solutions should be a lot longer than the actual article.

Barry: A reasonable solution to the casing problem would be to store the list of properly cased words in each terminal node. If words rarely differ by casing only (as is probably the case in practice), the list would typically be of length 1, maybe 2.

By: Barry Kelly

Barry Kelly — Wed, 02 Sep 2009 22:11:29 +0000

It’s worth pointing out that tries and structures similar to them have issues with case preserving but case insensitive operations. Consider the shape for sequences like:

aBcd
aBdd
abbd
abcd

… and then find all items with the prefix “abc”, considered case-insensitively.

Case preserving but insensitive comparisons normally sort uppercase characters directly before their corresponding lower-case characters, but the way tries store the data, they must either (1) normalize and lose the case data, failing the preservation requirement, or (2) store each letter case separately and have much greater difficulty finding all elements with the same prefix (considered case insensitively), as items with the same prefix may exist in multiple descendants and an in-order traversal of the trie will place the items out of lexicographical order.