Infobright Blog

In the world of databases, if it's not a date or a number, it's the ever ambiguous varchar.  What can you do with a varchar?  Seriously.  It's a random assortment of alpha-numerics slopped together with random other characters.  At best, we aspire to achieve high levels of text searching.  Cool tools exist to help with that search -- look at Sphinx.  Several community members use Sphinx+Infobright to power text searching.  However, it really doesn't solve the obvious problem: "My data type isn't a varchar.  It's really an [Email Address, URL, Stock Tick, Genomic Sequence, Foobar] type."  To make things simple, I'll call this datatype your "foobar type." 

While ICE supports just the three official main data types (date/number/varchar), we added a simple yet innovative method to storing and retrieving your foobar type.  We call the functionality "DomainExpert."  DomainExpert allows you to better define your varchar column to potentially improve compression, and most importantly, decompression.  If your query requires significant pulls from disk, the decompression aspect of DomainExpert can greatly improve your query performance.  Best of all, you set it once and you've got it forever.  There's very little overhead or maintenance required.

As an exmaple, take an out-of-the-box DomainExpert available within 4.0: URLs.  By rule, urls take the form of STRING://STRING?STRING.  Logically, we can represent this makeup using %s://%s?%s.  By knowing this makeup, we utilize our compression technology to further compact the data size.  When we pull the structure for any reason, we know exactly where particular elements exist.  For example, any query parameters will belong after the ? in this example. 

More information can be found on DomainExperts within the 4.0 ICE User Guide starting on page 78.  It explains how to setup your custom DomainExpert, assigning them to columns, and other required information.  If you do not have the guide, you can request it by registering at this link: ICE Registration.