SAND

Integration

Home »  Nucleus »  Integration

Simple Integration

  • SAND software uses standard commodity hardware and operating systems.
  • ETL and data load tools can load data into SAND as an ODBC compliant database.
  • Existing database schemas will not necessarily need to be modified to deliver against requirements.
  • Existing query tools will deploy against SAND through ODBC and standard SQL.
For the end user, SAND products appear as a standard RDBMS and use standard SQL commands to communicate with the underlying database. Integration with a SAND environment is provided using a standard ODBC/JDBC interface and is supported via a range of standard business intelligence front-end tools. End users do not require any SAND specific training and there are no specific requirements to use any particular front-end tools.

Existing data models or schemas can be loaded into the database with little change or more easily understood and managed data structures may be employed, such as denormalisation, for ease of user understanding or to combat the issues surrounding slowly changing dimensions.


Load & Go operation

  • Due to the fully automated indexing and column based design
    • The indexes are an inherent part of the database structure
    • Additional storage space for indexes is not required
    • Data is immediately available for querying once it has been loaded
    • Very little manual tuning of the environment is required
    • No additional indexes need to be created once the data is loaded
This ability is one of the major sources of cost and time savings which are enabled by SAND. It allows user requirements to be fully understood by loading data without needing to understand exactly how it will be used and to immediately execute the sort of queries they were previously unable to run. It also allows for the proposed reporting model to be refined in-line with the user’s requirements and expectations.

In client tests involving direct comparisons between SAND and other database systems, data was available for querying over 8 times faster within the fully indexed “load-and-go” environment provided by SAND when compared to the “load-and-index” approach used to create the partially indexed (key fields only) approach used by other database systems.


Unstructured Text Integration: Advanced Text Search

  • Enhances standard SQL to provide advanced text search capability
    • Match searches
      • Search for an exact word or phrase in a text field
        • SELECT c1 FROM t1 WHERE c1 MATCHES ‘”abc”‘
        • This would return rows where t1.c1 contains the word “abc
    • Phonic searches
      • “sounds like” matching based on double metaphone encoding
        • SELECT c1 FROM t1 WHERE c1 MATCHES ‘PHONIC “gray”‘
        • This would return rows where t1.c1 contains the word “grey” as well as “gray
    • Concept searches
      • Search for synonyms using a built-in thesaurus
        • SELECT c1 FROM t1 WHERE c1 MATCHES ‘CONCEPT “large”‘
        • This would return all records where t1.c1 contains the words “big”, “huge”, and so on, as well as “large”
    • Fuzzy searches
      • “spelled like” matching, allowing for typing errors and so on
        • SELECT c1 FROM t1 WHERE c1 MATCHES ‘FUZZY/1 “gray”‘
        • This would return rows where t1.c1 contains the words “grab”, “fray”, “ray”, “grady” and so on, as well as the search term “gray”
    • Stem searches
      • Search for all words sharing a common stem with the search term
        • SELECT c1 FROM t1 WHERE c1 MATCHES ‘STEM “scheduled”‘
        • This would return all records where t1.c1 contains the words “schedule”, “schedules”, “scheduling”, “re-schedule”, and so on, as well as “scheduled”
    • Proximity searches
      • Search for one word within or beyond a specified distance from another
      • BEFORE, AFTER, NEAR and NOT NEAR
        • SELECT c1 FROM t1 WHERE c1 MATCHES ‘((“cat”) NEAR/5 (“dog”))’
        • This would return the all records where “cat” appears within 5 words of dog. Specific instances of “dog” or “elephant” would not be returned
    • Relevance (“Bag of Words“) search
      • Involves calculating the numeric relevance of each searched record to a list of search terms, and filtering those records according to a user-defined threshold for relevance
      • This type of search is called a “bag of words” search; it operates in a manner similar to some of the modern Internet search engines, like Google.
        • SELECT … WHERE RELEVANCE(col1, ‘sand magma glass’) > 0.75