Thanks Ted. In terms of vision I see the data federation layer actually
a lot wider than the example I gave in my previous mail (adjunct and
core data warehouse).
Different types of storage engines excel at different types of use cases:
- A graph database is ideal for complex hierarchies and many to many
relationships, e.g. found in Master Data Management
- Inverted index and document value store for unstructured data
- self describing data formats are great for semi-structured data
- databases such as MapR-DB or RDBMS are great for structured data
What is missing, however, is a layer to query these through one engine
and stitch the results together on the fly.
On 13/02/2015 19:16, Ted Dunning wrote:
Drill definitely can serve as a database virtualization layer. Calcite was
used this way when it was just Optiq and Drill provides interesting
additional capabilities.
The emerging view of user needs seems to be tilting more towards the
semi-structured data capabilities of Drill rather than the virtualization
potential, however.
As an open community, Drill can be swayed by your input. If you want to
make this happen, you definitely can do it.
On Fri, Feb 13, 2015 at 4:28 AM, Uli Bethke <[email protected]> wrote:
The use case of the adjunct data warehouse requires a data federation
layer between production warehouse analytics on Hadoop and the rest of the
EDW on an RDBMS.
The incumbents (Teradata, Oracle, SAS etc.) have proprietary offerings in
this space. PrestoDB also allows for federation between Hadoop and Postgres
and MySQL.
In my opinion Drill would be ideally suited to be the federation tool of
choice.
I am wondering how the Drill roadmap prioritises this use case in terms of
connectors to popular EDW RDBMS engines?
thanks
uli
--
___________________________
Uli Bethke
Co-founder Sonra
p: +353 86 32 83 040
w: www.sonra.io
l: linkedin.com/in/ulibethke
t: twitter.com/ubethke
