However, we are interested in exploring ways that a deeper integration might be beneficial especially in the case of federated query processing across multiple database backends.
Can you elaborate a bit more on this? I can understand that there are common functionalities can be shared and make sense to share (parsing, relational algebras, etc). But I am more curious about what new scenarios and/or performance benefits can be achieved through the integration. I personally work on the infrastructure side, and am not the direct user of data system. But some of our customers do express interests in approximate query support. I’d like to talk to them and do some more survey. I would be happy to share with you my findings after that. Thanks. On May 7, 2018, at 11:15 AM, Michael Mior <[email protected]> wrote: Xiening, You are correct that VerdictDB is currently completely DBMS independent. However, we are interested in exploring ways that a deeper integration might be beneficial especially in the case of federated query processing across multiple database backends. The first step would be to simply allow Calcite and VerdictDB to work together. That is potentially highly useful in itself since then it should be possible to perform AQP over any Calcite-supported backend. If you'd be willing to discuss potential use cases further, I'd love to try to schedule a call with you. -- Michael Mior [email protected] Le lun. 7 mai 2018 à 13:30, Xiening Dai <[email protected]> a écrit : Hi Michael, AQP is valuable in our business scenarios. Often time our data scientist would issue exploratory queries to get a basic sense of the data (means, aggregation on certain groupings, etc). But from my understanding, Verdict DB is independent to the database system and the query planner/optimizer. I wonder what you want to achieve specifically through Calcite integration? On May 7, 2018, at 10:05 AM, Michael Mior <[email protected]> wrote: Edmon (and others), I'd be curious to hear more about your specific use cases if you're able to share. Especially those who have companies which may benefit from using AQP with Calcite to lower costs. -- Michael Mior [email protected] Le jeu. 3 mai 2018 à 18:58, Edmon Begoli <[email protected]> a écrit : I am excited that you are considering taking Calcite in this direction. Approximate querying and probabilistic databases are of great interest to me, and I might be able to provide some applied research scenarios. One domain that comes to mind where we had some use cases is a sensor data analysis. Thank you, Edmon On Thu, May 3, 2018 at 6:54 PM, Michael Mior <[email protected]> wrote: Hi all, I recently had a chat with the VerdictDB (http://verdictdb.org/) team about possible integration with Calcite. VerdictDB sits between an application and a database to enable the approximation of query results which are expected to be highly accurate while consuming significantly fewer resources on the backend. I'm curious to talk to anyone who might have a use case for this. Particularly those using Calcite to power analytics systems that can tolerate approximate results. We'll likely be looking at putting together a proof of concept in the next few weeks if there's any interest. Let me know! -- Michael Mior [email protected]
