All I know is in that case, I'm afraid. I surmise that Darion (who logged CALCITE-1737) wanted to change Calcite's Spark adapter to use the newer Dataframe/Dataset Spark API rather than the older RDD API.
In my opinion, the Spark adapter needs some love and testing far more than it needs to be rewritten for a new API. As is the case with most Calcite issues, there are no resources committed to fixing CALCITE-1737. Contributions are welcome. Julian On Fri, Jan 19, 2018 at 7:21 PM, Eric Vandenberg <[email protected]> wrote: > Hello, > > I’m researching big data projects and noticed Apache Calcite, looks like an > exciting SQL/framework with lots of potential in connecting different compute > engines including Spark. > > In particular, was curious about Calcite support for Dataframe/Dataset > operators such as map/filter/groupby/etc functional transformations. It > looks like there’s a project jira > (https://issues.apache.org/jira/browse/CALCITE-1737)<https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_CALCITE-2D1737-29&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=TjYOasVCvqxoFOP7yKFkysvpmQmHm5V7VcSlTkeo5Ug&m=FSaBByPogJOgk77YhJ89yI4mcmNF6QbCv64BM3Fg_Wk&s=f3iicLVyPjbz9S-MYmMQ0_QTmj1yCjSNDPRcS-FlIss&e=> > for this, but thought I’d ask if there is any update/progress on plans in > this area. Is this something that’s firmly in the roadmap for Calcite / > relative priority? Is this something there’s been a lot of requests/asks > for? Are there any design proposals, understanding of technical > blockers/issues to add such support? > > Appreciate any information/insight you’re able to share. > > Thanks, > Eric >
