The “file” adapter has some support for REST. Naturally the file adapter can read files of various formats (.csv, .json, compressed .csv, etc.) from the file system. But it can also read those same kinds of files over HTTP, and read data sets in HTML tables [1].
The logical next step is to allow parameters to those HTTP requests, and a way to convert filters, projects, sorts into parameters. There’s a jira case [2] where we can capture requirements. Contributions welcome. Julian [1] https://issues.apache.org/jira/browse/CALCITE-884 [2] https://issues.apache.org/jira/browse/CALCITE-4035 > On Oct 25, 2023, at 6:46 AM, Charles Givre <cgi...@gmail.com> wrote: > > Hi David, > I don't know if this is helpful or not, but we did this for Apache Drill. > (https://github.com/apache/drill/tree/master/contrib/storage-http) Over the > years it became extremely powerful, but I will say that it gets really > complicated, specifically dealing with pushdowns, OAuth, pagination etc. > I'd be happy to share our experiences as Drill does use Calcite for query > planning. > Best, > -- C > > >> On Oct 25, 2023, at 4:53 AM, David Radley <david_rad...@uk.ibm.com> wrote: >> >> Hi, >> We are looking at driving rest calls underneath SQL, for Apache Flink - >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-233%3A+Introduce+HTTP+Connector >> >> The discussion mentions >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-233%3A+Introduce+HTTP+Connector >> >> I wondered whether it would make sense to have a http rest implementation at >> the Calcite layer. I see that you have a Kafka folder, would it make sense >> to have an equivalent http rest folder, that would map json GET / POST json >> payloads to sql. Or does this seem like the wrong layer? >> >> WDYT? >> >> Kind regards, David >> >> >> >> Unless otherwise stated above: >> >> IBM United Kingdom Limited >> Registered in England and Wales with number 741598 >> Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU >
