I appreciate the responses. I'll give that a go based on your suggestions. Thank you
On Mon, 7 Nov 2022 at 22:57, Julian Hyde <[email protected]> wrote: > Sorry, rather busy so my answer will be partial. > > The easiest thing to put into DataContext is a table. If you can make your > ‘object’ look like a table and wire it into the schema/table hierarchy then > you can get it from the schema tree. > > You can’t officially share objects between executions. But you can use > your JSON string as a key to a cache (say a static Guava cache), so that > when you deserialize your JSON string you will end up with the same object > as when you last executed the query. > > > On Nov 7, 2022, at 2:51 PM, Courtney Robinson <[email protected]> wrote: > > > > Okay, I understand the reasoning. > >> > >> If you need complex non-primitive objects a really good pattern is to > pass > >> in a string representation of that object (say JSON) and then build the > >> objects using ‘root’ as a directory service. > >> > > I actually thought something like that was possible but I couldn't figure > > out how to get anything into the DataContext root parameter. > > I was successfully generating code like this > > (org.apache.lucene.search.Query)root.get("my_query") > > but couldn't figure out how to put query into DataContext root. > > > > In debug I saw that it was an anonymous class from inside > > CalciteConnectionImpl (this was with a JDBC based setup). Knowing what I > > know now, I guess that means it wraps the connection params? I'll > checkout > > the code later. > > > > I've since changed and followed the setup from the BOSS 21 lucene video > as > > I needed more control over the Lex and other options. > > > > What I have now was the other way I thought about doing it before > > sending that email (forgot to include it) > > > > When I create the planner I create a Context that when unwrap gets > called, > > it returns MyDBContext > > > >> public static class MyDBPlannerCtx { > >> private final Map<MyDBTable, QueryBuilder> builders = new > LinkedHashMap<>(); > >> > >> public void put(MyDBTable tbl, QueryBuilder builder) { > >> builders.put(tbl, builder); > >> } > >> > >> public QueryBuilder get(MyDBTable tbl) { > >> return builders.get(tbl); > >> } > >> } > >> > >> In the EnumerableConverter I then just do > > > > MyDBPlannerCtx ctx = getInput().getCluster() .getPlanner() .getContext() > >> .unwrap(MyDBPlannerCtx.class); > >> ctx.put(implementer.table, implementer.query); > > > > > > Later in the Queryable it has getTable().find() and in MyTable.find I > just > > do ctx.get(this) to get the Query > > > > This works...I get the right Query but there are a lot of unknowns...it's > > why I sent the first message. > > Right now, each query in the tests creates a planner and all the objects > > used so there's no chance of there being two connections (when I get to > > that point) getting their Query mixed up...the issue I see now is I don't > > feel like all these objects should be re-created for each query but it is > > not clear what the lifecycle of everything should be and what objects can > > be safely kept and used for multiple queries. > > > > Rather long winded but it boils down to the fact I have two ways of doing > > this. > > > > 1. As you said, JSON or similar combined with DataContext root - how do > > I put info into DataContext? > > 2. Is it safe to do the plan context as I have it working now and are > > any of these objects safe to re-use between two queries and or threads? > > 1. > > CalciteSchema, RelDataTypeFactory, CalciteCatalogReader, > SqlValidator > > 2. I'm thinking a single instance of these per catalog > > > > > > On Mon, 7 Nov 2022 at 21:44, Julian Hyde <[email protected]> wrote: > > > >> The goal of Enumerable convention is to create a query plan that is Java > >> source code. Along with that goes the idea that the plan can be run by > >> something like a ‘public static void main’ method, where the only > >> parameters are things you could pass from the command-line. > >> > >> We’re not literally that strict, but that should give you an idea of > what > >> we’re striving for. By not passing objects around we are simplifying > things > >> like running the code in a debugger, running the same plan several times > >> and/or in parallel. > >> > >> The one non-primitive argument is the ‘DataContext root’ parameter. This > >> is a map that contains all of the objects we need. It acts as a > directory > >> service, so we can look up any Table objects based on their path > >> (schema1.schema2.myTable). > >> > >> If you need complex non-primitive objects a really good pattern is to > pass > >> in a string representation of that object (say JSON) and then build the > >> objects using ‘root’ as a directory service. > >> > >> Julian > >> > >>> On Nov 7, 2022, at 1:32 PM, Courtney Robinson <[email protected]> > wrote: > >>> > >>> Hello all! > >>> We're looking at Calcite and I've watched the video of BOSS 21 > >>> with Stamatis and Julian. > >>> > >>> All the examples I've seen, including the lucene example from that > video > >>> and others like the ES adapter all end up serialising their values that > >> go > >>> into the generated code. > >>> > >>> ConstantExpression luceneQuery = Expressions.constant(((LuceneRel) > >>> input).implement().query*.toString()*); > >>> > >>> See the toString() here. What I am wondering and haven't been able to > >>> figure out is how do I pass the generated lucene Query object rather > >> than a > >>> String form of it? > >>> > >>> I've tried seeing if there was some common object and found the > >> DataContext > >>> that is available in the generated code but there's no way to add a > value > >>> into the DataContext from what I can see. > >>> I tried Expressions.dynamic but it's not implemented, couldn't quite > get > >>> Expressions.lambda to work either, throws NPE because `body` is > missing? > >>> > >>> I want to stick to one question here since I have a few but this is > >>> related. > >>> Is it possible to avoid the code generation here altogether? In the > basic > >>> lucene demo, the AbstractEnumerable is extended > >>> and Linq4j.enumerator(searchIndex()) gets returned from enumerator(). > >>> > >>> I guess what I'm trying to ask is if I can use what seems like the > >> simpler > >>> API (returning an enumerator) whilst providing the rules the other > >> example > >>> uses? > >>> > >>> Repo here for those who don't know what I'm referring to > >>> > >> > https://github.com/zabetak/calcite-tutorial/blob/main/solution/src/main/java/com/github/zabetak/calcite/tutorial/LuceneEnumerable.java#L71 > >>> > >>> > >>> I'm literally extending the lucene tutorial example so replies in that > >>> context (getting the generated lucene Query object passed around) are > >>> welcome and it's not far of what we're going to need to do later. > >>> > >>> -- > >>> Regards, > >>> Courtney - CEO, Hypi <https://hypi.io/custom-software-development/> > >>> Tel: 020 8123 2413 > >> > >> > > > > -- > > Regards, > > Courtney - CEO, Hypi <https://hypi.io/custom-software-development/> > > Tel: 020 8123 2413 > > -- Regards, Courtney - CEO, Hypi <https://hypi.io/custom-software-development/> Tel: 020 8123 2413
