Thanks for doing these experiments, Christian, and documenting what you found.
I think you’re running into the limitations of ReflectiveSchema. It works with POJOs (java classes with public fields) but hasn’t been tested for richer variations and therefore just doesn’t work. In many cases, it can be fixed (see https://issues.apache.org/jira/browse/CALCITE-6244 for example). I’m uneasy about extending RelDataType to return JavaRowFormat. That seems to be introducing a physical property into a logical data type; also, it is surfacing the details of one particular adapter (enumerable). Maybe there is a way to surface this information via annotations or the java.sql.Wrapper interface without extending RelDataType. Julian > On May 24, 2024, at 11:33 AM, Christian Beikov <christian.bei...@gmail.com> > wrote: > > Hello, > > in my recent experiments I ran into some issues when trying to unnest an > array of struct. > > The query is roughly this: select t.id, e.value1 from MyTable t, > unnest(t.structArray) e > > EnumerableTableScan#fieldExpression will then try to generate code that > converts the value of the "structArray" column to a List<List>, which is > where the problems start to arise. This code does not seem to be tested at > all, because currently generates a compile error, due to missing a cast to > "Iterable". It also assumes the data is already available in the > JavaRowFormat.CUSTOM representation, but AFAIU, it could be in any format. > > When using RelRecordType for structs, regular struct columns seem to expect > JavaRowFormat.ARRAY, but struct arrays don't seem to behave the same way. > > What is the expected data format that an enumerator should return for struct > arrays that are typed as RelRecordType? > > To support formats per type it might be nice to allow specifying the > JavaRowFormat on RelDataType. Wdyt? > > Also, is there a way to allow working with custom Java types for table/struct > rows? From looking at AbstractCursor#createAccessor, it seems the Aviatica > code currently only works with classes that expose public fields. > > Regards, > > Christian
