@Ildar: Yes. The current implementation requires that. So, I asked whether which one makes sense.
Best, Taewoo On Fri, Jul 14, 2017 at 12:09 AM, Ildar Absalyamov < [email protected]> wrote: > Taewoo, > > You’ve correctly identified the issue here: to make use of an enforced > index we must cast the record to a particular type, which is imposed by the > index. > > So, using your example, if we have an index on path “nested.one.title” the > indexed record must be castable to {…, “nested”: {…,”one”: {…,”title”: > string, …}, ...},…}. > As you have observed a case when there is no “nested” field in the > top-level type leads to exception, because it relies of a fact that there > is a compile-time type information for a field “nested”. This type > information is used to build a type for aforementioned cast operator. > Form the perspective of current implementation a runtime exception is a > bug, instead it should have caught this issue during compile time. > > > On Jul 13, 2017, at 23:10, Taewoo Kim <[email protected]> wrote: > > > > @Yingyi: thanks. > > > > @Mike: Yeah. My problem is how to associate the field type information. > > Ideally, the leaf level has the field to type hash map and the parent of > it > > has that hashmap in its record type. And its parent needs to have the > > necessary information to reach to this record type. If we don't need any > > pre-defined type at each level to create a multi-level enforced index, > then > > things will become more complex to me. :-) Anyway, we can discuss further > > to finalize the field type propagation implementation. > > > > Best, > > Taewoo > > > > On Thu, Jul 13, 2017 at 11:02 PM, Mike Carey <[email protected]> wrote: > > > >> Taewoo, > >> > >> To clarify further what should work: > >> - We should support nested indexes that go down multiple levels. > >> - We should (ideally) support their use in index-NL joins. > >> > >> Reflecting on our earlier conversation(s), I think I can see why you're > >> asking this. :-) The augmented type information that'll be needed to do > >> this completely/properly will actually have to associate types with > field > >> paths (not just with fields by name) - which is a slightly more > complicated > >> association. > >> > >> Cheers, > >> Mike > >> > >> > >> On 7/13/17 10:54 PM, Yingyi Bu wrote: > >> > >>> Hi Taewoo, > >>> > >>> The first query shouldn't fail because indexnl is just a hint. > >>> The second query should succeed because it's a valid indexing > statement. > >>> High nesting levels in open record like JSON is not uncommon. > >>> > >>> Best, > >>> Yingyi > >>> > >>> > >>> On Thu, Jul 13, 2017 at 10:51 PM, Taewoo Kim <[email protected]> > wrote: > >>> > >>> @Mike: In order to properly deal with the enforced index on a > nested-type > >>>> field, I need to make sure that whether my understanding (each nested > >>>> type > >>>> (except the leaf level0 has a record type for the next level) is > correct > >>>> or > >>>> not. Which one is a bug? The first one (without index) should fail? Or > >>>> the > >>>> second one (with an index) should succeed? > >>>> > >>>> Best, > >>>> Taewoo > >>>> > >>>> On Thu, Jul 13, 2017 at 9:58 PM, Yingyi Bu <[email protected]> > wrote: > >>>> > >>>> Indeed, it's a bug! > >>>>> > >>>>> Best, > >>>>> Yingyi > >>>>> > >>>>> On Thu, Jul 13, 2017 at 9:52 PM, Mike Carey <[email protected]> > wrote: > >>>>> > >>>>> Sounds like a bug to me. > >>>>>> > >>>>>> > >>>>>> > >>>>>> On 7/13/17 7:59 PM, Taewoo Kim wrote: > >>>>>> > >>>>>> Currently, I am working on a field type propagation without using > >>>>>>> initializing the OptimizableSubTree in the current index access > >>>>>>> > >>>>>> method. > >>>> > >>>>> I > >>>>> > >>>>>> am encountering an issue with an open-type enforced index. So, I > just > >>>>>>> > >>>>>> want > >>>>> > >>>>>> to make sure that my understanding is correct. It looks like we > can't > >>>>>>> > >>>>>> have > >>>>> > >>>>>> an enforced-index on a completely schemaless nested field. For > >>>>>>> > >>>>>> example, > >>>> > >>>>> the > >>>>>>> following doesn't generate any issue. > >>>>>>> > >>>>>>> // > >>>>>>> create type DBLPType as open {id: int32} > >>>>>>> create type CSXType as closed {id: int32} > >>>>>>> > >>>>>>> create dataset DBLP(DBLPType) primary key id; > >>>>>>> create dataset CSX(CSXType) primary key id; > >>>>>>> > >>>>>>> for $a in dataset('DBLP') > >>>>>>> for $b in dataset('CSX') > >>>>>>> where $a.nested.one.title /*+ indexnl */ = $b.nested.one.title > >>>>>>> return {"arec": $a, "brec": $b} > >>>>>>> // > >>>>>>> > >>>>>>> However, the following generates an exception. So, can we assume > that > >>>>>>> > >>>>>> to > >>>> > >>>>> create an enforced-index, except the leaf level, there should be a > >>>>>>> > >>>>>> defined > >>>>> > >>>>>> record type. For example, for this example, there should be "nested" > >>>>>>> > >>>>>> type > >>>>> > >>>>>> and "one" type. > >>>>>>> > >>>>>>> // > >>>>>>> create type DBLPType as open {id: int32} > >>>>>>> create type CSXType as closed {id: int32} > >>>>>>> > >>>>>>> create dataset DBLP(DBLPType) primary key id; > >>>>>>> create dataset CSX(CSXType) primary key id; > >>>>>>> > >>>>>>> create index title_index_DBLP on DBLP(nested.one.title: string?) > >>>>>>> > >>>>>> enforced; > >>>>> > >>>>>> create index title_index_CSX on CSX(nested.one.title: string?) > >>>>>>> > >>>>>> enforced; > >>>> > >>>>> for $a in dataset('DBLP') > >>>>>>> for $b in dataset('CSX') > >>>>>>> where $a.nested.one.title /*+ indexnl */ = $b.nested.one.title > >>>>>>> return {"arec": $a, "brec": $b} > >>>>>>> // > >>>>>>> > >>>>>>> Best, > >>>>>>> Taewoo > >>>>>>> > >>>>>>> > >>>>>>> > >> > > Best regards, > Ildar > >
