Agreed.
Best,
Taewoo
On Fri, Jul 14, 2017 at 4:09 PM, Yingyi Bu <buyin...@gmail.com> wrote:
> >> When we are encounter a field (“nested”) for which the is no
> compile-time information
> >> we should assume that the type of this field is completely open, i.e.,
> {}, and pass it down the chain.
>
> Correct, since it's enforced.
> The augmented enforced type maps should be recursively added into those
> nested ARecordType.
>
> Best,
> Yingyi
>
>
> On Fri, Jul 14, 2017 at 12:13 AM, Ildar Absalyamov <
> ildar.absalya...@gmail.com> wrote:
>
> > However, there should be a way to deal with this issue when the top-level
> > type is open.
> >
> > create type DBLPType as open {id: int32}
> > create index title_index_DBLP on DBLP(nested.one.title: string?)
> enforced;
> >
> > When we are encounter a field (“nested”) for which the is no compile-time
> > information we should assume that the type of this field is completely
> > open, i.e., {}, and pass it down the chain.
> >
> > > On Jul 14, 2017, at 00:09, Ildar Absalyamov <
> ildar.absalya...@gmail.com>
> > wrote:
> > >
> > > Taewoo,
> > >
> > > You’ve correctly identified the issue here: to make use of an enforced
> > index we must cast the record to a particular type, which is imposed by
> the
> > index.
> > >
> > > So, using your example, if we have an index on path “nested.one.title”
> > the indexed record must be castable to {…, “nested”: {…,”one”:
> {…,”title”:
> > string, …}, ...},…}.
> > > As you have observed a case when there is no “nested” field in the
> > top-level type leads to exception, because it relies of a fact that there
> > is a compile-time type information for a field “nested”. This type
> > information is used to build a type for aforementioned cast operator.
> > > Form the perspective of current implementation a runtime exception is a
> > bug, instead it should have caught this issue during compile time.
> > >
> > >> On Jul 13, 2017, at 23:10, Taewoo Kim <wangs...@gmail.com> wrote:
> > >>
> > >> @Yingyi: thanks.
> > >>
> > >> @Mike: Yeah. My problem is how to associate the field type
> information.
> > >> Ideally, the leaf level has the field to type hash map and the parent
> > of it
> > >> has that hashmap in its record type. And its parent needs to have the
> > >> necessary information to reach to this record type. If we don't need
> any
> > >> pre-defined type at each level to create a multi-level enforced index,
> > then
> > >> things will become more complex to me. :-) Anyway, we can discuss
> > further
> > >> to finalize the field type propagation implementation.
> > >>
> > >> Best,
> > >> Taewoo
> > >>
> > >> On Thu, Jul 13, 2017 at 11:02 PM, Mike Carey <dtab...@gmail.com>
> wrote:
> > >>
> > >>> Taewoo,
> > >>>
> > >>> To clarify further what should work:
> > >>> - We should support nested indexes that go down multiple levels.
> > >>> - We should (ideally) support their use in index-NL joins.
> > >>>
> > >>> Reflecting on our earlier conversation(s), I think I can see why
> you're
> > >>> asking this. :-) The augmented type information that'll be needed to
> do
> > >>> this completely/properly will actually have to associate types with
> > field
> > >>> paths (not just with fields by name) - which is a slightly more
> > complicated
> > >>> association.
> > >>>
> > >>> Cheers,
> > >>> Mike
> > >>>
> > >>>
> > >>> On 7/13/17 10:54 PM, Yingyi Bu wrote:
> > >>>
> > >>>> Hi Taewoo,
> > >>>>
> > >>>> The first query shouldn't fail because indexnl is just a hint.
> > >>>> The second query should succeed because it's a valid indexing
> > statement.
> > >>>> High nesting levels in open record like JSON is not uncommon.
> > >>>>
> > >>>> Best,
> > >>>> Yingyi
> > >>>>
> > >>>>
> > >>>> On Thu, Jul 13, 2017 at 10:51 PM, Taewoo Kim <wangs...@gmail.com>
> > wrote:
> > >>>>
> > >>>> @Mike: In order to properly deal with the enforced index on a
> > nested-type
> > >>>>> field, I need to make sure that whether my understanding (each
> nested
> > >>>>> type
> > >>>>> (except the leaf level0 has a record type for the next level) is
> > correct
> > >>>>> or
> > >>>>> not. Which one is a bug? The first one (without index) should fail?
> > Or
> > >>>>> the
> > >>>>> second one (with an index) should succeed?
> > >>>>>
> > >>>>> Best,
> > >>>>> Taewoo
> > >>>>>
> > >>>>> On Thu, Jul 13, 2017 at 9:58 PM, Yingyi Bu <buyin...@gmail.com>
> > wrote:
> > >>>>>
> > >>>>> Indeed, it's a bug!
> > >>>>>>
> > >>>>>> Best,
> > >>>>>> Yingyi
> > >>>>>>
> > >>>>>> On Thu, Jul 13, 2017 at 9:52 PM, Mike Carey <dtab...@gmail.com>
> > wrote:
> > >>>>>>
> > >>>>>> Sounds like a bug to me.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On 7/13/17 7:59 PM, Taewoo Kim wrote:
> > >>>>>>>
> > >>>>>>> Currently, I am working on a field type propagation without using
> > >>>>>>>> initializing the OptimizableSubTree in the current index access
> > >>>>>>>>
> > >>>>>>> method.
> > >>>>>
> > >>>>>> I
> > >>>>>>
> > >>>>>>> am encountering an issue with an open-type enforced index. So, I
> > just
> > >>>>>>>>
> > >>>>>>> want
> > >>>>>>
> > >>>>>>> to make sure that my understanding is correct. It looks like we
> > can't
> > >>>>>>>>
> > >>>>>>> have
> > >>>>>>
> > >>>>>>> an enforced-index on a completely schemaless nested field. For
> > >>>>>>>>
> > >>>>>>> example,
> > >>>>>
> > >>>>>> the
> > >>>>>>>> following doesn't generate any issue.
> > >>>>>>>>
> > >>>>>>>> //
> > >>>>>>>> create type DBLPType as open {id: int32}
> > >>>>>>>> create type CSXType as closed {id: int32}
> > >>>>>>>>
> > >>>>>>>> create dataset DBLP(DBLPType) primary key id;
> > >>>>>>>> create dataset CSX(CSXType) primary key id;
> > >>>>>>>>
> > >>>>>>>> for $a in dataset('DBLP')
> > >>>>>>>> for $b in dataset('CSX')
> > >>>>>>>> where $a.nested.one.title /*+ indexnl */ = $b.nested.one.title
> > >>>>>>>> return {"arec": $a, "brec": $b}
> > >>>>>>>> //
> > >>>>>>>>
> > >>>>>>>> However, the following generates an exception. So, can we assume
> > that
> > >>>>>>>>
> > >>>>>>> to
> > >>>>>
> > >>>>>> create an enforced-index, except the leaf level, there should be a
> > >>>>>>>>
> > >>>>>>> defined
> > >>>>>>
> > >>>>>>> record type. For example, for this example, there should be
> > "nested"
> > >>>>>>>>
> > >>>>>>> type
> > >>>>>>
> > >>>>>>> and "one" type.
> > >>>>>>>>
> > >>>>>>>> //
> > >>>>>>>> create type DBLPType as open {id: int32}
> > >>>>>>>> create type CSXType as closed {id: int32}
> > >>>>>>>>
> > >>>>>>>> create dataset DBLP(DBLPType) primary key id;
> > >>>>>>>> create dataset CSX(CSXType) primary key id;
> > >>>>>>>>
> > >>>>>>>> create index title_index_DBLP on DBLP(nested.one.title: string?)
> > >>>>>>>>
> > >>>>>>> enforced;
> > >>>>>>
> > >>>>>>> create index title_index_CSX on CSX(nested.one.title: string?)
> > >>>>>>>>
> > >>>>>>> enforced;
> > >>>>>
> > >>>>>> for $a in dataset('DBLP')
> > >>>>>>>> for $b in dataset('CSX')
> > >>>>>>>> where $a.nested.one.title /*+ indexnl */ = $b.nested.one.title
> > >>>>>>>> return {"arec": $a, "brec": $b}
> > >>>>>>>> //
> > >>>>>>>>
> > >>>>>>>> Best,
> > >>>>>>>> Taewoo
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>
> > >
> > > Best regards,
> > > Ildar
> > >
> >
> > Best regards,
> > Ildar
> >
> >
>
