Taewoo,
You’ve correctly identified the issue here: to make use of an enforced index we
must cast the record to a particular type, which is imposed by the index.
So, using your example, if we have an index on path “nested.one.title” the
indexed record must be castable to {…, “nested”: {…,”one”: {…,”title”: string,
…}, ...},…}.
As you have observed a case when there is no “nested” field in the top-level
type leads to exception, because it relies of a fact that there is a
compile-time type information for a field “nested”. This type information is
used to build a type for aforementioned cast operator.
Form the perspective of current implementation a runtime exception is a bug,
instead it should have caught this issue during compile time.
> On Jul 13, 2017, at 23:10, Taewoo Kim <[email protected]> wrote:
>
> @Yingyi: thanks.
>
> @Mike: Yeah. My problem is how to associate the field type information.
> Ideally, the leaf level has the field to type hash map and the parent of it
> has that hashmap in its record type. And its parent needs to have the
> necessary information to reach to this record type. If we don't need any
> pre-defined type at each level to create a multi-level enforced index, then
> things will become more complex to me. :-) Anyway, we can discuss further
> to finalize the field type propagation implementation.
>
> Best,
> Taewoo
>
> On Thu, Jul 13, 2017 at 11:02 PM, Mike Carey <[email protected]> wrote:
>
>> Taewoo,
>>
>> To clarify further what should work:
>> - We should support nested indexes that go down multiple levels.
>> - We should (ideally) support their use in index-NL joins.
>>
>> Reflecting on our earlier conversation(s), I think I can see why you're
>> asking this. :-) The augmented type information that'll be needed to do
>> this completely/properly will actually have to associate types with field
>> paths (not just with fields by name) - which is a slightly more complicated
>> association.
>>
>> Cheers,
>> Mike
>>
>>
>> On 7/13/17 10:54 PM, Yingyi Bu wrote:
>>
>>> Hi Taewoo,
>>>
>>> The first query shouldn't fail because indexnl is just a hint.
>>> The second query should succeed because it's a valid indexing statement.
>>> High nesting levels in open record like JSON is not uncommon.
>>>
>>> Best,
>>> Yingyi
>>>
>>>
>>> On Thu, Jul 13, 2017 at 10:51 PM, Taewoo Kim <[email protected]> wrote:
>>>
>>> @Mike: In order to properly deal with the enforced index on a nested-type
>>>> field, I need to make sure that whether my understanding (each nested
>>>> type
>>>> (except the leaf level0 has a record type for the next level) is correct
>>>> or
>>>> not. Which one is a bug? The first one (without index) should fail? Or
>>>> the
>>>> second one (with an index) should succeed?
>>>>
>>>> Best,
>>>> Taewoo
>>>>
>>>> On Thu, Jul 13, 2017 at 9:58 PM, Yingyi Bu <[email protected]> wrote:
>>>>
>>>> Indeed, it's a bug!
>>>>>
>>>>> Best,
>>>>> Yingyi
>>>>>
>>>>> On Thu, Jul 13, 2017 at 9:52 PM, Mike Carey <[email protected]> wrote:
>>>>>
>>>>> Sounds like a bug to me.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 7/13/17 7:59 PM, Taewoo Kim wrote:
>>>>>>
>>>>>> Currently, I am working on a field type propagation without using
>>>>>>> initializing the OptimizableSubTree in the current index access
>>>>>>>
>>>>>> method.
>>>>
>>>>> I
>>>>>
>>>>>> am encountering an issue with an open-type enforced index. So, I just
>>>>>>>
>>>>>> want
>>>>>
>>>>>> to make sure that my understanding is correct. It looks like we can't
>>>>>>>
>>>>>> have
>>>>>
>>>>>> an enforced-index on a completely schemaless nested field. For
>>>>>>>
>>>>>> example,
>>>>
>>>>> the
>>>>>>> following doesn't generate any issue.
>>>>>>>
>>>>>>> //
>>>>>>> create type DBLPType as open {id: int32}
>>>>>>> create type CSXType as closed {id: int32}
>>>>>>>
>>>>>>> create dataset DBLP(DBLPType) primary key id;
>>>>>>> create dataset CSX(CSXType) primary key id;
>>>>>>>
>>>>>>> for $a in dataset('DBLP')
>>>>>>> for $b in dataset('CSX')
>>>>>>> where $a.nested.one.title /*+ indexnl */ = $b.nested.one.title
>>>>>>> return {"arec": $a, "brec": $b}
>>>>>>> //
>>>>>>>
>>>>>>> However, the following generates an exception. So, can we assume that
>>>>>>>
>>>>>> to
>>>>
>>>>> create an enforced-index, except the leaf level, there should be a
>>>>>>>
>>>>>> defined
>>>>>
>>>>>> record type. For example, for this example, there should be "nested"
>>>>>>>
>>>>>> type
>>>>>
>>>>>> and "one" type.
>>>>>>>
>>>>>>> //
>>>>>>> create type DBLPType as open {id: int32}
>>>>>>> create type CSXType as closed {id: int32}
>>>>>>>
>>>>>>> create dataset DBLP(DBLPType) primary key id;
>>>>>>> create dataset CSX(CSXType) primary key id;
>>>>>>>
>>>>>>> create index title_index_DBLP on DBLP(nested.one.title: string?)
>>>>>>>
>>>>>> enforced;
>>>>>
>>>>>> create index title_index_CSX on CSX(nested.one.title: string?)
>>>>>>>
>>>>>> enforced;
>>>>
>>>>> for $a in dataset('DBLP')
>>>>>>> for $b in dataset('CSX')
>>>>>>> where $a.nested.one.title /*+ indexnl */ = $b.nested.one.title
>>>>>>> return {"arec": $a, "brec": $b}
>>>>>>> //
>>>>>>>
>>>>>>> Best,
>>>>>>> Taewoo
>>>>>>>
>>>>>>>
>>>>>>>
>>
Best regards,
Ildar
