Sounds kinda bad! Also, I wonder what happens when the compiler
encounters records in the dataset - whose type in the catalog doesn't
claim to have a given (but now indexed) open field - e.g., during a data
scan or an access via some other path? Can Bad Things Happen due to the
compiler not properly anticipating the casted form of the records?
(Maybe I am misunderstanding something, but we should probably take a
careful look at the test cases - and make sure we do things like add a
bunch of records, then add such an index, then add some more records,
then stress-test type-related things that come at the dataset (i) thru
the index, (ii) thru a primary dataset scan, and (iii) thru some other
index.)
On 9/22/15 4:06 PM, Taewoo Kim wrote:
I think this issue:https://issues.apache.org/jira/browse/ASTERIXDB-1109 is
related. Currently, index entries (SK, PK) are not deleted on an open-type
secondary index during a deletion. This issue was not surfaced due to the
fact that every search after a secondary index search had to go through the
primary index lookup.
Best,
Taewoo
On Tue, Sep 22, 2015 at 12:04 AM, Ildar Absalyamov <
[email protected]> wrote:
Abdullah,
If I remember correctly whenever a secondary open index is created all
existing records would be casted to a proper type to ensure that the index
creation is valid.
As for the overall correctness of casting operation, semantically creating
an open index is the same thing as altering the dataset type. The current
implementation allows only one open index of particular type created on a
single field. If we would have had “alter datatype” functionality the open
indexing would not be required at all.
On Sep 21, 2015, at 23:25, abdullah alamoudi <[email protected]> wrote:
More thoughts:
I assume the intention of the cast was just to make sure if the open
field
exists, it is of the specified type. Moreover, the un-casted record
should
be inserted into the index.
If my assumptions are not correct, please, let me know ASAP.
I have two thoughts on this:
1. Actually, insert plans show that the records being inserted into the
primary index is actually the casted record creating the issue described
above.
2. I don't believe this is the right way to ensure that the open field if
exists is of the right type. why not extract the field using field access
by name function and then verify the type using the field tag?
On Tue, Sep 22, 2015 at 9:11 AM, abdullah alamoudi <[email protected]>
wrote:
Hi Dev, @Ildar,
In the insert pipeline for datasets with open indexes, we introduce a
cast
function before the insert and so one would expect the records to look
like
the casted record type which I assume has {{the closed fields + a
nullable
field}}.
The question is, what happens to the previously existing records?, since
now the index has both, records of the original type and records of the
casted type.
Thanks,
Abdullah.
Best regards,
Ildar
