I think this issue:https://issues.apache.org/jira/browse/ASTERIXDB-1109 is related. Currently, index entries (SK, PK) are not deleted on an open-type secondary index during a deletion. This issue was not surfaced due to the fact that every search after a secondary index search had to go through the primary index lookup.
Best, Taewoo On Tue, Sep 22, 2015 at 12:04 AM, Ildar Absalyamov < [email protected]> wrote: > Abdullah, > > If I remember correctly whenever a secondary open index is created all > existing records would be casted to a proper type to ensure that the index > creation is valid. > As for the overall correctness of casting operation, semantically creating > an open index is the same thing as altering the dataset type. The current > implementation allows only one open index of particular type created on a > single field. If we would have had “alter datatype” functionality the open > indexing would not be required at all. > > > On Sep 21, 2015, at 23:25, abdullah alamoudi <[email protected]> wrote: > > > > More thoughts: > > I assume the intention of the cast was just to make sure if the open > field > > exists, it is of the specified type. Moreover, the un-casted record > should > > be inserted into the index. > > If my assumptions are not correct, please, let me know ASAP. > > > > I have two thoughts on this: > > 1. Actually, insert plans show that the records being inserted into the > > primary index is actually the casted record creating the issue described > > above. > > > > 2. I don't believe this is the right way to ensure that the open field if > > exists is of the right type. why not extract the field using field access > > by name function and then verify the type using the field tag? > > > > > > > > On Tue, Sep 22, 2015 at 9:11 AM, abdullah alamoudi <[email protected]> > > wrote: > > > >> Hi Dev, @Ildar, > >> > >> In the insert pipeline for datasets with open indexes, we introduce a > cast > >> function before the insert and so one would expect the records to look > like > >> the casted record type which I assume has {{the closed fields + a > nullable > >> field}}. > >> > >> The question is, what happens to the previously existing records?, since > >> now the index has both, records of the original type and records of the > >> casted type. > >> > >> Thanks, > >> Abdullah. > >> > > Best regards, > Ildar > >
