Good catch indeed. As Dmitry said, Metadata.Dataset needs to be updated which will also update Metadata.Index.
Cheers, Murtadha > On 25 Apr 2020, at 3:30 AM, Dmitry Lychagin > <[email protected]> wrote: > > Rui, > > Good catch! > Yes, I think the new `Synonym` dataset should be inserted into > Metadata.`Dataset`. > I'll fix it. > > Thanks, > -- Dmitry > > > On 4/24/20, 4:25 PM, "Rui Guo" <[email protected]> wrote: > > Hi, devs, > > I have a question about the backward-compatibility of the "*Synonym*" > metadata catalog. > > Assume an old data verse (which doesn't have the "Synonym" metadata > catalog) is loaded in a new AsterixDB (which supports the "Synonym" > feature), then, > > * In MetadataBootstrap, a local resource file (physical file on disk) will > be created for the Synonym metadata catalog. The related code is at > > https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java#L496 > where the MetadataPrimaryIndexes.SYNONYM_DATASET is whitelisted so that its > local resource can be created later. > > * However, in this case, it seems MetadataPrimaryIndexes.SYNONYM_DATASET is > not inserted into the Metadata.`Dataset` catalog. Note that the > "insertMetadataDatasets(mdTxnCtx, PRIMARY_INDEXES)" is called only when > "isNewUniverse", and for an old data verse, it is not called. Related code > is at > > https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java#L161 > . > > * Because MetadataPrimaryIndexes.SYNONYM_DATASET is not > in Metadata.`Dataset`, users ***cannot*** see it by selecting from > Metadata.`Dataset`. On the other hand, since the local resource file is > created correspondingly, the synonym ***feature works fine***. > > So, my question is: > * Should the Synonym dataset be inserted into Metadata.`Dataset` in the > above case where the old data verse doesn't it there? > * If so, then maybe we can add a related integration test case to guarantee > a better backward-compatibility here. > > Also, I'm now working on a new metadata catalog "FullTextEntity" for the > new full-text search features. My related codes to support > backward-compatibility where the new dataset is inserted into > Metadata.`Dataset` is at > > https://github.com/ruig2/asterixdb/blob/ruiguo/stopword--syntax/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java#L313-L330 > . > Any comments here? > > Many thanks, > Rui Guo >
