[jira] [Updated] (CARBONDATA-4169) Secondary Index coarse grain datamap to work for existing Secondary Index tables
[ https://issues.apache.org/jira/browse/CARBONDATA-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venugopal Reddy K updated CARBONDATA-4169: -- Attachment: Leverage Secondary Indexes with Presto Queries.pdf > Secondary Index coarse grain datamap to work for existing Secondary Index > tables > > > Key: CARBONDATA-4169 > URL: https://issues.apache.org/jira/browse/CARBONDATA-4169 > Project: CarbonData > Issue Type: Sub-task > Components: core, spark-integration >Reporter: Venugopal Reddy K >Priority: Minor > Attachments: Leverage Secondary Indexes with Presto Queries.pdf > > > With [[https://github.com/apache/carbondata/pull/4110]] user has to delete > all segments from the older version of existing secondary index table and > reload/sync all segments again to use them in the seconday index coarse grain > datamap. This restriction is due to the fact that PR expects > CarbonCommonConstants.INDEX_STATUS property for the index to be set to > IndexStatus.ENABLED in the index metadata when main table and respective > secondary index table segments are in sync. And IndexChooser picks only > IndexStatus.ENABLED indexes in the datamap pruning. It applies to bloom, > lucene as well. > But this property was not set for the secondary indexes in the older versions > as they were not part of datamap pruning. > > To support the secondary index coarse grain datamap pruning for existing > secondary indexes, need to support a way it happens automatically. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-4169) Secondary Index coarse grain datamap to work for existing Secondary Index tables
[ https://issues.apache.org/jira/browse/CARBONDATA-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venugopal Reddy K updated CARBONDATA-4169: -- Description: With [[https://github.com/apache/carbondata/pull/4110]] user has to delete all segments from the older version of existing secondary index table and reload/sync all segments again to use them in the seconday index coarse grain datamap. This restriction is due to the fact that PR expects CarbonCommonConstants.INDEX_STATUS property for the index to be set to IndexStatus.ENABLED in the index metadata when main table and respective secondary index table segments are in sync. And IndexChooser picks only IndexStatus.ENABLED indexes in the datamap pruning. It applies to bloom, lucene as well. But this property was not set for the secondary indexes in the older versions as they were not part of datamap pruning. To support the secondary index coarse grain datamap pruning for existing secondary indexes, need to support a way it happens automatically. was: With [[https://github.com/apache/carbondata/pull/4110],] user has to delete all segments from the older version of existing secondary index table and reload/sync all segments again to use them in the seconday index coarse grain datamap. This restriction is due to the fact that PR expects CarbonCommonConstants.INDEX_STATUS property for the index to be set to IndexStatus.ENABLED in the index metadata when main table and respective secondary index table segments are in sync. And IndexChooser picks only IndexStatus.ENABLED indexes in the datamap pruning. It applies to bloom, lucene as well. But this property was not set for the secondary indexes in the older versions as they were not part of datamap pruning. To support the secondary index coarse grain datamap pruning for existing secondary indexes, need to support a way it happens automatically. > Secondary Index coarse grain datamap to work for existing Secondary Index > tables > > > Key: CARBONDATA-4169 > URL: https://issues.apache.org/jira/browse/CARBONDATA-4169 > Project: CarbonData > Issue Type: Sub-task > Components: core, spark-integration >Reporter: Venugopal Reddy K >Priority: Minor > > With [[https://github.com/apache/carbondata/pull/4110]] user has to delete > all segments from the older version of existing secondary index table and > reload/sync all segments again to use them in the seconday index coarse grain > datamap. This restriction is due to the fact that PR expects > CarbonCommonConstants.INDEX_STATUS property for the index to be set to > IndexStatus.ENABLED in the index metadata when main table and respective > secondary index table segments are in sync. And IndexChooser picks only > IndexStatus.ENABLED indexes in the datamap pruning. It applies to bloom, > lucene as well. > But this property was not set for the secondary indexes in the older versions > as they were not part of datamap pruning. > > To support the secondary index coarse grain datamap pruning for existing > secondary indexes, need to support a way it happens automatically. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-4169) Secondary Index coarse grain datamap to work for existing Secondary Index tables
[ https://issues.apache.org/jira/browse/CARBONDATA-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venugopal Reddy K updated CARBONDATA-4169: -- Description: With [[https://github.com/apache/carbondata/pull/4110],] user has to delete all segments from the older version of existing secondary index table and reload/sync all segments again to use them in the seconday index coarse grain datamap. This restriction is due to the fact that PR expects CarbonCommonConstants.INDEX_STATUS property for the index to be set to IndexStatus.ENABLED in the index metadata when main table and respective secondary index table segments are in sync. And IndexChooser picks only IndexStatus.ENABLED indexes in the datamap pruning. It applies to bloom, lucene as well. But this property was not set for the secondary indexes in the older versions as they were not part of datamap pruning. To support the secondary index coarse grain datamap pruning for existing secondary indexes, need to support a way it happens automatically. was: With [PR 4110|[https://github.com/apache/carbondata/pull/4110],] user has to delete all segments from the older version of existing secondary index table and reload/sync all segments again to use them in the seconday index coarse grain datamap. This restriction is due to the fact that PR expects CarbonCommonConstants.INDEX_STATUS property for the index to be set to IndexStatus.ENABLED in the index metadata when main table and respective secondary index table segments are in sync. And IndexChooser picks only IndexStatus.ENABLED indexes in the datamap pruning. It applies to bloom, lucene as well. But this property was not set for the secondary indexes in the older versions as they were not part of datamap pruning. To support the secondary index coarse grain datamap pruning for existing secondary indexes, need to support a way it happens automatically. > Secondary Index coarse grain datamap to work for existing Secondary Index > tables > > > Key: CARBONDATA-4169 > URL: https://issues.apache.org/jira/browse/CARBONDATA-4169 > Project: CarbonData > Issue Type: Sub-task > Components: core, spark-integration >Reporter: Venugopal Reddy K >Priority: Minor > > With [[https://github.com/apache/carbondata/pull/4110],] user has to delete > all segments from the older version of existing secondary index table and > reload/sync all segments again to use them in the seconday index coarse grain > datamap. This restriction is due to the fact that PR expects > CarbonCommonConstants.INDEX_STATUS property for the index to be set to > IndexStatus.ENABLED in the index metadata when main table and respective > secondary index table segments are in sync. And IndexChooser picks only > IndexStatus.ENABLED indexes in the datamap pruning. It applies to bloom, > lucene as well. > But this property was not set for the secondary indexes in the older versions > as they were not part of datamap pruning. > > To support the secondary index coarse grain datamap pruning for existing > secondary indexes, need to support a way it happens automatically. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
