Refactored SegmentPropertiesFetcher to resolve pruning problem post the carbon schema restructure.

2018-05-03 Thread mohdshahidkhan
Hi Dev, Currently DataMap while pruning is considering SegmentProperties from the 0th block of Segment DataMap. It is not correct as post schema restructure if the table is updated (*Update/Delete record*) then within the segment, all the block will not have symmetric schema. To support schema

Column comment is not included in the carbon column schema.

2017-12-03 Thread mohdshahidkhan
Hi Dev, Column comment not included in the carbon column schema. It is maintained only in the hivemetastore catalog, so while creating table from existing existing carbon table the column comment can not retrieved. Regards, Mohammad Shahid Khan -- Sent from: http://apache-carbondata-dev-mai

Re: DDL for CarbonData table backup and recovery (new feature)

2017-11-28 Thread mohdshahidkhan
Hi Dev, The table level registration should be also be supported.' -- Register the carbon tables at table level: *REGISTER TABLE $tbName;* Use case: If user has 10 tables but wants to register only 2 or 3 table not all. Regards, Shahid -- Sent from: http://apache-carbondata-d

Re: DDL for CarbonData table backup and recovery (new feature)

2017-11-26 Thread mohdshahidkhan
Thanks for the clarification Naresh. Please find my answer. Actually if the export command is on CarbonData table, we can just zip the actual table folder & associated agg table folders into user mentioned location. It dont export Metadata Copy data from 1 cluster to other will still remain sam

Re: DDL for CarbonData table backup and recovery (new feature)

2017-11-23 Thread mohdshahidkhan
*Please update solution: Instead of passing the dbLocation, the database name will be passed in the Register DDL* CarbonData table backup and recovery Background Customer has created one CarbonData table which is already loaded very huge data, and now they install another cluster which want to use

Re: DDL for CarbonData table backup and recovery (new feature)

2017-11-23 Thread mohdshahidkhan
Hi Naresh, Hive export export the meta data as well as the table data also. We do not want to export the table data as it will tedious for TB's of data. We have table and table data in the store location but the table is not register with hive metastore. Regards, Shahid -- Sent from: http://a

Re: [DISCUSSION] Support Database Location Configuration while Creating Database

2017-10-06 Thread mohdshahidkhan
Hi Sea, 1. create database with location is supported by spark(hive) only, carbon will not have any own implementation for create database. It is mention here just for reference regarding the location attribute. 2. Why carbon want to keep tablePath = 'databaseLocation “/” + database_Name + "/" +

Is assigning negative surrogate value (-1) for the dictionary value which is not present in ALL_DICTIONARY_PATH is correct?

2017-05-16 Thread mohdshahidkhan
Is assigning negative surrogate value (-1) for the dictionary value which is not present in ALL_DICTIONARY_PATH is correct? if not correct then weather the data-load should failed or load action should be decided based on the configuration bad_records_action. In case of if ALL_DICTIONARY_PATH is p