Re: [DISCUSSION] Distributed Index Cache Server

+1 1. Add the impacted areas in design document. 2. If any executor goes down then update the index cache to executor mapping in driver accordingly. 3. Even though the cache would be divided based on index files, the minimum unit of cache need to be fixed. Example: 1 segment cache should belong

Re: [VOTE] Apache CarbonData 1.5.2(RC2) release

+1 Regards Manish Gupta -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [DISCUSS] Move to gitbox as per ASF infra team mail

+1 Regards Manish Gupta -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [Discussion] Make 'no_sort' as default sort_scope and keep sort_columns as 'empty' by default

Hi Ajantha +1 for the proposal. 1. I agree with Liang to remove empty SORT_COLUMNS option. This will give more calrity to the user about the property behavior. If configured we use LOCAL_SORT else we use NO_SORT. Internal behavior you can keep anything as per the implementation, it need nnot be

Re: [DISCUSSION] Complex Delimiter support as per Hive format

+1 We should modify the delimters as per hive. Also update the documentation as per the change. Regards Manish Gupta -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [Discussion]Alter table column rename feature

+1 We already have a DDL for data type change and the same can be used for rename column. The DDL is same as that of hive. Regards Manish Gupta -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

RE: [SUGGESTION]Support compaction no_sort

Hi Xuchuanyin The scope for this feature is to SORT the data during compaction when the data is loaded using NO_SORT option during data load operation. There are few users who want to maximize the data load speed and in turn fine tune the data further during off peak time (time when system is

Re: [VOTE] Apache CarbonData 1.5.0(RC2) release

+1 Regards Manish Gupta -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [DISCUSSION] Optimizing the writing of min max for a column

Hi Xuchuanyin The idea you have mentioned is good and correct. But I feel that the current implementation behavior is better because of the following reasons. 1. Code understanding will be good as per the current implementation. Looking at the thrift anyone can understand the design and come to

Re: [DISCUSSION] Support Standard Spark's FileFormat interface in Carbondata

+1 Regards Manish Gupta -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[DISCUSSION] Support Map type for complex type columns

Hi All, I am working on supporting complex type map columns. Please find below the scope for the same. *Scope:* 1. Create Table DDL support for complex map type. 1. Support loading of data for complex map type column [SDK, DataLoad DDL]. 2. Design to consider n level nested support for map

Re: [VOTE] Apache CarbonData 1.4.1(RC1) release

Agree with Liang. -1 Regards Manish Gupta -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [Discussion] Blocklet DataMap caching in driver

Thanks Ravi for the feedback. I completely agree with you that we need to develop the second solution ASAP. Please find my response below for your queries. 1. what if the query comes on noncached columns, will it start read from disk in driver side for minmax ? - If query is on a non-cached

Re: COMPACT error

Hi, >From the exception it seems there is some problem during processing the data in writer step. Can you please share the complete executor and driver logs to get some idea on the exact issue. Regards Manish Gupta -- Sent from:

Re: Cannot seek after EOF

Hi Can you please provide more details on the issue 1. Steps to reproduce the issue 2. Which Carbondata version you are using Regards Manish Gupta -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: insert carbondata table failed

Hi Feng, You can also refer the below links wherein the spark users have tried to resolve this issue by making changes in the configuration. This might help you. https://stackoverflow.com/questions/28901123/why-do-spark-jobs-fail-with-org-apache-spark-shuffle-metadatafetchfailedexceptio

Re: 2 problems with loading data into a carbon table

Hi Marek, >From the logs it seems that this is a bug in the code. You can raise a jira to track the issue. Regards Manish Gupta -- View this message in context:

Re: method not found issue when creating table

Hi Lionel, Carbon table creation flow is executed on the driver side, Executors do not participate in creation of carbon table. From the logs it seems that spark-catalyst jar is missing which is generally placed under $SPARK_HOME/jars OR $SPARK_HOME/lib directory. Please check if spark jars

Re: carbon data performance doubts

Hi Swapnil Please find my answers inline. 1. What is the use of *carbon.number.of.cores *property and how is it different from spark's executor cores? -carbon.number.of.cores is used for reading the footer and header of the carbondata file during query execution. Spark executor cores is a