Hi, It seems your data might contain the data which is longer than the short limit (> ~ 32000). Currently, carbondata cannot support the column value more than short limit.
Regards, Ravindra On 19 October 2017 at 15:44, lcxxsg <[email protected]> wrote: > Hi all, > I have trouble with carbondata. > carbondata1.2 + spark2.1 > > There is a table with more than 300 columns. I create the table like this: > > create table carbondata_table_name( > ds String, > event String, > partnercode String, > xx String, > ... > ) > stored by 'carbondata' > tblproperties( > 'DICTIONARY_INCLUDE'='ds, event, partnercode', > 'SORT_COLUMNS'='ds, event, partnercode’) > > Now,there is data of more than ten million and larger than 60G.I try to > insert them into this table.I set spark properties with > spark.executor.instances=20, spark.executor.cores=1, > spark.executor.memory=20g > and spark.driver.memory=20g. > I execute the instruction like "insert into carbondata_table_name select > from hive_table". > But, this job is failed.Message like this: > <http://apache-carbondata-user-mailing-list.3231.n8.nabble.com/file/t29/ > carbondata_1.png> > <http://apache-carbondata-user-mailing-list.3231.n8.nabble.com/file/t29/ > carbondata_2.png> > <http://apache-carbondata-user-mailing-list.3231.n8.nabble.com/file/t29/ > carbondata_3.png> > <http://apache-carbondata-user-mailing-list.3231.n8.nabble.com/file/t29/ > carbondata_4.png> > <http://apache-carbondata-user-mailing-list.3231.n8.nabble.com/file/t29/ > carbondata_5.png> > > anyone know how to solve the problem? Thanks! > > Regards > > > > -- > Sent from: http://apache-carbondata-user-mailing-list.3231.n8.nabble.com/ > -- Thanks & Regards, Ravi
