[jira] [Commented] (CARBONDATA-3797) Refresh materialized view command throws null pointer exception
[
https://issues.apache.org/jira/browse/CARBONDATA-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110901#comment-17110901
]
Chetan Bhat commented on CARBONDATA-3797:
-
Added other steps-queries
> Refresh materialized view command throws null pointer exception
> ---
>
> Key: CARBONDATA-3797
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3797
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
>Affects Versions: 2.0.0
> Environment: Spark 2.3.2, Spark 2.4.5
>Reporter: Chetan Bhat
>Priority: Major
>
> Refresh materialized view command throws null pointer exception
> CREATE TABLE uniqdata_mv(CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED as carbondata
> TBLPROPERTIES('local_dictionary_enable'='true','local_dictionary_threshold'='1000');
> LOAD DATA INPATH 'hdfs://hacluster/chetan/2000_UniqData.csv' into table
> uniqdata_mv OPTIONS('DELIMITER'=',',
> 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> CREATE MATERIALIZED VIEW mv1 as select cust_id, cust_name, count(cust_id)
> from uniqdata_mv group by cust_id, cust_name;
> refresh MATERIALIZED VIEW mv1;
> Error: java.lang.NullPointerException (state=,code=0)
>
> *Exception-*
> 2020-05-06 00:50:59,941 | ERROR | [HiveServer2-Background-Pool: Thread-1822]
> | Error executing query, currentState RUNNING, |
> org.apache.spark.internal.Logging$class.logError(Logging.scala:91)
> java.lang.NullPointerException
> at org.apache.carbondata.view.MVRefresher$.refresh(MVRefresher.scala:62)
> at
> org.apache.spark.sql.execution.command.view.CarbonRefreshMVCommand.processData(CarbonRefreshMVCommand.scala:52)
> at
> org.apache.spark.sql.execution.command.DataCommand$$anonfun$run$2.apply(package.scala:132)
> at
> org.apache.spark.sql.execution.command.DataCommand$$anonfun$run$2.apply(package.scala:132)
> at
> org.apache.spark.sql.execution.command.Auditable$class.runWithAudit(package.scala:104)
> at
> org.apache.spark.sql.execution.command.DataCommand.runWithAudit(package.scala:130)
> at org.apache.spark.sql.execution.command.DataCommand.run(package.scala:132)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
> at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:194)
> at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:194)
> at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3370)
> at
> org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:80)
> at
> org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:127)
> at
> org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:75)
> at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3369)
> at org.apache.spark.sql.Dataset.(Dataset.scala:194)
> at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:79)
> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:642)
> at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:694)
> at
> org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:232)
> at
> org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:175)
> at
> org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at
> org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:185)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.ru
[jira] [Updated] (CARBONDATA-3797) Refresh materialized view command throws null pointer exception
[
https://issues.apache.org/jira/browse/CARBONDATA-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Bhat updated CARBONDATA-3797:
Description:
Refresh materialized view command throws null pointer exception
CREATE TABLE uniqdata_mv(CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED as carbondata
TBLPROPERTIES('local_dictionary_enable'='true','local_dictionary_threshold'='1000');
LOAD DATA INPATH 'hdfs://hacluster/chetan/2000_UniqData.csv' into table
uniqdata_mv OPTIONS('DELIMITER'=',',
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
CREATE MATERIALIZED VIEW mv1 as select cust_id, cust_name, count(cust_id) from
uniqdata_mv group by cust_id, cust_name;
refresh MATERIALIZED VIEW mv1;
Error: java.lang.NullPointerException (state=,code=0)
*Exception-*
2020-05-06 00:50:59,941 | ERROR | [HiveServer2-Background-Pool: Thread-1822] |
Error executing query, currentState RUNNING, |
org.apache.spark.internal.Logging$class.logError(Logging.scala:91)
java.lang.NullPointerException
at org.apache.carbondata.view.MVRefresher$.refresh(MVRefresher.scala:62)
at
org.apache.spark.sql.execution.command.view.CarbonRefreshMVCommand.processData(CarbonRefreshMVCommand.scala:52)
at
org.apache.spark.sql.execution.command.DataCommand$$anonfun$run$2.apply(package.scala:132)
at
org.apache.spark.sql.execution.command.DataCommand$$anonfun$run$2.apply(package.scala:132)
at
org.apache.spark.sql.execution.command.Auditable$class.runWithAudit(package.scala:104)
at
org.apache.spark.sql.execution.command.DataCommand.runWithAudit(package.scala:130)
at org.apache.spark.sql.execution.command.DataCommand.run(package.scala:132)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:194)
at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:194)
at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3370)
at
org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:80)
at
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:127)
at
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:75)
at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3369)
at org.apache.spark.sql.Dataset.(Dataset.scala:194)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:79)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:642)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:694)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:232)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:175)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:171)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:185)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2020-05-06 00:50:59,941 | ERROR | [HiveServer2-Background-Pool: Thread-1822] |
Error running hive query: |
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:179)
org.apache.hive.service.cli.HiveSQLException: java.lang.NullPointerException
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:269)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkE
[jira] [Commented] (CARBONDATA-3797) Refresh materialized view command throws null pointer exception
[ https://issues.apache.org/jira/browse/CARBONDATA-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110881#comment-17110881 ] Akash R Nilugal commented on CARBONDATA-3797: - [~chetdb] can you please give the queries and steps for which it happened? > Refresh materialized view command throws null pointer exception > --- > > Key: CARBONDATA-3797 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3797 > Project: CarbonData > Issue Type: Bug > Components: data-query >Affects Versions: 2.0.0 > Environment: Spark 2.3.2, Spark 2.4.5 >Reporter: Chetan Bhat >Priority: Major > > Refresh materialized view command throws null pointer exception > refresh MATERIALIZED VIEW tab1; > Error: java.lang.NullPointerException (state=,code=0) > > *Exception-* > 2020-05-06 00:50:59,941 | ERROR | [HiveServer2-Background-Pool: Thread-1822] > | Error executing query, currentState RUNNING, | > org.apache.spark.internal.Logging$class.logError(Logging.scala:91) > java.lang.NullPointerException > at org.apache.carbondata.view.MVRefresher$.refresh(MVRefresher.scala:62) > at > org.apache.spark.sql.execution.command.view.CarbonRefreshMVCommand.processData(CarbonRefreshMVCommand.scala:52) > at > org.apache.spark.sql.execution.command.DataCommand$$anonfun$run$2.apply(package.scala:132) > at > org.apache.spark.sql.execution.command.DataCommand$$anonfun$run$2.apply(package.scala:132) > at > org.apache.spark.sql.execution.command.Auditable$class.runWithAudit(package.scala:104) > at > org.apache.spark.sql.execution.command.DataCommand.runWithAudit(package.scala:130) > at org.apache.spark.sql.execution.command.DataCommand.run(package.scala:132) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68) > at > org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79) > at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:194) > at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:194) > at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3370) > at > org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:80) > at > org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:127) > at > org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:75) > at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3369) > at org.apache.spark.sql.Dataset.(Dataset.scala:194) > at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:79) > at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:642) > at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:694) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:232) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:175) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:171) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:185) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > 2020-05-06 00:50:59,941 | ERROR | [HiveServer2-Background-Pool: Thread-1822] > | Error running hive query: | > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:179) > org.apache.hive.service.cli.HiveSQLException: java.lang.NullPointerException > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:269) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:175) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatemen
[jira] [Updated] (CARBONDATA-3827) Merge DDL is not working as per the mentioned syntax.
[
https://issues.apache.org/jira/browse/CARBONDATA-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanna Ravichandran updated CARBONDATA-3827:
--
Description:
This issue is seen with opensource jars. Spark 2.4.5 & Carbon 2.0.
Merge DDL is not working as per the mentioned syntax as in CARBONDATA-3597
Test queries:
drop table if exists uniqdata1;
CREATE TABLE uniqdata1 (cust_id int,cust_name String,active_emui_version
string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2
bigint,decimal_column1 decimal(30,10), decimal_column2
decimal(36,36),double_column1 double, double_column2 double,integer_column1
int) stored as carbondata;
load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into table
uniqdata1
options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
drop table if exists uniqdata;
CREATE TABLE uniqdata (cust_id int,cust_name String,active_emui_version
string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2
bigint,decimal_column1 decimal(30,10), decimal_column2
decimal(36,36),double_column1 double, double_column2 double,integer_column1
int) stored as carbondata;
load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into table
uniqdata
options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
merge into uniqdata1 as a using uniqdata as b on a.cust_id=b.cust_id; --not
working , getting parse exeption;
>merge into uniqdata1 as a using uniqdata as b on a.cust_id=b.cust_id;
Error: org.apache.spark.sql.AnalysisException: == Parser1:
org.apache.spark.sql.parser.CarbonExtensionSpark2SqlParser ==
[1.1] failure: identifier matching regex (?i)EXPLAIN expected
merge into uniqdata1 as a using uniqdata as b on a.cust_id=b.cust_id
^;
== Parser2: org.apache.spark.sql.execution.SparkSqlParser ==
mismatched input 'merge' expecting \{'(', 'SELECT', 'FROM', 'ADD', 'DESC',
'EMPOWER', 'WITH', 'VALUES', 'CREATE', 'TABLE', 'INSERT', 'DELETE', 'DESCRIBE',
'EXPLAIN', 'SHOW', 'USE', 'DROP', 'ALTER', 'MAP', 'SET', 'RESET', 'START',
'COMMIT', 'ROLLBACK', 'REDUCE', 'REFRESH', 'CLEAR', 'CACHE', 'UNCACHE', 'DFS',
'TRUNCATE', 'ANALYZE', 'LIST', 'REVOKE', 'GRANT', 'LOCK', 'UNLOCK', 'MSCK',
'EXPORT', 'IMPORT', 'LOAD', 'HEALTHCHECK'}(line 1, pos 0)
== SQL ==
merge into uniqdata1 as a using uniqdata as b on a.cust_id=b.cust_id
^^^; (state=,code=0)
was:
This issue is seen with opensource jars. Spark 2.4.5 & Carbon 2.0.
Merge DDL is not working as per the mentioned syntax as in CARBONDATA-3597
Test queries:
drop table if exists uniqdata1;
CREATE TABLE uniqdata1 (cust_id int,cust_name String,active_emui_version
string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2
bigint,decimal_column1 decimal(30,10), decimal_column2
decimal(36,36),double_column1 double, double_column2 double,integer_column1
int) stored as carbondata;
load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into table
uniqdata1
options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
drop table if exists uniqdata;
CREATE TABLE uniqdata (cust_id int,cust_name String,active_emui_version
string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2
bigint,decimal_column1 decimal(30,10), decimal_column2
decimal(36,36),double_column1 double, double_column2 double,integer_column1
int) stored as carbondata;
load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into table
uniqdata
options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
merge into uniqdata1 as a using uniqdata as b on a.cust_id=b.cust_id; --not
working;
Attached the screenshot for your reference.
!image-2020-05-18-21-30-31-344.png!
> Merge DDL is not working as per the mentioned syntax.
> -
>
> Key: CARBONDATA-3827
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3827
> Project: CarbonData
> Issue Type: Bug
>Reporter: Prasanna Ravichandran
>Priority: Major
>
> This issue is seen with opensource jars. Spark 2.4.5 & Carbon 2.0.
> Merge DDL is not working as per the mentioned syntax as in CARBONDATA-3597
> Test queries:
> drop table if exists uniqdata1;
> CREATE TABLE uniqdata1 (cust_id
[jira] [Updated] (CARBONDATA-3827) Merge DDL is not working as per the mentioned syntax.
[
https://issues.apache.org/jira/browse/CARBONDATA-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanna Ravichandran updated CARBONDATA-3827:
--
Description:
This issue is seen with opensource jars. Spark 2.4.5 & Carbon 2.0.
Merge DDL is not working as per the mentioned syntax as in CARBONDATA-3597
Test queries:
drop table if exists uniqdata1;
CREATE TABLE uniqdata1 (cust_id int,cust_name String,active_emui_version
string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2
bigint,decimal_column1 decimal(30,10), decimal_column2
decimal(36,36),double_column1 double, double_column2 double,integer_column1
int) stored as carbondata;
load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into table
uniqdata1
options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
drop table if exists uniqdata;
CREATE TABLE uniqdata (cust_id int,cust_name String,active_emui_version
string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2
bigint,decimal_column1 decimal(30,10), decimal_column2
decimal(36,36),double_column1 double, double_column2 double,integer_column1
int) stored as carbondata;
load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into table
uniqdata
options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
merge into uniqdata1 as a using uniqdata as b on a.cust_id=b.cust_id; --not
working;
Attached the screenshot for your reference.
!image-2020-05-18-21-30-31-344.png!
was:
This issue is seen with opensource jars. Spark 2.4.5 & Carbon 2.0.
Merge DDL is not working as per the mentioned syntax as in CARBONDATA-3597
Test queries:
drop table if exists uniqdata1;
CREATE TABLE uniqdata1 (cust_id int,cust_name String,active_emui_version
string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2
bigint,decimal_column1 decimal(30,10), decimal_column2
decimal(36,36),double_column1 double, double_column2 double,integer_column1
int) stored as carbondata;
load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into table
uniqdata1
options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
drop table if exists uniqdata;
CREATE TABLE uniqdata (cust_id int,cust_name String,active_emui_version string,
dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2
bigint,decimal_column1 decimal(30,10), decimal_column2
decimal(36,36),double_column1 double, double_column2 double,integer_column1
int) stored as carbondata;
load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into table
uniqdata
options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
merge into uniqdata1 as a using uniqdata as b on a.cust_id=b.cust_id; --not
working;
Attached the screenshot for your reference.
> Merge DDL is not working as per the mentioned syntax.
> -
>
> Key: CARBONDATA-3827
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3827
> Project: CarbonData
> Issue Type: Bug
>Reporter: Prasanna Ravichandran
>Priority: Major
>
> This issue is seen with opensource jars. Spark 2.4.5 & Carbon 2.0.
> Merge DDL is not working as per the mentioned syntax as in CARBONDATA-3597
> Test queries:
> drop table if exists uniqdata1;
> CREATE TABLE uniqdata1 (cust_id int,cust_name String,active_emui_version
> string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2
> bigint,decimal_column1 decimal(30,10), decimal_column2
> decimal(36,36),double_column1 double, double_column2 double,integer_column1
> int) stored as carbondata;
> load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into
> table uniqdata1
> options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
> drop table if exists uniqdata;
> CREATE TABLE uniqdata (cust_id int,cust_name String,active_emui_version
> string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2
> bigint,decimal_column1 decimal(30,10), decimal_column2
> decimal(36,36),double_column1 double, double_column2 double,integer_column1
> int) stored as ca
[jira] [Created] (CARBONDATA-3827) Merge DDL is not working as per the mentioned syntax.
Prasanna Ravichandran created CARBONDATA-3827:
-
Summary: Merge DDL is not working as per the mentioned syntax.
Key: CARBONDATA-3827
URL: https://issues.apache.org/jira/browse/CARBONDATA-3827
Project: CarbonData
Issue Type: Bug
Reporter: Prasanna Ravichandran
This issue is seen with opensource jars. Spark 2.4.5 & Carbon 2.0.
Merge DDL is not working as per the mentioned syntax as in CARBONDATA-3597
Test queries:
drop table if exists uniqdata1;
CREATE TABLE uniqdata1 (cust_id int,cust_name String,active_emui_version
string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2
bigint,decimal_column1 decimal(30,10), decimal_column2
decimal(36,36),double_column1 double, double_column2 double,integer_column1
int) stored as carbondata;
load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into table
uniqdata1
options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
drop table if exists uniqdata;
CREATE TABLE uniqdata (cust_id int,cust_name String,active_emui_version string,
dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2
bigint,decimal_column1 decimal(30,10), decimal_column2
decimal(36,36),double_column1 double, double_column2 double,integer_column1
int) stored as carbondata;
load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into table
uniqdata
options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
merge into uniqdata1 as a using uniqdata as b on a.cust_id=b.cust_id; --not
working;
Attached the screenshot for your reference.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CARBONDATA-3826) Merge DDL is not working as per the mentioned syntax.
Prasanna Ravichandran created CARBONDATA-3826:
-
Summary: Merge DDL is not working as per the mentioned syntax.
Key: CARBONDATA-3826
URL: https://issues.apache.org/jira/browse/CARBONDATA-3826
Project: CarbonData
Issue Type: Bug
Reporter: Prasanna Ravichandran
This issue is seen with opensource jars. Spark 2.4.5 & Carbon 2.0.
Merge DDL is not working as per the mentioned syntax as in CARBONDATA-3597
Test queries:
drop table if exists uniqdata1;
CREATE TABLE uniqdata1 (cust_id int,cust_name String,active_emui_version
string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2
bigint,decimal_column1 decimal(30,10), decimal_column2
decimal(36,36),double_column1 double, double_column2 double,integer_column1
int) stored as carbondata;
load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into table
uniqdata1
options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
drop table if exists uniqdata;
CREATE TABLE uniqdata (cust_id int,cust_name String,active_emui_version string,
dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2
bigint,decimal_column1 decimal(30,10), decimal_column2
decimal(36,36),double_column1 double, double_column2 double,integer_column1
int) stored as carbondata;
load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into table
uniqdata
options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
merge into uniqdata1 as a using uniqdata as b on a.cust_id=b.cust_id; --not
working;
Attached the screenshot for your reference.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
