[jira] [Updated] (CARBONDATA-2199) Exception occurs when change the datatype of measure having sort_column
[
https://issues.apache.org/jira/browse/CARBONDATA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravindra Pesala updated CARBONDATA-2199:
Fix Version/s: 1.3.1
> Exception occurs when change the datatype of measure having sort_column
> ---
>
> Key: CARBONDATA-2199
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2199
> Project: CarbonData
> Issue Type: Bug
> Components: spark-integration
>Affects Versions: 1.3.0
> Environment: spark 2.1
>Reporter: Jatin
>Assignee: Jatin
>Priority: Minor
> Fix For: 1.4.0, 1.3.1
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Use a measure columns in sort_column and change the datatype of that columns
> Steps to replicate
> CREATE TABLE non_partitiontable7(id Int,vin String,phonenumber Long,area
> String,salary Int, country String,logdate date)STORED BY
> 'org.apache.carbondata.format'TBLPROPERTIES('SORT_COLUMNS'='id,vin','sort_scope'='global_sort');
> insert into non_partitiontable7 select
> 1,'A42151477823',125371344,'OutSpace',1,'China','2017-02-12';
> insert into non_partitiontable7 select
> 1,'Y42151477823',125371344,'midasia',1,'China','2017-02-13';
> insert into non_partitiontable7 select
> 1,'B42151477823',125371346,'OutSpace',1,'US','2018-02-12';
> insert into non_partitiontable7 select
> 1,'C42151477823',125371348,'InnerSpace',10001,'UK','2019-02-12';
> select * from non_partitiontable7;
> alter table non_partitiontable7 add columns (c1 int);
> select * from non_partitiontable7;
> alter table non_partitiontable7 change id id bigint;
> select * from non_partitiontable7;
> Exception StackTrace
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 1 in stage 16.0 failed 4 times, most recent failure: Lost task 1.3 in
> stage 16.0 (TID 80, BLR123654, executor 3):
> java.lang.IllegalArgumentException: Wrong length: 4, expected 8
> at
> org.apache.carbondata.core.util.ByteUtil.explainWrongLengthOrOffset(ByteUtil.java:581)
> at org.apache.carbondata.core.util.ByteUtil.toLong(ByteUtil.java:553)
> at
> org.apache.carbondata.core.util.DataTypeUtil.getDataBasedOnRestructuredDataType(DataTypeUtil.java:847)
> at
> org.apache.carbondata.core.datastore.chunk.store.impl.unsafe.UnsafeVariableLengthDimesionDataChunkStore.fillRow(UnsafeVariableLengthDimesionDataChunkStore.java:181)
> at
> org.apache.carbondata.core.datastore.chunk.impl.VariableLengthDimensionDataChunk.fillConvertedChunkData(VariableLengthDimensionDataChunk.java:112)
> at
> org.apache.carbondata.core.scan.result.AbstractScannedResult.fillColumnarNoDictionaryBatch(AbstractScannedResult.java:256)
> at
> org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.scanAndFillResult(DictionaryBasedVectorResultCollector.java:163)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:128)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
> at
> org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:283)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:171)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:402)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
> Source)
> at
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at
> org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:395)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:234)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:228)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
> at org.apache.spark.scheduler.Task.run(Task.scal
[jira] [Updated] (CARBONDATA-2199) Exception occurs when change the datatype of measure having sort_column
[
https://issues.apache.org/jira/browse/CARBONDATA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravindra Pesala updated CARBONDATA-2199:
Affects Version/s: (was: 1.4.0)
1.3.0
> Exception occurs when change the datatype of measure having sort_column
> ---
>
> Key: CARBONDATA-2199
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2199
> Project: CarbonData
> Issue Type: Bug
> Components: spark-integration
>Affects Versions: 1.3.0
> Environment: spark 2.1
>Reporter: Jatin
>Assignee: Jatin
>Priority: Minor
> Fix For: 1.4.0, 1.3.1
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Use a measure columns in sort_column and change the datatype of that columns
> Steps to replicate
> CREATE TABLE non_partitiontable7(id Int,vin String,phonenumber Long,area
> String,salary Int, country String,logdate date)STORED BY
> 'org.apache.carbondata.format'TBLPROPERTIES('SORT_COLUMNS'='id,vin','sort_scope'='global_sort');
> insert into non_partitiontable7 select
> 1,'A42151477823',125371344,'OutSpace',1,'China','2017-02-12';
> insert into non_partitiontable7 select
> 1,'Y42151477823',125371344,'midasia',1,'China','2017-02-13';
> insert into non_partitiontable7 select
> 1,'B42151477823',125371346,'OutSpace',1,'US','2018-02-12';
> insert into non_partitiontable7 select
> 1,'C42151477823',125371348,'InnerSpace',10001,'UK','2019-02-12';
> select * from non_partitiontable7;
> alter table non_partitiontable7 add columns (c1 int);
> select * from non_partitiontable7;
> alter table non_partitiontable7 change id id bigint;
> select * from non_partitiontable7;
> Exception StackTrace
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 1 in stage 16.0 failed 4 times, most recent failure: Lost task 1.3 in
> stage 16.0 (TID 80, BLR123654, executor 3):
> java.lang.IllegalArgumentException: Wrong length: 4, expected 8
> at
> org.apache.carbondata.core.util.ByteUtil.explainWrongLengthOrOffset(ByteUtil.java:581)
> at org.apache.carbondata.core.util.ByteUtil.toLong(ByteUtil.java:553)
> at
> org.apache.carbondata.core.util.DataTypeUtil.getDataBasedOnRestructuredDataType(DataTypeUtil.java:847)
> at
> org.apache.carbondata.core.datastore.chunk.store.impl.unsafe.UnsafeVariableLengthDimesionDataChunkStore.fillRow(UnsafeVariableLengthDimesionDataChunkStore.java:181)
> at
> org.apache.carbondata.core.datastore.chunk.impl.VariableLengthDimensionDataChunk.fillConvertedChunkData(VariableLengthDimensionDataChunk.java:112)
> at
> org.apache.carbondata.core.scan.result.AbstractScannedResult.fillColumnarNoDictionaryBatch(AbstractScannedResult.java:256)
> at
> org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.scanAndFillResult(DictionaryBasedVectorResultCollector.java:163)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:128)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
> at
> org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:283)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:171)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:402)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
> Source)
> at
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at
> org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:395)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:234)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:228)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
> at o
[jira] [Updated] (CARBONDATA-2199) Exception occurs when change the datatype of measure having sort_column
[
https://issues.apache.org/jira/browse/CARBONDATA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jatin updated CARBONDATA-2199:
--
Priority: Minor (was: Major)
> Exception occurs when change the datatype of measure having sort_column
> ---
>
> Key: CARBONDATA-2199
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2199
> Project: CarbonData
> Issue Type: Bug
> Components: spark-integration
>Affects Versions: 1.4.0
> Environment: spark 2.1
>Reporter: Jatin
>Assignee: Jatin
>Priority: Minor
> Fix For: 1.4.0
>
>
> Use a measure columns in sort_column and change the datatype of that columns
> Steps to replicate
> CREATE TABLE non_partitiontable7(id Int,vin String,phonenumber Long,area
> String,salary Int, country String,logdate date)STORED BY
> 'org.apache.carbondata.format'TBLPROPERTIES('SORT_COLUMNS'='id,vin','sort_scope'='global_sort');
> insert into non_partitiontable7 select
> 1,'A42151477823',125371344,'OutSpace',1,'China','2017-02-12';
> insert into non_partitiontable7 select
> 1,'Y42151477823',125371344,'midasia',1,'China','2017-02-13';
> insert into non_partitiontable7 select
> 1,'B42151477823',125371346,'OutSpace',1,'US','2018-02-12';
> insert into non_partitiontable7 select
> 1,'C42151477823',125371348,'InnerSpace',10001,'UK','2019-02-12';
> select * from non_partitiontable7;
> alter table non_partitiontable7 add columns (c1 int);
> select * from non_partitiontable7;
> alter table non_partitiontable7 change id id bigint;
> select * from non_partitiontable7;
> Exception StackTrace
> Error: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 1 in stage 16.0 failed 4 times, most recent failure: Lost task 1.3 in
> stage 16.0 (TID 80, BLR123654, executor 3):
> java.lang.IllegalArgumentException: Wrong length: 4, expected 8
> at
> org.apache.carbondata.core.util.ByteUtil.explainWrongLengthOrOffset(ByteUtil.java:581)
> at org.apache.carbondata.core.util.ByteUtil.toLong(ByteUtil.java:553)
> at
> org.apache.carbondata.core.util.DataTypeUtil.getDataBasedOnRestructuredDataType(DataTypeUtil.java:847)
> at
> org.apache.carbondata.core.datastore.chunk.store.impl.unsafe.UnsafeVariableLengthDimesionDataChunkStore.fillRow(UnsafeVariableLengthDimesionDataChunkStore.java:181)
> at
> org.apache.carbondata.core.datastore.chunk.impl.VariableLengthDimensionDataChunk.fillConvertedChunkData(VariableLengthDimensionDataChunk.java:112)
> at
> org.apache.carbondata.core.scan.result.AbstractScannedResult.fillColumnarNoDictionaryBatch(AbstractScannedResult.java:256)
> at
> org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.scanAndFillResult(DictionaryBasedVectorResultCollector.java:163)
> at
> org.apache.carbondata.core.scan.collector.impl.RestructureBasedVectorResultCollector.collectVectorBatch(RestructureBasedVectorResultCollector.java:128)
> at
> org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.processNextBatch(DataBlockIteratorImpl.java:65)
> at
> org.apache.carbondata.core.scan.result.iterator.VectorDetailQueryResultIterator.processNextBatch(VectorDetailQueryResultIterator.java:46)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextBatch(VectorizedCarbonRecordReader.java:283)
> at
> org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader.nextKeyValue(VectorizedCarbonRecordReader.java:171)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.hasNext(CarbonScanRDD.scala:402)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.scan_nextBatch$(Unknown
> Source)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
> Source)
> at
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at
> org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:395)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:234)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:228)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
> at org.apache.spark.scheduler.Task.run(Task.scala:108)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.
