[jira] [Created] (CARBONDATA-4194) read from presto session throws error after delete operation from complex table from spark session

[Chetan Bhat (Jira)]

Chetan Bhat created CARBONDATA-4194:
---

 Summary: read from presto session throws error after delete 
operation from complex table from spark session
 Key: CARBONDATA-4194
 URL: https://issues.apache.org/jira/browse/CARBONDATA-4194
 Project: CarbonData
  Issue Type: Bug
  Components: presto-integration
Affects Versions: 2.1.1
 Environment: Spark 2.4.5, Presto SQL 316
Reporter: Chetan Bhat


*Queries executed -* 

>From Spark session create table with complex types , load data to table and 
>delete data from table

create table Struct_com19_PR4031_009 (CUST_ID string, YEAR int, MONTH int, AGE 
int, GENDER string, EDUCATED string, IS_MARRIED string, 
STRUCT_INT_DOUBLE_STRING_DATE 
struct,CARD_COUNT
 int,DEBIT_COUNT int, CREDIT_COUNT int, DEPOSIT double, HQ_DEPOSIT 
decimal(20,3)) stored as carbondata;
LOAD DATA INPATH 'hdfs://hacluster/chetan/Struct.csv' INTO table 
Struct_com19_PR4031_009 options ('DELIMITER'=',', 'QUOTECHAR'='"', 
'FILEHEADER'='CUST_ID,YEAR,MONTH,AGE,GENDER,EDUCATED,IS_MARRIED,STRUCT_INT_DOUBLE_STRING_DATE,CARD_COUNT,DEBIT_COUNT,CREDIT_COUNT,DEPOSIT,HQ_DEPOSIT','COMPLEX_DELIMITER_LEVEL_1'='$');
delete from Struct_com19_PR4031_009 where EDUCATED='MS';

 

>From Presto CLI execute the select queries.

select * from Struct_com19_PR4031_009 limit 1;
select count(*) from Struct_com19_PR4031_009;

 

*Issue : -* read from presto session throws error after delete operation from 
complex table from spark session

presto:ranjan> select * from Struct_com19_PR4031_009 limit 1;

Query 20210528_075917_1_swzys, FAILED, 1 node
Splits: 18 total, 0 done (0.00%)
0:00 [0 rows, 0B] [0 rows/s, 0B/s]

Query 20210528_075917_1_swzys failed: Error in Reading Data from Carbondata

 

*Log -*

org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: 
Error in Reading Data from Carbondata 
 at 
org.apache.carbondata.presto.CarbondataPageSource$CarbondataBlockLoader.load(CarbondataPageSource.java:491)
 at 
org.apache.carbondata.presto.CarbondataPageSource$CarbondataBlockLoader.load(CarbondataPageSource.java:467)
 at io.prestosql.spi.block.LazyBlock.assureLoaded(LazyBlock.java:276)
 at io.prestosql.spi.block.LazyBlock.getLoadedBlock(LazyBlock.java:267)
 at io.prestosql.spi.Page.getLoadedPage(Page.java:261)
 at 
io.prestosql.operator.TableScanOperator.getOutput(TableScanOperator.java:283)
 at io.prestosql.operator.Driver.processInternal(Driver.java:379)
 at io.prestosql.operator.Driver.lambda$processFor$8(Driver.java:283)
 at io.prestosql.operator.Driver.tryWithLock(Driver.java:675)
 at io.prestosql.operator.Driver.processFor(Driver.java:276)
 at 
io.prestosql.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1075)
 at 
io.prestosql.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
 at 
io.prestosql.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:484)
 at io.prestosql.$gen.Presto_31620210526_073226_1.run(Unknown Source)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: 
java.lang.ClassCastException
 at 
org.apache.carbondata.core.datastore.chunk.impl.DimensionRawColumnChunk.convertToDimColDataChunkAndFillVector(DimensionRawColumnChunk.java:140)
 at 
org.apache.carbondata.core.scan.scanner.LazyPageLoader.loadPage(LazyPageLoader.java:75)
 at 
org.apache.carbondata.core.scan.result.vector.impl.CarbonColumnVectorImpl.loadPage(CarbonColumnVectorImpl.java:531)
 at 
org.apache.carbondata.presto.CarbondataPageSource$CarbondataBlockLoader.load(CarbondataPageSource.java:483)
 ... 16 more
Caused by: java.lang.RuntimeException: java.lang.ClassCastException
 at 
org.apache.carbondata.core.datastore.chunk.impl.DimensionRawColumnChunk.convertToDimColDataChunkAndFillVector(DimensionRawColumnChunk.java:140)
 at 
org.apache.carbondata.core.scan.scanner.LazyPageLoader.loadPage(LazyPageLoader.java:75)
 at 
org.apache.carbondata.core.scan.result.vector.impl.CarbonColumnVectorImpl.loadPage(CarbonColumnVectorImpl.java:531)
 at 
org.apache.carbondata.core.datastore.page.encoding.compress.DirectCompressCodec$3.decodeAndFillVector(DirectCompressCodec.java:277)
 at 
org.apache.carbondata.core.datastore.page.encoding.compress.DirectCompressCodec$2.decodeAndFillVector(DirectCompressCodec.java:158)
 at 
org.apache.carbondata.core.datastore.chunk.reader.dimension.v3.DimensionChunkReaderV3.decodeDimensionByMeta(DimensionChunkReaderV3.java:260)
 at 
org.apache.carbondata.core.datastore.chunk.reader.dimension.v3.DimensionChunkReaderV3.decodeDimension(DimensionChunkReaderV3.java:307)
 at 
org.apache.carbondata.core.datastore.chunk.reader.dimension.v3.DimensionChunkReaderV3.decodeColumnPag

[jira] [Created] (CARBONDATA-4193) Fix compaction failure after alter add complex column.

[SHREELEKHYA GAMPA (Jira)]

SHREELEKHYA GAMPA created CARBONDATA-4193:
-

 Summary:  Fix compaction failure after alter add complex column.
 Key: CARBONDATA-4193
 URL: https://issues.apache.org/jira/browse/CARBONDATA-4193
 Project: CarbonData
  Issue Type: Bug
Reporter: SHREELEKHYA GAMPA


[Steps] :-

>From spark beeline/SQL/Shell/Submit the following queries are executed

drop table if exists alter_complex; create table alter_complex (a int, b 
string) stored as carbondata; insert into alter_complex select 1,'a'; insert 
into alter_complex select 1,'a'; insert into alter_complex select 1,'a'; insert 
into alter_complex select 1,'a'; insert into alter_complex select 1,'a'; select 
_from alter_complex; ALTER TABLE alter_complex ADD COLUMNS(struct1 
STRUCT); insert into alter_complex select 
3,'c',named_struct('s1',4,'s2','d'); insert into alter_complex select 
3,'c',named_struct('s1',4,'s2','d'); insert into alter_complex select 
3,'c',named_struct('s1',4,'s2','d'); insert into alter_complex select 
3,'c',named_struct('s1',4,'s2','d'); insert into alter_complex select 
3,'c',named_struct('s1',4,'s2','d'); select_ from alter_complex; alter table 
alter_complex compact 'minor'; OR alter table alter_complex compact 'major'; OR 
alter table alter_complex compact 'custom' where segment.id In (3,4,5,6);

[Expected Result] :- Compaction should be success after alter add complex 
column.

[Actual Issue] : - Compaction fails after alter add complex column.

 

!https://clouddevops.huawei.com/vision-file-storage/api/file/download/upload-v2/2021/3/26/c71035/ec9486ee659c4374a211db588b2f6b2a/image.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)