[
https://issues.apache.org/jira/browse/CARBONDATA-1535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xubo245 updated CARBONDATA-1535:
--------------------------------
Description:
When carbon.enable.auto.load.merge was set as true, the table that has complex
data type ARRAY<STRING> will happen error when compaction(auto merge).
as follow:
# auto merge
# sql("alter table array_table compact 'major'")
# sql("alter table array_table compact 'minor'")
{code:java}
java.lang.IndexOutOfBoundsException: 1
at
scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
at scala.collection.immutable.List.apply(List.scala:84)
at scala.collection.convert.Wrappers$SeqWrapper.get(Wrappers.scala:66)
at
org.apache.carbondata.core.datastore.block.SegmentProperties.readAllComplexTypeChildren(SegmentProperties.java:445)
at
org.apache.carbondata.core.datastore.block.SegmentProperties.fillDimensionAndMeasureDetails(SegmentProperties.java:390)
at
org.apache.carbondata.core.datastore.block.SegmentProperties.<init>(SegmentProperties.java:173)
at
org.apache.carbondata.spark.rdd.CarbonMergerRDD$$anon$1.<init>(CarbonMergerRDD.scala:161)
at
org.apache.carbondata.spark.rdd.CarbonMergerRDD.internalCompute(CarbonMergerRDD.scala:79)
at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:62)
{code}
was:
When carbon.enable.auto.load.merge was set as true, the table that has complex
data type ARRAY<STRING> will happen error when compaction(auto merge).
{code:java}
java.lang.IndexOutOfBoundsException: 1
at
scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
at scala.collection.immutable.List.apply(List.scala:84)
at scala.collection.convert.Wrappers$SeqWrapper.get(Wrappers.scala:66)
at
org.apache.carbondata.core.datastore.block.SegmentProperties.readAllComplexTypeChildren(SegmentProperties.java:445)
at
org.apache.carbondata.core.datastore.block.SegmentProperties.fillDimensionAndMeasureDetails(SegmentProperties.java:390)
at
org.apache.carbondata.core.datastore.block.SegmentProperties.<init>(SegmentProperties.java:173)
at
org.apache.carbondata.spark.rdd.CarbonMergerRDD$$anon$1.<init>(CarbonMergerRDD.scala:161)
at
org.apache.carbondata.spark.rdd.CarbonMergerRDD.internalCompute(CarbonMergerRDD.scala:79)
at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:62)
{code}
> Complex data type ARRAY<STRING> do not support auto merge/compaction
> ---------------------------------------------------------------------
>
> Key: CARBONDATA-1535
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1535
> Project: CarbonData
> Issue Type: Bug
> Reporter: xubo245
> Original Estimate: 720h
> Remaining Estimate: 720h
>
> When carbon.enable.auto.load.merge was set as true, the table that has
> complex data type ARRAY<STRING> will happen error when compaction(auto
> merge).
>
> as follow:
> # auto merge
> # sql("alter table array_table compact 'major'")
> # sql("alter table array_table compact 'minor'")
> {code:java}
> java.lang.IndexOutOfBoundsException: 1
> at
> scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
> at scala.collection.immutable.List.apply(List.scala:84)
> at scala.collection.convert.Wrappers$SeqWrapper.get(Wrappers.scala:66)
> at
> org.apache.carbondata.core.datastore.block.SegmentProperties.readAllComplexTypeChildren(SegmentProperties.java:445)
> at
> org.apache.carbondata.core.datastore.block.SegmentProperties.fillDimensionAndMeasureDetails(SegmentProperties.java:390)
> at
> org.apache.carbondata.core.datastore.block.SegmentProperties.<init>(SegmentProperties.java:173)
> at
> org.apache.carbondata.spark.rdd.CarbonMergerRDD$$anon$1.<init>(CarbonMergerRDD.scala:161)
> at
> org.apache.carbondata.spark.rdd.CarbonMergerRDD.internalCompute(CarbonMergerRDD.scala:79)
> at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:62)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
