[
https://issues.apache.org/jira/browse/CARBONDATA-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16146975#comment-16146975
]
Pallavi Singh commented on CARBONDATA-1422:
-------------------------------------------
Hi Zhichao Zhang,
Here is the stack-trace of the error:
17/08/30 15:12:30 INFO CatalystSqlParser: Parsing command: array<string>
17/08/30 15:12:30 AUDIT CarbonDataRDDFactory$:
[pallavi][hduser][Thread-136]Compaction request received for table
30aug_5.uniqdata
17/08/30 15:12:30 INFO HdfsFileLock: pool-24-thread-7 HDFS lock
path:hdfs://localhost:54310/user/hive/warehouse/carbon.store/30aug_5/uniqdata/compaction.lock
17/08/30 15:12:30 INFO CarbonDataRDDFactory$: pool-24-thread-7 Acquired the
compaction lock for table 30aug_5.uniqdata
17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified
for merge is 0
17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified
for merge is 1
17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified
for merge is 2
17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified
for merge is 3
17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified
for merge is 4
17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified
for merge is 5
17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified
for merge is 6
17/08/30 15:12:30 INFO DataManagementFunc$: pool-24-thread-7 loads identified
for merge is 7
17/08/30 15:12:30 INFO TableInfo: pool-28-thread-1 Table block size not
specified for 30aug_5_uniqdata. Therefore considering the default value 1024 MB
17/08/30 15:12:30 INFO DistributionUtil$: pool-28-thread-1 Requesting total
executors: 1
17/08/30 15:12:30 INFO CarbonMergerRDD: No.of Executors required=1 ,
spark.executor.instances=1, no.of.nodes where data present=1
17/08/30 15:12:30 INFO CarbonMergerRDD: Time taken to wait for executor
allocation is =0millis
17/08/30 15:12:30 INFO CarbonMergerRDD: for the node pallavi
17/08/30 15:12:30 INFO CarbonMergerRDD: Task ID is 0no. of blocks is 8
17/08/30 15:12:30 INFO CarbonMergerRDD: Identified no.of.Blocks:
8,parallelism: 4 , no.of.nodes: 1, no.of.tasks: 1
17/08/30 15:12:30 INFO CarbonMergerRDD: Time taken to identify Blocks to scan :
177
17/08/30 15:12:30 INFO CarbonMergerRDD: Node: pallavi, No.Of Blocks: 8
17/08/30 15:12:30 INFO SparkContext: Starting job: collect at
Compactor.scala:100
17/08/30 15:12:30 INFO DAGScheduler: Got job 4 (collect at Compactor.scala:100)
with 1 output partitions
17/08/30 15:12:30 INFO DAGScheduler: Final stage: ResultStage 4 (collect at
Compactor.scala:100)
17/08/30 15:12:30 INFO DAGScheduler: Parents of final stage: List()
17/08/30 15:12:30 INFO DAGScheduler: Missing parents: List()
17/08/30 15:12:30 INFO DAGScheduler: Submitting ResultStage 4
(CarbonMergerRDD[11] at RDD at CarbonRDD.scala:35), which has no missing parents
17/08/30 15:12:30 INFO MemoryStore: Block broadcast_6 stored as values in
memory (estimated size 14.2 KB, free 366.2 MB)
17/08/30 15:12:30 INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in
memory (estimated size 6.7 KB, free 366.2 MB)
17/08/30 15:12:30 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on
192.168.2.188:41587 (size: 6.7 KB, free: 366.3 MB)
17/08/30 15:12:30 INFO SparkContext: Created broadcast 6 from broadcast at
DAGScheduler.scala:996
17/08/30 15:12:30 INFO DAGScheduler: Submitting 1 missing tasks from
ResultStage 4 (CarbonMergerRDD[11] at RDD at CarbonRDD.scala:35)
17/08/30 15:12:30 INFO TaskSchedulerImpl: Adding task set 4.0 with 1 tasks
17/08/30 15:12:30 INFO TaskSetManager: Starting task 0.0 in stage 4.0 (TID 4,
192.168.2.188, executor 0, partition 0, ANY, 9296 bytes)
17/08/30 15:12:30 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on
192.168.2.188:38477 (size: 6.7 KB, free: 366.3 MB)
17/08/30 15:12:31 INFO TaskSetManager: Finished task 0.0 in stage 4.0 (TID 4)
in 266 ms on 192.168.2.188 (executor 0) (1/1)
17/08/30 15:12:31 INFO TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have
all completed, from pool
17/08/30 15:12:31 INFO DAGScheduler: ResultStage 4 (collect at
Compactor.scala:100) finished in 0.267 s
17/08/30 15:12:31 INFO DAGScheduler: Job 4 finished: collect at
Compactor.scala:100, took 0.276120 s
17/08/30 15:12:31 AUDIT Compactor$: [pallavi][hduser][Thread-143]Compaction
request failed for table 30aug_5.uniqdata
17/08/30 15:12:31 ERROR Compactor$: pool-28-thread-1 Compaction request failed
for table 30aug_5.uniqdata
17/08/30 15:12:31 ERROR DataManagementFunc$: pool-24-thread-7 Exception in
compaction thread java.lang.Exception: Compaction Failure in Merger Rdd.
java.util.concurrent.ExecutionException: java.lang.Exception: Compaction
Failure in Merger Rdd.
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at
org.apache.carbondata.spark.rdd.DataManagementFunc$$anonfun$executeCompaction$1.apply(DataManagementFunc.scala:193)
at
org.apache.carbondata.spark.rdd.DataManagementFunc$$anonfun$executeCompaction$1.apply(DataManagementFunc.scala:192)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at
org.apache.carbondata.spark.rdd.DataManagementFunc$.executeCompaction(DataManagementFunc.scala:192)
at
org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$$anon$2.run(CarbonDataRDDFactory.scala:273)
at
org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.startCompactionThreads(CarbonDataRDDFactory.scala:364)
at
org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.alterTableForCompaction(CarbonDataRDDFactory.scala:157)
at
org.apache.spark.sql.execution.command.AlterTableCompaction.processData(carbonTableSchema.scala:175)
at
org.apache.spark.sql.execution.command.AlterTableCompaction.run(carbonTableSchema.scala:139)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:74)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:135)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:132)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:113)
at
org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:87)
at
org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:87)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:185)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:64)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:592)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:699)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:220)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:163)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:160)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:173)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.Exception: Compaction Failure in Merger Rdd.
at
org.apache.carbondata.spark.rdd.Compactor$.triggerCompaction(Compactor.scala:143)
at
org.apache.carbondata.spark.rdd.Compactor.triggerCompaction(Compactor.scala)
at
org.apache.carbondata.spark.compaction.CompactionCallable.call(CompactionCallable.java:40)
at
org.apache.carbondata.spark.compaction.CompactionCallable.call(CompactionCallable.java:29)
... 4 more
17/08/30 15:12:31 ERROR CarbonDataRDDFactory$: pool-24-thread-7 Exception in
compaction thread java.lang.Exception: Compaction Failure in Merger Rdd.
17/08/30 15:12:31 INFO HdfsFileLock: pool-24-thread-7 Deleted the lock file
hdfs://localhost:54310/user/hive/warehouse/carbon.store/30aug_5/uniqdata/compaction.lock
17/08/30 15:12:31 ERROR CarbonDataRDDFactory$: pool-24-thread-7 Exception in
start compaction thread. Exception in compaction java.lang.Exception:
Compaction Failure in Merger Rdd.
17/08/30 15:12:31 ERROR AbstractDFSCarbonFile: pool-24-thread-7 Exception
occurred:File does not exist:
hdfs://localhost:54310/user/hive/warehouse/carbon.store/30aug_5/uniqdata/compaction.lock
17/08/30 15:12:31 ERROR HdfsFileLock: pool-24-thread-7 Not able to delete the
lock file because it is not existed in location
hdfs://localhost:54310/user/hive/warehouse/carbon.store/30aug_5/uniqdata/compaction.lock
17/08/30 15:12:31 ERROR SparkExecuteStatementOperation: Error executing query,
currentState RUNNING,
java.lang.RuntimeException: Compaction failed. Please check logs for more info.
Exception in compaction java.lang.Exception: Compaction Failure in Merger Rdd.
at scala.sys.package$.error(package.scala:27)
at
org.apache.spark.sql.execution.command.AlterTableCompaction.processData(carbonTableSchema.scala:184)
at
org.apache.spark.sql.execution.command.AlterTableCompaction.run(carbonTableSchema.scala:139)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:74)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:135)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:132)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:113)
at
org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:87)
at
org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:87)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:185)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:64)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:592)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:699)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:220)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:163)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:160)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:173)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
17/08/30 15:12:31 ERROR SparkExecuteStatementOperation: Error running hive
query:
org.apache.hive.service.cli.HiveSQLException: java.lang.RuntimeException:
Compaction failed. Please check logs for more info. Exception in compaction
java.lang.Exception: Compaction Failure in Merger Rdd.
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:258)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:163)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:160)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:173)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
> Major and Minor Compaction Failing
> ----------------------------------
>
> Key: CARBONDATA-1422
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1422
> Project: CarbonData
> Issue Type: Bug
> Affects Versions: 1.2.0
> Reporter: Pallavi Singh
> Fix For: 1.2.0
>
>
> The major and minor compaction is failing.
> The Compaction is failing in the default scenario where the table property
> dictionary_include is not specified which is the default behaviour.
> Please find the error logs below :
> 0: jdbc:hive2://localhost:10000> show segments for table uniqdata;
> +--------------------+----------+--------------------------+--------------------------+--+
> | SegmentSequenceId | Status | Load Start Time | Load End
> Time |
> +--------------------+----------+--------------------------+--------------------------+--+
> | 9 | Success | 2017-08-29 11:17:29.927 | 2017-08-29
> 11:17:30.555 |
> | 8 | Success | 2017-08-29 11:17:27.572 | 2017-08-29
> 11:17:28.363 |
> | 7 | Success | 2017-08-29 11:17:23.583 | 2017-08-29
> 11:17:25.112 |
> | 6 | Success | 2017-08-29 11:17:07.966 | 2017-08-29
> 11:17:09.322 |
> | 5 | Success | 2017-08-29 10:38:15.727 | 2017-08-29
> 10:38:16.548 |
> | 4 | Success | 2017-08-29 10:37:13.053 | 2017-08-29
> 10:37:13.888 |
> | 3 | Success | 2017-08-29 10:36:57.851 | 2017-08-29
> 10:36:59.08 |
> | 2 | Success | 2017-08-29 10:36:49.439 | 2017-08-29
> 10:36:50.373 |
> | 1 | Success | 2017-08-29 10:36:37.365 | 2017-08-29
> 10:36:38.768 |
> | 0 | Success | 2017-08-29 10:36:21.011 | 2017-08-29
> 10:36:26.1 |
> +--------------------+----------+--------------------------+--------------------------+--+
> 10 rows selected (0.081 seconds)
> 0: jdbc:hive2://localhost:10000> ALTER TABLE uniqdata COMPACT 'MINOR';
> Error: java.lang.RuntimeException: Compaction failed. Please check logs for
> more info. Exception in compaction java.lang.Exception: Compaction Failure in
> Merger Rdd. (state=,code=0)
> 0: jdbc:hive2://localhost:10000> ALTER TABLE uniqdata COMPACT 'MAJOR';
> Error: java.lang.RuntimeException: Compaction failed. Please check logs for
> more info. Exception in compaction java.lang.Exception: Compaction Failure in
> Merger Rdd. (state=,code=0)
> 0: jdbc:hive2://localhost:10000>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
