Hi, It seems like a bug in 1.1.1 version, can you try out on the latest master branch once.
Regards, Ravindra. On 24 August 2017 at 14:52, lk_hadoop <[email protected]> wrote: > @Ravindra carbondata1.1.1 spark2.1.0 yarn 2.7.3 and > catalog_returns_1_4.dat size is 5.5G > > Container: container_1499044620720_0086_01_000007 on XX_57375 > ============================================================ > ===================== > LogType:stderr > Log Upload Time:Thu Aug 24 16:03:58 +0800 2017 > LogLength:47927 > Log Contents: > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in [jar:file:/fs/data10/yarn/ > usercache/hadoop/filecache/209/carbondata.tar.gz/ > carbonlib/carbondata_2.11-1.1.1-shade-hadoop2.7.2.jar!/org/ > slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in [jar:file:/fs/data11/yarn/ > usercache/hadoop/filecache/210/__spark_libs__ > 5076531087226320409.zip/slf4j-log4j12-1.7.16.jar!/org/slf4j/ > impl/StaticLoggerBinder.class] > SLF4J: Found binding in [jar:file:/home/hadoop/dmp/ > hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7. > 10.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > 17/08/24 15:57:13 INFO executor.CoarseGrainedExecutorBackend: Started > daemon with process name: 84197@kafka03 > 17/08/24 15:57:13 INFO util.SignalUtils: Registered signal handler for TERM > 17/08/24 15:57:13 INFO util.SignalUtils: Registered signal handler for HUP > 17/08/24 15:57:13 INFO util.SignalUtils: Registered signal handler for INT > 17/08/24 15:57:13 INFO spark.SecurityManager: Changing view acls to: hadoop > 17/08/24 15:57:13 INFO spark.SecurityManager: Changing modify acls to: > hadoop > 17/08/24 15:57:13 INFO spark.SecurityManager: Changing view acls groups > to: > 17/08/24 15:57:13 INFO spark.SecurityManager: Changing modify acls groups > to: > 17/08/24 15:57:13 INFO spark.SecurityManager: SecurityManager: > authentication disabled; ui acls disabled; users with view permissions: > Set(hadoop); groups with view permissions: Set(); users with modify > permissions: Set(hadoop); groups with modify permissions: Set() > 17/08/24 15:57:14 INFO client.TransportClientFactory: Successfully created > connection to /192.168.0.142:51695 after 82 ms (0 ms spent in bootstraps) > 17/08/24 15:57:14 INFO spark.SecurityManager: Changing view acls to: hadoop > 17/08/24 15:57:14 INFO spark.SecurityManager: Changing modify acls to: > hadoop > 17/08/24 15:57:14 INFO spark.SecurityManager: Changing view acls groups > to: > 17/08/24 15:57:14 INFO spark.SecurityManager: Changing modify acls groups > to: > 17/08/24 15:57:14 INFO spark.SecurityManager: SecurityManager: > authentication disabled; ui acls disabled; users with view permissions: > Set(hadoop); groups with view permissions: Set(); users with modify > permissions: Set(hadoop); groups with modify permissions: Set() > 17/08/24 15:57:14 INFO client.TransportClientFactory: Successfully created > connection to /192.168.0.142:51695 after 1 ms (0 ms spent in bootstraps) > 17/08/24 15:57:14 INFO storage.DiskBlockManager: Created local directory > at /fs/data10/yarn/usercache/hadoop/appcache/application_ > 1499044620720_0086/blockmgr-ffa868ad-0ce0-43c5-90be-60ae014d8863 > 17/08/24 15:57:14 INFO storage.DiskBlockManager: Created local directory > at /fs/data11/yarn/usercache/hadoop/appcache/application_ > 1499044620720_0086/blockmgr-2fbb8304-06b4-4843-88be-ee7b497c92be > 17/08/24 15:57:14 INFO storage.DiskBlockManager: Created local directory > at /fs/data12/yarn/usercache/hadoop/appcache/application_ > 1499044620720_0086/blockmgr-9c04bab5-d599-4fdc-8a62-d8767037b18a > 17/08/24 15:57:14 INFO memory.MemoryStore: MemoryStore started with > capacity 47.8 GB > 17/08/24 15:57:14 INFO executor.CoarseGrainedExecutorBackend: Connecting > to driver: spark://[email protected]:51695 > 17/08/24 15:57:14 INFO executor.CoarseGrainedExecutorBackend: > Successfully registered with driver > 17/08/24 15:57:14 INFO executor.Executor: Starting executor ID 6 on host > xxx > 17/08/24 15:57:14 INFO util.Utils: Successfully started service > 'org.apache.spark.network.netty.NettyBlockTransferService' on port 60309. > 17/08/24 15:57:14 INFO netty.NettyBlockTransferService: Server created on > xxx > 17/08/24 15:57:14 INFO storage.BlockManager: Using > org.apache.spark.storage.RandomBlockReplicationPolicy for block > replication policy > 17/08/24 15:57:14 INFO storage.BlockManagerMaster: Registering > BlockManager BlockManagerId(6, xxx, 60309, None) > 17/08/24 15:57:14 INFO storage.BlockManagerMaster: Registered BlockManager > BlockManagerId(6, xxx, 60309, None) > 17/08/24 15:57:14 INFO storage.BlockManager: Initialized BlockManager: > BlockManagerId(6, xxx, 60309, None) > 17/08/24 15:57:14 INFO executor.Executor: Using REPL class URI: spark:// > 192.168.0.142:51695/classes > 17/08/24 15:59:21 INFO executor.CoarseGrainedExecutorBackend: Got > assigned task 3 > 17/08/24 15:59:21 INFO executor.CoarseGrainedExecutorBackend: Got > assigned task 9 > 17/08/24 15:59:21 INFO executor.CoarseGrainedExecutorBackend: Got > assigned task 15 > 17/08/24 15:59:21 INFO executor.CoarseGrainedExecutorBackend: Got > assigned task 21 > 17/08/24 15:59:21 INFO executor.Executor: Running task 8.0 in stage 0.0 > (TID 9) > 17/08/24 15:59:21 INFO executor.Executor: Running task 20.0 in stage 0.0 > (TID 21) > 17/08/24 15:59:21 INFO executor.Executor: Running task 14.0 in stage 0.0 > (TID 15) > 17/08/24 15:59:21 INFO executor.Executor: Running task 3.0 in stage 0.0 > (TID 3) > 17/08/24 15:59:21 INFO broadcast.TorrentBroadcast: Started reading > broadcast variable 1 > 17/08/24 15:59:21 INFO client.TransportClientFactory: Successfully created > connection to /192.168.0.142:48571 after 1 ms (0 ms spent in bootstraps) > 17/08/24 15:59:21 INFO memory.MemoryStore: Block broadcast_1_piece0 stored > as bytes in memory (estimated size 11.0 KB, free 47.8 GB) > 17/08/24 15:59:21 INFO broadcast.TorrentBroadcast: Reading broadcast > variable 1 took 108 ms > 17/08/24 15:59:21 INFO memory.MemoryStore: Block broadcast_1 stored as > values in memory (estimated size 29.6 KB, free 47.8 GB) > 17/08/24 15:59:21 INFO util.CarbonProperties: Executor task launch > worker-0 Property file path: /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/carbon.properties > 17/08/24 15:59:21 INFO util.CarbonProperties: Executor task launch > worker-0 ------Using Carbon.properties -------- > 17/08/24 15:59:21 INFO util.CarbonProperties: Executor task launch > worker-0 {carbon.graph.rowset.size=100000, carbon.enable.quick.filter=false, > carbon.number.of.cores=4, carbon.sort.file.buffer.size=20, > carbon.number.of.cores.while.compacting=2, > carbon.compaction.level.threshold=4,3, > carbon.lock.type=HDFSLOCK, carbon.number.of.cores.while.loading=6, > carbon.badRecords.location=/opt/Carbon/Spark/badrecords, > carbon.sort.size=500000, carbon.inmemory.record.size=120000, > carbon.enableXXHash=true, > carbon.ddl.base.hdfs.url=hdfs://kafka01:9000/opt/data, > carbon.major.compaction.size=1024, carbon.storelocation=hdfs:// > kafka01:9000/Opt/CarbonStore} > 17/08/24 15:59:21 INFO util.CarbonProperties: Executor task launch > worker-0 Carbon Current data file version: V3 > 17/08/24 15:59:21 INFO util.CarbonProperties: Executor task launch > worker-0 Executor start up wait time: 5 > 17/08/24 15:59:21 INFO util.CarbonProperties: Executor task launch > worker-0 Blocklet Size Configured value is "64 > 17/08/24 15:59:21 INFO rdd.NewHadoopRDD: Input split: > hdfs://kafka01:9000/tpcds/source/catalog_returns/catalog_returns_1_4.dat: > 5368709120+268435456 > 17/08/24 15:59:21 INFO rdd.NewHadoopRDD: Input split: > hdfs://kafka01:9000/tpcds/source/catalog_returns/catalog_returns_1_4.dat: > 2147483648+268435456 > 17/08/24 15:59:21 INFO rdd.NewHadoopRDD: Input split: > hdfs://kafka01:9000/tpcds/source/catalog_returns/catalog_returns_1_4.dat: > 3758096384+268435456 > 17/08/24 15:59:21 INFO rdd.NewHadoopRDD: Input split: > hdfs://kafka01:9000/tpcds/source/catalog_returns/catalog_returns_1_4.dat: > 805306368+268435456 > 17/08/24 15:59:21 INFO broadcast.TorrentBroadcast: Started reading > broadcast variable 0 > 17/08/24 15:59:21 INFO client.TransportClientFactory: Successfully created > connection to kafka01.youedata.com/192.168.0.140:37515 after 2 ms (0 ms > spent in bootstraps) > 17/08/24 15:59:21 INFO memory.MemoryStore: Block broadcast_0_piece0 stored > as bytes in memory (estimated size 24.1 KB, free 47.8 GB) > 17/08/24 15:59:21 INFO broadcast.TorrentBroadcast: Reading broadcast > variable 0 took 35 ms > 17/08/24 15:59:21 INFO memory.MemoryStore: Block broadcast_0 stored as > values in memory (estimated size 424.1 KB, free 47.8 GB) > 17/08/24 15:59:22 INFO client.TransportClientFactory: Successfully created > connection to /192.168.0.142:51695 after 2 ms (0 ms spent in bootstraps) > 17/08/24 15:59:22 INFO codegen.CodeGenerator: Code generated in 310.137373 > ms > 17/08/24 15:59:22 INFO codegen.CodeGenerator: Code generated in 19.689183 > ms > 17/08/24 15:59:23 INFO codegen.CodeGenerator: Code generated in 16.974435 > ms > 17/08/24 15:59:24 INFO codegen.CodeGenerator: Code generated in 174.032802 > ms > 17/08/24 15:59:39 INFO executor.Executor: Finished task 20.0 in stage 0.0 > (TID 21). 2515 bytes result sent to driver > 17/08/24 15:59:39 INFO executor.Executor: Finished task 14.0 in stage 0.0 > (TID 15). 1803 bytes result sent to driver > 17/08/24 15:59:39 INFO executor.Executor: Finished task 3.0 in stage 0.0 > (TID 3). 2515 bytes result sent to driver > 17/08/24 15:59:39 INFO executor.Executor: Finished task 8.0 in stage 0.0 > (TID 9). 1716 bytes result sent to driver > 17/08/24 16:01:21 INFO executor.CoarseGrainedExecutorBackend: Got > assigned task 24 > 17/08/24 16:01:21 INFO executor.Executor: Running task 0.0 in stage 2.0 > (TID 24) > 17/08/24 16:01:21 INFO spark.MapOutputTrackerWorker: Updating epoch to 1 > and clearing cache > 17/08/24 16:01:21 INFO broadcast.TorrentBroadcast: Started reading > broadcast variable 4 > 17/08/24 16:01:21 INFO memory.MemoryStore: Block broadcast_4_piece0 stored > as bytes in memory (estimated size 8.4 KB, free 47.8 GB) > 17/08/24 16:01:21 INFO broadcast.TorrentBroadcast: Reading broadcast > variable 4 took 7 ms > 17/08/24 16:01:21 INFO memory.MemoryStore: Block broadcast_4 stored as > values in memory (estimated size 24.9 KB, free 47.8 GB) > 17/08/24 16:01:21 INFO broadcast.TorrentBroadcast: Started reading > broadcast variable 3 > 17/08/24 16:01:21 INFO memory.MemoryStore: Block broadcast_3_piece0 stored > as bytes in memory (estimated size 46.5 KB, free 47.8 GB) > 17/08/24 16:01:21 INFO broadcast.TorrentBroadcast: Reading broadcast > variable 3 took 7 ms > 17/08/24 16:01:21 INFO memory.MemoryStore: Block broadcast_3 stored as > values in memory (estimated size 842.7 KB, free 47.8 GB) > 17/08/24 16:01:21 INFO rdd.NewCarbonDataLoadRDD: Input split: > kafka03.youedata.com > 17/08/24 16:01:21 INFO rdd.NewCarbonDataLoadRDD: The Block Count in this > node :28 > 17/08/24 16:01:21 WARN util.CarbonDataProcessorUtil: [Executor task launch > worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] sort scope is set to LOCAL_SORT > 17/08/24 16:01:21 INFO newflow.AbstractDataLoadProcessorStep: Thread-7 > Rows processed in step Input Processor : 0 > 17/08/24 16:01:21 INFO newflow.AbstractDataLoadProcessorStep: Thread-8 > Rows processed in step Data Converter : 0 > 17/08/24 16:01:21 INFO newflow.AbstractDataLoadProcessorStep: Thread-9 > Rows processed in step Sort Processor : 0 > 17/08/24 16:01:21 INFO newflow.AbstractDataLoadProcessorStep: Thread-10 > Rows processed in step Data Writer : 0 > 17/08/24 16:01:22 INFO cache.CacheProvider: [Executor task launch > worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Executor LRU cache size not > configured. Initializing with driver LRU cache size. > 17/08/24 16:01:22 INFO cache.CarbonLRUCache: [Executor task launch > worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] LRU cache size not configured. > Therefore default behavior will be considered and no LRU based eviction of > columns will be done > 17/08/24 16:01:31 INFO newflow.AbstractDataLoadProcessorStep: Thread-7 > Rows processed in step Input Processor : 0 > 17/08/24 16:01:31 INFO newflow.AbstractDataLoadProcessorStep: Thread-8 > Rows processed in step Data Converter : 0 > 17/08/24 16:01:31 INFO newflow.AbstractDataLoadProcessorStep: Thread-9 > Rows processed in step Sort Processor : 0 > 17/08/24 16:01:31 INFO newflow.AbstractDataLoadProcessorStep: Thread-10 > Rows processed in step Data Writer : 0 > 17/08/24 16:01:36 WARN util.CarbonDataProcessorUtil: [Executor task launch > worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] batch sort size is set to 0 > 17/08/24 16:01:36 INFO sortdata.SortParameters: [Executor task launch > worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Sort size for table: 500000 > 17/08/24 16:01:36 INFO sortdata.SortParameters: [Executor task launch > worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Number of intermediate file to be > merged: 20 > 17/08/24 16:01:36 INFO sortdata.SortParameters: [Executor task launch > worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] File Buffer Size: 1048576 > 17/08/24 16:01:36 INFO sortdata.SortParameters: [Executor task launch > worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] temp file location/fs/data10/yarn/ > usercache/hadoop/appcache/application_1499044620720_ > 0086/container_1499044620720_0086_01_000007/tmp/ > 4688236258085316/0/default/carbon_catalog_returns2/Fact/ > Part0/Segment_0/0/sortrowtmp > 17/08/24 16:01:36 WARN util.CarbonDataProcessorUtil: [Executor task launch > worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] sort scope is set to LOCAL_SORT > 17/08/24 16:01:36 INFO newflow.DataLoadExecutor: [Executor task launch > worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Data Loading is started for table > carbon_catalog_returns2 > 17/08/24 16:01:54 INFO newflow.AbstractDataLoadProcessorStep: Thread-10 > Rows processed in step Data Writer : 0 > 17/08/24 16:01:54 INFO newflow.AbstractDataLoadProcessorStep: Thread-7 > Rows processed in step Input Processor : 489000 > 17/08/24 16:01:54 INFO newflow.AbstractDataLoadProcessorStep: Thread-9 > Rows processed in step Sort Processor : 487000 > 17/08/24 16:01:54 INFO newflow.AbstractDataLoadProcessorStep: Thread-8 > Rows processed in step Data Converter : 487000 > 17/08/24 16:01:57 INFO sortdata.SortDataRows: pool-16-thread-2 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688269866804135.sorttemp is: 3069 > 17/08/24 16:01:57 INFO sortdata.SortDataRows: pool-16-thread-1 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688269560609898.sorttemp is: 3512 > 17/08/24 16:01:58 INFO sortdata.SortDataRows: pool-16-thread-3 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688270205697874.sorttemp is: 2751 > 17/08/24 16:02:00 INFO sortdata.SortDataRows: pool-16-thread-2 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688272980143789.sorttemp is: 2966 > 17/08/24 16:02:01 INFO sortdata.SortDataRows: pool-16-thread-1 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688273622524925.sorttemp is: 2815 > 17/08/24 16:02:02 INFO sortdata.SortDataRows: pool-16-thread-3 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688274076536424.sorttemp is: 2925 > 17/08/24 16:02:04 INFO sortdata.SortDataRows: pool-16-thread-2 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688275840260663.sorttemp is: 3445 > 17/08/24 16:02:04 INFO newflow.AbstractDataLoadProcessorStep: Thread-10 > Rows processed in step Data Writer : 0 > 17/08/24 16:02:04 INFO newflow.AbstractDataLoadProcessorStep: Thread-7 > Rows processed in step Input Processor : 5045000 > 17/08/24 16:02:04 INFO newflow.AbstractDataLoadProcessorStep: Thread-9 > Rows processed in step Sort Processor : 5042000 > 17/08/24 16:02:04 INFO newflow.AbstractDataLoadProcessorStep: Thread-8 > Rows processed in step Data Converter : 5043000 > 17/08/24 16:02:04 INFO sortdata.SortDataRows: pool-16-thread-1 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688276446874268.sorttemp is: 3305 > 17/08/24 16:02:05 INFO sortdata.SortDataRows: pool-16-thread-3 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688277415523789.sorttemp is: 3297 > 17/08/24 16:02:08 INFO sortdata.SortDataRows: pool-16-thread-2 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688279224093804.sorttemp is: 4084 > 17/08/24 16:02:08 INFO sortdata.SortDataRows: pool-16-thread-1 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688279682044167.sorttemp is: 3909 > 17/08/24 16:02:09 INFO sortdata.SortDataRows: pool-16-thread-3 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688280238732307.sorttemp is: 3877 > 17/08/24 16:02:10 INFO sortdata.SortDataRows: pool-16-thread-2 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688283131136222.sorttemp is: 2580 > 17/08/24 16:02:11 INFO sortdata.SortDataRows: pool-16-thread-1 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688283724960116.sorttemp is: 2806 > 17/08/24 16:02:12 INFO sortdata.SortDataRows: pool-16-thread-3 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688284161309296.sorttemp is: 2868 > 17/08/24 16:02:14 INFO newflow.AbstractDataLoadProcessorStep: Thread-10 > Rows processed in step Data Writer : 0 > 17/08/24 16:02:14 INFO newflow.AbstractDataLoadProcessorStep: Thread-7 > Rows processed in step Input Processor : 9505000 > 17/08/24 16:02:14 INFO newflow.AbstractDataLoadProcessorStep: Thread-9 > Rows processed in step Sort Processor : 9499000 > 17/08/24 16:02:14 INFO newflow.AbstractDataLoadProcessorStep: Thread-8 > Rows processed in step Data Converter : 9505000 > 17/08/24 16:02:14 INFO sortdata.SortDataRows: pool-16-thread-2 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688285856830704.sorttemp is: 3782 > 17/08/24 16:02:15 INFO sortdata.SortDataRows: pool-16-thread-1 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688286523938421.sorttemp is: 3657 > 17/08/24 16:02:15 INFO sortdata.SortDataRows: pool-16-thread-3 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688287952674573.sorttemp is: 3760 > 17/08/24 16:02:17 INFO sortdata.SortDataRows: pool-16-thread-2 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688289648034842.sorttemp is: 2804 > 17/08/24 16:02:18 INFO sortdata.SortDataRows: pool-16-thread-1 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688290214867296.sorttemp is: 3682 > 17/08/24 16:02:19 INFO sortdata.SortDataRows: pool-16-thread-3 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688290712710427.sorttemp is: 3387 > 17/08/24 16:02:19 INFO sortdata.SortDataRows: [Executor task launch > worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] File based sorting will be used > 17/08/24 16:02:21 INFO sortdata.SortDataRows: pool-16-thread-2 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688293177154616.sorttemp is: 3535 > 17/08/24 16:02:21 INFO sortdata.SortDataRows: pool-16-thread-1 Time taken > to sort and write sort temp file /fs/data10/yarn/usercache/ > hadoop/appcache/application_1499044620720_0086/container_ > 1499044620720_0086_01_000007/tmp/4688236258085316/0/ > default/carbon_catalog_returns2/Fact/Part0/Segment_0/ > 0/sortrowtmp/carbon_catalog_returns24688293732092549.sorttemp is: 2696 > 17/08/24 16:02:22 INFO impl.ParallelReadMergeSorterImpl: [Executor task > launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Record Processed For table: > carbon_catalog_returns2 > 17/08/24 16:02:22 INFO store.SingleThreadFinalSortFilesMerger: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Number of temp file: 24 > 17/08/24 16:02:22 INFO store.SingleThreadFinalSortFilesMerger: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] File Buffer Size: 873813 > 17/08/24 16:02:22 INFO store.SingleThreadFinalSortFilesMerger: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Started adding first record from > each file > 17/08/24 16:02:22 INFO store.SingleThreadFinalSortFilesMerger: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Heap Size24 > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Initializing writer executors > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Number of rows per column blocklet > 32000 > 17/08/24 16:02:22 INFO writer.AbstractFactDataWriter: [Executor task > launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total file size: 1073741824 and > dataBlock Size: 966367642 > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 32000 > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 64000 > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 96000 > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 128000 > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 160000 > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 192000 > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-5 Number Of records processed: 32000 > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-6 Number Of records processed: 32000 > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-4 Number Of records processed: 32000 > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-3 Number Of records processed: 32000 > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-1 Number Of records processed: 32000 > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 224000 > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-2 Number Of records processed: 32000 > 17/08/24 16:02:22 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 256000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 288000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 320000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 352000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 384000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-4 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-3 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-1 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-5 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 416000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-6 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-2 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 448000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 480000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 512000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 544000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 576000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-4 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 608000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-3 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 640000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-1 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-5 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 672000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-6 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 704000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 736000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-2 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 768000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-4 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total Number Of records added to > store: 800000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-3 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO v3.CarbonFactDataWriterImplV3: pool-44-thread-1 > Number of Pages for blocklet is: 19 :Rows Added: 608000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-1 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-5 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-6 Number Of records processed: 32000 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-2 Number Of records processed: 32000 > 17/08/24 16:02:23 ERROR store.CarbonFactDataHandlerColumnar: > pool-44-thread-1 Problem while writing the carbon data file > java.nio.BufferUnderflowException > at java.nio.Buffer.nextGetIndex(Buffer.java:506) > at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:412) > at org.apache.carbondata.core.util.CarbonMetadataUtil.compareMeasureData( > CarbonMetadataUtil.java:899) > at org.apache.carbondata.core.util.CarbonMetadataUtil.getBlockletIndex( > CarbonMetadataUtil.java:286) > at org.apache.carbondata.processing.store.writer.v3. > CarbonFactDataWriterImplV3.writeDataToFile(CarbonFactDataWriterImplV3. > java:508) > at org.apache.carbondata.processing.store.writer.v3. > CarbonFactDataWriterImplV3.writeDataToFile(CarbonFactDataWriterImplV3. > java:396) > at org.apache.carbondata.processing.store.writer.v3. > CarbonFactDataWriterImplV3.writeBlockletData(CarbonFactDataWriterImplV3. > java:345) > at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$ > Consumer.call(CarbonFactDataHandlerColumnar.java:1400) > at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$ > Consumer.call(CarbonFactDataHandlerColumnar.java:1380) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > 17/08/24 16:02:23 ERROR store.CarbonFactDataHandlerColumnar: > pool-43-thread-4 null > java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject. > reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ > ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) > at java.util.concurrent.ThreadPoolExecutor.awaitTermination( > ThreadPoolExecutor.java:1465) > at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar. > createNodeHolderObjectWithOutKettle(CarbonFactDataHandlerColumnar. > java:811) > at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar. > processDataRows(CarbonFactDataHandlerColumnar.java:648) > at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar. > access$1100(CarbonFactDataHandlerColumnar.java:86) > at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$ > Producer.call(CarbonFactDataHandlerColumnar.java:1362) > at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$ > Producer.call(CarbonFactDataHandlerColumnar.java:1339) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > 17/08/24 16:02:23 ERROR steps.DataWriterProcessorStepImpl: [Executor task > launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Failed for table: > carbon_catalog_returns2 in DataWriterProcessorStepImpl > org.apache.carbondata.processing.newflow.exception.CarbonDataLoadingException: > unable to generate the mdkey > at org.apache.carbondata.processing.newflow.steps. > DataWriterProcessorStepImpl.processBatch(DataWriterProcessorStepImpl. > java:181) > at org.apache.carbondata.processing.newflow.steps. > DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:111) > at org.apache.carbondata.processing.newflow.DataLoadExecutor.execute( > DataLoadExecutor.java:48) > at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD$$anon$1.<init>( > NewCarbonDataLoadRDD.scala:243) > at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD. > compute(NewCarbonDataLoadRDD.scala:220) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) > at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) > at org.apache.spark.scheduler.Task.run(Task.scala:99) > at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.util.concurrent.RejectedExecutionException: Task > java.util.concurrent.FutureTask@67098e0 rejected from > java.util.concurrent.ThreadPoolExecutor@5b91b608[Shutting down, pool size > = 1, active threads = 1, queued tasks = 0, completed tasks = 24] > at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution( > ThreadPoolExecutor.java:2047) > at java.util.concurrent.ThreadPoolExecutor.reject( > ThreadPoolExecutor.java:823) > at java.util.concurrent.ThreadPoolExecutor.execute( > ThreadPoolExecutor.java:1369) > at java.util.concurrent.AbstractExecutorService.submit( > AbstractExecutorService.java:134) > at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar. > addDataToStore(CarbonFactDataHandlerColumnar.java:466) > at org.apache.carbondata.processing.newflow.steps. > DataWriterProcessorStepImpl.processBatch(DataWriterProcessorStepImpl. > java:178) > ... 12 more > 17/08/24 16:02:23 INFO newflow.AbstractDataLoadProcessorStep: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total rows processed in step Data > Writer: 831000 > 17/08/24 16:02:23 INFO newflow.AbstractDataLoadProcessorStep: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total rows processed in step Sort > Processor: 11946858 > 17/08/24 16:02:23 INFO newflow.AbstractDataLoadProcessorStep: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total rows processed in step Data > Converter: 11946858 > 17/08/24 16:02:23 INFO newflow.AbstractDataLoadProcessorStep: [Executor > task launch worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] Total rows processed in step Input > Processor: 11946858 > 17/08/24 16:02:23 INFO rdd.NewCarbonDataLoadRDD: DataLoad failure > > org.apache.carbondata.processing.newflow.exception.CarbonDataLoadingException: > There is an unexpected error: unable to generate the mdkey > at org.apache.carbondata.processing.newflow.steps. > DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:125) > at org.apache.carbondata.processing.newflow.DataLoadExecutor.execute( > DataLoadExecutor.java:48) > at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD$$anon$1.<init>( > NewCarbonDataLoadRDD.scala:243) > at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD. > compute(NewCarbonDataLoadRDD.scala:220) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) > at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) > at org.apache.spark.scheduler.Task.run(Task.scala:99) > at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: > org.apache.carbondata.processing.newflow.exception.CarbonDataLoadingException: > unable to generate the mdkey > at org.apache.carbondata.processing.newflow.steps. > DataWriterProcessorStepImpl.processBatch(DataWriterProcessorStepImpl. > java:181) > at org.apache.carbondata.processing.newflow.steps. > DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:111) > ... 11 more > Caused by: java.util.concurrent.RejectedExecutionException: Task > java.util.concurrent.FutureTask@67098e0 rejected from > java.util.concurrent.ThreadPoolExecutor@5b91b608[Shutting down, pool size > = 1, active threads = 1, queued tasks = 0, completed tasks = 24] > at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution( > ThreadPoolExecutor.java:2047) > at java.util.concurrent.ThreadPoolExecutor.reject( > ThreadPoolExecutor.java:823) > at java.util.concurrent.ThreadPoolExecutor.execute( > ThreadPoolExecutor.java:1369) > at java.util.concurrent.AbstractExecutorService.submit( > AbstractExecutorService.java:134) > at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar. > addDataToStore(CarbonFactDataHandlerColumnar.java:466) > at org.apache.carbondata.processing.newflow.steps. > DataWriterProcessorStepImpl.processBatch(DataWriterProcessorStepImpl. > java:178) > ... 12 more > 17/08/24 16:02:23 ERROR rdd.NewCarbonDataLoadRDD: [Executor task launch > worker-4][partitionID:default_carbon_catalog_returns2_ > 03f3c47c-cd02-4774-ac7f-d7a2aa97c7ef] > org.apache.carbondata.processing.newflow.exception.CarbonDataLoadingException: > There is an unexpected error: unable to generate the mdkey > at org.apache.carbondata.processing.newflow.steps. > DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:125) > at org.apache.carbondata.processing.newflow.DataLoadExecutor.execute( > DataLoadExecutor.java:48) > at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD$$anon$1.<init>( > NewCarbonDataLoadRDD.scala:243) > at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD. > compute(NewCarbonDataLoadRDD.scala:220) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) > at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) > at org.apache.spark.scheduler.Task.run(Task.scala:99) > at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: > org.apache.carbondata.processing.newflow.exception.CarbonDataLoadingException: > unable to generate the mdkey > at org.apache.carbondata.processing.newflow.steps. > DataWriterProcessorStepImpl.processBatch(DataWriterProcessorStepImpl. > java:181) > at org.apache.carbondata.processing.newflow.steps. > DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:111) > ... 11 more > Caused by: java.util.concurrent.RejectedExecutionException: Task > java.util.concurrent.FutureTask@67098e0 rejected from > java.util.concurrent.ThreadPoolExecutor@5b91b608[Shutting down, pool size > = 1, active threads = 1, queued tasks = 0, completed tasks = 24] > at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution( > ThreadPoolExecutor.java:2047) > at java.util.concurrent.ThreadPoolExecutor.reject( > ThreadPoolExecutor.java:823) > at java.util.concurrent.ThreadPoolExecutor.execute( > ThreadPoolExecutor.java:1369) > at java.util.concurrent.AbstractExecutorService.submit( > AbstractExecutorService.java:134) > at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar. > addDataToStore(CarbonFactDataHandlerColumnar.java:466) > at org.apache.carbondata.processing.newflow.steps. > DataWriterProcessorStepImpl.processBatch(DataWriterProcessorStepImpl. > java:178) > ... 12 more > 17/08/24 16:02:23 INFO load.CarbonLoaderUtil: pool-71-thread-1 Deleted the > local store location/fs/data10/yarn/usercache/hadoop/appcache/ > application_1499044620720_0086/container_1499044620720_0086_01_000007/tmp/4688236258085316/0 > : TIme taken: 3 > 17/08/24 16:02:23 INFO store.CarbonFactDataHandlerColumnar: > pool-43-thread-4 Number Of records processed: 32000 > 17/08/24 16:02:23 ERROR executor.Executor: Exception in task 0.0 in stage > 2.0 (TID 24) > > org.apache.carbondata.processing.newflow.exception.CarbonDataLoadingException: > There is an unexpected error: unable to generate the mdkey > at org.apache.carbondata.processing.newflow.steps. > DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:125) > at org.apache.carbondata.processing.newflow.DataLoadExecutor.execute( > DataLoadExecutor.java:48) > at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD$$anon$1.<init>( > NewCarbonDataLoadRDD.scala:243) > at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD. > compute(NewCarbonDataLoadRDD.scala:220) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) > at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) > at org.apache.spark.scheduler.Task.run(Task.scala:99) > at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: > org.apache.carbondata.processing.newflow.exception.CarbonDataLoadingException: > unable to generate the mdkey > at org.apache.carbondata.processing.newflow.steps. > DataWriterProcessorStepImpl.processBatch(DataWriterProcessorStepImpl. > java:181) > at org.apache.carbondata.processing.newflow.steps. > DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:111) > ... 11 more > Caused by: java.util.concurrent.RejectedExecutionException: Task > java.util.concurrent.FutureTask@67098e0 rejected from > java.util.concurrent.ThreadPoolExecutor@5b91b608[Shutting down, pool size > = 1, active threads = 1, queued tasks = 0, completed tasks = 24] > at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution( > ThreadPoolExecutor.java:2047) > at java.util.concurrent.ThreadPoolExecutor.reject( > ThreadPoolExecutor.java:823) > at java.util.concurrent.ThreadPoolExecutor.execute( > ThreadPoolExecutor.java:1369) > at java.util.concurrent.AbstractExecutorService.submit( > AbstractExecutorService.java:134) > at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar. > addDataToStore(CarbonFactDataHandlerColumnar.java:466) > at org.apache.carbondata.processing.newflow.steps. > DataWriterProcessorStepImpl.processBatch(DataWriterProcessorStepImpl. > java:178) > ... 12 more > 17/08/24 16:03:57 INFO executor.CoarseGrainedExecutorBackend: Driver > commanded a shutdown > 17/08/24 16:03:57 INFO executor.CoarseGrainedExecutorBackend: Driver from > 192.168.0.142:51695 disconnected during shutdown > 17/08/24 16:03:57 INFO executor.CoarseGrainedExecutorBackend: Driver from > 192.168.0.142:51695 disconnected during shutdown > 17/08/24 16:03:57 INFO memory.MemoryStore: MemoryStore cleared > 17/08/24 16:03:57 INFO storage.BlockManager: BlockManager stopped > 17/08/24 16:03:57 INFO util.ShutdownHookManager: Shutdown hook called > End of LogType:stderr > > LogType:stdout > Log Upload Time:Thu Aug 24 16:03:58 +0800 2017 > LogLength:0 > Log Contents: > End of LogType:stdout > 2017-08-24 > ------------------------------ > lk_hadoop > ------------------------------ > > *发件人：*Ravindra Pesala <[email protected]> > *发送时间：*2017-08-24 22:34 > *主题：*Re: get error when load tpcds data catalog_returns > *收件人：*"user"<[email protected]> > *抄送：* > > Hi, > > Which version of carbon and spark are you using? How much data are you > loading and what is the machine configuration? > > I have tried loading catlog_returns with 20 MB data in my local machine > and it is successful. I used the latest master branch and spark-2.1 > version. > > Also please send the complete log as the log information you provided does > not say the actual cause. > > Regards, > Ravindra. > > On 24 August 2017 at 14:02, lk_hadoop <[email protected]> wrote: > >> hi,all >> I want to test carbondata by using tpc-ds data , I try to load >> table : catalog_returns >> I got error ： >> >> org.apache.carbondata.processing.newflow.exception.CarbonDataLoadingException: >> There is an unexpected error: unable to generate the mdkey >> at org.apache.carbondata.processing.newflow.steps.DataWriterPro >> cessorStepImpl.execute(DataWriterProcessorStepImpl.java:125) >> at org.apache.carbondata.processing.newflow.DataLoadExecutor. >> execute(DataLoadExecutor.java:48) >> at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD$$anon$ >> 1.<init>(NewCarbonDataLoadRDD.scala:243) >> at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD.compute >> (NewCarbonDataLoadRDD.scala:220) >> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) >> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) >> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) >> at org.apache.spark.scheduler.Task.run(Task.scala:99) >> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool >> Executor.java:1142) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo >> lExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: >> org.apache.carbondata.processing.newflow.exception.CarbonDataLoadingException: >> unable to generate the mdkey >> at org.apache.carbondata.processing.newflow.steps.DataWriterPro >> cessorStepImpl.processBatch(DataWriterProcessorStepImpl.java:181) >> at org.apache.carbondata.processing.newflow.steps.DataWriterPro >> cessorStepImpl.execute(DataWriterProcessorStepImpl.java:111) >> ... 11 more >> Caused by: java.util.concurrent.RejectedExecutionException: Task >> java.util.concurrent.FutureTask@67098e0 rejected from >> java.util.concurrent.ThreadPoolExecutor@5b91b608[Shutting down, pool >> size = 1, active threads = 1, queued tasks = 0, completed tasks = 24] >> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy. >> rejectedExecution(ThreadPoolExecutor.java:2047) >> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExe >> cutor.java:823) >> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolEx >> ecutor.java:1369) >> at java.util.concurrent.AbstractExecutorService.submit(Abstract >> ExecutorService.java:134) >> at org.apache.carbondata.processing.store.CarbonFactDataHandler >> Columnar.addDataToStore(CarbonFactDataHandlerColumnar.java:466) >> at org.apache.carbondata.processing.newflow.steps.DataWriterPro >> cessorStepImpl.processBatch(DataWriterProcessorStepImpl.java:178) >> ... 12 more >> >> my table creat with sql : >> create table if not exists tpcds_carbon.catalog_returns >> ( >> cr_returned_date_sk int , >> cr_returned_time_sk int , >> cr_item_sk int , >> cr_refunded_customer_sk int , >> cr_refunded_cdemo_sk int , >> cr_refunded_hdemo_sk int , >> cr_refunded_addr_sk int , >> cr_returning_customer_sk int , >> cr_returning_cdemo_sk int , >> cr_returning_hdemo_sk int , >> cr_returning_addr_sk int , >> cr_call_center_sk int , >> cr_catalog_page_sk int , >> cr_ship_mode_sk int , >> cr_warehouse_sk int , >> cr_reason_sk int , >> cr_order_number int , >> cr_return_quantity int , >> cr_return_amount decimal(7,2) , >> cr_return_tax decimal(7,2) , >> cr_return_amt_inc_tax decimal(7,2) , >> cr_fee decimal(7,2) , >> cr_return_ship_cost decimal(7,2) , >> cr_refunded_cash decimal(7,2) , >> cr_reversed_charge decimal(7,2) , >> cr_store_credit decimal(7,2) , >> cr_net_loss decimal(7,2) >> )STORED BY 'carbondata' >> TBLPROPERTIES ('DICTIONARY_INCLUDE'='cr_item_sk,cr_order_number') >> //because these two cols are the real PK >> >> and I load data with : >> >> carbon.sql("load data inpath 'hdfs://AAA:9000/tpcds/source/ >> catalog_returns/catalog_returns_1_4.dat' into table >> carbon_catalog_returns2 OPTIONS('DELIMITER'='|','fileh >> eader'='cr_returned_date_sk,cr_returned_time_sk,cr_item_ >> sk,cr_refunded_customer_sk,cr_refunded_cdemo_sk,cr_refunded_ >> hdemo_sk,cr_refunded_addr_sk,cr_returning_customer_sk,cr_ >> returning_cdemo_sk,cr_returning_hdemo_sk,cr_returning_addr_ >> sk,cr_call_center_sk,cr_catalog_page_sk,cr_ship_mode_ >> sk,cr_warehouse_sk,cr_reason_sk,cr_order_number,cr_return_ >> quantity,cr_return_amount,cr_return_tax,cr_return_amt_inc_ >> tax,cr_fee,cr_return_ship_cost,cr_refunded_cash,cr_ >> reversed_charge,cr_store_credit,cr_net_loss')") >> >> any one know what was wrong? >> >> >> >> >> 2017-08-24 >> ------------------------------ >> lk_hadoop >> > > > > -- > Thanks & Regards, > Ravi > > -- Thanks & Regards, Ravi
