[
https://issues.apache.org/jira/browse/CARBONDATA-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Bhat closed CARBONDATA-4049.
-----------------------------------
Resolution: Won't Fix
Issue is identified as that of refresh which is handled by Spark. As issue is
not a carbon issue its closed.
> Sometimes refresh table fails with error "table not found in database" error
> ----------------------------------------------------------------------------
>
> Key: CARBONDATA-4049
> URL: https://issues.apache.org/jira/browse/CARBONDATA-4049
> Project: CarbonData
> Issue Type: Bug
> Components: data-query
> Affects Versions: 2.1.0
> Environment: Spark 2.4.5
> Reporter: Chetan Bhat
> Priority: Minor
>
> In Carbon 2.1 version user creates a database.
> user copies a old version store such as 1.6.1 to HDFS folder of the database
> in the In Carbon 2.1 version
> In Spark-SQL or beeline the user accesses the database using the use db
> command.
> Refresh table command is executed on the old version store table and then the
> subsequent operations on the table are performed.
> Next refresh table command is tried to be executed on another old version
> store table .
>
> Issue : Sometimes refresh table fails with error "table not found in
> database" error.
> spark-sql> refresh table brinjal_deleteseg;
> *Error in query: Table or view 'brinjal_deleteseg' not found in database
> '1_6_1';*
>
> **Log -
> 2020-11-12 18:55:46,922 | INFO | [main] | Created broadcast 171 from
> broadCastHadoopConf at CarbonRDD.scala:58 |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,922 | INFO | [main] | Created broadcast 171 from
> broadCastHadoopConf at CarbonRDD.scala:58 |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,924 | INFO | [main] | Pushed Filters: |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,939 | INFO | [main] | Distributed Index server is enabled for
> 1_6_1.brinjal_update |
> org.apache.carbondata.core.util.CarbonProperties.isDistributedPruningEnabled(CarbonProperties.java:1742)2020-11-12
> 18:55:46,939 | INFO | [main] | Started block pruning ... |
> org.apache.carbondata.hadoop.api.CarbonInputFormat.getPrunedBlocklets(CarbonInputFormat.java:526)2020-11-12
> 18:55:46,940 | INFO | [main] | Distributed Index server is enabled for
> 1_6_1.brinjal_update |
> org.apache.carbondata.core.util.CarbonProperties.isDistributedPruningEnabled(CarbonProperties.java:1742)2020-11-12
> 18:55:46,945 | INFO | [main] | Successfully Created directory:
> hdfs://hacluster/tmp/indexservertmp/4b6353d4-65d7-4856-b3cd-b3bc11d15c55 |
> org.apache.carbondata.core.util.CarbonUtil.createTempFolderForIndexServer(CarbonUtil.java:3273)2020-11-12
> 18:55:46,945 | INFO | [main] | Temp folder path for Query ID:
> 4b6353d4-65d7-4856-b3cd-b3bc11d15c55 is
> org.apache.carbondata.core.datastore.filesystem.HDFSCarbonFile@b8f2e1bf |
> org.apache.carbondata.indexserver.DistributedIndexJob.execute(IndexJobs.scala:57)2020-11-12
> 18:55:46,946 | ERROR | [main] | Configured port for index server is not a
> valid number |
> org.apache.carbondata.core.util.CarbonProperties.getIndexServerPort(CarbonProperties.java:1779)java.lang.NumberFormatException:
> null at java.lang.Integer.parseInt(Integer.java:542) at
> java.lang.Integer.parseInt(Integer.java:615) at
> org.apache.carbondata.core.util.CarbonProperties.getIndexServerPort(CarbonProperties.java:1777)
> at
> org.apache.carbondata.indexserver.IndexServer$.serverPort$lzycompute(IndexServer.scala:88)
> at
> org.apache.carbondata.indexserver.IndexServer$.serverPort(IndexServer.scala:88)
> at
> org.apache.carbondata.indexserver.IndexServer$.getClient(IndexServer.scala:312)
> at
> org.apache.carbondata.indexserver.IndexServer$.getClient(IndexServer.scala:301)
> at
> org.apache.carbondata.indexserver.DistributedIndexJob$$anonfun$1.apply(IndexJobs.scala:83)
> at
> org.apache.carbondata.indexserver.DistributedIndexJob$$anonfun$1.apply(IndexJobs.scala:59)
> at
> org.apache.carbondata.spark.util.CarbonScalaUtil$.logTime(CarbonScalaUtil.scala:769)
> at
> org.apache.carbondata.indexserver.DistributedIndexJob.execute(IndexJobs.scala:58)
> at
> org.apache.carbondata.core.index.IndexUtil.executeIndexJob(IndexUtil.java:304)
> at
> org.apache.carbondata.hadoop.api.CarbonInputFormat.getDistributedSplit(CarbonInputFormat.java:431)
> at
> org.apache.carbondata.hadoop.api.CarbonInputFormat.getPrunedBlocklets(CarbonInputFormat.java:532)
> at
> org.apache.carbondata.hadoop.api.CarbonInputFormat.getDataBlocksOfSegment(CarbonInputFormat.java:477)
> at
> org.apache.carbondata.hadoop.api.CarbonTableInputFormat.getSplits(CarbonTableInputFormat.java:356)
> at
> org.apache.carbondata.hadoop.api.CarbonTableInputFormat.getSplits(CarbonTableInputFormat.java:204)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD.internalGetPartitions(CarbonScanRDD.scala:159)
> at
> org.apache.carbondata.spark.rdd.CarbonRDD.getPartitions(CarbonRDD.scala:68)
> at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:273) at
> org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:269) at
> scala.Option.getOrElse(Option.scala:121) at
> org.apache.spark.rdd.RDD.partitions(RDD.scala:269) at
> org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49)
> at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:273) at
> org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:269) at
> scala.Option.getOrElse(Option.scala:121) at
> org.apache.spark.rdd.RDD.partitions(RDD.scala:269) at
> org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49)
> at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:273) at
> org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:269) at
> scala.Option.getOrElse(Option.scala:121) at
> org.apache.spark.rdd.RDD.partitions(RDD.scala:269) at
> org.apache.spark.SparkContext.runJob(SparkContext.scala:2126) at
> org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:990) at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:385) at
> org.apache.spark.rdd.RDD.collect(RDD.scala:989) at
> org.apache.spark.sql.execution.SparkPlan.executeCollect(SparkPlan.scala:299)
> at
> org.apache.spark.sql.execution.SparkPlan.executeCollectPublic(SparkPlan.scala:326)
> at
> org.apache.spark.sql.execution.QueryExecution.hiveResultString(QueryExecution.scala:128)
> at
> org.apache.spark.sql.hive.thriftserver.SparkSQLDriver$$anonfun$run$1.apply(SparkSQLDriver.scala:64)
> at
> org.apache.spark.sql.hive.thriftserver.SparkSQLDriver$$anonfun$run$1.apply(SparkSQLDriver.scala:64)
> at
> org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:80)
> at
> org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:127)
> at
> org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:75)
> at
> org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run(SparkSQLDriver.scala:63)
> at
> org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:371)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376) at
> org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:274)
> at
> org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497) at
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845)
> at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161) at
> org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184) at
> org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929) at
> org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)2020-11-12
> 18:55:46,949 | ERROR | [main] | Exception occurred while getting splits using
> index server. Initiating Fall back to embedded mode |
> org.apache.carbondata.hadoop.api.CarbonInputFormat.getDistributedSplit(CarbonInputFormat.java:438)java.lang.NumberFormatException:
> null at java.lang.Integer.parseInt(Integer.java:542) at
> java.lang.Integer.parseInt(Integer.java:615) at
> org.apache.carbondata.core.util.CarbonProperties.getIndexServerPort(CarbonProperties.java:1777)
> at
> org.apache.carbondata.indexserver.IndexServer$.serverPort$lzycompute(IndexServer.scala:88)
> at
> org.apache.carbondata.indexserver.IndexServer$.serverPort(IndexServer.scala:88)
> at
> org.apache.carbondata.indexserver.IndexServer$.getClient(IndexServer.scala:312)
> at
> org.apache.carbondata.indexserver.IndexServer$.getClient(IndexServer.scala:301)
> at
> org.apache.carbondata.indexserver.DistributedIndexJob$$anonfun$1.apply(IndexJobs.scala:83)
> at
> org.apache.carbondata.indexserver.DistributedIndexJob$$anonfun$1.apply(IndexJobs.scala:59)
> at
> org.apache.carbondata.spark.util.CarbonScalaUtil$.logTime(CarbonScalaUtil.scala:769)
> at
> org.apache.carbondata.indexserver.DistributedIndexJob.execute(IndexJobs.scala:58)
> at
> org.apache.carbondata.core.index.IndexUtil.executeIndexJob(IndexUtil.java:304)
> at
> org.apache.carbondata.hadoop.api.CarbonInputFormat.getDistributedSplit(CarbonInputFormat.java:431)
> at
> org.apache.carbondata.hadoop.api.CarbonInputFormat.getPrunedBlocklets(CarbonInputFormat.java:532)
> at
> org.apache.carbondata.hadoop.api.CarbonInputFormat.getDataBlocksOfSegment(CarbonInputFormat.java:477)
> at
> org.apache.carbondata.hadoop.api.CarbonTableInputFormat.getSplits(CarbonTableInputFormat.java:356)
> at
> org.apache.carbondata.hadoop.api.CarbonTableInputFormat.getSplits(CarbonTableInputFormat.java:204)
> at
> org.apache.carbondata.spark.rdd.CarbonScanRDD.internalGetPartitions(CarbonScanRDD.scala:159)
> at
> org.apache.carbondata.spark.rdd.CarbonRDD.getPartitions(CarbonRDD.scala:68)
> at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:273) at
> org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:269) at
> scala.Option.getOrElse(Option.scala:121) at
> org.apache.spark.rdd.RDD.partitions(RDD.scala:269) at
> org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49)
> at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:273) at
> org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:269) at
> scala.Option.getOrElse(Option.scala:121) at
> org.apache.spark.rdd.RDD.partitions(RDD.scala:269) at
> org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49)
> at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:273) at
> org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:269) at
> scala.Option.getOrElse(Option.scala:121) at
> org.apache.spark.rdd.RDD.partitions(RDD.scala:269) at
> org.apache.spark.SparkContext.runJob(SparkContext.scala:2126) at
> org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:990) at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:385) at
> org.apache.spark.rdd.RDD.collect(RDD.scala:989) at
> org.apache.spark.sql.execution.SparkPlan.executeCollect(SparkPlan.scala:299)
> at
> org.apache.spark.sql.execution.SparkPlan.executeCollectPublic(SparkPlan.scala:326)
> at
> org.apache.spark.sql.execution.QueryExecution.hiveResultString(QueryExecution.scala:128)
> at
> org.apache.spark.sql.hive.thriftserver.SparkSQLDriver$$anonfun$run$1.apply(SparkSQLDriver.scala:64)
> at
> org.apache.spark.sql.hive.thriftserver.SparkSQLDriver$$anonfun$run$1.apply(SparkSQLDriver.scala:64)
> at
> org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:80)
> at
> org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:127)
> at
> org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:75)
> at
> org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run(SparkSQLDriver.scala:63)
> at
> org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:371)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376) at
> org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:274)
> at
> org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497) at
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845)
> at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161) at
> org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184) at
> org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929) at
> org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)2020-11-12
> 18:55:46,951 | INFO | [main] | Block broadcast_172 stored as values in
> memory (estimated size 370.9 KB, free 909.1 MB) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,960 | INFO | [main] | Block broadcast_172_piece0 stored as bytes in
> memory (estimated size 29.4 KB, free 909.0 MB) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,961 | INFO | [dispatcher-event-loop-1] | Added broadcast_172_piece0
> in memory on vm1:43460 (size: 29.4 KB, free: 912.0 MB) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,961 | INFO | [main] | Created broadcast 172 from
> broadCastHadoopConf at CarbonRDD.scala:58 |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,962 | INFO | [main] | Distributed Index server is enabled for
> 1_6_1.brinjal_update |
> org.apache.carbondata.core.util.CarbonProperties.isDistributedPruningEnabled(CarbonProperties.java:1742)2020-11-12
> 18:55:46,966 | INFO | [main] | Starting job: collect at
> IndexServer.scala:178 |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,967 | INFO | [dag-scheduler-event-loop] | Got job 87 (collect at
> IndexServer.scala:178) with 1 output partitions |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,967 | INFO | [dag-scheduler-event-loop] | Final stage: ResultStage
> 83 (collect at IndexServer.scala:178) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,967 | INFO | [dag-scheduler-event-loop] | Parents of final stage:
> List() |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,967 | INFO | [dag-scheduler-event-loop] | Missing parents: List() |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,967 | INFO | [dag-scheduler-event-loop] | Submitting ResultStage 83
> (DistributedPruneRDD[249] at RDD at CarbonRDD.scala:38), which has no missing
> parents |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,968 | INFO | [dag-scheduler-event-loop] | Block broadcast_173
> stored as values in memory (estimated size 17.7 KB, free 909.0 MB) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,969 | INFO | [dag-scheduler-event-loop] | Block
> broadcast_173_piece0 stored as bytes in memory (estimated size 8.7 KB, free
> 909.0 MB) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,970 | INFO | [dispatcher-event-loop-2] | Added broadcast_173_piece0
> in memory on vm1:43460 (size: 8.7 KB, free: 912.0 MB) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,970 | INFO | [dag-scheduler-event-loop] | Created broadcast 173
> from broadcast at DAGScheduler.scala:1163 |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,970 | INFO | [dag-scheduler-event-loop] | Submitting 1 missing
> tasks from ResultStage 83 (DistributedPruneRDD[249] at RDD at
> CarbonRDD.scala:38) (first 15 tasks are for partitions Vector(0)) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,970 | INFO | [dag-scheduler-event-loop] | Adding task set 83.0 with
> 1 tasks |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,971 | INFO | [dispatcher-event-loop-5] | Starting task 0.0 in stage
> 83.0 (TID 464, localhost, executor driver, partition 0, PROCESS_LOCAL, 9449
> bytes) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,971 | INFO | [Executor task launch worker for task 464] | Running
> task 0.0 in stage 83.0 (TID 464) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,974 | INFO | [Executor task launch worker for task 464] | Value for
> carbon.max.executor.threads.for.block.pruning is 4 |
> org.apache.carbondata.core.util.CarbonProperties.getNumOfThreadsForExecutorPruning(CarbonProperties.java:1809)2020-11-12
> 18:55:46,995 | INFO | [IndexPruningPool_1605178546974] | Constructing new
> SegmentProperties for table: 1_6_1_brinjal_update. Current size of segment
> properties holder list is: 2 |
> org.apache.carbondata.core.datastore.block.SegmentPropertiesAndSchemaHolder.addSegmentProperties(SegmentPropertiesAndSchemaHolder.java:115)2020-11-12
> 18:55:46,997 | INFO | [IndexPruningPool_1605178546974] | Removed entry from
> InMemory lru cache ::
> hdfs://hacluster/user/sparkhive/warehouse/1_6_1.db/brinjal_update/Fact/Part0/Segment_5/5_1605178541880.carbonindexmerge
> |
> org.apache.carbondata.core.cache.CarbonLRUCache.removeKey(CarbonLRUCache.java:189)2020-11-12
> 18:55:46,997 | INFO | [Executor task launch worker for task 464] | Time
> taken to collect 1 blocklets : 23 |
> org.apache.carbondata.indexserver.DistributedPruneRDD.internalCompute(DistributedPruneRDD.scala:118)2020-11-12
> 18:55:46,998 | INFO | [Executor task launch worker for task 464] | Finished
> task 0.0 in stage 83.0 (TID 464). 2475 bytes result sent to driver |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,999 | INFO | [task-result-getter-3] | Finished task 0.0 in stage
> 83.0 (TID 464) in 27 ms on localhost (executor driver) (1/1) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,999 | INFO | [task-result-getter-3] | Removed TaskSet 83.0, whose
> tasks have all completed, from pool |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,999 | INFO | [dag-scheduler-event-loop] | ResultStage 83 (collect
> at IndexServer.scala:178) finished in 0.031 s |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:46,999 | INFO | [main] | Job 87 finished: collect at
> IndexServer.scala:178, took 0.032684 s |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,001 | INFO | [main] | Block broadcast_174 stored as values in
> memory (estimated size 370.9 KB, free 908.6 MB) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,010 | INFO | [main] | Block broadcast_174_piece0 stored as bytes in
> memory (estimated size 29.4 KB, free 908.6 MB) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,011 | INFO | [dispatcher-event-loop-7] | Added broadcast_174_piece0
> in memory on vm1:43460 (size: 29.4 KB, free: 912.0 MB) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,011 | INFO | [main] | Created broadcast 174 from
> broadCastHadoopConf at CarbonRDD.scala:58 |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,016 | INFO | [main] | Starting job: collect at
> IndexServer.scala:205 |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,017 | INFO | [main] | Job 88 finished: collect at
> IndexServer.scala:205, took 0.000032 s |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,017 | INFO | [main] | Finished block pruning ... |
> org.apache.carbondata.hadoop.api.CarbonInputFormat.getPrunedBlocklets(CarbonInputFormat.java:622)2020-11-12
> 18:55:47,018 | INFO | [main] | Planning scan with bin packing, max size:
> 4194304 bytes, open cost is considered as scanning 4194304 bytes. |
> org.apache.carbondata.spark.rdd.CarbonScanRDD.distributeColumnarSplits(CarbonScanRDD.scala:366)2020-11-12
> 18:55:47,018 | INFO | [main] | Identified no.of.blocks: 5, no.of.tasks: 5,
> no.of.nodes: 0, parallelism: 8 |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,022 | INFO | [main] | Starting job: processCmd at
> CliDriver.java:376 |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,023 | INFO | [dag-scheduler-event-loop] | Got job 89 (processCmd at
> CliDriver.java:376) with 5 output partitions |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,023 | INFO | [dag-scheduler-event-loop] | Final stage: ResultStage
> 84 (processCmd at CliDriver.java:376) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,023 | INFO | [dag-scheduler-event-loop] | Parents of final stage:
> List() |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,023 | INFO | [dag-scheduler-event-loop] | Missing parents: List() |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,023 | INFO | [dag-scheduler-event-loop] | Submitting ResultStage 84
> (MapPartitionsRDD[248] at processCmd at CliDriver.java:376), which has no
> missing parents |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,025 | INFO | [dag-scheduler-event-loop] | Block broadcast_175
> stored as values in memory (estimated size 24.3 KB, free 908.6 MB) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,026 | INFO | [dag-scheduler-event-loop] | Block
> broadcast_175_piece0 stored as bytes in memory (estimated size 11.6 KB, free
> 908.6 MB) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,026 | INFO | [dispatcher-event-loop-6] | Added broadcast_175_piece0
> in memory on vm1:43460 (size: 11.6 KB, free: 912.0 MB) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,026 | INFO | [dag-scheduler-event-loop] | Created broadcast 175
> from broadcast at DAGScheduler.scala:1163 |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,027 | INFO | [dag-scheduler-event-loop] | Submitting 5 missing
> tasks from ResultStage 84 (MapPartitionsRDD[248] at processCmd at
> CliDriver.java:376) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
> | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,027 | INFO | [dag-scheduler-event-loop] | Adding task set 84.0 with
> 5 tasks |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,028 | INFO | [dispatcher-event-loop-1] | Starting task 0.0 in stage
> 84.0 (TID 465, localhost, executor driver, partition 0, ANY, 9185 bytes) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,028 | INFO | [dispatcher-event-loop-1] | Starting task 1.0 in stage
> 84.0 (TID 466, localhost, executor driver, partition 1, ANY, 9185 bytes) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,028 | INFO | [dispatcher-event-loop-1] | Starting task 2.0 in stage
> 84.0 (TID 467, localhost, executor driver, partition 2, ANY, 9185 bytes) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,029 | INFO | [dispatcher-event-loop-1] | Starting task 3.0 in stage
> 84.0 (TID 468, localhost, executor driver, partition 3, ANY, 9185 bytes) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,029 | INFO | [dispatcher-event-loop-1] | Starting task 4.0 in stage
> 84.0 (TID 469, localhost, executor driver, partition 4, ANY, 9185 bytes) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,029 | INFO | [Executor task launch worker for task 466] | Running
> task 1.0 in stage 84.0 (TID 466) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,029 | INFO | [Executor task launch worker for task 467] | Running
> task 2.0 in stage 84.0 (TID 467) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,029 | INFO | [Executor task launch worker for task 469] | Running
> task 4.0 in stage 84.0 (TID 469) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,029 | INFO | [Executor task launch worker for task 468] | Running
> task 3.0 in stage 84.0 (TID 468) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,029 | INFO | [Executor task launch worker for task 465] | Running
> task 0.0 in stage 84.0 (TID 465) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,033 | INFO | [Executor task launch worker for task 469] |
> Projection Columns: [imei, amsize, channelsid, activecountry, activecity,
> productiondate, deliverydate, gamepointid, deviceinformationid,
> deliverycharge] |
> org.apache.carbondata.core.scan.model.QueryModelBuilder.projectColumns(QueryModelBuilder.java:94)2020-11-12
> 18:55:47,033 | INFO | [Executor task launch worker for task 467] |
> Projection Columns: [imei, amsize, channelsid, activecountry, activecity,
> productiondate, deliverydate, gamepointid, deviceinformationid,
> deliverycharge] |
> org.apache.carbondata.core.scan.model.QueryModelBuilder.projectColumns(QueryModelBuilder.java:94)2020-11-12
> 18:55:47,033 | INFO | [Executor task launch worker for task 466] |
> Projection Columns: [imei, amsize, channelsid, activecountry, activecity,
> productiondate, deliverydate, gamepointid, deviceinformationid,
> deliverycharge] |
> org.apache.carbondata.core.scan.model.QueryModelBuilder.projectColumns(QueryModelBuilder.java:94)2020-11-12
> 18:55:47,033 | INFO | [Executor task launch worker for task 468] |
> Projection Columns: [imei, amsize, channelsid, activecountry, activecity,
> productiondate, deliverydate, gamepointid, deviceinformationid,
> deliverycharge] |
> org.apache.carbondata.core.scan.model.QueryModelBuilder.projectColumns(QueryModelBuilder.java:94)2020-11-12
> 18:55:47,033 | INFO | [Executor task launch worker for task 465] |
> Projection Columns: [imei, amsize, channelsid, activecountry, activecity,
> productiondate, deliverydate, gamepointid, deviceinformationid,
> deliverycharge] |
> org.apache.carbondata.core.scan.model.QueryModelBuilder.projectColumns(QueryModelBuilder.java:94)2020-11-12
> 18:55:47,034 | INFO | [Executor task launch worker for task 466] | Query
> will be executed on table: brinjal_update |
> org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.initQuery(AbstractQueryExecutor.java:122)2020-11-12
> 18:55:47,034 | INFO | [Executor task launch worker for task 467] | Query
> will be executed on table: brinjal_update |
> org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.initQuery(AbstractQueryExecutor.java:122)2020-11-12
> 18:55:47,034 | INFO | [Executor task launch worker for task 469] | Query
> will be executed on table: brinjal_update |
> org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.initQuery(AbstractQueryExecutor.java:122)2020-11-12
> 18:55:47,034 | INFO | [Executor task launch worker for task 466] | Query
> prefetch is: true |
> org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.getBlockExecutionInfoForBlock(AbstractQueryExecutor.java:479)2020-11-12
> 18:55:47,034 | INFO | [Executor task launch worker for task 465] | Query
> will be executed on table: brinjal_update |
> org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.initQuery(AbstractQueryExecutor.java:122)2020-11-12
> 18:55:47,034 | INFO | [Executor task launch worker for task 469] | Query
> prefetch is: true |
> org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.getBlockExecutionInfoForBlock(AbstractQueryExecutor.java:479)2020-11-12
> 18:55:47,034 | INFO | [Executor task launch worker for task 465] | Query
> prefetch is: true |
> org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.getBlockExecutionInfoForBlock(AbstractQueryExecutor.java:479)2020-11-12
> 18:55:47,034 | INFO | [Executor task launch worker for task 468] | Query
> will be executed on table: brinjal_update |
> org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.initQuery(AbstractQueryExecutor.java:122)2020-11-12
> 18:55:47,034 | INFO | [Executor task launch worker for task 467] | Query
> prefetch is: true |
> org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.getBlockExecutionInfoForBlock(AbstractQueryExecutor.java:479)2020-11-12
> 18:55:47,035 | INFO | [Executor task launch worker for task 468] | Query
> prefetch is: true |
> org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.getBlockExecutionInfoForBlock(AbstractQueryExecutor.java:479)2020-11-12
> 18:55:47,036 | INFO | [Executor task launch worker for task 466] | Vector
> based dictionary collector is used to scan and collect the data |
> org.apache.carbondata.core.scan.collector.ResultCollectorFactory.getScannedResultCollector(ResultCollectorFactory.java:78)2020-11-12
> 18:55:47,036 | INFO | [Executor task launch worker for task 466] | Direct
> page-wise vector fill collector is used to scan and collect the data |
> org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.<init>(DictionaryBasedVectorResultCollector.java:73)2020-11-12
> 18:55:47,037 | INFO | [Executor task launch worker for task 469] | Vector
> based dictionary collector is used to scan and collect the data |
> org.apache.carbondata.core.scan.collector.ResultCollectorFactory.getScannedResultCollector(ResultCollectorFactory.java:78)2020-11-12
> 18:55:47,037 | INFO | [Executor task launch worker for task 469] | Direct
> page-wise vector fill collector is used to scan and collect the data |
> org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.<init>(DictionaryBasedVectorResultCollector.java:73)2020-11-12
> 18:55:47,037 | INFO | [Executor task launch worker for task 465] | Vector
> based dictionary collector is used to scan and collect the data |
> org.apache.carbondata.core.scan.collector.ResultCollectorFactory.getScannedResultCollector(ResultCollectorFactory.java:78)2020-11-12
> 18:55:47,037 | INFO | [Executor task launch worker for task 465] | Direct
> page-wise vector fill collector is used to scan and collect the data |
> org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.<init>(DictionaryBasedVectorResultCollector.java:73)2020-11-12
> 18:55:47,037 | INFO | [Executor task launch worker for task 468] | Vector
> based dictionary collector is used to scan and collect the data |
> org.apache.carbondata.core.scan.collector.ResultCollectorFactory.getScannedResultCollector(ResultCollectorFactory.java:78)2020-11-12
> 18:55:47,037 | INFO | [Executor task launch worker for task 468] | Direct
> page-wise vector fill collector is used to scan and collect the data |
> org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.<init>(DictionaryBasedVectorResultCollector.java:73)2020-11-12
> 18:55:47,037 | INFO | [Executor task launch worker for task 467] | Vector
> based dictionary collector is used to scan and collect the data |
> org.apache.carbondata.core.scan.collector.ResultCollectorFactory.getScannedResultCollector(ResultCollectorFactory.java:78)2020-11-12
> 18:55:47,038 | INFO | [Executor task launch worker for task 467] | Direct
> page-wise vector fill collector is used to scan and collect the data |
> org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.<init>(DictionaryBasedVectorResultCollector.java:73)2020-11-12
> 18:55:47,050 | INFO | [Executor task launch worker for task 465] | Total
> off-heap working memory used after task 645a72eb-8d06-489b-940d-f7ca1e901bc7
> is 128073. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4,
> f251027c-707b-4df1-b35f-00eb74c8d77f, 5233fa5c-5173-4a9e-a8bf-8c024dac1509,
> a05de0f7-398e-45a5-a91d-a4e1045e9a98 |
> org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12
> 18:55:47,050 | INFO | [Executor task launch worker for task 467] | Total
> off-heap working memory used after task a05de0f7-398e-45a5-a91d-a4e1045e9a98
> is 128073. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4,
> f251027c-707b-4df1-b35f-00eb74c8d77f, 5233fa5c-5173-4a9e-a8bf-8c024dac1509 |
> org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12
> 18:55:47,050 | INFO | [Executor task launch worker for task 469] | Total
> off-heap working memory used after task f251027c-707b-4df1-b35f-00eb74c8d77f
> is 128073. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4,
> 5233fa5c-5173-4a9e-a8bf-8c024dac1509 |
> org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12
> 18:55:47,050 | INFO | [Executor task launch worker for task 467] | Total
> off-heap working memory used after task 11861831-3ae3-4dd7-98ce-352f4416e4d5
> is 128073. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4,
> 5233fa5c-5173-4a9e-a8bf-8c024dac1509 |
> org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12
> 18:55:47,050 | INFO | [Executor task launch worker for task 469] | Total
> off-heap working memory used after task c4193df5-ceb8-4ff9-8844-0d9c42f989d8
> is 128045. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4,
> 5233fa5c-5173-4a9e-a8bf-8c024dac1509 |
> org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12
> 18:55:47,051 | INFO | [Executor task launch worker for task 467] | Total
> off-heap working memory used after task 88e4dbd6-c19e-43e6-bc9b-42dfefc8378c
> is 128045. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4,
> 5233fa5c-5173-4a9e-a8bf-8c024dac1509 |
> org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12
> 18:55:47,051 | INFO | [Executor task launch worker for task 469] | Total
> off-heap working memory used after task f75746ea-783d-4098-91d5-1a93fda81f04
> is 128045. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4,
> 5233fa5c-5173-4a9e-a8bf-8c024dac1509 |
> org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12
> 18:55:47,051 | INFO | [Executor task launch worker for task 465] | Total
> off-heap working memory used after task ae4ac11d-7a43-426d-8578-ac77b6ddf74b
> is 128090. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4,
> 5233fa5c-5173-4a9e-a8bf-8c024dac1509 |
> org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12
> 18:55:47,051 | INFO | [Executor task launch worker for task 466] | Total
> off-heap working memory used after task 7a7a3012-c819-4e4c-9f08-54dd890850e4
> is 128090. Current running tasks are 5233fa5c-5173-4a9e-a8bf-8c024dac1509 |
> org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12
> 18:55:47,051 | INFO | [Executor task launch worker for task 465] | Total
> off-heap working memory used after task fb7aa505-1163-4fe2-936e-0de1e4f78271
> is 128090. Current running tasks are 5233fa5c-5173-4a9e-a8bf-8c024dac1509 |
> org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12
> 18:55:47,051 | INFO | [Executor task launch worker for task 466] | Total
> off-heap working memory used after task 27cb623e-3b1c-4f49-adaa-c11c19d766af
> is 128000. Current running tasks are 5233fa5c-5173-4a9e-a8bf-8c024dac1509 |
> org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12
> 18:55:47,051 | INFO | [Executor task launch worker for task 466] | Total
> off-heap working memory used after task b2d72c4c-7d5d-4030-9f79-54219d56474d
> is 0. Current running tasks are 5233fa5c-5173-4a9e-a8bf-8c024dac1509 |
> org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12
> 18:55:47,052 | INFO | [Executor task launch worker for task 465] | Finished
> task 0.0 in stage 84.0 (TID 465). 2766 bytes result sent to driver |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,051 | INFO | [Executor task launch worker for task 469] | Finished
> task 4.0 in stage 84.0 (TID 469). 2198 bytes result sent to driver |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,051 | INFO | [Executor task launch worker for task 467] | Finished
> task 2.0 in stage 84.0 (TID 467). 2444 bytes result sent to driver |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,052 | INFO | [Executor task launch worker for task 466] | Finished
> task 1.0 in stage 84.0 (TID 466). 2524 bytes result sent to driver |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,052 | INFO | [Executor task launch worker for task 468] | Total
> off-heap working memory used after task 5233fa5c-5173-4a9e-a8bf-8c024dac1509
> is 0. Current running tasks are |
> org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12
> 18:55:47,052 | INFO | [Executor task launch worker for task 468] | Total
> off-heap working memory used after task 4ea15806-df6e-4377-9cea-84f07586fe02
> is 0. Current running tasks are |
> org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12
> 18:55:47,052 | INFO | [task-result-getter-2] | Finished task 0.0 in stage
> 84.0 (TID 465) in 25 ms on localhost (executor driver) (1/5) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,052 | INFO | [Executor task launch worker for task 468] | Total
> off-heap working memory used after task 1086a9e4-a644-4340-8215-b8c3d936e32d
> is 0. Current running tasks are |
> org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12
> 18:55:47,053 | INFO | [task-result-getter-1] | Finished task 4.0 in stage
> 84.0 (TID 469) in 24 ms on localhost (executor driver) (2/5) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,053 | INFO | [Executor task launch worker for task 468] | Finished
> task 3.0 in stage 84.0 (TID 468). 2380 bytes result sent to driver |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,053 | INFO | [task-result-getter-0] | Finished task 2.0 in stage
> 84.0 (TID 467) in 25 ms on localhost (executor driver) (3/5) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,054 | INFO | [task-result-getter-3] | Finished task 1.0 in stage
> 84.0 (TID 466) in 26 ms on localhost (executor driver) (4/5) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,054 | INFO | [task-result-getter-2] | Finished task 3.0 in stage
> 84.0 (TID 468) in 26 ms on localhost (executor driver) (5/5) |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,054 | INFO | [task-result-getter-2] | Removed TaskSet 84.0, whose
> tasks have all completed, from pool |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,054 | INFO | [dag-scheduler-event-loop] | ResultStage 84
> (processCmd at CliDriver.java:376) finished in 0.030 s |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,055 | INFO | [main] | Job 89 finished: processCmd at
> CliDriver.java:376, took 0.032288 s |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12
> 18:55:47,071 | INFO | [main] | Time taken: 0.344 seconds, Fetched 534 row(s)
> |
> org.apache.hadoop.hive.ql.session.SessionState$LogHelper.printInfo(SessionState.java:951)2020-11-12
> 18:55:47,086 | AUDIT | [main] | \{"time":"November 12, 2020 6:55:47 PM
> CST","username":"root","opName":"SET","opId":"3304241283944785","opStatus":"START"}
> |
> org.apache.carbondata.processing.util.Auditor.logOperationStart(Auditor.java:74)2020-11-12
> 18:55:47,086 | INFO | [main] | The key carbon.enable.index.server with
> value false added in the session param |
> org.apache.carbondata.core.util.SessionParams.addProperty(SessionParams.java:102)2020-11-12
> 18:55:47,087 | AUDIT | [main] | \{"time":"November 12, 2020 6:55:47 PM
> CST","username":"root","opName":"SET","opId":"3304241283944785","opStatus":"SUCCESS","opTime":"1
> ms","table":"NA","extraInfo":{}} |
> org.apache.carbondata.processing.util.Auditor.logOperationEnd(Auditor.java:97)2020-11-12
> 18:55:47,094 | INFO | [main] | Time taken: 0.016 seconds, Fetched 1 row(s)
> |
> org.apache.hadoop.hive.ql.session.SessionState$LogHelper.printInfo(SessionState.java:951)2020-11-12
> 18:58:04,451 | AUDIT | [main] | \{"time":"November 12, 2020 6:58:04 PM
> CST","username":"root","opName":"REFRESH
> TABLE","opId":"3304378649144145","opStatus":"START"} |
> org.apache.carbondata.processing.util.Auditor.logOperationStart(Auditor.java:74)2020-11-12
> 18:58:04,452 | INFO | [main] | 0: get_database: 1_6_1 |
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logInfo(HiveMetaStore.java:746)2020-11-12
> 18:58:04,452 | INFO | [main] | ugi=root ip=unknown-ip-addr
> cmd=get_database: 1_6_1 |
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logAuditEvent(HiveMetaStore.java:371)2020-11-12
> 18:58:04,455 | INFO | [main] | 0: get_table : db=1_6_1
> tbl=brinjal_deleteseg |
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logInfo(HiveMetaStore.java:746)2020-11-12
> 18:58:04,455 | INFO | [main] | ugi=root ip=unknown-ip-addr cmd=get_table :
> db=1_6_1 tbl=brinjal_deleteseg |
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logAuditEvent(HiveMetaStore.java:371)2020-11-12
> 18:58:04,458 | INFO | [main] | 0: get_table : db=1_6_1
> tbl=brinjal_deleteseg |
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logInfo(HiveMetaStore.java:746)2020-11-12
> 18:58:04,458 | INFO | [main] | ugi=root ip=unknown-ip-addr cmd=get_table :
> db=1_6_1 tbl=brinjal_deleteseg |
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logAuditEvent(HiveMetaStore.java:371)2020-11-12
> 18:58:04,460 | INFO | [main] | 0: get_database: 1_6_1 |
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logInfo(HiveMetaStore.java:746)2020-11-12
> 18:58:04,460 | INFO | [main] | ugi=root ip=unknown-ip-addr
> cmd=get_database: 1_6_1 |
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logAuditEvent(HiveMetaStore.java:371)2020-11-12
> 18:58:04,462 | INFO | [main] | 0: get_database: 1_6_1 |
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logInfo(HiveMetaStore.java:746)2020-11-12
> 18:58:04,462 | INFO | [main] | ugi=root ip=unknown-ip-addr
> cmd=get_database: 1_6_1 |
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logAuditEvent(HiveMetaStore.java:371)2020-11-12
> 18:58:04,465 | INFO | [main] | 0: get_database: 1_6_1 |
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logInfo(HiveMetaStore.java:746)2020-11-12
> 18:58:04,465 | INFO | [main] | ugi=root ip=unknown-ip-addr
> cmd=get_database: 1_6_1 |
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logAuditEvent(HiveMetaStore.java:371)2020-11-12
> 18:58:04,467 | INFO | [main] | 0: get_table : db=1_6_1
> tbl=brinjal_deleteseg |
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logInfo(HiveMetaStore.java:746)2020-11-12
> 18:58:04,467 | INFO | [main] | ugi=root ip=unknown-ip-addr cmd=get_table :
> db=1_6_1 tbl=brinjal_deleteseg |
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logAuditEvent(HiveMetaStore.java:371)2020-11-12
> 18:58:04,472 | AUDIT | [main] | \{"time":"November 12, 2020 6:58:04 PM
> CST","username":"root","opName":"REFRESH
> TABLE","opId":"3304378649144145","opStatus":"FAILED","opTime":"21
> ms","table":"1_6_1.brinjal_deleteseg","extraInfo":{"Exception":"org.apache.spark.sql.catalyst.analysis.NoSuchTableException","Message":"Table
> or view 'brinjal_deleteseg' not found in database '1_6_1';"}} |
> org.apache.carbondata.processing.util.Auditor.logOperationEnd(Auditor.java:97)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
