[
https://issues.apache.org/jira/browse/SPARK-15078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295790#comment-15295790
]
Ovidiu Marcu commented on SPARK-15078:
--------------------------------------
Running tpcds1.4 queries (using https://github.com/databricks/spark-sql-perf
with val experiment = tpcds.runExperiment(tpcds.runnable) reveals some issues:
Running execution q1-v1.4 iteration: 1, StandardRun=true
Execution time: 7.130528s
Running execution q2-v1.4 iteration: 1, StandardRun=true
Execution time: 4.527452s
Running execution q3-v1.4 iteration: 1, StandardRun=true
Execution time: 1.642223s
Running execution q4-v1.4 iteration: 1, StandardRun=true
Execution time: 43.076536999999995s
Running execution q5-v1.4 iteration: 1, StandardRun=true
Execution time: 11.677014999999999s
Running execution q7-v1.4 iteration: 1, StandardRun=true
Execution time: 3.548009s
Running execution q8-v1.4 iteration: 1, StandardRun=true
Execution time: 2.637601s
Running execution q9-v1.4 iteration: 1, StandardRun=true
16/05/23 00:37:14 WARN BlockManager: Asked to remove block
broadcast_396_piece0, which does not exist
16/05/23 00:37:22 ERROR Executor: Exception in task 0.0 in stage 243.0 (TID
13067)
java.lang.NullPointerException
at
org.apache.spark.sql.execution.ScalarSubquery.dataType(subquery.scala:45)
at
org.apache.spark.sql.catalyst.expressions.CaseWhenBase.dataType(conditionalExpressions.scala:103)
at
org.apache.spark.sql.catalyst.expressions.Alias.toAttribute(namedExpressions.scala:165)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.immutable.List.map(List.scala:285)
at
org.apache.spark.sql.execution.ProjectExec.output(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.WholeStageCodegenExec.output(WholeStageCodegenExec.scala:289)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:61)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:60)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
16/05/23 00:37:22 WARN TaskSetManager: Lost task 0.0 in stage 243.0 (TID 13067,
localhost): java.lang.NullPointerException
at
org.apache.spark.sql.execution.ScalarSubquery.dataType(subquery.scala:45)
at
org.apache.spark.sql.catalyst.expressions.CaseWhenBase.dataType(conditionalExpressions.scala:103)
at
org.apache.spark.sql.catalyst.expressions.Alias.toAttribute(namedExpressions.scala:165)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.immutable.List.map(List.scala:285)
at
org.apache.spark.sql.execution.ProjectExec.output(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.WholeStageCodegenExec.output(WholeStageCodegenExec.scala:289)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:61)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:60)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
16/05/23 00:37:22 ERROR TaskSetManager: Task 0 in stage 243.0 failed 1 times;
aborting job
Execution 'q9-v1.4' failed: Job aborted due to stage failure: Task 0 in stage
243.0 failed 1 times, most recent failure: Lost task 0.0 in stage 243.0 (TID
13067, localhost): java.lang.NullPointerException
at
org.apache.spark.sql.execution.ScalarSubquery.dataType(subquery.scala:45)
at
org.apache.spark.sql.catalyst.expressions.CaseWhenBase.dataType(conditionalExpressions.scala:103)
at
org.apache.spark.sql.catalyst.expressions.Alias.toAttribute(namedExpressions.scala:165)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.immutable.List.map(List.scala:285)
at
org.apache.spark.sql.execution.ProjectExec.output(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.WholeStageCodegenExec.output(WholeStageCodegenExec.scala:289)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:61)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:60)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Driver stacktrace:
Running execution q11-v1.4 iteration: 1, StandardRun=true
16/05/23 00:37:51 WARN BlockManager: Asked to remove block
broadcast_529_piece0, which does not exist
Execution time: 24.977066s
Running execution q12-v1.4 iteration: 1, StandardRun=true
Execution time: 2.344853s
Running execution q13-v1.4 iteration: 1, StandardRun=true
16/05/23 00:38:02 WARN BlockManager: Asked to remove block
broadcast_585_piece0, which does not exist
Execution time: 5.225287000000001s
Running execution q15-v1.4 iteration: 1, StandardRun=true
16/05/23 00:38:06 WARN BlockManager: Asked to remove block
broadcast_590_piece0, which does not exist
Execution time: 2.5487399999999996s
Running execution q17-v1.4 iteration: 1, StandardRun=true
Execution time: 17.472356s
Running execution q18-v1.4 iteration: 1, StandardRun=true
16/05/23 00:38:34 WARN BlockManager: Asked to remove block
broadcast_679_piece0, which does not exist
16/05/23 00:38:34 WARN BlockManager: Asked to remove block
broadcast_678_piece3, which does not exist
Execution time: 5.795082000000001s
Running execution q19-v1.4 iteration: 1, StandardRun=true
Execution time: 1.919265s
Running execution q20-v1.4 iteration: 1, StandardRun=true
Execution time: 2.557582s
Running execution q21-v1.4 iteration: 1, StandardRun=true
Execution time: 1.640811s
Running execution q22-v1.4 iteration: 1, StandardRun=true
Execution time: 5.183359s
Running execution q25-v1.4 iteration: 1, StandardRun=true
16/05/23 00:38:52 ERROR BlockManagerSlaveEndpoint: Error in removing broadcast
755
java.lang.IllegalStateException: Task -1024 has already locked
broadcast_755_piece0 for writing
at
org.apache.spark.storage.BlockInfoManager.lockForWriting(BlockInfoManager.scala:232)
at
org.apache.spark.storage.BlockManager.removeBlock(BlockManager.scala:1296)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at
org.apache.spark.storage.BlockManager.removeBroadcast(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply$mcI$sp(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$1.apply(BlockManagerSlaveEndpoint.scala:82)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
16/05/23 00:38:52 WARN BlockManagerMaster: Failed to remove broadcast 755 with
removeFromMaster = true - Task -1024 has already locked broadcast_755_piece0
for writing
java.lang.IllegalStateException: Task -1024 has already locked
broadcast_755_piece0 for writing
at
org.apache.spark.storage.BlockInfoManager.lockForWriting(BlockInfoManager.scala:232)
at
org.apache.spark.storage.BlockManager.removeBlock(BlockManager.scala:1296)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at
org.apache.spark.storage.BlockManager.removeBroadcast(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply$mcI$sp(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$1.apply(BlockManagerSlaveEndpoint.scala:82)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
16/05/23 00:38:52 ERROR ContextCleaner: Error cleaning broadcast 755
org.apache.spark.SparkException: Exception thrown in awaitResult
at
org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.scala:77)
at
org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.scala:75)
at
scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
at
org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:59)
at
org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:59)
at scala.PartialFunction$OrElse.apply(PartialFunction.scala:167)
at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:83)
at
org.apache.spark.storage.BlockManagerMaster.removeBroadcast(BlockManagerMaster.scala:135)
at
org.apache.spark.broadcast.TorrentBroadcast$.unpersist(TorrentBroadcast.scala:260)
at
org.apache.spark.broadcast.TorrentBroadcastFactory.unbroadcast(TorrentBroadcastFactory.scala:45)
at
org.apache.spark.broadcast.BroadcastManager.unbroadcast(BroadcastManager.scala:60)
at
org.apache.spark.ContextCleaner.doCleanupBroadcast(ContextCleaner.scala:232)
at
org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1$$anonfun$apply$mcV$sp$2.apply(ContextCleaner.scala:188)
at
org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1$$anonfun$apply$mcV$sp$2.apply(ContextCleaner.scala:179)
at scala.Option.foreach(Option.scala:257)
at
org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1.apply$mcV$sp(ContextCleaner.scala:179)
at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1181)
at
org.apache.spark.ContextCleaner.org$apache$spark$ContextCleaner$$keepCleaning(ContextCleaner.scala:172)
at org.apache.spark.ContextCleaner$$anon$1.run(ContextCleaner.scala:67)
Caused by: java.lang.IllegalStateException: Task -1024 has already locked
broadcast_755_piece0 for writing
at
org.apache.spark.storage.BlockInfoManager.lockForWriting(BlockInfoManager.scala:232)
at
org.apache.spark.storage.BlockManager.removeBlock(BlockManager.scala:1296)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at
org.apache.spark.storage.BlockManager.removeBroadcast(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply$mcI$sp(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$1.apply(BlockManagerSlaveEndpoint.scala:82)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Execution time: 17.794742s
Running execution q26-v1.4 iteration: 1, StandardRun=true
Execution time: 2.847808s
Running execution q27-v1.4 iteration: 1, StandardRun=true
Execution time: 3.747907s
Running execution q28-v1.4 iteration: 1, StandardRun=true
Execution time: 14.445853999999999s
Running execution q29-v1.4 iteration: 1, StandardRun=true
Execution time: 17.120878s
Running execution q31-v1.4 iteration: 1, StandardRun=true
16/05/23 00:40:16 ERROR BlockManagerSlaveEndpoint: Error in removing broadcast
1015
java.lang.IllegalStateException: Task -1024 has already locked
broadcast_1015_piece0 for writing
at
org.apache.spark.storage.BlockInfoManager.lockForWriting(BlockInfoManager.scala:232)
at
org.apache.spark.storage.BlockManager.removeBlock(BlockManager.scala:1296)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at
org.apache.spark.storage.BlockManager.removeBroadcast(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply$mcI$sp(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$1.apply(BlockManagerSlaveEndpoint.scala:82)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
16/05/23 00:40:16 WARN BlockManagerMaster: Failed to remove broadcast 1015 with
removeFromMaster = true - Task -1024 has already locked broadcast_1015_piece0
for writing
java.lang.IllegalStateException: Task -1024 has already locked
broadcast_1015_piece0 for writing
at
org.apache.spark.storage.BlockInfoManager.lockForWriting(BlockInfoManager.scala:232)
at
org.apache.spark.storage.BlockManager.removeBlock(BlockManager.scala:1296)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at
org.apache.spark.storage.BlockManager.removeBroadcast(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply$mcI$sp(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$1.apply(BlockManagerSlaveEndpoint.scala:82)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
16/05/23 00:40:16 ERROR ContextCleaner: Error cleaning broadcast 1015
org.apache.spark.SparkException: Exception thrown in awaitResult
at
org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.scala:77)
at
org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.scala:75)
at
scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
at
org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:59)
at
org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:59)
at scala.PartialFunction$OrElse.apply(PartialFunction.scala:167)
at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:83)
at
org.apache.spark.storage.BlockManagerMaster.removeBroadcast(BlockManagerMaster.scala:135)
at
org.apache.spark.broadcast.TorrentBroadcast$.unpersist(TorrentBroadcast.scala:260)
at
org.apache.spark.broadcast.TorrentBroadcastFactory.unbroadcast(TorrentBroadcastFactory.scala:45)
at
org.apache.spark.broadcast.BroadcastManager.unbroadcast(BroadcastManager.scala:60)
at
org.apache.spark.ContextCleaner.doCleanupBroadcast(ContextCleaner.scala:232)
at
org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1$$anonfun$apply$mcV$sp$2.apply(ContextCleaner.scala:188)
at
org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1$$anonfun$apply$mcV$sp$2.apply(ContextCleaner.scala:179)
at scala.Option.foreach(Option.scala:257)
at
org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1.apply$mcV$sp(ContextCleaner.scala:179)
at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1181)
at
org.apache.spark.ContextCleaner.org$apache$spark$ContextCleaner$$keepCleaning(ContextCleaner.scala:172)
at org.apache.spark.ContextCleaner$$anon$1.run(ContextCleaner.scala:67)
Caused by: java.lang.IllegalStateException: Task -1024 has already locked
broadcast_1015_piece0 for writing
at
org.apache.spark.storage.BlockInfoManager.lockForWriting(BlockInfoManager.scala:232)
at
org.apache.spark.storage.BlockManager.removeBlock(BlockManager.scala:1296)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at
org.apache.spark.storage.BlockManager.removeBroadcast(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply$mcI$sp(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$1.apply(BlockManagerSlaveEndpoint.scala:82)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Execution time: 12.933843000000001s
Running execution q34-v1.4 iteration: 1, StandardRun=true
Execution time: 3.036113s
Running execution q36-v1.4 iteration: 1, StandardRun=true
16/05/23 00:40:25 WARN BlockManager: Asked to remove block
broadcast_1072_piece0, which does not exist
Execution time: 2.589373s
Running execution q37-v1.4 iteration: 1, StandardRun=true
Execution time: 2.993193s
Running execution q38-v1.4 iteration: 1, StandardRun=true
Execution time: 11.397585s
Running execution q39a-v1.4 iteration: 1, StandardRun=true
16/05/23 00:40:44 WARN BlockManager: Asked to remove block
broadcast_1127_piece0, which does not exist
Execution time: 6.968036s
Running execution q39b-v1.4 iteration: 1, StandardRun=true
Execution time: 5.015952s
Running execution q40-v1.4 iteration: 1, StandardRun=true
Execution time: 7.644813s
Running execution q42-v1.4 iteration: 1, StandardRun=true
Execution time: 1.0390460000000001s
Running execution q43-v1.4 iteration: 1, StandardRun=true
Execution time: 1.824691s
Running execution q44-v1.4 iteration: 1, StandardRun=true
16/05/23 00:41:17 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:41:17 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:41:17 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:41:17 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:41:18 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:41:18 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
Execution time: 4.376587000000001s
Running execution q46-v1.4 iteration: 1, StandardRun=true
Execution time: 3.245273s
Running execution q47-v1.4 iteration: 1, StandardRun=true
16/05/23 00:42:08 WARN BlockManager: Asked to remove block
broadcast_1383_piece0, which does not exist
Execution time: 34.106424s
Running execution q48-v1.4 iteration: 1, StandardRun=true
Execution time: 3.63227s
Running execution q49-v1.4 iteration: 1, StandardRun=true
16/05/23 00:42:14 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:14 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:14 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:14 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:14 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:14 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:15 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:15 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:15 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:15 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:15 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:15 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:16 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:16 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:16 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:16 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:16 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:16 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:42:33 ERROR BlockManagerSlaveEndpoint: Error in removing broadcast
1451
java.lang.IllegalStateException: Task -1024 has already locked
broadcast_1451_piece0 for writing
at
org.apache.spark.storage.BlockInfoManager.lockForWriting(BlockInfoManager.scala:232)
at
org.apache.spark.storage.BlockManager.removeBlock(BlockManager.scala:1296)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at
org.apache.spark.storage.BlockManager.removeBroadcast(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply$mcI$sp(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$1.apply(BlockManagerSlaveEndpoint.scala:82)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
16/05/23 00:42:33 WARN BlockManagerMaster: Failed to remove broadcast 1451 with
removeFromMaster = true - Task -1024 has already locked broadcast_1451_piece0
for writing
java.lang.IllegalStateException: Task -1024 has already locked
broadcast_1451_piece0 for writing
at
org.apache.spark.storage.BlockInfoManager.lockForWriting(BlockInfoManager.scala:232)
at
org.apache.spark.storage.BlockManager.removeBlock(BlockManager.scala:1296)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at
org.apache.spark.storage.BlockManager.removeBroadcast(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply$mcI$sp(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$1.apply(BlockManagerSlaveEndpoint.scala:82)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Execution time: 15.109112s
Running execution q50-v1.4 iteration: 1, StandardRun=true
16/05/23 00:42:33 ERROR ContextCleaner: Error cleaning broadcast 1451
org.apache.spark.SparkException: Exception thrown in awaitResult
at
org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.scala:77)
at
org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.scala:75)
at
scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
at
org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:59)
at
org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:59)
at scala.PartialFunction$OrElse.apply(PartialFunction.scala:167)
at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:83)
at
org.apache.spark.storage.BlockManagerMaster.removeBroadcast(BlockManagerMaster.scala:135)
at
org.apache.spark.broadcast.TorrentBroadcast$.unpersist(TorrentBroadcast.scala:260)
at
org.apache.spark.broadcast.TorrentBroadcastFactory.unbroadcast(TorrentBroadcastFactory.scala:45)
at
org.apache.spark.broadcast.BroadcastManager.unbroadcast(BroadcastManager.scala:60)
at
org.apache.spark.ContextCleaner.doCleanupBroadcast(ContextCleaner.scala:232)
at
org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1$$anonfun$apply$mcV$sp$2.apply(ContextCleaner.scala:188)
at
org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1$$anonfun$apply$mcV$sp$2.apply(ContextCleaner.scala:179)
at scala.Option.foreach(Option.scala:257)
at
org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1.apply$mcV$sp(ContextCleaner.scala:179)
at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1181)
at
org.apache.spark.ContextCleaner.org$apache$spark$ContextCleaner$$keepCleaning(ContextCleaner.scala:172)
at org.apache.spark.ContextCleaner$$anon$1.run(ContextCleaner.scala:67)
Caused by: java.lang.IllegalStateException: Task -1024 has already locked
broadcast_1451_piece0 for writing
at
org.apache.spark.storage.BlockInfoManager.lockForWriting(BlockInfoManager.scala:232)
at
org.apache.spark.storage.BlockManager.removeBlock(BlockManager.scala:1296)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at
org.apache.spark.storage.BlockManager.removeBroadcast(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply$mcI$sp(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$1.apply(BlockManagerSlaveEndpoint.scala:82)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Execution time: 9.809977s
Running execution q51-v1.4 iteration: 1, StandardRun=true
Execution time: 33.090788s
Running execution q52-v1.4 iteration: 1, StandardRun=true
Execution time: 1.291409s
Running execution q53-v1.4 iteration: 1, StandardRun=true
16/05/23 00:43:30 WARN BlockManager: Asked to remove block
broadcast_1531_piece0, which does not exist
Execution time: 3.432714s
Running execution q54-v1.4 iteration: 1, StandardRun=true
Execution time: 8.835495000000002s
Running execution q55-v1.4 iteration: 1, StandardRun=true
Execution time: 1.374749s
Running execution q57-v1.4 iteration: 1, StandardRun=true
Execution time: 34.045852s
Running execution q59-v1.4 iteration: 1, StandardRun=true
16/05/23 00:44:35 WARN BlockManager: Asked to remove block
broadcast_1715_piece0, which does not exist
Execution time: 5.568991s
Running execution q61-v1.4 iteration: 1, StandardRun=true
16/05/23 00:44:42 ERROR BlockManagerSlaveEndpoint: Error in removing block
broadcast_1763_piece0
java.lang.IllegalStateException: Task -1024 has already locked
broadcast_1763_piece0 for writing
at
org.apache.spark.storage.BlockInfoManager.lockForWriting(BlockInfoManager.scala:232)
at
org.apache.spark.storage.BlockManager.removeBlock(BlockManager.scala:1296)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$1.apply$mcZ$sp(BlockManagerSlaveEndpoint.scala:47)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$1.apply(BlockManagerSlaveEndpoint.scala:46)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$1.apply(BlockManagerSlaveEndpoint.scala:46)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$1.apply(BlockManagerSlaveEndpoint.scala:82)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Execution time: 2.809748s
Running execution q62-v1.4 iteration: 1, StandardRun=true
Execution time: 2.529738s
Running execution q63-v1.4 iteration: 1, StandardRun=true
Execution time: 3.1742730000000003s
Running execution q64-v1.4 iteration: 1, StandardRun=true
Execution time: 36.348962s
Running execution q65-v1.4 iteration: 1, StandardRun=true
Execution time: 9.292081s
Running execution q66-v1.4 iteration: 1, StandardRun=true
Execution time: 5.666406s
Running execution q67-v1.4 iteration: 1, StandardRun=true
Execution time: 14.164485s
Running execution q68-v1.4 iteration: 1, StandardRun=true
Execution time: 3.292982s
Running execution q71-v1.4 iteration: 1, StandardRun=true
Execution time: 4.630912s
Running execution q72-v1.4 iteration: 1, StandardRun=true
Execution time: 34.012607s
Running execution q73-v1.4 iteration: 1, StandardRun=true
Execution time: 2.367591s
Running execution q74-v1.4 iteration: 1, StandardRun=true
Execution time: 26.1375s
Running execution q75-v1.4 iteration: 1, StandardRun=true
16/05/23 00:48:56 WARN BlockManager: Asked to remove block
broadcast_2367_piece0, which does not exist
16/05/23 00:48:56 WARN BlockManager: Asked to remove block
broadcast_2361_piece0, which does not exist
Execution time: 63.607198s
Running execution q76-v1.4 iteration: 1, StandardRun=true
Execution time: 4.265584s
Running execution q77-v1.4 iteration: 1, StandardRun=true
Execution time: 6.935116s
Running execution q78-v1.4 iteration: 1, StandardRun=true
Execution time: 32.66075s
Running execution q79-v1.4 iteration: 1, StandardRun=true
Execution time: 4.030887s
Running execution q80-v1.4 iteration: 1, StandardRun=true
Execution time: 30.15457s
Running execution q82-v1.4 iteration: 1, StandardRun=true
Execution time: 4.198055s
Running execution q84-v1.4 iteration: 1, StandardRun=true
Execution time: 3.603463s
Running execution q85-v1.4 iteration: 1, StandardRun=true
16/05/23 00:51:03 WARN BlockManager: Asked to remove block
broadcast_2716_piece8, which does not exist
Execution time: 10.293922s
Running execution q86-v1.4 iteration: 1, StandardRun=true
Execution time: 2.3347469999999997s
Running execution q87-v1.4 iteration: 1, StandardRun=true
Execution time: 15.640649s
Running execution q88-v1.4 iteration: 1, StandardRun=true
16/05/23 00:51:40 WARN BlockManager: Asked to remove block
broadcast_2909_piece0, which does not exist
Execution time: 6.346291s
Running execution q89-v1.4 iteration: 1, StandardRun=true
Execution time: 4.1178490000000005s
Running execution q90-v1.4 iteration: 1, StandardRun=true
Execution time: 1.519127s
Running execution q91-v1.4 iteration: 1, StandardRun=true
Execution time: 1.500019s
Running execution q93-v1.4 iteration: 1, StandardRun=true
Execution time: 13.214564000000001s
Running execution q96-v1.4 iteration: 1, StandardRun=true
Execution time: 0.8901s
Running execution q97-v1.4 iteration: 1, StandardRun=true
Execution time: 7.265679s
Running execution q98-v1.4 iteration: 1, StandardRun=true
Execution time: 3.608366s
Running execution q99-v1.4 iteration: 1, StandardRun=true
Execution time: 3.0280929999999997s
Running execution qSsMax-v1.4 iteration: 1, StandardRun=true
Execution time: 2.646716s
##########################################################################################
Running execution q1-v1.4 iteration: 2, StandardRun=true
Execution time: 7.057716999999999s
Running execution q2-v1.4 iteration: 2, StandardRun=true
Execution time: 4.005946s
Running execution q3-v1.4 iteration: 2, StandardRun=true
Execution time: 1.371915s
Running execution q4-v1.4 iteration: 2, StandardRun=true
Execution time: 58.654839s
Running execution q5-v1.4 iteration: 2, StandardRun=true
Execution time: 14.457578999999999s
Running execution q7-v1.4 iteration: 2, StandardRun=true
Execution time: 4.062525s
Running execution q8-v1.4 iteration: 2, StandardRun=true
Execution time: 2.912124s
Running execution q9-v1.4 iteration: 2, StandardRun=true
16/05/23 00:54:55 ERROR Executor: Exception in task 0.0 in stage 1111.0 (TID
76531)
java.lang.NullPointerException
at
org.apache.spark.sql.execution.ScalarSubquery.dataType(subquery.scala:45)
at
org.apache.spark.sql.catalyst.expressions.CaseWhenBase.dataType(conditionalExpressions.scala:103)
at
org.apache.spark.sql.catalyst.expressions.Alias.toAttribute(namedExpressions.scala:165)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.immutable.List.map(List.scala:285)
at
org.apache.spark.sql.execution.ProjectExec.output(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.WholeStageCodegenExec.output(WholeStageCodegenExec.scala:289)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:61)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:60)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
16/05/23 00:54:55 WARN TaskSetManager: Lost task 0.0 in stage 1111.0 (TID
76531, localhost): java.lang.NullPointerException
at
org.apache.spark.sql.execution.ScalarSubquery.dataType(subquery.scala:45)
at
org.apache.spark.sql.catalyst.expressions.CaseWhenBase.dataType(conditionalExpressions.scala:103)
at
org.apache.spark.sql.catalyst.expressions.Alias.toAttribute(namedExpressions.scala:165)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.immutable.List.map(List.scala:285)
at
org.apache.spark.sql.execution.ProjectExec.output(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.WholeStageCodegenExec.output(WholeStageCodegenExec.scala:289)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:61)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:60)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
16/05/23 00:54:55 ERROR TaskSetManager: Task 0 in stage 1111.0 failed 1 times;
aborting job
Execution 'q9-v1.4' failed: Job aborted due to stage failure: Task 0 in stage
1111.0 failed 1 times, most recent failure: Lost task 0.0 in stage 1111.0 (TID
76531, localhost): java.lang.NullPointerException
at
org.apache.spark.sql.execution.ScalarSubquery.dataType(subquery.scala:45)
at
org.apache.spark.sql.catalyst.expressions.CaseWhenBase.dataType(conditionalExpressions.scala:103)
at
org.apache.spark.sql.catalyst.expressions.Alias.toAttribute(namedExpressions.scala:165)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.immutable.List.map(List.scala:285)
at
org.apache.spark.sql.execution.ProjectExec.output(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.WholeStageCodegenExec.output(WholeStageCodegenExec.scala:289)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:61)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:60)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Driver stacktrace:
Running execution q11-v1.4 iteration: 2, StandardRun=true
Execution time: 28.856122s
Running execution q12-v1.4 iteration: 2, StandardRun=true
Execution time: 2.6626950000000003s
Running execution q13-v1.4 iteration: 2, StandardRun=true
16/05/23 00:55:51 WARN BlockManager: Asked to remove block
broadcast_3579_piece0, which does not exist
Execution time: 5.9593869999999995s
Running execution q15-v1.4 iteration: 2, StandardRun=true
Execution time: 3.249051s
Running execution q17-v1.4 iteration: 2, StandardRun=true
Execution time: 19.453306s
Running execution q18-v1.4 iteration: 2, StandardRun=true
16/05/23 00:56:31 WARN BlockManager: Asked to remove block
broadcast_3673_piece0, which does not exist
Execution time: 5.284472s
Running execution q19-v1.4 iteration: 2, StandardRun=true
Execution time: 2.315645s
Running execution q20-v1.4 iteration: 2, StandardRun=true
Execution time: 2.90954s
Running execution q21-v1.4 iteration: 2, StandardRun=true
Execution time: 2.243049s
Running execution q22-v1.4 iteration: 2, StandardRun=true
Execution time: 6.339354s
Running execution q25-v1.4 iteration: 2, StandardRun=true
Execution time: 20.697432s
Running execution q26-v1.4 iteration: 2, StandardRun=true
Execution time: 2.745586s
Running execution q27-v1.4 iteration: 2, StandardRun=true
Execution time: 3.6981080000000004s
Running execution q28-v1.4 iteration: 2, StandardRun=true
Execution time: 14.774464s
Running execution q29-v1.4 iteration: 2, StandardRun=true
Execution time: 20.057629000000002s
Running execution q31-v1.4 iteration: 2, StandardRun=true
Execution time: 15.194232s
Running execution q34-v1.4 iteration: 2, StandardRun=true
16/05/23 00:58:43 WARN BlockManager: Asked to remove block
broadcast_4045_piece0, which does not exist
Execution time: 3.248439s
Running execution q36-v1.4 iteration: 2, StandardRun=true
Execution time: 3.067586s
Running execution q37-v1.4 iteration: 2, StandardRun=true
Execution time: 4.098272s
Running execution q38-v1.4 iteration: 2, StandardRun=true
16/05/23 00:59:15 WARN BlockManager: Asked to remove block
broadcast_4123_piece0, which does not exist
Execution time: 13.738226000000001s
Running execution q39a-v1.4 iteration: 2, StandardRun=true
16/05/23 00:59:27 WARN BlockManager: Asked to remove block
broadcast_4169_piece0, which does not exist
Execution time: 7.920738s
Running execution q39b-v1.4 iteration: 2, StandardRun=true
Execution time: 4.97345s
Running execution q40-v1.4 iteration: 2, StandardRun=true
Execution time: 8.193442s
Running execution q42-v1.4 iteration: 2, StandardRun=true
16/05/23 00:59:49 WARN BlockManager: Asked to remove block
broadcast_4250_piece0, which does not exist
Execution time: 0.9597060000000001s
Running execution q43-v1.4 iteration: 2, StandardRun=true
16/05/23 00:59:53 WARN BlockManager: Asked to remove block
broadcast_4264_piece0, which does not exist
Execution time: 2.0044020000000002s
Running execution q44-v1.4 iteration: 2, StandardRun=true
16/05/23 00:59:54 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:59:54 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:59:54 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:59:54 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:59:54 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 00:59:54 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
Execution time: 4.509967s
Running execution q46-v1.4 iteration: 2, StandardRun=true
Execution time: 3.235608s
Running execution q47-v1.4 iteration: 2, StandardRun=true
Execution time: 35.467439s
Running execution q48-v1.4 iteration: 2, StandardRun=true
Execution time: 3.392444s
Running execution q49-v1.4 iteration: 2, StandardRun=true
16/05/23 01:00:54 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:54 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:54 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:54 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:54 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:54 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:55 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:55 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:55 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:55 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:55 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:55 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:56 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:56 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:56 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:56 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:56 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:00:56 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
Execution time: 15.231178s
Running execution q50-v1.4 iteration: 2, StandardRun=true
16/05/23 01:01:26 WARN BlockManager: Asked to remove block
broadcast_4484_piece0, which does not exist
Execution time: 9.642467s
Running execution q51-v1.4 iteration: 2, StandardRun=true
Execution time: 32.018196s
Running execution q52-v1.4 iteration: 2, StandardRun=true
Execution time: 1.288138s
Running execution q53-v1.4 iteration: 2, StandardRun=true
Execution time: 3.441164s
Running execution q54-v1.4 iteration: 2, StandardRun=true
Execution time: 8.336381s
Running execution q55-v1.4 iteration: 2, StandardRun=true
Execution time: 1.334499s
Running execution q57-v1.4 iteration: 2, StandardRun=true
Execution time: 31.494508999999997s
Running execution q59-v1.4 iteration: 2, StandardRun=true
Execution time: 5.255688s
Running execution q61-v1.4 iteration: 2, StandardRun=true
16/05/23 01:03:20 WARN BlockManager: Asked to remove block
broadcast_4728_piece0, which does not exist
Execution time: 2.521154s
Running execution q62-v1.4 iteration: 2, StandardRun=true
Execution time: 2.552513s
Running execution q63-v1.4 iteration: 2, StandardRun=true
Execution time: 3.140888s
Running execution q64-v1.4 iteration: 2, StandardRun=true
Execution time: 37.702866s
Running execution q65-v1.4 iteration: 2, StandardRun=true
Execution time: 9.040861000000001s
Running execution q66-v1.4 iteration: 2, StandardRun=true
Execution time: 5.116697s
Running execution q67-v1.4 iteration: 2, StandardRun=true
Execution time: 15.144709s
Running execution q68-v1.4 iteration: 2, StandardRun=true
Execution time: 2.9580859999999998s
Running execution q71-v1.4 iteration: 2, StandardRun=true
Execution time: 4.633735s
Running execution q72-v1.4 iteration: 2, StandardRun=true
Execution time: 35.607901s
Running execution q73-v1.4 iteration: 2, StandardRun=true
Execution time: 2.2195970000000003s
Running execution q74-v1.4 iteration: 2, StandardRun=true
16/05/23 01:06:26 WARN BlockManager: Asked to remove block
broadcast_5267_piece0, which does not exist
Execution time: 25.844392s
Running execution q75-v1.4 iteration: 2, StandardRun=true
Execution time: 59.665131s
Running execution q76-v1.4 iteration: 2, StandardRun=true
Execution time: 3.68838s
Running execution q77-v1.4 iteration: 2, StandardRun=true
Execution time: 6.845957s
Running execution q78-v1.4 iteration: 2, StandardRun=true
Execution time: 30.298175s
Running execution q79-v1.4 iteration: 2, StandardRun=true
Execution time: 3.281832s
Running execution q80-v1.4 iteration: 2, StandardRun=true
Execution time: 30.226626s
Running execution q82-v1.4 iteration: 2, StandardRun=true
Execution time: 3.612107s
Running execution q84-v1.4 iteration: 2, StandardRun=true
Execution time: 4.26783s
Running execution q85-v1.4 iteration: 2, StandardRun=true
Execution time: 10.756991s
Running execution q86-v1.4 iteration: 2, StandardRun=true
16/05/23 01:09:40 WARN BlockManager: Asked to remove block
broadcast_5732_piece0, which does not exist
Execution time: 2.222719s
Running execution q87-v1.4 iteration: 2, StandardRun=true
Execution time: 15.259191999999999s
Running execution q88-v1.4 iteration: 2, StandardRun=true
Execution time: 6.685505s
Running execution q89-v1.4 iteration: 2, StandardRun=true
Execution time: 4.059044s
Running execution q90-v1.4 iteration: 2, StandardRun=true
Execution time: 1.388765s
Running execution q91-v1.4 iteration: 2, StandardRun=true
Execution time: 1.423935s
Running execution q93-v1.4 iteration: 2, StandardRun=true
Execution time: 12.459983s
Running execution q96-v1.4 iteration: 2, StandardRun=true
Execution time: 0.822059s
Running execution q97-v1.4 iteration: 2, StandardRun=true
16/05/23 01:10:56 WARN BlockManager: Asked to remove block
broadcast_6066_piece0, which does not exist
Execution time: 6.997206s
Running execution q98-v1.4 iteration: 2, StandardRun=true
Execution time: 3.6080680000000003s
Running execution q99-v1.4 iteration: 2, StandardRun=true
Execution time: 2.709227s
Running execution qSsMax-v1.4 iteration: 2, StandardRun=true
Execution time: 2.3527359999999997s
###################################################################################################
Running execution q1-v1.4 iteration: 3, StandardRun=true
Execution time: 7.094931s
Running execution q2-v1.4 iteration: 3, StandardRun=true
Execution time: 3.9405419999999998s
Running execution q3-v1.4 iteration: 3, StandardRun=true
Execution time: 1.131667s
Running execution q4-v1.4 iteration: 3, StandardRun=true
Execution time: 45.563398s
Running execution q5-v1.4 iteration: 3, StandardRun=true
Execution time: 13.418183s
Running execution q7-v1.4 iteration: 3, StandardRun=true
Execution time: 4.048009s
Running execution q8-v1.4 iteration: 3, StandardRun=true
Execution time: 3.172596s
Running execution q9-v1.4 iteration: 3, StandardRun=true
16/05/23 01:13:03 ERROR Executor: Exception in task 0.0 in stage 1981.0 (TID
140061)
java.lang.NullPointerException
at
org.apache.spark.sql.execution.ScalarSubquery.dataType(subquery.scala:45)
at
org.apache.spark.sql.catalyst.expressions.CaseWhenBase.dataType(conditionalExpressions.scala:103)
at
org.apache.spark.sql.catalyst.expressions.Alias.toAttribute(namedExpressions.scala:165)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.immutable.List.map(List.scala:285)
at
org.apache.spark.sql.execution.ProjectExec.output(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.WholeStageCodegenExec.output(WholeStageCodegenExec.scala:289)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:61)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:60)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
16/05/23 01:13:03 WARN TaskSetManager: Lost task 0.0 in stage 1981.0 (TID
140061, localhost): java.lang.NullPointerException
at
org.apache.spark.sql.execution.ScalarSubquery.dataType(subquery.scala:45)
at
org.apache.spark.sql.catalyst.expressions.CaseWhenBase.dataType(conditionalExpressions.scala:103)
at
org.apache.spark.sql.catalyst.expressions.Alias.toAttribute(namedExpressions.scala:165)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.immutable.List.map(List.scala:285)
at
org.apache.spark.sql.execution.ProjectExec.output(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.WholeStageCodegenExec.output(WholeStageCodegenExec.scala:289)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:61)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:60)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
16/05/23 01:13:03 ERROR TaskSetManager: Task 0 in stage 1981.0 failed 1 times;
aborting job
Execution 'q9-v1.4' failed: Job aborted due to stage failure: Task 0 in stage
1981.0 failed 1 times, most recent failure: Lost task 0.0 in stage 1981.0 (TID
140061, localhost): java.lang.NullPointerException
at
org.apache.spark.sql.execution.ScalarSubquery.dataType(subquery.scala:45)
at
org.apache.spark.sql.catalyst.expressions.CaseWhenBase.dataType(conditionalExpressions.scala:103)
at
org.apache.spark.sql.catalyst.expressions.Alias.toAttribute(namedExpressions.scala:165)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.ProjectExec$$anonfun$output$1.apply(basicPhysicalOperators.scala:33)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.immutable.List.map(List.scala:285)
at
org.apache.spark.sql.execution.ProjectExec.output(basicPhysicalOperators.scala:33)
at
org.apache.spark.sql.execution.WholeStageCodegenExec.output(WholeStageCodegenExec.scala:289)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:61)
at
org.apache.spark.sql.execution.DeserializeToObject$$anonfun$2.apply(objects.scala:60)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$23.apply(RDD.scala:774)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:318)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:282)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Driver stacktrace:
Running execution q11-v1.4 iteration: 3, StandardRun=true
Execution time: 27.974636999999998s
Running execution q12-v1.4 iteration: 3, StandardRun=true
Execution time: 2.482792s
Running execution q13-v1.4 iteration: 3, StandardRun=true
Execution time: 6.217131s
Running execution q15-v1.4 iteration: 3, StandardRun=true
Execution time: 3.012969s
Running execution q17-v1.4 iteration: 3, StandardRun=true
Execution time: 18.443051s
Running execution q18-v1.4 iteration: 3, StandardRun=true
16/05/23 01:14:32 ERROR BlockManagerSlaveEndpoint: Error in removing broadcast
6671
java.lang.IllegalStateException: Task -1024 has already locked
broadcast_6671_piece0 for writing
at
org.apache.spark.storage.BlockInfoManager.lockForWriting(BlockInfoManager.scala:232)
at
org.apache.spark.storage.BlockManager.removeBlock(BlockManager.scala:1296)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at
org.apache.spark.storage.BlockManager.removeBroadcast(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply$mcI$sp(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$1.apply(BlockManagerSlaveEndpoint.scala:82)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
16/05/23 01:14:32 WARN BlockManagerMaster: Failed to remove broadcast 6671 with
removeFromMaster = true - Task -1024 has already locked broadcast_6671_piece0
for writing
java.lang.IllegalStateException: Task -1024 has already locked
broadcast_6671_piece0 for writing
at
org.apache.spark.storage.BlockInfoManager.lockForWriting(BlockInfoManager.scala:232)
at
org.apache.spark.storage.BlockManager.removeBlock(BlockManager.scala:1296)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at
org.apache.spark.storage.BlockManager.removeBroadcast(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply$mcI$sp(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$1.apply(BlockManagerSlaveEndpoint.scala:82)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
16/05/23 01:14:32 ERROR ContextCleaner: Error cleaning broadcast 6671
org.apache.spark.SparkException: Exception thrown in awaitResult
at
org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.scala:77)
at
org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.scala:75)
at
scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
at
org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:59)
at
org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:59)
at scala.PartialFunction$OrElse.apply(PartialFunction.scala:167)
at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:83)
at
org.apache.spark.storage.BlockManagerMaster.removeBroadcast(BlockManagerMaster.scala:135)
at
org.apache.spark.broadcast.TorrentBroadcast$.unpersist(TorrentBroadcast.scala:260)
at
org.apache.spark.broadcast.TorrentBroadcastFactory.unbroadcast(TorrentBroadcastFactory.scala:45)
at org.apache.spaExecution time: 4.779654000000001s
Running execution q19-v1.4 iteration: 3, StandardRun=true
rk.broadcast.BroadcastManager.unbroadcast(BroadcastManager.scala:60)
at
org.apache.spark.ContextCleaner.doCleanupBroadcast(ContextCleaner.scala:232)
at
org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1$$anonfun$apply$mcV$sp$2.apply(ContextCleaner.scala:188)
at
org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1$$anonfun$apply$mcV$sp$2.apply(ContextCleaner.scala:179)
at scala.Option.foreach(Option.scala:257)
at
org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1.apply$mcV$sp(ContextCleaner.scala:179)
at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1181)
at
org.apache.spark.ContextCleaner.org$apache$spark$ContextCleaner$$keepCleaning(ContextCleaner.scala:172)
at org.apache.spark.ContextCleaner$$anon$1.run(ContextCleaner.scala:67)
Caused by: java.lang.IllegalStateException: Task -1024 has already locked
broadcast_6671_piece0 for writing
at
org.apache.spark.storage.BlockInfoManager.lockForWriting(BlockInfoManager.scala:232)
at
org.apache.spark.storage.BlockManager.removeBlock(BlockManager.scala:1296)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManager$$anonfun$removeBroadcast$2.apply(BlockManager.scala:1287)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at
org.apache.spark.storage.BlockManager.removeBroadcast(BlockManager.scala:1287)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply$mcI$sp(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$receiveAndReply$1$$anonfun$applyOrElse$4.apply(BlockManagerSlaveEndpoint.scala:66)
at
org.apache.spark.storage.BlockManagerSlaveEndpoint$$anonfun$1.apply(BlockManagerSlaveEndpoint.scala:82)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Execution time: 1.9448189999999999s
Running execution q20-v1.4 iteration: 3, StandardRun=true
Execution time: 2.749436s
Running execution q21-v1.4 iteration: 3, StandardRun=true
Execution time: 1.659196s
Running execution q22-v1.4 iteration: 3, StandardRun=true
Execution time: 5.851532s
Running execution q25-v1.4 iteration: 3, StandardRun=true
Execution time: 20.176004000000002s
Running execution q26-v1.4 iteration: 3, StandardRun=true
Execution time: 3.0417579999999997s
Running execution q27-v1.4 iteration: 3, StandardRun=true
Execution time: 3.95241s
Running execution q28-v1.4 iteration: 3, StandardRun=true
Execution time: 15.558399999999999s
Running execution q29-v1.4 iteration: 3, StandardRun=true
Execution time: 19.895013s
Running execution q31-v1.4 iteration: 3, StandardRun=true
Execution time: 15.363453s
Running execution q34-v1.4 iteration: 3, StandardRun=true
Execution time: 3.244011s
Running execution q36-v1.4 iteration: 3, StandardRun=true
Execution time: 3.261247s
Running execution q37-v1.4 iteration: 3, StandardRun=true
Execution time: 3.280498s
Running execution q38-v1.4 iteration: 3, StandardRun=true
Execution time: 12.059522000000001s
Running execution q39a-v1.4 iteration: 3, StandardRun=true
Execution time: 8.369285s
Running execution q39b-v1.4 iteration: 3, StandardRun=true
Execution time: 5.311641s
Running execution q40-v1.4 iteration: 3, StandardRun=true
Execution time: 9.636736s
Running execution q42-v1.4 iteration: 3, StandardRun=true
Execution time: 1.344935s
Running execution q43-v1.4 iteration: 3, StandardRun=true
Execution time: 2.116327s
Running execution q44-v1.4 iteration: 3, StandardRun=true
16/05/23 01:17:55 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:17:55 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:17:55 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:17:55 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:17:56 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:17:56 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
Execution time: 4.617470999999999s
Running execution q46-v1.4 iteration: 3, StandardRun=true
Execution time: 3.762496s
Running execution q47-v1.4 iteration: 3, StandardRun=true
Execution time: 36.76569s
Running execution q48-v1.4 iteration: 3, StandardRun=true
Execution time: 4.454999s
Running execution q49-v1.4 iteration: 3, StandardRun=true
16/05/23 01:19:01 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:01 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:01 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:01 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:01 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:01 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:02 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:02 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:02 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:02 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:02 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:02 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:02 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:02 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:02 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:02 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:02 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
16/05/23 01:19:02 WARN WindowExec: No Partition Defined for Window operation!
Moving all data to a single partition, this can cause serious performance
degradation.
Execution time: 15.890137000000001s
Running execution q50-v1.4 iteration: 3, StandardRun=true
Execution time: 9.908862s
Running execution q51-v1.4 iteration: 3, StandardRun=true
16/05/23 01:20:13 WARN BlockManager: Asked to remove block
broadcast_7509_piece0, which does not exist
Execution time: 33.958834s
Running execution q52-v1.4 iteration: 3, StandardRun=true
Execution time: 1.23084s
Running execution q53-v1.4 iteration: 3, StandardRun=true
Execution time: 3.361366s
Running execution q54-v1.4 iteration: 3, StandardRun=true
Execution time: 8.531334000000001s
Running execution q55-v1.4 iteration: 3, StandardRun=true
Execution time: 1.376814s
Running execution q57-v1.4 iteration: 3, StandardRun=true
Execution time: 35.25222s
Running execution q59-v1.4 iteration: 3, StandardRun=true
Execution time: 5.899032s
Running execution q61-v1.4 iteration: 3, StandardRun=true
Execution time: 2.863627s
Running execution q62-v1.4 iteration: 3, StandardRun=true
Execution time: 2.606057s
Running execution q63-v1.4 iteration: 3, StandardRun=true
Execution time: 2.898519s
Running execution q64-v1.4 iteration: 3, StandardRun=true
Execution time: 36.1879s
Running execution q65-v1.4 iteration: 3, StandardRun=true
Execution time: 8.676874s
Running execution q66-v1.4 iteration: 3, StandardRun=true
Execution time: 5.051299s
Running execution q67-v1.4 iteration: 3, StandardRun=true
Execution time: 13.10159s
Running execution q68-v1.4 iteration: 3, StandardRun=true
Execution time: 3.007121s
Running execution q71-v1.4 iteration: 3, StandardRun=true
Execution time: 3.958377s
Running execution q72-v1.4 iteration: 3, StandardRun=true
Execution time: 32.607965s
Running execution q73-v1.4 iteration: 3, StandardRun=true
Execution time: 2.033648s
Running execution q74-v1.4 iteration: 3, StandardRun=true
Execution time: 24.767559000000002s
Running execution q75-v1.4 iteration: 3, StandardRun=true
Execution time: 56.078994s
Running execution q76-v1.4 iteration: 3, StandardRun=true
Execution time: 3.5158069999999997s
Running execution q77-v1.4 iteration: 3, StandardRun=true
Execution time: 5.832628s
Running execution q78-v1.4 iteration: 3, StandardRun=true
Execution time: 28.71925s
Running execution q79-v1.4 iteration: 3, StandardRun=true
Execution time: 3.1472979999999997s
Running execution q80-v1.4 iteration: 3, StandardRun=true
Execution time: 28.924941999999998s
Running execution q82-v1.4 iteration: 3, StandardRun=true
Execution time: 3.431854s
Running execution q84-v1.4 iteration: 3, StandardRun=true
16/05/23 01:27:27 WARN BlockManager: Asked to remove block
broadcast_8678_piece0, which does not exist
Execution time: 3.752372s
Running execution q85-v1.4 iteration: 3, StandardRun=true
Execution time: 9.479683000000001s
Running execution q86-v1.4 iteration: 3, StandardRun=true
16/05/23 01:27:46 WARN BlockManager: Asked to remove block
broadcast_8725_piece0, which does not exist
Execution time: 2.2747979999999997s
Running execution q87-v1.4 iteration: 3, StandardRun=true
Execution time: 14.90815s
Running execution q88-v1.4 iteration: 3, StandardRun=true
Execution time: 6.881189999999999s
Running execution q89-v1.4 iteration: 3, StandardRun=true
Execution time: 3.861533s
Running execution q90-v1.4 iteration: 3, StandardRun=true
Execution time: 1.3322049999999999s
Running execution q91-v1.4 iteration: 3, StandardRun=true
Execution time: 2.099972s
Running execution q93-v1.4 iteration: 3, StandardRun=true
Execution time: 11.231245000000001s
Running execution q96-v1.4 iteration: 3, StandardRun=true
Execution time: 0.825515s
Running execution q97-v1.4 iteration: 3, StandardRun=true
Execution time: 7.247237s
Running execution q98-v1.4 iteration: 3, StandardRun=true
Execution time: 3.563375s
Running execution q99-v1.4 iteration: 3, StandardRun=true
Execution time: 2.685846s
Running execution qSsMax-v1.4 iteration: 3, StandardRun=true
Execution time: 2.165382s
> Add all TPCDS 1.4 benchmark queries for SparkSQL
> ------------------------------------------------
>
> Key: SPARK-15078
> URL: https://issues.apache.org/jira/browse/SPARK-15078
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Reporter: Sameer Agarwal
> Assignee: Sameer Agarwal
> Fix For: 2.0.0, 2.1.0
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
