[ 
https://issues.apache.org/jira/browse/SPARK-30805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17036680#comment-17036680
 ] 

Hyukjin Kwon commented on SPARK-30805:
--------------------------------------

Firstly, please avoid to set Critical+ which is reserved for committers. Seems 
like impossible to investigate or reproduce only given that the stacktrace.
I am leaving this resolved until anyone can reproduce.

> Failed to get locally stored broadcast data: broadcast_30
> ---------------------------------------------------------
>
>                 Key: SPARK-30805
>                 URL: https://issues.apache.org/jira/browse/SPARK-30805
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.4.4
>         Environment: spark 2.4.4
>            Reporter: Jiasi
>            Priority: Major
>
> the stack trace is below:
>  
> {quote}20/02/08 04:56:30 ERROR Utils: Exception 
> encounteredjava.io.EOFExceptionat 
> java.io.ObjectInputStream$BlockDataInputStream.readFully(ObjectInputStream.java:3049)at
>  java.io.ObjectInputStream.readFully(ObjectInputStream.java:1084)at 
> org.apache.spark.sql.execution.joins.UnsafeHashedRelation$$anonfun$readExternal$1$$anonfun$apply$mcV$sp$11.apply(HashedRelation.scala:216)at
>  
> org.apache.spark.sql.execution.joins.UnsafeHashedRelation$$anonfun$readExternal$1$$anonfun$apply$mcV$sp$11.apply(HashedRelation.scala:216)at
>  
> org.apache.spark.sql.execution.joins.UnsafeHashedRelation.org$apache$spark$sql$execution$joins$UnsafeHashedRelation$$read(HashedRelation.scala:259)at
>  
> org.apache.spark.sql.execution.joins.UnsafeHashedRelation$$anonfun$readExternal$1.apply$mcV$sp(HashedRelation.scala:216)at
>  
> org.apache.spark.sql.execution.joins.UnsafeHashedRelation$$anonfun$readExternal$1.apply(HashedRelation.scala:216)at
>  
> org.apache.spark.sql.execution.joins.UnsafeHashedRelation$$anonfun$readExternal$1.apply(HashedRelation.scala:216)at
>  org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1326)at 
> org.apache.spark.sql.execution.joins.UnsafeHashedRelation.readExternal(HashedRelation.scala:215)at
>  java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2062)at 
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2011)at 
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)at 
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)at 
> org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75)at
>  
> org.apache.spark.serializer.DeserializationStream$$anon$1.getNext(Serializer.scala:168)at
>  org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:73)at 
> org.apache.spark.storage.memory.MemoryStore.putIterator(MemoryStore.scala:221)at
>  
> org.apache.spark.storage.memory.MemoryStore.putIteratorAsValues(MemoryStore.scala:299)at
>  
> org.apache.spark.storage.BlockManager.maybeCacheDiskValuesInMemory(BlockManager.scala:1312)at
>  
> org.apache.spark.storage.BlockManager.getLocalValues(BlockManager.scala:612)at
>  
> org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1$$anonfun$apply$2.apply(TorrentBroadcast.scala:214)at
>  scala.Option.getOrElse(Option.scala:121)at 
> org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1.apply(TorrentBroadcast.scala:211)at
>  org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1326)at 
> org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:207)at
>  
> org.apache.spark.broadcast.TorrentBroadcast._value$lzycompute(TorrentBroadcast.scala:66)at
>  
> org.apache.spark.broadcast.TorrentBroadcast._value(TorrentBroadcast.scala:66)at
>  
> org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:96)at
>  org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)at 
> org.apache.spark.sql.execution.joins.BroadcastHashJoinExec.prepareBroadcast(BroadcastHashJoinExec.scala:119)at
>  
> org.apache.spark.sql.execution.joins.BroadcastHashJoinExec.codegenOuter(BroadcastHashJoinExec.scala:259)at
>  
> org.apache.spark.sql.execution.joins.BroadcastHashJoinExec.doConsume(BroadcastHashJoinExec.scala:102)at
>  
> org.apache.spark.sql.execution.CodegenSupport$class.constructDoConsumeFunction(WholeStageCodegenExec.scala:216)at
>  
> org.apache.spark.sql.execution.CodegenSupport$class.consume(WholeStageCodegenExec.scala:187)at
>  
> org.apache.spark.sql.execution.FilterExec.consume(basicPhysicalOperators.scala:85)at
>  
> org.apache.spark.sql.execution.FilterExec.doConsume(basicPhysicalOperators.scala:206)at
>  
> org.apache.spark.sql.execution.CodegenSupport$class.consume(WholeStageCodegenExec.scala:189)at
>  
> org.apache.spark.sql.execution.InputAdapter.consume(WholeStageCodegenExec.scala:374)at
>  
> org.apache.spark.sql.execution.InputAdapter.doProduce(WholeStageCodegenExec.scala:403)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:90)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.CodegenSupport$class.produce(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.InputAdapter.produce(WholeStageCodegenExec.scala:374)at
>  
> org.apache.spark.sql.execution.FilterExec.doProduce(basicPhysicalOperators.scala:125)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:90)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.CodegenSupport$class.produce(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.FilterExec.produce(basicPhysicalOperators.scala:85)at
>  
> org.apache.spark.sql.execution.joins.BroadcastHashJoinExec.doProduce(BroadcastHashJoinExec.scala:96)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:90)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.CodegenSupport$class.produce(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.joins.BroadcastHashJoinExec.produce(BroadcastHashJoinExec.scala:40)at
>  
> org.apache.spark.sql.execution.ProjectExec.doProduce(basicPhysicalOperators.scala:45)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:90)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.CodegenSupport$class.produce(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.ProjectExec.produce(basicPhysicalOperators.scala:35)at
>  
> org.apache.spark.sql.execution.WholeStageCodegenExec.doCodeGen(WholeStageCodegenExec.scala:544)at
>  
> org.apache.spark.sql.execution.WholeStageCodegenExec.doExecute(WholeStageCodegenExec.scala:598)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)at 
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec.prepareShuffleDependency(ShuffleExchangeExec.scala:92)at
>  
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec$$anonfun$doExecute$1.apply(ShuffleExchangeExec.scala:128)at
>  
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec$$anonfun$doExecute$1.apply(ShuffleExchangeExec.scala:119)at
>  org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:52)at 
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec.doExecute(ShuffleExchangeExec.scala:119)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)at 
> org.apache.spark.sql.execution.InputAdapter.inputRDDs(WholeStageCodegenExec.scala:391)at
>  org.apache.spark.sql.execution.SortExec.inputRDDs(SortExec.scala:121)at 
> org.apache.spark.sql.execution.WholeStageCodegenExec.doExecute(WholeStageCodegenExec.scala:627)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)at 
> org.apache.spark.sql.execution.joins.SortMergeJoinExec.doExecute(SortMergeJoinExec.scala:150)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)at 
> org.apache.spark.sql.execution.InputAdapter.inputRDDs(WholeStageCodegenExec.scala:391)at
>  
> org.apache.spark.sql.execution.ProjectExec.inputRDDs(basicPhysicalOperators.scala:41)at
>  
> org.apache.spark.sql.execution.WholeStageCodegenExec.doExecute(WholeStageCodegenExec.scala:627)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)at 
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec.prepareShuffleDependency(ShuffleExchangeExec.scala:92)at
>  
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec$$anonfun$doExecute$1.apply(ShuffleExchangeExec.scala:128)at
>  
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec$$anonfun$doExecute$1.apply(ShuffleExchangeExec.scala:119)at
>  org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:52)at 
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec.doExecute(ShuffleExchangeExec.scala:119)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)at 
> org.apache.spark.sql.execution.InputAdapter.inputRDDs(WholeStageCodegenExec.scala:391)at
>  
> org.apache.spark.sql.execution.ProjectExec.inputRDDs(basicPhysicalOperators.scala:41)at
>  
> org.apache.spark.sql.execution.WholeStageCodegenExec.doExecute(WholeStageCodegenExec.scala:627)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)at 
> org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:143)at
>  
> org.apache.spark.sql.hive.execution.SaveAsHiveFile$class.saveAsHiveFile(SaveAsHiveFile.scala:86)at
>  
> org.apache.spark.sql.hive.execution.InsertIntoHiveTable.saveAsHiveFile(InsertIntoHiveTable.scala:66)at
>  
> org.apache.spark.sql.hive.execution.InsertIntoHiveTable.processInsert(InsertIntoHiveTable.scala:195)at
>  
> org.apache.spark.sql.hive.execution.InsertIntoHiveTable.run(InsertIntoHiveTable.scala:99)at
>  
> org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:104)at
>  
> org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:102)at
>  
> org.apache.spark.sql.execution.command.DataWritingCommandExec.executeCollect(commands.scala:115)at
>  org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:194)at 
> org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:194)at 
> org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3370)at 
> org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:78)at
>  
> org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125)at
>  
> org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73)at
>  org.apache.spark.sql.Dataset.withAction(Dataset.scala:3369)at 
> org.apache.spark.sql.Dataset.<init>(Dataset.scala:194)at 
> org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:79)at 
> org.apache.spark.sql.SparkSession.sql(SparkSession.scala:642)at 
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at
>  
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at
>  java.lang.reflect.Method.invoke(Method.java:498)at 
> org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:684)20/02/08
>  04:56:30 ERROR Utils: Exception encounteredorg.apache.spark.SparkException: 
> Failed to get locally stored broadcast data: broadcast_30at 
> org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1$$anonfun$apply$2.apply(TorrentBroadcast.scala:226)at
>  scala.Option.getOrElse(Option.scala:121)at 
> org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1.apply(TorrentBroadcast.scala:211)at
>  org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1326)at 
> org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:207)at
>  
> org.apache.spark.broadcast.TorrentBroadcast._value$lzycompute(TorrentBroadcast.scala:66)at
>  
> org.apache.spark.broadcast.TorrentBroadcast._value(TorrentBroadcast.scala:66)at
>  
> org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:96)at
>  org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)at 
> org.apache.spark.sql.execution.joins.BroadcastHashJoinExec.prepareBroadcast(BroadcastHashJoinExec.scala:119)at
>  
> org.apache.spark.sql.execution.joins.BroadcastHashJoinExec.codegenOuter(BroadcastHashJoinExec.scala:259)at
>  
> org.apache.spark.sql.execution.joins.BroadcastHashJoinExec.doConsume(BroadcastHashJoinExec.scala:102)at
>  
> org.apache.spark.sql.execution.CodegenSupport$class.constructDoConsumeFunction(WholeStageCodegenExec.scala:216)at
>  
> org.apache.spark.sql.execution.CodegenSupport$class.consume(WholeStageCodegenExec.scala:187)at
>  
> org.apache.spark.sql.execution.FilterExec.consume(basicPhysicalOperators.scala:85)at
>  
> org.apache.spark.sql.execution.FilterExec.doConsume(basicPhysicalOperators.scala:206)at
>  
> org.apache.spark.sql.execution.CodegenSupport$class.consume(WholeStageCodegenExec.scala:189)at
>  
> org.apache.spark.sql.execution.InputAdapter.consume(WholeStageCodegenExec.scala:374)at
>  
> org.apache.spark.sql.execution.InputAdapter.doProduce(WholeStageCodegenExec.scala:403)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:90)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.CodegenSupport$class.produce(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.InputAdapter.produce(WholeStageCodegenExec.scala:374)at
>  
> org.apache.spark.sql.execution.FilterExec.doProduce(basicPhysicalOperators.scala:125)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:90)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.CodegenSupport$class.produce(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.FilterExec.produce(basicPhysicalOperators.scala:85)at
>  
> org.apache.spark.sql.execution.joins.BroadcastHashJoinExec.doProduce(BroadcastHashJoinExec.scala:96)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:90)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.CodegenSupport$class.produce(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.joins.BroadcastHashJoinExec.produce(BroadcastHashJoinExec.scala:40)at
>  
> org.apache.spark.sql.execution.ProjectExec.doProduce(basicPhysicalOperators.scala:45)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:90)at
>  
> org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.CodegenSupport$class.produce(WholeStageCodegenExec.scala:85)at
>  
> org.apache.spark.sql.execution.ProjectExec.produce(basicPhysicalOperators.scala:35)at
>  
> org.apache.spark.sql.execution.WholeStageCodegenExec.doCodeGen(WholeStageCodegenExec.scala:544)at
>  
> org.apache.spark.sql.execution.WholeStageCodegenExec.doExecute(WholeStageCodegenExec.scala:598)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)at 
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec.prepareShuffleDependency(ShuffleExchangeExec.scala:92)at
>  
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec$$anonfun$doExecute$1.apply(ShuffleExchangeExec.scala:128)at
>  
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec$$anonfun$doExecute$1.apply(ShuffleExchangeExec.scala:119)at
>  org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:52)at 
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec.doExecute(ShuffleExchangeExec.scala:119)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)at 
> org.apache.spark.sql.execution.InputAdapter.inputRDDs(WholeStageCodegenExec.scala:391)at
>  org.apache.spark.sql.execution.SortExec.inputRDDs(SortExec.scala:121)at 
> org.apache.spark.sql.execution.WholeStageCodegenExec.doExecute(WholeStageCodegenExec.scala:627)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)at 
> org.apache.spark.sql.execution.joins.SortMergeJoinExec.doExecute(SortMergeJoinExec.scala:150)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)at 
> org.apache.spark.sql.execution.InputAdapter.inputRDDs(WholeStageCodegenExec.scala:391)at
>  
> org.apache.spark.sql.execution.ProjectExec.inputRDDs(basicPhysicalOperators.scala:41)at
>  
> org.apache.spark.sql.execution.WholeStageCodegenExec.doExecute(WholeStageCodegenExec.scala:627)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)at 
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec.prepareShuffleDependency(ShuffleExchangeExec.scala:92)at
>  
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec$$anonfun$doExecute$1.apply(ShuffleExchangeExec.scala:128)at
>  
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec$$anonfun$doExecute$1.apply(ShuffleExchangeExec.scala:119)at
>  org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:52)at 
> org.apache.spark.sql.execution.exchange.ShuffleExchangeExec.doExecute(ShuffleExchangeExec.scala:119)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)at 
> org.apache.spark.sql.execution.InputAdapter.inputRDDs(WholeStageCodegenExec.scala:391)at
>  
> org.apache.spark.sql.execution.ProjectExec.inputRDDs(basicPhysicalOperators.scala:41)at
>  
> org.apache.spark.sql.execution.WholeStageCodegenExec.doExecute(WholeStageCodegenExec.scala:627)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)at
>  
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)at
>  
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at
>  org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)at 
> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)at 
> org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:143)at
>  
> org.apache.spark.sql.hive.execution.SaveAsHiveFile$class.saveAsHiveFile(SaveAsHiveFile.scala:86)at
>  
> org.apache.spark.sql.hive.execution.InsertIntoHiveTable.saveAsHiveFile(InsertIntoHiveTable.scala:66)at
>  
> org.apache.spark.sql.hive.execution.InsertIntoHiveTable.processInsert(InsertIntoHiveTable.scala:195)at
>  
> org.apache.spark.sql.hive.execution.InsertIntoHiveTable.run(InsertIntoHiveTable.scala:99)at
>  
> org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:104)at
>  
> org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:102)at
>  
> org.apache.spark.sql.execution.command.DataWritingCommandExec.executeCollect(commands.scala:115)at
>  org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:194)at 
> org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:194)at 
> org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3370)at 
> org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:78)at
>  
> org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125)at
>  
> org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73)at
>  org.apache.spark.sql.Dataset.withAction(Dataset.scala:3369)at 
> org.apache.spark.sql.Dataset.<init>(Dataset.scala:194)at 
> org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:79)at 
> org.apache.spark.sql.SparkSession.sql(SparkSession.scala:642)at 
> com.task.cbas.label.utils.SparkSQLUtil.dataSetToHive(SparkSQLUtil.java:134)at 
> com.task.cbas.label.task.TaskExecutorByDate.executeDs(TaskExecutorByDate.java:269)at
>  
> com.task.cbas.label.task.TaskExecutorByDate.main(TaskExecutorByDate.java:75)at
>  sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at
>  
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at
>  java.lang.reflect.Method.invoke(Method.java:498)at 
> org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:684)
> {quote}
>  this exception happened once,  than we restart the task, everything is ok, 
> but why this happened.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to