Github user viirya commented on the issue:

    https://github.com/apache/spark/pull/21745
  
    I tried to checkout the commit 82183f7b57f2a93e646c56a9e37fac64b348ff0b 
which is before AnalysisBarrier commit.
    
    ```scala
    scala> val df = Seq(("test1", 0), ("test2", 1)).toDF("name", "id")
    18/07/12 05:36:52 WARN ObjectStore: Failed to get database global_temp, 
returning NoSuchObjectException
    df: org.apache.spark.sql.DataFrame = [name: string, id: int]
    
    scala> df.select(df("name")).filter(df("id") === 0).show()
    org.apache.spark.sql.AnalysisException: Resolved attribute(s) id#6 missing 
from name#5 in operator !Filter (id#6 = 0).;;
    !Filter (id#6 = 0)
    +- Project [name#5]
       +- Project [_1#2 AS name#5, _2#3 AS id#6]
          +- LocalRelation [_1#2, _2#3]
    
      at 
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.failAnalysis(CheckAnalysis.scala:41)
      at 
org.apache.spark.sql.catalyst.analysis.Analyzer.failAnalysis(Analyzer.scala:89)
      at 
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:291)
      at 
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:80)
      at 
org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:127)
      at 
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.checkAnalysis(CheckAnalysis.scala:80)
      at 
org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:89)
      at 
org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:53)
      at org.apache.spark.sql.Dataset.<init>(Dataset.scala:168)
      at org.apache.spark.sql.Dataset.<init>(Dataset.scala:174)
      at org.apache.spark.sql.Dataset$.apply(Dataset.scala:65)
      at org.apache.spark.sql.Dataset.withTypedPlan(Dataset.scala:3240)
      at org.apache.spark.sql.Dataset.filter(Dataset.scala:1403)
      ... 49 elided
    ```
    
    Looks like it is already failed at that time?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to