[jira] [Commented] (SPARK-2474) In some cases, the Analyzer failed to resolve a table registered in OverrideCatalog

[Yin Huai (JIRA)]

    [ 
https://issues.apache.org/jira/browse/SPARK-2474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060828#comment-14060828
 ] 

Yin Huai commented on SPARK-2474:
---------------------------------

I think the problem is the lookupRelation in OverrideCatalog. 

The current version is 
{code}
abstract override def lookupRelation(
    databaseName: Option[String],
    tableName: String,
    alias: Option[String] = None): LogicalPlan = {
    val (dbName, tblName) = processDatabaseAndTableName(databaseName, tableName)
    val overriddenTable = overrides.get((dbName, tblName))

    // If an alias was specified by the lookup, wrap the plan in a subquery so 
that attributes are
    // properly qualified with this alias.
    val withAlias =
      overriddenTable.map(r => alias.map(a => Subquery(a, r)).getOrElse(r))

    withAlias.getOrElse(super.lookupRelation(dbName, tblName, alias))
  }
{code}
You can notice that we do not insert a Subquery for the tableName (i.e. 
Subquery(tableName, logicalPlan)). Seems the SimpleCatalog.lookupRelation does 
not have this issue because we have 
{code}
val tableWithQualifiers = Subquery(tblName, table)
{code}


> In some cases, the Analyzer failed to resolve a table registered in 
> OverrideCatalog
> -----------------------------------------------------------------------------------
>
>                 Key: SPARK-2474
>                 URL: https://issues.apache.org/jira/browse/SPARK-2474
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.0.1
>            Reporter: Yin Huai
>
> To reproduce the error, execute the following code in hive/console...
> {code}
> val m = hql("select key from src")
> m.registerAsTable("m")
> hql("select m.key from m")
> {code}
> Then, you will see
> {code}
> org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Unresolved 
> attributes: 'm.key, tree:
> Project ['m.key]
>  LowerCaseSchema 
>   Project [key#6]
>    LowerCaseSchema 
>     MetastoreRelation default, src, None
>       at 
> org.apache.spark.sql.catalyst.analysis.Analyzer$CheckResolution$$anonfun$apply$1.applyOrElse(Analyzer.scala:71)
>       at 
> org.apache.spark.sql.catalyst.analysis.Analyzer$CheckResolution$$anonfun$apply$1.applyOrElse(Analyzer.scala:69)
>       at 
> org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:165)
>       at 
> org.apache.spark.sql.catalyst.trees.TreeNode.transform(TreeNode.scala:156)
> ...
> {code}
> However, if you run
> {code}
> hql("select tmp.key from m tmp")
> {code}
> We are fine.
> {code}
> SchemaRDD[3] at RDD at SchemaRDD.scala:104
> == Query Plan ==
> HiveTableScan [key#8], (MetastoreRelation default, src, None), None
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)