Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/2570#discussion_r18279773
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateTableAsSelect.scala
---
@@ -56,7 +71,10 @@ case class CreateTableAsSelect(
}
override protected[sql] lazy val sideEffectResult: Seq[Row] = {
- insertIntoRelation(metastoreRelation).execute
+ // TODO ideally, we should get the output data ready first and then
+ // update the relation, just in case of failure occurs in data
+ // processing. Otherwise we may not able to get a consistent.
--- End diff --
If we populate metastore after evaluating the query, we also need to make
sure information stored in `CreateTableDesc` will be correctly set to
`tableInfo` in the `FileSinkDesc`. Also, if `TableDesc
org.apache.hadoop.hive.ql.plan.PlanUtils.getTableDesc(CreateTableDesc, String,
String)` will be used to create the `tableInfo`, the implementation of this
method in hive 0.12 cannot be used because of the bug mentioned in
https://issues.apache.org/jira/browse/HIVE-6083. Can you add a note at here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
