Stanislav Hadjiiski created SPARK-11777: -------------------------------------------
Summary: HiveContext.saveAsTable does not update the metastore on overwrite Key: SPARK-11777 URL: https://issues.apache.org/jira/browse/SPARK-11777 Project: Spark Issue Type: Bug Affects Versions: 1.5.1 Reporter: Stanislav Hadjiiski Consider the following code: {quote} case class Bean(cdata: String) val hiveContext = new org.apache.spark.sql.hive.HiveContext(sparkContext) val df = hiveContext.createDataFrame(Bean("test10") :: Bean("test20") :: Nil) df.write.mode(SaveMode.Overwrite).saveAsTable("db_name.table") {quote} This works as expected - if the table does not exist it is created, otherwise it's content is replaced. However, only in the first case the data is accessible through impala (i.e. outside of spark environment). To get it working after overwriting a {quote} REFRESH db_name.table {quote} should be issued in impala-shell. Neither {quote} hiveContext.refreshTable("db_name.table") {quote} nor {quote} hiveContext.sql("REFRESH TABLE db_name.table") {quote} fixes the issue. The same applies if the {{default}} database is used (and {{db_name.}} is omiited everywhere) -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org