[jira] [Commented] (SPARK-11777) HiveContext.saveAsTable does not update the metastore on overwrite

2015-11-19 Thread Stanislav Hadjiiski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013130#comment-15013130 ] Stanislav Hadjiiski commented on SPARK-11777: - long story short, yes, that is the case >

[jira] [Commented] (SPARK-11777) HiveContext.saveAsTable does not update the metastore on overwrite

2015-11-18 Thread Stanislav Hadjiiski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010907#comment-15010907 ] Stanislav Hadjiiski commented on SPARK-11777: - Spark updates the HDFS (which is directly used

[jira] [Commented] (SPARK-11777) HiveContext.saveAsTable does not update the metastore on overwrite

2015-11-18 Thread Stanislav Hadjiiski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010944#comment-15010944 ] Stanislav Hadjiiski commented on SPARK-11777: - I guess when HiveContext is used spark carries

[jira] [Commented] (SPARK-11777) HiveContext.saveAsTable does not update the metastore on overwrite

2015-11-18 Thread Stanislav Hadjiiski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010819#comment-15010819 ] Stanislav Hadjiiski commented on SPARK-11777: - I would like to refer you to the cloudera

[jira] [Commented] (SPARK-11777) HiveContext.saveAsTable does not update the metastore on overwrite

2015-11-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010831#comment-15010831 ] Sean Owen commented on SPARK-11777: --- I'm confused, because you say Spark did update the metastore. If

[jira] [Commented] (SPARK-11777) HiveContext.saveAsTable does not update the metastore on overwrite

2015-11-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010919#comment-15010919 ] Sean Owen commented on SPARK-11777: --- If there is no update to the metastore, what update can a REFRESH

[jira] [Commented] (SPARK-11777) HiveContext.saveAsTable does not update the metastore on overwrite

2015-11-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011133#comment-15011133 ] Sean Owen commented on SPARK-11777: --- This really could be my ignorance, but, refresh doesn't change the

[jira] [Commented] (SPARK-11777) HiveContext.saveAsTable does not update the metastore on overwrite

2015-11-18 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15012268#comment-15012268 ] Yin Huai commented on SPARK-11777: -- Let me see if I understand the problem. After overwrite, you need to

[jira] [Commented] (SPARK-11777) HiveContext.saveAsTable does not update the metastore on overwrite

2015-11-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15008608#comment-15008608 ] Sean Owen commented on SPARK-11777: --- Why do you say it's not updated or readable? you say it did

[jira] [Commented] (SPARK-11777) HiveContext.saveAsTable does not update the metastore on overwrite

2015-11-17 Thread Stanislav Hadjiiski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15008628#comment-15008628 ] Stanislav Hadjiiski commented on SPARK-11777: - Yes, it updates the HDFS, i.e. the new data is

[jira] [Commented] (SPARK-11777) HiveContext.saveAsTable does not update the metastore on overwrite

2015-11-17 Thread Stanislav Hadjiiski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15008588#comment-15008588 ] Stanislav Hadjiiski commented on SPARK-11777: - I was to change the HDFS contents manually I