[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2017-03-07 Thread Lefty Leverenz (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15899218#comment-15899218 ] Lefty Leverenz commented on HIVE-14270: --- Thanks for documenting the configs, Sergio. *

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-08-12 Thread Lefty Leverenz (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418580#comment-15418580 ] Lefty Leverenz commented on HIVE-14270: --- bq. ... wiki section about S3 or blobstore tables? The

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-08-11 Thread JIRA
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417513#comment-15417513 ] Sergio Peña commented on HIVE-14270: [~leftylev] is there a wiki section about S3 or blobstore tables?

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-08-11 Thread Ashutosh Chauhan (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417459#comment-15417459 ] Ashutosh Chauhan commented on HIVE-14270: - +1 Thanks for your persistence on this one, Sergio!

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-08-11 Thread JIRA
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417406#comment-15417406 ] Sergio Peña commented on HIVE-14270: [~ashutoshc] All tests are passing now, The ones failing are not

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-08-11 Thread Lefty Leverenz (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416662#comment-15416662 ] Lefty Leverenz commented on HIVE-14270: --- Thanks for the fix, Sergio. +1 for the configuration

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-08-11 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416644#comment-15416644 ] Hive QA commented on HIVE-14270: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-08-10 Thread JIRA
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415405#comment-15415405 ] Sergio Peña commented on HIVE-14270: Thanks [~leftylev] to catch it. You have a good eye on these

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-08-09 Thread Lefty Leverenz (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414740#comment-15414740 ] Lefty Leverenz commented on HIVE-14270: --- I requested a punctuation change on RB for the

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-08-09 Thread JIRA
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414136#comment-15414136 ] Sergio Peña commented on HIVE-14270: [~ashutoshc] I attached a new paths (it is on RB too) that added

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-08-08 Thread Ashutosh Chauhan (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412170#comment-15412170 ] Ashutosh Chauhan commented on HIVE-14270: - I think its better to be consistent in code then to

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-08-08 Thread JIRA
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412096#comment-15412096 ] Sergio Peña commented on HIVE-14270: [~ashutoshc] I need a little advice here. The tests failing are

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-08-08 Thread JIRA
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412030#comment-15412030 ] Sergio Peña commented on HIVE-14270: Good point. I'll add a flag to enable S3 scratch directories for

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-08-05 Thread Ashutosh Chauhan (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409680#comment-15409680 ] Ashutosh Chauhan commented on HIVE-14270: - Seems like failures are related. > Write temporary

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-08-05 Thread Ashutosh Chauhan (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409688#comment-15409688 ] Ashutosh Chauhan commented on HIVE-14270: - There is also a use case where data on S3 is much

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-08-04 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408615#comment-15408615 ] Hive QA commented on HIVE-14270: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-07-30 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400779#comment-15400779 ] Hive QA commented on HIVE-14270: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-07-30 Thread Lefty Leverenz (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400557#comment-15400557 ] Lefty Leverenz commented on HIVE-14270: --- I made some editorial comments on RB for the config

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-07-28 Thread Ashutosh Chauhan (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397993#comment-15397993 ] Ashutosh Chauhan commented on HIVE-14270: - [~spena] It will be good to have schemes come from

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-07-28 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397859#comment-15397859 ] Chris Nauroth commented on HIVE-14270: -- Separating development of a richer Hive-on-S3 integration

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-07-27 Thread JIRA
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396375#comment-15396375 ] Sergio Peña commented on HIVE-14270: Another thing I just thought about having a configuration

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-07-27 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396090#comment-15396090 ] Chris Nauroth commented on HIVE-14270: -- Any approach that involves matching on scheme is going to be

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-07-27 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395413#comment-15395413 ] Steve Loughran commented on HIVE-14270: --- Hadoop's FS API is a plugin point: there are more object

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-07-22 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15390473#comment-15390473 ] Hive QA commented on HIVE-14270: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-07-22 Thread JIRA
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15390278#comment-15390278 ] Sergio Peña commented on HIVE-14270: Thanks guys for the testing recommendations. I'll see what I can

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-07-22 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15389975#comment-15389975 ] Steve Loughran commented on HIVE-14270: --- Regarding S3 testing, have a look at what hadoop-aws does,

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-07-22 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15389895#comment-15389895 ] Steve Loughran commented on HIVE-14270: --- Having a hard coded list of URI schemas to use here is

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-07-22 Thread Thomas Poepping (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15389818#comment-15389818 ] Thomas Poepping commented on HIVE-14270: Can you open a reviewboard submission for this patch? >

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-07-22 Thread Thomas Poepping (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15389817#comment-15389817 ] Thomas Poepping commented on HIVE-14270: The unit tests you want would verify that the scheme of

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-07-21 Thread JIRA
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388609#comment-15388609 ] Sergio Peña commented on HIVE-14270: Only requirement on this patch is the scratch directory is not on

[jira] [Commented] (HIVE-14270) Write temporary data to HDFS when doing inserts on tables located on S3

2016-07-21 Thread JIRA
[ https://issues.apache.org/jira/browse/HIVE-14270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388608#comment-15388608 ] Sergio Peña commented on HIVE-14270: I run several tests using just 1 or 2 rows to validate the