[jira] [Updated] (HIVE-7220) Empty dir in external table causes issue (root_dir_external_table.q failure)
[
https://issues.apache.org/jira/browse/HIVE-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-7220:
Resolution: Fixed
Fix Version/s: 0.14.0
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks Szehon!
Empty dir in external table causes issue (root_dir_external_table.q failure)
Key: HIVE-7220
URL: https://issues.apache.org/jira/browse/HIVE-7220
Project: Hive
Issue Type: Bug
Reporter: Szehon Ho
Assignee: Szehon Ho
Fix For: 0.14.0
Attachments: HIVE-7220.2.patch, HIVE-7220.3.patch, HIVE-7220.4.patch,
HIVE-7220.5.patch, HIVE-7220.5.patch, HIVE-7220.patch
While looking at root_dir_external_table.q failure, which is doing a query on
an external table located at root ('/'), I noticed that latest Hadoop2
CombineFileInputFormat returns split representing empty directories (like
'/Users'), which leads to failure in Hive's CombineFileRecordReader as it
tries to open the directory for processing.
Tried with an external table in a normal HDFS directory, and it also returns
the same error. Looks like a real bug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7220) Empty dir in external table causes issue (root_dir_external_table.q failure)
[
https://issues.apache.org/jira/browse/HIVE-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho updated HIVE-7220:
Attachment: HIVE-7220.5.patch
I couldn't reproduce the dynpart_sort_optimization failure with this patch. As
the logs are gone, attaching again to see if I can get some idea.
Empty dir in external table causes issue (root_dir_external_table.q failure)
Key: HIVE-7220
URL: https://issues.apache.org/jira/browse/HIVE-7220
Project: Hive
Issue Type: Bug
Reporter: Szehon Ho
Assignee: Szehon Ho
Attachments: HIVE-7220.2.patch, HIVE-7220.3.patch, HIVE-7220.4.patch,
HIVE-7220.5.patch, HIVE-7220.5.patch, HIVE-7220.patch
While looking at root_dir_external_table.q failure, which is doing a query on
an external table located at root ('/'), I noticed that latest Hadoop2
CombineFileInputFormat returns split representing empty directories (like
'/Users'), which leads to failure in Hive's CombineFileRecordReader as it
tries to open the directory for processing.
Tried with an external table in a normal HDFS directory, and it also returns
the same error. Looks like a real bug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7220) Empty dir in external table causes issue (root_dir_external_table.q failure)
[
https://issues.apache.org/jira/browse/HIVE-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho updated HIVE-7220:
Attachment: HIVE-7220.5.patch
Addressed comments
Empty dir in external table causes issue (root_dir_external_table.q failure)
Key: HIVE-7220
URL: https://issues.apache.org/jira/browse/HIVE-7220
Project: Hive
Issue Type: Bug
Reporter: Szehon Ho
Assignee: Szehon Ho
Attachments: HIVE-7220.2.patch, HIVE-7220.3.patch, HIVE-7220.4.patch,
HIVE-7220.5.patch, HIVE-7220.patch
While looking at root_dir_external_table.q failure, which is doing a query on
an external table located at root ('/'), I noticed that latest Hadoop2
CombineFileInputFormat returns split representing empty directories (like
'/Users'), which leads to failure in Hive's CombineFileRecordReader as it
tries to open the directory for processing.
Tried with an external table in a normal HDFS directory, and it also returns
the same error. Looks like a real bug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7220) Empty dir in external table causes issue (root_dir_external_table.q failure)
[
https://issues.apache.org/jira/browse/HIVE-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gunther Hagleitner updated HIVE-7220:
-
Attachment: HIVE-7220.3.patch
Rebased. Only change was to move the new q file to the now separate file used
to configure mini mr files.
Empty dir in external table causes issue (root_dir_external_table.q failure)
Key: HIVE-7220
URL: https://issues.apache.org/jira/browse/HIVE-7220
Project: Hive
Issue Type: Bug
Reporter: Szehon Ho
Assignee: Szehon Ho
Attachments: HIVE-7220.2.patch, HIVE-7220.3.patch, HIVE-7220.patch
While looking at root_dir_external_table.q failure, which is doing a query on
an external table located at root ('/'), I noticed that latest Hadoop2
CombineFileInputFormat returns split representing empty directories (like
'/Users'), which leads to failure in Hive's CombineFileRecordReader as it
tries to open the directory for processing.
Tried with an external table in a normal HDFS directory, and it also returns
the same error. Looks like a real bug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7220) Empty dir in external table causes issue (root_dir_external_table.q failure)
[
https://issues.apache.org/jira/browse/HIVE-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gunther Hagleitner updated HIVE-7220:
-
Attachment: HIVE-7220.4.patch
Oops. Missed to include the new files. .4 has those.
Empty dir in external table causes issue (root_dir_external_table.q failure)
Key: HIVE-7220
URL: https://issues.apache.org/jira/browse/HIVE-7220
Project: Hive
Issue Type: Bug
Reporter: Szehon Ho
Assignee: Szehon Ho
Attachments: HIVE-7220.2.patch, HIVE-7220.3.patch, HIVE-7220.4.patch,
HIVE-7220.patch
While looking at root_dir_external_table.q failure, which is doing a query on
an external table located at root ('/'), I noticed that latest Hadoop2
CombineFileInputFormat returns split representing empty directories (like
'/Users'), which leads to failure in Hive's CombineFileRecordReader as it
tries to open the directory for processing.
Tried with an external table in a normal HDFS directory, and it also returns
the same error. Looks like a real bug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7220) Empty dir in external table causes issue (root_dir_external_table.q failure)
[
https://issues.apache.org/jira/browse/HIVE-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho updated HIVE-7220:
Status: Patch Available (was: Open)
Empty dir in external table causes issue (root_dir_external_table.q failure)
Key: HIVE-7220
URL: https://issues.apache.org/jira/browse/HIVE-7220
Project: Hive
Issue Type: Bug
Reporter: Szehon Ho
Assignee: Szehon Ho
Attachments: HIVE-7220.2.patch, HIVE-7220.patch
While looking at root_dir_external_table.q failure, which is doing a query on
an external table located at root ('/'), I noticed that latest Hadoop2
CombineFileInputFormat returns split representing empty directories (like
'/Users'), which leads to failure in Hive's CombineFileRecordReader as it
tries to open the directory for processing.
Tried with an external table in a normal HDFS directory, and it also returns
the same error. Looks like a real bug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7220) Empty dir in external table causes issue (root_dir_external_table.q failure)
[
https://issues.apache.org/jira/browse/HIVE-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho updated HIVE-7220:
Attachment: HIVE-7220.2.patch
I took another look at this, it was missing some things. The startOffset and
length arrays on the split correspond to the invalid paths, and must be pruned
as well if path is pruned. Giving it another try.
Empty dir in external table causes issue (root_dir_external_table.q failure)
Key: HIVE-7220
URL: https://issues.apache.org/jira/browse/HIVE-7220
Project: Hive
Issue Type: Bug
Reporter: Szehon Ho
Assignee: Szehon Ho
Attachments: HIVE-7220.2.patch, HIVE-7220.patch
While looking at root_dir_external_table.q failure, which is doing a query on
an external table located at root ('/'), I noticed that latest Hadoop2
CombineFileInputFormat returns split representing empty directories (like
'/Users'), which leads to failure in Hive's CombineFileRecordReader as it
tries to open the directory for processing.
Tried with an external table in a normal HDFS directory, and it also returns
the same error. Looks like a real bug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7220) Empty dir in external table causes issue (root_dir_external_table.q failure)
[
https://issues.apache.org/jira/browse/HIVE-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho updated HIVE-7220:
Status: Open (was: Patch Available)
Cancelling for now, unless there's interest in having a workaround in Hive. It
will not be necessary to pursue if MAPREDUCE-5756 is fixed.
Empty dir in external table causes issue (root_dir_external_table.q failure)
Key: HIVE-7220
URL: https://issues.apache.org/jira/browse/HIVE-7220
Project: Hive
Issue Type: Bug
Reporter: Szehon Ho
Assignee: Szehon Ho
Attachments: HIVE-7220.patch
While looking at root_dir_external_table.q failure, which is doing a query on
an external table located at root ('/'), I noticed that latest Hadoop2
CombineFileInputFormat returns split representing empty directories (like
'/Users'), which leads to failure in Hive's CombineFileRecordReader as it
tries to open the directory for processing.
Tried with an external table in a normal HDFS directory, and it also returns
the same error. Looks like a real bug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7220) Empty dir in external table causes issue (root_dir_external_table.q failure)
[
https://issues.apache.org/jira/browse/HIVE-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho updated HIVE-7220:
Attachment: HIVE-7220.patch
Adding some code to compensate for the directory being returned. Let's see if
it works
Empty dir in external table causes issue (root_dir_external_table.q failure)
Key: HIVE-7220
URL: https://issues.apache.org/jira/browse/HIVE-7220
Project: Hive
Issue Type: Bug
Reporter: Szehon Ho
Assignee: Szehon Ho
Attachments: HIVE-7220.patch
While looking at root_dir_external_table.q failure, which is doing a query on
an external table located at root ('/'), I noticed that latest Hadoop2
CombineFileInputFormat returns split representing empty directories (like
'/Users'), which leads to failure in Hive's CombineFileRecordReader as it
tries to open the directory for processing.
Tried with an external table in a normal HDFS directory, and it also returns
the same error. Looks like a real bug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7220) Empty dir in external table causes issue (root_dir_external_table.q failure)
[
https://issues.apache.org/jira/browse/HIVE-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho updated HIVE-7220:
Status: Patch Available (was: Open)
Empty dir in external table causes issue (root_dir_external_table.q failure)
Key: HIVE-7220
URL: https://issues.apache.org/jira/browse/HIVE-7220
Project: Hive
Issue Type: Bug
Reporter: Szehon Ho
Assignee: Szehon Ho
Attachments: HIVE-7220.patch
While looking at root_dir_external_table.q failure, which is doing a query on
an external table located at root ('/'), I noticed that latest Hadoop2
CombineFileInputFormat returns split representing empty directories (like
'/Users'), which leads to failure in Hive's CombineFileRecordReader as it
tries to open the directory for processing.
Tried with an external table in a normal HDFS directory, and it also returns
the same error. Looks like a real bug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
