[jira] [Commented] (HIVE-2457) Files in Avro-backed Hive tables do not have a .avro extension
[ https://issues.apache.org/jira/browse/HIVE-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13115096#comment-13115096 ] Hudson commented on HIVE-2457: -- Integrated in Hive-trunk-h0.21 #978 (See [https://builds.apache.org/job/Hive-trunk-h0.21/978/]) HIVE-2457. Files in Avro-backed Hive tables do not have a .avro extension (Tom White via cws) cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1176010 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java * /hive/trunk/ql/src/test/queries/clientpositive/input44.q * /hive/trunk/ql/src/test/results/clientpositive/input44.q.out Files in Avro-backed Hive tables do not have a .avro extension Key: HIVE-2457 URL: https://issues.apache.org/jira/browse/HIVE-2457 Project: Hive Issue Type: Improvement Components: Query Processor, Serializers/Deserializers Reporter: Tom White Assignee: Tom White Fix For: 0.9.0 Attachments: HIVE-2457.patch, HIVE-2457.patch When using the Avro SerDe (see HIVE-895, https://github.com/jghoman/haivvreo) the files created for an Avro table do not have a .avro extension, which causes problems for tools like Avro MapReduce or Sqoop which expect the extension. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2457) Files in Avro-backed Hive tables do not have a .avro extension
[ https://issues.apache.org/jira/browse/HIVE-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13113579#comment-13113579 ] jirapos...@reviews.apache.org commented on HIVE-2457: - bq. On 2011-09-21 00:26:28, Carl Steinbach wrote: bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java, line 892 bq. https://reviews.apache.org/r/1989/diff/1/?file=2#file2line892 bq. bq. Please add this configuration property to HiveConf and hive-default.xml bq. bq. Tom White wrote: bq. Does Hive have the concept of private configuration properties? This is one that would set by SerDe's, not by users, which is why I didn't add it to HiveConf/hive-default.xml. bq. bq. Carl Steinbach wrote: bq. No, it doesn't, but it should. I'll file a JIRA. In the meantime this property should still be included in hive-default and HiveConf. OK, I added the property to hive-default and HiveConf in the latest patch. - Tom --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1989/#review1982 --- On 2011-09-20 22:28:53, Carl Steinbach wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/1989/ bq. --- bq. bq. (Updated 2011-09-20 22:28:53) bq. bq. bq. Review request for hive. bq. bq. bq. Summary bq. --- bq. bq. Review for HIVE-2457 bq. bq. bq. This addresses bug HIVE-2457. bq. https://issues.apache.org/jira/browse/HIVE-2457 bq. bq. bq. Diffs bq. - bq. bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java 1173340 bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1173340 bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java 1173340 bq.trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java PRE-CREATION bq. bq. Diff: https://reviews.apache.org/r/1989/diff bq. bq. bq. Testing bq. --- bq. bq. bq. Thanks, bq. bq. Carl bq. bq. Files in Avro-backed Hive tables do not have a .avro extension Key: HIVE-2457 URL: https://issues.apache.org/jira/browse/HIVE-2457 Project: Hive Issue Type: Improvement Components: Query Processor, Serializers/Deserializers Reporter: Tom White Assignee: Tom White Attachments: HIVE-2457.patch, HIVE-2457.patch When using the Avro SerDe (see HIVE-895, https://github.com/jghoman/haivvreo) the files created for an Avro table do not have a .avro extension, which causes problems for tools like Avro MapReduce or Sqoop which expect the extension. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2457) Files in Avro-backed Hive tables do not have a .avro extension
[ https://issues.apache.org/jira/browse/HIVE-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13113633#comment-13113633 ] Carl Steinbach commented on HIVE-2457: -- +1. Will commit if tests pass. Files in Avro-backed Hive tables do not have a .avro extension Key: HIVE-2457 URL: https://issues.apache.org/jira/browse/HIVE-2457 Project: Hive Issue Type: Improvement Components: Query Processor, Serializers/Deserializers Reporter: Tom White Assignee: Tom White Attachments: HIVE-2457.patch, HIVE-2457.patch When using the Avro SerDe (see HIVE-895, https://github.com/jghoman/haivvreo) the files created for an Avro table do not have a .avro extension, which causes problems for tools like Avro MapReduce or Sqoop which expect the extension. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2457) Files in Avro-backed Hive tables do not have a .avro extension
[ https://issues.apache.org/jira/browse/HIVE-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13112826#comment-13112826 ] jirapos...@reviews.apache.org commented on HIVE-2457: - bq. On 2011-09-21 00:26:28, Carl Steinbach wrote: bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java, line 892 bq. https://reviews.apache.org/r/1989/diff/1/?file=2#file2line892 bq. bq. Please add this configuration property to HiveConf and hive-default.xml bq. bq. Tom White wrote: bq. Does Hive have the concept of private configuration properties? This is one that would set by SerDe's, not by users, which is why I didn't add it to HiveConf/hive-default.xml. No, it doesn't, but it should. I'll file a JIRA. In the meantime this property should still be included in hive-default and HiveConf. - Carl --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1989/#review1982 --- On 2011-09-20 22:28:53, Carl Steinbach wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/1989/ bq. --- bq. bq. (Updated 2011-09-20 22:28:53) bq. bq. bq. Review request for hive. bq. bq. bq. Summary bq. --- bq. bq. Review for HIVE-2457 bq. bq. bq. This addresses bug HIVE-2457. bq. https://issues.apache.org/jira/browse/HIVE-2457 bq. bq. bq. Diffs bq. - bq. bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java 1173340 bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1173340 bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java 1173340 bq.trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java PRE-CREATION bq. bq. Diff: https://reviews.apache.org/r/1989/diff bq. bq. bq. Testing bq. --- bq. bq. bq. Thanks, bq. bq. Carl bq. bq. Files in Avro-backed Hive tables do not have a .avro extension Key: HIVE-2457 URL: https://issues.apache.org/jira/browse/HIVE-2457 Project: Hive Issue Type: Improvement Components: Query Processor, Serializers/Deserializers Reporter: Tom White Assignee: Tom White Attachments: HIVE-2457.patch, HIVE-2457.patch When using the Avro SerDe (see HIVE-895, https://github.com/jghoman/haivvreo) the files created for an Avro table do not have a .avro extension, which causes problems for tools like Avro MapReduce or Sqoop which expect the extension. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2457) Files in Avro-backed Hive tables do not have a .avro extension
[ https://issues.apache.org/jira/browse/HIVE-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13109036#comment-13109036 ] jirapos...@reviews.apache.org commented on HIVE-2457: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1989/ --- Review request for hive. Summary --- Review for HIVE-2457 This addresses bug HIVE-2457. https://issues.apache.org/jira/browse/HIVE-2457 Diffs - trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java 1173340 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1173340 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java 1173340 trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java PRE-CREATION Diff: https://reviews.apache.org/r/1989/diff Testing --- Thanks, Carl Files in Avro-backed Hive tables do not have a .avro extension Key: HIVE-2457 URL: https://issues.apache.org/jira/browse/HIVE-2457 Project: Hive Issue Type: Improvement Components: Query Processor, Serializers/Deserializers Reporter: Tom White Attachments: HIVE-2457.patch When using the Avro SerDe (see HIVE-895, https://github.com/jghoman/haivvreo) the files created for an Avro table do not have a .avro extension, which causes problems for tools like Avro MapReduce or Sqoop which expect the extension. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2457) Files in Avro-backed Hive tables do not have a .avro extension
[
https://issues.apache.org/jira/browse/HIVE-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13109130#comment-13109130
]
jirapos...@reviews.apache.org commented on HIVE-2457:
-
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1989/#review1982
---
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
https://reviews.apache.org/r/1989/#comment4468
Please add this configuration property to HiveConf and hive-default.xml
trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java
https://reviews.apache.org/r/1989/#comment4479
In addition to the unit test it would also be nice to test this via
TestCliDriver.
It should be possible to verify this from the CLI by doing something like
this:
-- Set the filename suffix property. Then create a new table and stream
-- data into it. Then use the dfs cat command to dump the contents of
-- the raw files in the warehouse to stdout
hive dfs -cat ${hiveconf:hive.metastore.warehouse.dir}/tablename/*.avro;
- Carl
On 2011-09-20 22:28:53, Carl Steinbach wrote:
bq.
bq. ---
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1989/
bq. ---
bq.
bq. (Updated 2011-09-20 22:28:53)
bq.
bq.
bq. Review request for hive.
bq.
bq.
bq. Summary
bq. ---
bq.
bq. Review for HIVE-2457
bq.
bq.
bq. This addresses bug HIVE-2457.
bq. https://issues.apache.org/jira/browse/HIVE-2457
bq.
bq.
bq. Diffs
bq. -
bq.
bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java
1173340
bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1173340
bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java
1173340
bq.trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java
PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/1989/diff
bq.
bq.
bq. Testing
bq. ---
bq.
bq.
bq. Thanks,
bq.
bq. Carl
bq.
bq.
Files in Avro-backed Hive tables do not have a .avro extension
Key: HIVE-2457
URL: https://issues.apache.org/jira/browse/HIVE-2457
Project: Hive
Issue Type: Improvement
Components: Query Processor, Serializers/Deserializers
Reporter: Tom White
Assignee: Tom White
Attachments: HIVE-2457.patch
When using the Avro SerDe (see HIVE-895, https://github.com/jghoman/haivvreo)
the files created for an Avro table do not have a .avro extension, which
causes problems for tools like Avro MapReduce or Sqoop which expect the
extension.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
