[
https://issues.apache.org/jira/browse/HIVE-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joydeep Sen Sarma updated HIVE-85:
----------------------------------
Status: Patch Available (was: Open)
two new options are provided: hive.exec.compress.output and
hive.exec.compress.intermediate. documentation is included in
conf/hive-default.xml
patch includes some testing related changes as well (which i found necessary
for this stuff):
- update to QTestUtil to only overwrite files on -Doverwrite=true iff the files
actually differ
- update to SemanticAnalyzer to display boolean fields in explain plan. this is
causing some additional items to show up in explain plan outputs that were not
previously.
> separate compression options for different output types
> -------------------------------------------------------
>
> Key: HIVE-85
> URL: https://issues.apache.org/jira/browse/HIVE-85
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Joydeep Sen Sarma
> Assignee: Joydeep Sen Sarma
> Attachments: hive-85.patch.1, hive-85.patch.2
>
>
> currently hive uses mapred.output.compress to determine compression for all
> output files. however not all files are final output. at least three
> different kinds of output files are generated:
> 1. intermediate output files for next map-reduce job
> 2. files targeted for result hdfs directories or hive tables/partitions
> (which are just hdfs dirs)
> 3. files written to user local directories (downloading results)
> the plan is to provide three separate options for controlling 1,2,3
> separately. we may want to split (2) in case compression is determined by
> table metadata (and not session options).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
