[
https://issues.apache.org/jira/browse/HIVE-20165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547688#comment-16547688
]
Hive QA commented on HIVE-20165:
--------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 10s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-12667/patches/PreCommit-HIVE-Build-12667.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12667/yetus.txt |
| Powered by | Apache Yetus http://yetus.apache.org |
This message was automatically generated.
> Enable ZLIB for streaming ingest
> --------------------------------
>
> Key: HIVE-20165
> URL: https://issues.apache.org/jira/browse/HIVE-20165
> Project: Hive
> Issue Type: Bug
> Components: Streaming, Transactions
> Affects Versions: 4.0.0, 3.2.0
> Reporter: Prasanth Jayachandran
> Assignee: Prasanth Jayachandran
> Priority: Major
> Attachments: HIVE-20165.1.branch-3.patch, HIVE-20165.1.patch,
> HIVE-20165.2.patch, HIVE-20165.3.patch
>
>
> Per [~gopalv]'s recommendation tried running streaming ingest with and
> without zlib. Following are the numbers
>
> *Compression: NONE*
> Total rows committed: 93800000
> Throughput: *1563333* rows/second
> $ hdfs dfs -du -s -h /apps/hive/warehouse/prasanth.db/culvert
> *14.1 G* /apps/hive/warehouse/prasanth.db/culvert
>
> *Compression: ZLIB*
> Total rows committed: 92100000
> Throughput: *1535000* rows/second
> $ hdfs dfs -du -s -h /apps/hive/warehouse/prasanth.db/culvert
> *7.4 G* /apps/hive/warehouse/prasanth.db/culvert
>
> ZLIB is getting us 2x compression and only 2% lesser throughput. We should
> enable ZLIB by default for streaming ingest.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
