Xiaomeng Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15304 )
Change subject: IMPALA-9389: [DOCS] Support reading zstd text files ...................................................................... Patch Set 4: (2 comments) http://gerrit.cloudera.org:8080/#/c/15304/4/docs/topics/impala_txtfile.xml File docs/topics/impala_txtfile.xml: http://gerrit.cloudera.org:8080/#/c/15304/4/docs/topics/impala_txtfile.xml@633 PS4, Line 633: <title>Using bzip2, gzip, Snappy-Compressed, or zstd Text Files</title> I saw the other code review https://gerrit.cloudera.org/c/15310/, do we need to add deflate here as well? http://gerrit.cloudera.org:8080/#/c/15304/4/docs/topics/impala_txtfile.xml@653 PS4, Line 653: or zstd-compressed text file is processed, the node doing the : work reads the entire file into memory and then decompresses it. Therefore, the node must : have enough memory to hold both the compressed and uncompressed data from the text file For text zstd decompression, we're using streaming, which doesn't load all at once. It decompress as it read. To be notice is that this is not true for parquet, we're still using block decompression for parquet file. -- To view, visit http://gerrit.cloudera.org:8080/15304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic83137bd2c3a49398fb60cf1901f8b74ed111fce Gerrit-Change-Number: 15304 Gerrit-PatchSet: 4 Gerrit-Owner: Anonymous Coward <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Andrew Sherman <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Xiaomeng Zhang <[email protected]> Gerrit-Comment-Date: Fri, 28 Feb 2020 00:00:25 +0000 Gerrit-HasComments: Yes
