Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/18126 )
Change subject: IMPALA-11068: Add tuning flag to reduce scanner thread launch. ...................................................................... Patch Set 1: > Patch Set 1: > > Should we look into making the constant COMPRESSED_TEXT_COMPRESSION_RATIO > tunable in hdfs-scan-node.cc instead? COMPRESSED_TEXT_COMPRESSION_RATIO is only used when reading file with text format (THdfsFileFormat::TEXT). For non-text format, it will use hdfs_scanner_thread_max_estimated_bytes which default to 32MB. Customer hit OOM over parquet file format. They have try increase hdfs_scanner_thread_max_estimated_bytes higher, but that leads to performance regression for other queries. -- To view, visit http://gerrit.cloudera.org:8080/18126 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I03cadf1230eed00d69f2890c82476c6861e37466 Gerrit-Change-Number: 18126 Gerrit-PatchSet: 1 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Bikramjeet Vig <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Comment-Date: Sat, 08 Jan 2022 03:48:39 +0000 Gerrit-HasComments: No
