Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19807 )
Change subject: IMPALA-12091: Control scan parallelism by its processing cost ...................................................................... Patch Set 3: (4 comments) http://gerrit.cloudera.org:8080/#/c/19807/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19807/3//COMMIT_MSG@17 PS3, Line 17: 200 scan range just curious any reason to choose this number? Do we need to tune this number in future based on storage io throughput? http://gerrit.cloudera.org:8080/#/c/19807/3/be/src/exec/hdfs-scanner.h File be/src/exec/hdfs-scanner.h: http://gerrit.cloudera.org:8080/#/c/19807/3/be/src/exec/hdfs-scanner.h@388 PS3, Line 388: effective_dop nit: effective_dop() http://gerrit.cloudera.org:8080/#/c/19807/3/common/thrift/Query.thrift File common/thrift/Query.thrift: http://gerrit.cloudera.org:8080/#/c/19807/3/common/thrift/Query.thrift@632 PS3, Line 632: 157: optional i32 processing_cost_max_threads = 128; > Can we consolidate there 128 constants somewhere? Agree. The ranges of PROCESSING_COST_MAX_THREADS and PROCESSING_COST_MIN_THREADS are also bound to 128. We should define a constant 128 so later we can increase it easily. http://gerrit.cloudera.org:8080/#/c/19807/3/fe/src/main/java/org/apache/impala/analysis/Analyzer.java File fe/src/main/java/org/apache/impala/analysis/Analyzer.java: http://gerrit.cloudera.org:8080/#/c/19807/3/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@658 PS3, Line 658: = Is 0 valid value? -- To view, visit http://gerrit.cloudera.org:8080/19807 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If948e45455275d9a61a6cd5d6a30a8b98a7c729a Gerrit-Change-Number: 19807 Gerrit-PatchSet: 3 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]> Gerrit-Comment-Date: Wed, 03 May 2023 19:54:35 +0000 Gerrit-HasComments: Yes
