Dimitris Tsirogiannis has posted comments on this change. ( http://gerrit.cloudera.org:8080/8971 )
Change subject: IMPALA-5519: Allocate fragment's runtime filter memory from Buffer pool ...................................................................... Patch Set 6: (6 comments) Quick question about testing. How do we ensure that the min/max bounds are respected? Do we have any tests for that? http://gerrit.cloudera.org:8080/#/c/8971/6/common/thrift/PlanNodes.thrift File common/thrift/PlanNodes.thrift: http://gerrit.cloudera.org:8080/#/c/8971/6/common/thrift/PlanNodes.thrift@150 PS6, Line 150: Should be non-zero for bloom filters So, for other than bloom filters is it not set or is it set with a zero value? http://gerrit.cloudera.org:8080/#/c/8971/6/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java File fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java: http://gerrit.cloudera.org:8080/#/c/8971/6/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java@75 PS6, Line 75: table "associated table columns"? http://gerrit.cloudera.org:8080/#/c/8971/6/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java@75 PS6, Line 75: are bound by : * {MIN,MAX}_BLOOM_FILTER_SIZE, query options and the max buffer size that can be : * allocated by the bufferpool Maybe say that they are computed based on ndvs, query options and max buffer size and are bound by MIN/MAX_BLOOM_FILTER_SIZE. My point is to stress that their size can never be smaller/larger than the MIN/MAX values. http://gerrit.cloudera.org:8080/#/c/8971/6/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java@177 PS6, Line 177: based on ndvEstimate_ and bounded by the max : // and min runtime filter size specified in the query options. Should be non-zero for : // bloom filters. remove, you already have the description in the class comment above. http://gerrit.cloudera.org:8080/#/c/8971/6/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java@608 PS6, Line 608: filter.boundFilterSize(defaultFilterSize, maxFilterSize, minFilterSize); I'd prefer if you pass the default, max and min sizes to the RuntimeFilter.create and then to the constructor. Since these three are always used together you may create a tiny class to hold them and pass around. http://gerrit.cloudera.org:8080/#/c/8971/6/testdata/workloads/functional-planner/queries/PlannerTest/disable-codegen.test File testdata/workloads/functional-planner/queries/PlannerTest/disable-codegen.test: http://gerrit.cloudera.org:8080/#/c/8971/6/testdata/workloads/functional-planner/queries/PlannerTest/disable-codegen.test@59 PS6, Line 59: 174.39 Hm, why did these change? -- To view, visit http://gerrit.cloudera.org:8080/8971 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iea2759665fb2e8bef9433014a8d42a7ebf99ce1f Gerrit-Change-Number: 8971 Gerrit-PatchSet: 6 Gerrit-Owner: Bikramjeet Vig <[email protected]> Gerrit-Reviewer: Bikramjeet Vig <[email protected]> Gerrit-Reviewer: Dimitris Tsirogiannis <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Tue, 30 Jan 2018 19:02:33 +0000 Gerrit-HasComments: Yes
