Abhishek Rawat has posted comments on this change. ( http://gerrit.cloudera.org:8080/17979 )
Change subject: IMPALA-10791 Add batch reading for remote temporary files ...................................................................... Patch Set 15: (9 comments) Looks good to me. Some initial comments. http://gerrit.cloudera.org:8080/#/c/17979/12//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17979/12//COMMIT_MSG@10 PS12, Line 10: file in order to improve the reading performance for the spilled nit: reading -> read http://gerrit.cloudera.org:8080/#/c/17979/12//COMMIT_MSG@13 PS12, Line 13: Originally, the design is to use the local disk file as the buffer nit: is -> was http://gerrit.cloudera.org:8080/#/c/17979/14/be/src/runtime/io/disk-file-test.cc File be/src/runtime/io/disk-file-test.cc: http://gerrit.cloudera.org:8080/#/c/17979/14/be/src/runtime/io/disk-file-test.cc@30 PS14, Line 30: MemBlockStatus new_status, bool expect_suc); 'expect_success' is probably better for reading. http://gerrit.cloudera.org:8080/#/c/17979/14/be/src/runtime/io/disk-file-test.cc@33 PS14, Line 33: // last_status is MemBlock's last status is going to reach other than missing "it" in the description. last_status is MemBlock's last status 'it' is going to reach other than http://gerrit.cloudera.org:8080/#/c/17979/14/be/src/runtime/io/disk-file.h File be/src/runtime/io/disk-file.h: http://gerrit.cloudera.org:8080/#/c/17979/14/be/src/runtime/io/disk-file.h@45 PS14, Line 45: /// UNINIT is the default status, indicates the block is not uninitialized. typo? not uninitialized. => not initialized. http://gerrit.cloudera.org:8080/#/c/17979/14/be/src/runtime/io/disk-file.h@72 PS14, Line 72: // Allcate the memory for the MemBlock. typo "Allcate" http://gerrit.cloudera.org:8080/#/c/17979/15/be/src/runtime/tmp-file-mgr.cc File be/src/runtime/tmp-file-mgr.cc: http://gerrit.cloudera.org:8080/#/c/17979/15/be/src/runtime/tmp-file-mgr.cc@604 PS15, Line 604: admit_mem_limit I believe this is the memory limit for query processing, which is what I would expect we use and not use the 20% emergency memory. But, then this contradicts the commit message. http://gerrit.cloudera.org:8080/#/c/17979/15/common/thrift/metrics.json File common/thrift/metrics.json: http://gerrit.cloudera.org:8080/#/c/17979/15/common/thrift/metrics.json@2424 PS15, Line 2424: The used read memory buffer of all scratch directories Maybe re-word this as follows to reflect this is current usage? "The current total read memory buffer bytes for all scratch directories" http://gerrit.cloudera.org:8080/#/c/17979/15/common/thrift/metrics.json@2434 PS15, Line 2434: The high water mark for read memory buffer of all scratch directories. Better wording to include the unit/bytes: The high water mark for read memory buffer bytes of all scratch directories. -- To view, visit http://gerrit.cloudera.org:8080/17979 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1dcc5d0881ffaeff09c5c514306cd668373ad31b Gerrit-Change-Number: 17979 Gerrit-PatchSet: 15 Gerrit-Owner: Yida Wu <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Qifan Chen <[email protected]> Gerrit-Reviewer: Yida Wu <[email protected]> Gerrit-Comment-Date: Tue, 09 Aug 2022 23:25:02 +0000 Gerrit-HasComments: Yes
