Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19877 )
Change subject: IMPALA-12123 (part 2): Fix offsets for HDFS caching with subranges ...................................................................... IMPALA-12123 (part 2): Fix offsets for HDFS caching with subranges When using HDFS caching, subrange reads are copying memory from the HDFS cache buffer into the output buffer. When the HDFS cache buffer starts at an offset into the file, the subrange offsets need to be adjusted to point to the relative place within the cache buffer. This fixes the calculation to use the relative locations. Testing: - Modified the caching tests in disk-io-mgr-test to also test with reads at an offset - In a local minicluster, creating a table based on alltypes_tiny_pages.parquet, added HDFS caching for the table, ran the SQLs in parquet-page-index-alltypes-tiny-pages.test, and checked the HDFS caching reads in the profile. Change-Id: Icd0ccc677b090ef3a75047defb4d7bda4279dac6 Reviewed-on: http://gerrit.cloudera.org:8080/19877 Tested-by: Impala Public Jenkins <[email protected]> Reviewed-by: Joe McDonnell <[email protected]> --- M be/src/runtime/io/disk-io-mgr-test.cc M be/src/runtime/io/scan-range.cc 2 files changed, 58 insertions(+), 22 deletions(-) Approvals: Impala Public Jenkins: Verified Joe McDonnell: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/19877 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Icd0ccc677b090ef3a75047defb4d7bda4279dac6 Gerrit-Change-Number: 19877 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
