[
https://issues.apache.org/jira/browse/ORC-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yongzhi.shao updated ORC-2224:
------------------------------
Description:
Summary:
Vectored read path ignores orc.min.disk.seek.size and its tolerance, and pays
per-range async overhead; coalesce ranges, compact by tolerance, and read
single-range batches synchronously
Type: Bug
Component: Java (core reader)
Affects Version: 2.1.x, 3.0.0-SNAPSHOT (only when vectored IO is active: Hadoop
>= 3.3.5, zero-copy disabled)
Description:
h3. Problem
When the vectored read path (\{{RecordReaderUtils.readDiskRangesVectored}}) is
active:
# \{{orc.min.disk.seek.size}} is silently ignored. The sync path merges disk
ranges whose gap is within \{{minSeekSize}} (\{{findSingleRead}}), but the
vectored path builds one \{{FileRange}} per \{{BufferChunk}} without any
merging. As a result, \{{TestMinSeekSize.readAlternateColumnsWMinSeekSize}}
*fails on Hadoop >= 3.3.5* (reads 5.26% of the file instead of ~106%) and any
user-configured \{{orc.min.disk.seek.size}} has no effect.
# \{{orc.min.disk.seek.size.tolerance}} is silently ignored. The sync path
compacts merged reads whose extra-bytes fraction exceeds the tolerance into a
tightly-sized buffer (\{{ChunkReader.populateChunksReduceSize}}); the vectored
path keeps the over-read gap bytes resident in the range buffers for the
lifetime of the chunks.
# A batch that consists of a single range is still submitted to
\{{readVectored}}, which adds executor task + \{{CompletableFuture}} overhead
with no parallelism benefit — the caller blocks on it immediately.
h3. Observed impact
Local benchmark: 2M rows, 1954 stripes of ~1k rows each, ZLIB and NONE, warm
page cache, Hadoop 3.4.1, median of 15 iterations, ORC 2.1.2 before → after:
||Mode||ZLIB before||ZLIB after||NONE before||NONE after|
|full scan|216.9 ms|187.8 ms|97.3 ms|73.3 ms|
|projection|129.7 ms|105.4 ms|51.7 ms|32.9 ms|
|filter (SARG)|51.3 ms|28.2 ms|43.9 ms|21.5 ms|
|projection + filter|34.1 ms|16.8 ms|32.8 ms|14.9 ms|
Read bytes and row counts are identical in every cell. After the fix the
vectored path matches the non-vectored sync path (and ORC 1.5.3) on this
workload; the regression introduced by routing all reads through
\{{readVectored}} is eliminated. With default configuration
(\{{minSeekSize=0}}, \{{minSeekSizeTolerance=0.0}}) the same bytes are read as
before.
h3. Proposed changes
All inside \{{readDiskRangesVectored}}; the sync path is untouched:
# *Coalesce contiguous/overlapping ranges* within the same batch into one
\{{FileRange}} (gap <= 0 merge).
# *Honor \{{minSeekSize}}*: additionally merge ranges whose gap is within
\{{minSeekSize}}, mirroring the sync path's \{{findSingleRead}} semantics. This
fixes the behavioral divergence (and the \{{TestMinSeekSize}} failure).
# *Honor \{{minSeekSizeTolerance}}*: after the read, compact each merged range
whose extra-bytes fraction exceeds the tolerance into a tightly-sized buffer,
mirroring the sync path's \{{populateChunksReduceSize}}. Like the sync path,
this is a memory bound only — it does not change how many bytes are read from
disk.
# *Single-range short-circuit*: when coalescing yields exactly one range, read
it with a synchronous positioned \{{readFully}} instead of \{{readVectored}}.
Invariants preserved:
* already-filled chunks (\{{hasData()}}) are never crossed;
* merged spans are capped at \{{MAX_FILE_RANGE_LENGTH}} with \{{long}}
arithmetic (no \{{int}} wrap);
* single forward pass O(n); chunk backfill is O(chunks + ranges);
* with \{{minSeekSize=0}} and \{{minSeekSizeTolerance=0.0}} behavior is
identical to today.
h3. Tests
* \{{TestMinSeekSize}} now passes with Hadoop 3.4.1 (previously failing on the
vectored path; passes on Hadoop 3.3.4 where vectored IO is disabled —
confirming the vectored path as the cause).
* 18 new unit tests in \{{TestRecordReaderUtils}} covering: gap-0 merge, gap <=
minSeekSize merge, gap > minSeekSize stays split, hasData boundary,
MAX_FILE_RANGE_LENGTH cap, negative minSeekSize rejection, single-range sync
short-circuit, minSeekSize=0 equivalence, and tolerance-driven compaction
(above/within tolerance, single-range short-circuit path, tolerance=0.0
backward compatibility).
* \{{TestRecordReaderImpl.testHadoopVectoredIO}} updated (a full scan now
coalesces to a single range and reads synchronously; the test uses a first+last
column projection to keep exercising \{{readVectored}}), plus a new test
asserting single-range batches do not call \{{readVectored}}.
* Full \{{java/core}} suite: 1562 tests, 0 failures, 0 errors (21 skipped) on
main.
h3. Notes
* Discovered while investigating a 2.1.2 vs 1.5.3 reader slowdown on files with
many small stripes.
* Also fixes a main-line compilation break in the original patch: it referenced
\{{IOUtils.MAX_ARRAY_SIZE}}, a constant moved into \{{RecordReaderUtils}} by
ORC-2049; the final patch references \{{RecordReaderUtils.MAX_ARRAY_SIZE}}
directly.
* Out of scope (unchanged): footer reads, cross-stripe batching, zero-copy path.
was:
Summary:
Vectored read path ignores orc.min.disk.seek.size and pays per-range async
overhead; coalesce ranges and read single-range batches synchronously
Type: Bug
Component: Java (core reader)
Affects Version: 2.1.x, 3.0.0-SNAPSHOT (only when vectored IO is active: Hadoop
>= 3.3.5, zero-copy disabled)
Description:
h3. Problem
When the vectored read path ({{{}RecordReaderUtils.readDiskRangesVectored{}}})
is active:
# {{orc.min.disk.seek.size}} is silently ignored. The sync path merges disk
ranges whose gap is within {{minSeekSize}} ({{{}findSingleRead{}}}), but the
vectored path builds one {{FileRange}} per {{BufferChunk}} without any merging.
As a result, {{TestMinSeekSize.readAlternateColumnsWMinSeekSize}} *fails on
Hadoop >= 3.3.5* (reads 5.26% of the file instead of ~106%) and any
user-configured {{orc.min.disk.seek.size}} has no effect.
# Ranges that are contiguous or overlapping within the same batch are still
submitted as separate \{{FileRange}}s, paying one async task + future per range.
# A batch that consists of a single range is still submitted to
{{{}readVectored{}}}, which adds executor task + {{CompletableFuture}} overhead
with no parallelism benefit — the caller blocks on it immediately.
h3. Observed impact
Local benchmark: 2M rows, 1954 stripes of ~1k rows each, ZLIB and NONE, warm
page cache, Hadoop 3.4.1, median of 15 iterations, ORC 2.1.2 before → after:
||Mode||ZLIB before||ZLIB after||NONE before||NONE after||
|full scan|216.9 ms|187.8 ms|97.3 ms|73.3 ms|
|projection|129.7 ms|105.4 ms|51.7 ms|32.9 ms|
|filter (SARG)|51.3 ms|28.2 ms|43.9 ms|21.5 ms|
|projection + filter|34.1 ms|16.8 ms|32.8 ms|14.9 ms|
PR: [ORC-2224:Coalesce vectored read ranges honoring minSeekSize and read
single-range batches synchronously|https://github.com/apache/orc/pull/2718]
> [Java] Vectored read coalescing honors minSeekSize; single-range batches read
> synchronously
> -------------------------------------------------------------------------------------------
>
> Key: ORC-2224
> URL: https://issues.apache.org/jira/browse/ORC-2224
> Project: ORC
> Issue Type: Improvement
> Reporter: yongzhi.shao
> Priority: Major
>
> Summary:
> Vectored read path ignores orc.min.disk.seek.size and its tolerance, and pays
> per-range async overhead; coalesce ranges, compact by tolerance, and read
> single-range batches synchronously
> Type: Bug
> Component: Java (core reader)
> Affects Version: 2.1.x, 3.0.0-SNAPSHOT (only when vectored IO is active:
> Hadoop >= 3.3.5, zero-copy disabled)
> Description:
> h3. Problem
> When the vectored read path (\{{RecordReaderUtils.readDiskRangesVectored}})
> is active:
> # \{{orc.min.disk.seek.size}} is silently ignored. The sync path merges disk
> ranges whose gap is within \{{minSeekSize}} (\{{findSingleRead}}), but the
> vectored path builds one \{{FileRange}} per \{{BufferChunk}} without any
> merging. As a result, \{{TestMinSeekSize.readAlternateColumnsWMinSeekSize}}
> *fails on Hadoop >= 3.3.5* (reads 5.26% of the file instead of ~106%) and any
> user-configured \{{orc.min.disk.seek.size}} has no effect.
> # \{{orc.min.disk.seek.size.tolerance}} is silently ignored. The sync path
> compacts merged reads whose extra-bytes fraction exceeds the tolerance into a
> tightly-sized buffer (\{{ChunkReader.populateChunksReduceSize}}); the
> vectored path keeps the over-read gap bytes resident in the range buffers for
> the lifetime of the chunks.
> # A batch that consists of a single range is still submitted to
> \{{readVectored}}, which adds executor task + \{{CompletableFuture}} overhead
> with no parallelism benefit — the caller blocks on it immediately.
> h3. Observed impact
> Local benchmark: 2M rows, 1954 stripes of ~1k rows each, ZLIB and NONE, warm
> page cache, Hadoop 3.4.1, median of 15 iterations, ORC 2.1.2 before → after:
> ||Mode||ZLIB before||ZLIB after||NONE before||NONE after|
> |full scan|216.9 ms|187.8 ms|97.3 ms|73.3 ms|
> |projection|129.7 ms|105.4 ms|51.7 ms|32.9 ms|
> |filter (SARG)|51.3 ms|28.2 ms|43.9 ms|21.5 ms|
> |projection + filter|34.1 ms|16.8 ms|32.8 ms|14.9 ms|
> Read bytes and row counts are identical in every cell. After the fix the
> vectored path matches the non-vectored sync path (and ORC 1.5.3) on this
> workload; the regression introduced by routing all reads through
> \{{readVectored}} is eliminated. With default configuration
> (\{{minSeekSize=0}}, \{{minSeekSizeTolerance=0.0}}) the same bytes are read
> as before.
> h3. Proposed changes
> All inside \{{readDiskRangesVectored}}; the sync path is untouched:
> # *Coalesce contiguous/overlapping ranges* within the same batch into one
> \{{FileRange}} (gap <= 0 merge).
> # *Honor \{{minSeekSize}}*: additionally merge ranges whose gap is within
> \{{minSeekSize}}, mirroring the sync path's \{{findSingleRead}} semantics.
> This fixes the behavioral divergence (and the \{{TestMinSeekSize}} failure).
> # *Honor \{{minSeekSizeTolerance}}*: after the read, compact each merged
> range whose extra-bytes fraction exceeds the tolerance into a tightly-sized
> buffer, mirroring the sync path's \{{populateChunksReduceSize}}. Like the
> sync path, this is a memory bound only — it does not change how many bytes
> are read from disk.
> # *Single-range short-circuit*: when coalescing yields exactly one range,
> read it with a synchronous positioned \{{readFully}} instead of
> \{{readVectored}}.
> Invariants preserved:
> * already-filled chunks (\{{hasData()}}) are never crossed;
> * merged spans are capped at \{{MAX_FILE_RANGE_LENGTH}} with \{{long}}
> arithmetic (no \{{int}} wrap);
> * single forward pass O(n); chunk backfill is O(chunks + ranges);
> * with \{{minSeekSize=0}} and \{{minSeekSizeTolerance=0.0}} behavior is
> identical to today.
> h3. Tests
> * \{{TestMinSeekSize}} now passes with Hadoop 3.4.1 (previously failing on
> the vectored path; passes on Hadoop 3.3.4 where vectored IO is disabled —
> confirming the vectored path as the cause).
> * 18 new unit tests in \{{TestRecordReaderUtils}} covering: gap-0 merge, gap
> <= minSeekSize merge, gap > minSeekSize stays split, hasData boundary,
> MAX_FILE_RANGE_LENGTH cap, negative minSeekSize rejection, single-range sync
> short-circuit, minSeekSize=0 equivalence, and tolerance-driven compaction
> (above/within tolerance, single-range short-circuit path, tolerance=0.0
> backward compatibility).
> * \{{TestRecordReaderImpl.testHadoopVectoredIO}} updated (a full scan now
> coalesces to a single range and reads synchronously; the test uses a
> first+last column projection to keep exercising \{{readVectored}}), plus a
> new test asserting single-range batches do not call \{{readVectored}}.
> * Full \{{java/core}} suite: 1562 tests, 0 failures, 0 errors (21 skipped) on
> main.
> h3. Notes
> * Discovered while investigating a 2.1.2 vs 1.5.3 reader slowdown on files
> with many small stripes.
> * Also fixes a main-line compilation break in the original patch: it
> referenced \{{IOUtils.MAX_ARRAY_SIZE}}, a constant moved into
> \{{RecordReaderUtils}} by ORC-2049; the final patch references
> \{{RecordReaderUtils.MAX_ARRAY_SIZE}} directly.
> * Out of scope (unchanged): footer reads, cross-stripe batching, zero-copy
> path.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
