Are you proposing dropping SIZED from the spliterator for arrays? This
would undermine all the array-based optimizations (e.g., toArray), which
seems a bad trade. I realize the splitting heuristics are frustrating
for a number of use cases, but this seems like throwing the baby out
with the bathwater.
(Loom is coming, and that is a good time to revisit streams support for
blocking operations, which is a big part of what people complain about
with parallel streams.)
On 1/29/2022 11:38 AM, Tagir F.Valeev wrote:
See the bug description for details.
I propose a simple solution. Let's allow ArraySpliterator to be non-SIZED and
report artificial estimatedSize(), much bigger than the real one. This will allow
AbstractSpliterator and IteratorSpliterator to produce prefix whose size is
comparable to Long.MAX_VALUE (say, starting with Long.MAX_VALUE/2), and this will
enable further splitting of the prefix. This change will drastically improve
parallel streaming for affected streams of size <= 1024 and significantly
improve for streams of size 1025..20000. The cost is higher-grained splitting for
huge streams of unknown size. This might add a minor overhead for such scenarios
which, I believe, is completely tolerable.
No public API changes are necessary, sequential processing should not be
affected, except an extra field in ArraySpliterator which increases a footprint
by 8 bytes.
I added a simple test to ensure that at least two threads are actually used
when parallelizing Stream.iterate source. More testing ideas are welcome.
-------------
Commit messages:
- JDK-8280915 Better parallelization for AbstractSpliterator and
IteratorSpliterator when size is unknown
Changes:https://git.openjdk.java.net/jdk/pull/7279/files
Webrev:https://webrevs.openjdk.java.net/?repo=jdk&pr=7279&range=00
Issue:https://bugs.openjdk.java.net/browse/JDK-8280915
Stats: 128 lines in 2 files changed: 96 ins; 0 del; 32 mod
Patch:https://git.openjdk.java.net/jdk/pull/7279.diff
Fetch: git fetchhttps://git.openjdk.java.net/jdk pull/7279/head:pull/7279
PR:https://git.openjdk.java.net/jdk/pull/7279
