[
https://issues.apache.org/jira/browse/SPARK-36900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-36900:
-----------------------------
Description:
Execute
{code:java}
build/mvn clean install -pl core -am -Dtest=none
-DwildcardSuites=org.apache.spark.util.io.ChunkedByteBufferOutputStreamSuite
{code}
with JDK 17,
{code:java}
ChunkedByteBufferOutputStreamSuite:
- empty output
- write a single byte
- write a single near boundary
- write a single at boundary
- single chunk output
- single chunk output at boundary size
- multiple chunk output
- multiple chunk output at boundary size
*** RUN ABORTED ***
java.lang.OutOfMemoryError: Java heap space
at java.base/java.lang.Integer.valueOf(Integer.java:1081)
at scala.runtime.BoxesRunTime.boxToInteger(BoxesRunTime.java:67)
at
org.apache.spark.util.io.ChunkedByteBufferOutputStream.allocateNewChunkIfNeeded(ChunkedByteBufferOutputStream.scala:87)
at
org.apache.spark.util.io.ChunkedByteBufferOutputStream.write(ChunkedByteBufferOutputStream.scala:75)
at java.base/java.io.OutputStream.write(OutputStream.java:127)
at
org.apache.spark.util.io.ChunkedByteBufferOutputStreamSuite.$anonfun$new$22(ChunkedByteBufferOutputStreamSuite.scala:127)
at
org.apache.spark.util.io.ChunkedByteBufferOutputStreamSuite$$Lambda$179/0x00000008011a75d8.apply(Unknown
Source)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
{code}
> "SPARK-36464: size returns correct positive number even with over 2GB data"
> will oom with JDK17
> ------------------------------------------------------------------------------------------------
>
> Key: SPARK-36900
> URL: https://issues.apache.org/jira/browse/SPARK-36900
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Affects Versions: 3.3.0
> Reporter: Yang Jie
> Priority: Major
>
> Execute
>
> {code:java}
> build/mvn clean install -pl core -am -Dtest=none
> -DwildcardSuites=org.apache.spark.util.io.ChunkedByteBufferOutputStreamSuite
> {code}
> with JDK 17,
> {code:java}
> ChunkedByteBufferOutputStreamSuite:
> - empty output
> - write a single byte
> - write a single near boundary
> - write a single at boundary
> - single chunk output
> - single chunk output at boundary size
> - multiple chunk output
> - multiple chunk output at boundary size
> *** RUN ABORTED ***
> java.lang.OutOfMemoryError: Java heap space
> at java.base/java.lang.Integer.valueOf(Integer.java:1081)
> at scala.runtime.BoxesRunTime.boxToInteger(BoxesRunTime.java:67)
> at
> org.apache.spark.util.io.ChunkedByteBufferOutputStream.allocateNewChunkIfNeeded(ChunkedByteBufferOutputStream.scala:87)
> at
> org.apache.spark.util.io.ChunkedByteBufferOutputStream.write(ChunkedByteBufferOutputStream.scala:75)
> at java.base/java.io.OutputStream.write(OutputStream.java:127)
> at
> org.apache.spark.util.io.ChunkedByteBufferOutputStreamSuite.$anonfun$new$22(ChunkedByteBufferOutputStreamSuite.scala:127)
> at
> org.apache.spark.util.io.ChunkedByteBufferOutputStreamSuite$$Lambda$179/0x00000008011a75d8.apply(Unknown
> Source)
> at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
> at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
> at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
