This is an automated email from the ASF dual-hosted git repository.
pjfanning pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pekko.git
The following commit(s) were added to refs/heads/main by this push:
new 9d80429d4c fix: default the Jackson max-decompressed-size to unlimited
(#3515)
9d80429d4c is described below
commit 9d80429d4c521c71fc33e07bee8588b617390214
Author: PJ Fanning <[email protected]>
AuthorDate: Thu Sep 3 14:31:58 2026 +0100
fix: default the Jackson max-decompressed-size to unlimited (#3515)
* fix: default the Jackson max-decompressed-size to unlimited
Motivation:
A bounded default could reject a payload an existing system legitimately
exchanges, so a patch release carrying the 256 MiB default from #3491
could break running systems on upgrade. The bound should be opt-in,
matching the change made to pekko.serialization.max-decompressed-size
in #3502.
Modification:
Default pekko.serialization.jackson.compression.max-decompressed-size
(and the jackson3 equivalent) to -1, meaning no limit and matching the
behaviour of releases before #3491. A negative maximum skips the gzip
size check and the LZ4 declared-size check; a negative declared LZ4
size is still rejected, since it is malformed regardless of the limit.
Config's getBytes refuses negative numbers, so the setting is read as a
plain long first and as a memory size only when that is not a negative
number.
Result:
Jackson payload decompression is unbounded by default; configuring a
size such as 256 MiB bounds it.
Tests:
- sbt "serialization-jackson/testOnly
org.apache.pekko.serialization.jackson.*" - 122 passed
- sbt "serialization-jackson3/testOnly
org.apache.pekko.serialization.jackson3.*" - 120 passed
- sbt "serialization-jackson/scalafmtCheckAll"
"serialization-jackson3/scalafmtCheckAll" - clean
- sbt "serialization-jackson/mimaReportBinaryIssues" - no issues
References:
Refs #3491, Refs #3502
* also accept "unlimited" for max-decompressed-size
Motivation:
Review on #3515 noted that Pekko is inconsistent about unlimited
spellings and an explicit keyword is clearer than a magic number, while
the neighbouring read.max-document-length and read.max-token-count
settings use -1. Accept both.
Modification:
The setting reads "unlimited" or any negative number as no limit; the
reference.conf default is written as `unlimited`. Applied to both
serialization-jackson and serialization-jackson3, with a test each for
the keyword.
Result:
`max-decompressed-size = unlimited` and `= -1` both disable the bound.
Tests:
- sbt "serialization-jackson/testOnly
org.apache.pekko.serialization.jackson.*" - 123 passed
- sbt "serialization-jackson3/testOnly
org.apache.pekko.serialization.jackson3.*" - 121 passed
- sbt "serialization-jackson/scalafmtCheckAll"
"serialization-jackson3/scalafmtCheckAll" - clean
References:
Refs #3515
---
.../src/main/resources/reference.conf | 6 +++-
.../serialization/jackson/JacksonSerializer.scala | 20 +++++++++---
.../jackson/JacksonSerializerSpec.scala | 36 ++++++++++++++++++++++
.../src/main/resources/reference.conf | 6 +++-
.../serialization/jackson3/JacksonSerializer.scala | 20 +++++++++---
.../jackson3/JacksonSerializerSpec.scala | 36 ++++++++++++++++++++++
6 files changed, 114 insertions(+), 10 deletions(-)
diff --git a/serialization-jackson/src/main/resources/reference.conf
b/serialization-jackson/src/main/resources/reference.conf
index 5ad52a7110..593a4ad0aa 100644
--- a/serialization-jackson/src/main/resources/reference.conf
+++ b/serialization-jackson/src/main/resources/reference.conf
@@ -210,7 +210,11 @@ pekko.serialization.jackson {
# payload that decompresses to more than this is rejected rather than
# allocated, guarding against a small message that inflates without bound.
# This applies on deserialization regardless of the `algorithm` setting
above.
- max-decompressed-size = 256 MiB
+ # The default of `unlimited` applies no limit, preserving the behaviour of
+ # earlier releases; a negative number such as -1 also means unlimited. Set
a
+ # size such as `256 MiB` to bound decompression, choosing a value larger
than
+ # any payload the system legitimately exchanges.
+ max-decompressed-size = unlimited
}
# Whether the type should be written to the manifest.
diff --git
a/serialization-jackson/src/main/scala/org/apache/pekko/serialization/jackson/JacksonSerializer.scala
b/serialization-jackson/src/main/scala/org/apache/pekko/serialization/jackson/JacksonSerializer.scala
index 01fa6d016f..1ef5094b60 100644
---
a/serialization-jackson/src/main/scala/org/apache/pekko/serialization/jackson/JacksonSerializer.scala
+++
b/serialization-jackson/src/main/scala/org/apache/pekko/serialization/jackson/JacksonSerializer.scala
@@ -208,7 +208,16 @@ import pekko.util.OptionVal
""""off" or "gzip"""")
}
}
- private val maxDecompressedSize: Long =
conf.getBytes("compression.max-decompressed-size")
+ // "unlimited" or a negative number means no limit; getBytes refuses both,
so read them first
+ private val maxDecompressedSize: Long = {
+ val raw = conf.getString("compression.max-decompressed-size")
+ if (raw == "unlimited") -1L
+ else
+ raw.toLongOption match {
+ case Some(n) if n < 0 => n
+ case _ =>
conf.getBytes("compression.max-decompressed-size")
+ }
+ }
private val migrations: Map[String, JacksonMigration] = {
import scala.jdk.CollectionConverters._
conf.getConfig("migrations").root.unwrapped.asScala.toMap.map {
@@ -564,7 +573,10 @@ import pekko.util.OptionVal
// meta.length is the decompressed size declared on the wire; a small
// message can declare a huge (or negative) size and drive a large
// allocation, so bound it before decompressing.
- if (meta.length < 0 || meta.length > maxDecompressedSize)
+ if (meta.length < 0)
+ throw new IllegalArgumentException(
+ s"Compressed message declares a negative decompressed size
[${meta.length}] bytes")
+ if (maxDecompressedSize >= 0 && meta.length > maxDecompressedSize)
throw new IllegalArgumentException(
s"Compressed message declares decompressed size [${meta.length}]
bytes, which exceeds the maximum " +
s"of [$maxDecompressedSize] bytes
(pekko.serialization.jackson.compression.max-decompressed-size)")
@@ -576,7 +588,7 @@ import pekko.util.OptionVal
}
// gunzip with a bound on the decompressed size, so a small gzip payload
cannot
- // inflate without limit (a "zip bomb").
+ // inflate without limit (a "zip bomb"). A negative maximum applies no bound.
private def gunzip(in: GZIPInputStream): Array[Byte] = {
val out = new ByteArrayOutputStream()
val buffer = new Array[Byte](BufferSize)
@@ -584,7 +596,7 @@ import pekko.util.OptionVal
var n = in.read(buffer)
while (n != -1) {
total += n
- if (total > maxDecompressedSize)
+ if (maxDecompressedSize >= 0 && total > maxDecompressedSize)
throw new IllegalArgumentException(
s"Decompressed message exceeds the maximum of [$maxDecompressedSize]
bytes " +
"(pekko.serialization.jackson.compression.max-decompressed-size)")
diff --git
a/serialization-jackson/src/test/scala/org/apache/pekko/serialization/jackson/JacksonSerializerSpec.scala
b/serialization-jackson/src/test/scala/org/apache/pekko/serialization/jackson/JacksonSerializerSpec.scala
index 1265562c95..afc7fd8f0d 100644
---
a/serialization-jackson/src/test/scala/org/apache/pekko/serialization/jackson/JacksonSerializerSpec.scala
+++
b/serialization-jackson/src/test/scala/org/apache/pekko/serialization/jackson/JacksonSerializerSpec.scala
@@ -688,6 +688,42 @@ class JacksonJsonSerializerSpec extends
JacksonSerializerSpec("jackson-json") {
}
ex.getMessage should include("max-decompressed-size")
}
+
+ "apply no gzip decompression limit when max-decompressed-size is -1" in
withSystem("""
+ pekko.serialization.jackson.jackson-json.compression {
+ algorithm = gzip
+ compress-larger-than = 0 KiB
+ max-decompressed-size = -1
+ }
+ """) { sys =>
+ val msg = SimpleCommand("0" * (8 * 1024))
+ JacksonSerializer.isGZipped(serializeToBinary(msg, sys)) should ===(true)
+ checkSerialization(msg, sys)
+ }
+
+ "apply no lz4 decompression limit when max-decompressed-size is -1" in
withSystem("""
+ pekko.serialization.jackson.jackson-json.compression {
+ algorithm = lz4
+ compress-larger-than = 0 KiB
+ max-decompressed-size = -1
+ }
+ """) { sys =>
+ val msg = SimpleCommand("0" * (8 * 1024))
+ JacksonSerializer.isLZ4(serializeToBinary(msg, sys)) should ===(true)
+ checkSerialization(msg, sys)
+ }
+
+ "apply no decompression limit when max-decompressed-size is unlimited" in
withSystem("""
+ pekko.serialization.jackson.jackson-json.compression {
+ algorithm = gzip
+ compress-larger-than = 0 KiB
+ max-decompressed-size = unlimited
+ }
+ """) { sys =>
+ val msg = SimpleCommand("0" * (8 * 1024))
+ JacksonSerializer.isGZipped(serializeToBinary(msg, sys)) should ===(true)
+ checkSerialization(msg, sys)
+ }
}
"JacksonJsonSerializer without type in manifest" should {
diff --git a/serialization-jackson3/src/main/resources/reference.conf
b/serialization-jackson3/src/main/resources/reference.conf
index 0a94af43f2..18c37b5e9e 100644
--- a/serialization-jackson3/src/main/resources/reference.conf
+++ b/serialization-jackson3/src/main/resources/reference.conf
@@ -191,7 +191,11 @@ pekko.serialization.jackson3 {
# payload that decompresses to more than this is rejected rather than
# allocated, guarding against a small message that inflates without bound.
# This applies on deserialization regardless of the `algorithm` setting
above.
- max-decompressed-size = 256 MiB
+ # The default of `unlimited` applies no limit, preserving the behaviour of
+ # earlier releases; a negative number such as -1 also means unlimited. Set
a
+ # size such as `256 MiB` to bound decompression, choosing a value larger
than
+ # any payload the system legitimately exchanges.
+ max-decompressed-size = unlimited
}
# Whether the type should be written to the manifest.
diff --git
a/serialization-jackson3/src/main/scala/org/apache/pekko/serialization/jackson3/JacksonSerializer.scala
b/serialization-jackson3/src/main/scala/org/apache/pekko/serialization/jackson3/JacksonSerializer.scala
index 9c6181d358..c901176dc9 100644
---
a/serialization-jackson3/src/main/scala/org/apache/pekko/serialization/jackson3/JacksonSerializer.scala
+++
b/serialization-jackson3/src/main/scala/org/apache/pekko/serialization/jackson3/JacksonSerializer.scala
@@ -209,7 +209,16 @@ import pekko.util.OptionVal
""""off" or "gzip"""")
}
}
- private val maxDecompressedSize: Long =
conf.getBytes("compression.max-decompressed-size")
+ // "unlimited" or a negative number means no limit; getBytes refuses both,
so read them first
+ private val maxDecompressedSize: Long = {
+ val raw = conf.getString("compression.max-decompressed-size")
+ if (raw == "unlimited") -1L
+ else
+ raw.toLongOption match {
+ case Some(n) if n < 0 => n
+ case _ =>
conf.getBytes("compression.max-decompressed-size")
+ }
+ }
private val migrations: Map[String, JacksonMigration] = {
import scala.jdk.CollectionConverters._
conf.getConfig("migrations").root.unwrapped.asScala.toMap.map {
@@ -565,7 +574,10 @@ import pekko.util.OptionVal
// meta.length is the decompressed size declared on the wire; a small
// message can declare a huge (or negative) size and drive a large
// allocation, so bound it before decompressing.
- if (meta.length < 0 || meta.length > maxDecompressedSize)
+ if (meta.length < 0)
+ throw new IllegalArgumentException(
+ s"Compressed message declares a negative decompressed size
[${meta.length}] bytes")
+ if (maxDecompressedSize >= 0 && meta.length > maxDecompressedSize)
throw new IllegalArgumentException(
s"Compressed message declares decompressed size [${meta.length}]
bytes, which exceeds the maximum " +
s"of [$maxDecompressedSize] bytes
(pekko.serialization.jackson3.compression.max-decompressed-size)")
@@ -577,7 +589,7 @@ import pekko.util.OptionVal
}
// gunzip with a bound on the decompressed size, so a small gzip payload
cannot
- // inflate without limit (a "zip bomb").
+ // inflate without limit (a "zip bomb"). A negative maximum applies no bound.
private def gunzip(in: GZIPInputStream): Array[Byte] = {
val out = new ByteArrayOutputStream()
val buffer = new Array[Byte](BufferSize)
@@ -585,7 +597,7 @@ import pekko.util.OptionVal
var n = in.read(buffer)
while (n != -1) {
total += n
- if (total > maxDecompressedSize)
+ if (maxDecompressedSize >= 0 && total > maxDecompressedSize)
throw new IllegalArgumentException(
s"Decompressed message exceeds the maximum of [$maxDecompressedSize]
bytes " +
"(pekko.serialization.jackson3.compression.max-decompressed-size)")
diff --git
a/serialization-jackson3/src/test/scala/org/apache/pekko/serialization/jackson3/JacksonSerializerSpec.scala
b/serialization-jackson3/src/test/scala/org/apache/pekko/serialization/jackson3/JacksonSerializerSpec.scala
index 551e22426c..2e7d8cc86f 100644
---
a/serialization-jackson3/src/test/scala/org/apache/pekko/serialization/jackson3/JacksonSerializerSpec.scala
+++
b/serialization-jackson3/src/test/scala/org/apache/pekko/serialization/jackson3/JacksonSerializerSpec.scala
@@ -633,6 +633,42 @@ class JacksonJsonSerializerSpec extends
JacksonSerializerSpec("jackson-json") {
}
ex.getMessage should include("max-decompressed-size")
}
+
+ "apply no gzip decompression limit when max-decompressed-size is -1" in
withSystem("""
+ pekko.serialization.jackson3.jackson-json.compression {
+ algorithm = gzip
+ compress-larger-than = 0 KiB
+ max-decompressed-size = -1
+ }
+ """) { sys =>
+ val msg = SimpleCommand("0" * (8 * 1024))
+ JacksonSerializer.isGZipped(serializeToBinary(msg, sys)) should ===(true)
+ checkSerialization(msg, sys)
+ }
+
+ "apply no lz4 decompression limit when max-decompressed-size is -1" in
withSystem("""
+ pekko.serialization.jackson3.jackson-json.compression {
+ algorithm = lz4
+ compress-larger-than = 0 KiB
+ max-decompressed-size = -1
+ }
+ """) { sys =>
+ val msg = SimpleCommand("0" * (8 * 1024))
+ JacksonSerializer.isLZ4(serializeToBinary(msg, sys)) should ===(true)
+ checkSerialization(msg, sys)
+ }
+
+ "apply no decompression limit when max-decompressed-size is unlimited" in
withSystem("""
+ pekko.serialization.jackson3.jackson-json.compression {
+ algorithm = gzip
+ compress-larger-than = 0 KiB
+ max-decompressed-size = unlimited
+ }
+ """) { sys =>
+ val msg = SimpleCommand("0" * (8 * 1024))
+ JacksonSerializer.isGZipped(serializeToBinary(msg, sys)) should ===(true)
+ checkSerialization(msg, sys)
+ }
}
"JacksonJsonSerializer without type in manifest" should {
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
