if we do check on all code paths then we must not create a package scope getBigDecimalUnchecked but just do check for getBigDecimal() and be it no?
Romain Manni-Bucau @rmannibucau <https://x.com/rmannibucau> | .NET Blog <https://dotnetbirdie.github.io/> | Blog <https://rmannibucau.github.io/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064> Javaccino <https://javaccino.dev/> founder (Java/.NET service - contact via linkedin) ---------- Forwarded message --------- De : <[email protected]> Date: dim. 25 oct. 2026 à 09:46 Subject: (johnzon) branch master updated: Align BigDecimal scale handling To: [email protected] <[email protected]> This is an automated email from the ASF dual-hosted git repository. jungm pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/johnzon.git The following commit(s) were added to refs/heads/master by this push: new 80b26a01 Align BigDecimal scale handling 80b26a01 is described below commit 80b26a010184ccc00c10f3753a428dae3bd1147c Author: Markus Jung <[email protected]> AuthorDate: Sun Oct 25 09:46:04 2026 +0100 Align BigDecimal scale handling --- .../apache/johnzon/core/JohnzonJsonParserImpl.java | 13 +- .../apache/johnzon/core/JsonInMemoryParser.java | 5 + .../org/apache/johnzon/core/JsonNumberImpl.java | 15 +- .../org/apache/johnzon/core/JsonProviderImpl.java | 2 +- .../org/apache/johnzon/core/JsonReaderImpl.java | 11 +- .../apache/johnzon/core/JsonStreamParserImpl.java | 19 +- .../johnzon/core/JsonInMemoryParserTest.java | 23 ++ .../org/apache/johnzon/core/JsonNumberTest.java | 30 +++ .../apache/johnzon/core/JsonReaderImplTest.java | 4 +- .../johnzon/core/JsonStreamParserImplTest.java | 46 ++++ .../org/apache/johnzon/core/SerializationTest.java | 12 + .../org/apache/johnzon/jsonb/JohnzonBuilder.java | 8 +- .../org/apache/johnzon/jsonb/JohnzonJsonb.java | 5 +- .../org/apache/johnzon/jsonb/JsonbAccessMode.java | 22 +- .../jsonb/converter/JsonbNumberConverter.java | 12 + .../apache/johnzon/jsonb/BigDecimalScaleTest.java | 154 +++++++++++++ .../org/apache/johnzon/mapper/MapperBuilder.java | 38 +++- .../apache/johnzon/mapper/MappingParserImpl.java | 30 ++- .../mapper/converter/BigDecimalConverter.java | 40 +++- .../johnzon/mapper/map/LazyConverterMap.java | 29 ++- .../johnzon/mapper/NumberSerializationTest.java | 251 +++++++++++++++++++++ src/site/markdown/index.md | 1 + src/site/markdown/security.md | 25 +- src/site/markdown/vulnerabilities.md | 2 +- 24 files changed, 756 insertions(+), 41 deletions(-) diff --git a/johnzon-core/src/main/java/org/apache/johnzon/core/JohnzonJsonParserImpl.java b/johnzon-core/src/main/java/org/apache/johnzon/core/JohnzonJsonParserImpl.java index af0e9cf4..0c4b33f4 100644 --- a/johnzon-core/src/main/java/org/apache/johnzon/core/JohnzonJsonParserImpl.java +++ b/johnzon-core/src/main/java/org/apache/johnzon/core/JohnzonJsonParserImpl.java @@ -17,6 +17,7 @@ package org.apache.johnzon.core; +import java.math.BigDecimal; import java.util.AbstractMap; import java.util.Collections; import java.util.Map; @@ -56,6 +57,16 @@ public abstract class JohnzonJsonParserImpl implements JohnzonJsonParser { this.provider = provider; } + protected final BigDecimal checkBigDecimalScale(final BigDecimal value) { + provider.checkBigDecimalScale(value); + return value; + } + + // Only use this for safe primitive conversions or to construct a guarded JsonNumberImpl. + BigDecimal getBigDecimalUnchecked() { + return getBigDecimal(); + } + @Override public Event next() { @@ -109,7 +120,7 @@ public abstract class JohnzonJsonParserImpl implements JohnzonJsonParser { if (isFitLong()) { return new JsonLongImpl(getLong()); } - return new JsonNumberImpl(getBigDecimal(), provider::checkBigDecimalScale); + return new JsonNumberImpl(getBigDecimalUnchecked(), provider::checkBigDecimalScale); default: throw new IllegalStateException(current + " doesn't support getValue()"); } diff --git a/johnzon-core/src/main/java/org/apache/johnzon/core/JsonInMemoryParser.java b/johnzon-core/src/main/java/org/apache/johnzon/core/JsonInMemoryParser.java index 0655f700..91d5391d 100644 --- a/johnzon-core/src/main/java/org/apache/johnzon/core/JsonInMemoryParser.java +++ b/johnzon-core/src/main/java/org/apache/johnzon/core/JsonInMemoryParser.java @@ -307,6 +307,11 @@ class JsonInMemoryParser extends JohnzonJsonParserImpl { @Override public BigDecimal getBigDecimal() { + return checkBigDecimalScale(getBigDecimalUnchecked()); + } + + @Override + BigDecimal getBigDecimalUnchecked() { if (currentEvent != Event.VALUE_NUMBER) { throw new IllegalStateException("getBigDecimal is for numbers"); } diff --git a/johnzon-core/src/main/java/org/apache/johnzon/core/JsonNumberImpl.java b/johnzon-core/src/main/java/org/apache/johnzon/core/JsonNumberImpl.java index 158a1c35..82704ac6 100644 --- a/johnzon-core/src/main/java/org/apache/johnzon/core/JsonNumberImpl.java +++ b/johnzon-core/src/main/java/org/apache/johnzon/core/JsonNumberImpl.java @@ -19,15 +19,21 @@ package org.apache.johnzon.core; import jakarta.json.JsonNumber; +import java.io.IOException; +import java.io.ObjectInputStream; import java.io.Serializable; import java.math.BigDecimal; import java.math.BigInteger; import java.util.function.Consumer; final class JsonNumberImpl implements JsonNumber, Serializable { + private static final class DeserializationScaleChecker { + private static final Consumer<BigDecimal> INSTANCE = new JsonProviderImpl()::checkBigDecimalScale; + } + private final BigDecimal value; private transient Integer hashCode = null; - private transient Consumer<BigDecimal> maxBigDecimalScale = (bigDecimal) -> {}; // for deserialization? + private transient Consumer<BigDecimal> maxBigDecimalScale; JsonNumberImpl(final BigDecimal decimal, final Consumer<BigDecimal> maxBigDecimalScale) { this.maxBigDecimalScale = maxBigDecimalScale; @@ -38,8 +44,15 @@ final class JsonNumberImpl implements JsonNumber, Serializable { this.value = decimal; } + private void readObject(final ObjectInputStream in) throws IOException, ClassNotFoundException { + in.defaultReadObject(); + // The original provider callback is transient; restore the default limit after deserialization. + maxBigDecimalScale = DeserializationScaleChecker.INSTANCE; + } + @Override public Number numberValue() { + maxBigDecimalScale.accept(value); return value; } diff --git a/johnzon-core/src/main/java/org/apache/johnzon/core/JsonProviderImpl.java b/johnzon-core/src/main/java/org/apache/johnzon/core/JsonProviderImpl.java index fce0fac9..a38fd29b 100644 --- a/johnzon-core/src/main/java/org/apache/johnzon/core/JsonProviderImpl.java +++ b/johnzon-core/src/main/java/org/apache/johnzon/core/JsonProviderImpl.java @@ -268,7 +268,7 @@ public class JsonProviderImpl extends JsonProvider implements Serializable { // should be fine enough. Maybe we should externalize so users can pick something better if they need to // it becomes their responsibility to fix the limit and may expose them to a DoS attack final int limit = maxBigDecimalScale; - final int absScale = Math.abs(value.scale()); + final long absScale = Math.abs((long) value.scale()); if (absScale > limit) { throw new ArithmeticException(String.format( diff --git a/johnzon-core/src/main/java/org/apache/johnzon/core/JsonReaderImpl.java b/johnzon-core/src/main/java/org/apache/johnzon/core/JsonReaderImpl.java index fff2c46a..c46af05c 100644 --- a/johnzon-core/src/main/java/org/apache/johnzon/core/JsonReaderImpl.java +++ b/johnzon-core/src/main/java/org/apache/johnzon/core/JsonReaderImpl.java @@ -31,6 +31,7 @@ import jakarta.json.stream.JsonParsingException; import static java.util.Collections.emptyList; import static java.util.Collections.emptyMap; +import java.math.BigDecimal; import java.util.Arrays; public class JsonReaderImpl implements JsonReader { @@ -68,6 +69,12 @@ public class JsonReaderImpl implements JsonReader { this.rejectDuplicateKeysMode = rejectDuplicateKeys; } + private BigDecimal getBigDecimalUnchecked() { + return parser instanceof JohnzonJsonParserImpl + ? ((JohnzonJsonParserImpl) parser).getBigDecimalUnchecked() + : parser.getBigDecimal(); + } + @Override public JsonStructure read() { return JsonStructure.class.cast(readValue()); @@ -128,7 +135,7 @@ public class JsonReaderImpl implements JsonReader { if (parser.isFitLong()) { number = new JsonLongImpl(parser.getLong()); } else { - number = new JsonNumberImpl(parser.getBigDecimal(), provider::checkBigDecimalScale); + number = new JsonNumberImpl(getBigDecimalUnchecked(), provider::checkBigDecimalScale); } if (!subStreamReader && parser.hasNext()) { throw new JsonParsingException("Expected end of file", parser.getLocation()); @@ -202,7 +209,7 @@ public class JsonReaderImpl implements JsonReader { if (parser.isFitLong()) { add(current, currentKey, new JsonLongImpl(parser.getLong())); } else { - add(current, currentKey, new JsonNumberImpl(parser.getBigDecimal(), provider::checkBigDecimalScale)); + add(current, currentKey, new JsonNumberImpl(getBigDecimalUnchecked(), provider::checkBigDecimalScale)); } break; diff --git a/johnzon-core/src/main/java/org/apache/johnzon/core/JsonStreamParserImpl.java b/johnzon-core/src/main/java/org/apache/johnzon/core/JsonStreamParserImpl.java index cadceeba..fc909f8b 100644 --- a/johnzon-core/src/main/java/org/apache/johnzon/core/JsonStreamParserImpl.java +++ b/johnzon-core/src/main/java/org/apache/johnzon/core/JsonStreamParserImpl.java @@ -978,12 +978,12 @@ public class JsonStreamParserImpl extends JohnzonJsonParserImpl implements JsonC final Integer retVal = fallBackCopyBufferLength > 0 ? parseIntegerFromChars(fallBackCopyBuffer, 0, fallBackCopyBufferLength) : parseIntegerFromChars(buffer, startOfValueInBuffer, endOfValueInBuffer); if (retVal == null) { - return getBigDecimal().intValue(); + return getBigDecimalUnchecked().intValue(); } else { return retVal.intValue(); } } else { - return getBigDecimal().intValue(); + return getBigDecimalUnchecked().intValue(); } } @@ -999,12 +999,12 @@ public class JsonStreamParserImpl extends JohnzonJsonParserImpl implements JsonC final Long retVal = fallBackCopyBufferLength > 0 ? parseLongFromChars(fallBackCopyBuffer, 0, fallBackCopyBufferLength) : parseLongFromChars(buffer, startOfValueInBuffer, endOfValueInBuffer); if (retVal == null) { - return getBigDecimal().longValue(); + return getBigDecimalUnchecked().longValue(); } else { return retVal.longValue(); } } else { - return getBigDecimal().longValue(); + return getBigDecimalUnchecked().longValue(); } } @@ -1023,6 +1023,11 @@ public class JsonStreamParserImpl extends JohnzonJsonParserImpl implements JsonC @Override public BigDecimal getBigDecimal() { + return checkBigDecimalScale(getBigDecimalUnchecked()); + } + + @Override + BigDecimal getBigDecimalUnchecked() { if (previousEvent != VALUE_NUMBER) { throw new IllegalStateException(EVT_MAP[previousEvent] + " doesn't support getBigDecimal()"); // } else if (currentBigDecimalNumber != null) { @@ -1033,8 +1038,8 @@ public class JsonStreamParserImpl extends JohnzonJsonParserImpl implements JsonC combinePreviousFallbackBuffersToCurrent(); //if there a content in the value buffer read from them, if not use main buffer - return (/*currentBigDecimalNumber = */fallBackCopyBufferLength > 0 ? new BigDecimal(fallBackCopyBuffer, 0, - fallBackCopyBufferLength) : new BigDecimal(buffer, startOfValueInBuffer, (endOfValueInBuffer - startOfValueInBuffer))); + return /*currentBigDecimalNumber = */fallBackCopyBufferLength > 0 ? new BigDecimal(fallBackCopyBuffer, 0, + fallBackCopyBufferLength) : new BigDecimal(buffer, startOfValueInBuffer, (endOfValueInBuffer - startOfValueInBuffer)); } @Override @@ -1128,4 +1133,4 @@ public class JsonStreamParserImpl extends JohnzonJsonParserImpl implements JsonC return new JsonParsingException("General exception on " + location + ". Reason is [[" + message + "]]", location); } -} \ No newline at end of file +} diff --git a/johnzon-core/src/test/java/org/apache/johnzon/core/JsonInMemoryParserTest.java b/johnzon-core/src/test/java/org/apache/johnzon/core/JsonInMemoryParserTest.java index a219a860..416738b3 100644 --- a/johnzon-core/src/test/java/org/apache/johnzon/core/JsonInMemoryParserTest.java +++ b/johnzon-core/src/test/java/org/apache/johnzon/core/JsonInMemoryParserTest.java @@ -20,13 +20,36 @@ package org.apache.johnzon.core; import org.junit.Test; +import jakarta.json.JsonArray; import jakarta.json.stream.JsonParser; +import java.math.BigDecimal; import java.util.Collections; import static org.junit.Assert.assertEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; public class JsonInMemoryParserTest { + @Test + public void getBigDecimalChecksReceivingProviderScale() { + final JsonProviderImpl permissiveProvider = new JsonProviderImpl(); + permissiveProvider.setMaxBigDecimalScale(4); + final JsonArray array = permissiveProvider.createArrayBuilder() + .add(new BigDecimal("1e2")) + .add(new BigDecimal("1e3")) + .build(); + + final JsonProviderImpl strictProvider = new JsonProviderImpl(); + strictProvider.setMaxBigDecimalScale(2); + try (JsonParser parser = strictProvider.createParserFactory(Collections.emptyMap()).createParser(array)) { + assertEquals(JsonParser.Event.START_ARRAY, parser.next()); + assertEquals(JsonParser.Event.VALUE_NUMBER, parser.next()); + assertEquals(new BigDecimal("1e2"), parser.getBigDecimal()); + assertEquals(JsonParser.Event.VALUE_NUMBER, parser.next()); + assertThrows(ArithmeticException.class, parser::getBigDecimal); + } + } + @Test public void testSpecCurrentEvent() { JsonParser parser = new JsonInMemoryParser( diff --git a/johnzon-core/src/test/java/org/apache/johnzon/core/JsonNumberTest.java b/johnzon-core/src/test/java/org/apache/johnzon/core/JsonNumberTest.java index 77d2866e..05d6fd1d 100644 --- a/johnzon-core/src/test/java/org/apache/johnzon/core/JsonNumberTest.java +++ b/johnzon-core/src/test/java/org/apache/johnzon/core/JsonNumberTest.java @@ -32,9 +32,39 @@ import java.util.concurrent.TimeUnit; import static org.junit.Assert.assertFalse; import static org.junit.Assert.fail; +import static org.junit.jupiter.api.Assertions.assertThrows; public class JsonNumberTest { + @Test + public void numberValueChecksConfiguredScale() { + final JsonProviderImpl provider = new JsonProviderImpl(); + provider.setMaxBigDecimalScale(2); + + Assert.assertEquals(new BigDecimal("1e2"), provider.createValue(new BigDecimal("1e2")).numberValue()); + Assert.assertEquals(new BigDecimal("1e-2"), provider.createValue(new BigDecimal("1e-2")).numberValue()); + assertThrows(ArithmeticException.class, () -> provider.createValue(new BigDecimal("1e3")).numberValue()); + assertThrows(ArithmeticException.class, () -> provider.createValue(new BigDecimal("1e-3")).numberValue()); + } + + @Test(expected = ArithmeticException.class) + public void minimumScaleRejected() { + final JsonProviderImpl provider = new JsonProviderImpl(); + provider.createValue(new BigDecimal(BigInteger.ONE, Integer.MIN_VALUE)).bigDecimalValue(); + } + + @Test + public void exponentBeyondIntegerScaleRangeRejected() { + final JsonProviderImpl provider = new JsonProviderImpl(); + try (final jakarta.json.JsonReader reader = provider.createReader(new StringReader("[1e2147483648]"))) { + reader.readArray().getJsonNumber(0).bigDecimalValue(); + } catch (final ArithmeticException | NumberFormatException e) { + // Older JDKs reject this exponent while parsing; newer JDKs must reject its scale. + return; + } + fail("expected the minimum scale to be rejected"); + } + @Test public void nonZeroFractional() { final JsonNumber number = Json.createArrayBuilder() diff --git a/johnzon-core/src/test/java/org/apache/johnzon/core/JsonReaderImplTest.java b/johnzon-core/src/test/java/org/apache/johnzon/core/JsonReaderImplTest.java index 618c39c7..5a7e87e1 100644 --- a/johnzon-core/src/test/java/org/apache/johnzon/core/JsonReaderImplTest.java +++ b/johnzon-core/src/test/java/org/apache/johnzon/core/JsonReaderImplTest.java @@ -194,7 +194,7 @@ public class JsonReaderImplTest { final long longVal = jsonNumber.longValue(); final int intVal = jsonNumber.intValue(); final double doubleVal = jsonNumber.doubleValue(); - jsonNumber.numberValue(); + assertArithmeticException(jsonNumber::numberValue); jsonNumber.isIntegral(); jsonNumber.hashCode(); @@ -858,4 +858,4 @@ public class JsonReaderImplTest { return current; } -} \ No newline at end of file +} diff --git a/johnzon-core/src/test/java/org/apache/johnzon/core/JsonStreamParserImplTest.java b/johnzon-core/src/test/java/org/apache/johnzon/core/JsonStreamParserImplTest.java index 965d6e96..dbce75d1 100644 --- a/johnzon-core/src/test/java/org/apache/johnzon/core/JsonStreamParserImplTest.java +++ b/johnzon-core/src/test/java/org/apache/johnzon/core/JsonStreamParserImplTest.java @@ -19,6 +19,7 @@ package org.apache.johnzon.core; import jakarta.json.Json; +import jakarta.json.JsonNumber; import jakarta.json.JsonReader; import jakarta.json.JsonReaderFactory; import jakarta.json.spi.JsonProvider; @@ -29,6 +30,7 @@ import jakarta.json.stream.JsonParser; import java.io.ByteArrayInputStream; import java.io.IOException; import java.io.StringReader; +import java.math.BigDecimal; import java.nio.charset.StandardCharsets; import java.util.ArrayList; import java.util.List; @@ -38,8 +40,52 @@ import static java.util.Arrays.asList; import static java.util.Collections.emptyMap; import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertTrue; +import static org.junit.jupiter.api.Assertions.assertThrows; public class JsonStreamParserImplTest { + @Test + public void getBigDecimalChecksDefaultScale() { + final JsonProviderImpl provider = new JsonProviderImpl(); + assertEquals(new BigDecimal("1e1000"), parseBigDecimal(provider, "1e1000")); + assertThrows(ArithmeticException.class, () -> parseBigDecimal(provider, "1e1001")); + assertThrows(ArithmeticException.class, () -> parseBigDecimal(provider, "1e-1001")); + } + + @Test + public void getBigDecimalChecksConfiguredScale() { + final JsonProviderImpl provider = new JsonProviderImpl(); + provider.setMaxBigDecimalScale(2); + assertEquals(new BigDecimal("1e2"), parseBigDecimal(provider, "1e2")); + assertEquals(new BigDecimal("1e-2"), parseBigDecimal(provider, "1e-2")); + assertThrows(ArithmeticException.class, () -> parseBigDecimal(provider, "1e3")); + assertThrows(ArithmeticException.class, () -> parseBigDecimal(provider, "1e-3")); + } + + @Test + public void getValueKeepsCompactNumberGuarded() { + final JsonProviderImpl provider = new JsonProviderImpl(); + provider.setMaxBigDecimalScale(2); + try (JohnzonJsonParser parser = (JohnzonJsonParser) provider.createParser(new StringReader("[1e3]"))) { + assertEquals(JsonParser.Event.START_ARRAY, parser.next()); + assertEquals(JsonParser.Event.VALUE_NUMBER, parser.next()); + assertEquals(1000, parser.getInt()); + assertEquals(1000L, parser.getLong()); + assertThrows(ArithmeticException.class, parser::getBigDecimal); + final JsonNumber number = (JsonNumber) parser.getValue(); + assertEquals("1E+3", number.toString()); + assertThrows(ArithmeticException.class, number::bigDecimalValue); + assertThrows(ArithmeticException.class, number::numberValue); + } + } + + private BigDecimal parseBigDecimal(final JsonProviderImpl provider, final String value) { + try (JsonParser parser = provider.createParser(new StringReader("[" + value + "]"))) { + assertEquals(JsonParser.Event.START_ARRAY, parser.next()); + assertEquals(JsonParser.Event.VALUE_NUMBER, parser.next()); + return parser.getBigDecimal(); + } + } + @Test public void testSpecCurrentEvent() { String json = "{}"; diff --git a/johnzon-core/src/test/java/org/apache/johnzon/core/SerializationTest.java b/johnzon-core/src/test/java/org/apache/johnzon/core/SerializationTest.java index 3a19b191..4ddc1cd5 100644 --- a/johnzon-core/src/test/java/org/apache/johnzon/core/SerializationTest.java +++ b/johnzon-core/src/test/java/org/apache/johnzon/core/SerializationTest.java @@ -39,6 +39,7 @@ import java.util.Map; import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertNotSame; import static org.junit.Assert.assertTrue; +import static org.junit.jupiter.api.Assertions.assertThrows; public class SerializationTest { @Test @@ -57,9 +58,20 @@ public class SerializationTest { final JsonNumber deserialized = serialDeser(source); assertNotSame(source, deserialized); assertEquals(1.0, deserialized.doubleValue(), 0.); + assertEquals(new BigDecimal("1.0"), deserialized.numberValue()); + assertEquals(new BigDecimal("1.0"), deserialized.bigDecimalValue()); assertEquals("1.0", deserialized.toString()); } + @Test + public void deserializedNumberUsesDefaultScaleLimit() throws IOException, ClassNotFoundException { + final JsonNumber source = new JsonNumberImpl(new BigDecimal("1e1001"), + new JsonProviderImpl()::checkBigDecimalScale); + final JsonNumber deserialized = serialDeser(source); + assertThrows(ArithmeticException.class, deserialized::numberValue); + assertThrows(ArithmeticException.class, deserialized::bigDecimalValue); + } + @Test public void jsonLong() throws IOException, ClassNotFoundException { final JsonNumber source = new JsonLongImpl(1); diff --git a/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/JohnzonBuilder.java b/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/JohnzonBuilder.java index 460d41c8..092d2660 100644 --- a/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/JohnzonBuilder.java +++ b/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/JohnzonBuilder.java @@ -36,6 +36,7 @@ import org.apache.johnzon.mapper.ObjectConverter; import org.apache.johnzon.mapper.SerializeValueFilter; import org.apache.johnzon.mapper.access.AccessMode; import org.apache.johnzon.mapper.access.FieldAndMethodAccessMode; +import org.apache.johnzon.mapper.converter.BigDecimalConverter; import org.apache.johnzon.mapper.converter.LocaleConverter; import org.apache.johnzon.mapper.internal.AdapterKey; @@ -110,7 +111,8 @@ public class JohnzonBuilder implements JsonbBuilder { // todo: global spec toggle to disable all these ones at once? builder.setUseBigDecimalForObjectNumbers(config.getProperty("johnzon.use-big-decimal-for-object").map(this::toBool).orElse(true)); - builder.setMaxBigDecimalScale(config.getProperty("johnzon.max-big-decimal-scale").map(this::toInt).orElse(1000)); + final int maxBigDecimalScale = config.getProperty("johnzon.max-big-decimal-scale").map(this::toInt).orElse(BigDecimalConverter.DEFAULT_MAX_SCALE); + builder.setMaxBigDecimalScale(maxBigDecimalScale); builder.setSupportEnumContainerDeserialization( // https://github.com/eclipse-ee4j/jakartaee-tck/issues/103 toBool(System.getProperty("johnzon.support-enum-container-deserialization", config.getProperty("johnzon.support-enum-container-deserialization") @@ -226,7 +228,7 @@ public class JohnzonBuilder implements JsonbBuilder { isNillable, config.getProperty("johnzon.supportsPrivateAccess") .map(this::toBool) - .orElse(false))); + .orElse(false), maxBigDecimalScale)); builder.setAccessMode(accessMode); config.getProperty("johnzon.snippetMaxLength") @@ -489,4 +491,4 @@ public class JohnzonBuilder implements JsonbBuilder { protected abstract T doCreate(); } -} \ No newline at end of file +} diff --git a/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/JohnzonJsonb.java b/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/JohnzonJsonb.java index 5f33a27d..ba7779e2 100644 --- a/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/JohnzonJsonb.java +++ b/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/JohnzonJsonb.java @@ -42,6 +42,7 @@ import java.io.StringReader; import java.io.Writer; import java.lang.reflect.ParameterizedType; import java.lang.reflect.Type; +import java.math.BigDecimal; import java.math.BigInteger; import java.util.Collection; import java.util.List; @@ -523,9 +524,9 @@ public class JohnzonJsonb implements Jsonb, AutoCloseable, JsonbExtension { return (T) Short.valueOf((short) jsonNumber.intValue()); } if (BigInteger.class == type) { - return (T) jsonNumber.bigIntegerValue(); + return delegate.readObject(json, BigInteger.class); } - return (T) jsonNumber.bigDecimalValue(); + return delegate.readObject(json, BigDecimal.class); case OBJECT: case ARRAY: return delegate.readObject(JsonStructure.class.cast(json), type); diff --git a/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/JsonbAccessMode.java b/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/JsonbAccessMode.java index ef0e7069..c407c4b4 100644 --- a/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/JsonbAccessMode.java +++ b/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/JsonbAccessMode.java @@ -49,6 +49,7 @@ import org.apache.johnzon.mapper.access.FieldAccessMode; import org.apache.johnzon.mapper.access.FieldAndMethodAccessMode; import org.apache.johnzon.mapper.access.Meta; import org.apache.johnzon.mapper.access.MethodAccessMode; +import org.apache.johnzon.mapper.converter.BigDecimalConverter; import org.apache.johnzon.mapper.converter.ReversedAdapter; import org.apache.johnzon.mapper.internal.AdapterKey; import org.apache.johnzon.mapper.internal.ConverterAdapter; @@ -156,6 +157,7 @@ public class JsonbAccessMode implements AccessMode, Closeable, Cleanable<Class<? private final Types types = new Types(); private final boolean globalIsNillable; private final boolean supportsPrivateAccess; + private final int maxBigDecimalScale; // CHECKSTYLE:OFF public JsonbAccessMode(final PropertyNamingStrategy propertyNamingStrategy, final String orderValue, @@ -167,6 +169,23 @@ public class JsonbAccessMode implements AccessMode, Closeable, Cleanable<Class<? final boolean failOnMissingCreatorValues, final boolean globalIsNillable, final boolean supportsPrivateAccess) { + this(propertyNamingStrategy, orderValue, visibilityStrategy, caseSensitive, defaultConverters, factory, + jsonProvider, builderFactory, parserFactory, delegate, failOnMissingCreatorValues, + globalIsNillable, supportsPrivateAccess, BigDecimalConverter.DEFAULT_MAX_SCALE); + } + // CHECKSTYLE:ON + + // CHECKSTYLE:OFF + public JsonbAccessMode(final PropertyNamingStrategy propertyNamingStrategy, final String orderValue, + final PropertyVisibilityStrategy visibilityStrategy, final boolean caseSensitive, + final Map<AdapterKey, Adapter<?, ?>> defaultConverters, final JohnzonAdapterFactory factory, + final JsonProvider jsonProvider, final Supplier<JsonBuilderFactory> builderFactory, + final Supplier<JsonParserFactory> parserFactory, + final AccessMode delegate, + final boolean failOnMissingCreatorValues, + final boolean globalIsNillable, + final boolean supportsPrivateAccess, + final int maxBigDecimalScale) { // CHECKSTYLE:ON this.globalIsNillable = globalIsNillable; this.naming = propertyNamingStrategy; @@ -181,6 +200,7 @@ public class JsonbAccessMode implements AccessMode, Closeable, Cleanable<Class<? this.parserFactory = parserFactory; this.failOnMissingCreatorValues = failOnMissingCreatorValues; this.supportsPrivateAccess = supportsPrivateAccess; + this.maxBigDecimalScale = maxBigDecimalScale; } @Override @@ -543,7 +563,7 @@ public class JsonbAccessMode implements AccessMode, Closeable, Cleanable<Class<? converter = null; // todo: should we fallback on numberformat? } } else if (numberFormat != null) { // TODO: support lists? - converter = new ConverterAdapter<>(new JsonbNumberConverter(numberFormat), Number.class); + converter = new ConverterAdapter<>(new JsonbNumberConverter(numberFormat, maxBigDecimalScale), Number.class); } else { converter = new ConverterAdapter<>(new JsonbValueConverter(), Object.class); } diff --git a/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/converter/JsonbNumberConverter.java b/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/converter/JsonbNumberConverter.java index 5034c3e0..c86f4c34 100644 --- a/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/converter/JsonbNumberConverter.java +++ b/johnzon-jsonb/src/main/java/org/apache/johnzon/jsonb/converter/JsonbNumberConverter.java @@ -18,7 +18,10 @@ */ package org.apache.johnzon.jsonb.converter; +import org.apache.johnzon.mapper.converter.BigDecimalConverter; + import jakarta.json.bind.annotation.JsonbNumberFormat; +import java.math.BigDecimal; import java.text.DecimalFormat; import java.text.DecimalFormatSymbols; import java.text.NumberFormat; @@ -31,8 +34,14 @@ import java.util.function.Supplier; public class JsonbNumberConverter extends JsonbLocaleParserConverterBase<Number> { private final Supplier<NumberFormat> delegateFactory; // NumberFormat is not thread safe private final Queue<NumberFormat> pool = new ArrayBlockingQueue<>(30); // configurable? + private final int maxBigDecimalScale; public JsonbNumberConverter(final JsonbNumberFormat numberFormat) { + this(numberFormat, BigDecimalConverter.DEFAULT_MAX_SCALE); + } + + public JsonbNumberConverter(final JsonbNumberFormat numberFormat, final int maxBigDecimalScale) { + this.maxBigDecimalScale = maxBigDecimalScale; final String locale = numberFormat.locale(); final String format = numberFormat.value(); final boolean customLocale = !JsonbNumberFormat.DEFAULT_LOCALE.equals(locale); @@ -49,6 +58,9 @@ public class JsonbNumberConverter extends JsonbLocaleParserConverterBase<Number> @Override public String toString(final Number instance) { + if (instance instanceof BigDecimal) { + BigDecimalConverter.checkScale((BigDecimal) instance, maxBigDecimalScale); + } return execute(f -> f.format(instance)); } diff --git a/johnzon-jsonb/src/test/java/org/apache/johnzon/jsonb/BigDecimalScaleTest.java b/johnzon-jsonb/src/test/java/org/apache/johnzon/jsonb/BigDecimalScaleTest.java new file mode 100644 index 00000000..1d8db929 --- /dev/null +++ b/johnzon-jsonb/src/test/java/org/apache/johnzon/jsonb/BigDecimalScaleTest.java @@ -0,0 +1,154 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.johnzon.jsonb; + +import org.apache.johnzon.core.JsonProviderImpl; +import org.apache.johnzon.jsonb.api.experimental.JsonbExtension; +import org.junit.Test; + +import jakarta.json.JsonNumber; +import jakarta.json.bind.Jsonb; +import jakarta.json.bind.JsonbBuilder; +import jakarta.json.bind.JsonbConfig; +import jakarta.json.bind.annotation.JsonbNumberFormat; +import java.math.BigDecimal; +import java.math.BigInteger; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.fail; + +public class BigDecimalScaleTest { + @Test + public void defaultScaleLimitWithStringAdapter() throws Exception { + try (final Jsonb jsonb = JsonbBuilder.create(new JsonbConfig() + .setProperty("johnzon.use-bigdecimal-stringadapter", true))) { + assertEquals(new BigDecimal("1e1000"), jsonb.fromJson("{\"value\":1e1000}", Holder.class).value); + assertEquals(new BigDecimal("1e1000"), jsonb.fromJson("{\"value\":\"1e1000\"}", Holder.class).value); + assertScaleRejected(jsonb, "{\"value\":1e1001}"); + assertScaleRejected(jsonb, "{\"value\":\"1e1001\"}"); + } + } + + @Test + public void defaultScaleLimitWithoutStringAdapter() throws Exception { + try (final Jsonb jsonb = JsonbBuilder.create(new JsonbConfig() + .setProperty("johnzon.use-bigdecimal-stringadapter", false))) { + assertEquals(new BigDecimal("1e1000"), jsonb.fromJson("{\"value\":1e1000}", Holder.class).value); + assertScaleRejected(jsonb, "{\"value\":1e1001}"); + } + } + + @Test + public void configuredScaleLimit() throws Exception { + try (final Jsonb jsonb = JsonbBuilder.create(new JsonbConfig() + .setProperty("johnzon.use-bigdecimal-stringadapter", true) + .setProperty("johnzon.max-big-decimal-scale", 2000))) { + assertEquals(new BigDecimal("1e2000"), jsonb.fromJson("{\"value\":1e2000}", Holder.class).value); + assertEquals(new BigDecimal("1e2000"), jsonb.fromJson("{\"value\":\"1e2000\"}", Holder.class).value); + assertScaleRejected(jsonb, "{\"value\":1e2001}"); + assertScaleRejected(jsonb, "{\"value\":\"1e2001\"}"); + } + } + + @Test + public void stricterLimitWinsWithProvider() throws Exception { + final org.apache.johnzon.core.JsonProviderImpl provider = new org.apache.johnzon.core.JsonProviderImpl(); + provider.setMaxBigDecimalScale(100); + try (final Jsonb jsonb = JsonbBuilder.newBuilder().withProvider(provider).build()) { + assertEquals(new BigDecimal("1e-100"), jsonb.fromJson("{\"value\":1e-100}", Holder.class).value); + assertScaleRejected(jsonb, "{\"value\":1e-101}"); + } + try (final Jsonb jsonb = JsonbBuilder.newBuilder().withProvider(new org.apache.johnzon.core.JsonProviderImpl()) + .withConfig(new JsonbConfig().setProperty("johnzon.max-big-decimal-scale", 2)).build()) { + assertScaleRejected(jsonb, "{\"value\":1e3}"); + } + } + + @Test + public void fromJsonValueChecksExternallySuppliedNumbers() throws Exception { + final JsonProviderImpl permissiveProvider = new JsonProviderImpl(); + permissiveProvider.setMaxBigDecimalScale(100); + final JsonNumber accepted = permissiveProvider.createValue(new BigDecimal("1e2")); + final JsonNumber rejected = permissiveProvider.createValue(new BigDecimal("1e3")); + assertEquals(new BigDecimal("1e3"), rejected.bigDecimalValue()); + + try (final Jsonb jsonb = JsonbBuilder.create(new JsonbConfig() + .setProperty("johnzon.max-big-decimal-scale", 2))) { + final JsonbExtension extension = JsonbExtension.class.cast(jsonb); + assertEquals(new BigDecimal("1e2"), extension.fromJsonValue(accepted, BigDecimal.class)); + assertEquals(new BigDecimal("1e2"), extension.fromJsonValue(accepted, Number.class)); + assertEquals(new BigInteger("100"), extension.fromJsonValue(accepted, BigInteger.class)); + + assertScaleRejected(() -> extension.fromJsonValue(rejected, BigDecimal.class)); + assertScaleRejected(() -> extension.fromJsonValue(rejected, Number.class)); + assertScaleRejected(() -> extension.fromJsonValue(rejected, BigInteger.class)); + } + } + + @Test + public void formattedNumberChecksConfiguredScaleBeforeFormatting() throws Exception { + try (final Jsonb jsonb = JsonbBuilder.create(new JsonbConfig() + .setProperty("johnzon.max-big-decimal-scale", 3))) { + final FormattedHolder holder = new FormattedHolder(); + holder.value = new BigDecimal("1e3"); + assertEquals("{\"value\":\"1000\"}", jsonb.toJson(holder)); + + holder.value = new BigDecimal("1e4"); + assertScaleRejected(() -> jsonb.toJson(holder)); + } + } + + private static void assertScaleRejected(final Jsonb jsonb, final String json) { + try { + jsonb.fromJson(json, Holder.class); + } catch (final RuntimeException e) { + Throwable t = e; + while (t != null && !(t instanceof ArithmeticException)) { + t = t.getCause(); + } + assertNotNull("expected a scale rejection for " + json + " but got " + e, t); + return; + } + fail("expected " + json + " to be rejected"); + } + + private static void assertScaleRejected(final Runnable action) { + try { + action.run(); + } catch (final RuntimeException e) { + Throwable t = e; + while (t != null && !(t instanceof ArithmeticException)) { + t = t.getCause(); + } + assertNotNull("expected a scale rejection but got " + e, t); + return; + } + fail("expected a scale rejection"); + } + + public static class Holder { + public BigDecimal value; + } + + public static class FormattedHolder { + @JsonbNumberFormat("0") + public BigDecimal value; + } +} diff --git a/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/MapperBuilder.java b/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/MapperBuilder.java index 9b088963..5023e726 100644 --- a/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/MapperBuilder.java +++ b/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/MapperBuilder.java @@ -30,6 +30,7 @@ import org.apache.johnzon.mapper.access.FieldAccessMode; import org.apache.johnzon.mapper.access.FieldAndMethodAccessMode; import org.apache.johnzon.mapper.access.MethodAccessMode; import org.apache.johnzon.mapper.access.KnownNotOpenedJavaTypesAccessMode; +import org.apache.johnzon.mapper.converter.BigDecimalConverter; import org.apache.johnzon.mapper.converter.BooleanConverter; import org.apache.johnzon.mapper.converter.ByteConverter; import org.apache.johnzon.mapper.converter.CachedDelegateConverter; @@ -105,7 +106,10 @@ public class MapperBuilder { private Boolean deduplicateObjects = null; private boolean useJsRange; private boolean useBigDecimalForObjectNumbers; - private int maxBigDecimalScale = 1000; + private int maxBigDecimalScale = BigDecimalConverter.DEFAULT_MAX_SCALE; + private boolean adaptersShared; + private JsonProvider createdProvider; // owned by this builder, so it carries maxBigDecimalScale + private final Collection<Object> createdFactories = new ArrayList<>(); private boolean supportEnumContainerDeserialization = true; private Function<Class<?>, MapperConfig.CustomEnumConverter<?>> enumConverterFactory = type -> new EnumConverter(type); private boolean skipAccessModeWrapper; @@ -127,6 +131,7 @@ public class MapperBuilder { provider = JsonProvider.provider(); JsonProviderUtil.setMaxBigDecimalScale(provider, maxBigDecimalScale); this.provider = provider; + createdProvider = provider; } final Map<String, Object> config = new HashMap<String, Object>(); if (bufferStrategy != null) { @@ -141,6 +146,7 @@ public class MapperBuilder { config.put(JsonGenerator.PRETTY_PRINTING, true); } generatorFactory = provider.createGeneratorFactory(config); + createdFactories.add(generatorFactory); } if (readerFactory == null) { @@ -158,13 +164,16 @@ public class MapperBuilder { config.put("org.apache.johnzon.auto-adjust-buffer", true); } readerFactory = provider.createReaderFactory(config); + createdFactories.add(readerFactory); } } else if (this.provider == null) { this.provider = JsonProvider.provider(); JsonProviderUtil.setMaxBigDecimalScale(provider, maxBigDecimalScale); + createdProvider = provider; } if (builderFactory == null) { builderFactory = provider.createBuilderFactory(emptyMap()); + createdFactories.add(builderFactory); } if (accessMode == null) { @@ -229,6 +238,7 @@ public class MapperBuilder { adapters.put(new AdapterKey(boolean.class, String.class), adapters.get(new AdapterKey(Boolean.class, String.class))); } + adaptersShared = true; return new Mapper( readerFactory, generatorFactory, builderFactory, provider, new MapperConfig( @@ -545,8 +555,32 @@ public class MapperBuilder { return this; } + /** + * Sets the absolute scale limit for BigDecimal values read through the mapper's built-in conversions. + * JSON numbers must also pass the JSON-P provider's limit; a supplied provider or reader factory + * keeps its own limit, so the stricter one applies. A registered {@link BigDecimalConverter} + * adds its own limit for JSON numbers and replaces this one for JSON strings. + * Custom adapters are responsible for validating values they create themselves. + * The limit also applies before the default BigDecimal string adapter writes plain notation. + * @param maxBigDecimalScale the maximum absolute scale + * @return this builder + */ public MapperBuilder setMaxBigDecimalScale(final int maxBigDecimalScale) { + if (this.maxBigDecimalScale != maxBigDecimalScale && adaptersShared) { + // mappers built so far keep their own limit + adapters = adapters.copy(); + adaptersShared = false; + if (provider != null && provider == createdProvider) { + provider = null; + readerFactory = createdFactories.contains(readerFactory) ? null : readerFactory; + generatorFactory = createdFactories.contains(generatorFactory) ? null : generatorFactory; + builderFactory = createdFactories.contains(builderFactory) ? null : builderFactory; + } + createdProvider = null; + createdFactories.clear(); + } this.maxBigDecimalScale = maxBigDecimalScale; + adapters.setMaxBigDecimalScale(maxBigDecimalScale); return this; } @@ -589,4 +623,4 @@ public class MapperBuilder { this.mappingsFactory = mappingsFactory; return this; } -} \ No newline at end of file +} diff --git a/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/MappingParserImpl.java b/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/MappingParserImpl.java index 5969ea01..f1a54fc4 100644 --- a/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/MappingParserImpl.java +++ b/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/MappingParserImpl.java @@ -19,6 +19,7 @@ package org.apache.johnzon.mapper; import org.apache.johnzon.mapper.access.AccessMode; +import org.apache.johnzon.mapper.converter.BigDecimalConverter; import org.apache.johnzon.mapper.converter.CharacterConverter; import org.apache.johnzon.mapper.internal.AdapterKey; import org.apache.johnzon.mapper.internal.ConverterAdapter; @@ -202,10 +203,10 @@ public class MappingParserImpl implements MappingParser { return (T) Short.valueOf((short) number.intValue()); } if (targetType == BigDecimal.class || Number.class == targetType) { - return (T) number.bigDecimalValue(); + return (T) toBigDecimal(number); } if (targetType == BigInteger.class) { - return (T) number.bigIntegerValue(); + return (T) toBigDecimal(number).toBigInteger(); } } if (JsonArray.class.isInstance(jsonValue)) { @@ -318,7 +319,7 @@ public class MappingParserImpl implements MappingParser { final JsonValue jsonValue = value.getValue(); if (JsonNumber.class.isInstance(jsonValue) && any) { map.put(value.getKey(), config.isUseBigDecimalForObjectNumbers() ? - JsonNumber.class.cast(jsonValue).bigDecimalValue() : toNumberValue(JsonNumber.class.cast(jsonValue))); + toBigDecimal(JsonNumber.class.cast(jsonValue)) : toNumberValue(JsonNumber.class.cast(jsonValue))); } else if (JsonString.class.isInstance(jsonValue) && any) { map.put(value.getKey(), JsonString.class.cast(jsonValue).getString()); } else { @@ -513,11 +514,18 @@ public class MappingParserImpl implements MappingParser { return longValue; } if (config.isUseBigDecimalForFloats()) { - return jsonNumber.bigDecimalValue(); + return toBigDecimal(jsonNumber); } return jsonNumber.doubleValue(); } + // bigDecimalValue() applies the provider limit, which differs from the mapper one when the provider or reader factory is supplied + private BigDecimal toBigDecimal(final JsonNumber number) { + final BigDecimal value = number.bigDecimalValue(); + BigDecimalConverter.checkScale(value, config.getMaxBigDecimalScale()); + return value; + } + private Object convertTo(final Adapter converter, final JsonValue jsonValue, final JsonPointerTracker jsonPointer, final Type targetType) { final JsonValue.ValueType valueType = jsonValue != null ? jsonValue.getValueType() : null; @@ -562,6 +570,12 @@ public class MappingParserImpl implements MappingParser { } } if (NUMBER.equals(valueType)) { + if (ConverterAdapter.class.isInstance(converter) + && BigDecimalConverter.class == ConverterAdapter.class.cast(converter).getConverter().getClass()) { + final BigDecimal value = toBigDecimal(JsonNumber.class.cast(jsonValue)); + BigDecimalConverter.class.cast(ConverterAdapter.class.cast(converter).getConverter()).checkScale(value); + return value; + } if (key != null) { if (Long.class == key.getTo() || long.class == key.getTo()) { return converter.to (JsonNumber.class.cast(jsonValue).longValue()); @@ -572,9 +586,9 @@ public class MappingParserImpl implements MappingParser { } else if (Float.class == key.getTo() || float.class == key.getTo()) { return converter.to (JsonNumber.class.cast(jsonValue).doubleValue()); } else if (BigInteger.class == key.getTo()) { - return converter.to (JsonNumber.class.cast(jsonValue).bigIntegerValue()); + return converter.to (toBigDecimal(JsonNumber.class.cast(jsonValue)).toBigInteger()); } else if (BigDecimal.class == key.getTo() || Number.class == key.getTo()) { - return converter.to (JsonNumber.class.cast(jsonValue).bigDecimalValue()); + return converter.to (toBigDecimal(JsonNumber.class.cast(jsonValue))); } else if (JsonNumber.class == key.getTo()) { return converter.to(JsonNumber.class.cast(jsonValue)); } @@ -719,11 +733,11 @@ public class MappingParserImpl implements MappingParser { } if (type == BigInteger.class) { - return number.bigIntegerValue(); + return toBigDecimal(number).toBigInteger(); } if (type == BigDecimal.class || Number.class == type) { - return number.bigDecimalValue(); + return toBigDecimal(number); } if (type == Integer.class || type == int.class) { diff --git a/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/converter/BigDecimalConverter.java b/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/converter/BigDecimalConverter.java index 891f60b1..397f2fb7 100644 --- a/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/converter/BigDecimalConverter.java +++ b/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/converter/BigDecimalConverter.java @@ -23,15 +23,53 @@ import org.apache.johnzon.mapper.Converter; import java.math.BigDecimal; public class BigDecimalConverter implements Converter<BigDecimal> { + public static final int DEFAULT_MAX_SCALE = Integer.getInteger("johnzon.max-big-decimal-scale", 1000); + + private final int maxScale; + + public BigDecimalConverter() { + this(DEFAULT_MAX_SCALE); + } + + // parses JSON strings the JSON-P provider never saw as numbers, so its max-big-decimal-scale check does not apply + public BigDecimalConverter(final int maxScale) { + this.maxScale = maxScale; + } + @Override public String toString(final BigDecimal instance) { // when using the converter, user expects the decimal notation // otherwise, JsonNumber will give the E (scientific) notation + checkScale(instance); return instance.toPlainString(); } @Override public BigDecimal fromString(final String text) { - return new BigDecimal(text); + final BigDecimal value = new BigDecimal(text); + checkScale(value); + return value; + } + + /** + * Checks a decimal obtained without parsing a string, for example from a JSON-P provider. + * @param value the decimal to validate + * @throws ArithmeticException if its absolute scale exceeds this converter's limit + */ + public void checkScale(final BigDecimal value) { + checkScale(value, maxScale); + } + + /** + * @param value the decimal to validate + * @param maxScale the maximum absolute scale + * @throws ArithmeticException if the absolute scale of the value exceeds maxScale + */ + public static void checkScale(final BigDecimal value, final int maxScale) { + if (Math.abs((long) value.scale()) > maxScale) { + throw new ArithmeticException(String.format( + "BigDecimal scale (%d) limit exceeds maximum allowed (%d)", + value.scale(), maxScale)); + } } } diff --git a/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/map/LazyConverterMap.java b/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/map/LazyConverterMap.java index 363bad23..e7419cd1 100644 --- a/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/map/LazyConverterMap.java +++ b/johnzon-mapper/src/main/java/org/apache/johnzon/mapper/map/LazyConverterMap.java @@ -89,6 +89,7 @@ public class LazyConverterMap extends ConcurrentHashMap<AdapterKey, Adapter<?, ? private static final boolean IJSON_BIG_NUMBER_DEFAULT = !Boolean.getBoolean("johnzon.use-big-number-stringadapter.disabled"); private boolean useBigIntegerStringAdapter = IJSON_BIG_NUMBER_DEFAULT; private boolean useBigDecimalStringAdapter = IJSON_BIG_NUMBER_DEFAULT; + private int maxBigDecimalScale = BigDecimalConverter.DEFAULT_MAX_SCALE; public void setUseShortISO8601Format(final boolean useShortISO8601Format) { this.useShortISO8601Format = useShortISO8601Format; @@ -106,6 +107,26 @@ public class LazyConverterMap extends ConcurrentHashMap<AdapterKey, Adapter<?, ? this.useBigIntegerStringAdapter = useBigIntegerStringAdapter; } + public void setMaxBigDecimalScale(final int maxBigDecimalScale) { + if (this.maxBigDecimalScale == maxBigDecimalScale) { + return; + } + this.maxBigDecimalScale = maxBigDecimalScale; + replaceAll((key, adapter) -> adapter instanceof BigDecimalStringAdapter + ? new BigDecimalStringAdapter(maxBigDecimalScale) : adapter); + } + + public LazyConverterMap copy() { + final LazyConverterMap copy = new LazyConverterMap(); + copy.useShortISO8601Format = useShortISO8601Format; + copy.dateTimeFormatter = dateTimeFormatter; + copy.useBigIntegerStringAdapter = useBigIntegerStringAdapter; + copy.useBigDecimalStringAdapter = useBigDecimalStringAdapter; + copy.maxBigDecimalScale = maxBigDecimalScale; + forEach(copy::put); + return copy; + } + @Override public Adapter<?, ?> get(final Object key) { final Adapter<?, ?> found = super.get(key); @@ -166,7 +187,7 @@ public class LazyConverterMap extends ConcurrentHashMap<AdapterKey, Adapter<?, ? return add(key, new ConverterAdapter<>(new StringConverter(), String.class)); } if (from == BigDecimal.class && useBigDecimalStringAdapter) { - return add(key, new ConverterAdapter<>(new BigDecimalConverter(), BigDecimal.class)); + return add(key, new BigDecimalStringAdapter(maxBigDecimalScale)); } if (from == BigInteger.class && useBigIntegerStringAdapter) { return add(key, new ConverterAdapter<>(new BigIntegerConverter(), BigInteger.class)); @@ -608,6 +629,12 @@ public class LazyConverterMap extends ConcurrentHashMap<AdapterKey, Adapter<?, ? (instance.isSet(Calendar.SECOND) && instance.get(Calendar.SECOND) != 0); } + private static class BigDecimalStringAdapter extends ConverterAdapter<BigDecimal> { + private BigDecimalStringAdapter(final int maxScale) { + super(new BigDecimalConverter(maxScale), BigDecimal.class); + } + } + private <T extends Calendar> T fromCalendar(final String text, final Function<ZonedDateTime, T> calendarSupplier) { switch (text.length()) { case 10: { diff --git a/johnzon-mapper/src/test/java/org/apache/johnzon/mapper/NumberSerializationTest.java b/johnzon-mapper/src/test/java/org/apache/johnzon/mapper/NumberSerializationTest.java index 4f2b01d7..8e07e64f 100644 --- a/johnzon-mapper/src/test/java/org/apache/johnzon/mapper/NumberSerializationTest.java +++ b/johnzon-mapper/src/test/java/org/apache/johnzon/mapper/NumberSerializationTest.java @@ -67,6 +67,41 @@ public class NumberSerializationTest { "0.000000733915", result); } + @Test + public void bigDecimalConverterChecksScaleBeforePlainString() { + final BigDecimalConverter converter = new BigDecimalConverter(2); + assertEquals("100", converter.toString(new BigDecimal("1e2"))); + assertEquals("0.01", converter.toString(new BigDecimal("1e-2"))); + assertRejected(() -> converter.toString(new BigDecimal("1e3"))); + assertRejected(() -> converter.toString(new BigDecimal("1e-3"))); + } + + @Test + public void bigDecimalStringAdapterChecksScaleWhenWriting() { + try (final Mapper mapper = new MapperBuilder().setMaxBigDecimalScale(2).build()) { + final BigDecimalHolder holder = new BigDecimalHolder(); + holder.score = new BigDecimal("1e2"); + assertEquals("{\"score\":\"100\"}", mapper.writeObjectAsString(holder)); + holder.score = new BigDecimal("1e3"); + assertRejected(() -> mapper.writeObjectAsString(holder)); + } + } + + @Test + public void bigIntegerFromJsonValueChecksMapperScale() { + final org.apache.johnzon.core.JsonProviderImpl sourceProvider = new org.apache.johnzon.core.JsonProviderImpl(); + sourceProvider.setMaxBigDecimalScale(2000); + try (final Mapper mapper = new MapperBuilder().setMaxBigDecimalScale(2) + .setUseBigIntegerStringAdapter(false).build()) { + assertEquals(new BigInteger("100"), mapper.readObject( + sourceProvider.createValue(new BigDecimal("1e2")), BigInteger.class)); + assertRejected(() -> mapper.readObject( + sourceProvider.createValue(new BigDecimal("1e3")), BigInteger.class)); + assertRejected(() -> mapper.readObject(sourceProvider.createObjectBuilder() + .add("value", new BigDecimal("1e3")).build(), BigIntegerHolder.class)); + } + } + /** * Bug fix: useBigDecimalStringAdapter and useBigIntegerStringAdapter flags * were swapped in LazyConverterMap. Each flag must control its own type. @@ -166,6 +201,202 @@ public class NumberSerializationTest { } } + @Test + public void bigDecimalNumberPropertyRejectsScaleAboveDefaultLimit() { + try (final Mapper mapper = new MapperBuilder().build()) { + assertScaleRejected(mapper, "{\"score\":1e1001}"); + assertScaleRejected(mapper, "{\"score\":1e-1001}"); + } + } + + @Test + public void bigDecimalStringPropertyRejectsScaleAboveDefaultLimit() { + try (final Mapper mapper = new MapperBuilder().build()) { + assertScaleRejected(mapper, "{\"score\":\"1e1001\"}"); + assertScaleRejected(mapper, "{\"score\":\"1E+1001\"}"); + } + } + + @Test + public void bigDecimalNumberPropertyRejectsScaleWithoutAdapter() { + try (final Mapper mapper = new MapperBuilder().setUseBigDecimalStringAdapter(false).build()) { + assertScaleRejected(mapper, "{\"score\":1e1001}"); + } + } + + @Test + public void bigDecimalPropertyWithinScale() { + try (final Mapper mapper = new MapperBuilder().build()) { + assertEquals(new BigDecimal("1.25"), readScore(mapper, "{\"score\":1.25}")); + assertEquals(new BigDecimal("1.25"), readScore(mapper, "{\"score\":\"1.25\"}")); + assertEquals(new BigDecimal("1e1000"), readScore(mapper, "{\"score\":1e1000}")); + assertEquals(new BigDecimal("1e1000"), readScore(mapper, "{\"score\":\"1e1000\"}")); + } + } + + @Test + public void bigDecimalPropertyConfiguredScale() { + try (final Mapper mapper = new MapperBuilder().setMaxBigDecimalScale(2000).build()) { + assertEquals(new BigDecimal("1e2000"), readScore(mapper, "{\"score\":1e2000}")); + assertEquals(new BigDecimal("1e2000"), readScore(mapper, "{\"score\":\"1e2000\"}")); + assertScaleRejected(mapper, "{\"score\":\"1e2001\"}"); + } + } + + @Test + public void minimumBigDecimalScaleRejected() { + final BigDecimal value = new BigDecimal(BigInteger.ONE, Integer.MIN_VALUE); + try { + new BigDecimalConverter(1000).checkScale(value); + fail("expected the minimum scale to be rejected"); + } catch (final ArithmeticException expected) { + // The scale's absolute value must not overflow an int. + } + } + + @Test + public void numericBigDecimalHonorsConverterLimit() { + try (final Mapper mapper = new MapperBuilder() + .addConverter(BigDecimal.class, new BigDecimalConverter(2)).build()) { + assertEquals(new BigDecimal("1e2"), readScore(mapper, "{\"score\":1e2}")); + assertEquals(new BigDecimal("1e-2"), readScore(mapper, "{\"score\":1e-2}")); + assertScaleRejected(mapper, "{\"score\":1e3}"); + assertScaleRejected(mapper, "{\"score\":1e-3}"); + assertScaleRejected(mapper, "{\"score\":\"1e3\"}"); + } + } + + @Test + public void numericBigDecimalHonorsMapperLimitWithExplicitProvider() { + final org.apache.johnzon.core.JsonProviderImpl provider = new org.apache.johnzon.core.JsonProviderImpl(); + try (final Mapper mapper = new MapperBuilder().setProvider(provider).setMaxBigDecimalScale(2).build()) { + assertScaleRejected(mapper, "{\"score\":1e3}"); + assertScaleRejected(mapper, "{\"score\":\"1e3\"}"); + assertEquals(1000, provider.getMaxBigDecimalScale()); + } + } + + @Test + public void cachedBigDecimalAdapterHonorsScaleChanges() { + final MapperBuilder builder = new MapperBuilder().setMaxBigDecimalScale(4); + builder.getAdapters().get(new AdapterKey(BigDecimal.class, String.class)); + try (final Mapper mapper = builder.setMaxBigDecimalScale(2).build()) { + assertScaleRejected(mapper, "{\"score\":\"1e3\"}"); + } + try (final Mapper mapper = builder.setMaxBigDecimalScale(4).build()) { + assertEquals(new BigDecimal("1e3"), readScore(mapper, "{\"score\":\"1e3\"}")); + } + try (final Mapper mapper = builder.setMaxBigDecimalScale(2).build()) { + assertScaleRejected(mapper, "{\"score\":\"1e3\"}"); + } + } + + @Test + public void earlyBigDecimalAdapterHasDefaultLimit() { + final MapperBuilder builder = new MapperBuilder(); + final Adapter<?, ?> adapter = builder.getAdapters().get(new AdapterKey(BigDecimal.class, String.class)); + try { + ((Adapter<BigDecimal, String>) adapter).to("1e1001"); + fail("expected the default scale limit before build"); + } catch (final ArithmeticException expected) { + // An early lookup must not create an unlimited converter. + } + try (final Mapper mapper = builder.build()) { + assertScaleRejected(mapper, "{\"score\":\"1e1001\"}"); + } + } + + @Test + public void scaleChangesPreserveRegisteredConverter() { + try (final Mapper mapper = new MapperBuilder() + .addConverter(BigDecimal.class, new BigDecimalConverter(4)).setMaxBigDecimalScale(2).build()) { + assertEquals(new BigDecimal("1e3"), readScore(mapper, "{\"score\":\"1e3\"}")); + assertScaleRejected(mapper, "{\"score\":\"1e5\"}"); + } + } + + @Test + public void numericBigDecimalHonorsStricterProviderLimit() { + final org.apache.johnzon.core.JsonProviderImpl provider = new org.apache.johnzon.core.JsonProviderImpl(); + provider.setMaxBigDecimalScale(100); + try (final Mapper mapper = new MapperBuilder().setProvider(provider).setMaxBigDecimalScale(1000).build()) { + assertEquals(new BigDecimal("1e-100"), readScore(mapper, "{\"score\":1e-100}")); + assertScaleRejected(mapper, "{\"score\":1e-101}"); + assertRejected(() -> mapper.readObject("{\"scores\":[1e-101]}", BigDecimalListHolder.class)); + } + } + + @Test + public void builtMapperKeepsScaleWhenBuilderChanges() { + final MapperBuilder builder = new MapperBuilder().setMaxBigDecimalScale(2); + try (final Mapper first = builder.build()) { + try (final Mapper second = builder.setMaxBigDecimalScale(4).build()) { + assertScaleRejected(first, "{\"score\":\"1e3\"}"); + assertScaleRejected(first, "{\"score\":1e3}"); + assertEquals(new BigDecimal("1e3"), readScore(second, "{\"score\":\"1e3\"}")); + assertEquals(new BigDecimal("1e3"), readScore(second, "{\"score\":1e3}")); + } + } + } + + @Test + public void nestedBigDecimalsHonorMapperLimit() { + try (final Mapper mapper = new MapperBuilder().setProvider(new org.apache.johnzon.core.JsonProviderImpl()) + .setMaxBigDecimalScale(2).setUseBigDecimalForObjectNumbers(true).build()) { + final BigDecimalListHolder list = mapper.readObject("{\"scores\":[1e2]}", BigDecimalListHolder.class); + assertEquals(new BigDecimal("1e2"), list.scores.get(0)); + assertRejected(() -> mapper.readObject("{\"scores\":[1e3]}", BigDecimalListHolder.class)); + assertRejected(() -> mapper.readObject("{\"scores\":[1e3]}", BigDecimalArrayHolder.class)); + assertRejected(() -> mapper.readObject("{\"scores\":{\"a\":1e3}}", BigDecimalMapHolder.class)); + assertRejected(() -> mapper.readObject("{\"scores\":{\"a\":1e3}}", ObjectMapHolder.class)); + assertRejected(() -> mapper.readObject("1e3", BigDecimal.class)); + } + try (final Mapper mapper = new MapperBuilder().setMaxBigDecimalScale(2).setUseBigDecimalStringAdapter(false).build()) { + assertScaleRejected(mapper, "{\"score\":1e3}"); + } + } + + @Test + public void defaultBigDecimalConverterIsBounded() { + try (final Mapper mapper = new MapperBuilder().addConverter(BigDecimal.class, new BigDecimalConverter()).build()) { + assertEquals(new BigDecimal("1e1000"), readScore(mapper, "{\"score\":\"1e1000\"}")); + assertScaleRejected(mapper, "{\"score\":\"1e1001\"}"); + } + } + + private static void assertRejected(final Runnable read) { + try { + read.run(); + } catch (final RuntimeException e) { + Throwable t = e; + while (t != null && !ArithmeticException.class.isInstance(t)) { + t = t.getCause(); + } + assertNotNull("expected a scale rejection but got " + e, t); + return; + } + fail("expected a scale rejection"); + } + + private static BigDecimal readScore(final Mapper mapper, final String json) { + final BigDecimalHolder holder = mapper.readObject(json, BigDecimalHolder.class); + return holder.score; + } + + private static void assertScaleRejected(final Mapper mapper, final String json) { + try { + mapper.readObject(json, BigDecimalHolder.class); + } catch (final RuntimeException e) { + Throwable t = e; + while (t != null && !ArithmeticException.class.isInstance(t)) { + t = t.getCause(); + } + assertNotNull("expected a scale rejection for " + json + " but got " + e, t); + return; + } + fail("expected " + json + " to be rejected"); + } + public static class Holder { public long value; } @@ -177,4 +408,24 @@ public class NumberSerializationTest { public static class BigDecimalHolder { public BigDecimal score; } + + public static class BigIntegerHolder { + public BigInteger value; + } + + public static class BigDecimalListHolder { + public java.util.List<BigDecimal> scores; + } + + public static class BigDecimalArrayHolder { + public BigDecimal[] scores; + } + + public static class BigDecimalMapHolder { + public java.util.Map<String, BigDecimal> scores; + } + + public static class ObjectMapHolder { + public java.util.Map<String, Object> scores; + } } diff --git a/src/site/markdown/index.md b/src/site/markdown/index.md index 2c86a27c..c9a56815 100644 --- a/src/site/markdown/index.md +++ b/src/site/markdown/index.md @@ -360,6 +360,7 @@ to use for deserialization. JsonbConfig specific properties: * johnzon.use-big-decimal-for-object: true to use BigDecimal for numbers not typed (Object), false to adjust the type to the number size, true by default. +* johnzon.max-big-decimal-scale: Maximum absolute `BigDecimal` scale for built-in conversions, 1000 by default. * johnzon.support-enum-container-deserialization: prevent EnumMap/EnumSet instantiation, true by default. * johnzon.attributeOrder: Comparator instance to sort properties by name. * johnzon.deduplicateObjects: should instances be deduplicated. diff --git a/src/site/markdown/security.md b/src/site/markdown/security.md index afed84ae..86cd142b 100644 --- a/src/site/markdown/security.md +++ b/src/site/markdown/security.md @@ -40,14 +40,23 @@ The private security mailing address is: security (at) apache (dot) org Known vulnerabilities are listed on the [vulnerabilities](./vulnerabilities.html) page. -## BigInteger and Java - -JSON-P/JSON-B exposes API using `BigDecimal` and `BigInteger`. -The bridge between these two types is `BigDecimal#toBigInteger` which has a slow implementation in Java without careness or scale max validation. - -Johnzon does some sanity checks on this value but at some point we recommend you to stay away from these API and handle big numbers using `String` type and parse them yourself since you are the only ones knowing the correct functional and relevant validation of the scale before a instantiation. - -If you know you don't need such big types, prefer using plain primitives (or wrappers). +<a id="BigInteger_and_Java"></a> +## BigDecimal and BigInteger + +JSON-P, Mapper, and JSON-B can represent JSON numbers as `BigDecimal` or `BigInteger`. +Converting a `BigDecimal` with a large absolute scale to `BigInteger` or plain decimal notation can take substantial time and memory. + +Johnzon's built-in numeric conversions check the absolute `BigDecimal` scale before these operations. +The default maximum is 1000. The `johnzon.max-big-decimal-scale` system property changes this default; +`JsonProviderImpl#setMaxBigDecimalScale`, `MapperBuilder#setMaxBigDecimalScale`, and the JSON-B +`johnzon.max-big-decimal-scale` property configure their respective components. On built-in paths, +JSON numbers converted to `BigDecimal` or `BigInteger` through a separately configured Johnzon JSON-P +provider must satisfy both the provider's limit and the Mapper's limit. + +The scale limit does not restrict the number of digits or the total JSON input size. Apply suitable input-size +limits and validate numeric range and precision for your application. Custom adapters are responsible for +validating numbers they construct or format. If you do not need large numeric types, prefer primitive types +or their wrappers. ## JSON Schema validation diff --git a/src/site/markdown/vulnerabilities.md b/src/site/markdown/vulnerabilities.md index 2ef6f360..715d5e8c 100644 --- a/src/site/markdown/vulnerabilities.md +++ b/src/site/markdown/vulnerabilities.md @@ -41,4 +41,4 @@ Exploitation needs a mapped POJO with a `Class` attribute and a class with a har * **Announcement:** [lists.apache.org]( https://lists.apache.org/thread/qbg14djo95gfpk7o560lr8wcrzfyw43l) Denial of service: JSON input with very large numbers (e.g. `1e20000000`) deserialized into `BigDecimal` can cause a slow conversion. -Johnzon 1.2.21 limits the `BigDecimal` scale to 1000 by default. See also [BigInteger and Java](./security.html#BigInteger_and_Java). +Johnzon 1.2.21 limits the `BigDecimal` scale to 1000 by default. See also [BigDecimal and BigInteger](./security.html#BigDecimal_and_BigInteger).
