This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/fory.git
The following commit(s) were added to refs/heads/main by this push:
new 93a30f51b feat(java): add Json object field cache (#3862)
93a30f51b is described below
commit 93a30f51b66833e6a8893556468e8deb2ded995b
Author: Shawn Yang <[email protected]>
AuthorDate: Thu Jul 16 14:19:18 2026 +0530
feat(java): add Json object field cache (#3862)
## Why?
## What does this PR do?
## Related issues
## AI Contribution Checklist
- [ ] Substantial AI assistance was used in this PR: `yes` / `no`
- [ ] If `yes`, I included a completed [AI Contribution
Checklist](https://github.com/apache/fory/blob/main/AI_POLICY.md#9-contributor-checklist-for-ai-assisted-prs)
in this PR description and the required `AI Usage Disclosure`.
- [ ] If `yes`, my PR description includes the required `ai_review`
summary and screenshot evidence or equivalent persisted links of the
final clean AI review results from both fresh reviewers described in
`AI_POLICY.md`, the Fory-guided reviewer and the independent general
reviewer, on the current PR diff or current HEAD after the latest code
changes.
## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
---
docs/guide/java/json-support.md | 9 +-
java/fory-json/README.md | 11 +-
.../main/java/org/apache/fory/json/ForyJson.java | 3 +
.../java/org/apache/fory/json/ForyJsonBuilder.java | 25 +-
.../main/java/org/apache/fory/json/JsonConfig.java | 23 +-
.../java/org/apache/fory/json/codec/MapCodec.java | 22 +-
.../apache/fory/json/reader/FieldNameCache.java | 123 ++++++
.../org/apache/fory/json/reader/JsonReader.java | 32 +-
.../apache/fory/json/reader/Latin1JsonReader.java | 144 ++++++
.../apache/fory/json/reader/Utf16JsonReader.java | 186 +++++++-
.../apache/fory/json/reader/Utf8JsonReader.java | 123 ++++++
.../fory/json/resolver/JsonSharedRegistry.java | 47 +-
.../fory/json/resolver/JsonTypeResolver.java | 6 +
.../org/apache/fory/json/JsonAnyPropertyTest.java | 13 +
.../apache/fory/json/JsonAsyncCompilationTest.java | 1 +
.../apache/fory/json/JsonFieldNameCacheTest.java | 481 +++++++++++++++++++++
.../java/org/apache/fory/json/JsonTestSupport.java | 12 +
.../fory/json/reader/FieldNameCacheTest.java | 105 +++++
18 files changed, 1339 insertions(+), 27 deletions(-)
diff --git a/docs/guide/java/json-support.md b/docs/guide/java/json-support.md
index 2ab106e3f..042320821 100644
--- a/docs/guide/java/json-support.md
+++ b/docs/guide/java/json-support.md
@@ -291,13 +291,16 @@ are rejected.
| `withPropertyNamingStrategy` | `LOWER_CAMEL_CASE`
| Naming of properties without explicit names |
| `withClassLoader` | Snapshotted context loader, then Fory loader
| Resolve annotation subtype class names |
| `maxDepth` | `20`
| Maximum nested object/array depth |
+| `withMaxCachedFieldNames` | `DEFAULT_MAX_CACHED_FIELD_NAMES` (`8192`)
| Field-name cache entries per reader; zero disables it |
| `withConcurrencyLevel` | `max(1, 2 * processors)`
| Reusable operation-state count |
| `withBufferSizeLimitBytes` | 2 MiB
| Reusable capacity retained by each pooled writer |
| `registerCodec` | None
| Exact-class complete-value codec |
| `withTypeChecker` | None
| Application policy in addition to Fory's disallow list |
-Depth, concurrency, and retained buffer limits must be positive. The buffer
setting does not limit
-output size. Builder changes after `build()` do not mutate an existing runtime.
+Depth, concurrency, and retained buffer limits must be positive. The
cached-field-name limit applies
+independently to each reader; zero disables the cache, and the setting does
not limit accepted
+input. The buffer setting does not limit output size. Builder changes after
`build()` do not mutate
+an existing runtime.
In a GraalVM native image, runtime code generation and asynchronous
compilation are automatically
disabled. Every other builder option keeps the behavior described above.
@@ -1108,7 +1111,7 @@ Circular graphs eventually fail `maxDepth`; they are not
reconstructed.
| ---------------------------------- |
-----------------------------------------------------------------------------------------------
|
| `ForyJsonException` | Check JSON grammar, target type,
mapping support, depth, trailing content, or output cause |
| `InsecureException` | Check Fory's disallow list and the
configured type checker |
-| Builder `IllegalArgumentException` | Use positive depth, concurrency, and
retained-buffer values |
+| Builder `IllegalArgumentException` | Check the configured depth,
concurrency, retained-buffer, and cached-field-name limits |
| Declared write fails | Remove wildcard/type variables and pass
an assignable value; primitive declarations reject null |
| Immutable value is empty | Use a record, valid creator, or custom
codec |
| `JsonValue` read fails | Add one plain `String` creator, or
register an exact custom codec |
diff --git a/java/fory-json/README.md b/java/fory-json/README.md
index 0bb3c34a7..ed7503975 100644
--- a/java/fory-json/README.md
+++ b/java/fory-json/README.md
@@ -345,14 +345,17 @@ original key type. Null map keys are rejected.
| `withPropertyNamingStrategy(strategy)` | `LOWER_CAMEL_CASE`
| Name properties without an explicit `JsonProperty` name
|
| `withClassLoader(loader)` | Snapshotted thread context loader,
then Fory JSON loader | Resolve annotation-declared subtype class names
|
| `maxDepth(int)` | `20`
| Maximum nested object/array depth for reads and writes
|
+| `withMaxCachedFieldNames(int)` | `DEFAULT_MAX_CACHED_FIELD_NAMES`
(`8192`) | Field-name cache entries per reader; zero disables
caching |
| `withConcurrencyLevel(int)` | `max(1, 2 * processors)`
| Number of reusable concurrent operation states
|
| `withBufferSizeLimitBytes(int)` | 2 MiB
| Maximum reusable capacity retained by each pooled writer
|
| `registerCodec(type, codec)` | None
| Replace the exact class's complete JSON codec
|
| `withTypeChecker(checker)` | No custom checker
| Apply an application type policy in addition to Fory's
disallow list |
-Depth, concurrency level, and buffer retention limit must be positive. The
buffer retention setting
-does not limit JSON input or output size; it only limits reusable writer
storage retained after an
-operation. Apply request/body size limits at the transport boundary when
parsing untrusted input.
+Depth, concurrency level, and buffer retention limit must be positive. The
cached-field-name limit
+applies independently to each reader. It does not limit accepted JSON input;
zero disables this
+cache. The buffer retention setting does not limit JSON input or output size;
it only limits
+reusable writer storage retained after an operation. Apply request/body size
limits at the
+transport boundary when parsing untrusted input.
Builder mutation after `build()` does not modify an existing `ForyJson`
runtime.
@@ -1249,7 +1252,7 @@ native or xlang protocol when reference identity or
cycles are required.
| ----------------------------------------- |
---------------------------------------------------------------------------------------------------------------------------------------------------
|
| `ForyJsonException` while parsing | Invalid JSON grammar, type
mismatch, unsupported mapping, depth violation, or trailing content; inspect
the message and target type |
| `InsecureException` | Fory's disallow list or the
configured `JsonTypeChecker` rejected a class
|
-| `IllegalArgumentException` from a builder | Depth, concurrency level, or
retained buffer limit is not positive
|
+| `IllegalArgumentException` from a builder | Check the configured depth,
concurrency, retained-buffer, and cached-field-name limits
|
| Declared write is rejected | The value is not assignable to
the declared type, the type contains a wildcard/type variable, or null was
supplied for a primitive |
| Immutable value is not populated | Use a record, a valid
`JsonCreator`, or an exact custom codec
|
| `JsonValue` read fails | Add one plain `String`
`JsonCreator`, or register an exact custom codec
|
diff --git a/java/fory-json/src/main/java/org/apache/fory/json/ForyJson.java
b/java/fory-json/src/main/java/org/apache/fory/json/ForyJson.java
index c2989c3a8..d9ed4f2c6 100644
--- a/java/fory-json/src/main/java/org/apache/fory/json/ForyJson.java
+++ b/java/fory-json/src/main/java/org/apache/fory/json/ForyJson.java
@@ -78,6 +78,9 @@ public final class ForyJson {
/** Default maximum nested JSON object/array depth accepted while reading or
writing. */
public static final int DEFAULT_MAX_DEPTH = 20;
+ /** Default maximum number of short, unescaped ASCII field names cached by
each JSON reader. */
+ public static final int DEFAULT_MAX_CACHED_FIELD_NAMES = 8192;
+
private final JsonConfig config;
private final JsonSharedRegistry sharedRegistry;
private final int secondaryPoolSize;
diff --git
a/java/fory-json/src/main/java/org/apache/fory/json/ForyJsonBuilder.java
b/java/fory-json/src/main/java/org/apache/fory/json/ForyJsonBuilder.java
index eed84065d..554c23f0b 100644
--- a/java/fory-json/src/main/java/org/apache/fory/json/ForyJsonBuilder.java
+++ b/java/fory-json/src/main/java/org/apache/fory/json/ForyJsonBuilder.java
@@ -36,10 +36,11 @@ import org.apache.fory.platform.GraalvmSupport;
*
* <p>Defaults omit null object fields, enable code generation and
asynchronous compilation where
* supported, use JavaBean property discovery, use {@link
PropertyNamingStrategy#LOWER_CAMEL_CASE},
- * snapshot the current thread context class loader, allow a nesting depth of
20, use twice the
- * available processors as the pooled-state concurrency level, retain writer
buffers up to 2 MiB,
- * and install no custom type checker. Field mode disables getter and setter
discovery but continues
- * to discover eligible instance fields across the class hierarchy.
+ * snapshot the current thread context class loader, allow a nesting depth of
20, cache up to 8192
+ * common field names in each reader, use twice the available processors as
the pooled-state
+ * concurrency level, retain writer buffers up to 2 MiB, and install no custom
type checker. Field
+ * mode disables getter and setter discovery but continues to discover
eligible instance fields
+ * across the class hierarchy.
*/
public final class ForyJsonBuilder {
private boolean writeNullFields;
@@ -49,6 +50,7 @@ public final class ForyJsonBuilder {
private PropertyNamingStrategy propertyNamingStrategy =
PropertyNamingStrategy.LOWER_CAMEL_CASE;
private ClassLoader classLoader;
private int maxDepth = ForyJson.DEFAULT_MAX_DEPTH;
+ private int maxCachedFieldNames = ForyJson.DEFAULT_MAX_CACHED_FIELD_NAMES;
private int concurrencyLevel = Math.max(1,
Runtime.getRuntime().availableProcessors() * 2);
private int bufferSizeLimitBytes = 2 * 1024 * 1024;
private JsonTypeChecker typeChecker;
@@ -133,6 +135,20 @@ public final class ForyJsonBuilder {
return this;
}
+ /**
+ * Sets the maximum number of unescaped ASCII object field names of up to 16
characters cached by
+ * each JSON reader.
+ *
+ * <p>The default is {@link ForyJson#DEFAULT_MAX_CACHED_FIELD_NAMES}. The
supported range is 0
+ * through 536870912, inclusive; zero disables field-name caching. Other
field names are parsed
+ * normally without being cached. The limit does not restrict accepted JSON
input.
+ */
+ public ForyJsonBuilder withMaxCachedFieldNames(int maxCachedFieldNames) {
+ JsonConfig.validateMaxCachedFieldNames(maxCachedFieldNames);
+ this.maxCachedFieldNames = maxCachedFieldNames;
+ return this;
+ }
+
/** Sets the number of reusable execution states available to concurrent
root operations. */
public ForyJsonBuilder withConcurrencyLevel(int concurrencyLevel) {
if (concurrencyLevel < 1) {
@@ -200,6 +216,7 @@ public final class ForyJsonBuilder {
propertyNamingStrategy,
fixedClassLoader,
maxDepth,
+ maxCachedFieldNames,
concurrencyLevel,
bufferSizeLimitBytes,
codecRegistry,
diff --git a/java/fory-json/src/main/java/org/apache/fory/json/JsonConfig.java
b/java/fory-json/src/main/java/org/apache/fory/json/JsonConfig.java
index 83c58ef28..e4e4b91c4 100644
--- a/java/fory-json/src/main/java/org/apache/fory/json/JsonConfig.java
+++ b/java/fory-json/src/main/java/org/apache/fory/json/JsonConfig.java
@@ -33,10 +33,12 @@ import org.apache.fory.json.resolver.CodecRegistry;
* The type checker uses identity semantics because checker instances may
carry different user
* policy despite sharing a class. {@link #getCodegenHash()} identifies only
settings that can
* change generated source; runtime-only settings such as depth and
asynchronous scheduling do not
- * fragment generated class names. Concurrency and retained writer-buffer
limits are also
- * runtime-only and do not fragment generated class names.
+ * fragment generated class names. Concurrency, per-reader field-name cache,
and retained
+ * writer-buffer limits are also runtime-only and do not fragment generated
class names.
*/
public final class JsonConfig {
+ private static final int MAX_CACHED_FIELD_NAMES = 1 << 29;
+
private final boolean writeNullFields;
private final boolean codegenEnabled;
private final boolean asyncCompilationEnabled;
@@ -44,6 +46,7 @@ public final class JsonConfig {
private final PropertyNamingStrategy propertyNamingStrategy;
private final ClassLoader classLoader;
private final int maxDepth;
+ private final int maxCachedFieldNames;
private final int concurrencyLevel;
private final int bufferSizeLimitBytes;
private final CodecRegistry codecRegistry;
@@ -61,6 +64,7 @@ public final class JsonConfig {
PropertyNamingStrategy propertyNamingStrategy,
ClassLoader classLoader,
int maxDepth,
+ int maxCachedFieldNames,
int concurrencyLevel,
int bufferSizeLimitBytes,
CodecRegistry codecRegistry,
@@ -73,6 +77,8 @@ public final class JsonConfig {
Objects.requireNonNull(propertyNamingStrategy,
"propertyNamingStrategy");
this.classLoader = Objects.requireNonNull(classLoader, "classLoader");
this.maxDepth = maxDepth;
+ validateMaxCachedFieldNames(maxCachedFieldNames);
+ this.maxCachedFieldNames = maxCachedFieldNames;
this.concurrencyLevel = concurrencyLevel;
this.bufferSizeLimitBytes = bufferSizeLimitBytes;
this.codecRegistry = codecRegistry;
@@ -114,6 +120,17 @@ public final class JsonConfig {
return maxDepth;
}
+ public int maxCachedFieldNames() {
+ return maxCachedFieldNames;
+ }
+
+ static void validateMaxCachedFieldNames(int maxCachedFieldNames) {
+ if (maxCachedFieldNames < 0 || maxCachedFieldNames >
MAX_CACHED_FIELD_NAMES) {
+ throw new IllegalArgumentException(
+ "maxCachedFieldNames must be between 0 and " +
MAX_CACHED_FIELD_NAMES);
+ }
+ }
+
public int concurrencyLevel() {
return concurrencyLevel;
}
@@ -150,6 +167,7 @@ public final class JsonConfig {
&& propertyNamingStrategy == that.propertyNamingStrategy
&& classLoader == that.classLoader
&& maxDepth == that.maxDepth
+ && maxCachedFieldNames == that.maxCachedFieldNames
&& concurrencyLevel == that.concurrencyLevel
&& bufferSizeLimitBytes == that.bufferSizeLimitBytes
&& typeChecker == that.typeChecker
@@ -165,6 +183,7 @@ public final class JsonConfig {
result = 31 * result + propertyNamingStrategy.hashCode();
result = 31 * result + System.identityHashCode(classLoader);
result = 31 * result + maxDepth;
+ result = 31 * result + maxCachedFieldNames;
result = 31 * result + concurrencyLevel;
result = 31 * result + bufferSizeLimitBytes;
result = 31 * result + System.identityHashCode(typeChecker);
diff --git
a/java/fory-json/src/main/java/org/apache/fory/json/codec/MapCodec.java
b/java/fory-json/src/main/java/org/apache/fory/json/codec/MapCodec.java
index 5b4f8ad12..305df441b 100644
--- a/java/fory-json/src/main/java/org/apache/fory/json/codec/MapCodec.java
+++ b/java/fory-json/src/main/java/org/apache/fory/json/codec/MapCodec.java
@@ -75,6 +75,11 @@ public abstract class MapCodec<T extends Map<?, ?>>
implements JsonValueCodec<T>
public Object fromName(String name) {
return name;
}
+
+ @Override
+ public Object readName(JsonReader reader) {
+ return reader.readFieldName();
+ }
};
private static final MapKeyCodec OBJECT_KEY_CODEC =
new MapKeyCodec() {
@@ -96,6 +101,11 @@ public abstract class MapCodec<T extends Map<?, ?>>
implements JsonValueCodec<T>
public Object fromName(String name) {
return name;
}
+
+ @Override
+ public Object readName(JsonReader reader) {
+ return reader.readFieldName();
+ }
};
private final MapFactory factory;
@@ -184,7 +194,7 @@ public abstract class MapCodec<T extends Map<?, ?>>
implements JsonValueCodec<T>
reader.expectNextToken('{');
if (!reader.consumeNextToken('}')) {
do {
- Object key = STRING_KEY_CODEC.readName(reader);
+ Object key = reader.readFieldName();
reader.expectNextToken(':');
map.put(key, codec.readLatin1(reader));
} while (reader.consumeNextToken(','));
@@ -202,7 +212,7 @@ public abstract class MapCodec<T extends Map<?, ?>>
implements JsonValueCodec<T>
reader.expectNextToken('{');
if (!reader.consumeNextToken('}')) {
do {
- Object key = STRING_KEY_CODEC.readName(reader);
+ Object key = reader.readFieldName();
reader.expectNextToken(':');
map.put(key, codec.readUtf16(reader));
} while (reader.consumeNextToken(','));
@@ -220,7 +230,7 @@ public abstract class MapCodec<T extends Map<?, ?>>
implements JsonValueCodec<T>
reader.expectNextToken('{');
if (!reader.consumeNextToken('}')) {
do {
- Object key = STRING_KEY_CODEC.readName(reader);
+ Object key = reader.readFieldName();
reader.expectNextToken(':');
map.put(key, codec.readUtf8(reader));
} while (reader.consumeNextToken(','));
@@ -470,7 +480,7 @@ public abstract class MapCodec<T extends Map<?, ?>>
implements JsonValueCodec<T>
reader.expectNextToken('{');
if (!reader.consumeNextToken('}')) {
do {
- String key = reader.readString();
+ String key = reader.readFieldName();
reader.expectNextToken(':');
map.put(key, readLatin1Value(reader));
} while (reader.consumeNextToken(','));
@@ -490,7 +500,7 @@ public abstract class MapCodec<T extends Map<?, ?>>
implements JsonValueCodec<T>
reader.expectNextToken('{');
if (!reader.consumeNextToken('}')) {
do {
- String key = reader.readString();
+ String key = reader.readFieldName();
reader.expectNextToken(':');
map.put(key, readUtf16Value(reader));
} while (reader.consumeNextToken(','));
@@ -510,7 +520,7 @@ public abstract class MapCodec<T extends Map<?, ?>>
implements JsonValueCodec<T>
reader.expectNextToken('{');
if (!reader.consumeNextToken('}')) {
do {
- String key = reader.readString();
+ String key = reader.readFieldName();
reader.expectNextToken(':');
map.put(key, readUtf8Value(reader));
} while (reader.consumeNextToken(','));
diff --git
a/java/fory-json/src/main/java/org/apache/fory/json/reader/FieldNameCache.java
b/java/fory-json/src/main/java/org/apache/fory/json/reader/FieldNameCache.java
new file mode 100644
index 000000000..ce1ae13f9
--- /dev/null
+++
b/java/fory-json/src/main/java/org/apache/fory/json/reader/FieldNameCache.java
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.fory.json.reader;
+
+import java.util.Objects;
+import org.apache.fory.json.resolver.JsonSharedRegistry.CachedFieldName;
+
+/** Fixed two-candidate cache of shared field-name entries owned by one JSON
reader. */
+final class FieldNameCache {
+ private final int maxEntries;
+ private final int mask;
+ private long[] hashes;
+ private CachedFieldName[] entries;
+ private int size;
+
+ FieldNameCache(int maxEntries) {
+ // JsonConfig owns range validation; its upper bound keeps this doubled
slot count positive.
+ this.maxEntries = maxEntries;
+ int requiredSlots = maxEntries << 1;
+ int slots = Integer.highestOneBit(requiredSlots - 1) << 1;
+ mask = slots - 1;
+ }
+
+ CachedFieldName get(long hash) {
+ CachedFieldName[] localEntries = entries;
+ if (localEntries == null) {
+ return null;
+ }
+ int home = slot(hash);
+ CachedFieldName entry = localEntries[home];
+ if (entry == null) {
+ // Entries are never removed or replaced, so a hash cannot occupy the
adjacent slot while
+ // its home remains empty.
+ return null;
+ }
+ if (hashes[home] == hash) {
+ return entry;
+ }
+ int adjacent = (home + 1) & mask;
+ entry = localEntries[adjacent];
+ return entry != null && hashes[adjacent] == hash ? entry : null;
+ }
+
+ boolean canPut(long hash) {
+ if (size >= maxEntries) {
+ return false;
+ }
+ CachedFieldName[] localEntries = entries;
+ if (localEntries == null) {
+ return true;
+ }
+ int home = slot(hash);
+ CachedFieldName entry = localEntries[home];
+ if (entry == null) {
+ return true;
+ }
+ if (hashes[home] == hash) {
+ return false;
+ }
+ int adjacent = (home + 1) & mask;
+ entry = localEntries[adjacent];
+ return entry == null;
+ }
+
+ void put(long hash, CachedFieldName entry) {
+ Objects.requireNonNull(entry, "entry");
+ CachedFieldName[] localEntries = entries;
+ if (localEntries == null) {
+ hashes = new long[mask + 1];
+ localEntries = new CachedFieldName[mask + 1];
+ entries = localEntries;
+ }
+ int home = slot(hash);
+ CachedFieldName cached = localEntries[home];
+ if (cached == null) {
+ if (size < maxEntries) {
+ hashes[home] = hash;
+ localEntries[home] = entry;
+ size++;
+ }
+ return;
+ }
+ if (hashes[home] == hash) {
+ return;
+ }
+ int adjacent = (home + 1) & mask;
+ cached = localEntries[adjacent];
+ if (cached == null) {
+ if (size < maxEntries) {
+ hashes[adjacent] = hash;
+ localEntries[adjacent] = entry;
+ size++;
+ }
+ return;
+ }
+ if (hashes[adjacent] == hash) {
+ return;
+ }
+ }
+
+ private int slot(long hash) {
+ int spread = (int) (hash ^ (hash >>> 32));
+ spread ^= spread >>> 16;
+ return spread & mask;
+ }
+}
diff --git
a/java/fory-json/src/main/java/org/apache/fory/json/reader/JsonReader.java
b/java/fory-json/src/main/java/org/apache/fory/json/reader/JsonReader.java
index 625f3046f..0e27da04a 100644
--- a/java/fory-json/src/main/java/org/apache/fory/json/reader/JsonReader.java
+++ b/java/fory-json/src/main/java/org/apache/fory/json/reader/JsonReader.java
@@ -135,6 +135,7 @@ public abstract class JsonReader {
private final JsonTypeResolver typeResolver;
protected int position;
private final int maxDepth;
+ protected final FieldNameCache fieldNameCache;
private int depth;
/**
@@ -233,6 +234,9 @@ public abstract class JsonReader {
protected JsonReader(JsonConfig config, JsonTypeResolver typeResolver) {
this.typeResolver = Objects.requireNonNull(typeResolver, "typeResolver");
maxDepth = config.maxDepth();
+ // The configured limit belongs to each reader; pooled-state concurrency
must not divide it.
+ int maxEntries = config.maxCachedFieldNames();
+ fieldNameCache = maxEntries == 0 ? null : new FieldNameCache(maxEntries);
}
/**
@@ -248,6 +252,32 @@ public abstract class JsonReader {
public abstract String readString();
+ /**
+ * Reads a JSON object field name that the caller retains as a String key.
+ *
+ * <p>The returned String has normal value semantics. Implementations may
reuse its reference for
+ * common names, but callers must not depend on reference identity.
+ */
+ public abstract String readFieldName();
+
+ protected static long fieldNameHash(int length, long word0, long word1) {
+ if (length == 0) {
+ return JsonFieldNameHash.MAGIC_HASH_CODE;
+ }
+ if (length <= Long.BYTES) {
+ return word0;
+ }
+ long hash = JsonFieldNameHash.hashPacked(word0, Long.BYTES);
+ for (int i = Long.BYTES; i < length; i++) {
+ hash = JsonFieldNameHash.update(hash, (char) ((word1 >>> ((i -
Long.BYTES) << 3)) & 0xff));
+ }
+ return hash;
+ }
+
+ protected static long fieldNameWord(long word, int length) {
+ return length == Long.BYTES ? word : word & ((1L << (length << 3)) - 1);
+ }
+
@Internal
public final int position() {
return position;
@@ -257,7 +287,7 @@ public abstract class JsonReader {
public final String materializeFieldName(int start) {
int current = position;
position = start;
- String name = readString();
+ String name = readFieldName();
position = current;
return name;
}
diff --git
a/java/fory-json/src/main/java/org/apache/fory/json/reader/Latin1JsonReader.java
b/java/fory-json/src/main/java/org/apache/fory/json/reader/Latin1JsonReader.java
index 9f2ab035e..23895d0c1 100644
---
a/java/fory-json/src/main/java/org/apache/fory/json/reader/Latin1JsonReader.java
+++
b/java/fory-json/src/main/java/org/apache/fory/json/reader/Latin1JsonReader.java
@@ -30,6 +30,8 @@ import org.apache.fory.json.meta.JsonFieldInfo;
import org.apache.fory.json.meta.JsonFieldNameHash;
import org.apache.fory.json.meta.JsonFieldTable;
import org.apache.fory.json.meta.JsonSubtypeScanInfo;
+import org.apache.fory.json.resolver.JsonSharedRegistry;
+import org.apache.fory.json.resolver.JsonSharedRegistry.CachedFieldName;
import org.apache.fory.json.resolver.JsonTypeResolver;
import org.apache.fory.memory.LittleEndian;
import org.apache.fory.memory.NativeByteOrder;
@@ -1605,6 +1607,148 @@ public final class Latin1JsonReader extends JsonReader {
return readStringToken();
}
+ @Override
+ public String readFieldName() {
+ FieldNameCache cache = fieldNameCache;
+ if (cache == null) {
+ return readString();
+ }
+ return readCachedFieldName(cache);
+ }
+
+ private String readCachedFieldName(FieldNameCache cache) {
+ skipWhitespaceFast();
+ byte[] bytes = input;
+ int inputLength = bytes.length;
+ if (position >= inputLength || bytes[position++] != '"') {
+ throw error("Expected string");
+ }
+ int start = position;
+ if (start + Long.BYTES <= inputLength) {
+ long word0 = LittleEndian.getInt64(bytes, start);
+ long stopMask = asciiStringStopMask(word0);
+ if (stopMask != 0) {
+ int length = Long.numberOfTrailingZeros(stopMask) >>> 3;
+ int stop = start + length;
+ int ch = bytes[stop];
+ if (ch != '"') {
+ return readStringStop(start, stop, ch);
+ }
+ position = stop + 1;
+ word0 = fieldNameWord(word0, length);
+ long hash = length == 0 ? JsonFieldNameHash.MAGIC_HASH_CODE : word0;
+ CachedFieldName entry = cache.get(hash);
+ if (entry != null) {
+ return entry.matches(length, word0, 0) ? entry.name() :
newLatin1String(start, stop);
+ }
+ if (!cache.canPut(hash)) {
+ return newLatin1String(start, stop);
+ }
+ return readFieldNameMiss(cache, start, stop, length, word0, 0, hash);
+ }
+ return readFieldNameAfterWord0(cache, start, word0, inputLength);
+ }
+ return readFieldNameTail(cache, start, start, 0, 0, 0);
+ }
+
+ private String readFieldNameAfterWord0(
+ FieldNameCache cache, int start, long word0, int inputLength) {
+ int offset = start + Long.BYTES;
+ if (offset + Long.BYTES <= inputLength) {
+ long word1 = LittleEndian.getInt64(input, offset);
+ long stopMask = asciiStringStopMask(word1);
+ if (stopMask != 0) {
+ int length = Long.numberOfTrailingZeros(stopMask) >>> 3;
+ int stop = offset + length;
+ int ch = input[stop];
+ if (ch != '"') {
+ return readStringStop(start, stop, ch);
+ }
+ position = stop + 1;
+ return resolveFieldName(
+ cache, start, stop, Long.BYTES + length, word0,
fieldNameWord(word1, length));
+ }
+ offset += Long.BYTES;
+ if (offset < inputLength && input[offset] == '"') {
+ position = offset + 1;
+ return resolveFieldName(cache, start, offset, 16, word0, word1);
+ }
+ // Keep the beyond-cache-width scanner out of the short-name and
ordinary-string hot paths.
+ return readLongFieldName(start, offset, inputLength);
+ }
+ return readFieldNameTail(cache, start, offset, Long.BYTES, word0, 0);
+ }
+
+ private String readLongFieldName(int start, int offset, int inputLength) {
+ byte[] bytes = input;
+ int wordEnd = inputLength - Long.BYTES;
+ while (offset <= wordEnd) {
+ long stopMask = asciiStringStopMask(LittleEndian.getInt64(bytes,
offset));
+ if (stopMask == 0) {
+ offset += Long.BYTES;
+ continue;
+ }
+ int stop = offset + (Long.numberOfTrailingZeros(stopMask) >>> 3);
+ int ch = bytes[stop];
+ if (ch == '"') {
+ position = stop + 1;
+ return newLatin1String(start, stop);
+ }
+ return readStringStop(start, stop, ch);
+ }
+ return readStringTokenTail(start, offset, inputLength);
+ }
+
+ private String readFieldNameTail(
+ FieldNameCache cache, int start, int offset, int length, long word0,
long word1) {
+ int inputLength = input.length;
+ while (offset < inputLength) {
+ int ch = input[offset] & 0xff;
+ if (ch == '"') {
+ position = offset + 1;
+ return resolveFieldName(cache, start, offset, length, word0, word1);
+ }
+ if (ch == '\\' || ch >= 0x80 || ch < 0x20) {
+ return readStringStop(start, offset, input[offset]);
+ }
+ if (length < Long.BYTES) {
+ word0 |= ((long) ch) << (length << 3);
+ } else {
+ word1 |= ((long) ch) << ((length - Long.BYTES) << 3);
+ }
+ length++;
+ offset++;
+ }
+ throw error("Unterminated string");
+ }
+
+ private String resolveFieldName(
+ FieldNameCache cache, int start, int end, int length, long word0, long
word1) {
+ long hash = fieldNameHash(length, word0, word1);
+ CachedFieldName entry = cache.get(hash);
+ if (entry != null) {
+ return entry.matches(length, word0, word1) ? entry.name() :
newLatin1String(start, end);
+ }
+ if (!cache.canPut(hash)) {
+ return newLatin1String(start, end);
+ }
+ return readFieldNameMiss(cache, start, end, length, word0, word1, hash);
+ }
+
+ private String readFieldNameMiss(
+ FieldNameCache cache, int start, int end, int length, long word0, long
word1, long hash) {
+ JsonSharedRegistry registry = typeResolver().sharedRegistry();
+ CachedFieldName entry = registry.cachedFieldName(hash);
+ if (entry != null) {
+ cache.put(hash, entry);
+ return entry.matches(length, word0, word1) ? entry.name() :
newLatin1String(start, end);
+ }
+ String candidate = newLatin1String(start, end);
+ entry = registry.cacheFieldName(hash, candidate, word0, word1);
+ cache.put(hash, entry);
+ return entry.matches(length, word0, word1) ? entry.name() : candidate;
+ }
+
@Override
public String readNullableString() {
skipWhitespaceFast();
diff --git
a/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf16JsonReader.java
b/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf16JsonReader.java
index 38e96824a..65f18223b 100644
---
a/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf16JsonReader.java
+++
b/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf16JsonReader.java
@@ -29,6 +29,8 @@ import org.apache.fory.json.meta.JsonFieldInfo;
import org.apache.fory.json.meta.JsonFieldNameHash;
import org.apache.fory.json.meta.JsonFieldTable;
import org.apache.fory.json.meta.JsonSubtypeScanInfo;
+import org.apache.fory.json.resolver.JsonSharedRegistry;
+import org.apache.fory.json.resolver.JsonSharedRegistry.CachedFieldName;
import org.apache.fory.json.resolver.JsonTypeResolver;
import org.apache.fory.memory.LittleEndian;
import org.apache.fory.memory.NativeByteOrder;
@@ -63,6 +65,7 @@ public final class Utf16JsonReader extends JsonReader {
private static final long UTF16_BACKSLASH_CHARS = 0x005c005c005c005cL;
private static final long UTF16_CONTROL_LIMIT = 0x0020002000200020L;
private static final long UTF16_NON_LATIN_BYTES = 0xFF00FF00FF00FF00L;
+ private static final long UTF16_NON_ASCII_BITS = 0xFF80FF80FF80FF80L;
private static final long UTF16_SURROGATE_MASK = 0xf800f800f800f800L;
private static final long UTF16_SURROGATE_PREFIX = 0xd800d800d800d800L;
private static final long LONG_MAX_DIV_10 = Long.MAX_VALUE / 10;
@@ -1525,6 +1528,176 @@ public final class Utf16JsonReader extends JsonReader {
return readStringToken();
}
+ @Override
+ public String readFieldName() {
+ FieldNameCache cache = fieldNameCache;
+ if (cache == null) {
+ return readString();
+ }
+ return readCachedFieldName(cache);
+ }
+
+ private String readCachedFieldName(FieldNameCache cache) {
+ skipWhitespaceFast();
+ if (position >= length || charAtFast(position++) != '"') {
+ throw error("Expected string");
+ }
+ int start = position;
+ if (LITTLE_ENDIAN && bytes != null) {
+ return readFieldNameWords(cache, start);
+ }
+ return readFieldNameTail(cache, start, start, 0, 0, 0);
+ }
+
+ private String readFieldNameWords(FieldNameCache cache, int start) {
+ byte[] localBytes = bytes;
+ int inputLength = length;
+ if (start + 4 > inputLength) {
+ return readFieldNameTail(cache, start, start, 0, 0, 0);
+ }
+ long word = LittleEndian.getInt64(localBytes, start << 1);
+ long stopMask =
+ utf16StringStopMask(word, word & UTF16_NON_LATIN_BYTES) | (word &
UTF16_NON_ASCII_BITS);
+ if (stopMask != 0) {
+ int chars = Long.numberOfTrailingZeros(stopMask) >>> 4;
+ int stop = start + chars;
+ if (charAtFast(stop) != '"') {
+ return continueFieldName(start, stop);
+ }
+ position = stop + 1;
+ return resolveShortFieldName(
+ cache, start, stop, chars, fieldNameWord(packAsciiUtf16(word),
chars));
+ }
+ long word0 = packAsciiUtf16(word);
+ int offset = start + 4;
+ if (offset + 4 > inputLength) {
+ return readFieldNameTail(cache, start, offset, 4, word0, 0);
+ }
+ word = LittleEndian.getInt64(localBytes, offset << 1);
+ stopMask =
+ utf16StringStopMask(word, word & UTF16_NON_LATIN_BYTES) | (word &
UTF16_NON_ASCII_BITS);
+ if (stopMask != 0) {
+ int chars = Long.numberOfTrailingZeros(stopMask) >>> 4;
+ int stop = offset + chars;
+ if (charAtFast(stop) != '"') {
+ return continueFieldName(start, stop);
+ }
+ position = stop + 1;
+ word0 |= fieldNameWord(packAsciiUtf16(word), chars) << 32;
+ return resolveShortFieldName(cache, start, stop, 4 + chars, word0);
+ }
+ word0 |= packAsciiUtf16(word) << 32;
+ return readFieldNameAfterWord0(cache, start, offset + 4, word0);
+ }
+
+ private String readFieldNameAfterWord0(FieldNameCache cache, int start, int
offset, long word0) {
+ byte[] localBytes = bytes;
+ long word1 = 0;
+ int nameLength = Long.BYTES;
+ while (nameLength < 16 && offset + 4 <= length) {
+ long word = LittleEndian.getInt64(localBytes, offset << 1);
+ long stopMask =
+ utf16StringStopMask(word, word & UTF16_NON_LATIN_BYTES) | (word &
UTF16_NON_ASCII_BITS);
+ if (stopMask != 0) {
+ int chars = Long.numberOfTrailingZeros(stopMask) >>> 4;
+ int stop = offset + chars;
+ char ch = charAtFast(stop);
+ if (ch != '"') {
+ return continueFieldName(start, stop);
+ }
+ position = stop + 1;
+ long packed = fieldNameWord(packAsciiUtf16(word), chars);
+ word1 |= packed << ((nameLength - Long.BYTES) << 3);
+ return resolveFieldName(cache, start, stop, nameLength + chars, word0,
word1);
+ }
+ long packed = packAsciiUtf16(word);
+ word1 |= packed << ((nameLength - Long.BYTES) << 3);
+ nameLength += 4;
+ offset += 4;
+ }
+ if (nameLength == 16) {
+ if (offset < length && charAtFast(offset) == '"') {
+ position = offset + 1;
+ return resolveFieldName(cache, start, offset, nameLength, word0,
word1);
+ }
+ return continueFieldName(start, offset);
+ }
+ return readFieldNameTail(cache, start, offset, nameLength, word0, word1);
+ }
+
+ private String resolveShortFieldName(
+ FieldNameCache cache, int start, int end, int nameLength, long word0) {
+ long hash = nameLength == 0 ? JsonFieldNameHash.MAGIC_HASH_CODE : word0;
+ CachedFieldName entry = cache.get(hash);
+ if (entry != null) {
+ return entry.matches(nameLength, word0, 0) ? entry.name() :
input.substring(start, end);
+ }
+ if (!cache.canPut(hash)) {
+ return input.substring(start, end);
+ }
+ return readFieldNameMiss(cache, start, end, nameLength, word0, 0, hash);
+ }
+
+ private String readFieldNameTail(
+ FieldNameCache cache, int start, int offset, int nameLength, long word0,
long word1) {
+ while (offset < length) {
+ char ch = charAtFast(offset);
+ if (ch == '"') {
+ position = offset + 1;
+ return resolveFieldName(cache, start, offset, nameLength, word0,
word1);
+ }
+ if (ch == '\\' || ch > 0x7f || ch < 0x20) {
+ return continueFieldName(start, offset);
+ }
+ if (nameLength == 16) {
+ return continueFieldName(start, offset);
+ }
+ if (nameLength < Long.BYTES) {
+ word0 |= ((long) ch) << (nameLength << 3);
+ } else {
+ word1 |= ((long) ch) << ((nameLength - Long.BYTES) << 3);
+ }
+ nameLength++;
+ offset++;
+ }
+ throw error("Unterminated string");
+ }
+
+ private String continueFieldName(int start, int offset) {
+ position = offset;
+ if (LITTLE_ENDIAN && bytes != null) {
+ return readUtf16StringToken(start, offset, false);
+ }
+ return readStringLoop(start, false);
+ }
+
+ private String readFieldNameMiss(
+ FieldNameCache cache, int start, int end, int length, long word0, long
word1, long hash) {
+ JsonSharedRegistry registry = typeResolver().sharedRegistry();
+ CachedFieldName entry = registry.cachedFieldName(hash);
+ if (entry != null) {
+ cache.put(hash, entry);
+ return entry.matches(length, word0, word1) ? entry.name() :
input.substring(start, end);
+ }
+ String candidate = input.substring(start, end);
+ entry = registry.cacheFieldName(hash, candidate, word0, word1);
+ cache.put(hash, entry);
+ return entry.matches(length, word0, word1) ? entry.name() : candidate;
+ }
+
+ private String resolveFieldName(
+ FieldNameCache cache, int start, int end, int length, long word0, long
word1) {
+ long hash = fieldNameHash(length, word0, word1);
+ CachedFieldName entry = cache.get(hash);
+ if (entry != null) {
+ return entry.matches(length, word0, word1) ? entry.name() :
input.substring(start, end);
+ }
+ if (!cache.canPut(hash)) {
+ return input.substring(start, end);
+ }
+ return readFieldNameMiss(cache, start, end, length, word0, word1, hash);
+ }
+
@Override
public String readNullableString() {
skipWhitespaceFast();
@@ -1597,16 +1770,14 @@ public final class Utf16JsonReader extends JsonReader {
private String readStringAfterQuote() {
int start = position;
if (LITTLE_ENDIAN && bytes != null) {
- return readUtf16StringToken(start);
+ return readUtf16StringToken(start, start, false);
}
return readStringLoop(start, false);
}
- private String readUtf16StringToken(int start) {
+ private String readUtf16StringToken(int start, int offset, boolean nonLatin)
{
byte[] localBytes = bytes;
int inputLength = length;
- int offset = start;
- boolean nonLatin = false;
int doubleWordEnd = inputLength - 8;
while (offset <= doubleWordEnd) {
long word = LittleEndian.getInt64(localBytes, offset << 1);
@@ -2415,6 +2586,13 @@ public final class Utf16JsonReader extends JsonReader {
return LITTLE_ENDIAN ? word : word << 8;
}
+ private static long packAsciiUtf16(long word) {
+ return (word & 0xff)
+ | ((word >>> 8) & 0xff00)
+ | ((word >>> 16) & 0xff0000)
+ | ((word >>> 24) & 0xff000000L);
+ }
+
private static long utf16WordMask(int length) {
return length == 4 ? -1L : (1L << (length << 4)) - 1;
}
diff --git
a/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf8JsonReader.java
b/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf8JsonReader.java
index fe0d181ef..3aaa77900 100644
---
a/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf8JsonReader.java
+++
b/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf8JsonReader.java
@@ -30,6 +30,8 @@ import org.apache.fory.json.meta.JsonFieldInfo;
import org.apache.fory.json.meta.JsonFieldNameHash;
import org.apache.fory.json.meta.JsonFieldTable;
import org.apache.fory.json.meta.JsonSubtypeScanInfo;
+import org.apache.fory.json.resolver.JsonSharedRegistry;
+import org.apache.fory.json.resolver.JsonSharedRegistry.CachedFieldName;
import org.apache.fory.json.resolver.JsonTypeResolver;
import org.apache.fory.memory.LittleEndian;
import org.apache.fory.memory.NativeByteOrder;
@@ -1698,6 +1700,127 @@ public final class Utf8JsonReader extends JsonReader {
return readStringToken();
}
+ @Override
+ public String readFieldName() {
+ FieldNameCache cache = fieldNameCache;
+ if (cache == null) {
+ return readString();
+ }
+ return readCachedFieldName(cache);
+ }
+
+ private String readCachedFieldName(FieldNameCache cache) {
+ skipWhitespaceFast();
+ byte[] bytes = input;
+ int inputLength = bytes.length;
+ if (position >= inputLength || bytes[position++] != '"') {
+ throw error("Expected string");
+ }
+ int start = position;
+ if (start + Long.BYTES <= inputLength) {
+ long word0 = LittleEndian.getInt64(bytes, start);
+ long stopMask = stringStopMask(word0);
+ if (stopMask != 0) {
+ int length = Long.numberOfTrailingZeros(stopMask) >>> 3;
+ int stop = start + length;
+ int b = bytes[stop];
+ if (b != '"') {
+ return readStringStop(start, stop, b);
+ }
+ position = stop + 1;
+ word0 = fieldNameWord(word0, length);
+ long hash = length == 0 ? JsonFieldNameHash.MAGIC_HASH_CODE : word0;
+ CachedFieldName entry = cache.get(hash);
+ if (entry != null) {
+ return entry.matches(length, word0, 0) ? entry.name() :
newLatin1String(start, stop);
+ }
+ if (!cache.canPut(hash)) {
+ return newLatin1String(start, stop);
+ }
+ return readFieldNameMiss(cache, start, stop, length, word0, 0, hash);
+ }
+ return readFieldNameAfterWord0(cache, start, word0, inputLength);
+ }
+ return readFieldNameTail(cache, start, start, 0, 0, 0);
+ }
+
+ private String readFieldNameAfterWord0(
+ FieldNameCache cache, int start, long word0, int inputLength) {
+ int offset = start + Long.BYTES;
+ if (offset + Long.BYTES <= inputLength) {
+ long word1 = LittleEndian.getInt64(input, offset);
+ long stopMask = stringStopMask(word1);
+ if (stopMask != 0) {
+ int length = Long.numberOfTrailingZeros(stopMask) >>> 3;
+ int stop = offset + length;
+ int b = input[stop];
+ if (b != '"') {
+ return readStringStop(start, stop, b);
+ }
+ position = stop + 1;
+ return resolveFieldName(
+ cache, start, stop, Long.BYTES + length, word0,
fieldNameWord(word1, length));
+ }
+ offset += Long.BYTES;
+ if (offset < inputLength && input[offset] == '"') {
+ position = offset + 1;
+ return resolveFieldName(cache, start, offset, 16, word0, word1);
+ }
+ return readStringTokenLongTail(start, offset, inputLength);
+ }
+ return readFieldNameTail(cache, start, offset, Long.BYTES, word0, 0);
+ }
+
+ private String readFieldNameTail(
+ FieldNameCache cache, int start, int offset, int length, long word0,
long word1) {
+ int inputLength = input.length;
+ while (offset < inputLength) {
+ int ch = input[offset] & 0xff;
+ if (ch == '"') {
+ position = offset + 1;
+ return resolveFieldName(cache, start, offset, length, word0, word1);
+ }
+ if (ch == '\\' || ch >= 0x80 || ch < 0x20) {
+ return readStringStop(start, offset, input[offset]);
+ }
+ if (length < Long.BYTES) {
+ word0 |= ((long) ch) << (length << 3);
+ } else {
+ word1 |= ((long) ch) << ((length - Long.BYTES) << 3);
+ }
+ length++;
+ offset++;
+ }
+ throw error("Unterminated string");
+ }
+
+ private String resolveFieldName(
+ FieldNameCache cache, int start, int end, int length, long word0, long
word1) {
+ long hash = fieldNameHash(length, word0, word1);
+ CachedFieldName entry = cache.get(hash);
+ if (entry != null) {
+ return entry.matches(length, word0, word1) ? entry.name() :
newLatin1String(start, end);
+ }
+ if (!cache.canPut(hash)) {
+ return newLatin1String(start, end);
+ }
+ return readFieldNameMiss(cache, start, end, length, word0, word1, hash);
+ }
+
+ private String readFieldNameMiss(
+ FieldNameCache cache, int start, int end, int length, long word0, long
word1, long hash) {
+ JsonSharedRegistry registry = typeResolver().sharedRegistry();
+ CachedFieldName entry = registry.cachedFieldName(hash);
+ if (entry != null) {
+ cache.put(hash, entry);
+ return entry.matches(length, word0, word1) ? entry.name() :
newLatin1String(start, end);
+ }
+ String candidate = newLatin1String(start, end);
+ entry = registry.cacheFieldName(hash, candidate, word0, word1);
+ cache.put(hash, entry);
+ return entry.matches(length, word0, word1) ? entry.name() : candidate;
+ }
+
@Override
public String readNullableString() {
skipWhitespaceFast();
diff --git
a/java/fory-json/src/main/java/org/apache/fory/json/resolver/JsonSharedRegistry.java
b/java/fory-json/src/main/java/org/apache/fory/json/resolver/JsonSharedRegistry.java
index 3c1da0bfc..52d0e6025 100644
---
a/java/fory-json/src/main/java/org/apache/fory/json/resolver/JsonSharedRegistry.java
+++
b/java/fory-json/src/main/java/org/apache/fory/json/resolver/JsonSharedRegistry.java
@@ -138,9 +138,12 @@ import org.apache.fory.util.record.RecordUtils;
*
* <p>Accepted type-check results are cached by class name up to a bounded
8192-entry shared cache.
* Once full, new names are checked on every resolution rather than growing
attacker-controlled
- * state. Source-generated model companions and JIT-generated classes are
shared here; concrete JIT
- * codec instances, ordinary type bindings, JIT locks, and callbacks remain
resolver-local. A fresh
- * generic {@link JsonJITContext} is therefore created for every pooled JSON
state.
+ * state. Common short field names admitted by reader-local caches are
published here for
+ * best-effort String reference reuse across readers. Reader-local admission
is the only field-name
+ * capacity gate; the shared field-name map has no explicit limit.
Source-generated model companions
+ * and JIT-generated classes are shared here; concrete JIT codec instances,
ordinary type bindings,
+ * JIT locks, and callbacks remain resolver-local. A fresh generic {@link
JsonJITContext} is
+ * therefore created for every pooled JSON state.
*/
public final class JsonSharedRegistry {
private static final int TYPE_CHECK_CACHE_LIMIT = 8192;
@@ -179,6 +182,7 @@ public final class JsonSharedRegistry {
private final ConcurrentHashMap<Class<? extends MapKeyCodec>, MapKeyCodec>
mapKeyCodecs;
private final ConcurrentHashMap<Class<?>, GeneratedJsonCodec<?>>
generatedCodecs;
private final Set<Class<?>> typesWithoutGeneratedCodec;
+ private final ConcurrentHashMap<Long, CachedFieldName> cachedFieldNames;
public JsonSharedRegistry(JsonConfig config) {
this(config, null);
@@ -206,6 +210,7 @@ public final class JsonSharedRegistry {
mapKeyCodecs = new ConcurrentHashMap<>();
generatedCodecs = new ConcurrentHashMap<>();
typesWithoutGeneratedCodec = ConcurrentHashMap.newKeySet();
+ cachedFieldNames = new ConcurrentHashMap<>();
boolean codegenEnabled = config.codegenEnabled();
codegen = codegenEnabled ? new JsonCodegen(config.getCodegenHash(),
classLoader) : null;
asyncCompilationEnabled = codegenEnabled &&
config.asyncCompilationEnabled();
@@ -213,6 +218,42 @@ public final class JsonSharedRegistry {
registerExactCodecs();
}
+ /** Returns the immutable cached entry for {@code hash}, or null when none
was published. */
+ @Internal
+ public CachedFieldName cachedFieldName(long hash) {
+ return cachedFieldNames.get(hash);
+ }
+
+ /** Publishes one already validated short ASCII field name, or returns the
existing hash owner. */
+ @Internal
+ public CachedFieldName cacheFieldName(long hash, String name, long word0,
long word1) {
+ CachedFieldName candidate = new CachedFieldName(name, word0, word1);
+ CachedFieldName existing = cachedFieldNames.putIfAbsent(hash, candidate);
+ return existing == null ? candidate : existing;
+ }
+
+ /** Immutable field-name hash owner retained for best-effort cross-reader
reference reuse. */
+ @Internal
+ public static final class CachedFieldName {
+ private final String name;
+ private final long word0;
+ private final long word1;
+
+ private CachedFieldName(String name, long word0, long word1) {
+ this.name = name;
+ this.word0 = word0;
+ this.word1 = word1;
+ }
+
+ public String name() {
+ return name;
+ }
+
+ public boolean matches(int length, long candidateWord0, long
candidateWord1) {
+ return name.length() == length && word0 == candidateWord0 && word1 ==
candidateWord1;
+ }
+ }
+
GeneratedJsonCodec<?> generatedCodec(Class<?> type) {
if (type.getDeclaredAnnotation(JsonType.class) == null) {
return null;
diff --git
a/java/fory-json/src/main/java/org/apache/fory/json/resolver/JsonTypeResolver.java
b/java/fory-json/src/main/java/org/apache/fory/json/resolver/JsonTypeResolver.java
index 6717af985..3f66da527 100644
---
a/java/fory-json/src/main/java/org/apache/fory/json/resolver/JsonTypeResolver.java
+++
b/java/fory-json/src/main/java/org/apache/fory/json/resolver/JsonTypeResolver.java
@@ -110,6 +110,12 @@ public final class JsonTypeResolver {
canonicalObjectTypeInfos = new IdentityMap<>();
}
+ /** Returns the shared registry that owns this resolver and its reader cache
domain. */
+ @Internal
+ public JsonSharedRegistry sharedRegistry() {
+ return sharedRegistry;
+ }
+
@Internal
public void lockJIT() {
jitContext.lock();
diff --git
a/java/fory-json/src/test/java/org/apache/fory/json/JsonAnyPropertyTest.java
b/java/fory-json/src/test/java/org/apache/fory/json/JsonAnyPropertyTest.java
index 2bdfeb111..a2a62fe24 100644
--- a/java/fory-json/src/test/java/org/apache/fory/json/JsonAnyPropertyTest.java
+++ b/java/fory-json/src/test/java/org/apache/fory/json/JsonAnyPropertyTest.java
@@ -121,6 +121,19 @@ public class JsonAnyPropertyTest extends
ForyJsonTestModels {
assertGeneratedCapabilities(json, FinalAny.class);
}
+ @Test
+ public void retainedKeyIdentity() {
+ ForyJson json = newJson();
+ MutableAny latin1 = json.fromJson("{\"common\":1}", MutableAny.class);
+ String canonical = latin1.properties.keySet().iterator().next();
+
+ MutableAny utf16 = json.fromJson("{\"common\":2,\"键\":3}",
MutableAny.class);
+ MutableAny utf8 =
+ json.fromJson("{\"common\":4}".getBytes(StandardCharsets.UTF_8),
MutableAny.class);
+ assertSame(utf16.properties.keySet().iterator().next(), canonical);
+ assertSame(utf8.properties.keySet().iterator().next(), canonical);
+ }
+
@Test
public void directionalField() {
ForyJson json = newJson();
diff --git
a/java/fory-json/src/test/java/org/apache/fory/json/JsonAsyncCompilationTest.java
b/java/fory-json/src/test/java/org/apache/fory/json/JsonAsyncCompilationTest.java
index 1f927bce7..2aabfb2a2 100644
---
a/java/fory-json/src/test/java/org/apache/fory/json/JsonAsyncCompilationTest.java
+++
b/java/fory-json/src/test/java/org/apache/fory/json/JsonAsyncCompilationTest.java
@@ -878,6 +878,7 @@ public class JsonAsyncCompilationTest {
PropertyNamingStrategy.LOWER_CAMEL_CASE,
JsonAsyncCompilationTest.class.getClassLoader(),
ForyJson.DEFAULT_MAX_DEPTH,
+ ForyJson.DEFAULT_MAX_CACHED_FIELD_NAMES,
1,
2 * 1024 * 1024,
codecs,
diff --git
a/java/fory-json/src/test/java/org/apache/fory/json/JsonFieldNameCacheTest.java
b/java/fory-json/src/test/java/org/apache/fory/json/JsonFieldNameCacheTest.java
new file mode 100644
index 000000000..a10c9768b
--- /dev/null
+++
b/java/fory-json/src/test/java/org/apache/fory/json/JsonFieldNameCacheTest.java
@@ -0,0 +1,481 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.fory.json;
+
+import static org.testng.Assert.assertEquals;
+import static org.testng.Assert.assertNotEquals;
+import static org.testng.Assert.assertNotNull;
+import static org.testng.Assert.assertNotSame;
+import static org.testng.Assert.assertNull;
+import static org.testng.Assert.assertSame;
+import static org.testng.Assert.assertThrows;
+import static org.testng.Assert.assertTrue;
+
+import java.nio.charset.StandardCharsets;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicReferenceArray;
+import org.apache.fory.json.meta.JsonFieldNameHash;
+import org.apache.fory.json.reader.Latin1JsonReader;
+import org.apache.fory.json.resolver.JsonSharedRegistry;
+import org.apache.fory.json.resolver.JsonSharedRegistry.CachedFieldName;
+import org.apache.fory.reflect.TypeRef;
+import org.testng.annotations.Test;
+
+public class JsonFieldNameCacheTest {
+ @Test
+ public void configuration() {
+ JsonConfig defaults = JsonTestSupport.config(ForyJson.builder().build());
+ assertEquals(defaults.maxCachedFieldNames(),
ForyJson.DEFAULT_MAX_CACHED_FIELD_NAMES);
+ assertThrows(
+ IllegalArgumentException.class, () ->
ForyJson.builder().withMaxCachedFieldNames(-1));
+ assertThrows(
+ IllegalArgumentException.class,
+ () -> ForyJson.builder().withMaxCachedFieldNames(Integer.MAX_VALUE));
+
+ JsonConfig first =
+ JsonTestSupport.config(
+
ForyJson.builder().withConcurrencyLevel(1).withMaxCachedFieldNames(1).build());
+ JsonConfig second =
+ JsonTestSupport.config(
+
ForyJson.builder().withConcurrencyLevel(1).withMaxCachedFieldNames(2).build());
+ assertNotEquals(first, second);
+ assertNotEquals(first.hashCode(), second.hashCode());
+ assertEquals(first.getCodegenHash(), second.getCodegenHash());
+ assertEquals(first.maxCachedFieldNames(), 1);
+ assertEquals(second.maxCachedFieldNames(), 2);
+ assertEquals(JsonTestSupport.config(newJson(0)).maxCachedFieldNames(), 0);
+ assertEquals(ForyJson.DEFAULT_MAX_CACHED_FIELD_NAMES, 8192);
+ }
+
+ @Test
+ public void representations() {
+ ForyJson json = newJson(64);
+ String latin1 = firstKey(json.fromJson("{\"shared\":1}",
JsonObject.class));
+ String utf16 = firstKey(json.fromJson("{\"shared\":\"你好\"}",
JsonObject.class));
+ String utf8 =
+ firstKey(
+ json.fromJson("{\"shared\":1}".getBytes(StandardCharsets.UTF_8),
JsonObject.class));
+ assertSame(utf16, latin1);
+ assertSame(utf8, latin1);
+ }
+
+ @Test
+ public void retainedKeyPaths() {
+ ForyJson json = newJson(64);
+ String canonical = firstKey(json.fromJson("{\"common\":1}",
JsonObject.class));
+
+ Object dynamic = json.fromJson("{\"common\":1}", Object.class);
+ assertSame(firstKey((Map<?, ?>) dynamic), canonical);
+ assertSame(
+ firstKey(json.fromJson("{\"common\":1}", new TypeRef<Map<String,
Integer>>() {})),
+ canonical);
+ assertSame(
+ firstKey(json.fromJson("{\"common\":\"v\"}", new TypeRef<Map<String,
String>>() {})),
+ canonical);
+ Map<String, TypedFields> generic =
+ json.fromJson("{\"common\":{\"value\":1}}", new TypeRef<Map<String,
TypedFields>>() {});
+ assertSame(firstKey(generic), canonical);
+ assertEquals(generic.get("common").value, 1);
+ assertSame(
+ firstKey(json.fromJson("{\"common\":1}", new TypeRef<Map<Object,
Object>>() {})),
+ canonical);
+ }
+
+ @Test
+ public void convertedKeysAreNotCached() {
+ ForyJson json = newJson(64);
+ Map<Integer, Integer> values =
+ json.fromJson("{\"7\":1}", new TypeRef<Map<Integer, Integer>>() {});
+ assertEquals(values.get(7), Integer.valueOf(1));
+ assertNull(
+ JsonTestSupport.primaryTypeResolver(json)
+ .sharedRegistry()
+ .cachedFieldName(JsonFieldNameHash.hash("7")));
+ }
+
+ @Test
+ public void eligibilityBoundaries() {
+ assertCached(newJson(64), "");
+ assertCached(newJson(64), "a");
+ assertCached(newJson(64), "12345678");
+ assertCached(newJson(64), "123456789");
+ assertCached(newJson(64), "1234567890123456");
+
+ assertNotCached(newJson(64), "12345678901234567");
+ assertNotCached(newJson(64), "café");
+ assertNotCached(newJson(64), "键");
+ ForyJson utf8 = newJson(64);
+ byte[] multibyteName = "{\"键\":1}".getBytes(StandardCharsets.UTF_8);
+ String utf8First = firstKey(utf8.fromJson(multibyteName,
JsonObject.class));
+ String utf8Second = firstKey(utf8.fromJson(multibyteName,
JsonObject.class));
+ assertEquals(utf8Second, "键");
+ assertNotSame(utf8Second, utf8First);
+ ForyJson json = newJson(64);
+ String escapedFirst = firstKey(json.fromJson("{\"na\\u006de\":1}",
JsonObject.class));
+ String escapedSecond = firstKey(json.fromJson("{\"na\\u006de\":1}",
JsonObject.class));
+ assertEquals(escapedFirst, "name");
+ assertNotSame(escapedSecond, escapedFirst);
+ }
+
+ @Test
+ public void longFieldNames() {
+ ForyJson json = newJson(64);
+ assertNotCached(json, "abcdefghijklmnopqrstuvwxyz0123456789");
+ assertNotCached(json, "abcdefghijklmnopé");
+
+ String first = firstKey(json.fromJson("{\"abcdefghijklmnop\\u0071\":1}",
JsonObject.class));
+ String second = firstKey(json.fromJson("{\"abcdefghijklmnop\\u0071\":1}",
JsonObject.class));
+ assertEquals(first, "abcdefghijklmnopq");
+ assertEquals(second, first);
+ assertNotSame(second, first);
+ }
+
+ @Test
+ public void disabledAndLocalFull() {
+ ForyJson disabled = newJson(0);
+ assertNotCached(disabled, "name");
+ assertNull(
+ JsonTestSupport.primaryTypeResolver(disabled)
+ .sharedRegistry()
+ .cachedFieldName(JsonFieldNameHash.hash("name")));
+
+ ForyJson bounded = newJson(1);
+ assertCached(bounded, "first");
+ assertNotCached(bounded, "second");
+ assertCached(bounded, "first");
+ }
+
+ @Test
+ public void localFullFallbacks() {
+ ForyJson utf8 = newJson(1);
+ byte[] utf8First = "{\"first\":1}".getBytes(StandardCharsets.UTF_8);
+ byte[] utf8Second = "{\"second\":1}".getBytes(StandardCharsets.UTF_8);
+ String utf8Canonical = firstKey(utf8.fromJson(utf8First,
JsonObject.class));
+ String utf8Second1 = firstKey(utf8.fromJson(utf8Second, JsonObject.class));
+ String utf8Second2 = firstKey(utf8.fromJson(utf8Second, JsonObject.class));
+ assertNotSame(utf8Second2, utf8Second1);
+ byte[] utf8Long =
"{\"abcdefghijklmnop\":1}".getBytes(StandardCharsets.UTF_8);
+ String utf8Long1 = firstKey(utf8.fromJson(utf8Long, JsonObject.class));
+ String utf8Long2 = firstKey(utf8.fromJson(utf8Long, JsonObject.class));
+ assertEquals(utf8Long2, "abcdefghijklmnop");
+ assertNotSame(utf8Long2, utf8Long1);
+ assertSame(firstKey(utf8.fromJson(utf8First, JsonObject.class)),
utf8Canonical);
+
+ ForyJson utf16 = newJson(1);
+ String utf16Canonical = firstKey(utf16.fromJson("{\"first\":\"中文\"}",
JsonObject.class));
+ String utf16Second1 = firstKey(utf16.fromJson("{\"second\":\"中文\"}",
JsonObject.class));
+ String utf16Second2 = firstKey(utf16.fromJson("{\"second\":\"中文\"}",
JsonObject.class));
+ assertNotSame(utf16Second2, utf16Second1);
+ String utf16Long1 =
firstKey(utf16.fromJson("{\"abcdefghijklmnop\":\"中文\"}", JsonObject.class));
+ String utf16Long2 =
firstKey(utf16.fromJson("{\"abcdefghijklmnop\":\"中文\"}", JsonObject.class));
+ assertEquals(utf16Long2, "abcdefghijklmnop");
+ assertNotSame(utf16Long2, utf16Long1);
+ assertSame(firstKey(utf16.fromJson("{\"first\":\"中文\"}",
JsonObject.class)), utf16Canonical);
+ }
+
+ @Test
+ public void localLimitSkipsShared() {
+ ForyJson json = newJson(1, 2);
+ JsonSharedRegistry registry =
JsonTestSupport.primaryTypeResolver(json).sharedRegistry();
+ Latin1JsonReader secondary =
+ (Latin1JsonReader) JsonTestSupport.secondaryStateField(json, 0,
"latin1Reader");
+ String sharedName = secondary.reset(latin1Bytes("\"b\"")).readFieldName();
+ CachedFieldName shared =
registry.cachedFieldName(JsonFieldNameHash.hash("b"));
+ assertNotNull(shared);
+ assertSame(shared.name(), sharedName);
+ String first = firstKey(json.fromJson("{\"a\":1}", JsonObject.class));
+
+ String fallback1 = firstKey(json.fromJson("{\"b\":1}", JsonObject.class));
+ String fallback2 = firstKey(json.fromJson("{\"b\":1}", JsonObject.class));
+ assertNotSame(fallback1, shared.name());
+ assertNotSame(fallback2, shared.name());
+ assertNotSame(fallback2, fallback1);
+ assertSame(firstKey(json.fromJson("{\"a\":1}", JsonObject.class)), first);
+ assertSame(registry.cachedFieldName(JsonFieldNameHash.hash("b")), shared);
+ CachedFieldName primaryEntry =
registry.cachedFieldName(JsonFieldNameHash.hash("a"));
+ assertNotNull(primaryEntry);
+ assertSame(primaryEntry.name(), first);
+ }
+
+ @Test
+ public void adjacentSlotHit() {
+ ForyJson json = newJson(2);
+ String home = firstKey(json.fromJson("{\"a\":1}", JsonObject.class));
+ String adjacent = firstKey(json.fromJson("{\"e\":1}", JsonObject.class));
+
+ assertSame(firstKey(json.fromJson("{\"e\":2}", JsonObject.class)),
adjacent);
+ assertSame(firstKey(json.fromJson("{\"a\":2}", JsonObject.class)), home);
+ }
+
+ @Test
+ public void localConflictSkipsShared() {
+ ForyJson json = newJson(3, 2);
+ JsonSharedRegistry registry =
JsonTestSupport.primaryTypeResolver(json).sharedRegistry();
+ Latin1JsonReader secondary =
+ (Latin1JsonReader) JsonTestSupport.secondaryStateField(json, 0,
"latin1Reader");
+ String sharedName = secondary.reset(latin1Bytes("\"q\"")).readFieldName();
+ CachedFieldName shared =
registry.cachedFieldName(JsonFieldNameHash.hash("q"));
+ assertNotNull(shared);
+ assertSame(shared.name(), sharedName);
+ String home = firstKey(json.fromJson("{\"a\":1}", JsonObject.class));
+ String adjacent = firstKey(json.fromJson("{\"i\":1}", JsonObject.class));
+
+ String fallback1 = firstKey(json.fromJson("{\"q\":1}", JsonObject.class));
+ String fallback2 = firstKey(json.fromJson("{\"q\":1}", JsonObject.class));
+ assertNotSame(fallback1, shared.name());
+ assertNotSame(fallback2, shared.name());
+ assertNotSame(fallback2, fallback1);
+ assertSame(firstKey(json.fromJson("{\"a\":2}", JsonObject.class)), home);
+ assertSame(firstKey(json.fromJson("{\"i\":2}", JsonObject.class)),
adjacent);
+ assertSame(registry.cachedFieldName(JsonFieldNameHash.hash("q")), shared);
+ }
+
+ @Test
+ public void pooledReadersShareNames() {
+ ForyJson json =
+ ForyJson.builder()
+ .withCodegen(false)
+ .withAsyncCompilation(false)
+ .withConcurrencyLevel(2)
+ .withMaxCachedFieldNames(1)
+ .build();
+ Latin1JsonReader primary =
+ (Latin1JsonReader) JsonTestSupport.primaryStateField(json,
"latin1Reader");
+ Latin1JsonReader secondary =
+ (Latin1JsonReader) JsonTestSupport.secondaryStateField(json, 0,
"latin1Reader");
+
+ String first = primary.reset(latin1Bytes("\"shared\"")).readFieldName();
+ String second = secondary.reset(latin1Bytes("\"shared\"")).readFieldName();
+ assertSame(second, first);
+ }
+
+ @Test
+ public void typedFieldsAreNotCached() {
+ assertTypedFieldsNotCached(false);
+ assertTypedFieldsNotCached(true);
+ }
+
+ @Test
+ public void valuesAreNotCached() {
+ ForyJson json = newJson(64);
+ JsonObject first = json.fromJson("{\"name\":\"name\"}", JsonObject.class);
+ JsonObject second = json.fromJson("{\"name\":\"name\"}", JsonObject.class);
+ String key = firstKey(first);
+ assertNotSame(first.get("name"), key);
+ assertNotSame(second.get("name"), first.get("name"));
+ assertSame(firstKey(second), key);
+ }
+
+ @Test
+ public void separateRuntimes() {
+ String first = firstKey(newJson(64).fromJson("{\"name\":1}",
JsonObject.class));
+ String second = firstKey(newJson(64).fromJson("{\"name\":1}",
JsonObject.class));
+ assertNotSame(second, first);
+ }
+
+ @Test
+ public void malformedNames() {
+ ForyJson json = newJson(64);
+ assertThrows(ForyJsonException.class, () ->
json.fromJson("{\"bad\\q\":1}", JsonObject.class));
+ assertThrows(
+ ForyJsonException.class,
+ () -> json.fromJson("{\"abcdefghijklmnop\\q\":1}", JsonObject.class));
+ assertThrows(
+ ForyJsonException.class, () -> json.fromJson("{\"bad\u0001\":1}",
JsonObject.class));
+ assertThrows(
+ ForyJsonException.class,
+ () -> json.fromJson("{\"abcdefghijklmnop\u0001\":1}",
JsonObject.class));
+ assertThrows(
+ ForyJsonException.class, () -> json.fromJson("{\"abcdefghijklmnopq",
JsonObject.class));
+ assertThrows(ForyJsonException.class, () ->
json.fromJson("{\"\ud800\":1}", JsonObject.class));
+
+ byte[] invalidUtf8 = {'{', '"', (byte) 0xc0, (byte) 0x80, '"', ':', '1',
'}'};
+ assertThrows(ForyJsonException.class, () -> json.fromJson(invalidUtf8,
JsonObject.class));
+ JsonSharedRegistry registry =
JsonTestSupport.primaryTypeResolver(json).sharedRegistry();
+ assertNull(registry.cachedFieldName(JsonFieldNameHash.hash("bad")));
+ assertNull(registry.cachedFieldName(JsonFieldNameHash.hash("")));
+ }
+
+ @Test
+ public void sharedHashCollision() {
+ JsonConfig config = JsonTestSupport.config(newJson(4));
+ JsonSharedRegistry registry = new JsonSharedRegistry(config);
+ CachedFieldName first = registry.cacheFieldName(1L, "a", 'a', 0);
+ CachedFieldName second = registry.cacheFieldName(1L, "b", 'b', 0);
+ assertSame(second, first);
+ assertEquals(first.name(), "a");
+ assertSame(registry.cachedFieldName(1L), first);
+ }
+
+ @Test
+ public void readerHashCollision() {
+ ForyJson json = newJson(8);
+ String expected = "collisionKey";
+ String different = "differentKey";
+ long hash = JsonFieldNameHash.hash(expected);
+ long[] words = pack(different);
+ JsonSharedRegistry registry =
JsonTestSupport.primaryTypeResolver(json).sharedRegistry();
+ CachedFieldName collision =
+ registry.cacheFieldName(hash, new String(different), words[0],
words[1]);
+
+ assertCollisionFallback(
+ collision,
+ firstKey(json.fromJson("{\"collisionKey\":1}", JsonObject.class)),
+ firstKey(json.fromJson("{\"collisionKey\":2}", JsonObject.class)));
+ assertCollisionFallback(
+ collision,
+ firstKey(json.fromJson("{\"collisionKey\":\"中文\"}", JsonObject.class)),
+ firstKey(json.fromJson("{\"collisionKey\":\"中文\"}",
JsonObject.class)));
+ byte[] utf8 = "{\"collisionKey\":1}".getBytes(StandardCharsets.UTF_8);
+ assertCollisionFallback(
+ collision,
+ firstKey(json.fromJson(utf8, JsonObject.class)),
+ firstKey(json.fromJson(utf8, JsonObject.class)));
+ assertSame(registry.cachedFieldName(hash), collision);
+ }
+
+ @Test
+ public void concurrentPublication() throws Exception {
+ JsonConfig config = JsonTestSupport.config(newJson(1));
+ JsonSharedRegistry registry = new JsonSharedRegistry(config);
+ int threads = 8;
+ ExecutorService executor = Executors.newFixedThreadPool(threads);
+ CountDownLatch start = new CountDownLatch(1);
+ CountDownLatch done = new CountDownLatch(threads);
+ AtomicReferenceArray<CachedFieldName> results = new
AtomicReferenceArray<>(threads);
+ String name = "name";
+ long hash = JsonFieldNameHash.hash(name);
+ long[] words = pack(name);
+ for (int i = 0; i < threads; i++) {
+ final int index = i;
+ executor.execute(
+ () -> {
+ try {
+ start.await();
+ results.set(
+ index, registry.cacheFieldName(hash, new String(name),
words[0], words[1]));
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ } finally {
+ done.countDown();
+ }
+ });
+ }
+ start.countDown();
+ assertTrue(done.await(10, TimeUnit.SECONDS));
+ executor.shutdown();
+ executor.awaitTermination(10, TimeUnit.SECONDS);
+ for (int i = 1; i < threads; i++) {
+ assertSame(results.get(i), results.get(0));
+ }
+ assertSame(registry.cachedFieldName(hash), results.get(0));
+ }
+
+ private static ForyJson newJson(int maxCachedFieldNames) {
+ return newJson(maxCachedFieldNames, 1);
+ }
+
+ private static ForyJson newJson(int maxCachedFieldNames, int
concurrencyLevel) {
+ return ForyJson.builder()
+ .withCodegen(false)
+ .withAsyncCompilation(false)
+ .withConcurrencyLevel(concurrencyLevel)
+ .withMaxCachedFieldNames(maxCachedFieldNames)
+ .build();
+ }
+
+ private static void assertTypedFieldsNotCached(boolean codegen) {
+ ForyJson json =
+ ForyJson.builder()
+ .withCodegen(codegen)
+ .withAsyncCompilation(false)
+ .withConcurrencyLevel(1)
+ .withMaxCachedFieldNames(64)
+ .build();
+ TypedFields value = json.fromJson("{\"value\":1}", TypedFields.class);
+ assertEquals(value.value, 1);
+ assertNull(
+ JsonTestSupport.primaryTypeResolver(json)
+ .sharedRegistry()
+ .cachedFieldName(JsonFieldNameHash.hash("value")));
+ }
+
+ private static void assertCached(ForyJson json, String name) {
+ String first = firstKey(json.fromJson(jsonForName(name),
JsonObject.class));
+ String second = firstKey(json.fromJson(jsonForName(name),
JsonObject.class));
+ assertSame(second, first);
+ CachedFieldName entry =
+ JsonTestSupport.primaryTypeResolver(json)
+ .sharedRegistry()
+ .cachedFieldName(JsonFieldNameHash.hash(name));
+ assertNotNull(entry);
+ assertSame(entry.name(), first);
+ }
+
+ private static void assertNotCached(ForyJson json, String name) {
+ String first = firstKey(json.fromJson(jsonForName(name),
JsonObject.class));
+ String second = firstKey(json.fromJson(jsonForName(name),
JsonObject.class));
+ assertEquals(first, name);
+ assertEquals(second, name);
+ assertNotSame(second, first);
+ }
+
+ private static void assertCollisionFallback(
+ CachedFieldName collision, String first, String second) {
+ assertEquals(first, "collisionKey");
+ assertEquals(second, "collisionKey");
+ assertNotSame(first, collision.name());
+ assertNotSame(second, collision.name());
+ assertNotSame(second, first);
+ }
+
+ private static String jsonForName(String name) {
+ return "{\"" + name + "\":1}";
+ }
+
+ private static byte[] latin1Bytes(String json) {
+ return json.getBytes(StandardCharsets.ISO_8859_1);
+ }
+
+ private static String firstKey(Map<?, ?> map) {
+ return (String) map.keySet().iterator().next();
+ }
+
+ private static long[] pack(String name) {
+ long word0 = 0;
+ long word1 = 0;
+ for (int i = 0; i < name.length(); i++) {
+ if (i < Long.BYTES) {
+ word0 |= ((long) name.charAt(i)) << (i << 3);
+ } else {
+ word1 |= ((long) name.charAt(i)) << ((i - Long.BYTES) << 3);
+ }
+ }
+ return new long[] {word0, word1};
+ }
+
+ public static class TypedFields {
+ public int value;
+ }
+}
diff --git
a/java/fory-json/src/test/java/org/apache/fory/json/JsonTestSupport.java
b/java/fory-json/src/test/java/org/apache/fory/json/JsonTestSupport.java
index 1a95060b4..515d43029 100644
--- a/java/fory-json/src/test/java/org/apache/fory/json/JsonTestSupport.java
+++ b/java/fory-json/src/test/java/org/apache/fory/json/JsonTestSupport.java
@@ -43,6 +43,7 @@ final class JsonTestSupport {
PropertyNamingStrategy.LOWER_CAMEL_CASE,
JsonTestSupport.class.getClassLoader(),
ForyJson.DEFAULT_MAX_DEPTH,
+ ForyJson.DEFAULT_MAX_CACHED_FIELD_NAMES,
1,
2 * 1024 * 1024,
new CodecRegistry(),
@@ -135,6 +136,17 @@ final class JsonTestSupport {
}
}
+ static Object secondaryStateField(ForyJson json, int index, String name) {
+ try {
+ AtomicReferenceArray<?> slots = (AtomicReferenceArray<?>) field(json,
"slots");
+ Object pooledState = slots.get(index);
+ Object state = field(pooledState, "state");
+ return field(state, name);
+ } catch (ReflectiveOperationException e) {
+ throw new AssertionError(e);
+ }
+ }
+
static JsonConfig config(ForyJson json) {
try {
return (JsonConfig) field(json, "config");
diff --git
a/java/fory-json/src/test/java/org/apache/fory/json/reader/FieldNameCacheTest.java
b/java/fory-json/src/test/java/org/apache/fory/json/reader/FieldNameCacheTest.java
new file mode 100644
index 000000000..5b164b1b5
--- /dev/null
+++
b/java/fory-json/src/test/java/org/apache/fory/json/reader/FieldNameCacheTest.java
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.fory.json.reader;
+
+import static org.testng.Assert.assertFalse;
+import static org.testng.Assert.assertNull;
+import static org.testng.Assert.assertSame;
+import static org.testng.Assert.assertTrue;
+
+import java.lang.reflect.Field;
+import org.apache.fory.json.ForyJson;
+import org.apache.fory.json.meta.JsonFieldNameHash;
+import org.apache.fory.json.resolver.JsonSharedRegistry;
+import org.apache.fory.json.resolver.JsonSharedRegistry.CachedFieldName;
+import org.testng.annotations.Test;
+
+public class FieldNameCacheTest {
+ private static final JsonSharedRegistry REGISTRY = registry();
+
+ @Test
+ public void fixedCandidatesAndLimit() {
+ FieldNameCache cache = new FieldNameCache(3);
+ CachedFieldName home = entry("home");
+ CachedFieldName adjacent = entry("adjacent");
+ CachedFieldName fallback = entry("fallback");
+ CachedFieldName unrelated = entry("unrelated");
+
+ assertNull(cache.get(0L));
+ assertTrue(cache.canPut(0L));
+ cache.put(0L, home);
+ assertSame(cache.get(0L), home);
+
+ assertTrue(cache.canPut(8L));
+ cache.put(8L, adjacent);
+ assertSame(cache.get(8L), adjacent);
+
+ assertFalse(cache.canPut(16L));
+ cache.put(16L, fallback);
+ assertNull(cache.get(16L));
+ assertSame(cache.get(0L), home);
+ assertSame(cache.get(8L), adjacent);
+
+ assertTrue(cache.canPut(2L));
+ cache.put(2L, unrelated);
+ assertSame(cache.get(2L), unrelated);
+ assertFalse(cache.canPut(3L));
+ cache.put(3L, fallback);
+ assertNull(cache.get(3L));
+ }
+
+ @Test
+ public void idempotentPut() {
+ FieldNameCache cache = new FieldNameCache(1);
+ CachedFieldName first = entry("first");
+ CachedFieldName second = entry("second");
+
+ cache.put(1L, first);
+ cache.put(1L, first);
+ cache.put(1L, second);
+
+ assertSame(cache.get(1L), first);
+ assertFalse(cache.canPut(1L));
+ }
+
+ private static JsonSharedRegistry registry() {
+ try {
+ ForyJson json = ForyJson.builder().withCodegen(false).build();
+ Field field = ForyJson.class.getDeclaredField("sharedRegistry");
+ field.setAccessible(true);
+ return (JsonSharedRegistry) field.get(json);
+ } catch (NoSuchFieldException | IllegalAccessException e) {
+ throw new AssertionError(e);
+ }
+ }
+
+ private static CachedFieldName entry(String name) {
+ long word0 = 0;
+ long word1 = 0;
+ for (int i = 0; i < name.length(); i++) {
+ if (i < Long.BYTES) {
+ word0 |= ((long) name.charAt(i)) << (i << 3);
+ } else {
+ word1 |= ((long) name.charAt(i)) << ((i - Long.BYTES) << 3);
+ }
+ }
+ return REGISTRY.cacheFieldName(JsonFieldNameHash.hash(name), name, word0,
word1);
+ }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
