This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 62a23dd55a69 [SPARK-50780][SQL] Use `overrideStdFeatures` instead of
`setFeatureMask` in `JacksonParser`
62a23dd55a69 is described below
commit 62a23dd55a69fd426a36d8fb7067e3114f17e0d8
Author: yangjie01 <[email protected]>
AuthorDate: Fri Jan 10 08:28:17 2025 -0800
[SPARK-50780][SQL] Use `overrideStdFeatures` instead of `setFeatureMask` in
`JacksonParser`
### What changes were proposed in this pull request?
In https://github.com/apache/spark/pull/49018, the restoration logic for
feature flags was fixed using the `setFeatureMask` method. However, the
`setFeatureMask` method has been deprecated since Jackson 2.7, so this pr
reimplements the relevant logic using `overrideStdFeatures`.
### Why are the changes needed?
Clean up the use of deprecated APIs.
https://github.com/FasterXML/jackson-core/blob/0d2b0f39200d466f49f1abb06d9027053d41483d/src/main/java/com/fasterxml/jackson/core/JsonParser.java#L999-L1035
```
/**
* Bulk set method for (re)setting states of all standard {link
Feature}s
*
* param mask Bit mask that defines set of features to enable
*
* return This parser, to allow call chaining
*
* since 2.3
* deprecated Since 2.7, use {link #overrideStdFeatures(int, int)}
instead
*/
Deprecated
public JsonParser setFeatureMask(int mask) {
_features = mask;
return this;
}
/**
* Bulk set method for (re)setting states of features specified by
<code>mask</code>.
* Functionally equivalent to
*<code>
* int oldState = getFeatureMask();
* int newState = (oldState & ~mask) | (values & mask);
* setFeatureMask(newState);
*</code>
* but preferred as this lets caller more efficiently specify actual
changes made.
*
* param values Bit mask of set/clear state for features to change
* param mask Bit mask of features to change
*
* return This parser, to allow call chaining
*
* since 2.6
*/
public JsonParser overrideStdFeatures(int values, int mask) {
int newState = (_features & ~mask) | (values & mask);
return setFeatureMask(newState);
}
```
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass GitHub Actions
- Specialized tests have already been added in
https://github.com/apache/spark/pull/49018: "feature mask should remain
unchanged" in `JacksonParserSuite`.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #49434 from LuciferYang/setFeatureMask.
Authored-by: yangjie01 <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../org/apache/spark/sql/catalyst/json/JacksonParser.scala | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala
index 19e2c4228236..1cd4b4cd29bc 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala
@@ -293,7 +293,8 @@ class JacksonParser(
case _: StringType => (parser: JsonParser) => {
// This must be enabled if we will retrieve the bytes directly from the
raw content:
val oldFeature = parser.getFeatureMask
- parser.setFeatureMask(oldFeature |
JsonParser.Feature.INCLUDE_SOURCE_IN_LOCATION.getMask)
+ val featureToAdd = JsonParser.Feature.INCLUDE_SOURCE_IN_LOCATION.getMask
+ parser.overrideStdFeatures(oldFeature | featureToAdd, featureToAdd)
val result = parseJsonToken[UTF8String](parser, dataType) {
case VALUE_STRING =>
UTF8String.fromString(parser.getText)
@@ -338,8 +339,11 @@ class JacksonParser(
UTF8String.fromBytes(writer.toByteArray)
}
}
- // Reset back to the original configuration:
- parser.setFeatureMask(oldFeature)
+ // Reset back to the original configuration using `~0` as the mask,
+ // which is a bitmask with all bits set, effectively allowing all
features
+ // to be reset. This ensures that every feature is restored to its
previous
+ // state as defined by `oldFeature`.
+ parser.overrideStdFeatures(oldFeature, ~0)
result
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
