rzo1 commented on PR #1868:
URL: https://github.com/apache/stormcrawler/pull/1868#issuecomment-4266130210
@dpol1 opensearch-java 3.x stores hit.source() as a jakarta.json.JsonObject
(backed by Parsson). JsonData.to(Class) has this shortcut in JsonDataImpl:
` if (clazz.isAssignableFrom(value.getClass())) return (T) value;
`
Since Object.class/Map.class are assignable from JsonObject,
hit.source().to(Object.class) returns the raw Parsson JsonObject whose values
are JsonString/JsonNumber, not plain String. The (String) keyValues.get(...)
then blows up.
Could you check if the following does solve it?
```
Index:
external/opensearch-java/src/test/java/org/apache/stormcrawler/opensearch/bolt/AbstractOpenSearchTest.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git
a/external/opensearch-java/src/test/java/org/apache/stormcrawler/opensearch/bolt/AbstractOpenSearchTest.java
b/external/opensearch-java/src/test/java/org/apache/stormcrawler/opensearch/bolt/AbstractOpenSearchTest.java
---
a/external/opensearch-java/src/test/java/org/apache/stormcrawler/opensearch/bolt/AbstractOpenSearchTest.java
(revision 6fd51e4ae8b73d468ad7af1d0dde18ecde9ef50c)
+++
b/external/opensearch-java/src/test/java/org/apache/stormcrawler/opensearch/bolt/AbstractOpenSearchTest.java
(date 1776410127001)
@@ -27,7 +27,7 @@
public abstract class AbstractOpenSearchTest {
private static final String OPENSEARCH_VERSION =
- System.getProperty("opensearch-version", "2.19.5");
+ System.getProperty("opensearch-version", "3.5.0");
public static final String PASSWORD = "This1sAPassw0rd";
Index:
external/opensearch-java/src/main/java/org/apache/stormcrawler/opensearch/persistence/HybridSpout.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git
a/external/opensearch-java/src/main/java/org/apache/stormcrawler/opensearch/persistence/HybridSpout.java
b/external/opensearch-java/src/main/java/org/apache/stormcrawler/opensearch/persistence/HybridSpout.java
---
a/external/opensearch-java/src/main/java/org/apache/stormcrawler/opensearch/persistence/HybridSpout.java
(revision 6fd51e4ae8b73d468ad7af1d0dde18ecde9ef50c)
+++
b/external/opensearch-java/src/main/java/org/apache/stormcrawler/opensearch/persistence/HybridSpout.java
(date 1776410122100)
@@ -24,6 +24,7 @@
import java.time.ZoneOffset;
import java.time.format.DateTimeFormatter;
import java.util.Date;
+import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
@@ -214,9 +215,12 @@
for (Hit<JsonData> hit : hits) {
numDocs++;
+ // Passing LinkedHashMap (not Object/Map) forces the
JsonpMapper to deserialize
+ // via Jackson, producing plain Java values
(String/Number/List/Map). With
+ // Object/Map the raw jakarta.json.JsonObject is returned
and values remain
+ // JsonValue instances (e.g. parsson JsonStringImpl),
breaking (String) casts.
@SuppressWarnings("unchecked")
- Map<String, Object> sourceAsMap =
- (Map<String, Object>) hit.source().to(Object.class);
+ Map<String, Object> sourceAsMap =
hit.source().to(LinkedHashMap.class);
String pfield = partitionField;
Map<String, Object> fieldSource = sourceAsMap;
Index:
external/opensearch-java/src/main/java/org/apache/stormcrawler/opensearch/persistence/AggregationSpout.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git
a/external/opensearch-java/src/main/java/org/apache/stormcrawler/opensearch/persistence/AggregationSpout.java
b/external/opensearch-java/src/main/java/org/apache/stormcrawler/opensearch/persistence/AggregationSpout.java
---
a/external/opensearch-java/src/main/java/org/apache/stormcrawler/opensearch/persistence/AggregationSpout.java
(revision 6fd51e4ae8b73d468ad7af1d0dde18ecde9ef50c)
+++
b/external/opensearch-java/src/main/java/org/apache/stormcrawler/opensearch/persistence/AggregationSpout.java
(date 1776410114080)
@@ -26,6 +26,7 @@
import java.util.Date;
import java.util.HashSet;
import java.util.Iterator;
+import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
@@ -288,8 +289,12 @@
TopHitsAggregate topHits =
entry.aggregations().get("docs").topHits();
for (Hit<JsonData> hit : topHits.hits().hits()) {
+ // Passing LinkedHashMap (not Object/Map) forces the
JsonpMapper to deserialize
+ // via Jackson, producing plain Java values
(String/Number/List/Map). With
+ // Object/Map the raw jakarta.json.JsonObject is returned
and values remain
+ // JsonValue instances (e.g. parsson JsonStringImpl),
breaking (String) casts.
@SuppressWarnings("unchecked")
- Map<String, Object> keyValues = (Map<String, Object>)
hit.source().to(Object.class);
+ Map<String, Object> keyValues =
hit.source().to(LinkedHashMap.class);
LOG.debug("{} -> id [{}], _source [{}]", logIdprefix,
hit.id(), keyValues);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
