agturley commented on code in PR #11299:
URL: https://github.com/apache/nifi/pull/11299#discussion_r3336090099
##########
nifi-extension-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchJson.java:
##########
@@ -450,17 +526,36 @@ public void onTrigger(final ProcessContext context, final
ProcessSession session
final IndexOperationRequest opRequest;
final long docBytes;
if (o == IndexOperationRequest.Operation.Index ||
o == IndexOperationRequest.Operation.Create) {
- final String id = extractId(trimmedLine,
documentIdField, flowFileIdAttribute);
final byte[] rawJsonBytes;
- if (suppressingWriter != null) {
- // Parse to Map so NON_NULL/NON_EMPTY
inclusion filters apply during serialization.
- // JsonNode tree serialization bypasses
JsonInclude filters.
- rawJsonBytes =
suppressingWriter.writeValueAsBytes(mapReader.readValue(trimmedLine));
+ final String id;
+ final String docIndex;
+ final boolean stripId = !retainIdentifierField
&& StringUtils.isNotBlank(documentIdField);
+ final boolean stripIdx = !retainIndexField &&
StringUtils.isNotBlank(documentIndexField);
+ final boolean needsTimestamp =
StringUtils.isNotBlank(documentTimestampField);
+ if (suppressingWriter != null || stripId ||
stripIdx || needsTimestamp) {
+ // Map is needed anyway — extract both
fields from the Map directly.
+ final Map<String, Object> contentMap =
mapReader.readValue(trimmedLine);
+ id = resolveId(contentMap,
documentIdField, flowFileIdAttribute);
+ docIndex = resolveIndex(contentMap,
documentIndexField, index);
+ if (stripId) {
+ contentMap.remove(documentIdField);
+ }
+ if (stripIdx) {
+ contentMap.remove(documentIndexField);
+ }
+ applyTimestamp(contentMap,
documentTimestampField, retainTimestampField);
Review Comment:
Added unit tests covering Index Field, Timestamp Field, and all three Retain
toggles across NDJSON, JSON Array, and Single JSON — extraction,
retain-by-default, stripping when Retain=false, and fallback to the Index
property when the field is absent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
