ChrisSamo632 commented on a change in pull request #4691:
URL: https://github.com/apache/nifi/pull/4691#discussion_r723727008
##########
File path:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchHttpRecord.java
##########
@@ -266,9 +283,11 @@
descriptors.add(RECORD_WRITER);
descriptors.add(LOG_ALL_ERRORS);
descriptors.add(ID_RECORD_PATH);
+ descriptors.add(AT_TIMESTAMP_RECORD_PATH);
descriptors.add(INDEX);
descriptors.add(TYPE);
descriptors.add(INDEX_OP);
+ descriptors.add(AT_TIMESTAMP);
Review comment:
I was following the existing pattern of properties - the direct value
properties (e.g. index, type) all appear together at the top of the processor;
the Record Path lookup properties then appear in a separate group later on
We could re-organise all the properties, but I didn't really want to do that
initially at least
What do you think?
##########
File path:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchHttpRecord.java
##########
@@ -209,6 +207,25 @@
.required(true)
.build();
+ static final PropertyDescriptor AT_TIMESTAMP = new
PropertyDescriptor.Builder()
+ .name("put-es-record-at-timestamp")
+ .displayName("@timestamp Value")
+ .description("The value to use as the @timestamp field (required
for Elasticsearch Data Streams)")
+ .required(false)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+ .addValidator(StandardValidators.NON_EMPTY_EL_VALIDATOR)
+ .build();
+
+ static final PropertyDescriptor AT_TIMESTAMP_RECORD_PATH = new
PropertyDescriptor.Builder()
+ .name("put-es-record-at-timestamp-path")
+ .displayName("@timestamp Record Path")
+ .description("A RecordPath pointing to a field in the record(s)
that contains the @timestamp for the document " +
+ "(required for Elasticsearch Data Streams). If left blank
the @timestamp will be determined using the main property type")
+ .required(false)
+ .addValidator(new RecordPathValidator())
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+ .build();
Review comment:
Not quite sure I follow what you mean, but I've re-worded the property
description to be more inline with the other existing properties on the
processor, does it make more sense now?
##########
File path:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchHttpRecord.java
##########
@@ -209,6 +207,25 @@
.required(true)
.build();
+ static final PropertyDescriptor AT_TIMESTAMP = new
PropertyDescriptor.Builder()
+ .name("put-es-record-at-timestamp")
+ .displayName("@timestamp Value")
+ .description("The value to use as the @timestamp field (required
for Elasticsearch Data Streams)")
+ .required(false)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+ .addValidator(StandardValidators.NON_EMPTY_EL_VALIDATOR)
+ .build();
+
+ static final PropertyDescriptor AT_TIMESTAMP_RECORD_PATH = new
PropertyDescriptor.Builder()
+ .name("put-es-record-at-timestamp-path")
+ .displayName("@timestamp Record Path")
+ .description("A RecordPath pointing to a field in the record(s)
that contains the @timestamp for the document " +
+ "(required for Elasticsearch Data Streams). If left blank
the @timestamp will be determined using the main property type")
Review comment:
Copy & paste error here I think... tried to keep the property
descriptions more or less the same, but think I messed something up a little -
updated to match the existing Index/Type properties, does this make more sense?
##########
File path:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchHttpRecord.java
##########
@@ -405,11 +424,17 @@ public void onTrigger(final ProcessContext context, final
ProcessSession session
this.nullSuppression = context.getProperty(SUPPRESS_NULLS).getValue();
- final String id_path =
context.getProperty(ID_RECORD_PATH).evaluateAttributeExpressions(flowFile).getValue();
- final RecordPath recordPath = StringUtils.isEmpty(id_path) ? null :
recordPathCache.getCompiled(id_path);
+ final String idPath =
context.getProperty(ID_RECORD_PATH).evaluateAttributeExpressions(flowFile).getValue();
+ final RecordPath recordPath = StringUtils.isEmpty(idPath) ? null :
recordPathCache.getCompiled(idPath);
final StringBuilder sb = new StringBuilder();
final Charset charset =
Charset.forName(context.getProperty(CHARSET).evaluateAttributeExpressions(flowFile).getValue());
+ final String atTimestamp =
context.getProperty(AT_TIMESTAMP).evaluateAttributeExpressions(flowFile).getValue();
+ final String atTimestampPath =
context.getProperty(AT_TIMESTAMP_RECORD_PATH).isSet()
+ ?
context.getProperty(AT_TIMESTAMP_RECORD_PATH).evaluateAttributeExpressions(flowFile).getValue()
+ : null;
Review comment:
I just copied the existing code for the other property values to be
fair, but I think you could be right and all the code like this could be
simplified
##########
File path:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchHttpRecord.java
##########
@@ -209,6 +207,25 @@
.required(true)
.build();
+ static final PropertyDescriptor AT_TIMESTAMP = new
PropertyDescriptor.Builder()
+ .name("put-es-record-at-timestamp")
+ .displayName("@timestamp Value")
+ .description("The value to use as the @timestamp field (required
for Elasticsearch Data Streams)")
+ .required(false)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+ .addValidator(StandardValidators.NON_EMPTY_EL_VALIDATOR)
+ .build();
+
+ static final PropertyDescriptor AT_TIMESTAMP_RECORD_PATH = new
PropertyDescriptor.Builder()
+ .name("put-es-record-at-timestamp-path")
+ .displayName("@timestamp Record Path")
+ .description("A RecordPath pointing to a field in the record(s)
that contains the @timestamp for the document " +
+ "(required for Elasticsearch Data Streams). If left blank
the @timestamp will be determined using the main property type")
Review comment:
Silly me, forgot that I had the two processors with the same properties
and descriptions! 🙄
##########
File path:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchHttpRecord.java
##########
@@ -266,9 +283,11 @@
descriptors.add(RECORD_WRITER);
descriptors.add(LOG_ALL_ERRORS);
descriptors.add(ID_RECORD_PATH);
+ descriptors.add(AT_TIMESTAMP_RECORD_PATH);
descriptors.add(INDEX);
descriptors.add(TYPE);
descriptors.add(INDEX_OP);
+ descriptors.add(AT_TIMESTAMP);
Review comment:
I was looking at the wrong processor, d'oh, my bad! 🙄
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
