mattyb149 commented on a change in pull request #4301:
URL: https://github.com/apache/nifi/pull/4301#discussion_r431851903
##########
File path:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateRecord.java
##########
@@ -207,6 +225,8 @@
properties.add(SCHEMA_TEXT);
properties.add(ALLOW_EXTRA_FIELDS);
properties.add(STRICT_TYPE_CHECKING);
+ properties.add(MAX_VALIDATION_DETAILS_LENGTH);
Review comment:
Although the order of properties is not guaranteed (in the UI, e.g.),
they tend to show up in the order they were added. Though not a requirement, I
recommend switching the order of these properties just for clarity, totally up
to you though.
##########
File path:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestValidateRecord.java
##########
@@ -593,5 +593,49 @@ public void testValidateMaps() throws IOException,
InitializationException, Malf
assertEquals(2, ( (Map<?,?>) ((Record)
data[2]).getValue("points")).size());
}
}
+
+ @Test
+ public void testValidationsDetailsAttributeForInvalidRecords() throws
InitializationException, UnsupportedEncodingException, IOException {
+ final String schema = new
String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/schema/person-with-name-string.avsc")),
"UTF-8");
+
+ final CSVReader csvReader = new CSVReader();
+ runner.addControllerService("reader", csvReader);
+ runner.setProperty(csvReader,
SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY,
SchemaAccessUtils.SCHEMA_TEXT_PROPERTY);
+ runner.setProperty(csvReader, SchemaAccessUtils.SCHEMA_TEXT, schema);
+ runner.setProperty(csvReader, CSVUtils.FIRST_LINE_IS_HEADER, "false");
+ runner.setProperty(csvReader, CSVUtils.QUOTE_MODE,
CSVUtils.QUOTE_MINIMAL.getValue());
+ runner.setProperty(csvReader, CSVUtils.TRAILING_DELIMITER, "false");
+ runner.enableControllerService(csvReader);
+
+ final MockRecordWriter validWriter = new MockRecordWriter("valid",
false);
+ runner.addControllerService("writer", validWriter);
+ runner.enableControllerService(validWriter);
+
+ final MockRecordWriter invalidWriter = new MockRecordWriter("invalid",
true);
+ runner.addControllerService("invalid-writer", invalidWriter);
+ runner.enableControllerService(invalidWriter);
+
+ runner.setProperty(ValidateRecord.RECORD_READER, "reader");
+ runner.setProperty(ValidateRecord.RECORD_WRITER, "writer");
+ runner.setProperty(ValidateRecord.INVALID_RECORD_WRITER,
"invalid-writer");
+ runner.setProperty(ValidateRecord.ALLOW_EXTRA_FIELDS, "false");
+ runner.setProperty(ValidateRecord.MAX_VALIDATION_DETAILS_LENGTH, "20");
+ runner.setProperty(ValidateRecord.VALIDATION_DETAILS_ATTRIBUTE_NAME,
"valDetails");
+
+ final String content = "1, John Doe\n"
+ + "2, Jane Doe\n"
+ + "Three, Jack Doe\n";
+
+ runner.enqueue(content);
+ runner.run();
+
+ runner.assertTransferCount(ValidateRecord.REL_INVALID, 1);
+ runner.assertTransferCount(ValidateRecord.REL_FAILURE, 0);
+
+ final MockFlowFile invalidFlowFile =
runner.getFlowFilesForRelationship(ValidateRecord.REL_INVALID).get(0);
+ invalidFlowFile.assertAttributeEquals("record.count", "1");
+ invalidFlowFile.assertContentEquals("invalid\n\"Three\",\"Jack
Doe\"\n");
+ invalidFlowFile.assertAttributeExists("valDetails");
Review comment:
Since you are setting the max details length to 20, probably a good idea
to verify that here. If the output is deterministic for this test, you could
also verify the actual attribute value.
##########
File path:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateRecord.java
##########
@@ -180,6 +180,24 @@
.defaultValue("true")
.required(true)
.build();
+ static final PropertyDescriptor MAX_VALIDATION_DETAILS_LENGTH = new
PropertyDescriptor.Builder()
+ .name("maximum-validation-details-length")
+ .displayName("Maximum Validation Details Length")
+ .description("Specifies the maximum number of characters that
validation details value can have. Any characters beyond the max will be
truncated.")
Review comment:
Just for completeness, it would be good to mention in this description
that this property is only used if `Validation Details Attribute Name` is set.
Also since it's being evaluated at the same time as the attribute name, what do
you think about supporting FlowFile attributes for the expression evaluation?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
