dan-s1 commented on code in PR #9196:
URL: https://github.com/apache/nifi/pull/9196#discussion_r1735357584
##########
nifi-extension-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestValidateRecord.java:
##########
@@ -593,6 +594,47 @@ public void testValidateMaps() throws IOException,
InitializationException, Malf
}
}
+ @Test
+ void testTimeZoneWithTimeStamp() throws Exception {
+ final TimeZone defaultTimezone = TimeZone.getDefault();
+ //Ensure the timezone is UTC in order to validate the use of a
timezone in the timestamp pattern is taken into account.
+ TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
+ final String timestampWithTimeZonePattern = "EEE, dd MMM yyyy
HH:mm:ssZ";
+ final String schema = new
String(Files.readAllBytes(Paths.get("src/test/resources/TestValidateRecord/timestampWithTimeZone.avsc")),
"UTF-8");
+ final CSVReader csvReader = new CSVReader();
+ runner.addControllerService("reader", csvReader);
+ runner.setProperty(csvReader,
SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY,
SchemaAccessUtils.SCHEMA_TEXT_PROPERTY);
+ runner.setProperty(csvReader, SchemaAccessUtils.SCHEMA_TEXT, schema);
+ runner.setProperty(csvReader, CSVUtils.FIRST_LINE_IS_HEADER, "true");
+ runner.setProperty(csvReader, CSVUtils.VALUE_SEPARATOR, "◆");
+ runner.setProperty(csvReader, DateTimeUtils.TIMESTAMP_FORMAT,
timestampWithTimeZonePattern);
+ runner.enableControllerService(csvReader);
+
+ final CSVRecordSetWriter csvWriter = new CSVRecordSetWriter();
+ runner.addControllerService("writer", csvWriter);
+ runner.setProperty(csvWriter, "Schema Write Strategy",
"full-schema-attribute");
+ runner.setProperty(csvWriter, DateTimeUtils.TIMESTAMP_FORMAT,
timestampWithTimeZonePattern);
+ runner.enableControllerService(csvWriter);
+
+ runner.setProperty(ValidateRecord.RECORD_READER, "reader");
+ runner.setProperty(ValidateRecord.RECORD_WRITER, "writer");
+ runner.setProperty(ValidateRecord.ALLOW_EXTRA_FIELDS, "false");
+ runner.setProperty(ValidateRecord.MAX_VALIDATION_DETAILS_LENGTH,
"4000");
+ runner.setProperty(ValidateRecord.VALIDATION_DETAILS_ATTRIBUTE_NAME,
"valDetails");
+
+ final String content =
"apache_date◆apache_ip_source◆apache_method◆apache_path◆apache_query_string◆apache_response_code◆apache_referer◆apache_user_agent\n"
+
+ "Wed, 24 Jul 2024
15:04:23+0200◆10.4.3.20◆GET◆/path◆?test=toto◆200◆-◆";
+
+ runner.enqueue(content);
+ runner.run();
+
+ runner.assertTransferCount(ValidateRecord.REL_VALID, 1);
+ final MockFlowFile validFlowFile =
runner.getFlowFilesForRelationship(ValidateRecord.REL_VALID).get(0);
+ //Validate timezone is taken into account
+ assertTrue(validFlowFile.getContent().contains("Wed, 24 Jul 2024
13:04:23+0000"));
Review Comment:
@exceptionfactory You were %100 right it took a few iterations to get this
right. But after this
[search](https://github.com/search?q=repo%3Aapache%2Fnifi++Excel&type=pullrequests&p=2)
I remembered I had locale issues when developing the Excel Record Reader in
#7194. When I set the locales I realized I could not use this `"Wed, 24 Jul
2024 15:04:23+0200"` so I ended up removing the three character day of the
week, the three day month and the `+0200` timezone. I also tweaked the
timezone regular expression to cover all cases.
Please let me know what else needs to be done. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
