I have a question about the expected behavior of convertSimpleIfPossible in CSVRecordReader.java (NiFi 1.5.0).
I have a custom CSV file that I am taking to an avro schema using ValidateRecord. The schema contains a logical date type and the CSV has the date in the format MM/DD/YYYY. I expected to provide the date string in the controller element for the CSV reader and have everything parse happily, but it ends up throwing an exception when it tries to parse things in the avro writer (String->Date). I don't think I should be blaming the avro writer because I expected the CSV reader to parse the date for me. I did a little digging in the CSVRecordReader.java, and I see everything flows through convertSimpleIfPossible when parsing the data, and each data type is checked with DataTypeUtils.isCompatibleDataType prior to actually trying to perform the conversion. The date string doesn't use the user provided format in the call to DataTypeUtils.isCompatibleDataType, but instead uses the default for date types. The validation ends up failing when it uses the default date string (YYYY-MM-DD), so it won't use LAZY_DATE_FORMAT as I expected. Am I totally off base, or it this unexpected behavior? Thanks. -Derek
