Hi Derek,

By looking at the code briefly, I guess you are using ValidateRecord
processor with CSVReader and AvroWriter..
As you pointed out, it seems DataTypeUtils.isCompatibleDataType does
not use the date format user defined at CSVReader.

Is it possible for you to share followings for us to reproduce and
understand it better?
- Sample input CSV file
- NiFi flow template using CSVReader and AvroWriter


On Wed, Feb 14, 2018 at 7:11 AM, Derek Straka <de...@asterius.io> wrote:
> I have a question about the expected behavior of convertSimpleIfPossible in
> CSVRecordReader.java (NiFi 1.5.0).
> I have a custom CSV file that I am taking to an avro schema using
> ValidateRecord.  The schema contains a logical date type and the CSV has
> the date in the format MM/DD/YYYY.  I expected to provide the date string
> in the controller element for the CSV reader and have everything parse
> happily, but it ends up throwing an exception when it tries to parse things
> in the avro writer (String->Date).  I don't think I should be blaming the
> avro writer because I expected the CSV reader to parse the date for me.
> I did a little digging in the CSVRecordReader.java, and I see everything
> flows through convertSimpleIfPossible when parsing the data, and each data
> type is checked with DataTypeUtils.isCompatibleDataType prior to actually
> trying to perform the conversion.
> The date string doesn't use the user provided format in the call to
> DataTypeUtils.isCompatibleDataType, but instead uses the default for date
> types.  The validation ends up failing when it uses the default date string
> (YYYY-MM-DD), so it won't use LAZY_DATE_FORMAT as I expected.  Am I totally
> off base, or it this unexpected behavior?
> Thanks.
> -Derek

Reply via email to