Re: CSV record parsing with custom date formats

As a possible work around, there were date functions added to record path in 1.5.0, so if you had a schema that treated the field as a string, you could reformat the column in place using UpdateRecord to get it into whatever format it needs to be in. On Tue, Feb 13, 2018 at 9:17 PM Koji Kawamura

Re: CSV record parsing with custom date formats

Hi Derek, By looking at the code briefly, I guess you are using ValidateRecord processor with CSVReader and AvroWriter.. As you pointed out, it seems DataTypeUtils.isCompatibleDataType does not use the date format user defined at CSVReader. Is it possible for you to share followings for us to

CSV record parsing with custom date formats

I have a question about the expected behavior of convertSimpleIfPossible in CSVRecordReader.java (NiFi 1.5.0). I have a custom CSV file that I am taking to an avro schema using ValidateRecord. The schema contains a logical date type and the CSV has the date in the format MM/DD/. I expected

Policies and access to core attributes

In order to use Provenance to view details of a flow file, a user must belong to the 'view the data' policy for a given component(s) along the flow. For example, the lineage graph will show "UNKNOWN" for any component which the user does not possess 'view the data'. Not only can the user not

Re: Will you accept contributions in Scala?

I completely second Mike's sentiment here as well as cautionary statements by other contributors over this thread's history (Matt Burgess' post in particular). Besides fueling flame wars and religious inquisition, you cannot open the door to Scala without opening it to other (JVM languages at

Re: Will you accept contributions in Scala?

Milan, I don't think you can do that without creating a lot of fuel for a flame war. I have personally never met a Scala developer who was incapable of writing decent Java. The same is true of Groovy. I think anyone who finds the requirement to use Java over their language of choice to be a deal

Re: Will you accept contributions in Scala?

I think we should not add blindly any language but should be open to add couple of language like Scala. In Bigdata world Scala/Python/Java are widely accepted. Thanks, Milan Das Interset On 2/13/18, 10:20 AM, "Weiss, Adam" wrote: I think it makes the most

Re: Will you accept contributions in Scala?

I think it makes the most sense to me for us to publish a separate repo with a module and nar build for now and post when it's available in the users group. Thanks for the discussion everyone, hopefully we can start making some helpful contributions soon. -Adam On 2018/02/10 23:43:31, Tony