[
https://issues.apache.org/jira/browse/NIFI-12426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17790794#comment-17790794
]
David Handermann commented on NIFI-12426:
-----------------------------------------
Thanks for providing the JoltTransform as a potential option, that is a helpful
point of reference for existing flows.
Although DateTimeFormatter is largely compatible with SimpleDateFormat
expressions, there are a few subtle differences, which was one of the reasons
for not moving to it directly on the 1.x branch. It might be possible to
consider a way to opt-in to DateTimeFormatter, but it would take some
evaluation to consider how complicated it would be to implement a conditional
approach.
There is still additional work to do on the main branch to move away from
SimpleDateFormat in places, so the best strategy seems to be completing more of
those changes and evaluating potential solutions for the 1.x branch at that
time. Definitely open to other ideas on the best solution, as it is difficult
to work around in cases where the input has microsecond or nanosecond precision
as you highlighted.
> Support microseconds in RegexDateTimeMatcher
> --------------------------------------------
>
> Key: NIFI-12426
> URL: https://issues.apache.org/jira/browse/NIFI-12426
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Matt Burgess
> Assignee: Matt Burgess
> Priority: Major
> Fix For: 1.25.0, 2.0.0
>
>
> If a timestamp in the input has microseconds and a RecordReader is using
> Infer Schema, the data type will be inferred as a string rather than a
> timestamp regardless of the Timestamp Format property in the reader. Although
> SimpleDateFormat doesn't support microseconds, it is forgiving in the parsing
> of a timestamp string and accepts ".SSSSSS" as a milliseconds format even
> though the microseconds will not be honored.
> However when inferring the schema, the input must also pass the
> RegexDateTimeMatcher which checks that it "looks like" a timestamp and within
> the legitimate length boundaries. This matcher enforces a 3-digit length of
> milliseconds and will fail to match input with microseconds. This matcher
> should accept 6 digits of fractional seconds and allow the other matchers to
> proceed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
