Re: RFC3339 parsing with Expression Language

[Marton Szasz]

We've already talked about this with Martin and other contributors,
but leaving my 2c here for the wider discussion.

I'm leaning towards the second option, having a separate EL function,
because parsing the RFC-compliant dates does not directly map to
strptime-like format strings used in toDate. In python, they also have
separate functions for parsing like strptime [1] (like the EL toDate)
and parsing RFC 3339 or ISO 8601 dates [2]. Most other MiNiFi C++ devs
are leaning towards the first option, extending toDate with an
overload that doesn't take a format string.

Ultimately I think we should follow whatever interface the NiFi devs
prefer, so if the NiFi Expression Language is ever extended with a
similar RFC/ISO date parser facility in the future, the MiNiFi C++
Expression Language would match that. This way users have an easier
time converting their flows between the two implementations.

Thanks,
Marton


[1] https://docs.python.org/3/library/datetime.html#datetime.datetime.strptime
[2] 
https://docs.python.org/3/library/datetime.html#datetime.datetime.fromisoformat
and https://stackoverflow.com/a/15228038

On Thu, 2 Mar 2023 at 13:09, Martin Zink  wrote:
>
> When dealing with timestamps coming from third parties, sometimes we cannot
> rely on exact timestamp formats, so it is difficult to parse them using
> toDate
> .
> (e.g. Syslog messages allow a subset of RFC3339 for timestamps).
> To circumvent this I started working on an expression language function
> that could parse all RFC3339 compliant timestamps with expression
> language in MiNiFi C++, but I feel this functionality would also be a great
> addition to NiFi.
>
> It would be great if the expression language would remain consistent
> between MiNiFi C++ and NiFi, and I'm not quite sure where this
> functionality should be accessible from.
>
>- We could either change the current toDate
>
> 
> to
>accept zero arguments and calling it without argument would try to parse
>the string as RFC3339 compliant.
>- or we could introduce a new expression language function e.g.
>parseRfc3339
>
>
> What's the community's stance on this?
>
> Thanks,
> Martin

RFC3339 parsing with Expression Language

[Martin Zink]

When dealing with timestamps coming from third parties, sometimes we cannot
rely on exact timestamp formats, so it is difficult to parse them using
toDate
.
(e.g. Syslog messages allow a subset of RFC3339 for timestamps).
To circumvent this I started working on an expression language function
that could parse all RFC3339 compliant timestamps with expression
language in MiNiFi C++, but I feel this functionality would also be a great
addition to NiFi.

It would be great if the expression language would remain consistent
between MiNiFi C++ and NiFi, and I'm not quite sure where this
functionality should be accessible from.

   - We could either change the current toDate
   

to
   accept zero arguments and calling it without argument would try to parse
   the string as RFC3339 compliant.
   - or we could introduce a new expression language function e.g.
   parseRfc3339


What's the community's stance on this?

Thanks,
Martin