Hi Kevin,
Thank you for your mail and suggestions. I was also able to achieve the
same by using the below expression
${current_ts:substring(0,19):toDate("yyyy-MM-dd'T'HH:mm:ss","UTC"):toNumber():plus(${current_ts:substring(20,23)})}
So I am taking out the millisecond part of the microsecond time and adding
into the unix time generated from yyyy-MM-dd'T'HH:mm:ss. While this worked
for the one of the case i was working with, sadly it failed for another
one. As it turns out I need to have a microsecond precision so e.g. I have
one transaction that is coming at current_ts: 2018-11-11T00:17:27.937000
and then I have another one coming at
current_ts: 2018-11-11T00:17:27.937001 . The Unix timestamp converted from
the above expression is in millisecond (13 digit epoch number) so as i
understand it cannot be more precise then this. If it take out the
microsecond precision the target system which is Hbase in my case considers
it as the same transaction and over writes the record received at the
earlier time. Hbase timestamp too can only accept unix time in millisecond
(13 digit epoch number) precision. So i am lost at this point how to order
these transactions :(. Please do me know if you have any suggestion on this.
Regards,
Faisal
On Tue, Dec 4, 2018 at 1:49 AM Kevin Doran <kdoran.apa...@gmail.com> wrote:
> Hi Faisal,
>
>
>
> It appears whatever is writing these date strings that you have for inputs
> is writing microseconds, not milliseconds,
>
>
>
> So when you are using `.SSSSSS`, that is, in this case `937000`
> microseconds being interpreted by the expression language java parser as
> 937000 milliseconds (15 minutes 37 seconds), hence the “slightly off”
> factor you are experiencing for the minutes/seconds of the result.
>
>
>
> The difference in hour/day can be explained by timezone. Your input
> timestring must be localtime, but you are treating it as UTC. Specify the
> correct timezone when calling toDate() and that will shift the hours.
>
>
>
> So, my suggestion is to run these date strings through and expression that
> first “converts” microseconds to milliseconds via truncation (using
> substring to drop off the trailing “000” is equivalent to dividing by 1000
> and dropping the remainder/fraction), and then using your expression with
> the timezone specified. Given that your example was off by 9 hours (in
> addition to the ~15 minutes), I used UTC+9 time zone and got this
> expression, which worked for me:
>
>
>
> ${current_ts:substring(0,${current_ts:length():minus(3)}):toDate("yyyy-MM-dd'T'HH:mm:ss.SSSSSS",
> "AWDT"):toNumber()}
>
>
>
> Lastly, depending on how you are using the result of toNumber(), keep in
> mind that some systems expect *seconds since epoch* (not *milliseconds*,
> which toNumber() outputs) for a Unix timestamp.
>
>
>
> Cheers,
> Kevin
>
>
>
> *From: *Faisal Durrani <te04.0...@gmail.com>
> *Reply-To: *"users@nifi.apache.org" <users@nifi.apache.org>
> *Date: *Sunday, December 2, 2018 at 21:59
> *To: *"users@nifi.apache.org" <users@nifi.apache.org>
> *Subject: *Expression language - Convert ISO 8601 to unixtimestamp
>
>
>
> Hi Guys,
>
>
>
> I am having trouble converting the ISO 8601 to a unixtimestamp . Here is
> what i have tried
>
>
>
> current_ts: 2018-11-11T00:17:27.937000
>
>
>
> Using updateAttribute, I have configured the below property
>
> ${current_ts:toDate("yyyy-MM-dd'T'HH:mm:ss.SSSSSS"):toNumber()}
>
> This gives the Output value : 1541863984000
>
> if converted back from epochcoverter, its giving
>
>
>
> *Assuming that this timestamp is in milliseconds:*
>
> *GMT*: Saturday, 10 November 2018 15:33:04
>
> Your time zone: Sunday, 11 November 2018 00:33:04 GMT+09:00
> <https://www.epochconverter.com/timezones?q=1541863984>
>
> *Relative*: 22 days ago
>
>
>
> I have also tried using
> ${current_ts:toDate("yyyy-MM-dd'T'HH:mm:ss.SSSSSS", "UTC"):toNumber()} but
> the result is the same and its completely wrong. Kindly help finding out
> correct expression.
>
>
>
> Regards,
>
> Faisal
>
>
>
