Re: Event Timestamp corrupted by timezone

[Jingsong Li]

Hi Wojciech,

You can try 1.9/1.10 with blink planner.
As Timo said, the timestamp is TimestampType without time zone in new type
system and Blink planner support it.

And you should use LocalDateTime in sink/down stream, LocalDateTime has no
time zone.

Best,
Jingsong Lee

On Tue, Dec 10, 2019 at 11:09 PM Timo Walther  wrote:

> Hi,
>
> I hope we can solve this issues with the new type system. The core
> problem is the old planner uses java.sql.Timestamp which depends on the
> timezone of the current machine. I would recommend to set everything to
> UTC if possible for now.
>
> Regards,
> Timo
>
>
> On 03.12.19 18:49, Lasse Nedergaard wrote:
> > Hi.
> >
> > We have the same Challenges. I asked on Flink forward and it’s a known
> > problem. We input in utc but Flink output in local machine time. We have
> > created a function that converts it back to utc before collecting to
> > down stream.
> >
> > Med venlig hilsen / Best regards
> > Lasse Nedergaard
> >
> >
> > Den 3. dec. 2019 kl. 15.16 skrev Wojciech Indyk  > >:
> >
> >> Hi!
> >> I use Flink 1.8 with Scala. I think I've found a problem with event
> >> timestamps in TableAPI. When I mark my timestamp: Long as .rowtime and
> >> then save it back to stream as sql.Timestamp I will get wrong .getTime
> >> result. The gist for reproduction is here:
> >> https://gist.github.com/woj-i/b1dfbb71590b7f1c0c58be1f9e41c610
> >> When I change my timezome from GMT+1 to GMT everything works ok.
> >> I've found this post from March
> >>
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/event-time-timezone-is-not-correct-tt26457.html
> >> but it's not resolved. The most relevant ticket I've found
> >> https://issues.apache.org/jira/browse/FLINK-8353 seems to not include
> >> the problem I described.
> >>
> >> 1. Can you confirm it's a bug?
> >> 2. Should I post this bug somewhere to be at least planned to solve?
> >> 3. Can you recommend me a workaround for the described problem?
> >>
> >> --
> >> Kind regards/ Pozdrawiam,
> >> Wojciech Indyk
>
>

-- 
Best, Jingsong Lee

Re: Event Timestamp corrupted by timezone

[Timo Walther]

Hi,

I hope we can solve this issues with the new type system. The core 
problem is the old planner uses java.sql.Timestamp which depends on the 
timezone of the current machine. I would recommend to set everything to 
UTC if possible for now.


Regards,
Timo


On 03.12.19 18:49, Lasse Nedergaard wrote:

Hi.

We have the same Challenges. I asked on Flink forward and it’s a known 
problem. We input in utc but Flink output in local machine time. We have 
created a function that converts it back to utc before collecting to 
down stream.


Med venlig hilsen / Best regards
Lasse Nedergaard


Den 3. dec. 2019 kl. 15.16 skrev Wojciech Indyk >:



Hi!
I use Flink 1.8 with Scala. I think I've found a problem with event 
timestamps in TableAPI. When I mark my timestamp: Long as .rowtime and 
then save it back to stream as sql.Timestamp I will get wrong .getTime 
result. The gist for reproduction is here: 
https://gist.github.com/woj-i/b1dfbb71590b7f1c0c58be1f9e41c610

When I change my timezome from GMT+1 to GMT everything works ok.
I've found this post from March 
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/event-time-timezone-is-not-correct-tt26457.html 
but it's not resolved. The most relevant ticket I've found 
https://issues.apache.org/jira/browse/FLINK-8353 seems to not include 
the problem I described.


1. Can you confirm it's a bug?
2. Should I post this bug somewhere to be at least planned to solve?
3. Can you recommend me a workaround for the described problem?

--
Kind regards/ Pozdrawiam,
Wojciech Indyk

Re: Event Timestamp corrupted by timezone

[Lasse Nedergaard]

Hi. 

We have the same Challenges. I asked on Flink forward and it’s a known problem. 
We input in utc but Flink output in local machine time. We have created a 
function that converts it back to utc before collecting to down stream. 

Med venlig hilsen / Best regards
Lasse Nedergaard


> Den 3. dec. 2019 kl. 15.16 skrev Wojciech Indyk :
> 
> Hi!
> I use Flink 1.8 with Scala. I think I've found a problem with event 
> timestamps in TableAPI. When I mark my timestamp: Long as .rowtime and then 
> save it back to stream as sql.Timestamp I will get wrong .getTime result. The 
> gist for reproduction is here: 
> https://gist.github.com/woj-i/b1dfbb71590b7f1c0c58be1f9e41c610
> When I change my timezome from GMT+1 to GMT everything works ok.
> I've found this post from March 
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/event-time-timezone-is-not-correct-tt26457.html
>  but it's not resolved. The most relevant ticket I've found 
> https://issues.apache.org/jira/browse/FLINK-8353 seems to not include the 
> problem I described.
> 
> 1. Can you confirm it's a bug?
> 2. Should I post this bug somewhere to be at least planned to solve?
> 3. Can you recommend me a workaround for the described problem? 
> 
> --
> Kind regards/ Pozdrawiam,
> Wojciech Indyk

Event Timestamp corrupted by timezone

[Wojciech Indyk]

Hi!
I use Flink 1.8 with Scala. I think I've found a problem with event
timestamps in TableAPI. When I mark my timestamp: Long as .rowtime and then
save it back to stream as sql.Timestamp I will get wrong .getTime result.
The gist for reproduction is here:
https://gist.github.com/woj-i/b1dfbb71590b7f1c0c58be1f9e41c610
When I change my timezome from GMT+1 to GMT everything works ok.
I've found this post from March
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/event-time-timezone-is-not-correct-tt26457.html
but it's not resolved. The most relevant ticket I've found
https://issues.apache.org/jira/browse/FLINK-8353 seems to not include the
problem I described.

1. Can you confirm it's a bug?
2. Should I post this bug somewhere to be at least planned to solve?
3. Can you recommend me a workaround for the described problem?

--
Kind regards/ Pozdrawiam,
Wojciech Indyk