[
https://issues.apache.org/jira/browse/NIFI-8423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17326478#comment-17326478
]
Josef Zahner commented on NIFI-8423:
------------------------------------
Hi Joe, after the screenshot above (where p-li-nifi-05 & p-li-nifi-10 showed
the wrong time I've just restarted p-li-nifi-05. Then every single node showed
the same timestamp. Very unpredictable.
Today I've restarted all NiFi nodes (not the OS, just the application) multiple
times and the result is still shocking, even though I've configured the
timezone manually in bootstrap.conf. What I get in the UI on top right:
* UTC keyword, but time is in CEST in the UI
* CEST keyword, but time is in UTC in the UI
* CEST keyword and time is really CEST
Additionally the logmessages from the nodes directly on the PGs/Processors can
show another time than the UI on the top right.
What I found is that if I'm stopping all my ListSFTP (and any other
processor/input which could load/generate flowfiles) and the cluster doesn't
use any thread, most of the time the cluster UI on the top right is showing the
correct timezone and time. So if there is no load on the cluster, it's very
likely the the UI time & timezone is correct. If everything is up and running
it's nearly impossible after a restart of all NiFi at the same time, to get a
correct timezone.
To sum up, there is clearly huge bug which leads to this behavior and in our
case it seems to be load dependent. I have screenshots for all the cases above.
And it's not possible in my eyes that just one node or the OS is causing the
issue as it looks random to me where the issue is.
Question, I'm restarting always everything at the same time. What is best
practices to restart a cluster? Node by node or with a big bang?
> Timezone wrong in UI for an 8 node cluster
> ------------------------------------------
>
> Key: NIFI-8423
> URL: https://issues.apache.org/jira/browse/NIFI-8423
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
> Affects Versions: 1.13.2
> Environment: 8 Node NiFi Cluster on CentOS 7
> OpenJDK 1.8.0_282
> Local timezone: Europe/Zurich (CEST or UTC+2h)
> Reporter: Josef Zahner
> Priority: Critical
> Labels: centos, cluster, openjdk, timezone
> Attachments: Screenshot 2021-04-20 at 14.24.17.png, Screenshot
> 2021-04-20 at 14.34.36.png, Screenshot 2021-04-20 at 15.00.22.png,
> image-2021-04-13-15-14-02-162.png, image-2021-04-13-15-14-06-930.png,
> image-2021-04-13-15-14-56-690.png, manual_configured_timezone_gui_output.png,
> nifi-app_log.png
>
>
> We just upgraded to NiFi 1.13.2 and Java 1.8.0_282
> On our 8 node NiFi 1.13.2 cluster with timezone Europe/Zurich (CEST/CET), we
> have the issue that the UI does display the correct timezone (CEST, so UTC
> +2h), but in fact the time is displayed as UTC. NTP is enabled and working.
> The OS configuration/location is everywhere the same (doesn't matter if
> single or cluster NiFi). My tests below are all done at around 15:xx:xx local
> time (CEST).
> As you can see below, the timezone seems to be correct, but the time itself
> within NiFi is 2h behind (so in fact UTC) compared to Windows. In earlier
> NiFi/java versions it was enough to multiple times restart the cluster, but
> on the newest versions this doesn't help anymore. It shows most of the time
> CEST with the wrong time or directly UTC.
> !image-2021-04-13-15-14-06-930.png!
>
> The single NiFi instances or the 2 node clusters are always fine. The issue
> exists only on our 8 node cluster.
> NiFi Single Node Screenshot, which is fine (CEST, so UTC + 2h):
> !image-2021-04-13-15-14-02-162.png!
>
> If we set the -Duser.timezone to "Europe/Zurich" in bootstrap.conf, the UI
> shows no summer time, so only GMT+1 instead of GMT+2. As well not what we
> want.
> {code:java}
> java.arg.20=-Duser.timezone="Europe/Zurich"{code}
> !image-2021-04-13-15-14-56-690.png!
>
> What Matt below suggested has been verified, all servers (single nodes as
> well as clusters) are reporting the same time/timezone.
> [https://community.cloudera.com/t5/Support-Questions/NiFi-clock-is-off-by-one-hour-daylight-savings-problem/td-p/192942]
>
> So the question remains, where on a NiFi cluster comes the time from the UI
> and what could cause it that it is wrong? Sometimes I get UTC, sometimes I'm
> getting CEST but the time is anyhow UTC instead of CEST... I really need to
> have the correct time in the UI as I don't know what the impact could be on
> our dataflows.
>
> Any help would be really appreciated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
