Hi,
I'm using Apache Kudu 1.4.0
Yesterday, 6 tservers die at the same time. Following message is logged for
each tserver.
F0614 14:58:32.868551 111454 hybrid_clock.cc:227]
Couldn't get the current time: Clock unsynchronized.
Status: Service unavailable:
Error reading clock. Clock considered unsynchronized
We are already using ntpd, and in /var/log/messages, ntpd related message
is logged.
Jun 14 14:58:38 hostname ntpdate[10231]: step time server ip_addr offset
-0.000168 sec
We use our own ntp service. I don't know what's the exact reason, but It's
suspicious that our ntp service is malfunctioned or network is not good
temporarily.
The problem is that this could happen again and again.
So, I'm considering modifying source code of Kudu from LOG(FATAL) to
LOG(WARN) so that tserver does not exit on unsync.
uint64_t now_usec;
uint64_t error_usec;
Status s = WalltimeWithError(&now_usec, &error_usec);
if (PREDICT_FALSE(!s.ok())) {
LOG(FATAL) << Substitute("Couldn't get the current time: Clock
unsynchronized. "
"Status: $0", s.ToString());
}
So, I question is that is it OK modifying LOG(FATAL) to LOG(WARN) of above
code? and wanted to know this can preventing from dying of tserver when
clock unsynced?
Thanks.
Jason,
Regard
