[
https://issues.apache.org/jira/browse/IMPALA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749157#comment-16749157
]
Michael Brown commented on IMPALA-8091:
---------------------------------------
This is a lot of diagnostic info...
{noformat}
I0116 18:56:30.146215 30799 master_main.cc:80] Initializing master server...
I0116 18:56:30.146333 30799 system_ntp.cc:122] Waiting up to
--ntp_initial_sync_wait_secs=60 seconds for the clock to synchronize
W0116 18:56:30.147728 30799 system_ntp.cc:132] Could not find ntp-wait; trying
chrony waitsync instead: Not found: Unable to find binary: ntp-wait
E0116 18:57:29.300457 30799 system_ntp.cc:157] Dumping NTP diagnostics
E0116 18:57:29.302417 30799 system_ntp.cc:102] /sbin/ntptime
------------------------------------------
stdout:
ntp_gettime() returns code 5 (ERROR)
time dfea6d99.4d59a2ec Wed, Jan 16 2019 18:57:29.302, (.302149739),
maximum error 16000000 us, estimated error 16000000 us, TAI offset 0
ntp_adjtime() returns code 5 (ERROR)
modes 0x0 (),
offset 0.000 us, frequency 129.100 ppm, interval 1 s,
maximum error 16000000 us, estimated error 16000000 us,
status 0x2041 (PLL,UNSYNC,NANO),
time constant 6, precision 0.001 us, tolerance 500 ppm,
E0116 18:57:29.330603 30799 system_ntp.cc:102] /sbin/ntpq -n -c timeout 1000 -c
readvar -c sysinfo -c lpeers -c opeers -c version
------------------------------------------
stdout:
associd=0 status=c618 leap_alarm, sync_ntp, 1 event, no_sys_peer,
version="ntpd 4.2.6p5@1.2349-o Wed Apr 12 21:24:06 UTC 2017 (1)",
processor="x86_64", system="Linux/3.10.0-693.5.2.el7.x86_64", leap=11,
stratum=2, precision=-24, rootdelay=65.438, rootdisp=43.467,
refid=96.126.122.39,
reftime=dfea6d72.e0d7d2b9 Wed, Jan 16 2019 18:56:50.878,
clock=dfea6d99.4f1265e8 Wed, Jan 16 2019 18:57:29.308, peer=64381, tc=6,
mintc=3, offset=0.000, frequency=129.100, sys_jitter=11.220,
clk_jitter=12.567, clk_wander=0.221
remote refid st t when poll reach delay offset jitter
==============================================================================
+66.228.48.38 127.67.113.92 2 u 39 64 7 70.236 -35.592 28.619
+208.75.89.4 63.145.169.3 2 u 37 64 7 64.246 -20.776 29.110
*96.126.122.39 .GPS. 1 u 38 64 7 63.424 -31.810 28.640
+69.89.207.199 212.215.1.157 2 u 36 64 7 37.729 -37.207 28.643
remote local st t when poll reach delay offset disp
==============================================================================
+66.228.48.38 172.28.204.72 2 u 39 64 7 70.236 -35.592 0.913
+208.75.89.4 172.28.204.72 2 u 37 64 7 64.246 -20.776 0.921
*96.126.122.39 172.28.204.72 1 u 38 64 7 63.424 -31.810 0.906
+69.89.207.199 172.28.204.72 2 u 36 64 7 37.729 -37.207 0.914
ntpq 4.2.6p5@1.2349-o Wed Apr 12 21:24:08 UTC 2017 (1)
stderr:
***Command `sysinfo' unknown
E0116 18:57:29.333056 30799 system_ntp.cc:102] /sbin/ntpdc -n -c timeout 1000
-c peers -c sysinfo -c sysstats -c version
------------------------------------------
stdout:
remote local st poll reach delay offset disp
=======================================================================
*96.126.122.39 172.28.204.72 1 64 7 0.06342 -0.031810 0.03009
=208.75.89.4 172.28.204.72 2 64 7 0.06424 -0.020776 0.03033
=69.89.207.199 172.28.204.72 2 64 7 0.03772 -0.037207 0.03023
=66.228.48.38 172.28.204.72 2 64 7 0.07022 -0.035592 0.03021
system peer: 96.126.122.39
system peer mode: client
leap indicator: 11
stratum: 2
precision: -24
root distance: 0.06543 s
root dispersion: 0.04346 s
reference ID: [96.126.122.39]
reference time: dfea6d72.e0d7d2b9 Wed, Jan 16 2019 18:56:50.878
system flags: auth ntp kernel stats
jitter: 0.011215 s
stability: 0.000 ppm
broadcastdelay: 0.000000 s
authdelay: 0.000000 s
time since restart: 1459
time since reset: 1459
packets received: 180
packets processed: 124
current version: 124
previous version: 0
declined: 0
access denied: 0
bad length or format: 0
bad authentication: 0
rate exceeded: 0
ntpdc 4.2.6p5@1.2349-o Wed Apr 12 21:24:08 UTC 2017 (1)
E0116 18:57:29.336165 30799 system_ntp.cc:102] /usr/bin/chronyc -n tracking
------------------------------------------
stdout:
506 Cannot talk to daemon
E0116 18:57:29.339103 30799 system_ntp.cc:102] /usr/bin/chronyc -n sources
------------------------------------------
stdout:
506 Cannot talk to daemon
F0116 18:57:29.339143 30799 master_main.cc:81] Check failed: _s.ok() Bad
status: Runtime error: Cannot initialize clock: failed to wait for clock sync
using command '/usr/bin/chronyc waitsync 60 0 0 1': /usr/bin/chronyc: process
exited with non-zero status 1
{noformat}
> minicluster kudu failure: Cannot initialize clock: failed to wait for clock
> sync
> --------------------------------------------------------------------------------
>
> Key: IMPALA-8091
> URL: https://issues.apache.org/jira/browse/IMPALA-8091
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 3.2.0
> Reporter: David Knupp
> Assignee: Thomas Tauber-Marshall
> Priority: Critical
> Labels: broken-build
>
> *Console log*:
> {noformat}
> 19:10:04 19:10:04 Error executing impala SQL:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql
> See:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
> 19:10:04 Background task Loading functional-query data (pid 4831) failed.
> 19:10:04 Background task Loading TPC-H data (pid 4832) failed.
> 19:10:04 ERROR in
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh
> at line 85: CM_HOST=${2-}
> [...]
> 19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core
> files: ['./core.1547693849.30799.kudu-master',
> './core.1547693849.30783.kudu-tserver',
> './core.1547693849.30767.kudu-tserver',
> './core.1547693849.30808.kudu-tserver']
> 19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
> 19:10:05 [New LWP 30834]
> 19:10:05 [New LWP 30835]
> 19:10:05 [New LWP 30838]
> 19:10:05 [New LWP 30837]
> 19:10:05 [New LWP 30836]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x00007f9a2cc611f7 in ?? ()
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary
> path through GDB:
> /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c
> 19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to
> determine binary because multiple candidate binaries were found and none of
> their paths contained 'latest' to disambiguate:
> 19:10:05 Core:./core.1547693849.30799.kudu-master
> 19:10:05
> Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master',
> './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master']
> 19:10:05
> 19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783]
> 19:10:05 [New LWP 30810]
> 19:10:05 [New LWP 30812]
> 19:10:05 [New LWP 30811]
> 19:10:05 [New LWP 30824]
> 19:10:05 [New LWP 30820]
> 19:10:05 Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> 19:10:05 Program terminated with signal SIGABRT, Aborted.
> 19:10:05 #0 0x00007f0b81fb11f7 in ?? ()
> {noformat}
> *Backtraces*:
> {noformat}
> CORE: ./core.1547693849.30799.kudu-master
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x00007f9a2cc611f7 in ?? ()
> #0 0x00007f9a2cc611f7 in ?? ()
> #1 0x00007f9a2cc628e8 in ?? ()
> #2 0x0000000000000020 in ?? ()
> #3 0x0000000000000000 in ?? ()
> CORE: ./core.1547693849.30783.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x00007f0b81fb11f7 in ?? ()
> #0 0x00007f0b81fb11f7 in ?? ()
> #1 0x00007f0b81fb28e8 in ?? ()
> #2 0x0000000000000020 in ?? ()
> #3 0x0000000000000000 in ?? ()
> CORE: ./core.1547693849.30767.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x00007f60b1e2f1f7 in ?? ()
> #0 0x00007f60b1e2f1f7 in ?? ()
> #1 0x00007f60b1e308e8 in ?? ()
> #2 0x0000000000000020 in ?? ()
> #3 0x0000000000000000 in ?? ()
> CORE: ./core.1547693849.30808.kudu-tserver
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x00007fa3cb5591f7 in ?? ()
> #0 0x00007fa3cb5591f7 in ?? ()
> #1 0x00007fa3cb55a8e8 in ?? ()
> #2 0x0000000000000020 in ?? ()
> #3 0x0000000000000000 in ?? ()
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
