Hi,
As I mentioned, I want to add a link from the console log back to the jenkins
job, which may help find the matching jenkins page. It just got added today, so
now the console log will start like:
Started by user anonymous<https://jenkins02.trafodion.org/user/null>
[EnvInject] - Loading node environment variables.
Building remotely on
slave-cdh51-01<https://jenkins02.trafodion.org/computer/slave-cdh51-01> (redhat
redhat_cdh5.1) in workspace /home/jenkins/workspace/core-regress-seabase-cdh5.1
[core-regress-seabase-cdh5.1] $ /bin/sh /tmp/hudson2494869012577528422.sh
Detailed logs:
http://logs.trafodion.org/10/810/2/gate/core-regress-seabase-cdh5.1/4ee874b/
Jenkins build:
https://jenkins02.trafodion.org/job/core-regress-seabase-cdh5.1/622/
Also, Chris put together this page, which is very helpful:
https://wiki.trafodion.org/wiki/index.php/History_of_Daily_Builds
The equivalent part of the hive set-up for the test nodes is in the
trafodion/infra repo:
https://github.com/trafodion/infra/blob/master/modules/traf/files/tpcds/init_hive_tpcds.sql
Anyone is free to submit a change, or let me know if you want me to take care
of it. Hmm, I may need to look at how to make sure it gets to nodes that
already have the TPCDS tables set up. I'll take a look at that tomorrow.
-Steve
From: Zeller, Hans
Sent: Monday, December 15, 2014 19:04
To: Varnau, Steve (Trafodion); Johnson, Stacey;
[email protected]
Cc: Sharma, Anoop
Subject: RE: Daily build 2014-12-14 08:30:00 UTC of trafodion/core -- Test
Failures
Hi Steve, thank you, the Jenkins page looks nice. What is the best way to get
to that page?
Also, I just talked to Anoop about the Hive failures due to the missing
STORE_ORC file. He said that he changed the install_local_hadoop file to create
this table as part of the initial setup. Do you have your own script to create
these Hive tables? I think he just added this line to the Hive commands that
are executed by install_local_hadoop:
create table store_orc stored as orc as select * from store;
Should I move the TPC-DS Hive setup commands into a separate script so you can
call that script from the Jenkins/Puppet environment?
Thanks,
Hans
From: Varnau, Steve (Trafodion)
Sent: Monday, December 15, 2014 4:24 PM
To: Zeller, Hans; Johnson, Stacey;
[email protected]<mailto:[email protected]>
Subject: RE: Daily build 2014-12-14 08:30:00 UTC of trafodion/core -- Test
Failures
For the phoenix tests (junit tests), it is easier to look at them on jenkins
web. For example:
https://jenkins02.trafodion.org/job/phoenix_part1_T2-cdh5.1/41/testReport/
You can drill down to specific tests, java stack traces, look at recent history
of tests, etc. Unfortunately, I guess it is not obvious how to find that page
from looking at the logs. I think I can add a link at the top of the console
log that would link back there.
For the various errors in the console logs, I don't have any immediate ideas on
how to make those easier to find. It doesn't take me long to find the errors
there, but I'm very used to looking at them. General location is the end of the
log and then back several pages to account for the instance shutdown and
log-uploading that happens at the end of the job.
-Steve
From: Zeller, Hans
Sent: Monday, December 15, 2014 15:59
To: Varnau, Steve (Trafodion); Johnson, Stacey;
[email protected]<mailto:[email protected]>
Subject: RE: Daily build 2014-12-14 08:30:00 UTC of trafodion/core -- Test
Failures
Hi Steve, thanks for this analysis. Would you expect the firefighters to go
through the detailed log to find out the reason of the failure? A related
question: For Phoenix tests, I see about 20 output files of a single line each
in the "surefire-reports" directory. To find out what happened, the only way I
know is to open each of these individually.
Hans
From: Varnau, Steve (Trafodion)
Sent: Monday, December 15, 2014 2:13 PM
To: Zeller, Hans; Johnson, Stacey;
[email protected]<mailto:[email protected]>
Subject: RE: Daily build 2014-12-14 08:30:00 UTC of trafodion/core -- Test
Failures
Hi Hans,
In all the jobs where you did not find test log files, it was either because it
never got to the testing phase or all the tests passed, but a core file was
found. These are the errors buried in the console logs.
core-regress-core-cm5.1
Error: requested mxo server processes did not come up
core-regress-catman1-cm5.1
WARNING: Core files found in
/home/jenkins/workspace/core-regress-catman1-cm5.1 :
-rw------- 1 trafodion trafodion 536719360 Dec 14 11:08
rundir/catman1/core.slave-cm51-03.3889.tdm_arkcmp
phoenix_part1_T4-cm5.1
[ERROR] Failed to execute goal on project phoenix_test: Could not resolve
dependencies for project com.hp.phoenix:phoenix_test:jar:1.0: Could not find
artifact org.trafodion.jdbc.t4.T4Driver:t4driver:jar:1 at specified path
/home/jenkins/workspace/phoenix_part1_T4-cm5.1/traf_run/export/lib/jdbcT4.jar
-> [Help 1]
phoenix_part2_T4-cm5.1
[ERROR] Failed to execute goal on project phoenix_test: Could not resolve
dependencies for project com.hp.phoenix:phoenix_test:jar:1.0: Could not find
artifact org.trafodion.jdbc.t4.T4Driver:t4driver:jar:1 at specified path
/home/jenkins/workspace/phoenix_part2_T4-cm5.1/traf_run/export/lib/jdbcT4.jar
-> [Help 1]
phoenix_part1_T2-cm5.1
# A fatal error has been detected by the Java Runtime
Environment:
#
# SIGSEGV (0xb) at pc=0x00007ffff739110c, pid=16856, tid=140737313330944
#
# JRE version: OpenJDK Runtime Environment (7.0_71-b14) (build
1.7.0_71-mockbuild_2014_10_17_22_23-b00)
# Java VM: OpenJDK 64-Bit Server VM (24.65-b04 mixed mode linux-amd64
compressed oops)
# Derivative: IcedTea 2.5.3
# Distribution: Built on CentOS release 6.5 (Final) (Fri Oct 17 22:23:16 UTC
2014)
# Problematic frame:
# V [libjvm.so+0x84610c]
jdbc_test-cm5.1
Error: requested mxo server processes did not come up
-Steve
From: Trafodion-firefighters
[mailto:trafodion-firefighters-bounces+steve.varnau=hp....@lists.launchpad.net]
On Behalf Of Zeller, Hans
Sent: Monday, December 15, 2014 11:27
To: Johnson, Stacey;
[email protected]<mailto:[email protected]>
Subject: Re: [Trafodion-firefighters] Daily build 2014-12-14 08:30:00 UTC of
trafodion/core -- Test Failures
Hi,
Here are a few comments on these failures, see below.
Hans
From: Trafodion-firefighters
[mailto:trafodion-firefighters-bounces+hans.zeller=hp....@lists.launchpad.net]
On Behalf Of Johnson, Stacey
Sent: Monday, December 15, 2014 10:31 AM
To:
[email protected]<mailto:[email protected]>
Subject: [Trafodion-firefighters] Daily build 2014-12-14 08:30:00 UTC of
trafodion/core -- Test Failures
Build failed.
- traf-pub-release http://logs.trafodion.org/daily/traf-pub-release/d559f4c :
SUCCESS in 37m 35s
- traf-pub-debug http://logs.trafodion.org/daily/traf-pub-debug/189851e :
SUCCESS in 29m 08s
- core-regress-core-cdh5.1
http://logs.trafodion.org/daily/core-regress-core-cdh5.1/7576814 : SUCCESS in
1h 54m 30s
- core-regress-core-cm5.1
http://logs.trafodion.org/daily/core-regress-core-cm5.1/f67a4d9 : FAILURE in 1h
01m 49s
[HZ] Log and diff files are missing
- core-regress-core-hdp2.1
http://logs.trafodion.org/daily/core-regress-core-hdp2.1/85df4a9 : SUCCESS in
1h 48m 34s
- core-regress-charsets-cdh5.1
http://logs.trafodion.org/daily/core-regress-charsets-cdh5.1/cc335b2 : SUCCESS
in 1h 11m 46s
- core-regress-charsets-cm5.1
http://logs.trafodion.org/daily/core-regress-charsets-cm5.1/a43f540 : SUCCESS
in 1h 38m 22s
- core-regress-charsets-hdp2.1
http://logs.trafodion.org/daily/core-regress-charsets-hdp2.1/4c63691 : SUCCESS
in 1h 09m 21s
- core-regress-qat-cdh5.1
http://logs.trafodion.org/daily/core-regress-qat-cdh5.1/b324e2e : SUCCESS in 1h
15m 22s
- core-regress-qat-cm5.1
http://logs.trafodion.org/daily/core-regress-qat-cm5.1/30566dc : SUCCESS in 1h
38m 24s
- core-regress-qat-hdp2.1
http://logs.trafodion.org/daily/core-regress-qat-hdp2.1/2e3e945 : SUCCESS in 1h
09m 57s
- core-regress-udr-cdh5.1
http://logs.trafodion.org/daily/core-regress-udr-cdh5.1/4eb08ed : FAILURE in 1h
02m 56s
[HZ] Seeing a syntax error in a UDF invocation in TEST108 that uses the
REPLICATE PARTITION syntax. I can fix that one.
- core-regress-udr-cm5.1
http://logs.trafodion.org/daily/core-regress-udr-cm5.1/10c0988 : FAILURE in 1h
28m 06s
- core-regress-udr-hdp2.1
http://logs.trafodion.org/daily/core-regress-udr-hdp2.1/58e3670 : FAILURE in 1h
00m 49s
- core-regress-catman1-cdh5.1
http://logs.trafodion.org/daily/core-regress-catman1-cdh5.1/3a4f277 : SUCCESS
in 2h 02m 01s
- core-regress-catman1-cm5.1
http://logs.trafodion.org/daily/core-regress-catman1-cm5.1/83aa516 : FAILURE in
2h 32m 33s
[HZ] Log and diff files are missing
- core-regress-catman1-hdp2.1
http://logs.trafodion.org/daily/core-regress-catman1-hdp2.1/348ab50 : SUCCESS
in 1h 56m 21s
- core-regress-compGeneral-cdh5.1
http://logs.trafodion.org/daily/core-regress-compGeneral-cdh5.1/b634ae8 :
SUCCESS in 1h 36m 05s
- core-regress-compGeneral-cm5.1
http://logs.trafodion.org/daily/core-regress-compGeneral-cm5.1/cc1aae9 :
SUCCESS in 2h 08m 58s
- core-regress-compGeneral-hdp2.1
http://logs.trafodion.org/daily/core-regress-compGeneral-hdp2.1/fe148be :
SUCCESS in 1h 29m 42s
- core-regress-executor-cdh5.1
http://logs.trafodion.org/daily/core-regress-executor-cdh5.1/4ddd6c2 : SUCCESS
in 1h 50m 37s
- core-regress-executor-cm5.1
http://logs.trafodion.org/daily/core-regress-executor-cm5.1/671d366 : FAILURE
in 2h 05m 40s
- core-regress-executor-hdp2.1
http://logs.trafodion.org/daily/core-regress-executor-hdp2.1/e42fc3c : SUCCESS
in 1h 47m 45s
- core-regress-fullstack2-cdh5.1
http://logs.trafodion.org/daily/core-regress-fullstack2-cdh5.1/3a36c62 :
SUCCESS in 43m 01s
- core-regress-fullstack2-cm5.1
http://logs.trafodion.org/daily/core-regress-fullstack2-cm5.1/9255eca : SUCCESS
in 1h 04m 49s
- core-regress-fullstack2-hdp2.1
http://logs.trafodion.org/daily/core-regress-fullstack2-hdp2.1/5cd1322 :
SUCCESS in 41m 38s
- core-regress-hive-cdh5.1
http://logs.trafodion.org/daily/core-regress-hive-cdh5.1/d78b894 : FAILURE in
1h 23m 48s
[HZ] There seem to be two failures: TEST003 has become non-deterministic, it
sometimes gives an incorrect answer right after inserting data into a table.
TEST020 has a bigger difference, the STORE_ORC file is missing. That could be
something in the setup. The test passes on my workstation.
- core-regress-hive-cm5.1
http://logs.trafodion.org/daily/core-regress-hive-cm5.1/56480a8 : FAILURE in 1h
48m 14s
- core-regress-hive-hdp2.1
http://logs.trafodion.org/daily/core-regress-hive-hdp2.1/abc3dbf : FAILURE in
1h 25m 34s
- core-regress-seabase-cdh5.1
http://logs.trafodion.org/daily/core-regress-seabase-cdh5.1/e8fe62a : SUCCESS
in 1h 43m 02s
- core-regress-seabase-cm5.1
http://logs.trafodion.org/daily/core-regress-seabase-cm5.1/515ec07 : FAILURE in
3h 20m 57s
- core-regress-seabase-hdp2.1
http://logs.trafodion.org/daily/core-regress-seabase-hdp2.1/23c7318 : SUCCESS
in 1h 40m 56s
- phoenix_part1_T4-cdh5.1
http://logs.trafodion.org/daily/phoenix_part1_T4-cdh5.1/30e7036 : SUCCESS in 1h
46m 17s
- phoenix_part2_T4-cdh5.1
http://logs.trafodion.org/daily/phoenix_part2_T4-cdh5.1/85a12ce : SUCCESS in 1h
41m 42s
- phoenix_part1_T4-cm5.1
http://logs.trafodion.org/daily/phoenix_part1_T4-cm5.1/03d001f : FAILURE in 51m
14s
[HZ] Log and diff files are missing.
- phoenix_part2_T4-cm5.1
http://logs.trafodion.org/daily/phoenix_part2_T4-cm5.1/1afcbbb : FAILURE in 51m
36s
[HZ] Can't find log files
- phoenix_part1_T4-hdp2.1
http://logs.trafodion.org/daily/phoenix_part1_T4-hdp2.1/778cd13 : SUCCESS in 1h
41m 22s
- phoenix_part2_T4-hdp2.1
http://logs.trafodion.org/daily/phoenix_part2_T4-hdp2.1/7783de4 : SUCCESS in 1h
42m 17s
- phoenix_part1_T2-cdh5.1
http://logs.trafodion.org/daily/phoenix_part1_T2-cdh5.1/fa49382 : SUCCESS in 2h
32m 05s (non-voting)
- phoenix_part2_T2-cdh5.1
http://logs.trafodion.org/daily/phoenix_part2_T2-cdh5.1/aa0ed0e : SUCCESS in 2h
21m 30s (non-voting)
- phoenix_part1_T2-cm5.1
http://logs.trafodion.org/daily/phoenix_part1_T2-cm5.1/c49a07a : FAILURE in 45m
09s (non-voting)
[HZ] Can't find log files
- phoenix_part2_T2-cm5.1
http://logs.trafodion.org/daily/phoenix_part2_T2-cm5.1/0dbadaa : FAILURE in 52m
06s (non-voting)
- phoenix_part1_T2-hdp2.1
http://logs.trafodion.org/daily/phoenix_part1_T2-hdp2.1/d4b1e83 : FAILURE in 2h
24m 23s (non-voting)
- phoenix_part2_T2-hdp2.1
http://logs.trafodion.org/daily/phoenix_part2_T2-hdp2.1/ee8c674 : SUCCESS in 2h
16m 57s (non-voting)
- pyodbc_test-cdh5.1 http://logs.trafodion.org/daily/pyodbc_test-cdh5.1/cd5f3af
: SUCCESS in 40m 34s
- pyodbc_test-cm5.1 http://logs.trafodion.org/daily/pyodbc_test-cm5.1/87defcc :
SUCCESS in 52m 56s
- pyodbc_test-hdp2.1 http://logs.trafodion.org/daily/pyodbc_test-hdp2.1/bb59075
: SUCCESS in 37m 15s
- jdbc_test-cdh5.1 http://logs.trafodion.org/daily/jdbc_test-cdh5.1/4a2efd0 :
FAILURE in 43m 06s
[HZ] Failure in TestBasic, number of columns expected is 0, got 4
- jdbc_test-cm5.1 http://logs.trafodion.org/daily/jdbc_test-cm5.1/f1f42c4 :
FAILURE in 1h 12m 09s
- jdbc_test-hdp2.1 http://logs.trafodion.org/daily/jdbc_test-hdp2.1/3048734 :
FAILURE in 40m 29s
--
Mailing list: https://launchpad.net/~trafodion-firefighters
Post to : [email protected]
Unsubscribe : https://launchpad.net/~trafodion-firefighters
More help : https://help.launchpad.net/ListHelp
