------------------------------------------------------------
revno: 756
committer: Matthias Klose <[email protected]>
branch nick: openjdk8
timestamp: Tue 2019-10-01 09:38:06 +0200
message:
[ Matthias Klose ]
* debian/tests/control: Depend on g++ instead of build-essential or libc6-dev.
* Bump standards vesion.
[ Tiago Stürmer Daitx ]
* Improve and fix build tests and autopkgtests:
- Update debian/tests/hotspot,jdk,langtools to ignore
jtreg-autopkgtest.sh return code.
- Create debian/tests/jtdiff-autopkgtest.in as it depends
on debian/rules variables.
- debian/control.in, debian/control: add default-jre-headless
to Build-Depends with a nocheck clause as jtreg requires
a JRE in /usr/lib/jvm/default-java.
- debian/tests/control:
+ Add zip and unzip test dependencies required by jdk's
test/sun/security/tools/jarsigner/diffend.sh and
test/sun/security/tools/jarsigner/emptymanifest.sh.
+ Depend on default-jre-headless so jtreg will use the
JRE from /usr/lib/jvm/default-java.
- debian/tests/jtdiff-autopkgtest.sh:
+ Fail only if an actual regression is detected.
+ Add the super-diff comparison from jtdiff.
+ Save failed jtr files for all runs.
- debian/tests/jtreg-autopkgtest.sh:
+ Enable retry of failed tests to trim out flaky tests.
+ Fix unbound variable.
+ Keep .jtr files from failed tests only.
- debian/patches/jdk-problem-list.diff: ignore failing tests
that require more investigation.
- debian/rules:
+ Preserve all JTreport directories in the test output
directory.
+ Use JDK_DIR instead of JDK_TO_TEST for autopkgtest
generation.
+ Package all .jtr files from JTwork as jtreg-autopkgtest.sh
makes sure it contains only failed tests.
* debian/tests/jdk: add our custom debian/tests/jdk-problem-list.txt to the
exclusion list.
* debian/tests/jdk-problem-list.txt: custom exclusion rules for jdk tests
that fail to run during a build or autopkgtest run.
* debian/rules: remove debian/patches/jdk-problem-list.diff.
* debian/patches/jdk-problem-list.diff: jtreg allows for extra exclusion
files thus there's no need to patch upstream's exclusion list.
* debian/tests/control: mark all autopkgtests as flaky.
* debian/tests/hotspot-archs: generated by debian/rules, contains a list of
archs that supports a hotspot vm.
* debian/tests/jdk: run only when the host arch is a hotspot vm - allow
override through an environment variable.
* debian/rules: update gen-autopkgtests to echo supported hotspot archs.
added:
debian/tests/hotspot-archs
debian/tests/jdk-problem-list.txt
debian/tests/jtdiff-autopkgtest.in
modified:
debian/changelog
debian/control
debian/control.in
debian/rules
debian/tests/control
debian/tests/hotspot
debian/tests/jdk
debian/tests/jtdiff-autopkgtest.sh
debian/tests/jtreg-autopkgtest.in
debian/tests/jtreg-autopkgtest.sh
debian/tests/langtools
--
lp:~openjdk/openjdk/openjdk8
https://code.launchpad.net/~openjdk/openjdk/openjdk8
Your team Debian Java Maintainers is subscribed to branch
lp:~openjdk/openjdk/openjdk8.
To unsubscribe from this branch go to
https://code.launchpad.net/~openjdk/openjdk/openjdk8/+edit-subscription
=== modified file 'debian/changelog'
--- debian/changelog 2019-10-01 06:30:46 +0000
+++ debian/changelog 2019-10-01 07:38:06 +0000
@@ -1,11 +1,61 @@
-openjdk-8 (8u232-b07-1) UNRELEASED; urgency=medium
+openjdk-8 (8u232-b07-1) unstable; urgency=medium
* Update to 8u232-b07 (early access build).
+
+ [ Matthias Klose ]
* Refresh patches.
* openjdk-8-jdk-headless: Add Breaks/Replaces for moved clhsdb binary.
LP: #1845873.
-
- -- Matthias Klose <[email protected]> Mon, 30 Sep 2019 14:03:48 +0200
+ * debian/tests/control: Depend on g++ instead of build-essential or libc6-dev.
+ * Bump standards vesion.
+
+ [ Tiago Stürmer Daitx ]
+ * Improve and fix build tests and autopkgtests:
+ - Update debian/tests/hotspot,jdk,langtools to ignore
+ jtreg-autopkgtest.sh return code.
+ - Create debian/tests/jtdiff-autopkgtest.in as it depends
+ on debian/rules variables.
+ - debian/control.in, debian/control: add default-jre-headless
+ to Build-Depends with a nocheck clause as jtreg requires
+ a JRE in /usr/lib/jvm/default-java.
+ - debian/tests/control:
+ + Add zip and unzip test dependencies required by jdk's
+ test/sun/security/tools/jarsigner/diffend.sh and
+ test/sun/security/tools/jarsigner/emptymanifest.sh.
+ + Depend on default-jre-headless so jtreg will use the
+ JRE from /usr/lib/jvm/default-java.
+ - debian/tests/jtdiff-autopkgtest.sh:
+ + Fail only if an actual regression is detected.
+ + Add the super-diff comparison from jtdiff.
+ + Save failed jtr files for all runs.
+ - debian/tests/jtreg-autopkgtest.sh:
+ + Enable retry of failed tests to trim out flaky tests.
+ + Fix unbound variable.
+ + Keep .jtr files from failed tests only.
+ - debian/patches/jdk-problem-list.diff: ignore failing tests
+ that require more investigation.
+ - debian/rules:
+ + Preserve all JTreport directories in the test output
+ directory.
+ + Use JDK_DIR instead of JDK_TO_TEST for autopkgtest
+ generation.
+ + Package all .jtr files from JTwork as jtreg-autopkgtest.sh
+ makes sure it contains only failed tests.
+ * debian/tests/jdk: add our custom debian/tests/jdk-problem-list.txt to the
+ exclusion list.
+ * debian/tests/jdk-problem-list.txt: custom exclusion rules for jdk tests
+ that fail to run during a build or autopkgtest run.
+ * debian/rules: remove debian/patches/jdk-problem-list.diff.
+ * debian/patches/jdk-problem-list.diff: jtreg allows for extra exclusion
+ files thus there's no need to patch upstream's exclusion list.
+ * debian/tests/control: mark all autopkgtests as flaky.
+ * debian/tests/hotspot-archs: generated by debian/rules, contains a list of
+ archs that supports a hotspot vm.
+ * debian/tests/jdk: run only when the host arch is a hotspot vm - allow
+ override through an environment variable.
+ * debian/rules: update gen-autopkgtests to echo supported hotspot archs.
+
+ -- Matthias Klose <[email protected]> Tue, 01 Oct 2019 08:47:18 +0200
openjdk-8 (8u232-b04-1) experimental; urgency=medium
=== modified file 'debian/control'
--- debian/control 2019-09-30 12:07:15 +0000
+++ debian/control 2019-10-01 07:38:06 +0000
@@ -1,12 +1,13 @@
Source: openjdk-8
Section: java
Priority: optional
-Maintainer: OpenJDK Team <[email protected]>
+Maintainer: Ubuntu Developers <[email protected]>
+XSBC-Original-Maintainer: OpenJDK Team <[email protected]>
Uploaders: Matthias Klose <[email protected]>
Build-Depends: debhelper (>= 9), quilt, m4, lsb-release, zip, unzip,
sharutils, gawk, cpio, pkg-config, procps, wdiff, tzdata,
xvfb<!nocheck>, xauth<!nocheck>, xfonts-base<!nocheck>, libgl1-mesa-dri [!x32]<!nocheck>, xfwm4<!nocheck>, x11-xkb-utils<!nocheck>, dbus-x11<!nocheck>,
- jtreg<!nocheck>, testng<!nocheck>, time,
+ jtreg<!nocheck>, testng<!nocheck>, default-jre-headless<!nocheck>, time,
fastjar (>= 2:0.96-0ubuntu2),
autoconf (>= 2.69), automake, autotools-dev, ant, ant-optional,
g++-8,
@@ -15,7 +16,7 @@
libffi-dev,
zlib1g-dev, libattr1-dev, libpng-dev, libjpeg-dev, libgif-dev, libpulse-dev (>= 0.9.12) [!alpha], systemtap-sdt-dev [!sh4],
libnss3-dev (>= 2:3.17.1), openjdk-8-jdk-headless <cross>, dpkg-dev (>= 1.17.14),
-Standards-Version: 4.4.0
+Standards-Version: 4.4.1
Homepage: http://openjdk.java.net/
Vcs-Bzr: http://bazaar.launchpad.net/~openjdk/openjdk/openjdk8
Vcs-Browser: https://code.launchpad.net/~openjdk/openjdk/openjdk8
=== modified file 'debian/control.in'
--- debian/control.in 2019-09-30 12:07:15 +0000
+++ debian/control.in 2019-10-01 07:38:06 +0000
@@ -1,12 +1,13 @@
Source: @basename@
Section: java
Priority: optional
-Maintainer: OpenJDK Team <[email protected]>
+Maintainer: Ubuntu Developers <[email protected]>
+XSBC-Original-Maintainer: OpenJDK Team <[email protected]>
Uploaders: Matthias Klose <[email protected]>
Build-Depends: debhelper (>= 9), quilt, m4, lsb-release, zip, unzip,
sharutils, gawk, cpio, pkg-config, procps, wdiff, tzdata,
xvfb@nocheck@, xauth@nocheck@, xfonts-base@nocheck@, libgl1-mesa-dri [!x32]@nocheck@, xfwm4@nocheck@, x11-xkb-utils@nocheck@, dbus-x11@nocheck@,
- jtreg@nocheck@, testng@nocheck@, time,
+ jtreg@nocheck@, testng@nocheck@, default-jre-headless@nocheck@, time,
@bd_fastjar@
@bd_autotools@ @bd_ant@
@bd_gcc@
@@ -15,7 +16,7 @@
@bd_zero@ @bd_shark@
@bd_syslibs@ @bd_pulsejava@ @bd_systemtap@
@bd_nss@ @bd_cross@
-Standards-Version: 4.4.0
+Standards-Version: 4.4.1
Homepage: http://openjdk.java.net/
Vcs-Bzr: http://bazaar.launchpad.net/~openjdk/openjdk/openjdk8
Vcs-Browser: https://code.launchpad.net/~openjdk/openjdk/openjdk8
=== modified file 'debian/rules'
--- debian/rules 2019-10-01 06:30:46 +0000
+++ debian/rules 2019-10-01 07:38:06 +0000
@@ -933,12 +933,14 @@
debian/tests/%.sh: debian/tests/%.in debian/rules
sed \
- -e 's,@JDK_TO_TEST@,/$(basedir),g' \
+ -e 's,@JDK_DIR@,/$(basedir),g' \
-e 's/@jtreg_archs@/$(jtreg_archs)/g' \
+ -e 's:@doc_dir@:/usr/share/doc/$(p_jrehl)/:g' \
$< > $@;
chmod +x $@
-gen-autopkgtests: debian/tests/jtreg-autopkgtest.sh
+gen-autopkgtests: debian/tests/jtdiff-autopkgtest.sh debian/tests/jtreg-autopkgtest.sh
+ echo $(hotspot_archs) > debian/tests/hotspot-archs
packaging-files:
for f in debian/*.in debian/patches/*.in; do \
@@ -1241,13 +1243,10 @@
@echo "END jtreg-summary-$(VMNAME)"
-for i in hotspot langtools jdk; do \
- test -f jtreg-test-output/check-$$i-$(VMNAME).log || continue; \
- for t in $$(egrep '^(FAILED|Error)' jtreg-test-output/check-$$i-$(VMNAME).log | sed 's/.* \(.*\)\.[^.][^.]*$$/\1/'); do \
- echo jtreg-test-output/$$i/JTwork/$$t.jtr; \
- done; \
+ find jtreg-test-output/$$i/JTwork/ -name '*.jtr'; \
done > jtreg-test-output/failed_tests-$(VMNAME).list
- GZIP=-9vn tar -C . -c -z -f jtreg-test-output/failed_tests-$(VMNAME).tar.gz -T jtreg-test-output/failed_tests-$(VMNAME).list
- GZIP=-9vn tar -C . -c -z -f jtreg-test-output/jtreport-$(VMNAME).tar.gz jtreg-test-output/*/JTreport/
+ GZIP=-9vn tar --ignore-failed-read -C . -c -z -f jtreg-test-output/failed_tests-$(VMNAME).tar.gz -T jtreg-test-output/failed_tests-$(VMNAME).list
+ GZIP=-9vn tar -C . -c -z -f jtreg-test-output/jtreport-$(VMNAME).tar.gz $$(find jtreg-test-output -name JTreport)
else
echo "jtreg harness not run for this build" > jtreg-test-output/jtreg_output-$(VMNAME)
endif
@@ -1260,19 +1259,19 @@
check-hotspot: stamps/build
mkdir -p jtreg-test-output/hotspot
JTREG_OPTIONS=$(JTREG_OPTIONS) VMNAME=$(VMNAME) JDK_TO_TEST=$(builddir)/$(sdkimg) \
- AUTOPKGTEST_TMP=/tmp/ AUTOPKGTEST_ARTIFACTS=jtreg-test-output/hotspot/ \
+ AUTOPKGTEST_TMP=/tmp/ AUTOPKGTEST_ARTIFACTS=jtreg-test-output/ \
$(TIME) debian/tests/hotspot | tee jtreg-test-output/[email protected]
check-langtools: stamps/build
mkdir -p jtreg-test-output/langtools
JTREG_OPTIONS=$(JTREG_OPTIONS) VMNAME=$(VMNAME) JDK_TO_TEST=$(builddir)/$(sdkimg) \
- AUTOPKGTEST_TMP=/tmp/ AUTOPKGTEST_ARTIFACTS=jtreg-test-output/langtools/ \
+ AUTOPKGTEST_TMP=/tmp/ AUTOPKGTEST_ARTIFACTS=jtreg-test-output/ \
$(TIME) debian/tests/langtools | tee jtreg-test-output/[email protected]
check-jdk: stamps/build
mkdir -p jtreg-test-output/jdk
JTREG_OPTIONS=$(JTREG_OPTIONS) VMNAME=$(VMNAME) JDK_TO_TEST=$(builddir)/$(sdkimg) \
- AUTOPKGTEST_TMP=/tmp/ AUTOPKGTEST_ARTIFACTS=jtreg-test-output/jdk/ \
+ AUTOPKGTEST_TMP=/tmp/ AUTOPKGTEST_ARTIFACTS=jtreg-test-output/ \
$(TIME) debian/tests/jdk | tee jtreg-test-output/[email protected]
jtregcheck: $(TESTS_TO_RUN)
=== modified file 'debian/tests/control'
--- debian/tests/control 2019-03-17 17:29:36 +0000
+++ debian/tests/control 2019-10-01 07:38:06 +0000
@@ -1,7 +1,7 @@
Tests: hotspot, langtools
-Depends: @, jtreg, testng, gcc
-Restrictions: allow-stderr, skippable
+Depends: @, default-jre-headless, jtreg, testng, g++
+Restrictions: allow-stderr, skippable, flaky
Tests: jdk
-Depends: @, jtreg, testng, gcc, xfwm4, xvfb, dbus-x11
-Restrictions: allow-stderr, skippable
+Depends: @, default-jre-headless, jtreg, testng, g++, xfwm4, xvfb, dbus-x11, unzip, zip
+Restrictions: allow-stderr, skippable, flaky
=== modified file 'debian/tests/hotspot'
--- debian/tests/hotspot 2018-12-05 08:04:39 +0000
+++ debian/tests/hotspot 2019-10-01 07:38:06 +0000
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/bin/bash
set -o errexit
set -o errtrace
set -o pipefail
@@ -6,5 +6,5 @@
[ -d hotspot/test ] || tar xf hotspot.tar.xz hotspot/test/
-debian/tests/jtreg-autopkgtest.sh -dir:hotspot/test compiler gc runtime sanity serviceability
+debian/tests/jtreg-autopkgtest.sh hotspot -dir:hotspot/test compiler gc runtime sanity serviceability || true
debian/tests/jtdiff-autopkgtest.sh hotspot
=== added file 'debian/tests/hotspot-archs'
--- debian/tests/hotspot-archs 1970-01-01 00:00:00 +0000
+++ debian/tests/hotspot-archs 2019-10-01 07:38:06 +0000
@@ -0,0 +1,1 @@
+amd64 i386 arm64 ppc64 ppc64el sparc64 armhf kfreebsd-amd64 kfreebsd-i386
=== modified file 'debian/tests/jdk'
--- debian/tests/jdk 2019-03-17 17:29:36 +0000
+++ debian/tests/jdk 2019-10-01 07:38:06 +0000
@@ -19,11 +19,24 @@
export XAUTHORITY="${HOME}/.Xauthority"
export DISPLAY=:10
+host_arch="${DEB_HOST_ARCH:-$(dpkg --print-architecture)}"
+
+if ! grep -q -w "${host_arch}" debian/tests/hotspot-archs; then
+ echo "Warning: architecture not listed in debian/tests/hotspot-archs. The jdk testsuite might timeout."
+ if [ -z "${FORCE_JDK_TEST+x}" ]; then
+ echo "Skipping test. Set environment variable FORCE_JDK_TEST to override this check."
+ exit 77
+ fi
+fi
+
debian/tests/start-xvfb.sh 10 &
sleep 3
-debian/tests/jtreg-autopkgtest.sh -exclude:jdk/test/ProblemList.txt -dir:jdk/test \
+debian/tests/jtreg-autopkgtest.sh jdk \
+ -dir:jdk/test \
+ -exclude:jdk/test/ProblemList.txt \
+ -exclude:debian/tests/jdk-problem-list.txt \
:jdk_core :jdk_svc :jdk_beans :jdk_imageio :jdk_sound :jdk_sctp \
- javax/accessibility com/sun/java/swing javax/print sun/pisces com/sun/awt
+ javax/accessibility com/sun/java/swing javax/print sun/pisces com/sun/awt || true
debian/tests/jtdiff-autopkgtest.sh jdk
=== added file 'debian/tests/jdk-problem-list.txt'
--- debian/tests/jdk-problem-list.txt 1970-01-01 00:00:00 +0000
+++ debian/tests/jdk-problem-list.txt 2019-10-01 07:38:06 +0000
@@ -0,0 +1,20 @@
+############################################################################
+
+# Debian - tests to be fixed
+
+############################################################################
+# Execution failed: `main' threw exception:
+# java.awt.AWTError:
+# Error trying to install Assistive Technology:
+# org.GNOME.Accessibility.AtkWrapper java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessClassInPackage.org.GNOME.Accessibility")
+# Need to either disable AtkWrapper or add it to the security policy
+java/beans/PropertyEditor/TestFontClass.java linux-all
+
+# Execution failed: `main' threw exception: java.lang.Exception: At least one cacert test failed
+# Need to update tests to conform to Debian/Ubuntu certificates
+sun/security/lib/cacerts/VerifyCACerts.java linux-all
+
+# timing-out, exit 1, need to investigate why
+#sun/tools/clhsdb/Basic.sh linux-all
+#sun/tools/hsdb/Basic.sh linux-all
+
=== added file 'debian/tests/jtdiff-autopkgtest.in'
--- debian/tests/jtdiff-autopkgtest.in 1970-01-01 00:00:00 +0000
+++ debian/tests/jtdiff-autopkgtest.in 2019-10-01 07:38:06 +0000
@@ -0,0 +1,52 @@
+#!/bin/bash
+set -o errexit
+set -o errtrace
+set -o pipefail
+set -o nounset
+
+testsuite=$1
+shift
+
+if [ -z "${AUTOPKGTEST_TMP+x}" ] || [ -z "${AUTOPKGTEST_ARTIFACTS+x}" ]; then
+ echo "Environment variables AUTOPKGTEST_TMP and AUTOPKGTEST_ARTIFACTS must be set" >&2
+ exit 1
+fi
+
+host_arch=${DEB_HOST_ARCH:-$(dpkg --print-architecture)}
+
+vmname=${VMNAME:-hotspot}
+
+jt_report_tb="@doc_dir@/test-${host_arch}/jtreport-${vmname}.tar.gz"
+
+if [ ! -f "${jt_report_tb}" ]; then
+ echo "Unable to compare jtreg results: no build jtreport found for ${vmname}/${host_arch}."
+ echo "Reason: '${jt_report_tb}' does not exist."
+ exit 77
+fi
+
+# create directories to hold the results
+mkdir -p "${AUTOPKGTEST_ARTIFACTS}/${testsuite}"
+mkdir -p "${AUTOPKGTEST_TMP}/openjdk-pkg-jtreg-report"
+
+current_report_dir="${AUTOPKGTEST_ARTIFACTS}/${testsuite}"
+previous_report_dir="${AUTOPKGTEST_TMP}/openjdk-pkg-jtreg-report/${testsuite}"
+
+# extract testsuite results from openjdk package
+[ -d "${previous_report_dir}" ] || \
+ tar -xf "${jt_report_tb}" --strip-components=1 -C "${AUTOPKGTEST_TMP}/openjdk-pkg-jtreg-report"
+
+
+jtdiff -o "${current_report_dir}/jtdiff.html" "${previous_report_dir}/JTreport" "${current_report_dir}/JTreport" || true
+jtdiff "${previous_report_dir}/JTreport" "${current_report_dir}/JTreport" | tee "${current_report_dir}/jtdiff.txt" || true
+
+# create jdiff super-diff structure
+jtdiff_dir="${AUTOPKGTEST_TMP}/jtdiff-${testsuite}/${host_arch}"
+mkdir -p "${jtdiff_dir}/"{1,2} "${current_report_dir}/jtdiff-super"
+ln -sf "${previous_report_dir}/"[0-9] "${jtdiff_dir}/1/"
+ln -sf "${current_report_dir}/"[0-9] "${jtdiff_dir}/2/"
+
+# run jtdiff super-diff
+jtdiff -o "${current_report_dir}/jtdiff-super/" -s "${AUTOPKGTEST_TMP}/jtdiff-${testsuite}/" || true
+
+# fail if we detect a regression
+if egrep '^(pass|---) +(fail|error)' "${current_report_dir}/jtdiff.txt"; then exit 1; else exit 0; fi
=== modified file 'debian/tests/jtdiff-autopkgtest.sh'
--- debian/tests/jtdiff-autopkgtest.sh 2019-04-29 12:53:16 +0000
+++ debian/tests/jtdiff-autopkgtest.sh 2019-10-01 07:38:06 +0000
@@ -14,18 +14,9 @@
host_arch=${DEB_HOST_ARCH:-$(dpkg --print-architecture)}
-# don't mess around with JT_* env vars. If JDK_TO_TEST is set, then the
-# script is called from the build, if not, from the autopkg tests
-if [ -z "$JDK_TO_TEST" ]; then
- JDK_TO_TEST=@JDK_TO_TEST@
-fi
-JVERSION=$($JDK_TO_TEST/bin/java -version 2>&1| sed -n '1s/.*"\(.*\)".*/\1/p')
-JMAJOR=$(echo $JVERSION | sed 's/\..*//')
-
vmname=${VMNAME:-hotspot}
-jt_report_tb="/usr/share/doc/openjdk-$JMAJOR-jdk/test-${host_arch}/jtreport-${vmname}.tar.gz"
-build_report_dir="${AUTOPKGTEST_TMP}/jtreg-test-output/${testsuite}/JTreport"
+jt_report_tb="/usr/share/doc/openjdk-8-jre-headless//test-${host_arch}/jtreport-${vmname}.tar.gz"
if [ ! -f "${jt_report_tb}" ]; then
echo "Unable to compare jtreg results: no build jtreport found for ${vmname}/${host_arch}."
@@ -33,9 +24,29 @@
exit 77
fi
-# extract testsuite results from original openjdk build
-[ -d "${build_report_dir}" ] || \
- tar -xf "${jt_report_tb}" -C "${AUTOPKGTEST_TMP}"
-
-jtdiff -o "${AUTOPKGTEST_ARTIFACTS}/jtdiff.html" "${build_report_dir}" "${AUTOPKGTEST_ARTIFACTS}/JTreport" || true
-jtdiff "${build_report_dir}" "${AUTOPKGTEST_ARTIFACTS}/JTreport" | tee "${AUTOPKGTEST_ARTIFACTS}/jtdiff.txt"
+# create directories to hold the results
+mkdir -p "${AUTOPKGTEST_ARTIFACTS}/${testsuite}"
+mkdir -p "${AUTOPKGTEST_TMP}/openjdk-pkg-jtreg-report"
+
+current_report_dir="${AUTOPKGTEST_ARTIFACTS}/${testsuite}"
+previous_report_dir="${AUTOPKGTEST_TMP}/openjdk-pkg-jtreg-report/${testsuite}"
+
+# extract testsuite results from openjdk package
+[ -d "${previous_report_dir}" ] || \
+ tar -xf "${jt_report_tb}" --strip-components=1 -C "${AUTOPKGTEST_TMP}/openjdk-pkg-jtreg-report"
+
+
+jtdiff -o "${current_report_dir}/jtdiff.html" "${previous_report_dir}/JTreport" "${current_report_dir}/JTreport" || true
+jtdiff "${previous_report_dir}/JTreport" "${current_report_dir}/JTreport" | tee "${current_report_dir}/jtdiff.txt" || true
+
+# create jdiff super-diff structure
+jtdiff_dir="${AUTOPKGTEST_TMP}/jtdiff-${testsuite}/${host_arch}"
+mkdir -p "${jtdiff_dir}/"{1,2} "${current_report_dir}/jtdiff-super"
+ln -sf "${previous_report_dir}/"[0-9] "${jtdiff_dir}/1/"
+ln -sf "${current_report_dir}/"[0-9] "${jtdiff_dir}/2/"
+
+# run jtdiff super-diff
+jtdiff -o "${current_report_dir}/jtdiff-super/" -s "${AUTOPKGTEST_TMP}/jtdiff-${testsuite}/" || true
+
+# fail if we detect a regression
+if egrep '^(pass|---) +(fail|error)' "${current_report_dir}/jtdiff.txt"; then exit 1; else exit 0; fi
=== modified file 'debian/tests/jtreg-autopkgtest.in'
--- debian/tests/jtreg-autopkgtest.in 2019-04-29 12:53:16 +0000
+++ debian/tests/jtreg-autopkgtest.in 2019-10-01 07:38:06 +0000
@@ -4,17 +4,18 @@
set -o pipefail
set -o nounset
+testsuite=$1
+shift
+
if [ -z "${AUTOPKGTEST_TMP+x}" ] || [ -z "${AUTOPKGTEST_ARTIFACTS+x}" ]; then
echo "Environment variables AUTOPKGTEST_TMP and AUTOPKGTEST_ARTIFACTS must be set" >&2
exit 1
fi
-host_arch=${DEB_HOST_ARCH:-$(dpkg --print-architecture)}
+host_arch="${DEB_HOST_ARCH:-$(dpkg --print-architecture)}"
-# don't mess around with JT_* env vars. If JDK_TO_TEST is set, then the
-# script is called from the build, if not, from the autopkg tests
-if [ -z "$JDK_TO_TEST" ]; then
- JDK_TO_TEST=$(echo @JDK_TO_TEST@ | sed "s/-[^-]*$/-$host_arch/")
+if [ -z "${JDK_TO_TEST+x}" ]; then
+ JDK_TO_TEST=$(echo @JDK_DIR@ | sed "s/-[^-]*$/-$host_arch/")
fi
jtreg_version="$(dpkg-query -W jtreg | cut -f2)"
@@ -29,13 +30,13 @@
fi
# check java binary
-if [ ! -x "${JDK_TO_TEST}/bin/java" ]; then
+if [ ! -x "${JDK_TO_TEST}/bin/java" ]; then
echo "Error: '${JDK_TO_TEST}/bin/java' is not an executable." >&2
exit 1
fi
# restrict the tests to a few archs (set from debian/rules)
-if ! echo "${host_arch}" | grep -E "^($(echo @jtreg_archs@ | tr ' ' '|'))$"; then
+if ! echo "${host_arch}" | grep -qE "^($(echo @jtreg_archs@ | tr ' ' '|'))$"; then
echo "Error: ${host_arch} is not on the jtreg_archs list, ignoring it."
exit 77
fi
@@ -78,14 +79,43 @@
trap "cleanup" EXIT INT TERM ERR
-jtreg ${jt_options} \
- -verbose:summary \
- -automatic \
- -retain:none \
- -ignore:quiet \
- -agentvm \
- -timeout:5 \
- -workDir:"${AUTOPKGTEST_ARTIFACTS}/JTwork" \
- -reportDir:"${AUTOPKGTEST_ARTIFACTS}/JTreport" \
- -jdk:${JDK_TO_TEST} \
- $@
+jtwork_dir="${AUTOPKGTEST_TMP}/${testsuite}/JTwork"
+output_dir="${AUTOPKGTEST_ARTIFACTS}/${testsuite}/"
+
+# retry tests with "fail" or "error" status at most 3 times
+for i in 0 1 2 3; do
+ # save each try under its own folder to preserve history
+ report_path="${i}/JTreport"
+ report_dir="${output_dir}/${report_path}"
+ jtreg ${jt_options} \
+ -verbose:summary \
+ -automatic \
+ -retain:none \
+ -ignore:quiet \
+ -agentvm \
+ -timeout:5 \
+ -workDir:"${jtwork_dir}" \
+ -reportDir:"${report_dir}" \
+ -jdk:${JDK_TO_TEST} \
+ ${on_retry:-} $@ \
+ && exit_code=0 && break || exit_code=$?
+
+ # copy .jtr files from failed tests for latter debugging
+ find "${jtwork_dir}" -name '*.jtr' -exec egrep -q '^execStatus=[^Pass]' {} \; -printf "%P\n" \
+ | while IF= read -r jtr; do
+ mkdir -p "$(dirname "${output_dir}/JTwork/${jtr}")"
+ cp --update --preserve --backup=numbered "${jtwork_dir}/${jtr}" "${output_dir}/JTwork/$jtr"
+ done
+
+ # break if jtdiff reports no difference from previous run
+ # also deletes the just created JTreport
+ # DISABLED: don't use it for now as flaky tests could still pass given more retries
+ #jtdiff "${output_dir}/JTreport" "$report_dir" >/dev/null 2>&1 \
+ # && rm -rf "${report_dir}" && break
+
+ # link latest JTreport to output_dir
+ ln -sf -t "${output_dir}" "${report_path}"
+ on_retry="-status:fail,error"
+done
+
+exit $exit_code
=== modified file 'debian/tests/jtreg-autopkgtest.sh'
--- debian/tests/jtreg-autopkgtest.sh 2019-04-29 12:53:16 +0000
+++ debian/tests/jtreg-autopkgtest.sh 2019-10-01 07:38:06 +0000
@@ -4,16 +4,17 @@
set -o pipefail
set -o nounset
+testsuite=$1
+shift
+
if [ -z "${AUTOPKGTEST_TMP+x}" ] || [ -z "${AUTOPKGTEST_ARTIFACTS+x}" ]; then
echo "Environment variables AUTOPKGTEST_TMP and AUTOPKGTEST_ARTIFACTS must be set" >&2
exit 1
fi
-host_arch=${DEB_HOST_ARCH:-$(dpkg --print-architecture)}
+host_arch="${DEB_HOST_ARCH:-$(dpkg --print-architecture)}"
-# don't mess around with JT_* env vars. If JDK_TO_TEST is set, then the
-# script is called from the build, if not, from the autopkg tests
-if [ -z "$JDK_TO_TEST" ]; then
+if [ -z "${JDK_TO_TEST+x}" ]; then
JDK_TO_TEST=$(echo /usr/lib/jvm/java-8-openjdk-amd64 | sed "s/-[^-]*$/-$host_arch/")
fi
@@ -29,13 +30,13 @@
fi
# check java binary
-if [ ! -x "${JDK_TO_TEST}/bin/java" ]; then
+if [ ! -x "${JDK_TO_TEST}/bin/java" ]; then
echo "Error: '${JDK_TO_TEST}/bin/java' is not an executable." >&2
exit 1
fi
# restrict the tests to a few archs (set from debian/rules)
-if ! echo "${host_arch}" | grep -E "^($(echo amd64 i386 arm64 ppc64 ppc64el sparc64 armhf kfreebsd-amd64 kfreebsd-i386 alpha arm64 armel armhf ia64 mips mipsel mips64 mips64el powerpc powerpcspe ppc64 ppc64el s390x sh4 x32 | tr ' ' '|'))$"; then
+if ! echo "${host_arch}" | grep -qE "^($(echo amd64 i386 arm64 ppc64 ppc64el sparc64 armhf kfreebsd-amd64 kfreebsd-i386 alpha arm64 armel armhf ia64 mips mipsel mips64 mips64el powerpc powerpcspe ppc64 ppc64el s390x sh4 x32 | tr ' ' '|'))$"; then
echo "Error: ${host_arch} is not on the jtreg_archs list, ignoring it."
exit 77
fi
@@ -78,14 +79,43 @@
trap "cleanup" EXIT INT TERM ERR
-jtreg ${jt_options} \
- -verbose:summary \
- -automatic \
- -retain:none \
- -ignore:quiet \
- -agentvm \
- -timeout:5 \
- -workDir:"${AUTOPKGTEST_ARTIFACTS}/JTwork" \
- -reportDir:"${AUTOPKGTEST_ARTIFACTS}/JTreport" \
- -jdk:${JDK_TO_TEST} \
- $@
+jtwork_dir="${AUTOPKGTEST_TMP}/${testsuite}/JTwork"
+output_dir="${AUTOPKGTEST_ARTIFACTS}/${testsuite}/"
+
+# retry tests with "fail" or "error" status at most 3 times
+for i in 0 1 2 3; do
+ # save each try under its own folder to preserve history
+ report_path="${i}/JTreport"
+ report_dir="${output_dir}/${report_path}"
+ jtreg ${jt_options} \
+ -verbose:summary \
+ -automatic \
+ -retain:none \
+ -ignore:quiet \
+ -agentvm \
+ -timeout:5 \
+ -workDir:"${jtwork_dir}" \
+ -reportDir:"${report_dir}" \
+ -jdk:${JDK_TO_TEST} \
+ ${on_retry:-} $@ \
+ && exit_code=0 && break || exit_code=$?
+
+ # copy .jtr files from failed tests for latter debugging
+ find "${jtwork_dir}" -name '*.jtr' -exec egrep -q '^execStatus=[^Pass]' {} \; -printf "%P\n" \
+ | while IF= read -r jtr; do
+ mkdir -p "$(dirname "${output_dir}/JTwork/${jtr}")"
+ cp --update --preserve --backup=numbered "${jtwork_dir}/${jtr}" "${output_dir}/JTwork/$jtr"
+ done
+
+ # break if jtdiff reports no difference from previous run
+ # also deletes the just created JTreport
+ # DISABLED: don't use it for now as flaky tests could still pass given more retries
+ #jtdiff "${output_dir}/JTreport" "$report_dir" >/dev/null 2>&1 \
+ # && rm -rf "${report_dir}" && break
+
+ # link latest JTreport to output_dir
+ ln -sf -t "${output_dir}" "${report_path}"
+ on_retry="-status:fail,error"
+done
+
+exit $exit_code
=== modified file 'debian/tests/langtools'
--- debian/tests/langtools 2018-12-05 08:04:39 +0000
+++ debian/tests/langtools 2019-10-01 07:38:06 +0000
@@ -6,5 +6,5 @@
[ -d langtools/test ] || tar xf langtools.tar.xz langtools/test/
-debian/tests/jtreg-autopkgtest.sh -dir:langtools/test com tools lib
+debian/tests/jtreg-autopkgtest.sh langtools -dir:langtools/test com tools lib || true
debian/tests/jtdiff-autopkgtest.sh langtools
__
This is the maintainer address of Debian's Java team
<https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-java-maintainers>.
Please use
[email protected] for discussions and questions.
