[Xen-devel] [xen-unstable bisection] complete test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm

2020-02-03 Thread osstest service owner 
branch xen-unstable
xenbranch xen-unstable
job test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm
testid debian-hvm-install

Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  xen git://xenbits.xen.org/xen.git
  Bug introduced:  75259239d85d6e522c164f1f00ace89bb2dbb3e6
  Bug not present: bc7b5343053c323e6edf71377d983f0b303a9637
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/146703/


  commit 75259239d85d6e522c164f1f00ace89bb2dbb3e6
  Author: Paul Durrant 
  Date:   Fri Jan 31 15:01:44 2020 +
  
  libxl_create: make 'soft reset' explicit
  
  The 'soft reset' code path in libxl__domain_make() is currently taken if a
  valid domid is passed into the function. A subsequent patch will enable
  higher levels of the toolstack to determine the domid of newly created or
  restored domains and therefore this criteria for choosing 'soft reset'
  will no longer be usable.
  
  This patch adds an extra boolean option to libxl__domain_make() to specify
  whether it is being invoked in soft reset context and appropriately
  modifies callers to choose the right value. To facilitate this, a new
  'soft_reset' boolean field is added to struct libxl__domain_create_state
  and the 'domid_soft_reset' field is renamed to 'domid' in anticipation of
  its wider remit. For the moment do_domain_create() will always set
  domid to INVALID_DOMID and hence we can add an assertion into
  libxl__domain_create() that, if it is not called in soft reset context,
  the passed in domid is exactly that value.
  
  Whilst in the neighbourhood, some checks of 'restore_fd > -1' have been
  replaced by 'restore_fd >= 0' to be more conventional and consistent with
  checks of 'restore_fd < 0'.
  
  Signed-off-by: Paul Durrant 
  Acked-by: Ian Jackson 


For bisection revision-tuple graph see:
   
http://logs.test-lab.xenproject.org/osstest/results/bisect/xen-unstable/test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm.debian-hvm-install.html
Revision IDs in each graph node refer, respectively, to the Trees above.


Running cs-bisection-step 
--graph-out=/home/logs/results/bisect/xen-unstable/test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm.debian-hvm-install
 --summary-out=tmp/146703.bisection-summary --basis-template=146563 
--blessings=real,real-bisect xen-unstable 
test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm debian-hvm-install
Searching for failure / basis pass:
 146676 fail [host=huxelrebe0] / 146625 [host=chardonnay1] 146611 
[host=elbling1] 146600 [host=fiano0] 146584 [host=fiano1] 146578 [host=pinot0] 
146563 [host=albana0] 146555 [host=debina1] 146543 [host=pinot1] 146534 
[host=huxelrebe1] 146526 [host=albana1] 146514 [host=chardonnay0] 146505 ok.
Failure / basis pass flights: 146676 / 146505
(tree with no url: minios)
(tree with no url: ovmf)
(tree with no url: seabios)
Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
Latest c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
d0d8ad39ecb51cd7497cd524484fe09f50876798 
933ebad2470a169504799a1d95b8e410bd9847ef 
41d8869003e96d8b7250ad1d0246371d6929aca6
Basis pass c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
d0d8ad39ecb51cd7497cd524484fe09f50876798 
933ebad2470a169504799a1d95b8e410bd9847ef 
f190e634daba1a40570700b3e7697d497874c66f
Generating revisions with ./adhoc-revtuple-generator  
git://xenbits.xen.org/linux-pvops.git#c3038e718a19fc596f7b1baba0f83d5146dc7784-c3038e718a19fc596f7b1baba0f83d5146dc7784
 
git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860
 
git://xenbits.xen.org/qemu-xen-traditional.git#d0d8ad39ecb51cd7497cd524484fe09f50876798-d0d8ad39ecb51cd7497cd524484fe09f50876798
 git://xenbits.xen.org/qemu-xen.git#933ebad2470a169504799a1d95b8e41\
 0bd9847ef-933ebad2470a169504799a1d95b8e410bd9847ef 
git://xenbits.xen.org/xen.git#f190e634daba1a40570700b3e7697d497874c66f-41d8869003e96d8b7250ad1d0246371d6929aca6
Loaded 5001 nodes in revision graph
Searching for test results:
 146505 pass c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
d0d8ad39ecb51cd7497cd524484fe09f50876798 
933ebad2470a169504799a1d95b8e410bd9847ef 
f190e634daba1a40570700b3e7697d497874c66f
 146514 [host=chardonnay0]
 146526 [host=albana1]
 146534 [host=huxelrebe1]
 146543

[Xen-devel] [xen-unstable bisection] complete test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm

2020-01-20 Thread osstest service owner 
branch xen-unstable
xenbranch xen-unstable
job test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm
testid debian-hvm-install

Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  xen git://xenbits.xen.org/xen.git
  Bug introduced:  aacc143006429de46932aabae17c13846c71fa45
  Bug not present: 2572c7d76e1aee9b11a23c548cee69b15a35401f
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/146337/


  commit aacc143006429de46932aabae17c13846c71fa45
  Author: Andrew Cooper 
  Date:   Thu Jan 2 21:37:36 2020 +
  
  tools/libxl: Plumb domain_create_state down into libxl__build_pre()
  
  To fix CPUID handling, libxl__build_pre() is going to have to distinguish
  between a brand new VM vs one which is being migrated-in/resumed.
  
  No functional change.
  
  Signed-off-by: Andrew Cooper 
  Acked-by: Ian Jackson 


For bisection revision-tuple graph see:
   
http://logs.test-lab.xenproject.org/osstest/results/bisect/xen-unstable/test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm.debian-hvm-install.html
Revision IDs in each graph node refer, respectively, to the Trees above.


Running cs-bisection-step 
--graph-out=/home/logs/results/bisect/xen-unstable/test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm.debian-hvm-install
 --summary-out=tmp/146337.bisection-summary --basis-template=146058 
--blessings=real,real-bisect xen-unstable 
test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm debian-hvm-install
Searching for failure / basis pass:
 146286 fail [host=italia0] / 146094 [host=fiano0] 146050 [host=albana0] 146039 
[host=albana1] 146030 [host=huxelrebe1] 146018 [host=elbling1] 146006 
[host=chardonnay0] 145982 [host=debina1] 145955 [host=chardonnay1] 145903 
[host=fiano1] 145879 [host=huxelrebe0] 145851 [host=fiano0] 145826 
[host=rimava1] 145796 [host=pinot1] 145773 [host=pinot0] 145749 
[host=huxelrebe1] 145725 ok.
Failure / basis pass flights: 146286 / 145725
(tree with no url: minios)
(tree with no url: ovmf)
(tree with no url: seabios)
Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
Latest c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
d0d8ad39ecb51cd7497cd524484fe09f50876798 
933ebad2470a169504799a1d95b8e410bd9847ef 
1eeedaf5a0d9ed6324f3bd5b700bb22eb4355341
Basis pass b98aebd298246df37b472c52a2ee1023256d02e3 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
d0d8ad39ecb51cd7497cd524484fe09f50876798 
933ebad2470a169504799a1d95b8e410bd9847ef 
0dd92688080202adcc43dcb3486d4143110a66d5
Generating revisions with ./adhoc-revtuple-generator  
git://xenbits.xen.org/linux-pvops.git#b98aebd298246df37b472c52a2ee1023256d02e3-c3038e718a19fc596f7b1baba0f83d5146dc7784
 
git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860
 
git://xenbits.xen.org/qemu-xen-traditional.git#d0d8ad39ecb51cd7497cd524484fe09f50876798-d0d8ad39ecb51cd7497cd524484fe09f50876798
 git://xenbits.xen.org/qemu-xen.git#933ebad2470a169504799a1d95b8e41\
 0bd9847ef-933ebad2470a169504799a1d95b8e410bd9847ef 
git://xenbits.xen.org/xen.git#0dd92688080202adcc43dcb3486d4143110a66d5-1eeedaf5a0d9ed6324f3bd5b700bb22eb4355341
adhoc-revtuple-generator: tree discontiguous: linux-pvops
Loaded 5002 nodes in revision graph
Searching for test results:
 145691 [host=albana1]
 145725 pass b98aebd298246df37b472c52a2ee1023256d02e3 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
d0d8ad39ecb51cd7497cd524484fe09f50876798 
933ebad2470a169504799a1d95b8e410bd9847ef 
0dd92688080202adcc43dcb3486d4143110a66d5
 145749 [host=huxelrebe1]
 145773 [host=pinot0]
 145796 [host=pinot1]
 145826 [host=rimava1]
 145851 [host=fiano0]
 145879 [host=huxelrebe0]
 145903 [host=fiano1]
 145955 [host=chardonnay1]
 146006 [host=chardonnay0]
 145982 [host=debina1]
 146030 [host=huxelrebe1]
 146018 [host=elbling1]
 146039 [host=albana1]
 146050 [host=albana0]
 146094 [host=fiano0]
 146205 fail irrelevant
 146119 []
 146176 fail irrelevant
 146288 pass c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
d0d8ad39ecb51cd7497cd524484fe09f50876798 
933ebad2470a169504799a1d95b8e410bd9847ef 
af01d330cbb350a7f8cd11189c49cd7904d2a9ff
 146237 pass b98aebd298246df37b472c52a2ee1023256d02e3 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
d0d8ad39ecb51cd7497cd524484fe09f50876798 
933ebad2470a169504799a1d95b8e410bd9847ef 
0dd92688080202adcc43dcb3486d4143110a66d5
 146242 fail irrelevant
 146271 pass irrelevant

[Xen-devel] [xen-unstable bisection] complete test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm

2019-10-30 Thread osstest service owner 
branch xen-unstable
xenbranch xen-unstable
job test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm
testid debian-hvm-install

Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  xen git://xenbits.xen.org/xen.git
  Bug introduced:  ad011ad08843f60f9ae17b9ae4aa5907674d72af
  Bug not present: 95596f6ab18feb825006ef8f272041f1d94e6bd1
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/143389/


  commit ad011ad08843f60f9ae17b9ae4aa5907674d72af
  Author: Ian Jackson 
  Date:   Mon Oct 7 17:59:15 2019 +0100
  
  libxl/xl: Overhaul passthrough setting logic
  
  LIBXL_PASSTHROUGH_UNKNOWN (aka "ENABLED" in an earlier uncommitted
  version of this code) is doing double duty.  We actually need all of
  the following to be specifiable:
* "default": enable PT iff we have devices to
  pass through specified in the initial config file.
* "enabled" (and fail if the platform doesn't support it).
* "disabled" (and reject future PT hotplug).
* "share_pt"/"sync_pt": enable PT and set a specific PT mode.
  
  Defaulting and error checking should be done in libxl.  So, we make
  several changes here.
  
  We introduce "enabled", and rename "unknown" to "default".
  
  We move all of the error checking and defaulting code from xl into
  libxl.  Now, libxl__domain_config_setdefault has all of the necessary
  information to get this right.  So we can do it all there.  Choosing
  the specific mode is arch-specific.
  
  We can also arrange to have only one place each which calculates
  (i) whether passthrough needs to be enabled because pt devices were
  specified (ii) whether pt_share can be used (for each arch).
  
  xl now only has to parse the enum in the same way as it parses all
  other enums.
  
  This change fixes a regression from earlier 4.13-pre: until recent
  changes, passthrough was only enabled by default if passthrough
  devices were specified.  We restore this behaviour.
  
  Signed-off-by: Ian Jackson 
  CC: Stefano Stabellini 
  CC: Julien Grall 
  CC: Volodymyr Babchuk 
  CC: Andrew Cooper 
  CC: Paul Durrant 
  CC: Jan Beulich 
  Release-acked-by: Juergen Gross 
  Acked-by: Anthony PERARD 


For bisection revision-tuple graph see:
   
http://logs.test-lab.xenproject.org/osstest/results/bisect/xen-unstable/test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm.debian-hvm-install.html
Revision IDs in each graph node refer, respectively, to the Trees above.


Running cs-bisection-step 
--graph-out=/home/logs/results/bisect/xen-unstable/test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm.debian-hvm-install
 --summary-out=tmp/143389.bisection-summary --basis-template=142750 
--blessings=real,real-bisect xen-unstable 
test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm debian-hvm-install
Searching for failure / basis pass:
 143288 fail [host=debina0] / 143133 [host=chardonnay0] 143036 [host=pinot1] 
143018 [host=fiano1] 142973 [host=italia1] 142907 [host=baroque0] 142865 
[host=pinot0] 142777 [host=albana0] 142750 [host=rimava1] 142722 
[host=chardonnay1] 142683 [host=debina1] 142642 [host=huxelrebe1] 142598 
[host=chardonnay0] 142563 ok.
Failure / basis pass flights: 143288 / 142563
(tree with no url: minios)
(tree with no url: ovmf)
(tree with no url: seabios)
Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
Latest b98aebd298246df37b472c52a2ee1023256d02e3 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
d0d8ad39ecb51cd7497cd524484fe09f50876798 
933ebad2470a169504799a1d95b8e410bd9847ef 
df663157c638d9778fa3ada9859f968fb240
Basis pass 42327896f194f256e5a361e0069985bc8d209b42 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
d0d8ad39ecb51cd7497cd524484fe09f50876798 
933ebad2470a169504799a1d95b8e410bd9847ef 
4ca8eab5ce1893b3048b06921f12157d33ab60f7
Generating revisions with ./adhoc-revtuple-generator  
git://xenbits.xen.org/linux-pvops.git#42327896f194f256e5a361e0069985bc8d209b42-b98aebd298246df37b472c52a2ee1023256d02e3
 
git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860
 
git://xenbits.xen.org/qemu-xen-traditional.git#d0d8ad39ecb51cd7497cd524484fe09f50876798-d0d8ad39ecb51cd7497cd524484fe09f50876798
 git://xenbits.xen.org/qemu-xen.git#933ebad2470a169504799a1d95b8e41\
 0bd9847ef-933ebad2470a169504799a1d95b8e410bd9847ef

[Xen-devel] [xen-unstable bisection] complete test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm

2018-04-04 Thread osstest service owner 
branch xen-unstable
xenbranch xen-unstable
job test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm
testid debian-hvm-install

Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  xen git://xenbits.xen.org/xen.git
  Bug introduced:  55e0590e4bed56db0ea628826409572c94c54ebf
  Bug not present: ca45928e46e300c5de70a779c2a84d1f0e77b8d2
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/121764/


  commit 55e0590e4bed56db0ea628826409572c94c54ebf
  Author: Wei Liu 
  Date:   Tue Mar 27 17:20:50 2018 +0100
  
  Config.mk: update mini-os commit
  
  Signed-off-by: Wei Liu 


For bisection revision-tuple graph see:
   
http://logs.test-lab.xenproject.org/osstest/results/bisect/xen-unstable/test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm.debian-hvm-install.html
Revision IDs in each graph node refer, respectively, to the Trees above.


Running cs-bisection-step 
--graph-out=/home/logs/results/bisect/xen-unstable/test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm.debian-hvm-install
 --summary-out=tmp/121764.bisection-summary --basis-template=121272 
--blessings=real,real-bisect xen-unstable 
test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm debian-hvm-install
Searching for failure / basis pass:
 121721 fail [host=italia0] / 121307 [host=baroque0] 121272 [host=elbling0] 
121059 [host=elbling1] 120988 [host=baroque1] 120943 [host=chardonnay0] 120859 
[host=pinot0] 120253 [host=huxelrebe1] 120189 [host=pinot1] 120120 
[host=chardonnay1] 120076 [host=chardonnay0] 120037 [host=fiano0] 120001 
[host=elbling0] 119970 [host=huxelrebe0] 119879 [host=italia1] 119785 
[host=baroque1] 119713 [host=pinot0] 119651 [host=huxelrebe1] 119592 
[host=elbling1] 119521 [host=baroque0] 119451 ok.
Failure / basis pass flights: 121721 / 119451
(tree with no url: minios)
(tree with no url: ovmf)
(tree with no url: seabios)
Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
Latest c44cfe06dfe2a5f54527e87a48c92a6595d070cc 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
c8ea0457495342c417c3dc033bba25148b279f60 
5c3fdee026a204a59cb392e43a313ab558de9682 
641f9ce2fab1b85479c564d9b27dfeb18a93ed87
Basis pass 7f3bd8db99746a60bcae1ec4059a4756d19b63c2 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
c8ea0457495342c417c3dc033bba25148b279f60 
2b033e396f4fa0981bae1213cdacd15775655a97 
24470b99c1671dca531c2cf5747eda2f8892ecbc
Generating revisions with ./adhoc-revtuple-generator  
git://xenbits.xen.org/linux-pvops.git#7f3bd8db99746a60bcae1ec4059a4756d19b63c2-c44cfe06dfe2a5f54527e87a48c92a6595d070cc
 
git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860
 
git://xenbits.xen.org/qemu-xen-traditional.git#c8ea0457495342c417c3dc033bba25148b279f60-c8ea0457495342c417c3dc033bba25148b279f60
 
git://xenbits.xen.org/qemu-xen.git#2b033e396f4fa0981bae1213cdacd15775655a97-5c3fdee026a204a59cb392e43a313ab558de9682
 
git://xenbits.xen.org/xen.git#24470b99c1671dca531c2cf5747eda2f8892ecbc-641f9ce2fab1b85479c564d9b27dfeb18a93ed87
adhoc-revtuple-generator: tree discontiguous: linux-pvops
Loaded 5096 nodes in revision graph
Searching for test results:
 119451 pass 7f3bd8db99746a60bcae1ec4059a4756d19b63c2 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
c8ea0457495342c417c3dc033bba25148b279f60 
2b033e396f4fa0981bae1213cdacd15775655a97 
24470b99c1671dca531c2cf5747eda2f8892ecbc
 119521 [host=baroque0]
 119592 [host=elbling1]
 119651 [host=huxelrebe1]
 119713 [host=pinot0]
 119785 [host=baroque1]
 119970 [host=huxelrebe0]
 119879 [host=italia1]
 120001 [host=elbling0]
 120076 [host=chardonnay0]
 120037 [host=fiano0]
 120189 [host=pinot1]
 120120 [host=chardonnay1]
 120287 [host=pinot0]
 120253 [host=huxelrebe1]
 120405 [host=pinot0]
 120626 [host=pinot0]
 120767 [host=pinot0]
 120859 [host=pinot0]
 120943 [host=chardonnay0]
 120988 [host=baroque1]
 121059 [host=elbling1]
 121272 [host=elbling0]
 121307 [host=baroque0]
 121322 fail irrelevant
 121342 fail irrelevant
 121404 fail irrelevant
 121682 fail c44cfe06dfe2a5f54527e87a48c92a6595d070cc 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
c8ea0457495342c417c3dc033bba25148b279f60 
5c3fdee026a204a59cb392e43a313ab558de9682 
6bbcb226cebac90f8ce5ac901e000bfd3ad783c5
 121699 blocked irrelevant
 121709 pass irrelevant
 121719 pass 7f3bd8db99746a60bcae1ec4059a4756d19b63c2 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
c8ea0457495342c417c3dc033bba25148b279f60