Bug#1065395: marked as done (spirv-llvm-translator-14: autopkgtest on s390x uses huge amount of disk space)
Your message dated Sun, 17 Mar 2024 12:04:05 + with message-id and subject line Bug#1065395: fixed in spirv-llvm-translator-18 18.1.0-1 has caused the Debian Bug report #1065395, regarding spirv-llvm-translator-14: autopkgtest on s390x uses huge amount of disk space to be marked as done. This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact ow...@bugs.debian.org immediately.) -- 1065395: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1065395 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems --- Begin Message --- Source: spirv-llvm-translator-14 Version: 14.0.0-10 Severity: serious X-Debbugs-CC: debian...@lists.debian.org User: debian...@lists.debian.org Usertags: issue Dear maintainers, Since a couple of days, our workers on s390x are dying because some test is filling up all disk space. Several days ago, I wrongly suspected src:fenics-dolfinx (bug #1064995) and added it to our reject-list. It didn't solve the issue, so today I spend more time on finding the culprit. Basically every spike above 40% in the graph [1] is a moment that we see issues like: Feb 28 05:38:18 ci-worker-s390x-01 debci[1738391]: gzip: /tmp/debci-worker-43383540-cNnbLE372K/autopkgtest-incoming/testing/s390x/f/fenics-dolfinx/43383540/log.gz: No space left on device Feb 28 05:38:18 ci-worker-s390x-01 debci[1424101]: E: Test for package fenics-dolfinx produced no exit code, aborting One of the suspects started to be spirv-llvm-translator-14, so I ran its autopkgtest manually, while logging disk use every 10 seconds (I started slightly delayed because I monitored the wrong partition first). As you can see below, during the test it grows from 17 GB (at the end) to its peak at 179 GB. That's not acceptable on our infrastructure. One file I happened to spot on the way was build/test/test_output/DebugInfo/Generic/Output/two-cus-from-same-file.ll.tmp: -rw-r--r-- 1 root root 41G Mar 3 19:18 two-cus-from-same-file.ll.tmp I have added spirv-llvm-translator-14 to our reject-list on s390x. As this seems to be a rather new issue, I'm wondering if it's due to: * Add build-needed autopkgtest for spirv-headers compat check. Or maybe something in the toolchain that broke on s390x? Paul [1] https://ci.debian.net/munin/ci-worker-s390x-01/ci-worker-s390x-01/df.html /dev/mapper/3600507630affd250004a 196G 40G 146G 22% /scratch /dev/mapper/3600507630affd250004a 196G 49G 138G 27% /scratch /dev/mapper/3600507630affd250004a 196G 57G 130G 31% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 66G 121G 36% /scratch /dev/mapper/3600507630affd250004a 196G 67G 120G 36% /scratch /dev/mapper/3600507630affd250004a 196G 70G 117G 38% /scratch /dev/mapper/3600507630affd250004a 196G 73G 114G 40% /scratch /dev/mapper/3600507630affd250004a 196G 76G 111G 41% /scratch /dev/mapper/3600507630affd250004a 196G 79G 108G 43% /scratch /dev/mapper/3600507630affd250004a 196G 83G 104G 45% /scratch /dev/mapper/3600507630affd250004a 196G 85G 101G 46% /scratch /dev/mapper/3600507630affd250004a 196G 88G 98G 48% /scratch /dev/mapper/3600507630affd250004a 196G 92G 95G 50% /scratch /dev/mapper/3600507630affd250004a 196G 95G 92G 51% /scratch /dev/mapper/3600507630affd250004a 196G 98G 89G 53% /scratch /dev/mapper/3600507630affd250004a 196G 101G 86G 54% /scratch /dev/mapper/3600507630affd250004a 196G 104G 83G 56% /scratch /dev/mapper/3600507630affd250004a 196G 107G 80G 58% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 66G 121G 36% /scratch /dev/mapper/3600507630affd250004a 196G 68G 118G 37% /scratch /dev/mapper/3600507630affd250004a 196G 72G 115G 39% /scratch /dev/mapper/3600507630affd250004a 196G 75G 112G 41% /scratch /dev/mapper/3600507630affd250004a 196G 78G 109G 42% /scratch /dev/mapper/3600507630affd250004a 196G 81G 106G 44% /scratch /dev/mapper/3600507630affd250004a 196G 85G 102G 46% /scratch /dev/mapper/3600507630affd250004a 196G 87G 99G 47% /scratch /dev/mapper/3600507630affd250004a 196G 90G 96G 49%
Bug#1065395: marked as done (spirv-llvm-translator-14: autopkgtest on s390x uses huge amount of disk space)
Your message dated Sat, 16 Mar 2024 07:06:53 + with message-id and subject line Bug#1065395: fixed in spirv-llvm-translator-17 17.0.0-5 has caused the Debian Bug report #1065395, regarding spirv-llvm-translator-14: autopkgtest on s390x uses huge amount of disk space to be marked as done. This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact ow...@bugs.debian.org immediately.) -- 1065395: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1065395 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems --- Begin Message --- Source: spirv-llvm-translator-14 Version: 14.0.0-10 Severity: serious X-Debbugs-CC: debian...@lists.debian.org User: debian...@lists.debian.org Usertags: issue Dear maintainers, Since a couple of days, our workers on s390x are dying because some test is filling up all disk space. Several days ago, I wrongly suspected src:fenics-dolfinx (bug #1064995) and added it to our reject-list. It didn't solve the issue, so today I spend more time on finding the culprit. Basically every spike above 40% in the graph [1] is a moment that we see issues like: Feb 28 05:38:18 ci-worker-s390x-01 debci[1738391]: gzip: /tmp/debci-worker-43383540-cNnbLE372K/autopkgtest-incoming/testing/s390x/f/fenics-dolfinx/43383540/log.gz: No space left on device Feb 28 05:38:18 ci-worker-s390x-01 debci[1424101]: E: Test for package fenics-dolfinx produced no exit code, aborting One of the suspects started to be spirv-llvm-translator-14, so I ran its autopkgtest manually, while logging disk use every 10 seconds (I started slightly delayed because I monitored the wrong partition first). As you can see below, during the test it grows from 17 GB (at the end) to its peak at 179 GB. That's not acceptable on our infrastructure. One file I happened to spot on the way was build/test/test_output/DebugInfo/Generic/Output/two-cus-from-same-file.ll.tmp: -rw-r--r-- 1 root root 41G Mar 3 19:18 two-cus-from-same-file.ll.tmp I have added spirv-llvm-translator-14 to our reject-list on s390x. As this seems to be a rather new issue, I'm wondering if it's due to: * Add build-needed autopkgtest for spirv-headers compat check. Or maybe something in the toolchain that broke on s390x? Paul [1] https://ci.debian.net/munin/ci-worker-s390x-01/ci-worker-s390x-01/df.html /dev/mapper/3600507630affd250004a 196G 40G 146G 22% /scratch /dev/mapper/3600507630affd250004a 196G 49G 138G 27% /scratch /dev/mapper/3600507630affd250004a 196G 57G 130G 31% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 66G 121G 36% /scratch /dev/mapper/3600507630affd250004a 196G 67G 120G 36% /scratch /dev/mapper/3600507630affd250004a 196G 70G 117G 38% /scratch /dev/mapper/3600507630affd250004a 196G 73G 114G 40% /scratch /dev/mapper/3600507630affd250004a 196G 76G 111G 41% /scratch /dev/mapper/3600507630affd250004a 196G 79G 108G 43% /scratch /dev/mapper/3600507630affd250004a 196G 83G 104G 45% /scratch /dev/mapper/3600507630affd250004a 196G 85G 101G 46% /scratch /dev/mapper/3600507630affd250004a 196G 88G 98G 48% /scratch /dev/mapper/3600507630affd250004a 196G 92G 95G 50% /scratch /dev/mapper/3600507630affd250004a 196G 95G 92G 51% /scratch /dev/mapper/3600507630affd250004a 196G 98G 89G 53% /scratch /dev/mapper/3600507630affd250004a 196G 101G 86G 54% /scratch /dev/mapper/3600507630affd250004a 196G 104G 83G 56% /scratch /dev/mapper/3600507630affd250004a 196G 107G 80G 58% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 66G 121G 36% /scratch /dev/mapper/3600507630affd250004a 196G 68G 118G 37% /scratch /dev/mapper/3600507630affd250004a 196G 72G 115G 39% /scratch /dev/mapper/3600507630affd250004a 196G 75G 112G 41% /scratch /dev/mapper/3600507630affd250004a 196G 78G 109G 42% /scratch /dev/mapper/3600507630affd250004a 196G 81G 106G 44% /scratch /dev/mapper/3600507630affd250004a 196G 85G 102G 46% /scratch /dev/mapper/3600507630affd250004a 196G 87G 99G 47% /scratch /dev/mapper/3600507630affd250004a 196G 90G 96G 49%
Bug#1065395: marked as done (spirv-llvm-translator-14: autopkgtest on s390x uses huge amount of disk space)
Your message dated Fri, 15 Mar 2024 23:44:17 + with message-id and subject line Bug#1065395: fixed in spirv-llvm-translator-16 16.0.0-7 has caused the Debian Bug report #1065395, regarding spirv-llvm-translator-14: autopkgtest on s390x uses huge amount of disk space to be marked as done. This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact ow...@bugs.debian.org immediately.) -- 1065395: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1065395 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems --- Begin Message --- Source: spirv-llvm-translator-14 Version: 14.0.0-10 Severity: serious X-Debbugs-CC: debian...@lists.debian.org User: debian...@lists.debian.org Usertags: issue Dear maintainers, Since a couple of days, our workers on s390x are dying because some test is filling up all disk space. Several days ago, I wrongly suspected src:fenics-dolfinx (bug #1064995) and added it to our reject-list. It didn't solve the issue, so today I spend more time on finding the culprit. Basically every spike above 40% in the graph [1] is a moment that we see issues like: Feb 28 05:38:18 ci-worker-s390x-01 debci[1738391]: gzip: /tmp/debci-worker-43383540-cNnbLE372K/autopkgtest-incoming/testing/s390x/f/fenics-dolfinx/43383540/log.gz: No space left on device Feb 28 05:38:18 ci-worker-s390x-01 debci[1424101]: E: Test for package fenics-dolfinx produced no exit code, aborting One of the suspects started to be spirv-llvm-translator-14, so I ran its autopkgtest manually, while logging disk use every 10 seconds (I started slightly delayed because I monitored the wrong partition first). As you can see below, during the test it grows from 17 GB (at the end) to its peak at 179 GB. That's not acceptable on our infrastructure. One file I happened to spot on the way was build/test/test_output/DebugInfo/Generic/Output/two-cus-from-same-file.ll.tmp: -rw-r--r-- 1 root root 41G Mar 3 19:18 two-cus-from-same-file.ll.tmp I have added spirv-llvm-translator-14 to our reject-list on s390x. As this seems to be a rather new issue, I'm wondering if it's due to: * Add build-needed autopkgtest for spirv-headers compat check. Or maybe something in the toolchain that broke on s390x? Paul [1] https://ci.debian.net/munin/ci-worker-s390x-01/ci-worker-s390x-01/df.html /dev/mapper/3600507630affd250004a 196G 40G 146G 22% /scratch /dev/mapper/3600507630affd250004a 196G 49G 138G 27% /scratch /dev/mapper/3600507630affd250004a 196G 57G 130G 31% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 66G 121G 36% /scratch /dev/mapper/3600507630affd250004a 196G 67G 120G 36% /scratch /dev/mapper/3600507630affd250004a 196G 70G 117G 38% /scratch /dev/mapper/3600507630affd250004a 196G 73G 114G 40% /scratch /dev/mapper/3600507630affd250004a 196G 76G 111G 41% /scratch /dev/mapper/3600507630affd250004a 196G 79G 108G 43% /scratch /dev/mapper/3600507630affd250004a 196G 83G 104G 45% /scratch /dev/mapper/3600507630affd250004a 196G 85G 101G 46% /scratch /dev/mapper/3600507630affd250004a 196G 88G 98G 48% /scratch /dev/mapper/3600507630affd250004a 196G 92G 95G 50% /scratch /dev/mapper/3600507630affd250004a 196G 95G 92G 51% /scratch /dev/mapper/3600507630affd250004a 196G 98G 89G 53% /scratch /dev/mapper/3600507630affd250004a 196G 101G 86G 54% /scratch /dev/mapper/3600507630affd250004a 196G 104G 83G 56% /scratch /dev/mapper/3600507630affd250004a 196G 107G 80G 58% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 66G 121G 36% /scratch /dev/mapper/3600507630affd250004a 196G 68G 118G 37% /scratch /dev/mapper/3600507630affd250004a 196G 72G 115G 39% /scratch /dev/mapper/3600507630affd250004a 196G 75G 112G 41% /scratch /dev/mapper/3600507630affd250004a 196G 78G 109G 42% /scratch /dev/mapper/3600507630affd250004a 196G 81G 106G 44% /scratch /dev/mapper/3600507630affd250004a 196G 85G 102G 46% /scratch /dev/mapper/3600507630affd250004a 196G 87G 99G 47% /scratch /dev/mapper/3600507630affd250004a 196G 90G 96G 49%
Bug#1065395: marked as done (spirv-llvm-translator-14: autopkgtest on s390x uses huge amount of disk space)
Your message dated Thu, 14 Mar 2024 19:52:23 + with message-id and subject line Bug#1065395: fixed in spirv-llvm-translator-15 15.0.0-7 has caused the Debian Bug report #1065395, regarding spirv-llvm-translator-14: autopkgtest on s390x uses huge amount of disk space to be marked as done. This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact ow...@bugs.debian.org immediately.) -- 1065395: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1065395 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems --- Begin Message --- Source: spirv-llvm-translator-14 Version: 14.0.0-10 Severity: serious X-Debbugs-CC: debian...@lists.debian.org User: debian...@lists.debian.org Usertags: issue Dear maintainers, Since a couple of days, our workers on s390x are dying because some test is filling up all disk space. Several days ago, I wrongly suspected src:fenics-dolfinx (bug #1064995) and added it to our reject-list. It didn't solve the issue, so today I spend more time on finding the culprit. Basically every spike above 40% in the graph [1] is a moment that we see issues like: Feb 28 05:38:18 ci-worker-s390x-01 debci[1738391]: gzip: /tmp/debci-worker-43383540-cNnbLE372K/autopkgtest-incoming/testing/s390x/f/fenics-dolfinx/43383540/log.gz: No space left on device Feb 28 05:38:18 ci-worker-s390x-01 debci[1424101]: E: Test for package fenics-dolfinx produced no exit code, aborting One of the suspects started to be spirv-llvm-translator-14, so I ran its autopkgtest manually, while logging disk use every 10 seconds (I started slightly delayed because I monitored the wrong partition first). As you can see below, during the test it grows from 17 GB (at the end) to its peak at 179 GB. That's not acceptable on our infrastructure. One file I happened to spot on the way was build/test/test_output/DebugInfo/Generic/Output/two-cus-from-same-file.ll.tmp: -rw-r--r-- 1 root root 41G Mar 3 19:18 two-cus-from-same-file.ll.tmp I have added spirv-llvm-translator-14 to our reject-list on s390x. As this seems to be a rather new issue, I'm wondering if it's due to: * Add build-needed autopkgtest for spirv-headers compat check. Or maybe something in the toolchain that broke on s390x? Paul [1] https://ci.debian.net/munin/ci-worker-s390x-01/ci-worker-s390x-01/df.html /dev/mapper/3600507630affd250004a 196G 40G 146G 22% /scratch /dev/mapper/3600507630affd250004a 196G 49G 138G 27% /scratch /dev/mapper/3600507630affd250004a 196G 57G 130G 31% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 66G 121G 36% /scratch /dev/mapper/3600507630affd250004a 196G 67G 120G 36% /scratch /dev/mapper/3600507630affd250004a 196G 70G 117G 38% /scratch /dev/mapper/3600507630affd250004a 196G 73G 114G 40% /scratch /dev/mapper/3600507630affd250004a 196G 76G 111G 41% /scratch /dev/mapper/3600507630affd250004a 196G 79G 108G 43% /scratch /dev/mapper/3600507630affd250004a 196G 83G 104G 45% /scratch /dev/mapper/3600507630affd250004a 196G 85G 101G 46% /scratch /dev/mapper/3600507630affd250004a 196G 88G 98G 48% /scratch /dev/mapper/3600507630affd250004a 196G 92G 95G 50% /scratch /dev/mapper/3600507630affd250004a 196G 95G 92G 51% /scratch /dev/mapper/3600507630affd250004a 196G 98G 89G 53% /scratch /dev/mapper/3600507630affd250004a 196G 101G 86G 54% /scratch /dev/mapper/3600507630affd250004a 196G 104G 83G 56% /scratch /dev/mapper/3600507630affd250004a 196G 107G 80G 58% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 66G 121G 36% /scratch /dev/mapper/3600507630affd250004a 196G 68G 118G 37% /scratch /dev/mapper/3600507630affd250004a 196G 72G 115G 39% /scratch /dev/mapper/3600507630affd250004a 196G 75G 112G 41% /scratch /dev/mapper/3600507630affd250004a 196G 78G 109G 42% /scratch /dev/mapper/3600507630affd250004a 196G 81G 106G 44% /scratch /dev/mapper/3600507630affd250004a 196G 85G 102G 46% /scratch /dev/mapper/3600507630affd250004a 196G 87G 99G 47% /scratch /dev/mapper/3600507630affd250004a 196G 90G 96G 49%
Bug#1065395: marked as done (spirv-llvm-translator-14: autopkgtest on s390x uses huge amount of disk space)
Your message dated Wed, 06 Mar 2024 11:50:46 + with message-id and subject line Bug#1065395: fixed in spirv-llvm-translator-14 14.0.0-12 has caused the Debian Bug report #1065395, regarding spirv-llvm-translator-14: autopkgtest on s390x uses huge amount of disk space to be marked as done. This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact ow...@bugs.debian.org immediately.) -- 1065395: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1065395 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems --- Begin Message --- Source: spirv-llvm-translator-14 Version: 14.0.0-10 Severity: serious X-Debbugs-CC: debian...@lists.debian.org User: debian...@lists.debian.org Usertags: issue Dear maintainers, Since a couple of days, our workers on s390x are dying because some test is filling up all disk space. Several days ago, I wrongly suspected src:fenics-dolfinx (bug #1064995) and added it to our reject-list. It didn't solve the issue, so today I spend more time on finding the culprit. Basically every spike above 40% in the graph [1] is a moment that we see issues like: Feb 28 05:38:18 ci-worker-s390x-01 debci[1738391]: gzip: /tmp/debci-worker-43383540-cNnbLE372K/autopkgtest-incoming/testing/s390x/f/fenics-dolfinx/43383540/log.gz: No space left on device Feb 28 05:38:18 ci-worker-s390x-01 debci[1424101]: E: Test for package fenics-dolfinx produced no exit code, aborting One of the suspects started to be spirv-llvm-translator-14, so I ran its autopkgtest manually, while logging disk use every 10 seconds (I started slightly delayed because I monitored the wrong partition first). As you can see below, during the test it grows from 17 GB (at the end) to its peak at 179 GB. That's not acceptable on our infrastructure. One file I happened to spot on the way was build/test/test_output/DebugInfo/Generic/Output/two-cus-from-same-file.ll.tmp: -rw-r--r-- 1 root root 41G Mar 3 19:18 two-cus-from-same-file.ll.tmp I have added spirv-llvm-translator-14 to our reject-list on s390x. As this seems to be a rather new issue, I'm wondering if it's due to: * Add build-needed autopkgtest for spirv-headers compat check. Or maybe something in the toolchain that broke on s390x? Paul [1] https://ci.debian.net/munin/ci-worker-s390x-01/ci-worker-s390x-01/df.html /dev/mapper/3600507630affd250004a 196G 40G 146G 22% /scratch /dev/mapper/3600507630affd250004a 196G 49G 138G 27% /scratch /dev/mapper/3600507630affd250004a 196G 57G 130G 31% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 66G 121G 36% /scratch /dev/mapper/3600507630affd250004a 196G 67G 120G 36% /scratch /dev/mapper/3600507630affd250004a 196G 70G 117G 38% /scratch /dev/mapper/3600507630affd250004a 196G 73G 114G 40% /scratch /dev/mapper/3600507630affd250004a 196G 76G 111G 41% /scratch /dev/mapper/3600507630affd250004a 196G 79G 108G 43% /scratch /dev/mapper/3600507630affd250004a 196G 83G 104G 45% /scratch /dev/mapper/3600507630affd250004a 196G 85G 101G 46% /scratch /dev/mapper/3600507630affd250004a 196G 88G 98G 48% /scratch /dev/mapper/3600507630affd250004a 196G 92G 95G 50% /scratch /dev/mapper/3600507630affd250004a 196G 95G 92G 51% /scratch /dev/mapper/3600507630affd250004a 196G 98G 89G 53% /scratch /dev/mapper/3600507630affd250004a 196G 101G 86G 54% /scratch /dev/mapper/3600507630affd250004a 196G 104G 83G 56% /scratch /dev/mapper/3600507630affd250004a 196G 107G 80G 58% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 66G 121G 36% /scratch /dev/mapper/3600507630affd250004a 196G 68G 118G 37% /scratch /dev/mapper/3600507630affd250004a 196G 72G 115G 39% /scratch /dev/mapper/3600507630affd250004a 196G 75G 112G 41% /scratch /dev/mapper/3600507630affd250004a 196G 78G 109G 42% /scratch /dev/mapper/3600507630affd250004a 196G 81G 106G 44% /scratch /dev/mapper/3600507630affd250004a 196G 85G 102G 46% /scratch /dev/mapper/3600507630affd250004a 196G 87G 99G 47% /scratch /dev/mapper/3600507630affd250004a 196G 90G 96G 49%
Bug#1065395: marked as done (spirv-llvm-translator-14: autopkgtest on s390x uses huge amount of disk space)
Your message dated Tue, 05 Mar 2024 04:35:30 + with message-id and subject line Bug#1065395: fixed in spirv-llvm-translator-14 14.0.0-11 has caused the Debian Bug report #1065395, regarding spirv-llvm-translator-14: autopkgtest on s390x uses huge amount of disk space to be marked as done. This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact ow...@bugs.debian.org immediately.) -- 1065395: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1065395 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems --- Begin Message --- Source: spirv-llvm-translator-14 Version: 14.0.0-10 Severity: serious X-Debbugs-CC: debian...@lists.debian.org User: debian...@lists.debian.org Usertags: issue Dear maintainers, Since a couple of days, our workers on s390x are dying because some test is filling up all disk space. Several days ago, I wrongly suspected src:fenics-dolfinx (bug #1064995) and added it to our reject-list. It didn't solve the issue, so today I spend more time on finding the culprit. Basically every spike above 40% in the graph [1] is a moment that we see issues like: Feb 28 05:38:18 ci-worker-s390x-01 debci[1738391]: gzip: /tmp/debci-worker-43383540-cNnbLE372K/autopkgtest-incoming/testing/s390x/f/fenics-dolfinx/43383540/log.gz: No space left on device Feb 28 05:38:18 ci-worker-s390x-01 debci[1424101]: E: Test for package fenics-dolfinx produced no exit code, aborting One of the suspects started to be spirv-llvm-translator-14, so I ran its autopkgtest manually, while logging disk use every 10 seconds (I started slightly delayed because I monitored the wrong partition first). As you can see below, during the test it grows from 17 GB (at the end) to its peak at 179 GB. That's not acceptable on our infrastructure. One file I happened to spot on the way was build/test/test_output/DebugInfo/Generic/Output/two-cus-from-same-file.ll.tmp: -rw-r--r-- 1 root root 41G Mar 3 19:18 two-cus-from-same-file.ll.tmp I have added spirv-llvm-translator-14 to our reject-list on s390x. As this seems to be a rather new issue, I'm wondering if it's due to: * Add build-needed autopkgtest for spirv-headers compat check. Or maybe something in the toolchain that broke on s390x? Paul [1] https://ci.debian.net/munin/ci-worker-s390x-01/ci-worker-s390x-01/df.html /dev/mapper/3600507630affd250004a 196G 40G 146G 22% /scratch /dev/mapper/3600507630affd250004a 196G 49G 138G 27% /scratch /dev/mapper/3600507630affd250004a 196G 57G 130G 31% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 66G 121G 36% /scratch /dev/mapper/3600507630affd250004a 196G 67G 120G 36% /scratch /dev/mapper/3600507630affd250004a 196G 70G 117G 38% /scratch /dev/mapper/3600507630affd250004a 196G 73G 114G 40% /scratch /dev/mapper/3600507630affd250004a 196G 76G 111G 41% /scratch /dev/mapper/3600507630affd250004a 196G 79G 108G 43% /scratch /dev/mapper/3600507630affd250004a 196G 83G 104G 45% /scratch /dev/mapper/3600507630affd250004a 196G 85G 101G 46% /scratch /dev/mapper/3600507630affd250004a 196G 88G 98G 48% /scratch /dev/mapper/3600507630affd250004a 196G 92G 95G 50% /scratch /dev/mapper/3600507630affd250004a 196G 95G 92G 51% /scratch /dev/mapper/3600507630affd250004a 196G 98G 89G 53% /scratch /dev/mapper/3600507630affd250004a 196G 101G 86G 54% /scratch /dev/mapper/3600507630affd250004a 196G 104G 83G 56% /scratch /dev/mapper/3600507630affd250004a 196G 107G 80G 58% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 65G 122G 35% /scratch /dev/mapper/3600507630affd250004a 196G 66G 121G 36% /scratch /dev/mapper/3600507630affd250004a 196G 68G 118G 37% /scratch /dev/mapper/3600507630affd250004a 196G 72G 115G 39% /scratch /dev/mapper/3600507630affd250004a 196G 75G 112G 41% /scratch /dev/mapper/3600507630affd250004a 196G 78G 109G 42% /scratch /dev/mapper/3600507630affd250004a 196G 81G 106G 44% /scratch /dev/mapper/3600507630affd250004a 196G 85G 102G 46% /scratch /dev/mapper/3600507630affd250004a 196G 87G 99G 47% /scratch /dev/mapper/3600507630affd250004a 196G 90G 96G 49%
