Re: Potential bug in openjdk-11 on mips64el

[Olek Wojnar]

On Sun, Feb 28, 2021 at 12:05 PM Olek Wojnar  wrote:

> Update: I see that a new version of OpenJDK 11 is currently building for
> mips64el. Does this new version address the directory deletion issue?
>
> If so, then yay and thanks. If not, I'll go ahead and just report a bug
> against the Debian package so it's easier to track. I'm thinking Severity
> Important, does that sound fair?
>

The build still fails with the same error so I reported the bug. [1] Please
comment there if you have any additional information to help address it.
Thanks!

-Olek

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=983878

>

Re: Potential bug in openjdk-11 on mips64el

[Olek Wojnar]

Update: I see that a new version of OpenJDK 11 is currently building for
mips64el. Does this new version address the directory deletion issue?

If so, then yay and thanks. If not, I'll go ahead and just report a bug
against the Debian package so it's easier to track. I'm thinking Severity
Important, does that sound fair?

Thanks!

-Olek

>

Re: Potential bug in openjdk-11 on mips64el

[Olek Wojnar]

Thanks for the ideas so far!

On Sat, Feb 20, 2021 at 3:53 AM Ao Qi  wrote:

> I tried to dpkg-buildpackage bazel-bootstrap, but I failed to
> reproduce the issue. It seems a long time is needed. Is there a
> simpler way to reproduce the problem?
>

When you say that you failed to reproduce the issue, do you mean that the
bazel-bootstrap package was built correctly or that you experienced a
different error? Regarding a simple example, this [1] is what causes the
problem so adapting this snippet to delete a test directory should allow
you to reproduce the problem. A guava dependency should allow you to
satisfy this: `import com.google.common.io.MoreFiles`. But I'm not
exactly a Java expert so I may be missing something here.

Regarding the long build times, if you are able to build in a persistent
environment that should speed up the build. Bazel only rebuilds changed
code so reusing the same build environment should speed up subsequent
builds. (I have done this when building software with Bazel but not with
building Bazel itself because I always build Debian packages in a clean
environment)

Just FYI, we had a workaround for another issue. I don't know if it
> has anything to do with this problem.
>

That looks like it may be related (from what I can tell)! I assume that
workaround is not in the current mips64el OpenJDK in Debian? If you can
confirm on your system that your workaround fixes the problem shown in [1]
then it would be great to have that fix available in Debian!

On Tue, Feb 16, 2021 at 12:29 AM Ao Qi  wrote:
> >
> > On Mon, Feb 15, 2021 at 10:22 PM Aleksey Shipilev 
> wrote:
> > >
> > > On 2/15/21 1:40 AM, Olek Wojnar wrote:
> > > > On Sat, Feb 13, 2021 at 2:21 PM  raphael.jo...@free.fr>> wrote:
> > > > It selects the "Zero" VM, which I assume is the one used on
> mips64el.
> > > > https://openjdk.java.net/projects/zero/
> > > >
> > > > Ah, thanks for the explanation! It helped me to appropriately adjust
> build-depends. Hmm, it looks
> > > > like Zero is not supported on MIPS at the moment, but perhaps that
> site is just out-of-date?
> > >
> > > There only Zero on mips64el for all current OpenJDKs. Zero for
> mips64el should be supported since
> > > JDK 10, see JDK-8186313:
> https://bugs.openjdk.java.net/browse/JDK-8186313
> > >
> > > ...so openjdk-11 is supposed to work.
>

Ok, I made a mistake previously and only activated Zero for the bootstrap
Bazel build and not for the main Bazel build. The latest version in
experimental should have both enabled [2] but it is still failing with the
same error.

> > Thing is, there is hardly anyone who supports mips64el in OpenJDK
> upstream. I think Debian folks are
> > > the most heavy users of it :) So, while you can submit a bug upstream,
> I don't think there is a high
> > > chance anyone picks it up. You can try to ask here:
>

That's unfortunate. :( Would filing a bug in Debian make more sense then?
That way, if we can figure out a fix, it can at least be applied to Debian.


> > I would like to pick it up. However, I am on vacation at the moment
> > and did not have a mips64el machine on hand. If time permits, I will
> > return to the company in three days to follow up on this issue.
>

Thanks for the follow-up! It sounds like we may have a better idea of
what's going on now.

> > It looks to me that Zero mips64el is another instance of
> SecureDirectoryStream bug tail:
> > >
> https://bugs.openjdk.java.net/issues/?jql=text%20~%20%22SecureDirectoryStream%22
> > >
> > > If there is the mips64el machine where this reproduces, the next step
> would be trying the mainline
> > > JDK NIO tests, something like:
> > >
> > >   $ git clone https://github.com/openjdk/jdk
> > >   $ cd jdk
> > >   $ wget https://builds.shipilev.net/jtreg/jtreg.zip
> > >   $ unzip jtreg.zip
> > >   $ ./configure --with-debug-level=fastdebug --with-jtreg=./jtreg
> > >   $ make run-test TEST=java/nio
> > >
> > > ...and if that does not yield failures, then MCVE would be needed.
>

Does anyone here have easy access to a mips64el machine to try this?


-Olek

[1]
https://salsa.debian.org/bazel-team/bazel-bootstrap/-/blob/master/src/java_tools/buildjar/java/com/google/devtools/build/buildjar/VanillaJavaBuilder.java#L374-383
[2]
https://buildd.debian.org/status/fetch.php?pkg=bazel-bootstrap=mips64el=3.5.1%2Bds-3%7Eexp4=1613632868=0

Re: Potential bug in openjdk-11 on mips64el

[Ao Qi]

I tried to dpkg-buildpackage bazel-bootstrap, but I failed to
reproduce the issue. It seems a long time is needed. Is there a
simpler way to reproduce the problem?

Just FYI, we had a workaround for another issue. I don't know if it
has anything to do with this problem.

diff -r 86de6cbb5d97 -r 0cde37559d27
src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c
--- a/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c Thu
May 30 15:01:09 2019 +0800
+++ b/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c Thu
Jun 06 17:51:54 2019 +0800
@@ -568,7 +568,15 @@
 JNU_ThrowInternalError(env, "should not reach here");
 return;
 }
+
+#ifdef __mips__
+// __NR_newfstatat is incorrect on some OS
+// workaround it using glibc's fstatat64
+RESTARTABLE(fstatat64((int)dfd, path, , (int)flag), err);
+#else
 RESTARTABLE((*my_fstatat64_func)((int)dfd, path, , (int)flag), err);
+#endif
+
 if (err == -1) {
 throwUnixException(env, errno);
 } else {

Cheers,
Ao Qi

On Tue, Feb 16, 2021 at 12:29 AM Ao Qi  wrote:
>
> On Mon, Feb 15, 2021 at 10:22 PM Aleksey Shipilev  wrote:
> >
> > On 2/15/21 1:40 AM, Olek Wojnar wrote:
> > > On Sat, Feb 13, 2021 at 2:21 PM  > > > wrote:
> > > It selects the "Zero" VM, which I assume is the one used on mips64el.
> > > https://openjdk.java.net/projects/zero/
> > >
> > > Ah, thanks for the explanation! It helped me to appropriately adjust 
> > > build-depends. Hmm, it looks
> > > like Zero is not supported on MIPS at the moment, but perhaps that site 
> > > is just out-of-date?
> >
> > There only Zero on mips64el for all current OpenJDKs. Zero for mips64el 
> > should be supported since
> > JDK 10, see JDK-8186313: https://bugs.openjdk.java.net/browse/JDK-8186313
> >
> > ...so openjdk-11 is supposed to work.
> >
> > > On Sat, Feb 13, 2021 at 6:56 PM Tiago Daitx  > > > wrote:
> > >
> > >
> > > This seems to indicate that the isDirectory call [4] is getting wrong
> > > file attributes. Following the trail, those file attributes come from
> > > a fstatat call [5] which ultimately come from the native stack in
> > > UnixNativeDispatcher.c [6]. The actual way that fstatat is called
> > > depends on the build flags and defines from the arch, so somebody with
> > > better knowledge might want to take a look at UnixNativeDispatcher.c
> > > [6] and tell if OpenJDK is picking the right function to call for
> > > mips64el.
> > >
> > >
> > > Thanks for the excellent analysis! So is this indeed a bug in OpenJDK on 
> > > mips64el then? I'm happy to
> > > file the bug, referencing all of the information in this thread, if 
> > > that's the probable culprit here.
> >
> > Thing is, there is hardly anyone who supports mips64el in OpenJDK upstream. 
> > I think Debian folks are
> > the most heavy users of it :) So, while you can submit a bug upstream, I 
> > don't think there is a high
> > chance anyone picks it up. You can try to ask here:
> >https://mail.openjdk.java.net/mailman/listinfo/mips-port
>
> I would like to pick it up. However, I am on vacation at the moment
> and did not have a mips64el machine on hand. If time permits, I will
> return to the company in three days to follow up on this issue.
>
> Cheers,
> Ao Qi
>
> >
> > It looks to me that Zero mips64el is another instance of 
> > SecureDirectoryStream bug tail:
> >
> > https://bugs.openjdk.java.net/issues/?jql=text%20~%20%22SecureDirectoryStream%22
> >
> > If there is the mips64el machine where this reproduces, the next step would 
> > be trying the mainline
> > JDK NIO tests, something like:
> >
> >   $ git clone https://github.com/openjdk/jdk
> >   $ cd jdk
> >   $ wget https://builds.shipilev.net/jtreg/jtreg.zip
> >   $ unzip jtreg.zip
> >   $ ./configure --with-debug-level=fastdebug --with-jtreg=./jtreg
> >   $ make run-test TEST=java/nio
> >
> > ...and if that does not yield failures, then MCVE would be needed.
> >
> > --
> > Thanks,
> > -Aleksey

Re: Potential bug in openjdk-11 on mips64el

[Ao Qi]

On Mon, Feb 15, 2021 at 10:22 PM Aleksey Shipilev  wrote:
>
> On 2/15/21 1:40 AM, Olek Wojnar wrote:
> > On Sat, Feb 13, 2021 at 2:21 PM  > > wrote:
> > It selects the "Zero" VM, which I assume is the one used on mips64el.
> > https://openjdk.java.net/projects/zero/
> >
> > Ah, thanks for the explanation! It helped me to appropriately adjust 
> > build-depends. Hmm, it looks
> > like Zero is not supported on MIPS at the moment, but perhaps that site is 
> > just out-of-date?
>
> There only Zero on mips64el for all current OpenJDKs. Zero for mips64el 
> should be supported since
> JDK 10, see JDK-8186313: https://bugs.openjdk.java.net/browse/JDK-8186313
>
> ...so openjdk-11 is supposed to work.
>
> > On Sat, Feb 13, 2021 at 6:56 PM Tiago Daitx  > > wrote:
> >
> >
> > This seems to indicate that the isDirectory call [4] is getting wrong
> > file attributes. Following the trail, those file attributes come from
> > a fstatat call [5] which ultimately come from the native stack in
> > UnixNativeDispatcher.c [6]. The actual way that fstatat is called
> > depends on the build flags and defines from the arch, so somebody with
> > better knowledge might want to take a look at UnixNativeDispatcher.c
> > [6] and tell if OpenJDK is picking the right function to call for
> > mips64el.
> >
> >
> > Thanks for the excellent analysis! So is this indeed a bug in OpenJDK on 
> > mips64el then? I'm happy to
> > file the bug, referencing all of the information in this thread, if that's 
> > the probable culprit here.
>
> Thing is, there is hardly anyone who supports mips64el in OpenJDK upstream. I 
> think Debian folks are
> the most heavy users of it :) So, while you can submit a bug upstream, I 
> don't think there is a high
> chance anyone picks it up. You can try to ask here:
>https://mail.openjdk.java.net/mailman/listinfo/mips-port

I would like to pick it up. However, I am on vacation at the moment
and did not have a mips64el machine on hand. If time permits, I will
return to the company in three days to follow up on this issue.

Cheers,
Ao Qi

>
> It looks to me that Zero mips64el is another instance of 
> SecureDirectoryStream bug tail:
>
> https://bugs.openjdk.java.net/issues/?jql=text%20~%20%22SecureDirectoryStream%22
>
> If there is the mips64el machine where this reproduces, the next step would 
> be trying the mainline
> JDK NIO tests, something like:
>
>   $ git clone https://github.com/openjdk/jdk
>   $ cd jdk
>   $ wget https://builds.shipilev.net/jtreg/jtreg.zip
>   $ unzip jtreg.zip
>   $ ./configure --with-debug-level=fastdebug --with-jtreg=./jtreg
>   $ make run-test TEST=java/nio
>
> ...and if that does not yield failures, then MCVE would be needed.
>
> --
> Thanks,
> -Aleksey

Re: Potential bug in openjdk-11 on mips64el

[Aleksey Shipilev]

On 2/15/21 1:40 AM, Olek Wojnar wrote:

On Sat, Feb 13, 2021 at 2:21 PM mailto:raphael.jo...@free.fr>> wrote:
It selects the "Zero" VM, which I assume is the one used on mips64el.
https://openjdk.java.net/projects/zero/

Ah, thanks for the explanation! It helped me to appropriately adjust build-depends. Hmm, it looks 
like Zero is not supported on MIPS at the moment, but perhaps that site is just out-of-date?


There only Zero on mips64el for all current OpenJDKs. Zero for mips64el should be supported since 
JDK 10, see JDK-8186313: https://bugs.openjdk.java.net/browse/JDK-8186313


...so openjdk-11 is supposed to work.

On Sat, Feb 13, 2021 at 6:56 PM Tiago Daitx > wrote:



This seems to indicate that the isDirectory call [4] is getting wrong
file attributes. Following the trail, those file attributes come from
a fstatat call [5] which ultimately come from the native stack in
UnixNativeDispatcher.c [6]. The actual way that fstatat is called
depends on the build flags and defines from the arch, so somebody with
better knowledge might want to take a look at UnixNativeDispatcher.c
[6] and tell if OpenJDK is picking the right function to call for
mips64el.


Thanks for the excellent analysis! So is this indeed a bug in OpenJDK on mips64el then? I'm happy to 
file the bug, referencing all of the information in this thread, if that's the probable culprit here.


Thing is, there is hardly anyone who supports mips64el in OpenJDK upstream. I think Debian folks are 
the most heavy users of it :) So, while you can submit a bug upstream, I don't think there is a high 
chance anyone picks it up. You can try to ask here:

  https://mail.openjdk.java.net/mailman/listinfo/mips-port

It looks to me that Zero mips64el is another instance of SecureDirectoryStream 
bug tail:
  
https://bugs.openjdk.java.net/issues/?jql=text%20~%20%22SecureDirectoryStream%22

If there is the mips64el machine where this reproduces, the next step would be trying the mainline 
JDK NIO tests, something like:


 $ git clone https://github.com/openjdk/jdk
 $ cd jdk
 $ wget https://builds.shipilev.net/jtreg/jtreg.zip
 $ unzip jtreg.zip
 $ ./configure --with-debug-level=fastdebug --with-jtreg=./jtreg
 $ make run-test TEST=java/nio

...and if that does not yield failures, then MCVE would be needed.

--
Thanks,
-Aleksey

Re: Potential bug in openjdk-11 on mips64el

[Matthias Klose]

On 2/15/21 1:40 AM, Olek Wojnar wrote:
> Done! [1] Unfortunately, I don't see anything useful there but perhaps one
> of you will.
> 
> -Olek
> 
> [1]
> https://gist.githubusercontent.com/olekw/32e54c0829d739e8fa88893a853c0fa8/raw/b2fce4d2ab77555a3d28c22441f1de3cb2d99f38/bazel-bootstrap-zero-jre

The log just shows that the compiler is called once with -J-zero, so I assume
that you use the hotspot VM for anything else.

You could also try to edit
/usr/lib/jvm/java-1.11.0-openjdk-amd64/lib/jvm.cfg
moving the zero line to  the top, and then use the zero VM by default:

$ java --version
openjdk 11.0.10 2021-01-19
OpenJDK Runtime Environment (build 11.0.10+9-Ubuntu-0ubuntu1)
OpenJDK 64-Bit Zero VM (build 11.0.10+9-Ubuntu-0ubuntu1, interpreted mode)

Re: Potential bug in openjdk-11 on mips64el

[Olek Wojnar]

Thank you all for your inputs and suggestions!

On Sat, Feb 13, 2021 at 2:43 AM Matthias Klose  wrote:

>
> you could try to debug with java -zero / javac -J-zero on amd64.


Done! [1] Unfortunately, I don't see anything useful there but perhaps one
of you will.

On Sat, Feb 13, 2021 at 2:21 PM  wrote:

> It selects the "Zero" VM, which I assume is the one used on mips64el.
>
> https://openjdk.java.net/projects/zero/


Ah, thanks for the explanation! It helped me to appropriately adjust
build-depends. Hmm, it looks like Zero is not supported on MIPS at the
moment, but perhaps that site is just out-of-date?

On Sat, Feb 13, 2021 at 6:56 PM Tiago Daitx 
wrote:

>
> This seems to indicate that the isDirectory call [4] is getting wrong
> file attributes. Following the trail, those file attributes come from
> a fstatat call [5] which ultimately come from the native stack in
> UnixNativeDispatcher.c [6]. The actual way that fstatat is called
> depends on the build flags and defines from the arch, so somebody with
> better knowledge might want to take a look at UnixNativeDispatcher.c
> [6] and tell if OpenJDK is picking the right function to call for
> mips64el.
>

Thanks for the excellent analysis! So is this indeed a bug in OpenJDK on
mips64el then? I'm happy to file the bug, referencing all of the
information in this thread, if that's the probable culprit here.

-Olek

[1]
https://gist.githubusercontent.com/olekw/32e54c0829d739e8fa88893a853c0fa8/raw/b2fce4d2ab77555a3d28c22441f1de3cb2d99f38/bazel-bootstrap-zero-jre

Re: Potential bug in openjdk-11 on mips64el

[Tiago Daitx]

On Sat, Feb 13, 2021 at 8:56 PM Tiago Daitx  wrote:
>
> Hi,
>
> This particular exception is being thrown by the call to unlinkat [1]
> which calls native unlinkat. Man page for unlinkat describes EISDIR
> error [2] as:
> "EISDIR pathname refers to a directory, and AT_REMOVEDIR was not
> specified in flags."
>
> Assuming unlinkat is telling the truth and this is indeed a directory,
> it can be seen from the stacktrace that deleteRecursivelySecure is
> calling deleteFile [3] instead of deleteDir (deleteDir does set
> AT_REMOVERDIR).
>
> This seems to indicate that the isDirectory call [4] is getting wrong
> file attributes. Following the trail, those file attributes come from
> a fstatat call [5] which ultimately come from the native stack in
> UnixNativeDispatcher.c [6]. The actual way that fstatat is called
> depends on the build flags and defines from the arch, so somebody with
> better knowledge might want to take a look at UnixNativeDispatcher.c
> [6] and tell if OpenJDK is picking the right function to call for
> mips64el.
>
> References:
> [1] unlinkat call
> https://sources.debian.org/src/openjdk-11/11.0.10+9-1/src/java.base/unix/classes/sun/nio/fs/UnixSecureDirectoryStream.java/#L200
> [2] EISDIR in OpenJDK:
> https://sources.debian.org/src/openjdk-11/11.0.10+9-1/src/hotspot/share/runtime/os.cpp/#L1504
> [3] 
> https://salsa.debian.org/java-team/guava-libraries/-/blob/master/guava/src/com/google/common/io/MoreFiles.java#L630
> [4] 
> https://salsa.debian.org/java-team/guava-libraries/-/blob/master/guava/src/com/google/common/io/MoreFiles.java#L335
> [5] 
> https://sources.debian.org/src/openjdk-11/11.0.10+9-1/src/java.base/unix/classes/sun/nio/fs/UnixFileAttributes.java/#L90

I missed the most important reference

[6] 
https://sources.debian.org/src/openjdk-11/11.0.10+9-1/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c/

-- 
Tiago Stürmer Daitx
Software Engineer
tiago.da...@canonical.com

PGP Key: 4096R/F5B213BE (hkp://keyserver.ubuntu.com)
Fingerprint = 45D0 FE5A 8109 1E91 866E  8CA4 1931 8D5E F5B2 13BE

Re: Potential bug in openjdk-11 on mips64el

[Tiago Daitx]

Hi,

This particular exception is being thrown by the call to unlinkat [1]
which calls native unlinkat. Man page for unlinkat describes EISDIR
error [2] as:
"EISDIR pathname refers to a directory, and AT_REMOVEDIR was not
specified in flags."

Assuming unlinkat is telling the truth and this is indeed a directory,
it can be seen from the stacktrace that deleteRecursivelySecure is
calling deleteFile [3] instead of deleteDir (deleteDir does set
AT_REMOVERDIR).

This seems to indicate that the isDirectory call [4] is getting wrong
file attributes. Following the trail, those file attributes come from
a fstatat call [5] which ultimately come from the native stack in
UnixNativeDispatcher.c [6]. The actual way that fstatat is called
depends on the build flags and defines from the arch, so somebody with
better knowledge might want to take a look at UnixNativeDispatcher.c
[6] and tell if OpenJDK is picking the right function to call for
mips64el.

References:
[1] unlinkat call
https://sources.debian.org/src/openjdk-11/11.0.10+9-1/src/java.base/unix/classes/sun/nio/fs/UnixSecureDirectoryStream.java/#L200
[2] EISDIR in OpenJDK:
https://sources.debian.org/src/openjdk-11/11.0.10+9-1/src/hotspot/share/runtime/os.cpp/#L1504
[3] 
https://salsa.debian.org/java-team/guava-libraries/-/blob/master/guava/src/com/google/common/io/MoreFiles.java#L630
[4] 
https://salsa.debian.org/java-team/guava-libraries/-/blob/master/guava/src/com/google/common/io/MoreFiles.java#L335
[5] 
https://sources.debian.org/src/openjdk-11/11.0.10+9-1/src/java.base/unix/classes/sun/nio/fs/UnixFileAttributes.java/#L90


cheers,

-- 
Tiago Stürmer Daitx
Software Engineer
tiago.da...@canonical.com

PGP Key: 4096R/F5B213BE (hkp://keyserver.ubuntu.com)
Fingerprint = 45D0 FE5A 8109 1E91 866E  8CA4 1931 8D5E F5B2 13BE

Re: Potential bug in openjdk-11 on mips64el

[raphael . jolly]

It selects the "Zero" VM, which I assume is the one used on mips64el.

https://openjdk.java.net/projects/zero/


Raphael

Olek Wojnar wrote:

Hi Matthias,

On Sat, Feb 13, 2021 at 2:43 AM Matthias Klose  wrote:

>
> On 2/12/21 10:26 PM, Olek Wojnar wrote:
> >
> > Any insights or suggestions would be greatly appreciated!
>
> you could try to debug with java -zero / javac -J-zero on amd64.
>

Sure! What does that flag do, anyway? Something optimization-related? I'm
not familiar with it and a quick Internet search yielded only a
mail-archive.com reference to this email. ;)

-Olek

Re: Potential bug in openjdk-11 on mips64el

[Olek Wojnar]

Hi Matthias,

On Sat, Feb 13, 2021 at 2:43 AM Matthias Klose  wrote:

>
> On 2/12/21 10:26 PM, Olek Wojnar wrote:
> >
> > Any insights or suggestions would be greatly appreciated!
>
> you could try to debug with java -zero / javac -J-zero on amd64.
>

Sure! What does that flag do, anyway? Something optimization-related? I'm
not familiar with it and a quick Internet search yielded only a
mail-archive.com reference to this email. ;)

-Olek

Re: Potential bug in openjdk-11 on mips64el

[Matthias Klose]

[also forwarding to the Debian mips porters]

On 2/12/21 10:26 PM, Olek Wojnar wrote:
> Hello Java Team and OpenJDK Team,
> 
> I'm hesitant to start filing potentially serious bugs at this point in the
> release cycle so please let me know if there's something I'm missing in
> this situation.
> 
> I've been trying to get the bazel-bootstrap package to build on mips64el.
> It's *almost* completing now but it runs into an error when trying to clean
> an output directory.[1] This is using `deleteRecursively` from Guava [2]
> which is supposed to delete files and directories recursively, as the name
> implies. However, it throws an error [3] complaining that the target is a
> directory. This error makes no sense and the code works just fine on all
> other architectures supported by Bazel (amd64, arm64, ppc64el, s390x,
> ppc64, and riscv64).
> 
> Any insights or suggestions would be greatly appreciated!

you could try to debug with java -zero / javac -J-zero on amd64.

Matthias

> -Olek
> 
> [1]
> https://salsa.debian.org/bazel-team/bazel-bootstrap/-/blob/olek-mips-riscv-3/src/java_tools/buildjar/java/com/google/devtools/build/buildjar/VanillaJavaBuilder.java#L379
> [2]
> https://salsa.debian.org/java-team/guava-libraries/-/blob/master/guava/src/com/google/common/io/MoreFiles.java#L521
> [3]
> https://buildd.debian.org/status/fetch.php?pkg=bazel-bootstrap=mips64el=3.5.1%2Bds-3%7Eexp2=1612583842=0
>