srowen commented on a change in pull request #32253:
URL: https://github.com/apache/spark/pull/32253#discussion_r618549812
##########
File path: mllib-local/pom.xml
##########
@@ -75,48 +75,12 @@
<type>test-jar</type>
<scope>test</scope>
</dependency>
+
+ <dependency>
+ <groupId>dev.ludovic.netlib</groupId>
+ <artifactId>blas</artifactId>
+ </dependency>
</dependencies>
- <profiles>
- <profile>
- <id>netlib-lgpl</id>
Review comment:
First some background - I think the issue with licensing comes from the
submodules that are pulled in by `all`, like:
https://github.com/fommil/netlib-java/blob/master/native_ref/xbuilds/linux-x86_64/pom.xml
netlib-java generates and compiles native code. So things like
https://mvnrepository.com/artifact/com.github.fommil.netlib/netlib-native_system-linux-x86_64/1.1
ship a `.so` and I believe it's that code or what it links to that is LGPL.
I've honestly forgotten the details but seemed pretty clear (it may be, at
least, libgfortran). At least, I recall going over this with a fine-toothed
comb with Cloudera's legal department and they agreed, FWIW.
Anyway, that's the reason behind all this bother with profiles. The wrapper
in netlib-java is OK to include in distributions and compile against. But
adding in the dependencies that bring in native code is not. Hence it's under a
profile that downstream users can turn on for their own builds if desired.
So, my question is, where is the native code coming in here to do the
acceleration? I understand Java 16+ has it built in, that's the whole point of
your change, but we still want to let people flip on dependencies that will
build in bindings to OpenBLAS, etc. Where would that be now? because I see
you're just depending on netlib-java `core`, which doesn't have that. I'm
asking if that's what `blas` is, etc, but not sure that's it either.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
