Thanks Jody,
Having spent a few hours on it – the issue is caused by Spark using Ivy for
package management.
Pulling sub-dependencies (like JAI) defaults to Maven Central and Ivy tries to
pull the JAR if the .pom/.xml files are present. Hence it fails because the
binary isn’t there.
Workaround is to manually copy the jai_core-1.1.3.jar file to the appropriate
~/.ivy2 directory. E.g.:
mkdir -p ${HOME}/.ivy2/cache/javax.media/jai_core/jars
cd ${HOME}/.ivy2/cache/javax.media/jai_core/jars
wget
https://s3-ap-southeast-2.amazonaws.com/minus34.com/opensource/jai_core-1.1.3.jar
Cheers,
Hugh
From: Jody Garnett <jody.garn...@gmail.com>
Date: Monday, 15 February 2021 at 6:19 pm
To: Hugh Saalmans <hugh.saalm...@iag.com.au>
Cc: Andrea Aime <andrea.a...@geo-solutions.it>,
"geotools-gt2-users@lists.sourceforge.net"
<geotools-gt2-users@lists.sourceforge.net>
Subject: Re: [External] Re: [Geotools-gt2-users] Including GeoTools in Shaded
JARs - Allowed under LGPL?
Last I checked we could download jai-core from maven, it is a bit of an odd
situation as it is a binary distribution license (not open source).
For a while maven eas being careful to only distribute things like that as a
pom.xml (giving you instructions to install the jar into your local repository
yourself). However search of maven central seems okay:
- https://search.maven.org/artifact/javax.media/jai_core/1.1.3/jar - search
result
- https://repo1.maven.org/maven2/javax/media/jai_core/1.1.3/ - browse page
--
Jody Garnett
On Sun, 14 Feb 2021 at 20:02, Hugh Saalmans
<hugh.saalm...@iag.com.au<mailto:hugh.saalm...@iag.com.au>> wrote:
Thanks Jody,
The Apache Sedona team have subsequently provided an alternative to using a
shaded JAR; by adding code to your script to download the GeoTools JARs on
demand when you start an Apache Spark session. (I wasn’t aware you can set
multiple repos to download JARs in Spark).
One issue though, related to GeoTools v24.0, the jai-core 1.1.3 JAR is missing
from Maven?! The .pom file is still there, just not the JAR…
I know it’s not an OSGeo package, but any ideas what’s going on there?
Cheers,
Hugh
From: Jody Garnett <jody.garn...@gmail.com<mailto:jody.garn...@gmail.com>>
Date: Saturday, 13 February 2021 at 2:27 pm
To: Andrea Aime
<andrea.a...@geo-solutions.it<mailto:andrea.a...@geo-solutions.it>>
Cc: Hugh Saalmans <hugh.saalm...@iag.com.au<mailto:hugh.saalm...@iag.com.au>>,
"geotools-gt2-users@lists.sourceforge.net<mailto:geotools-gt2-users@lists.sourceforge.net>"
<geotools-gt2-users@lists.sourceforge.net<mailto:geotools-gt2-users@lists.sourceforge.net>>
Subject: [External] Re: [Geotools-gt2-users] Including GeoTools in Shaded JARs
- Allowed under LGPL?
Great summary Andrea.
Hugh we actually have a FAQ entry on using GeoTools in a shaded jar, or uber
jar, some care is required to ensure you combine the META-INF/services files
(so it is quite tricky to shade).
--
Jody Garnett
On Mon, 8 Feb 2021 at 00:03, Andrea Aime
<andrea.a...@geo-solutions.it<mailto:andrea.a...@geo-solutions.it>> wrote:
Hi Hugh,
with the premise that I'm not a lawyer, I cannot think of a reason why you could
not redistribute GeoTools binaries in a shaded jar.
The LGPL does not require visibility of the project, or advertisement that it's
even
being used (it's just a nice to have), what is mandated, is that if you
redistribute
the binary, then you make the sources available as well (just point to the
github
tag for the version you're distributing), and if you make any change to the
GeoTools sources,
then the modified sources also need to be made available, and they are still
covered
by the LGPL.
Cheers
Andrea
On Mon, Feb 8, 2021 at 7:54 AM Hugh Saalmans
<hugh.saalm...@iag.com.au<mailto:hugh.saalm...@iag.com.au>> wrote:
Hi,
I’d like to check there’s no issue from the OSGeo side making a set of shaded
JARs with GeoTools embedded publicly available? I don’t know enough about the
LGPL license to know what the impact is of combining Apache and LGPL licensed
software into a set of binaries.
The Apache Software Foundation allow this from their end, but won’t allow the
binaries to be part of an ASF project’s official website or repo for the
reasons below.
Background
Apache Sedona v1.0.0 (formerly GeoSpark) was released over the weekend, giving
Apache Spark users access to “Spark native” geospatial functions and types. As
some of you would know – it uses GeoTools under the hood.
However, due to the Apache Software Foundation’s rule that LGPL licensed
software can’t be included in official releases; the Sedona team can’t publish
binaries that include GeoTools – knocking out a big chunk of Sedona
functionality in the official binaries.
To get full functionality – there’s a fair amount of complexity to install
Sedona with Spark as it would require an end user to do a Maven build. I’m
concerned that Pyspark users, with limited Java/Scala experience, will avoid
Sedona for this reason.
Hence, I’d like to host Apache Sedona shaded JARs with GeoTools embedded; in
either AWS S3 or my employer’s public GitHub repo. …or is there a better place
for them?
Cheers,
Hugh
_____________________________________________________________________
The information transmitted in this message and its attachments (if any) is
intended
only for the person or entity to which it is addressed.
The message may contain confidential and/or privileged material. Any review,
retransmission, dissemination or other use of, or taking of any action in
reliance
upon this information, by persons or entities other than the intended recipient
is
prohibited.
If you have received this in error, please contact the sender and delete this
e-mail
and associated material from any computer.
The intended recipient of this e-mail may only use, reproduce, disclose or
distribute
the information contained in this e-mail and any attached files, with the
permission
of the sender.
This message has been scanned for viruses.
_____________________________________________________________________
_______________________________________________
GeoTools-GT2-Users mailing list
GeoTools-GT2-Users@lists.sourceforge.net<mailto:GeoTools-GT2-Users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
--
Regards, Andrea Aime
== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V
for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions
S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax:
+39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it
http://twitter.com/geosolutions_it
------------------------------------------------------- Con riferimento alla
normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento
generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza
inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è
un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo
scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo,
ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene
notizia. This email is intended only for the person or entity to which it is
addressed and may contain information that is privileged, confidential or
otherwise protected from disclosure. We remind that - as provided by European
Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or
the information herein by anyone other than the intended recipient is
prohibited. If you have received this email by mistake, please notify us
immediately by telephone or e-mail.
_______________________________________________
GeoTools-GT2-Users mailing list
GeoTools-GT2-Users@lists.sourceforge.net<mailto:GeoTools-GT2-Users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
_____________________________________________________________________
The information transmitted in this message and its attachments (if any) is
intended
only for the person or entity to which it is addressed.
The message may contain confidential and/or privileged material. Any review,
retransmission, dissemination or other use of, or taking of any action in
reliance
upon this information, by persons or entities other than the intended recipient
is
prohibited.
If you have received this in error, please contact the sender and delete this
e-mail
and associated material from any computer.
The intended recipient of this e-mail may only use, reproduce, disclose or
distribute
the information contained in this e-mail and any attached files, with the
permission
of the sender.
This message has been scanned for viruses.
_____________________________________________________________________
_____________________________________________________________________
The information transmitted in this message and its attachments (if any) is
intended
only for the person or entity to which it is addressed.
The message may contain confidential and/or privileged material. Any review,
retransmission, dissemination or other use of, or taking of any action in
reliance
upon this information, by persons or entities other than the intended recipient
is
prohibited.
If you have received this in error, please contact the sender and delete this
e-mail
and associated material from any computer.
The intended recipient of this e-mail may only use, reproduce, disclose or
distribute
the information contained in this e-mail and any attached files, with the
permission
of the sender.
This message has been scanned for viruses.
_____________________________________________________________________
_______________________________________________
GeoTools-GT2-Users mailing list
GeoTools-GT2-Users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
