Hi Andrea,
using Java 11 (OpenJDK 11 from AdoptOpenJDK) I get the same results as
you.
I have isolated this two errors as the origin:
java.lang.NullPointerException
at
java.xml/com.sun.org.apache.xerces.internal.dom.ParentNode.nodeLi
stGetLength(ParentNode.java:697)
at
java.xml/com.sun.org.apache.xerces.internal.dom.ParentNode.getLen
gth(ParentNode.java:725)
at
org.geotools.xsd.impl.BindingPropertyExtractor.getSequenceElement
(BindingPropertyExtractor.java:397)
at
org.geotools.xsd.impl.BindingPropertyExtractor.isUnboundedSequenc
e(BindingPropertyExtractor.java:363)
at
org.geotools.xsd.impl.BindingPropertyExtractor.properties(Binding
PropertyExtractor.java:82)
at org.geotools.xsd.Encoder.encode(Encoder.java:990)
... and:
java.util.ConcurrentModificationException
at
java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:14
93)
at
java.base/java.util.HashMap$EntryIterator.next(HashMap.java:1526)
at
java.base/java.util.HashMap$EntryIterator.next(HashMap.java:1524)
at
org.geoserver.catalog.impl.ModificationProxyCloner.cloneMap(Modif
icationProxyCloner.java:274)
at
org.geoserver.catalog.impl.ModificationProxy.invoke(ModificationP
roxy.java:108)
at com.sun.proxy.$Proxy43.getGML(Unknown Source)
at
org.geoserver.wfs.xml.GML3OutputFormat.write(GML3OutputFormat.jav
a:219)
at
org.geoserver.wfs.WFSGetFeatureOutputFormat.write(WFSGetFeatureOu
tputFormat.java:198)
at org.geoserver.ows.Dispatcher.response(Dispatcher.java:1031)
Chatting whit Fernando looks like he made some interesting discoveries,
I will let him follow up on this thread :)
Best regards,
Nuno Oliveira
On Sat, 2019-10-12 at 17:45 +0200, Andrea Aime wrote:
> Hi,
> tried a few more things on the build server, with no success:
> Reduced build parallelism to 1
> Switched to Maven 3.6.2
> Removed all QA checks
> Skipping formatting checks
> I've also noticed that the normal build has the java.tmpdir set, so
> copied that setting over.
> Even with all of the above applied at the same time, the build still
> failed.
> I've also tried different positions for the local maven repo (the
> current build uses a separate one,
> maybe to make sure there is no conflict with java 8 builds) but that
> did not help either.
>
> No idea of what else to try....
>
> Cheers
> Andrea
>
>
> On Thu, Oct 10, 2019 at 6:47 PM Andrea Aime
> s.it> wrote:
> > Been building the master branch with OpenJDK 11 from AdoptOpenJDK
> > all day, tried to make
> > a few runs every now and then, found a image comparison failure
> > that I've fixed, with that,
> > no more failures, no file system issues... seems an Amazon fleet
> > specific mystery...
> >
> > Anyone that can reproduce these issues?
> >
> > Cheers
> > Andrea
> >
> >
> > On Thu, Oct 10, 2019 at 11:14 AM Andrea Aime
> > ions.it> wrote:
> > > Hi,
> > > Java 11 builds keep on being plagued by random failures, but
> > > lately, they are almost guaranteed.
> > > The starting point for the build fail is not always the same, it
> > > happens in different modules, but once
> > > there is an error, everything falls apart.
> > >
> > > See for example this build:
> > > https://build.geoserver.org/view/geotools/job/geotools-java11/489
> > > /consoleFull
> > > After the errors in mosaic, the log of the feature-pregeneralized
> > > module, while not reporting errors,
> > > state the following:
> > >
> > > [INFO] --- maven-resources-plugin:3.1.0:resources (default-
> > > resources) @ gt-feature-pregeneralized ---
> > > [INFO] Using 'UTF-8' encoding to copy filtered resources.
> > > [INFO] skip non existing resourceDirectory /tmp/jenkins-
> > > c559144f/workspace/java11/geotools/modules/plugin/feature-
> > > pregeneralized/src/main/resources
> > > [INFO]
> > > [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile)
> > > @ gt-feature-pregeneralized ---
> > > [INFO] No sources to compile
> > > [INFO]
> > > [INFO] --- maven-resources-plugin:3.1.0:testResources (default-
> > > testResources) @ gt-feature-pregeneralized ---
> > > [INFO] Using 'UTF-8' encoding to copy filtered resources.
> > > [INFO] skip non existing resourceDirectory /tmp/jenkins-
> > > c559144f/workspace/java11/geotools/modules/plugin/feature-
> > > pregeneralized/src/test/resources
> > >
> > > This is however not true, there definitely sources, and there are
> > > files there:
> > > https://github.com/geotools/geotools/tree/84d6dc79f544430bdb6bdec
> > > 7fa3a7fb89bb65229/modules/plugin/feature-
> > > pregeneralized/src/main/resources
> > > https://github.com/geotools/geotools/tree/84d6dc79f544430bdb6bdec
> > > 7fa3a7fb89bb65229/modules/plugin/feature-
> > > pregeneralized/src/test/resources
> > >
> > > So it looks like the file system suddenly gives up, or at least,
> > > Java is not able to see it any longer.
> > > I tried to see if there is a "too many open files" error
> > > somewhere, but could not find any.
> > >
> > > Looking at the errors, it seems to be affecting both surefire
> > > (which is running in its own separate JVMs) and the main
> > > maven JVM, which is running compiles and the like, indeed, right
> > > after the failures, it also attempts re-downloading dependencies
> > > that were already downloaded, like jackson jars, see for example
> > > this build console output, search for jackson-core 2.9.10 (there
> > > is also a 2.9.6 in the build, used by maven itself), you can see
> > > it being succesfully downloaded at the beginning, while
> > > building gt-main, and then after the first failure, it tries to
> > > re-download it again, as if the maven local repository got
> > > emptied somehow.
> > >
> > > The GeoTools failures to build are likely affecting also
> > > GeoServer, as a revert of a commit that is causing a concurrency
> > > failure in the WFS tests
> > > never got baked into a java 11 jar for the GeoServer java 11
> > > build to use.
> > >
> > > I'm not seeing a smoking gun, but maybe the above description may
> > > trigger a light in some of your reading this message.
> > > What would cause this, that does not happen with a Java 8 build.
> > >
> > > Btw, I've noticed that the Java 11 build is not the latest, it's
> > > a 11.0.3 while 11.0.4 is out already.. checked the changelog
> > > but don't see anything file related:
> > > https://www.oracle.com/technetwork/java/javase/2col/11-0-4-oracle
> > > -bugfixes-5480115.html
> > >
> > > Cheers
> > > Andrea
> > >
> > > ----
> > > GeoServer Professional Services from the experts! Visit http://go
> > > o.gl/it488V for more information.
> > > ==
> > >
> > > Ing. Andrea Aime
> > > @geowolf
> > > Technical Lead
> > >
> > > GeoSolutions S.A.S.
> > > Via di Montramito 3/A
> > > 55054 Massarosa (LU)
> > > phone: +39 0584 962313
> > > fax: +39 0584 1660272
> > > mob: +39 339 8844549
> > >
> > > http://www.geo-solutions.it
> > > http://twitter.com/geosolutions_it
> > >
> > >
> > > -------------------------------------------------------
> > >
> > > Con riferimento alla normativa sul trattamento dei dati personali
> > > (Reg. UE 2016/679 - Regolamento generale sulla protezione dei
> > > dati “GDPR”), si precisa che ogni circostanza inerente alla
> > > presente email (il suo contenuto, gli eventuali allegati, etc.) è
> > > un dato la cui conoscenza è riservata al/i solo/i destinatario/i
> > > indicati dallo scrivente. Se il messaggio Le è giunto per errore,
> > > è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le
> > > sarei comunque grato se potesse darmene notizia.
> > >
> > > This email is intended only for the person or entity to which it
> > > is addressed and may contain information that is privileged,
> > > confidential or otherwise protected from disclosure. We remind
> > > that - as provided by European Regulation 2016/679 “GDPR” -
> > > copying, dissemination or use of this e-mail or the information
> > > herein by anyone other than the intended recipient is prohibited.
> > > If you have received this email by mistake, please notify us
> > > immediately by telephone or e-mail.
> > >
> >
> > --
> > Regards,
> > Andrea Aime
> > ==
> > GeoServer Professional Services from the experts! Visit http://goo.
> > gl/it488V for more information.
> > ==
> >
> > Ing. Andrea Aime
> > @geowolf
> > Technical Lead
> >
> > GeoSolutions S.A.S.
> > Via di Montramito 3/A
> > 55054 Massarosa (LU)
> > phone: +39 0584 962313
> > fax: +39 0584 1660272
> > mob: +39 339 8844549
> >
> > http://www.geo-solutions.it
> > http://twitter.com/geosolutions_it
> >
> >
> > -------------------------------------------------------
> >
> > Con riferimento alla normativa sul trattamento dei dati personali
> > (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati
> > “GDPR”), si precisa che ogni circostanza inerente alla presente
> > email (il suo contenuto, gli eventuali allegati, etc.) è un dato la
> > cui conoscenza è riservata al/i solo/i destinatario/i indicati
> > dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o
> > a cancellarlo, ogni altra operazione è illecita. Le sarei comunque
> > grato se potesse darmene notizia.
> >
> > This email is intended only for the person or entity to which it is
> > addressed and may contain information that is privileged,
> > confidential or otherwise protected from disclosure. We remind that
> > - as provided by European Regulation 2016/679 “GDPR” - copying,
> > dissemination or use of this e-mail or the information herein by
> > anyone other than the intended recipient is prohibited. If you have
> > received this email by mistake, please notify us immediately by
> > telephone or e-mail.
> >
>
> --
> Regards,
> Andrea Aime
> ==
> GeoServer Professional Services from the experts! Visit http://goo.gl
> /it488V for more information.
> ==
>
> Ing. Andrea Aime
> @geowolf
> Technical Lead
>
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054 Massarosa (LU)
> phone: +39 0584 962313
> fax: +39 0584 1660272
> mob: +39 339 8844549
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
>
> -------------------------------------------------------
>
> Con riferimento alla normativa sul trattamento dei dati personali
> (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati
> “GDPR”), si precisa che ogni circostanza inerente alla presente email
> (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui
> conoscenza è riservata al/i solo/i destinatario/i indicati dallo
> scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a
> cancellarlo, ogni altra operazione è illecita. Le sarei comunque
> grato se potesse darmene notizia.
>
> This email is intended only for the person or entity to which it is
> addressed and may contain information that is privileged,
> confidential or otherwise protected from disclosure. We remind that -
> as provided by European Regulation 2016/679 “GDPR” - copying,
> dissemination or use of this e-mail or the information herein by
> anyone other than the intended recipient is prohibited. If you have
> received this email by mistake, please notify us immediately by
> telephone or e-mail.
> _______________________________________________
> Geoserver-devel mailing list
> geoserver-de...@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geoserver-devel
--
Regards,
Nuno Oliveira
==
GeoServer Professional Services from the
experts!
Visit http://goo.gl/it488V for more information.
==
Nuno Miguel Carvalho Oliveira
@nmcoliveira
Software Engineer
GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
Con riferimento alla normativa sul trattamento dei dati
personali (Reg. UE 2016/679 - Regolamento generale sulla
protezione dei dati “GDPR”), si precisa che ogni
circostanza inerente alla presente email (il suo contenuto,
gli eventuali allegati, etc.) è un dato la cui conoscenza
è riservata al/i solo/i destinatario/i indicati dallo
scrivente. Se il messaggio Le è giunto per errore, è
tenuta/o a cancellarlo, ogni altra operazione è illecita.
Le sarei comunque grato se potesse darmene notizia.
This email is intended only for the person or entity to
which it is addressed and may contain information that
is privileged, confidential or otherwise protected from
disclosure. We remind that - as provided by European
Regulation 2016/679 “GDPR” - copying, dissemination or
use of this e-mail or the information herein by anyone
other than the intended recipient is prohibited. If you
have received this email by mistake, please notify
us immediately by telephone or e-mail.
_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
