Re: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
> On Oct 18, 2016, at 8:34 PM, Uwe Schindler wrote:
>
> ...
> Nevertheless, the "original" issue with the symlinked home directory should
> be solved separately. I made a proposal to Max (Weijun Wang) how to fix this
> while reading the policy file. We fixed the problem locally by fixing the
> Jenkins User account running the tests to not have a symlinked user.home dir
> anymore.
I still feel hesitated to grant an extra permission for all FilePermission in a
policy file, because that might not be what the user always wanted.
How about adding a modifier to the line, something like
permission java.io.FilePermission "${user.home}${/}.ivy2${/}cache${/}-",
"read", canonicalized;
which means when the permission is created its name should be canonicalized.
With this modifier, if the canonicalized name is different, it will not permit
access using the symlink.
The format is backward compatible with jdk8 because the modifier will be
ignored.
Thanks
Max
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
On 18/10/2016 13:34, Uwe Schindler wrote: Hi, Thanks Alan for the clarification and the documentation fixes! This does not solve the "returns null if not found or otherwise denied" issue, but this is a problem from the early Java days, which cannot be fixed anymore (to throw useful Exception if resource not found). We can't change long standing behavior, also there are can be security concerns with revealing whether a resource exists. To fix our own code (we also have some getResource[AsStream]() calls in Lucene's codebase without doPrivileged), I opened the following issue: https://issues.apache.org/jira/browse/LUCENE-7502 We cannot fix this in 3rd party JARs, so we cannot remove our extra permission. Nevertheless, the "original" issue with the symlinked home directory should be solved separately. I made a proposal to Max (Weijun Wang) how to fix this while reading the policy file. We fixed the problem locally by fixing the Jenkins User account running the tests to not have a symlinked user.home dir anymore. I will leave this to Max to answer. -Alan - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
Hi, Thanks Alan for the clarification and the documentation fixes! This does not solve the "returns null if not found or otherwise denied" issue, but this is a problem from the early Java days, which cannot be fixed anymore (to throw useful Exception if resource not found). To fix our own code (we also have some getResource[AsStream]() calls in Lucene's codebase without doPrivileged), I opened the following issue: https://issues.apache.org/jira/browse/LUCENE-7502 We cannot fix this in 3rd party JARs, so we cannot remove our extra permission. Nevertheless, the "original" issue with the symlinked home directory should be solved separately. I made a proposal to Max (Weijun Wang) how to fix this while reading the policy file. We fixed the problem locally by fixing the Jenkins User account running the tests to not have a symlinked user.home dir anymore. Uwe - Uwe Schindler uschind...@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ > -Original Message- > From: Alan Bateman [mailto:alan.bate...@oracle.com] > Sent: Tuesday, October 18, 2016 12:40 PM > To: Uwe Schindler ; dev@lucene.apache.org; > 'Wang Weijun' > Cc: 'Dalibor Topic' ; 'Balchandra Vaidya' > ; 'Muneer Kolarkunnu' > ; 'Rory O'Donnell' > ; Mandy Chung > Subject: Re: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - > Build # 18064 - Unstable! > > > > On 18/10/2016 10:25, Uwe Schindler wrote: > > : > > From my perspective this looks wrong, because there is no security > implications documented on Class#getResource, so it is completely unclear > that you actually need a doPrivileged when calling > Class#getResource[AsStream](). This is a separate issue and has nothing to > do with your changes. It was and still is broken, IMHO. > > > Just to say that Class#getResources has always done a permission check > but the spec didn't properly document this. We have improved the spec in > the jigsaw/jake forest (as part of the effort to specify how resources > in modules are located) and this includes specifying the longstanding > permission check [1]. I can't say yet when this will be in JSR because > the changes in this area are tied to a number of issues under discussion > in JSR 376. > > -Alan > > [1] > http://download.java.net/java/jigsaw/docs/api/java/lang/Class.html#getRes > ource-java.lang.String- > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
On 18/10/2016 10:25, Uwe Schindler wrote: : From my perspective this looks wrong, because there is no security implications documented on Class#getResource, so it is completely unclear that you actually need a doPrivileged when calling Class#getResource[AsStream](). This is a separate issue and has nothing to do with your changes. It was and still is broken, IMHO. Just to say that Class#getResources has always done a permission check but the spec didn't properly document this. We have improved the spec in the jigsaw/jake forest (as part of the effort to specify how resources in modules are located) and this includes specifying the longstanding permission check [1]. I can't say yet when this will be in JSR because the changes in this area are tied to a number of issues under discussion in JSR 376. -Alan [1] http://download.java.net/java/jigsaw/docs/api/java/lang/Class.html#getResource-java.lang.String- - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
gt; >> permissions
> on
> >> both /a and /x, you will still see a failure if the code access /b.
> >>>
> >>> --Max
> >>>
> >>>> On Oct 17, 2016, at 11:03 PM, Uwe Schindler
> >> wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>> I attached the debugging logs ("all") for the working (conaonicalized)
> and
> >> non-working case to this mail. Please keep in mind that around the
> >> getResource() is no doPrivileged, so it looks like it uses privileges of
> >> outer
> JAR.
> >>>>
> >>>> I also checked, we have the following explicitely in our policy file,
> >>>> which
> >> explicitly allows everything in the IVY cache (although it should not be
> >> needed) - so it should really work anyways!!!
> >>>>
> >>>> permission java.io.FilePermission "${user.home}${/}.ivy2${/}cache${/}-
> ",
> >> "read";
> >>>>
> >>>> But I think I know the issue - which is really stupid - but it’s a fact
> affecting
> >> mny people! Here is what the policy file parsing says above the
> >> permission given before (please compare first line and second line):
> >>>>
> >>>> [junit4] 2>Active Principals: []
> >>>> [junit4] 2> policy: granting ("java.io.FilePermission"
> >> "/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/-" "read")
> >>>> [junit4] 2> policy: granting ("java.io.FilePermission"
> >> "/var/lib/jenkins/.ivy2/cache/-" "read")
> >>>> [junit4] 2> policy: granting ("java.io.FilePermission"
> >> "/home/jenkins/tools/java/64bit/jdk-9-ea+140/-" "read,execute")
> >>>>
> >>>> Unfortunately, the user changed its home directory, the old one
> >> (var/lib/jenkins) symlinked to the new one, but /etc/passwd was not
> >> updated. So ${user.home} looks different. This is the reason why it did not
> >> happen locally on my windows machine!
> >>>>
> >>>> IMHO: When reading the policy file, it should canonicalize all paths,
> >> especially stuff like ${user.home}. At runtime when actually checking the
> >> permissions it should not canonicalize for performance reasons. Not sure
> if
> >> this is a bug, but this could hit lots of people. I know that other Jenkins
> >> servers where Lucene runs their tests (not yet on Java 9) are setup in
> similar
> >> ways!
> >>>>
> >>>> Uwe
> >>>>
> >>>> -
> >>>> Uwe Schindler
> >>>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>>> http://www.thetaphi.de
> >>>> eMail: u...@thetaphi.de
> >>>>
> >>>>> -Original Message-
> >>>>> From: Wang Weijun [mailto:weijun.w...@oracle.com]
> >>>>> Sent: Monday, October 17, 2016 4:03 PM
> >>>>> To: dev@lucene.apache.org
> >>>>> Cc: Dalibor Topic ; Balchandra Vaidya
> >>>>> ; Muneer Kolarkunnu
> >>>>> ; Rory O'Donnell
> >>>>>
> >>>>> Subject: Re: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-
> ea+140)
> >> -
> >>>>> Build # 18064 - Unstable!
> >>>>>
> >>>>> OK, I'll try it tomorrow.
> >>>>>
> >>>>> At the same time can you try -Djava.security.debug=all or -
> >>>>> Djava.security.debug=scl and see if it shows what permission is
> granted?
> >>>>>
> >>>>> Thanks
> >>>>> Max
> >>>>>
> >>>>>> On Oct 17, 2016, at 9:07 PM, Uwe Schindler
>
> >>>>> wrote:
> >>>>>>
> >>>>>> FYI,
> >>>>>>
> >>>>>> With:
> >>>>>> $ ant test -Dtests.useSecurityManager=true -Dargs='-
> >>>>> Djava.security.debug=access -
> >> Djdk.io.permissionsUseCanonicalPath=true'
> >>>>>>
> >>>>>> I can see the following line:
> >>>>>>
> >>>>>> [junit4] 2> access: access allowed ("java.io.FilePermission"
> >>>>> "/home/jenkins/.ivy2/cache/org.carrot2/morfologik-
> >>>>> polish/bundles/morfologik-po
Re: Class#getResource returns null in JDK9 b140 if security manager is enabled (was: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!)
arsing the policy file,
> create 2 file permissions: One as given in the policy and an additional one
> with the canonical name. This does not solve all problems, but helps around
> issues like the one we encountered.
>
> I changed the setup of the Jenkins machine that hit this issue first to not
> have a symlinked entry in /etc/passwd - instead I placed the real path there
> - so ${user.home} is right. I will see if the issues are gone. Nevertheless I
> have just brought this into your attention, so we can figure out what could
> get wrong on people's systems after this change. I will also figure out with
> my colleagues how to solve the permission checks in 3rd party jars -
> especially as they did not do anything wrong - why should one wrap
> Class#getResource() with doPrivileged?!
> --snip--
>
> Maybe we can discuss the ideas on the public mailing list. Was quite hard to
> figure out (with lots of debugging output) until I discovered the problem.
>
> Uwe
>
> -
> Uwe Schindler
> uschind...@apache.org
> ASF Member, Apache Lucene PMC / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
>> -Original Message-
>> From: Sean Mullan [mailto:sean.mul...@oracle.com]
>> Sent: Monday, October 17, 2016 7:33 PM
>> To: Uwe Schindler ; dev@lucene.apache.org
>> Cc: 'jdk9-dev' ; 'Dawid Weiss'
>> ; balchandra.vai...@oracle.com
>> Subject: Re: Class#getResource returns null in JDK9 b140 if security manager
>> is enabled (was: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-
>> ea+140) - Build # 18064 - Unstable!)
>>
>> Weijun Wang is the best person to respond as he is the RE of JDK-8164705
>> - right now it is the middle of the night for him, but I would expect a
>> response from him once he comes online.
>>
>> --Sean
>>
>> On 10/16/2016 06:09 PM, Uwe Schindler wrote:
>>> Hi again,
>>>
>>> with jdk.io.permissionsUseCanonicalPath=true it also works, so it is related
>> to the new FilePermission code, so my first guess was true, the issue is JDK-
>> 8164705.
>>>
>>> Uwe
>>>
>>>> (I cc'ed jdk-dev@openjdk, reader there please read the previous mails
>>>> below, too).
>>>>
>>>> I analyzed the problem, although I don't know exactly why it happens:
>>>> - On Windows it does not happen on my machine (no idea why!)
>>>> - On Linux it happens when tests are running with security manager (this
>> is
>>>> the default for Lucene and Jenkins does this)
>>>> - On Linux it does not happen if I run Lucene tests with "-
>>>> Dtests.useSecurityManager=false"
>>>>
>>>> This makes me think it is related to this: "Remove pathname
>> canonicalization
>>>> from FilePermission" (https://bugs.openjdk.java.net/browse/JDK-
>> 8164705)
>>>>
>>>> What seems to happen: The code calls Class.getResource to get back an
>> URL.
>>>> As the JAR file is somehow outside of the FilePermissions given to the test
>>>> suite, it seems to fail. Maybe because some of the checks failed,
>>>> Class.getResource then returns a null reference, because it was not able
>> to
>>>> access the JAR file.
>>>>
>>>> Were there some changes related to this: URLClassLoader and
>> FilePermission
>>>> checks?
>>>>
>>>> How should we proceed?
>>>>
>>>> Uwe
>>>>
>>>> -
>>>> Uwe Schindler
>>>> uschind...@apache.org
>>>> ASF Member, Apache Lucene PMC / Committer
>>>> Bremen, Germany
>>>> http://lucene.apache.org/
>>>>
>>>>> -Original Message-
>>>>> From: Uwe Schindler [mailto:u...@thetaphi.de]
>>>>> Sent: Sunday, October 16, 2016 10:10 PM
>>>>> To: dev@lucene.apache.org
>>>>> Cc: dalibor.to...@oracle.com; balchandra.vai...@oracle.com; 'Muneer
>>>>> Kolarkunnu' ; 'Dawid Weiss'
>>>>>
>>>>> Subject: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140)
>> -
>>>>> Build # 18064 - Unstable!
>>>>>
>>>>> Hi,
>>>>>
>>>>> I reverted the Lucene builds to build Java 9 138 for now. I will later
>>>>> check
>> if
>>>>> this also happens with build 139, which I have to download first. I will
>> also
>>>>> debug l
Re: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
> On Oct 17, 2016, at 11:51 PM, Uwe Schindler wrote:
>
> Hi,
>
>> One more thing:
>>
>> You said "(although it should not be needed). Can you confirm that? I mean,
>> if you remove that line, does the test still passes with -
>> Djdk.io.permissionsUseCanonicalPath=true? If so, we will need to look into
>> other reasons, since I did see in many other places file permission on
>> /home/jenkins//morfologik-polish-2.1.0.jar itself is granted.
>
> See my other mail! It should not be needed, because Class#getResource does
> not throw or document any Security implications. But in fact if you don't
> wrap a doPrivileged around the call, it will also fail without our extra
> permission. It was added to work around this issue with 3rd party JARs.
You mean this was observed before the new FilePermission change in 8164705?
If so, this means before the code change, you need FilePermissions on both
/var/lib/.../- and /home/jenkins/.../thejar to make sure the jar is accessible.
Here it seems the 2nd one should be enough but actually not, and the 1st is
added to guarantee it. And after the code change, the 1st is useless and the
test starts to fail.
I would consider adding a compatibility layer to make the 1st also useful, but
at the same time, if we can fix the 2nd, that will be very helpful.
Thanks
Max
>
> I just blew up with the symlink on home dir.
>
> Uwe
>
>> Thanks
>> Max
>>
>>
>>> On Oct 17, 2016, at 11:17 PM, Wang Weijun
>> wrote:
>>>
>>> Yes, this is where the problem is.
>>>
>>> So it looks like the permission is granted in a policy file instead of being
>> granted by the class loader itself. In this case, the path of the permission
>> must match how you access that file.
>>>
>>> I'll think about your suggestion. However, can you guarantee the code
>> always accesses the file using the canonicalized path? For example, suppose
>> the actual file is /x, both /a and /b symlink to it, and you grant the
>> permission
>> on /a in a policy file. Even if I canonicalize /a to /x and grant
>> permissions on
>> both /a and /x, you will still see a failure if the code access /b.
>>>
>>> --Max
>>>
>>>> On Oct 17, 2016, at 11:03 PM, Uwe Schindler
>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I attached the debugging logs ("all") for the working (conaonicalized) and
>> non-working case to this mail. Please keep in mind that around the
>> getResource() is no doPrivileged, so it looks like it uses privileges of
>> outer JAR.
>>>>
>>>> I also checked, we have the following explicitely in our policy file, which
>> explicitly allows everything in the IVY cache (although it should not be
>> needed) - so it should really work anyways!!!
>>>>
>>>> permission java.io.FilePermission "${user.home}${/}.ivy2${/}cache${/}-",
>> "read";
>>>>
>>>> But I think I know the issue - which is really stupid - but it’s a fact
>>>> affecting
>> mny people! Here is what the policy file parsing says above the
>> permission given before (please compare first line and second line):
>>>>
>>>> [junit4] 2> Active Principals: []
>>>> [junit4] 2> policy: granting ("java.io.FilePermission"
>> "/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/-" "read")
>>>> [junit4] 2> policy: granting ("java.io.FilePermission"
>> "/var/lib/jenkins/.ivy2/cache/-" "read")
>>>> [junit4] 2> policy: granting ("java.io.FilePermission"
>> "/home/jenkins/tools/java/64bit/jdk-9-ea+140/-" "read,execute")
>>>>
>>>> Unfortunately, the user changed its home directory, the old one
>> (var/lib/jenkins) symlinked to the new one, but /etc/passwd was not
>> updated. So ${user.home} looks different. This is the reason why it did not
>> happen locally on my windows machine!
>>>>
>>>> IMHO: When reading the policy file, it should canonicalize all paths,
>> especially stuff like ${user.home}. At runtime when actually checking the
>> permissions it should not canonicalize for performance reasons. Not sure if
>> this is a bug, but this could hit lots of people. I know that other Jenkins
>> servers where Lucene runs their tests (not yet on Java 9) are setup in
>> similar
>> ways!
>>>>
>>>> Uwe
>>>>
>>>&
Re: Class#getResource returns null in JDK9 b140 if security manager is enabled (was: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!)
Sorry forgot to hit reply-all... On 10/17/2016 01:33 PM, Sean Mullan wrote: Weijun Wang is the best person to respond as he is the RE of JDK-8164705 - right now it is the middle of the night for him, but I would expect a response from him once he comes online. --Sean On 10/16/2016 06:09 PM, Uwe Schindler wrote: Hi again, with jdk.io.permissionsUseCanonicalPath=true it also works, so it is related to the new FilePermission code, so my first guess was true, the issue is JDK-8164705. Uwe (I cc'ed jdk-dev@openjdk, reader there please read the previous mails below, too). I analyzed the problem, although I don't know exactly why it happens: - On Windows it does not happen on my machine (no idea why!) - On Linux it happens when tests are running with security manager (this is the default for Lucene and Jenkins does this) - On Linux it does not happen if I run Lucene tests with "- Dtests.useSecurityManager=false" This makes me think it is related to this: "Remove pathname canonicalization from FilePermission" (https://bugs.openjdk.java.net/browse/JDK-8164705) What seems to happen: The code calls Class.getResource to get back an URL. As the JAR file is somehow outside of the FilePermissions given to the test suite, it seems to fail. Maybe because some of the checks failed, Class.getResource then returns a null reference, because it was not able to access the JAR file. Were there some changes related to this: URLClassLoader and FilePermission checks? How should we proceed? Uwe - Uwe Schindler uschind...@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Sunday, October 16, 2016 10:10 PM To: dev@lucene.apache.org Cc: dalibor.to...@oracle.com; balchandra.vai...@oracle.com; 'Muneer Kolarkunnu' ; 'Dawid Weiss' Subject: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable! Hi, I reverted the Lucene builds to build Java 9 138 for now. I will later check if this also happens with build 139, which I have to download first. I will also debug locally. The code fails because this code hits "null" on getResource() at morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:34) https://github.com/morfologik/morfologik- stemming/blob/master/morfologik- polish/src/main/java/morfologik/stemming/polish/PolishStemmer.java#L32 This is impossible to happen, because the dict file is in same package. I have no idea why this only fails here and not at other places in Lucene. The main difference looks like the use of URL instead of getResourceAsStream() like other places in Lucene. So this seems to be a major regression in Java 9 build 140. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Sunday, October 16, 2016 8:38 PM To: dev@lucene.apache.org Cc: dalibor.to...@oracle.com; balchandra.vai...@oracle.com; 'Muneer Kolarkunnu' ; 'Dawid Weiss' ; dev@lucene.apache.org Subject: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable! Hi, this seems to be a new regression in Java 9 ea build 140. Interestingly this only affects 2 libraries (morphologic and commons-codec phonetic). We use loading of resources from classloaders at many places; it is unclear to me, why it only fails here. I will look into the code, but this is outside of Lucene. I think it might be some crazyness like using context class loader in non- proper ways or similar. Maybe it is a new bug in JDK 9 build 139 or build 140 (the last working one was build 138). Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Policeman Jenkins Server [mailto:jenk...@thetaphi.de] Sent: Sunday, October 16, 2016 8:20 PM To: dev@lucene.apache.org Subject: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable! Importance: Low Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18064/ Java: 32bit/jdk-9-ea+140 -server -XX:+UseParallelGC 24 tests failed. FAILED: org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testSingleTok ens Error Message: Could not read dictionary data. Stack Trace: java.lang.RuntimeException: Could not read dictionary data. at __randomizedtesting.SeedInfo.seed([27A360EA9CD9A4E8:A1C1C9729AE78F9 A]:0) at morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:38) at org.apache.lucene.analysis.morfologik.MorfologikAnalyzer.(Morfologik Analyzer.java:52) at org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.getTestAnaly zer(TestMorfologikAnalyzer.java:39) at org.apache.lucene.analysis.morfologik.T
Re: Class#getResource returns null in JDK9 b140 if security manager is enabled (was: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!)
Weijun Wang is the best person to respond as he is the RE of JDK-8164705 - right now it is the middle of the night for him, but I would expect a response from him once he comes online. --Sean On 10/16/2016 06:09 PM, Uwe Schindler wrote: Hi again, with jdk.io.permissionsUseCanonicalPath=true it also works, so it is related to the new FilePermission code, so my first guess was true, the issue is JDK-8164705. Uwe (I cc'ed jdk-dev@openjdk, reader there please read the previous mails below, too). I analyzed the problem, although I don't know exactly why it happens: - On Windows it does not happen on my machine (no idea why!) - On Linux it happens when tests are running with security manager (this is the default for Lucene and Jenkins does this) - On Linux it does not happen if I run Lucene tests with "- Dtests.useSecurityManager=false" This makes me think it is related to this: "Remove pathname canonicalization from FilePermission" (https://bugs.openjdk.java.net/browse/JDK-8164705) What seems to happen: The code calls Class.getResource to get back an URL. As the JAR file is somehow outside of the FilePermissions given to the test suite, it seems to fail. Maybe because some of the checks failed, Class.getResource then returns a null reference, because it was not able to access the JAR file. Were there some changes related to this: URLClassLoader and FilePermission checks? How should we proceed? Uwe - Uwe Schindler uschind...@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Sunday, October 16, 2016 10:10 PM To: dev@lucene.apache.org Cc: dalibor.to...@oracle.com; balchandra.vai...@oracle.com; 'Muneer Kolarkunnu' ; 'Dawid Weiss' Subject: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable! Hi, I reverted the Lucene builds to build Java 9 138 for now. I will later check if this also happens with build 139, which I have to download first. I will also debug locally. The code fails because this code hits "null" on getResource() at morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:34) https://github.com/morfologik/morfologik- stemming/blob/master/morfologik- polish/src/main/java/morfologik/stemming/polish/PolishStemmer.java#L32 This is impossible to happen, because the dict file is in same package. I have no idea why this only fails here and not at other places in Lucene. The main difference looks like the use of URL instead of getResourceAsStream() like other places in Lucene. So this seems to be a major regression in Java 9 build 140. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Sunday, October 16, 2016 8:38 PM To: dev@lucene.apache.org Cc: dalibor.to...@oracle.com; balchandra.vai...@oracle.com; 'Muneer Kolarkunnu' ; 'Dawid Weiss' ; dev@lucene.apache.org Subject: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable! Hi, this seems to be a new regression in Java 9 ea build 140. Interestingly this only affects 2 libraries (morphologic and commons-codec phonetic). We use loading of resources from classloaders at many places; it is unclear to me, why it only fails here. I will look into the code, but this is outside of Lucene. I think it might be some crazyness like using context class loader in non- proper ways or similar. Maybe it is a new bug in JDK 9 build 139 or build 140 (the last working one was build 138). Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Policeman Jenkins Server [mailto:jenk...@thetaphi.de] Sent: Sunday, October 16, 2016 8:20 PM To: dev@lucene.apache.org Subject: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable! Importance: Low Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18064/ Java: 32bit/jdk-9-ea+140 -server -XX:+UseParallelGC 24 tests failed. FAILED: org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testSingleTok ens Error Message: Could not read dictionary data. Stack Trace: java.lang.RuntimeException: Could not read dictionary data. at __randomizedtesting.SeedInfo.seed([27A360EA9CD9A4E8:A1C1C9729AE78F9 A]:0) at morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:38) at org.apache.lucene.analysis.morfologik.MorfologikAnalyzer.(Morfologik Analyzer.java:52) at org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.getTestAnaly zer(TestMorfologikAnalyzer.java:39) at org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testSingleTok ens(TestMorfologikAn
Re: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
On 17/10/2016 13:16, Uwe Schindler wrote:
Hi,
yes I checked more already: The issue is caused by the mentioned change
(canonicalize of FilePermission). According to the docs of SecurityManager and
FilePermission, code should always be able to read stuff below the classpath
where the code was loaded from (in our case its part of a JAR file). So there
is no need to add permissions for this, it should work automatically.
So the following code must work without any extra permissions:
URL url = this.getClass().getResource("somefilenexttoclassfile");
InputStream is = url.openStream();
Interestingly the first line already returns "null", means "resource not
found", you don't get any SecurityException! As said before the code works without any
problems if I pass the special JDK property jdk.io.permissionsUseCanonicalPath=true to the code.
This is why I said that JDK-8164705 is causing the issue.
I will write a short reproducer and post it here. The code should work with
SecurityManager enabled and empty policy file, as the resource is covered by
the rule (everything below code source).
The getResourceXXX methods are specified to return null when denied by
the security manager so you can't distinguish it from not found. If you
can get trace output with -Djava.security.debug=failure,access then it
might help diagnose this.
It's probably best to follow-up on security-dev rather than jdk9-dev as
that is the mailing list where permission classes are maintained.
-Alan
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
Hi,
> One more thing:
>
> You said "(although it should not be needed). Can you confirm that? I mean,
> if you remove that line, does the test still passes with -
> Djdk.io.permissionsUseCanonicalPath=true? If so, we will need to look into
> other reasons, since I did see in many other places file permission on
> /home/jenkins//morfologik-polish-2.1.0.jar itself is granted.
See my other mail! It should not be needed, because Class#getResource does not
throw or document any Security implications. But in fact if you don't wrap a
doPrivileged around the call, it will also fail without our extra permission.
It was added to work around this issue with 3rd party JARs.
I just blew up with the symlink on home dir.
Uwe
> Thanks
> Max
>
>
> > On Oct 17, 2016, at 11:17 PM, Wang Weijun
> wrote:
> >
> > Yes, this is where the problem is.
> >
> > So it looks like the permission is granted in a policy file instead of being
> granted by the class loader itself. In this case, the path of the permission
> must match how you access that file.
> >
> > I'll think about your suggestion. However, can you guarantee the code
> always accesses the file using the canonicalized path? For example, suppose
> the actual file is /x, both /a and /b symlink to it, and you grant the
> permission
> on /a in a policy file. Even if I canonicalize /a to /x and grant permissions
> on
> both /a and /x, you will still see a failure if the code access /b.
> >
> > --Max
> >
> >> On Oct 17, 2016, at 11:03 PM, Uwe Schindler
> wrote:
> >>
> >> Hi,
> >>
> >> I attached the debugging logs ("all") for the working (conaonicalized) and
> non-working case to this mail. Please keep in mind that around the
> getResource() is no doPrivileged, so it looks like it uses privileges of
> outer JAR.
> >>
> >> I also checked, we have the following explicitely in our policy file, which
> explicitly allows everything in the IVY cache (although it should not be
> needed) - so it should really work anyways!!!
> >>
> >> permission java.io.FilePermission "${user.home}${/}.ivy2${/}cache${/}-",
> "read";
> >>
> >> But I think I know the issue - which is really stupid - but it’s a fact
> >> affecting
> mny people! Here is what the policy file parsing says above the
> permission given before (please compare first line and second line):
> >>
> >> [junit4] 2> Active Principals: []
> >> [junit4] 2> policy: granting ("java.io.FilePermission"
> "/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/-" "read")
> >> [junit4] 2> policy: granting ("java.io.FilePermission"
> "/var/lib/jenkins/.ivy2/cache/-" "read")
> >> [junit4] 2> policy: granting ("java.io.FilePermission"
> "/home/jenkins/tools/java/64bit/jdk-9-ea+140/-" "read,execute")
> >>
> >> Unfortunately, the user changed its home directory, the old one
> (var/lib/jenkins) symlinked to the new one, but /etc/passwd was not
> updated. So ${user.home} looks different. This is the reason why it did not
> happen locally on my windows machine!
> >>
> >> IMHO: When reading the policy file, it should canonicalize all paths,
> especially stuff like ${user.home}. At runtime when actually checking the
> permissions it should not canonicalize for performance reasons. Not sure if
> this is a bug, but this could hit lots of people. I know that other Jenkins
> servers where Lucene runs their tests (not yet on Java 9) are setup in similar
> ways!
> >>
> >> Uwe
> >>
> >> -
> >> Uwe Schindler
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >> http://www.thetaphi.de
> >> eMail: u...@thetaphi.de
> >>
> >>> -Original Message-
> >>> From: Wang Weijun [mailto:weijun.w...@oracle.com]
> >>> Sent: Monday, October 17, 2016 4:03 PM
> >>> To: dev@lucene.apache.org
> >>> Cc: Dalibor Topic ; Balchandra Vaidya
> >>> ; Muneer Kolarkunnu
> >>> ; Rory O'Donnell
> >>>
> >>> Subject: Re: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140)
> -
> >>> Build # 18064 - Unstable!
> >>>
> >>> OK, I'll try it tomorrow.
> >>>
> >>> At the same time can you try -Djava.security.debug=all or -
> >>> Djava.security.debug=scl and see if it shows what permission is granted?
> >>>
> >>> Thanks
> >>&g
Re: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
One more thing:
You said "(although it should not be needed). Can you confirm that? I mean, if
you remove that line, does the test still passes with
-Djdk.io.permissionsUseCanonicalPath=true? If so, we will need to look into
other reasons, since I did see in many other places file permission on
/home/jenkins//morfologik-polish-2.1.0.jar itself is granted.
Thanks
Max
> On Oct 17, 2016, at 11:17 PM, Wang Weijun wrote:
>
> Yes, this is where the problem is.
>
> So it looks like the permission is granted in a policy file instead of being
> granted by the class loader itself. In this case, the path of the permission
> must match how you access that file.
>
> I'll think about your suggestion. However, can you guarantee the code always
> accesses the file using the canonicalized path? For example, suppose the
> actual file is /x, both /a and /b symlink to it, and you grant the permission
> on /a in a policy file. Even if I canonicalize /a to /x and grant permissions
> on both /a and /x, you will still see a failure if the code access /b.
>
> --Max
>
>> On Oct 17, 2016, at 11:03 PM, Uwe Schindler wrote:
>>
>> Hi,
>>
>> I attached the debugging logs ("all") for the working (conaonicalized) and
>> non-working case to this mail. Please keep in mind that around the
>> getResource() is no doPrivileged, so it looks like it uses privileges of
>> outer JAR.
>>
>> I also checked, we have the following explicitely in our policy file, which
>> explicitly allows everything in the IVY cache (although it should not be
>> needed) - so it should really work anyways!!!
>>
>> permission java.io.FilePermission "${user.home}${/}.ivy2${/}cache${/}-",
>> "read";
>>
>> But I think I know the issue - which is really stupid - but it’s a fact
>> affecting mny people! Here is what the policy file parsing says above
>> the permission given before (please compare first line and second line):
>>
>> [junit4] 2> Active Principals: []
>> [junit4] 2> policy: granting ("java.io.FilePermission"
>> "/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/-" "read")
>> [junit4] 2> policy: granting ("java.io.FilePermission"
>> "/var/lib/jenkins/.ivy2/cache/-" "read")
>> [junit4] 2> policy: granting ("java.io.FilePermission"
>> "/home/jenkins/tools/java/64bit/jdk-9-ea+140/-" "read,execute")
>>
>> Unfortunately, the user changed its home directory, the old one
>> (var/lib/jenkins) symlinked to the new one, but /etc/passwd was not updated.
>> So ${user.home} looks different. This is the reason why it did not happen
>> locally on my windows machine!
>>
>> IMHO: When reading the policy file, it should canonicalize all paths,
>> especially stuff like ${user.home}. At runtime when actually checking the
>> permissions it should not canonicalize for performance reasons. Not sure if
>> this is a bug, but this could hit lots of people. I know that other Jenkins
>> servers where Lucene runs their tests (not yet on Java 9) are setup in
>> similar ways!
>>
>> Uwe
>>
>> -
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: u...@thetaphi.de
>>
>>> -Original Message-
>>> From: Wang Weijun [mailto:weijun.w...@oracle.com]
>>> Sent: Monday, October 17, 2016 4:03 PM
>>> To: dev@lucene.apache.org
>>> Cc: Dalibor Topic ; Balchandra Vaidya
>>> ; Muneer Kolarkunnu
>>> ; Rory O'Donnell
>>>
>>> Subject: Re: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) -
>>> Build # 18064 - Unstable!
>>>
>>> OK, I'll try it tomorrow.
>>>
>>> At the same time can you try -Djava.security.debug=all or -
>>> Djava.security.debug=scl and see if it shows what permission is granted?
>>>
>>> Thanks
>>> Max
>>>
>>>> On Oct 17, 2016, at 9:07 PM, Uwe Schindler
>>> wrote:
>>>>
>>>> FYI,
>>>>
>>>> With:
>>>> $ ant test -Dtests.useSecurityManager=true -Dargs='-
>>> Djava.security.debug=access -Djdk.io.permissionsUseCanonicalPath=true'
>>>>
>>>> I can see the following line:
>>>>
>>>> [junit4] 2> access: access allowed ("java.io.FilePermission"
>>> "/home/jenkins/.ivy2/cache/org.carrot2/morfologik-
>>> polish/bundles/mo
Re: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
> On Oct 17, 2016, at 6:24 PM, Rory O'Donnell wrote: > > Adding Max to discussion. > > Rgds,Rory > > > On 17/10/2016 08:26, Dawid Weiss wrote: >> Yeah, this doesn't look so good. This class and the resource it tries >> to access are within the same JAR file -- if I understand correctly >> the JDK issue you quoted as the source of the problem, the code here >> should be able to pass regardless of file restrictions/ policies? Correct. A class should always be able to read a file that's on its classpath, no matter if it's inside a jar or a .class file. Can someone tell me how to reproduce this error? I have never played with Lucene before. Also, please try adding -Djava.security.debug=access and see if you can get some extra debug info. Thanks Max >> >> Dawid >> >> On Sun, Oct 16, 2016 at 10:09 PM, Uwe Schindler wrote: >>> Hi, >>> >>> I reverted the Lucene builds to build Java 9 138 for now. I will later >>> check if this also happens with build 139, which I have to download first. >>> I will also debug locally. >>> >>> The code fails because this code hits "null" on getResource() at >>> morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:34) >>> >>> https://github.com/morfologik/morfologik-stemming/blob/master/morfologik-polish/src/main/java/morfologik/stemming/polish/PolishStemmer.java#L32 >>> >>> This is impossible to happen, because the dict file is in same package. I >>> have no idea why this only fails here and not at other places in Lucene. >>> The main difference looks like the use of URL instead of >>> getResourceAsStream() like other places in Lucene. >>> >>> So this seems to be a major regression in Java 9 build 140. >>> >>> Uwe >>> >>> - >>> Uwe Schindler >>> H.-H.-Meier-Allee 63, D-28213 Bremen >>> http://www.thetaphi.de >>> eMail: u...@thetaphi.de >>> >>>> -----Original Message----- >>>> From: Uwe Schindler [mailto:u...@thetaphi.de] >>>> Sent: Sunday, October 16, 2016 8:38 PM >>>> To: dev@lucene.apache.org >>>> Cc: dalibor.to...@oracle.com; balchandra.vai...@oracle.com; 'Muneer >>>> Kolarkunnu' ; 'Dawid Weiss' >>>> ; dev@lucene.apache.org >>>> Subject: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - >>>> Build # 18064 - Unstable! >>>> >>>> Hi, >>>> >>>> this seems to be a new regression in Java 9 ea build 140. Interestingly >>>> this >>>> only affects 2 libraries (morphologic and commons-codec phonetic). We use >>>> loading of resources from classloaders at many places; it is unclear to me, >>>> why it only fails here. I will look into the code, but this is outside of >>>> Lucene. I >>>> think it might be some crazyness like using context class loader in >>>> non-proper >>>> ways or similar. >>>> >>>> Maybe it is a new bug in JDK 9 build 139 or build 140 (the last working one >>>> was build 138). >>>> >>>> Uwe >>>> >>>> - >>>> Uwe Schindler >>>> H.-H.-Meier-Allee 63, D-28213 Bremen >>>> http://www.thetaphi.de >>>> eMail: u...@thetaphi.de >>>> >>>>> -Original Message- >>>>> From: Policeman Jenkins Server [mailto:jenk...@thetaphi.de] >>>>> Sent: Sunday, October 16, 2016 8:20 PM >>>>> To: dev@lucene.apache.org >>>>> Subject: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - >>>>> Build >>>> # >>>>> 18064 - Unstable! >>>>> Importance: Low >>>>> >>>>> Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18064/ >>>>> Java: 32bit/jdk-9-ea+140 -server -XX:+UseParallelGC >>>>> >>>>> 24 tests failed. >>>>> FAILED: >>>>> >>>> org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testSingleTok >>>>> ens >>>>> >>>>> Error Message: >>>>> Could not read dictionary data. >>>>> >>>>> Stack Trace: >>>>> java.lang.RuntimeException: Could not read dictionary data. >>>>> at >>>>> >>>> __randomizedtesting.SeedInfo.seed([27A360EA9CD9A4E8:A1C1C9729AE78F9 >>>&
RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
Hi,
yes I checked more already: The issue is caused by the mentioned change
(canonicalize of FilePermission). According to the docs of SecurityManager and
FilePermission, code should always be able to read stuff below the classpath
where the code was loaded from (in our case its part of a JAR file). So there
is no need to add permissions for this, it should work automatically.
So the following code must work without any extra permissions:
URL url = this.getClass().getResource("somefilenexttoclassfile");
InputStream is = url.openStream();
Interestingly the first line already returns "null", means "resource not
found", you don't get any SecurityException! As said before the code works
without any problems if I pass the special JDK property
jdk.io.permissionsUseCanonicalPath=true to the code. This is why I said that
JDK-8164705 is causing the issue.
I will write a short reproducer and post it here. The code should work with
SecurityManager enabled and empty policy file, as the resource is covered by
the rule (everything below code source).
Uwe
-
Uwe Schindler
uschind...@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
> -Original Message-
> From: Dawid Weiss [mailto:dawid.we...@gmail.com]
> Sent: Monday, October 17, 2016 9:26 AM
> To: Uwe Schindler
> Cc: dev@lucene.apache.org; Dalibor Topic ;
> Balchandra Vaidya ; Muneer Kolarkunnu
>
> Subject: Re: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) -
> Build # 18064 - Unstable!
>
> Yeah, this doesn't look so good. This class and the resource it tries
> to access are within the same JAR file -- if I understand correctly
> the JDK issue you quoted as the source of the problem, the code here
> should be able to pass regardless of file restrictions/ policies?
>
> Dawid
>
> On Sun, Oct 16, 2016 at 10:09 PM, Uwe Schindler wrote:
> > Hi,
> >
> > I reverted the Lucene builds to build Java 9 138 for now. I will later
> > check if
> this also happens with build 139, which I have to download first. I will also
> debug locally.
> >
> > The code fails because this code hits "null" on getResource() at
> morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:34)
> >
> > https://github.com/morfologik/morfologik-
> stemming/blob/master/morfologik-
> polish/src/main/java/morfologik/stemming/polish/PolishStemmer.java#L32
> >
> > This is impossible to happen, because the dict file is in same package. I
> have no idea why this only fails here and not at other places in Lucene. The
> main difference looks like the use of URL instead of getResourceAsStream()
> like other places in Lucene.
> >
> > So this seems to be a major regression in Java 9 build 140.
> >
> > Uwe
> >
> > -
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: u...@thetaphi.de
> >
> >> -Original Message-
> >> From: Uwe Schindler [mailto:u...@thetaphi.de]
> >> Sent: Sunday, October 16, 2016 8:38 PM
> >> To: dev@lucene.apache.org
> >> Cc: dalibor.to...@oracle.com; balchandra.vai...@oracle.com; 'Muneer
> >> Kolarkunnu' ; 'Dawid Weiss'
> >> ; dev@lucene.apache.org
> >> Subject: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) -
> >> Build # 18064 - Unstable!
> >>
> >> Hi,
> >>
> >> this seems to be a new regression in Java 9 ea build 140. Interestingly
> >> this
> >> only affects 2 libraries (morphologic and commons-codec phonetic). We
> use
> >> loading of resources from classloaders at many places; it is unclear to me,
> >> why it only fails here. I will look into the code, but this is outside of
> Lucene. I
> >> think it might be some crazyness like using context class loader in non-
> proper
> >> ways or similar.
> >>
> >> Maybe it is a new bug in JDK 9 build 139 or build 140 (the last working one
> >> was build 138).
> >>
> >> Uwe
> >>
> >> -
> >> Uwe Schindler
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >> http://www.thetaphi.de
> >> eMail: u...@thetaphi.de
> >>
> >> > -Original Message-
> >> > From: Policeman Jenkins Server [mailto:jenk...@thetaphi.de]
> >> > Sent: Sunday, October 16, 2016 8:20 PM
> >> > To: dev@lucene.apache.org
> >> > Subject: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) -
> Build
> >> #
> >> > 18064 - Unstable!
> >> > Importance: Low
>
Re: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
Adding Max to discussion. Rgds,Rory On 17/10/2016 08:26, Dawid Weiss wrote: Yeah, this doesn't look so good. This class and the resource it tries to access are within the same JAR file -- if I understand correctly the JDK issue you quoted as the source of the problem, the code here should be able to pass regardless of file restrictions/ policies? Dawid On Sun, Oct 16, 2016 at 10:09 PM, Uwe Schindler wrote: Hi, I reverted the Lucene builds to build Java 9 138 for now. I will later check if this also happens with build 139, which I have to download first. I will also debug locally. The code fails because this code hits "null" on getResource() at morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:34) https://github.com/morfologik/morfologik-stemming/blob/master/morfologik-polish/src/main/java/morfologik/stemming/polish/PolishStemmer.java#L32 This is impossible to happen, because the dict file is in same package. I have no idea why this only fails here and not at other places in Lucene. The main difference looks like the use of URL instead of getResourceAsStream() like other places in Lucene. So this seems to be a major regression in Java 9 build 140. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Sunday, October 16, 2016 8:38 PM To: dev@lucene.apache.org Cc: dalibor.to...@oracle.com; balchandra.vai...@oracle.com; 'Muneer Kolarkunnu' ; 'Dawid Weiss' ; dev@lucene.apache.org Subject: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable! Hi, this seems to be a new regression in Java 9 ea build 140. Interestingly this only affects 2 libraries (morphologic and commons-codec phonetic). We use loading of resources from classloaders at many places; it is unclear to me, why it only fails here. I will look into the code, but this is outside of Lucene. I think it might be some crazyness like using context class loader in non-proper ways or similar. Maybe it is a new bug in JDK 9 build 139 or build 140 (the last working one was build 138). Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Policeman Jenkins Server [mailto:jenk...@thetaphi.de] Sent: Sunday, October 16, 2016 8:20 PM To: dev@lucene.apache.org Subject: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable! Importance: Low Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18064/ Java: 32bit/jdk-9-ea+140 -server -XX:+UseParallelGC 24 tests failed. FAILED: org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testSingleTok ens Error Message: Could not read dictionary data. Stack Trace: java.lang.RuntimeException: Could not read dictionary data. at __randomizedtesting.SeedInfo.seed([27A360EA9CD9A4E8:A1C1C9729AE78F9 A]:0) at morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:38) at org.apache.lucene.analysis.morfologik.MorfologikAnalyzer.(Morfologik Analyzer.java:52) at org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.getTestAnaly zer(TestMorfologikAnalyzer.java:39) at org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testSingleTok ens(TestMorfologikAnalyzer.java:44) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9- ea/Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9- ea/NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9- ea/DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(java.base@9- ea/Method.java:535) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(Randomize dRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Rando mizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(Rando mizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(Rand omizedRunner.java:921) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRule SetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAf terRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThr eadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleI gnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure. java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(State mentAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.r un(
Re: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
Yeah, this doesn't look so good. This class and the resource it tries to access are within the same JAR file -- if I understand correctly the JDK issue you quoted as the source of the problem, the code here should be able to pass regardless of file restrictions/ policies? Dawid On Sun, Oct 16, 2016 at 10:09 PM, Uwe Schindler wrote: > Hi, > > I reverted the Lucene builds to build Java 9 138 for now. I will later check > if this also happens with build 139, which I have to download first. I will > also debug locally. > > The code fails because this code hits "null" on getResource() at > morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:34) > > https://github.com/morfologik/morfologik-stemming/blob/master/morfologik-polish/src/main/java/morfologik/stemming/polish/PolishStemmer.java#L32 > > This is impossible to happen, because the dict file is in same package. I > have no idea why this only fails here and not at other places in Lucene. The > main difference looks like the use of URL instead of getResourceAsStream() > like other places in Lucene. > > So this seems to be a major regression in Java 9 build 140. > > Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > >> -Original Message- >> From: Uwe Schindler [mailto:u...@thetaphi.de] >> Sent: Sunday, October 16, 2016 8:38 PM >> To: dev@lucene.apache.org >> Cc: dalibor.to...@oracle.com; balchandra.vai...@oracle.com; 'Muneer >> Kolarkunnu' ; 'Dawid Weiss' >> ; dev@lucene.apache.org >> Subject: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - >> Build # 18064 - Unstable! >> >> Hi, >> >> this seems to be a new regression in Java 9 ea build 140. Interestingly this >> only affects 2 libraries (morphologic and commons-codec phonetic). We use >> loading of resources from classloaders at many places; it is unclear to me, >> why it only fails here. I will look into the code, but this is outside of >> Lucene. I >> think it might be some crazyness like using context class loader in >> non-proper >> ways or similar. >> >> Maybe it is a new bug in JDK 9 build 139 or build 140 (the last working one >> was build 138). >> >> Uwe >> >> - >> Uwe Schindler >> H.-H.-Meier-Allee 63, D-28213 Bremen >> http://www.thetaphi.de >> eMail: u...@thetaphi.de >> >> > -Original Message- >> > From: Policeman Jenkins Server [mailto:jenk...@thetaphi.de] >> > Sent: Sunday, October 16, 2016 8:20 PM >> > To: dev@lucene.apache.org >> > Subject: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build >> # >> > 18064 - Unstable! >> > Importance: Low >> > >> > Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18064/ >> > Java: 32bit/jdk-9-ea+140 -server -XX:+UseParallelGC >> > >> > 24 tests failed. >> > FAILED: >> > >> org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testSingleTok >> > ens >> > >> > Error Message: >> > Could not read dictionary data. >> > >> > Stack Trace: >> > java.lang.RuntimeException: Could not read dictionary data. >> > at >> > >> __randomizedtesting.SeedInfo.seed([27A360EA9CD9A4E8:A1C1C9729AE78F9 >> > A]:0) >> > at >> > morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:38) >> > at >> > >> org.apache.lucene.analysis.morfologik.MorfologikAnalyzer.(Morfologik >> > Analyzer.java:52) >> > at >> > >> org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.getTestAnaly >> > zer(TestMorfologikAnalyzer.java:39) >> > at >> > >> org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testSingleTok >> > ens(TestMorfologikAnalyzer.java:44) >> > at >> > jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9- >> > ea/Native Method) >> > at >> > jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9- >> > ea/NativeMethodAccessorImpl.java:62) >> > at >> > jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9- >> > ea/DelegatingMethodAccessorImpl.java:43) >> > at java.lang.reflect.Method.invoke(java.base@9- >> > ea/Method.java:535) >> > at >> > >> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(Randomize >> > dRunner.java:1764) >> > at
RE: Class#getResource returns null in JDK9 b140 if security manager is enabled (was: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!)
Hi again, with jdk.io.permissionsUseCanonicalPath=true it also works, so it is related to the new FilePermission code, so my first guess was true, the issue is JDK-8164705. Uwe > (I cc'ed jdk-dev@openjdk, reader there please read the previous mails > below, too). > > I analyzed the problem, although I don't know exactly why it happens: > - On Windows it does not happen on my machine (no idea why!) > - On Linux it happens when tests are running with security manager (this is > the default for Lucene and Jenkins does this) > - On Linux it does not happen if I run Lucene tests with "- > Dtests.useSecurityManager=false" > > This makes me think it is related to this: "Remove pathname canonicalization > from FilePermission" (https://bugs.openjdk.java.net/browse/JDK-8164705) > > What seems to happen: The code calls Class.getResource to get back an URL. > As the JAR file is somehow outside of the FilePermissions given to the test > suite, it seems to fail. Maybe because some of the checks failed, > Class.getResource then returns a null reference, because it was not able to > access the JAR file. > > Were there some changes related to this: URLClassLoader and FilePermission > checks? > > How should we proceed? > > Uwe > > - > Uwe Schindler > uschind...@apache.org > ASF Member, Apache Lucene PMC / Committer > Bremen, Germany > http://lucene.apache.org/ > > > -Original Message- > > From: Uwe Schindler [mailto:u...@thetaphi.de] > > Sent: Sunday, October 16, 2016 10:10 PM > > To: dev@lucene.apache.org > > Cc: dalibor.to...@oracle.com; balchandra.vai...@oracle.com; 'Muneer > > Kolarkunnu' ; 'Dawid Weiss' > > > > Subject: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - > > Build # 18064 - Unstable! > > > > Hi, > > > > I reverted the Lucene builds to build Java 9 138 for now. I will later > > check if > > this also happens with build 139, which I have to download first. I will > > also > > debug locally. > > > > The code fails because this code hits "null" on getResource() at > > morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:34) > > > > https://github.com/morfologik/morfologik- > > stemming/blob/master/morfologik- > > > polish/src/main/java/morfologik/stemming/polish/PolishStemmer.java#L32 > > > > This is impossible to happen, because the dict file is in same package. I > have > > no idea why this only fails here and not at other places in Lucene. The main > > difference looks like the use of URL instead of getResourceAsStream() like > > other places in Lucene. > > > > So this seems to be a major regression in Java 9 build 140. > > > > Uwe > > > > - > > Uwe Schindler > > H.-H.-Meier-Allee 63, D-28213 Bremen > > http://www.thetaphi.de > > eMail: u...@thetaphi.de > > > > > -Original Message- > > > From: Uwe Schindler [mailto:u...@thetaphi.de] > > > Sent: Sunday, October 16, 2016 8:38 PM > > > To: dev@lucene.apache.org > > > Cc: dalibor.to...@oracle.com; balchandra.vai...@oracle.com; 'Muneer > > > Kolarkunnu' ; 'Dawid Weiss' > > > ; dev@lucene.apache.org > > > Subject: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - > > > Build # 18064 - Unstable! > > > > > > Hi, > > > > > > this seems to be a new regression in Java 9 ea build 140. Interestingly > > > this > > > only affects 2 libraries (morphologic and commons-codec phonetic). We > > use > > > loading of resources from classloaders at many places; it is unclear to > > > me, > > > why it only fails here. I will look into the code, but this is outside of > Lucene. > > I > > > think it might be some crazyness like using context class loader in non- > > proper > > > ways or similar. > > > > > > Maybe it is a new bug in JDK 9 build 139 or build 140 (the last working > > > one > > > was build 138). > > > > > > Uwe > > > > > > - > > > Uwe Schindler > > > H.-H.-Meier-Allee 63, D-28213 Bremen > > > http://www.thetaphi.de > > > eMail: u...@thetaphi.de > > > > > > > -Original Message- > > > > From: Policeman Jenkins Server [mailto:jenk...@thetaphi.de] > > > > Sent: Sunday, October 16, 2016 8:20 PM > > > > To: dev@lucene.apache.org > > > > Subject: [JENKINS-EA] Lucene-Solr-master-
Class#getResource returns null in JDK9 b140 if security manager is enabled (was: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!)
Hi, (I cc'ed jdk-dev@openjdk, reader there please read the previous mails below, too). I analyzed the problem, although I don't know exactly why it happens: - On Windows it does not happen on my machine (no idea why!) - On Linux it happens when tests are running with security manager (this is the default for Lucene and Jenkins does this) - On Linux it does not happen if I run Lucene tests with "-Dtests.useSecurityManager=false" This makes me think it is related to this: "Remove pathname canonicalization from FilePermission" (https://bugs.openjdk.java.net/browse/JDK-8164705) What seems to happen: The code calls Class.getResource to get back an URL. As the JAR file is somehow outside of the FilePermissions given to the test suite, it seems to fail. Maybe because some of the checks failed, Class.getResource then returns a null reference, because it was not able to access the JAR file. Were there some changes related to this: URLClassLoader and FilePermission checks? How should we proceed? Uwe - Uwe Schindler uschind...@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ > -Original Message- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Sunday, October 16, 2016 10:10 PM > To: dev@lucene.apache.org > Cc: dalibor.to...@oracle.com; balchandra.vai...@oracle.com; 'Muneer > Kolarkunnu' ; 'Dawid Weiss' > > Subject: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - > Build # 18064 - Unstable! > > Hi, > > I reverted the Lucene builds to build Java 9 138 for now. I will later check > if > this also happens with build 139, which I have to download first. I will also > debug locally. > > The code fails because this code hits "null" on getResource() at > morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:34) > > https://github.com/morfologik/morfologik- > stemming/blob/master/morfologik- > polish/src/main/java/morfologik/stemming/polish/PolishStemmer.java#L32 > > This is impossible to happen, because the dict file is in same package. I have > no idea why this only fails here and not at other places in Lucene. The main > difference looks like the use of URL instead of getResourceAsStream() like > other places in Lucene. > > So this seems to be a major regression in Java 9 build 140. > > Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > -Original Message- > > From: Uwe Schindler [mailto:u...@thetaphi.de] > > Sent: Sunday, October 16, 2016 8:38 PM > > To: dev@lucene.apache.org > > Cc: dalibor.to...@oracle.com; balchandra.vai...@oracle.com; 'Muneer > > Kolarkunnu' ; 'Dawid Weiss' > > ; dev@lucene.apache.org > > Subject: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - > > Build # 18064 - Unstable! > > > > Hi, > > > > this seems to be a new regression in Java 9 ea build 140. Interestingly this > > only affects 2 libraries (morphologic and commons-codec phonetic). We > use > > loading of resources from classloaders at many places; it is unclear to me, > > why it only fails here. I will look into the code, but this is outside of > > Lucene. > I > > think it might be some crazyness like using context class loader in non- > proper > > ways or similar. > > > > Maybe it is a new bug in JDK 9 build 139 or build 140 (the last working one > > was build 138). > > > > Uwe > > > > ----- > > Uwe Schindler > > H.-H.-Meier-Allee 63, D-28213 Bremen > > http://www.thetaphi.de > > eMail: u...@thetaphi.de > > > > > -Original Message- > > > From: Policeman Jenkins Server [mailto:jenk...@thetaphi.de] > > > Sent: Sunday, October 16, 2016 8:20 PM > > > To: dev@lucene.apache.org > > > Subject: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - > Build > > # > > > 18064 - Unstable! > > > Importance: Low > > > > > > Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18064/ > > > Java: 32bit/jdk-9-ea+140 -server -XX:+UseParallelGC > > > > > > 24 tests failed. > > > FAILED: > > > > > > org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testSingleTok > > > ens > > > > > > Error Message: > > > Could not read dictionary data. > > > > > > Stack Trace: > > > java.lang.RuntimeException: Could not read dictionary data. > > > at > > > > > > __randomizedtesting.SeedInfo.seed([27A3
RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
Hi, I reverted the Lucene builds to build Java 9 138 for now. I will later check if this also happens with build 139, which I have to download first. I will also debug locally. The code fails because this code hits "null" on getResource() at morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:34) https://github.com/morfologik/morfologik-stemming/blob/master/morfologik-polish/src/main/java/morfologik/stemming/polish/PolishStemmer.java#L32 This is impossible to happen, because the dict file is in same package. I have no idea why this only fails here and not at other places in Lucene. The main difference looks like the use of URL instead of getResourceAsStream() like other places in Lucene. So this seems to be a major regression in Java 9 build 140. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Sunday, October 16, 2016 8:38 PM > To: dev@lucene.apache.org > Cc: dalibor.to...@oracle.com; balchandra.vai...@oracle.com; 'Muneer > Kolarkunnu' ; 'Dawid Weiss' > ; dev@lucene.apache.org > Subject: RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - > Build # 18064 - Unstable! > > Hi, > > this seems to be a new regression in Java 9 ea build 140. Interestingly this > only affects 2 libraries (morphologic and commons-codec phonetic). We use > loading of resources from classloaders at many places; it is unclear to me, > why it only fails here. I will look into the code, but this is outside of > Lucene. I > think it might be some crazyness like using context class loader in non-proper > ways or similar. > > Maybe it is a new bug in JDK 9 build 139 or build 140 (the last working one > was build 138). > > Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > -Original Message- > > From: Policeman Jenkins Server [mailto:jenk...@thetaphi.de] > > Sent: Sunday, October 16, 2016 8:20 PM > > To: dev@lucene.apache.org > > Subject: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build > # > > 18064 - Unstable! > > Importance: Low > > > > Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18064/ > > Java: 32bit/jdk-9-ea+140 -server -XX:+UseParallelGC > > > > 24 tests failed. > > FAILED: > > > org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testSingleTok > > ens > > > > Error Message: > > Could not read dictionary data. > > > > Stack Trace: > > java.lang.RuntimeException: Could not read dictionary data. > > at > > > __randomizedtesting.SeedInfo.seed([27A360EA9CD9A4E8:A1C1C9729AE78F9 > > A]:0) > > at > > morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:38) > > at > > > org.apache.lucene.analysis.morfologik.MorfologikAnalyzer.(Morfologik > > Analyzer.java:52) > > at > > > org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.getTestAnaly > > zer(TestMorfologikAnalyzer.java:39) > > at > > > org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testSingleTok > > ens(TestMorfologikAnalyzer.java:44) > > at > > jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9- > > ea/Native Method) > > at > > jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9- > > ea/NativeMethodAccessorImpl.java:62) > > at > > jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9- > > ea/DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(java.base@9- > > ea/Method.java:535) > > at > > > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(Randomize > > dRunner.java:1764) > > at > > > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Rando > > mizedRunner.java:871) > > at > > > com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(Rando > > mizedRunner.java:907) > > at > > > com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(Rand > > omizedRunner.java:921) > > at > > > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRule > > SetupTeardownChained.java:49) > > at > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAf > > terRule.java:45) > > at > > > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThr > > eadAndTestName.java:48) > > at > >
RE: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
Hi, this seems to be a new regression in Java 9 ea build 140. Interestingly this only affects 2 libraries (morphologic and commons-codec phonetic). We use loading of resources from classloaders at many places; it is unclear to me, why it only fails here. I will look into the code, but this is outside of Lucene. I think it might be some crazyness like using context class loader in non-proper ways or similar. Maybe it is a new bug in JDK 9 build 139 or build 140 (the last working one was build 138). Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Policeman Jenkins Server [mailto:jenk...@thetaphi.de] > Sent: Sunday, October 16, 2016 8:20 PM > To: dev@lucene.apache.org > Subject: [JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # > 18064 - Unstable! > Importance: Low > > Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18064/ > Java: 32bit/jdk-9-ea+140 -server -XX:+UseParallelGC > > 24 tests failed. > FAILED: > org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testSingleTok > ens > > Error Message: > Could not read dictionary data. > > Stack Trace: > java.lang.RuntimeException: Could not read dictionary data. > at > __randomizedtesting.SeedInfo.seed([27A360EA9CD9A4E8:A1C1C9729AE78F9 > A]:0) > at > morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:38) > at > org.apache.lucene.analysis.morfologik.MorfologikAnalyzer.(Morfologik > Analyzer.java:52) > at > org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.getTestAnaly > zer(TestMorfologikAnalyzer.java:39) > at > org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testSingleTok > ens(TestMorfologikAnalyzer.java:44) > at > jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9- > ea/Native Method) > at > jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9- > ea/NativeMethodAccessorImpl.java:62) > at > jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9- > ea/DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(java.base@9- > ea/Method.java:535) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(Randomize > dRunner.java:1764) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Rando > mizedRunner.java:871) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(Rando > mizedRunner.java:907) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(Rand > omizedRunner.java:921) > at > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRule > SetupTeardownChained.java:49) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAf > terRule.java:45) > at > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThr > eadAndTestName.java:48) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleI > gnoreAfterMaxFailures.java:64) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure. > java:47) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(State > mentAdapter.java:36) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.r > un(ThreadLeakControl.java:367) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask > (ThreadLeakControl.java:809) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadL > eakControl.java:460) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(Ran > domizedRunner.java:880) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Rando > mizedRunner.java:781) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(Rando > mizedRunner.java:816) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(Rando > mizedRunner.java:827) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAf > terRule.java:45) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(State > mentAdapter.java:36) > at > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreCla > ssName.java:41) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet > hodsRule$1.evaluate(NoShadowingO
[JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+140) - Build # 18064 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18064/ Java: 32bit/jdk-9-ea+140 -server -XX:+UseParallelGC 24 tests failed. FAILED: org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testSingleTokens Error Message: Could not read dictionary data. Stack Trace: java.lang.RuntimeException: Could not read dictionary data. at __randomizedtesting.SeedInfo.seed([27A360EA9CD9A4E8:A1C1C9729AE78F9A]:0) at morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:38) at org.apache.lucene.analysis.morfologik.MorfologikAnalyzer.(MorfologikAnalyzer.java:52) at org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.getTestAnalyzer(TestMorfologikAnalyzer.java:39) at org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testSingleTokens(TestMorfologikAnalyzer.java:44) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:535) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) Caused by: java.io.IOException: Polish dictionary resource not found. at morfologik.stemming.polish.PolishStemmer.(PolishStemmer.java:34) ... 39 more FAILED: org.apache.lucene.analysis.morfologik.TestMorfologikAnalyzer.testRandom Error Message: Could not read dictionary data. Stack Trace: java.lang.Runtime
