Re: Annoying but harmless exceptions due to filepermissions when running tests
I think the best option is to learn to ignore the harmless error message in
test logs……
This was one of those “if there were just a few, we can fix the test to set
itself up in a writable directory”. But Hoss is right, it’s pretty common to
lots and lots and lots of tests and it’s just a minor annoyance when trying to
read through the output of a test that fails, it has no functional significance.
So I closed the JIRA “invalid” and y’all should go back to your regularly
scheduled programming ;)
Sorry for the noise
Erick
> On Sep 2, 2020, at 9:54 PM, Alexandre Rafalovitch wrote:
>
> There is a flag to disable package manager. Can that code path avoid creating
> directory? Or maybe it does already.
>
> Then tests that don't test that specifically could have disable flag on.
>
> Regards,
>Alex
>
> On Wed., Sep. 2, 2020, 9:41 p.m. Noble Paul, wrote:
> The filestore dir is where the packages live. If we move it to another
> location, existing installations might fail. So, it's a backward incompatible
> change.
>
> What are our options?
>
> Is it possible to have these directories precreated in the distro/code to
> ensure that these warnings don't come.
>
> On Thu, Sep 3, 2020, 4:58 AM Erick Erickson wrote:
> Oh bother. Somehow I thought I’d looked and only a handful of tests reported
> this.
>
> So I looked again and I _wish_ I was able to blame drugs cause you’re right,
> there
> are over a thousand of them.
>
> Never mind...
>
> > On Sep 1, 2020, at 8:55 PM, Chris Hostetter
> > wrote:
> >
> >
> > : Hmmm, that’s kind of an dilemma then. Are you saying that
> > : he test can see that the directory appears writable then tries
> > : to write to it then gets tripped up by the framework?
> > :
> > : Seems to me that a test that tries to write, thinks it can and then
> > : can’t should fail anyway.
> > :
> > : Well, I don’t think there are very many tests that have this problem
> > : anyway, so maybe I can examine them one-by-one and not
> > : introduce new failures...
> >
> > You keep using the phrase "the test" in the context of (trying to) create
> > these directories ("userfiles" and "filestore") ... the "test CODE" isn't
> > making any choices about trying to write those files -- the choice is
> > being made by the "CoreContainer CODE".
> >
> > These features were added with the explicit implementation choice to _TRY_
> > to write the "usersfiles" (and/or "filestore") directory to "solr home" IF
> > POSSIBLE, and if so then enable a bunch of features -- if NOT then log a
> > WARNing and don't enable those features.
> >
> > So what you're seeing here isn't an artifact/result of any particular
> > choices "a test" or "the test" makes -- it's a concious choice of the
> > developer who added this feature to solr. These WARN messages that show
> > up in tests where the solr home dir isn't writable (which is actaully the
> > vast majority of tests because of how the test framework works) are the
> > same types of WARN messages that a "real" solr deployment might get if
> > their solr home dir isn't wriable (ie: maybe the use ${solr.data.dir} to
> > point to a diff drive).
> >
> >
> >
> > :
> > : > On Aug 31, 2020, at 1:29 PM, Chris Hostetter
> > wrote:
> > : >
> > : >
> > : > Some tests "create" a new solr home dir and copy config files there,
> > but
> > : > you'll see this type of WARN logging for any test that just uses the
> > test
> > : > configs "in place" because of how the code is designed to _try_ and
> > create
> > : > a userfiles directory in the solr home if it's writable.
> > : >
> > : >
> > : > : Date: Sat, 29 Aug 2020 09:25:17 -0400
> > : > : From: Erick Erickson
> > : > : Reply-To: dev@lucene.apache.org
> > : > : To: dev@lucene.apache.org
> > : > : Subject: Re: Annoying but harmless exceptions due to filepermissions
> > when
> > : > : running tests
> > : > :
> > : > : Well, as Uwe and I discussed offline, he’s right and I’m wrong.
> > : > :
> > : > : In CoreContainer [364] there’s code like this:
> > : > :
> > : > : Path userFilesPath = return solrHome.resolve("userfiles"); // TODO
> > make configurable on cfg?
> > : > : try {
> > : > : Files.createDirectories(userFilesPath); // does nothing if already
> > exists
> &
Re: Annoying but harmless exceptions due to filepermissions when running tests
There is a flag to disable package manager. Can that code path avoid
creating directory? Or maybe it does already.
Then tests that don't test that specifically could have disable flag on.
Regards,
Alex
On Wed., Sep. 2, 2020, 9:41 p.m. Noble Paul, wrote:
> The filestore dir is where the packages live. If we move it to another
> location, existing installations might fail. So, it's a backward
> incompatible change.
>
> What are our options?
>
> Is it possible to have these directories precreated in the distro/code to
> ensure that these warnings don't come.
>
> On Thu, Sep 3, 2020, 4:58 AM Erick Erickson
> wrote:
>
>> Oh bother. Somehow I thought I’d looked and only a handful of tests
>> reported this.
>>
>> So I looked again and I _wish_ I was able to blame drugs cause you’re
>> right, there
>> are over a thousand of them.
>>
>> Never mind...
>>
>> > On Sep 1, 2020, at 8:55 PM, Chris Hostetter
>> wrote:
>> >
>> >
>> > : Hmmm, that’s kind of an dilemma then. Are you saying that
>> > : he test can see that the directory appears writable then tries
>> > : to write to it then gets tripped up by the framework?
>> > :
>> > : Seems to me that a test that tries to write, thinks it can and then
>> > : can’t should fail anyway.
>> > :
>> > : Well, I don’t think there are very many tests that have this problem
>> > : anyway, so maybe I can examine them one-by-one and not
>> > : introduce new failures...
>> >
>> > You keep using the phrase "the test" in the context of (trying to)
>> create
>> > these directories ("userfiles" and "filestore") ... the "test CODE"
>> isn't
>> > making any choices about trying to write those files -- the choice is
>> > being made by the "CoreContainer CODE".
>> >
>> > These features were added with the explicit implementation choice to
>> _TRY_
>> > to write the "usersfiles" (and/or "filestore") directory to "solr home"
>> IF
>> > POSSIBLE, and if so then enable a bunch of features -- if NOT then log
>> a
>> > WARNing and don't enable those features.
>> >
>> > So what you're seeing here isn't an artifact/result of any particular
>> > choices "a test" or "the test" makes -- it's a concious choice of the
>> > developer who added this feature to solr. These WARN messages that
>> show
>> > up in tests where the solr home dir isn't writable (which is actaully
>> the
>> > vast majority of tests because of how the test framework works) are the
>> > same types of WARN messages that a "real" solr deployment might get if
>> > their solr home dir isn't wriable (ie: maybe the use ${solr.data.dir}
>> to
>> > point to a diff drive).
>> >
>> >
>> >
>> > :
>> > : > On Aug 31, 2020, at 1:29 PM, Chris Hostetter <
>> hossman_luc...@fucit.org> wrote:
>> > : >
>> > : >
>> > : > Some tests "create" a new solr home dir and copy config files
>> there, but
>> > : > you'll see this type of WARN logging for any test that just uses
>> the test
>> > : > configs "in place" because of how the code is designed to _try_ and
>> create
>> > : > a userfiles directory in the solr home if it's writable.
>> > : >
>> > : >
>> > : > : Date: Sat, 29 Aug 2020 09:25:17 -0400
>> > : > : From: Erick Erickson
>> > : > : Reply-To: dev@lucene.apache.org
>> > : > : To: dev@lucene.apache.org
>> > : > : Subject: Re: Annoying but harmless exceptions due to
>> filepermissions when
>> > : > : running tests
>> > : > :
>> > : > : Well, as Uwe and I discussed offline, he’s right and I’m wrong.
>> > : > :
>> > : > : In CoreContainer [364] there’s code like this:
>> > : > :
>> > : > : Path userFilesPath = return solrHome.resolve("userfiles"); //
>> TODO make configurable on cfg?
>> > : > : try {
>> > : > : Files.createDirectories(userFilesPath); // does nothing if
>> already exists
>> > : > : } catch (Exception e) {
>> > : > : log.warn("Unable to create [{}]. Features requiring this
>> directory may fail.", userFilesPath, e);
>> > : > : }
>> > : > :
>> > : > : So I assumed it would happen on most every test, at leas
Re: Annoying but harmless exceptions due to filepermissions when running tests
The filestore dir is where the packages live. If we move it to another
location, existing installations might fail. So, it's a backward
incompatible change.
What are our options?
Is it possible to have these directories precreated in the distro/code to
ensure that these warnings don't come.
On Thu, Sep 3, 2020, 4:58 AM Erick Erickson wrote:
> Oh bother. Somehow I thought I’d looked and only a handful of tests
> reported this.
>
> So I looked again and I _wish_ I was able to blame drugs cause you’re
> right, there
> are over a thousand of them.
>
> Never mind...
>
> > On Sep 1, 2020, at 8:55 PM, Chris Hostetter
> wrote:
> >
> >
> > : Hmmm, that’s kind of an dilemma then. Are you saying that
> > : he test can see that the directory appears writable then tries
> > : to write to it then gets tripped up by the framework?
> > :
> > : Seems to me that a test that tries to write, thinks it can and then
> > : can’t should fail anyway.
> > :
> > : Well, I don’t think there are very many tests that have this problem
> > : anyway, so maybe I can examine them one-by-one and not
> > : introduce new failures...
> >
> > You keep using the phrase "the test" in the context of (trying to)
> create
> > these directories ("userfiles" and "filestore") ... the "test CODE"
> isn't
> > making any choices about trying to write those files -- the choice is
> > being made by the "CoreContainer CODE".
> >
> > These features were added with the explicit implementation choice to
> _TRY_
> > to write the "usersfiles" (and/or "filestore") directory to "solr home"
> IF
> > POSSIBLE, and if so then enable a bunch of features -- if NOT then log a
> > WARNing and don't enable those features.
> >
> > So what you're seeing here isn't an artifact/result of any particular
> > choices "a test" or "the test" makes -- it's a concious choice of the
> > developer who added this feature to solr. These WARN messages that show
> > up in tests where the solr home dir isn't writable (which is actaully
> the
> > vast majority of tests because of how the test framework works) are the
> > same types of WARN messages that a "real" solr deployment might get if
> > their solr home dir isn't wriable (ie: maybe the use ${solr.data.dir} to
> > point to a diff drive).
> >
> >
> >
> > :
> > : > On Aug 31, 2020, at 1:29 PM, Chris Hostetter <
> hossman_luc...@fucit.org> wrote:
> > : >
> > : >
> > : > Some tests "create" a new solr home dir and copy config files there,
> but
> > : > you'll see this type of WARN logging for any test that just uses the
> test
> > : > configs "in place" because of how the code is designed to _try_ and
> create
> > : > a userfiles directory in the solr home if it's writable.
> > : >
> > : >
> > : > : Date: Sat, 29 Aug 2020 09:25:17 -0400
> > : > : From: Erick Erickson
> > : > : Reply-To: dev@lucene.apache.org
> > : > : To: dev@lucene.apache.org
> > : > : Subject: Re: Annoying but harmless exceptions due to
> filepermissions when
> > : > : running tests
> > : > :
> > : > : Well, as Uwe and I discussed offline, he’s right and I’m wrong.
> > : > :
> > : > : In CoreContainer [364] there’s code like this:
> > : > :
> > : > : Path userFilesPath = return solrHome.resolve("userfiles"); // TODO
> make configurable on cfg?
> > : > : try {
> > : > : Files.createDirectories(userFilesPath); // does nothing if
> already exists
> > : > : } catch (Exception e) {
> > : > : log.warn("Unable to create [{}]. Features requiring this
> directory may fail.", userFilesPath, e);
> > : > : }
> > : > :
> > : > : So I assumed it would happen on most every test, at least in cloud
> mode. But when I tried to make it happen on a different test, there was no
> exception.
> > : > :
> > : > : I’ll have to poke some more to see what’s really happening…
> > : > :
> > : > : Never Mind….
> > : > :
> > : > : > On Aug 29, 2020, at 8:59 AM, Uwe Schindler
> wrote:
> > : > : >
> > : > : > Hi,
> > : > : >
> > : > : > this is a bug in the test! It should never ever modify files
> outside its sandbox. It should not even modify files in build directory. It
> is fully valid to reject those writes - has nothing to do with users, it's
> just forbidd
Re: Annoying but harmless exceptions due to filepermissions when running tests
Oh bother. Somehow I thought I’d looked and only a handful of tests reported
this.
So I looked again and I _wish_ I was able to blame drugs cause you’re right,
there
are over a thousand of them.
Never mind...
> On Sep 1, 2020, at 8:55 PM, Chris Hostetter wrote:
>
>
> : Hmmm, that’s kind of an dilemma then. Are you saying that
> : he test can see that the directory appears writable then tries
> : to write to it then gets tripped up by the framework?
> :
> : Seems to me that a test that tries to write, thinks it can and then
> : can’t should fail anyway.
> :
> : Well, I don’t think there are very many tests that have this problem
> : anyway, so maybe I can examine them one-by-one and not
> : introduce new failures...
>
> You keep using the phrase "the test" in the context of (trying to) create
> these directories ("userfiles" and "filestore") ... the "test CODE" isn't
> making any choices about trying to write those files -- the choice is
> being made by the "CoreContainer CODE".
>
> These features were added with the explicit implementation choice to _TRY_
> to write the "usersfiles" (and/or "filestore") directory to "solr home" IF
> POSSIBLE, and if so then enable a bunch of features -- if NOT then log a
> WARNing and don't enable those features.
>
> So what you're seeing here isn't an artifact/result of any particular
> choices "a test" or "the test" makes -- it's a concious choice of the
> developer who added this feature to solr. These WARN messages that show
> up in tests where the solr home dir isn't writable (which is actaully the
> vast majority of tests because of how the test framework works) are the
> same types of WARN messages that a "real" solr deployment might get if
> their solr home dir isn't wriable (ie: maybe the use ${solr.data.dir} to
> point to a diff drive).
>
>
>
> :
> : > On Aug 31, 2020, at 1:29 PM, Chris Hostetter
> wrote:
> : >
> : >
> : > Some tests "create" a new solr home dir and copy config files there, but
> : > you'll see this type of WARN logging for any test that just uses the test
> : > configs "in place" because of how the code is designed to _try_ and
> create
> : > a userfiles directory in the solr home if it's writable.
> : >
> : >
> : > : Date: Sat, 29 Aug 2020 09:25:17 -0400
> : > : From: Erick Erickson
> : > : Reply-To: dev@lucene.apache.org
> : > : To: dev@lucene.apache.org
> : > : Subject: Re: Annoying but harmless exceptions due to filepermissions
> when
> : > : running tests
> : > :
> : > : Well, as Uwe and I discussed offline, he’s right and I’m wrong.
> : > :
> : > : In CoreContainer [364] there’s code like this:
> : > :
> : > : Path userFilesPath = return solrHome.resolve("userfiles"); // TODO make
> configurable on cfg?
> : > : try {
> : > : Files.createDirectories(userFilesPath); // does nothing if already
> exists
> : > : } catch (Exception e) {
> : > : log.warn("Unable to create [{}]. Features requiring this directory
> may fail.", userFilesPath, e);
> : > : }
> : > :
> : > : So I assumed it would happen on most every test, at least in cloud
> mode. But when I tried to make it happen on a different test, there was no
> exception.
> : > :
> : > : I’ll have to poke some more to see what’s really happening…
> : > :
> : > : Never Mind….
> : > :
> : > : > On Aug 29, 2020, at 8:59 AM, Uwe Schindler wrote:
> : > : >
> : > : > Hi,
> : > : >
> : > : > this is a bug in the test! It should never ever modify files outside
> its sandbox. It should not even modify files in build directory. It is fully
> valid to reject those writes - has nothing to do with users, it's just
> forbidden by the test framework. Modifying this file is harmful, as it may
> affect later tests.
> : > : >
> : > : > So the correct way is to copy those files to the solr container
> running inside test's sandbox. That's one of the main advantages of the Test
> sandbox: No write access anywhere outside the test, see policy file.
> : > : >
> : > : > Uwe
> : > : >
> : > : > -
> : > : > Uwe Schindler
> : > : > Achterdiek 19, D-28357 Bremen
> : > : > https://www.thetaphi.de
> : > : > eMail: u...@thetaphi.de
> : > : >
> : > : >> -Original Message-
> : > : >> From: Erick Erickson
> : > : >> Sent: Saturday, August 29, 2020 2:54 PM
> : >
Re: Annoying but harmless exceptions due to filepermissions when running tests
: Hmmm, that’s kind of an dilemma then. Are you saying that
: he test can see that the directory appears writable then tries
: to write to it then gets tripped up by the framework?
:
: Seems to me that a test that tries to write, thinks it can and then
: can’t should fail anyway.
:
: Well, I don’t think there are very many tests that have this problem
: anyway, so maybe I can examine them one-by-one and not
: introduce new failures...
You keep using the phrase "the test" in the context of (trying to) create
these directories ("userfiles" and "filestore") ... the "test CODE" isn't
making any choices about trying to write those files -- the choice is
being made by the "CoreContainer CODE".
These features were added with the explicit implementation choice to _TRY_
to write the "usersfiles" (and/or "filestore") directory to "solr home" IF
POSSIBLE, and if so then enable a bunch of features -- if NOT then log a
WARNing and don't enable those features.
So what you're seeing here isn't an artifact/result of any particular
choices "a test" or "the test" makes -- it's a concious choice of the
developer who added this feature to solr. These WARN messages that show
up in tests where the solr home dir isn't writable (which is actaully the
vast majority of tests because of how the test framework works) are the
same types of WARN messages that a "real" solr deployment might get if
their solr home dir isn't wriable (ie: maybe the use ${solr.data.dir} to
point to a diff drive).
:
: > On Aug 31, 2020, at 1:29 PM, Chris Hostetter
wrote:
: >
: >
: > Some tests "create" a new solr home dir and copy config files there, but
: > you'll see this type of WARN logging for any test that just uses the test
: > configs "in place" because of how the code is designed to _try_ and create
: > a userfiles directory in the solr home if it's writable.
: >
: >
: > : Date: Sat, 29 Aug 2020 09:25:17 -0400
: > : From: Erick Erickson
: > : Reply-To: dev@lucene.apache.org
: > : To: dev@lucene.apache.org
: > : Subject: Re: Annoying but harmless exceptions due to filepermissions when
: > : running tests
: > :
: > : Well, as Uwe and I discussed offline, he’s right and I’m wrong.
: > :
: > : In CoreContainer [364] there’s code like this:
: > :
: > : Path userFilesPath = return solrHome.resolve("userfiles"); // TODO make
configurable on cfg?
: > : try {
: > : Files.createDirectories(userFilesPath); // does nothing if already
exists
: > : } catch (Exception e) {
: > : log.warn("Unable to create [{}]. Features requiring this directory may
fail.", userFilesPath, e);
: > : }
: > :
: > : So I assumed it would happen on most every test, at least in cloud mode.
But when I tried to make it happen on a different test, there was no exception.
: > :
: > : I’ll have to poke some more to see what’s really happening…
: > :
: > : Never Mind….
: > :
: > : > On Aug 29, 2020, at 8:59 AM, Uwe Schindler wrote:
: > : >
: > : > Hi,
: > : >
: > : > this is a bug in the test! It should never ever modify files outside
its sandbox. It should not even modify files in build directory. It is fully
valid to reject those writes - has nothing to do with users, it's just
forbidden by the test framework. Modifying this file is harmful, as it may
affect later tests.
: > : >
: > : > So the correct way is to copy those files to the solr container running
inside test's sandbox. That's one of the main advantages of the Test sandbox:
No write access anywhere outside the test, see policy file.
: > : >
: > : > Uwe
: > : >
: > : > -
: > : > Uwe Schindler
: > : > Achterdiek 19, D-28357 Bremen
: > : > https://www.thetaphi.de
: > : > eMail: u...@thetaphi.de
: > : >
: > : >> -Original Message-
: > : >> From: Erick Erickson
: > : >> Sent: Saturday, August 29, 2020 2:54 PM
: > : >> To: dev@lucene.apache.org
: > : >> Subject: Annoying but harmless exceptions due to filepermissions when
running
: > : >> tests
: > : >>
: > : >> These exceptions are handled in the code and don’t affect running
tests, but
: > : >> they can be a distraction when trying to figure out what’s causing a
failure.
: > : >> When CoreContainer is being initialized, these two paths get
“Permission
: > : >> Denied” errors since they try to create directories/files.
: > : >>
: > : >> java.security.AccessControlException: access denied
("java.io.FilePermission"
: > : >>
"/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
: > : >> o
Re: Annoying but harmless exceptions due to filepermissions when running tests
Hmmm, that’s kind of an dilemma then. Are you saying that
he test can see that the directory appears writable then tries
to write to it then gets tripped up by the framework?
Seems to me that a test that tries to write, thinks it can and then
can’t should fail anyway.
Well, I don’t think there are very many tests that have this problem
anyway, so maybe I can examine them one-by-one and not
introduce new failures...
> On Aug 31, 2020, at 1:29 PM, Chris Hostetter wrote:
>
>
> Some tests "create" a new solr home dir and copy config files there, but
> you'll see this type of WARN logging for any test that just uses the test
> configs "in place" because of how the code is designed to _try_ and create
> a userfiles directory in the solr home if it's writable.
>
>
> : Date: Sat, 29 Aug 2020 09:25:17 -0400
> : From: Erick Erickson
> : Reply-To: dev@lucene.apache.org
> : To: dev@lucene.apache.org
> : Subject: Re: Annoying but harmless exceptions due to filepermissions when
> : running tests
> :
> : Well, as Uwe and I discussed offline, he’s right and I’m wrong.
> :
> : In CoreContainer [364] there’s code like this:
> :
> : Path userFilesPath = return solrHome.resolve("userfiles"); // TODO make
> configurable on cfg?
> : try {
> : Files.createDirectories(userFilesPath); // does nothing if already exists
> : } catch (Exception e) {
> : log.warn("Unable to create [{}]. Features requiring this directory may
> fail.", userFilesPath, e);
> : }
> :
> : So I assumed it would happen on most every test, at least in cloud mode.
> But when I tried to make it happen on a different test, there was no
> exception.
> :
> : I’ll have to poke some more to see what’s really happening…
> :
> : Never Mind….
> :
> : > On Aug 29, 2020, at 8:59 AM, Uwe Schindler wrote:
> : >
> : > Hi,
> : >
> : > this is a bug in the test! It should never ever modify files outside its
> sandbox. It should not even modify files in build directory. It is fully
> valid to reject those writes - has nothing to do with users, it's just
> forbidden by the test framework. Modifying this file is harmful, as it may
> affect later tests.
> : >
> : > So the correct way is to copy those files to the solr container running
> inside test's sandbox. That's one of the main advantages of the Test sandbox:
> No write access anywhere outside the test, see policy file.
> : >
> : > Uwe
> : >
> : > -
> : > Uwe Schindler
> : > Achterdiek 19, D-28357 Bremen
> : > https://www.thetaphi.de
> : > eMail: u...@thetaphi.de
> : >
> : >> -Original Message-
> : >> From: Erick Erickson
> : >> Sent: Saturday, August 29, 2020 2:54 PM
> : >> To: dev@lucene.apache.org
> : >> Subject: Annoying but harmless exceptions due to filepermissions when
> running
> : >> tests
> : >>
> : >> These exceptions are handled in the code and don’t affect running tests,
> but
> : >> they can be a distraction when trying to figure out what’s causing a
> failure.
> : >> When CoreContainer is being initialized, these two paths get “Permission
> : >> Denied” errors since they try to create directories/files.
> : >>
> : >> java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> : >>
> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
> : >> ore" "write”)
> : >>
> : >> java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> : >>
> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/userf
> : >> iles" "write”)
> : >>
> : >> In both cases, the code logs a warning like "Features requiring this
> directory
> : >> may fail”.
> : >>
> : >> “build” is permitted this way, so I guess gradle runs as some other user?
> : >>
> : >> drwxr-xr-x 11 Erick staff352 Aug 28 09:30 build
> : >>
> : >> Any hints on an easy way to avoid these? It’s not worth much effort I
> don’t
> : >> think since they’re handled, but if it’s easy I’d like to not see them.
> : >>
> : >>
> : >>
> : >> -
> : >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> : >> For additional commands, e-mail: dev-h...@lucene.apache.org
> : >
> : >
> : > -
> : > T
Re: Annoying but harmless exceptions due to filepermissions when running tests
Some tests "create" a new solr home dir and copy config files there, but
you'll see this type of WARN logging for any test that just uses the test
configs "in place" because of how the code is designed to _try_ and create
a userfiles directory in the solr home if it's writable.
: Date: Sat, 29 Aug 2020 09:25:17 -0400
: From: Erick Erickson
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: Re: Annoying but harmless exceptions due to filepermissions when
: running tests
:
: Well, as Uwe and I discussed offline, he’s right and I’m wrong.
:
: In CoreContainer [364] there’s code like this:
:
: Path userFilesPath = return solrHome.resolve("userfiles"); // TODO make
configurable on cfg?
: try {
: Files.createDirectories(userFilesPath); // does nothing if already exists
: } catch (Exception e) {
: log.warn("Unable to create [{}]. Features requiring this directory may
fail.", userFilesPath, e);
: }
:
: So I assumed it would happen on most every test, at least in cloud mode. But
when I tried to make it happen on a different test, there was no exception.
:
: I’ll have to poke some more to see what’s really happening…
:
: Never Mind….
:
: > On Aug 29, 2020, at 8:59 AM, Uwe Schindler wrote:
: >
: > Hi,
: >
: > this is a bug in the test! It should never ever modify files outside its
sandbox. It should not even modify files in build directory. It is fully valid
to reject those writes - has nothing to do with users, it's just forbidden by
the test framework. Modifying this file is harmful, as it may affect later
tests.
: >
: > So the correct way is to copy those files to the solr container running
inside test's sandbox. That's one of the main advantages of the Test sandbox:
No write access anywhere outside the test, see policy file.
: >
: > Uwe
: >
: > -
: > Uwe Schindler
: > Achterdiek 19, D-28357 Bremen
: > https://www.thetaphi.de
: > eMail: u...@thetaphi.de
: >
: >> -Original Message-
: >> From: Erick Erickson
: >> Sent: Saturday, August 29, 2020 2:54 PM
: >> To: dev@lucene.apache.org
: >> Subject: Annoying but harmless exceptions due to filepermissions when
running
: >> tests
: >>
: >> These exceptions are handled in the code and don’t affect running tests,
but
: >> they can be a distraction when trying to figure out what’s causing a
failure.
: >> When CoreContainer is being initialized, these two paths get “Permission
: >> Denied” errors since they try to create directories/files.
: >>
: >> java.security.AccessControlException: access denied
("java.io.FilePermission"
: >>
"/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
: >> ore" "write”)
: >>
: >> java.security.AccessControlException: access denied
("java.io.FilePermission"
: >>
"/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/userf
: >> iles" "write”)
: >>
: >> In both cases, the code logs a warning like "Features requiring this
directory
: >> may fail”.
: >>
: >> “build” is permitted this way, so I guess gradle runs as some other user?
: >>
: >> drwxr-xr-x 11 Erick staff352 Aug 28 09:30 build
: >>
: >> Any hints on an easy way to avoid these? It’s not worth much effort I don’t
: >> think since they’re handled, but if it’s easy I’d like to not see them.
: >>
: >>
: >>
: >> -
: >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: >> For additional commands, e-mail: dev-h...@lucene.apache.org
: >
: >
: > -
: > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: > For additional commands, e-mail: dev-h...@lucene.apache.org
: >
:
:
: -
: To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: For additional commands, e-mail: dev-h...@lucene.apache.org
:
:
-Hoss
http://www.lucidworks.com/
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Annoying but harmless exceptions due to filepermissions when running tests
Well, as Uwe and I discussed offline, he’s right and I’m wrong.
In CoreContainer [364] there’s code like this:
Path userFilesPath = return solrHome.resolve("userfiles"); // TODO make
configurable on cfg?
try {
Files.createDirectories(userFilesPath); // does nothing if already exists
} catch (Exception e) {
log.warn("Unable to create [{}]. Features requiring this directory may
fail.", userFilesPath, e);
}
So I assumed it would happen on most every test, at least in cloud mode. But
when I tried to make it happen on a different test, there was no exception.
I’ll have to poke some more to see what’s really happening…
Never Mind….
> On Aug 29, 2020, at 8:59 AM, Uwe Schindler wrote:
>
> Hi,
>
> this is a bug in the test! It should never ever modify files outside its
> sandbox. It should not even modify files in build directory. It is fully
> valid to reject those writes - has nothing to do with users, it's just
> forbidden by the test framework. Modifying this file is harmful, as it may
> affect later tests.
>
> So the correct way is to copy those files to the solr container running
> inside test's sandbox. That's one of the main advantages of the Test sandbox:
> No write access anywhere outside the test, see policy file.
>
> Uwe
>
> -
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: u...@thetaphi.de
>
>> -Original Message-
>> From: Erick Erickson
>> Sent: Saturday, August 29, 2020 2:54 PM
>> To: dev@lucene.apache.org
>> Subject: Annoying but harmless exceptions due to filepermissions when running
>> tests
>>
>> These exceptions are handled in the code and don’t affect running tests, but
>> they can be a distraction when trying to figure out what’s causing a failure.
>> When CoreContainer is being initialized, these two paths get “Permission
>> Denied” errors since they try to create directories/files.
>>
>> java.security.AccessControlException: access denied ("java.io.FilePermission"
>> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
>> ore" "write”)
>>
>> java.security.AccessControlException: access denied ("java.io.FilePermission"
>> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/userf
>> iles" "write”)
>>
>> In both cases, the code logs a warning like "Features requiring this
>> directory
>> may fail”.
>>
>> “build” is permitted this way, so I guess gradle runs as some other user?
>>
>> drwxr-xr-x 11 Erick staff352 Aug 28 09:30 build
>>
>> Any hints on an easy way to avoid these? It’s not worth much effort I don’t
>> think since they’re handled, but if it’s easy I’d like to not see them.
>>
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Annoying but harmless exceptions due to filepermissions when running tests
Hi,
this is a bug in the test! It should never ever modify files outside its
sandbox. It should not even modify files in build directory. It is fully valid
to reject those writes - has nothing to do with users, it's just forbidden by
the test framework. Modifying this file is harmful, as it may affect later
tests.
So the correct way is to copy those files to the solr container running inside
test's sandbox. That's one of the main advantages of the Test sandbox: No write
access anywhere outside the test, see policy file.
Uwe
-
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: u...@thetaphi.de
> -Original Message-
> From: Erick Erickson
> Sent: Saturday, August 29, 2020 2:54 PM
> To: dev@lucene.apache.org
> Subject: Annoying but harmless exceptions due to filepermissions when running
> tests
>
> These exceptions are handled in the code and don’t affect running tests, but
> they can be a distraction when trying to figure out what’s causing a failure.
> When CoreContainer is being initialized, these two paths get “Permission
> Denied” errors since they try to create directories/files.
>
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
> ore" "write”)
>
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/userf
> iles" "write”)
>
> In both cases, the code logs a warning like "Features requiring this directory
> may fail”.
>
> “build” is permitted this way, so I guess gradle runs as some other user?
>
> drwxr-xr-x 11 Erick staff352 Aug 28 09:30 build
>
> Any hints on an easy way to avoid these? It’s not worth much effort I don’t
> think since they’re handled, but if it’s easy I’d like to not see them.
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
