Re: [DISCUSS] SIP-6 Solr should own the bootstrap process

[Jan Høydahl]

I agree Uwe. The scope of SIP-6 can be to let SolrBootstrap load existing 
web.xml through WebappContext, and spin off a new JIRA for the 
ServletContextHandler approach.

See the updated SIP-6 text which reflects this.

Jan

> 25. mar. 2020 kl. 18:50 skrev Uwe Schindler :
> 
> We could use 
> <https://www.eclipse.org/jetty/javadoc/9.4.14.v20181114/org/eclipse/jetty/webapp/WebAppContext.html#addOverrideDescriptor-java.lang.String-
>  
> <https://www.eclipse.org/jetty/javadoc/9.4.14.v20181114/org/eclipse/jetty/webapp/WebAppContext.html#addOverrideDescriptor-java.lang.String->>
>  to allow to override our config with a custom user-specified web.xml.
>  
> public void addOverrideDescriptor​(java.lang.String overrideDescriptor)
> The override descriptor is a web.xml format file that is applied to the 
> context after the standard WEB-INF/web.xml
>  
> Uwe
>  
> -
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de <https://www.thetaphi.de/>
> eMail: u...@thetaphi.de <mailto:u...@thetaphi.de>
>  
> From: Uwe Schindler  
> Sent: Wednesday, March 25, 2020 6:44 PM
> To: dev@lucene.apache.org
> Subject: RE: [DISCUSS] SIP-6 Solr should own the bootstrap process
>  
> Hi,
>  
> I agree, this should be a change in 9.0 only.
>  
> In general we can start with web.xml still existing (the bootstrap code would 
> use a Jetty “WebAppContext” initially). At a later stage we can replace the 
> WebappContext by a ServletContextHandler (which is the superclass and offers 
> no webapp anymore) and we can configure everything on our own.
>  
> Another idea would be to allow the end user to suply his own web.xml 
> somewhere in a directory, where he can add extra filters if really needed. On 
> bootstrap we setup our own WebappContext and tell it to load the user’s 
> web.xml, but linking our own servlets and filters and assets would be done in 
> our code. I don’t really like this approach, because this would disallow to 
> replace jetty by async Netty in the future.
>  
> Uwe
>  
> -
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de <https://www.thetaphi.de/>
> eMail: u...@thetaphi.de <mailto:u...@thetaphi.de>
>  
> From: Mike Drob mailto:md...@apache.org>> 
> Sent: Wednesday, March 25, 2020 6:07 PM
> To: dev@lucene.apache.org <mailto:dev@lucene.apache.org>
> Subject: Re: [DISCUSS] SIP-6 Solr should own the bootstrap process
>  
> I think we like to pretend that Jetty and web.xml are implementation details, 
> but I'm not sure how good of a job we do.
>  
> Whether we need to provide backwards compatibility or not is the wrong 
> question I think, because when we break compatibility we are telling a subset 
> of our users that it will not be easy for them to upgrade. I don't know who 
> out there has customizations in web.xml (maybe security filters is the most 
> common?). I don't know what the migration path for those users would look 
> like if they want to add stuff to an embedded jetty that we're running - do 
> we need to provide system properties they can set? Do we need to give them a 
> separate xml file that they can use similar syntax for?
>  
> I'm definitely against changing that for 8.6, and as much as I want this 
> change for 9.0 I really want to see an alternative that makes sense proposed 
> before we start tearing through all of this. I'll spend a bit more time 
> thinking about this and see if I can come up with any ideas as well.
>  
> Mike
>  
> On Wed, Mar 25, 2020 at 11:29 AM Jan Høydahl  <mailto:jan@cominvent.com>> wrote:
>> I agree. I changed the SIP page and now have four bullet points defined as 
>> in scope for the SIP:
>>  
>>> The most important change in this SIP will be
>>> ·Create a solr/bootstrap module with a solr.jar  to replace 
>>> start.jar  as the entrypoint (java -jar solr.jar ...)
>>> ·Hide jetty even more, i.e. rename jetty.port  → port , and perhaps 
>>> clean up other Sysprops as well
>>> ·Get rid of web.xml and instead build our contexts and paths from 
>>> Java code, driven by Solr config
>>> ·Be able to lock Solr's memory to prevent swapping (SOLR-14335)
>>  
>> Not sure how big of a job the web.xml thing is, but there is definitely a 
>> potential of unifying test and production code in this area and thus get 
>> much better test coverage of some of this.
>>  
>> However, removing web.xml is perhaps such a big change that we’ll have to 
>> target 9.0 while my initial goal of just creating a bootstrapper to lock 
>> memory could have been me

RE: [DISCUSS] SIP-6 Solr should own the bootstrap process

[Uwe Schindler]

We could use 
<https://www.eclipse.org/jetty/javadoc/9.4.14.v20181114/org/eclipse/jetty/webapp/WebAppContext.html#addOverrideDescriptor-java.lang.String->
 to allow to override our config with a custom user-specified web.xml.

 

public void addOverrideDescriptor​(java.lang.String overrideDescriptor)

The override descriptor is a web.xml format file that is applied to the context 
after the standard WEB-INF/web.xml

 

Uwe

 

-

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: u...@thetaphi.de

 

From: Uwe Schindler  
Sent: Wednesday, March 25, 2020 6:44 PM
To: dev@lucene.apache.org
Subject: RE: [DISCUSS] SIP-6 Solr should own the bootstrap process

 

Hi,

 

I agree, this should be a change in 9.0 only.

 

In general we can start with web.xml still existing (the bootstrap code would 
use a Jetty “WebAppContext” initially). At a later stage we can replace the 
WebappContext by a ServletContextHandler (which is the superclass and offers no 
webapp anymore) and we can configure everything on our own.

 

Another idea would be to allow the end user to suply his own web.xml somewhere 
in a directory, where he can add extra filters if really needed. On bootstrap 
we setup our own WebappContext and tell it to load the user’s web.xml, but 
linking our own servlets and filters and assets would be done in our code. I 
don’t really like this approach, because this would disallow to replace jetty 
by async Netty in the future.

 

Uwe

 

-

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: u...@thetaphi.de <mailto:u...@thetaphi.de> 

 

From: Mike Drob mailto:md...@apache.org> > 
Sent: Wednesday, March 25, 2020 6:07 PM
To: dev@lucene.apache.org <mailto:dev@lucene.apache.org> 
Subject: Re: [DISCUSS] SIP-6 Solr should own the bootstrap process

 

I think we like to pretend that Jetty and web.xml are implementation details, 
but I'm not sure how good of a job we do.

 

Whether we need to provide backwards compatibility or not is the wrong question 
I think, because when we break compatibility we are telling a subset of our 
users that it will not be easy for them to upgrade. I don't know who out there 
has customizations in web.xml (maybe security filters is the most common?). I 
don't know what the migration path for those users would look like if they want 
to add stuff to an embedded jetty that we're running - do we need to provide 
system properties they can set? Do we need to give them a separate xml file 
that they can use similar syntax for?

 

I'm definitely against changing that for 8.6, and as much as I want this change 
for 9.0 I really want to see an alternative that makes sense proposed before we 
start tearing through all of this. I'll spend a bit more time thinking about 
this and see if I can come up with any ideas as well.

 

Mike

 

On Wed, Mar 25, 2020 at 11:29 AM Jan Høydahl mailto:jan@cominvent.com> > wrote:

I agree. I changed the SIP page and now have four bullet points defined as in 
scope for the SIP:

 

The most important change in this SIP will be

*Create a solr/bootstrap module with a solr.jar  to replace start.jar  
as the entrypoint (java -jar solr.jar ...)

*Hide jetty even more, i.e. rename jetty.port  → port , and perhaps 
clean up other Sysprops as well

*Get rid of web.xml and instead build our contexts and paths from Java 
code, driven by Solr config

*Be able to lock Solr's memory to prevent swapping (SOLR-14335)

 

Not sure how big of a job the web.xml thing is, but there is definitely a 
potential of unifying test and production code in this area and thus get much 
better test coverage of some of this.

 

However, removing web.xml is perhaps such a big change that we’ll have to 
target 9.0 while my initial goal of just creating a bootstrapper to lock memory 
could have been merged to 8.x as well?? Or do we consider web.xml an 
implementation detail for which we don’t need to provide backward compatibility?

 

I’m a bit back and forth with myself on this. Perhaps if we reduce scope to 
just the bootstrap module, mem locking and some sysprop renames, it can be done 
quickly and end up in 8.6 already, then increment from there?

 

Jan

 

25. mar. 2020 kl. 16:13 skrev Mike Drob mailto:md...@apache.org> >:

 

I'm in favor of this approach mainly because of the line mentioned in the Test 
Plan:

> If we move servlet initialization into Java code and get rid of web.xml, we 
> may be able to start Solr in the same way both in tests and in real life!

 

I feel that this should be a top level goal, along with taking another look at 
EmbeddedSolrServer (perhaps it moves out of SolrJ?). Right now, I feel like we 
effectively have no testing of the configurations that go into the various xml 
files - can't speak for others, but it made me reluctant to make changes to 
them.

 

 

Mik

RE: [DISCUSS] SIP-6 Solr should own the bootstrap process

[Uwe Schindler]

Hi,

 

I agree, this should be a change in 9.0 only.

 

In general we can start with web.xml still existing (the bootstrap code would 
use a Jetty “WebAppContext” initially). At a later stage we can replace the 
WebappContext by a ServletContextHandler (which is the superclass and offers no 
webapp anymore) and we can configure everything on our own.

 

Another idea would be to allow the end user to suply his own web.xml somewhere 
in a directory, where he can add extra filters if really needed. On bootstrap 
we setup our own WebappContext and tell it to load the user’s web.xml, but 
linking our own servlets and filters and assets would be done in our code. I 
don’t really like this approach, because this would disallow to replace jetty 
by async Netty in the future.

 

Uwe

 

-

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: u...@thetaphi.de

 

From: Mike Drob  
Sent: Wednesday, March 25, 2020 6:07 PM
To: dev@lucene.apache.org
Subject: Re: [DISCUSS] SIP-6 Solr should own the bootstrap process

 

I think we like to pretend that Jetty and web.xml are implementation details, 
but I'm not sure how good of a job we do.

 

Whether we need to provide backwards compatibility or not is the wrong question 
I think, because when we break compatibility we are telling a subset of our 
users that it will not be easy for them to upgrade. I don't know who out there 
has customizations in web.xml (maybe security filters is the most common?). I 
don't know what the migration path for those users would look like if they want 
to add stuff to an embedded jetty that we're running - do we need to provide 
system properties they can set? Do we need to give them a separate xml file 
that they can use similar syntax for?

 

I'm definitely against changing that for 8.6, and as much as I want this change 
for 9.0 I really want to see an alternative that makes sense proposed before we 
start tearing through all of this. I'll spend a bit more time thinking about 
this and see if I can come up with any ideas as well.

 

Mike

 

On Wed, Mar 25, 2020 at 11:29 AM Jan Høydahl mailto:jan@cominvent.com> > wrote:

I agree. I changed the SIP page and now have four bullet points defined as in 
scope for the SIP:

 

The most important change in this SIP will be

*Create a solr/bootstrap module with a solr.jar  to replace start.jar  
as the entrypoint (java -jar solr.jar ...)

*Hide jetty even more, i.e. rename jetty.port  → port , and perhaps 
clean up other Sysprops as well

*Get rid of web.xml and instead build our contexts and paths from Java 
code, driven by Solr config

*Be able to lock Solr's memory to prevent swapping (SOLR-14335)

 

Not sure how big of a job the web.xml thing is, but there is definitely a 
potential of unifying test and production code in this area and thus get much 
better test coverage of some of this.

 

However, removing web.xml is perhaps such a big change that we’ll have to 
target 9.0 while my initial goal of just creating a bootstrapper to lock memory 
could have been merged to 8.x as well?? Or do we consider web.xml an 
implementation detail for which we don’t need to provide backward compatibility?

 

I’m a bit back and forth with myself on this. Perhaps if we reduce scope to 
just the bootstrap module, mem locking and some sysprop renames, it can be done 
quickly and end up in 8.6 already, then increment from there?

 

Jan





25. mar. 2020 kl. 16:13 skrev Mike Drob mailto:md...@apache.org> >:

 

I'm in favor of this approach mainly because of the line mentioned in the Test 
Plan:

> If we move servlet initialization into Java code and get rid of web.xml, we 
> may be able to start Solr in the same way both in tests and in real life!

 

I feel that this should be a top level goal, along with taking another look at 
EmbeddedSolrServer (perhaps it moves out of SolrJ?). Right now, I feel like we 
effectively have no testing of the configurations that go into the various xml 
files - can't speak for others, but it made me reluctant to make changes to 
them.

 

 

Mike

 

On Tue, Mar 24, 2020 at 6:36 PM Jan Høydahl mailto:jan@cominvent.com> > wrote:

This SIP has changed name to SIP-6 and I have changed the email subject of this 
thread, please keep this new subject going forward.

New SIP link: 
https://cwiki.apache.org/confluence/display/SOLR/SIP-6+Solr+should+own+the+bootstrap+process

 

Jan





24. mar. 2020 kl. 22:48 skrev Uwe Schindler mailto:u...@thetaphi.de> >:

 

Of course it's a single jvm. It's only a replacement for start.jar. the mock-up 
is already there.

Uwe

Am March 24, 2020 9:45:33 PM UTC schrieb Bram Van Dam mailto:bram.van...@intix.eu> >:

On 24/03/2020 16:01, Jan Høydahl wrote:

It has been proposed many times to make Solr a truly standalone app and that we 
should start Jetty from within Solr and not t

RE: [DISCUSS] SIP-6 Solr should own the bootstrap process

[Uwe Schindler]

Hi,

 

I think Solr should ship with default CORS Filter enabled (it’s part of Jetty). 
E.g. Elasticsearch also ships with a suitable CORS configuration by default. 
You can just change some details (e.g. restrict hostnames, default is “*”) or 
dis/enable it completely. We can also be much more selective which endpoints 
allow CORS at all, this limits security risks.

 

Uwe

 

-

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: u...@thetaphi.de

 

From: Eric Pugh  
Sent: Wednesday, March 25, 2020 5:34 PM
To: dev@lucene.apache.org
Subject: Re: [DISCUSS] SIP-6 Solr should own the bootstrap process

 

Would being able to manage CORS settings be part of this?  

 

Now that Solr is ramping up it’s default security posture, there are many 
applications that used to work that don’t.  Today, changing Solr’s CORS 
settings is very cumbersome, requiring changing an embedded web.xml.

 

Making it easier to set your CORS type settings would be great.

 

Eric





On Mar 25, 2020, at 12:29 PM, Jan Høydahl mailto:jan@cominvent.com> > wrote:

 

I agree. I changed the SIP page and now have four bullet points defined as in 
scope for the SIP:

 

The most important change in this SIP will be

*Create a solr/bootstrap module with a solr.jar  to replace start.jar  
as the entrypoint (java -jar solr.jar ...)

*Hide jetty even more, i.e. rename jetty.port  → port , and perhaps 
clean up other Sysprops as well

*Get rid of web.xml and instead build our contexts and paths from Java 
code, driven by Solr config

*Be able to lock Solr's memory to prevent swapping (SOLR-14335)

 

Not sure how big of a job the web.xml thing is, but there is definitely a 
potential of unifying test and production code in this area and thus get much 
better test coverage of some of this.

 

However, removing web.xml is perhaps such a big change that we’ll have to 
target 9.0 while my initial goal of just creating a bootstrapper to lock memory 
could have been merged to 8.x as well?? Or do we consider web.xml an 
implementation detail for which we don’t need to provide backward compatibility?

 

I’m a bit back and forth with myself on this. Perhaps if we reduce scope to 
just the bootstrap module, mem locking and some sysprop renames, it can be done 
quickly and end up in 8.6 already, then increment from there?

 

Jan





25. mar. 2020 kl. 16:13 skrev Mike Drob mailto:md...@apache.org> >:

 

I'm in favor of this approach mainly because of the line mentioned in the Test 
Plan:

> If we move servlet initialization into Java code and get rid of web.xml, we 
> may be able to start Solr in the same way both in tests and in real life!

 

I feel that this should be a top level goal, along with taking another look at 
EmbeddedSolrServer (perhaps it moves out of SolrJ?). Right now, I feel like we 
effectively have no testing of the configurations that go into the various xml 
files - can't speak for others, but it made me reluctant to make changes to 
them.

 

 

Mike

 

On Tue, Mar 24, 2020 at 6:36 PM Jan Høydahl mailto:jan@cominvent.com> > wrote:

This SIP has changed name to SIP-6 and I have changed the email subject of this 
thread, please keep this new subject going forward.

New SIP link: 
https://cwiki.apache.org/confluence/display/SOLR/SIP-6+Solr+should+own+the+bootstrap+process

 

Jan





24. mar. 2020 kl. 22:48 skrev Uwe Schindler mailto:u...@thetaphi.de> >:

 

Of course it's a single jvm. It's only a replacement for start.jar. the mock-up 
is already there.

Uwe

Am March 24, 2020 9:45:33 PM UTC schrieb Bram Van Dam mailto:bram.van...@intix.eu> >:

On 24/03/2020 16:01, Jan Høydahl wrote:

It has been proposed many times to make Solr a truly standalone app and that we 
should start Jetty from within Solr and not the other way around :)


Do you see this running in a single JVM, i.e. you run solr.jar and it
then starts an embedded Jetty server? Or would solr.jar exec a new Java
process and take it from there?

If it's all in one JVM, then the startup scripts probably won't be all
that different. Memory settings, OOM killers etc will still have to be
set up before using startup shellscripts and whatnot. If it's just a
small bootstrap jar which execs another JVM, a lot of the complicated
shell foo could probably be simplified.

 - Bram


  _  

To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org 
<mailto:dev-unsubscr...@lucene.apache.org> 
For additional commands, e-mail: dev-h...@lucene.apache.org 
<mailto:dev-h...@lucene.apache.org> 


--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de <https://www.thetaphi.de/> 

 

 

 

___

Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | 
http://www.opensourceconnections.com <http://www.opensourceconnections.com/>  | 
My Free/Busy <http://tinyu

Re: [DISCUSS] SIP-6 Solr should own the bootstrap process

[Mike Drob]

I think we like to pretend that Jetty and web.xml are implementation
details, but I'm not sure how good of a job we do.

Whether we need to provide backwards compatibility or not is the wrong
question I think, because when we break compatibility we are telling a
subset of our users that it will not be easy for them to upgrade. I don't
know who out there has customizations in web.xml (maybe security filters is
the most common?). I don't know what the migration path for those users
would look like if they want to add stuff to an embedded jetty that we're
running - do we need to provide system properties they can set? Do we need
to give them a separate xml file that they can use similar syntax for?

I'm definitely against changing that for 8.6, and as much as I want this
change for 9.0 I really want to see an alternative that makes sense
proposed before we start tearing through all of this. I'll spend a bit more
time thinking about this and see if I can come up with any ideas as well.

Mike

On Wed, Mar 25, 2020 at 11:29 AM Jan Høydahl  wrote:

> I agree. I changed the SIP page and now have four bullet points defined as
> in scope for the SIP:
>
> The most important change in this SIP will be
>
>- Create a solr/bootstrap module with a solr.jar  to replace start.jar  as
>the entrypoint (java -jar solr.jar ...)
>- Hide jetty even more, i.e. rename jetty.port  → port , and perhaps
>clean up other Sysprops as well
>- Get rid of web.xml and instead build our contexts and paths from
>Java code, driven by Solr config
>- Be able to lock Solr's memory to prevent swapping (SOLR-14335)
>
>
> Not sure how big of a job the web.xml thing is, but there is definitely a
> potential of unifying test and production code in this area and thus get
> much better test coverage of some of this.
>
> However, removing web.xml is perhaps such a big change that we’ll have to
> target 9.0 while my initial goal of just creating a bootstrapper to lock
> memory could have been merged to 8.x as well?? Or do we consider web.xml an
> implementation detail for which we don’t need to provide backward
> compatibility?
>
> I’m a bit back and forth with myself on this. Perhaps if we reduce scope
> to just the bootstrap module, mem locking and some sysprop renames, it can
> be done quickly and end up in 8.6 already, then increment from there?
>
> Jan
>
> 25. mar. 2020 kl. 16:13 skrev Mike Drob :
>
> I'm in favor of this approach mainly because of the line mentioned in the
> Test Plan:
>
> > If we move servlet initialization into Java code and get rid of web.xml,
> we may be able to start Solr in the same way both in tests and in real life!
>
> I feel that this should be a top level goal, along with taking another
> look at EmbeddedSolrServer (perhaps it moves out of SolrJ?). Right now, I
> feel like we effectively have no testing of the configurations that go into
> the various xml files - can't speak for others, but it made me reluctant to
> make changes to them.
>
>
> Mike
>
> On Tue, Mar 24, 2020 at 6:36 PM Jan Høydahl  wrote:
>
>> This SIP has changed name to SIP-6 and I have changed the email subject
>> of this thread, please keep this new subject going forward.
>> New SIP link:
>> https://cwiki.apache.org/confluence/display/SOLR/SIP-6+Solr+should+own+the+bootstrap+process
>>
>> Jan
>>
>> 24. mar. 2020 kl. 22:48 skrev Uwe Schindler :
>>
>> Of course it's a single jvm. It's only a replacement for start.jar. the
>> mock-up is already there.
>>
>> Uwe
>>
>> Am March 24, 2020 9:45:33 PM UTC schrieb Bram Van Dam <
>> bram.van...@intix.eu>:
>>>
>>> On 24/03/2020 16:01, Jan Høydahl wrote:
>>>
 It has been proposed many times to make Solr a truly standalone app and 
 that we should start Jetty from within Solr and not the other way around :)

>>>
>>> Do you see this running in a single JVM, i.e. you run solr.jar and it
>>> then starts an embedded Jetty server? Or would solr.jar exec a new Java
>>> process and take it from there?
>>>
>>> If it's all in one JVM, then the startup scripts probably won't be all
>>> that different. Memory settings, OOM killers etc will still have to be
>>> set up before using startup shellscripts and whatnot. If it's just a
>>> small bootstrap jar which execs another JVM, a lot of the complicated
>>> shell foo could probably be simplified.
>>>
>>>  - Bram
>>> --
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org 
>>> 
>>> For additional commands, e-mail: dev-h...@lucene.apache.org 
>>> 
>>>
>>>
>> --
>> Uwe Schindler
>> Achterdiek 19, 28357 Bremen
>> https://www.thetaphi.de
>>
>>
>>
>

Re: [DISCUSS] SIP-6 Solr should own the bootstrap process

[Eric Pugh]

Would being able to manage CORS settings be part of this?  

Now that Solr is ramping up it’s default security posture, there are many 
applications that used to work that don’t.  Today, changing Solr’s CORS 
settings is very cumbersome, requiring changing an embedded web.xml.

Making it easier to set your CORS type settings would be great.

Eric

> On Mar 25, 2020, at 12:29 PM, Jan Høydahl  wrote:
> 
> I agree. I changed the SIP page and now have four bullet points defined as in 
> scope for the SIP:
> 
>> The most important change in this SIP will be
>> Create a solr/bootstrap module with a solr.jar  to replace start.jar  as the 
>> entrypoint (java -jar solr.jar ...)
>> Hide jetty even more, i.e. rename jetty.port  → port , and perhaps clean up 
>> other Sysprops as well
>> Get rid of web.xml and instead build our contexts and paths from Java code, 
>> driven by Solr config
>> Be able to lock Solr's memory to prevent swapping (SOLR-14335)
> 
> Not sure how big of a job the web.xml thing is, but there is definitely a 
> potential of unifying test and production code in this area and thus get much 
> better test coverage of some of this.
> 
> However, removing web.xml is perhaps such a big change that we’ll have to 
> target 9.0 while my initial goal of just creating a bootstrapper to lock 
> memory could have been merged to 8.x as well?? Or do we consider web.xml an 
> implementation detail for which we don’t need to provide backward 
> compatibility?
> 
> I’m a bit back and forth with myself on this. Perhaps if we reduce scope to 
> just the bootstrap module, mem locking and some sysprop renames, it can be 
> done quickly and end up in 8.6 already, then increment from there?
> 
> Jan
> 
>> 25. mar. 2020 kl. 16:13 skrev Mike Drob > >:
>> 
>> I'm in favor of this approach mainly because of the line mentioned in the 
>> Test Plan:
>> > If we move servlet initialization into Java code and get rid of web.xml, 
>> > we may be able to start Solr in the same way both in tests and in real 
>> > life!
>> 
>> I feel that this should be a top level goal, along with taking another look 
>> at EmbeddedSolrServer (perhaps it moves out of SolrJ?). Right now, I feel 
>> like we effectively have no testing of the configurations that go into the 
>> various xml files - can't speak for others, but it made me reluctant to make 
>> changes to them.
>> 
>> 
>> Mike
>> 
>> On Tue, Mar 24, 2020 at 6:36 PM Jan Høydahl > > wrote:
>> This SIP has changed name to SIP-6 and I have changed the email subject of 
>> this thread, please keep this new subject going forward.
>> New SIP link: 
>> https://cwiki.apache.org/confluence/display/SOLR/SIP-6+Solr+should+own+the+bootstrap+process
>>  
>> 
>> 
>> Jan
>> 
>>> 24. mar. 2020 kl. 22:48 skrev Uwe Schindler >> >:
>>> 
>>> Of course it's a single jvm. It's only a replacement for start.jar. the 
>>> mock-up is already there.
>>> 
>>> Uwe
>>> 
>>> Am March 24, 2020 9:45:33 PM UTC schrieb Bram Van Dam >> >:
>>> On 24/03/2020 16:01, Jan Høydahl wrote:
>>> It has been proposed many times to make Solr a truly standalone app and 
>>> that we should start Jetty from within Solr and not the other way around :)
>>> 
>>> Do you see this running in a single JVM, i.e. you run solr.jar and it
>>> then starts an embedded Jetty server? Or would solr.jar exec a new Java
>>> process and take it from there?
>>> 
>>> If it's all in one JVM, then the startup scripts probably won't be all
>>> that different. Memory settings, OOM killers etc will still have to be
>>> set up before using startup shellscripts and whatnot. If it's just a
>>> small bootstrap jar which execs another JVM, a lot of the complicated
>>> shell foo could probably be simplified.
>>> 
>>>  - Bram
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org 
>>> 
>>> For additional commands, e-mail: dev-h...@lucene.apache.org 
>>> 
>>> 
>>> 
>>> --
>>> Uwe Schindler
>>> Achterdiek 19, 28357 Bremen
>>> https://www.thetaphi.de 
> 

___
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | 
http://www.opensourceconnections.com  | 
My Free/Busy   
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 


This e-mail and all contents, including attachments, is considered to be 
Company Confidential unless explicitly stated otherwise, regardless of whether 
attachments are marked as such.

Re: [DISCUSS] SIP-6 Solr should own the bootstrap process

[Jan Høydahl]

I agree. I changed the SIP page and now have four bullet points defined as in 
scope for the SIP:

> The most important change in this SIP will be
> Create a solr/bootstrap module with a solr.jar  to replace start.jar  as the 
> entrypoint (java -jar solr.jar ...)
> Hide jetty even more, i.e. rename jetty.port  → port , and perhaps clean up 
> other Sysprops as well
> Get rid of web.xml and instead build our contexts and paths from Java code, 
> driven by Solr config
> Be able to lock Solr's memory to prevent swapping (SOLR-14335)

Not sure how big of a job the web.xml thing is, but there is definitely a 
potential of unifying test and production code in this area and thus get much 
better test coverage of some of this.

However, removing web.xml is perhaps such a big change that we’ll have to 
target 9.0 while my initial goal of just creating a bootstrapper to lock memory 
could have been merged to 8.x as well?? Or do we consider web.xml an 
implementation detail for which we don’t need to provide backward compatibility?

I’m a bit back and forth with myself on this. Perhaps if we reduce scope to 
just the bootstrap module, mem locking and some sysprop renames, it can be done 
quickly and end up in 8.6 already, then increment from there?

Jan

> 25. mar. 2020 kl. 16:13 skrev Mike Drob :
> 
> I'm in favor of this approach mainly because of the line mentioned in the 
> Test Plan:
> > If we move servlet initialization into Java code and get rid of web.xml, we 
> > may be able to start Solr in the same way both in tests and in real life!
> 
> I feel that this should be a top level goal, along with taking another look 
> at EmbeddedSolrServer (perhaps it moves out of SolrJ?). Right now, I feel 
> like we effectively have no testing of the configurations that go into the 
> various xml files - can't speak for others, but it made me reluctant to make 
> changes to them.
> 
> 
> Mike
> 
> On Tue, Mar 24, 2020 at 6:36 PM Jan Høydahl  > wrote:
> This SIP has changed name to SIP-6 and I have changed the email subject of 
> this thread, please keep this new subject going forward.
> New SIP link: 
> https://cwiki.apache.org/confluence/display/SOLR/SIP-6+Solr+should+own+the+bootstrap+process
>  
> 
> 
> Jan
> 
>> 24. mar. 2020 kl. 22:48 skrev Uwe Schindler > >:
>> 
>> Of course it's a single jvm. It's only a replacement for start.jar. the 
>> mock-up is already there.
>> 
>> Uwe
>> 
>> Am March 24, 2020 9:45:33 PM UTC schrieb Bram Van Dam > >:
>> On 24/03/2020 16:01, Jan Høydahl wrote:
>> It has been proposed many times to make Solr a truly standalone app and that 
>> we should start Jetty from within Solr and not the other way around :)
>> 
>> Do you see this running in a single JVM, i.e. you run solr.jar and it
>> then starts an embedded Jetty server? Or would solr.jar exec a new Java
>> process and take it from there?
>> 
>> If it's all in one JVM, then the startup scripts probably won't be all
>> that different. Memory settings, OOM killers etc will still have to be
>> set up before using startup shellscripts and whatnot. If it's just a
>> small bootstrap jar which execs another JVM, a lot of the complicated
>> shell foo could probably be simplified.
>> 
>>  - Bram
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org 
>> 
>> For additional commands, e-mail: dev-h...@lucene.apache.org 
>> 
>> 
>> 
>> --
>> Uwe Schindler
>> Achterdiek 19, 28357 Bremen
>> https://www.thetaphi.de 

Re: [DISCUSS] SIP-6 Solr should own the bootstrap process

[Mike Drob]

I'm in favor of this approach mainly because of the line mentioned in the
Test Plan:

> If we move servlet initialization into Java code and get rid of web.xml,
we may be able to start Solr in the same way both in tests and in real life!

I feel that this should be a top level goal, along with taking another look
at EmbeddedSolrServer (perhaps it moves out of SolrJ?). Right now, I feel
like we effectively have no testing of the configurations that go into the
various xml files - can't speak for others, but it made me reluctant to
make changes to them.


Mike

On Tue, Mar 24, 2020 at 6:36 PM Jan Høydahl  wrote:

> This SIP has changed name to SIP-6 and I have changed the email subject of
> this thread, please keep this new subject going forward.
> New SIP link:
> https://cwiki.apache.org/confluence/display/SOLR/SIP-6+Solr+should+own+the+bootstrap+process
>
> Jan
>
> 24. mar. 2020 kl. 22:48 skrev Uwe Schindler :
>
> Of course it's a single jvm. It's only a replacement for start.jar. the
> mock-up is already there.
>
> Uwe
>
> Am March 24, 2020 9:45:33 PM UTC schrieb Bram Van Dam <
> bram.van...@intix.eu>:
>>
>> On 24/03/2020 16:01, Jan Høydahl wrote:
>>
>>> It has been proposed many times to make Solr a truly standalone app and 
>>> that we should start Jetty from within Solr and not the other way around :)
>>>
>>
>> Do you see this running in a single JVM, i.e. you run solr.jar and it
>> then starts an embedded Jetty server? Or would solr.jar exec a new Java
>> process and take it from there?
>>
>> If it's all in one JVM, then the startup scripts probably won't be all
>> that different. Memory settings, OOM killers etc will still have to be
>> set up before using startup shellscripts and whatnot. If it's just a
>> small bootstrap jar which execs another JVM, a lot of the complicated
>> shell foo could probably be simplified.
>>
>>  - Bram
>> --
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org 
>> 
>> For additional commands, e-mail: dev-h...@lucene.apache.org 
>> 
>>
>>
> --
> Uwe Schindler
> Achterdiek 19, 28357 Bremen
> https://www.thetaphi.de
>
>
>