From sysv to systemd

[Marius Ghita]

Greetings everyone. I would be interested to help transition Solr from SysV
to systemd.

Before coming here I've done a rudimentary search on the bug tracker to see
systemd related issues.

https://issues.apache.org/jira/browse/SOLR-10905
It is a 2017 issue supporting the same change. With a linked systemd
service file in the comment section, which I'll come back to.[1]

https://issues.apache.org/jira/browse/SOLR-9769
As we would not be calling SysV scripts directly this wouldn't be an issue
anymore, as systemd already knows the given service has stopped and
wouldn't need to call the ExecStop command anymore.

https://issues.apache.org/jira/browse/SOLR-9177
Based on the last comment this issue seems fixed, but not marked as such in
the tracker? While not necessarily up there on my list, the advantage of
running Solr in the foreground would simplify Solr log access via
journalctl


While trying out Solr 8.5.0 on CentOS 8 a few days ago I've noticed that
there are a couple of hoops one has to jump through to get the service to
start via SysV (issues that are just not present with systemd).

Systemd generates automated service files as wrappers for SysV scripts, and
that's the case for Solr as well.


*"Apr 15 12:52:14 localhost.localdomain solr[9084]: /etc/rc.d/init.d/solr:
line 75: su: command not found"*

The first issue I've had is that for some reason the su command path is not
resolved. su is located in /bin and /usr/bin, both are defined in $PATH if
I print it out from within the init.d/solr file. Patched manually by
specifying an absolute path.

Then I had a couple of SELinux violations, for which I had to run a series
of systemctl start solr, audit2allow -a -M, semodule -i until I've managed
to generate a type enforcement policy that would allow Solr to start. The
resulting TE file:

module solrsu 1.0;
require {
  type lastlog_t;
  type su_exec_t;
  type init_t;
  class file { execute execute_no_trans getattr lock map open read write };
  class passwd rootok;
}
#= init_t ==
allow init_t lastlog_t:file lock;
allow init_t lastlog_t:file { open read write };
allow init_t self:passwd rootok;
allow init_t su_exec_t:file { execute execute_no_trans getattr map open
read };

For someone familiar with SELinux they might see that a large part of the
policy has to deal with the su execution by an init script.
Of course with systemd, none of this SELinux wizardry is required, as - for
better or worse - systemd runs with GODMODE cheats.


[1] Back to the initial service file, alternatively I propose the following
base service file.

[Unit]
Description=Apache SOLR

[Service]
Type=forking
User=solr
Environment=SOLR_INCLUDE=/etc/default/solr.in.sh
ExecStart=/opt/solr/bin/solr start
ExecStop=/opt/solr/bin/solr stop
Restart=on-failure
LimitNOFILE=65000
LimitNPROC=65000
TimeoutSec=180s

[Install]
WantedBy=multi-user.target

We don't need all those Before/After/Conflicts, users can customize further
the service via systemd drop-ins
https://coreos.com/os/docs/latest/using-systemd-drop-in-units.html

Specify Type=forking and remove PIDFile. Systemd can guess the PIDFile just
fine as Solr spawns a single process. Also, I had issues with the PIDFile
and permissions, I didn't want to debug further :)
https://www.freedesktop.org/software/systemd/man/systemd.service.html#GuessMainPID=

TimeoutSec= set to 180 seconds. Wait for start/stop the same amount in
systemd as the solr bin

multi-user.target is enough as it's the equivalent of runlevel 3
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/sect-managing_services_with_systemd-targets
and graphical.target (runlevel 5) inherits already services of lower targets

I've included the LimitNOFILE/LimitNPROC directives, as it is already a
suggestion that shows up during startup and why not set up suggested
defaults.

Is there any opposition to this change? And how can I help to move it
forward?

Re: Lucene/Solr 8.5.1 bugfix release

[Adrien Grand]

Solr doesn't use addIndexes(Directory) so this is not relevant to Solr
indeed.

On Wed, Apr 15, 2020 at 11:40 AM Ignacio Vera  wrote:

> I updated the Solr's release announcement so there is no changes. I think
> the Lucene's bug fix is not relevant for Solr.
>
> On Wed, Apr 15, 2020 at 11:23 AM Jan Høydahl 
> wrote:
>
>> If RC1 is released with a non-working SOLR-14359, then please change
>> Solr’s release announcement. Not sure whether the Lucene-bugfix is valid
>> for Solr users, in that case it can be highlighted instead?
>>
>> Jan
>>
>> 7. apr. 2020 kl. 10:13 skrev Ignacio Vera :
>>
>> Here are the drafts for the release notes, let me know if there is
>> something you wish to change:
>>
>> Lucene:
>>
>>
>> https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=148645634=e835ddb5-3bb9-4b33-b6ad-1770e0a95327=shareui=1586247034772
>>
>> Solr:
>>
>>
>> https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=148645636=c4a8eb5f-08d0-40db-a877-b94adb383061=shareui=1586247001694
>>
>> As I reminder I am planning to build the first RC tomorrow, Wednesday
>> April 8th.
>>
>>
>>
>> On Mon, Apr 6, 2020 at 8:37 AM Ignacio Vera  wrote:
>>
>>>  Thanks Jan,  I see SOLR-14359
>>>  has already
>>> been back ported to branch 8.5, I am ok with  SOLR-14317
>>>  backporting as well.
>>> I am planning to build the first RC this coming Wednesday. Let me know if
>>> that works for you.
>>>
>>>
>>>
>>> On Sat, Apr 4, 2020 at 12:25 AM Jan Høydahl 
>>> wrote:
>>>
 Also this coould be a backport candidate: SOLR-14317
  HttpClusterStateProvider
 throws exception when only one node down

 Jan

 3. apr. 2020 kl. 22:29 skrev Jan Høydahl :

 I plan to merge this to branch_8_5

*SOLR-14359
  Admin UI has "Select an
 option" for collections and cores drop-downs*

 Jan

 3. apr. 2020 kl. 14:15 skrev jim ferenczi :

 +1, thanks Ignacio.
 I merged the fix for LUCENE-9300
  and backported to
 the 8.5 branch.

 Le jeu. 2 avr. 2020 à 21:48, Adrien Grand  a écrit :

> My general take on this is that it's ok to upgrade a dependency in a
> patch release if the dependency upgrade itself is a new patch release of
> the same minor version. The changelog of Tika 1.24 seems to include not
> only bug fixes but also some enhancements[1], so I'd rather do a 8.6
> release in the near future than backport this dependency upgrade to 8.5.
>
> [1] https://tika.apache.org/1.24/index.html
>
> On Thu, Apr 2, 2020 at 9:33 PM Cassandra Targett <
> casstarg...@gmail.com> wrote:
>
>> Should we consider backporting SOLR-14367 (the most recent Tika
>> upgrade)? It addresses a CVE in Tika, and while I think we usually avoid
>> changing 3rd party component versions in patch releases, but maybe we
>> should in this case? The upgrade also looks like it was pretty
>> straightforward (drop-in replacement).
>>
>> Cassandra
>> On Apr 2, 2020, 12:47 PM -0500, Ignacio Vera ,
>> wrote:
>>
>> Hi,
>>
>> I propose a quick 8.5.1 bugfix release and I volunteer as RM. The
>> main motivation for this release is LUCENE-9300 where Jim addressed a
>> serious bug that can lead to data corruption when merging indices via
>> IW#addIndices.
>>
>> If there are no objections I am planning to create a RC early next
>> week.
>>
>> Best regards,
>>
>> Ignacio
>>
>>
>>
>>
>
> --
> Adrien
>



>>

-- 
Adrien

[RESULT] [VOTE] Release Lucene/Solr 8.5.1 RC1

[Ignacio Vera]

It's been >72h since the vote was initiated and the result is:


+1  8  (8 binding)

 0  0

-1  0


This vote has PASSED

Re: Lucene/Solr 8.5.1 bugfix release

[Ignacio Vera]

I updated the Solr's release announcement so there is no changes. I think
the Lucene's bug fix is not relevant for Solr.

On Wed, Apr 15, 2020 at 11:23 AM Jan Høydahl  wrote:

> If RC1 is released with a non-working SOLR-14359, then please change
> Solr’s release announcement. Not sure whether the Lucene-bugfix is valid
> for Solr users, in that case it can be highlighted instead?
>
> Jan
>
> 7. apr. 2020 kl. 10:13 skrev Ignacio Vera :
>
> Here are the drafts for the release notes, let me know if there is
> something you wish to change:
>
> Lucene:
>
>
> https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=148645634=e835ddb5-3bb9-4b33-b6ad-1770e0a95327=shareui=1586247034772
>
> Solr:
>
>
> https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=148645636=c4a8eb5f-08d0-40db-a877-b94adb383061=shareui=1586247001694
>
> As I reminder I am planning to build the first RC tomorrow, Wednesday
> April 8th.
>
>
>
> On Mon, Apr 6, 2020 at 8:37 AM Ignacio Vera  wrote:
>
>>  Thanks Jan,  I see SOLR-14359
>>  has already been back
>> ported to branch 8.5, I am ok with  SOLR-14317
>>  backporting as well.
>> I am planning to build the first RC this coming Wednesday. Let me know if
>> that works for you.
>>
>>
>>
>> On Sat, Apr 4, 2020 at 12:25 AM Jan Høydahl 
>> wrote:
>>
>>> Also this coould be a backport candidate: SOLR-14317
>>>  HttpClusterStateProvider
>>> throws exception when only one node down
>>>
>>> Jan
>>>
>>> 3. apr. 2020 kl. 22:29 skrev Jan Høydahl :
>>>
>>> I plan to merge this to branch_8_5
>>>
>>>*SOLR-14359  Admin
>>> UI has "Select an option" for collections and cores drop-downs*
>>>
>>> Jan
>>>
>>> 3. apr. 2020 kl. 14:15 skrev jim ferenczi :
>>>
>>> +1, thanks Ignacio.
>>> I merged the fix for LUCENE-9300
>>>  and backported to
>>> the 8.5 branch.
>>>
>>> Le jeu. 2 avr. 2020 à 21:48, Adrien Grand  a écrit :
>>>
 My general take on this is that it's ok to upgrade a dependency in a
 patch release if the dependency upgrade itself is a new patch release of
 the same minor version. The changelog of Tika 1.24 seems to include not
 only bug fixes but also some enhancements[1], so I'd rather do a 8.6
 release in the near future than backport this dependency upgrade to 8.5.

 [1] https://tika.apache.org/1.24/index.html

 On Thu, Apr 2, 2020 at 9:33 PM Cassandra Targett 
 wrote:

> Should we consider backporting SOLR-14367 (the most recent Tika
> upgrade)? It addresses a CVE in Tika, and while I think we usually avoid
> changing 3rd party component versions in patch releases, but maybe we
> should in this case? The upgrade also looks like it was pretty
> straightforward (drop-in replacement).
>
> Cassandra
> On Apr 2, 2020, 12:47 PM -0500, Ignacio Vera ,
> wrote:
>
> Hi,
>
> I propose a quick 8.5.1 bugfix release and I volunteer as RM. The main
> motivation for this release is LUCENE-9300 where Jim addressed a serious
> bug that can lead to data corruption when merging indices via 
> IW#addIndices.
>
> If there are no objections I am planning to create a RC early next
> week.
>
> Best regards,
>
> Ignacio
>
>
>
>

 --
 Adrien

>>>
>>>
>>>
>

Re: Lucene/Solr 8.5.1 bugfix release

[Jan Høydahl]

If RC1 is released with a non-working SOLR-14359, then please change Solr’s 
release announcement. Not sure whether the Lucene-bugfix is valid for Solr 
users, in that case it can be highlighted instead?

Jan

> 7. apr. 2020 kl. 10:13 skrev Ignacio Vera :
> 
> Here are the drafts for the release notes, let me know if there is something 
> you wish to change:
> 
> Lucene:
> 
> https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=148645634=e835ddb5-3bb9-4b33-b6ad-1770e0a95327=shareui=1586247034772
>  
> 
> 
> Solr:
> 
> https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=148645636=c4a8eb5f-08d0-40db-a877-b94adb383061=shareui=1586247001694
>  
> 
> 
> As I reminder I am planning to build the first RC tomorrow, Wednesday April 
> 8th. 
> 
> 
> 
> On Mon, Apr 6, 2020 at 8:37 AM Ignacio Vera  > wrote:
>  Thanks Jan,  I see SOLR-14359 
>  has already been back 
> ported to branch 8.5, I am ok with  SOLR-14317 
>  backporting as well. I am 
> planning to build the first RC this coming Wednesday. Let me know if that 
> works for you.
> 
>  
> 
> On Sat, Apr 4, 2020 at 12:25 AM Jan Høydahl  > wrote:
> Also this coould be a backport candidate: SOLR-14317 
>  HttpClusterStateProvider 
> throws exception when only one node down
> 
> Jan
> 
>> 3. apr. 2020 kl. 22:29 skrev Jan Høydahl > >:
>> 
>> I plan to merge this to branch_8_5
>> 
>>SOLR-14359  Admin UI 
>> has "Select an option" for collections and cores drop-downs
>> 
>> Jan
>> 
>>> 3. apr. 2020 kl. 14:15 skrev jim ferenczi >> >:
>>> 
>>> +1, thanks Ignacio.
>>> I merged the fix for LUCENE-9300 
>>>  and backported to the 
>>> 8.5 branch.
>>> 
>>> Le jeu. 2 avr. 2020 à 21:48, Adrien Grand >> > a écrit :
>>> My general take on this is that it's ok to upgrade a dependency in a patch 
>>> release if the dependency upgrade itself is a new patch release of the same 
>>> minor version. The changelog of Tika 1.24 seems to include not only bug 
>>> fixes but also some enhancements[1], so I'd rather do a 8.6 release in the 
>>> near future than backport this dependency upgrade to 8.5.
>>> 
>>> [1] https://tika.apache.org/1.24/index.html 
>>> 
>>> On Thu, Apr 2, 2020 at 9:33 PM Cassandra Targett >> > wrote:
>>> Should we consider backporting SOLR-14367 (the most recent Tika upgrade)? 
>>> It addresses a CVE in Tika, and while I think we usually avoid changing 3rd 
>>> party component versions in patch releases, but maybe we should in this 
>>> case? The upgrade also looks like it was pretty straightforward (drop-in 
>>> replacement).
>>> 
>>> Cassandra
>>> On Apr 2, 2020, 12:47 PM -0500, Ignacio Vera >> >, wrote:
 Hi,
 
 I propose a quick 8.5.1 bugfix release and I volunteer as RM. The main 
 motivation for this release is LUCENE-9300 where Jim addressed a serious 
 bug that can lead to data corruption when merging indices via 
 IW#addIndices.
 
 If there are no objections I am planning to create a RC early next week.
 
 Best regards,
 
 Ignacio
 
  
>>> 
>>> 
>>> -- 
>>> Adrien
>> 
> 

Re: 7.7.3 bugfix release

[Noble Paul]

Hi, Please merge all the required changes to the branch branch_7_7

I shall cut the branch in a day or two

On Mon, Apr 6, 2020 at 6:14 PM jim ferenczi  wrote:
>
> Hi Paul,
> Ignacio have started the release process for a bug fix release of 8.5.1 last 
> week.
> We cannot have two releases at the same time so would you agree to start 
> 7.7.3 after 8.5.1 is out ?
> I'd also like to backport LUCENE-9300 in 7.7 (the reason why we started a. 
> 8.5.1 release) so don't hesitate if you need help or to delegate the release 
> if you don't have the time at the moment.
>
> - Jim
>
>
> Le mar. 18 févr. 2020 à 18:35, Houston Putman  a 
> écrit :
>>
>> I've backported SOLR-13v69. After you add in SOLR-14013 Noble, we should be 
>> good to go with 7.7.3 I think.
>>
>> - Houston
>>
>> On Fri, Feb 14, 2020 at 1:17 PM Jan Høydahl  wrote:
>>>
>>> Falde alarm, I needed to update my branch :)
>>>
>>> Jan Høydahl
>>>
>>> 14. feb. 2020 kl. 19:11 skrev Jan Høydahl :
>>>
>>> What commit hash is the backport of SOLR-13971? I cannot find it and there 
>>> is no CHANGES entry…?
>>>
>>> 14. feb. 2020 kl. 17:52 skrev Ishan Chattopadhyaya 
>>> :
>>>
>>> +1, Houston. That's my understanding as well. Please go ahead with the 
>>> backport.
>>>
>>> On Fri, 14 Feb, 2020, 9:02 PM Houston Putman,  
>>> wrote:

 It looks like CVE-2019-17558 / SOLR-13971 has already been taken care of: 
 https://issues.apache.org/jira/browse/SOLR-13971?focusedCommentId=17014356=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17014356

 So now CVE-2019-0193 / SOLR-13669 should be the only blocker. By the 
 description in the JIRA, it looks like backporting 
 https://github.com/apache/lucene-solr/commit/025f8763549151397284af28091cfd360307baa2
  should be enough. Is this correct, or am I missing something?

 - HOuston

 On Thu, Feb 13, 2020 at 12:59 PM Jan Høydahl  wrote:

 I’m afraid I don’t have the bandwidth the next couple of weeks.

 Jan Høydahl

 > 13. feb. 2020 kl. 16:27 skrev Noble Paul :
 >
 > Do you wish to backport them?
 >
 >> On Thu, Feb 13, 2020 at 7:55 PM Jan Høydahl  
 >> wrote:
 >>
 >> According to NVD, there are at least two published CVEs that affects 
 >> 7.7.2 (CVE-2019-17558 / SOLR-13971 and CVE-2019-0193 / SOLR-13669). We 
 >> cannot release 7.7.3 with these still present.
 >>
 >> Jan
 >>
 >> 13. feb. 2020 kl. 06:42 skrev Noble Paul :
 >>
 >> I'm planning to back port  SOLR-14013 and do a bug fix release soon.
 >> Please let me know if there is anything hat you wish to be included
 >>
 >> --
 >> -
 >> Noble Paul
 >>
 >> -
 >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 >> For additional commands, e-mail: dev-h...@lucene.apache.org
 >>
 >>
 >
 >
 > --
 > -
 > Noble Paul
 >
 > -
 > To unsubscribe, %-mail: dev-unsubscr...@lucene.apache.org
 > For additional commands, e-mail: dev-h...@lucene.apache.org
 >

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

>>>


-- 
-
Noble Paul

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org