[jira] [Commented] (SOLR-14335) Lock Solr's memory to prevent swapping
[ https://issues.apache.org/jira/browse/SOLR-14335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17064647#comment-17064647 ] Jan Høydahl commented on SOLR-14335: Updated PR. Should now work on Windows too. Moved start.jar into server/lib/ to make sure that is the only entry point we support Added JNA lib and added code for mlock from Cassandra Enable with env.var `SOLR_MEMORY_LOCK=true` - possible to set in solr.in.sh Some issues with locking on macOS. Have not tested on Linux yet, but it should work? More feedback welcome > Lock Solr's memory to prevent swapping > -- > > Key: SOLR-14335 > URL: https://issues.apache.org/jira/browse/SOLR-14335 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Jan Høydahl >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > Followup from SOLR-10306. While sometimes you are able to disable or reduce > swap on the host, other times that is not so easy. Having a native option to > lock Solr's memory would be beneficial. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14335) Lock Solr's memory to prevent swapping
[ https://issues.apache.org/jira/browse/SOLR-14335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062711#comment-17062711 ] Dawid Weiss commented on SOLR-14335: >From gradle's point of view it'd make more sense to have a separate launcher >project which would depend on jetty-start. Then you can build a solr.jar with >proper manifest (classpath) and launch it as -jar solr.jar. And put everything >together in the packaging project... But I don't want to be the person to do >it in ant, to be honest. > Lock Solr's memory to prevent swapping > -- > > Key: SOLR-14335 > URL: https://issues.apache.org/jira/browse/SOLR-14335 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Jan Høydahl >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > Followup from SOLR-10306. While sometimes you are able to disable or reduce > swap on the host, other times that is not so easy. Having a native option to > lock Solr's memory would be beneficial. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14335) Lock Solr's memory to prevent swapping
[
https://issues.apache.org/jira/browse/SOLR-14335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062704#comment-17062704
]
Jan Høydahl commented on SOLR-14335:
Ok, I started a POC in [GitHub Pull Request
#1364|https://github.com/apache/lucene-solr/pull/1364]
The PR introduces a SolrBootstrap class and then replaces {{-jar start.jar}}
with {{-cp ".:solr.jar:start.jar"}} in bin/solr. All it does so far is to
demonstrate that we can start Solr with out own bootstrap class which then
delegates to Jetty's Main class. The SolrBootstrap class currently just prints
a line to stdout.
Next steps (help welcome):
* hook in some JNA code to lock memory
* ivy support (needs to place the solr.jar in server folder)
* replace all -jar start.jar in both bin/solr and bin/solr.cmd
* add a sys param and env.var to control memory locking on/off (or should we
always attempt to lock?)
> Lock Solr's memory to prevent swapping
> --
>
> Key: SOLR-14335
> URL: https://issues.apache.org/jira/browse/SOLR-14335
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Jan Høydahl
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Followup from SOLR-10306. While sometimes you are able to disable or reduce
> swap on the host, other times that is not so easy. Having a native option to
> lock Solr's memory would be beneficial.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14335) Lock Solr's memory to prevent swapping
[ https://issues.apache.org/jira/browse/SOLR-14335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17060048#comment-17060048 ] Uwe Schindler commented on SOLR-14335: -- This library seems to do way to much, especially it tries to access JVM internals. We should use the plain simple case to lock to the JVM memory, so I'd suggest to use simple JNA like Elasticsearch or [~hossman]'s agent tool. Please don't try to hook into JVM internals! That's likely to break. In the future we can use Java 15+ Project Panama to do this, but not at that stage! > Lock Solr's memory to prevent swapping > -- > > Key: SOLR-14335 > URL: https://issues.apache.org/jira/browse/SOLR-14335 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Jan Høydahl >Priority: Major > > Followup from SOLR-10306. While sometimes you are able to disable or reduce > swap on the host, other times that is not so easy. Having a native option to > lock Solr's memory would be beneficial. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14335) Lock Solr's memory to prevent swapping
[
https://issues.apache.org/jira/browse/SOLR-14335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17060043#comment-17060043
]
Jan Høydahl commented on SOLR-14335:
In slack, [~uschindler] suggests we can create a new
{{org.apache.solr.bootstrap.Main}} class responsible for doing the necessary
JNA calls to do the locking. Guess we can borrow the
[NativeLibrary.java|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/NativeLibrary.java]
class from Cassandra which implements a {{tryMlockall()}} method that seems to
do the right thing for various platforms?
> Lock Solr's memory to prevent swapping
> --
>
> Key: SOLR-14335
> URL: https://issues.apache.org/jira/browse/SOLR-14335
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Jan Høydahl
>Priority: Major
>
> Followup from SOLR-10306. While sometimes you are able to disable or reduce
> swap on the host, other times that is not so easy. Having a native option to
> lock Solr's memory would be beneficial.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
