Re: Testing/packaging question
Hi,
and thanks for your hints. I've done some additional research and found
that there doesn't really seem to be any possibility of an embedded solr
server in solrpy.
Jetty, then. It'd all be probably kinda easy if it weren't for the way
things are unbundled in debian. I've recently posted to the debian-java
ML, but received no replay -- maybe because it's so solr-specific (so,
sorry for cross-posting here). It's kinda long as I've tried to capture
the issues I've run into so far. I'd really appreciate any help -- maybe
there are some solr ninjas here that know how to work around the
problems I'm experiencing.
[...]
The tests are basically a python file and a corresponding schema.xml.
Normally, it's suggested to just use solr's example directory (with
integrated jetty) and replace its schema.xml with the one provided by
solrpy, and then run it via java -jar start.jar
On debian however, solr and jetty are unbundled (for good reason), and
i
can't of course just replace solr's example schema.xml (which gets
installed to /etc/solr/conf).
I've started by looking for ways to run solr on jetty as non-root by
basically running
java -jar /usr/share/jetty/start.jar
But that is highly non-trivial, as jetty's configuration files seem to
point to solr's in a pretty hardcoded way.
I was able to change jetty's logs and home location by
java -jar -Djetty.home=/usr/share/jetty
-Djetty.logs=/home/me/solr/logs /usr/share/jetty/start.jar /etc/jetty/jetty.xml
I can run jetty as a non-root user by issuing that command from
within /usr/share/jetty, but it fails to start solr, as I haven't yet
found a way to make solr use another datadir than /var/lib/solr/data
--
which is hardcoded in /etc/solr/conf/solrconfig.xml.
I think I might be able to change the latter by having the solr-common
package change its /etc/solr/conf/solrconfig.xml to read
dataDir${solr.data.dir}/dataDir
instead of
dataDir/var/lib/solr/data/dataDir
and put a solrcore.properties into solr.home
#solrcore.properties
data.dir=/data/solrindex
( as seen over at
http://wiki.apache.org/solr/SolrConfigXml#System_property_substitution )
-- or just pass -Dsolr.data.dir=/data/dir from within some startup
script.
Would such a modification of solr-common have any chance to be
accepted
by the java team?
---
With schema.xml, I'm rather clueless. -Dsolr.solr.home=/home/me/solr/
doesn't do the trick as this stuff is also hardcoded
(in /usr/share/solr/WEB-INF/jetty-web.xml -- which is part of
solr-jetty). Actually, that's the only purpose of jetty-web.xml, which
is evaluated because of jetty's context settings (I think). Moreover,
I'm trying not to duplicate the other contents of that directory, so
setting solr.solr.home to a different location might not be what I
want
at all.
I think my most relevant questions are:
1. Is anyone able to successfully run jetty with solr
via /usr/share/jetty/start.jar as a non-root user (ideally from within
any directory, not just from within /usr/share/jetty)? If yes: how?
2. How can I make solr use my schema.xml instead of the one installed?
3. I haven't tried the whole thing in tomcat yet, so does anyone know
if
it would be easier there -- maybe because configuration files aren't
that much entangled and hardcoded there? (In the end, I might have to
do
the whole thing also for tomcat anyway as I'm build-depending on
solr-jetty | solr-tomcat which conflict each other).
I'm kinda desperate with the load of configuration files, so I'd
really
appreciate any help that gets me closer to get this thing done.
Kind regards
Bernhard
Am Donnerstag, den 04.11.2010, 23:57 +0100 schrieb Peter Karich:
Hi,
don't know if the python package provides one but solrj offers to start
solr embedded (|EmbeddedSolrServer|) and
setting up different schema + config is possible. for this see:
https://karussell.wordpress.com/2010/06/10/how-to-test-apache-solrj/
if you need an 'external solr' (via jetty and java -jar start.jar) while
tests running see this:
http://java.dzone.com/articles/getting-know-solr
Regards,
Peter.
Hi,
I'm pretty much of a Solr newbie currently packaging solrpy for Debian;
see
http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/
In order to run solrpy's supplied tests at build time, I'd need Solr to
know about the schema.xml that comes with the tests.
Can anyone tell me how do that properly? I'd basically need Solr to
temporarily recognize that schema.xml without permanently installing it
-- is there any way to do this, eg via environment variables?
TIA
Bernhard Reiter
RE: Testing/packaging question
Hi, I have up to now focussed on Jetty as it's already bundled with solr. The main issue there seems to be the way it's unbundled by Debian; I figure things might be similar with Tomcat, depending on how entangled configuration is there. Before I dig deeper into the Tomcat option: would you mind sending me the script you mention? Maybe that could clear up things a bit. Regards Bernhard Am Donnerstag, den 04.11.2010, 17:16 -0500 schrieb Turner, Robbin J: You can setup your own tomcat instance which would contain just configurations you need. You won't even have to recreate all the tomcat configuration and binaries, just the ones that were not defaults. So, if you lookup multiple tomcat configuration instance (google it), and then you'll have a set of directories. You'll need to have your own startup script that points to your configurations. You can use the current startup script as a model, then in your build procedures (I've done all this with a script) have this added to the system so you can preform restart. You'd have to have a couple of other environment variables set: export CATALINA_BASE=/path/to/your/tomcat/instance/conf/files export CATALINA_HOME=/path/to/default/installation/bin/files export SOLR_HOME=/path/to/solr/dataNconf Good luck From: Bernhard Reiter [ock...@raz.or.at] Sent: Thursday, November 04, 2010 5:49 PM To: solr-user@lucene.apache.org Subject: RE: Testing/packaging question Thanks for your instructions. Unfortunately, I need to do all that as part of my package's (python-solrpy) build procedure, so I can't change any global configuration, such as in the catalina subdirectories. I've already sensed that restarting tomcat is also just too system-invasive and would include changing its (system-wide) configuration. Are there any other ways to use solr for running the tests from http://pypi.python.org/packages/source/s/solrpy/solrpy-0.9.3.tar.gz without having to change any system configuration? Maybe via a user Tomcat instance such as provided by the tomcat6-user debian package? Thanks for your help! Bernhard Am Donnerstag, den 04.11.2010, 16:15 -0500 schrieb Turner, Robbin J: You need to either add that to catalina.sh or create a setenv.sh in the CATALINA_HOME/bin directory. Then you can restart tomcat. So, setenv.sh would contain the following: export JAVA_HOME=/path/to/jre export JAVA_OPTS==$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml If you were setting the export in your own environment and then issuing the restart, tomcat was not picking up your local environment because it's running as root. You don't want to change root's environment. You could also, create a context.xml in you CATALINA_HOME/conf/CATALINA/localhost. You should be able to find those instruction on/through the Solr FAQ. Hope this helps. From: Bernhard Reiter [ock...@raz.or.at] Sent: Thursday, November 04, 2010 4:49 PM To: solr-user@lucene.apache.org Subject: Re: Testing/packaging question Hi, I'm now trying to export JAVA_OPTS=$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml and restarting tomcat (v6 package from ubuntu maverick) via sudo /etc/init.d/tomcat6 restart but solr still doesn't seem to find that schema.xml, as it complains about unknown fields when running the tests that require that schema.xml Can someone please tell me what I'm doing wrong -- and what I should be doing? TIA again, Bernhard Am Montag, den 01.11.2010, 19:01 +0100 schrieb Bernhard Reiter: Hi, I'm pretty much of a Solr newbie currently packaging solrpy for Debian; see http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/ In order to run solrpy's supplied tests at build time, I'd need Solr to know about the schema.xml that comes with the tests. Can anyone tell me how do that properly? I'd basically need Solr to temporarily recognize that schema.xml without permanently installing it -- is there any way to do this, eg via environment variables? TIA Bernhard Reiter
Re: Testing/packaging question
Hi, I'm now trying to export JAVA_OPTS=$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml and restarting tomcat (v6 package from ubuntu maverick) via sudo /etc/init.d/tomcat6 restart but solr still doesn't seem to find that schema.xml, as it complains about unknown fields when running the tests that require that schema.xml Can someone please tell me what I'm doing wrong -- and what I should be doing? TIA again, Bernhard Am Montag, den 01.11.2010, 19:01 +0100 schrieb Bernhard Reiter: Hi, I'm pretty much of a Solr newbie currently packaging solrpy for Debian; see http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/ In order to run solrpy's supplied tests at build time, I'd need Solr to know about the schema.xml that comes with the tests. Can anyone tell me how do that properly? I'd basically need Solr to temporarily recognize that schema.xml without permanently installing it -- is there any way to do this, eg via environment variables? TIA Bernhard Reiter
RE: Testing/packaging question
I believe it should point to the directory above, where conf and lib are located (though I have a multi-core setup). Mine is set to: /usr/local/jboss-5.1.0.GA/server/solr/solr_data/ And in solr_data the solr.xml defines the two cores, but in each core directory, is a conf, data, and lib directory, which contains the schema.xml. -Original Message- From: Bernhard Reiter [mailto:ock...@raz.or.at] Sent: Thursday, November 04, 2010 3:49 PM To: solr-user@lucene.apache.org Subject: Re: Testing/packaging question Hi, I'm now trying to export JAVA_OPTS=$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml and restarting tomcat (v6 package from ubuntu maverick) via sudo /etc/init.d/tomcat6 restart but solr still doesn't seem to find that schema.xml, as it complains about unknown fields when running the tests that require that schema.xml Can someone please tell me what I'm doing wrong -- and what I should be doing? TIA again, Bernhard Am Montag, den 01.11.2010, 19:01 +0100 schrieb Bernhard Reiter: Hi, I'm pretty much of a Solr newbie currently packaging solrpy for Debian; see http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/ In order to run solrpy's supplied tests at build time, I'd need Solr to know about the schema.xml that comes with the tests. Can anyone tell me how do that properly? I'd basically need Solr to temporarily recognize that schema.xml without permanently installing it -- is there any way to do this, eg via environment variables? TIA Bernhard Reiter DISCLAIMER: This electronic message, including any attachments, files or documents, is intended only for the addressee and may contain CONFIDENTIAL, PROPRIETARY or LEGALLY PRIVILEGED information. If you are not the intended recipient, you are hereby notified that any use, disclosure, copying or distribution of this message or any of the information included in or with it is unauthorized and strictly prohibited. If you have received this message in error, please notify the sender immediately by reply e-mail and permanently delete and destroy this message and its attachments, along with any copies thereof. This message does not create any contractual obligation on behalf of the sender or Law Bulletin Publishing Company. Thank you.
RE: Testing/packaging question
You need to either add that to catalina.sh or create a setenv.sh in the CATALINA_HOME/bin directory. Then you can restart tomcat. So, setenv.sh would contain the following: export JAVA_HOME=/path/to/jre export JAVA_OPTS==$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml If you were setting the export in your own environment and then issuing the restart, tomcat was not picking up your local environment because it's running as root. You don't want to change root's environment. You could also, create a context.xml in you CATALINA_HOME/conf/CATALINA/localhost. You should be able to find those instruction on/through the Solr FAQ. Hope this helps. From: Bernhard Reiter [ock...@raz.or.at] Sent: Thursday, November 04, 2010 4:49 PM To: solr-user@lucene.apache.org Subject: Re: Testing/packaging question Hi, I'm now trying to export JAVA_OPTS=$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml and restarting tomcat (v6 package from ubuntu maverick) via sudo /etc/init.d/tomcat6 restart but solr still doesn't seem to find that schema.xml, as it complains about unknown fields when running the tests that require that schema.xml Can someone please tell me what I'm doing wrong -- and what I should be doing? TIA again, Bernhard Am Montag, den 01.11.2010, 19:01 +0100 schrieb Bernhard Reiter: Hi, I'm pretty much of a Solr newbie currently packaging solrpy for Debian; see http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/ In order to run solrpy's supplied tests at build time, I'd need Solr to know about the schema.xml that comes with the tests. Can anyone tell me how do that properly? I'd basically need Solr to temporarily recognize that schema.xml without permanently installing it -- is there any way to do this, eg via environment variables? TIA Bernhard Reiter
RE: Testing/packaging question
The thing is, I only have a schema.xml -- no data, no lib directories. See the tests subdirectory in the solrpy package: http://pypi.python.org/packages/source/s/solrpy/solrpy-0.9.3.tar.gz Bernhard Am Donnerstag, den 04.11.2010, 15:59 -0500 schrieb Olson, Ron: I believe it should point to the directory above, where conf and lib are located (though I have a multi-core setup). Mine is set to: /usr/local/jboss-5.1.0.GA/server/solr/solr_data/ And in solr_data the solr.xml defines the two cores, but in each core directory, is a conf, data, and lib directory, which contains the schema.xml. -Original Message- From: Bernhard Reiter [mailto:ock...@raz.or.at] Sent: Thursday, November 04, 2010 3:49 PM To: solr-user@lucene.apache.org Subject: Re: Testing/packaging question Hi, I'm now trying to export JAVA_OPTS=$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml and restarting tomcat (v6 package from ubuntu maverick) via sudo /etc/init.d/tomcat6 restart but solr still doesn't seem to find that schema.xml, as it complains about unknown fields when running the tests that require that schema.xml Can someone please tell me what I'm doing wrong -- and what I should be doing? TIA again, Bernhard Am Montag, den 01.11.2010, 19:01 +0100 schrieb Bernhard Reiter: Hi, I'm pretty much of a Solr newbie currently packaging solrpy for Debian; see http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/ In order to run solrpy's supplied tests at build time, I'd need Solr to know about the schema.xml that comes with the tests. Can anyone tell me how do that properly? I'd basically need Solr to temporarily recognize that schema.xml without permanently installing it -- is there any way to do this, eg via environment variables? TIA Bernhard Reiter DISCLAIMER: This electronic message, including any attachments, files or documents, is intended only for the addressee and may contain CONFIDENTIAL, PROPRIETARY or LEGALLY PRIVILEGED information. If you are not the intended recipient, you are hereby notified that any use, disclosure, copying or distribution of this message or any of the information included in or with it is unauthorized and strictly prohibited. If you have received this message in error, please notify the sender immediately by reply e-mail and permanently delete and destroy this message and its attachments, along with any copies thereof. This message does not create any contractual obligation on behalf of the sender or Law Bulletin Publishing Company. Thank you.
RE: Testing/packaging question
Thanks for your instructions. Unfortunately, I need to do all that as part of my package's (python-solrpy) build procedure, so I can't change any global configuration, such as in the catalina subdirectories. I've already sensed that restarting tomcat is also just too system-invasive and would include changing its (system-wide) configuration. Are there any other ways to use solr for running the tests from http://pypi.python.org/packages/source/s/solrpy/solrpy-0.9.3.tar.gz without having to change any system configuration? Maybe via a user Tomcat instance such as provided by the tomcat6-user debian package? Thanks for your help! Bernhard Am Donnerstag, den 04.11.2010, 16:15 -0500 schrieb Turner, Robbin J: You need to either add that to catalina.sh or create a setenv.sh in the CATALINA_HOME/bin directory. Then you can restart tomcat. So, setenv.sh would contain the following: export JAVA_HOME=/path/to/jre export JAVA_OPTS==$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml If you were setting the export in your own environment and then issuing the restart, tomcat was not picking up your local environment because it's running as root. You don't want to change root's environment. You could also, create a context.xml in you CATALINA_HOME/conf/CATALINA/localhost. You should be able to find those instruction on/through the Solr FAQ. Hope this helps. From: Bernhard Reiter [ock...@raz.or.at] Sent: Thursday, November 04, 2010 4:49 PM To: solr-user@lucene.apache.org Subject: Re: Testing/packaging question Hi, I'm now trying to export JAVA_OPTS=$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml and restarting tomcat (v6 package from ubuntu maverick) via sudo /etc/init.d/tomcat6 restart but solr still doesn't seem to find that schema.xml, as it complains about unknown fields when running the tests that require that schema.xml Can someone please tell me what I'm doing wrong -- and what I should be doing? TIA again, Bernhard Am Montag, den 01.11.2010, 19:01 +0100 schrieb Bernhard Reiter: Hi, I'm pretty much of a Solr newbie currently packaging solrpy for Debian; see http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/ In order to run solrpy's supplied tests at build time, I'd need Solr to know about the schema.xml that comes with the tests. Can anyone tell me how do that properly? I'd basically need Solr to temporarily recognize that schema.xml without permanently installing it -- is there any way to do this, eg via environment variables? TIA Bernhard Reiter
RE: Testing/packaging question
You can setup your own tomcat instance which would contain just configurations you need. You won't even have to recreate all the tomcat configuration and binaries, just the ones that were not defaults. So, if you lookup multiple tomcat configuration instance (google it), and then you'll have a set of directories. You'll need to have your own startup script that points to your configurations. You can use the current startup script as a model, then in your build procedures (I've done all this with a script) have this added to the system so you can preform restart. You'd have to have a couple of other environment variables set: export CATALINA_BASE=/path/to/your/tomcat/instance/conf/files export CATALINA_HOME=/path/to/default/installation/bin/files export SOLR_HOME=/path/to/solr/dataNconf Good luck From: Bernhard Reiter [ock...@raz.or.at] Sent: Thursday, November 04, 2010 5:49 PM To: solr-user@lucene.apache.org Subject: RE: Testing/packaging question Thanks for your instructions. Unfortunately, I need to do all that as part of my package's (python-solrpy) build procedure, so I can't change any global configuration, such as in the catalina subdirectories. I've already sensed that restarting tomcat is also just too system-invasive and would include changing its (system-wide) configuration. Are there any other ways to use solr for running the tests from http://pypi.python.org/packages/source/s/solrpy/solrpy-0.9.3.tar.gz without having to change any system configuration? Maybe via a user Tomcat instance such as provided by the tomcat6-user debian package? Thanks for your help! Bernhard Am Donnerstag, den 04.11.2010, 16:15 -0500 schrieb Turner, Robbin J: You need to either add that to catalina.sh or create a setenv.sh in the CATALINA_HOME/bin directory. Then you can restart tomcat. So, setenv.sh would contain the following: export JAVA_HOME=/path/to/jre export JAVA_OPTS==$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml If you were setting the export in your own environment and then issuing the restart, tomcat was not picking up your local environment because it's running as root. You don't want to change root's environment. You could also, create a context.xml in you CATALINA_HOME/conf/CATALINA/localhost. You should be able to find those instruction on/through the Solr FAQ. Hope this helps. From: Bernhard Reiter [ock...@raz.or.at] Sent: Thursday, November 04, 2010 4:49 PM To: solr-user@lucene.apache.org Subject: Re: Testing/packaging question Hi, I'm now trying to export JAVA_OPTS=$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml and restarting tomcat (v6 package from ubuntu maverick) via sudo /etc/init.d/tomcat6 restart but solr still doesn't seem to find that schema.xml, as it complains about unknown fields when running the tests that require that schema.xml Can someone please tell me what I'm doing wrong -- and what I should be doing? TIA again, Bernhard Am Montag, den 01.11.2010, 19:01 +0100 schrieb Bernhard Reiter: Hi, I'm pretty much of a Solr newbie currently packaging solrpy for Debian; see http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/ In order to run solrpy's supplied tests at build time, I'd need Solr to know about the schema.xml that comes with the tests. Can anyone tell me how do that properly? I'd basically need Solr to temporarily recognize that schema.xml without permanently installing it -- is there any way to do this, eg via environment variables? TIA Bernhard Reiter
Re: Testing/packaging question
Hi, don't know if the python package provides one but solrj offers to start solr embedded (|EmbeddedSolrServer|) and setting up different schema + config is possible. for this see: https://karussell.wordpress.com/2010/06/10/how-to-test-apache-solrj/ if you need an 'external solr' (via jetty and java -jar start.jar) while tests running see this: http://java.dzone.com/articles/getting-know-solr Regards, Peter. Hi, I'm pretty much of a Solr newbie currently packaging solrpy for Debian; see http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/ In order to run solrpy's supplied tests at build time, I'd need Solr to know about the schema.xml that comes with the tests. Can anyone tell me how do that properly? I'd basically need Solr to temporarily recognize that schema.xml without permanently installing it -- is there any way to do this, eg via environment variables? TIA Bernhard Reiter
Testing/packaging question
Hi, I'm pretty much of a Solr newbie currently packaging solrpy for Debian; see http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/ In order to run solrpy's supplied tests at build time, I'd need Solr to know about the schema.xml that comes with the tests. Can anyone tell me how do that properly? I'd basically need Solr to temporarily recognize that schema.xml without permanently installing it -- is there any way to do this, eg via environment variables? TIA Bernhard Reiter
