Re: Thoughts on production deployment?
There's an RPM project on GitHub that comes close: https://github.com/boogieshafer/jetty-solr-rpm On Fri, Feb 1, 2013 at 6:19 AM, Michael Della Bitta michael.della.bi...@appinions.com wrote: When I was referring to the different version of Jetty, I meant Jetty Plus, which the wiki mentions. Is this no longer true? My Chef recipe makes assumptions about the OS and EBS volumes being available, which can easily be fixed. Michael Thanks for jumping in guys. I agree the SolrJetty page needs just a little updating -- I commented at the bottom of SOLR-3159 about that. Michael and Paul, are your chef and ant recipes generic enough to share? My next install is going to be on RHEL 6, so I can take a crack at an install script that'll work there. It wouldn't be hard to translate between shell and chef. Michael: The problem with adding a dependency on Jetty in your chef recipe is that it's going to grab whatever version of Jetty was blessed by the distro maintainers on your target platform. -- Scott Stults | Founder Solutions Architect | OpenSource Connections, LLC | 434.409.2780 http://www.opensourceconnections.com
Re: Thoughts on production deployment?
Thanks for jumping in guys. I agree the SolrJetty page needs just a little updating -- I commented at the bottom of SOLR-3159 about that. Michael and Paul, are your chef and ant recipes generic enough to share? My next install is going to be on RHEL 6, so I can take a crack at an install script that'll work there. It wouldn't be hard to translate between shell and chef. Michael: The problem with adding a dependency on Jetty in your chef recipe is that it's going to grab whatever version of Jetty was blessed by the distro maintainers on your target platform.
Re: Thoughts on production deployment?
When I was referring to the different version of Jetty, I meant Jetty Plus, which the wiki mentions. Is this no longer true? My Chef recipe makes assumptions about the OS and EBS volumes being available, which can easily be fixed. Michael Thanks for jumping in guys. I agree the SolrJetty page needs just a little updating -- I commented at the bottom of SOLR-3159 about that. Michael and Paul, are your chef and ant recipes generic enough to share? My next install is going to be on RHEL 6, so I can take a crack at an install script that'll work there. It wouldn't be hard to translate between shell and chef. Michael: The problem with adding a dependency on Jetty in your chef recipe is that it's going to grab whatever version of Jetty was blessed by the distro maintainers on your target platform.
Thoughts on production deployment?
Part of this is a rant, part is a plea to others who've run successful production deployments. Solr is a second-class citizen when it comes to production deployment. Every recipe I've seen (RPM, DEB, chef, or puppet) makes assumptions that in one way or another run afoul of best-practices when it comes to production use. And if you're not using one of these recipe formats to deploy Solr you're building a SnowflakeServer (Martin Fowler's term). Granted, Solr _can_ be deployed into any vanilla JEE container, so the deployment spec responsibility may be erroneously assigned to whichever you choose. BUT, if you want to get the maximum out of Solr you'll want to put it on its own box, running in its own tuned container, and that container should be the one that Solr's been tested on repeatedly by an army of build bots. Right now that blessed container is Jetty version 8.1.2.v20120308. So the first problem with the recipes is that they make a generic dependency of Jetty or Tomcat. The assumption there is that either can be treated as a generic OS facility to be shared with other apps. That's not true because Solr is the driving force behind which version is deployed. The container can't be up- or downgraded without affecting Solr, and any other app running in there needs to be aware that Solr is taking first priority. The next problem is that most recipes don't make a distinction between collections. Solr configuration goes in one folder, Solr data goes in another, and the logs and container stuff gets scattered likewise. In reality, every collection can be configured differently and there is no generic Solr data. Lastly, the package maintainers of all the major OS distributions have ignored Solr since around version 1.4. That means if you want a newer version you're going to download a tarball and make another snowflake. This might be attributable to thinking of Solr as just another web app that doesn't need special packaging. Regardless, the consequence is that the only people who are deploying Solr according to best-practices are those intimately familiar with Solr. So what's the best way to fix this situation? Solr already ships with everything it needs except Java and a start-up script. Maybe the first step is to include a generic install.sh script that has a couple distro-specific support scripts. That would be fairly agnostic toward package management systems and it would be useful to sysadmins right away. It would also help package maintainers update their build specs. What do _you_ think? -Scott
Re: Thoughts on production deployment?
On Thu, Jan 31, 2013 at 5:13 AM, Scott Stults sstu...@opensourceconnections.com wrote: Right now that blessed container is Jetty version 8.1.2.v20120308. I'd really like some confirmation from the devs that there really is a blessed status for a given container that provides advantages over others. From what I understand, Jetty's considered one option out of many, and isn't considered to be head and shoulders above any other. We have a Chef regime here, and I've written Tomcat and Solr recipes to be played against Ubuntu 12.04 Server. I chose Tomcat mostly because I have the most experience administrating and configuring it, and I would assume familiarity with operations would be a pretty important factor. Michael Della Bitta Appinions 18 East 41st Street, 2nd Floor New York, NY 10017-6271 www.appinions.com Where Influence Isn’t a Game
Re: Thoughts on production deployment?
We have a Chef regime here, and I've written Tomcat and Solr recipes
to be played against Ubuntu 12.04 Server.
We do mostly the same: chef to install Tomcat (with configuration
appropriate to Solr), but then instead of deploying Solr via chef, we use
an ant script to package and deploy a war that includes Solr + some custom
Lucene extensions, then also deploy our {schema,solrconfig}.xml files. This
is a little easier for us than doing everything in chef, since we can more
easily push updates to our custom extensions.
I'll also note that our current process mirrors what we do for the
front-end app server (written in Ruby). We use chef to set up the box, then
Capistrano to deploy the app. We push app updates several times a week, but
rarely need to run chef after the initial setup.
But I'd love to know if there is an easier way to do it.
Paul
--
_
Pulchritudo splendor veritatis.
Re: Thoughts on production deployment?
On Jan 31, 2013, at 10:15 AM, Michael Della Bitta michael.della.bi...@appinions.com wrote: I'd really like some confirmation from the devs that there really is a blessed status for a given container that provides advantages over others. IMO: jetty is what all of our unit/integration tests are run in, jetty is what we configure to work well out of the box and add workarounds to, jetty is what the devs run, jetty is very likely what most of the users run simply because we ship with it, most of the bug reports we get around containers involve jetty (because of the previous most likely). I'd say jetty doesn't get anymore blessed than that. If you want to run another container, fine, but I would pick jetty myself - specifically, the one we ship with without darn good reason. - Mark
Re: Thoughts on production deployment?
That's surprising to me, mostly because a number of the Solr wiki pages don't really make that strong of a case for it: http://wiki.apache.org/solr/SolrInstall http://wiki.apache.org/solr/SolrTomcat http://wiki.apache.org/solr/SolrJetty Would it make sense to spell that out somewhere? I do notice that it seems like the version of Jetty that ships with Solr isn't the preferred one according to the wiki, so that would be an extra dependency for a config management system like Chef. Are there any other configuration choices that are blessed like this? JDK versions or sources (oracle vs. open), for example? Thanks, Michael Michael Della Bitta Appinions 18 East 41st Street, 2nd Floor New York, NY 10017-6271 www.appinions.com Where Influence Isn’t a Game On Thu, Jan 31, 2013 at 5:07 PM, Mark Miller markrmil...@gmail.com wrote: On Jan 31, 2013, at 10:15 AM, Michael Della Bitta michael.della.bi...@appinions.com wrote: I'd really like some confirmation from the devs that there really is a blessed status for a given container that provides advantages over others. IMO: jetty is what all of our unit/integration tests are run in, jetty is what we configure to work well out of the box and add workarounds to, jetty is what the devs run, jetty is very likely what most of the users run simply because we ship with it, most of the bug reports we get around containers involve jetty (because of the previous most likely). I'd say jetty doesn't get anymore blessed than that. If you want to run another container, fine, but I would pick jetty myself - specifically, the one we ship with without darn good reason. - Mark
Re: Thoughts on production deployment?
On 1/31/2013 3:21 PM, Michael Della Bitta wrote: I do notice that it seems like the version of Jetty that ships with Solr isn't the preferred one according to the wiki, so that would be an extra dependency for a config management system like Chef. Near as I can tell, the versions of jetty that shipped with 4.0 (8.1.2) and 4.1 (8.1.7) are unmodified. The config is somewhat specialized, but Jetty itself is not changed. I upgraded my 4.1-SNAPSHOT install to 8.1.7 before the committers did without any problems. The Jetty 6 versions included with 1.x and 3.x releases were patched for one or more bugs - the upstream package from mortbay wouldn't be the right thing to use. I have a RHEL/CentOS-friendly jetty init script with config options that can be overridden by a file in /etc/sysconfig. I could probably also come up with one for Debian (sysvinit). The newest Fedora releases use systemd, but systemd is backward compatible with RHEL/CentOS init scripts. Outside of these distributions, I know very little. Recent Ubuntu releases use upstart, about which I am completely clueless. If there's interest, I can make my init script more generic, make one for debian, and try to come up with an installation script to go with it. If someone knows upstart, they can use my work as a base. Thanks, Shawn
