On Tue, Jan 04, 2011 at 12:38:09AM +0100, Diez Roggisch wrote: > -----Ursprüngliche Nachricht----- > Von: "Toshio Kuratomi" <[email protected]> > Gesendet: 03.01.2011 23:59:23 > An: [email protected] > Betreff: Re: [TurboGears] Re: TurboGears joins the PylonsProject > > >On Mon, Jan 03, 2011 at 09:36:22PM +0100, Diez Roggisch wrote: > >> >Distribution is one of the axis considered for choosing a development > >> >platform. We have heard from multiple consumers that egg files > >> >deployment and management is not acceptable and they would consider > >> >the product only if it is available in .rpm or .deb package format. > >> > > >> >Even from developers point of view, being able to get an environment > >> >where they can start getting their hands wet is also important. (See > >> >wiki updates on documents on the install page the number of updates > >> >and workarounds for people to get TG2 stack installed) > >> > >> We have developers up and running (if they have a working linux) in under > >> half an hour. > >> > >> And of course a DEB or RPM is nice, if that's the way you chose to deploy. > >> But that has nothing to do with my point. You can certainly create a DEB > >> containing a full versionset, with dependend eggs. The problem is to try & > >> use the shipped dependencies of your distro. That's a desaster waiting to > >> happen. > >> > >Generalizations are bad because they ignore that there's different needs > >that different strategies are geared for and the different tools that > >underly those strategies. > > This discussion is in the context of TG2 development and deployment. > Yes, and all of the examples I've given are ways to develop and deploy TG. Because there are many different ways that you can deploy depending on the circumstances that occur. Whereas your harping on the idea that deploying on distro packages is unreasonable and a sure recipe for disaster is just false.
> > > >If you're creating a web application for use inside of an organization where > >you're also the full-time sysadmin for the service, virtualenvs may be the > >best way to manage that. > > I'm not the sysadmin. But I am of course working together with him. And he's > quite happy with the solution so far :) > <nod> Yep, this is definitely a valid way to deploy for certain scenarios. > > > >If you're handing off to a dedicated IT department that deploys to a stable > >system (one where API compatibility is ensured even when packages are > >updated), building to use the system packages that will be available on that > >system may be a requirement imposed by IT. > > It may be a requirement imposed by IT, but then it's not or only very hard > fulfillable in the context of TG2. > > Take the current ubuntu LTS (certainly a preferred choice for setting up a > production system today). It contains zope.sqlalchemy, which is needed by TG2: > > https://launchpad.net/ubuntu/lucid/+package/python-zope.sqlalchemy > > The stable (LTS!!!!) version here is 0.4-7. Now let's take a look what a > recent TG2 installation of a small project of mine: > > tequila:home deets$ ll > ~/.virtualenvs2.5/wordpuzzle/lib/python2.5/site-packages/ | grep zope.sqlalch > drwxr-xr-x 4 deets staff 136 Oct 31 22:05 > zope.sqlalchemy-0.6-py2.5.egg/ > > Bummer. Two minor version numbers higher. > > So I can't go with the distro's version. I possibly event can't chose to not > install it, as it might be a dependency of some other tool that needs in in > that very version. > > My solution here is to use a virtualenv, with --no-site-packages option to > prevent sys.path pollution. > > Which is yours? > Actually, using the system packages is precisely what I do in my deployments. The two targets that I have at the moment are RHEL5 with the EPEL5 repositories and the current latest Fedora releases. Which just proves that in some circumstances, it is possible to achieve just this. > > >If you're developing an application that will be released for people in many > >different organizations to run (for example Drupal or mediawiki [Is it bad > >that all the apps that I can think of that fit this role off the top of my > >head are php? :-( ]) then you have to write code that can deal with a wide > >variety of different versions since the route most users will take to get > >this software will be what's packaged for their operating system. > > > >A little unrelated, putting all of the eggs you depend on into a single > >deb/rpm really is a recipe for disaster as it means that you need to manage > >the builds for all of the packages should you need to make changes to just > >one. It's much better to take the packages from your distro and modify them > >to suit your needs and then create a package repository to pull your packages > >from. That way you: 1) Gain the benefit of seeing how your system > >maintainers have built their packages to work around quirks that they've > >discovered in each of the packages. 2) When you discover something that you > >must fix, you're able to fix only that one package and issue an update > >rather than updating everything in the rpm, possibly introducing unexpected > >changes. > > You see, all of this is fun & nice on a meta-level. But so far, whenever this > discussion has crept up here or on c.l.py or wherever, it was just on exactly > that level. I OTOH have to solve (and solved) the actual problem. I'm happy > to be convinced that there is a better way that is more conformant with the > OS dependency management. > The OS dependency management is really not what I'm trying to be illusttrative of here. I'm trying to show that deploying a single rpm of all your dependencies is problematic compared to deploying with an rpm for each of your dependencies. This is not a comparison of system package managers vs virtualenvs. > > So far, the only alternative I've been pointed to is using zc.buildout - > which goes even *further* in not only providing eggs, but instead building > whole libraries, including python itself. > I haven't looked at zc.buildout, so I wouldn't know... > So given the above example, how would you go about this? > > - start creating a DEB package I work with rpm distros but the basics are the same. I'm going to write down what I would do here based on the needs of my system administrators. However, if your environment is not as strict, there's shortcuts that can be taken in various places. I can talk about that a little as well. Let's say that I'm developing for RHEL5 and the SQLAlchemy version there is sqlalchemy-0.3 (in fact, that version comes from the EPEL repository but the difference in policy there is minimal). Now the hot new app that I'm developing is going to make use of features in SQLAlchemy-0.6 so what am I going to do? 1) get the source of that package: git clone git://pkgs.fedoraproject.org/python-sqlalchemy 2) Update the package to build SQLAlchemy 0.6 instead. In this case, I am even lucky enough that the Fedora version of the package (which is SQLAlchemy-0.6) is easily comparable to the EPEL-5 version so I can simply merge many of the changes from the Fedora version to the EPEL-5 version. 3) Modify the spec to build a parallel installable version instead. In the case of SQLAlchemy, the recipe would look something like this: - Name: python-sqlalchemy + Name: python-sqlalchemy0.6 %build - CFLAGS="%{optflags}" %{__python} setup.py --with-cextensions build + CFLAGS="$RPM_OPT_FLAGS" %{__python} setup.py--with-cextensions bdist_egg %install - %{__python} setup.py --with-cextensions install --skip-build --root %{buildroot} + easy_install -m --prefix %{buildroot}%{_usr} dist/*.egg 4) Build the package with something like:: rpmbuild -ba python-sqlalchemy0.6.spec If I needed to build for RHEL5 but I wasn't on RHEL5, I'd get a free account that can use the Fedora BuildSystem for scratch builds and use that:: koji build --scratch dist-5E-epel python-sqlalchemy0.6-0.6.1-1.src.rpm 5) Here's one of hte points where shortcuts can be taken. I personally would then get this package into EPEL so that others could benefit from having this package available. But if you didn't want to do that and your site policies allow it, you can also create a private repository where you can host the package and pull it in via the system package managers. > - manage to install it without provoking a conflict with the system's version In either case, once Step 5 above is complete, installation is: yum install -y python-sqlalchemy0.6 There will be no conflicts because of the way we installed took care to use eggs to install in parallel. > - manage to have it picked up when starting the TG2-app which requires it > If developing the app still or the deployment method is using paster:: 1) make sure SQLAlchemy >= 0.6 is in setup.py 2) python setup.py egg_info 3) cp /usr/bin/paster . 4) edit ./paster so the __requires__ line looks like this: __requires__ = ['PasteScript==1.7.3', 'appname'] 5) ./paster development.ini If deploying via mod_wsgi, edit the app.wsgi script so it has this at the top: import __main__ __main__.__requires__ = 'appname' import pkg_resources > Bonus points for pointing out how you still benefit from the original > zope.sqlalchemy package being under distro control, getting potential bug and > security fixes (for the otherwise outdated version). > With this setup, the original python-sqlalchemy package continues to be under distro control, getting the updates that the distro deems necessary. -Toshio
pgp4yUk4qKgoc.pgp
Description: PGP signature
