Re: [Zope-dev] Packaging Zope for Fedora

[Andreas Jung]

--On 27. März 2008 20:42:50 +0200 Marius Gedminas <[EMAIL PROTECTED]> wrote:


On Wed, Mar 26, 2008 at 09:20:27PM +0100, Dieter Maurer wrote:

Timothy Selivanow wrote at 2008-3-25 17:12 -0700:
> ...
> Now when I say "rip out", I don't mean repackage (make a sub RPM), I
> mean remove from the RPM that I am making.  I don't want to provide a
> "new" Docutils.

That Zope ships with its own "Docutils" comes from the fact
that the standard one has a big security hole.


Which one?  The one that lets you embed any file on the filesystem into
a web page?

  http://docutils.sourceforge.net/docs/howto/security.html

I didn't know Zope's bundled version of docutils fixed that.  In any
case, the src/docutils in the Zope 3.2 tree either doesn't have the fix,
or it doesn't work.  I tested it and ended up closing that hole in an
application myself.


At least Zope 2 uses Docutils with the related options disabled. No
idea about Zope 3.2.

-aj

pgpK98dCDd36X.pgp
Description: PGP signature
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Packaging Zope for Fedora

[Marius Gedminas]

On Wed, Mar 26, 2008 at 09:20:27PM +0100, Dieter Maurer wrote:
> Timothy Selivanow wrote at 2008-3-25 17:12 -0700:
> > ...
> >Now when I say "rip out", I don't mean repackage (make a sub RPM), I
> >mean remove from the RPM that I am making.  I don't want to provide a
> >"new" Docutils.
> 
> That Zope ships with its own "Docutils" comes from the fact
> that the standard one has a big security hole.

Which one?  The one that lets you embed any file on the filesystem into
a web page?

  http://docutils.sourceforge.net/docs/howto/security.html

I didn't know Zope's bundled version of docutils fixed that.  In any
case, the src/docutils in the Zope 3.2 tree either doesn't have the fix,
or it doesn't work.  I tested it and ended up closing that hole in an
application myself.

Marius Gedminas
-- 
Alan Turing thought about criteria to settle the question of whether
machines can think, a question of which we now know that it is about
as relevant as the question of whether submarines can swim.
-- Dijkstra


signature.asc
Description: Digital signature
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Packaging Zope for Fedora

[Dieter Maurer]

Timothy Selivanow wrote at 2008-3-25 17:12 -0700:
> ...
>Now when I say "rip out", I don't mean repackage (make a sub RPM), I
>mean remove from the RPM that I am making.  I don't want to provide a
>"new" Docutils.

That Zope ships with its own "Docutils" comes from the fact
that the standard one has a big security hole.
Security issues are harmless for normal "Docutils" use
but are critical when they can be exploited from a hostile
internet



-- 
Dieter
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Packaging Zope for Fedora

[Jens Vagelpohl]

On Mar 26, 2008, at 01:12 , Timothy Selivanow wrote:
I would rather have a package that upstream approves of and  
participates

in, than blindly create it in a silo not caring.  It never turns out
well in the long run with the latter.


You may be fighting windmills here. Previous attempts at creating  
packaged versions, like RPMs, have flopped badly.


On the one hand they tend to be behind the current release quickly. On  
the other hand they tend to introduce their own idea of how things  
should be laid out, which invariably clashes with what the developers/ 
supporters use. They tend to introduce scripts and scaffolding that do  
not exist in the standard tarball, which will lead to blank stares,  
such as start/stop scripts. Also, the goals that are cited for a  
packaged version, such as reproduceability of a given setup, are  
reached by the other installation mechanisms that are used as well, be  
it manually building a tarball or using helpers like zc.buildout.


What it all boils down to (and has boiled down to in the past) is that  
people will be told to abandon the prepackaged version as soon as they  
need any kind of support on the "official" zope mailing lists. The  
notion that you build Zope (and in most cases Python as well) yourself  
instead of using anything pre-packaged is so deeply ingrained that no  
one has ever created a successful distribution-supplied package. Don't  
get me wrong, I'm not making any kind of value judgment, I'm just  
telling you what to expect. You may already be in the situation you  
are trying to avoid because the upstream may never approve of and  
support what you're attempting to do.


jens


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Packaging Zope for Fedora

[Timothy Selivanow]

On Sat, 2008-03-22 at 08:05 +0100, Andreas Jung wrote:
> In general we want to get rid of 3rd-party packages we don't want to 
> maintain on our own.

That's perfectly sane :)  There is no sustainable reason to lug around
some other project's code in yours in the Open Source world.

> Right now we have several 3rd-party packages like 
> Docutils in our svn for several reasons. Some of those packages are 
> basically frozen (because we made local changes e.g. for security reasons).
> So if you rip out those packages you have to guarantee that they will 
> _never_ be updated by a new official package (because local changes might 
> get lost). On the other hand you must ensure that changes to our local
> 3rd-party packages will be available through your packaging mechanism.
> As said: this is a challenging task for us right now - it will be even more 
> challenging and more error-prone for outsiders.

Now when I say "rip out", I don't mean repackage (make a sub RPM), I
mean remove from the RPM that I am making.  I don't want to provide a
"new" Docutils.  I'm actually trying to place zope in the standard (for
Redhat) python site-packages location while guaranteeing that the zope
RPM doesn't provide more than it needs, and doesn't clobber other
packages, like the native Docutils.

If I knew what versions the 3rd party pieces were, I could make a diff
(or the Zope project could provide one) and I could get those into the
official Fedora RPMs (esp. if they are security fixes).  Fedora doesn't
like keeping those patches forever, so I would also get those pushed
upstream.

> The trend in all Zope-related projects (Zope 2,3, Grok, Plone) is 
> definitely: self-contained installation. Why self-contained? Different 
> projects require different modules in different versions.

That is completely understandable from a Service Provider standpoint,
but not a distro.  However, coming from a Service Provider background
(my day job, as it were), I definitely like reproducibility, and a
distro package (i.e. RPM) is a good place to start.  It's very annoying
and time consuming to replicate by hand the environment that is going to
appear on several servers, esp. over long periods of time (not to
mention upgrades/security, etc...)

> Messing a Python 
> installation with e.g. ZODB from different versions will end up in an 
> disaster. So the message of the Zope community is in general: keep your 
> Python installation clean and use zc.buildout or virtualenv for installing
> your other modules within an isolated environment. This is meanwhile 
> considered best-practice. Using Python eggs and easy_install you have _the
> official_ tools in your hand for installing 3rd-party packages..that's the 
> Pythonic way to do it, not using some distro packaging tool.

One could argue that the distro over-rides all.  In a distro role, your
job is to provide the least common denominator, not cater to every crazy
developer and their environment.  They should be more than capable of
maintaining that themselves (install several different versions of
python, zope, et al. in their home dir if need be).

> This was never my point. For me it would make more sense that the packages 
> would take the official release and re-package it as a whole without 
> splitting it up. Dependency management should happen on our side as a whole,
> not on the distribution site.

Yes, but I still have to meet those dependencies.  I was just hoping
that there would be pieces (sounds like ZODB  and zope.publisher would
be good candidates from some of the other threads) that would be useful
outside of the whole Zope Application Server scope.  I assumed that as a
(open source) software project, Zope would be delighted that people were
using their code in new and creative ways.  I wanted to help facilitate
that by offering those pieces as a stand-alone library.

> I don't know how important Zope distro packages are for users. My personal
> impression is: not so important.

Developers, yes, not very important.  /Users/ on the other hand are a
completely different animal, and packages are _very_ important.  I
suppose it's a matter of semantics on what exactly a user is in which
scope.

> Zope packages at this point might be good for getting into touch with Zope 
> but 
> for deployment packages are basically no used - but correct me if I should 
> be wrong. Since a while we have tools in place to bootstrap Zope 2/3 
> projects, Plone installations and Grok installations very easily and in a 
> reliable way.

It matters in an unattended, (e.g.) 100+ server enviro where
reproducibility is a must.

> I know that those approaches don't comply directly with your 
> package mechanisms but I would expect that the "offical" installation and 
> deployment stories by the Zope community should be taken into account (in 
> some way).

Absolutely.  Those are important.  I would be delighted to read some of
those if someone knew/had some.

> One last thought: you might check out the pac

Re: [Zope-dev] Packaging Zope for Fedora

[Andreas Jung]

--On 21. März 2008 13:39:08 -0700 Timothy Selivanow 
<[EMAIL PROTECTED]> wrote:



On Fri, 2008-03-21 at 20:09 +0100, Andreas Jung wrote:

Hi,

speaking as the Zope 2 release manager: I am strongly opposed against
splitting Zope yourself into different packages and modules. With Zope
2 depending from various Zope 3 packages (roughly 80-90) we have
already the situation to keep track which packages belong together.


My concern is more with striping out the non-Zope pieces.  If there are
pieces of Zope that are useful outside of Zope (e.g. in some other
project, or someones personal code...) I thought it would be nice to
offer those as a separate piece so that someone could use just that
piece.  Having 80-90 different RPMs _would_ be rather unmaintainable.
I'd only pull out the first few useful ones, and then others by-demand.
My biggest concern is removing the non-Zope parts.


In general we want to get rid of 3rd-party packages we don't want to 
maintain on our own. Right now we have several 3rd-party packages like 
Docutils in our svn for several reasons. Some of those packages are 
basically frozen (because we made local changes e.g. for security reasons).
So if you rip out those packages you have to guarantee that they will 
_never_ be updated by a new official package (because local changes might 
get lost). On the other hand you must ensure that changes to our local

3rd-party packages will be available through your packaging mechanism.
As said: this is a challenging task for us right now - it will be even more 
challenging and more error-prone for outsiders.




As far as Zope2 is concerned, I'll cross that bridge later.  Newest
technology is /generally/ the focus of Fedora.


This will become even more complicated when each Zope 3 package will have
its own life-cycle. I doubt that you as a package can keep track in a
reliable way with those requirements. It is somewhat hard for me to
follow.


The Eclipse project has done an awesome job, and so has Fedora in
keeping up with it (they actually work very closely together).  I'm not
concerned with ripping Zope /completely/ apart, but rather I
thought /some/ parts might be useful by themselves, like the ZODB for
example.


Since Zope (2+3) are officially only blessed for Python 2.4 you'll have 
problems if you're going with the newest Python 2.5 or even 2.6 version.
The trend in all Zope-related projects (Zope 2,3, Grok, Plone) is 
definitely: self-contained installation. Why self-contained? Different 
projects require different modules in different versions. Messing a Python 
installation with e.g. ZODB from different versions will end up in an 
disaster. So the message of the Zope community is in general: keep your 
Python installation clean and use zc.buildout or virtualenv for installing
your other modules within an isolated environment. This is meanwhile 
considered best-practice. Using Python eggs and easy_install you have _the
official_ tools in your hand for installing 3rd-party packages..that's the 
Pythonic way to do it, not using some distro packaging tool.







So why can't you leave the Zope source packages as they are? Splitting up
Zope into even more packages is even more error-prone.  Re-packaged Zope
distributions always raised more problems in the past than they really
solved. The deployment for Zope-based applications is nowadays based on
zc.buildout.


So, are you saying that you would rather Zope not be in Fedora, or any
other distro, and just leave it up to the user?


This was never my point. For me it would make more sense that the packages 
would take the official release and re-package it as a whole without 
splitting it up. Dependency management should happen on our side as a whole,
not on the distribution site. I don't know how important Zope distro 
packages are for users. My personal impression is: not so important. Zope 
packages at this point might be good for getting into touch with Zope but 
for deployment packages are basically no used - but correct me if I should 
be wrong. Since a while we have tools in place to bootstrap Zope 2/3 
projects, Plone installations and Grok installations very easily and in a 
reliable way. I know that those approaches don't comply directly with your 
package mechanisms but I would expect that the "offical" installation and 
deployment stories by the Zope community should be taken into account (in 
some way).






Serious solutions providers never go with distribution-specific packages.


Some do, but that's not the focus of me packaging Zope.  Since Fedora's
focus is not for production (yes, people do use Fedora in production,
e.g. NASA, but that is their onus), but rather development and blazing
new trails.  This would provide more code and utilities for people to
use and experiment with.



See above. Distro specific package are perhaps another change for gettting 
into touch with Zope but I am not sure how important that is. I can imagine 
that people hear of Zope technology and

Re: [Zope-dev] Packaging Zope for Fedora

[Marius Gedminas]

Disclaimer: my knowledge is not complete, and my track record of writing
emails late at night is not very good.  Take this with a big grain of
salt.

On Fri, Mar 21, 2008 at 11:57:31AM -0700, Timothy Selivanow wrote:
> As the subject implies, I'm working on a package for Fedora.  I've come
> across some missing dependencies, and some that are different versions.
> Because of the way that Python packages are put together (shared library
> environment), I'm trying to strip out everything that is not Zope
> specific (e.g. mechanize, docutils, etc.) that way Zope won't clobber
> already existing packages, and new packages won't clobber Zope.

Be careful: I wouldn't be surprised if the foreign packages in the Zope
tree have bugfixes applied to them.  I trust those fixes were also sent
to their respective upstreams as well, but it may make it a bit more
difficult to find out which upstream version is really needed.

> Here is my list (that I've discovered so far...) and some notes and
> questions.  I would appreciate any information on resolving these.  My
> list was taken from Zope-3.4.0c1/build/lib.linux-x86_64-2.5 (in my arch
> specific case...) after doing a `python install.py build` (side note,
> why is it called install.py and not setup.py?).

Good question.  It's called setup.py in the Subversion tree -- but the
tarball is not based on the old monolithic SVN tree any more.

> BTrees --
>   Does not currently exist in Fedora as a separate package.
>   Is this of Zope origin? (If not, where? Required version?)
>   If Zope origin, is it useful outside of Zope?

It is part of ZODB.  It may be useful outside of Zope.

> persistent --
>   Does not currently exist in Fedora as a separate package.
>   Is this of Zope origin? (If not, where? Required version?)
>   If Zope origin, is it useful outside of Zope?

ZODB again.

> transaction --
>   Does not currently exist in Fedora as a separate package.
>   Is this of Zope origin? (If not, where? Required version?)
>   If not Zope origin, is it useful outside of Zope?

ZODB again.

> ThreadedAsync --
>   Does not currently exist in Fedora as a separate package.
>   Is this of Zope origin? (If not, where? Required version?)
>   If Zope origin, is it useful outside of Zope?
 
ZODB again.

> ClientForm --
>   This currently exists in Fedora (version 0.2.7)
>   Diffing the directories and files reveals differences.
>   What version is required for Zope? What about newer versions?
>
> mechanize --
>   This currently exists in Fedora (version 0.1.6)
>   Diffing the directories and files reveals differences.
>   What version is required for Zope? What about newer versions?
> 

I'm pretty sure these two are only used for zope.testbrowser.  It ought
to be sufficient to take the latest upstream version and see if
zope.testbrowser's test suite passes with it.

Benji York is the one with most knowledge about zope.testbrowser and its
dependencies.  If he offers different advice, listen to him, not to me.

> twisted --
>   This currently exists in Fedora (version 2.4.0)
>   Diffing the directories reveals that is it mostly compatible,
>   however, missing some features (I have a list).
>   What parts are required by Zope?
>   Some of the missing parts are no longer being maintained, or are
>   incompatible with newest Twisted (2.5.0). Thoughts?

AFAIK twisted is only bundled with Zope 3 for convenience.  It is used
for its HTTP server as one of the alternatives (the other ones being
ZServer and WSGI), but even that part is not used by default.

> pytz --
>   This currently exists in Fedora (version 2006p)
>   Diffing the directories and files reveals differences.
>   What version is required for Zope? What about newer versions?
>   The Zope versions do not have the .py extension.  Possible
>   problems with Zope if they do (i.e., lib resolving)?

I think any non-broken pytz version should work.  (Ubuntu once shipped a
completely broken one.)

> RestrictedPython --
>   Does not currently exist in Fedora as a separate package.
>   Is this it? 
>   Required version for Zope? What about newer versions?

I honestly have no idea.

> Also, there is an existing package in Fedora called
> python-zope-interface, version 3.0.1 I assume (in F8), and I'm wondering
> if it would be worth keeping separate (or at least available, updated,
> maybe with a Conflicts: zope) in Fedora and are there other parts of
> Zope (e.g. ZODB, ZEO) that would be useful by themselves?

Yes and no.

The Zope 3 upstream is now split into hundreds of little packages
(Python eggs), each registered on the Python Package Index, with
intricate and sometimes suboptimal dependency lists.  If there's an easy
way to convert a Python egg to an RPM, you may want to look at this, but
beware: the dust hasn't fully settled yet.  

Re: [Zope-dev] Packaging Zope for Fedora

[Timothy Selivanow]

On Fri, 2008-03-21 at 20:09 +0100, Andreas Jung wrote:
> Hi,
> 
> speaking as the Zope 2 release manager: I am strongly opposed against 
> splitting Zope yourself into different packages and modules. With Zope
> 2 depending from various Zope 3 packages (roughly 80-90) we have
> already the situation to keep track which packages belong together.

My concern is more with striping out the non-Zope pieces.  If there are
pieces of Zope that are useful outside of Zope (e.g. in some other
project, or someones personal code...) I thought it would be nice to
offer those as a separate piece so that someone could use just that
piece.  Having 80-90 different RPMs _would_ be rather unmaintainable.
I'd only pull out the first few useful ones, and then others by-demand.
My biggest concern is removing the non-Zope parts.

As far as Zope2 is concerned, I'll cross that bridge later.  Newest
technology is /generally/ the focus of Fedora.

> This will become even more complicated when each Zope 3 package will have
> its own life-cycle. I doubt that you as a package can keep track in a
> reliable way with those requirements. It is somewhat hard for me to follow.

The Eclipse project has done an awesome job, and so has Fedora in
keeping up with it (they actually work very closely together).  I'm not
concerned with ripping Zope /completely/ apart, but rather I
thought /some/ parts might be useful by themselves, like the ZODB for
example.

> So why can't you leave the Zope source packages as they are? Splitting up
> Zope into even more packages is even more error-prone.  Re-packaged Zope
> distributions always raised more problems in the past than they really solved.
> The deployment for Zope-based applications is nowadays based on zc.buildout.

So, are you saying that you would rather Zope not be in Fedora, or any
other distro, and just leave it up to the user?

> Serious solutions providers never go with distribution-specific packages.

Some do, but that's not the focus of me packaging Zope.  Since Fedora's
focus is not for production (yes, people do use Fedora in production,
e.g. NASA, but that is their onus), but rather development and blazing
new trails.  This would provide more code and utilities for people to
use and experiment with.


--Tim
 __ 
/ As of next Tuesday, C will be flushed in favor of COBOL. \
\ Please update your programs. /
 -- 
  \
   \   \
\ /\
( )
  .( o ).

___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Packaging Zope for Fedora

[Andreas Jung]

Hi,

speaking as the Zope 2 release manager: I am strongly opposed against 
splitting Zope
yourself into different packages and modules. With Zope 2 depending from 
various Zope 3 packages (roughly 80-90) we have already the situation to 
keep track which packages belong together. This will become even more 
complicated when each Zope 3 package will have its own life-cycle.

I doubt that you as a package can keep track in a reliable way with those
requirements. It is somewhat hard for me to follow. So why can't you leave 
the Zope source packages as they are? Splitting up Zope into even more 
packages is even more error-prone.  Re-packaged Zope distributions always 
raised more problems in the past than they really solved. The deployment 
for Zope-based applications is nowadays based on zc.buildout. Serious 
solutions providers never go with distribution-specific packages.


Andreas



--On 21. März 2008 11:57:31 -0700 Timothy Selivanow 
<[EMAIL PROTECTED]> wrote:



As the subject implies, I'm working on a package for Fedora.  I've come
across some missing dependencies, and some that are different versions.
Because of the way that Python packages are put together (shared library
environment), I'm trying to strip out everything that is not Zope
specific (e.g. mechanize, docutils, etc.) that way Zope won't clobber
already existing packages, and new packages won't clobber Zope.

Here is my list (that I've discovered so far...) and some notes and
questions.  I would appreciate any information on resolving these.  My
list was taken from Zope-3.4.0c1/build/lib.linux-x86_64-2.5 (in my arch
specific case...) after doing a `python install.py build` (side note,
why is it called install.py and not setup.py?).

BTrees --
Does not currently exist in Fedora as a separate package.
Is this of Zope origin? (If not, where? Required version?)
If Zope origin, is it useful outside of Zope?

ClientForm --
This currently exists in Fedora (version 0.2.7)
Diffing the directories and files reveals differences.
What version is required for Zope? What about newer versions?

twisted --
This currently exists in Fedora (version 2.4.0)
Diffing the directories reveals that is it mostly compatible,
however, missing some features (I have a list).
What parts are required by Zope?
Some of the missing parts are no longer being maintained, or are
incompatible with newest Twisted (2.5.0). Thoughts?

mechanize --
This currently exists in Fedora (version 0.1.6)
Diffing the directories and files reveals differences.
What version is required for Zope? What about newer versions?

persistent --
Does not currently exist in Fedora as a separate package.
Is this of Zope origin? (If not, where? Required version?)
If Zope origin, is it useful outside of Zope?

pytz --
This currently exists in Fedora (version 2006p)
Diffing the directories and files reveals differences.
What version is required for Zope? What about newer versions?
The Zope versions do not have the .py extension.  Possible
problems with Zope if they do (i.e., lib resolving)?

RestrictedPython --
Does not currently exist in Fedora as a separate package.
Is this it? 
Required version for Zope? What about newer versions?

ThreadedAsync --
Does not currently exist in Fedora as a separate package.
Is this of Zope origin? (If not, where? Required version?)
If Zope origin, is it useful outside of Zope?

transaction --
Does not currently exist in Fedora as a separate package.
Is this of Zope origin? (If not, where? Required version?)
If not Zope origin, is it useful outside of Zope?


Also, there is an existing package in Fedora called
python-zope-interface, version 3.0.1 I assume (in F8), and I'm wondering
if it would be worth keeping separate (or at least available, updated,
maybe with a Conflicts: zope) in Fedora and are there other parts of
Zope (e.g. ZODB, ZEO) that would be useful by themselves?

I'm sorry if this seems daunting (it is on my side ;) and/or demanding,
but I would like this packaged the best possible way for both projects.
Who knows, this might show up in future RHEL versions (maybe I'm
delusional ;)


Thank you for your time and patience.

--Tim
 
_  / According to Kentucky state law, every person must take a bath at
least \ \ once a year.
/
 
-\
   \   \
\ /\
( )
  .( o ).

___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org