On Sat, 14 Jan 2006 13:44:45 +0100, Andrea Arcangeli <[EMAIL PROTECTED]> wrote:
On Fri, Jan 13, 2006 at 07:32:21PM +0200, Tristan Seligmann wrote:
I don't follow this reasoning. Twisted totally depends on Python, should
Twisted go in the Python SVN repository? Should Python go in the Twisted
SVN repository?
It's only a matter of size.
No... it isn't?
Different open source projects have different requirements, and different
audiences. They also have different release cycles and different expectations
of maturity. Finally they have different release managers and different
pressures which affect releases.
If it wasn't because of the checkout size (both for space it takes, and
network bandwidth it requires), it could as well be all in a single SVN.
But those are significant sized project so there is different people
working on them, so splitting make administration a bit simpler.
Exactly the same things are true of Divmod and Twisted; Divmod, like Twisted,
contains several projects (Nevow Axiom Epsilon ...), it's just that Divmod
started out with the goal of having them well-separated, whereas Twisted didn't
realize that until many years into its life.
But this is not true for epsilon, epsilon a is a few bytes, nevow is 14M
Twisted is 51M. Furthermore if epsilon contains twisted fixes needed by
divmod to become productive with old twisted version there's no chance
that the fine Nevow can depend on such a dirty package that workaround
twisted bugs.
Epsilon only contains bug-fixes that have already been contributed to Twisted.
It only installs them if a package specifically requires a fix. If Nevow wants
to require unreleased fixes to Twisted, Epsilon would be one way to go about
it; rather than requiring users to upgrade to unreleased versions of Twisted so
that we can do a release of Nevow, Nevow's developers can simply drop in the
appropriate Twisted fix, and then use it. The fix will only be enabled when it
is requested, so it will not conflict with other libraries which depend on the
buggy behavior unless they are being used by the same process.
If such a thing were necessary, Nevow could quite easily depend upon it. That
probably will not be a requirement until Nevow is mature enough to have a 1.0
release, however.
One way to avoid this would be for you to volunteer to be the release manager
of Twisted and be very responsive to release requests. If arbitrary Divmod
project releases could easily depend upon Twisted releases being made very
quickly, a work-around package would be less important.
I have one twisted-workaround lib in my app, but the whole point of this
lib is __not__ to require people to run apt get. If they require to run
apt get to install epsilon to workaround twisted bugs, then they should
have run apt get to upgrade twisted instead!!!
That requires that every version of Epsilon (and Nevow, and Axiom) have a
paired release of Twisted.
The release manager for Twisted (Christopher Armstrong) knows the Divmod team
well, and vice versa. He certainly wouldn't mind doing us a favor every so
often.
However, when we need to publish a release of Divmod, perhaps for some internal goal,
perhaps because one of our contracting clients requires it, we don't employ Chris. We
can't call him up and say, "do a release now, becuase it is your job". Maybe
Twisted can't release the appropriate subproject for some reason - a release goal hasn't
been met, several buildslaves have been taken offline for days' worth of maintenance -
whatever.
The whole point of a workaround-lib that fixes bugs in older twisted
releases is to avoid running apt get at all and to run the app, so to
_remove_ dependencies, not to add new ones. The only possible reason one
can want to workaround twisted bugs is to remove dependencies, there's
no chance one would ever add a epsilon dependency to install with apt to
workaround a twisted bug. In suse we even use binary diff deltas to
optimize the upgrade of big packages, just replace epsilon with a good
package manager that supports diff deltas (dunno if apt supports that).
Before we get to apt or yast or whatever, we have to produce upstream releases.
Producing an upstream release of Twisted is substantially more work than
producing an upstream release of a Divmod product, even Nevow. Producing a
release of Epsilon requires at most an hour or two of work.
Furthremore Nevow is a fine open source project, it's not Divmod
product. If Divmod want to customize Nevow to run on older Twisted
release they have to _fork_ it, the same way the distro fork the kernel
to customize it for their customers (i.e. freeze and stabilize 2.6 for
months, backport stric fixes etc..). You can't screw the official nevow
repository at your commercial gain. That would be something I'm against,
because I only care about the code, so if code becomes worse in any way
due divmod commercial constraints or schedule, that concerns me a lot.
I don't care where those packages are located.
Hahahahahahahaha.
You may be confused by the fact that Nevow discussion takes place on this list,
because it is a popular Twisted-capable web framework. However: go to
nevow.com. Where does that take you? What is the *first word* on the page
that you see? What is the SVN URL for Nevow? Are you detecting a theme yet?
I really appreciate the open-source contributions from the Nevow community. They have been
substantial, and it certainly wouldn't be the product it is today if we hadn't had them.
However, its full name is "*Divmod* Nevow". Divmod has paid for every stage of
its development, and written the vast majority of its code (>80%). Its primary author,
Donovan Preston, stopped working on it when he left Divmod, and maintenance has been taken
over by JP Calderone, who currently works -- surprise, surprise -- at Divmod.
However I'd prefer Twisted and Nevow to be in the same repository, be it
divmod.org or twistedmatrix.com. Apparently the .com one is the
no-profit community one, and the .org one is the commercial one, weird.
Yeah, the story behind that is kind of weird. twistedmatrix.com is an older
domain, that I bought when I was 16 or so, and *everybody* was registering .com
names. Domain squatters have long since purchased .org and .net. When Divmod
was registered, we got the .org for our open-source works and the .com for our
commercial service.
So apparently the .com one is more appropriate for nevow as well.
No, it isn't, because Divmod develops Nevow, and the Twisted team develops
Twisted.
Then you can fork Nevow from twistedmatrix.com into divmod.org and
customize it with epsilon (or you can create a branch called divmod into
the twistedmatrix.com to add the epsilon dependency to fix bugs).
None of this fixes the release-management problem I've described repeatedly,
which doesn't seem to concern you at all. I understand that you don't use a
package manager, or even downloaded tarballs of released version of projects,
but that is an extreme minority position. In many companies where it is
allowed to use open source at all, it is *strictly* disallowed to use
un-released and/or unpackaged versions on production machines.
You all have checkin rights in both, so I don't see why should you care
too. If you want more stuff into divmod.org to get the credit of its
development that's fine with me, merge Twisted into divmod.org too then,
if that can help writing better code and to remove any interest in
creating divmod packages like epsilon that workaround twisted bugs.
Not true. Moe, for example, has checkin rights to Divmod and not Twisted.
There are plenty of people who have commit access to Twisted who would not even
want to work on any Divmod projects. I certainly don't think that all Twisted
committers should have automatic commit access to Divmod, and I definitely
don't think that Twisted committers would be happy if a job at Divmod
automatically resulted in getting Twisted commit with no feedback from the rest
of the community.
I'm very serious when I say this, it's a matter of perception, you can't
base Nevow on a tiny package that totally depends on twisted and that
even includes twisted workarounds. Perhaps if that was a _stable_ branch
of nevow, that would be remotely acceptable. But like every open source
project out there, the _trunk_ must be aligned for the long term, and
definitely not for the short term gain of some commercial entity.
It's worth noting, by the way, that we *did* honor the community's request to
elide that dependency when it wasn't providing any significant benefit, and
Nevow doesn't depend on Epsilon yet. If and when it does, there will be a good
reason.
Many mistakes happened already, the obsolescence of too many apis is the
proof. I understand that one always want to learn from its own mistakes
and that you won't believe others until you scratched you head on it,
but you may as well decide to trust me and not to add silly dependencies.
This argument boils down to "you have made mistakes (unrelated to what we are
discussing) in the past, so you should not take any actions which may have consequences,
because they too might be mistakes". If I listened to this, I would never do
anything, I would be afraid to even get out of bed in the morning.
The important thing is not yast2 -i or apt get that will get everything
right automaticaly, the important thing is that "python setup.py" will
just work. Requiring a "python setup.py install" of a tiny package is
silly.
So - you want the convenience of a package manager (installations that work
instantly and with no manual intervention to install dependencies) but you also
want to be on the absolute bleeding edge of SVN development?
Sorry, but you can't have it both ways. Packaging systems are a lot of work.
Dependencies are hard to manage. Communication between projects is hard -
Divmod and Twisted are managed by very similar and very friendly groups of
people (which overlap significantly) and even *that* communication is hard.
If you want this kind of ease, help us produce eggs, or debs, or improve our installation
instructions. I thought this thread began as a helpful (albeit misguided) suggestion,
but it is quickly devolving into a simple whine: "why doesn't divmod do everything
the way I want?"
Let's say you did convince everyone involved that this was a really good idea,
and the various repositories should be merged. It is still probably several
weeks of work to do this. What about bugtrackers? What about repository
performance (doubling the size of the repository in one commit is not going to
make SVN happy). What about merge history on outstanding branches?
Note that there is people out there that must be in violent agreement
with me. Please try to run setup.py install of zope3, does it ask for
anything? No, it just runs. Then go in src/twisted. Hey surprise,
there's twisted in there. That twisted copy takes 28M on my harddisk,
not 700K like zope.interface. To avoid a dependency they waste 28M of
HD, and yet you refuse to copy zope.interface into twisted repository
and you require an external checkout or rpm install (and I believe you're
losing users because of that, again it's a matter of perception and
solidity and self-containment of the project). Not to tell about epsilon
that is just a few bytes.
Twisted already includes a zope.interface tarball in sumo releases. The
expectation is that if you are working from the repository, you should not be
doing setup.py install anyway - that pollutes your installed directory with an
unspecified development version.
Your main point, that people are in "violent agreement" with, is that obstacles
to installation should be removed. The Divmod team agrees with that too, which is why
Epsilon and Nevow are *already* in the same repository, and in fact, if you use
Combinator to set them up, they will both be in you sys.path automatically.
In the event that we did make Nevow depend upon Epsilon, it would take you less
time to contribute all the appropriate patches to bundle a release of Epsilon
inside Nevow, or copy it during setup.py install from an SVN export, than it
has to write all the messages in this thread. I suggest you conserve your
energy and do that.
Epsilon repository should be deleted, the good parts of it must be
merged into twisted repository as twisted.epsilon or twisted.python or
whatever twisted.*, and the bug workarounds must be dropped. This is
about the community development, I don't care if you fork one of those
packages for your needs, but you can only mess in the divmod-fork, you're not
allowed to screwup the trunk developed by the community. In the
community developed one the fixes must go into mainline twisted, not in
a epsilon package!
1. Epsilon isn't a separate repository, it's a package within Divmod trunk.
2. "You're not allowed to screwup the trunk developed by the community?" Pardon me, but
we are an integral part of the community. 'svn blame' does not seem to think that *you* are. This
really had me laughing. "You are not allowed to commit things I do not like to your projects,
Divmod! Go to your room!"
3. The Twisted fixes, which Divmod developers we put versions into Epsilon *to make
installing Divmod projects easier*, which is something that you want. Do you even have
any idea what these fixes are? Have you read any of the code you are claiming to be such
an expert on, that we should "trust you" about?
This as long as you think a community really exists around twisted and
nevow, and that you're not the only users of twisted and nevow. Your
behaviour would be perfectly ok if Divmod was the only nevow user.
We do not always see eye-to-eye with the community on what is best for Nevow, but other
productive contributors (who do not have anything to do with Divmod) have already
disagreed with you publicly. They do not seem to have a problem with our
"behavior".
Note that I'm overall very happy with the development happening, sadly
when I have to send emails is because I think something is going wrong,
time is limited and when things goes well one doesn't need to send
emails.
Thank you kindly.
But as much as I like the development, sometime I see some
frightening nosenses too (new formless api wasted effort, atop wasted
effort, many twisted api going obsolete) and this epsilon thing is one
of those nosense. So this time I'm trying to help before you scratch
your head into it by yourself and before you risk to scare people away
from the project. And again, there is people who understand things that
lives very close to you, you just have to look at the zope folks merging
twisted in to avoid external checkouts to understand this, how can
explain the fact they merged twisted and they waste 28M of disk and
bandwidth when you refuse to merge a few bytes into twisted just becasue
it's not a divmod.org?
Since you keep repeating yourself, I will keep repeating myself to make sure you hear the
answer: Twisted fixes are merged to Twisted before they are replicaetd in Epsilon. There
is only one such fix so far. Read epsilon.hotfix for details. If Epsilon discovers that
it is running on a Twisted version with the appropriate fix already made, it does
nothing. The hotfix facility is just a way for bits of Divmod code to say "I really
need this particular bugfix/feature in library XXX, called YYY, but you do not need to
upgrade that library to SVN"
I cannot care less who gains what and what url I have to checkout, I
only care about code and there's no chance I can agree with an epsilon
dependency if it's the nevow trunk you're talking about, you're free to
add it in a nevow-divmod trunk, that's perfect!
So please trust me on this. Thanks!
Andrea, I have a lot of respect for you, but in this case you quite literally
do not know what you are talking about. You also haven't expressed yourself
very clearly: what is it you want exactly? *Why* is an Epsilon dependency a
problem? The reasons you have given -- it's harder to install, I can't run it
from an SVN checkout, I have to use apt, Divmod is not giving changes back to
Twisted -- are all wrong. Sometimes they are confusing and wrong, sometimes
they are insulting and wrong, but there are very few actual facts you have
offered. Your suggestions for a remedy are also extremely labor-intensive on
the part of the Twisted community, the Nevow community, and Divmod. While we
would not implement these changes either way, it would have been a nice show of
good faith for you to offer to do some of the work required yourself.
What started this discussion was MG objecting to an unnecessary dependency. He
was correct: every dependency has a cost, and therefore there must be a benefit
to establishing the dependency that exceeds that cost. The change being
discussed was basically zero benefit, so even if the cost was tiny it should
not be introduced. I understood this criticism, I agreed with it, and I
corrected the mistake. What you're saying, though, is that adding a dependency
on a project which is small on disk is the WORST thing that ANY project can do
EVER. That is simply not true. You are objecting to a hypothetical dependency
which is being introduced for some hypothetical reason, which since it is
hypothetical, could be totally awesome.
I know you also think the zope.interface dependency was a bad idea - again, you
find yourself in a tiny minority. Everyone who knew the old Twisted components
system seems to agree that the Zope people did a *much* better job and it is a
great thing that we are using their components system. Everyone who knew
Twisted pre-interfaces seems to agree that introducing them (even the crappy,
pre-zope interfaces) simplified a lot of things and improved the documentation.
_______________________________________________
Twisted-web mailing list
[email protected]
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
