Re: [Distutils] setup.cfg new format proposal

[sstein...@gmail.com]

On Sep 15, 2009, at 7:52 PM, David Lyon wrote:


And imo the discussion has gone way off track..

The use case isn't abstract. All this discussion is about trying
to rewrite two lines of code.

-- setup.py --
"""
   if sys.platform == 'win32':
   setup.dependencies.add('win32com','162')

   setup()

"""

That would be the simplest way to do it in code.

It appears as if "if sys.platform == 'win32':"
is an evil line of code - that certain people
want to go to great lengths to stomp out.

It's one line of code for crying out loud...

We don't need a mini-language just because we
don't like writing it the shortest way...

talk about platform bias gone utterly crazy...


Assuming that it really is that simple...

+1 on everything above.

A 1 line change is much better than a 30 message debate, BNF diagram,  
and DSL for a simple case.  If it doesn't work out for some reason,  
some _real_ reason, debate it then in context, if it ever comes up  
again.


S


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] setup.cfg new format proposal

[sstein...@gmail.com]

On Sep 15, 2009, at 8:42 PM, exar...@twistedmatrix.com wrote:

What use cases do we have?  There's the one described above, which  
lots of people have been talking about.  I think there's another one  
related to target Python version - eg, on Python 2.3, depend on  
simplejson, but on Python 2.6, don't.  What else?


Ok, so let's work backwards from use cases to tests to code.  Sounds  
like something I heard once, somewhere...


Do we start by breaking  it down by:

Python Version

Platform

and drill down from there?

Seems to me, from there there are lots of details to handle but,  
without those big chunks out of the way, there's really no way to  
meaningfully discuss any further details.


Anyone have any more "big chunks?"

S




___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[sstein...@gmail.com]

On Oct 3, 2009, at 1:00 PM, Ned Deily wrote:


This is not a good experience for users.  Unless I'm missing something
(and I hope I am), this issue really can't be hand-waved away.


What would you suggest?

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[sstein...@gmail.com]

On Oct 3, 2009, at 4:08 PM, Lennart Regebro wrote:


2009/10/3 Ned Deily :
This is not a good experience for users.  Unless I'm missing  
something

(and I hope I am), this issue really can't be hand-waved away.


How about some sort of an announcement/warning on the setuptools site  
itself?


I know the code's not going to get updated but how about a simple  
warning and suggestion to move on to Distribute?


S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[sstein...@gmail.com]

On Oct 5, 2009, at 7:44 AM, Lennart Regebro wrote:


2009/10/5 Ronald Oussoren :
This is a problem, it means 2.6.3 is not a simple drop-in  
replacement for 2.6.2 but requires the replacement of another  
component as well.  That can be a problem in organizations with  
strict configuration management where you cannot install new  
software without going to lots of red tape (and that might involve  
lawyers when you install a new package instead of upgrading an  
existing one).


Sure, but that would have happened sooner or later anyway. Is it
really so bad that it happens now instead of say, in half a year? I
don't see why it's such a big deal. Sorry, but I don't.


It's not the happening, it's the happening without a deprecation,  
warnings, announcements etc. deserved by a change that affects so much  
of the Python eco-system.


S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[sstein...@gmail.com]

On Oct 5, 2009, at 9:38 AM, Barry Warsaw wrote:


I apologize for my part in this, but moving forward I think that if  
it's possible to patch and release a setuptools that works with  
Python 2.6.3 /and/ earlier Python 2.6.x's, then that should happen  
asap.  If that's not possible, then we might need to revert the  
distutils change in a quick Python 2.6.4.


I thought the whole point of the Distribute project was that we  
couldn't _get_ a new setuptools release and, so, had to fork.


Hopefully, with this motivation, we'll get a new setuptools "one more  
time!" and make the transition to Distribute a little more controlled.


Fast, but controlled.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[sstein...@gmail.com]

On Oct 7, 2009, at 3:18 AM, Florian Schulze wrote:


This way one - who stumbled upon the bitbucket site - does not have  
to pull the source tree and look in docs/index.rst in order to get  
the URL to the bug tracker. (I had a bug to report a couple of days).


The launchpad bugtracker is for virtualenv, not this fork. Sadly I  
just realized bitbucket.org doesn't have an integrated issue tracker  
like github :(


Uh...it's cleverly hidden under the "Issues"  tab on the project page.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[sstein...@gmail.com]

On Oct 7, 2009, at 10:51 AM, Florian Schulze wrote:



On 07.10.2009, at 16:21, sstein...@gmail.com wrote:



On Oct 7, 2009, at 3:18 AM, Florian Schulze wrote:


This way one - who stumbled upon the bitbucket site - does not  
have to pull the source tree and look in docs/index.rst in order  
to get the URL to the bug tracker. (I had a bug to report a  
couple of days).


The launchpad bugtracker is for virtualenv, not this fork. Sadly I  
just realized bitbucket.org doesn't have an integrated issue  
tracker like github :(


Uh...it's cleverly hidden under the "Issues"  tab on the project  
page.


I had to enable it in the cleverly hidden "Admin" tab first.


Told'ya it was cleverly hidden. ;-)

I forgot about that; I think I had exactly the same "Where the heck's  
the issue tracker?" moment when I started my first repository.


Why it's not enabled by default is one of the deep mysteries...

Glad you found it!

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.6c10 release imminent; please test

[sstein...@gmail.com]

On Oct 12, 2009, at 4:25 PM, P.J. Eby wrote:


But before you do that, be sure to uninstall Distribute completely


Damn if I understand this...such a long time waiting for all these bug  
fixes...so little action, so much angst...


Then that all that effort going into the Distribute fork, so much  
effort expended carefully working around so that Distribute would be a  
bug-fixed, drop-in replacement for the orphaned setuptools.


Then suddenly out of the blue, everything is fixed in setuptools 06c10  
and we must "uninstall Distribute completely" get these fixes.


If the changes you made really were superior in any way, I trust that  
Tarek and co. are smart, and egoless enough to see that and will make  
those changes part of Distribute.


Too little, too late, no thanks, I'll just be sticking with Distribute  
from now on.


S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.6c10 release imminent; please test

[sstein...@gmail.com]

On Oct 12, 2009, at 7:48 PM, P.J. Eby wrote:


At 07:28 PM 10/12/2009 -0400, sstein...@gmail.com wrote:

we must "uninstall Distribute completely" get these fixes.


That's Distribute's doing, not mine.  As I understand it, their  
package includes a 'setuptools' package, and if it's on your  
sys.path, then installing the new version of setuptools will be a no- 
op.  If they hadn't done that, there'd be no problem.  See the  
Distribute documentation.


In any case, the update is not intended for people who are happy to  
have Distribute, but the people who are unhappy about having to  
switch, or deal with its workarounds...  or just wish the whole  
discussion would go away.



Then suddenly out of the blue


It may appear sudden to you, if you haven't been reading Python- 
Dev.  There's been quite a bit of discussion about an urgent bug  
that Tarek introduced in Python 2.6.3.  It's mainly because of that  
bug that I took the time to go ahead and get a bunch of other  
pending bugs cleaned up and checked in.


No, I've been reading Python-Dev right along.

Yes, Python 2.6.3 included a change that broke a year-old, orphaned  
product, in severe need of bug fixes that lots of people just happen  
to depend on.


The fall-down was in the testing done before the Python release and  
I'm sure more testing will be done in that area before the 2.6.4  
bugfix release.


I find it kind of comical, and a little pathetic, that you think you  
can just whip out a bug fix after a year of frustration and everone's  
just going to forget history and sign up for more of the same.


As Alex said in response to my previous message:

Too little, too late, no thanks, I'll just be sticking with  
Distribute from now on.
Several developers and an open development process vs a lone coder  
with a closed codebase? That's not really a choice at all...


Sorry, it doesn't look like anyone wants to play with you any more;  
you can just keep your ball.


Maybe you could submit some patches into the open Distribute process.   
Now _that_ would be helpful.


S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] the virtualenv-distribute mess

[sstein...@gmail.com]

On Oct 13, 2009, at 3:15 AM, Reinout van Rees wrote:


On 2009-10-12, P.J. Eby  wrote:

At 08:09 AM 10/12/2009 +, Reinout van Rees wrote:
OTOH, grumbl ... horrible breakage ... essential piece of  
infrastructure ...

allowed to persist  I'm pretty grumpy right now.


Relax, take a deep breath, and then easy_install setuptools==dev or
setuptools==dev06.  ;-)


Relax, take a deep breath, go to #distutils in irc and be amazed at  
the speed

by which Tarek is able to fix things.

I'm helping Tarek debug stuff now so that I can switch to  
distribute :-)


Yes, now that people are allowed to help, things should improve rapidly.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Setuptools 0.6c10 release imminent; please test

[sstein...@gmail.com]

On Oct 13, 2009, at 3:32 AM, Jeff Rush wrote:
Too little, too late, no thanks, I'll just be sticking with  
Distribute

from now on.
Several developers and an open development process vs a lone coder  
with

a closed codebase? That's not really a choice at all...

Sorry, it doesn't look like anyone wants to play with you any more;  
you

can just keep your ball.


Please don't speak for others - it's rude.  Only speak for yourself.


I apologize -- I didn't mean to speak for anyone who hasn't already  
spoken.  I was specifically speaking to the "Several developers..."


 Thank you for telling me how to behave, which is not rude at all.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Proposal for Distribute 0.7

[sstein...@gmail.com]

 that there are other people on the Distribute team who I'd  
seriously consider as committers on setuptools or even as a chief  
maintainer of the setuptools 0.6 line (if not more).

...asked me who those team members are


I'm asking.  Who are they and, if you are willing to give them access,  
have you offered it and they've declined, or you were waiting  
until...what?   You had Tarek's acknowledgement or permission?


We, regular Python users, have been asking for you to let someone help  
with setuptools for years since you obviously have other priorities  
and the various issues in setuptools have affected many of us in  
various ways.


Are you really willing to let anyone help?  Really?

Do it, then, and let's just let this asinine infighting go down in  
history as one of the stupidest things a language community has ever  
done to its own credibility in public and move on.


Not having an evolving, comprehensive packaging system, that is  
keeping up with the ever-changing requirements of our daily work is  
really hurting the language, the community, and us, as working Python  
programmers, every day.


Thanks,

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Proposal for Distribute 0.7

[sstein...@gmail.com]

On Oct 15, 2009, at 3:55 PM, P.J. Eby wrote:

[a whole bunch of stuff...]

Understood and I've observed the public portions of all of this.  Much  
of it has not been pretty.


Personality conflicts aside, I wonder if it would be possible to move  
some portion of the base of both setuptools and Distribute to some  
sort of shared foundation.


That would reduce the amount of duplicate work that I can see coming  
out of this divergence and, perhaps, provide a basis on top of which  
setuptools and Distribute can continue to evolve.


This kind of "shared utility" project could, perhaps, be undertaken by  
someone that the two team leaders both know and trust and could become  
a shared resource instead of having the two teams competing for the  
extremely valuable (and severely limited) pool of people you would  
trust to work on this.


The thing that has bothered me for years is that, while the code  
that's being installed is generally of higher quality in the Python  
world, the tools for doing so in the Ruby world have pulled ahead and  
now do a better job of installing code that's not as good.


Unfortunately, that can give the Ruby solution a visible head start  
that often times can't be overcome in the minds of management,  
regardless of the relative quality of the resultant products.


Thanks,

S



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Proposal for Distribute 0.7

[sstein...@gmail.com]

Warning: 3 part message.

It's all related so I decided to bundle it rather than starting  
several new threads on the already over-burdened distutils list.


On Oct 15, 2009, at 5:06 PM, Tarek Ziadé wrote:


Now, ssteinerX, what do you mean exactly by shared foundation ?


Well, at the moment, setuptools 6.10rc and Distribute 0.6.6 share only  
one identical directory; tests. (tests in the root directory which  
only contains shlib_test which I'm not even sure is used by anyone)


Many other files are only different by a few lines, some additions to  
Distribute are just for Python 3 compatibility.


Some of the changes are only in the way message text is distributed  
over a couple of lines (the kind of clean-up people naturally do as  
they're assimilating a product they didn't write), some extra  
comments, sometimes the wording (Couldn't vs. Could not) of an error  
message.


Some of the files (e.g. setuptools/archive_util.py) are only different  
by a bit of whitespace.


In taking a closer look, the two products are still very close to each  
other but it would take a careful person a solid few days or a week to  
merge them back together into a single code-base, if that would even  
be desirable.


I had thought going forward would be to take some of the utility type  
functionality that is unlikely to change (archive_util.py for example)  
and move them to a shared utility library.


This is probably unrealistic given the dynamics of the current  
situation, but it was a nice idea ;-).


Thanks,

S

P.S.

While I was in there, I did a little actual work.  Not much, but now I  
feel like I've made a contribution, however small... ;-).   Does this  
make me a "contributor?"


This is from the archive pulled down by:

# wget 
http://pypi.python.org/packages/source/d/distribute/distribute-0.6.6.tar.gz

While I was pawing around, I noticed that, in distribute-0.6.6/ 
setuptools/command/alias.py, line 12, the <> operator is used.


Since that's not allowed in Python 3.x, the test suite must not cover  
this 'cause Python 3 would barf.


So...I whipped out my command line and:

find . -name "*.py" -exec grep "<>" {} \; -print

I get:

(~/src/distribute-0.6.6/setuptools)# find . -name "*.py" -exec grep  
"<>" {} \; -print

if arg.split()<>[arg]:
if self.remove and len(self.args)<>1:
./command/alias.py
if safe is None or bool(safe)<>flag:
./command/bdist_egg.py
if self.verbose<>self.distribution.verbose:
if k<>'target_version':
if len(parts)<>2 or not name.endswith('.pth'):
./command/easy_install.py
str(version)<>"unknown" and version >=  
self.requested_version

./depends.py
if p<>package and not p.startswith(pfx)
if p<>package and not p.startswith(pfx)
if p.name<>package and not p.name.startswith(pfx)
./dist.py
if cs.hexdigest()<>info[4:]:
./package_index.py

Those aren't going to fly in Python 3 and can be safely removed for  
2.x as well.  A patch would be silly, just go fix it ;-).


Thanks,

S

P.S.S.

Part of the problem we'd have with merging the two products to form a  
single codebase is that there are some differences between the ways  
the tickets have been closed in the two products.


Distribute's fixes are committed against the bugs in its tracker so  
it's pretty easy to see what change fixed what.  Not so much in  
setuptools where many things are in one big mega-commit.


The latest commit you've made in setuptools package is still  
cryptic to us because

it fixes many things at once.


Sorry about that, but it was pretty much the only way to get it done  
in a weekend, without the overheads of separate commit messages, doc  
changes, and backporting killing me.  Those overheads were the main  
reason I wasn't making changes more often; I dreaded the amount of  
work involved.


Also, as it happened, I was able to fix multiple problems with  
single changes this way.


I don't mean to take another pot-shot a PJE, but the facts is the  
facts and this shows clearly why the product with the open development  
process should be the one supported by the community.


Also, I've made it pretty plain for a long time that if Ian Bicking  
or Jim Fulton were ever willing to take it over, I'd hand it *all*  
over -- 0.7 as well as 0.6 -- and happily retire from the  
distribution tools business.


I'd love to see this be the last release of setuptools so if you're  
the aforementioned "Ian" or "Jim", could you please, please, very  
temporarily, accept the burden of "setuptools maintainer" so it can  
die an elegant, compatible, controlled, peaceful death, being merged  
into and supplanted by Distribute after long and good service to the  
Python world?


We really need a good, modern, open, constantly evolving packaging  
system to keep Python competitive.


Thanks,

S

___
Distutils-SIG maill

Re: [Distutils] Proposal for Distribute 0.7

[sstein...@gmail.com]

On Oct 16, 2009, at 4:24 AM, Tarek Ziadé wrote:


On Fri, Oct 16, 2009 at 3:39 AM, sstein...@gmail.com
 wrote:
Some of the files (e.g. setuptools/archive_util.py) are only  
different by a

bit of whitespace.


archive_util, in the mid term (2.7/3.2) will dissapear, in favor of
changes in Distutils side.


Ok, it was just a datapoint, not a suggestion that it be retained.

In taking a closer look, the two products are still very close to  
each other
but it would take a careful person a solid few days or a week to  
merge them
back together into a single code-base, if that would even be  
desirable.


We had a "setuptools-compatible" branch for a while when PJE said he
might use our work to release setuptools, but that didn't happened  
and we moved on.

[please no flame here,it's just a fact : it didn't happen. period]

At  this point, the code base is backward compatible, but is slightly
changing yes (we are working in bug fixes so..)


Ok, after reviewing the code, I can see that maintaining "common code"  
would not actually be a good idea because it's not worth retaining.


While I was in there, I did a little actual work.  Not much, but  
now I feel
like I've made a contribution, however small... ;-).   Does this  
make me a

"contributor?"


Yes absolutely. Then if you keep them coming and you will be able the
have a contributor access.


Thanks.  At the moment, I, frankly, wouldn't touch that code with a 10  
foot pole.  I had never actually sat down to do anything resembling a  
"code review" on the setuptools code and, after doing so, I really  
think it should just be scrapped as quickly as possible.  Dragging any  
of it along, for whatever reason, is just going to slow progress.  The  
test coverage is pretty much non-existent and, without that, trying to  
figure out the intent of this code is pointlessly difficult.  It'd  
just be easier to write the tests, write the code, and move on.



As, for the quality of the code, the current quality is very low in
many ways, so, for obvious bug patches, we trust anyone that is
already running a project succesfully, and we are counting on the fact
that many eyes are now watching the changesets being done.

Remember that 0.6.x is bugfix only.


Understood.


The only hard part in setuptools is understanding what the code is
doing because it's hard to read, overdesigned and overcomplex and with
giant monolithic modules.


I don't know that I'd call what I saw "designed."   Complex, yes, but  
design implies a plan and known direction, communicated by design  
documents with tests that prove the code is working as expected.


This is not designed by any objective standard I'd use.

Maybe PJE is some sort of savant who can keep the myriad details of a  
piece of code like this in his head but "design" is not that.



This is from the archive pulled down by:

   # wget
http://pypi.python.org/packages/source/d/distribute/distribute-0.6.6.tar.gz

While I was pawing around, I noticed that, in
distribute-0.6.6/setuptools/command/alias.py, line 12, the <>  
operator is

used.

Those aren't going to fly in Python 3 and can be safely removed for  
2.x as

well.  A patch would be silly, just go fix it ;-).


Thanks ! we will fix it. The code base is not well covered in the
tests, so this problem was not tackled. Notice that the latest  
released, if run under Python
3, will convert in the fly the code, using 2to3, so make sure the  
tests are run after 2to3 (that's

what is supposed to happen)


Yes, I'm sure 2to3 can fix it, but the less work it has to do, the  
fewer chances for mistakes in the automatic conversion.  Not that I'd  
expect this to break something but having the 2to3 log less cluttered  
with simple things like this that can be easily done by hand cuts down  
on the amount of stuff that has to be reviewed in case of a problem.



If you find anything else, please come in #distutils (freenode)


Will do...

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Proposal for Distribute 0.7

[sstein...@gmail.com]

On Oct 16, 2009, at 11:41 AM, P.J. Eby wrote:


At 10:46 AM 10/16/2009 -0400, sstein...@gmail.com wrote:

I don't know that I'd call what I saw "designed."   Complex, yes, but
design implies a plan and known direction, communicated by design
documents with tests that prove the code is working as expected.

This is not designed by any objective standard I'd use.


You are correct; setuptools itself was not particularly designed.   
Eggs are, entry points are, a few other odds and ends were designed,  
but setuptools itself (and *especially* easy_install) are a  
collection of hacks upon hacks, mostly done in a rush to get  
something out the door...  which as soon as it became popular,  
locked into a cycle of not having enough time to do improvements,  
because maintenance was taking so much time.


Yup.  Understood.  It's a vicious circle/cycle that I'm sure we've all  
been trapped in at one time or another; just not quite so publicly or  
with such a large impact on so many people.  And to do it for free,  
besides!  Betcha' you'll think thrice next time...and say nah, no  
thanks...



Maybe PJE is some sort of savant who can keep the myriad details of a
piece of code like this in his head


Nope.  Why do you think I've not been thrilled about working on  
it?  ;-)


Welp...here's to hoping you're freed from that burden partially of  
your own making if not choosing...


S



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [venv] Re: two questions about virtualenv

[sstein...@gmail.com]

On Oct 16, 2009, at 12:25 PM, Brandon Craig Rhodes wrote:


Jannis Leidel  writes:


That'd be very much appreciated, I'm sure the experience of the
distribute people would be helpful, too.


I'm not sure I quite approve of their approach, since it makes
installing Distribute under Python 3 take a very long time - which,  
as a
result, currently makes creating a virtualenv under Python 3 take a  
very

long time as well.  Someone was complaining to me about that just
earlier this week.


Doesn't that also mean that the Python 3 version of Distribute is  
generated live on the installation system?  That doesn't seem right  
because that leaves the possibility that the code generated by 2to3 is  
different from the code that passed the (hypothetical, at this point)  
tests before the distribution was created.


I'm going to cross-post this to the distutils list as well.  If the  
idea is to stabilize Distribute, get some tests in place, etc. then  
this method of distribution  (running 2to3 in situ, as it were) is not  
the correct way to do it from a "shipping the tested product"  
perspective.


S


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [venv] Re: two questions about virtualenv

[sstein...@gmail.com]

On Oct 16, 2009, at 12:59 PM, Lennart Regebro wrote:
I'm going to cross-post this to the distutils list as well.  If the  
idea is
to stabilize Distribute, get some tests in place, etc. then this  
method of
distribution  (running 2to3 in situ, as it were) is not the correct  
way to

do it from a "shipping the tested product" perspective.


I don't agree with that statement.


This is not analogous to running a C compiler, it's analogous to  
running a pre-processor maybe, but the fact that the distribution  
doesn't run the tests on the exact code delivered in the package gives  
me agita.


S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [venv] Re: two questions about virtualenv

[sstein...@gmail.com]

On Oct 16, 2009, at 2:09 PM, Tres Seaver wrote:


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

sstein...@gmail.com wrote:

On Oct 16, 2009, at 12:59 PM, Lennart Regebro wrote:

I'm going to cross-post this to the distutils list as well.  If the
idea is to stabilize Distribute, get some tests in place, etc.  
then this
method of distribution  (running 2to3 in situ, as it were) is not  
the correct

way to do it from a "shipping the tested product" perspective.

I don't agree with that statement.


This is not analogous to running a C compiler, it's analogous to
running a pre-processor maybe, but the fact that the distribution
doesn't run the tests on the exact code delivered in the package  
gives

me agita.


You can't run tests on an sdist:  at a minimus, you have to build it  
and
run the tests with the build directory on your path.  Tests run  
inside a
"develop" egg (a VCS checkout) are always potentially influenced by  
the

environment they are run in, which is why we need buildbots (or the
equivalent) for each supported platform.


Buildbots seem to be in short supply  though they should be easy  
enough to come by in this day and age of penny-an-hour cloud computing.


I think I'll see if maybe I can just whip up something about this...

S







___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [venv] Re: two questions about virtualenv

[sstein...@gmail.com]

On Oct 16, 2009, at 3:56 PM, exar...@twistedmatrix.com wrote:
Buildbots seem to be in short supply  though they should be easy  
enough to come by in this day and age of penny-an-hour cloud  
computing.


I think I'll see if maybe I can just whip up something about this...


Not sure if you're aware, but the latest version of buildbot  
includes support for managing ec2-based slaves.  That's an easy way  
to get all the slaves you can afford.


No, I did not know that, thanks!

Thing is, with ec2's high prices, the number of servers I could afford  
to put up would be pretty limited.  I was thinking more along the  
lines of options at Rackspace and others (@ Rackspace, for example,  
1.5 cents/hr for 256MB 10 GB server).


Maybe the best thing to do is just extend buildbot to handle other  
server types where the prices aren't quite so steep.


Thanks for the heads up!

S



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Buildbot throwing a gear after default setup?

[sstein...@gmail.com]

I've just downloaded and configured the Distribute buildbot and it  
appears that a dependency is missing somewhere.  I get this (just the  
errors excerpted).


Just wanted to capture this before I figure out what needs to be done...

test_make_tarball  
(distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped  
'requires zlib'
test_make_zipfile  
(distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped  
'Requires zlib'
test_tarfile_root_owner  
(distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped  
'Requires zlib'
test_tarfile_vs_tar  
(distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped  
'Requires zlib'


==
ERROR: test_make_archive_owner_group  
(distutils.tests.test_archive_util.ArchiveUtilTestCase)

--
Traceback (most recent call last):
  File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ 
Lib/distutils/tests/test_archive_util.py", line 224, in  
test_make_archive_owner_group

group=group)
  File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ 
Lib/distutils/archive_util.py", line 236, in make_archive

filename = func(base_name, base_dir, **kwargs)
  File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ 
Lib/distutils/archive_util.py", line 163, in make_zipfile

compression=zipfile.ZIP_DEFLATED)
  File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ 
Lib/zipfile.py", line 675, in __init__

"Compression requires the (missing) zlib module"
RuntimeError: Compression requires the (missing) zlib module

--
Ran 158 tests in 1.242s

FAILED (errors=1, skipped=11)
test test_distutils crashed -- : [Errno 2]  
No such file or directory

Traceback (most recent call last):
  File "Lib/test/regrtest.py", line 767, in runtest_inner
  File "Lib/test/regrtest.py", line 723, in __exit__
  File "Lib/test/regrtest.py", line 686, in get_cwd
OSError: [Errno 2] No such file or directory
1 test failed:
test_distutils


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Buildbot throwing a gear after default setup?

[sstein...@gmail.com]

On Oct 16, 2009, at 6:58 PM, Nathan Yergler wrote:


Looks like you're using a build of Python that wasn't compiled with
zlib support (--with-zlib, IIRC).


Right -- that's the python that the buildbot compiled from source,  
that's why it's a bug ;-).


S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Buildbot throwing a gear after default setup?

[sstein...@gmail.com]

On Oct 16, 2009, at 7:04 PM, Tarek Ziadé wrote:


On Sat, Oct 17, 2009 at 12:58 AM, Nathan Yergler
 wrote:

Looks like you're using a build of Python that wasn't compiled with
zlib support (--with-zlib, IIRC).

2009/10/16 sstein...@gmail.com :
I've just downloaded and configured the Distribute buildbot and it  
appears that a dependency is missing somewhere.  I get this (just  
the errors excerpted).


Nathan is right, but this Distutils test was supposed to check if zlib
is supported before its run,

I have fixed it in r75450. So you should be OK now.

Thanks for noticing !


Cool, no problem.  I also reported it in the buildbot project on  
BitBucket.


Odd thing is:

sstei...@ubuntu:~/distutils-builbot$ cat bin/test
#!/usr/bin/python
# -*- coding: utf-8 -*-
...

sstei...@ubuntu:~/distutils-builbot$ /usr/bin/python
Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import zlib

works fine, so I'm not sure why it thinks it's not able to get zlib.

S


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Buildbot throwing a gear after default setup?

[sstein...@gmail.com]

On Oct 16, 2009, at 7:21 PM, Tarek Ziadé wrote:


On Sat, Oct 17, 2009 at 1:13 AM, sstein...@gmail.com
 wrote:


On Oct 16, 2009, at 7:04 PM, Tarek Ziadé wrote:


On Sat, Oct 17, 2009 at 12:58 AM, Nathan Yergler
 wrote:


Looks like you're using a build of Python that wasn't compiled with
zlib support (--with-zlib, IIRC).

2009/10/16 sstein...@gmail.com :


I've just downloaded and configured the Distribute buildbot and it
appears that a dependency is missing somewhere.  I get this  
(just the errors

excerpted).


Nathan is right, but this Distutils test was supposed to check if  
zlib

is supported before its run,

I have fixed it in r75450. So you should be OK now.

Thanks for noticing !


Cool, no problem.  I also reported it in the buildbot project on  
BitBucket.


Odd thing is:

sstei...@ubuntu:~/distutils-builbot$ cat bin/test
#!/usr/bin/python
# -*- coding: utf-8 -*-
...

sstei...@ubuntu:~/distutils-builbot$ /usr/bin/python
Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more  
information.

import zlib


works fine, so I'm not sure why it thinks it's not able to get zlib.


Python gets build from scratch everytime here, and "test" just runs  
a subprocess

to call the "other" python located in the buildbot.

IOW the way python is built in the bbot, is now using your system's
zlib support...


No, the Python I called was the system python in /usr/bin, I see that  
test is grabbing the trunk checkout python in python = os.path.join 
(root, 'parts', 'trunk-checkout', 'python', 'python') which is NOT  
built with zlib.


The error is in the build of the trunk checkout -- zlib is in my  
system python just fine; it's just not in the python built from the  
trunk checkout:


sstei...@ubuntu:~/distutils-builbot$ parts/trunk-checkout/python/python
Python 2.7a0 (trunk:75449, Oct 16 2009, 16:08:27)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import zlib
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named zlib
>>>



S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Buildbot throwing a gear after default setup?

[sstein...@gmail.com]

On Oct 16, 2009, at 7:26 PM, Tarek Ziadé wrote:

On Sat, Oct 17, 2009 at 1:21 AM, Tarek Ziadé   
wrote:


IOW the way python is built in the bbot, is now using your system's
zlib support...


Arggg... I am the typo master...

s/now/not.

So as Tres said maybe Ubuntu ships with a Python with zlib support,
but doesn't provide zlib headers.


Ok, so the python you build from the trunk checkout doesn't have zlib  
support.


I couldn't find an apt-get'able one, it looks like you have to build  
from source, but the lack of zlib support should be picked up before  
the unit tests fail.


S


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Buildbot throwing a gear after default setup?

[sstein...@gmail.com]

On Oct 16, 2009, at 7:57 PM, Tarek Ziadé wrote:


On Sat, Oct 17, 2009 at 1:39 AM, sstein...@gmail.com
 wrote:


On Oct 16, 2009, at 7:26 PM, Tarek Ziadé wrote:


On Sat, Oct 17, 2009 at 1:21 AM, Tarek Ziadé 
wrote:


IOW the way python is built in the bbot, is now using your system's
zlib support...


Arggg... I am the typo master...

s/now/not.

So as Tres said maybe Ubuntu ships with a Python with zlib support,
but doesn't provide zlib headers.


Ok, so the python you build from the trunk checkout doesn't have zlib
support.


Yup.



I couldn't find an apt-get'able one, it looks like you have to  
build from
source, but the lack of zlib support should be picked up before the  
unit

tests fail.


Yes that's what I've fixed, so it should not fail anymore.


Where can I get that?  I pulled into distutils-buildbot and nothing  
came down.



I'll digg in z...@ubuntu support in any case.



sstei...@ubuntu:~/distutils-builbot$ aptitude search zlib

p   zlib-bin  -  
compression library - sample programs
p   zlib-gst  - Zlib  
bindings for GNU Smalltalk
i   zlib1g-  
compression library - runtime
p   zlib1g-dbg-  
compression library - development
p   zlib1g-dev-  
compression library - development
p   zlibc - An on- 
fly auto-uncompressing C library


# sudo aptitude install zlib-bin zlib-dev zlib-dbg zlibc

Not sure if they're all needed...

l'll rebuild as soon as I know where to get the changes.

S







___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Proposal for Distribute 0.7

[sstein...@gmail.com]

On Oct 16, 2009, at 8:10 PM, Reinout van Rees wrote:


On 2009-10-16, P.J. Eby  wrote:

At 10:46 AM 10/16/2009 -0400, sstein...@gmail.com wrote:


This is not designed by any objective standard I'd use.


You are correct; setuptools itself was not particularly
designed.  Eggs are, entry points are, a few other odds and ends were
designed, but setuptools itself (and *especially* easy_install) are a
collection of hacks upon hacks, mostly done in a rush to get
something out the door...  which as soon as it became popular, locked
into a cycle of not having enough time to do improvements, because
maintenance was taking so much time.


Ah, the perils of success :-)  And you've had a lot of success with  
it.  I
still remember the more-or-less painful days of having to extract  
and install

everything by hand.

And I remember being yealous at perl's "oh, just install x y and z"  
dependency

handling experience.

Hurray for having eggs, pypi and friends!


Here's to catching up with Perl and Ruby and maybe even passing them,  
finally.


S


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Buildbot throwing a gear after default setup?

[sstein...@gmail.com]

Yes, I found those and installed -- not sure if the message went to  
the list, too.


I'm waiting on how to get updated code from Tarek to re-run the whole  
thing.


Thanks,

S

On Oct 16, 2009, at 10:36 PM, Tres Seaver wrote:


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

sstein...@gmail.com wrote:

On Oct 16, 2009, at 7:26 PM, Tarek Ziadé wrote:



So as Tres said maybe Ubuntu ships with a Python with zlib support,
but doesn't provide zlib headers.


Ok, so the python you build from the trunk checkout doesn't have zlib
support.

I couldn't find an apt-get'able one, it looks like you have to build
from source, but the lack of zlib support should be picked up before
the unit tests fail.


$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=9.04
DISTRIB_CODENAME=jaunty
DISTRIB_DESCRIPTION="Ubuntu 9.04"
$ apt-cache search zlib | grep ^zlib.*dev
zlib1g-dbg - compression library - development
zlib1g-dev - compression library - development


Tres.
- --
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkrZLbQACgkQ+gerLs4ltQ6RTQCeIDYNZNJ/xp5owC4xva1wCtPP
jqEAnAim9iziE6rXbHtoxvsy94ZUNis7
=MxaL
-END PGP SIGNATURE-


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Buildbot throwing a gear after default setup?

[sstein...@gmail.com]

On Oct 17, 2009, at 4:34 AM, Tarek Ziadé wrote:


On Sat, Oct 17, 2009 at 4:58 AM, sstein...@gmail.com
 wrote:
Yes, I found those and installed -- not sure if the message went to  
the

list, too.

I'm waiting on how to get updated code from Tarek to re-run the  
whole thing.




Hey,

My update was done on python trunk.


Ah, wasn't sure where to get it; I wasn't sure whether you'd changed  
the buildbot code on bitbucket or something else.



It's a buildbot so once your buildout is created, (python bootstrap.py
&& bin/buildout)
you have new binaries in bin/:

- one to launch the buildbot server (bin/master)
- one to launch the buildbot slave on your machine (bin/ziade.org)

once they are launched (daemons), you can go to http://localhost: 
9080/.


The slave will automatically run the tests by:

- doing "svn up" on the python trunk (that's where you will get the
change I've done on Python trunk)
- building python (configure + make)
- run the python tests via bin/test


bin/test was what complained about missing zlib.  I ran it by hand to  
see if it was working.



Let me know how it goes.


I thought I had apt-get installed all the zlib components so I wasn't  
expecting problems but I guess not.  It now throws a different gear in  
a different way, so I guess I got your changes :-)


Creating tar archive
Traceback (most recent call last):
  File "setup.py", line 15, in 
packages=['extensions', 'extensions.command']
  File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ 
Lib/distutils/core.py", line 149, in setup

dist.run_commands()
  File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ 
Lib/distutils/dist.py", line 926, in run_commands

self.run_command(cmd)
  File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ 
Lib/distutils/dist.py", line 945, in run_command

cmd_obj.run()
  File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ 
Lib/distutils/command/sdist.py", line 166, in run

self.make_distribution()
  File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ 
Lib/distutils/command/sdist.py", line 465, in make_distribution

owner=self.owner, group=self.group)
  File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ 
Lib/distutils/cmd.py", line 392, in make_archive

owner=owner, group=group)
  File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ 
Lib/distutils/archive_util.py", line 236, in make_archive

filename = func(base_name, base_dir, **kwargs)
  File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ 
Lib/distutils/archive_util.py", line 101, in make_tarball
tar = tarfile.open(archive_name, 'w|%s' % tar_compression 
[compress])
  File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ 
Lib/tarfile.py", line 1668, in open

_Stream(name, filemode, comptype, fileobj, bufsize),
  File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ 
Lib/tarfile.py", line 417, in __init__

raise CompressionError("zlib module is not available")
tarfile.CompressionError: zlib module is not available
Exception AttributeError: "_Stream instance has no attribute 'cmp'" in  
0x7fadedc8ec68>> ignored
Could not create distro with /home/ssteiner/distutils-builbot/parts/ 
trunk-checkout/python/python2.7







___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Buildbot throwing a gear after default setup?

[sstein...@gmail.com]

On Oct 17, 2009, at 4:54 AM, Tarek Ziadé wrote:

On Sat, Oct 17, 2009 at 10:34 AM, Tarek Ziadé  
 wrote:


It's a buildbot so once your buildout is created, (python  
bootstrap.py

&& bin/buildout)


precisely:
python bootstrap.py && bin/buildout -c buildbot.cfg


sstei...@ubuntu:~/distutils-builbot$ python bootstrap.py && bin/ 
buildout -c buildbot.cfg

Downloading 
http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg
Getting distribution for 'Twisted'.
twisted/runner/portmap.c:10:20: error: Python.h: No such file or  
directory
twisted/runner/portmap.c:14: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or  
‘__attribute__’ before ‘*’ token
twisted/runner/portmap.c:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or  
‘__attribute__’ before ‘*’ token
twisted/runner/portmap.c:45: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or  
‘__attribute__’ before ‘PortmapMethods’

twisted/runner/portmap.c: In function ‘initportmap’:
twisted/runner/portmap.c:55: warning: implicit declaration of function  
‘Py_InitModule’
twisted/runner/portmap.c:55: error: ‘PortmapMethods’ undeclared (first  
use in this function)
twisted/runner/portmap.c:55: error: (Each undeclared identifier is  
reported only once

twisted/runner/portmap.c:55: error: for each function it appears in.)
error: Setup script exited with error: command 'gcc' failed with exit  
status 1
An error occured when trying to install Twisted 8.2.0.Look above this  
message for any errors thatwere output by easy_install.

While:
  Installing.
  Getting section master.
  Initializing section master.
  Installing recipe collective.buildbot.
  Getting distribution for 'Twisted'.
Error: Couldn't install: Twisted 8.2.0

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Buildbot throwing a gear after default setup?

[sstein...@gmail.com]

On Oct 17, 2009, at 12:24 PM, Tarek Ziadé wrote:


On Sat, Oct 17, 2009 at 4:49 PM, sstein...@gmail.com
 wrote:


precisely:
python bootstrap.py && bin/buildout -c buildbot.cfg


sstei...@ubuntu:~/distutils-builbot$ python bootstrap.py && bin/ 
buildout -c

buildbot.cfg
Downloading
http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg
Getting distribution for 'Twisted'.
twisted/runner/portmap.c:10:20: error: Python.h: No such file or  
directory


Buildbot needs Twisted, so it needs the 'python-dev' to compile on  
ubuntu.


$ apt-get install python-dev
$ apt-get install build-essentials


Is there an easy way of getting bootstrap.py to run these (and get  
Zlib) on Ubuntu since they're not installed on a default image?


S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Buildbot throwing a gear after default setup?

[sstein...@gmail.com]

On Oct 17, 2009, at 12:24 PM, Tarek Ziadé wrote:


On Sat, Oct 17, 2009 at 4:49 PM, sstein...@gmail.com
 wrote:


precisely:
python bootstrap.py && bin/buildout -c buildbot.cfg


sstei...@ubuntu:~/distutils-builbot$ python bootstrap.py && bin/ 
buildout -c

buildbot.cfg
Downloading
http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg
Getting distribution for 'Twisted'.
twisted/runner/portmap.c:10:20: error: Python.h: No such file or  
directory


Buildbot needs Twisted, so it needs the 'python-dev' to compile on  
ubuntu.


$ apt-get install python-dev
$ apt-get install build-essentials
(should be enough)


That got me further (through installing Twisted) but now:

# sstei...@ubuntu:~/distutils-builbot$ python bootstrap.py && bin/ 
buildout -c buildbot.cfg

Downloading 
http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg
Getting distribution for 'Twisted'.

[Twisted builds...]

Got Twisted 8.2.0.
Getting distribution for 'virtualenv'.
Got virtualenv 1.3.4.
Getting distribution for 'zc.recipe.egg'.
Got zc.recipe.egg 1.2.2.
Getting distribution for 'PasteDeploy'.
warning: no files found matching 'docs/*.html'
warning: no previously-included files found matching 'docs/rebuild'
Got PasteDeploy 1.3.3.
Getting distribution for 'Paste>=1.3'.
Got Paste 1.7.2.
While:
  Installing.
  Getting section master.
  Initializing section master.
  Loading zc.buildout recipe entry collective.buildbot:master.

An internal error occured due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "/home/ssteiner/distutils-builbot/eggs/zc.buildout-1.4.1- 
py2.6.egg/zc/buildout/buildout.py", line 1659, in main

getattr(buildout, command)(args)
  File "/home/ssteiner/distutils-builbot/eggs/zc.buildout-1.4.1- 
py2.6.egg/zc/buildout/buildout.py", line 416, in install

[self[part]['recipe'] for part in install_parts]
  File "/home/ssteiner/distutils-builbot/eggs/zc.buildout-1.4.1- 
py2.6.egg/zc/buildout/buildout.py", line 963, in __getitem__

options._initialize()
  File "/home/ssteiner/distutils-builbot/eggs/zc.buildout-1.4.1- 
py2.6.egg/zc/buildout/buildout.py", line 1047, in _initialize
recipe_class = _install_and_load(reqs, 'zc.buildout', entry,  
buildout)
  File "/home/ssteiner/distutils-builbot/eggs/zc.buildout-1.4.1- 
py2.6.egg/zc/buildout/buildout.py", line 1008, in _install_and_load

req.project_name, group, entry)
  File "/home/ssteiner/distutils-builbot/eggs/setuptools-0.6c9- 
py2.6.egg/pkg_resources.py", line 277, in load_entry_point

return get_distribution(dist).load_entry_point(group, name)
  File "/home/ssteiner/distutils-builbot/eggs/setuptools-0.6c9- 
py2.6.egg/pkg_resources.py", line 2180, in load_entry_point

return ep.load()
  File "/home/ssteiner/distutils-builbot/eggs/setuptools-0.6c9- 
py2.6.egg/pkg_resources.py", line 1913, in load
entry = __import__(self.module_name, globals(),globals(),  
['__name__'])
  File "/home/ssteiner/distutils-builbot/eggs/ 
collective.buildbot-0.3.5-py2.6.egg/collective/buildbot/__init__.py",  
line 48, in 

from buildbot.steps.shell import WarningCountingShellCommand
  File "/usr/local/lib/python2.6/dist-packages/buildbot/steps/ 
shell.py", line 5, in 
from buildbot.process.buildstep import LoggingBuildStep,  
RemoteShellCommand
  File "/usr/local/lib/python2.6/dist-packages/buildbot/process/ 
buildstep.py", line 9, in 

from twisted.web.util import formatFailure
ImportError: No module named web.util
sstei...@ubuntu:~/distutils-builbot$

I guess it's a good thing we're doing this!  ;-O

S


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Buildbot throwing a gear after default setup?

[sstein...@gmail.com]

On Oct 17, 2009, at 12:24 PM, Tarek Ziadé wrote:

Just to be clear...I'm not just giving up when the build falls down, I  
just do want to make sure to report everything I do so that we can get  
the buildbot to configure whatever's not there automatically instead  
of having to do anything manually.


We want to be able to fire these up, on a remote, freshly minted Cloud  
Server, and have it pull in everything it needs to run with no  
operator intervention, run tests, and self-destruct all within less  
than an hour (i.e. 1.5 cents worth of compute power).


Right?

S

P.S. To get Twisted to build, I had to:

#  sudo apt-get install zlib1g-dev zlib1g-dev

s


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Buildbot throwing a gear after default setup?

[sstein...@gmail.com]

On Oct 17, 2009, at 3:47 PM, exar...@twistedmatrix.com wrote:


On 04:24 pm, ziade.ta...@gmail.com wrote:

On Sat, Oct 17, 2009 at 4:49 PM, sstein...@gmail.com
 wrote:


precisely:
python bootstrap.py && bin/buildout -c buildbot.cfg


sstei...@ubuntu:~/distutils-builbot$ python bootstrap.py && bin/ 
buildout -c

buildbot.cfg
Downloading
http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg
Getting distribution for 'Twisted'.
twisted/runner/portmap.c:10:20: error: Python.h: No such file or  
directory


Buildbot needs Twisted, so it needs the 'python-dev' to compile on  
ubuntu.


$ apt-get install python-dev
$ apt-get install build-essentials
(should be enough)


Almost all of Twisted's extension modules are optional (and portmap  
is hardly useful, I doubt anyone would miss it if it were gone).   
This failure reminds me that it would be nice if distutils had a way  
to express this.


So, is this something I have to install in addition to the way  
Twisted's being installed now?


I'm working on setting up the buildbot automatically on a cloud server  
and all these errors are what happen if you start from a plain-vanilla  
Ubuntu 9.0.4 and try to just bootstrap the distutils buildbot.


S


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Buildbot throwing a gear after default setup?

[sstein...@gmail.com]

On Oct 18, 2009, at 11:26 AM, exar...@twistedmatrix.com wrote:

So, is this something I have to install in addition to the way  
Twisted's being installed now?


portmap?  No, it's an extension module that's part of Twisted.  Once  
you have python-dev installed it should be okay.


Yah, I'm thinking that this is supposed to be building against the  
Python 2.7 that's built in the course of building the buildbot so I  
think the bug has less to do with the host environment and more to do  
with the buildbot not using its own self-contained stuff to build what  
it needs but I haven't looked too closely yet.


S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 386

[sstein...@gmail.com]

On Oct 20, 2009, at 8:56 AM, Chris Withers wrote:
I'm sure I can't be the only person suffering from PEP overload when  
it comes to packaging.


+1 on PEP overload

Any chance we could at least get dev/post markers in PEP386 and get  
it done and out of the way?


+1 on getting the versioning number scheme nailed down.  The reference  
implementation looks pretty good from a quick review and the tests,  
while not insanely exhaustive, seem to cover everything that would  
come up in practice.  If anyone disagrees with that last assessment,  
how about providing a test case?


I have a feeling that PEP345 and PEP390 along with David's  
alternative proposal are all related in such a way that the best  
thing ot do is bottom out the latter two first


I'm not sure anymore what "bottom out the latter two first" means, but  
getting 386 out of the way would certainly make everything else easier.


S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 345

[sstein...@gmail.com]

On Oct 20, 2009, at 9:53 AM, Tarek Ziadé wrote:


(other than that 386 should be accepted and implemented asap...)


Its implemented, we just need a consensus


+1, ship it!

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] First attempt: close but no data files!

[sstein...@gmail.com]

On Oct 22, 2009, at 8:47 PM, Kaelin Colclasure wrote:


Greetings,

I am attempting my first Python contribution and have run into a  
speed bump getting a working setup.py using setuptools.  
Specifically, I cannot coax it to install my package data files into  
the site-packages directory. Here is a link to my project where my  
setup.py is available for browsing:




S close!


How about some sort of description of what you're trying to do...

• Reproduce steps: Clearly mention the steps to reproduce the bug.
	• Expected result: How application should behave on above mentioned  
steps.
	• Actual result: What is the actual result on running above steps  
i.e. the bug behavior.


Thanks,

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Distribute support added in zc.buildout

[sstein...@gmail.com]

On Oct 24, 2009, at 5:43 AM, kiorky wrote:




Lennart Regebro a écrit :

2009/10/24 kiorky :
If tomorrow frameworks i use, rely on that for deployment, it  
would be hard to

ignore it.


Yes. But hard to ignore does not equal forced. And especially in open
source, where you always have the ability to improve or rewrite.


In french that is called "mauvaise foi".
A nearly translation is  "You're acting in bad faith".


Ok, why don't y'all go run off and have an argument in French and come  
back when you're done...and ready to play nice.


S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Distribute 0.6.7 this week

[sstein...@gmail.com]

On Oct 27, 2009, at 4:53 AM, Tarek Ziadé wrote:


Hello

We are going to release 0.6.7 this week. Among some new bugs fixes, we
want to provide a specific change that will
allow virtualenv to be released with an option to use Distribute
instead of Setuptools, like zc.buildout.

$ virtualenv --distribute ENV

If you want to see another bug fixed in this release,  speak up !
(http://bitbucket.org/tarek/distribute/issues)


I don't necessarily have a request bug-wise, but maybe this would be a  
good time to coordinate the upgrading of the test suite with the  
release.


Maybe something like a TEST_README documenting the tests that prove  
the that the closed ticket's bugs have a test that proves that the  
former bad behaviour was fixed.


S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Distribute 0.6.7 this week

[sstein...@gmail.com]

On Oct 27, 2009, at 10:21 AM, Tarek Ziadé wrote:

Maybe something like a TEST_README documenting the tests that prove  
the that

the closed ticket's bugs have a test that proves that the former bad
behaviour was fixed.


Maybe a simpler way would be to use Bitbucket issue tracker features ?


Are you suggesting maybe getting at that through the (undocumented)  
API, or something else?


For the last two bugs I've fixed, they have a corresponding test in  
the commit.

And if you go to the issue, you have a link to the commit with a diff,
so you can see it.

That's done with the special "fixes #N" bit in the commit messages,
where N is the issue number.  Bitbuckets links the tyicket with the  
commits when its pushed there.  So we could maybe add the link to  
the issue in CHANGES ?


I'm not sure what mechanisms we have for this, do you know?


Although some parts are cleary undertested yet, so even if it's
unpleasant to add tests in an undertested code base, I am +1 in  
making the tests mandatory when the code is not yet covered for now on


That was what we discussed at the sprint and I'm doing what I feel is  
my job as QA PITA (pain in the ass) by insisting that A> we have tests  
for anything fixed and especially not covered and B> that we document  
the test related to the fix so that whomever reported the bug can  
verify that the test does indeed cover the use-case they reported.


What is the next step?

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [issue89] easy_install silently drop symlinks when auto-extracting tarball source distributions

[sstein...@gmail.com]

On Nov 2, 2009, at 5:40 AM, Martin v. Löwis wrote:

If the subject can be changed with the word "setuptools" in it, I'll
accomodate with it.


Did you ask for that specifically because I said it would be difficult
to provide?


Uh, personal animosities aside, Tarek didn't ask for that, Chris did.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] People want CPAN :-)

[sstein...@gmail.com]

On Nov 7, 2009, at 3:20 AM, Ben Finney wrote:


Guido van Rossum  writes:

On Fri, Nov 6, 2009 at 2:52 PM, David Lyon  
 wrote:



I think buildbot-style test runs for PyPI packages would raise average
package quality on PyPI.


Please excuse the cross-post but I wanted to make sure that all these  
"CPAN for Python" discussions got this message and I've lost track of  
which list which part of what discussion had occurred on.


We are currently extending our distutils/Distribute test system to  
include installation of a broad range of packages as part of the pre- 
release process for a future release of Distribute and as part of our  
"smoke" test for distutils/Distribute.Eventually, the goal is to  
integrate this with our buildbot system but that's a ways off.


Our goal is to install a range of packages and, where practicable,  
actually run and record any errors with the packages' individual test  
suites.


Right now, our "smoke" test only does Twisted and numpy.  We've  
discussed how to collect test results from Twisted trial and we'll be  
working on similar things for other test runners (nose et al.).  For  
Twisted, we're going to install and test both the current release  
version and an svn checkout from trunk.


It would be an extension of that concept to install and test *all*  
packages from PyPI but would, obviously, take considerable horsepower  
(and time) to run such an exhaustive test (especially if we're talking  
about 2.4?, 2.5, 2.6, 2.7, and 3.1+.


Right now I'm extending the configuration file for our smoke test to  
allow for various test runners (e.g. nose, twisted trial, etc.) so we  
can "smoke out" more installation problems and/or failed tests after  
installation.


For the first pass, I'm just focusing on Twisted and trial, then  
numpy, then finding packages that support nose so that I can collect  
the data on what ran, what passed, and what didn't.  I'm planning on  
collecting this all in a database and making some simple API so that  
it can be mined by very simple apps later.


At the point where that infrastructure is in place, we could pretty  
easily mine the data to do all kinds of crazy things people have  
mentioned like:


* A ranking system of test coverage
* Complexity analysis
* Test coverage
	* Run pylint, pyflakes, 2to3, whatever automated measurement tools  
over the code
	* Send test failure messages to maintainers (maybe with opt-in in the  
new meta-data).

* Whatever!

We're actively working on this right now; anyone who wants to lend a  
hand is welcome to contact me off-list and we can talk about what  
types of things we are needing and where we could use a hand.


All in all, I think this could be a big leap forward for the Python  
distribution ecosystem whether or not we eventually write the PyPan I  
wished for as a new Perl refugee.


Thanks,

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Python-Dev] People want CPAN :-)

[sstein...@gmail.com]

On Nov 7, 2009, at 10:08 AM, Jesse Noller wrote:

On Sat, Nov 7, 2009 at 9:30 AM, sstein...@gmail.com > wrote:

On Nov 7, 2009, at 3:20 AM, Ben Finney wrote:


Guido van Rossum  writes:

On Fri, Nov 6, 2009 at 2:52 PM, David Lyon >

wrote:


I think buildbot-style test runs for PyPI packages would raise  
average

package quality on PyPI.


Please excuse the cross-post but I wanted to make sure that all  
these "CPAN
for Python" discussions got this message and I've lost track of  
which list

which part of what discussion had occurred on.

We are currently extending our distutils/Distribute test system to  
include
installation of a broad range of packages as part of the pre- 
release process
for a future release of Distribute and as part of our "smoke" test  
for
distutils/Distribute.Eventually, the goal is to integrate this  
with our

buildbot system but that's a ways off.


Who is "we"?



We is the people working on Distribute/distutils.

S


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] People want CPAN :-)

[sstein...@gmail.com]

On Nov 7, 2009, at 7:10 PM, Kevin Teague wrote:

Being able to update individual modules within the standard library  
would absolutely rock my world and would have removed the urgency from  
2.6.3/4 updates.


Just update what's necessary.  What a concept.

Yes, please.  The sooner the better.

As long as the testing is at least as thorough, and reverting is very  
easy, why _wouldn't_ this be a good idea?


I gotta say, all this talk of CPAN reminds me of when I first started  
using Python and was mystified/mortified at the total lack of  
something similar.  I kind of got used to it but this is bringing back  
the itch in a major way...


S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] People want CPAN :-)

[sstein...@gmail.com]

On Nov 11, 2009, at 4:39 AM, Lennart Regebro wrote:


If nobody is willing to write the code, the code is not needed.


I think it would be more accurate to say that nobody is deciding that  
the "need" is sufficient to invest the resources to fill it.


There are lots of things that many people would be willing and able to  
write, that would fill a great "need", and that many people would  
benefit from, but that aren't getting written because there's no one  
willing to invest the time or money.


In come cases people are both willing and able, but unable to invest  
the time due to the pressing need to make a living.  I'd love to work  
on open source all day, but my wife and kids get cranky if they don't  
eat for a couple of days.


People don't always agree on what is the "rational" need to fill, and  
sometimes what we, as programmers "need", and know would make us more  
productive, is not what the people controlling the pursestrings are  
willing to finance, even if it would ultimately benefit them  
financially.


S



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] People want CPAN :-)

[sstein...@gmail.com]

On Nov 11, 2009, at 7:38 AM, Lennart Regebro wrote:


2009/11/11 sstein...@gmail.com :

On Nov 11, 2009, at 4:39 AM, Lennart Regebro wrote:


If nobody is willing to write the code, the code is not needed.


I think it would be more accurate to say that nobody is deciding  
that the

"need" is sufficient to invest the resources to fill it.


More accurate, but longer and redundant. :)


I don't think it's redundant because a lot of "needs" in Python go  
unmet, not due to people's inability or unwillingness, but due to a  
lack of time/funding which amount to the same thing.


S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Making a tar.gz source distribution

[sstein...@gmail.com]

On Nov 16, 2009, at 11:10 AM, Ram Rachum wrote:

I'm trying to make a tar.gz source distribution. I saw in the docs  
that a tar

program is needed. I am on Windows. Where do I get one?


You've got to be joking.

Try Google.

http://www.google.com/search?client=safari&rls=en&q=windows+tar+program&ie=UTF-8&oe=UTF-8

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 386 status - last round here ?

[sstein...@gmail.com]

On Nov 24, 2009, at 4:47 AM, Tarek Ziadé wrote:

> Hello,
> 
> PEP 386 seem to be ready, and I would like to push if for feedback at
> python-dev, just before PEP 345 is pushed.
> 
> My only concern is now to make sure the PEP motivations and explanations are 
> crystal clear.
> 
> Anyone see any problem ? or have any concern with this PEP ?

I think the motivation and explanations are clear.

There are, however, some issues that should be addressed before it's accepted 
as "final."

1>  There seems to be a typo on line 29 of verlib.py where it says 'f' < 
'b', shouldn't that be 'b' < 'f' ?

2>  The explanation for "suggest_rational_version" is stubbed out in the 
README.txt file:

XXX explain here suggest_rational_version

it should be documented what we expect it to do and that leads into...

3>  test_suggest_rational_version needs a more comprehensive test suite 
(like all the version numbers from PyPi).  Right now it only has 11 cases and 
they're all pretty mild though the doc asserts success rates on PyPi that are 
much higher.  This should be done so that we can insure that the algorithm 
stays stable (and hopefully improves) in future versions.  (I will create this 
expanded test using PyPi data, if requested).

4>  The [#requrires] reference is missing from the README.txt file

5>  There is no cross reference to any other non-python project using this 
scheme.  There is a note that  "During Pycon, members of the Python, Ubuntu and 
Fedora community worked on a version standard that would be acceptable for 
everyone"  but there is no evidence that anyone else has agreed to this 
standard and, particularly, to this reference implementation.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 386 status - last round here ?

[sstein...@gmail.com]

On Nov 24, 2009, at 3:40 PM, Tarek Ziadé wrote:
>> 2>  The explanation for "suggest_rational_version" is stubbed out in the 
>> README.txt file:
>> 
>>XXX explain here suggest_rational_version
>> 
>>it should be documented what we expect it to do and that leads into...
>> 
> 
> This README.txt is a bit outdated. It's an old copy of PEP 386 (which 
> explains suggest_rational_version)
>> 4>  The [#requrires] reference is missing from the README.txt file
> I'll update it using the latest PEP 376 text


Ok, I'll look for those when I pull later tonight?

> 
>> 3>  test_suggest_rational_version needs a more comprehensive test suite 
>> (like all the version numbers from PyPi).  Right now it only has 11 cases 
>> and they're all pretty mild though the doc asserts success rates on PyPi 
>> that are much higher.  This should be done so that we can insure that the 
>> algorithm stays stable (and hopefully improves) in future versions.  (I will 
>> create this expanded test using PyPi data, if requested).
> 
> That would be great,

May I have access please?

>> 5>  There is no cross reference to any other non-python project using 
>> this scheme.  There is a note that  "During Pycon, members of the Python, 
>> Ubuntu and Fedora community worked on a version standard that would be 
>> acceptable for everyone"  but there is no evidence that anyone else has 
>> agreed to this standard and, particularly, to this reference implementation.
> 
> I don't think any project uses this scheme yet, besides some Python projects. 
> What happened is that when we worked on this scheme, Toshio (Fedora) and 
> Matthias (Ubuntu/Debian) made sure it would be acceptable for their needs to 
> translate it in their own schemes automatically.
> 
> For instance, RPMs want to avoid date-based versions, and Debian packages 
> avoids "-" characters.

Ok, it would be great to have an example of an RPM or deb package showing the 
direct translation but no big deal.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 386 status - last round here ?

[sstein...@gmail.com]

On Nov 25, 2009, at 12:44 PM, M.-A. Lemburg wrote:

> M.-A. Lemburg wrote:
>> Could we extend the pseudo-format of the versions a little to also
>> include variants which use more than just one character and also
>> allow hyphens and spaces to be used for additional clarity ?
>> 
>> VERSION_RE = re.compile(r'''
>>^
>>(?P\d+\.\d+)  # minimum 'N.N'
>>(?P(?:\.\d+)*)   # any number of extra '.N' segments
>>(?:
>>[- .]*
>>(?P[a-z]+)# 'a'=alpha, 'b'=beta, 'c'=release candidate
>>[- .]*
>>(?P\d+(?:\.\d+)*)
>>)?
>>(?P
>>([- .]*post[- .]*(?P\d+))?
>>([- .]*dev[- .]*(?P\d+))?
>>)?
>>$''', re.VERBOSE + re.I)
>> 
>> The pre-release marker would then be interpreted in alphabetical
>> order, ie. 'alpha' < 'beta' < 'rc'.
>> 
>> This minor change would broaden the scope of the scheme somewhat
>> and make it more compatible to what's being used outside the
>> python-dev sphere (esp. with respect to 'c' standing for release
>> candidate... unless you happen to read it as gamma ;-).
> 
> ... and even python-dev has switched to "rc" for these:

While I like the use of "rc" for release candidate, doesn't this lead to some 
ambiguity where a,b,c are seen as interim releases with "rc" sorting after all 
of these?

I don't have an objection to [a-q] OR 'rc' if we're reserving "rc" for release 
candidate and only "rc" for that purpose.

Otherwise, the sorting becomes ambiguous so if "c" means candidate then "rc" 
can't also or we don't know, for sure, that blah-rc comes AFTER blah-c; it's 
ambiguous.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 386 status - last round here ?

[sstein...@gmail.com]

On Nov 27, 2009, at 7:31 AM, Tarek Ziadé wrote:

> On Fri, Nov 27, 2009 at 1:24 PM, Tarek Ziadé  wrote:
> [..]
>>> don't worry about Debian, we'll simply replace "-" with "~" (we use "~"
>>> and "+" right now[0]). I'm not sure about rpm, but I bet it has
>>> something similar and it will be much easier for us to simply handle two
>>> characters instead of recognizing that dev1 < a1 < b1 < c1 == rc1 ...
>> 
>> It's different from RPMs, since they use a strcmp(), segment  by segment,
>> so I think they have to extract the dev/post suffixes and to put them in 
>> front
>> as an epoch marker maybe ? (ccing Toshio)
>> 
> That makes me think that a nice add-on to the lib and the PEP would be
> to provide APIs to translate a Python PEP 386 version to a Debian/Ubuntu or 
> RPM ones -
> and any major packaging system out there. (whatever scheme we pick)

Wouldn't it be cool if the package that goes along with this PEP became the 
standard version checker used by ALL of these distributions?

It's not like they don't have Python interpreters around...

What would that API have to look like?  Maybe start a doc in the repo to hold 
that spec as we develop it?

I'm still very interested in the increment_version functionality we talked 
about earlier so that we could have our build process automatically up our 
release version numbers so we have a standard way of maintaining incremental 
versioning.

S



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 386 status - last round here ?

[sstein...@gmail.com]

On Nov 27, 2009, at 9:41 AM, Tarek Ziadé wrote:
>> I'm still very interested in the increment_version functionality we talked 
>> about earlier so that we could have our build process automatically up our 
>> release version numbers so we have a standard way of maintaining incremental 
>> versioning.
> 
> That would be an interesting feature

I think I'll start it a separate project, calling into the API, that can just 
be put into any build process.

If it works well enough stand-alone, maybe we'll incorporate it later.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 386 status - last round here ?

[sstein...@gmail.com]

On Nov 27, 2009, at 9:58 AM, Piotr Ozarowski wrote:

> [sstein...@gmail.com, 2009-11-27]
>>> That makes me think that a nice add-on to the lib and the PEP would be
>>> to provide APIs to translate a Python PEP 386 version to a Debian/Ubuntu or 
>>> RPM ones -
>>> and any major packaging system out there. (whatever scheme we pick)
>> 
>> Wouldn't it be cool if the package that goes along with this PEP
>> became the standard version checker used by ALL of these
>> distributions?
> 
> used where?

I'm thinking of a tool, callable within *any* build process, for verifying that 
the version conforms to the "RationalVersion" scheme, for example.  

Especially, I'm interested in a tool that would automatically increment version 
numbers for e.g. in process development versions where each build needs a new 
version number (maybe in several places e.g. setup.py, documentation, history 
file, etc.).

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Common version-comparison semantics for peace love and harmony

[sstein...@gmail.com]

On Nov 28, 2009, at 4:50 AM, Laura Creighton wrote:
>> At some point, we all agree that MAJOR.MINOR.MICRO is an accepted
>> standard and we are arguing about pre/post/dev releases.
> 
> We have no way to enforce this on the world.  The arguments here may
> have gone away, but out in the wild, I think we need to accept that
> people are going to name their packages whatever they like.  We
> can suggest in the strongest possible terms that they use this
> numbering convention, but it will still just be a suggestion.

I think the PEP should lay out a standard scheme, the Python standard 
library (distutils) should have the tools to support it, and either conform to 
it or not at your product and your user's peril.  

Other than providing tools to suggest a PEP compliant version for you that is 
as close to your current version as possible, I don't think we should support 
ANY other versioning scheme but the rational one laid out in the PEP, period.  
Once the bikeshed is built and painted, throw away all the other exotic 
materials and paint colors.

How about if we make it a suggestion like this:

If you want your tools to be installable in a reliable way, using 
standard Python installation tools,  so that your users always know that they 
are getting the exact version that they want, and so they can install things 
from PyPi, the Python standard code repository, we suggest that you follow the 
PEP 386 versioning scheme.

The PEP 386 scheme will be supported by a broad range of tools 
including Python's distutils, the Distribute installation system, and any other 
tools using the Python standard library routines for versioning.

Otherwise, we suggest that you devote a large amount of time preparing 
an extremely detailed  explanation about why you chose not to follow a 
community standard and also either provide your own comprehensive installation 
tools, or explain exactly how to use standard Python installation tools with 
your non-standard versioning scheme.

Also, please write a FAQ about exactly how to determine exactly which 
version of your product is installed and how to determine what the "next" and 
"previous" versions are since the users can't determine that using the standard 
Python tools or reading the Python documentation.

Ok?  It's just a suggestion.

Feel free to make it as difficult for yourself and your users by 
inventing whatever scheme seems best to you and best of luck with that.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Bundling pkg_resources

[sstein...@gmail.com]

On Nov 30, 2009, at 2:39 PM, Tarek Ziadé wrote:

> maintainer that has been AWOL for most of the time in the past 2 years)

AWOL is an acronym for "absent without leave" or "absent without official 
leave".

Yes, it sucked that setuptools got abandoned but nobody had to give him leave.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Problem with buildout and zc.recipe.cmmi

[sstein...@gmail.com]

On Dec 12, 2009, at 11:26 AM, Tarek Ziadé wrote:

> There are hundreds of way to break things :)
> 
> IOW everytime a change is made, there's a potential risk. Functional
> tests may help

Tarek,

What are you using for a test suite pre-release now?

I started on the multi-package installer we talked about but we never 
got back together to get it into the release testing.

Steve


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Problem with buildout and zc.recipe.cmmi

[sstein...@gmail.com]

On Dec 12, 2009, at 12:34 PM, Tarek Ziadé wrote:

> On Sat, Dec 12, 2009 at 6:29 PM, sstein...@gmail.com
>  wrote:
>> 
>> On Dec 12, 2009, at 11:26 AM, Tarek Ziadé wrote:
>> 
>>> There are hundreds of way to break things :)
>>> 
>>> IOW everytime a change is made, there's a potential risk. Functional
>>> tests may help
>> 
>> Tarek,
>> 
>>What are you using for a test suite pre-release now?
> 
> I run the test command using python then python3 :
> 
> $ python setup.py test
> $ python3 setup.py test
> 
> the latter triggers the 2to3 script and launch the tests in build/
> 
>> 
>>I started on the multi-package installer we talked about but we never 
>> got back together to get it into the release testing.
> 
> 
> That would be a good timing to continue then ;)

I'm thinking you're right...

Do we have a buildout that fails to put into the loop?

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Problem with buildout and zc.recipe.cmmi

[sstein...@gmail.com]

On Dec 12, 2009, at 12:34 PM, Tarek Ziadé wrote:

> That would be a good timing to continue then ;)

The work I did is in the distutils-buildbot branch "smoke_with_mirrors".

Basically, what I did was set up a configuration file that contained the 
packages we were going to install, what versions they are to be installed on.

What still needs to be sone is to iterate through that configuration, doing the 
actual install/run test suite.

Could you take a quick look and see if you agree with the approach I took?

Thanks,

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [buildout] "Couldn't find index page for" in tests

[sstein...@gmail.com]

On Dec 14, 2009, at 9:05 AM, Chris Withers wrote:

> Hi All,
> 
> I see a few tests for buildout recipes that use zc.buildout.testing but also 
> use a RENormalizer to hide "Couldn't find index page for" errors..
> 
> Why is this error showing up and why is it okay to hide it?
> 
> not-joining-the-cargo-cult-ly yours,

Which cargo cult are you not joining?  

Perhaps we could start a news/support group.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Distribute testing

[sstein...@gmail.com]

Tarek,

We recently talked about incorporating the testing scheme I started in 
distutils-buildbot directly into the release path for Distutils to avoid issues 
like the ones with the most recent release.

I am putting this out on the list for comments; I'm NOT interested in 
bikeshedding this to death.  Anyone else is welcome to do their own project if 
they feel strongly enough about it.  I'm going to take opinions into account 
but I want to have the tool built before the next release, not the next 
millennium release of Python.

The way I started with the project in distutils-buildbot 
(http://bitbucket.org/tarek/distutils-buildbot/ smoke-and-mirrors branch, 
bin/smoke2.py, bin/smoke2.cfg, and bin/smoke_base_products.cfg) was to assume a 
brand new system e.g. a newly initialized Cloud Server and to build out 
everything necessary to run the tests under Python 2.4, 2.5, 2.6, 2.7dev, 3.1, 
and 3.2dev.  

While this approach will be fine when we move it to building on-demand 
buildbots, it is much too time consuming both in development time and real-time 
processing to use for this release cycle.  It would be much better (for now) to 
just make sure you're running it on a machine that can do the job.

The main goal is to come up with a simple way to run as many full-scope 
tests i.e. from install through full test-suite on as many large products, with 
as many versions of Python as practical.

What we had originally started was to build and run the test suites for 
(spec'd in bin/smoke_base_products.cfg):

Twisted 
numpy
tarek_extensions
warped

I had stopped where it reads the configuration files and knows on what 
versions of Python to test which products.

At this point, I'd like to move this into Distribute proper and get it 
into the "pre-release" test process.

So...

First, we need to decide where to put the testing sub-project.  I'm 
asuming it can just live in the /tests subdirectory, maybe with a subdir to 
hold this whole project as its own module.

Here's what I'd like to see it do:

Run pre-test checks i.e. verify that the machine is capable of running 
the tests:

1>  Assume that `python2.4`, `python2.5`, `python2.6`, 
`python3.1` will invoke a properly   set up Python interpreter.  This is beyond 
the scope of this project; might be taken up at a later time.

2>  Create a simple way to verify installed utilities 
before proceeding with tests (anyone have one of these around?).
Something like:
ASSERT(`nosetests -V` == `nosetests version 0.11.1`)

3>  Allow a certain amount of corrective action to be able 
to be taken to satisfy the assertions above (like, install nose version 0.1.11 
from PyPi, for example) for each supported version of Python.

Then:
for each product to be tested:
for each Python version this product is to be tested on:
install the product
run its test suite
record any failures

Pretty simple, but will avoid any of the sorts of issues we just had 
with the most recent release, is very easy to add new products to (just add a 
configuration section), and can be easily moved out to the buildbots when it 
does what we want.

Comments etc. gladly accepted as long as there are no bikeshedding 
tools (hammers, paint-brushes, Erlang) in hand.

Thanks,

S
aka/Steve Steiner
aka/ssteinerX





___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Distribute testing

[sstein...@gmail.com]

On Dec 20, 2009, at 6:54 AM, Tarek Ziadé wrote:

> On Sat, Dec 19, 2009 at 6:33 PM, sstein...@gmail.com
>  wrote:
> [..]
>> 
>>While this approach will be fine when we move it to building 
>> on-demand buildbots, it is much too time consuming both in development time 
>> and real-time processing to use for this release cycle.  It would be much 
>> better (for now) to just make sure you're running it on a machine that can 
>> do the job.
> 
> *or* to update an existing test environment. e.g. check if python != trunk is 
> already present.

Yes, but what I mean is that I'm not going to be downloading and building all 
supported versions of Python itself in this version as I will in the buildbot 
version.

Updating the dev versions of 2.7 and 3.2 to latest trunk will be in this 
version as will updating any peripheral tools required for testing (pip etc.).

>>First, we need to decide where to put the testing sub-project.  I'm 
>> asuming it can just live in the /tests subdirectory, maybe with a subdir to 
>> hold this whole project as its own module.
> 
> Sounds good

Ok, I'll do that.  I'm going to put this in one working chunk at a time so that 
you can use whatever's there right away.  As time goes on, more tests will be 
added but my intention is to keep what's checked into the main branch always 
working.

I'll make a new branch

How, exactly, do you run the tests that you run before you post to pypi so that 
I can integrate it properly?

>>Here's what I'd like to see it do:
>> 
>> [snip...]
>>for each product to be tested:
>>for each Python version this product is to be tested 
>> on:
>>install the product
>>run its test suite
>>record any failures
>> 
> 
> Sounds good too ! Notice that we also need to try those tests in
> various environments, besides python versions: virtualenv, no virtualenv, 
> setuptools present before
> we start, not present, etc

Yes, I understand what might be possible at some point in the future but, for 
now, I just want to get the machinery running and in place to support all of 
those different scenarios at some point in the future.

While I'd love to do a comprehensive test suite, I just don't have the time to 
devote to getting every combination and permutation done it in one shot so my 
intention is to set up the hooks, get some useful tests into the process that 
cover recent issues, and make it so that anyone can add new tests with a 
minimum of configuration.

So...what command should I hook it into so I can quickly get this into the 
testing process before the next release?

Thanks,

S






___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Python people want CPAN and how the latter came about

[sstein...@gmail.com]

On Dec 23, 2009, at 9:59 PM, exar...@twistedmatrix.com wrote:

> What's with the interest in having packages hosted on PyPI?
> 
> I'm not specifically opposed to this, but I don't see any reason it would 
> benefit anyone either.  It'd be awesome if someone could explain this.  
> Perhaps if the answer were included somewhere on PyPI or in the distutils 
> documentation, more people would see the light and upload their packages.

I really don't think it has to do with 'having packages hosted on PyPI' as it 
is in having a single way to install any Python 'thing.'

Right now, installing e.g. Twisted, requires finding the website, figuring out 
which exact file to download, then figuring out exactly how to get it 
installed.  

Yes, 'we' know how to do it, or can figure it out pretty easily (with the value 
of 'we' being pretty much anyone reading this list), but the argument seems to 
be whether "Joe Average Non-Programmer" could figure it out, or should even 
have to.

A simple:

# super-duper-python-thing-just-like-cpan-only-better -i Twisted

Should do it, and it should find the proper mirror/source repository itself 
without further operator intervention or annoyance.

At least that's what I get from all this...

Steve
aka/S
aka/ssteinerX

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Python people want CPAN and how the latter came about

[sstein...@gmail.com]

On Dec 24, 2009, at 2:02 AM, Lennart Regebro wrote:

> On Thu, Dec 24, 2009 at 06:06, sstein...@gmail.com  
> wrote:
>> Right now, installing e.g. Twisted, requires finding the website, figuring 
>> out which exact file to download, then figuring out exactly how to get it 
>> installed.
> 
> Nope.
> 
>> # super-duper-python-thing-just-like-cpan-only-better -i Twisted
>> 
>> Should do it
> 
> $ /opt/python26/bin/virtualenv twist

[snip]

Ok, so easy_install works for Twisted.  Yay.

> Installed 
> /tmp/twist/lib/python2.6/site-packages/zope.interface-3.5.3-py2.6-linux-i686.egg
> Finished processing dependencies for Twisted
> 
> Done!
> 
>> At least that's what I get from all this...
> 
> We have *that*. Two versions of it even, as pip would likely work as well.

Ok, there's easy_install, the direct download route, and pip.

I think that's exactly *not*:

# super-duper-python-thing-just-like-cpan-only-better -i Twisted

And, I picked Twisted 'cause it was already in the discussion.  

There are at least three ways to install it and I'm really concerned that what 
you just seemed to suggest was that the much-maligned-and-soon-to-be-deprecated 
`easy_install` is the `super-duper-python-thing-just-like-cpan-only-better` of 
today?

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Python people want CPAN and how the latter came about

[sstein...@gmail.com]

On Dec 24, 2009, at 9:06 AM, Lennart Regebro wrote:

>> I'm really concerned that what you just seemed to suggest was that the 
>> much-maligned-and-soon-to-be-deprecated `easy_install` is the 
>> `super-duper-python-thing-just-like-cpan-only-better` of today?
> 
> That's a strange concern.

I don't think it's strange to be concerned that one of the SDPTJLCOB options, 
and the one that you used as your example, is going to go away.

If pip is going to be the SDPTJLCOB of the future, with an facade that 
substitutes for easy_install in the same way that Distribute does for 
setuptools then great, that's progress!

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Python people want CPAN and how the latter came about

[sstein...@gmail.com]

On Dec 24, 2009, at 10:16 AM, Lennart Regebro wrote:

> On Thu, Dec 24, 2009 at 15:46, sstein...@gmail.com  
> wrote:
>> I don't think it's strange to be concerned that one of the SDPTJLCOB 
>> options, and the one that you used as your example, is going to go away.
> 
> Why? Pip does the same thing. They are commands. And it's not likely
> to go away anytime soon, what probably happens is that people will use
> pip more and more until pretty much noone uses easy_install. It
> completely and utterly fails me to see why this would be anything to
> worry about.

I guess what I mean is that I'd like to make sure that while moving to pip, 
that easy_install, as a command name, not as an implementation, gets brought 
along in the same way that Distribute has brought along setuptools 
compatibility.  IOW in such a way that the improvements in the new code are 
available without breaking any existing configurations/scripts/workflows etc.

>> If pip is going to be the SDPTJLCOB of the future, with an facade that 
>> substitutes for easy_install in the same way that Distribute does for 
>> setuptools then great, that's progress!
> 
> It'll have an easy_install facade if someone needs it badly enough to write 
> one. But I can't imagine why they would.

I imagine it would be useful to have a facade because easy_install is familiar 
and ubiquitous and shouldn't be left rotting in place as pip takes over the 
world and becomes the SDPTJLCOB any more than Distribute left the rest of 
setuptools exposed to cause problems once Distribute was installed.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Python people want CPAN and how the latter came about

[sstein...@gmail.com]

On Dec 24, 2009, at 5:50 PM, Lennart Regebro wrote:
> easy_install is a command, basically a wrapper around setuptools
> install functionality. I doubt many scripts would use it in any more
> complex way than calling it, in which case moving to pip is a matter
> of replacing the command.

Yes, I'm saying that it would be smart to implement a replacement easy_install 
with pip doing the work in the background instead of setuptools so that no 
changes are necessary.

>> I imagine it would be useful to have a facade because easy_install is 
>> familiar and ubiquitous and shouldn't be left rotting in place as pip takes 
>> over the world and becomes the SDPTJLCOB any more than Distribute left the 
>> rest of setuptools exposed to cause problems once Distribute was installed.
> 
> I don't even understand that sentence, let alone what you are trying
> to say. Distribute is a setuptools fork.

Distribute is a setuptools fork and is intended to replace it in the Python 
ecosystem.  In order to accomplish this, it transparently replaces setuptools.

> Pip is not an easy_install fork. I don't really see the parallell.

The parallel is that as Distribute transparently replaces setuptools, the new 
easy_install, with pip doing the behind-the-scenes work, should transparently 
replace the existing easy_install.

# easy_install Twisted 

should still work just as it does now, it'll just do a better job of it with 
the new machinery running in the background.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Python people want CPAN and how the latter came about

[sstein...@gmail.com]

On Dec 26, 2009, at 7:15 AM, Laura Creighton wrote:

>  Right now I don't know any way to say 'under no circumstances,
> ever, let easy_install near my code because it will do very bad things
> to it'.

Uh...I think you just did.

> I liked things a whole lot better when pypi was about being a package
> index, and _only_ about being a package index, and where those people
> who had ideas about improving the user experience were free to go out
> there and write their own programs to do the same, but where none of
> these has any sort of 'official recognition' and where, of course,
> others who didn't want that sort of experience were free to
> ignore the whole thing.

I think that, in the whole CPAN-ification of PyPI discussion, an absurd amount 
feature creep has come into the discussion.   I think the ratings discussion 
was the tiny crystal that started the whole gigantic snowball.

At the bottom of everything CPAN's repository is just a glorified, rsync-able 
FTP site with a bunch of stuff in directories.  Everything on top of that is 
window dressing.

The PyPI discussions seem to be tending toward mixing the window dressing with 
the framing, to use a building analogy, and what that will result in is a weak 
frame and ugly windows.  A building that slowly (or quickly) falls down under 
its own weight, and looks bad doing it.

I think that splitting 

> package storage and pointers to off-repository storage (for those who 
don't upload to PyPI)
> metadata about the stored packages
> tools for creating stored packages
> tools for retrieving stored packages
> tools for installing packages

would go a long way towards unobfuscating this whole discussion.

Yes, I'm sure someone will disagree with some fine-point of that division but 
isn't that what  woodshedding is all about?

S






___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Python people want CPAN and how the latter came about

[sstein...@gmail.com]

On Dec 26, 2009, at 4:17 PM, Stephen Waterbury wrote:

> sstein...@gmail.com wrote:
>> I think that splitting   > package storage and pointers to 
>> off-repository storage (for those who don't upload to PyPI)
>>  > metadata about the stored packages
>>  > tools for creating stored packages
>>  > tools for retrieving stored packages
>>  > tools for installing packages
>> would go a long way towards unobfuscating this whole discussion.
>> Yes, I'm sure someone will disagree with some fine-point of that
>> division but isn't that what  woodshedding is all about?
> 
> OT (diction):
> Hmm ... I think you meant "bikeshedding" (energetic discussion of
> meaningless details) -- "woodshedding" *is* an actual slang term, but
> one that originated among musicians, meaning "to practice one's skills
> alone" (i.e., to go off to a woodshed where no one can hear you).

I know that, I'm a musician, too..it was kind of a play on the fact that simple 
bikeshedding seems to turn into woodshedding i.e. it goes beyond the normal 
bikeshedding and becomes beating a dead bikeshed...  I couldn't come up with a 
conjoined word fast enough so I just let it go as I wrote it.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How Python can have CPAN.

[sstein...@gmail.com]

On Dec 27, 2009, at 3:27 AM, Lennart Regebro wrote:

> On Sat, Dec 26, 2009 at 18:40, Lennart Regebro  wrote:
>> 3. Making it easier to mirror/replicate all metadata.
> 
> I've now written a script that does this via XML-RPC. It's dead easy,
> actually. It's however also slow, and looks like it takes at least
> five hours. But that's to download all metadata, and there is quite a
> lot of it. :) Syncing after the initial download is reasonably fast.

How hard would it be to set up a cron to tar up a daily snapshot so that the 
initial download was quick (no API calls), then you'd only need an update from 
the last snapshot?

Thanks, 

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How Python can have CPAN.

[sstein...@gmail.com]

On Dec 27, 2009, at 5:47 AM, Lennart Regebro wrote:

> On Sat, Dec 26, 2009 at 18:40, Lennart Regebro  wrote:
>> 1. Ask those who do not upload why they don't upload, and see if we can fix 
>> it.
>> 2. Ask those who do not want to upload why they don't provide a download URL.
> 
> Out of a total of 8522 packages on PyPI, there are 203 packages (2.4%)
> whose latest release does not provide either a package on PyPI, nor a
> download url. Of these 16 does not provide any contact data.

I'll update the PyPI checker in the distutilsversion test_pypi_versions.py to 
provide that information as well.  I think I'll make a separate project 
(uploaded to PyPI, promise) out of it so we have a shared way to get statistics 
on PyPI projects.  Is your code for this in the PyPI code repository or were 
these just quick one-off's?

If there's no data on PyPI and no download url then wouldn't those be 
"non-packages?"   And if there's no contact info, "non-package" by "nobody?"  
Sounds like a song title.

The survey should include a "Project abandoned" or "Nothing to upload" or "It 
was just a mistake" and an "Ok to delete immediately".  

Any non-project where the owner gives permission to delete, where the e-mail 
bounces, or there's no response in a month, just delete it.

That 2.4% is pretty small compared to somewhere like SourceForge where every 
other project seems to be abandoned or worse and there sure are a lot of them...

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How Python can have CPAN.

[sstein...@gmail.com]

On Dec 27, 2009, at 11:27 AM, Lennart Regebro wrote:

> On Sun, Dec 27, 2009 at 16:54, sstein...@gmail.com  
> wrote:
>> I'll update the PyPI checker in the distutilsversion test_pypi_versions.py 
>> to provide that information as well.  I think I'll make a separate project 
>> (uploaded to PyPI, promise) out of it so we have a shared way to get 
>> statistics on PyPI projects.  Is your code for this in the PyPI code 
>> repository or were these just quick one-off's?
> 
> Quick one-offs. Here, in fact:
> 
> Takes an hour or two to run.
> 
> from xmlrpc import client
> PYPIURL = 'http://pypi.python.org/pypi'
> pypi = client.Server(PYPIURL)
> for package in pypi.list_packages():
>for release in pypi.package_releases(package, False):
>release_urls = pypi.release_urls(package, release)
>if release_urls:
>continue
>release_data = pypi.release_data(package, release)
>if not release_data.get('download_url', ''):
>print("Package %s, Release: %s" % (package, release))
>print("Did not have releases on PyPI or download URL")
>print("Author: %s <%s>" % (release_data['author'],
>   release_data['author_email']))
>print("Maintainer: %s <%s>" % (release_data['maintainer'],
> 
> release_data['maintainer_email']))

Ok, thanks, I'll throw that into the code, in some form.  

The tarred download would be really handy for this utility as, if there's no 
.pkl of the data, or the user requests it, I pull  fresh copy.  

Right now, my query is very limited (I'm only looking for version info) and 
only takes a couple of minutes to build.  

Since I'm going to add more capabilities, having a quick way to refresh the 
whole thing would be great.

I'll put my version up in the new project and maybe we can work together to get 
it into some PyPI code or to store the version I build somewhere though 
building it right on the server would seem to be much faster (if memory 
intensive).

Thanks!

S
aka/Steve Steiner
aka/ssteinerX





___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How Python can have CPAN.

[sstein...@gmail.com]

On Dec 27, 2009, at 1:34 PM, Martin v. Löwis wrote:

>> The tarred download would be really handy for this utility as, if
>> there's no .pkl of the data, or the user requests it, I pull  fresh
>> copy.
> 
> How difficult would it be for you to provide such data, for anybody
> interested in using them?

Really easy, that's what I'm saying, let's cooperate in getting this put 
somewhere.

I have plenty of hosting resources and can put a cron job up to keep it up to 
date.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How Python can have CPAN.

[sstein...@gmail.com]

On Dec 27, 2009, at 2:10 PM, Lennart Regebro wrote:

> On Sun, Dec 27, 2009 at 17:50, sstein...@gmail.com  
> wrote:
>> The tarred download would be really handy for this utility as, if there's no 
>> .pkl of the data, or the user requests it, I pull  fresh copy.
> 
> Right... Anyone want to host such a tarred download? :-)
> I pretty much got the code, but don't feel like setting up and hosting
> it right now. Maybe later.

I'll finish up the code, get it set up on a cron job to keep it up to date, and 
host it.  

I'll probably just put the finished data up on S3, no big deal and I can run 
the cron job on one of our servers.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] PyPI Data Synchronizer

[sstein...@gmail.com]

On Dec 27, 2009, at 2:34 PM, Lennart Regebro wrote in Re: [Distutils] How 
Python can have CPAN.:

> I attached the file I used for the mirroring.

Cool.  I'll just integrate it with the current version of the PyPI version 
extractor from the distutilsversion project and upload it to PyPI as a new 
project.  

I'll make sure to put up a usable client class that pulls from the stored 
version and updates it, with appropriate caching similar to what you did in 
your code (and the way that I did it in my version as well).

That way, anyone wishing to write a tool has a drop-in class that will use 
minimal resources to get a full copy of the PyPI data.

I'll put a copy on one of our servers, set a sync job to run every day then 
push the result to S3.

I'll publish the URL on this list (once, it won't change).

Feel free to put a link to it from the PyPI site, and anyone needing the data 
can just pull it from there, presumably using the supplied class.

I'll probably start this tonight, I'll post it as soon as it's working.

S




___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PyPI Data Synchronizer

[sstein...@gmail.com]

On Dec 27, 2009, at 4:49 PM, Martin v. Löwis wrote:

>> I'll make sure to put up a usable client class that pulls from the
>> stored version and updates it, with appropriate caching similar to
>> what you did in your code (and the way that I did it in my version as
>> well).
> 
> Please make sure to change the User-Agent header in the requests, so
> that we can attribute accesses to this tool.

Ok, I'll identify the tool and my cron'd collector specifically so we can track 
resource usage for the tool in general and specifically for my collector, just 
in case it becomes an issue down the road.

Steve
aka/S
aka/ssteinerX


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How Python can have CPAN.

[sstein...@gmail.com]

On Dec 27, 2009, at 6:42 PM, david.l...@preisshare.net wrote:

>> On Sun, Dec 27, 2009 at 17:50, sstein...@gmail.com 
>> wrote:
>>> The tarred download would be really handy for this utility as, if
>>> there's no .pkl of the data, or the user requests it, I pull  fresh
>>> copy.
>> 
>> Right... Anyone want to host such a tarred download? :-)
>> I pretty much got the code, but don't feel like setting up and hosting
>> it right now. Maybe later.
> 
> I have some spare on a custom domain. What do you need ?

I have it covered.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] How Python can have CPAN.

[sstein...@gmail.com]

On Dec 30, 2009, at 1:48 PM, Sebastien Douche wrote:

> On Sun, Dec 27, 2009 at 11:47, Lennart Regebro  wrote:
> 
>> Out of a total of 8522 packages on PyPI, there are 203 packages (2.4%)
>> whose latest release does not provide either a package on PyPI, nor a
>> download url. Of these 16 does not provide any contact data.
> 
> Hi Lennart,
> Glad to see someone is interested by a PyPI mirror, I have one here
> and it's a pity.
> 
> Statistics (from the creation of the mirror / proxy. The goal is to
> avoid external download, like an internal debian mirror):
> 2009-12-15 21:37:20,855 DEBUG  Found (cached): 0
> 2009-12-15 21:37:20,855 DEBUG  Stored (downloaded):15367
> 2009-12-15 21:37:20,855 DEBUG  Not found (404):188
> 2009-12-15 21:37:20,855 DEBUG  Invalid packages:   0
> 2009-12-15 21:37:20,855 DEBUG  Invalid URLs:   54
> 2009-12-15 21:37:20,855 DEBUG  Runtime:208m38s
> 
> The root issue (for me) is: packages out of the PyPI. A lot of broken
> links, broken html pages or stupid scripts (cf. old SourceForge).

I will put a way of getting this data out, thanks for the heads up.

> Some examples:
> WARNING Unload downloading http://wiki.woodpecker.org.cn/moin/UliPad (timed 
> out)
> WARNING Unload downloading http://launchpad.net/mcrepogen/+download
> (The read operation timed out)
> WARNING Unload downloading http://launchpad.net/mcrepogen (The read
> operation timed out)
> WARNING Unload downloading https://launchpad.net/lovely.tal (The read
> operation timed out)
> WARNING Unload downloading ffnet.sourceforge.net (unknown url type:
> ffnet.sourceforge.net)
> WARNING Unload downloading http://pysqlite.org/ ((-3, 'Temporary
> failure in name resolution'))
> 
> 
>> Is there significant interest in doing this?
> 
> YES! ;)
> 
> In that case, what answer
>> options should we have?
> 
> Always upload a version to PyPI, the only way to have a reliable,
> solid and smart PyPI and an easy way to proxy-ing. Think the case
> where SF is down: No docutils. Zope server down: no Zope 2, no Zope3,
> no ZTK, no buildout... With a full mirror I don't care...
> 
> Note: I'm very happy when I see a distribution with:
> - a description
> - a summary (with examples if necessary)
> - a changelog (quick way to see what's new)
> - the name of the author and email (or maintainer)
> - contain files (with distribution name = package name, not MyPackage
> and mypackage)
> 
> 
> Like this :
> http://pypi.python.org/pypi/collective.portlet.relateditems/0.3.0
> 
> And not this:
> http://pypi.python.org/pypi/django-sphinxdoc/0.2

I have put this into my working spec document which I'll be publishing with the 
first version of the code (which won't have all the options implemented, but 
they'll be in the plan/issue tracker in case anyone wants to help).

Steve





___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [distribute] Strategy for a DVCS extension

[sstein...@gmail.com]

On Dec 30, 2009, at 1:59 PM, Sebastien Douche wrote:

> Hi folks :)
> I want to know the strategy to be followed in a DVCS extension. A
> small improvement I want to code is unique area for the version (and
> not version.cfg *and* setup.py)
> 
> What do you think? Any ideas welcome

Sorry, I'm not sure what you're proposing or asking.

A DVCS extension of what?

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] current preferred way to specify dependencies? future?

[sstein...@gmail.com]

On Jan 11, 2010, at 8:01 PM, David Lyon wrote:
> When I run the same thing:
> 
>> import platform
>> platform.machine()
>> 'x86'

Just as a data point, I get:

import platform
platform.machine()
'i386'

on a dual processor quad core Mac Pro.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] distribute 0.6.10 and convert_2to3_doctests

[sstein...@gmail.com]

On Jan 29, 2010, at 7:32 PM, Ben Finney wrote:

> "P.J. Eby"  writes:
> 
>> At 04:05 PM 1/29/2010 -0500, Barry Warsaw wrote:
>>> The important thing is to have exactly one place to set the package's
>>> version number.
>> 
>> Put it in setup.py, then.  If you absolutely must have a __version__
>> at runtime, use this to extract it from the installation metadata:
>> 
>>__version__ = pkg_resources.require('MyProject')[0].version
>> 
>> Mostly, though, I don't bother with having a __version__ in my modules
>> or packages any more, since you can just do the above if you want to
>> check the installed version of something.
> 
> That assumes that the only things that will need to query the package's
> version are Python modules. That's often not the case, especially when
> there are other tools (e.g. shell programs, make files, or any program
> not written in Python) that are part of the same package.
> 
> Better would be to have a *non-executable* data file containing the
> version string and other such package meta-data. “Query the metadata”
> should not necessarily imply “parse or execute a bunch of Python code”.

I'd love to have a standard, documented way to set/query module versions.  

I actually started this e-mail about a week ago when I went looking for a 
"standard" way to version something the other day and, after I couldn't find a 
definitive answer by Googling, went looking in the stdlib for guidance.

I started with two core modules, sys and distutils, and tried just doing the 
same things to both modules:

>>> import sys

>>> sys.version
'2.6.4 (r264:75821M, Oct 27 2009, 19:48:32) \n[GCC 4.0.1 (Apple Inc. build 
5493)]'

>>> sys.version_info
(2, 6, 4, 'final', 0)

>>> sys.__version__
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'module' object has no attribute '__version__'

#

>>> import distutils

>>> distutils.version


>>> distutils.version_info
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'module' object has no attribute 'version_info'

>>> distutils.__version__
'2.6.4'

The standard library, and modules in PyPI, are all over the place on it so 
maybe a completely new API/method is in order.

S

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig