Re: [Python-Dev] accept the wheel PEPs 425, 426, 427

2012-10-26 Thread Ronald Oussoren 

On 24 Oct, 2012, at 14:48, Daniel Holth  wrote:

> On Wed, Oct 24, 2012 at 7:28 AM, Ronald Oussoren  
> wrote:
>> 
>> On 18 Oct, 2012, at 19:29, Daniel Holth  wrote:
>> 
>>> I'd like to submit the Wheel PEPs 425 (filename metadata), 426
>>> (Metadata 1.3), and 427 (wheel itself) for acceptance. The format has
>>> been stable since May and we are preparing a patch to support it in
>>> pip, but we need to earn consensus before including it in the most
>>> widely used installer.
>> 
>> PEP 427:
>> 
>> * The installation section mentions that .py files should be compiled to 
>> .pyc/.pyo files, and that "Uninstallers should be smart enough to remove 
>> .pyc even if it is not mentioned in RECORD.".
>> 
>>   Wouldn't it be better to add the compiled files to the RECORD file? That 
>> would break the digital signature, but I'm not sure if verifying the 
>> signature post-installation is useful (or if it's even
>>   intended to work).
> 
> The trouble with mentioning .pyc files in RECORD is that someone can
> install Python 3.4, and suddenly you have additional .pyc files,
> approximately __pycache__/pyfile.cp34.pyc. So you should remove more
> than what you installed anyway.
> 
> You can't verify the signature post-installation. #!python and RECORD
> have been rewritten at this point.
> 
>> * Why is urlsafe_b64encode_nopad used to encode the hash in the record file, 
>> instead of the normal hex encoding that's directly supported by the hash 
>> module and system tools?
> 
> It's nice and small. The encoder is just
> base64.urlsafe_b64encode(digest).rstrip('=')

But is the size difference really important? The wheel file itself is 
compressed, and the additional amount of space needed on installation shouldn't 
be a problem.  The advantage of using hexdigest is that both the "classic" MD5 
checksum and the new tagged checksums you propose then use the same encoding 
for the signature.

> 
>> * The way to specify the required public key in package requirements in ugly 
>> (it looks like an abuse of setuptools' extras mechanism). Is there really no 
>> nicer way to specify this?
>> 
>> * As was noted before there is no threat model for the signature feature, 
>> which makes it hard to evaluate if the feature.  In particular, what is the 
>> advantage of this over PGP signatures of wheels? (PyPI already supports 
>> detached signatures, and such signatures are used more widely in the OSS 
>> world)
>> 
>> * RECORD.p7s is not described at all. I'm assuming this is intented to be a 
>> X.509 signature of RECORD in pkcs7 format. Why PKCS7 and not PEM? The latter 
>> seems to be easier to work with.
> 
> I am very confused about the idea that
> not-downloading-the-archive-you-expected (pypi accounts getting
> hacked, man-in-the-middle attacks, simply using the wrong index) is an
> unrealistic threat.

You don't mention the threats you try to protect against in the PEP.   Users 
are still somewhat vulnerable to the attacks to mention when the download new 
software, they still have to get the public key somewhere. In the example of 
using a requirements.txt file with public keys I'd still have to get that file 
from somewhere and maybe that location was attacked.

> 
> It might help to think of the wheel signing scheme as a more powerful
> version of the current #md5=digest instead of comparing it to PGP or
> TLS. An md5 sum verifies the integrity of a single archive, the wheel
> signing key verifies the integrity of any number of archives. Like the
> archive digest, wheel just explains how to attach the signature to the
> archive. A system for [automatically] trusting any particular key
> could be built separately.
> 
> Wheel's signing scheme is similar to jarsigner. The big advantage over
> PGP is that they are attached and less likely to get lost. PyPI still
> supports detached signatures, even on wheel files, but they are
> unpopular. Wheel gives you an additional different option.

RPM uses embedded PGP signatures, and those are easy enough to use.PGP 
signatures on PyPI require a PGP installation on the users machine, your scheme 
at least has the advantage of not needing additional software. 

> 
> Since the signature is over the unpacked contents, you can also change
> the compression algorithm in the zipfile or append another signature
> without invalidating the existing signature.
> 
> The simplified certificate model is inspired by SPKI/SDSI
> (http://world.std.com/~cme/html/spki.html), Convergence
> (http://convergence.io/) TACK (http://tack.io), and the general
> discussion about the brokenness of the certificate authority system.

I've added these to my reading list. I know just enough of crypto/signatures to 
be dangerous, which might explain why I worry about something that isn't old 
and used a lot.

> You get the raw public key without a claim that it represents anything
> or anyone.

Simularly to the CA system :-)

> 
> PKCS7 is the format that a US government user would be required to use
> with

Re: [Python-Dev] [Distutils] accept the wheel PEPs 425, 426, 427

2012-10-26 Thread Paul Moore 
On 26 October 2012 08:54, Ronald Oussoren  wrote:
>>
>> It's nice and small. The encoder is just
>> base64.urlsafe_b64encode(digest).rstrip('=')
>
> But is the size difference really important? The wheel file itself is 
> compressed, and the additional
> amount of space needed on installation shouldn't be a problem.  The advantage 
> of using hexdigest
> is that both the "classic" MD5 checksum and the new tagged checksums you 
> propose then use
> the same encoding for the signature.

I agree. This encoding seems to be a micro-optimisation with no real
justification. I'd prefer to see hexdigest used, as (a) it means md5
is not a special case, and (b) there's not a proliferation of 1-line
functions in use code doing that b64encode/strip dance.

With hexdigest, the syntax is just [algorithm=]hexdigest, where
algorithm defaults to md5. Much simpler to describe and work with.

Paul.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Bug Day in October

2012-10-26 Thread Todd Rovito 
On Wed, Oct 24, 2012 at 3:45 PM, Maciej Szulik  wrote:
> On 10/24/2012 03:19 AM, Éric Araujo wrote:
>>
>> Hello,
>>
>> Le 12/10/2012 13:50, Petri Lehtinen a écrit :
>>>
>>> It's two and a half weeks left, but I've not seen any announcements
>>> yet!
>>
>> Indeed, work and other commitments took over, so we (Montréal-Python)
>> decided to move the bug day instead of announcing it late.  The date
>> that would work for us is November 3rd.
>>
>> Brian, is it okay for Boston?
>> Maciej, what about your group?
>> Comitters, who could join on IRC?
>>
>> Sorry for the false start.
>>
>
> Eric,
> We have a meeting tomorrow, I'll talk to guys, but because we're
> starting Silesian Python Group, there won't be much interest yet.
> I'll try to work on that ;) maybe some time in future I could organize
> this kind of event and invite all of you to join us.
> Nonetheless I'll try to join both events, on this and next Saturday
> on IRC.

I will see if I can get some folks from Py Ohio to join in on November
3rd for sure I will be available to help.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

2012-10-26 Thread Python tracker 

ACTIVITY SUMMARY (2012-10-19 - 2012-10-26)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open3824 ( +9)
  closed 24283 (+34)
  total  28107 (+43)

Open issues with patches: 1730 


Issues opened (32)
==

#16274: test_asyncore failing on Solaris 10 2.7 (nitrogen/s10)
http://bugs.python.org/issue16274  reopened by trent

#16288: TextTestResult uses TestCase.__str__() which isn't customisabl
http://bugs.python.org/issue16288  opened by rbcollins

#16290: PyComplex_AsCComplex should allow __complex__ to return float.
http://bugs.python.org/issue16290  opened by mark.dickinson

#16291: Enable darwin-host cross compilation. Tested with build=linux,
http://bugs.python.org/issue16291  opened by Ray.Donnelly

#16292: Cross compilation fixes (general)
http://bugs.python.org/issue16292  opened by Ray.Donnelly

#16293: curses.ungetch raises OverflowError when given -1
http://bugs.python.org/issue16293  opened by Julian

#16296: Patch to fix building on Win32/64 under VS 2010
http://bugs.python.org/issue16296  opened by silverbacknet

#16298: httplib.HTTPResponse.read could potentially leave the socket o
http://bugs.python.org/issue16298  opened by eranrund

#16300: Windows raises ValueError for file:///file/not/exists instead 
http://bugs.python.org/issue16300  opened by orsenthil

#16301: localhost() and thishost() in urllib/request.py
http://bugs.python.org/issue16301  opened by orsenthil

#16304: re: Match Objects always have a boolean value of  True
http://bugs.python.org/issue16304  opened by vrutsky

#16306: Multiple error line for unknown command line parameter
http://bugs.python.org/issue16306  opened by tweksteen

#16307: multiprocess.pool.map_async callables not working
http://bugs.python.org/issue16307  opened by Bbb

#16308: Undocumented (?) behaviour change in argparse from 3.2.3 to 3.
http://bugs.python.org/issue16308  opened by telmich

#16309: "PYTHONPATH=" different from no PYTHONPATH at all
http://bugs.python.org/issue16309  opened by arigo

#16310: zipfile: allow surrogates in filenames
http://bugs.python.org/issue16310  opened by stefanholek

#16311: Use _PyUnicodeWriter API in text decoders
http://bugs.python.org/issue16311  opened by haypo

#16312: New command line option supported by unittest.main() for runni
http://bugs.python.org/issue16312  opened by piotr.dobrogost

#16316: Support xz compression in mimetypes module
http://bugs.python.org/issue16316  opened by serhiy.storchaka

#16317: Mention lzma module in tutorial
http://bugs.python.org/issue16317  opened by serhiy.storchaka

#16318: FTP_TLS in ftplib not supporting prot_p storlines in FTP7.5
http://bugs.python.org/issue16318  opened by Jeremy.Brock

#16320: Establish order in bytes/string dependencies
http://bugs.python.org/issue16320  opened by serhiy.storchaka

#16321: Move eq.h out of stringlib
http://bugs.python.org/issue16321  opened by serhiy.storchaka

#16322: time.tzname on Python 3.3.0 for Windows is decoded by wrong en
http://bugs.python.org/issue16322  opened by msmhrt

#16323: Wrong C API documentation for locale encoding
http://bugs.python.org/issue16323  opened by serhiy.storchaka

#16324: MIMEText __init__ does not support Charset instance
http://bugs.python.org/issue16324  opened by claudep

#16325: PEP 8 refers to reference to PEP 8
http://bugs.python.org/issue16325  opened by serhiy.storchaka

#16326: distutils build_ext fails to set library_dirs in 2.7.2 on Linu
http://bugs.python.org/issue16326  opened by Andy.Salnikov

#16327: subprocess.Popen leaks file descriptors on os.fork() failure
http://bugs.python.org/issue16327  opened by Mark.Gius

#16328: win_add2path.py sets wrong user path
http://bugs.python.org/issue16328  opened by Emil.Styrke

#16329: mimetypes does not support webm type
http://bugs.python.org/issue16329  opened by giampaolo.rodola

#16330: Use surrogate-related macros
http://bugs.python.org/issue16330  opened by serhiy.storchaka



Most recent 15 issues with no replies (15)
==

#16330: Use surrogate-related macros
http://bugs.python.org/issue16330

#16326: distutils build_ext fails to set library_dirs in 2.7.2 on Linu
http://bugs.python.org/issue16326

#16323: Wrong C API documentation for locale encoding
http://bugs.python.org/issue16323

#16321: Move eq.h out of stringlib
http://bugs.python.org/issue16321

#16320: Establish order in bytes/string dependencies
http://bugs.python.org/issue16320

#16317: Mention lzma module in tutorial
http://bugs.python.org/issue16317

#16316: Support xz compression in mimetypes module
http://bugs.python.org/issue16316

#16312: New command line option supported by unittest.main() for runni
http://bugs.python.org/issue16312

#16298: httplib.HTTPResponse.read could potentially leave the socket o
http://bugs.python.org/issue16298

#16287: Sporadic test_utime() failures on Solaris

[Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again, but this time more solid numbers)

2012-10-26 Thread Brett Cannon 
I re-ran the unladen benchmarks on my work machine and w/o the -b option
flipped on (i.e. more thorough benchmark numbers). I figured I would share
them now instead of after my PyCon Argentina talk in case people decide to
dig into the results now, find a pathological problem in CPython, and then
fix it before I give my presentation (if you have trouble running a
benchmark or it isn't available in the repo because it's one I hacked
together, just ask and I can help you run the benchmark if you want to try
to speed things up). I have colour-coded benchmarks based on whether it is
faster or slower in Python 3.3 (sorry for those of you who hate HTML email).

But the tl;dr message is that Python 3.3 looks good compared to Python 2.7
(the median benchmark score is 5% slower).

Worst benchmark is nosite_startup, best is telco. The benchmarks people
might want to analyze (i.e. more than 20% slower in Python 3.3) are
mako_v2, threaded_count, normal_startup, iterative_count, pathlib,
formatted_logging, and simple_logging.

###

Report on Linux 3.2.5-gg987 #1 SMP Fri Sep 14 02:36:36 PDT 2012 x86_64
x86_64
Total CPU cores: 12

### 2to3 ###
9.32 -> 8.98: 1.04x faster

### call_method ###
Min: 0.417756 -> 0.355247: 1.18x faster
Avg: 0.419688 -> 0.356382: 1.18x faster
Significant (t=92.85)
Stddev: 0.00604 -> 0.00577: 1.0479x smaller
Timeline: b'http://tinyurl.com/8soruze'

### call_method_slots ###
Min: 0.417611 -> 0.358451: 1.17x faster
Avg: 0.420761 -> 0.359676: 1.17x faster
Significant (t=88.70)
Stddev: 0.00605 -> 0.00588: 1.0291x smaller
Timeline: b'http://tinyurl.com/8uu8234'

### call_method_unknown ###
Min: 0.459057 -> 0.359327: 1.28x faster
Avg: 0.462929 -> 0.360410: 1.28x faster
Significant (t=137.99)
Stddev: 0.00698 -> 0.00583: 1.1969x smaller
Timeline: b'http://tinyurl.com/9mo7h24'

### call_simple ###
Min: 0.341689 -> 0.265289: 1.29x faster
Avg: 0.343003 -> 0.266503: 1.29x faster
Significant (t=124.20)
Stddev: 0.00555 -> 0.00511: 1.0859x smaller
Timeline: b'http://tinyurl.com/9pnn7q4'

### chameleon ###
Min: 0.072232 -> 0.062713: 1.15x faster
Avg: 0.074588 -> 0.064261: 1.16x faster
Significant (t=33.74)
Stddev: 0.00284 -> 0.00245: 1.1599x smaller
Timeline: b'http://tinyurl.com/8my8afl'

### chaos ###
Min: 0.313727 -> 0.367015: 1.17x slower
Avg: 0.317568 -> 0.371473: 1.17x slower
Significant (t=-26.72)
Stddev: 0.00962 -> 0.01053: 1.0942x larger
Timeline: b'http://tinyurl.com/9y2u6kh'

### django ###
Min: 0.798331 -> 0.855461: 1.07x slower
Avg: 0.801109 -> 0.860996: 1.07x slower
Significant (t=-87.43)
Stddev: 0.00336 -> 0.00348: 1.0356x larger
Timeline: b'http://tinyurl.com/9sf95pq'

### fannkuch ###
Min: 1.364705 -> 1.327680: 1.03x faster
Avg: 1.380412 -> 1.337467: 1.03x faster
Significant (t=10.48)
Stddev: 0.02056 -> 0.02040: 1.0077x smaller
Timeline: b'http://tinyurl.com/9r2vq6g'

### fastpickle ###
Min: 0.763479 -> 0.805715: 1.06x slower
Avg: 0.770036 -> 0.810855: 1.05x slower
Significant (t=-12.73)
Stddev: 0.01618 -> 0.01589: 1.0180x smaller
Timeline: b'http://tinyurl.com/9rvqo4s'

### fastunpickle ###
Min: 0.588694 -> 0.663616: 1.13x slower
Avg: 0.596622 -> 0.672418: 1.13x slower
Significant (t=-23.22)
Stddev: 0.01503 -> 0.01752: 1.1656x larger
Timeline: b'http://tinyurl.com/9eggn34'

### float ###
Min: 0.363234 -> 0.344408: 1.05x faster
Avg: 0.376159 -> 0.354165: 1.06x faster
Significant (t=8.76)
Stddev: 0.01282 -> 0.01227: 1.0455x smaller
Timeline: b'http://tinyurl.com/8d6rcb8'

### formatted_logging ###
Min: 0.330988 -> 0.400309: 1.21x slower
Avg: 0.335522 -> 0.408920: 1.22x slower
Significant (t=-33.48)
Stddev: 0.00989 -> 0.01194: 1.2076x larger
Timeline: b'http://tinyurl.com/9ll7dqk'

### genshi ###
Min: 0.229140 -> 0.251766: 1.10x slower
Avg: 0.232124 -> 0.257252: 1.11x slower
Significant (t=-40.24)
Stddev: 0.00516 -> 0.00564: 1.0925x larger
Timeline: b'http://tinyurl.com/9dpuuaw'

### go ###
Min: 0.632778 -> 0.710382: 1.12x slower
Avg: 0.636143 -> 0.716748: 1.13x slower
Significant (t=-37.61)
Stddev: 0.00186 -> 0.01504: 8.0815x larger
Timeline: b'http://tinyurl.com/8s7vw74'

### hexiom2 ###
Min: 150.982155 -> 154.702444: 1.02x slower
Avg: 151.194622 -> 154.780953: 1.02x slower
Significant (t=-15.83)
Stddev: 0.30047 -> 0.11103: 2.7063x smaller
Timeline: b'http://tinyurl.com/8rkkduv'

### iterative_count ###
Min: 0.117036 -> 0.156752: 1.34x slower
Avg: 0.120802 -> 0.172218: 1.43x slower
Significant (t=-34.92)
Stddev: 0.00542 -> 0.00889: 1.6422x larger
Timeline: b'http://tinyurl.com/9x9rtnk'

### json_dump_v2 ###
Min: 3.449868 -> 3.522645: 1.02x slower
Avg: 3.467124 -> 3.541902: 1.02x slower
Significant (t=-13.20)
Stddev: 0.02701 -> 0.02960: 1.0959x larger
Timeline: b'http://tinyurl.com/8bsz64a'

### json_load ###
Min: 0.981740 -> 0.567611: 1.73x faster
Avg: 0.986729 -> 0.572975: 1.72x faster
Significant (t=128.95)
Stddev: 0.01796 -> 0.01386: 1.2955x smaller
Timeline: b'http://tinyurl.com/93txokx'

### mako_v2 ###
Min: 0.083660 -> 0.243323: 2.91x slower

Re: [Python-Dev] Anyone have a Universal setter example?

2012-10-26 Thread Oleg Broytman 
Hello.

   Let me remind you this mailing list is to work on developing Python
(adding new features to Python itself and fixing bugs); if you're having
problems using Python, please find another forum. Probably
python-list/comp.lang.python mailing list/news group is the best place;
there are Python developers who participate in it; you may get a faster,
and probably more complete, answer there. See
http://www.python.org/community/ for other lists/news groups/fora. Thank
you for understanding.

On Fri, Oct 26, 2012 at 05:01:09PM -0400, Robert Steckroth 
 wrote:
> 1. Is it appropriate to post usage questions regarding the Python C
> API to this list like the questions below?

  I think not until you ask a question in python-list/comp.lang.python
and don't get any reply.

Oleg.
-- 
 Oleg Broytmanhttp://phdru.name/[email protected]
   Programmers don't die, they just GOSUB without RETURN.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Anyone have a Universal setter example?

2012-10-26 Thread Tres Seaver 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 10/26/2012 05:01 PM, Robert Steckroth wrote:

> 1. Is it appropriate to post usage questions regarding the Python C 
> API to this list like the questions below?

The C-API SIG would likely be a better forum:

  [email protected]
  gmane:  gmane.comp.python.capi


Tres.
- -- 
===
Tres Seaver  +1 540-429-0999  [email protected]
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlCLA2oACgkQ+gerLs4ltQ6JnACg004jmBpUDRX1TBHaFNkg7JW0
7twAoI/s228R0TnaBpqiImJsjGg1hJKp
=IAPl
-END PGP SIGNATURE-

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com