ANN: eGenix mxODBC Connect 2.0.5 - Python ODBC Database Interface
ANNOUNCING eGenix.com mxODBC Connect Python ODBC Database Interface Version 2.0.5 mxODBC Connect is our commercially supported client-server product for connecting Python applications to relational databases in a truly platform independent way. This announcement is also available on our web-site for online reading: http://www.egenix.com/company/news/eGenix-mxODBC-Connect-2.0.5-GA.html INTRODUCTION The mxODBC Connect Database Interface for Python allows users to easily connect Python applications to all major databases on the market today in a highly portable, convenient and secure way. Python Database Connectivity the Easy Way - Unlike our mxODBC Python extension, mxODBC Connect is designed as client-server application, so you no longer need to find production quality ODBC drivers for all the platforms you target with your Python application. Instead you use an easy to install royalty-free Python client library which connects directly to the mxODBC Connect database server over the network. This makes mxODBC Connect a great basis for writing cross-platform multi-tier database applications and utilities in Python, especially if you run applications that need to communicate with databases such as MS SQL Server and MS Access, Oracle Database, IBM DB2 and Informix, Sybase ASE and Sybase Anywhere, MySQL, PostgreSQL, SAP MaxDB and many more, that run on Windows or Linux machines. Ideal for Database Driven Client Applications - By removing the need to install and configure ODBC drivers on the client side and dealing with complicated network setups for each set of drivers, mxODBC Connect greatly simplifies deployment of database driven client applications, while at the same time making the network communication between client and database server more efficient and more secure. For more information, please have a look at the mxODBC Connect product page, in particular, the full list of available features. For more information, please see the product page: http://www.egenix.com/products/python/mxODBCConnect/ NEWS The 2.0.5 release of mxODBC Connect includes the following enhancements and fixes: Security Enhancements - * Upgraded client and server to the most recent eGenix pyOpenSSL Distribution 0.13.3.1.0.1.7 (see http://egenix.com/go54 for details) to address the recently found Heartbleed Bug in OpenSSL 1.0.1 - 1.0.1f: - CVE-2014-0160 (Heartbleed Bug): A missing bounds check in the handling of the TLS heartbeat extension can be used to reveal up to 64kB of memory to a connected client or server. This issue did not affect versions of OpenSSL prior to 1.0.1. For information, also have a look at the Heartbleed Bug website: http://heartbleed.com/ For the full set of changes, please check the mxODBC Connect change log: http://www.egenix.com/products/python/mxODBCConnect/changelog.html mxODBC Connect 2.0 Highlights - mxODBC Connect 2.0 was released on 2012-08-20. These are the most important highlights: * mxODBC Connect Server now uses mxODBC 3.2 internally and makes its API available in the mxODBC Connect Client. This is a major step forward from the mxODBC 3.0 version used in mxODBC Connect Server 1.0. * We've added native Windows x64 builds. * mxODBC Connect Client now integrates directly with gevent, allowing client applications to run asynchronous tasks while performing remote database queries. Please see the release announcement for full details: http://www.egenix.com/company/news/eGenix-mxODBC-Connect-2.0.0-GA.html UPGRADING You are encouraged to upgrade to this latest mxODBC Connect release. When upgrading, please always upgrade both the server and the client installations to the same version - even for patch level releases. Customers who have purchased mxODBC Connect 2.0 licenses can continue to use their licenses with this patch level release. Customers who have purchased mxODBC Connect 1.x licenses can request 20% discount coupons for upgrade purchases. Please contact the eGenix.com Sales Team (sa...@egenix.com) with your existing license serials for details. Users of our stand-alone mxODBC product will have to purchase new licenses from our online shop in order to use mxODBC Connect. You can request 30-day evaluation licenses by visiting our web-site or writing to sa...@egenix.com, stating your name (or the name of the company) and the number of eval licenses that you need.
Wing IDE 5.0.6 released
Hi, Wingware has released version 5.0.6 of Wing IDE, our cross-platform integrated development environment for the Python programming language. Wing IDE includes a professional quality code editor with vi, emacs, visual studio, and other key bindings, auto-completion, call tips, goto-definition, find uses, refactoring, context-aware auto-editing, a powerful graphical debugger, version control, unit testing, search, and many other features. For details see http://wingware.com/ Changes in this minor release include: * New auto-editing operation for starting a new block when ':' is pressed while a range of lines is selected. This also auto-enters 'except:' if 'try' is then entered. * Offer completion on argument names in a def * New Editor Block Comment Style preference option to use PEP8 style indented '# ' for commenting out code * On OS X, Wing no longer quits when the last window is closed, to match the standard for Mac applications * Restart correctly after downloading patches * Pick up changes made in the debug dialog when debugging a named entry point * Avoid hiding file selection auto-completer popup too soon * Set sys.path[0] correctly in Python Shell on OS X and Linux * Fix cleanup after auto-invocation that spans multiple lines * About 15 other bug fixes For details see http://wingware.com/pub/wingide/5.0.6/CHANGELOG.txt A summary of new features in Wing 5: * Redesigned GUI based on Qt and PySide * Native GUI on OS X (and better overall OS-native look and feel) * Tools and editors can be dragged around * Toolbar and editor and Project context menus are configurable * Optional mode that opens different sets of files in each editor split * Sharable color palettes and syntax highlighting configurations * Auto-editing is on by default (except some operations that have a learning curve) * Named file sets * Sharable launch configurations * Named entry points * More control over unit testing environment * Lockable editor splits * Initial preferences dialog for new users * Support for Python 3.4 * Support for Django 1.6 * Support for matplotlib on Anaconda and with MacOSX backend * Improved Source Assistant with PEP 287 docstring rendering and return types * Improved integrated and PDF documentation For more information on what's new in Wing 5, see http://wingware.com/wingide/whatsnew Free trial: http://wingware.com/wingide/trial Downloads: http://wingware.com/downloads Feature list: http://wingware.com/wingide/features Sales: http://wingware.com/store/purchase Upgrades: https://wingware.com/store/upgrade Questions? Don't hesitate to email us at supp...@wingware.com. Thanks, -- Stephan Deibel Wingware | Python IDE The Intelligent Development Environment for Python Programmers wingware.com -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
ANN: Python Meeting Düsseldorf - 29.04.2014
[This announcement is in German since it targets a local user group meeting in Düsseldorf, Germany] ANKÜNDIGUNG Python Meeting Düsseldorf http://pyddf.de/ Ein Treffen von Python Enthusiasten und Interessierten in ungezwungener Atmosphäre. Dienstag, 29.04.2014, 19:00 Uhr Raum 1, 2.OG im Bürgerhaus Stadtteilzentrum Bilk Düsseldorfer Arcaden, Bachstr. 145, 40217 Düsseldorf Diese Nachricht ist auch online verfügbar: http://www.egenix.com/company/news/Python-Meeting-Duesseldorf-2014-04-29 NEUIGKEITEN * Bereits angemeldete Vorträge: Charlie Clark Status openpyxl, bzw. Lösung neuer Probleme IndexedList - eine Liste optimiert für in Abfragen Bericht von der PyCon 2014 in Montreal Marc-Andre Lemburg Python Code mit lib2to3 modernisieren DDOS Attacken mit Python bekämpfen Bericht von der FOSDEM 2014 Wir suchen noch weitere Vorträge. Bei Interesse, bitte unter i...@pyddf.de melden. * Geänderte Startzeit: Dieses Mal treffen wir uns erst um 19:00 Uhr im Bürgerhaus in den Düsseldorfer Arcaden, da wir keinen Termin für 18 Uhr bekommen haben. Hier eine kurze Beschreibung: Das Bürgerhaus teilt sich den Eingang mit dem Schwimmbad und befindet sich an der Seite der Tiefgarageneinfahrt der Düsseldorfer Arcaden. Über dem Eingang steht ein großes Schwimm'in Bilk Logo. Hinter der Tür direkt links zu den zwei Aufzügen, dann in den 2. Stock hochfahren. Der Eingang zum Raum 1 liegt direkt links, wenn man aus dem Aufzug kommt. Google Street View: http://bit.ly/11sCfiw EINLEITUNG Das Python Meeting Düsseldorf ist eine regelmäßige Veranstaltung in Düsseldorf, die sich an Python Begeisterte aus der Region wendet: * http://pyddf.de/ Einen guten Überblick über die Vorträge bietet unser YouTube-Kanal, auf dem wir die Vorträge nach den Meetings veröffentlichen: * http://www.youtube.com/pyddf/ Veranstaltet wird das Meeting von der eGenix.com GmbH, Langenfeld, in Zusammenarbeit mit Clark Consulting Research, Düsseldorf: * http://www.egenix.com/ * http://www.clark-consulting.eu/ PROGRAMM Das Python Meeting Düsseldorf nutzt eine Mischung aus Open Space und Lightning Talks, wobei die Gewitter bei uns auch schon mal 20 Minuten dauern können ;-). Lightning Talks können vorher angemeldet werden, oder auch spontan während des Treffens eingebracht werden. Ein Beamer mit XGA Auflösung steht zur Verfügung. Folien bitte als PDF auf USB Stick mitbringen. Lightning Talk Anmeldung bitte formlos per EMail an i...@pyddf.de KOSTENBETEILIGUNG Das Python Meeting Düsseldorf wird von Python Nutzern für Python Nutzer veranstaltet. Um die Kosten zumindest teilweise zu refinanzieren, bitten wir die Teilnehmer um einen Beitrag in Höhe von EUR 10,00 inkl. 19% Mwst, Schüler und Studenten zahlen EUR 5,00 inkl. 19% Mwst. Wir möchten alle Teilnehmer bitten, den Betrag in bar mitzubringen. ANMELDUNG Da wir nur für ca. 20 Personen Sitzplätze haben, möchten wir bitten, sich per EMail anzumelden. Damit wird keine Verpflichtung eingegangen. Es erleichtert uns allerdings die Planung. Meeting Anmeldung bitte formlos per EMail an i...@pyddf.de WEITERE INFORMATIONEN Weitere Informationen finden Sie auf der Webseite des Meetings: http://pyddf.de/ Mit freundlichen Grüßen, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Inconsistent viewkeys behaviour
Is there any reference for this strange behaviour on Python 2: set() dict().viewkeys() Traceback (most recent call last): File stdin, line 1, in module TypeError: can only compare to a set dict().viewkeys() set() False ? -- https://mail.python.org/mailman/listinfo/python-list
Re: Sorting list alphabetically
Terry Reedy wrote: On 4/27/2014 6:40 AM, Kev Dwyer wrote: Igor Korot wrote: Hi, ALL, I need to perform a subj. Looking at the Google I found following thread with explanation: http://stackoverflow.com/questions/36139/how-do-i-sort-a-list-of- strings- in-python However, doing this in my python shell produces an error: C:\Documents and Settings\Igor.FORDANWORK\My Documents\GitHub\webapppython Python 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. import locale locale.setlocale(locale.LC_ALL, 'en_US.UTF-8') Traceback (most recent call last): File stdin, line 1, in module File c:\python27\lib\locale.py, line 547, in setlocale return _setlocale(category, locale) locale.Error: unsupported locale setting What is wrong with this? Thank you. Hello Igor, Windows maintains it's own names for locales, so you need to supply the Windows name if you're workong on Windows. You might find these links helpful: http://stackoverflow.com/questions/19709026/how-can-i-list-all-available- windows-locales-in-python-console This one says to look at locale.locala_alias, but that is not helpful. for k, v in la.items(): if v.startswith ('en') and 'UTF' in v: print(k, # , v) universal.utf8@ucs4 # en_US.UTF-8 But that local does not work on my machine. locale.setlocale(locale.LC_ALL, 'en_US.UTF-8') Traceback (most recent call last): File pyshell#20, line 1, in module locale.setlocale(locale.LC_ALL, 'en_US.UTF-8') File C:\Programs\Python34\lib\locale.py, line 592, in setlocale return _setlocale(category, locale) locale.Error: unsupported locale setting locale.locale_alias must not be machine limited. https://mail.python.org/pipermail/python-list/2009-February/525427.html This merely says to look at a now dead link. My mistake for not checking the link; this one works: http://msdn.microsoft.com/en-us/library/hzz3tw78 Hello Terry, Regarding your second point, my mistake in not checking the link: I'd seen a similar one elsewhere and assumed they were the same. This link should work: http://msdn.microsoft.com/en-us/library/hzz3tw78 As to your first point, you're right, it seems setlocale(locale.LC_ALL, 'en_US.UTF-8') doesn't work on Windows. It seems the locale name needs to be one of the aliases provided at http://msdn.microsoft.com/en-us/library/39cwe7zf, so for example locale.setlocale(locale.LC_ALL, 'usa') returns 'English_United States.1252'. Do you know if this is something people programming on Windows should just be aware of, or is there a case for a hint of some kind in the locale module documentation? Cheers, Kev -- https://mail.python.org/mailman/listinfo/python-list
Re: Help with changes in traceback stack from Python 2.7 to Python 3.x
On Mon, 28 Apr 2014 12:19:02 +1000, Chris Angelico wrote: On Mon, Apr 28, 2014 at 12:11 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: Suppose we could pass variables directly to the constructor, like this: a = b = 2 L = [1, 2, 3] dctA = dict(a, b, L[1], 2, 1+1) Obviously all five values are 2, but what are the keys? The dict construct receives five arguments, all bound to the object 2. (That might be the same object five times, or five distinct objects, all with the same value of 2.) Just to nit-pick: A conforming Python implementation might pass up to four distinct objects, but the first two MUST be the same object. :) So it does, thanks for the correction. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Unicode in Python
Le samedi 26 avril 2014 15:38:29 UTC+2, Ian a écrit : On Apr 26, 2014 3:46 AM, Frank Millman fr...@chagford.com wrote: wxjm...@gmail.com wrote in message news:03bb12d8-93be-4ef6-94ae-4a02789ae...@googlegroups.com... == I wrote once 90 % of Python 2 apps (a generic term) supposed to process text, strings are not working. In Python 3, that's 100 %. It is somehow only by chance, apps may give the illusion they are properly working. It is quite frustrating when you make these statements without explaining what you mean by 'not working'. As far as anybody has been able to determine, what jmf means by not working is that strings containing the EURO character are handled less efficiently than strings that do not contain it in certain contrived test cases. Python 2.7 + cp1252: - Solid and coherent system (nothing to do with the Euro). Python 3: - It missed the unicode shift. - Covering the whole unicode range will not make Python a unicode compliant product. - Flexible String Representation (a problem per se), a mathematical absurditiy which does the opposite of the coding schemes endorsed by Unicord.org (sheet of paper and pencil!) - Very deeply buggy (quadrature of the circle problem). Positive side: - A very nice tool to teach the coding of characters and unicode. jmf -- https://mail.python.org/mailman/listinfo/python-list
Re: possible bug in re expression?
On 25/04/2014 19:32, Terry Reedy wrote: .. I suppose that one could argue that '{' alone should be treated as special immediately, and not just when a matching '}' is found, and should disable other special meanings. I wonder what JS does if there is no matching '}'? well in fact I suspect this is my mistranslation of the original new RegExp('.{1,' + (+size) + '}', 'g') my hacked up translator doesn't know what that means. I suspect that (+size) is an attempt to force size to an integer prior to it being forced to a string. I used to believe that conversion was always written 0-x, but experimentally (+3) ends up as 3 not 3. Naively, I imagined that re would complain about ambiguous regular expressions, but in the regexp world n problems -- n+1 problems almost surely so I should have anticipated it :) Does this in fact that almost any broken regexp specification will silently fail because re will reset and consider any metacharacter as literal? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Inconsistent viewkeys behaviour
On 4/28/2014 2:22 AM, Joshua Landau wrote: Is there any reference for this strange behaviour on Python 2: set() dict().viewkeys() Traceback (most recent call last): File stdin, line 1, in module TypeError: can only compare to a set dict().viewkeys() set() False The left operand determines the result. The manual specifies that and do not have to be consistent. But I suspect that when 3.x dict.keys() was backported to 2.7.0, no one thought to update set, whereas the backported key view code already had the comparison. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Proper deletion of selected items during map iteration in for loop: Thanks to all
Chris Angelico ros...@gmail.com wrote: # Snapshot of keys: for k in list(d): if f(k): del d[k] No extra loop at the end, no switching out and in of contents, just one little change in the loop header. Obviously you don't want to do this when you're deleting two out of three billion, but for smallish dicts, that won't make a visible change. Even if you have three billion keys, the extra memory needed to create a list that references those keys is going to be a lot less than the memory used by the keys themselves. For example if the keys are 6 character strings then each string needs I think at least 45 bytes (64 bit Python 2.x, up to double that in Python 3.x) but the list only needs one 8 byte pointer per key. I would always choose this simple solution until such time as it is proved to be a problem. -- Duncan Booth -- https://mail.python.org/mailman/listinfo/python-list
Re: possible bug in re expression?
On Mon, 28 Apr 2014 10:47:54 +0100, Robin Becker wrote: Does this in fact that almost any broken regexp specification will silently fail because re will reset and consider any metacharacter as literal? Well, I don't know about almost any, but at least some broken regexes will explicitly fail: py import re py re.search('*', 123*4) Traceback (most recent call last): File stdin, line 1, in module File /usr/local/lib/python3.3/re.py, line 161, in search return _compile(pattern, flags).search(string) [...] File /usr/local/lib/python3.3/sre_parse.py, line 552, in _parse raise error(nothing to repeat) sre_constants.error: nothing to repeat (For brevity I have abbreviated the traceback.) -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
problem with regex
i want to find a specific urls from a txt file but i have some issus. First when i take just two lines from the file with copy paste and assign it to a variable like this and it works only with triple quotes test='''_*_n.jpg,timelineCoverPhoto:{\focus\:{\x\:0.5,\y\:0.386925795053},\photo\:{\__type__\:{\name\:\Photo\},\image_lowres\:{\uri\:\https://fbcdn-photos-f-a.akamaihd.net/*-*-*/*_*_*_a.jpg\,\width\:180,\height\:179}}},subscribeStatus:IS_SUBSCRIBED,smallPictureUrl:https://fbcdn-profile-a.akamaihd.net/*-*-*/s100x100/*_*_*_s.jpg,contactId:*==,contactType:USER,friendshipStatus:ARE_FRIENDS,graphApiWriteId:contact_*:*:*,hugePictureUrl:https://fbcdn-profile-a.akamaihd.net/hprofile-ak-frc3/*_*_*_n.jpg,profileFbid:1284503586,isMobilePushable:NO,lookupKey:null,name:{displayName:* *,firstName:*,lastName:*},nameSearchTokens:[*,*],phones:[],phoneticName:{displayName:null,firstName:null,lastName:null},isMemorialized:false,communicationRank:1.1144714,canViewerSendGift:false,canMessage:true} *=={bigPictureUrl:https://fbcdn-profile-a.akamaihd.net/hprofile-ak-ash3/*.*.*.*/s200x200/*_*_*_n.jpg,timelineCoverPhoto:{\focus\:{\x\:0.5,\y\:0.49137931034483},\photo\:{\__type__\:{\name\:\Photo\},\image_lowres\:{\uri\:\https://fbcdn-photos-h-a.akamaihd.net/*-*-*/*_*_*_a.jpg\,\width\:180,\height\:135}}},subscribeStatus:IS_SUBSCRIBED,smallPictureUrl:https://fbcdn-profile-a.akamaihd.net/*-*-*/*.*.*.*/s100x100/*_*_*_a.jpg,contactId:*==,contactType:USER,friendshipStatus:ARE_FRIENDS,graphApiWriteId:contact_*:*:*,hugePictureUrl:https://fbcdn-profile-a.akamaihd.net/hprofile-ak-ash3/c0.0.540.540/*_*_*_n.jpg,profileFbid:*,isMobilePushable:YES,lookupKey:null,name:{displayName:* *,firstName:*,lastName:*},nameSearchTokens:[*,*],phones:[],phoneticName:{displayName:null,firstName:null,lastName:null},isMemorialized:false,communicationRank:1.2158813,canViewerSendGift:false,canMessage:true}''' uri = re.findall(r'''uri\:\https://fbcdn-(a-z|photos)?([^\' ]+)''',test) print uri it works fine and i have my result [('photos', '-f-a.akamaihd.net/*-*-*/*_*_*_a.jpg'), ('photos', '-h-a.akamaihd.net/*-*-*/*_*_*_a.jpg')] but if a take those lines and save it into a txt file like the original is without the quotes and do the following datafile=open('a.txt','r') data_array='' for line in datafile: data_array=data_array+line uri = re.findall(r'''uri\:\https://fbcdn-(a-z|photos)?([^\' ]+)''',data_array) after printing uri it gives an empty list,. what to do to make it work for the lines of a txt file -- https://mail.python.org/mailman/listinfo/python-list
problem with regex
i want to find a specific urls from a txt file but i have some issus. First when i take just two lines from the file with copy paste and assign it to a variable like this and it works only with triple quotes test='''_*_n.jpg,timelineCoverPhoto:{\focus\:{\x\:0.5,\y\:0.386925795053},\photo\:{\__type__\:{\name\:\Photo\},\image_lowres\:{\uri\:\https://fbcdn-photos-f-a.akamaihd.net/*-*-*/*_*_*_a.jpg\,\width\:180,\height\:179}}},subscribeStatus:IS_SUBSCRIBED,smallPictureUrl:https://fbcdn-profile-a.akamaihd.net/*-*-*/s100x100/*_*_*_s.jpg,contactId:*==,contactType:USER,friendshipStatus:ARE_FRIENDS,graphApiWriteId:contact_*:*:*,hugePictureUrl:https://fbcdn-profile-a.akamaihd.net/hprofile-ak-frc3/*_*_*_n.jpg,profileFbid:*,isMobilePushable:NO,lookupKey:null,name:{displayName:* *,firstName:*,lastName:*},nameSearchTokens:[*,*],phones:[],phoneticName:{displayName:null,firstName:null,lastName:null},isMemorialized:false,communicationRank:1.1144714,canViewerSendGift:false,canMessage:true} *=={bigPictureUrl:https://fbcdn-profile-a.akamaihd.net/hprofile-ak-ash3/*.*.*.*/s200x200/*_*_*_n.jpg,timelineCoverPhoto:{\focus\:{\x\:0.5,\y\:0.49137931034483},\photo\:{\__type__\:{\name\:\Photo\},\image_lowres\:{\uri\:\https://fbcdn-photos-h-a.akamaihd.net/*-*-*/*_*_*_a.jpg\,\width\:180,\height\:135}}},subscribeStatus:IS_SUBSCRIBED,smallPictureUrl:https://fbcdn-profile-a.akamaihd.net/*-*-*/*.*.*.*/s100x100/*_*_*_a.jpg,contactId:*==,contactType:USER,friendshipStatus:ARE_FRIENDS,graphApiWriteId:contact_*:*:*,hugePictureUrl:https://fbcdn-profile-a.akamaihd.net/hprofile-ak-ash3/c0.0.540.540/*_*_*_n.jpg,profileFbid:*,isMobilePushable:YES,lookupKey:null,name:{displayName:* *,firstName:*,lastName:*},nameSearchTokens:[*,*],phones:[],phoneticName:{displayName:null,firstName:null,lastName:null},isMemorialized:false,communicationRank:1.2158813,canViewerSendGift:false,canMessage:true}''' uri = re.findall(r'''uri\:\https://fbcdn-(a-z|photos)?([^\' ]+)''',test) print uri it works fine and i have my result [('photos', '-f-a.akamaihd.net/*-*-*/*_*_*_a.jpg'), ('photos', '-h-a.akamaihd.net/*-*-*/*_*_*_a.jpg')] but if a take those lines and save it into a txt file like the original is without the quotes and do the following datafile=open('a.txt','r') data_array='' for line in datafile: data_array=data_array+line uri = re.findall(r'''uri\:\https://fbcdn-(a-z|photos)?([^\' ]+)''',data_array) after printing uri it gives an empty list,. what to do to make it work for the lines of a txt file -- https://mail.python.org/mailman/listinfo/python-list
Re: problem with regex
In article caeba811-441e-42a0-9b2b-c743205b1...@googlegroups.com, dimm...@gmail.com wrote: i want to find a specific urls from a txt file but i have some issus. First when i take just two lines from the file with copy paste and assign it to a variable like this and it works only with triple quotes test='''long string elided''' [...] but if a take those lines and save it into a txt file like the original is without the quotes [it doesn't work] I suspect this has nothing to do with regular expressions, but it's just about string management. The first thing you want to do is verify that the text you are reading in from the file is the same as the text you have in triple quotes. So, write a program like this: test='''long string elided''' datafile=open('a.txt','r') data_array='' for line in datafile: data_array=data_array+line print test == data_array If that prints True, then you've got the same text in both cases (and you can go on to looking for other problems). I suspect it will print False, though. So, now your task is to figure out where those two strings differ. Maybe something like: for c1, c2 in zip(test, data_array): print c1 == c2, repr(c1), repr(c2) and look for the first place they're not the same. Hopefully that will give you a clue what's going wrong. -- https://mail.python.org/mailman/listinfo/python-list
Convert numpy array to single number
I have a numpy array consisting of 1s and zeros for representing binary numbers: e.g. binary array([ 1., 0., 1., 0.]) I wish the array to be in the form 1010, so it can be manipulated. I do not want to use built in binary converters as I am trying to build my own. -- https://mail.python.org/mailman/listinfo/python-list
Re: possible bug in re expression?
On 28/04/2014 12:49, Steven D'Aprano wrote: .. Well, I don't know about almost any, but at least some broken regexes will explicitly fail: py import re sre_constants.error: nothing to repeat (For brevity I have abbreviated the traceback.) so there is intent to catch some specification errors. I've abandoned this translation anyhow as all that was intended was to split the string into non-overlapping strings of size at most k. I find this works faster than the regexp even if the regexp is pre-compiled. [p[i:i+k] for i in xrange(0,len(p),k)] -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Convert numpy array to single number
On Mon, 28 Apr 2014 06:04:02 -0700, mboyd02255 wrote: I have a numpy array consisting of 1s and zeros for representing binary numbers: e.g. binary array([ 1., 0., 1., 0.]) I wish the array to be in the form 1010, so it can be manipulated. I do not want to use built in binary converters as I am trying to build my own. Did you have a question, or are you just sharing? -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
Significant digits in a float?
I'm using Python 2.7 I have a bunch of floating point values. For example, here's a few (printed as reprs): 38.0 41.2586 40.752801 49.25 33.7951994 36.8371996 34.1489 45.5 Fundamentally, these numbers have between 0 and 4 decimal digits of precision, and I want to be able to intuit how many each has, ignoring the obvious floating point roundoff problems. Thus, I want to map: 38.0 == 0 41.2586 == 4 40.752801 == 4 49.25 == 2 33.7951994 == 4 36.8371996 == 4 34.1489 == 4 45.5 == 1 Is there any clean way to do that? The best I've come up with so far is to str() them and parse the remaining string to see how many digits it put after the decimal point. The numbers are given to me as Python floats; I have no control over that. I'm willing to accept that fact that I won't be able to differentiate between float(38.0) and float(38.). Both of those map to 1, which is OK for my purposes. --- Roy Smith r...@panix.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Significant digits in a float?
On 4/28/14 12:00 PM, Roy Smith wrote: Fundamentally, these numbers have between 0 and 4 decimal digits of precision, and I want to be able to intuit how many each has, ignoring the obvious floating point roundoff problems. Thus, I want to map: 38.0 == 0 41.2586 == 4 40.752801 == 4 49.25 == 2 33.7951994 == 4 36.8371996 == 4 34.1489 == 4 45.5 == 1 Is there any clean way to do that? The best I've come up with so far is to str() them and parse the remaining string to see how many digits it put after the decimal point. That sounds like a pretty clean way: len(str(num).partition(.)[2]), though it also sounds like you understand all of the inaccuracies in that technique. -- Ned Batchelder, http://nedbatchelder.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Significant digits in a float?
On Tue, Apr 29, 2014 at 2:00 AM, Roy Smith r...@panix.com wrote: I have a bunch of floating point values. For example, here's a few (printed as reprs): 38.0 41.2586 40.752801 49.25 33.7951994 36.8371996 34.1489 45.5 Fundamentally, these numbers have between 0 and 4 decimal digits of precision... Terminology question: Why do you count only what's after the decimal point? I would describe these as having between 2 and 6 significant figures. Will they always have two digits before the decimal, or does your precision really care only about what's after it? Of course, there would still remain the problem of describing 49.25 with 6 sig figs, in the same way that it's hard to explain to someone that you really do need to leave in five minutes (but if you said you had to leave in six minutes, they'd believe you to be accurate to the minute). But I assume that's not part of the question. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Heartbleed and the windows distributions on python.org
I am building a cherrypy app that is testing as vulnerable to the heartbleed exploit. The app is running on the 64 bit 3.3.5 Windows distribution of python. An updated version of 64 bit Python 3.3.x for Windows or an updated pyopenssl? I am kind of surprised the distribution on python.org hasen't been updated. Here is the thread on the issue from the cherrypy-users group. -- https://mail.python.org/mailman/listinfo/python-list
Re: Significant digits in a float?
On Monday, April 28, 2014 12:28:59 PM UTC-4, Chris Angelico wrote: Terminology question: Why do you count only what's after the decimal point? I would describe these as having between 2 and 6 significant figures. Will they always have two digits before the decimal, or does your precision really care only about what's after it? Hmmm, yeah, I was being sloppy in my description. Definitely would have gotten points off in physics class for being so sloppy :-) No, there's not always two digits before the decimal. Could be anywhere from 0 to 3 digits before. What I'm really after is how many digits are after the decimal point. -- https://mail.python.org/mailman/listinfo/python-list
Re: Significant digits in a float?
On Monday, April 28, 2014 12:07:14 PM UTC-4, Ned Batchelder wrote: On 4/28/14 12:00 PM, Roy Smith wrote: 38.0 == 0 [...] Is there any clean way to do that? The best I've come up with so far is to str() them and parse the remaining string to see how many digits it put after the decimal point. That sounds like a pretty clean way: len(str(num).partition(.)[2]), though it also sounds like you understand all of the inaccuracies in Well, it's actually, a little uglier, because I want to map 38.0 ==0, so I need to special case that. The other annoying thing about using str() is its behavior isn't well defined. It looks like it does the right thing, but I imagine the details could change in a different implementation. -- https://mail.python.org/mailman/listinfo/python-list
Re: Significant digits in a float?
On 4/28/14 2:39 PM, Roy Smith wrote: On Monday, April 28, 2014 12:07:14 PM UTC-4, Ned Batchelder wrote: On 4/28/14 12:00 PM, Roy Smith wrote: 38.0 == 0 [...] Is there any clean way to do that? The best I've come up with so far is to str() them and parse the remaining string to see how many digits it put after the decimal point. That sounds like a pretty clean way: len(str(num).partition(.)[2]), though it also sounds like you understand all of the inaccuracies in Well, it's actually, a little uglier, because I want to map 38.0 ==0, so I need to special case that. Ah, right. The other annoying thing about using str() is its behavior isn't well defined. It looks like it does the right thing, but I imagine the details could change in a different implementation. I don't have a reference, but in recent Pythons, str() was specifically changed to guarantee that it produces the shortest string that when re-interpreted as a float, produces the same float. -- Ned Batchelder, http://nedbatchelder.com -- https://mail.python.org/mailman/listinfo/python-list
Re:Significant digits in a float?
Roy Smith r...@panix.com Wrote in message: I'm using Python 2.7 I have a bunch of floating point values. For example, here's a few (printed as reprs): 38.0 41.2586 40.752801 49.25 33.7951994 36.8371996 34.1489 45.5 Fundamentally, these numbers have between 0 and 4 decimal digits of precision, and I want to be able to intuit how many each has, ignoring the obvious floating point roundoff problems. Thus, I want to map: 38.0 == 0 41.2586 == 4 40.752801 == 4 49.25 == 2 33.7951994 == 4 36.8371996 == 4 34.1489 == 4 45.5 == 1 Is there any clean way to do that? The best I've come up with so far is to str() them and parse the remaining string to see how many digits it put after the decimal point. The numbers are given to me as Python floats; I have no control over that. I'm willing to accept that fact that I won't be able to differentiate between float(38.0) and float(38.). Both of those map to 1, which is OK for my purposes. Ignoring the unexpected terminology, you seem to be looking for the number of decimal places, and you're not interested in 2100 == -2 If you know something about the possible range of the numbers, and/or you know the valid range of decimal places, then you should convert to string in a way that will round the 3rd, 5th, and 6th values. Then if the string has no decimal, the answer is 0. If there are any trailing zeroes, strip them. Then just count digits after the decimal point. Without such limits, there can be no unique algorithm, and thus no correct code. -- DaveA -- https://mail.python.org/mailman/listinfo/python-list
Re: Heartbleed and the windows distributions on python.org
In article d576956a-5bcc-4508-bac1-87e954b7e...@googlegroups.com, Timothy McDonald tmcdon...@gmail.com wrote: I am building a cherrypy app that is testing as vulnerable to the heartbleed exploit. The app is running on the 64 bit 3.3.5 Windows distribution of python. An updated version of 64 bit Python 3.3.x for Windows or an updated pyopenssl? I am kind of surprised the distribution on python.org hasen't been updated. The current release of Python 3 is 3.4.0. A 3.4.1 maintenance release, with OpenSSL updated in the Windows installer, is planned for final release in mid-May. Python 3.3.x is now in security-fix-only mode which means only source fixes for security problems are released as needed and no further binary installers for Windows or OS X are produced. (The Python 2 Windows installer is not affected since it bundles an older, pre-heartbleed version of OpenSSL.) -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list
Re:Convert numpy array to single number
mboyd02...@gmail.com Wrote in message: I have a numpy array consisting of 1s and zeros for representing binary numbers: e.g. binary array([ 1., 0., 1., 0.]) I wish the array to be in the form 1010, so it can be manipulated. I do not want to use built in binary converters as I am trying to build my own. One thousand and ten is an int, not an array. So is 10, but it seems a more likely value you might be trying for. As for using the builtin binary converters, you'd better be a lot more specific, as you cannot even print an int in decimal form without utilizing them. The number is in binary already. Please copy the exact problem and constraints, as well as what you've written so far and what's wrong with it. -- DaveA -- https://mail.python.org/mailman/listinfo/python-list
Re:problem with regex
dimm...@gmail.com Wrote in message: i want to find a specific urls from a txt file but i have some issus. First when i take just two lines from the file with copy paste and assign it to a variable like this and it works only with triple quotes test='''_*_n.jpg,timelineCoverPhoto:{\focus\:{\x\:0.5,\y\:0.386925795053},\photo\:{\__type__\:{\name\:\Photo\},\image_lowres\:{\uri\:\https://fbcdn-photos-f-a.akamaihd.net/*-*-* Why did you start a second thread with similar content two minutes after the first? Do you expect us to compare the two messages and figure out what you changed, or were you just impatient for a response? I only check in here about 6 times a day, and I imagine some might be even less often. Your test string literal has lots of backslashes in it, which get interpreted into escape sequences in a literal, but not in a file. If that's really what the file looks like, you're going to want to use a raw string. I agree with Roy, you're probably not getting the same string the two ways. -- DaveA -- https://mail.python.org/mailman/listinfo/python-list
Re: Help with changes in traceback stack from Python 2.7 to Python 3.x
On 4/27/14 5:51 PM, Andrew Konstantaras wrote: I guess I am missing something big as I am looking for a shorthand way of doing the following: dctA = dict(x=x, y=y, ... n=n) Yes, your makeDict(x, y) is a shorthand for dict(x=x, y=y), but there are many things you can do with dict that you can't do with makeDict. What is the makeDict equivalent of: dict(x=12, y=self.y, z=a+b) The code you have allows you more compact expression, but it brings fragility and surprise. This is, as I understand it a very natural way of using a dictionary. It seems that this syntax is unnecessarily redundant and hence my goal of writing something more compact. Perhaps the way I am doing it is a little unorthodox, but the ultimate use of a dictionary is, as I understand it, completely in line with how dictionaries were designed to be used. In my other code, I often use these dictionaries to pass arguments to functions and return results. It allows me great flexibility without breaking existing code. I pack a dictionary before passing and unpack when retrieving. Perhaps you want to create a class instead? If you find yourself passing more than a handful of arguments to a function, and especially more than a handful of values returned from a function, then a class with methods might be a better way to combine state and behavior. Also, keep in mind that you can return a tuple from a function if you want to return two or three values and assign them to names: x, y, z = compute_xyz() You mention unpacking your dictionary after the function call. How do you do that? Isn't that a cumbersome and repetitive operation? I will give the locals approach a try, it seems a little more clumsy than simply passing the variables to the function. Thanks again for your input. ---Andrew BTW, it's a little easier to follow the threads of conversation if you put your responses after the text you are responding to. This is known as bottom-posting, and is preferred to top-posting as you did here. -- Ned Batchelder, http://nedbatchelder.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Help with changes in traceback stack from Python 2.7 to Python 3.x
On Apr 28, 2014, at 12:36 PM, Ned Batchelder n...@nedbatchelder.com wrote: On 4/27/14 5:51 PM, Andrew Konstantaras wrote: I guess I am missing something big as I am looking for a shorthand way of doing the following: dctA = dict(x=x, y=y, ... n=n) Yes, your makeDict(x, y) is a shorthand for dict(x=x, y=y), but there are many things you can do with dict that you can't do with makeDict. What is the makeDict equivalent of: dict(x=12, y=self.y, z=a+b) The code you have allows you more compact expression, but it brings fragility and surprise. This is, as I understand it a very natural way of using a dictionary. It seems that this syntax is unnecessarily redundant and hence my goal of writing something more compact. Perhaps the way I am doing it is a little unorthodox, but the ultimate use of a dictionary is, as I understand it, completely in line with how dictionaries were designed to be used. In my other code, I often use these dictionaries to pass arguments to functions and return results. It allows me great flexibility without breaking existing code. I pack a dictionary before passing and unpack when retrieving. Perhaps you want to create a class instead? If you find yourself passing more than a handful of arguments to a function, and especially more than a handful of values returned from a function, then a class with methods might be a better way to combine state and behavior. Also, keep in mind that you can return a tuple from a function if you want to return two or three values and assign them to names: x, y, z = compute_xyz() You mention unpacking your dictionary after the function call. How do you do that? Isn't that a cumbersome and repetitive operation? One of the reasons I like using the dictionary over the tuple is that if I have a function that grows in functionality (i.e., I decide to return more data for new scenarios), I can just add the new object to the dictionary and I don't have to worry about changing the code every where else that used to call the function. Actually, that is one of the nice features of using a dictionary, I can check if the key is there and if it is pull it out. As I was dusting off this old code, I considered trying to implement this functionality through by creating a class. I never liked going through the stack, it seemed hacky, but I came to love using dictionaries as they have some great built in features. Again, thanks for the advice and I will focus on the class implementation. I will give the locals approach a try, it seems a little more clumsy than simply passing the variables to the function. Thanks again for your input. ---Andrew BTW, it's a little easier to follow the threads of conversation if you put your responses after the text you are responding to. This is known as bottom-posting, and is preferred to top-posting as you did here. -- Ned Batchelder, http://nedbatchelder.com -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Sorting list alphabetically
On 4/28/2014 2:33 AM, Kev Dwyer wrote: Hello Terry, Regarding your second point, my mistake in not checking the link: I'd seen a similar one elsewhere and assumed they were the same. This link should work: http://msdn.microsoft.com/en-us/library/hzz3tw78 As to your first point, you're right, it seems setlocale(locale.LC_ALL, 'en_US.UTF-8') doesn't work on Windows. From what I read of the link above and http://msdn.microsoft.com/en-US/goglobal/bb896001.aspx given therein, I am going to guess that .UTF-8 is not supported for any language. It seems the locale name needs to be one of the aliases provided at http://msdn.microsoft.com/en-us/library/39cwe7zf, so for example locale.setlocale(locale.LC_ALL, 'usa') returns 'English_United States.1252'. Do you know if this is something people programming on Windows should just be aware of, or is there a case for a hint of some kind in the locale module documentation? *Definitely the latter. Perhaps you could open an issue with a specific suggestion. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Soap list and soap users on this list
Hi Joseph, Sorry for the late response, I seem to have missed this post. On 04/17/14 21:34, Joseph L. Casale wrote: I've been looking at Spyne to produce a service that can accept a request formatted as follows: ?xml version='1.0' encoding='UTF-8'? SOAP-ENV:Envelope xmlns:SOAP-ENV=http://...; xmlns:xsi=http:/... xmlns:xsd=http://...; SOAP-ENV:Body modifyRequest returnData=everything xmlns=urn:... attr ID=.../ data /data /modifyRequest /SOAP-ENV:Body /SOAP-ENV:Envelope https://gist.github.com/plq/11384113 Unfortunately, you need the latest Spyne from https://github.com/arskom/spyne, this doesn't work with 2.10 2.11 is due around end of may, beginning of june. Ping back if you got any other questions. Best, Burak -- https://mail.python.org/mailman/listinfo/python-list
RE: Soap list and soap users on this list
https://gist.github.com/plq/11384113 Unfortunately, you need the latest Spyne from https://github.com/arskom/spyne, this doesn't work with 2.10 2.11 is due around end of may, beginning of june. Ping back if you got any other questions. Burak, Thanks a ton! I've just pulled this down and started working through it. Much appreciated! jlc -- https://mail.python.org/mailman/listinfo/python-list
Re: Help with changes in traceback stack from Python 2.7 to Python 3.x
On Tue, Apr 29, 2014 at 5:50 AM, Andrew Konstantaras akon...@icloud.com wrote: Actually, that is one of the nice features of using a dictionary, I can check if the key is there and if it is pull it out. As I was dusting off this old code, I considered trying to implement this functionality through by creating a class. I never liked going through the stack, it seemed hacky, but I came to love using dictionaries as they have some great built in features. In that case, consider using function named arguments. You can simply list a whole lot of args with defaults, and then provide values for the exact ones you want: def f(a=1, b=2, c=3, d=4, e=5): print(My args are,a,b,c,d,e) f() # all defaults, like an empty dict f(d=123) # all defaults but one f(10,20,e=7) # specify the early ones positionally if you always pass those This is a really REALLY handy feature. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: MacOS 10.9.2: threading error using python.org 2.7.6 distribution
On Sunday, 27 April 2014 10:33:38 UTC-4, Chris Angelico wrote: In most contexts, thread unsafe simply means that you can't use the same facilities simultaneously from two threads (eg a lot of database connection libraries are thread unsafe with regard to a single connection, as they'll simply write to a pipe or socket and then read a response from it). But processes and threads are, on many systems, linked. Just the act of spinning off a new thread and then forking can potentially cause problems. Those are the exact sorts of issues that you'll see when you switch OSes, as it's the underlying thread/process model that's significant. (Particularly of note is that Windows is *very* different from Unix-based systems, in that subprocess management is not done by forking. But not applicable here.) Thanks, I'll keep all that in mind. I have to wonder how much of a problem it is here though, since I was able to demonstrate a functioning fork inside a new thread further up in the discussion. I have a new development that I find interesting, and I'm wondering if you still think it's the same problem. I have taken that threading object and turned it into a normal function definition. It's still forking the external tool, but it's doing so in the main thread, and it is finished execution before any other threads are created. And I'm still getting the same error. Turns out it's not coming from the threading module, but from the subprocess module instead. Specifically, like 709 of /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py which is this: try: self._execute_child(args, executable, preexec_fn, close_fds, cwd, env, universal_newlines, startupinfo, creationflags, shell, to_close, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) except Exception: I get the Warning: No stack to get attribute from twice when that self._execute_child() call is made. I've tried stepping into it to narrow it down further, but I'm getting weird behaviour from the debugger that I've never seen before once I do that. It's making it hard to track down exactly where the error is occurring. Interestingly, it's not actually raising an exception there. The except block is not being run. -- https://mail.python.org/mailman/listinfo/python-list
Re: MacOS 10.9.2: threading error using python.org 2.7.6 distribution
On Tue, Apr 29, 2014 at 8:50 AM, Matthew Pounsett matt.pouns...@gmail.com wrote: Thanks, I'll keep all that in mind. I have to wonder how much of a problem it is here though, since I was able to demonstrate a functioning fork inside a new thread further up in the discussion. Yeah, it's really hard to pin down sometimes. I once discovered a problem whereby I was unable to spin off subprocesses that did certain things, but I could do a trivial subprocess (I think I fork/exec'd to the echo command or something) and that worked fine. Turned out to be a bug in one of my signal handlers, but the error was being reported at the point of the forking. I have a new development that I find interesting, and I'm wondering if you still think it's the same problem. I have taken that threading object and turned it into a normal function definition. It's still forking the external tool, but it's doing so in the main thread, and it is finished execution before any other threads are created. And I'm still getting the same error. Interesting. That ought to eliminate all possibility of thread-vs-process issues. Can you post the smallest piece of code that exhibits the same failure? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Inconsistent viewkeys behaviour
Terry Reedy wrote: The left operand determines the result. The manual specifies that and do not have to be consistent. But I suspect that when 3.x dict.keys() was backported to 2.7.0, no one thought to update set, whereas the backported key view code already had the comparison. The question is why set() is raising an exception instead of returning NotImplemented to give the other operand a chance. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: Significant digits in a float?
On Mon, 28 Apr 2014 12:00:23 -0400, Roy Smith wrote: [...] Fundamentally, these numbers have between 0 and 4 decimal digits of precision, I'm surprised that you have a source of data with variable precision, especially one that varies by a factor of TEN THOUSAND. The difference between 0 and 4 decimal digits is equivalent to measuring some lengths to the nearest metre, some to the nearest centimetre, and some to the nearest 0.1 of a millimetre. That's very unusual and I don't know what justification you have for combining such a mix of data sources. One possible interpretation of your post is that you have a source of floats, where all the numbers are actually measured to the same precision, and you've simply misinterpreted the fact that some of them look like they have less precision. Since you indicate that 4 decimal digits is the maximum, I'm going with 4 decimal digits. So if your data includes the float 23.5, that's 23.5 measured to a precision of four decimal places (that is, it's 23.5000, not 23.5001 or 23.4999). On the other hand, if you're getting your values as *strings*, that's another story. If you can trust the strings, they'll tell you how many decimal places: 23.5 is only one decimal place, 23.5000 is four. But then what to make of your later example? 40.752801 == 4 Python floats (C doubles) are quite capable of distinguishing between 40.7528 and 40.752801. They are distinct numbers: py 40.752801 - 40.7528 7.105427357601002e-15 so if a number is recorded as 40.752801 presumably it is because it was measured as 40.752801. (How that precision can be justified, I don't know! Does it come from the Large Hadron Collider?) If it were intended to be 40.7528, I expect it would have be recorded as 40.7528. What reason do you have to think that something recorded to 14 decimal places was only intended to have been recorded to 4? Without knowing more about how your data is generated, I can't advise you much, but the whole scenario as you have described it makes me think that *somebody* is doing something wrong. Perhaps you need to explain why you're doing this, as it seems numerically broken. Is there any clean way to do that? The best I've come up with so far is to str() them and parse the remaining string to see how many digits it put after the decimal point. I really think you need to go back to the source. Trying to infer the precision of the measurements from the accident of the string formatting seems pretty dubious to me. But I suppose if you wanted to infer the number of digits after the decimal place, excluding trailing zeroes (why, I do not understand), up to a maximum of four digits, then you could do: s = %.4f % number # rounds to four decimal places s = s.rstrip(0) # ignore trailing zeroes, whether significant or not count = len(s.split(.)[1]) Assuming all the numbers fit in the range where they are shown in non- exponential format. If you have to handle numbers like 1.23e19 as well, you'll have to parse the string more carefully. (Keep in mind that most floats above a certain size are all integer-valued.) The numbers are given to me as Python floats; I have no control over that. If that's the case, what makes you think that two floats from the same data set were measured to different precision? Given that you don't see strings, only floats, I would say that your problem is unsolvable. Whether I measure something to one decimal place and get 23.5, or four decimal places and get 23.5000, the float you see will be the same. Perhaps you ought to be using Decimal rather than float. Floats have a fixed precision, while Decimals can be configured. Then the right way to answer your question is to inspect the number: py from decimal import Decimal as D py x = D(23.5000) py x.as_tuple() DecimalTuple(sign=0, digits=(2, 3, 5, 0, 0, 0), exponent=-4) The number of decimal digits precision is -exponent. I'm willing to accept that fact that I won't be able to differentiate between float(38.0) and float(38.). Both of those map to 1, which is OK for my purposes. That seems... well, bizarre and wrong are the only words that come to mind. If I were recording data as 38. and you told me I had measured it to only one decimal place accuracy, I wouldn't be too pleased. Maybe if I understood the context better? How about 38.12 and 38.1200? By the way, you contradict yourself here. Earlier, you described 38.0 as having zero decimal places (which is wrong). Here you describe it as having one, which is correct, and then in a later post you describe it as having zero decimal places again. -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Significant digits in a float?
Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes: By the way, you contradict yourself here. Earlier, you described 38.0 as having zero decimal places (which is wrong). Here you describe it as having one, which is correct, and then in a later post you describe it as having zero decimal places again. I get the impression that this is at the core of the misunderstanding. Having a number's representation ending in “….0” does not mean zero decimal places; it has exactly one. The value's representation contains the digit “0” after the decimal point, but that digit is significant to the precision of the representation. If the problem could be stated such that “38.0” and “38” and “38.000” are consistently described with the correct number of decimal digits of precision (in those examples: one, zero, and three), maybe the discussion would make more sense. -- \ “Men never do evil so completely and cheerfully as when they do | `\it from religious conviction.” —Blaise Pascal (1623–1662), | _o__) Pensées, #894. | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: Significant digits in a float?
On Tue, 29 Apr 2014 13:23:07 +1000, Ben Finney wrote: Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes: By the way, you contradict yourself here. Earlier, you described 38.0 as having zero decimal places (which is wrong). Here you describe it as having one, which is correct, and then in a later post you describe it as having zero decimal places again. I get the impression that this is at the core of the misunderstanding. Having a number's representation ending in “….0” does not mean zero decimal places; it has exactly one. The value's representation contains the digit “0” after the decimal point, but that digit is significant to the precision of the representation. If the problem could be stated such that “38.0” and “38” and “38.000” are consistently described with the correct number of decimal digits of precision (in those examples: one, zero, and three), maybe the discussion would make more sense. It's actually trickier than that. Digits of precision can refer to measurement error, or to the underlying storage type. Python floats are C doubles, so they have 64 bits of precision (approximately 17 decimal digits, if I remember correctly) regardless of the precision of the measurement. The OP (Roy) is, I think, trying to guess the measurement precision after the fact, given a float. If the measurement error really does differ from value to value, I don't think he'll have much luck: given a float like 23.0, all we can say is that it has *at least* zero significant decimal places. 23.1 has at least one, 23. has at least four. If you can put an upper bound on the precision, as Roy indicates he can, then perhaps a reasonable approach is to convert to a string rounded to four decimal places, then strip trailing zeroes: py x = 1234.1 # actual internal is closer to 1234.009 py (%.4f % x).rstrip('0') '1234.1' then count the number of digits after the dot. (This assumes that the string formatting routines are correctly rounded, which they should be on *most* platforms.) But again, this only gives a lower bound to the number of significant digits -- it's at least one, but might be more. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
[issue13583] sqlite3.Row doesn't support slice indexes
Jessica McKellar added the comment: Thanks for the ticket and patch, xapple! I updated the patch to address the compiler warning and use assertEqual. While testing, I noticed that slicing with steps wasn't supported, so I expanded the sqlite3.Row slicing code to support steps, and added some additional tests. The slicing code in Modules/_sqlite/row.c:pysqlite_row_subscript is unfortunately pretty redundant with the slicing code in Objects/tupleobject.c. It'd be better to either be able to factor the code from both into a function (but I couldn't see how to do this without making it part of the public API), or have tuple, sqlite.Row, etc. implement a shared slicing interface. Perhaps we should defer that decision to a future ticket, though. Note that even after this patch, sqlite.Row instances don't support negative indices. * This patch passes `make patchcheck`. * The full test suite passes with this patch. * There are no build warnings related to the patch. -- keywords: +needs review nosy: +jesstess versions: +Python 3.5 -Python 3.3 Added file: http://bugs.python.org/file35070/issue13583.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13583 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13583] sqlite3.Row doesn't support slice indexes
Jessica McKellar added the comment: I've also uploaded a short script that sets up an in-memory sqlite database that fetches Rows, for easy manual testing. -- Added file: http://bugs.python.org/file35071/sqlite3_slicing_demo.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13583 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21369] Extended modes for tarfile.TarFile()
Lars Gustäbel added the comment: That was a design decision. What would be the advantage of having the TarFile class offer the compression itself? -- assignee: - lars.gustaebel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21369 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18472] Update PEP 8 to encourage modern conventions
Changes by Florent Xicluna florent.xicl...@gmail.com: -- nosy: +flox ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18472 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21233] Add *Calloc functions to CPython memory allocation API
STINNER Victor added the comment: It looks like Windows supports also lazy initialization of memory pages initialized to zero. According to my microbenchmark on Linux and Windows, only bytes(n) and bytearray(n) are really faster with use_calloc.patch. Most changes of use_calloc.patch are maybe useless since all bytes are initilized to zero, but just after that they are replaced with new bytes. Results of bench_alloc2.py on Windows 7: original vs calloc-4.patch+use_calloc.patch: Common platform: Timer: time.perf_counter Python unicode implementation: PEP 393 Bits: int=32, long=32, long long=64, size_t=32, void*=32 Platform: Windows-7-6.1.7601-SP1 CFLAGS: None Timer info: namespace(adjustable=False, implementation='QueryPerformanceCounter( )', monotonic=True, resolution=1e-08) Platform of campaign orig: SCM: hg revision=4b97092aa4bd branch=default date=2014-04-27 18:02 +0100 Date: 2014-04-28 09:35:30 Python version: 3.5.0a0 (default, Apr 28 2014, 09:33:30) [MSC v.1600 32 bit (Int el)] Timer precision: 4.47 us Platform of campaign calloc: SCM: hg revision=4f0aaa8804c6 tag=tip branch=default date=2014-04-28 09:27 +020 0 Date: 2014-04-28 09:37:37 Python version: 3.5.0a0 (default:4f0aaa8804c6, Apr 28 2014, 09:37:03) [MSC v.160 0 32 bit (Intel)] Timer precision: 4.44 us ---+-+ Tests | orig | calloc ---+-+ object() | 121 ns (*) | 109 ns (-10%) b'A' * 10 | 77 ns (*) | 79 ns b'A' * 10**3 | 159 ns (*) | 168 ns (+5%) b'A' * 10**6 | 428 us (*) | 415 us 'A' * 10 | 87 ns (*) | 89 ns 'A' * 10**3 | 175 ns (*) | 177 ns 'A' * 10**6 | 429 us (*) | 454 us (+6%) 'A' * 10**8 | 48.4 ms (*) | 49 ms (None,) * 10**0 | 49 ns (*) | 51 ns (None,) * 10**1 | 115 ns (*) | 99 ns (-14%) (None,) * 10**2 | 433 ns (*) | 422 ns (None,) * 10**3 | 3.58 us (*) | 3.57 us (None,) * 10**4 | 34.9 us (*) | 34.9 us (None,) * 10**5 | 347 us (*) | 351 us (None,) * 10**6 | 5.14 ms (*) | 4.85 ms (-6%) (None,) * 10**7 | 53.2 ms (*) | 50.2 ms (-6%) (None,) * 10**8 | 563 ms (*) | 515 ms (-9%) ([None] * 10)[1:-1] | 217 ns (*) | 217 ns ([None] * 10**3)[1:-1] | 3.89 us (*) | 3.92 us ([None] * 10**6)[1:-1] | 5.13 ms (*) | 5.17 ms ([None] * 10**8)[1:-1] | 634 ms (*) | 533 ms (-16%) bytes(10) | 193 ns (*) | 206 ns (+7%) bytes(10**3) | 266 ns (*) | 296 ns (+12%) bytes(10**6) | 414 us (*) | 3.89 us (-99%) bytes(10**8) | 44.2 ms (*) | 4.56 us (-100%) bytearray(10) | 229 ns (*) | 243 ns (+6%) bytearray(10**3) | 301 ns (*) | 330 ns (+10%) bytearray(10**6) | 421 us (*) | 3.89 us (-99%) bytearray(10**8) | 44.4 ms (*) | 4.56 us (-100%) ---+-+ Total | 1.4 sec (*) | 1.16 sec (-17%) ---+-+ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21233 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21372] multiprocessing.util.register_after_fork inconsistency
New submission from Marc Schlaich: multiprocessing.util.register_after_fork does not behave consistently on Windows because the `_afterfork_registry` is not transferred to the subprocess. The following example fails on Windows while it works perfectly on Linux: import multiprocessing.util def hook(*args): print (args) def func(): print ('func') if __name__ == '__main__': multiprocessing.util.register_after_fork(hook, hook) p = multiprocessing.Process(target=func) p.start() -- components: Windows messages: 217347 nosy: schlamar priority: normal severity: normal status: open title: multiprocessing.util.register_after_fork inconsistency type: behavior versions: Python 2.7, Python 3.3, Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21372 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21233] Add *Calloc functions to CPython memory allocation API
STINNER Victor added the comment: Changes on the pickle module don't look like an interesting optimization. It even looks slower. $ python perf.py -b fastpickle,fastunpickle,pickle,pickle_dict,pickle_list,slowpickle,slowunpickle,unpickle ../default/python.orig ../default/python.calloc ... Report on Linux selma 3.13.9-200.fc20.x86_64 #1 SMP Fri Apr 4 12:13:05 UTC 2014 x86_64 x86_64 Total CPU cores: 4 ### fastpickle ### Min: 0.364510 - 0.374144: 1.03x slower Avg: 0.367882 - 0.377714: 1.03x slower Significant (t=-11.54) Stddev: 0.00493 - 0.00347: 1.4209x smaller The following not significant results are hidden, use -v to show them: fastunpickle, pickle_dict, pickle_list. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21233 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21233] Add *Calloc functions to CPython memory allocation API
STINNER Victor added the comment: Patch version 5. This patch is ready for a review. Summary of calloc-5.patch: - add the following functions: * void* PyMem_RawCalloc(size_t nelem, size_t elsize) * void* PyMem_Calloc(size_t nelem, size_t elsize) * void* PyObject_Calloc(size_t nelem, size_t elsize) * PyObject* _PyObject_GC_Calloc(size_t basicsize) - add void* calloc(void *ctx, size_t nelem, size_t elsize) field to the PyMemAllocator structure - optimize bytes(n) and bytearray(n) to allocate objects using calloc() instead of malloc() - update tracemalloc to trace also calloc() - document new functions and add unit tests for the calloc hook (in _testcapi) Changes between versions 4 and 5: - revert all changes except bytes(n) and bytearray(n) of use_calloc.patch: they were useless according to benchmarks - _PyObject_GC_Calloc() now takes a single parameter - add versionadded and versionchanged fields in the documentation According to benchmarks, calloc() is only useful for large allocation (1 MB?) if only a part of the memory block is modified (to non-zero bytes) just after the allocation. Untouched memory pages don't use physical memory and don't use RSS memory pages, but it is possible to read their content (null bytes). Using calloc() instead of malloc()+memset(0) doens't look to be faster (it may be a little bit slower) if all bytes are set just after the allocation. I chose to only use one parameter for _PyObject_GC_Calloc() because this function is used to allocate Python objects. A structure of a Python object must start with PyObject_HEAD or PyObject_VAR_HEAD and so the total size of an object cannot be expressed as NELEM * ELEMSIZE. I have no use case for _PyObject_GC_Calloc(), but it makes sense to use it to allocate a large Python object tracked by the GC and using a single memory block for the Python header + data. PyObject_Calloc() simply use memset(0) for small objects (= 512 bytes). It delegates the allocation to PyMem_RawCalloc(), and so indirectly to calloc(), for larger objects. Note: use_calloc.patch is no more needed, I merged the two patches since only bytes(n) and bytearray(n) now use calloc(). -- Added file: http://bugs.python.org/file35072/calloc-5.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21233 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20305] Android's incomplete locale.h implementation prevents cross-compilation
Changes by Stefan Krah stefan-use...@bytereef.org: -- nosy: +lizhenhua ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20305 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21371] struct lconv does not have decimal_point on android platform
Stefan Krah added the comment: This looks like a duplicate. -- nosy: +skrah resolution: - duplicate stage: - resolved status: open - closed superseder: - Android's incomplete locale.h implementation prevents cross-compilation ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21371 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21233] Add *Calloc functions to CPython memory allocation API
STINNER Victor added the comment: Demo of calloc-5.patch on Linux. Thanks to calloc(), bytes(50 * 1024 * 1024) doesn't allocate memory for null bytes and so the RSS memory is unchanged (+148 kB, not +50 MB), but tracemalloc says that 50 MB were allocated. $ ./python -X tracemalloc Python 3.5.0a0 (default:4b97092aa4bd+, Apr 28 2014, 10:40:53) [GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux Type help, copyright, credits or license for more information. import os, tracemalloc os.system(grep RSS /proc/%s/status % os.getpid()) VmRSS: 10736 kB 0 before = tracemalloc.get_traced_memory()[0] large = bytes(50 * 1024 * 1024) import sys sys.getsizeof(large) / 1024. 51200.0478515625 (tracemalloc.get_traced_memory()[0] - before) / 1024. 51198.1962890625 os.system(grep RSS /proc/%s/status % os.getpid()) VmRSS: 10884 kB 0 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21233 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5404] Cross-compiling Python
Stefan Krah added the comment: Ping. Is this still an issue for anyone in 3.4? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5404 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21220] Enhance obmalloc allocation strategy
Kristján Valur Jónsson added the comment: This significantly helps fragmentation in programs with dynamic memory usage, e.g. long running programs. On which programs? The fragmentation of the memory depends a lot on how the program allocates memory. For example, if a program has no temporary memory peak, it should not be a victim of the memory fragmentation. Long running programs. E.g. web servers and so on. Where there is a churn in memory usage. As objects are allocated and released with some sort of churn. New objects will be allocated from lower-address pages, where higher address pages are increasingly likely to be freed as no-longer used objects are released. This is the second best thing to a sliding-block allocator and is motivated by the same requirements that makes such an sliding-block allocator (such as pypy uses) desirable in the first place. To measure the improvment of such memory allocator, more benchmarks (speed and fragmentation) should be run than a single test (memcruch.py included in the test) written to benchmark the allocator. Yes. Memcrunch was specifically written to simulate a case where objects are continuously created and released, such as might be expected in a server, with a peak in memory usage followed by lower memory usage, and to demonstrate that the pages allocated during the peak will be released later as memory churn causes memory usage to migrate toward lower addresses. However, following Antoine's advice I ran the Benchmarks testsuite and found an adverse effect in the n-body benchmark. That can have two causes: a) the insertion cost of the block when a block moves from 'full' to 'used'. This is a rare event and should be unimportant. I will instrument this for this test and see if it is really the reason b) Cache effects because a newly 'used' block is not immediately allocated from. Again, it happens rarely that a block is linked at the head so this shouldn't be significant. Because of this, this change isn't yet ready to be applied. If however I manage to change the policy so that memory usage becomes better while still preserving performance of the benchmark tests, I will report back :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21220 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20188] ALPN support for TLS
Changes by Cory Benfield c...@lukasa.co.uk: -- nosy: +Lukasa ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20188 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21369] Extended modes for tarfile.TarFile()
Sworddragon added the comment: The TarFile class provides more options. Alternatively a file object could be used but this means additional code (and maybe IO overhead). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21369 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19977] Use surrogateescape error handler for sys.stdin and sys.stdout on UNIX for the C locale
Antoine Pitrou added the comment: We should not overcomplicate this. I suggest that we simply use utf-8 under the C locale. Do you mean utf8/strict or utf8/surrogateescape? utf8/strict doesn't work (os.listdir raises an unicode error) if your system is configured to use latin1 (ex: filenames are stored in this encoding), but unfortunately your program is running in an empty environment (so will use the POSIX locale). The issue is about stdin and stdout, I'm not sure why os.listdir would be affected. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19977 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13559] Use sendfile where possible in httplib
Changes by Giampaolo Rodola' g.rod...@gmail.com: -- dependencies: +Add a new socket.sendfile() method versions: +Python 3.5 -Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13559 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20306] Lack of pw_gecos field in Android's struct passwd causes cross-compilation for the pwd module to fail
Antoine Pitrou added the comment: Interesting. It seems pw_gecos isn't mandated by POSIX: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/pwd.h.html I wonder if there's a better way to do this (autoconf check?) than an Android-specific #ifdef, though. -- nosy: +pitrou versions: +Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20306 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21233] Add *Calloc functions to CPython memory allocation API
Stefan Krah added the comment: With the latest patch the decimal benchmark with a lot of small allocations is consistently 2% slower. Large factorials (where the operands are initialized to zero for the number-theoretic transform) have the same performance with and without the patch. It would be interesting to see some NumPy benchmarks (Nathaniel?). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21233 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21369] Extended modes for tarfile.TarFile()
Lars Gustäbel added the comment: You can pass keyword arguments to tarfile.open(), which will be passed to the TarFile constructor. You can also use pass fileobj arguments to tarfile.open(). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21369 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13204] sys.flags.__new__ crashes
Antoine Pitrou added the comment: Thanks for the patch, Jessica. It seems to work under Windows here. -- versions: +Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13204 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13204] sys.flags.__new__ crashes
Roundup Robot added the comment: New changeset 7052fdd90a11 by Antoine Pitrou in branch '3.4': Issue #13204: Calling sys.flags.__new__ would crash the interpreter, now it raises a TypeError. http://hg.python.org/cpython/rev/7052fdd90a11 New changeset a14012352f65 by Antoine Pitrou in branch 'default': Issue #13204: Calling sys.flags.__new__ would crash the interpreter, now it raises a TypeError. http://hg.python.org/cpython/rev/a14012352f65 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13204 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21369] Extended modes for tarfile.TarFile()
Sworddragon added the comment: Interesting, after reading the documentation again I would now assume that is what **kwargs is for. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21369 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13204] sys.flags.__new__ crashes
Antoine Pitrou added the comment: I've committed the patch to 3.4 and 3.5. I'm closing the issue, I don't think fixing 2.7 is important at this point. -- resolution: - fixed stage: patch review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13204 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21369] Extended modes for tarfile.TarFile()
Lars Gustäbel added the comment: Jup. That's it. -- priority: normal - low resolution: - not a bug stage: - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21369 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21305] PEP 466: update os.urandom
Donald Stufft added the comment: Depleting /dev/urandom isn't actually a thing. /dev/urandom on all modern *nix OSs uses a fast PRNG which is secure as long as it has received enough bytes of initial entropy. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21305 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21305] PEP 466: update os.urandom
Charles-François Natali added the comment: Depleting /dev/urandom isn't actually a thing. /dev/urandom on all modern *nix OSs uses a fast PRNG which is secure as long as it has received enough bytes of initial entropy. I didn't say deplete /dev/urandom, I said that when reading from /dev/urandom you're depleting your entropy pool. So reading from /dev/urandom won't block, but it can starve processes that read from /dev/random, and that's a problem. See https://groups.google.com/forum/#!msg/fa.linux.kernel/Ocl01d8TzT0/KDCon2ZUm1AJ I think since 2.6 Linux uses two different entropy pools for /dev/random and /dev/urandom, but that might not be true for every OS. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21305 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1820] Enhance Object/structseq.c to match namedtuple and tuple api
Stefan Krah added the comment: Is accessing _fields a common operation? Personally I'd use a PyGetSetDef and generate the tuple on access (perhaps cache the result). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1820 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13096] ctypes: segfault with large POINTER type names
Meador Inge added the comment: Thanks for the review and reminder about this issue, jesstess. I will apply the patch later today. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13096 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20306] Lack of pw_gecos field in Android's struct passwd causes cross-compilation for the pwd module to fail
Shiz added the comment: Ah, yes, if it's not actually mandated by POSIX, something like HAVE_PASSWD_PW_GECOS would be more appropriate. I'll rework the patch into something more generic. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20306 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21305] PEP 466: update os.urandom
Donald Stufft added the comment: I don't think what you're worrying about here is something that has a high chance of happening, if it even occurs in the wild at all. To be clear in order for that to matter at all in the context of this ticket, some software would need to be reading from /dev/random (which almost zero software should be doing) on a system where you have a high number of threads or async handlers all reading from /dev/urandom at the same time and reading enough data off of it to drop the entropy estimation in the primary pool down below whatever amount of data that the other process is attempting to read from /dev/random. In that case no trouble will occur and the process reading from /dev/random will just block waiting on additional entropy to be collected so that the entropy estimation is high enough to fulfill the request. AFAIK there are zero practical concerns from reading as much as you want off of /dev/urandom as often as you want. What this change does is make it safe to just simply use os.urandom in order to generate random bytes in a Python application. The current situation is such that any use of os.urandom is potentially a place where your application will crash in highly concurrent environments. This will drive people to either: A) Poorly reimplement the persistent FD option, especially troublesome on Windows because the simple approach to doing so will flat out not work on Windows B) Use a userspace CSPRNG, this is considered ill advised by most reputable cryptographer's as most of them have had issues at one point in time or another, and they all generally depend on the security of /dev/urandom anyways so if /dev/urandom fall they fall as well. Using os.urandom is the *right* thing to do for getting random in an application, but the current implementation effectively punishes people who use it if their application is highly concurrent. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21305 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21373] robotparser: Automatically call modified function in read()
New submission from Maxime Lorant: For the moment, RobotFileParser (on both Python 2.x and 3.x) has a method modified, but it is never called in the class itself, hence the last_checked attribute is always at 0 if the user doesn't call modified() explicitly. I would suggest to add a call to modified() at the end of the read() method. It makes more sense to have a last_checked value (returns in mtime()) updated by the class itself. Especially when the doc says: Returns the time the ``robots.txt`` file was last fetched.. Currently this sentence isn't true, since the value has to be updated by the user. -- components: Library (Lib) files: robotparser.diff keywords: patch messages: 217370 nosy: mlorant, orsenthil priority: normal severity: normal status: open title: robotparser: Automatically call modified function in read() type: enhancement versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file35073/robotparser.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21373 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20962] Rather modest chunk size in gzip.GzipFile
William Tisäter added the comment: That makes sense. I proceeded and updated `Lib/gzip.py` to use `io.DEFAULT_BUFFER_SIZE` instead. This will change the existing behaviour in two ways: * Start using 1024 * 8 as buffer size instead of 1024. * Add one more kwarg (`buffer_size`) to `GzipFile.__init__()`. Ps. This is my first patch, tell me if I'm missing something. -- Added file: http://bugs.python.org/file35074/20962_default-buffer-size.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20962 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21305] PEP 466: update os.urandom
Charles-François Natali added the comment: Using os.urandom is the *right* thing to do for getting random in an application, but the current implementation effectively punishes people who use it if their application is highly concurrent. And I argue that this scenario is almost as likely as the one you depict above: we never had a bug report before, and if you have a look at the the bug report which led to the change in question, it's not clear at all that all threads were indeed reading from /dev/urandom when EMFILE was raised. Since reading from /dev/urandom shouldn't block, it's not clear at all how a realistic workload would actually hit the file descriptor limit because RLIMIT_NOFILE threads are reading from /dev/urandom. But don't get me wrong, I'm not saying this change is useless, it actually makes sense to use a persistent FD. But backporting always has a risk, which has to be balanced. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21305 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19940] ssl.cert_time_to_seconds() returns wrong results if local timezone is not UTC
akira added the comment: I've updated the patch: - fixed the code example in the documentation to use int instead of float result - removed assertion on the int returned type (float won't lose precision for the practical dates but guaranteeing an integer would be nice) - reworded the scary comment - removed tests that test the tests Ready for review. -- Added file: http://bugs.python.org/file35075/ssl_cert_time_to_seconds-ps6.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19940 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21305] PEP 466: update os.urandom
Donald Stufft added the comment: But backporting always has a risk, which has to be balanced. Sure, which is why a PEP was written, discussed and accepted to find that balance. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21305 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21233] Add *Calloc functions to CPython memory allocation API
STINNER Victor added the comment: With the latest patch the decimal benchmark with a lot of small allocations is consistently 2% slower. Does your benchmark use bytes(int) or bytearray(int)? If not, I guess that your benchmark is not reliable because only these two functions are changed by calloc-5.patch, except if there is a bug in my patch. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21233 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21362] concurrent.futures does not validate that max_workers is proper
Jim Jewett added the comment: I confirm the bug. The patch looks good. -- nosy: +Jim.Jewett ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21362 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8776] Bytes version of sys.argv
Nick Coghlan added the comment: I'd like to revisit this after PEP 432 is in place, since having to do this dance for arg processing when running on Linux in the POSIX locale is somewhat lame: argv = sys.argv encoding = locale.getpreferredencoding() # Hope nobody changed the locale! fixed_encoding = read_encoding_from(/etc/locale.conf) # For example argvb = [arg.encode(encoding, surrogateescape) for arg in argv] fixed_argv = [arg.decode(fixed_encoding, surrogateescape) for arg in argvb] (For stricter parsing, leave out the second surrogateescape) Now, if PEP 432 resolves the system encoding issue such that we are able to use the right encoding even when locale.getpreferredencoding() returns the wrong answer, then it may not be worthwhile to also provide sys.argvb (especially since it won't help hybrid 2/3 code). On the other hand, like os.environb, it does make it easier for POSIX-only code paths that wants to handle boundary encoding issues directly to stick with consuming the binary data directly and avoid the interpreter's automatic conversion to the text domain. Note also that os.environb is only available when os.supports_bytes_environ is True, so it would make sense to only provide sys.argvb in the circumstances where we provide os.environb. -- assignee: - ncoghlan nosy: +ncoghlan resolution: wont fix - later status: closed - open versions: +Python 3.5 -Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8776 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1599254] mailbox: other programs' messages can vanish without trace
Jim Jewett added the comment: pinging David Watson: What is the status? If I understand correctly, (and I may well not), you have already opened other issues for parts of this, and (only) the final patch is ready for patch (and hopefully) commit review. Is this correct? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1599254 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20974] email module docs say not compatible with current python version
Jim Jewett added the comment: I don't know for sure if the compatibility claims are correct, but the patch looks good. -- stage: - commit review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20974 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21233] Add *Calloc functions to CPython memory allocation API
Stefan Krah added the comment: Hmm, obmalloc.c changed as well, so already the gcc optimizer can take different paths and produce different results. Also I did set mpd_callocfunc to PyMem_Calloc(). 2% slowdown is far from being a tragic result, so I guess we can ignore that. The bytes() speedup is very nice. Allocations that took one second are practically instant now. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21233 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21374] DecimalTuple.__module__ is set to _frozen_importlib
New submission from Stefan Krah: x = Decimal(9).as_tuple() import pickle pickle.dumps(x) Traceback (most recent call last): File stdin, line 1, in module _pickle.PicklingError: Can't pickle class '_frozen_importlib.DecimalTuple': attribute lookup DecimalTuple on _frozen_importlib failed -- components: Extension Modules messages: 217381 nosy: skrah priority: normal severity: normal stage: needs patch status: open title: DecimalTuple.__module__ is set to _frozen_importlib type: behavior versions: Python 3.4, Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21374 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21233] Add *Calloc functions to CPython memory allocation API
Charles-François Natali added the comment: Also I did set mpd_callocfunc to PyMem_Calloc(). 2% slowdown is far from being a tragic result, so I guess we can ignore that. Agreed. The bytes() speedup is very nice. Allocations that took one second are practically instant now. Indeed. Victor, thanks for the great work! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21233 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19354] test_format fails on RHEL-6
Changes by Stefan Krah stefan-use...@bytereef.org: -- resolution: - out of date stage: - resolved status: pending - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19354 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21370] segfault from simple traceback.format_exc call
Stefan Krah added the comment: I cannot reproduce this. Which platform? Does it happen with Python 3.4? -- nosy: +skrah ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21370 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21354] PyCFunction_New no longer exposed by python DLL breaking bdist_wininst installers
Andrew Svetlov added the comment: Ok, I'll take a look. Sorry, probably I've missed python3.def file. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21354 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19977] Use surrogateescape error handler for sys.stdin and sys.stdout on UNIX for the C locale
Nick Coghlan added the comment: Victor was referring to code like print(os.listdir()). Those are the motivating cases for ensuring round trips from system APIs to the standard streams work correctly. There's also the problem that sys.argv currently relies on the locale encoding directly, because the filesystem encoding hasn't been worked out at that point (see issue 8776). So this current change will also make print(sys.argv) work more reliably in the POSIX locale. The conclusion I have come to is that any further decoupling of Python 3 from the locale encoding will actually depend on getting the PEP 432 bootstrapping changes implemented, reviewed and the PEP approved, so we have more interpreter infrastructure in place by the time the interpreter starts trying to figure out all these boundary encoding issues. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19977 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19977] Use surrogateescape error handler for sys.stdin and sys.stdout on UNIX for the C locale
Antoine Pitrou added the comment: The conclusion I have come to is that any further decoupling of Python 3 from the locale encoding will actually depend on getting the PEP 432 bootstrapping changes implemented, reviewed and the PEP approved, so we have more interpreter infrastructure in place by the time the interpreter starts trying to figure out all these boundary encoding issues. Yeah. My proposal had more to do with the fact that we should some day switch to utf-8 by default on all POSIX systems, regardless of what the system advertises as best encoding. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19977 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19977] Use surrogateescape error handler for sys.stdin and sys.stdout on UNIX for the C locale
Nick Coghlan added the comment: Antoine Pitrou added the comment: Yeah. My proposal had more to do with the fact that we should some day switch to utf-8 by default on all POSIX systems, regardless of what the system advertises as best encoding. Yeah, that seems like a plausible future to me as well, and knowing it's a step along that path actually gives me more motivation to get back to working on the startup issues :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19977 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1533105] NetBSD build with --with-pydebug causes SIGSEGV
Stefan Krah added the comment: Given msg84518 and msg118909 I think this can be closed. -- resolution: - out of date stage: patch review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1533105 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21375] Fix and test overflow behavior in the C version of heapq
New submission from Raymond Hettinger: The Py_ssizet indexes can overflow the childpos variable: childpos = 2*pos + 1;/* leftmost child position */ while (childpos endpos) ... http://googleresearch.blogspot.com/2006/06/extra-extra-read-all-about-it-nearly.html -- assignee: rhettinger components: Extension Modules messages: 217389 nosy: rhettinger priority: normal severity: normal stage: needs patch status: open title: Fix and test overflow behavior in the C version of heapq type: behavior versions: Python 2.7, Python 3.4, Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21375 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21376] asyncio docs refer to wrong TimeoutError
New submission from Philip Sequeira: Example: https://docs.python.org/3.4/library/asyncio-task.html TimeoutError is mentioned several times, and links to the OSError subclass. However, the actual TimeoutError raised by asyncio stuff is the one from concurrent.futures, which is not compatible. The docs as they are seem to suggest that something like except TimeoutError would be appropriate, when in fact that would not produce the expected behavior; except asyncio.TimeoutError is what you'd want. -- assignee: docs@python components: Documentation messages: 217390 nosy: docs@python, qmega priority: normal severity: normal status: open title: asyncio docs refer to wrong TimeoutError versions: Python 3.4, Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21376 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20962] Rather modest chunk size in gzip.GzipFile
Antoine Pitrou added the comment: So I'd suggest, instead of using an hardcoded value, to simply reuse io.DEFAULT_BUFFER_SIZE. That way, if some day we decide to change it, all user code wil benefit from the change. I don't think io.DEFAULT_BUFFER_SIZE makes much sense as a heuristic for the gzip module (or compressed files in general). Perhaps gzip should get its own DEFAULT_BUFFER_SIZE? -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20962 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1284316] Win32: Security problem with default installation directory
Changes by Steve Dower steve.do...@microsoft.com: -- nosy: +steve.dower ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1284316 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19940] ssl.cert_time_to_seconds() returns wrong results if local timezone is not UTC
Antoine Pitrou added the comment: Thanks for the updated patch, Akira! I'm gonna take a look right now. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19940 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20962] Rather modest chunk size in gzip.GzipFile
Charles-François Natali added the comment: I don't think io.DEFAULT_BUFFER_SIZE makes much sense as a heuristic for the gzip module (or compressed files in general). Perhaps gzip should get its own DEFAULT_BUFFER_SIZE? Do you mean from a namespace point of vue, or from a performance point of view? Because this size is used to read/write from underlying the file object, so using the io default would make sense, no? Sure, it might not be optimal for compressed files, but I gues that the optimal value is function of the compression-level block size and many other factors which are just too varied to come up with a reasonable heuristic. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20962 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19940] ssl.cert_time_to_seconds() returns wrong results if local timezone is not UTC
Antoine Pitrou added the comment: I've committed the patch. Thank you very much for contributing! -- resolution: - fixed stage: patch review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19940 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20962] Rather modest chunk size in gzip.GzipFile
Antoine Pitrou added the comment: Sure, it might not be optimal for compressed files, but I gues that the optimal value is function of the compression-level block size and many other factors which are just too varied to come up with a reasonable heuristic. Well, I think that compressed files in general would benefit from a larger buffer size than plain binary I/O, but that's just a hunch. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20962 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19940] ssl.cert_time_to_seconds() returns wrong results if local timezone is not UTC
Roundup Robot added the comment: New changeset 7191c37238d5 by Antoine Pitrou in branch 'default': Issue #19940: ssl.cert_time_to_seconds() now interprets the given time string in the UTC timezone (as specified in RFC 5280), not the local timezone. http://hg.python.org/cpython/rev/7191c37238d5 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19940 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com