unexpected syntax errors
A user is getting this error New issue 8: bad raise syntax https://bitbucket.org/rptlab/reportlab/issue/8/bad-raise-syntax File "/usr/lib/python2.7/site-packages/svg2rlg.py", line 16, in from reportlab.graphics import renderPDF File "/usr/lib64/python2.7/site-packages/reportlab/graphics/renderPDF.py", line 168 raise ValueError, 'bad value for textAnchor '+str(text_anchor) ^ SyntaxError: invalid syntax however, I believe that this older syntax is allowed in python 2.7. We've had other issues like this raised from various distros which are apparently making changes to 2.7 which change the external behaviour eg spelling corrections to attribute names. Could this be one of those? -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
Re: Writing Extensions for Python 3 in C
On 18/06/2013 11:24, Aditya Avinash wrote: Hi. This is the last place where I want to ask a question. I have searched for lots of tutorials and documentation on the web but, didn't find a decent one to develop extensions for Python 3 using a custom compiler (mingw32, nvcc). Please help me. PS: Don't point me to Python Documentation. It is not good for beginners. It doesn't elaborate about calls and implementation. Try getting the source code for the Python you intend to build for. Then look in the Modules directory for the sample extensions xxlimited.c, xxmodule.c & xxsubtype.c. They may give you some idea of how to proceed. Of course all the other .c codes in there are the actual extensions that Python uses so they are also good examples. I started with "Extending and Embedding the Python Interpreter" from the python documentation though. -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
substituting proxy
Before attempting to reinvent the wheel has anyone created an http(s) proxy that can replace the content for specific requests. Context: I have access to the client's test site, but a lot of the requests are dynamic and save html complete etc etc doesn't work properly. In addition lots of the content comes from a cloud server which seems to object unless I take great care over spoofing of host names & referrers etc. I would like to debug stuff we supply, but that's hard unless I have the whole kit & kaboodle. I thought a proxy that could substitute for our javascript file(s) might work. Has anyone done this in twisted pymiproxy etc etc? -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
looping versus comprehension
An email in reportlab-us...@reportlab.com claimed that the following loop in a charting module was taking a long time I use ReportLab 2.6 but I also found the problem in ReportLab daily from 01/29/2013 in /src/reportlab/graphics/charts/lineplots.py: 276 # Iterate over data columns. 277 if self.joinedLines: 278 points = [] 279 for xy in row: 280 points += [xy[0], xy[1]] If I use a list comprehension instead, the plot is generated within seconds or minutes: 278 points = [[xy[0], xy[1]] for xy in row] however, when I tried an experiment in python 2.7 using the script below I find that the looping algorithms perform better. A naive loop using list += list would appear to be an O(n**2) operation, but python seems to be doing better than that. Also why does the append version fail so dismally. Is my test coded wrongly or is pre-allocation of the list making this better than expected? C:\code\tests>tpoints 86000 86 START n=86000 existing algorithm took 0.08 seconds existing algorithm using list took 0.12 seconds existing algorithm using list assuming length 2 took 0.12 seconds map(list,row) took 0.16 seconds [list(xy) for xy in row] took 0.28 seconds [[xy[0],xy[1]] for xy in row] took 0.22 seconds append algorithm took 0.19 seconds END n=86000 START n=86 existing algorithm took 0.86 seconds existing algorithm using list took 1.33 seconds existing algorithm using list assuming length 2 took 1.25 seconds map(list,row) took 3.44 seconds [list(xy) for xy in row] took 3.03 seconds [[xy[0],xy[1]] for xy in row] took 2.70 seconds append algorithm took 2.48 seconds END n=86 # import sys, time def main(n): print 20*'#','START n=%s'%n,20*'#' row = [(i,i+1) for i in xrange(2*n)] print 'existing algorithm', t0 = time.time() points = [] for xy in row: points += [xy[0],xy[1]] t1 = time.time() print 'took %.2f seconds' % (t1-t0) print 'existing algorithm using list', t0 = time.time() points = [] for xy in row: points += list(xy[:2]) t1 = time.time() print 'took %.2f seconds' % (t1-t0) print 'existing algorithm using list assuming length 2', t0 = time.time() points = [] for xy in row: points += list(xy) t1 = time.time() print 'took %.2f seconds' % (t1-t0) print 'map(list,row)', t0 = time.time() points = map(list,row) t1 = time.time() print 'took %.2f seconds' % (t1-t0) print '[list(xy) for xy in row]', t0 = time.time() points = [list(xy) for xy in row] t1 = time.time() print 'took %.2f seconds' % (t1-t0) print '[[xy[0],xy[1]] for xy in row]', t0 = time.time() points = [[xy[0],xy[1]] for xy in row] t1 = time.time() print 'took %.2f seconds' % (t1-t0) print 'append algorithm', t0 = time.time() points = [].append for xy in row: points([xy[0],xy[1]]) points = points.__self__ t1 = time.time() print 'took %.2f seconds' % (t1-t0) print 20*'#','END n=%s'%n,20*'#','\n\n' if __name__=='__main__': if len(sys.argv)==1: N = [86000] else: N = map(int,sys.argv[1:]) for n in N: main(n) # -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
Re: looping versus comprehension
On 30/01/2013 15:49, Chris Angelico wrote: On Thu, Jan 31, 2013 at 1:58 AM, Robin Becker wrote: however, when I tried an experiment in python 2.7 using the script below I find that the looping algorithms perform better. A naive loop using list += list would appear to be an O(n**2) operation, but python seems to be doing better than that. Also why does the append version fail so dismally. Is my test coded wrongly or is pre-allocation of the list making this better than expected? First off, are you aware that your first three blocks of code and your last four produce different results? The first ones flatten the list, the others simply convert tuples to lists. With n = 3: points = [] for xy in row: points += [xy[0],xy[1]] points [0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6] map(list,row) [[0, 1], [1, 2], [2, 3], [3, 4], [4, 5], [5, 6]] Once that's sorted out, then timings can be played with. But it's worth noting that list appending is not going to be O(N*N), because it's going to allow room for expansion. ChrisA No I missed that :( the list is a flattened one. That'll teach me not to copy the code from the users without checking. Now you point it out it's clear that his code is doing something different. Presumably it's not drawing the same thing at all :) no wonder it got much faster. -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
client ssl verification
I'm trying to do client ssl verification with code that looks like the sample
below. I am able to reach and read urls that are secure and have no client
certificate requirement OK. If I set explicit_check to True then verbose output
indicates that the server certs are being checked fine ie I see the correct cert
details and am able to check them.
However, when I try to reach an apache location like
sslverifyclient require
sslverifydepth 10
I am getting an error from urllib2 that goes like this
urllib2.py", line 1148, in do_open
raise URLError(err)
URLError: routines:SSL3_READ_BYTES:tlsv1 alert unknown ca
I am using the server.crt and server.key (both in PEM format) from the target
server itself; I reasoned that should be the easiest combo for the client &
server to match, but I am obviously wrong. Any obvious stupidities to be pointed
out? I suppose I could create a new cert/key based on a self signed ca, but that
would not work properly for the other parts of the server.
import socket, ssl, fnmatch, datetime, urllib2, httplib
verbose=False
# wraps https connections with ssl certificate verification
class SecuredHTTPSHandler(urllib2.HTTPSHandler):
def
__init__(self,key_file=None,cert_file=None,ca_certs=None,explicit_check=False):
class SecuredHTTPSConnection(httplib.HTTPSConnection):
def connect(self):
# overrides the version in httplib so that we do
# certificate verification
sock = socket.create_connection((self.host, self.port), self.timeout)
if self._tunnel_host:
self.sock = sock
self._tunnel()
# wrap the socket using verification with the root
# certs in ca_certs
if verbose:
print ca_certs, key_file, cert_file
self.sock = ssl.wrap_socket(sock,
cert_reqs=ssl.CERT_REQUIRED,
ca_certs=ca_certs,
keyfile=key_file,
certfile=cert_file,
)
if explicit_check:
cert = self.sock.getpeercert()
if verbose:
import pprint
pprint.pprint(cert)
for key,field in cert.iteritems():
if key=='subject':
sd = dict([x[0] for x in field])
certhost = sd.get('commonName')
if not fnmatch.fnmatch(self.host,certhost):
raise ssl.SSLError("Host name '%s' doesn't match certificate host
'%s'"
% (self.host, certhost))
if verbose:
print 'matched "%s" to "%s"' % (self.host,certhost)
elif key=='notAfter':
now = datetime.datetime.now()
crttime = datetime.datetime.strptime(field,'%b %d %H:%M:%S %Y %Z')
if verbose:
print 'crttime=%s now=%s' % (crttime,now)
if now>=crttime:
raise ssl.SSLError("Host '%s' certificate expired on %s"
% (self.host, field))
self.specialized_conn_class = SecuredHTTPSConnection
urllib2.HTTPSHandler.__init__(self)
def https_open(self, req):
return self.do_open(self.specialized_conn_class, req)
def secureDataGet(uri,ca_certs='cacert.pem',key_file=None,cert_file=None,
explicit_check=False):
https_handler = SecuredHTTPSHandler(key_file=key_file,cert_file=cert_file,
ca_certs=ca_certs,explicit_check=explicit_check)
url_opener = urllib2.build_opener(https_handler)
handle = url_opener.open(uri)
response = handle.readlines()
handle.close()
return response
--
Robin Becker
--
http://mail.python.org/mailman/listinfo/python-list
How good is security via hashing
A python web process is producing files that are given randomized names of the form hh-MMDDhhmmss-.pdf where rrr.. is a 128bit random number (encoded as base62). The intent of the random part is to prevent recipients of one file from being able to guess the names of others. The process was originally a cgi script which meant each random number was produced thusly pid is process id, dur is 4 bytes from /dev/urandom. random.seed(long(time.time()*someprimeint)|(pid<<64)|(dur<<32)) rrr = random.getrandbits(128) is this algorithm safe? Is it safe if the process is switched to fastcgi and the initialization is only carried out once and then say 50 rrr values are generated. -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
Re: How good is security via hashing
On 07/06/2011 11:26, Nitin Pawar wrote: Have you tried using UUID module? Its pretty handy and comes with base64 encoding function which gives extremely high quality randon strings ref: http://stackoverflow.com/questions/621649/python-and-random-keys-of-21-char-max .. I didn't actually ask for a suitable method for doing this; I assumed that Tim Peters' algorithm (at least I think he's behind most of the python random support) is pretty good so that the bits produced are indeed fairly good approximations to random. I guess what I'm asking is whether any sequence that's using random to generate random numbers is predictable if enough samples are drawn. In this case assuming that fastcgi is being used can I observe a sequence of generated numbers and work out the state of the generator. If that is possible then the sequence becomes deterministic and such a scheme is useless. If I use cgi then we're re-initializing the sequence hopefully using some other unrelated randomness for each number. Uuid apparently uses machine internals etc etc to try and produce randomness, but urandom and similar can block so are probably not entirely suitable. -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
Re: How good is security via hashing
/dev/urandom does not block, that's the point of it as compared to / dev/random. Jean-Paul my mistake, I thought it was the other way round, on FreeBSD they're the same anyway which is what we test on. -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
Re: How good is security via hashing
On 07/06/2011 12:40, Jean-Paul Calderone wrote: astcgi and the initialization is only carried out once and then say 50 rrr values are generated. How much randomness do you actually have in this scheme? The PID is probably difficult for an attacker to know, but it's allocated roughly monotonically with a known wrap-around value. The time is probably roughly known, so it also contributes less than its full bits to the randomness. Only dur is really unpredictable. So you have something somewhat above 4 bytes of randomness in your seed - perhaps 8 or 10. That's much less than even the fairly small 16 bytes of "randomness" you expose in the filename. I'm sure you're right about the limited amount of entropy in the initial state, but how much state can be in the prng? The random module is entirely deterministic, so once the seed is known the value you produce is known too. Is 10 bytes enough to thwart your attackers? Hard to say, what does an attack look like? An attacker could try to gain information from seeing others' results by guessing the filename. an attack would consist of generating a sample file via a web query which might take 1 or 2 seconds; the sequence number could then be seen and if the state established future filenames could be guessed if fastcgi is in operation. In a cgi type scheme that requires searching over the pid space, the time space and some random bits from the OS. I'm not sure such an attack is realistic given the size of the space even in the initial seed. If you want the full 16 bytes of unpredictability, why don't you just read 16 bytes from /dev/urandom and forget about all the other stuff? Jean-Paul I have a vague memory that the original author felt that entropy might run out or something like that so reading from /dev/urandom always was not a good idea. FreeBSD re-uses the entropy, but the end target is Solaris so I'm not really sure about the details of /dev/urandom. -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
Re: How good is security via hashing
On 07/06/2011 21:42, Paul Rubin wrote: geremy condra writes: # adds random junk to the filename- should make it hard to guess rrr = os.urandom(16) fname += base64.b64encode(rrr) Don't use b64 output in a filename -- it can have slashes in it! :-( Simplest is to use old fashioned hexadeimal for stuff like that, unless the number of chars is a significant problem. Go for a more complicated encoding if you must. we have been using base62 ie 0-9A-Za-z just to reduce the name length. -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
Re: How to read barcoded value from PDF
On 28/06/2011 06:59, Asif Jamadar wrote:
Hi,
...
In Reportlab I can do the following code to generate barcode and to get the
value of that barcode
barcode=code39.Extended39("123456789",barWidth=0.2*mm,barHeight=8*mm)
bc = Paragraph("Barcode value: %s" % barcode.value, STYLES['Normal'])
document.append(bc)
But how can I achieve this from the existing PDF document??
.
you might consider asking on the reportlab list as there is considerable
experience there about pdf in general.
It's unlikely that you will be able to easily discern which string/text in the
pdf corresponds to the barcode values that you are interested in.
PDF does allow things called annotations which reportlab can generate.
Alternatively you can generate an invisible string which may make more sense
than the simple barcode value. So when you draw the barcode you also need to add
the magic string using some prefix/postfix that allows easy extraction with
pypdf or similar eg
"===radamajfisa===123456789===radamajfisa===". Your text extractor should be
able to find this without too much trouble.
--
Robin Becker
--
http://mail.python.org/mailman/listinfo/python-list
Re: Implicit initialization is EVIL!
On 03/07/2011 23:21, Chris Angelico wrote: . var(0x14205359) x # Don't forget to provide an address where the object will be located x=42 did you forget to specify the memory bank and computer (and presumably planet etc etc) -molly-coddled-ly yrs- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
Re: Implicit initialization is EVIL!
On 05/07/2011 16:33, nn wrote: .. Ah, I see we have a mainframe programmer among us ... :-) so long since I manipulated the switches of that old pdp-8 -anciently yrs- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
Re: reportlab import error after dundled using py2exe
On 22/07/2011 03:55, SANKAR . wrote:
Hi all,
C:\Python26\dist>DELchek.exe
Traceback (most recent call last):
File "DELchek.py", line 12, in
File "reportlab\pdfgen\canvas.pyc", line 25, in<
File "reportlab\pdfbase\pdfdoc.pyc", line 22, in
File "reportlab\pdfbase\pdfmetrics.pyc", line 23,
File "reportlab\pdfbase\_fontdata.pyc", line 158,
ImportError: No module named _fontdata_enc_winansi
But I could see the '_fontdata_enc_winansi' module in reportlab folder.
Could someone help me to fix this.
.
You can try asking this in the reportlab list
reportlab-us...@lists2.reportlab.com
but perhaps this is more about py2exe than reportlab. The modules
_fontdata_enc_* & _fontdata_widths_* are imported dynamically in _fontdata.py
rather than explicitly. I suspect that py2exe needs to be given a hint that this
is going on. However, I'm uncertain as to why this should be required since even
if the imports are being dynamically imported that is done as soon as _fontdata
is imported (ie it's part of the module code) so those modules should be seen by
the setup.py.
If you don't have reportlab explicitly imported as part of the packages try
adding this to the packages list
packages=[
'reportlab',
'reportlab.graphics.charts',
'reportlab.graphics.samples',
'reportlab.graphics.widgets',
'reportlab.graphics.barcode',
'reportlab.graphics',
'reportlab.lib',
'reportlab.pdfbase',
'reportlab.pdfgen',
'reportlab.platypus',
],
that's what we use to make the distributions and seems to work.
--
Robin Becker
--
http://mail.python.org/mailman/listinfo/python-list
Re: arbitrary precision linear algebra
On 02/03/2011 16:39, Ben123 wrote: ... Languages can't support infinitely large or small numbers, so try to multiply the inner variables by 10^n to increase their values if this will not involve on the method. For example, I did this when was calculating geometric means of computer benchmarks. Currently I have values between 1 and 1E-300 (not infinitely small). I don't see how scaling by powers of 10 will increase precision. In such way you will be storing the number of zeros as n. Are you saying python cares whether I express a number as 0.001 or scaled by 10^5 to read 100? If this is the case, I'm still stuck. I need the full range of eigenvalues from 1 to 1E-300, so the entire range could be scaled by 1E300 but I would still need better precision than 1E19 ... If you enter a number as 1e-19 then python will treat as a float; by default I think that float is IEEE double precision so you're getting a 48 bit mantissa (others may have better details). That means you've already lost any idea of arbitrary precision. When you say you have numbers like 1E-300 are those actually numerically zero or have you some valid inputs that vary over a huge range. It should be possible to compute determinants/inverses etc to arbitrary precision as those are known to be polynomial functions of the input elements. However, eigenvalues are not. eg [0 2] [1 0] has eigenvalues +/- sqrt(2) so even though you can represent the matrix in finite precision the eigenvalues require infinite precision. Eigenvalues are roots of a polynomial in the elements and root solving may require an infinite number of steps so it will be difficult with arbitrary matrices to keep arbitrary precision. -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
Re: Generate PDF with Tamil font problem
On 05/03/2011 05:20, sathe...@e-ndicus.com wrote: Hi All, I am using python's reportlab to print some unicode Tamil characters 'பே'. I added necessary unicode font to reportlab. But It prints the output as 'ேப' (in reverse order). This issue happens for multi-byte characters, whereas for character 'ப' is printed as it is. I am struggling to figure out the issue. Any help would see me on track. ... you can ask at the reportlab mailing list reportlab-us...@reportlab.com; the use of numeric entities is supported in paragraph text, but not in canvas.drawString and similar. There you need to use either unicode or a utf8 encoded byte string. -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
full reload of fastcgi
A recent problem with a fastcgi script leads me to ask if there's any well defined way to fully reload a python fastcgi process. I know that people do tricks like unloading/reloading all modules, but it seems there's no obvious way to get python to restart itself. I believe that the server is apache based with mod_fcgid. The script in question suffers from two problems 1) it runs out of a zip file and the software can be upgraded by moving a new zip file into place. 2) A cron job pulls new data into place which the script should notice. The last is easy we can just re-read the data files before acting on the request, but it seems much harder to reload all modules from the zipfile. An easy way out is to just halt the running script when a software change is detected. That means we must run with the old software for at least one request and could be dangerous if an import is required during the last request. Normally the upgrade process is supposed to move software into place and then new data and then restart the fastcgi scripts; for whatever reason the error seems to have been that the process was restarted with old data around. Any ideas? -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
Re: Abandoning Python
On 21/05/2011 16:49, John J Lee wrote: I still like Python after using it for over a decade, but there are things I don't like. .. a relatively new one that's going about is cobra, http://cobra-language.com/, it appears to have some of the features you indicate eg speed, some kind of interfaces (contracts I think), but it needs net or mono. -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
windows 8 versus urllib2 certificate verify
I have an application built on 32 bit windows 7 with python 2.7.10. The application runs fine on windows 7 and older windows machines, but it is failing to connect properly using urllib2 when run on windows 8. The error CERTIFICATE_VERIFY_FAILED indcates this is some issue with urllib2 not being able to verify the remote certificate. This is pyinstaller so the python and python library seem to be constant. I thought I understood that python uses its own cert path, but obviously I am wrong. Googling sort of implies I might need certifi to be installed, but is that true? Why does this fail only on windows 8? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: windows 8 versus urllib2 certificate verify
On 12/09/2017 08:35, dieter wrote: Robin Becker writes: Certificate verification generally depends on local configuration: specifically, the set of installed trusted root certificates. I do not know about Windows, but often the root certificates installed for other packages, e.g. the browser, are used. And different versions of the same package may install different sets of trusted certificates. It might also be that your new environment lacks one of the packages from your old environment - and it has provided the required root certificate. Of course, the problem might also be caused by a general problem. Try to access a "common" https site via urllib2, one you do not have any problems to connect with a browser (or another "http" utility like "wget" or "curl") from the same machines. If this works, the problem is assiciated with the specific certificate. The certs are fine at least in all the browsers I have access to. It's pretty hard to use the python that's built in to a pyinstaller exe so I have used the certifi method and passed certifi.where() into the urlopen calls. That seems to work, but obviously if the compiled in root certs become too old problems will recur. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: How do I check all variables returned buy the functions exists
On 16/09/2017 01:58, Steve D'Aprano wrote: If you want to test for None specifically: if any(v is None for v in values): print "at least one value was None" ... for some reason that seems slow on my machine when compared with if None in values: . C:\usr\share\robin\pythonDoc>python -m timeit -s"values=(1,2,None)" "any(v is None for v in values)" 100 loops, best of 3: 0.62 usec per loop C:\usr\share\robin\pythonDoc>python -m timeit -s"values=(None,2,None)" "any(v is None for v in values)" 100 loops, best of 3: 0.504 usec per loop C:\usr\share\robin\pythonDoc>python -m timeit -s"values=(None,2,None)" "None in values" 1000 loops, best of 3: 0.0309 usec per loop C:\usr\share\robin\pythonDoc>python -m timeit -s"values=(1,2,None)" "None in values" 1000 loops, best of 3: 0.097 usec per loop it also seems a bit less obvious -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Even Older Man Yells At Whippersnappers
On 20/09/2017 10:54, Chris Angelico wrote: What, you take silicon that someone else created?! ChrisA well I had germanium for flipflops and dekatron tubes with neon for counters never built anything digital with valves though -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: The "loop and a half"
Given the prevalence of the loop and a half idea in python I wonder why we don't have a "do" or "loop" statement to start loops without a test. C:\Python27\Lib>grep "while True" *.py | wc -l 99 C:\Python27\Lib>grep "while 1" *.py | wc -l 117 C:\Python36\Lib>grep "while True" *.py | wc -l 131 C:\Python36\Lib>grep "while 1" *.py | wc -l 44 How much does the while True actually cost compared to nothing? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: The "loop and a half"
On 04/10/2017 11:57, Rhodri James wrote: On 04/10/17 10:01, Robin Becker wrote: Given the prevalence of the loop and a half idea in python I wonder why we don't have a "do" or "loop" statement to start loops without a test. See PEP 315. Guido's rejection note is here: https://mail.python.org/pipermail/python-ideas/2013-June/021610.html seems fair enough; I suppose the cost is negligible or perhaps there's peephole optimization for this common case. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: why del is not a function or method?
On 16/10/2017 16:37, Xue Feng via Python-list wrote:
Hi,
I wonder why 'del' is not a function or method. Most operations can be used
as follows
len(team)
or
team.append("tom")
But, I think
del team[2]
is somewhat uncommon. Why does not it take a syntax we are famillar with?
It can look like a function
x = 3
del(x)
x
Traceback (most recent call last):
File "", line 1, in
NameError: name 'x' is not defined
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
efficient way to get a sufficient set of identifying attributes
Given a list of objects with attributes a0, a1, a2,an-1 is there an efficient way to find sets of attributes which can be used to distinguish members of the list? As example a list of people might have firstName, lastName, nationality, postcode, phonenumber, as attributes. The probe items may have some of these attributes, but which should be used to test. Presumably the information in any attribute is highest if the number of distinct occurrences is the the same as the list length and pairs of attributes are more likely to be unique, but is there some proper way to go about determining what tests to use? A particular problem might be dynamic in that the list may be being constructed from the probes. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: efficient way to get a sufficient set of identifying attributes
On 19/10/2017 16:42, Stefan Ram wrote: Robin Becker writes: Presumably the information in any attribute is highest if the number of distinct occurrences is the the same as the list length and pairs of attributes are more likely to be unique, but is there some proper way to go about determining what tests to use? When there is a list |>>> list = [ 'b', 'b', 'c', 'd', 'c', 'b' ] |>>> l = len( list ) , the length of its set can be obtained: |>>> s = len( set( list )) . The entries are unique if the length of the set is the length of the list |>>> l == s |False And the ratio between the length of the set and the length of the list can be used to quantify the amount of repetiton. |>>> s / l |0.5 ... this sort of makes sense for single attributes, but ignores the possibility of combining the attributes to make the checks more discerning. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: efficient way to get a sufficient set of identifying attributes
On 19/10/2017 17:50, Stefan Ram wrote: Robin Becker writes: ... this sort of makes sense for single attributes, but ignores the possibility of combining the attributes to make the checks more discerning. What I wrote also applies to compound attributes (sets of base attributes). When there are n base attributes, one can form 2^n-1 compound attributes from them, or 2^n-1-n proper compound attributes. Therefore, a combinatoric explosion might impede the brute-force approach. A heuristics might start to explore combinations of keys with the best s/l ratio first and/or use preferences for certain fields set by a human. all good In database design, the keys are usually chosen by a human database designer using world knowledge. It sounds as if you want to have the computer make such a choice using only the information in the table as knowledge. I think I am tending towards the chosen by real world knowledge approach :( Your "identifying attributes" are called "super keys" in database science. You probably want minimal identifying attribute sets (without unneeded attributes), which are called "candidate keys". thanks for this and the reference below. So, now you can find and read literature, such as: Journal of Al-Nahrain University Vol.13 (2), June, 2010, pp.247-255 Science 247 Automatic Discovery Of Candidate In The Relational Databases Keys By Using Attributes Sets Closure Yasmeen F. Al-ward Department of Computer Science, College of Science, Al-Nahrain University. (The title was copied by me as found, the contents is in the web and makes more sense than the title.) -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Let's talk about debuggers!
On 25/10/2017 15:08, Michele Simionato wrote: pdb plus plus: https://pypi.python.org/pypi/pdbpp I like the idea, but in putty at least changing the terminal size causes pdb++ to detach immediately from the process and mess up the screen. I think this is caused by (5, 'Input/output error') here /home/rptlab/devel/otr/local/lib/python2.7/site-packages/pyrepl/fancy_termios.py in tcsetattr def copy(self): return self.__class__(self.as_list()) def tcgetattr(fd): return TermState(termios.tcgetattr(fd)) def tcsetattr(fd, when, attrs): termios.tcsetattr(fd, when, attrs.as_list()) error is here class Term(TermState): TS__init__ = TermState.__init__ def __init__(self, fd=0): self.TS__init__(termios.tcgetattr(fd)) self.fd = fd -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
headless python app for android/ios
In the past we have developed reportlab applications for use on android/ios devices. We used Kivy for the gui and the kivy setup did allow us to create a working reportlab pdf producer under the kivy gui. It was not exactly easy, but in the end we had a working PDF producer. A possible requirement is for pdf production using reportlab, but with others providing the gui using possible something called electron. I know very little about what is actually possible to provide api's in python under ios/android. Can the front end spawn a python process or must we run some kind of background app with listening sockets etc etc? Can applications call out to the world to download updated templates and so on? Any pointers would be useful. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Python Templating Language
On 17/12/2017 06:41, Abdur-Rahmaan Janhangeer wrote: Hi all, Can somebody point out to me some py-based template languages interpreters resources? Thank you ! https://bitbucket.org/rptlab/preppy -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
unicode direction control characters
I'm seeing some strange characters in web responses eg u'\u200e28\u200e/\u200e09\u200e/\u200e1962' for a date of birth. The code \u200e is LEFT-TO-RIGHT MARK according to unicodedata.name. I tried unicodedata.normalize, but it leaves those characters there. Is there any standard way to deal with these? I assume that some browser+settings combination is putting these in eg perhaps the language is normally right to left but numbers are not. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: unicode direction control characters
On 02/01/2018 15:18, Chris Angelico wrote: On Wed, Jan 3, 2018 at 1:30 AM, Robin Becker wrote: I'm seeing some strange characters in web responses eg u'\u200e28\u200e/\u200e09\u200e/\u200e1962' for a date of birth. The code \u200e is LEFT-TO-RIGHT MARK according to unicodedata.name. I tried unicodedata.normalize, but it leaves those characters there. Is there any standard way to deal with these? I assume that some browser+settings combination is putting these in eg perhaps the language is normally right to left but numbers are not. Unicode normalization is a different beast altogether. You could probably just remove the LTR marks and run with the rest, though, as they don't seem to be important in this string. ChrisA I guess I'm really wondering whether the BIDI control characters have any semantic meaning. Most numbers seem to be LTR. If I saw u'\u200f12' it seems to imply that the characters should be displayed '21', but I don't know whether the number is 12 or 21. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Tips or strategies to understanding how CPython works under the hood
http://aosabook.org/en/500L/a-python-interpreter-written-in-python.html -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
py2exe output flagged as malware
I see this has happened to others in the past. I'm using 32 bit python 2.7.10 with py2exe 3.3 on windows 7. The exes work fine, but when I try to download into windows 10 I'm getting the exes immediately removed as malware. Is there any way to prevent this. It's very bad for python programs to get this kind of reputation. Anyone know if the same happens with PyInstaller etc etc? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: py2exe output flagged as malware
On 28/02/2018 11:46, alister via Python-list wrote: On Wed, 28 Feb 2018 09:53:09 +, Robin Becker wrote: I see this has happened to others in the past. I'm using 32 bit python 2.7.10 with py2exe 3.3 on windows 7. The exes work fine, but when I try to download into windows 10 I'm getting the exes immediately removed as malware. Is there any way to prevent this. It's very bad for python programs to get this kind of reputation. Anyone know if the same happens with PyInstaller etc etc? I would suggest you contact the Anti-virus vendor, Are you sure you have download a clean copy of py2exe. The anti-virus vendor is Microsoft; I imagine I will get short shrift from them regarding this issue. Turns out my py2exe script was just pyinstaller under the hood. Apologies to py2exe. I guess someone changed the build script script to use a different python packager. I upgraded pyinstaller using the very latest pip and now the version of pyinstaller at least is 3.3.1. I don't actually know how to check the validity of the installed code or the binary stubs. After rebuilding with 3.3.1 the new exe is 10 bytes shorter and is not recognized as malware by windows 10. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: py2exe output flagged as malware
On 28/02/2018 16:25, Hartmut Goebel wrote: Am 28.02.2018 um 16:47 schrieb Robin Becker: I upgraded pyinstaller using the very latest pip and now the version of pyinstaller at least is 3.3.1. I don't actually know how to check the validity of the installed code or the binary stubs. The current archives are PyPI are PGP/GnuPG-signed, as the ones at github. so presumably I can locate the downloaded tar zip and hash it to check. Of course if pip is already busted it can out fox me anyhow. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
is mypy failing here
I haven't used dataclasses or typing very much, but while playing about I found this didn't give me an expected error (.py312) robin@minikat:~/devel/reportlab $ cat tmp/examples/tdc.py && python tmp/examples/tdc.py && mypy tmp/examples/tdc.py ## from dataclasses import dataclass @dataclass class DC: a: str b: str def main(): dc = DC(DC, "B") print(dc) if __name__ == "__main__": main() ## DC(a=, b='B') Success: no issues found in 1 source file (.py312) robin@minikat:~/devel/reportlab DC.a is supposed to be a str and I expected mypy to indicate a type error should typing work for this case? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: is mypy failing here
On 24/11/2022 14:10, Thomas Passin wrote: . C:\temp\python>py -V Python 3.10.4 C:\temp\python>py tdc.py DC(a=, b='B') C:\temp\python>mypy tdc.py tdc.py:10: error: Argument 1 to "DC" has incompatible type "Type[DC]"; expected "str" [arg-type] Found 1 error in 1 file (checked 1 source file) Forgot the mypy version: C:\Users\tom>mypy --version mypy 0.910 interesting; I'm on archlinux and neither the system python 3.10.8 / mypy 0.982 gives an error. I did try running in my self build 3.10.8 with latest mypy 0.991 and mypy 0.910 and I still don't get an error. I'll break out the windows 10 laptop and see what happens there. You ran with the py runner. I wonder if that does something special. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: is mypy failing here
On 24/11/2022 13:50, Kirill Ratkin via Python-list wrote: mypy --strict gives you detail info. Thanks Kirill, it seems --strict does find the errors. One of those is that on line 9 I have to add a return type ie def main() -> None: . if that is added then mypy without --strict also finds the real typing error. So it seems the tool fails in the simplest cases if you forget some typing. Interesting that it works in windows without --strict though. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
C extension custom types in abi3 module
I am trying to split off reportlab C extensions to simplify installations and
make use of more advanced packages.
A simple extension is easily converted to being an abi3 module. However, another has a custom type which uses the old
style mechanisms here
https://docs.python.org/3.7/extending/newtypes_tutorial.html#the-basics
I made a simple setup based on this abi3 example modified to allow switching
between abi3 and normal build
https://github.com/joerick/python-abi3-package-sample
I made a tiny change to the code example in the newtypes tutorial page the code
is here
https://github.com/MrBitBucket/custom-type
In a python 3.7 - 3.12a3 environment I find I can build the wheel OK with
ABI3_WHEEL=0 pip wheel -w dist .
but I get lots of compile errors if I switch to an abi3 build with
ABI3_WHEEL=1 pip wheel -w dist .
looking at the errors
src/_custom.c:10:1: error: variable ‘CustomType’ has initializer but
incomplete type
10 | static PyTypeObject CustomType = {
| ^~
In file included from
/home/robin/LOCAL/3.7.16/include/python3.7m/Python.h:90,
from src/_custom.c:2:
/home/robin/LOCAL/3.7.16/include/python3.7m/object.h:90:5: error: extra
brace group at end of initializer
90 | { PyObject_HEAD_INIT(type) size },
| ^
src/_custom.c:11:9: note: in expansion of macro ‘PyVarObject_HEAD_INIT’
11 | PyVarObject_HEAD_INIT(NULL, 0)
it looks like I have to use a different mechanism to setup custom types in the
abi3 world.
I looked in Modules/xxlimited_35.c, but that seems much more complex and
provides for a type which supports GC.
Are there any ABI3 examples using the old style strategy?
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: C extension custom types in abi3 module
On 08/12/2022 12:52, Robin Becker wrote: I am trying to split off reportlab C extensions to simplify installations and make use of more advanced packages. A simple extension is easily converted to being an abi3 module. However, another has a custom type which uses the old style mechanisms here it looks like I have to use a different mechanism to setup custom types in the abi3 world. In the docs I see this "Also, since PyTypeObject is only part of the Limited API as an opaque struct, any extension modules using static types must be compiled for a specific Python minor version." So it seems I must switch to using a heap allocated type or keep compiling in the old way. I looked in Modules/xxlimited_35.c, but that seems much more complex and provides for a type which supports GC. Are there any ABI3 examples using the old style strategy? -- Robin Becker -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
lxml with python-3.12.0a5
I'm trying to test python-3.12.0a5 and need to install lxml.
My wheel build for lxml fails with errors like this
src/lxml/etree.c: In function ‘__Pyx_PyIndex_AsSsize_t’:
src/lxml/etree.c:270404:45: error: ‘PyLongObject’ {aka ‘struct _longobject’}
has no member named ‘ob_digit’
270404 | const digit* digits = ((PyLongObject*)b)->ob_digit;
| ^~
I'm using archlinux which has gcc 12.2.1. I tested and lxml will build with
3.11.2.
I imagine this is part of ongoing changes to the python core, but perhaps
there's some simple workaround.
Anyone know how to get around this problem. I did try rebuilding the cpython
stuff using make, but that also failed.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: lxml with python-3.12.0a5
On 23/02/2023 18:09, Mats Wichmann wrote: I seem to always have trouble with lxml (which I know doesn't help). The cause would seem to be this: GH-101291: Refactor the `PyLongObject` struct into object header and PyLongValue struct. (GH-101292) So it looks to me like cython was affected, and has patched themselves. It's possible regenerating with a patched cython will clear up the build problem (something which the lxml project takes pains to tell you that you don't really want to do :) Thanks Mats, I build a cython wheel from git latest and after installing into python-3.12.0a5 I used latest lxml source and python setup.py bdist_wheel --with-cython which built without error. The installed lxml seems fine (at least for reportlab tests). -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.12.0 alpha 6 released
On 08/03/2023 04:37, Thomas Wouters wrote:
I'm pleased to announce the release of Python 3.12 alpha 6.
https://www.python.org/downloads/release/python-3120a6/
*This is an early developer preview of Python 3.12.*
Major new features of the 3.12 series, compared to 3.11
I was able to test reportlab with the 3.12.0a5 release (I build following the Archlinux pkgbuild), but am unable to do
so with 3.12.0a6 because of problems with cython/lxml/freetype-py (I think).
With an optimized build of a6 I was getting segfaults which I think were caused by incompatible C extensions from the
pip cache so decided to rebuild the requirements.
With latest cython git (allegedly 3.0.0b1) I see errors related to tstate eg
Cython/Compiler/Parsing.c:86861:34: error: ‘PyThreadState’ {aka ‘struct _ts’}
has no member named ‘curexc_traceback’
86861 | PyObject* tmp_tb = tstate->curexc_traceback;
I guess that this is caused by changes in what we are allowed to see of Python
internal structure.
If anyone knows of a way to advance further I can try to experiment.
Python 3.12 is still in development. This release, 3.12.0a6 is the sixth of
seven planned alpha releases.
Alpha releases are intended to make it easier to test the current state of
new features and bug fixes and to test the release process.
During the alpha phase, features may be added up until the start of the
beta phase (2023-05-08) and, if necessary, may be modified or deleted up
until the release candidate phase (2023-07-31). Please keep in mind that
this is a preview release and its use is not recommended for production
environments.
Many new features for Python 3.12 are still being planned and written.
Among the new major new features and changes so far:
Your release team,
Thomas Wouters
Ned Deily
Steve Dower
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: [RELEASE] Python 3.12.0 beta 1 released.
On 22/05/2023 22:04, Thomas Wouters wrote:
> I'm pleased to announce the release of Python 3.12 beta 1 (and feature
> freeze for Python 3.12).
>
...
I see a major difference between 3.12.0a7 and 3.12.0b1
Basically in preppy an importer is defined to handle imports of '.prep' files.
This worked as expected in the a7 version and fails in the b1. I put in some prints in the code and I see these calls
for the a7 run> $ ~/LOCAL/3.12.0a7/bin/python3 test_import.py
> sys.meta_path.insert(.PreppyImporter object at 0x7fa870b84080>) -->
[.PreppyImporter object at 0x7fa870b84080>, <_distutils_hack.DistutilsMetaFinder object
at 0x7fa871290fb0>, , , '_frozen_importlib_external.PathFinder'>]
> PreppyImporter.find_module('sample001',None)
> PreppyImporter.load_module('sample001')
> 4
> .
> --
> Ran 1 test in 0.004s
>
> OK
In 3.12.0b1 although the importer is inserted into sys.meta_path the
find_module/load_module methods are never called.
and the import fails.
So is this an expected change in the way importers behave or a bug?
> $ ~/LOCAL/3.12.0b1/bin/python3 test_import.py
> sys.meta_path.insert(.PreppyImporter object at 0x7fc866ecb110>) -->
[.PreppyImporter object at 0x7fc866ecb110>, ,
, ]
> E
> ==
> ERROR: testImport1 (__main__.ImportTestCase.testImport1)
> --
> Traceback (most recent call last):
> File "/home/robin/devel/reportlab/REPOS/preppy/tmp/test_import.py", line
13, in testImport1
> import sample001
> ModuleNotFoundError: No module named 'sample001'
>
> --
> Ran 1 test in 0.001s
>
> FAILED (errors=1)
..
> Your release team,
> Thomas Wouters
> Ned Deily
> Steve Dower
--
https://mail.python.org/mailman/listinfo/python-list
Re: [RELEASE] Python 3.12.0 beta 1 released.
On 25/05/2023 12:23, Robin Becker wrote: On 22/05/2023 22:04, Thomas Wouters wrote: > I'm pleased to announce the release of Python 3.12 beta 1 (and feature > freeze for Python 3.12). > ... I see a major difference between 3.12.0a7 and 3.12.0b1 Basically in preppy an importer is defined to handle imports of '.prep' files. This worked as expected in the a7 version and fails in the b1. I put in some prints in the code and I see these calls for the a7 run> $ ~/LOCAL/3.12.0a7/bin/python3 test_import.py > sys.meta_path.insert(.PreppyImporter object at 0x7fa870b84080>) --> . I think this might be caused by the removal of the find_module method of importlib.abc.MetaPathFinder. So I guess I have to implement a modernised importer. Apologies for noise if that is the case. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
advice on debugging a segfault
I have a segfault in the 3.10 alpha 4 when running the reportlab document generation; all the other tests seem to have worked. I would like to ask experts here how to approach getting the location of the problem. I run recent archlinux. Below is the output of a test run with -Xdev -Xtracemalloc; the main process is almost finished so the error appears to come from trying to free resources $ python -Xdev -Xtracemalloc genuserguide.py /home/robin/devel/reportlab/.py310/lib/python3.10/distutils/__init__.py:1: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp Built story contains 1843 flowables... Saved "/home/robin/devel/reportlab/REPOS/reportlab/docs/reportlab-userguide.pdf" [ Top 10 ] :630: size=10.5 MiB, count=105832, average=104 B /home/robin/devel/reportlab/reportlab/lib/abag.py:19: size=7161 KiB, count=27126, average=270 B /home/robin/devel/reportlab/reportlab/platypus/paraparser.py:3141: size=3364 KiB, count=5980, average=576 B /home/robin/devel/reportlab/.py310/lib/python3.10/site-packages/pyphen/__init__.py:160: size=2905 KiB, count=41797, average=71 B /home/robin/devel/reportlab/reportlab/platypus/paragraph.py:776: size=1386 KiB, count=32208, average=44 B /home/robin/devel/reportlab/reportlab/platypus/paragraph.py:92: size=1384 KiB, count=26248, average=54 B /home/robin/devel/reportlab/.py310/lib/python3.10/copy.py:280: size=1232 KiB, count=8827, average=143 B /home/robin/devel/reportlab/reportlab/platypus/frames.py:155: size=1101 KiB, count=1173, average=962 B :241: size=915 KiB, count=8146, average=115 B /home/robin/devel/reportlab/reportlab/platypus/paragraph.py:773: size=881 KiB, count=16104, average=56 B sys:1: ResourceWarning: unclosed file <_io.BufferedReader name='../images/replogo.gif'> sys:1: ResourceWarning: unclosed file <_io.BufferedReader name='../images/testimg.gif'> Debug memory block at address p=0x5617c33effe0: API '' 0 bytes originally requested The 7 pad bytes at p-7 are not all FORBIDDENBYTE (0xfd): at p-7: 0x00 *** OUCH at p-6: 0x00 *** OUCH at p-5: 0x00 *** OUCH at p-4: 0x00 *** OUCH at p-3: 0x00 *** OUCH at p-2: 0x00 *** OUCH at p-1: 0x00 *** OUCH Because memory is corrupted at the start, the count of bytes requested may be bogus, and checking the trailing pad bytes may segfault. The 8 pad bytes at tail=0x5617c33effe0 are not all FORBIDDENBYTE (0xfd): at tail+0: 0x00 *** OUCH at tail+1: 0x00 *** OUCH at tail+2: 0x00 *** OUCH at tail+3: 0x00 *** OUCH at tail+4: 0x00 *** OUCH at tail+5: 0x00 *** OUCH at tail+6: 0x00 *** OUCH at tail+7: 0x00 *** OUCH Enable tracemalloc to get the memory block allocation traceback Fatal Python error: _PyMem_DebugRawFree: bad ID: Allocated using API '', verified using API 'o' Python runtime state: finalizing (tstate=0x5617c53c8b30) Current thread 0x7fd5742b3740 (most recent call first): Aborted (core dumped) for comparison here is the 3.9.1 output> $ python -Xdev -Xtracemalloc genuserguide.py /home/robin/devel/reportlab/.py39/lib/python3.9/distutils/__init__.py:1: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp /home/robin/devel/reportlab/.py39/lib/python3.9/site-packages/fontTools/misc/py23.py:11: DeprecationWarning: The py23 module has been deprecated and will be removed in the next release. Please update your code. warnings.warn( Built story contains 1843 flowables... Saved "/home/robin/devel/reportlab/REPOS/reportlab/docs/reportlab-userguide.pdf" [ Top 10 ] :587: size=12.4 MiB, count=128010, average=102 B /home/robin/devel/reportlab/reportlab/lib/abag.py:19: size=7132 KiB, count=26951, average=271 B /home/robin/devel/reportlab/reportlab/platypus/paraparser.py:3141: size=3364 KiB, count=5980, average=576 B /home/robin/devel/reportlab/.py39/lib/python3.9/site-packages/pyphen/__init__.py:160: size=2905 KiB, count=41797, average=71 B /home/robin/devel/reportlab/reportlab/platypus/paragraph.py:776: size=1386 KiB, count=32208, average=44 B /home/robin/devel/reportlab/reportlab/platypus/paragraph.py:92: size=1384 KiB, count=26248, average=54 B /home/robin/devel/reportlab/.py39/lib/python3.9/copy.py:279: size=1230 KiB, count=8827, average=143 B /home/robin/devel/reportlab/reportlab/platypus/frames.py:155: size=1039 KiB, count=957, average=1112 B :228: size=959 KiB, count=8503, average=116 B /home/robin/devel/reportlab/reportlab/platypus/paragraph.py:773: size=881 KiB, count=16104, average=56 B sys:1: ResourceWarning: unclosed file <_io.BufferedReader name='../images/replogo.gif'> sys:1: ResourceWarning: unclosed file <_io.BufferedReader name='../images/testimg.gif'> -- https://mail.python.org/mailman/listinfo/python-list
Re: advice on debugging a segfault
On 17/01/2021 21:35, Stestagg wrote: I would normally agree, except... This is a refcount issue (I was able to reproduce the problem, gbd shows a free error ) And I wouldn't recommend DGBing a refcount issue as a beginner to debugging. The other mailing list identified a PIL bug that messes up the refcount for True, but this refcount issue is for some tuple object, so may possibly be a different problem. Steve .. thanks for all the advice and apologies for the noise. It turns out that Victor Stinner's recommendation in the dev list was the solution as I don't see the segfault with pillow built from latest git Pillow @ git+git://github.com/python-pillow/pillow.git@8dc5f692dbd9a418d8c441f0e2aa09a3f5c03508 I did realize that this must be a late issue in the process as the pdf was being produced and written as expected and that is normally the end of things -- not segfaulting-ly yrs Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
idlelib re-use
I googled in vain for instances where parts of idlelib are re-used in a simplistic way. I would like to use the editor functionality in a tkinter window and also probably run code in a subprocess. Are there any examples around that do these sorts of things? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: idlelib re-use
Thanks, On 28/01/2021 19:57, Terry Reedy wrote: On 1/28/2021 5:53 AM, Robin Becker wrote: I googled in vain for instances where parts of idlelib are re-used in a simplistic way. I would like to use the editor functionality in a tkinter window and also probably run code in a subprocess. Are there any examples around that do these sorts of things? turtledemo reuses IDLE's colorizer and read-only textviews. I have seen occasional hints on stackoverflow of other such uses. One barrier to reuse is that the parts are highly interconnected, with numerous import loops. (Changing the form of some imports from 'import x' to 'from x import y' can make IDLE startup fail.) Some objects, like EditorWindow, are too monolithic. You cannot put a toplevel inside another toplevel. yes I found this out trying to reuse the editor window and shell. Another, for those thinking long term, is that implementation modules in idlelib are defined as private since 3.6 (PEP-434, idlelib.__init__). I pushed for this in order to be able to refactor to reduce interconnections, and break some things apart, and also to switch to ttk widgets. For instance, breaking EditorFrame apart from EditorWindow would allow me to put multiple editors on multiple tabs of a notebook in an application window. It would also allow others to put an editor window in their tkinter window. +1 -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
new match statement and types
I'm trying out new features in 3.10.0a6 looking at the new match statement I tried it with various cases. Knowing that
the class of a class is 'type' I tried using match to distinguish between classes and instances so I tried various
versions of
###
class A:
pass
class B:
pass
def tmc(C):
match C:
case type(__name__='A'):
print(f'{C} is an A' )
case type():
print(f'{C} is a type' )
case A():
print(f'{C} is an A instance')
case _:
print(f'{C} is an instance')
if __name__=='__main__':
tmc(A)
tmc(B)
tmc(A())
tmc(B())
###
the above seems to work and produces
is an A
is a type
<__main__.A object at 0x7fe5a2248fd0> is an A instance
<__main__.B object at 0x7fe5a2248fd0> is an instance
is this the right approach to this problem of distinguishing instances ?
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python script accessing own source code
... with open(__file__) as myself: print(myself.read(), end='') very nice, but accessing code that's already seems quite easy. I think the real problem is to get a python script name that creates and writes itself. So I would ask if any one has the solution to the self writing script python find-tomorrows-lotto-numbers.py since GvR has been shown to have time traveling abilities such a script could paradoxically appear acausally. -- yrs-not-too-seriously Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Python script accessing own source code
On 12/05/2021 20:17, Mirko via Python-list wrote: Am 12.05.2021 um 20:41 schrieb Robin Becker: ... ... since GvR has been shown to have time traveling abilities such a script could paradoxically appear acausally. -- yrs-not-too-seriously Robin Becker Not sure, if that's what you mean, but writing a self-replicating script is easy too: actually I was really joking about self creating scripts that do something useful like finding future lotto numbers. the artificial programmer servant is some way off import os import sys with open(os.path.abspath(__file__)) as myself: with open(sys.argv[1], "w") as yourself: yourself.write(myself.read()) Give it a filename as a command-line argument and it will write itself to that file. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
error from pypa build
I'm building a pure python wheel in a python3.9 virtualenv. If I use python setup.py bdist_wheel I do get a wheel named rlextra-3.6.0-py3-none-any.whl I'm trying out building a modern python 3 only wheel so I followed instructions here https://packaging.python.org/tutorials/packaging-projects/#creating-the-package-files and see the error below which is not very informative. Any ideas of what's wrong? I still get the error even after upgrading pip so I don't think that is the problem. > (.py39) user@pc:~/devel/reportlab/REPOS/rlextra > $ pip install build > Collecting build > Downloading build-0.5.1-py2.py3-none-any.whl (15 kB) > Collecting pep517>=0.9.1 > .. > Installing collected packages: tomli, pyparsing, toml, pep517, packaging, build > Successfully installed build-0.5.1 packaging-21.0 pep517-0.11.0 pyparsing-2.4.7 toml-0.10.2 tomli-1.2.0 > WARNING: You are using pip version 21.1.3; however, version 21.2.2 is available. > You should consider upgrading via the '/home/user/devel/reportlab/.py39/bin/python39 -m pip install --upgrade pip' command. > (.py39) user@pc:~/devel/reportlab/REPOS/rlextra > $ ls > build dist docs examples LICENSE.txt README.txt setup.cfg setup.py src tests tmp > (.py39) user@pc:~/devel/reportlab/REPOS/rlextra > $ python -m build --wheel > Traceback (most recent call last): > File "/home/user/devel/reportlab/.py39/lib/python3.9/site-packages/build/__main__.py", line 302, in main > build_call(args.srcdir, outdir, distributions, config_settings, not args.no_isolation, args.skip_dependency_check) > File "/home/user/devel/reportlab/.py39/lib/python3.9/site-packages/build/__main__.py", line 145, in build_package > _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check) > File "/home/user/devel/reportlab/.py39/lib/python3.9/site-packages/build/__main__.py", line 101, in _build > return _build_in_isolated_env(builder, outdir, distribution, config_settings) > File "/home/user/devel/reportlab/.py39/lib/python3.9/site-packages/build/__main__.py", line 77, in _build_in_isolated_env > with IsolatedEnvBuilder() as env: > File "/home/user/devel/reportlab/.py39/lib/python3.9/site-packages/build/env.py", line 92, in __enter__ > executable, scripts_dir = _create_isolated_env_venv(self._path) > File "/home/user/devel/reportlab/.py39/lib/python3.9/site-packages/build/env.py", line 221, in _create_isolated_env_venv > pip_distribution = next(iter(metadata.distributions(name='pip', path=[purelib]))) > StopIteration > > ERROR > (.py39) user@pc:~/devel/reportlab/REPOS/rlextra thanks for any assistance -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
basic auth request
While porting an ap from python2.7 to python3 I see this
base64string = base64.b64encode('%s:%s' % (wsemail, wspassword))
request.add_header("Authorization", "Basic %s" % base64string)
in python3.x I find this works
base64string = base64.b64encode(('%s:%s' % (wsemail,
wspassword)).encode('ascii')).decode('ascii')
request.add_header("Authorization", "Basic %s" % base64string)
but I find the conversion to and from ascii irksome. Is there a more direct way
to create the basic auth value?
As an additional issue I find I have no clear idea what encoding is allowed for
the components of a basic auth input.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: basic auth request
On 17/08/2021 22:47, Jon Ribbens via Python-list wrote: ... That's only true if you're not using HTTPS - and you should *never* not be using HTTPS, and that goes double if forms are being filled in and double again if passwords are being supplied. I think I agree with most of the replies; I understood from reading the rfc that the charset is utf8 (presumably without ':') and that basic auth is considered insecure. It is being used over https so should avoid the simplest net scanning. I googled a bunch of ways to do this, but many come down to 1) using the requests package or 2) setting up an opener. Both of these seem to be much more complex than is required to add the header. I thought there might be a shortcut or more elegant way to replace the old code, but it seems not thanks -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
what to do with multiple BOMs
Channeling unicode text experts and xml people: I have xml entity with initial bytes ff fe ff fe which the file command says is UTF-16, little-endian text. I agree, but what should be done about the additional BOM. A test output made many years ago seems to keep the extra BOM. The xml context is xml file 014.xml ]> &e;\xef\xbb\xbfdata' which implies seems as though the extra BOM in the entity has been kept and processed into a different BOM meaning utf8. I think the test file is wrong and that multiple BOM chars in the entiry should have been removed. Am I right? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
c extension finding the module in object initialization
I have a c extension which is intended to implement a module the looks structurally like this a = 1 b = 2 class T: def __init__(self): self.a = a self.b = b so when an object of type T is instantiated it can set up defaults based on the current module values of a and b. In the past using old style single phase module creation the init function for the type could look up the module by using the PyState_FindModule function. In the new world where we can implement c extensions which might work with multiple interpreters (multi-phase creation). The docs say PyState_FindModule won't work for those and indeed it returns NULL so is useless. Is there a way I can set the current module onto the type definition during the module's exec function? I thought it would be easy to add at the point in the exec where I'm doing this to the module, m, TType.tp_base = &PyBaseObject_Type; if(PyType_Ready(&TType)<0) goto fail; if(PyModule_AddObject(m,"T", (PyObject *)&TType)<0) goto fail; but I don't see the place in the type where I can add these sorts of dynamic attributes. The basic_size is for the created objects. The created type does have a __dict__ which is a mappingproxy. I wondered when that actually gets instantiated. I can see that the type has a __module__ attribute after the module is imported and I suppose if I can find the 'current' interpreter I might use the __module__ to locate the module object via PyImport_GetModuleDict. Any expertise or advice gratefully received. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: c extension finding the module in object initialization
Hi Marc, Thanks for the suggestion, On 27/09/2021 09:38, Marc-Andre Lemburg wrote: Hi Robin, seeing that no one replied to your question, I'd suggest to ask this on the Python C-API ML: https://mail.python.org/mailman3/lists/capi-sig.python.org/ That's where the experts are, including the ones who implemented the mutli-phase logic. Cheers, I think I have this working using ob=PyImport_GetModuleDict() followed by PyDict_GetItemString(ob,"modulename"), but I will ask there to see if there's a more direct route. In Python >=3.7 there's PyImport_GetModule, but that seems more complex than is actually required for this simple case and has to wait until 3.6 dies for me :( -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
lxml parsing with validation and target?
I'm having a problem using lxml.etree to make a treebuilding parser that validates; I have test code where invalid xml
is detected and an error raised when the line below target=ET.TreeBuilder(), is commented out.
The validation error looks as expected > python tlxml.py invalid.rml
re.compile('^.*(?:\\W|\\b)(?Pdynamic_rml\\.dtd|rml\\.dtd|rml_0_2\\.dtd|rml_0_3\\.dtd|rml_1_0\\.dtd)$',
re.MULTILINE)
Resolving url='../rml.dtd' context= dtdPath='rml.dtd'
Traceback (most recent call last):
File "/home/robin/devel/reportlab/REPOS/rlextra/tmp/tlxml.py", line 78, in
tree = ET.parse(sys.argv[1],parser)
File "src/lxml/etree.pyx", line 3521, in lxml.etree.parse
File "src/lxml/parser.pxi", line 1859, in lxml.etree._parseDocument
File "src/lxml/parser.pxi", line 1885, in lxml.etree._parseDocumentFromURL
File "src/lxml/parser.pxi", line 1789, in lxml.etree._parseDocFromFile
File "src/lxml/parser.pxi", line 1177, in
lxml.etree._BaseParser._parseDocFromFile
File "src/lxml/parser.pxi", line 615, in
lxml.etree._ParserContext._handleParseResultDoc
File "src/lxml/parser.pxi", line 725, in lxml.etree._handleParseResult
File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError
File "invalid.rml", line 23
lxml.etree.XMLSyntaxError: No declaration for attribute x of element place1,
line 23, column 55
when I have the target=etree.TreeBuilder() active the validation does not work and the tree is formed and passed to the
primitive tuple tree builder so the output looks like
$ python tlxml.py invalid.rml
Resolving url='../rml.dtd' context= dtdPath='rml.dtd'
('document',
{'filename': 'test_000_simple.pdf', 'invariant': '1'},
['\n\n',
('stylesheet',
>
None,
44),
'\n\t\t\n\t\t'],
40),
'\n'],
35),
'\n\n'],
2)
If I use the standard example EchoTarget the validation also fails. So I assume that the target argument makes the
validation fail. Is there a way to get validation to work with a target?
The code is
##
from pprint import pprint
from lxml import etree as ET
import sys, os, re
from rlextra.rml2pdf.rml2pdf import CompatibleDTDNames as rmlDTDPat
rmlDTDPat = re.compile('^.*(?:\\W|\\b)(?P%s)$' % '|'.join((re.escape(_) for
_ in rmlDTDPat)),re.M)
class TT:
def __init__(self):
pass
def __call__(self,e):
return (e.tag,e.attrib or None,self.content(e),e.sourceline)
def content(self,e):
t = e.text
if len(e)==0 and t is None:
return t
else:
r = [].append
if t is not None: r(t)
for c in e:
r(self(c))
t = c.tail
if t is not None:
r(t)
return r.__self__
class RMLDTDResolver(ET.Resolver):
__dtds = None
def resolve(self, url, id, context):
m = rmlDTDPat.match(url)
if m:
if self.__dtds is None:
from rlextra import rml2pdf
self.__dtds = {}
for fn in ('rml.dtd','dynamic_rml.dtd'):
with
open(os.path.join(os.path.dirname(rml2pdf.__file__),fn),'r') as _:
self.__dtds[fn] = _.read()
fn = m.group('fn')
dtdPath = 'rml.dtd' if fn.startswith('rml') else 'dynamic.dtd'
print(f"Resolving url={url!r} context={context!r} {dtdPath=}")
return self.resolve_string(
self.__dtds[dtdPath],
context,
)
else:
return None
parser = ET.XMLParser(
load_dtd=True,
dtd_validation=True,
attribute_defaults=True,
no_network=True,
remove_comments=True,
remove_pis=True,
strip_cdata=True,
resolve_entities=True,
target=ET.TreeBuilder(), #if commented the parser
validates
)
parser.resolvers.add(RMLDTDResolver())
tree = ET.parse(sys.argv[1],parser)
pprint(TT()(tree))
##
--
https://mail.python.org/mailman/listinfo/python-list
Re: lxml parsing with validation and target?
On 02/11/2021 12:55, Robin Becker wrote: I'm having a problem using lxml.etree to make a treebuilding parser that validates; I have test code where invalid xml is detected and an error raised when the line below target=ET.TreeBuilder(), is commented out. . I managed to overcome this problem by utilizing the non-targeted parser with returns an _ElementTree object. I can then convert to tuple tree using code like this class TT: def __init__(self): pass def __call__(self,tree): if not tree: return return self.maketuple(next(tree.iter())) def maketuple(self,e): return (e.tag,e.attrib or None,self.content(e),e.sourceline) def content(self,e): t = e.text kids = e.getchildren() if len(kids)==0 and t is None: return t else: r = [].append if t is not None: r(t) for c in kids: r(self.maketuple(c)) t = c.tail if t is not None: r(t) return r.__self__ -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
preserving entities with lxml
I have a puzzle over how lxml & entities should be 'preserved' code below illustrates. To preserve I change & --> &
in the source and add resolve_entities=False to the parser definition. The escaping means we only have one kind of
entity & which means lxml will preserve it. For whatever reason lxml won't preserve character entities eg !.
The simple parse from string and conversion tostring shows that the parsing at
least took notice of it.
However, I want to create a tuple tree so have to use tree.text,
tree.getchildren() and tree.tail for access.
When I use those I expected to have to undo the escaping to get back the original entities, but it seems they are
already done.
Good for me, but if the tree knows how it was created (tostring shows that) why
is it ignored with attribute access?
if __name__=='__main__':
from lxml import etree as ET
#initial xml
xml = b'a &mysym; < & >
! A'
#escaped xml
xxml = xml.replace(b'&',b'&')
myparser = ET.XMLParser(resolve_entities=False)
tree = ET.fromstring(xxml,parser=myparser)
#use tostring
print(f'using tostring\n{xxml=!r}\n{ET.tostring(tree)=!r}\n')
#now access the items using text & children & text
print(f'using
attributes\n{tree.text=!r}\n{tree.getchildren()=!r}\n{tree.tail=!r}')
when run I see this
$ python tmp/tlp.py
using tostring
xxml=b'a &mysym; < & >
! A'
ET.tostring(tree)=b'a &mysym; < &
> ! A'
using attributes
tree.text='a &mysym; < & > ! A'
tree.getchildren()=[]
tree.tail=None
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: preserving entities with lxml
On 12/01/2022 20:49, Dieter Maurer wrote: ... when run I see this $ python tmp/tlp.py using tostring xxml=b'a &mysym; < & > ! A' ET.tostring(tree)=b'a &mysym; < & > ! A' using attributes tree.text='a &mysym; < & > ! A' tree.getchildren()=[] tree.tail=None Apparently, the `resolve_entities=False` was not effective: otherwise, your tree content should have more structure (especially some entity reference children). except that the tree knows not to expand the entities using ET.tostring so in some circumstances resolve_entities=False does work. I expected that the tree would contain the parsed (unexpanded) values, but referencing the actual tree.text/tail/attrib doesn't give the expected results. There's no criticism here, it makes my life a bit easier. If I had wanted the unexpanded values in the attrib/text/tail it would be more of a problem. `&#` is not an entity reference but a character reference. It may rightfully be treated differently from entity references. I understand the difference, but lxml (and perhaps libxml2) doesn't provide a way to turn off character reference expansion. This makes using lxml for source transformation a bit harder since the original text is not preserved. -- https://mail.python.org/mailman/listinfo/python-list
Re: preserving entities with lxml
On 13/01/2022 09:29, Dieter Maurer wrote: Robin Becker wrote at 2022-1-13 09:13 +: On 12/01/2022 20:49, Dieter Maurer wrote: ... Apparently, the `resolve_entities=False` was not effective: otherwise, your tree content should have more structure (especially some entity reference children). except that the tree knows not to expand the entities using ET.tostring so in some circumstances resolve_entities=False does work. I think this is a misunderstanding: `tostring` will represent the text character `&` as `&`. aaa, thanks I see now. So tostring is actually restoring some of the entities which on input are normally expanded. If that means resolve_entities=False does not work at all then I guess there's no need to use it at all. The initial transform & --> & does what I need as it is reversed on output of the tree fragments. Wonder what resolve_entities is actually used for then? All the docs seem to say resolve_entities - replace entities by their text value (default: True) I assumed False would mean that they would pass through the parse -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
lxml empty versus self closed tag
I'm using lxml.etree.XMLParser and would like to distinguish from I seem to have e.getchildren()==[] and e.text==None for both cases. Is there a way to get the first to have e.text=='' -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: lxml empty versus self closed tag
On 02/03/2022 18:39, Dieter Maurer wrote: Robin Becker wrote at 2022-3-2 15:32 +: I'm using lxml.etree.XMLParser and would like to distinguish from I seem to have e.getchildren()==[] and e.text==None for both cases. Is there a way to get the first to have e.text=='' I do not think so (at least not without a DTD): I have a DTD which has so I guess the empty case is allowed as well as the self closed. I am converting from an older parser which has text=='' for and text==None for the self closed version. I don't think I really need to make the distinction. However, I wonder how lxml can present an empty string content deliberately or if that always has to be a semantic decision. ` ag/>' is just a shorthand notation for '' and the difference has no influence on the DOM. Note that `lxml` is just a Python binding for `libxml2`. All the parsing is done by this library. yes I think I knew that -- https://mail.python.org/mailman/listinfo/python-list
strange problem building non-pure wheel for apple M1 arm64
I use cibuildwheel to build extensions with a github action. For the macos 11.0 arm64 build I get a strange message from the load command. So I am looking for assistance to try and figure out what is going wrong. The cibuild action uses the latest pip 21.2.4 and latest setuptools etc. I use brew to install freetype version 2.11.1. The compilations look like this gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -g -arch arm64 -DRENDERPM_FT -DLIBART_COMPILATION -DLIBART_VERSION=2.3.21 -Isrc/rl_addons/renderPM -Isrc/rl_addons/renderPM/libart_lgpl -Isrc/rl_addons/renderPM/gt1 -I/usr/local/include/freetype2 -I/Library/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c src/rl_addons/renderPM/_renderPM.c -o build/temp.macosx-11.0-arm64-3.9/src/rl_addons/renderPM/_renderPM.o this is the load command on multiple lines for readability the strange error is gcc -bundle -undefined dynamic_lookup -g -arch arm64 build/temp.macosx-11.0-arm64-3.9/src/rl_addons/renderPM/_renderPM.o build/temp.macosx-11.0-arm64-3.9/src/rl_addons/renderPM/gt1/gt1-dict.o build/temp.macosx-11.0-arm64-3.9/src/rl_addons/renderPM/gt1/gt1-namecontext.o '''''''other compiled code build/temp.macosx-11.0-arm64-3.9/src/rl_addons/renderPM/libart_lgpl/art_vpath_dash.o -L/usr/local/lib -L/usr/lib -L/Library/Frameworks/Python.framework/Versions/3.9/lib -lfreetype -o build/lib.macosx-11.0-arm64-3.9/reportlab/graphics/_renderPM.cpython-39-darwin.so ld: warning: ignoring file /usr/local/lib/libfreetype.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64 The above message seems bizarre; everything is compiled for arm64, but gcc doesn't want to use an arm64 dylib. Can macos experts assist? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: strange problem building non-pure wheel for apple M1 arm64
On 08/03/2022 16:08, Christian Gollwitzer wrote: Am 07.03.22 um 17:22 schrieb Robin Becker: I use brew to install freetype version 2.11.1. I find via google that homebrew/apple have split the installation of intel and arm64 into /usr/local and /opt/homebrew so I must modify the include_dirs & library_dirs in setup.py -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Reportlab / platypus bug?
On 14/03/2022 18:17, Les wrote: Unfortunately, the reportlab-users mailing list is unavailable (I cannot subscribe). There is paid support but since I already have a workaround, I won't pay for this. I think this is a documentation error of the reportlab package. (They do not mention that stories cannot be reused.) I think we can say that my original problem is solved, because I have a workaround that always works. Schachner, Joseph ezt írta (időpont: 2022. márc. 14., H 19:09): Hi Les, so far as I know the reportlab-users list is still running it is hosted (nad has been for many years) at https://pairlist2.pair.net/mailman/listinfo/reportlab-users is that the address you used? I see messages in the archives so some people can use it.-- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Reportlab / platypus bug?
.. Hi Les, so far as I know the reportlab-users list is still running it is hosted (nad has been for many years) at https://pairlist2.pair.net/mailman/listinfo/reportlab-users is that the address you used? I see messages in the archives so some people can use it.-- Robin Becker as a test I subscribed under my private email address and the list responded pretty quickly; the request confirmation email ended up in spam though. I believe the list is a fairly old version of mailman, but I don't have any access to the server. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Reportlab / platypus bug?
On 15/03/2022 13:20, Les wrote: Robin Becker ezt írta (időpont: 2022. márc. 15., K, 14:06): Hi Les, so far as I know the reportlab-users list is still running it is hosted (nad has been for many years) at https://pairlist2.pair.net/mailman/listinfo/reportlab-users is that the address you used? I see messages in the archives so some people can use it.-- Yes, it is. I tried to subscribe (twice), but never got the confirmation email. Checked in the spam folder too, but it is nowhere to be found. Hi again, Les. I just looked at the list using the admin login and find you are already a member nagy...@gmail.com Laszlo Nagy in fact you are number 1 in the N's. Perhaps you could just try sending a test email. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: How to split value where is comma ?
On 08/09/2016 03:54, D'Arcy J.M. Cain wrote: On Wed, 7 Sep 2016 16:04:40 + Joaquin Alzola wrote: .. This email is confidential and may be subject to privilege. If you are not the intended recipient, please do not copy or disclose its content but contact the sender immediately upon receipt. Do you realize how stupid it is to put this on a message sent all around the world? I have worked places where they put stuff like this at the bottom of emails sent to the person sitting next to them :) -raising entropy-ly yrs- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: [Python-ideas] Inconsistencies
On 11/09/2016 21:30, Sven R. Kunze wrote: On 10.09.2016 15:00, Chris Angelico wrote: Some things are absolute hard facts. There is no way in which 1 will ever be greater than 2, ergo "1 is less than 2" is strictly true, and not a matter of opinion. If you hear someone trying to claim otherwise, would you let him have his opinion, or would you treat it as incorrect? I don't know exactly if it's clear that one would need to make a distinction between real/physical-world facts and pure-logic facts. "1 < 2" is by definition "true" (construction of natural numbers) not by real-world evidence. IIRC, the quote is about real-world matters. as I understand it, the universe is said by some to be a long lived random fluctuation of nothing and it is said that observers make a real difference to reality. The existence of arithmetic etc is somewhat moot under such assumptions. Also presumably there are other constructions of 'our' familiar arithmetic. Perhaps someone could probably make an arithmetic where most of standard ZF is true except for 1<2. Gödel definitely says there are holes in arithmetic :) -possibly non-existently yrs- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: [RELEASE] Python 3.6.0b2 is now available
On 11/10/2016 02:12, Ned Deily wrote: On behalf of the Python development community and the Python 3.6 release team, I'm happy to announce the availability of Python 3.6.0b2. 3.6.0b2 is the second of four planned beta releases of Python 3.6, the next major release of Python, and marks the end of the feature development phase for 3.6. ... thanks -- Ned Deily n...@python.org -- [] I notice an extra space at the windows command prompt compared with 3.5, is that deliberate? C:\ux\XB33>\python35\python Python 3.5.0 (v3.5.0:374f501f4567, Sep 13 2015, 02:27:37) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. import sys C:\ux\XB33\repos\pyRXP>..\..\py36_amd64\Scripts\python.exe Python 3.6.0b2 (default, Oct 10 2016, 21:15:32) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. import sys -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
htmlparser charrefs
For various reasons I am using HTMLParser to do transcription of some xml. I need to keep charrefs as is so for Python > 3.4 I pass in convert_charrefs =False to the constructor. This seems to work as intended for data, but I notice that a change in Python 3.4 prevents me from keeping the charrefs which are in attribute strings. Is it intentional that we can no longer use HTMLParser.unescape? It seems to prevent correct interpretation of the convert_charrefs constructor argument. The unescaping is done, but in a module level function which means I can no longer override that functionality safely. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: [RELEASE] Python 3.6.0 is released!
On 23/12/2016 10:34, Ned Deily wrote: On behalf of the Python development community and the Python 3.6 release team, I am pleased to announce the availability of Python 3.6.0. Python 3.6.0 is the newest major release of the Python language, and it contains many new features and optimizations. See the "What’s New In Python 3.6" document for more information: https://docs.python.org/3.6/whatsnew/3.6.html You can download Python 3.6.0 here: https://www.python.org/downloads/release/python-360/ Also, most third-party distributors of Python should be making 3.6.0 packages available soon. Maintenance releases for the 3.6 series will follow at regular intervals starting in the first quarter of 2017. We hope you enjoy Python 3.6.0! ... Thanks for this release; makes my Christmas just a bit more stressed :( or perhaps happier :) Anyhow I am trying to figure out this error when building windows versions C:\ux\XB33\py36_x86\lib\site-packages\wheel\pep425tags.py:77: RuntimeWarning: Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect warn=(impl == 'cp')): C:\ux\XB33\py36_x86\lib\site-packages\wheel\pep425tags.py:81: RuntimeWarning: Config variable 'WITH_PYMALLOC' is unset, Python ABI tag may be incorrect warn=(impl == 'cp')): I guess this must mean I need to set something somewhere, but what? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Enumerating all 3-tuples
It's possible to generalize the cantor pairing function to triples, but that may not give you what you want. Effectively you can generate an arbitrary number of triples using an iterative method. My sample code looked like this import math def cantor_pair(k1,k2): return (((k1+k2)*(k1+k2+1))>>1) + k2 def inverse_cantor_pair(z): w = int((math.sqrt(8*z+1)-1)/2.0) t = (w*(w+1))>>1 j = z - t i = w - j return i, j def cantor_triple(i,j,k): return cantor_pair(cantor_pair(i,j),k) def inverse_cantor_triple(z): j,k = inverse_cantor_pair(z) i,j = inverse_cantor_pair(j) return i,j,k if __name__=='__main__': for z in xrange(100): i,j,k = inverse_cantor_triple(z) print z, repr((i,j,k)) or changing the construction import math def cantor_pair(k1,k2): return (((k1+k2)*(k1+k2+1))>>1) + k2 def inverse_cantor_pair(z): w = int((math.sqrt(8*z+1)-1)/2.0) t = (w*(w+1))>>1 j = z - t i = w - j return i, j def cantor_triple(i,j,k): return cantor_pair(i,cantor_pair(j,k)) def inverse_cantor_triple(z): i,k = inverse_cantor_pair(z) j,k = inverse_cantor_pair(k) return i,j,k if __name__=='__main__': for z in xrange(100): i,j,k = inverse_cantor_triple(z) print z, repr((i,j,k)), cantor_triple(i,j,k)==z this give different outcomes, but both appear to be a correct mapping of non-negative integers to triplets. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Enumerating all 3-tuples
On 12/03/2018 13:17, Robin Becker wrote: It's possible to generalize the cantor pairing function to triples, but that may not give you what you want. Effectively you can generate an arbitrary number of triples using an iterative method. My sample code looked like this ct mapping of non-negative integers to triplets. An alternative approach gives more orderly sequences using a variable base number construction class Tupilator(object): def __init__(self,degree=3): self.i = 0 self.n = 2 self.degree = degree def next(self): x = self.i v = [] a =v.append n = self.n if not x: a(0) while x>0: x, d = divmod(x,n) a(d) if sum(v)==self.degree*(n-1): self.n += 1 pad = self.degree - len(v) if pad>0: v += pad*[0] self.i += 1 return tuple(v) if __name__=='__main__': t = Tupilator() for z in xrange(100): print z, t.next() 0 (0, 0, 0) 1 (1, 0, 0) 2 (0, 1, 0) 3 (1, 1, 0) 4 (0, 0, 1) 5 (1, 0, 1) 6 (0, 1, 1) 7 (1, 1, 1) 8 (2, 2, 0) 9 (0, 0, 1) 10 (1, 0, 1) 11 (2, 0, 1) 12 (0, 1, 1) 13 (1, 1, 1) 14 (2, 1, 1) 15 (0, 2, 1) 16 (1, 2, 1) 17 (2, 2, 1) 18 (0, 0, 2) 19 (1, 0, 2) 20 (2, 0, 2) 21 (0, 1, 2) 22 (1, 1, 2) 23 (2, 1, 2) 24 (0, 2, 2) 25 (1, 2, 2) 26 (2, 2, 2) 27 (3, 2, 1) 28 (0, 3, 1) 29 (1, 3, 1) 30 (2, 3, 1) 31 (3, 3, 1) 32 (0, 0, 2) 33 (1, 0, 2) 34 (2, 0, 2) 35 (3, 0, 2) 36 (0, 1, 2) 37 (1, 1, 2) 38 (2, 1, 2) 39 (3, 1, 2) 40 (0, 2, 2) 80 (0, 1, 3) 81 (1, 1, 3) 82 (2, 1, 3) 83 (3, 1, 3) 84 (4, 1, 3) 85 (0, 2, 3) 86 (1, 2, 3) 87 (2, 2, 3) 88 (3, 2, 3) 89 (4, 2, 3) 90 (0, 3, 3) 91 (1, 3, 3) 92 (2, 3, 3) 93 (3, 3, 3) 94 (4, 3, 3) 95 (0, 4, 3) 96 (1, 4, 3) 97 (2, 4, 3) 98 (3, 4, 3) 99 (4, 4, 3) -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Enumerating all 3-tuples
On 12/03/2018 18:05, Chris Angelico wrote: On Tue, Mar 13, 2018 at 2:54 AM, Robin Becker wrote: On 12/03/2018 13:17, Robin Becker wrote: An alternative approach gives more orderly sequences using a variable base number construction 4 (0, 0, 1) 9 (0, 0, 1) 18 (0, 0, 2) 32 (0, 0, 2) I spy duplicates. ChrisA oh well there goes that idea :( -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Enumerating all 3-tuples
On 13/03/2018 11:14, Steven D'Aprano wrote: On Mon, 12 Mar 2018 13:17:15 +0000, Robin Becker wrote: It's possible to generalize the cantor pairing function to triples, but that may not give you what you want. Effectively you can generate an arbitrary number of triples using an iterative method. My sample code looked like this import math def cantor_pair(k1,k2): return (((k1+k2)*(k1+k2+1))>>1) + k2 def inverse_cantor_pair(z): w = int((math.sqrt(8*z+1)-1)/2.0) t = (w*(w+1))>>1 j = z - t i = w - j return i, j I definitely want to avoid any round trips through float in order to use sqrt. But thanks for the code, I'll certainly steal it, er I mean study it for future reference :-) well I guess Cantor didn't worry about rounding errors :) For high z there's an integer square root function which seems to work pretty well here http://code.activestate.com/recipes/577821-integer-square-root-function/ I'm not sure if there are any other sensible invertible pairing functions on non-negative integers; this page mentions a couple implemented in matlab https://uk.mathworks.com/matlabcentral/fileexchange/44253-three-different-bijections-or-pairing-functions-between-n-and-n%5E2--including-cantor-polynomials- and this describes the elegant pairing more http://szudzik.com/ElegantPairing.pdf It seems reasonable that a mapping N x N --> N should require a square root in the inverse. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
curious asymmetry in range limiting
I don't know how I never came across this before, but there's a curious asymmetry in the way ranges are limited Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> s = '0123456789' >>> print(repr(s[-5:5])) '' >>> print(repr(s[5:15])) '56789' >>> why is the underflow start index treated so differently from the limit index overflow? I suppose there must be some reason, but it eludes me. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: curious asymmetry in range limiting
So eventually I tried this C:\code\hg-repos\reportlab>\python36\python -c"s='0123456789';print(repr(s[-5:15]))" '56789' C:\code\hg-repos\reportlab>\python36\python -c"s='0123456789';print(repr(s[-6:15]))" '456789' and the light dawned :) seems the negative indexes rules apply to both On 20/03/2018 14:21, Robin Becker wrote: I don't know how I never came across this before, but there's a curious asymmetry in the way ranges are limited Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> s = '0123456789' >>> print(repr(s[-5:5])) '' >>> print(repr(s[5:15])) '56789' >>> why is the underflow start index treated so differently from the limit index overflow? I suppose there must be some reason, but it eludes me. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Enumerating all 3-tuples
On 21/03/2018 05:14, Steven D'Aprano wrote:
On Tue, 13 Mar 2018 23:56:37 +0100, Denis Kasak wrote:
[...]
The triples can be viewed as a pair of a pair and a natural number:
(1,1),1 (1,1),2 (1,1),3 ...
(2,1),1 (2,1),2 (2,1),3 ...
(1,2),1 (1,2),2 (1,2),3 ...
[...]
This leads fairly naturally to the implementation.
from itertools import accumulate, count
def c(i):
"""
...
That looks interesting, I haven't had a chance to play with it in a lot
of detail yet, but it looks to me like every time you generate a triple,
it starts counting up from 1.
So iterating over c3(1), c3(2), c3(4), ... c3(something big) is going to
have O(N**2) performance. It's like the old joke about the Polish painter:
This cantor inverse is much faster; it uses the integer sqrt from here
http://code.activestate.com/recipes/577821-integer-square-root-function/
so far as I can tell it works for the cantor below
def cantor(k1,k2):
return (((k1+k2)*(k1+k2+1))>>1) + k2
def isqrt(x):
if x < 0:
raise ValueError('square root not defined for negative numbers')
n = int(x)
if n == 0:
return 0
a, b = divmod(n.bit_length(), 2)
x = 2**(a+b)
while True:
y = (x + n//x)//2
if y >= x:
return x
x = y
def inverse_cantor(z):
w = int((isqrt(8*z+1)-1)//2)
t = (w*(w+1))>>1
j = z - t
i = w - j
return i, j
but it doesn't agree with Denis' inverse I guess because this is defined on non-negative integers, but his functions are defined
on positive integers.
http://global.joelonsoftware.com/English/Articles/BacktoBasics.html
Since that's exactly what I need to do, that might be a problem.
On the other hand, I doubt I'll need to generate more than a few thousand
of these, so it might be fast enough. I guess I have to run some
benchmarks to find out.
But however they turn out, I appreciate your time and code. Thanks heaps!
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: Multi-threading with a simple timer?
On 03/07/2018 07:12, Gregory Ewing wrote:
import signal, sys
def timeout(*args):
print("Too late!")
sys.exit(0)
signal.signal(signal.SIGALRM, timeout)
signal.setitimer(signal.ITIMER_REAL, 15)
data = input("Enter something: ")
print("You entered: ", data)
This doesn't work in windows (SIGALRM not available see
https://stackoverflow.com/questions/6947065/right-way-to-run-some-code-with-timeout-in-python)
For completeness I think it needs the handler restoring; this seemed to work for me in linux; I did try using SIG_DFL, but
apparently that just prints 'Alarm clock' and exits at least
import signal, sys, time
def timeout(*args):
print("Too late!")
sys.exit(0)
signal.signal(signal.SIGALRM, timeout)
signal.setitimer(signal.ITIMER_REAL, 15)
data = input("Enter something: ")
signal.signal(signal.SIGALRM,signal.SIG_IGN)
print("You entered: ", data)
for i in reversed(xrange(15)):
print i
time.sleep(1)
print 'finished!'
if I leave out the signal.signal(signal.SIGALRM,signal.SIG_IGN) then the
timeout function gets called anyway.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Python 3.7 configuration issue
On a Ubuntu trusty system I ran ./configure --prefix=/home/rptlab/PYTHON make && make install and get an error related to the ctypes module not being importable. I needed to do sudo apt-get install libffi-dev ./configure --prefix=/home/rptlab/PYTHON --with-system-ffi make && make install Something in setup.py seems to want ctypes irrespective of what I give to configure; I don't actually know what the alternative to --with-system-ffi does, but it didn't work for me. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.7 configuration issue
On 04/07/2018 05:59, dieter wrote: .. "libffi" contains the so called "foreign function interface" (an interface that allows code written in one language to call code written in another language). Likely, "ctypes" depends on it. Since some time, "ctypes" is part of Python's standard library. It is likely, that the Python test suite contains tests verifying that it could be build successfully. Your observations seem to indicate that "ctypes" contains its own "foreign function interface" specification and that this specification does not fit to the libraries available on your system. The "with-system-ffi" likely tells the Python generation process to use the system "foreign function interface" rather than its own. I don't disagree with the above. I think the issue is that the configure process did not say anything about this. If ctypes is required and cannot be built then ./configure .. should probably tell me; if that's not possible then the make step should do so and fail to complete. Failing at make install seems a bit late. I have to admit that I no longer understand how python builds its extensions; there used to be a step where you configured Modules/Setup*, but that seems no longer required. Any how as an experiment it seems that if I do have the libffi-dev package installed then the with-system-ffi flag is not required and the make install does work. It might be that make silently allows extension builds to fail even if they are 'required' and that later causes an error in the install phase. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: about main()
On 05/07/2018 21:43, Jim Lee wrote: ... identifying the poisonous berries. I would respect your analogy more if every compiler used today were forty years old and not being developed by anyone other than its original creator(s). ChrisA It's not about compilers - it's about skills. As programming becomes more and more specialized, it becomes harder and harder to find programmers with a skill set broad enough to be adaptable to a different task. -Jim I suspect compiler writing is a task that an AI could be taught. The villagers will shout "hey siri I need a compiler" and one will be provided, of course by that time they will all have forgotten about eating berries in favour of soma or equivalent. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
test for absence of infinite loop
A user reported an infinite loop in reportlab. I determined a possible cause and fix and would like to test for absence of the loop. Is there any way to check for presence/absence of an infinite loop in python? I imagine we could do something like call an external process and see if it takes too long, but that seems a bit flaky. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: test for absence of infinite loop
On 17/07/2018 10:32, Chris Angelico wrote: .. All you gotta do is solve the halting problem... https://en.wikipedia.org/wiki/Halting_problem ChrisA ah so it's easy :) -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: test for absence of infinite loop
On 17/07/2018 12:16, Cameron Simpson wrote: On 17Jul2018 10:10, Robin Becker wrote: A user reported an infinite loop in reportlab. I determined a possible cause and fix and would like to test for absence of the loop. Is there any way to check for presence/absence of an infinite loop in python? I imagine we could do something like call an external process and see if it takes too long, but that seems a bit flaky. While others have kindly pointed you at the halting problem (unsolvable in the general case) you can usually verify that the specific problem you fixed is fixed. Can you figure out how long the task should run with your fix in a test case? Not as time, but in loop iterations? Put a counter in the loop and check that its value doesn't exceed that. well I understand the problem about not halting. However as you point out in a fixed case I know that the test should take fractions of a second to complete. I certainly don't want to put instrumentation into the source code. It's relatively easy to imagine polling termination of a separate thread/process, but that's not particularly reliable. I don't know if there is a way to ask a python interpeter how many instructions it has carried out. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Are dicts supposed to raise comparison errors
A bitbucket user complains that python 3.6.6 with -Wall -b prints warnings for
some reportlab code; the
example boils down to the following
##
C:\code\hg-repos\reportlab\tmp>cat tb.py
if __name__=='__main__':
d={'a':1}
d[b'a'] = d['a']
##
C:\code\hg-repos\reportlab\tmp>\python36\python -Wall -b tb.py
tb.py:3: BytesWarning: Comparison between bytes and string
d[b'a'] = d['a']
I had always assumed that dicts didn't care about the type of keys although some types might cause issue with hashability, but
obviously the implementation seems to be comparing b'a' with 'a' (I suppose because they hash to the same chain).
Is this code erroneous or is the warning spurious or wrong?
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 31/07/2018 09:16, Paul Moore wrote:
On 31 July 2018 at 08:40, Robin Becker wrote:
A bitbucket user complains that python 3.6.6 with -Wall -b prints warnings
for some reportlab code; the
example boils down to the following
##
C:\code\hg-repos\reportlab\tmp>cat tb.py
if __name__=='__main__':
d={'a':1}
d[b'a'] = d['a']
##
..
v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
b'a' == 'a'
True
b'a' == u'a'
True
which is basically the sort of thing that -b should warn about.
Specifically the quoted code would end up with a dictionary with 2
entries on Python 3, but 1 entry on Python 2.
Paul
yes but I didn't do the compare so this warning seems entirely spurious and wrong. It's not an error to put 1 and 1.0 and 'a' into
a dict. Should I get a warning if the hashes of two different types happen to clash so that an int needs to be checked against a
string?
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 31/07/2018 16:52, Chris Angelico wrote: On Wed, Aug 1, 2018 at 1:28 AM, MRAB wrote: On 2018-07-31 08:40, Robin Becker wrote: A bitbucket user complains that python 3.6.6 with -Wall -b prints warnings . The warning looks wrong to be. In Python 2, u'a' and b'a' would be treated as the same key, but in Python 3 they are distinct and can co-exist. Something for Python's bug tracker, I think! It's a warning specifically requested by the -b option. The two keys in question have the same hash, which means they have to be compared directly; they will compare unequal, but because of the -b flag, the comparison triggers a warning. If that warning is spurious, *don't use the -b option*. ChrisA I looked at the language documentation for 3.7 mappings These represent finite sets of objects indexed by nearly arbitrary values. The only types of values not acceptable as keys are values containing lists or dictionaries or other mutable types that are compared by value rather than by object identity, the reason being that the efficient implementation of dictionaries requires a key’s hash value to remain constant. Numeric types used for keys obey the normal rules for numeric comparison: if two numbers compare equal (e.g., 1 and 1.0) then they can be used interchangeably to index the same dictionary entry. it says explicitly that numeric keys will use numeric comparison, but no mention is made of strings/bytes etc etc and there's an implication that object identity is used rather than comparison. In python 3.x b'a' is not the same as 'a' either the documentation is lacking some reference to strings/bytes or the warning is wrong. Using the excuse that normal comparison is being used seems a bit feeble. It would clearly improve speed if object identity were to be used, but I suppose that's not the case for other reasons. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 01/08/2018 09:52, Chris Angelico wrote:
On Wed, Aug 1, 2018 at 6:36 PM, Robin Becker wrote:
On 31/07/2018 16:52, Chris Angelico wrote:
..
it says explicitly that numeric keys will use numeric comparison, but no
.
Technically, the comparison used is:
a is b or a == b
in other words, identity will match, but mainly, equality is used. The
identity check improves performance in many common cases, and also
avoids pathological cases involving float("nan"), but in general
discussion, it's assumed that value rather than identity is the
comparison used.
ChrisA
If the testing were done in that order then no comparison warning would occur
as inspection shows 'a' is not b'a'
> C:\code\hg-repos\reportlab\tmp>\python37\python.exe -b -Wall
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
'a' is b'a'
False
so presumably the testing goes something like
a is b or (a==b if comparable(a,b) else False)
even so I still think errors/warnings created internally by implementers should not be exposed. It's not a big deal. I'm a bit
surprised that we don't have a mapping which uses only identity as that would be faster.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
.. Nope, that would be the effect of "and", not "or". "a" is b"b" and "fallback" False "a" is b"b" or "fallback" 'fallback' You seem to be caught in your wrong mental model. I recommend that you let the matter rest for a day or so, and then look at it with a fresh eye. .. my bad; not worrying enough about real problems -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
messing with bytes I discover that this doesn't warn with python -b
if __name__=='__main__':
class nbytes(bytes):
def __eq__(self,other):
return bytes.__eq__(self,other) if isinstance(other,bytes) else
False
def __hash__(self):
return bytes.__hash__(self)
d={'a':1}
d[nbytes(b'a')] = d['a']
print(d)
d={nbytes(b'a'):1}
d['a'] = d[b'a']
print(d)
C:\code\hg-repos\reportlab\tmp>\python37\python -b tb1.py
{'a': 1, b'a': 1}
{b'a': 1, 'a': 1}
I expected one of the assignments to warn.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 01/08/2018 14:38, Chris Angelico wrote: t's a warning designed to help people port code from Py2 to Py3. It's not meant to catch every possible comparison. Unless you are actually porting Py2 code and are worried that you'll be accidentally comparing bytes and text, just*don't use the -b switch* and there will be no problems. I don't understand what the issue is here. I don't either, I have never used the -b flag until the issue was raised on bitbucket. If someone is testing a program with reportlab and uses that flag then they get a lot of warnings from this dictionary assignment. Probably the code needs tightening so that we insist on using native strings everywhere; that's quite hard for py2/3 compatible code. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 01/08/2018 18:19, Peter Otten wrote: I've looked into the actual code which has # paraparser.py f = isPy3 and asBytes or asUnicode K = list(known_entities.keys()) for k in K: known_entities[f(k)] = known_entities[k] It looks like known_entities starts out with the default string type, i. e. unicode in py3 and bytes in py2. While in py2 the code has no effect in py3 it adds the corresponding keys of type bytes. However, known_entities is then used in HTMLParser.handle_entity_ref(self, name) which passes the name as unicode in py3. > I didn't try, but I would suspect that the module keeps working as expected when you remove the lines quoted above. I did try and all the tests pass in 2.7.x & >=3.3; the commit message says "fix entityref handling" and happened during the python 3.x porting. I suppose there was some issue, but its entirely probable that some later change (eg parser) has fixed the original problem and made this code redundant. If I'm correct running the program with the -b flag has at least helped in cleaning up the code in this case. and I will try running all tests under that flag; it's sure to find more issues. In other cases it might detect sources of bugs, so IMHO it's better to have a close look at and possibly rewrite code that triggers the warning rather than to disable it. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
