Re: [Python-Dev] bitfields - short - and xlc compiler
On 3/20/2016 4:04 PM, Andrew Barnert via Python-Dev wrote: Agreed. But I think the test is reasonable on at least MSVC, gcc, clang, and icc. So what you need is some way to run the test on those compilers, but not on compilers that can't handle it. The test could be conditioned on the compiler used. >>> platform.python_compiler() 'MSC v.1900 64 bit (AMD64)' The doc could say something about the feature only being available with certain compilers. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bitfields - short - and xlc compiler
On Mar 20, 2016, at 09:07, Michael Felt wrote: > >> On 2016-03-18 05:57, Andrew Barnert via Python-Dev wrote: >> Yeah, C99 (6.7.2.1) allows "a qualified or unqualified version of _Bool, >> signed int, unsigned int, or some other implementation-defined type", and >> same for C11. This means that a compiler could easily allow an >> implementation-defined type that's identical to and interconvertible with >> short, say "i16", to be used in bitfields, but not short itself. >> >> And yet, gcc still allows short "even in strictly conforming mode" (4.9), >> and it looks like Clang and Intel do the same. >> >> Meanwhile, MSVC specifically says it's illegal ("The type-specifier for the >> declarator must be unsigned int, signed int, or int") but then defines the >> semantics (you can't have a 17-bit short, bit fields act as the underlying >> type when accessed, alignment is forced to a boundary appropriate for the >> underlying type). They do mention that allowing char and long types is a >> Microsoft extension, but still nothing about short, even though it's used in >> most of the examples on the page. >> >> Anyway, is the question what ctypes should do? If a platform's compiler >> allows "short M: 1", especially if it has potentially different alignment >> than "int M: 1", ctypes on that platform had better make ("M", c_short, 1) >> match the former, right? >> >> So it sounds like you need some configure switch to test that your compiler >> doesn't allow short bit fields, so your ctypes build at least skips that >> part of _ctypes_test.c and test_bitfields.py, and maybe even doesn't allow >> them in Python code. >> >> >> test_short fails om AIX when using xlC in any case. How terrible is >> this? > a) this does not look solveable using xlC, and I expect from the comment > above re: MSVC, that it will, or should also fail there. > And, imho, if anything is to done, it is a decision to be made by "Python". Sure, but isn't that exactly why you're posting to this list? > b) aka - it sounds like a defect, at least in the test. Agreed. But I think the test is reasonable on at least MSVC, gcc, clang, and icc. So what you need is some way to run the test on those compilers, but not on compilers that can't handle it. So it sounds like you need a flag coming from autoconf that can be tested in C (and probably in Python as well) that tells you whether the compiler can handle it. And I don't think there is any such flag. Which means someone would have to add the configure test. And if people who use MSVC, gcc, and clang are all unaffected, I'm guessing that someone would have to be someone who cares about xlC or some other compiler, like you. The alternative would be to just change the docs to make it explicit that using non-int bitfields isn't supported but may work in platform-specific ways. If you got everyone to agree to that, surely you could just remove the tests, right? But if people are actually writing C code that follows the examples on the MSVC bitfield docs page, and need to talk to that code from ctypes, I don't know if it would be acceptable to stop officially supporting that. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Counting references to Py_None
[Facundo Batista ] > I'm seeing that our code increases the reference counting to Py_None, > and I find this a little strange: isn't Py_None eternal and will never > die? Yes, but it's immortal in CPython because its reference count never falls to 0 (it's created with a reference count of 1 to begin with). That's the only thing that makes it immortal. > What's the point of counting its references? Uniformity and simplicity: code using a PyObject* increments and decrements reference counts appropriately with no concern for what _kind_ of object is being pointed at. All objects (including None) are treated exactly the same way for refcount purposes. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Counting references to Py_None
On Sun, 20 Mar 2016 at 09:44 Facundo Batista wrote: > Hello! > > I'm seeing that our code increases the reference counting to Py_None, > and I find this a little strange: isn't Py_None eternal and will never > die? > Semantically yes, but we have to technically make that happen. :) > > What's the point of counting its references? > It's just another Python object, so if you return it then the code that receives it may very well decref it because it always DECREFs the object returned. And if we didn't keep its count accurately it would eventually hit zero and constantly have its dealloc function checked for. We could add a magical "never dies" value for the refcount, but that adds another `if` branch in all the code instead of simply treating Py_None like any other object and properly tracking its reference. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Counting references to Py_None
On Sun, Mar 20, 2016 at 01:43:27PM -0300, Facundo Batista wrote: > Hello! > > I'm seeing that our code increases the reference counting to Py_None, > and I find this a little strange: isn't Py_None eternal and will never > die? > > What's the point of counting its references? Avoiding a branch on every single Py_DECREF / Py_INCREF? > > Thanks! > > -- > .Facundo > > Blog: http://www.taniquetil.com.ar/plog/ > PyAr: http://www.python.org/ar/ > Twitter: @facundobatista > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/dw%2Bpython-dev%40hmmz.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Counting references to Py_None
Hello! I'm seeing that our code increases the reference counting to Py_None, and I find this a little strange: isn't Py_None eternal and will never die? What's the point of counting its references? Thanks! -- .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ Twitter: @facundobatista ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bitfields - short - and xlc compiler
On 2016-03-18 05:57, Andrew Barnert via Python-Dev wrote: Yeah, C99 (6.7.2.1) allows "a qualified or unqualified version of _Bool, signed int, unsigned int, or some other implementation-defined type", and same for C11. This means that a compiler could easily allow an implementation-defined type that's identical to and interconvertible with short, say "i16", to be used in bitfields, but not short itself. And yet, gcc still allows short "even in strictly conforming mode" (4.9), and it looks like Clang and Intel do the same. Meanwhile, MSVC specifically says it's illegal ("The type-specifier for the declarator must be unsigned int, signed int, or int") but then defines the semantics (you can't have a 17-bit short, bit fields act as the underlying type when accessed, alignment is forced to a boundary appropriate for the underlying type). They do mention that allowing char and long types is a Microsoft extension, but still nothing about short, even though it's used in most of the examples on the page. Anyway, is the question what ctypes should do? If a platform's compiler allows "short M: 1", especially if it has potentially different alignment than "int M: 1", ctypes on that platform had better make ("M", c_short, 1) match the former, right? So it sounds like you need some configure switch to test that your compiler doesn't allow short bit fields, so your ctypes build at least skips that part of _ctypes_test.c and test_bitfields.py, and maybe even doesn't allow them in Python code. >> test_short fails om AIX when using xlC in any case. How terrible is this? a) this does not look solveable using xlC, and I expect from the comment above re: MSVC, that it will, or should also fail there. And, imho, if anything is to done, it is a decision to be made by "Python". b) aka - it sounds like a defect, at least in the test. c) what danger is there to existing Python code if "short" is expected, per legacy when compilers did (and GCC still does - verified that when I compile with gcc the test does not signal failure) So, more with regard to c) - is there something I could/should be looking at in Python itself, in order to message that the code is not supported by the compiler? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] What does a double coding cookie mean?
On Thu, Mar 17, 2016 at 9:50 AM, Serhiy Storchaka wrote: > On 17.03.16 16:55, Guido van Rossum wrote: >> >> On Thu, Mar 17, 2016 at 5:04 AM, Serhiy Storchaka >> wrote: Should we recommend that everyone use tokenize.detect_encoding()? >>> >>> >>> Likely. However the interface of tokenize.detect_encoding() is not very >>> simple. >> >> >> I just found that out yesterday. You have to give it a readline() >> function, which is cumbersome if all you have is a (byte) string and >> you don't want to split it on lines just yet. And the readline() >> function raises SyntaxError when the encoding isn't right. I wish >> there were a lower-level helper that just took a line and told you >> what the encoding in it was, if any. Then the rest of the logic can be >> handled by the caller (including the logic of trying up to two lines). > > > The simplest way to detect encoding of bytes string: > > lines = data.splitlines() > encoding = tokenize.detect_encoding(iter(lines).__next__)[0] This will raise SyntaxError if the encoding is unknown. That needs to be caught in mypy's case and then it needs to get the line number from the exception. I tried this and it was too painful, so now I've just changed the regex that mypy uses to use non-eager matching (https://github.com/python/mypy/commit/b291998a46d580df412ed28af1ba1658446b9fe5). > If you don't want to split all data on lines, the most efficient way in > Python 3.5 is: > > encoding = tokenize.detect_encoding(io.BytesIO(data).readline)[0] > > In Python 3.5 io.BytesIO(data) has constant complexity. Ditto with the SyntaxError though. > In older versions for detecting encoding without copying data or splitting > all data on lines you should write line iterator. For example: > > def iterlines(data): > start = 0 > while True: > end = data.find(b'\n', start) + 1 > if not end: > break > yield data[start:end] > start = end > yield data[start:] > > encoding = tokenize.detect_encoding(iterlines(data).__next__)[0] > > or > > it = (m.group() for m in re.finditer(b'.*\n?', data)) > encoding = tokenize.detect_encoding(it.__next__) > > I don't know what approach is more efficient. Having my own regex was simpler. :-( -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] What does a double coding cookie mean?
On 03/17/2016 04:54 PM, Glenn Linderman wrote: On 3/16/2016 12:59 AM, Serhiy Storchaka wrote: Actually "must match the regular expression" is not correct, because re.match() implies anchoring at the start. I have proposed more correct regular expression in other branch of this thread. "match" doesn't imply anchoring at the start. "re.match()" does (and as a result is very confusing to newbies to Python re, that have used other regexp systems). It still confuses me from time to time. :( -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] What does a double coding cookie mean?
On 20 March 2016 at 07:46, Glenn Linderman wrote: > Diagnosing ambiguous conditions, even including my example above, might be > useful... for a few files... is it worth the effort? What % of .py sources > have coding specifications? What % of those have two? And there's a decent argument for leaving detecting such cases to linters rather than the tokeniser. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com