[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

STINNER Victor added the comment: New changeset 3243f8c1fb16b6de73f1d7a30f5d09047553bce3 by Victor Stinner in branch '2.7': bpo-29565: Corrected ctypes passing of large structs by value on Windows AMD64 (GH-168) (GH-8625)

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Change by STINNER Victor : -- pull_requests: +8131 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Change by Éric Araujo : -- pull_requests: -5529 ___ Python tracker ___ ___

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Change by miss-islington : -- pull_requests: +5529 ___ Python tracker ___

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Vinay Sajip added the comment: New changeset a86339b83fbd0932e0529a3c91935e997a234582 by GitHub in branch 'master': Fixed bpo-29565: Corrected ctypes passing of large structs by value on Windows AMD64. (#168) https://github.com/python/cpython/commit/a86339b83fbd0932e0529a3c91935e997a234582

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Changes by Vinay Sajip : -- pull_requests: +132 ___ Python tracker ___ ___

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Rafał Chłodnicki added the comment: In case it makes any difference, my query was prompted by this issue being noticed in Sublime Text editor. Its author bundles Python 3.3 so the bug reproduced there and it took a while to figure out what is the problem. Now that the Sublime Text issue was

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Steve Dower added the comment: It may be exploitable, but I doubt it can be exploited without providing arbitrary Python code. Anyone still running Python 3.3 on Windows must be building it from source. If they haven't noticed crashes from this issue yet, they aren't using the feature, and

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Vinay Sajip added the comment: > This bug is clearly not a security issue. I'm not sure it's all that clear - the bug could cause a crash (observed in practice - not theoretical), which perhaps could be exploited. See Steve Dower's message msg272008 in this thread. That's why I added the

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Christian Heimes added the comment: Small correction, 3.3 will get security updates until 2017. This bug is clearly not a security issue. The patch should not have landed in 3.3 in the first patch. https://www.python.org/dev/peps/pep-0398/#id7 --

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Christian Heimes added the comment: 3.3 no longer receives any updates, neither security nor feature updates. 3.3.7 was the final release of 3.3. -- nosy: +christian.heimes ___ Python tracker

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Rafał Chłodnicki added the comment: Only a part of this fix was backported to 3.3 branch (in http://bugs.python.org/issue20160#msg272059). The other important part was only backported to 3.4 and up in http://bugs.python.org/issue20160#msg230723 . Should this still be fixed? -- nosy:

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Roundup Robot added the comment: New changeset 4d33bccb59a8 by Vinay Sajip in branch '3.3': Issue #20160: Handled passing of large structs to callbacks correctly. https://hg.python.org/cpython/rev/4d33bccb59a8 New changeset 190ebf99bf45 by Vinay Sajip in branch '3.4': Issue #20160: Merged fix

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Roundup Robot added the comment: New changeset 09475e6135d0 by Vinay Sajip in branch '2.7': Issue #20160: Handled passing of large structs to callbacks correctly. https://hg.python.org/cpython/rev/09475e6135d0 -- ___ Python tracker

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Patrick Stewart added the comment: Actually the current released version of libffi (3.2.1) is even more severely broken on win64, you can't return structs at all. (patch here

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Steve Dower added the comment: Yeah, it's more a loose end than a real concern. Helps make the case for reintegrating current libffi builds, as IIRC they've had the fix for a long time, but we don't have anyone willing to do the integration work right now. --

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Larry Hastings added the comment: 3.4 is also in security-fixes-only mode, which also means it's in no-binary-installers mode. Good luck making the case that "this bugfix, which took us more than 2.5 years to finalize, is so critical that the release team must immediately issue binary

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Vinay Sajip added the comment: According to PEP 398, we should patch the source for security updates for 3.3 until September 2017, though no new binary release needs to be made. I'm not sure if expedited binary releases are needed for 3.4 and 3.5. I will look at applying the patch in 2.7 and

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Steve Dower added the comment: I'd also suggest that this is potentially exploitable (if the Python code calling into ctypes is already vulnerable - I don't think you could reasonably manufacture an exploit simply from this bug) and it should probably go into all active security branches. I

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Steve Dower added the comment: Looks good to me. Sorry for not getting back to this after I said I would. Every time I remembered I didn't have the time available... -- ___ Python tracker

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Vinay Sajip added the comment: I've uploaded a new patch which adds a test. I've confirmed that *without* the patch to Modules/_ctypes/libffi_msvc/ffi.c, the test passes on win32 and fails on amd64. *With* the patch to Modules/_ctypes/libffi_msvc/ffi.c, the test passes on both win32 and

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Changes by Vinay Sajip : -- assignee: steve.dower -> vinay.sajip ___ Python tracker ___

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Patrick Stewart added the comment: There's some confusion above about clang - that's completely irrelevant, the problem was using ctypes to call into libclang, not using clang to compile anything. The problem applies to any callback function that returns a struct larger than 8 bytes with any

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Changes by Vinay Sajip : -- versions: +Python 3.6 ___ Python tracker ___ ___

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Changes by Vinay Sajip : -- nosy: +vinay.sajip ___ Python tracker ___ ___

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Patrick Stewart added the comment: This is still a problem, and the suggested fix seems to work. There is another error a few lines above where it is allocating 4 bytes for a pointer. I've attached a new patch with both fixes, but without Bob's tests. This issue is a duplicate of 17310

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Changes by Mark Lawrence : -- nosy: -BreamoreBoy ___ Python tracker ___ ___

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Changes by Christoph Sarnowski : -- nosy: +Christoph Sarnowski ___ Python tracker ___

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

John Ehresman added the comment: I've confirmed that the test included in the 3/25/15 patch fails without the change to ffi.c. I think the 11/5/14 change fixed the bug for calling into a C function and converting the return value, but did not address the callback case. The new patch seems

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Bob added the comment: What I see is that structs lager that 8 bytes are not passed correctly to a callback funtion. I've attached a patchfile that includes my fix and a test to demonstrate the problem. -- Added file: http://bugs.python.org/file38687/large_struct_callback.patch

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Bob added the comment: I was looking into http://lists.cs.uiuc.edu/pipermail/llvmbugs/2012-September/025152.html 'Use of libclang python bindings on Windows 7 64 bit fails with memory access violation' It appears to be 90% fixed with this patch, but I believe there is still a problem when

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Mark Lawrence added the comment: This https://mail.python.org/pipermail/python-dev/2014-December/137631.html seems relevant. Follow up here https://mail.python.org/pipermail/python-dev/2015-March/138731.html -- nosy: +BreamoreBoy ___ Python

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

mattip added the comment: Bob is relating to a dead issue, the email was from 2012 and this issue was fixed in Nov 2014. His fix is wrong and unnecessary, which he would have discovered had he run the tests that were added with that patch. This issue was only left open in order to allow

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Steve Dower added the comment: It could be, though I admit I don't know ctypes or libffi that well. Should be easy enough to set up a repro for this (we'll add a function to _ctypes_test eventually that calls back and passes a struct) - I'll get to it sooner or later, but if someone else gets

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

mattip added the comment: This is the workflow for developing a patch for python https://docs.python.org/devguide This is the workflow for adding a patch to an issue https://docs.python.org/devguide/patch.html If there is an issue with python, please demonstrate it in a way that we can follow,

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Bob added the comment: ok, forgive me for asking a newbe question ot two, but when you say 'fixed in nov 2014' are you refering to the release mentioned below or something else, possibly something that didn't make it into 2.7.9? I dont really understand what 'resync libffi with the upstream

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Robert Kuska added the comment: I have created #23249. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20160 ___ ___ Python-bugs-list mailing

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

mattip added the comment: it seems the problem is that the bundled libffi version in Modules/_ctypes/libffi needs updating. The fix for aarch64 is a simple one-liner (tested on 2.7 in a aarch64 vm), for both HEAD and 2.7 (attached), but perhaps a better fix would be to update the bundled

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Robert Kuska added the comment: FYI The bug was found in libffi. I have tried and rebuilt python also with *bundled libffi* with the *same result* (=test failure). There is more info in the bugzilla mentioned in my previous post along with the libffi patch. --

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Steve Dower added the comment: You're running on Fedora, not Windows, so this is not the right issue. You should open a new one and probably post on python-dev to get some attention (since there's no useful nosy lists right now and ctypes has no maintainer). --

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Robert Kuska added the comment: This commit (probably) breaks aarch64 python build. See https://bugzilla.redhat.com/show_bug.cgi?id=1174037 Build was done with libffi 3.1.6, I have also tried with latest upstream libffi version with same result. (gdb) b ReturnRect Function ReturnRect not

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Steve Dower added the comment: This change only has an effect of you're building with Visual Studio and our fork of libffi. You seem to have an unrelated issue. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20160

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Matthias Klose added the comment: steve, please can we keep this issue open until this is forwarded and accepted upstream? -- nosy: +doko resolution: fixed - remind status: closed - open ___ Python tracker rep...@bugs.python.org

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Steve Dower added the comment: Upstream as in libffi? I'm fairly sure they've fixed it already, but their current code bears little relation to what we have, so it's not easy to tell and I didn't look all that closely. Who's doing the forwarding? --

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Roundup Robot added the comment: New changeset f75b0470168b by Steve Dower in branch '2.7': Issue #20160: broken ctypes calling convention on MSVC / 64-bit Windows (large structs). Patch by mattip https://hg.python.org/cpython/rev/f75b0470168b -- nosy: +python-dev

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Roundup Robot added the comment: New changeset cd36ba22602d by Steve Dower in branch '3.4': Issue #20160: broken ctypes calling convention on MSVC / 64-bit Windows (large structs) Patch by mattip https://hg.python.org/cpython/rev/cd36ba22602d New changeset b701eb69260d by Steve Dower in branch

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Steve Dower added the comment: Done. Thanks mattip! -- resolution: - fixed stage: - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20160 ___

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Steve Dower added the comment: Had a look at where libffi is at, and we're such a long way from them now that I don't think we can easily merge (a.k.a. I don't want to be the one to do it :) ) I want to run this patch through some builds with the right compilers, but then I'll check it in.

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Steve Dower added the comment: Patch looks good to me, but given this issue, #11835, #22733, and probably more, should we be integrating from libffi (which apparently has fixes for some of these) more often? I know nothing about how we move code between that and ctypes. -- nosy:

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

mattip added the comment: Here is a fix for arguments and return values, based on the support in cffi, for python2.7 I added a test in test_win32, removed the too-early attempt to fix in callproc.c, and merged in most of the changes in lib_msvc from cffi's code base. These changes handle the

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

mattip added the comment: and here is the promised patch for tip -- Added file: http://bugs.python.org/file35249/issue_20160_tip.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20160 ___

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Changes by Meador Inge mead...@gmail.com: -- nosy: +meador.inge ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20160 ___ ___ Python-bugs-list

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

New submission from Mark Dickinson: The argument-passing code for passing structs larger than 8 bytes is broken on 64-bit Windows, leading to potential segmentation faults or other unpredictable behaviour. According to http://msdn.microsoft.com/en-us/library/zthk2dkh.aspx structs not of

[issue20160] broken ctypes calling convention on MSVC / 64-bit Windows (large structs)

Larry Hastings added the comment: For the record, I'd be very happy to accept a patch for this into 3.4 at any time. -- nosy: +larry ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20160 ___