Re: [Python-Dev] libffi embedded in CPython

2015-03-25 Thread Nick Coghlan
On 26 March 2015 at 12:40, Zachary Ware wrote: > On Mar 25, 2015 9:28 PM, "Nick Coghlan" wrote: >> >> On 26 March 2015 at 01:57, Steve Dower wrote: >> > Zachary Ware wrote: >> >> On Mar 25, 2015 4:22 AM, "Paul Moore" wrote: >> >>> On a related note, is there any information available on how the

Re: [Python-Dev] libffi embedded in CPython

2015-03-25 Thread Zachary Ware
On Mar 25, 2015 9:28 PM, "Nick Coghlan" wrote: > > On 26 March 2015 at 01:57, Steve Dower wrote: > > Zachary Ware wrote: > >> On Mar 25, 2015 4:22 AM, "Paul Moore" wrote: > >>> On a related note, is there any information available on how the > >>> "externals" repo is maintained? In particular, s

Re: [Python-Dev] libffi embedded in CPython

2015-03-25 Thread Nick Coghlan
On 26 March 2015 at 01:57, Steve Dower wrote: > Zachary Ware wrote: >> On Mar 25, 2015 4:22 AM, "Paul Moore" wrote: >>> On a related note, is there any information available on how the >>> "externals" repo is maintained? In particular, should things in there >>> be exact copies of upstream, or is

Re: [Python-Dev] libffi embedded in CPython

2015-03-25 Thread Steve Dower
Zachary Ware wrote: > On Mar 25, 2015 4:22 AM, "Paul Moore" wrote: >> On a related note, is there any information available on how the >> "externals" repo is maintained? In particular, should things in there >> be exact copies of upstream, or is it OK to include extra data (in >> this case, the re

Re: [Python-Dev] libffi embedded in CPython

2015-03-25 Thread Zachary Ware
On Mar 25, 2015 4:22 AM, "Paul Moore" wrote: > > On 25 March 2015 at 09:09, Antoine Pitrou wrote: > > I'm not sure we guarantee anything. In any case, it's only a small > > proportion of the kind of crashes you can get by messing the signature. > > Fair point. I guess what I'm asking is, would it

Re: [Python-Dev] libffi embedded in CPython

2015-03-25 Thread Antoine Pitrou
On Wed, 25 Mar 2015 09:22:01 + Paul Moore wrote: > On 25 March 2015 at 09:09, Antoine Pitrou wrote: > > I'm not sure we guarantee anything. In any case, it's only a small > > proportion of the kind of crashes you can get by messing the signature. > > Fair point. I guess what I'm asking is, w

Re: [Python-Dev] libffi embedded in CPython

2015-03-25 Thread Paul Moore
On 25 March 2015 at 09:09, Antoine Pitrou wrote: > I'm not sure we guarantee anything. In any case, it's only a small > proportion of the kind of crashes you can get by messing the signature. Fair point. I guess what I'm asking is, would it be OK to remove the code that checks for a stack size di

Re: [Python-Dev] libffi embedded in CPython

2015-03-25 Thread Antoine Pitrou
On Wed, 25 Mar 2015 08:15:18 + Paul Moore wrote: > On 25 March 2015 at 08:05, Maciej Fijalkowski wrote: > > Linux crashes. The mechanism for detecting the number of arguments is > > only available on windows (note that this is a band-aid anyway, since > > if your arguments are of the wrong ki

Re: [Python-Dev] libffi embedded in CPython

2015-03-25 Thread Paul Moore
On 25 March 2015 at 08:05, Maciej Fijalkowski wrote: > Linux crashes. The mechanism for detecting the number of arguments is > only available on windows (note that this is a band-aid anyway, since > if your arguments are of the wrong kind you segfault anyway). We do > have two copies of libffi one

Re: [Python-Dev] libffi embedded in CPython

2015-03-25 Thread Maciej Fijalkowski
On Tue, Mar 24, 2015 at 11:31 PM, Paul Moore wrote: > On 12 March 2015 at 17:44, Paul Moore wrote: >> On 12 March 2015 at 17:26, Brett Cannon wrote: I'm all for ditching our 'libffi_msvc' in favor of adding libffi as another 'external' for the Windows build. I have managed to get

Re: [Python-Dev] libffi embedded in CPython

2015-03-24 Thread Paul Moore
On 12 March 2015 at 17:44, Paul Moore wrote: > On 12 March 2015 at 17:26, Brett Cannon wrote: >>> I'm all for ditching our 'libffi_msvc' in favor of adding libffi as >>> another 'external' for the Windows build. I have managed to get >>> _ctypes to build on Windows using vanilla libffi sources,

Re: [Python-Dev] libffi embedded in CPython

2015-03-14 Thread Matthias Klose
On 03/11/2015 06:27 PM, Brett Cannon wrote: > On Mon, Dec 22, 2014 at 4:49 PM Jim J. Jewett wrote: > >> >> >> On Thu, Dec 18, 2014, at 14:13, Maciej Fijalkowski wrote: >>> ... http://bugs.python.org/issue23085 ... >>> is there any reason any more for libffi being included in CPython? >> >> >> Pau

Re: [Python-Dev] libffi embedded in CPython

2015-03-12 Thread Thomas Wouters
On Thu, Mar 12, 2015 at 6:26 PM, Brett Cannon wrote: > > > For UNIX OSs we could probably rely on the system libffi then. > Yes, it's possible to use the system libffi -- that's what most linux distros already do -- but only if you use dynamic linking. If you want to statically link libffi (usefu

Re: [Python-Dev] libffi embedded in CPython

2015-03-12 Thread Maciej Fijalkowski
On Thu, Mar 12, 2015 at 8:35 PM, Ned Deily wrote: > In article > , >> For UNIX OSs we could probably rely on the system libffi then. What's the >> situation on OS X? Anyone know if it has libffi, or would be need to be >> pulled in to be used like on Windows? > > Ronald (in http://bugs.python.org/

Re: [Python-Dev] libffi embedded in CPython

2015-03-12 Thread Paul Moore
On 12 March 2015 at 18:54, Zachary Ware wrote: > On Thu, Mar 12, 2015 at 12:44 PM, Paul Moore wrote: >> I'd be willing to contemplate helping out on the Windows side of >> things, if nobody else steps up (with the proviso that I have little >> free time, and I'm saying this without much idea of w

Re: [Python-Dev] libffi embedded in CPython

2015-03-12 Thread Zachary Ware
On Thu, Mar 12, 2015 at 12:44 PM, Paul Moore wrote: > I'd be willing to contemplate helping out on the Windows side of > things, if nobody else steps up (with the proviso that I have little > free time, and I'm saying this without much idea of what's involved > :-)) If Zachary can give a bit more

Re: [Python-Dev] libffi embedded in CPython

2015-03-12 Thread Ned Deily
In article , > For UNIX OSs we could probably rely on the system libffi then. What's the > situation on OS X? Anyone know if it has libffi, or would be need to be > pulled in to be used like on Windows? Ronald (in http://bugs.python.org/issue23534): "On OSX the internal copy of libffi that's used

Re: [Python-Dev] libffi embedded in CPython

2015-03-12 Thread Paul Moore
On 12 March 2015 at 17:26, Brett Cannon wrote: >> I'm all for ditching our 'libffi_msvc' in favor of adding libffi as >> another 'external' for the Windows build. I have managed to get >> _ctypes to build on Windows using vanilla libffi sources, prepared >> using their configure script from withi

Re: [Python-Dev] libffi embedded in CPython

2015-03-12 Thread Brett Cannon
On Thu, Mar 12, 2015 at 10:39 AM Zachary Ware wrote: > I started this message about 3 months ago; at this point I'm just > getting it posted so it stops rotting in my Drafts folder. > Thanks for looking into this! > > On Mon, Dec 22, 2014 at 3:49 PM, Jim J. Jewett > wrote: > > On Thu, Dec 18,

Re: [Python-Dev] libffi embedded in CPython

2015-03-12 Thread Zachary Ware
I started this message about 3 months ago; at this point I'm just getting it posted so it stops rotting in my Drafts folder. On Mon, Dec 22, 2014 at 3:49 PM, Jim J. Jewett wrote: > On Thu, Dec 18, 2014, at 14:13, Maciej Fijalkowski wrote: >> ... http://bugs.python.org/issue23085 ... >> is there a

Re: [Python-Dev] libffi embedded in CPython

2015-03-12 Thread Thomas Wouters
On Thu, Mar 12, 2015 at 1:20 AM, Guido van Rossum wrote: > Can we please decouple the ctypes deprecation discussion from efforts to > upgrade cffi? They can coexist just fine, and they don't even really solve > the same problem. > ctypes and cffi do actually solve the same problem, just not in t

Re: [Python-Dev] libffi embedded in CPython

2015-03-12 Thread Brett Cannon
On Wed, Mar 11, 2015 at 8:20 PM Guido van Rossum wrote: > Can we please decouple the ctypes deprecation discussion from efforts to > upgrade cffi? They can coexist just fine, and they don't even really solve > the same problem. > I mostly proposed deprecating ctypes because we were not keeping u

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Guido van Rossum
Can we please decouple the ctypes deprecation discussion from efforts to upgrade cffi? They can coexist just fine, and they don't even really solve the same problem. On Wed, Mar 11, 2015 at 3:20 PM, Brett Cannon wrote: > > > On Wed, Mar 11, 2015 at 6:03 PM Paul Moore wrote: > >> On 11 March 201

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Antoine Pitrou
On Wed, 11 Mar 2015 22:55:33 + Paul Moore wrote: > On 11 March 2015 at 22:33, Maciej Fijalkowski wrote: > > You're missing my point. Ripping off the libffi from CPython is a good > > idea to start with. Maybe deprecating ctypes is *also* a good idea, > > but it's a separate discussion point.

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Paul Moore
On 11 March 2015 at 22:33, Maciej Fijalkowski wrote: > You're missing my point. Ripping off the libffi from CPython is a good > idea to start with. Maybe deprecating ctypes is *also* a good idea, > but it's a separate discussion point. It certainly does not solve the > libffi problem. OK, so let'

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Maciej Fijalkowski
On Thu, Mar 12, 2015 at 12:31 AM, Antoine Pitrou wrote: > On Wed, 11 Mar 2015 23:10:14 +0200 > Maciej Fijalkowski wrote: >> > >> > Well, sure. The point is, such bugs are unlikely to appear at a fast >> > rate... Also, I don't understand why libffi issues would affect cffi >> > any less than it a

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Maciej Fijalkowski
On Thu, Mar 12, 2015 at 12:20 AM, Brett Cannon wrote: > > > On Wed, Mar 11, 2015 at 6:03 PM Paul Moore wrote: >> >> On 11 March 2015 at 21:45, Maciej Fijalkowski wrote: >> >> Is it possible to use cffi without a C compiler/headers as easily than >> >> ctypes? >> > >> > yes, it has two modes, one

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Antoine Pitrou
On Wed, 11 Mar 2015 23:10:14 +0200 Maciej Fijalkowski wrote: > > > > Well, sure. The point is, such bugs are unlikely to appear at a fast > > rate... Also, I don't understand why libffi issues would affect cffi > > any less than it affects ctypes, at least in the compiler-less mode of > > operatio

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Antoine Pitrou
On Wed, 11 Mar 2015 22:20:58 + Brett Cannon wrote: > > You're ignoring that it's not maintained, According to which metric/criterion? changeset: 94932:86c9ef950288 user:Steve Dower date:Tue Mar 10 09:56:38 2015 -0700 summary: Issue #23606: Disable ctypes.util.find_lib

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Brett Cannon
On Wed, Mar 11, 2015 at 6:03 PM Paul Moore wrote: > On 11 March 2015 at 21:45, Maciej Fijalkowski wrote: > >> Is it possible to use cffi without a C compiler/headers as easily than > >> ctypes? > > > > yes, it has two modes, one that does that and the other that does > > extra safety at the cost

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Paul Moore
On 11 March 2015 at 21:45, Maciej Fijalkowski wrote: >> Is it possible to use cffi without a C compiler/headers as easily than >> ctypes? > > yes, it has two modes, one that does that and the other that does > extra safety at the cost of a C compiler So if someone were to propose a practical appr

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Maciej Fijalkowski
On Wed, Mar 11, 2015 at 11:34 PM, Victor Stinner wrote: > > Le 11 mars 2015 18:29, "Brett Cannon" a écrit : >> I'm going to propose a somewhat controversial idea: let's deprecate the >> ctypes module. > > In the past I tried to deprecate many functions or modules because they are > rarely or neve

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Victor Stinner
Le 11 mars 2015 18:29, "Brett Cannon" a écrit : > I'm going to propose a somewhat controversial idea: let's deprecate the ctypes module. In the past I tried to deprecate many functions or modules because they are rarely or never used. Many developers prefered to keep them. By the way, I still wan

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Maciej Fijalkowski
On Wed, Mar 11, 2015 at 8:31 PM, Wes Turner wrote: > > On Mar 11, 2015 12:55 PM, "Maciej Fijalkowski" wrote: >> >> On Wed, Mar 11, 2015 at 7:50 PM, Antoine Pitrou >> wrote: >> > On Wed, 11 Mar 2015 17:27:58 + >> > Brett Cannon wrote: >> >> >> >> Did anyone ever step forward to do this? I'm

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Maciej Fijalkowski
On Wed, Mar 11, 2015 at 8:05 PM, Antoine Pitrou wrote: > On Wed, 11 Mar 2015 19:54:58 +0200 > Maciej Fijalkowski wrote: >> > >> > Depends what you call "dangerous". C code doesn't rot quicker than pure >> > Python code :-) Also, libffi really offers a wrapper around platform >> > ABIs, which rare

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Wes Turner
On Mar 11, 2015 12:55 PM, "Maciej Fijalkowski" wrote: > > On Wed, Mar 11, 2015 at 7:50 PM, Antoine Pitrou wrote: > > On Wed, 11 Mar 2015 17:27:58 + > > Brett Cannon wrote: > >> > >> Did anyone ever step forward to do this? I'm a bit worried about the > >> long-term viability of ctypes if we

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Antoine Pitrou
On Wed, 11 Mar 2015 19:54:58 +0200 Maciej Fijalkowski wrote: > > > > Depends what you call "dangerous". C code doesn't rot quicker than pure > > Python code :-) Also, libffi really offers a wrapper around platform > > ABIs, which rarely change. > > And yet, lesser known ABIs in libffi contain bug

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Paul Moore
On 11 March 2015 at 17:27, Brett Cannon wrote: > I'm going to propose a somewhat controversial idea: let's deprecate the > ctypes module. We now have things like cffi and Cython for people who need > to interface with C code. Both of those projects are maintained. And they > are not overly difficu

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Maciej Fijalkowski
On Wed, Mar 11, 2015 at 7:50 PM, Antoine Pitrou wrote: > On Wed, 11 Mar 2015 17:27:58 + > Brett Cannon wrote: >> >> Did anyone ever step forward to do this? I'm a bit worried about the >> long-term viability of ctypes if we don't have a maintainer or at least >> someone making sure we are sta

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Antoine Pitrou
On Wed, 11 Mar 2015 17:27:58 + Brett Cannon wrote: > > Did anyone ever step forward to do this? I'm a bit worried about the > long-term viability of ctypes if we don't have a maintainer or at least > someone making sure we are staying up-to-date with upstream libffi. The > ctypes module is a

Re: [Python-Dev] libffi embedded in CPython

2015-03-11 Thread Brett Cannon
On Mon, Dec 22, 2014 at 4:49 PM Jim J. Jewett wrote: > > > On Thu, Dec 18, 2014, at 14:13, Maciej Fijalkowski wrote: > > ... http://bugs.python.org/issue23085 ... > > is there any reason any more for libffi being included in CPython? > > > Paul Moore wrote: > > Probably the easiest way of moving

Re: [Python-Dev] libffi embedded in CPython

2014-12-22 Thread Jim J. Jewett
On Thu, Dec 18, 2014, at 14:13, Maciej Fijalkowski wrote: > ... http://bugs.python.org/issue23085 ... > is there any reason any more for libffi being included in CPython? Paul Moore wrote: > Probably the easiest way of moving this forward would be for someone > to identify the CPython-specific

Re: [Python-Dev] libffi embedded in CPython

2014-12-19 Thread Nick Coghlan
On 19 December 2014 at 23:01, Antoine Pitrou wrote: > On Fri, 19 Dec 2014 09:52:26 + > Paul Moore wrote: > > On 19 December 2014 at 08:26, Maciej Fijalkowski > wrote: > > > I would like to add that "not doing anything" is not a good strategy > > > either, because you accumulate bugs that ge

Re: [Python-Dev] libffi embedded in CPython

2014-12-19 Thread Antoine Pitrou
On Fri, 19 Dec 2014 09:52:26 + Paul Moore wrote: > On 19 December 2014 at 08:26, Maciej Fijalkowski wrote: > > I would like to add that "not doing anything" is not a good strategy > > either, because you accumulate bugs that get fixed upstream (I'm > > pretty sure all the problems from cpytho

Re: [Python-Dev] libffi embedded in CPython

2014-12-19 Thread Christian Heimes
On 19.12.2014 10:52, Paul Moore wrote: > Probably the easiest way of moving this forward would be for someone > to identify the CPython-specific patches in the current version, and > check if they are addressed in the latest libffi version. They haven't > been applied as they are, I gather, but may

Re: [Python-Dev] libffi embedded in CPython

2014-12-19 Thread Paul Moore
On 19 December 2014 at 08:26, Maciej Fijalkowski wrote: > I would like to add that "not doing anything" is not a good strategy > either, because you accumulate bugs that get fixed upstream (I'm > pretty sure all the problems from cpython got fixed in upstream > libffi, but not all libffi fixes mad

Re: [Python-Dev] libffi embedded in CPython

2014-12-19 Thread Maciej Fijalkowski
On Thu, Dec 18, 2014 at 10:36 PM, Jim J. Jewett wrote: > > > On Thu, Dec 18, 2014, at 14:13, Maciej Fijalkowski wrote: >> ... http://bugs.python.org/issue23085 ... >> is there any reason any more for libffi being included in CPython? > > [And why a fork, instead of just treating it as an external

Re: [Python-Dev] libffi embedded in CPython

2014-12-18 Thread Benjamin Peterson
On Thu, Dec 18, 2014, at 15:36, Jim J. Jewett wrote: > > > On Thu, Dec 18, 2014, at 14:13, Maciej Fijalkowski wrote: > > ... http://bugs.python.org/issue23085 ... > > is there any reason any more for libffi being included in CPython? > > [And why a fork, instead of just treating it as an exter

Re: [Python-Dev] libffi embedded in CPython

2014-12-18 Thread Jim J. Jewett
On Thu, Dec 18, 2014, at 14:13, Maciej Fijalkowski wrote: > ... http://bugs.python.org/issue23085 ... > is there any reason any more for libffi being included in CPython? [And why a fork, instead of just treating it as an external dependency] Benjamin Peterson responded: > It has some sort of

Re: [Python-Dev] libffi embedded in CPython

2014-12-18 Thread Benjamin Peterson
On Thu, Dec 18, 2014, at 14:50, Maciej Fijalkowski wrote: > well, the problem is essentially that libffi gets patched (e.g. for > ARM) and it does not make it's way to CPython quickly. This is > unlikely to be a security issue (for a variety of reasons, including > ctypes), but it's still an issu

Re: [Python-Dev] libffi embedded in CPython

2014-12-18 Thread Benjamin Peterson
On Thu, Dec 18, 2014, at 14:50, Maciej Fijalkowski wrote: > well, the problem is essentially that libffi gets patched (e.g. for > ARM) and it does not make it's way to CPython quickly. This is > unlikely to be a security issue (for a variety of reasons, including > ctypes), but it's still an issu

Re: [Python-Dev] libffi embedded in CPython

2014-12-18 Thread Maciej Fijalkowski
well, the problem is essentially that libffi gets patched (e.g. for ARM) and it does not make it's way to CPython quickly. This is unlikely to be a security issue (for a variety of reasons, including ctypes), but it's still an issue I think. Segfaults related to e.g. stack alignment are hard to deb

Re: [Python-Dev] libffi embedded in CPython

2014-12-18 Thread Benjamin Peterson
On Thu, Dec 18, 2014, at 14:13, Maciej Fijalkowski wrote: > After reading this http://bugs.python.org/issue23085 and remembering > struggling having our own patches into cpython's libffi (but not into > libffi itself), I wonder, is there any reason any more for libffi > being included in CPython?

Re: [Python-Dev] libffi embedded in CPython

2014-12-18 Thread Maciej Fijalkowski
On Thu, Dec 18, 2014 at 9:17 PM, Steve Dower wrote: > Maciej Fijalkowski wrote: >> After reading this http://bugs.python.org/issue23085 and remembering >> struggling >> having our own patches into cpython's libffi (but not into libffi itself), I >> wonder, is there any reason any more for libffi

Re: [Python-Dev] libffi embedded in CPython

2014-12-18 Thread Steve Dower
Maciej Fijalkowski wrote: > After reading this http://bugs.python.org/issue23085 and remembering > struggling > having our own patches into cpython's libffi (but not into libffi itself), I > wonder, is there any reason any more for libffi being included in CPython? We use it for ctypes, so there'

[Python-Dev] libffi embedded in CPython

2014-12-18 Thread Maciej Fijalkowski
After reading this http://bugs.python.org/issue23085 and remembering struggling having our own patches into cpython's libffi (but not into libffi itself), I wonder, is there any reason any more for libffi being included in CPython? Cheers, fijal ___ Pyth