[issue22194] access to cdecimal / libmpdec API

Stefan Krah added the comment: Closing in favor of #41324, which adds just the most important functions. -- resolution: -> duplicate stage: -> resolved status: open -> closed superseder: -> Add a minimal decimal capsule API ___ Python tracker

[issue22194] access to cdecimal / libmpdec API

Change by Paul Ganssle : -- nosy: +p-ganssle ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pyt

[issue22194] access to cdecimal / libmpdec API

Antoine Pitrou added the comment: For the record, we have realized that pure C code brought less than a 2x speedup compared to straightforward Python code, so cdecimal support has become a low-priority concern for Numba. For the curious, the C code is from libmpdec's own benchmark (Web site se

[issue22194] access to cdecimal / libmpdec API

Stefan Krah added the comment: Of course Windows is a problem. I do not know how to implement --with-system-libmpdec on Windows. -- ___ Python tracker ___ ___

[issue22194] access to cdecimal / libmpdec API

Stefan Krah added the comment: libmpdec has always been a separate project, predating the integration into Python-3.3. Before the Python-3.3 release, Jim Jewett suggested a cleaner library/module separation (and he was right, it made the code much more readable). Then distributors wanted --with

[issue22194] access to cdecimal / libmpdec API

Martin v. Löwis added the comment: I'd like to focus this issue a bit. Antoine originally proposed that non-Python code might want to access libmpdec. However, given that this is now a separate project (as it seems), I don't think it's Python's task to make the API available. If it is a separa

[issue22194] access to cdecimal / libmpdec API

Changes by Alexander Belopolsky : -- nosy: +belopolsky ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://

[issue22194] access to cdecimal / libmpdec API

Stefan Krah added the comment: The compatibility discussion was for the cdecimal-2.3 package that's hosted on PyPI and used for Python 2.7. IOW, will people use a capsule API that's Python-3 only? Compatibility with pypy would be esoteric indeed. :) --

[issue22194] access to cdecimal / libmpdec API

Larry Hastings added the comment: Are there any other modules where the capsule API works in both CPython and PyPy? I thought capsule APIs were decidedly implementation-specific. Not that I'm not for it in theory. But this is some crazy uncharted hyper-compatibility territory we're talking a

[issue22194] access to cdecimal / libmpdec API

Antoine Pitrou added the comment: If there aren't today, there will be in a few years time, and they'll be glad they can support Python 3.5. -- ___ Python tracker ___ __

[issue22194] access to cdecimal / libmpdec API

Stefan Krah added the comment: Sure, if there are people who write python3-only C modules (I can't think of one right now). -- ___ Python tracker ___

[issue22194] access to cdecimal / libmpdec API

Antoine Pitrou added the comment: Le 03/09/2014 18:33, Stefan Krah a écrit : > > > Yes, cdecimal on PyPI is slower, has a different ABI, uses libmpdec-2.3, > has subtle differences in the context handling, cannot subclass the > context, ... ;) > > I think a common C API for 2.7 is only possibl

[issue22194] access to cdecimal / libmpdec API

Stefan Krah added the comment: Antoine Pitrou wrote: > > large parts of the cdecimal on PyPI (which uses the incompatible > > libmpdec-2.3) would need to be rewritten. > > Ah, so it has an incompatible ABI? That will complicate things a bit :-) Yes, cdecimal on PyPI is slower, has a different

[issue22194] access to cdecimal / libmpdec API

Antoine Pitrou added the comment: > Note that active symbol hiding has always only worked for gcc (but I think on > Windows and AIX the symbols are hidden by default anyway). Does it mean a separate Windows and AIX solution should be found? I think if we can't make the mpd symbols available in

[issue22194] access to cdecimal / libmpdec API

Antoine Pitrou added the comment: > large parts of the cdecimal on PyPI (which uses the incompatible > libmpdec-2.3) would need to be rewritten. Ah, so it has an incompatible ABI? That will complicate things a bit :-) -- ___ Python tracker

[issue22194] access to cdecimal / libmpdec API

Stefan Krah added the comment: Well, we have two issues now: 1) Make the _decimal API available via capsule. 2) Make the libmpdec symbols public (i.e. remove "GCC visibility push(hidden)" from Modules/_decimal/libmpdec/mpdecimal.h. The question here is now whether 2) is safe. Note th

[issue22194] access to cdecimal / libmpdec API

Larry Hastings added the comment: I think this is a duplicate of #15237. -- nosy: +larry ___ Python tracker ___ ___ Python-bugs-list m

[issue22194] access to cdecimal / libmpdec API

Stefan Krah added the comment: Thanks, Stefan. So everyone agrees that Capsule is the right way for the API. Then this issue is about making the libmpdec symbols public. I've tried to produce a collision with duplicate symbols as outlined in msg176486, but I haven't been successful (on Linux)

[issue22194] access to cdecimal / libmpdec API

Changes by Stefan Krah : Removed file: http://bugs.python.org/file36446/module_get_symbol.diff ___ Python tracker ___ ___ Python-bugs-list mai

[issue22194] access to cdecimal / libmpdec API

Stefan Behnel added the comment: > Stefan (Behnel), could you comment on the strategy that you had in mind? > Is it similar to module_get_symbol.diff or entirely different? I agree with Antoine that a Capsule would work better here (and yes, the performance problem of capsules is only with case

[issue22194] access to cdecimal / libmpdec API

Antoine Pitrou added the comment: Le 24/08/2014 11:11, Stefan Krah a écrit : > > Antoine Pitrou wrote: >> (which means that, perhaps, the answer is to make the mpd_ prefix >> configurable with a #define?) > > I don't know 100% what you have in mind, but Debian and Arch already ship > --with-sys

[issue22194] access to cdecimal / libmpdec API

Stefan Krah added the comment: Antoine Pitrou wrote: > (which means that, perhaps, the answer is to make the mpd_ prefix > configurable with a #define?) I don't know 100% what you have in mind, but Debian and Arch already ship --with-system-libmpdec, so only the mpd_* functions would be availa

[issue22194] access to cdecimal / libmpdec API

Antoine Pitrou added the comment: > MvL, in #4555 (msg176486). Ok, I'm cc'ing Martin then :-) Note RTLD_LOCAL seems to be the default with dlopen(). Now I don't know how that behaves when you have a chained library loading, e.g.: Apache --> dlopen(Python dll) --> dlopen(_decimal dll) _decim

[issue22194] access to cdecimal / libmpdec API

Stefan Krah added the comment: Ah yes, the array of function pointers is directly accessible. I did not look close enough -- reading the word "spam" 100x in the docs always makes me skim the text. ;) -- ___ Python tracker

[issue22194] access to cdecimal / libmpdec API

Antoine Pitrou added the comment: Le 24/08/2014 04:56, Stefan Krah a écrit : > > I'm not sure about OS X, but I would be surprised if it did not work. > > To my limited knowledge, Capsules are slow, see also here (the penultimate > paragraph): They are slow if you have to lookup and unwrap a cap

[issue22194] access to cdecimal / libmpdec API

Stefan Krah added the comment: Antoine Pitrou wrote: > Who are those people? #16745 was opened by you :-) MvL, in #4555 (msg176486). > > Platform specific maybe, but no hack: I was thinking about storing the DSO > > handle in the PyModuleObject struct and add functions to lookup symbols. > >

[issue22194] access to cdecimal / libmpdec API

Antoine Pitrou added the comment: Le 23/08/2014 16:42, Stefan Krah a écrit : > >> That's what I meant. The issue here is that Python's libmpdec is not exposed >> to third-party code at all. Also there should probably be a (thin?) API to >> get >> at the underlying mpdec object from a cdecimal Py

[issue22194] access to cdecimal / libmpdec API

Stefan Krah added the comment: > That's what I meant. The issue here is that Python's libmpdec is not exposed > to third-party code at all. Also there should probably be a (thin?) API to get > at the underlying mpdec object from a cdecimal PyObject (apologies for the > poor > wording, I'm actual

[issue22194] access to cdecimal / libmpdec API

Changes by Antoine Pitrou : -- nosy: +lemburg ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pyth

[issue22194] access to cdecimal / libmpdec API

Stefan Behnel added the comment: > (for the record, the context is that we would like to support decimal objects > efficiently in Numba) Same for Cython, although I guess we wouldn't do more than shipping the necessary declarations and (likely) also enable auto-coercion between the libmpdec d

[issue22194] access to cdecimal / libmpdec API

Antoine Pitrou added the comment: Relatedly, is the libmpdec ABI stable? That is, if I build a separate libmpdec, can I expect it to handle cdecimal's innards fine, or will there be problems? -- ___ Python tracker

[issue22194] access to cdecimal / libmpdec API

Antoine Pitrou added the comment: Note that you could expose a C API even if decimal didn't become built-in, see Include/datetime.h for example. -- ___ Python tracker ___ __

[issue22194] access to cdecimal / libmpdec API

Antoine Pitrou added the comment: > 3) It's not clear whether users would not be served better by > using functions from libmpdec directly (much faster, > probably less error handling). That's what I meant. The issue here is that Python's libmpdec is not exposed to third-party code at

[issue22194] access to cdecimal / libmpdec API

Stefan Krah added the comment: I'm a little unsure what to do with the API, see also #15237: 1) I'm not too fond of the Capsule method, especially because it *seems* possible to get at the symbols directly on Linux and Windows (provided that they aren't static of course). 2) I wou

[issue22194] access to cdecimal / libmpdec API

Changes by Stefan Krah : Added file: http://bugs.python.org/file36378/api-demo[1].c ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue22194] access to cdecimal / libmpdec API

New submission from Antoine Pitrou: Currently cdecimal exports no C API that I know of, and it makes sure the libmpdec symbols are kept private in the .so file. It would be nice for external C code (or, in general, non-Python code) to be able to access cdecimal objects, and make operations on