On Thu, Aug 18, 2016 at 12:05 PM, Jason Newton <nev...@gmail.com> wrote: > Accidentally posted to an already-opened tab for the cython-users ML > yesterday, moving to here. Following up from a github opened issue here: > > https://github.com/cython/cython/issues/1440 > > I was hoping we could give a way to drop straight into C/C++ inside of > Cython pyx files. > > Why? > > -It [helps] avoid needing to declare every class/function in cython, a > somewhat daunting/impossible task that I think everyone hates. Have you > libraries like Eigen or others that use complex template based techniques? > How about those with tons of [member] functions to boot or getting C++ > inheritance involved.
I agree that this is a pain, and better tooling should be developed to (mostly?) eliminate this (e.g. see the recent thread at https://groups.google.com/forum/#!topic/cython-users/c8ChI6jERzY ). Of course having symbols magically appear (e.g. due some #Include, who knows which) has its downsides too, which is why import * is often discouraged in Python too. > -It works around having the Cython compiler know about all of C++'s nuances > - as an advanced C++ developer these are painful and it is a 2nd class > citizen to Cython's simpler C-support - that's no good. Just right now I > was bitten by yet another template argument bug and it's clear C++ template > arguments have been kind of dicy since support appeared. Yes, C++ is an extraordinarily complicated language, and exposing all of that would add significant amounts of complexity to Cython itself, and perhaps more importantly increase the barrier of entry to reading Cython code. One of the guiding principles is that we try to stay as close to Python as possible (if you know Python, you can probably read Cython, and with a minimal amount of C knowledge start writing it) and much of C++ simply isn't Pythonic. Though, as you discovered, there are some basic things like non-type template arguments that we would like to have. If there are other specific C++ constructs that are commonly used but impossible to express in Cython it'd be useful to know. > -It would allow single source files - I think this is important for runtime > compiled quasi-JIT/AOC fragments, like OpenCL/PyOpenCL/PyCUDA provide Not quite following what you're saying here. > The idea is that Cython glue makes the playing field for extracting data > easy, but that once it's extracted to a cdef variable for instance, cython > doesn't need to know what happens. Maybe in a way sort of like the GCC asm > extension. Hopefully simpler variable passing though. Cython uses "mangled" names (e.g. with a __pyx prefix) to avoid any possible conflicts. Specifying what/how to mangle could get as ugly as GCC's asm variable passing. And embedded variable declarations, let alone control flow statements (especially return, break, ...) could get really messy. It obscures analysis Cython can do on the code, such as whether variables are used or what values they may take. Little code snippets are not always local either, e.g. do they often need to refer to variables (or headers) referenced elsewhere. And they must all be mutually compatible. That's aside from the jarring nature of interleaving Python and C++ code. Would semicolons be required? How would parsers (including IDEs) handle the C++ snippets? Or would they be placed in opaque strings/comments (which I'd rather avoid)? Feels like you want PyInline or weave.inline, but at compile time. > The alternative to not having this construct is a concrete wall. I'll have > to segment up files for C++ for the rest of time until I get function forms > that Cython can handle. At that point I just say screw it and use boost > python. Of course cython does the definitions, data extraction, and > compilation so much easier than boost.python. It would be a shame to not > consider this plight C++ developers have been cornered into and you can't > say the C++ libraries are broken, it is the strategy Cython is taking that > cannot work. Some C++ libraries are not very amenable to being used outside a C++ context. They don't expose an "FFI-friendly" interface and calling them from other languages is more painful. That doesn't mean they're broken, just C++ centric. The strategy that Cython uses is that you use (as close to possible) Python structures and syntax to write your bindings. This is great for people who know and understand Python, but for those who would rather write their code (including bindings) in C++ there's Boost.Python and others (including writing C extensions by hand). If your library is full of C++isms, another option is to create a C wrapper, and expose that to Python (which is what people did before there was any C++ support). If exposing these as Python functions, with automatic conversion to/from Python types is the primary value you're getting from Cython, simply declare these as cpdef functions in your extern blocks and wrappers will automatically be created. If your C wrappers are simple enough, you might even be able to auto-generate the .pxd and .pyx files with some of the tools listed at https://github.com/cython/cython/wiki/AutoPxd . Yes, you'd have your C++ code in a C++ file and "Python" code in a .pyx file rather than interleaving them, but the amount of code you write would be the same. > I did some examination of how this could be implemented - my idea was to > base on the print/exec statements handling and simply emit their arguments > from the Nodes to the code generator. Proposing inline_c or inline_ as the > statement. Such a statement could be used to do macros, pull in includes, > and modify/declare c-variables. > > -Jason > > _______________________________________________ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > _______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
