zbentley commented on issue #85: URL: https://github.com/apache/pulsar-client-python/issues/85#issuecomment-1571083980
> a fully-native Python client is only more friendly for Python developers, not Python users. I disagree. A lot of issues with python binary extensions in general boil down to them being implemented in non-Python languages. Those issues fall into three major categories: - Issues building and installing a module that uses an extension. Because cross-platform support (especially for older Pythons, especially on Macs) is tricky, many folks end up invoking a compile/build toolchain as part of `pip install`. `pulsar-client` doesn't do this because it doesn't offer a source wheel, meaning that those users are just left out in the cold. But even if a source wheel were offered, many users (not developers, users) would then struggle with compilation/installation issues. This has been true for very nearly every single binary-extension-based Python project I've ever worked with--they're just never quite as seamless to install for all developers on all platforms as native code. This even extends to binary-based modules with really robust packaging support, like `numpy`; the issue rate is just plain higher. - Issues linking and using an extension-based module. Having complex code called from Python in a non-Python language, with non-Python semantics around garbage collection, concurrency, GIL management, and the like is just plain hard. It's even harder than usual for `pulsar-client`, since the C++ driver at the core of this package resembles an operating system or async runtime more than it resembles a network driver. This results in a very large volume of pretty bad bugs over time, including bugs around garbage collection, GIL deadlocks invoking Python loggers, issues with Python callbacks supplied to e.g. `send_async` blocking for longer than expected ... the list goes on. Those aren't code quality issues with either the C++ *or* the Python code; they're simply issues with the _interoperation_ between two highly complex language runtimes. While fixes have been found for most of them, the rate and severity of those issues really damages my ability to recommend this library to other Python users. - Situations where Python runtime makes assumptions that extension runtimes do not. While Python might be a higher-level language than C++, many of the conventions around how python code is run in the community are ... if not "poorly supported" then at least "unusual" in the presence of pybind/boost/threaded C++ code. Python code is frequently async/awaited, forked (without execcing, as a primary means of adding concurrency to Python programs everywhere), threaded (even with the GIL, many folks can use threads for IO-ful operations), and embedded as a runtime inside applications written in another language, such as Golang or C. In all of those environments, using an extension-based module in Python is riskier than in a normal `python main.py` single-threaded invocation. And those aren't terribly weird contexts; many of those are very common in the python community! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
