Github user holdenk commented on the pull request:
https://github.com/apache/spark/pull/13064#issuecomment-218836044
So the release notes since the last version we used are:
0.10.1:
> Major performance fix: the Python side is now using default buffering
when reading responses from the Java side. This is particularly important if
you transfer large parameters (large strings or byte arrays). A simple
benchmark found that repeatedly sending 10 MB strings went from 99 seconds to 1
second. Thanks to @kaytwo for finding this bug and suggesting a fix.
> Both the Java and the Python libraries are now available as OSGi bundles.
Thanks to kichwacoders for funding the work.
> The 0.10.0 jar uploaded to PyPI wrongly required Java 8. The Java
compatibility has been restored to 1.6. Thansk to @agronholm for finding this
bug.
> Added the __version__ attribute in the py4j package to conform to PEP396.
Thanks to @lessthanoptimal for reporting this bug.
> tickets closed for 0.10.1 release
>
0.10.0:
> Added a new threading model that is more efficient with indirect
recursion between Java and Python and that enables users to control which
thread will execute calls. Thanks to kichwacoders for funding the
implementation and providing the initial idea.
> Added TLS support to encrypt the communication between both sides. Thanks
to @njwhite.
> Added initial byte stream support so Python can consume Java byte streams
more efficiently. Support is still preliminary and subject to change in the
future, but it provides a good base to build on. See these Python unit test and
Java example class for a small example. Thanks to @njwhite.
> Java side: converted build script from ant to gradle. Introduced Java
coding conventions and static code analysis. See Java Coding Conventions for
more details.
> Java side: it is now possible to build a osgi bundle and an Eclipse
update site from Py4J source. See using Py4J with Eclipse
> tickets closed for 0.10.0 release
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
