Sorry, tried running your code and I'm able to reproduce the behavior (the leak fixed in ARROW-1017 is still fixed). I will investigate a bit later this evening to find the problem, but we will probably need to make another RC. Thanks for catching this!
On Thu, May 18, 2017 at 5:52 PM, Wes McKinney <wesmck...@gmail.com> wrote: > Do you definitely have the ARROW-1017 patch? If your > libarrow_python.so is out of date then you will still have the memory > leak > > On Thu, May 18, 2017 at 5:51 PM, Bryan Cutler <cutl...@gmail.com> wrote: >> -1 >> >> I think there might be a memory leak with Python RecordBatchFileReader. In >> repeated calls to a reader, memory usage increases. I filed ARROW-1053 >> here https://issues.apache.org/jira/browse/ARROW-1053 and included some >> code to reproduce. I noticed this while testing the RC with SPARK-13534, >> so this would be a blocker for me. >> >> On Thu, May 18, 2017 at 9:18 AM, Wes McKinney <wesmck...@gmail.com> wrote: >> >>> +1 (binding) >>> >>> * Verified GPG signature >>> * Built and ran Java unit tests, C++ unit tests, GLib bindings and >>> tests, and Python bindings and tests on Ubuntu 14.04, gcc 4.9. Tested >>> pyarrow Parquet extension against parquet-cpp-1.1.0-rc0 >>> * Built and ran C++ and Python tests with Visual Studio 2015, >>> including Parquet Python extension/tests >>> >>> On Thu, May 18, 2017 at 10:51 AM, Justin Erenkrantz >>> <jus...@erenkrantz.com> wrote: >>> > +1. >>> > >>> > I verified the PGP sig (signed with 1735623D from [1]), MD5 and SHA1 >>> sigs. >>> > >>> > I successfully ran the test suites for Java (mvn test) and C++ (make >>> test) >>> > on both MacOS 10.12.4 and Ubuntu 17.04/x86_64. I got a bit stuck on >>> > building the Python module though. >>> > >>> > We might want to update the release process to create the SHA sig as >>> .sha1 >>> > rather than .sha. (httpd has done this.) Perhaps we might also want to >>> > upgrade it to SHA256 if we're worried about collisions? =) (I don't >>> know >>> > how many older distros have OpenSSL that supports SHA256 - but, RHEL 7 >>> and >>> > Ubuntu 16.04 do; but, MacOS 10.12 doesn't.) >>> > >>> > Separately, I'll probably submit some PRs for improving some of the build >>> > docs to help the next poor schlub - but, none of that should be >>> considered >>> > a blocker for release. >>> > >>> > Cheers. -- justin >>> > >>> > 1. https://dist.apache.org/repos/dist/release/arrow/KEYS >>> > >>> > On Wed, May 17, 2017 at 4:07 PM, Wes McKinney <wesmck...@gmail.com> >>> wrote: >>> > >>> >> Hello all, >>> >> >>> >> I'd like to propose the 1st release candidate (rc0) of Apache Arrow >>> version >>> >> 0.4.0. It covers a total of 75 resolved JIRAs [1]. Thanks to everyone >>> who >>> >> contributed to this release! >>> >> >>> >> The source release rc0 is hosted at [2]. >>> >> >>> >> This release candidate is based on commit >>> >> fea6b71468618d22ece16250ff75f23ba2f18914 >>> >> >>> >> The vote will be open for the next ~72 hours ending at 16:15 Eastern US >>> >> Time, >>> >> May 20, 2017. >>> >> >>> >> [ ] +1 Release this as Apache Arrow 0.4.0 >>> >> [ ] +0 >>> >> [ ] -1 Do not release this as Apache Arrow 0.4.0 because... >>> >> >>> >> Thanks, >>> >> Wes >>> >> >>> >> How to validate a release signature: >>> >> https://httpd.apache.org/dev/verification.html >>> >> >>> >> [1]: >>> >> https://issues.apache.org/jira/issues/?jql=project%20% >>> >> 3D%20ARROW%20AND%20status%20in%20(Resolved%2C%20Closed)% >>> >> 20AND%20fixVersion%20%3D%200.4.0 >>> >> [2]: https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-0. >>> 4.0-rc0/ >>> >> [3]: https://github.com/apache/arrow/tree/ >>> fea6b71468618d22ece16250ff75f2 >>> >> 3ba2f18914 >>> >> >>>
