Re: Making a bugfix 0.11.1 release
I was able to verify with the script from master (which includes ARROW-3430) so I started a vote with the RC0. Please vote. Thank you! On Sat, Oct 20, 2018 at 5:06 PM Wes McKinney wrote: > > Looks like I need > https://github.com/apache/arrow/commit/274237cfd396a8cb3424c12ef75e7267e0480cb9. > I'll cherry-pick that into the maint branch and start making a new RC > On Sat, Oct 20, 2018 at 4:55 PM Wes McKinney wrote: > > > > I created 0.11.1 RC0 but verification (verify-release-candidate.sh > > source 0.11.1 0) fails with > > > > + export > > GI_TYPELIB_PATH=/tmp/arrow-0.11.1.Cpdsj/install/lib/girepository-1.0: > > + GI_TYPELIB_PATH=/tmp/arrow-0.11.1.Cpdsj/install/lib/girepository-1.0: > > + bundle --version > > Bundler version 1.16.5 > > + bundle install --path vendor/bundle > > >>> Could not locate Gemfile > > + cleanup > > + rm -fr /tmp/arrow-0.11.1.Cpdsj > > > > @Kou -- what should I do? > > > > Thanks > > Wes > > On Sat, Oct 20, 2018 at 2:16 PM Wes McKinney wrote: > > > > > > I'm working on the source release now. We need to try to determine the > > > cause of the spurious RAT errors: > > > > > > https://gist.github.com/wesm/490dc50c494cc914ae28b4fd897a73eb > > > On Sat, Oct 20, 2018 at 11:27 AM Wes McKinney wrote: > > > > > > > > Seems that Travis CI builds are triggering Appveyor builds, so things > > > > show up as failed > > > > > > > > https://github.com/wesm/crossbow/branches/all?utf8=%E2%9C%93=build-8 > > > > > > > > On Sat, Oct 20, 2018 at 10:18 AM Uwe L. Korn wrote: > > > > > > > > > > I have triggered the wheel builds on my crossbow repo with build-25, > > > > > feel free to use them. > > > > > > > > > > Uwe > > > > > > > > > > On Sat, Oct 20, 2018, at 3:52 PM, Wes McKinney wrote: > > > > > > I'm having problems with Crossbow. I am going to try a few things > > > > > > (going through the setup process "from scratch" -- new tokens, new > > > > > > local repositories, etc.) but if I can't figure it out today, a > > > > > > different PMC with a working Crossbow setup may need to cut the > > > > > > release candidate artifacts > > > > > > On Fri, Oct 19, 2018 at 10:48 AM Wes McKinney > > > > > > wrote: > > > > > > > > > > > > > > I prepared the maintenance branch here > > > > > > > > > > > > > > https://github.com/apache/arrow/tree/maint-0.11.x > > > > > > > > > > > > > > I'm not fully set up with to create a release candidate yet with > > > > > > > Crossbow but I'll work on it today and try to get a vote started > > > > > > > by > > > > > > > EOD > > > > > > > On Fri, Oct 19, 2018 at 3:45 AM Antoine Pitrou > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > I would recommend cherry-picking a minimal number of patches > > > > > > > > for the > > > > > > > > bugfix and for packaging to work. It's better not to include > > > > > > > > API > > > > > > > > additions or changes. > > > > > > > > > > > > > > > > Regards > > > > > > > > > > > > > > > > Antoine. > > > > > > > > > > > > > > > > > > > > > > > > Le 17/10/2018 à 03:32, Wes McKinney a écrit : > > > > > > > > > hi folks, > > > > > > > > > > > > > > > > > > As a result of ARROW-3514, we need to release new Python > > > > > > > > > packages > > > > > > > > > quite urgently since major functionality (Parquet writing on > > > > > > > > > many > > > > > > > > > Linux platforms) is broken out of the box > > > > > > > > > > > > > > > > > > https://github.com/apache/arrow/commit/66d9a30a26e1659d9e992037339515e59a6ae518 > > > > > > > > > > > > > > > > > > We have a couple options: > > > > > > > > > > > > > > > > > > * Release from master > > > > > > > > > * Release 0.11.0 + minimum patches to include the ARROW-3514 > > > > > > > > > fix and > > > > > > > > > any follow up patches to fix packaging > > > > > > > > > > > > > > > > > > There is the option to "not" release but it could cause > > > > > > > > > confusion for > > > > > > > > > people because PyPI does not allow replacing wheels; a new > > > > > > > > > version > > > > > > > > > number has to be created. > > > > > > > > > > > > > > > > > > What would folks like to do? Who can help with the RM duties? > > > > > > > > > Since a > > > > > > > > > 72 hour vote is a _should_ rather than _must_, we could > > > > > > > > > reasonably > > > > > > > > > close the release vote in < 72 hours and push out packages > > > > > > > > > faster if > > > > > > > > > it is scope limited to the zlib bug fix > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Wes > > > > > > > > >
Re: Making a bugfix 0.11.1 release
Looks like I need https://github.com/apache/arrow/commit/274237cfd396a8cb3424c12ef75e7267e0480cb9. I'll cherry-pick that into the maint branch and start making a new RC On Sat, Oct 20, 2018 at 4:55 PM Wes McKinney wrote: > > I created 0.11.1 RC0 but verification (verify-release-candidate.sh > source 0.11.1 0) fails with > > + export GI_TYPELIB_PATH=/tmp/arrow-0.11.1.Cpdsj/install/lib/girepository-1.0: > + GI_TYPELIB_PATH=/tmp/arrow-0.11.1.Cpdsj/install/lib/girepository-1.0: > + bundle --version > Bundler version 1.16.5 > + bundle install --path vendor/bundle > >>> Could not locate Gemfile > + cleanup > + rm -fr /tmp/arrow-0.11.1.Cpdsj > > @Kou -- what should I do? > > Thanks > Wes > On Sat, Oct 20, 2018 at 2:16 PM Wes McKinney wrote: > > > > I'm working on the source release now. We need to try to determine the > > cause of the spurious RAT errors: > > > > https://gist.github.com/wesm/490dc50c494cc914ae28b4fd897a73eb > > On Sat, Oct 20, 2018 at 11:27 AM Wes McKinney wrote: > > > > > > Seems that Travis CI builds are triggering Appveyor builds, so things > > > show up as failed > > > > > > https://github.com/wesm/crossbow/branches/all?utf8=%E2%9C%93=build-8 > > > > > > On Sat, Oct 20, 2018 at 10:18 AM Uwe L. Korn wrote: > > > > > > > > I have triggered the wheel builds on my crossbow repo with build-25, > > > > feel free to use them. > > > > > > > > Uwe > > > > > > > > On Sat, Oct 20, 2018, at 3:52 PM, Wes McKinney wrote: > > > > > I'm having problems with Crossbow. I am going to try a few things > > > > > (going through the setup process "from scratch" -- new tokens, new > > > > > local repositories, etc.) but if I can't figure it out today, a > > > > > different PMC with a working Crossbow setup may need to cut the > > > > > release candidate artifacts > > > > > On Fri, Oct 19, 2018 at 10:48 AM Wes McKinney > > > > > wrote: > > > > > > > > > > > > I prepared the maintenance branch here > > > > > > > > > > > > https://github.com/apache/arrow/tree/maint-0.11.x > > > > > > > > > > > > I'm not fully set up with to create a release candidate yet with > > > > > > Crossbow but I'll work on it today and try to get a vote started by > > > > > > EOD > > > > > > On Fri, Oct 19, 2018 at 3:45 AM Antoine Pitrou > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > I would recommend cherry-picking a minimal number of patches for > > > > > > > the > > > > > > > bugfix and for packaging to work. It's better not to include API > > > > > > > additions or changes. > > > > > > > > > > > > > > Regards > > > > > > > > > > > > > > Antoine. > > > > > > > > > > > > > > > > > > > > > Le 17/10/2018 à 03:32, Wes McKinney a écrit : > > > > > > > > hi folks, > > > > > > > > > > > > > > > > As a result of ARROW-3514, we need to release new Python > > > > > > > > packages > > > > > > > > quite urgently since major functionality (Parquet writing on > > > > > > > > many > > > > > > > > Linux platforms) is broken out of the box > > > > > > > > > > > > > > > > https://github.com/apache/arrow/commit/66d9a30a26e1659d9e992037339515e59a6ae518 > > > > > > > > > > > > > > > > We have a couple options: > > > > > > > > > > > > > > > > * Release from master > > > > > > > > * Release 0.11.0 + minimum patches to include the ARROW-3514 > > > > > > > > fix and > > > > > > > > any follow up patches to fix packaging > > > > > > > > > > > > > > > > There is the option to "not" release but it could cause > > > > > > > > confusion for > > > > > > > > people because PyPI does not allow replacing wheels; a new > > > > > > > > version > > > > > > > > number has to be created. > > > > > > > > > > > > > > > > What would folks like to do? Who can help with the RM duties? > > > > > > > > Since a > > > > > > > > 72 hour vote is a _should_ rather than _must_, we could > > > > > > > > reasonably > > > > > > > > close the release vote in < 72 hours and push out packages > > > > > > > > faster if > > > > > > > > it is scope limited to the zlib bug fix > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Wes > > > > > > > >
Re: Making a bugfix 0.11.1 release
I created 0.11.1 RC0 but verification (verify-release-candidate.sh source 0.11.1 0) fails with + export GI_TYPELIB_PATH=/tmp/arrow-0.11.1.Cpdsj/install/lib/girepository-1.0: + GI_TYPELIB_PATH=/tmp/arrow-0.11.1.Cpdsj/install/lib/girepository-1.0: + bundle --version Bundler version 1.16.5 + bundle install --path vendor/bundle >>> Could not locate Gemfile + cleanup + rm -fr /tmp/arrow-0.11.1.Cpdsj @Kou -- what should I do? Thanks Wes On Sat, Oct 20, 2018 at 2:16 PM Wes McKinney wrote: > > I'm working on the source release now. We need to try to determine the > cause of the spurious RAT errors: > > https://gist.github.com/wesm/490dc50c494cc914ae28b4fd897a73eb > On Sat, Oct 20, 2018 at 11:27 AM Wes McKinney wrote: > > > > Seems that Travis CI builds are triggering Appveyor builds, so things > > show up as failed > > > > https://github.com/wesm/crossbow/branches/all?utf8=%E2%9C%93=build-8 > > > > On Sat, Oct 20, 2018 at 10:18 AM Uwe L. Korn wrote: > > > > > > I have triggered the wheel builds on my crossbow repo with build-25, feel > > > free to use them. > > > > > > Uwe > > > > > > On Sat, Oct 20, 2018, at 3:52 PM, Wes McKinney wrote: > > > > I'm having problems with Crossbow. I am going to try a few things > > > > (going through the setup process "from scratch" -- new tokens, new > > > > local repositories, etc.) but if I can't figure it out today, a > > > > different PMC with a working Crossbow setup may need to cut the > > > > release candidate artifacts > > > > On Fri, Oct 19, 2018 at 10:48 AM Wes McKinney > > > > wrote: > > > > > > > > > > I prepared the maintenance branch here > > > > > > > > > > https://github.com/apache/arrow/tree/maint-0.11.x > > > > > > > > > > I'm not fully set up with to create a release candidate yet with > > > > > Crossbow but I'll work on it today and try to get a vote started by > > > > > EOD > > > > > On Fri, Oct 19, 2018 at 3:45 AM Antoine Pitrou > > > > > wrote: > > > > > > > > > > > > > > > > > > I would recommend cherry-picking a minimal number of patches for the > > > > > > bugfix and for packaging to work. It's better not to include API > > > > > > additions or changes. > > > > > > > > > > > > Regards > > > > > > > > > > > > Antoine. > > > > > > > > > > > > > > > > > > Le 17/10/2018 à 03:32, Wes McKinney a écrit : > > > > > > > hi folks, > > > > > > > > > > > > > > As a result of ARROW-3514, we need to release new Python packages > > > > > > > quite urgently since major functionality (Parquet writing on many > > > > > > > Linux platforms) is broken out of the box > > > > > > > > > > > > > > https://github.com/apache/arrow/commit/66d9a30a26e1659d9e992037339515e59a6ae518 > > > > > > > > > > > > > > We have a couple options: > > > > > > > > > > > > > > * Release from master > > > > > > > * Release 0.11.0 + minimum patches to include the ARROW-3514 fix > > > > > > > and > > > > > > > any follow up patches to fix packaging > > > > > > > > > > > > > > There is the option to "not" release but it could cause confusion > > > > > > > for > > > > > > > people because PyPI does not allow replacing wheels; a new version > > > > > > > number has to be created. > > > > > > > > > > > > > > What would folks like to do? Who can help with the RM duties? > > > > > > > Since a > > > > > > > 72 hour vote is a _should_ rather than _must_, we could reasonably > > > > > > > close the release vote in < 72 hours and push out packages faster > > > > > > > if > > > > > > > it is scope limited to the zlib bug fix > > > > > > > > > > > > > > Thanks, > > > > > > > Wes > > > > > > >
Re: Making a bugfix 0.11.1 release
I'm working on the source release now. We need to try to determine the cause of the spurious RAT errors: https://gist.github.com/wesm/490dc50c494cc914ae28b4fd897a73eb On Sat, Oct 20, 2018 at 11:27 AM Wes McKinney wrote: > > Seems that Travis CI builds are triggering Appveyor builds, so things > show up as failed > > https://github.com/wesm/crossbow/branches/all?utf8=%E2%9C%93=build-8 > > On Sat, Oct 20, 2018 at 10:18 AM Uwe L. Korn wrote: > > > > I have triggered the wheel builds on my crossbow repo with build-25, feel > > free to use them. > > > > Uwe > > > > On Sat, Oct 20, 2018, at 3:52 PM, Wes McKinney wrote: > > > I'm having problems with Crossbow. I am going to try a few things > > > (going through the setup process "from scratch" -- new tokens, new > > > local repositories, etc.) but if I can't figure it out today, a > > > different PMC with a working Crossbow setup may need to cut the > > > release candidate artifacts > > > On Fri, Oct 19, 2018 at 10:48 AM Wes McKinney wrote: > > > > > > > > I prepared the maintenance branch here > > > > > > > > https://github.com/apache/arrow/tree/maint-0.11.x > > > > > > > > I'm not fully set up with to create a release candidate yet with > > > > Crossbow but I'll work on it today and try to get a vote started by > > > > EOD > > > > On Fri, Oct 19, 2018 at 3:45 AM Antoine Pitrou > > > > wrote: > > > > > > > > > > > > > > > I would recommend cherry-picking a minimal number of patches for the > > > > > bugfix and for packaging to work. It's better not to include API > > > > > additions or changes. > > > > > > > > > > Regards > > > > > > > > > > Antoine. > > > > > > > > > > > > > > > Le 17/10/2018 à 03:32, Wes McKinney a écrit : > > > > > > hi folks, > > > > > > > > > > > > As a result of ARROW-3514, we need to release new Python packages > > > > > > quite urgently since major functionality (Parquet writing on many > > > > > > Linux platforms) is broken out of the box > > > > > > > > > > > > https://github.com/apache/arrow/commit/66d9a30a26e1659d9e992037339515e59a6ae518 > > > > > > > > > > > > We have a couple options: > > > > > > > > > > > > * Release from master > > > > > > * Release 0.11.0 + minimum patches to include the ARROW-3514 fix and > > > > > > any follow up patches to fix packaging > > > > > > > > > > > > There is the option to "not" release but it could cause confusion > > > > > > for > > > > > > people because PyPI does not allow replacing wheels; a new version > > > > > > number has to be created. > > > > > > > > > > > > What would folks like to do? Who can help with the RM duties? Since > > > > > > a > > > > > > 72 hour vote is a _should_ rather than _must_, we could reasonably > > > > > > close the release vote in < 72 hours and push out packages faster if > > > > > > it is scope limited to the zlib bug fix > > > > > > > > > > > > Thanks, > > > > > > Wes > > > > > >
Re: Making a bugfix 0.11.1 release
Seems that Travis CI builds are triggering Appveyor builds, so things show up as failed https://github.com/wesm/crossbow/branches/all?utf8=%E2%9C%93=build-8 On Sat, Oct 20, 2018 at 10:18 AM Uwe L. Korn wrote: > > I have triggered the wheel builds on my crossbow repo with build-25, feel > free to use them. > > Uwe > > On Sat, Oct 20, 2018, at 3:52 PM, Wes McKinney wrote: > > I'm having problems with Crossbow. I am going to try a few things > > (going through the setup process "from scratch" -- new tokens, new > > local repositories, etc.) but if I can't figure it out today, a > > different PMC with a working Crossbow setup may need to cut the > > release candidate artifacts > > On Fri, Oct 19, 2018 at 10:48 AM Wes McKinney wrote: > > > > > > I prepared the maintenance branch here > > > > > > https://github.com/apache/arrow/tree/maint-0.11.x > > > > > > I'm not fully set up with to create a release candidate yet with > > > Crossbow but I'll work on it today and try to get a vote started by > > > EOD > > > On Fri, Oct 19, 2018 at 3:45 AM Antoine Pitrou wrote: > > > > > > > > > > > > I would recommend cherry-picking a minimal number of patches for the > > > > bugfix and for packaging to work. It's better not to include API > > > > additions or changes. > > > > > > > > Regards > > > > > > > > Antoine. > > > > > > > > > > > > Le 17/10/2018 à 03:32, Wes McKinney a écrit : > > > > > hi folks, > > > > > > > > > > As a result of ARROW-3514, we need to release new Python packages > > > > > quite urgently since major functionality (Parquet writing on many > > > > > Linux platforms) is broken out of the box > > > > > > > > > > https://github.com/apache/arrow/commit/66d9a30a26e1659d9e992037339515e59a6ae518 > > > > > > > > > > We have a couple options: > > > > > > > > > > * Release from master > > > > > * Release 0.11.0 + minimum patches to include the ARROW-3514 fix and > > > > > any follow up patches to fix packaging > > > > > > > > > > There is the option to "not" release but it could cause confusion for > > > > > people because PyPI does not allow replacing wheels; a new version > > > > > number has to be created. > > > > > > > > > > What would folks like to do? Who can help with the RM duties? Since a > > > > > 72 hour vote is a _should_ rather than _must_, we could reasonably > > > > > close the release vote in < 72 hours and push out packages faster if > > > > > it is scope limited to the zlib bug fix > > > > > > > > > > Thanks, > > > > > Wes > > > > >
Re: Making a bugfix 0.11.1 release
I have triggered the wheel builds on my crossbow repo with build-25, feel free to use them. Uwe On Sat, Oct 20, 2018, at 3:52 PM, Wes McKinney wrote: > I'm having problems with Crossbow. I am going to try a few things > (going through the setup process "from scratch" -- new tokens, new > local repositories, etc.) but if I can't figure it out today, a > different PMC with a working Crossbow setup may need to cut the > release candidate artifacts > On Fri, Oct 19, 2018 at 10:48 AM Wes McKinney wrote: > > > > I prepared the maintenance branch here > > > > https://github.com/apache/arrow/tree/maint-0.11.x > > > > I'm not fully set up with to create a release candidate yet with > > Crossbow but I'll work on it today and try to get a vote started by > > EOD > > On Fri, Oct 19, 2018 at 3:45 AM Antoine Pitrou wrote: > > > > > > > > > I would recommend cherry-picking a minimal number of patches for the > > > bugfix and for packaging to work. It's better not to include API > > > additions or changes. > > > > > > Regards > > > > > > Antoine. > > > > > > > > > Le 17/10/2018 à 03:32, Wes McKinney a écrit : > > > > hi folks, > > > > > > > > As a result of ARROW-3514, we need to release new Python packages > > > > quite urgently since major functionality (Parquet writing on many > > > > Linux platforms) is broken out of the box > > > > > > > > https://github.com/apache/arrow/commit/66d9a30a26e1659d9e992037339515e59a6ae518 > > > > > > > > We have a couple options: > > > > > > > > * Release from master > > > > * Release 0.11.0 + minimum patches to include the ARROW-3514 fix and > > > > any follow up patches to fix packaging > > > > > > > > There is the option to "not" release but it could cause confusion for > > > > people because PyPI does not allow replacing wheels; a new version > > > > number has to be created. > > > > > > > > What would folks like to do? Who can help with the RM duties? Since a > > > > 72 hour vote is a _should_ rather than _must_, we could reasonably > > > > close the release vote in < 72 hours and push out packages faster if > > > > it is scope limited to the zlib bug fix > > > > > > > > Thanks, > > > > Wes > > > >
Re: Making a bugfix 0.11.1 release
I'm having problems with Crossbow. I am going to try a few things (going through the setup process "from scratch" -- new tokens, new local repositories, etc.) but if I can't figure it out today, a different PMC with a working Crossbow setup may need to cut the release candidate artifacts On Fri, Oct 19, 2018 at 10:48 AM Wes McKinney wrote: > > I prepared the maintenance branch here > > https://github.com/apache/arrow/tree/maint-0.11.x > > I'm not fully set up with to create a release candidate yet with > Crossbow but I'll work on it today and try to get a vote started by > EOD > On Fri, Oct 19, 2018 at 3:45 AM Antoine Pitrou wrote: > > > > > > I would recommend cherry-picking a minimal number of patches for the > > bugfix and for packaging to work. It's better not to include API > > additions or changes. > > > > Regards > > > > Antoine. > > > > > > Le 17/10/2018 à 03:32, Wes McKinney a écrit : > > > hi folks, > > > > > > As a result of ARROW-3514, we need to release new Python packages > > > quite urgently since major functionality (Parquet writing on many > > > Linux platforms) is broken out of the box > > > > > > https://github.com/apache/arrow/commit/66d9a30a26e1659d9e992037339515e59a6ae518 > > > > > > We have a couple options: > > > > > > * Release from master > > > * Release 0.11.0 + minimum patches to include the ARROW-3514 fix and > > > any follow up patches to fix packaging > > > > > > There is the option to "not" release but it could cause confusion for > > > people because PyPI does not allow replacing wheels; a new version > > > number has to be created. > > > > > > What would folks like to do? Who can help with the RM duties? Since a > > > 72 hour vote is a _should_ rather than _must_, we could reasonably > > > close the release vote in < 72 hours and push out packages faster if > > > it is scope limited to the zlib bug fix > > > > > > Thanks, > > > Wes > > >
Re: Making a bugfix 0.11.1 release
I prepared the maintenance branch here https://github.com/apache/arrow/tree/maint-0.11.x I'm not fully set up with to create a release candidate yet with Crossbow but I'll work on it today and try to get a vote started by EOD On Fri, Oct 19, 2018 at 3:45 AM Antoine Pitrou wrote: > > > I would recommend cherry-picking a minimal number of patches for the > bugfix and for packaging to work. It's better not to include API > additions or changes. > > Regards > > Antoine. > > > Le 17/10/2018 à 03:32, Wes McKinney a écrit : > > hi folks, > > > > As a result of ARROW-3514, we need to release new Python packages > > quite urgently since major functionality (Parquet writing on many > > Linux platforms) is broken out of the box > > > > https://github.com/apache/arrow/commit/66d9a30a26e1659d9e992037339515e59a6ae518 > > > > We have a couple options: > > > > * Release from master > > * Release 0.11.0 + minimum patches to include the ARROW-3514 fix and > > any follow up patches to fix packaging > > > > There is the option to "not" release but it could cause confusion for > > people because PyPI does not allow replacing wheels; a new version > > number has to be created. > > > > What would folks like to do? Who can help with the RM duties? Since a > > 72 hour vote is a _should_ rather than _must_, we could reasonably > > close the release vote in < 72 hours and push out packages faster if > > it is scope limited to the zlib bug fix > > > > Thanks, > > Wes > >
Re: Making a bugfix 0.11.1 release
I would recommend cherry-picking a minimal number of patches for the bugfix and for packaging to work. It's better not to include API additions or changes. Regards Antoine. Le 17/10/2018 à 03:32, Wes McKinney a écrit : > hi folks, > > As a result of ARROW-3514, we need to release new Python packages > quite urgently since major functionality (Parquet writing on many > Linux platforms) is broken out of the box > > https://github.com/apache/arrow/commit/66d9a30a26e1659d9e992037339515e59a6ae518 > > We have a couple options: > > * Release from master > * Release 0.11.0 + minimum patches to include the ARROW-3514 fix and > any follow up patches to fix packaging > > There is the option to "not" release but it could cause confusion for > people because PyPI does not allow replacing wheels; a new version > number has to be created. > > What would folks like to do? Who can help with the RM duties? Since a > 72 hour vote is a _should_ rather than _must_, we could reasonably > close the release vote in < 72 hours and push out packages faster if > it is scope limited to the zlib bug fix > > Thanks, > Wes >
Re: Making a bugfix 0.11.1 release
Hi Antoine, Thanks for the quick response! This helps to clear up my confusion. Best Regards, Kevin Gurney From: Antoine Pitrou Sent: Thursday, October 18, 2018 9:54:47 AM To: dev@arrow.apache.org Subject: Re: Making a bugfix 0.11.1 release Le 18/10/2018 à 15:44, Kevin Gurney a écrit : > Hi All, > > We are working with the arrow version 0.9.0 C++ libraries in conjunction with > separate parquet-cpp version 1.4.0. > > Questions: > > 1. Does this zlib issue affect all clients of the arrow C++ libraries or > just the Python PyArrow code? To be clear: this is a packaging issue and only affects the PyArrow binary wheels (i.e. if you type "pip install pyarrow"). It should not affect people who self-compile Arrow or PyArrow; and it probably doesn't affect people who download other binaries, either (such as Conda packages). Regards Antoine.
Re: Making a bugfix 0.11.1 release
Le 18/10/2018 à 15:44, Kevin Gurney a écrit : > Hi All, > > We are working with the arrow version 0.9.0 C++ libraries in conjunction with > separate parquet-cpp version 1.4.0. > > Questions: > > 1. Does this zlib issue affect all clients of the arrow C++ libraries or > just the Python PyArrow code? To be clear: this is a packaging issue and only affects the PyArrow binary wheels (i.e. if you type "pip install pyarrow"). It should not affect people who self-compile Arrow or PyArrow; and it probably doesn't affect people who download other binaries, either (such as Conda packages). Regards Antoine.
Re: Making a bugfix 0.11.1 release
Hi All, We are working with the arrow version 0.9.0 C++ libraries in conjunction with separate parquet-cpp version 1.4.0. Questions: 1. Does this zlib issue affect all clients of the arrow C++ libraries or just the Python PyArrow code? 2. Does this zlib compression issue also affect the arrow version 0.9.0 C++ libraries (before parquet-cpp was merged in), or only the latest arrow version 0.11.0 C++ libraries (with parquet-cpp merged in)? Best Regards, Kevin Gurney From: Krisztián Szűcs Sent: Thursday, October 18, 2018 5:31:01 AM To: dev@arrow.apache.org Subject: Re: Making a bugfix 0.11.1 release I've added the two zlib issues to 0.11.1 version: https://issues.apache.org/jira/projects/ARROW/versions/12344316 On Wed, Oct 17, 2018 at 10:51 PM Wes McKinney wrote: > Got it, thank you for clarifying. It wasn't clear whether the bug > would occur in the build environment (CentOS 5 + devtoolset-2) as well > as other Linux environments. > On Wed, Oct 17, 2018 at 4:16 PM Antoine Pitrou wrote: > > > > > > Le 17/10/2018 à 20:38, Wes McKinney a écrit : > > > hi folks, > > > > > > Since the Python wheels are being installed 10,000 times per day or > > > more, I don't think we should allow them to be broken for much longer. > > > > > > What additional patches need to be done before an RC can be cut? Since > > > I'm concerned about the broken patches undermining the project's > > > reputation, I can adjust my priorities to start a release vote later > > > today or first thing tomorrow morning. Seems like > > > https://issues.apache.org/jira/browse/ARROW-3535 might be the last > > > item, and I can prepare a maintenance branch with the cherry-picked > > > fixes > > > > > > Was there a determination as to why our CI systems did not catch the > > > blocker ARROW-3514? > > > > Because it was not exercised by the test suite. My take is that the bug > > would only happen with specific data, e.g. tiny and/or entirely > > incompressible. I don't think general gzip compression of Parquet files > > was broken. > > > > Regards > > > > Antoine. >
Re: Making a bugfix 0.11.1 release
I've added the two zlib issues to 0.11.1 version: https://issues.apache.org/jira/projects/ARROW/versions/12344316 On Wed, Oct 17, 2018 at 10:51 PM Wes McKinney wrote: > Got it, thank you for clarifying. It wasn't clear whether the bug > would occur in the build environment (CentOS 5 + devtoolset-2) as well > as other Linux environments. > On Wed, Oct 17, 2018 at 4:16 PM Antoine Pitrou wrote: > > > > > > Le 17/10/2018 à 20:38, Wes McKinney a écrit : > > > hi folks, > > > > > > Since the Python wheels are being installed 10,000 times per day or > > > more, I don't think we should allow them to be broken for much longer. > > > > > > What additional patches need to be done before an RC can be cut? Since > > > I'm concerned about the broken patches undermining the project's > > > reputation, I can adjust my priorities to start a release vote later > > > today or first thing tomorrow morning. Seems like > > > https://issues.apache.org/jira/browse/ARROW-3535 might be the last > > > item, and I can prepare a maintenance branch with the cherry-picked > > > fixes > > > > > > Was there a determination as to why our CI systems did not catch the > > > blocker ARROW-3514? > > > > Because it was not exercised by the test suite. My take is that the bug > > would only happen with specific data, e.g. tiny and/or entirely > > incompressible. I don't think general gzip compression of Parquet files > > was broken. > > > > Regards > > > > Antoine. >
Re: Making a bugfix 0.11.1 release
Got it, thank you for clarifying. It wasn't clear whether the bug would occur in the build environment (CentOS 5 + devtoolset-2) as well as other Linux environments. On Wed, Oct 17, 2018 at 4:16 PM Antoine Pitrou wrote: > > > Le 17/10/2018 à 20:38, Wes McKinney a écrit : > > hi folks, > > > > Since the Python wheels are being installed 10,000 times per day or > > more, I don't think we should allow them to be broken for much longer. > > > > What additional patches need to be done before an RC can be cut? Since > > I'm concerned about the broken patches undermining the project's > > reputation, I can adjust my priorities to start a release vote later > > today or first thing tomorrow morning. Seems like > > https://issues.apache.org/jira/browse/ARROW-3535 might be the last > > item, and I can prepare a maintenance branch with the cherry-picked > > fixes > > > > Was there a determination as to why our CI systems did not catch the > > blocker ARROW-3514? > > Because it was not exercised by the test suite. My take is that the bug > would only happen with specific data, e.g. tiny and/or entirely > incompressible. I don't think general gzip compression of Parquet files > was broken. > > Regards > > Antoine.
Re: Making a bugfix 0.11.1 release
Le 17/10/2018 à 20:38, Wes McKinney a écrit : > hi folks, > > Since the Python wheels are being installed 10,000 times per day or > more, I don't think we should allow them to be broken for much longer. > > What additional patches need to be done before an RC can be cut? Since > I'm concerned about the broken patches undermining the project's > reputation, I can adjust my priorities to start a release vote later > today or first thing tomorrow morning. Seems like > https://issues.apache.org/jira/browse/ARROW-3535 might be the last > item, and I can prepare a maintenance branch with the cherry-picked > fixes > > Was there a determination as to why our CI systems did not catch the > blocker ARROW-3514? Because it was not exercised by the test suite. My take is that the bug would only happen with specific data, e.g. tiny and/or entirely incompressible. I don't think general gzip compression of Parquet files was broken. Regards Antoine.
Re: Making a bugfix 0.11.1 release
Seems like perhaps we should create a Dockerized environment for testing the wheels. I opened https://issues.apache.org/jira/browse/ARROW-3546 On Wed, Oct 17, 2018 at 2:38 PM Wes McKinney wrote: > > hi folks, > > Since the Python wheels are being installed 10,000 times per day or > more, I don't think we should allow them to be broken for much longer. > > What additional patches need to be done before an RC can be cut? Since > I'm concerned about the broken patches undermining the project's > reputation, I can adjust my priorities to start a release vote later > today or first thing tomorrow morning. Seems like > https://issues.apache.org/jira/browse/ARROW-3535 might be the last > item, and I can prepare a maintenance branch with the cherry-picked > fixes > > Was there a determination as to why our CI systems did not catch the > blocker ARROW-3514? > > Thanks, > Wes > On Tue, Oct 16, 2018 at 9:32 PM Wes McKinney wrote: > > > > hi folks, > > > > As a result of ARROW-3514, we need to release new Python packages > > quite urgently since major functionality (Parquet writing on many > > Linux platforms) is broken out of the box > > > > https://github.com/apache/arrow/commit/66d9a30a26e1659d9e992037339515e59a6ae518 > > > > We have a couple options: > > > > * Release from master > > * Release 0.11.0 + minimum patches to include the ARROW-3514 fix and > > any follow up patches to fix packaging > > > > There is the option to "not" release but it could cause confusion for > > people because PyPI does not allow replacing wheels; a new version > > number has to be created. > > > > What would folks like to do? Who can help with the RM duties? Since a > > 72 hour vote is a _should_ rather than _must_, we could reasonably > > close the release vote in < 72 hours and push out packages faster if > > it is scope limited to the zlib bug fix > > > > Thanks, > > Wes
Re: Making a bugfix 0.11.1 release
hi folks, Since the Python wheels are being installed 10,000 times per day or more, I don't think we should allow them to be broken for much longer. What additional patches need to be done before an RC can be cut? Since I'm concerned about the broken patches undermining the project's reputation, I can adjust my priorities to start a release vote later today or first thing tomorrow morning. Seems like https://issues.apache.org/jira/browse/ARROW-3535 might be the last item, and I can prepare a maintenance branch with the cherry-picked fixes Was there a determination as to why our CI systems did not catch the blocker ARROW-3514? Thanks, Wes On Tue, Oct 16, 2018 at 9:32 PM Wes McKinney wrote: > > hi folks, > > As a result of ARROW-3514, we need to release new Python packages > quite urgently since major functionality (Parquet writing on many > Linux platforms) is broken out of the box > > https://github.com/apache/arrow/commit/66d9a30a26e1659d9e992037339515e59a6ae518 > > We have a couple options: > > * Release from master > * Release 0.11.0 + minimum patches to include the ARROW-3514 fix and > any follow up patches to fix packaging > > There is the option to "not" release but it could cause confusion for > people because PyPI does not allow replacing wheels; a new version > number has to be created. > > What would folks like to do? Who can help with the RM duties? Since a > 72 hour vote is a _should_ rather than _must_, we could reasonably > close the release vote in < 72 hours and push out packages faster if > it is scope limited to the zlib bug fix > > Thanks, > Wes
Making a bugfix 0.11.1 release
hi folks, As a result of ARROW-3514, we need to release new Python packages quite urgently since major functionality (Parquet writing on many Linux platforms) is broken out of the box https://github.com/apache/arrow/commit/66d9a30a26e1659d9e992037339515e59a6ae518 We have a couple options: * Release from master * Release 0.11.0 + minimum patches to include the ARROW-3514 fix and any follow up patches to fix packaging There is the option to "not" release but it could cause confusion for people because PyPI does not allow replacing wheels; a new version number has to be created. What would folks like to do? Who can help with the RM duties? Since a 72 hour vote is a _should_ rather than _must_, we could reasonably close the release vote in < 72 hours and push out packages faster if it is scope limited to the zlib bug fix Thanks, Wes
