Thanks Krisztián, that sounds like a prudent approach. I agree with Antoine that it's not the worst thing if we have to revert and do a patch release after that handles the timezone issues properly, using the extra time to work out the details. Chances are we'll need to do a patch release anyway :)
Neal On Mon, Jul 20, 2020 at 9:45 AM Krisztián Szűcs <szucs.kriszt...@gmail.com> wrote: > Until Micah is working on compatibility flags in his patch, I'm going > to apply the reversion patch and cut RC2. > > Since RC2 will take at least 6-8 hours to build and three additional > days for voting we'll have enough time to sort this issue out and > decide to either cut RC3 including the timezone fix patch [1] or keep > RC2. > > [1]: https://github.com/apache/arrow/pull/7805 > > On Mon, Jul 20, 2020 at 4:48 PM Antoine Pitrou <anto...@python.org> wrote: > > > > > > If the release condition is for the regression to be fixed in less than > > 24 hours (less than 12 hours now?), I think we should simply revert the > > original PR and work on a fix more leisurely for 1.1.0 (or even 1.0.1). > > > > Unless it really causes havoc for Spark users, in which case a > > circumvention should be found. > > > > Regards > > > > Antoine. > > > > > > Le 20/07/2020 à 16:46, Krisztián Szűcs a écrit : > > > If I understand correctly we used to just store the timestamp and the > > > timezone if an explicit arrow type was passed during the python->arrow > > > conversion, but the timestamp values were not changed in any way. > > > Micah's current patch changes the python->arrow conversion behavior to > > > normalize all values to utc timestamps. > > > > > > While it's definitely an improvement over the previously ignored > > > timezones, I'm not sure that it won't cause unexpected regressions in > > > the users' codebases. > > > I'm still trying to better understand the issue and its compatibility > > > implications, but my intuition tells me that we should apply the > > > reversion instead and properly handle the datetime value conversions > > > in an upcoming minor release. > > > > > > Either way we should move this conversation to the pull request [1], > > > because the code snippets pasted here are hardly readable. > > > > > > [1]: https://github.com/apache/arrow/pull/7805 > > > > > > On Mon, Jul 20, 2020 at 9:40 AM Sutou Kouhei <k...@clear-code.com> > wrote: > > >> > > >> Done: > https://github.com/apache/arrow/pull/7805#issuecomment-660855376 > > >> > > >> We can use ...-3.8-... not ...-3.7-... because we don't have > > >> ...-3.7-... task in > > >> https://github.com/apache/arrow/blob/master/dev/tasks/tasks.yml. > > >> > > >> In < > cak7z5t8hqcsd3meg42cuzkscpjr3zndsvrjmm8vied0gzto...@mail.gmail.com> > > >> "Re: [VOTE] Release Apache Arrow 1.0.0 - RC1" on Mon, 20 Jul 2020 > 00:14:00 -0700, > > >> Micah Kornfield <emkornfi...@gmail.com> wrote: > > >> > > >>> FYI, I'm not sure if it is a permissions issue or I've done > something wrong > > >>> but github-actions does not seem to be responding to "@github-actions > > >>> <https://github.com/github-actions> crossbow submit > > >>> test-conda-python-3.7-spark-master" when I enter it. If someone > could kick > > >>> off the spark integration test I would be grateful. > > >>> > > >>> On Mon, Jul 20, 2020 at 12:09 AM Micah Kornfield < > emkornfi...@gmail.com> > > >>> wrote: > > >>> > > >>>> Thanks Bryan. I cherry-picked your change onto my change [1] which > now > > >>>> honors timezone aware datetime objects on ingestion. I've kicked > off the > > >>>> spark integration tests. > > >>>> > > >>>> If this change doesn't work I think the correct course of action is > to > > >>>> provide an environment variable in python to turn back to the old > behavior > > >>>> (ignoring timezones on conversion). I think honoring timezone > information > > >>>> where possible is a strict improvement but I agree we should give > users an > > >>>> option to not break if they wish to upgrade to the latest version. > I need > > >>>> to get some sleep but I will have another PR posted tomorrow > evening if the > > >>>> current one doesn't unblock the release. > > >>>> > > >>>> [1] https://github.com/apache/arrow/pull/7805 > > >>>> > > >>>> On Sun, Jul 19, 2020 at 10:50 PM Bryan Cutler <cutl...@gmail.com> > wrote: > > >>>> > > >>>>> I'd rather not see ARROW-9223 reverted, if possible. I will put up > my > > >>>>> hacked patch to Spark for this so we can test against it if > needed, and > > >>>>> could share my branch if anyone else wants to test it locally. > > >>>>> > > >>>>> On Sun, Jul 19, 2020 at 7:35 PM Micah Kornfield < > emkornfi...@gmail.com> > > >>>>> wrote: > > >>>>> > > >>>>>> I'll spend some time tonight on it and if I can't get round trip > working > > >>>>>> I'll handle reverting > > >>>>>> > > >>>>>> On Sunday, July 19, 2020, Wes McKinney <wesmck...@gmail.com> > wrote: > > >>>>>> > > >>>>>>> On Sun, Jul 19, 2020 at 7:33 PM Neal Richardson > > >>>>>>> <neal.p.richard...@gmail.com> wrote: > > >>>>>>>> > > >>>>>>>> It sounds like you may have identified a pyarrow bug, which > sounds > > >>>>> not > > >>>>>>>> good, though I don't know enough about the broader context to > know > > >>>>>>> whether > > >>>>>>>> this is (1) worse than 0.17 or (2) release blocking. I defer to > > >>>>> y'all > > >>>>>> who > > >>>>>>>> know better. > > >>>>>>>> > > >>>>>>>> If there are quirks in how Spark handles timezone-naive > timestamps, > > >>>>>>>> shouldn't the fix/workaround go in pyspark, not pyarrow? For > what > > >>>>> it's > > >>>>>>>> worth, I dealt with similar Spark timezone issues in R recently: > > >>>>>>>> https://github.com/sparklyr/sparklyr/issues/2439 I handled > with it > > >>>>> (in > > >>>>>>>> sparklyr, not the arrow R package) by always setting a timezone > when > > >>>>>>>> sending data to Spark. Not ideal but it made the numbers > "right". > > >>>>>>> > > >>>>>>> Since people are running this code in production we need to be > careful > > >>>>>>> about disrupting them. Unfortunately I'm at the limit of how > much time > > >>>>>>> I can spend on this, but releasing with ARROW-9223 as is (without > > >>>>>>> being partially or fully reverted) makes me deeply > uncomfortable. So I > > >>>>>>> hope the matter can be resolved. > > >>>>>>> > > >>>>>>>> Neal > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> On Sun, Jul 19, 2020 at 5:13 PM Wes McKinney < > wesmck...@gmail.com> > > >>>>>>> wrote: > > >>>>>>>> > > >>>>>>>>> Honestly I think reverting is the best option. This change > > >>>>> evidently > > >>>>>>>>> needs more time to "season" and perhaps this is motivation to > > >>>>> enhance > > >>>>>>>>> test coverage in a number of places. > > >>>>>>>>> > > >>>>>>>>> On Sun, Jul 19, 2020 at 7:11 PM Wes McKinney < > wesmck...@gmail.com > > >>>>>> > > >>>>>>> wrote: > > >>>>>>>>>> > > >>>>>>>>>> I am OK with any solution that doesn't delay the production of > > >>>>> the > > >>>>>>>>>> next RC by more than 24 hours > > >>>>>>>>>> > > >>>>>>>>>> On Sun, Jul 19, 2020 at 7:08 PM Micah Kornfield < > > >>>>>>> emkornfi...@gmail.com> > > >>>>>>>>> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>> If I read the example right it looks like constructing from > > >>>>>> python > > >>>>>>>>> types > > >>>>>>>>>>> isn't keeping timezones into in place? I can try make a > patch > > >>>>>> that > > >>>>>>>>> fixes > > >>>>>>>>>>> that tonight or would the preference be to revert my patch > > >>>>> (note > > >>>>>> I > > >>>>>>>>> think > > >>>>>>>>>>> another bug with a prior bug was fixed in my PR as well) > > >>>>>>>>>>> > > >>>>>>>>>>> -Micah > > >>>>>>>>>>> > > >>>>>>>>>>> On Sunday, July 19, 2020, Wes McKinney <wesmck...@gmail.com> > > >>>>>>> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>>> I think I see the problem now: > > >>>>>>>>>>>> > > >>>>>>>>>>>> In [40]: parr > > >>>>>>>>>>>> Out[40]: > > >>>>>>>>>>>> 0 {'f0': 1969-12-31 16:00:00-08:00} > > >>>>>>>>>>>> 1 {'f0': 1969-12-31 16:00:00.000001-08:00} > > >>>>>>>>>>>> 2 {'f0': 1969-12-31 16:00:00.000002-08:00} > > >>>>>>>>>>>> dtype: object > > >>>>>>>>>>>> > > >>>>>>>>>>>> In [41]: parr[0]['f0'] > > >>>>>>>>>>>> Out[41]: datetime.datetime(1969, 12, 31, 16, 0, > > >>>>>> tzinfo=<DstTzInfo > > >>>>>>>>>>>> 'America/Los_Angeles' PST-1 day, 16:00:00 STD>) > > >>>>>>>>>>>> > > >>>>>>>>>>>> In [42]: pa.array(parr) > > >>>>>>>>>>>> Out[42]: > > >>>>>>>>>>>> <pyarrow.lib.StructArray object at 0x7f0893706a60> > > >>>>>>>>>>>> -- is_valid: all not null > > >>>>>>>>>>>> -- child 0 type: timestamp[us] > > >>>>>>>>>>>> [ > > >>>>>>>>>>>> 1969-12-31 16:00:00.000000, > > >>>>>>>>>>>> 1969-12-31 16:00:00.000001, > > >>>>>>>>>>>> 1969-12-31 16:00:00.000002 > > >>>>>>>>>>>> ] > > >>>>>>>>>>>> > > >>>>>>>>>>>> In [43]: pa.array(parr).field(0).type > > >>>>>>>>>>>> Out[43]: TimestampType(timestamp[us]) > > >>>>>>>>>>>> > > >>>>>>>>>>>> On 0.17.1 > > >>>>>>>>>>>> > > >>>>>>>>>>>> In [8]: arr = pa.array([0, 1, 2], type=pa.timestamp('us', > > >>>>>>>>>>>> 'America/Los_Angeles')) > > >>>>>>>>>>>> > > >>>>>>>>>>>> In [9]: arr > > >>>>>>>>>>>> Out[9]: > > >>>>>>>>>>>> <pyarrow.lib.TimestampArray object at 0x7f9dede69d00> > > >>>>>>>>>>>> [ > > >>>>>>>>>>>> 1970-01-01 00:00:00.000000, > > >>>>>>>>>>>> 1970-01-01 00:00:00.000001, > > >>>>>>>>>>>> 1970-01-01 00:00:00.000002 > > >>>>>>>>>>>> ] > > >>>>>>>>>>>> > > >>>>>>>>>>>> In [10]: struct_arr = pa.StructArray.from_arrays([arr], > > >>>>>>> names=['f0']) > > >>>>>>>>>>>> > > >>>>>>>>>>>> In [11]: struct_arr > > >>>>>>>>>>>> Out[11]: > > >>>>>>>>>>>> <pyarrow.lib.StructArray object at 0x7f9ded0016e0> > > >>>>>>>>>>>> -- is_valid: all not null > > >>>>>>>>>>>> -- child 0 type: timestamp[us, tz=America/Los_Angeles] > > >>>>>>>>>>>> [ > > >>>>>>>>>>>> 1970-01-01 00:00:00.000000, > > >>>>>>>>>>>> 1970-01-01 00:00:00.000001, > > >>>>>>>>>>>> 1970-01-01 00:00:00.000002 > > >>>>>>>>>>>> ] > > >>>>>>>>>>>> > > >>>>>>>>>>>> In [12]: struct_arr.to_pandas() > > >>>>>>>>>>>> Out[12]: > > >>>>>>>>>>>> 0 {'f0': 1970-01-01 00:00:00} > > >>>>>>>>>>>> 1 {'f0': 1970-01-01 00:00:00.000001} > > >>>>>>>>>>>> 2 {'f0': 1970-01-01 00:00:00.000002} > > >>>>>>>>>>>> dtype: object > > >>>>>>>>>>>> > > >>>>>>>>>>>> In [13]: pa.array(struct_arr.to_pandas()) > > >>>>>>>>>>>> Out[13]: > > >>>>>>>>>>>> <pyarrow.lib.StructArray object at 0x7f9ded003210> > > >>>>>>>>>>>> -- is_valid: all not null > > >>>>>>>>>>>> -- child 0 type: timestamp[us] > > >>>>>>>>>>>> [ > > >>>>>>>>>>>> 1970-01-01 00:00:00.000000, > > >>>>>>>>>>>> 1970-01-01 00:00:00.000001, > > >>>>>>>>>>>> 1970-01-01 00:00:00.000002 > > >>>>>>>>>>>> ] > > >>>>>>>>>>>> > > >>>>>>>>>>>> In [14]: pa.array(struct_arr.to_pandas()).type > > >>>>>>>>>>>> Out[14]: StructType(struct<f0: timestamp[us]>) > > >>>>>>>>>>>> > > >>>>>>>>>>>> So while the time zone is getting stripped in both cases, > > >>>>> the > > >>>>>>> failure > > >>>>>>>>>>>> to round trip is a problem. If we are going to attach the > > >>>>> time > > >>>>>>> zone > > >>>>>>>>> in > > >>>>>>>>>>>> to_pandas() then we need to respect it when going the other > > >>>>>> way. > > >>>>>>>>>>>> > > >>>>>>>>>>>> This looks like a regression to me and so I'm inclined to > > >>>>>> revise > > >>>>>>> my > > >>>>>>>>>>>> vote on the release to -0/-1 > > >>>>>>>>>>>> > > >>>>>>>>>>>> On Sun, Jul 19, 2020 at 6:46 PM Wes McKinney < > > >>>>>>> wesmck...@gmail.com> > > >>>>>>>>> wrote: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Ah I forgot that this is a "feature" of nanosecond > > >>>>> timestamps > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> In [21]: arr = pa.array([0, 1, 2], type=pa.timestamp('us', > > >>>>>>>>>>>>> 'America/Los_Angeles')) > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> In [22]: struct_arr = pa.StructArray.from_arrays([arr], > > >>>>>>>>> names=['f0']) > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> In [23]: struct_arr.to_pandas() > > >>>>>>>>>>>>> Out[23]: > > >>>>>>>>>>>>> 0 {'f0': 1969-12-31 16:00:00-08:00} > > >>>>>>>>>>>>> 1 {'f0': 1969-12-31 16:00:00.000001-08:00} > > >>>>>>>>>>>>> 2 {'f0': 1969-12-31 16:00:00.000002-08:00} > > >>>>>>>>>>>>> dtype: object > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> So this is working as intended, such as it is > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> On Sun, Jul 19, 2020 at 6:40 PM Wes McKinney < > > >>>>>>> wesmck...@gmail.com> > > >>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> There seems to be other broken StructArray stuff > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> In [14]: arr = pa.array([0, 1, 2], > > >>>>> type=pa.timestamp('ns', > > >>>>>>>>>>>>>> 'America/Los_Angeles')) > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> In [15]: struct_arr = pa.StructArray.from_arrays([arr], > > >>>>>>>>> names=['f0']) > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> In [16]: struct_arr > > >>>>>>>>>>>>>> Out[16]: > > >>>>>>>>>>>>>> <pyarrow.lib.StructArray object at 0x7f089370f590> > > >>>>>>>>>>>>>> -- is_valid: all not null > > >>>>>>>>>>>>>> -- child 0 type: timestamp[ns, tz=America/Los_Angeles] > > >>>>>>>>>>>>>> [ > > >>>>>>>>>>>>>> 1970-01-01 00:00:00.000000000, > > >>>>>>>>>>>>>> 1970-01-01 00:00:00.000000001, > > >>>>>>>>>>>>>> 1970-01-01 00:00:00.000000002 > > >>>>>>>>>>>>>> ] > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> In [17]: struct_arr.to_pandas() > > >>>>>>>>>>>>>> Out[17]: > > >>>>>>>>>>>>>> 0 {'f0': 0} > > >>>>>>>>>>>>>> 1 {'f0': 1} > > >>>>>>>>>>>>>> 2 {'f0': 2} > > >>>>>>>>>>>>>> dtype: object > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> All in all it appears that this part of the project > > >>>>> needs > > >>>>>>> some > > >>>>>>>>> TLC > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> On Sun, Jul 19, 2020 at 6:16 PM Wes McKinney < > > >>>>>>>>> wesmck...@gmail.com> > > >>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Well, the problem is that time zones are really > > >>>>> finicky > > >>>>>>>>> comparing > > >>>>>>>>>>>>>>> Spark (which uses a localtime interpretation of > > >>>>>> timestamps > > >>>>>>>>> without > > >>>>>>>>>>>>>>> time zone) and Arrow (which has naive timestamps -- a > > >>>>>>> concept > > >>>>>>>>> similar > > >>>>>>>>>>>>>>> but different from the SQL concept TIMESTAMP WITHOUT > > >>>>> TIME > > >>>>>>> ZONE > > >>>>>>>>> -- and > > >>>>>>>>>>>>>>> tz-aware timestamps). So somewhere there is a time > > >>>>> zone > > >>>>>>> being > > >>>>>>>>>>>> stripped > > >>>>>>>>>>>>>>> or applied/localized which may result in the > > >>>>> transferred > > >>>>>>> data > > >>>>>>>>> to/from > > >>>>>>>>>>>>>>> Spark being shifted by the time zone offset. I think > > >>>>> it's > > >>>>>>>>> important > > >>>>>>>>>>>>>>> that we determine what the problem is -- if it's a > > >>>>>> problem > > >>>>>>>>> that has > > >>>>>>>>>>>> to > > >>>>>>>>>>>>>>> be fixed in Arrow (and it's not clear to me that it > > >>>>> is) > > >>>>>>> it's > > >>>>>>>>> worth > > >>>>>>>>>>>>>>> spending some time to understand what's going on to > > >>>>> avoid > > >>>>>>> the > > >>>>>>>>>>>>>>> possibility of patch release on account of this. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> On Sun, Jul 19, 2020 at 6:12 PM Neal Richardson > > >>>>>>>>>>>>>>> <neal.p.richard...@gmail.com> wrote: > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> If it’s a display problem, should it block the > > >>>>> release? > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Sent from my iPhone > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> On Jul 19, 2020, at 3:57 PM, Wes McKinney < > > >>>>>>>>> wesmck...@gmail.com> > > >>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> I opened https://issues.apache.org/ > > >>>>>>> jira/browse/ARROW-9525 > > >>>>>>>>>>>> about the > > >>>>>>>>>>>>>>>>> display problem. My guess is that there are other > > >>>>>>> problems > > >>>>>>>>>>>> lurking > > >>>>>>>>>>>>>>>>> here > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> On Sun, Jul 19, 2020 at 5:54 PM Wes McKinney < > > >>>>>>>>>>>> wesmck...@gmail.com> wrote: > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> hi Bryan, > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> This is a display bug > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> In [6]: arr = pa.array([0, 1, 2], > > >>>>>>> type=pa.timestamp('ns', > > >>>>>>>>>>>>>>>>>> 'America/Los_Angeles')) > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> In [7]: arr.view('int64') > > >>>>>>>>>>>>>>>>>> Out[7]: > > >>>>>>>>>>>>>>>>>> <pyarrow.lib.Int64Array object at 0x7fd1b8aaef30> > > >>>>>>>>>>>>>>>>>> [ > > >>>>>>>>>>>>>>>>>> 0, > > >>>>>>>>>>>>>>>>>> 1, > > >>>>>>>>>>>>>>>>>> 2 > > >>>>>>>>>>>>>>>>>> ] > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> In [8]: arr > > >>>>>>>>>>>>>>>>>> Out[8]: > > >>>>>>>>>>>>>>>>>> <pyarrow.lib.TimestampArray object at > > >>>>>> 0x7fd1b8aae6e0> > > >>>>>>>>>>>>>>>>>> [ > > >>>>>>>>>>>>>>>>>> 1970-01-01 00:00:00.000000000, > > >>>>>>>>>>>>>>>>>> 1970-01-01 00:00:00.000000001, > > >>>>>>>>>>>>>>>>>> 1970-01-01 00:00:00.000000002 > > >>>>>>>>>>>>>>>>>> ] > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> In [9]: arr.to_pandas() > > >>>>>>>>>>>>>>>>>> Out[9]: > > >>>>>>>>>>>>>>>>>> 0 1969-12-31 16:00:00-08:00 > > >>>>>>>>>>>>>>>>>> 1 1969-12-31 16:00:00.000000001-08:00 > > >>>>>>>>>>>>>>>>>> 2 1969-12-31 16:00:00.000000002-08:00 > > >>>>>>>>>>>>>>>>>> dtype: datetime64[ns, America/Los_Angeles] > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> the repr of TimestampArray doesn't take into > > >>>>> account > > >>>>>>> the > > >>>>>>>>>>>> timezone > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> In [10]: arr[0] > > >>>>>>>>>>>>>>>>>> Out[10]: <pyarrow.TimestampScalar: > > >>>>>>> Timestamp('1969-12-31 > > >>>>>>>>>>>>>>>>>> 16:00:00-0800', tz='America/Los_Angeles')> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> So if it's incorrect, the problem is happening > > >>>>>>> somewhere > > >>>>>>>>> before > > >>>>>>>>>>>> or > > >>>>>>>>>>>>>>>>>> while the StructArray is being created. If I had > > >>>>> to > > >>>>>>> guess > > >>>>>>>>> it's > > >>>>>>>>>>>> caused > > >>>>>>>>>>>>>>>>>> by the tzinfo of the datetime.datetime values not > > >>>>>>> being > > >>>>>>>>> handled > > >>>>>>>>>>>> in the > > >>>>>>>>>>>>>>>>>> way that they were before > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> On Sun, Jul 19, 2020 at 5:19 PM Wes McKinney < > > >>>>>>>>>>>> wesmck...@gmail.com> wrote: > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> Well this is not good and pretty disappointing > > >>>>>> given > > >>>>>>>>> that we > > >>>>>>>>>>>> had nearly a month to sort through the implications of > > >>>>> Micah’s > > >>>>>>>>> patch. We > > >>>>>>>>>>>> should try to resolve this ASAP > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> On Sun, Jul 19, 2020 at 5:10 PM Bryan Cutler < > > >>>>>>>>>>>> cutl...@gmail.com> wrote: > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> +0 (non-binding) > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> I ran verification script for binaries and then > > >>>>>>> source, > > >>>>>>>>> as > > >>>>>>>>>>>> below, and both > > >>>>>>>>>>>>>>>>>>>> look good > > >>>>>>>>>>>>>>>>>>>> ARROW_TMPDIR=/tmp/arrow-test TEST_DEFAULT=0 > > >>>>>>>>> TEST_SOURCE=1 > > >>>>>>>>>>>> TEST_CPP=1 > > >>>>>>>>>>>>>>>>>>>> TEST_PYTHON=1 TEST_JAVA=1 > > >>>>> TEST_INTEGRATION_CPP=1 > > >>>>>>>>>>>> TEST_INTEGRATION_JAVA=1 > > >>>>>>>>>>>>>>>>>>>> dev/release/verify-release-candidate.sh source > > >>>>>>> 1.0.0 1 > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> I tried to patch Spark locally to verify the > > >>>>>> recent > > >>>>>>>>> change in > > >>>>>>>>>>>> nested > > >>>>>>>>>>>>>>>>>>>> timestamps and was not able to get things > > >>>>> working > > >>>>>>> quite > > >>>>>>>>>>>> right, but I'm not > > >>>>>>>>>>>>>>>>>>>> sure if the problem is in Spark, Arrow or my > > >>>>>> patch - > > >>>>>>>>> hence my > > >>>>>>>>>>>> vote of +0. > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> Here is what I'm seeing > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> ``` > > >>>>>>>>>>>>>>>>>>>> (Input as datetime) > > >>>>>>>>>>>>>>>>>>>> datetime.datetime(2018, 3, 10, 0, 0) > > >>>>>>>>>>>>>>>>>>>> datetime.datetime(2018, 3, 15, 0, 0) > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> (Struct Array) > > >>>>>>>>>>>>>>>>>>>> -- is_valid: all not null > > >>>>>>>>>>>>>>>>>>>> -- child 0 type: timestamp[us, > > >>>>>>> tz=America/Los_Angeles] > > >>>>>>>>>>>>>>>>>>>> [ > > >>>>>>>>>>>>>>>>>>>> 2018-03-10 00:00:00.000000, > > >>>>>>>>>>>>>>>>>>>> 2018-03-10 00:00:00.000000 > > >>>>>>>>>>>>>>>>>>>> ] > > >>>>>>>>>>>>>>>>>>>> -- child 1 type: timestamp[us, > > >>>>>>> tz=America/Los_Angeles] > > >>>>>>>>>>>>>>>>>>>> [ > > >>>>>>>>>>>>>>>>>>>> 2018-03-15 00:00:00.000000, > > >>>>>>>>>>>>>>>>>>>> 2018-03-15 00:00:00.000000 > > >>>>>>>>>>>>>>>>>>>> ] > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> (Flattened Arrays) > > >>>>>>>>>>>>>>>>>>>> types [TimestampType(timestamp[us, > > >>>>>>>>> tz=America/Los_Angeles]), > > >>>>>>>>>>>>>>>>>>>> TimestampType(timestamp[us, > > >>>>>>> tz=America/Los_Angeles])] > > >>>>>>>>>>>>>>>>>>>> [<pyarrow.lib.TimestampArray object at > > >>>>>>> 0x7ffbbd88f520> > > >>>>>>>>>>>>>>>>>>>> [ > > >>>>>>>>>>>>>>>>>>>> 2018-03-10 00:00:00.000000, > > >>>>>>>>>>>>>>>>>>>> 2018-03-10 00:00:00.000000 > > >>>>>>>>>>>>>>>>>>>> ], <pyarrow.lib.TimestampArray object at > > >>>>>>> 0x7ffba958be50> > > >>>>>>>>>>>>>>>>>>>> [ > > >>>>>>>>>>>>>>>>>>>> 2018-03-15 00:00:00.000000, > > >>>>>>>>>>>>>>>>>>>> 2018-03-15 00:00:00.000000 > > >>>>>>>>>>>>>>>>>>>> ]] > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> (Pandas Conversion) > > >>>>>>>>>>>>>>>>>>>> [ > > >>>>>>>>>>>>>>>>>>>> 0 2018-03-09 16:00:00-08:00 > > >>>>>>>>>>>>>>>>>>>> 1 2018-03-09 16:00:00-08:00 > > >>>>>>>>>>>>>>>>>>>> dtype: datetime64[ns, America/Los_Angeles], > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> 0 2018-03-14 17:00:00-07:00 > > >>>>>>>>>>>>>>>>>>>> 1 2018-03-14 17:00:00-07:00 > > >>>>>>>>>>>>>>>>>>>> dtype: datetime64[ns, America/Los_Angeles]] > > >>>>>>>>>>>>>>>>>>>> ``` > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> Based on output of existing a correct timestamp > > >>>>>>> udf, it > > >>>>>>>>> looks > > >>>>>>>>>>>> like the > > >>>>>>>>>>>>>>>>>>>> pyarrow Struct Array values are wrong and > > >>>>> that's > > >>>>>>> carried > > >>>>>>>>>>>> through the > > >>>>>>>>>>>>>>>>>>>> flattened arrays, causing the Pandas values to > > >>>>>> have > > >>>>>>> a > > >>>>>>>>>>>> negative offset. > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> Here is output from a working udf with > > >>>>> timestamp, > > >>>>>>> the > > >>>>>>>>> pyarrow > > >>>>>>>>>>>> Array > > >>>>>>>>>>>>>>>>>>>> displays in UTC time, I believe. > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> ``` > > >>>>>>>>>>>>>>>>>>>> (Timestamp Array) > > >>>>>>>>>>>>>>>>>>>> type timestamp[us, tz=America/Los_Angeles] > > >>>>>>>>>>>>>>>>>>>> [ > > >>>>>>>>>>>>>>>>>>>> [ > > >>>>>>>>>>>>>>>>>>>> 1969-01-01 09:01:01.000000 > > >>>>>>>>>>>>>>>>>>>> ] > > >>>>>>>>>>>>>>>>>>>> ] > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> (Pandas Conversion) > > >>>>>>>>>>>>>>>>>>>> 0 1969-01-01 01:01:01-08:00 > > >>>>>>>>>>>>>>>>>>>> Name: _0, dtype: datetime64[ns, > > >>>>>> America/Los_Angeles] > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> (Timezone Localized) > > >>>>>>>>>>>>>>>>>>>> 0 1969-01-01 01:01:01 > > >>>>>>>>>>>>>>>>>>>> Name: _0, dtype: datetime64[ns] > > >>>>>>>>>>>>>>>>>>>> ``` > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> I'll have to dig in further at another time and > > >>>>>>> debug > > >>>>>>>>> where > > >>>>>>>>>>>> the values go > > >>>>>>>>>>>>>>>>>>>> wrong. > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> On Sat, Jul 18, 2020 at 9:51 PM Micah > > >>>>> Kornfield < > > >>>>>>>>>>>> emkornfi...@gmail.com> > > >>>>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> +1 (binding) > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> Ran wheel and binary tests on ubuntu 19.04 > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> On Fri, Jul 17, 2020 at 2:25 PM Neal > > >>>>> Richardson < > > >>>>>>>>>>>>>>>>>>>>> neal.p.richard...@gmail.com> > > >>>>>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> +1 (binding) > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> In addition to the usual verification on > > >>>>>>>>>>>>>>>>>>>>>> https://github.com/apache/arrow/pull/7787, > > >>>>> I've > > >>>>>>>>>>>> successfully staged the > > >>>>>>>>>>>>>>>>>>>>> R > > >>>>>>>>>>>>>>>>>>>>>> binary artifacts on Windows ( > > >>>>>>>>>>>>>>>>>>>>>> https://github.com/r-windows/ > > >>>>>>> rtools-packages/pull/126 > > >>>>>>>>> ), > > >>>>>>>>>>>> macOS ( > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>> https://github.com/autobrew/homebrew-core/pull/12 > > >>>>>>> ), > > >>>>>>>>> and > > >>>>>>>>>>>> Linux ( > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>> https://github.com/ursa-labs/arrow-r-nightly/actions/runs/ > > >>>>>>>>>>>> 172977277) > > >>>>>>>>>>>>>>>>>>>>> using > > >>>>>>>>>>>>>>>>>>>>>> the release candidate. > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> And I agree with the judgment about skipping > > >>>>> a > > >>>>>> JS > > >>>>>>>>> release > > >>>>>>>>>>>> artifact. Looks > > >>>>>>>>>>>>>>>>>>>>>> like there hasn't been a code change since > > >>>>>>> October so > > >>>>>>>>>>>> there's no point. > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> Neal > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 17, 2020 at 10:37 AM Wes > > >>>>> McKinney < > > >>>>>>>>>>>> wesmck...@gmail.com> > > >>>>>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> I see the JS failures as well. I think it > > >>>>> is a > > >>>>>>>>> failure > > >>>>>>>>>>>> localized to > > >>>>>>>>>>>>>>>>>>>>>>> newer Node versions since our JavaScript CI > > >>>>>> works > > >>>>>>>>> fine. I > > >>>>>>>>>>>> don't think > > >>>>>>>>>>>>>>>>>>>>>>> it should block the release given the lack > > >>>>> of > > >>>>>>>>> development > > >>>>>>>>>>>> activity in > > >>>>>>>>>>>>>>>>>>>>>>> JavaScript [1] -- if any JS devs are > > >>>>> concerned > > >>>>>>> about > > >>>>>>>>>>>> publishing an > > >>>>>>>>>>>>>>>>>>>>>>> artifact then we can skip pushing it to NPM > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> @Ryan it seems it may be something > > >>>>> environment > > >>>>>>>>> related on > > >>>>>>>>>>>> your > > >>>>>>>>>>>>>>>>>>>>>>> machine, I'm on Ubuntu 18.04 and have not > > >>>>> seen > > >>>>>>> this. > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> On > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>> * Python 3.8 wheel's tests are failed. > > >>>>> 3.5, > > >>>>>> 3.6 > > >>>>>>>>> and 3.7 > > >>>>>>>>>>>>>>>>>>>>>>>> are passed. It seems that -larrow and > > >>>>>>>>> -larrow_python > > >>>>>>>>>>>> for > > >>>>>>>>>>>>>>>>>>>>>>>> Cython are failed. > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> I suspect this is related to > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> https://github.com/apache/arrow/commit/ > > >>>>>>>>>>>> 120c21f4bf66d2901b3a353a1f67bac3c3355924#diff- > > >>>>>>>>>>>> 0f69784b44040448d17d0e4e8a641fe8 > > >>>>>>>>>>>>>>>>>>>>>>> , > > >>>>>>>>>>>>>>>>>>>>>>> but I don't think it's a blocking issue > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> [1]: > > >>>>>>>>> https://github.com/apache/arrow/commits/master/js > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 17, 2020 at 9:42 AM Ryan Murray > > >>>>> < > > >>>>>>>>>>>> rym...@dremio.com> wrote: > > >>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>> I've tested Java and it looks good. However > > >>>>>> the > > >>>>>>>>> verify > > >>>>>>>>>>>> script keeps > > >>>>>>>>>>>>>>>>>>>>> on > > >>>>>>>>>>>>>>>>>>>>>>>> bailing with protobuf related errors: > > >>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>> 'cpp/build/orc_ep-prefix/src/orc_ep-build/c++/src/orc_ > > >>>>>>>>>>>> proto.pb.cc' > > >>>>>>>>>>>>>>>>>>>>> and > > >>>>>>>>>>>>>>>>>>>>>>>> friends cant find protobuf definitions. A > > >>>>> bit > > >>>>>>> odd as > > >>>>>>>>>>>> cmake can see > > >>>>>>>>>>>>>>>>>>>>>>> protobuf > > >>>>>>>>>>>>>>>>>>>>>>>> headers and builds directly off master work > > >>>>>> just > > >>>>>>>>> fine. > > >>>>>>>>>>>> Has anyone > > >>>>>>>>>>>>>>>>>>>>> else > > >>>>>>>>>>>>>>>>>>>>>>>> experienced this? I am on ubutnu 18.04 > > >>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 17, 2020 at 10:49 AM Antoine > > >>>>>> Pitrou > > >>>>>>> < > > >>>>>>>>>>>> anto...@python.org> > > >>>>>>>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>> +1 (binding). I tested on Ubuntu 18.04. > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>> * Wheels verification went fine. > > >>>>>>>>>>>>>>>>>>>>>>>>> * Source verification went fine with CUDA > > >>>>>>> enabled > > >>>>>>>>> and > > >>>>>>>>>>>>>>>>>>>>>>>>> TEST_INTEGRATION_JS=0 TEST_JS=0. > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>> I didn't test the binaries. > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Regards > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Antoine. > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>> Le 17/07/2020 à 03:41, Krisztián Szűcs a > > >>>>>> écrit > > >>>>>>> : > > >>>>>>>>>>>>>>>>>>>>>>>>>> Hi, > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> I would like to propose the second > > >>>>> release > > >>>>>>>>> candidate > > >>>>>>>>>>>> (RC1) of > > >>>>>>>>>>>>>>>>>>>>>> Apache > > >>>>>>>>>>>>>>>>>>>>>>>>>> Arrow version 1.0.0. > > >>>>>>>>>>>>>>>>>>>>>>>>>> This is a major release consisting of 826 > > >>>>>>>>> resolved JIRA > > >>>>>>>>>>>>>>>>>>>>> issues[1]. > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> The verification of the first release > > >>>>>>> candidate > > >>>>>>>>> (RC0) > > >>>>>>>>>>>> has failed > > >>>>>>>>>>>>>>>>>>>>>>> [0], and > > >>>>>>>>>>>>>>>>>>>>>>>>>> the packaging scripts were unable to > > >>>>> produce > > >>>>>>> two > > >>>>>>>>>>>> wheels. Compared > > >>>>>>>>>>>>>>>>>>>>>>>>>> to RC0 this release candidate includes > > >>>>>>> additional > > >>>>>>>>>>>> patches for the > > >>>>>>>>>>>>>>>>>>>>>>>>>> following bugs: ARROW-9506, ARROW-9504, > > >>>>>>>>> ARROW-9497, > > >>>>>>>>>>>>>>>>>>>>>>>>>> ARROW-9500, ARROW-9499. > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> This release candidate is based on > > >>>>> commit: > > >>>>>>>>>>>>>>>>>>>>>>>>>> bc0649541859095ee77d03a7b891ea8d6e2fd641 > > >>>>> [2] > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> The source release rc1 is hosted at [3]. > > >>>>>>>>>>>>>>>>>>>>>>>>>> The binary artifacts are hosted at > > >>>>>>> [4][5][6][7]. > > >>>>>>>>>>>>>>>>>>>>>>>>>> The changelog is located at [8]. > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> Please download, verify checksums and > > >>>>>>> signatures, > > >>>>>>>>> run > > >>>>>>>>>>>> the unit > > >>>>>>>>>>>>>>>>>>>>>> tests, > > >>>>>>>>>>>>>>>>>>>>>>>>>> and vote on the release. See [9] for how > > >>>>> to > > >>>>>>>>> validate a > > >>>>>>>>>>>> release > > >>>>>>>>>>>>>>>>>>>>>>> candidate. > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> The vote will be open for at least 72 > > >>>>> hours. > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> [ ] +1 Release this as Apache Arrow 1.0.0 > > >>>>>>>>>>>>>>>>>>>>>>>>>> [ ] +0 > > >>>>>>>>>>>>>>>>>>>>>>>>>> [ ] -1 Do not release this as Apache > > >>>>> Arrow > > >>>>>>> 1.0.0 > > >>>>>>>>>>>> because... > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>>> [0]: > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>> https://github.com/apache/arrow/pull/7778#issuecomment- > > >>>>>>>>>>>> 659065370 > > >>>>>>>>>>>>>>>>>>>>>>>>>> [1]: > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/ > > >>>>>>> jira/issues/?jql=project%20% > > >>>>>>>>>>>> 3D%20ARROW%20AND%20status%20in%20%28Resolved%2C% > > >>>>>>> 20Closed%29%20AND% > > >>>>>>>>>>>> 20fixVersion%20%3D%201.0.0 > > >>>>>>>>>>>>>>>>>>>>>>>>>> [2]: > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> https://github.com/apache/arrow/tree/ > > >>>>>>>>>>>> bc0649541859095ee77d03a7b891ea8d6e2fd641 > > >>>>>>>>>>>>>>>>>>>>>>>>>> [3]: > > >>>>>>>>>>>>>>>>>>>>>>> https://dist.apache.org/repos/ > > >>>>>>>>>>>> dist/dev/arrow/apache-arrow-1.0.0-rc1 > > >>>>>>>>>>>>>>>>>>>>>>>>>> [4]: https://bintray.com/apache/ > > >>>>>>>>>>>> arrow/centos-rc/1.0.0-rc1 > > >>>>>>>>>>>>>>>>>>>>>>>>>> [5]: https://bintray.com/apache/ > > >>>>>>>>>>>> arrow/debian-rc/1.0.0-rc1 > > >>>>>>>>>>>>>>>>>>>>>>>>>> [6]: https://bintray.com/apache/ > > >>>>>>>>>>>> arrow/python-rc/1.0.0-rc1 > > >>>>>>>>>>>>>>>>>>>>>>>>>> [7]: https://bintray.com/apache/ > > >>>>>>>>>>>> arrow/ubuntu-rc/1.0.0-rc1 > > >>>>>>>>>>>>>>>>>>>>>>>>>> [8]: > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> https://github.com/apache/arrow/blob/ > > >>>>>>>>>>>> bc0649541859095ee77d03a7b891ea8d6e2fd641/CHANGELOG.md > > >>>>>>>>>>>>>>>>>>>>>>>>>> [9]: > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> https://cwiki.apache.org/ > > >>>>>>> confluence/display/ARROW/How+ > > >>>>>>>>>>>> to+Verify+Release+Candidates > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > >>>> >
