[RESULT][VOTE][Julia] Release Apache Arrow Julia 2.4.1 RC0

2022-11-17 Thread Sutou Kouhei 
Hi,

The vote carries with 4 +1 binding votes and 2 +1
non-binding votes.

I'll publish this release to
https://dist.apache.org/repos/dist/release/arrow/ .

Thanks,
-- 
kou


In <20221117.082136.809288014636861866@clear-code.com>
  "[VOTE][Julia] Release Apache Arrow Julia 2.4.1 RC0" on Thu, 17 Nov 2022 
08:21:36 +0900 (JST),
  Sutou Kouhei  wrote:

> Hi,
> 
> I would like to propose the following release candidate (RC0) of
> Apache Arrow Julia version 2.4.1.
> 
> This release candidate is based on commit:
> 23258f12bb4b28eb3846d0d3a91a54e2628254d1 [1]
> 
> The source release rc0 is hosted at [2].
> 
> Please download, verify checksums and signatures, run the unit tests,
> and vote on the release. See [3] for how to validate a release candidate.
> 
> The vote will be open for at least 24 hours.
> 
> [ ] +1 Release this as Apache Arrow Julia 2.4.1
> [ ] +0
> [ ] -1 Do not release this as Apache Arrow Julia 2.4.1 because...
> 
> [1]: 
> https://github.com/apache/arrow-julia/tree/23258f12bb4b28eb3846d0d3a91a54e2628254d1
> [2]: 
> https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-julia-2.4.1-rc0/
> [3]: 
> https://github.com/apache/arrow-julia/blob/main/dev/release/README.md#verify
> 
> Thanks,
> -- 
> kou

[DISCUSS] JSON Canonical Extension Type

2022-11-17 Thread Pradeep Gollakota 
Hi folks!

I put together this specification for canonicalizing the JSON type in Arrow.

## Introduction
JSON is a widely used text based data interchange format. There are many
use cases where a user has a column whose contents are a JSON encoded
string. BigQuery's [JSON Type][1] and Parquet’s [JSON Logical Type][2] are
two such examples.

The JSON specification is defined in [RFC-8259][3]. However, many of the
most popular parsers support non standard extensions. Examples of non
standard extensions to JSON include comments, unquoted keys, trailing
commas, etc.

## Extension Specification
* The name of the extension is `arrow.json`
* The storage type of the extension is `utf8`
* The extension type has no parameters
* The metadata MUST be either empty or a valid JSON object
- There is no canonical metadata
- Implementations MAY include implementation-specific metadata by using
a namespaced key. For example `{"google.bigquery": {"my": "metadata"}}`
* Implementations...
- MUST produce valid UTF-8 encoded text
- SHOULD produce valid standard JSON
- MAY produce valid non-standard JSON
- MUST support parsing standard JSON
- MAY support parsing non standard JSON
- SHOULD pass through contents that they do not understand

## Forward compatibility
In the future we might allow this logical type to annotate a byte storage
type with a different text encoding.  Implementations consuming JSON
logical types should verify this.

[1]:
https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#json_type
[2]:
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#json
[3]: https://datatracker.ietf.org/doc/html/rfc8259

Re: [VOTE] Release Apache Arrow 10.0.1 - RC0

2022-11-17 Thread Raúl Cumplido 
Hi,

+1

I've run:

TEST_DEFAULT=0 TEST_SOURCE=1 dev/release/verify-release-candidate.sh 10.0.1
0
TEST_DEFAULT=0 TEST_WHEELS=1 dev/release/verify-release-candidate.sh 10.0.1
0
TEST_DEFAULT=0 TEST_BINARIES=1 dev/release/verify-release-candidate.sh
10.0.1 0

For the yum verification to succeed I had to apply as suggested by Kou this
verification fix:
https://github.com/apache/arrow/pull/14661


Tested with:
  * Python 3.10.6 and Python 3.11.0
  * gcc version 11.3.0 (Ubuntu 11.3.0-1ubuntu1~22.04)
  * NVIDIA CUDA compiler cuda_11.5.r11.5/compiler.30672275_0
  * openjdk "17.0.5" 2022-10-18
  * ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux-gnu]
  * Ubuntu 22.04 LTS

Thanks,
Raúl


El jue, 17 nov 2022 a las 21:21, Sutou Kouhei ()
escribió:

> Hi,
>
> > I had to skip debian:bookworm because of an apparent
> > upstream issue where "lsb_release" gives "No LSB modules
> > are available".
>
> "docker pull debian:bookworm" will fix it.
>
> > I skipped YUM verification because the
> > gobject-introspection gem failed to build.
>
> https://github.com/apache/arrow/pull/14661 will fix it.
>
>
> Thanks,
> --
> kou
>
> In <81a43c88-7260-4714-805c-3c79d7ece...@app.fastmail.com>
>   "Re: [VOTE] Release Apache Arrow 10.0.1 - RC0" on Thu, 17 Nov 2022
> 11:58:30 -0500,
>   "David Li"  wrote:
>
> > +1 (binding), AMD64/Linux
> >
> > I had to skip debian:bookworm because of an apparent upstream issue
> where "lsb_release" gives "No LSB modules are available".  I skipped YUM
> verification because the gobject-introspection gem failed to build.
> >
> > On Thu, Nov 17, 2022, at 09:45, Rok Mihevc wrote:
> >> +1
> >>
> >> Passed on M1 with:
> >> TEST_DEFAULT=0 TEST_SOURCE=1 TEST_PYTHON=0 TEST_GLIB=0 TEST_RUBY=0
> >> dev/release/verify-release-candidate.sh 10.0.1 0
> >>
> >>
> >> With TEST_PYTHON=1 it reported a CMake issue:
> >>
> >> -- ArrowPythonFlight version: 10.0.1
> >>
> >> -- Found the ArrowPythonFlight shared library:
> >>
> /private/var/folders/bc/h5_8w_395x32tv7b7f9lnhz4gn/T/arrow-10.0.1.X.BpuqwhAp/apache-arrow-10.0.1/python/build/dist/lib/libarrow_python_flight.1000.1.0.dylib
> >>
> >> -- Found the ArrowPythonFlight import library:
> >> ARROW_PYTHON_FLIGHT_IMPORT_LIB-NOTFOUND
> >>
> >> -- Found the ArrowPythonFlight static library:
> >>
> /private/var/folders/bc/h5_8w_395x32tv7b7f9lnhz4gn/T/arrow-10.0.1.X.BpuqwhAp/apache-arrow-10.0.1/python/build/dist/lib/libarrow_python_flight.a
> >>
> >> CMake Error at CMakeLists.txt:553 (find_package):
> >>
> >>   By not providing "FindArrowSubstrait.cmake" in CMAKE_MODULE_PATH this
> >>
> >>   project has asked CMake to find a package configuration file provided
> by
> >>
> >>   "ArrowSubstrait", but CMake did not find one.
> >>
> >>
> >>   Could not find a package configuration file provided by
> "ArrowSubstrait"
> >>
> >>   with any of the following names:
> >>
> >>
> >> ArrowSubstraitConfig.cmake
> >>
> >> arrowsubstrait-config.cmake
> >>
> >>
> >>   Add the installation prefix of "ArrowSubstrait" to CMAKE_PREFIX_PATH
> or
> >> set
> >>
> >>   "ArrowSubstrait_DIR" to a directory containing one of the above
> files.  If
> >>
> >>   "ArrowSubstrait" provides a separate development package or SDK, be
> sure
> >> it
> >>
> >>   has been installed.
> >>
> >>
> >>
> >> With TEST_RUBY=1 it failed with:
> >>
> >>
> >> Installing gobject-introspection 4.0.3 with native extensions
> >>
> >> Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
> >>
> >>
> >> Both seem more related to my specific environment issues than anything
> else.
> >>
> >>
> >> Rok
> >>
> >> On Thu, Nov 17, 2022 at 8:52 AM Sutou Kouhei 
> wrote:
> >>
> >>> +1
> >>>
> >>> I ran the followings on Debian GNU/Linux sid:
> >>>
> >>>   * TEST_DEFAULT=0 \
> >>>   TEST_SOURCE=1 \
> >>>   LANG=C \
> >>>   TZ=UTC \
> >>>   CUDAToolkit_ROOT=/usr \
> >>>   ARROW_CMAKE_OPTIONS="-DBoost_NO_BOOST_CMAKE=ON
> >>> -Dxsimd_SOURCE=BUNDLED" \
> >>>   dev/release/verify-release-candidate.sh 10.0.1 0
> >>>
> >>>   * TEST_DEFAULT=0 \
> >>>   TEST_APT=1 \
> >>>   LANG=C \
> >>>   dev/release/verify-release-candidate.sh 10.0.1 0
> >>>
> >>>   * TEST_DEFAULT=0 \
> >>>   TEST_BINARY=1 \
> >>>   LANG=C \
> >>>   dev/release/verify-release-candidate.sh 10.0.1 0
> >>>
> >>>   * TEST_DEFAULT=0 \
> >>>   TEST_JARS=1 \
> >>>   LANG=C \
> >>>   dev/release/verify-release-candidate.sh 10.0.1 0
> >>>
> >>>   * TEST_DEFAULT=0 \
> >>>   TEST_WHEELS=1 \
> >>>   LANG=C \
> >>>   dev/release/verify-release-candidate.sh 10.0.1 0
> >>>
> >>>   * TEST_DEFAULT=0 \
> >>>   TEST_YUM=1 \
> >>>   LANG=C \
> >>>   dev/release/verify-release-candidate.sh 10.0.1 0
> >>>
> >>> with:
> >>>
> >>>   * .NET SDK (6.0.402)
> >>>   * Python 3.10.8
> >>>   * gcc (Debian 12.2.0-9) 12.2.0
> >>>   * nvidia-cuda-dev 11.6.2-2
> >>>   * openjdk version "18.0.2-ea" 2022-07-19
> >>>   * ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux-gnu]
>

Re: [VOTE] Release Apache Arrow 10.0.1 - RC0

2022-11-17 Thread Sutou Kouhei 
Hi,

> I had to skip debian:bookworm because of an apparent
> upstream issue where "lsb_release" gives "No LSB modules
> are available".

"docker pull debian:bookworm" will fix it.

> I skipped YUM verification because the
> gobject-introspection gem failed to build.

https://github.com/apache/arrow/pull/14661 will fix it.


Thanks,
-- 
kou

In <81a43c88-7260-4714-805c-3c79d7ece...@app.fastmail.com>
  "Re: [VOTE] Release Apache Arrow 10.0.1 - RC0" on Thu, 17 Nov 2022 11:58:30 
-0500,
  "David Li"  wrote:

> +1 (binding), AMD64/Linux
> 
> I had to skip debian:bookworm because of an apparent upstream issue where 
> "lsb_release" gives "No LSB modules are available".  I skipped YUM 
> verification because the gobject-introspection gem failed to build.
> 
> On Thu, Nov 17, 2022, at 09:45, Rok Mihevc wrote:
>> +1
>>
>> Passed on M1 with:
>> TEST_DEFAULT=0 TEST_SOURCE=1 TEST_PYTHON=0 TEST_GLIB=0 TEST_RUBY=0
>> dev/release/verify-release-candidate.sh 10.0.1 0
>>
>>
>> With TEST_PYTHON=1 it reported a CMake issue:
>>
>> -- ArrowPythonFlight version: 10.0.1
>>
>> -- Found the ArrowPythonFlight shared library:
>> /private/var/folders/bc/h5_8w_395x32tv7b7f9lnhz4gn/T/arrow-10.0.1.X.BpuqwhAp/apache-arrow-10.0.1/python/build/dist/lib/libarrow_python_flight.1000.1.0.dylib
>>
>> -- Found the ArrowPythonFlight import library:
>> ARROW_PYTHON_FLIGHT_IMPORT_LIB-NOTFOUND
>>
>> -- Found the ArrowPythonFlight static library:
>> /private/var/folders/bc/h5_8w_395x32tv7b7f9lnhz4gn/T/arrow-10.0.1.X.BpuqwhAp/apache-arrow-10.0.1/python/build/dist/lib/libarrow_python_flight.a
>>
>> CMake Error at CMakeLists.txt:553 (find_package):
>>
>>   By not providing "FindArrowSubstrait.cmake" in CMAKE_MODULE_PATH this
>>
>>   project has asked CMake to find a package configuration file provided by
>>
>>   "ArrowSubstrait", but CMake did not find one.
>>
>>
>>   Could not find a package configuration file provided by "ArrowSubstrait"
>>
>>   with any of the following names:
>>
>>
>> ArrowSubstraitConfig.cmake
>>
>> arrowsubstrait-config.cmake
>>
>>
>>   Add the installation prefix of "ArrowSubstrait" to CMAKE_PREFIX_PATH or
>> set
>>
>>   "ArrowSubstrait_DIR" to a directory containing one of the above files.  If
>>
>>   "ArrowSubstrait" provides a separate development package or SDK, be sure
>> it
>>
>>   has been installed.
>>
>>
>>
>> With TEST_RUBY=1 it failed with:
>>
>>
>> Installing gobject-introspection 4.0.3 with native extensions
>>
>> Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
>>
>>
>> Both seem more related to my specific environment issues than anything else.
>>
>>
>> Rok
>>
>> On Thu, Nov 17, 2022 at 8:52 AM Sutou Kouhei  wrote:
>>
>>> +1
>>>
>>> I ran the followings on Debian GNU/Linux sid:
>>>
>>>   * TEST_DEFAULT=0 \
>>>   TEST_SOURCE=1 \
>>>   LANG=C \
>>>   TZ=UTC \
>>>   CUDAToolkit_ROOT=/usr \
>>>   ARROW_CMAKE_OPTIONS="-DBoost_NO_BOOST_CMAKE=ON
>>> -Dxsimd_SOURCE=BUNDLED" \
>>>   dev/release/verify-release-candidate.sh 10.0.1 0
>>>
>>>   * TEST_DEFAULT=0 \
>>>   TEST_APT=1 \
>>>   LANG=C \
>>>   dev/release/verify-release-candidate.sh 10.0.1 0
>>>
>>>   * TEST_DEFAULT=0 \
>>>   TEST_BINARY=1 \
>>>   LANG=C \
>>>   dev/release/verify-release-candidate.sh 10.0.1 0
>>>
>>>   * TEST_DEFAULT=0 \
>>>   TEST_JARS=1 \
>>>   LANG=C \
>>>   dev/release/verify-release-candidate.sh 10.0.1 0
>>>
>>>   * TEST_DEFAULT=0 \
>>>   TEST_WHEELS=1 \
>>>   LANG=C \
>>>   dev/release/verify-release-candidate.sh 10.0.1 0
>>>
>>>   * TEST_DEFAULT=0 \
>>>   TEST_YUM=1 \
>>>   LANG=C \
>>>   dev/release/verify-release-candidate.sh 10.0.1 0
>>>
>>> with:
>>>
>>>   * .NET SDK (6.0.402)
>>>   * Python 3.10.8
>>>   * gcc (Debian 12.2.0-9) 12.2.0
>>>   * nvidia-cuda-dev 11.6.2-2
>>>   * openjdk version "18.0.2-ea" 2022-07-19
>>>   * ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux-gnu]
>>>
>>> Notes:
>>>
>>>   * https://github.com/apache/arrow/pull/14661 is needed for TEST_YUM=1
>>>
>>>
>>> Thanks,
>>> --
>>> kou
>>>
>>>
>>> In <20221117.141054.1339663249834765324@clear-code.com>
>>>   "[VOTE] Release Apache Arrow 10.0.1 - RC0" on Thu, 17 Nov 2022 14:10:54
>>> +0900 (JST),
>>>   Sutou Kouhei  wrote:
>>>
>>> > Hi,
>>> >
>>> > I would like to propose the following release candidate (RC0) of Apache
>>> > Arrow version 10.0.1. This is a release consisting of 30
>>> > resolved JIRA issues[1].
>>> >
>>> > This release candidate is based on commit:
>>> > a6eabc2b890030578131aecc5e85900597d694a4 [2]
>>> >
>>> > The source release rc0 is hosted at [3].
>>> > The binary artifacts are hosted at [4][5][6][7][8][9][10][11].
>>> > The changelog is located at [12].
>>> >
>>> > Please download, verify checksums and signatures, run the unit tests,
>>> > and vote on the release. See [13] for how to validate a release
>>> candidate.
>>> >
>>> > See also a verification result on GitHub pull request [14].
>>> >
>>> > The vote will

Re: [DISCUSS]: Interim plan for new users reporting issues before GitHub migration

2022-11-17 Thread MAURICIO ANDRES VARGAS SEPULVEDA 
Hi!

+Inf to Nic's point

Asking to write a Gh issue seems to be the easiest

Get Outlook for Android

From: Nic 
Sent: Thursday, November 17, 2022 3:12:36 PM
To: dev@arrow.apache.org 
Subject: [DISCUSS]: Interim plan for new users reporting issues before GitHub 
migration

Hi folks,

The recent Infra policy change preventing people from signing themselves up
for JIRA accounts in order to prevent spam [1] was originally scheduled for
6th November, and though it was slightly delayed, it appears now to be in
force.

There is ongoing work to do a migration from JIRA to GitHub Issues[2], but
this may take a few weeks or move.

What should we do in the interim? Should we ask new participants to use
GitHub issues for their issue reports, or provide some path for new users
to contact the PMC (via the user mailing list or zulip?) to open an account?

I propose we go with the former and move straight to GitHub Issues even if
we're not fully there yet with the migration - this may be the simplest
option in terms of reducing the number of times we ask people to change
what they're doing, even if the GitHub Issues usage will change
significantly in that time.

Whatever we choose, I suppose we should also announce this to users
somewhere - Steph Hazlitt has started this doc [3] listing possible places
to mention this.

Thanks,

Nic

[1] https://lists.apache.org/thread/l545m95xmf3w47oxwqxvg811or7b93tb
[2] https://github.com/apache/arrow/issues/14542
[3]
https://docs.google.com/document/d/1UaSJs-oyuq8QvlUPoQ9GeiwP19LK5ZzF_5-HLfHDCIg/edit?usp=sharing

[DISCUSS]: Interim plan for new users reporting issues before GitHub migration

2022-11-17 Thread Nic 
Hi folks,

The recent Infra policy change preventing people from signing themselves up
for JIRA accounts in order to prevent spam [1] was originally scheduled for
6th November, and though it was slightly delayed, it appears now to be in
force.

There is ongoing work to do a migration from JIRA to GitHub Issues[2], but
this may take a few weeks or move.

What should we do in the interim? Should we ask new participants to use
GitHub issues for their issue reports, or provide some path for new users
to contact the PMC (via the user mailing list or zulip?) to open an account?

I propose we go with the former and move straight to GitHub Issues even if
we're not fully there yet with the migration - this may be the simplest
option in terms of reducing the number of times we ask people to change
what they're doing, even if the GitHub Issues usage will change
significantly in that time.

Whatever we choose, I suppose we should also announce this to users
somewhere - Steph Hazlitt has started this doc [3] listing possible places
to mention this.

Thanks,

Nic

[1] https://lists.apache.org/thread/l545m95xmf3w47oxwqxvg811or7b93tb
[2] https://github.com/apache/arrow/issues/14542
[3]
https://docs.google.com/document/d/1UaSJs-oyuq8QvlUPoQ9GeiwP19LK5ZzF_5-HLfHDCIg/edit?usp=sharing

Re: [VOTE] Release Apache Arrow 10.0.1 - RC0

2022-11-17 Thread David Li 
+1 (binding), AMD64/Linux

I had to skip debian:bookworm because of an apparent upstream issue where 
"lsb_release" gives "No LSB modules are available".  I skipped YUM verification 
because the gobject-introspection gem failed to build.

On Thu, Nov 17, 2022, at 09:45, Rok Mihevc wrote:
> +1
>
> Passed on M1 with:
> TEST_DEFAULT=0 TEST_SOURCE=1 TEST_PYTHON=0 TEST_GLIB=0 TEST_RUBY=0
> dev/release/verify-release-candidate.sh 10.0.1 0
>
>
> With TEST_PYTHON=1 it reported a CMake issue:
>
> -- ArrowPythonFlight version: 10.0.1
>
> -- Found the ArrowPythonFlight shared library:
> /private/var/folders/bc/h5_8w_395x32tv7b7f9lnhz4gn/T/arrow-10.0.1.X.BpuqwhAp/apache-arrow-10.0.1/python/build/dist/lib/libarrow_python_flight.1000.1.0.dylib
>
> -- Found the ArrowPythonFlight import library:
> ARROW_PYTHON_FLIGHT_IMPORT_LIB-NOTFOUND
>
> -- Found the ArrowPythonFlight static library:
> /private/var/folders/bc/h5_8w_395x32tv7b7f9lnhz4gn/T/arrow-10.0.1.X.BpuqwhAp/apache-arrow-10.0.1/python/build/dist/lib/libarrow_python_flight.a
>
> CMake Error at CMakeLists.txt:553 (find_package):
>
>   By not providing "FindArrowSubstrait.cmake" in CMAKE_MODULE_PATH this
>
>   project has asked CMake to find a package configuration file provided by
>
>   "ArrowSubstrait", but CMake did not find one.
>
>
>   Could not find a package configuration file provided by "ArrowSubstrait"
>
>   with any of the following names:
>
>
> ArrowSubstraitConfig.cmake
>
> arrowsubstrait-config.cmake
>
>
>   Add the installation prefix of "ArrowSubstrait" to CMAKE_PREFIX_PATH or
> set
>
>   "ArrowSubstrait_DIR" to a directory containing one of the above files.  If
>
>   "ArrowSubstrait" provides a separate development package or SDK, be sure
> it
>
>   has been installed.
>
>
>
> With TEST_RUBY=1 it failed with:
>
>
> Installing gobject-introspection 4.0.3 with native extensions
>
> Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
>
>
> Both seem more related to my specific environment issues than anything else.
>
>
> Rok
>
> On Thu, Nov 17, 2022 at 8:52 AM Sutou Kouhei  wrote:
>
>> +1
>>
>> I ran the followings on Debian GNU/Linux sid:
>>
>>   * TEST_DEFAULT=0 \
>>   TEST_SOURCE=1 \
>>   LANG=C \
>>   TZ=UTC \
>>   CUDAToolkit_ROOT=/usr \
>>   ARROW_CMAKE_OPTIONS="-DBoost_NO_BOOST_CMAKE=ON
>> -Dxsimd_SOURCE=BUNDLED" \
>>   dev/release/verify-release-candidate.sh 10.0.1 0
>>
>>   * TEST_DEFAULT=0 \
>>   TEST_APT=1 \
>>   LANG=C \
>>   dev/release/verify-release-candidate.sh 10.0.1 0
>>
>>   * TEST_DEFAULT=0 \
>>   TEST_BINARY=1 \
>>   LANG=C \
>>   dev/release/verify-release-candidate.sh 10.0.1 0
>>
>>   * TEST_DEFAULT=0 \
>>   TEST_JARS=1 \
>>   LANG=C \
>>   dev/release/verify-release-candidate.sh 10.0.1 0
>>
>>   * TEST_DEFAULT=0 \
>>   TEST_WHEELS=1 \
>>   LANG=C \
>>   dev/release/verify-release-candidate.sh 10.0.1 0
>>
>>   * TEST_DEFAULT=0 \
>>   TEST_YUM=1 \
>>   LANG=C \
>>   dev/release/verify-release-candidate.sh 10.0.1 0
>>
>> with:
>>
>>   * .NET SDK (6.0.402)
>>   * Python 3.10.8
>>   * gcc (Debian 12.2.0-9) 12.2.0
>>   * nvidia-cuda-dev 11.6.2-2
>>   * openjdk version "18.0.2-ea" 2022-07-19
>>   * ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux-gnu]
>>
>> Notes:
>>
>>   * https://github.com/apache/arrow/pull/14661 is needed for TEST_YUM=1
>>
>>
>> Thanks,
>> --
>> kou
>>
>>
>> In <20221117.141054.1339663249834765324@clear-code.com>
>>   "[VOTE] Release Apache Arrow 10.0.1 - RC0" on Thu, 17 Nov 2022 14:10:54
>> +0900 (JST),
>>   Sutou Kouhei  wrote:
>>
>> > Hi,
>> >
>> > I would like to propose the following release candidate (RC0) of Apache
>> > Arrow version 10.0.1. This is a release consisting of 30
>> > resolved JIRA issues[1].
>> >
>> > This release candidate is based on commit:
>> > a6eabc2b890030578131aecc5e85900597d694a4 [2]
>> >
>> > The source release rc0 is hosted at [3].
>> > The binary artifacts are hosted at [4][5][6][7][8][9][10][11].
>> > The changelog is located at [12].
>> >
>> > Please download, verify checksums and signatures, run the unit tests,
>> > and vote on the release. See [13] for how to validate a release
>> candidate.
>> >
>> > See also a verification result on GitHub pull request [14].
>> >
>> > The vote will be open for at least 72 hours.
>> >
>> > [ ] +1 Release this as Apache Arrow 10.0.1
>> > [ ] +0
>> > [ ] -1 Do not release this as Apache Arrow 10.0.1 because...
>> >
>> > [1]:
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20%28Resolved%2C%20Closed%29%20AND%20fixVersion%20%3D%2010.0.1
>> > [2]:
>> https://github.com/apache/arrow/tree/a6eabc2b890030578131aecc5e85900597d694a4
>> > [3]:
>> https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-10.0.1-rc0
>> > [4]: https://apache.jfrog.io/artifactory/arrow/almalinux-rc/
>> > [5]: https://apache.jfrog.io/artifactory/arrow/amazon-linux-rc/
>> > [6]:

Re: [VOTE][Julia] Release Apache Arrow Julia 2.4.1 RC0

2022-11-17 Thread Neal Richardson 
+1

Verified on Intel macOS 13

Neal

On Thu, Nov 17, 2022 at 9:00 AM Raúl Cumplido 
wrote:

> +1
>
> Tested on Ubuntu 22.04.1 LTS
>
> El jue, 17 nov 2022 a las 14:42, David Li ()
> escribió:
>
> > +1 (binding)
> >
> > Tested on AMD64/Linux (Ubuntu 18.04)
> >
> > On Thu, Nov 17, 2022, at 07:18, Eric Hanson wrote:
> > > +1
> > >
> > > (tested on m1)
> > >
> > > On 2022/11/16 23:21:36 Sutou Kouhei wrote:
> > >> Hi,
> > >>
> > >> I would like to propose the following release candidate (RC0) of
> > >> Apache Arrow Julia version 2.4.1.
> > >>
> > >> This release candidate is based on commit:
> > >> 23258f12bb4b28eb3846d0d3a91a54e2628254d1 [1]
> > >>
> > >> The source release rc0 is hosted at [2].
> > >>
> > >> Please download, verify checksums and signatures, run the unit tests,
> > >> and vote on the release. See [3] for how to validate a release
> > candidate.
> > >>
> > >> The vote will be open for at least 24 hours.
> > >>
> > >> [ ] +1 Release this as Apache Arrow Julia 2.4.1
> > >> [ ] +0
> > >> [ ] -1 Do not release this as Apache Arrow Julia 2.4.1 because...
> > >>
> > >> [1]:
> >
> https://github.com/apache/arrow-julia/tree/23258f12bb4b28eb3846d0d3a91a54e2628254d1
> > >> [2]:
> >
> https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-julia-2.4.1-rc0/
> > >> [3]:
> >
> https://github.com/apache/arrow-julia/blob/main/dev/release/README.md#verify
> > >>
> > >> Thanks,
> > >> --
> > >> kou
> > >>
> >
>

Re: [VOTE] Release Apache Arrow 10.0.1 - RC0

2022-11-17 Thread Rok Mihevc 
+1

Passed on M1 with:
TEST_DEFAULT=0 TEST_SOURCE=1 TEST_PYTHON=0 TEST_GLIB=0 TEST_RUBY=0
dev/release/verify-release-candidate.sh 10.0.1 0


With TEST_PYTHON=1 it reported a CMake issue:

-- ArrowPythonFlight version: 10.0.1

-- Found the ArrowPythonFlight shared library:
/private/var/folders/bc/h5_8w_395x32tv7b7f9lnhz4gn/T/arrow-10.0.1.X.BpuqwhAp/apache-arrow-10.0.1/python/build/dist/lib/libarrow_python_flight.1000.1.0.dylib

-- Found the ArrowPythonFlight import library:
ARROW_PYTHON_FLIGHT_IMPORT_LIB-NOTFOUND

-- Found the ArrowPythonFlight static library:
/private/var/folders/bc/h5_8w_395x32tv7b7f9lnhz4gn/T/arrow-10.0.1.X.BpuqwhAp/apache-arrow-10.0.1/python/build/dist/lib/libarrow_python_flight.a

CMake Error at CMakeLists.txt:553 (find_package):

  By not providing "FindArrowSubstrait.cmake" in CMAKE_MODULE_PATH this

  project has asked CMake to find a package configuration file provided by

  "ArrowSubstrait", but CMake did not find one.


  Could not find a package configuration file provided by "ArrowSubstrait"

  with any of the following names:


ArrowSubstraitConfig.cmake

arrowsubstrait-config.cmake


  Add the installation prefix of "ArrowSubstrait" to CMAKE_PREFIX_PATH or
set

  "ArrowSubstrait_DIR" to a directory containing one of the above files.  If

  "ArrowSubstrait" provides a separate development package or SDK, be sure
it

  has been installed.



With TEST_RUBY=1 it failed with:


Installing gobject-introspection 4.0.3 with native extensions

Gem::Ext::BuildError: ERROR: Failed to build gem native extension.


Both seem more related to my specific environment issues than anything else.


Rok

On Thu, Nov 17, 2022 at 8:52 AM Sutou Kouhei  wrote:

> +1
>
> I ran the followings on Debian GNU/Linux sid:
>
>   * TEST_DEFAULT=0 \
>   TEST_SOURCE=1 \
>   LANG=C \
>   TZ=UTC \
>   CUDAToolkit_ROOT=/usr \
>   ARROW_CMAKE_OPTIONS="-DBoost_NO_BOOST_CMAKE=ON
> -Dxsimd_SOURCE=BUNDLED" \
>   dev/release/verify-release-candidate.sh 10.0.1 0
>
>   * TEST_DEFAULT=0 \
>   TEST_APT=1 \
>   LANG=C \
>   dev/release/verify-release-candidate.sh 10.0.1 0
>
>   * TEST_DEFAULT=0 \
>   TEST_BINARY=1 \
>   LANG=C \
>   dev/release/verify-release-candidate.sh 10.0.1 0
>
>   * TEST_DEFAULT=0 \
>   TEST_JARS=1 \
>   LANG=C \
>   dev/release/verify-release-candidate.sh 10.0.1 0
>
>   * TEST_DEFAULT=0 \
>   TEST_WHEELS=1 \
>   LANG=C \
>   dev/release/verify-release-candidate.sh 10.0.1 0
>
>   * TEST_DEFAULT=0 \
>   TEST_YUM=1 \
>   LANG=C \
>   dev/release/verify-release-candidate.sh 10.0.1 0
>
> with:
>
>   * .NET SDK (6.0.402)
>   * Python 3.10.8
>   * gcc (Debian 12.2.0-9) 12.2.0
>   * nvidia-cuda-dev 11.6.2-2
>   * openjdk version "18.0.2-ea" 2022-07-19
>   * ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux-gnu]
>
> Notes:
>
>   * https://github.com/apache/arrow/pull/14661 is needed for TEST_YUM=1
>
>
> Thanks,
> --
> kou
>
>
> In <20221117.141054.1339663249834765324@clear-code.com>
>   "[VOTE] Release Apache Arrow 10.0.1 - RC0" on Thu, 17 Nov 2022 14:10:54
> +0900 (JST),
>   Sutou Kouhei  wrote:
>
> > Hi,
> >
> > I would like to propose the following release candidate (RC0) of Apache
> > Arrow version 10.0.1. This is a release consisting of 30
> > resolved JIRA issues[1].
> >
> > This release candidate is based on commit:
> > a6eabc2b890030578131aecc5e85900597d694a4 [2]
> >
> > The source release rc0 is hosted at [3].
> > The binary artifacts are hosted at [4][5][6][7][8][9][10][11].
> > The changelog is located at [12].
> >
> > Please download, verify checksums and signatures, run the unit tests,
> > and vote on the release. See [13] for how to validate a release
> candidate.
> >
> > See also a verification result on GitHub pull request [14].
> >
> > The vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this as Apache Arrow 10.0.1
> > [ ] +0
> > [ ] -1 Do not release this as Apache Arrow 10.0.1 because...
> >
> > [1]:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20%28Resolved%2C%20Closed%29%20AND%20fixVersion%20%3D%2010.0.1
> > [2]:
> https://github.com/apache/arrow/tree/a6eabc2b890030578131aecc5e85900597d694a4
> > [3]:
> https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-10.0.1-rc0
> > [4]: https://apache.jfrog.io/artifactory/arrow/almalinux-rc/
> > [5]: https://apache.jfrog.io/artifactory/arrow/amazon-linux-rc/
> > [6]: https://apache.jfrog.io/artifactory/arrow/centos-rc/
> > [7]: https://apache.jfrog.io/artifactory/arrow/debian-rc/
> > [8]: https://apache.jfrog.io/artifactory/arrow/java-rc/10.0.1-rc0
> > [9]: https://apache.jfrog.io/artifactory/arrow/nuget-rc/10.0.1-rc0
> > [10]: https://apache.jfrog.io/artifactory/arrow/python-rc/10.0.1-rc0
> > [11]: https://apache.jfrog.io/artifactory/arrow/ubuntu-rc/
> > [12]:
> https://github.com/apache/arrow/blob/a6eabc2b890030578131aecc5e85900597d694a4/CHANGELOG.md
> > [13]:
>

Re: [VOTE][Julia] Release Apache Arrow Julia 2.4.1 RC0

2022-11-17 Thread Raúl Cumplido 
+1

Tested on Ubuntu 22.04.1 LTS

El jue, 17 nov 2022 a las 14:42, David Li () escribió:

> +1 (binding)
>
> Tested on AMD64/Linux (Ubuntu 18.04)
>
> On Thu, Nov 17, 2022, at 07:18, Eric Hanson wrote:
> > +1
> >
> > (tested on m1)
> >
> > On 2022/11/16 23:21:36 Sutou Kouhei wrote:
> >> Hi,
> >>
> >> I would like to propose the following release candidate (RC0) of
> >> Apache Arrow Julia version 2.4.1.
> >>
> >> This release candidate is based on commit:
> >> 23258f12bb4b28eb3846d0d3a91a54e2628254d1 [1]
> >>
> >> The source release rc0 is hosted at [2].
> >>
> >> Please download, verify checksums and signatures, run the unit tests,
> >> and vote on the release. See [3] for how to validate a release
> candidate.
> >>
> >> The vote will be open for at least 24 hours.
> >>
> >> [ ] +1 Release this as Apache Arrow Julia 2.4.1
> >> [ ] +0
> >> [ ] -1 Do not release this as Apache Arrow Julia 2.4.1 because...
> >>
> >> [1]:
> https://github.com/apache/arrow-julia/tree/23258f12bb4b28eb3846d0d3a91a54e2628254d1
> >> [2]:
> https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-julia-2.4.1-rc0/
> >> [3]:
> https://github.com/apache/arrow-julia/blob/main/dev/release/README.md#verify
> >>
> >> Thanks,
> >> --
> >> kou
> >>
>

Re: [VOTE][Julia] Release Apache Arrow Julia 2.4.1 RC0

2022-11-17 Thread David Li 
+1 (binding)

Tested on AMD64/Linux (Ubuntu 18.04)

On Thu, Nov 17, 2022, at 07:18, Eric Hanson wrote:
> +1
>
> (tested on m1)
>
> On 2022/11/16 23:21:36 Sutou Kouhei wrote:
>> Hi,
>> 
>> I would like to propose the following release candidate (RC0) of
>> Apache Arrow Julia version 2.4.1.
>> 
>> This release candidate is based on commit:
>> 23258f12bb4b28eb3846d0d3a91a54e2628254d1 [1]
>> 
>> The source release rc0 is hosted at [2].
>> 
>> Please download, verify checksums and signatures, run the unit tests,
>> and vote on the release. See [3] for how to validate a release candidate.
>> 
>> The vote will be open for at least 24 hours.
>> 
>> [ ] +1 Release this as Apache Arrow Julia 2.4.1
>> [ ] +0
>> [ ] -1 Do not release this as Apache Arrow Julia 2.4.1 because...
>> 
>> [1]: 
>> https://github.com/apache/arrow-julia/tree/23258f12bb4b28eb3846d0d3a91a54e2628254d1
>> [2]: 
>> https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-julia-2.4.1-rc0/
>> [3]: 
>> https://github.com/apache/arrow-julia/blob/main/dev/release/README.md#verify
>> 
>> Thanks,
>> -- 
>> kou
>>

Re: [VOTE][Julia] Release Apache Arrow Julia 2.4.1 RC0

2022-11-17 Thread Eric Hanson 
+1

(tested on m1)

On 2022/11/16 23:21:36 Sutou Kouhei wrote:
> Hi,
> 
> I would like to propose the following release candidate (RC0) of
> Apache Arrow Julia version 2.4.1.
> 
> This release candidate is based on commit:
> 23258f12bb4b28eb3846d0d3a91a54e2628254d1 [1]
> 
> The source release rc0 is hosted at [2].
> 
> Please download, verify checksums and signatures, run the unit tests,
> and vote on the release. See [3] for how to validate a release candidate.
> 
> The vote will be open for at least 24 hours.
> 
> [ ] +1 Release this as Apache Arrow Julia 2.4.1
> [ ] +0
> [ ] -1 Do not release this as Apache Arrow Julia 2.4.1 because...
> 
> [1]: 
> https://github.com/apache/arrow-julia/tree/23258f12bb4b28eb3846d0d3a91a54e2628254d1
> [2]: 
> https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-julia-2.4.1-rc0/
> [3]: 
> https://github.com/apache/arrow-julia/blob/main/dev/release/README.md#verify
> 
> Thanks,
> -- 
> kou
>

Re: Array::GetValue ?

2022-11-17 Thread Antoine Pitrou 



Uh, you're right.  We may want another base class, not sure how it 
should be named though (also, we may want to be careful with multiple 
inheritance?).


Regards

Antoine.


Le 17/11/2022 à 06:15, Micah Kornfield a écrit :


std::string_view FlatArray::GetValueBytes(int64_t index)



I think this would be problematic for Boolean?

On Tue, Nov 15, 2022 at 11:01 AM John Muehlhausen  wrote:


If that covers primitive and binary(string) types, that would work for me.

On Tue, Nov 15, 2022 at 13:50 Antoine Pitrou  wrote:



Then perhaps we can define a method:

std::string_view FlatArray::GetValueBytes(int64_t index)

?


Le 15/11/2022 à 19:39, John Muehlhausen a écrit :

I had a use-case where untyped access to bytes would have been

sufficient,

vs branching depending on array type.  This is what brought the idea to
mind.

On Tue, Nov 15, 2022 at 02:34 Jin Shang 

wrote:



Hi John,

In addition to Micah’s reply, does the member method Value(int64_t
i)[1][2][3] satisfy your need? It is defined for all array types with

a

primitive value representation, i.e. all primitive arrays and binary

arrays.


[1]




https://github.com/js8544/arrow/blob/master/cpp/src/arrow/array/array_primitive.h#L50

<




https://github.com/js8544/arrow/blob/master/cpp/src/arrow/array/array_primitive.h#L50



[2]




https://github.com/js8544/arrow/blob/master/cpp/src/arrow/array/array_primitive.h#L109

<




https://github.com/js8544/arrow/blob/master/cpp/src/arrow/array/array_primitive.h#L109



[3]




https://github.com/js8544/arrow/blob/master/cpp/src/arrow/array/array_binary.h#L87

<




https://github.com/js8544/arrow/blob/master/cpp/src/arrow/array/array_binary.h#L87






2022年11月15日 13:06，Micah Kornfield  写道：

Hi John,

There are a couple of edge cases that need to be discussed to move

the

function to the base array class (which IIUC is this proposal):
1. boolean
2. struct
3.  lists/LargeList
4.  DictionaryArray

FlatArray [1] seems like a better place for this method if there is
consensus on adding it.

Cheers,
Micah

[1]






https://github.com/apache/arrow/blob/master/cpp/src/arrow/array/array_base.h#L219


On Mon, Nov 14, 2022 at 11:46 AM John Muehlhausen 

wrote:



There exists:
const uint8_t* BaseBinaryArray::GetValue(int64_t i, offset_type*
out_length) const

What about adding:
const uint8_t* Array::GetValue(int64_t i, offset_type* out_length)

const


This would allow GetValue to get the untyped bytes/length of any

value?

E.g. out_length would be set to sizeof(T) for arrays of primitive

type

T?


For FixedSizeBinaryArray the existing GetValue would still be a

valid

overload.

-John