Re: [VOTE] Release Apache Arrow nanoarrow 0.5.0

2024-05-22 Thread Dane Pitkin
+1 (non-binding) Verified on MacOS 14 aarch64. On Wed, May 22, 2024 at 2:55 PM Bryce Mecum wrote: > +1 (non-binding) > > Verified on: > > - macOS aarch64 > - Debian 12 x86_64 inside a conda environment (note I had to install > Python 3.11 separately from the instructions, not sure I missed a >

Re: [DISCUSS] Drop Java 8 support

2024-05-21 Thread Dane Pitkin
ned Apache Parquet and haven't seen someone mentioned if/when > Apache Parquet would transition. > > > > On Fri, May 17, 2024 at 9:07 AM Dane Pitkin wrote: > > > Fokko, thank you for these datapoints! It's great to see how other low > > level Java OSS projects are approa

Re: [DISCUSS] Drop Java 8 support

2024-05-17 Thread Dane Pitkin
i 2024 om 07:44 schreef Jean-Baptiste Onofré : > >> Hi Dane >> >> Do we have a formal consensus about Java version in regards of arrow >> version ? >> I agree with the plan but just wondering if it’s ok from everyone with the >> community. >> >>

Re: [DISCUSS] Drop Java 8 support

2024-05-16 Thread Dane Pitkin
issue[1] to reflect this. [1]https://github.com/apache/arrow/issues/38051 On Wed, May 8, 2024 at 5:46 PM Dane Pitkin wrote: > Thank you all for your valuable input. The consensus from my understanding > is that dropping Java 8 is not contentious, so we will move forward here. > > We won

Re: [DISCUSS] Drop Java 8 support

2024-05-08 Thread Dane Pitkin
sts us is one extra CI job. > > Hope some of this is helpful - apologies for the high latency, busy as > always!! > > Martin. > > > > On 1 May 2024, at 22:38, Dane Pitkin > wrote: > > > > Thanks, Martin. It's great to hear of real-world use cases. Do you

Re: [ANNOUNCE] New Arrow committer: Dane Pitkin

2024-05-08 Thread Dane Pitkin
; >> > Congrats! > >> > > >> > Am Di., 7. Mai 2024 um 23:19 Uhr schrieb Bryce Mecum < > bryceme...@gmail.com > >> > >: > >> > > >> > > Congrats Dane! > >> > > > >> > > On Tue, May 7, 2024 at 5:53 

Re: [DISCUSS] Drop Java 8 support

2024-05-01 Thread Dane Pitkin
gt; Otherwise people will just move to 11 and then have the pain of migration > > again when we drop that (which will happen soon regardless imo). > > > > Am Di., 30. Apr. 2024 um 16:18 Uhr schrieb Dane Pitkin > > : > > > >> Thanks, JB. Are we aware of any downstr

Re: [DISCUSS] Drop Java 8 support

2024-04-30 Thread Dane Pitkin
t; integration (as what we’re doing currently somewhat shoe-horns module > information alongside JDK8 bytecode). > > > > From: Dane Pitkin > > Date: Thursday, April 25, 2024 at 1:02 PM > > To: dev@arrow.apache.org > > Subject: [DISCUSS] Drop Java 8 support >

Re: [DISCUSS] Drop Java 8 support

2024-04-26 Thread Dane Pitkin
port-roadmap.html>. I > echo'd this on the Iceberg dev list, and also suggested this with Avro 1.12 > <https://lists.apache.org/thread/6vbd3w5qk7mpb5lyrfyf2s0z1cymjt5w>. > > Kind regards, > Fokko > > > > Op do 25 apr 2024 om 22:02 schreef Dane Pitkin > >: > &g

[DISCUSS] Drop Java 8 support

2024-04-25 Thread Dane Pitkin
Hi all, I would like to revisit the discussion of dropping Java 8 (and maybe 11) from Arrow's Java implementation. See GH issue[1] below. This was also discussed in the last Arrow community sync meeting on 2024-04-24. For context, this was discussed[2] last year on this mailing list. We decided

Re: Upgrading Java version in build toolchain

2024-04-05 Thread Dane Pitkin
I think we can revisit the discussion soon for dropping Java 8 altogether, since Spark will release 4.0 in ~June supporting Java 17+ at runtime. I'm curious how big of an effort it would be to get your proposal implemented. Would you be willing to draft a PR so we can see what types of changes

Re: [VOTE] Release Apache Arrow ADBC 0.11.0 - RC0

2024-03-28 Thread Dane Pitkin
+1 (non-binding) Verified on MacOS 14.4.1 aarch64 with Conda using: DOCKER_DEFAULT_PLATFORM=linux/amd64 USE_CONDA=1 ./dev/release/verify-release-candidate.sh 0.11.0 0 On Thu, Mar 28, 2024 at 11:07 AM David Li wrote: > Hello, > > I would like to propose the following release candidate (RC0) of

Re: [ANNOUNCE] New Arrow committer: Bryce Mecum

2024-03-18 Thread Dane Pitkin
Congratulations, Bryce!! On Mon, Mar 18, 2024 at 9:18 AM David Li wrote: > Congrats Bryce! > > On Mon, Mar 18, 2024, at 08:52, Ian Cook wrote: > > Congratulations Bryce! > > > > Ian > > > > On Sun, Mar 17, 2024 at 22:24 Nic Crane wrote: > > > >> On behalf of the Arrow PMC, I'm happy to

Re: [VOTE] Release Apache Arrow ADBC 0.10.0 - RC1

2024-02-21 Thread Dane Pitkin
+1 (non-binding) Verified on Mac M1 using conda. On Tue, Feb 20, 2024 at 11:27 PM Dewey Dunnington wrote: > +1! > > I ran USE_CONDA=1 dev/release/verify-release-candidate.sh 0.10.0 1 on > MacOS Sonoma (M1). > > On Tue, Feb 20, 2024 at 9:43 AM Jean-Baptiste Onofré > wrote: > > > > +1 (non

Re: Arrow 15 parquet nanosecond change

2024-02-21 Thread Dane Pitkin
It is possible to change the default Parquet version when instantiating PyArrow's ParquetWriter[1]. Here's the PR[2] that upgraded the default Parquet format version from 2.4 -> 2.6, which contains nanosecond support. It was released in Arrow v13. [1]

[DISCUSS] Arrow 15.0.1 patch release

2024-02-13 Thread Dane Pitkin
Hi all, Arrow Java identified an issue[1] in the 15.0.0 release. There is an undefined symbol in the dataset module that causes a linking error at run time. The issue is resolved[2] and I'd like to propose a patch release. We also have an open issue to implement testing to prevent this from

Re: [VOTE] Release Apache Arrow nanoarrow 0.4.0 - RC0

2024-01-29 Thread Dane Pitkin
+1 (non-binding) Verified on MacOS 14 using conda. On Mon, Jan 29, 2024 at 10:11 AM Dewey Dunnington wrote: > Hello, > > I would like to propose the following release candidate (rc0) of > Apache Arrow nanoarrow [0] version 0.4.0. This release consists of 46 > resolved GitHub issues from 5

Re: [DISC] Improve Arrow Release verification process

2024-01-19 Thread Dane Pitkin
I agree that this is a great time to look at improving the verification process. One solution I've seen work fairly well is to convert large bash scripts into a lightweight ETL pipeline that caches the results/status of each node as it executes. That way, restarting a pipeline at the right

Re: [ANNOUNCE] New Arrow committer: Felipe Oliveira Carvalho

2023-12-07 Thread Dane Pitkin
Congrats, Felipe! On Thu, Dec 7, 2023 at 11:41 AM hsseo0501 wrote: > Congrats. Felipe :)내 Galaxy에서 보냄 > 원본 이메일 발신: Ian Cook 날짜: 23/12/8 > 오전 1:24 (GMT+09:00) 받은 사람: dev@arrow.apache.org 제목: Re: [ANNOUNCE] New > Arrow committer: Felipe Oliveira Carvalho Congratulations

Re: [ANNOUNCE] New Arrow committer: James Duong

2023-11-16 Thread Dane Pitkin
Congrats, James! On Thu, Nov 16, 2023 at 4:23 AM Alenka Frim wrote: > Congratulations! > > On Thu, Nov 16, 2023 at 8:46 AM Joris Van den Bossche < > jorisvandenboss...@gmail.com> wrote: > > > Congrats! > > > > On Thu, 16 Nov 2023 at 08:44, Sutou Kouhei wrote: > > > > > > On behalf of the Arrow

Re: [ANNOUNCE] New Arrow PMC member: Raúl Cumplido

2023-11-13 Thread Dane Pitkin
Congrats, Raul! On Mon, Nov 13, 2023 at 2:45 PM Kevin Gurney wrote: > Congratulations, Raúl! > > > From: Nic Crane > Sent: Monday, November 13, 2023 2:31 PM > To: dev@arrow.apache.org > Subject: Re: [ANNOUNCE] New Arrow PMC member: Raúl Cumplido > > Congrats

Re: [VOTE] Release Apache Arrow ADBC 0.8.0 - RC0

2023-11-07 Thread Dane Pitkin
+1 (non-binding) Verified on M1 MacOS 13 with: USE_CONDA=1 TEST_YUM=0 TEST_APT=0 ./dev/release/verify-release-candidate.sh 0.8.0 0 On Tue, Nov 7, 2023 at 9:10 AM Dewey Dunnington wrote: > +1! > > I ran: TEST_APT=0 TEST_YUM=0 USE_CONDA=1 > dev/release/verify-release-candidate.sh 0.8.0 0 > > On

Re: [VOTE] Release Apache Arrow 14.0.0 - RC2

2023-10-27 Thread Dane Pitkin
Has anyone fully verified on MacOS yet? Last time I tried, I was still getting stuck on the original Go issue after syncing Kou's latest changes (I might need to try a fresh install of arrow). On Wed, Oct 25, 2023 at 9:34 AM David Li wrote: > +1 (binding) > > Tested on Debian 12 'bookworm' /

Re: [ANNOUNCE] New Arrow committer: Xuwei Fu

2023-10-26 Thread Dane Pitkin
Congratulations, Xuwei! On Thu, Oct 26, 2023 at 9:34 AM Joris Van den Bossche < jorisvandenboss...@gmail.com> wrote: > Congrats! > > On Wed, 25 Oct 2023 at 08:23, Ian Joiner wrote: > > > > Congrats! > > > > On Mon, Oct 23, 2023 at 2:33 AM Sutou Kouhei wrote: > > > > > On behalf of the Arrow

Re: Request for Assistance with MATLAB CI Workflow Issue

2023-10-24 Thread Dane Pitkin
Hey Divyansh, I suggest discussing this on the PR[1] itself. You will get the best discussion there, where comments can be added to the code review directly. It looks like there is a MATLAB reviewer that has already left feedback that may be helpful. [1]https://github.com/apache/arrow/pull/38274

Re: [ANNOUNCE] New Arrow committer: Curt Hagenlocher

2023-10-16 Thread Dane Pitkin
Congrats Curt! On Mon, Oct 16, 2023 at 12:00 PM Kevin Gurney wrote: > Congratulations, Curt! > > From: Weston Pace > Sent: Sunday, October 15, 2023 5:32 PM > To: dev@arrow.apache.org > Subject: Re: [ANNOUNCE] New Arrow committer: Curt Hagenlocher > >

Re: [ANNOUNCE] New Arrow PMC member: Jonathan Keane

2023-10-16 Thread Dane Pitkin
Congrats Jon!! On Mon, Oct 16, 2023 at 7:04 AM Krisztián Szűcs wrote: > Congrats Jon! > > On Mon, Oct 16, 2023 at 11:20 AM Alenka Frim > wrote: > > > > Yay, congratulations Jon!! > > > > On Mon, Oct 16, 2023 at 10:27 AM vin jake wrote: > > > > > Congrats Jon! > > > > > > On Sun, Oct 15, 2023

Re: [Java][Discuss]: consensus for JDK 8 deprecation

2023-10-12 Thread Dane Pitkin
My > > > employer has leveraged Apache Arrow in the internal engine > > > and depends on Arrow Java in the Java SDK. For end users > > > who cannot get away with JDK 8, we might need to prepare > > > different Java SDKs and use features that are available in t

Re: [Java][Discuss]: consensus for JDK 8 deprecation

2023-10-10 Thread Dane Pitkin
> always make a patch release of an older version. > > > > On Thu, Oct 5, 2023 at 3:13 PM Dane Pitkin > > > wrote: > > > > > I also learned today that Apache Spark has dropped support for Java 8 > and > > > 11 for their next release (v4.0)[1]. Shou

Re: [Java][Discuss]: consensus for JDK 8 deprecation

2023-10-05 Thread Dane Pitkin
I also learned today that Apache Spark has dropped support for Java 8 and 11 for their next release (v4.0)[1]. Should we consider dropping Java 11 as well? [1]https://github.com/apache/spark/pull/43005 -Dane On Thu, Oct 5, 2023 at 3:30 PM Dane Pitkin wrote: > I created a GH issue[1] propos

Re: [Java][Discuss]: consensus for JDK 8 deprecation

2023-10-05 Thread Dane Pitkin
[2]https://en.wikipedia.org/wiki/Java_Platform_Module_System -Dane On Fri, Sep 15, 2023 at 12:26 PM Dane Pitkin wrote: > - As a low level library, users have to add specific flags to use >> Java 9 and up with Arrow to resolve issues with java.nio. This has >> been annoying fo

Re: [VOTE] Release Apace Arrow nanoarrow 0.3.0 - RC0

2023-09-26 Thread Dane Pitkin
+1 (non-binding) I verified successfully on MacOS 13.5 (aarch64) with: cd dev/release && ./verify-release-candidate.sh 0.3.0 0 On Tue, Sep 26, 2023 at 5:30 PM Sutou Kouhei wrote: > +1 > > I ran the following command line on Debian GNU/Linux sid: > > CMAKE_PREFIX_PATH=/tmp/local \ >

Re: [VOTE] Release Apache Arrow ADBC 0.7.0 - RC0

2023-09-20 Thread Dane Pitkin
+1 (non-binding) I verified it successfully on macOS/aarch64 with: $ export DOCKER_DEFAULT_PLATFORM=linux/amd64 > && ARROW_TMPDIR=/tmp/adbc-0.7.0 USE_CONDA=1 > ./dev/release/verify-release-candidate.sh 0.7.0 0 On Wed, Sep 20, 2023 at 1:04 PM David Li wrote: > Hello, > > I would like to

Re: [Java][Discuss]: consensus for JDK 8 deprecation

2023-09-15 Thread Dane Pitkin
> > - As a low level library, users have to add specific flags to use > Java 9 and up with Arrow to resolve issues with java.nio. This has > been annoying for our customers constantly. If this is not resolved, > I would say we may see a lot of complaints in the future. > I filed issue 37739[1]

Re: [Python][Discuss] PyArrow Dataset as a Python protocol

2023-09-01 Thread Dane Pitkin
The Python Substrait package[1] is on PyPi[2] and currently has python wrappers for the Substrait protobuf objects. I think this will be a great opportunity to identify helper features that users of this protocol would like to see. I'll be keeping an eye out as this develops, but also feel free to

Re: [VOTE] Release Apache Arrow 13.0.0 - RC2

2023-08-02 Thread Dane Pitkin
During the Arrow community meeting today, we decided to include the fix for the wide-dataframe performance regression due to its severity. The ETA is end-of-the-week for the patch. You can review the meeting notes here[1]. [1]

Re: [DISCUSS] Canonical alternative layout proposal

2023-07-13 Thread Dane Pitkin
I am in favor of this proposal. IMO the Arrow project is the right place to standardize both the interoperability *and operability* of columnar data layouts. Data engines are a core component of the Arrow ecosystem and the project should be able to grow with these data engines as they converge on

Re: [ANNOUNCE] New Arrow PMC member: Dewey Dunnington

2023-06-23 Thread Dane Pitkin
Congrats Dewey! On Fri, Jun 23, 2023 at 9:15 AM Nic Crane wrote: > Well-deserved Dewey, congratulations! > > On Fri, 23 Jun 2023 at 11:53, Vibhatha Abeykoon > wrote: > > > Congratulations Dewey! > > > > On Fri, Jun 23, 2023 at 4:16 PM Alenka Frim > .invalid> > > wrote: > > > > >

Re: [VOTE] Release Apache Arrow nanoarrow 0.2.0 - RC1

2023-06-20 Thread Dane Pitkin
+1 (non-binding) Verified on MacOS (M1) using conda. A couple of nuances: * Had to uninstall gnupg in conda and used brew's gnupg instead (same issue Will found). * I initially encountered some intermittent CMake build timeouts with gtest, but haven't been able to reproduce. On Tue, Jun 20,