Arrow community meeting June 5 at 16:00 UTC

2024-06-04 Thread Ian Cook
Our next biweekly Arrow community meeting is tomorrow at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Re: [DISCUSS] Apache Arrow LinkedIn page

2024-05-24 Thread Ian Cook
nkedIn > membership/permissions model? > > > > Le 24/05/2024 à 18:04, Ian Cook a écrit : > > Following the discussion [1] earlier this year about the status of the > > Apache Arrow Twitter / X account [2], I have seen several news stories > > citing declines in us

[DISCUSS] Apache Arrow LinkedIn page

2024-05-24 Thread Ian Cook
Following the discussion [1] earlier this year about the status of the Apache Arrow Twitter / X account [2], I have seen several news stories citing declines in use of X and increases in use of LinkedIn (for example [3]). Anecdotally I have seen that the types of conversations about open source

Arrow community meeting May 22 at 16:00 UTC

2024-05-21 Thread Ian Cook
Our next biweekly Arrow community meeting is tomorrow at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Arrow community meeting May 8 at 16:00 UTC

2024-05-07 Thread Ian Cook
Our next biweekly Arrow community meeting is tomorrow at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Re: [VOTE][Format] JSON canonical extension type

2024-05-07 Thread Ian Cook
Thanks Rok and Pradeep for your work to advance this proposal. I spoke to the DuckDB maintainers about this. DuckDB has a JSON extension which defines a JSON column type. They intend to have DuckDB's Arrow integrations recognize this arrow.json extension name on input and set it on output. Ian

Re: [ANNOUNCE] New Arrow committer: Dane Pitkin

2024-05-07 Thread Ian Cook
Congratulations Dane! On Tue, May 7, 2024 at 10:10 AM Alenka Frim wrote: > Yay, congratulations Dane!! > > On Tue, May 7, 2024 at 4:00 PM Rok Mihevc wrote: > > > Congrats Dane! > > > > Rok > > > > On Tue, May 7, 2024 at 3:57 PM wish maple > wrote: > > > > > Congrats! > > > > > > Best, > > >

Re: [Discuss] Extension types based on canonical extension types?

2024-04-30 Thread Ian Cook
her extension type as you wouldn't be able > > to include the metadata for the second-level extension part. > > > > i.e. you'd be able to have "ARROW:extension:name" => "HLLSKETCH", but you > > wouldn't be able to *also* have "ARROW:extension:name&quo

[Discuss] Extension types based on canonical extension types?

2024-04-30 Thread Ian Cook
The vote on adding a JSON canonical extension type [1] got me wondering: Is it possible to define an extension type that is based on a canonical extension type? If so, how? For example, say I wanted to define a (non-canonical) HLLSKETCH extension type that corresponds to the type that Redshift

Re: [VOTE][Format] JSON canonical extension type

2024-04-29 Thread Ian Cook
+1 (non-binding) I added a comment in the PR suggesting that we explicitly refer to RFC-8259 in CanonicalExtensions.rst. On Mon, Apr 29, 2024 at 1:21 PM Micah Kornfield wrote: > +1, I added a comment to the PR because I think we should recommend > implementations specifically reject parsing

Re: ADBC - OS-level driver manager

2024-04-23 Thread Ian Cook
Ha—no, I was thinking of a special ADBC-specific environment variable, which would work irrespective of the OS. On Tue, Apr 23, 2024 at 21:38 Matt Topol wrote: > An environment variable like LD_LIBRARY_PATH perhaps? =p > > On Tue, Apr 23, 2024, 8:40 PM Ian Cook wrote: > > >

Re: ADBC - OS-level driver manager

2024-04-23 Thread Ian Cook
orm-specific > guidance on a standard list seems reasonable. > > On Wed, Apr 24, 2024, at 02:45, Ian Cook wrote: > > I wonder if there is a relatively simple way to solve this problem. The > > ADBC driver manager libraries already make it possible to dynamically > load > &

Arrow community meeting April 24 at 16:00 UTC

2024-04-23 Thread Ian Cook
Our next biweekly Arrow community meeting is tomorrow at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Re: ADBC - OS-level driver manager

2024-04-23 Thread Ian Cook
I wonder if there is a relatively simple way to solve this problem. The ADBC driver manager libraries already make it possible to dynamically load drivers, and I believe these libraries already allow the user to specify which driver to use by passing either a bare filename or a full file path. So

Re: [ANNOUNCE] New Arrow committer: Sarah Gilmore

2024-04-11 Thread Ian Cook
Congrats Sarah! On Thu, Apr 11, 2024 at 12:31 Bryce Mecum wrote: > Congratulations! > > On Thu, Apr 11, 2024 at 3:13 AM Sutou Kouhei wrote: > > > > Hi, > > > > On behalf of the Arrow PMC, I'm happy to announce that Sarah > > Gilmore has accepted an invitation to become a committer on > >

Arrow community meeting April 10 at 16:00 UTC

2024-04-09 Thread Ian Cook
Our next biweekly Arrow community meeting is tomorrow at 16:00 UTC / 12:00 EDT. I will not be able to attend tomorrow. Could someone please volunteer to lead the meeting and take notes in the Google Doc? The Zoom meeting will work as usual; it does not require a host to start it. Zoom meeting

Re: [INFO] Subsets of pyarrow package with pyarrow-core < pyarrow < pyarrow-all on conda

2024-04-09 Thread Ian Cook
Thanks Raúl for taking care to make this minimally disruptive. This might be an inconvenience for some users of PyArrow, but I think the benefits outweigh the inconvenience. Ian On Tue, Apr 9, 2024 at 11:17 AM Raúl Cumplido wrote: > Hi, > > As part of the effort to reduce the footprint of

Re: [DISCUSS] Versioning and releases for apache/arrow components

2024-04-09 Thread Ian Cook
I think it is worthwhile to pursue this, but I fear that if we do not proceed very carefully, unforeseen complications could arise, creating even greater work for the release managers. > In general I think that this is not something we neither need to nor want to implement from 0 to 100. >

Re: [ANNOUNCE] New Committer Joel Lubinitsky

2024-04-01 Thread Ian Cook
Congratulations Joel! On Mon, Apr 1, 2024 at 11:08 AM wish maple wrote: > Congrats Joel! > > Best, > Xuwei Fu > > Matt Topol 于2024年4月1日周一 22:59写道: > > > On behalf of the Arrow PMC, I'm happy to announce that Joel Lubinitsky > has > > accepted an invitation to become a committer on Apache

Arrow community meeting March 27 at 16:00 UTC

2024-03-26 Thread Ian Cook
Our next biweekly Arrow community meeting is tomorrow at 16:00 UTC / 12:00 EDT. *** For attendees in countries that have not yet switched to Daylight Saving Time, please note that the time is one hour earlier than usual in your local time zone. *** Zoom meeting URL:

Re: ADBC - OS-level driver manager

2024-03-20 Thread Ian Cook
I have given this some thought and discussed it with some colleagues at Voltron Data. Something like this could be valuable in managed environments where there is a need to centrally define data sources across a fleet of systems. Perhaps it would also be valuable for individual system-level

Re: [DISCUSS] Conventions for transporting Arrow data over HTTP

2024-03-18 Thread Ian Cook
: https://github.com/apache/arrow/issues/40465 Ian On Tue, Mar 5, 2024 at 11:01 PM Ian Cook wrote: > Update on recent progress in this Arrow-over-HTTP project: > > I cleaned up the minimal examples of HTTP clients and servers and > moved them into a directory in the Arrow Experiments r

Re: [ANNOUNCE] New Arrow committer: Bryce Mecum

2024-03-18 Thread Ian Cook
Congratulations Bryce! Ian On Sun, Mar 17, 2024 at 22:24 Nic Crane wrote: > On behalf of the Arrow PMC, I'm happy to announce that Bryce Mecum has > accepted an invitation to become a committer on Apache Arrow. Welcome, and > thank you for your contributions! > > Nic >

Arrow community meeting March 13 at 16:00 UTC

2024-03-13 Thread Ian Cook
Our next biweekly Arrow community meeting is today at 16:00 UTC / 12:00 EDT. *** For attendees in countries that have not yet switched to Daylight Saving Time, please note that the time is one hour earlier than usual in your local time zone. *** Zoom meeting URL:

Re: [DISCUSS][MATLAB] Proposed "Category B" License for Bundling MATLAB MEX Build Artifacts in Official Arrow Release

2024-03-12 Thread Ian Cook
apache.org > > Cc: Kevin Gurney > > Subject: Re: [DISCUSS][MATLAB] Proposed "Category B" License for Bundling > > MATLAB MEX Build Artifacts in Official Arrow Release > > > > Hi Ian, > > > > Thanks for the feedback! We will proceed with the ASF Lega

Re: [DISCUSS] Conventions for transporting Arrow data over HTTP

2024-03-05 Thread Ian Cook
more languages (especially Rust) before we move on to developing richer types of examples. Is anyone interested in contributing additional minimal examples? Thanks, Ian On Wed, Dec 6, 2023 at 2:29 PM Ian Cook wrote: > > I just remembered that there is an unused "Arrow Experiments"

Arrow community meeting February 28 at 17:00 UTC

2024-02-27 Thread Ian Cook
Our next biweekly Arrow community meeting is tomorrow at 17:00 UTC / 12:00 EST. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Re: Arrow community meeting February 14 at 17:00 UTC

2024-02-14 Thread Ian Cook
JB > > On Wed, Feb 14, 2024 at 5:07 PM Ian Cook wrote: > > > > Thanks JB. I'll start by asking in today's meeting whether the usual > > attendees are available in the hour preceding the current meeting > > time. > > > > Ian > > > > On Wed, F

Re: Arrow community meeting February 14 at 17:00 UTC

2024-02-14 Thread Ian Cook
ng and the Iceberg Community Meeting. > > Thanks ! > Regards > JB > > On Wed, Feb 14, 2024 at 2:39 AM Ian Cook wrote: > > > > Our next biweekly Arrow community meeting is tomorrow at 17:00 UTC / 12:00 > > EST. > > > > Zoom meeting URL: > > https:

Arrow community meeting February 14 at 17:00 UTC

2024-02-13 Thread Ian Cook
Our next biweekly Arrow community meeting is tomorrow at 17:00 UTC / 12:00 EST. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Re: [DISCUSS][MATLAB] Proposed "Category B" License for Bundling MATLAB MEX Build Artifacts in Official Arrow Release

2024-01-26 Thread Ian Cook
suspected this may be the case and will be > sure to include all the relevant information when we file the Jira issue. > > Best, > > Sarah and Kevin > > From: Roman Shaposhnik > Sent: Friday, January 19, 2024 12:15 PM > To: dev@arrow.apache.org > Subject: Re: [DISCUSS][

Re: [DISCUSS][MATLAB] Proposed "Category B" License for Bundling MATLAB MEX Build Artifacts in Official Arrow Release

2024-01-18 Thread Ian Cook
Hi Sarah, Thanks for pursuing this. The ASF 3rd Party License Policy lists a number of standard, off-the-shelf licenses that are compatible with Category B, but the policy does not include any provision for custom-written licenses. This appears to be a custom-written license. Is that correct?

Arrow community meeting January 17 at 17:00 UTC

2024-01-16 Thread Ian Cook
Our next biweekly Arrow community meeting is tomorrow at 17:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Arrow community meeting January 3 at 17:00 UTC

2024-01-02 Thread Ian Cook
Our first biweekly Arrow community meeting of 2024 is tomorrow at 17:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Arrow community meeting December 20 at 17:00 UTC

2023-12-20 Thread Ian Cook
Our last biweekly Arrow community meeting of 2023 is today at 17:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Re: [ANNOUNCE] New Arrow committer: Felipe Oliveira Carvalho

2023-12-07 Thread Ian Cook
Congratulations Felipe!!! On Thu, Dec 7, 2023 at 10:43 AM Benjamin Kietzman wrote: > > On behalf of the Arrow PMC, I'm happy to announce that Felipe Oliveira > Carvalho > has accepted an invitation to become a committer on Apache > Arrow. Welcome, and thank you for your contributions! > > Ben

Re: [DISCUSS] Conventions for transporting Arrow data over HTTP

2023-12-06 Thread Ian Cook
com/apache/arrow-experiments [2] https://lists.apache.org/thread/cw14s874pwplzf9ycnvfwtwq0xq17npg Ian On Wed, Dec 6, 2023 at 1:45 PM Ian Cook wrote: > > Antoine, > > Thank you for taking a look. I agree—these are basic examples intended > to prove the concept and answer fundamental

Re: [DISCUSS] Conventions for transporting Arrow data over HTTP

2023-12-06 Thread Ian Cook
ervices. Especially, one > question is how to send both an application-specific POST request and an > Arrow stream, or an application-specific GET response and an Arrow > stream. This might necessitate some kind of framing layer, or a > standardized delimiter. > > Regards > > Antoine.

Arrow community meeting December 6 at 17:00 UTC

2023-12-05 Thread Ian Cook
Our next biweekly Arrow community meeting is tomorrow at 17:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

[DISCUSS] Conventions for transporting Arrow data over HTTP

2023-12-05 Thread Ian Cook
equest/response pair. Groonga > > can't return only the specified range result after the > > response is returned. > > > > > - recommendations about compression > > > > In the case that network is the bottleneck, LZ4 or Zstandard > > compression will i

Re: [ANNOUNCE] New Arrow PMC chair: Andy Grove

2023-11-28 Thread Ian Cook
Thank you Andy and Andrew for your service to the Arrow community! On Tue, Nov 28, 2023 at 10:36 AM Andy Grove wrote: > Thank you all. I look forward to helping the project in this new role over > the next year. > > Thanks to Andrew for handling this for the past year. > > On Tue, Nov 28, 2023

Arrow community meeting November 22 at 17:00 UTC

2023-11-21 Thread Ian Cook
Our next biweekly Arrow community meeting is tomorrow at 17:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Re: [DISCUSS] Protocol for exchanging Arrow data over REST APIs

2023-11-20 Thread Ian Cook
need to be REST-ful. > >> Something like JSON-RPC might fit well with the existing model for Arrow > >> over the wire that's been implemented in things like Flight/FlightSQL. > >> > >> Something else I've been interested in (I think Matt Topol has done > work in &

Re: [DISCUSS] Protocol for exchanging Arrow data over REST APIs

2023-11-18 Thread Ian Cook
Hi Kou, I think it is too early to make a specific proposal. I hope to use this discussion to collect more information about existing approaches. If several viable approaches emerge from this discussion, then I think we should make a document listing them, like you suggest. Thank you for the

Re: [ANNOUNCE] New Arrow committer: James Duong

2023-11-17 Thread Ian Cook
Congratulations James! On Thu, Nov 16, 2023 at 3:45 AM Sutou Kouhei wrote: > On behalf of the Arrow PMC, I'm happy to announce that James Duong > has accepted an invitation to become a committer on Apache > Arrow. Welcome, and thank you for your contributions! > > -- > kou > > >

[DISCUSS] Protocol for exchanging Arrow data over REST APIs

2023-11-17 Thread Ian Cook
Several recent discussions have highlighted the lack of an established specification / protocol for sending Arrow-formatted data through REST APIs. I would like to start a discussion here to gauge interest and gather ideas about this. For background: Flight RPC provides a framework for building

Re: [ANNOUNCE] New Arrow PMC member: Raúl Cumplido

2023-11-13 Thread Ian Cook
Congratulations Raúl! On Mon, Nov 13, 2023 at 2:28 PM Andrew Lamb wrote: > > The Project Management Committee (PMC) for Apache Arrow has invited > Raúl Cumplido to become a PMC member and we are pleased to announce > that Raúl Cumplido has accepted. > > Please join me in congratulating them. >

Arrow community meeting November 8 at 17:00 UTC

2023-11-07 Thread Ian Cook
Our next biweekly Arrow community meeting is tomorrow at 17:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Arrow community meeting October 25 at 16:00 UTC

2023-10-25 Thread Ian Cook
Our next biweekly Arrow community meeting is today at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Re: [ANNOUNCE] New Arrow committer: Xuwei Fu

2023-10-23 Thread Ian Cook
Congratulations Xuwei! On Mon, Oct 23, 2023 at 12:46 AM Sutou Kouhei wrote: > > On behalf of the Arrow PMC, I'm happy to announce that Xuwei Fu > has accepted an invitation to become a committer on Apache > Arrow. Welcome, and thank you for your contributions! > > -- > kou

Re: [ANNOUNCE] New Arrow committer: Curt Hagenlocher

2023-10-15 Thread Ian Cook
Congratulations Curt! On Sun, Oct 15, 2023 at 05:32 Andrew Lamb wrote: > On behalf of the Arrow PMC, I'm happy to announce that Curt Hagenlocher > has accepted an invitation to become a committer on Apache > Arrow. Welcome, and thank you for your contributions! > > Andrew >

Re: [ANNOUNCE] New Arrow PMC member: Jonathan Keane

2023-10-14 Thread Ian Cook
Congratulations Jonathan! On Sat, Oct 14, 2023 at 13:24 Andrew Lamb wrote: > The Project Management Committee (PMC) for Apache Arrow has invited > Jonathan Keane to become a PMC member and we are pleased to announce > that Jonathan Keane has accepted. > > Congratulations and welcome! > > Andrew

Arrow community meeting October 11 at 16:00 UTC

2023-10-11 Thread Ian Cook
Our next biweekly Arrow community meeting is today at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Re: [DISCUSS][C++] Raw pointer string views

2023-09-29 Thread Ian Cook
I strongly agree with Ben's assertion that "the risk of a parallel ecosystem… is more likely to be provoked by excluding a user's vital use case [than by implementing support for an unofficial layout variant]" in the C++ library. But there seems to be a consensus here that there is a real risk of

Re: [VOTE][Format] Add ListView and LargeListView Arrays to Arrow Format

2023-09-29 Thread Ian Cook
+1 (non-binding) Thanks very much Felipe for your persistence and your commitment to addressing the numerous questions and comments that have been raised since the beginning of the discussion on this in April. On Fri, Sep 29, 2023 at 12:34 PM Benjamin Kietzman wrote: > > +1 > > On Fri, Sep 29,

Arrow community meeting September 27 at 16:00 UTC

2023-09-27 Thread Ian Cook
Our next biweekly Arrow community meeting is today at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Arrow community meeting September 13 at 16:00 UTC

2023-09-12 Thread Ian Cook
Our next biweekly Arrow community meeting is tomorrow at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Arrow community meeting August 30 at 16:00 UTC

2023-08-30 Thread Ian Cook
Our next biweekly Arrow community meeting is today at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Re: [Python][Discuss] PyArrow Dataset as a Python protocol

2023-08-29 Thread Ian Cook
; > That wouldn't remove the feature from DuckDB, would it? It would just > > > > mean > > > > > that we recognize that PyArrow expressions don't have well-defined > > > > > semantics that we are committing to at this time. As long as we have > > &

Re: Sort a Table In C++?

2023-08-17 Thread Ian Cook
Li, Here's a standalone C++ example that constructs a Table and executes an Acero ExecPlan to sort it: https://gist.github.com/ianmcook/2aa9aa82e61c3ea4405450b93cf80fbc Ian On Thu, Aug 17, 2023 at 4:50 PM Li Jin wrote: > > Hi, > > I am writing some C++ test and found myself in need for an c++

Re: [Vote][Format] C Data Interface Format string for REE

2023-08-16 Thread Ian Cook
+1 (non-binding) On Wed, Aug 16, 2023 at 10:16 AM Matt Topol wrote: > > Hey All, > > As proposed by Felipe [1] I'm starting a vote on the proposed update to the > Format Spec of adding "+r" as the format string for passing Run-End Encoded > arrays through the Arrow C Data Interface. > > A PR

Arrow community meeting August 16 at 16:00 UTC

2023-08-16 Thread Ian Cook
Our next biweekly Arrow community meeting is today at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Arrow community meeting August 2 at 16:00 UTC

2023-08-02 Thread Ian Cook
Our next biweekly Arrow community meeting is today at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Re: [QUESTION] Syndication site(s) for Apache Arrow related content?

2023-07-21 Thread Ian Cook
+1. Something like this would be quite valuable, especially if it highlighted successful real-world applications of Arrow and helped to disseminate to a broader audience the news of Arrow's increasing adoption as a standard. We should name it "This IntervalUnit in Arrow" On Fri, Jul 21, 2023

Arrow community meeting July 19 at 16:00 UTC

2023-07-19 Thread Ian Cook
Our next biweekly Arrow community meeting is today at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Re: [DISCUSS] Canonical alternative layout proposal

2023-07-13 Thread Ian Cook
Thank you Weston for proposing this solution and Neal for describing its context and implications. I agree with the other replies here—this seems like an elegant solution to a growing need that could, if left unaddressed, increase the fragmentation of the ecosystem and reduce the centrality of the

Arrow community meeting July 5 at 16:00 UTC

2023-07-04 Thread Ian Cook
Our next biweekly Arrow community meeting is tomorrow at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Re: [Python][Discuss] PyArrow Dataset as a Python protocol

2023-06-27 Thread Ian Cook
" and there are > > > some parallels between RecordBatch/RecordBatchReader and > > > Fragment/Dataset. > > > > > > > This is a good point. I can add a section describing the differences. The > > main ones I can think of are that: (1) Datasets are &q

Re: [Python][Discuss] PyArrow Dataset as a Python protocol

2023-06-23 Thread Ian Cook
Thanks Will for this proposal! For anyone familiar with PyArrow, this idea has a clear intuitive logic to it. It provides an expedient solution to the current lack of a practical means for interchanging "unmaterialized dataframes" between different Python libraries. To elaborate on that: If you

Re: [ANNOUNCE] New Arrow PMC member: Dewey Dunnington

2023-06-23 Thread Ian Cook
Congratulations Dewey! On Fri, Jun 23, 2023 at 10:03 AM Matt Topol wrote: > > Congrats Dewey!! > > On Fri, Jun 23, 2023, 9:35 AM Dane Pitkin > wrote: > > > Congrats Dewey! > > > > On Fri, Jun 23, 2023 at 9:15 AM Nic Crane wrote: > > > > > Well-deserved Dewey, congratulations! > > > > > > On

Arrow community meeting June 21 at 16:00 UTC

2023-06-21 Thread Ian Cook
Our next biweekly Arrow community meeting is today at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Re: [Parquet C++] Plan to bump default write version from 2.4 -> 2.6 (include nanoseconds LogicalType)

2023-06-15 Thread Ian Cook
It will still be possible to write files using Parquet 2.4 by explicitly specifying the 2.4 version to the Parquet writer, correct? If yes, that provides a simple workaround for users who encounter compatibility issues. However we should take care to document this as a potentially breaking

Arrow community meeting June 7 at 16:00 UTC

2023-06-07 Thread Ian Cook
Our next biweekly Arrow community meeting is today at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 Meeting notes will be captured in this Google Doc:

Re: [DISCUSS][Format] Starting the draft implementation of the ArrayView array format

2023-06-06 Thread Ian Cook
arrow compatible > * Receives and emits both primary and alternative layouts - arrow > compatible† > > † - with the caveat that the primary layout must be emitted if the user > does not > specifically request the alternative layout. > > [1] > https://arrow.apache.org/docs/form

Re: [DISCUSS][Format] Starting the draft implementation of the ArrayView array format

2023-06-06 Thread Ian Cook
To clarify why we cannot simply propose adding ListView as a new “canonical extension type”: The extension type mechanism in Arrow depends on the underlying data being organized in an existing Arrow layout—that way an implementation that does not support the extension type can still handle the

Re: [VOTE][Format] Add experimental ArrowDeviceArray to C-Data API

2023-05-31 Thread Ian Cook
+1 (non-binding). Thanks very much Matt for all the work you did here to solicit input from other stakeholder communities. On Mon, May 22, 2023 at 12:02 PM Matt Topol wrote: > Hello, > > Now that there's a rough consensus and a toy example POC[1], I would like > to propose an official

Arrow community meeting May 24 at 16:00 UTC

2023-05-23 Thread Ian Cook
Hi all, Our biweekly Arrow community meeting is tomorrow at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 The notes for this and future instances of this meeting will be captured in this

Re: [DISCUSS][Format] Starting the draft implementation of the ArrayView array format

2023-05-19 Thread Ian Cook
would allow us to bitmask out mvcc rows that have been deleted / have > not yet been committed / have been rolled back, etc. > > - Brent > > On Mon, May 15, 2023, 06:55 Ian Cook wrote: > > > I think it would be easier for us all to weigh the costs and benefits > &g

Re: [DISCUSS] Interest in a 12.0.1 patch?

2023-05-18 Thread Ian Cook
There is also a major issue with the 12.0.0 R package that has now been fixed in the repo [2] and needs to be resubmitted to CRAN soon. The R package developers are supportive of a 12.0.1 patch release happening soon so that the resubmission of the R package to CRAN can also include the fix for

Re: [DISCUSS][Gandiva] changes in bundled double-conversion

2023-05-18 Thread Ian Cook
ot; on Mon, 1 May > 2023 19:01:31 -0400, > Ian Cook wrote: > > > Hi Kou, > > > > Thank you. I think this is a reasonable approach. > > > > I added a comment asking if the PR author can please update the PR by > > porting the changes from PR #9816.

Re: [ANNOUNCE] New Arrow committer: Gang Wu

2023-05-15 Thread Ian Cook
Congratulations Gang! On Mon, May 15, 2023 at 9:47 AM vin jake wrote: > > Congrats Gang! > > On Mon, May 15, 2023 at 9:33 PM Sutou Kouhei wrote: > > > On behalf of the Arrow PMC, I'm happy to announce that Gang > > Wu has accepted an invitation to become a committer on > > Apache Arrow.

Re: [DISCUSS][Format] Starting the draft implementation of the ArrayView array format

2023-05-15 Thread Ian Cook
I think it would be easier for us all to weigh the costs and benefits of adding this proposed ListView layout to the Arrow specification and implementing it in the various Arrow libraries if we could all see some benchmarks demonstrating the performance/efficiency benefits compared to Arrow’s

Re: Arrow community meeting May 10 at 16:00 UTC

2023-05-10 Thread Ian Cook
Below is a summary of the notes from today's meeting. Attendees: - Ian Cook - Raúl Cumplido - Dewey Dunnington - Xuwei Fu - Will Jones - David Li - Ashish Paliwal - Dane Pitkin - Matthew Topol - Joris Van den Bossche Discussion: Arrow 12.0.0 release - Release is complete - Most post-release

Arrow community meeting May 10 at 16:00 UTC

2023-05-09 Thread Ian Cook
Hi all, Our biweekly Arrow community meeting is tomorrow at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 The notes for this and future instances of this meeting will be captured in this

Re: [ANNOUNCE] New Arrow PMC member: Matt Topol

2023-05-03 Thread Ian Cook
Congratulations Matt!!! On Wed, May 3, 2023 at 9:55 PM Yibo Cai wrote: > > Congrats Matt! > > On 5/4/23 07:07, Krisztián Szűcs wrote: > > Congrats Matt! > > > > On Wed, May 3, 2023 at 11:44 PM Rok Mihevc wrote: > >> > >> Congrats Matt. Well deserved! > >> > >> Rok > >> > >> On Wed, May 3, 2023

Re: [DISCUSS][Gandiva] changes in bundled double-conversion

2023-05-01 Thread Ian Cook
ng our changes with upstream. > > Does anyone want to upstream our changes? It seems that our > changes break a compatibility. So I think that we need to > explain our use-case to upstream. > > > Thanks, > -- > kou > > In > "Re: [DISCUSS][Gandiva] chan

Re: [DISCUSS][Gandiva] changes in bundled double-conversion

2023-05-01 Thread Ian Cook
Looking at PR #9816 which is the PR that introduced downstream changes to our vendored copy of double-conversion, it appears that the changes were quite small: two files modified, fewer than 10 lines of added code, plus some comments [1]. If this is correct, then I think the easiest path forward

Re: [WEBSITE] [DISCUSS] Arrow-Site blog post

2023-04-28 Thread Ian Cook
Hi Matt, I reviewed it and left a few very minor comments. Looks great to me. Do any PMC members wish to chime in? If not, it seems OK to give it 72 hours from the time of your email here and then merge it. Thanks, Ian On Fri, Apr 28, 2023 at 11:41 AM Matt Topol wrote: > > Hey All, > >

Re: Arrow community meeting April 26 at 16:00 UTC

2023-04-27 Thread Ian Cook
Below is a summary of the notes from yesterday's meeting: Attendees: - Ian Cook - Raúl Cumplido - Xuwei Fu - Will Jones - Bryce Mecum - Rok Mihevc - Sri Nadukudy - Matthew Topol Discussion: Arrow 12.0.0 release - RC0 has been proposed [1] - There were a lot of CI failures at the time

Re: [DISCUSS][Format] Starting the draft implementation of the ArrayView array format

2023-04-26 Thread Ian Cook
+1 to what Weston and Joris suggested regarding the name. "ListView" seems like the best name to use for this layout in Arrow. My understanding is that the primary benefit of this ListView layout over Arrow's existing List layouts [1] is that ListView allows for buffer alignment [2] without

Arrow community meeting April 26 at 16:00 UTC

2023-04-25 Thread Ian Cook
Hi all, Our biweekly Arrow community meeting is tomorrow at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 The notes for this and future instances of this meeting will be captured in this

Re: Arrow community meeting April 12 at 16:00 UTC

2023-04-12 Thread Ian Cook
Below is a summary of the notes from today's meeting: Attendees: - Ian Cook - Raúl Cumplido - Xuwei Fu - Will Jones - Bryce Mecum - Rok Mihevc - Sri Nadukudy - Ashish Paliwal - Dane Pitkin - David Dali Susanibar Arce - Matthew Topol - Joris Van den Bossche - Jacob Wujciak Discussion: 12.0.0

Arrow community meeting April 12 at 16:00 UTC

2023-04-11 Thread Ian Cook
Hi all, Our biweekly Arrow community meeting is tomorrow at 16:00 UTC / 12:00 EDT. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 The notes for this and future instances of this meeting will be captured in this

Re: Arrow community meeting March 29 at 16:00 UTC

2023-04-01 Thread Ian Cook
Below is a summary of the notes from this week's meeting: Attendees: - Ian Cook - Will Jones - David Li - Rok Mihevc - Sri Nadukudy - Dane Pitkin Discussion: Questions about versioning, packaging, releasing the Rust ADBC API [1] - ADBC drivers are packaged in native language, and can

Arrow community meeting March 29 at 16:00 UTC

2023-03-28 Thread Ian Cook
Hi all, Our biweekly Arrow community meeting is tomorrow at 16:00 UTC / 12:00 EDT. I expect that this meeting might run shorter than usual because all the attendees from Voltron Data will need to leave to join another meeting at 12:30 EDT. Zoom meeting URL:

Re: Arrow community meeting March 15 at 16:00 UTC

2023-03-15 Thread Ian Cook
Below is a summary of the notes from this week's meeting Attendees: - Ian Cook - Raúl Cumplido - Alenka Frim - Will Jones - David Li - Bryce Mecum - Rok Mihevc - Sri Nadukudy - Weston Pace - Dane Pitkin - Matthew Topol - Joris Van den Bossche Discussion: Arrow 12.0.0 Release - Planned date

Arrow community meeting March 15 at 16:00 UTC

2023-03-15 Thread Ian Cook
Hi all, Our biweekly Arrow community meeting is today at 16:00 UTC / 12:00 EST. For attendees in countries that have not yet switched to Daylight Saving Time, please note that the time is one hour earlier than usual in your local time zone. Zoom meeting URL:

Re: [ANNOUNCE] New Arrow PMC member: Will Jones

2023-03-13 Thread Ian Cook
Congratulations Will! On Mon, Mar 13, 2023 at 1:58 PM Andrew Lamb wrote: > > The Project Management Committee (PMC) for Apache Arrow has invited > Will Jones to become a PMC member and we are pleased to announce > that Will Jones has accepted. > > Congratulations and welcome!

Re: Arrow community meeting March 1 at 17:00 UTC

2023-03-01 Thread Ian Cook
Below is a summary of the notes from this week's meeting Attendees: - Ian Cook - Raúl Cumplido - Dewey Dunnington - Ian Joiner - Will Jones - David Li - Bryce Mecum - Rok Mihevc - Sri Nadukudy - Weston Pace - Dane Pitkin Discussion: Fixed Shape Tensor canonical ExtensionType

Arrow community meeting March 1 at 17:00 UTC

2023-02-28 Thread Ian Cook
Hi all, Our biweekly Arrow community meeting is tomorrow at 17:00 UTC / 12:00 EST. Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 958092 The notes for this and future instances of this meeting will be captured in this

  1   2   3   >