I have submitted the board report. The final report is copy/pasted below for your reference.
Thank you again to everyone who helped make this happen. Andrew ## Description: The mission of Apache Arrow is the creation and maintenance of software related to columnar in-memory processing and data interchange. More information can be found at https://arrow.apache.org/overview/ ## Project Status: Current project status: Ongoing (high activity) Issues for the board: None ## Membership Data: Apache Arrow was founded 2016-01-19 (8 years ago) There are currently 99 committers and 50 PMC members in this project. The Committer-to-PMC ratio is roughly 7:4. Community changes, past quarter: - No new PMC members. Last addition was Dewey Dunnington on 2023-06-22. - Oleks V. was added as committer on 2023-09-30 - Metehan Yildirim was added as committer on 2023-08-29 ## Project Activity: We added a new array layout, [Utf8View] to the Arrow spec, which allows more efficient variable length string handling. There has also been much [Utf8ViewDiscussion] about how to use this new layout to interoperate better with the broader ecosystem We have started adding cross-language integration tests for the [C Data Interface], ensuring that implementations comply with the spec and are concretely interoperable. This is in addition to the existing integration tests for the [IPC format] and for the [Flight RPC] protocol. We registered our DOAP file: https://projects.apache.org/project.html?arrow [Utf8View]: https://s.apache.org/ldd1b [Utf8ViewDiscussion]: https://s.apache.org/a1v8o [C Data Interface]: https://arrow.apache.org/docs/format/CDataInterface.html [IPC format]: https://s.apache.org/5uwhb [Flight RPC]: https://arrow.apache.org/docs/format/Flight.html [ListView and LargeListView]: We have also finalized adding alternative layouts to variable-length list arrays that require an extra buffer with sizes, a layout found in other systems such as DuckDB and Velox. This in turn allows offsets to be out of order. Elements of the child array do not have to be stored in the same order they logically appear in the list elements of the parent array. [C Data Interface]: ArrowDeviceArray and ArrowDeviceArrayStream structs were added to the spec to allow support for Arrow Arrays whose buffers are on a non-CPU device. ## Sub Project Updates Arrow has several subprojects, as listed on https://arrow.apache.org/ ### ADBC Version 0.7.0 was released, along with a 1.1.0 revision of the API specification it implements. ### Arrow Flight We added support for long-running queries use case. ### Arrow Flight SQL There have been a few discussions about small improvements as systems implement FlightSQL more broadly ### Arrow Flight SQL adapter for PostgreSQL This is a new sub project that adds Arrow Flight SQL endpoint to PostgreSQL. We have released the first version: https://arrow.apache.org/blog/2023/09/13/flight-sql-postgresql-0.1.0-release/ ### DataFusion DataFusion continues releasing regularly. We are working on a paper describing the system for ACM SIGMOD, and in general are trying to scale the project as it grows in popularity. We have added User Defined Window functions, much improved List and Struct support, Spillable Hashing, better join performance, parallel writes, and a new library user guide. ### Acero No Update ### nanoarrow We released version 0.3.0, which focused on solidifying the C library and IPC implementation. ## Language Area Updates Arrow has at least 13 different language implementations, as explained in https://arrow.apache.org/overview/ Arrow 13.0.0 was released from the monorepo: https://arrow.apache.org/blog/2023/08/24/13.0.0-release/ ### C++ StringView and ListView data types were added to the C++ implementation, along with ArrowDeviceArray and ArrowDeviceArrayStreams for non-CPU devices. ### C# The C# implementation gained support for Flight SQL servers and clients. It also added support for fixed-size lists, dense and sparse unions and maps, and is now complete enough to express the types used in ADBC schemas. Its C API support has been hooked up to the formal C Data interface integration testing. ### Go StringView and ListView data types are added to the Go implementation. ### Java Support for Java 21 has been added. ### JavaScript ### Julia No highlight. ### Rust The Rust implementation reworked the computation kernels, and is in the process of adding StringView. ### C (GLib) We have added new bindings continually as usual. ### MATLAB We started to use GitHub Projects to organize the MATLAB bindings tasks: https://github.com/orgs/apache/projects/289. We have also continued to expand the MATLAB bindings by adding tabular types (i.e. RecordBatch and Table), more Array types (struct, date, time, timestamp, …), and CSV file I/O. We are actively working towards cutting an initial “0.1” release of the MATLAB bindings in the coming weeks. ### Python ### R Added bindings to the C++ functionality for reading ND-JSON, and continued to work to ensure dplyr compatibility. ### Ruby We have added new bindings continually as usual. ### Swift No highlight. ## Recent Releases: * NANOARROW-0.3.0 was released on 2023-09-29. * RS-OS-0.7.1 was released on 2023-09-29. * ADBC-0.7.0 was released on 2023-09-23. * RS-47.0.0 was released on 2023-09-22. * RS-DATAFUSION-PYTHON-31.0.0 was released on 2023-09-18. * FLIGHT-SQL-POSTGRESQL-0.1.0 was released on 2023-09-13. * RS-DATAFUSION-31.0.0 was released on 2023-09-11. * ADBC-0.6.0 was released on 2023-08-28. * RS-DATAFUSION-30.0.0 was released on 2023-08-25. * RS-46.0.0 was released on 2023-08-24. * 13.0.0 was released on 2023-08-23. * RS-OS-0.7.0 was released on 2023-08-18. * RS-DATAFUSION-29.0.0 was released on 2023-08-14. * RS-45.0.0 was released on 2023-08-02. * RS-DATAFUSION-28.0.0 was released on 2023-07-25. * RS-44.0.0 was released on 2023-07-18. ## Community Health: Community communication continues to be strong. There have been 5 blog posts published to https://arrow.apache.org/blog/ in the last 3 months. The mailing lists are active * dev@arrow.apache.org had a 32% decrease in traffic in the past quarter (545 emails compared to 798) * u...@arrow.apache.org had a 37% decrease in traffic in the past quarter (76 emails compared to 120) For the mono repo: * 2266 commits in the past quarter (-2% change) * 249 code contributors in the past quarter (no change) * 1788 PRs opened on GitHub, past quarter (-12% change) * 1682 PRs closed on GitHub, past quarter (-16% change) * 1499 issues opened on GitHub, past quarter (-8% change) * 1211 issues closed on GitHub, past quarter (-11% change) On Mon, Oct 9, 2023 at 9:53 AM Andrew Lamb <al...@influxdata.com> wrote: > Thank you to everyone who has already contributed updates to the overall > project update. It is amazingly broad and is as always neat to see a bit of > what each area is working on. > > As a reminder, I plan to submit this report this Wednesday October 11, > 2023 (my original email said October 14). If anyone has additional > commentary to add, please do so by replying to this email or adding it > directly to the document[1] > > Thanks, > Andrew > > [1] > https://docs.google.com/document/d/1MU5cxzVuAIuDb6KXOAkwT4ze7IBGHKks_l92gxZeTbg/edit > > > > On Thu, Oct 5, 2023 at 11:53 AM Kevin Gurney <kgur...@mathworks.com.invalid> > wrote: > >> Hi All, >> >> @Andrew - thanks for organizing this! >> @Kou - thank you for adding notes about the MATLAB bindings! >> >> Sarah (Cc'd) and I added a few more details about progress on the MATLAB >> bindings. >> >> Best Regards, >> >> Kevin Gurney >> >> ________________________________________ >> From: Sutou Kouhei <k...@clear-code.com> >> Sent: Wednesday, October 4, 2023 5:49 PM >> To: dev@arrow.apache.org >> Subject: Re: [CROWDSOURCING] 2023 ASF Board Report -- October 11, 2023 >> >> Hi Andrew, >> >> Thanks for preparing this! I've added some comments for >> Flight RPC, Apache Arrow Flight SQL adapter for PostgreSQL, >> C GLib, MATLAB, Ruby, Swift and so on. >> >> >> Thanks, >> -- >> ko >> >> In <cafhtnrzntty3xqv7r1hgxljdpnvsy2u+sl8xdlc0isqvdht...@mail.gmail.com> >> "[CROWDSOURCING] 2023 ASF Board Report -- October 11, 2023" on Thu, 28 >> Sep 2023 06:33:22 -0400, >> Andrew Lamb <al...@influxdata.com> wrote: >> >> > Hello Arrow Community, >> > >> > Please add any comments or board content directly to [1] or reply to >> > this email and I will incorporate your comments. You can see what we >> > currently have at the end of this email. >> > >> > One of the responsibilities of being part of the Apache Software >> Foundation >> > (ASF) is to regularly summarize the state of the project in a quarterly >> > update to the ASF board. I plan to submit the next report on October 14, >> > 2023 >> > >> > While this is partly an administrative reporting exercise, I think it is >> > also valuable to reflect on past accomplishments and think about goals >> for >> > the future. >> > >> > Historically, Arrow has crowd sourced the content which has worked well. >> > It would be especially interesting and valuable for members of the >> various >> > language >> > implementation communities and subprojects could provide a sentence or >> two >> > updates >> > >> > Thank you, >> > Andrew >> > >> > [1]: >> > >> https://docs.google.com/document/d/1MU5cxzVuAIuDb6KXOAkwT4ze7IBGHKks_l92gxZeTbg/edit >> < >> https://docs.google.com/document/d/1MU5cxzVuAIuDb6KXOAkwT4ze7IBGHKks_l92gxZeTbg/edit >> > >> > >> > -- >> > >> > Current content >> > >> > ## Description: >> > The mission of Apache Arrow is the creation and maintenance of software >> > related to columnar in-memory processing and data interchange. More >> > information can be found at https://arrow.apache.org/overview/< >> https://arrow.apache.org/overview> >> > >> > ## Project Status: >> > >> > Current project status: Ongoing (high activity) >> > >> > Issues for the board: None >> > >> > ## Membership Data: >> > >> > Apache Arrow was founded 2016-01-19 (8 years ago) >> > There are currently 98 committers and 50 PMC members in this project. >> > The Committer-to-PMC ratio is roughly 7:4. >> > >> > Community changes, past quarter: >> > - No new PMC members. Last addition was Dewey Dunnington on 2023-06-22. >> > - Kevin Gurney was added as committer on 2023-07-04 >> > - Metehan Yildirim was added as committer on 2023-08-29 >> > >> > >> > ## Project Activity: >> > >> > We added a new array layout, [Utf8View] to the Arrow spec, which allows >> > more efficient variable length string handling. >> > >> > >> > >> > [Utf8View]: >> https://lists.apache.org/thread/wt9j3q7qd59cz44kyh1zkts8s6wo1dn6< >> https://lists.apache.org/thread/wt9j3q7qd59cz44kyh1zkts8s6wo1dn6> >> > >> > >> > ## Sub Project Updates >> > Arrow has several subprojects, as listed on https://arrow.apache.org/< >> https://arrow.apache.org> >> > >> > ### ADBC >> > >> > >> > >> > >> > ### Arrow Flight >> > >> > >> > >> > >> > ### Arrow Flight SQL >> > >> > >> > >> > >> > ### DataFusion >> > >> > >> > >> > ### Acero >> > >> > >> > >> > ## Language Area Updates >> > >> > Arrow has at least 12 different language implementations, as explained >> in >> > https://arrow.apache.org/overview/<https://arrow.apache.org/overview/> >> > >> > Arrow 12.0.0 was released from the monorepo: >> > https://arrow.apache.org/blog/2023/05/02/12.0.0-release/< >> https://arrow.apache.org/blog/2023/05/02/12.0.0-release> >> > >> > ### C++ >> > >> > >> > >> > ### C# >> > >> > >> > >> > ### Go >> > >> > >> > >> > ### Java >> > >> > >> > ### JavaScript >> > >> > ### Julia >> > >> > >> > >> > ### nanoarrow >> > >> > >> > >> > ### Rust >> > >> > The Rust implementation is in the process of adding StringView. >> > >> > ### C (GLib) >> > >> > >> > ### MATLAB >> > >> > >> > >> > ### Python >> > >> > >> > >> > ### R >> > >> > >> > >> > ### Ruby >> > >> > >> > ### Swift >> > >> > >> > ## Community Health: >> > Community communication continues to be strong. >> > >> > There have been 5 blog posts published to >> https://arrow.apache.org/blog/<https://arrow.apache.org/blog> in >> > the last 3 months. >> > >> > The mailing lists are active >> > >> > * dev@arrow.apache.org had a 29% decrease in traffic in the past >> quarter >> > (568 emails compared to 789) >> > * u...@arrow.apache.org had a 52% decrease in traffic in the past >> quarter >> > (65 emails compared to 133) >> > >> > >> > For the mono repo: >> > >> > * 2328 commits in the past quarter (2% increase) >> > * 246 code contributors in the past quarter (-3% change) >> > * 1802 PRs opened on GitHub, past quarter (-13% change) >> > * 1740 PRs closed on GitHub, past quarter (-14% change) >> > * 1515 issues opened on GitHub, past quarter (-10% change) >> > * 1234 issues closed on GitHub, past quarter (-12% change) >> >> >>
