tustvold commented on code in PR #305:
URL: https://github.com/apache/arrow-site/pull/305#discussion_r1103860013
##########
_posts/2023-01-26-rust-32.0.0.md:
##########
@@ -0,0 +1,193 @@
+---
+layout: post
+title: "January 2023 Rust Apache Arrow Highlights"
+date: "2023-01-26 00:00:00 +0000"
+author: pmc
+categories: [release]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+# Introduction
+
+With the recent release of [32.0.0](https://crates.io/crates/arrow/32.0.0) of
the Rust implementation of [Apache Arrow](https://arrow.apache.org/), it seemed
timely to highlight some of the communities work since the [last
update](https://arrow.apache.org/blog/2022/06/16/rust-16.0.0/).
+
+The most recent list of detailed changes can always be found in the
[CHANGELOG](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md), with
the full historical list available
[here](https://github.com/apache/arrow-rs/blob/master/CHANGELOG-old.md).
+
+# Arrow
+
+[arrow](https://crates.io/crates/arrow) and
[arrow-flight](https://crates.io/crates/arrow-flight) are native Rust
implementations of Apache Arrow. Apache Arrow defines a language-independent
columnar memory format for flat and hierarchical data, organized for efficient
analytic operations on modern hardware like CPUs and GPUs. The Arrow memory
format also supports zero-copy reads for lightning-fast data access without
serialization overhead.
+
+The [Rust language](https://www.rust-lang.org/) offers best in class
performance, memory safety, and the developer productivity of a modern
programming language. These features make Rust an excellent choice for building
modern high performance analytical systems. When combined, Rust and the Apache
Arrow Ecosystem are a compelling toolkit for building the next generation of
systems.
+
+The [repository](https://github.com/apache/arrow-rs) recently passed 1400
stars on github, and the community has been focused on performance and feature
completeness.
+
+
+**Major Highlights**
+
+* **New CSV and JSON Readers**: The CSV and JSON readers have been revamped.
Their performance has more than doubled, and they now support push-driven
parsing, and better streaming object storage integration.
+* **Faster Build Times and Reduced Codegen**: The `arrow` crate has been split
into multiple smaller crates, and large kernels have been moved behind optional
feature flags. These changes allow downstream projects to choose a smaller
dependency footprint and build times, if desired.
+* **Support for Copy-On-Write**: Arrow arrays now support copy-on-write, via
the
[`into_builder`](https://docs.rs/arrow/32.0.0/arrow/array/struct.ArrayData.html#method.into_builder)
methods
+* **Comparable Row Format**: [Much faster multi-column Sorting and
Grouping](https://arrow.apache.org/blog/2022/11/07/multi-column-sorts-in-arrow-rust-part-1/)
is now possible with the the new spillable, comparable
[row-format](https://docs.rs/arrow-row/32.0.0/arrow_row/index.html)
+* **FlightSQL Support**:
[FlightSQL](https://arrow.apache.org/docs/format/FlightSql.html)
[support](https://docs.rs/arrow-flight/32.0.0/arrow_flight/sql/index.html) has
been expanded
+* **Mid-Level Flight Client**: A new
[FlightClight](https://docs.rs/arrow-flight/32.0.0/arrow_flight/client/struct.FlightClient.html)
is available that handles lower level protocol details, and easier to use
[encoding](https://docs.rs/arrow-flight/32.0.0/arrow_flight/encode/struct.FlightDataEncoderBuilder.html)
and
[decoding](https://docs.rs/arrow-flight/32.0.0/arrow_flight/decode/struct.FlightDataDecoder.html)
APIs.
Review Comment:
```suggestion
* **Mid-Level Flight Client**: A new
[FlightClight](https://docs.rs/arrow-flight/32.0.0/arrow_flight/client/struct.FlightClient.html)
is available that handles lower level protocol details, and easier to use
[encoding](https://docs.rs/arrow-flight/32.0.0/arrow_flight/encode/struct.FlightDataEncoderBuilder.html)
and
[decoding](https://docs.rs/arrow-flight/32.0.0/arrow_flight/decode/struct.FlightDataDecoder.html)
APIs.
```
##########
_posts/2023-01-26-rust-32.0.0.md:
##########
@@ -0,0 +1,193 @@
+---
+layout: post
+title: "January 2023 Rust Apache Arrow Highlights"
+date: "2023-01-26 00:00:00 +0000"
+author: pmc
+categories: [release]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+# Introduction
+
+With the recent release of [32.0.0](https://crates.io/crates/arrow/32.0.0) of
the Rust implementation of [Apache Arrow](https://arrow.apache.org/), it seemed
timely to highlight some of the communities work since the [last
update](https://arrow.apache.org/blog/2022/06/16/rust-16.0.0/).
+
+The most recent list of detailed changes can always be found in the
[CHANGELOG](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md), with
the full historical list available
[here](https://github.com/apache/arrow-rs/blob/master/CHANGELOG-old.md).
+
+# Arrow
+
+[arrow](https://crates.io/crates/arrow) and
[arrow-flight](https://crates.io/crates/arrow-flight) are native Rust
implementations of Apache Arrow. Apache Arrow defines a language-independent
columnar memory format for flat and hierarchical data, organized for efficient
analytic operations on modern hardware like CPUs and GPUs. The Arrow memory
format also supports zero-copy reads for lightning-fast data access without
serialization overhead.
+
+The [Rust language](https://www.rust-lang.org/) offers best in class
performance, memory safety, and the developer productivity of a modern
programming language. These features make Rust an excellent choice for building
modern high performance analytical systems. When combined, Rust and the Apache
Arrow Ecosystem are a compelling toolkit for building the next generation of
systems.
+
+The [repository](https://github.com/apache/arrow-rs) recently passed 1400
stars on github, and the community has been focused on performance and feature
completeness.
+
+
+**Major Highlights**
+
+* **New CSV and JSON Readers**: The CSV and JSON readers have been revamped.
Their performance has more than doubled, and they now support push-driven
parsing, and better streaming object storage integration.
+* **Faster Build Times and Reduced Codegen**: The `arrow` crate has been split
into multiple smaller crates, and large kernels have been moved behind optional
feature flags. These changes allow downstream projects to choose a smaller
dependency footprint and build times, if desired.
+* **Support for Copy-On-Write**: Arrow arrays now support copy-on-write, via
the
[`into_builder`](https://docs.rs/arrow/32.0.0/arrow/array/struct.ArrayData.html#method.into_builder)
methods
+* **Comparable Row Format**: [Much faster multi-column Sorting and
Grouping](https://arrow.apache.org/blog/2022/11/07/multi-column-sorts-in-arrow-rust-part-1/)
is now possible with the the new spillable, comparable
[row-format](https://docs.rs/arrow-row/32.0.0/arrow_row/index.html)
+* **FlightSQL Support**:
[FlightSQL](https://arrow.apache.org/docs/format/FlightSql.html)
[support](https://docs.rs/arrow-flight/32.0.0/arrow_flight/sql/index.html) has
been expanded
+* **Mid-Level Flight Client**: A new
[FlightClight](https://docs.rs/arrow-flight/32.0.0/arrow_flight/client/struct.FlightClient.html)
is available that handles lower level protocol details, and easier to use
[encoding](https://docs.rs/arrow-flight/32.0.0/arrow_flight/encode/struct.FlightDataEncoderBuilder.html)
and
[decoding](https://docs.rs/arrow-flight/32.0.0/arrow_flight/decode/struct.FlightDataDecoder.html)
APIs.
+* **IPC File Compression**: Arrow IPC file
[compression](https://docs.rs/arrow-ipc/32.0.0/arrow_ipc/gen/Message/struct.CompressionType.html)
with ZSTD and LZ4 is now fully supported.
+* **Full Decimal Support**: 256-bit decimals and negative scales can be
created and manipulated using many kernels, such as arithmetic.
+* **Improved Dictionary Support**: Dictionaries are now transparently
supported in most kernels.
+* **Improved Temporal Support**: Timestamps with Timezones and other temporal
types are supported in many more kernels.
+* **Improved Generics**: Improved generics allow writing code generic over all
arrays, or all arrays with the same layout
+* **Downcast Macros**: Various
[helper](https://docs.rs/arrow/32.0.0/arrow/macro.downcast_primitive_array.html)
[macros](https://docs.rs/arrow/32.0.0/arrow/macro.downcast_dictionary_array.html)
are now available to simplify dynamic dispatch to statically typed
implementations.
+
+# Parquet
+
+[Apache Parquet](https://parquet.apache.org/) is an open source,
column-oriented data file format designed for efficient data storage and
retrieval. It provides efficient data compression and encoding schemes with
enhanced performance to handle complex data in bulk. The Apache Parquet
implementation in Rust is one of the [fastest and most sophisticated
](https://arrow.apache.org/blog/2022/12/26/querying-parquet-with-millisecond-latency/)
open source implementations available.
+
+
+**Major Highlights**
+
+* **Arbitrarily Nested Schema**: Arbitrarily nested schemas can be read to and
written from arrow, as described in the series of [blog
posts](https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/)
on the topic.
+* **Predicate Pushdown**: The [arrow
reader](https://docs.rs/parquet/32.0.0/parquet/arrow/index.html) now supports
advanced predicate pushdown, including late materialization, as described in a
[the blog
posts](https://arrow.apache.org/blog/2022/12/26/querying-parquet-with-millisecond-latency/).
See
[RowSelection](https://docs.rs/parquet/32.0.0/parquet/arrow/arrow_reader/struct.RowSelection.html)
for the code.
+* **Bloom Filter Support**: Support for both [reading and writing bloom
filters](https://docs.rs/parquet/32.0.0/parquet/bloom_filter/index.html) has
been added.
+* **CLI Tools**: additional [CLI
tools](https://github.com/apache/arrow-rs/tree/master/parquet/src/bin) have
been added to introspect and manipulate parquet data.
+
+# Object Store
+
+Modern analytic workloads increasingly make use of blob storage facilities,
such as S3, to store large volumes of queryable data. A native Rust object
storage implementation that works well with the Rust Ecosystem and general and
the Arrow IO abstractions, is an important building block for many applications.
+The [object_store](https://docs.rs/object_store/latest/object_store/) crate
was [donated to the Apache Arrow project in July
2022](https://www.influxdata.com/blog/rust-object-store-donation/) to fill this
need, and while it follows a separate release schedule than the `arrow` and
`parquet` crates, it forms an integral part of the overarching Arrow IO story.
+
+Recent improvements include the following:
+
+**Major Highlights**
+
+* **Streaming Upload**: Multipart uploads are now supported.
+* **Minimised dependency footprint**: Upstream SDKs are no longer used,
improving consistency and reducing dependencies.
+* **HTTP / WebDAV Support**: Applications can access arbitrary HTTP servers
supporting `WebDAV`.
+* **Configurable Networking**: Socks proxies, and advanced HTTP client
configuration are now supported.
+* **Serializable Configuration**: Configuration information can now be easily
serialized and deserialized.
+* **Configurable Authentication**: Additional authentication options are now
available for the various cloud providers.
Review Comment:
```suggestion
* **Additional Authentication**: Additional authentication options are now
available for the various cloud providers.
```
It was configurable before, there is just now more, perhaps?
##########
_posts/2023-01-26-rust-32.0.0.md:
##########
@@ -0,0 +1,193 @@
+---
+layout: post
+title: "January 2023 Rust Apache Arrow Highlights"
+date: "2023-01-26 00:00:00 +0000"
+author: pmc
+categories: [release]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+# Introduction
+
+With the recent release of [32.0.0](https://crates.io/crates/arrow/32.0.0) of
the Rust implementation of [Apache Arrow](https://arrow.apache.org/), it seemed
timely to highlight some of the communities work since the [last
update](https://arrow.apache.org/blog/2022/06/16/rust-16.0.0/).
+
+The most recent list of detailed changes can always be found in the
[CHANGELOG](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md), with
the full historical list available
[here](https://github.com/apache/arrow-rs/blob/master/CHANGELOG-old.md).
+
+# Arrow
+
+[arrow](https://crates.io/crates/arrow) and
[arrow-flight](https://crates.io/crates/arrow-flight) are native Rust
implementations of Apache Arrow. Apache Arrow defines a language-independent
columnar memory format for flat and hierarchical data, organized for efficient
analytic operations on modern hardware like CPUs and GPUs. The Arrow memory
format also supports zero-copy reads for lightning-fast data access without
serialization overhead.
+
+The [Rust language](https://www.rust-lang.org/) offers best in class
performance, memory safety, and the developer productivity of a modern
programming language. These features make Rust an excellent choice for building
modern high performance analytical systems. When combined, Rust and the Apache
Arrow Ecosystem are a compelling toolkit for building the next generation of
systems.
+
+The [repository](https://github.com/apache/arrow-rs) recently passed 1400
stars on github, and the community has been focused on performance and feature
completeness.
+
+
+**Major Highlights**
+
+* **New CSV and JSON Readers**: The CSV and JSON readers have been revamped.
Their performance has more than doubled, and they now support push-driven
parsing, and better streaming object storage integration.
Review Comment:
```suggestion
* **New CSV and JSON Readers**: The CSV and JSON readers have been revamped.
Their performance has more than doubled, and they now support push-driven
parsing facilitating async streaming decode from object storage.
```
I think we should highlight why you would care about push-driving parsing
:smile:
##########
_posts/2023-01-26-rust-32.0.0.md:
##########
@@ -0,0 +1,193 @@
+---
+layout: post
+title: "January 2023 Rust Apache Arrow Highlights"
+date: "2023-01-26 00:00:00 +0000"
+author: pmc
+categories: [release]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+# Introduction
+
+With the recent release of [32.0.0](https://crates.io/crates/arrow/32.0.0) of
the Rust implementation of [Apache Arrow](https://arrow.apache.org/), it seemed
timely to highlight some of the communities work since the [last
update](https://arrow.apache.org/blog/2022/06/16/rust-16.0.0/).
+
+The most recent list of detailed changes can always be found in the
[CHANGELOG](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md), with
the full historical list available
[here](https://github.com/apache/arrow-rs/blob/master/CHANGELOG-old.md).
+
+# Arrow
+
+[arrow](https://crates.io/crates/arrow) and
[arrow-flight](https://crates.io/crates/arrow-flight) are native Rust
implementations of Apache Arrow. Apache Arrow defines a language-independent
columnar memory format for flat and hierarchical data, organized for efficient
analytic operations on modern hardware like CPUs and GPUs. The Arrow memory
format also supports zero-copy reads for lightning-fast data access without
serialization overhead.
+
+The [Rust language](https://www.rust-lang.org/) offers best in class
performance, memory safety, and the developer productivity of a modern
programming language. These features make Rust an excellent choice for building
modern high performance analytical systems. When combined, Rust and the Apache
Arrow Ecosystem are a compelling toolkit for building the next generation of
systems.
+
+The [repository](https://github.com/apache/arrow-rs) recently passed 1400
stars on github, and the community has been focused on performance and feature
completeness.
+
+
+**Major Highlights**
+
+* **New CSV and JSON Readers**: The CSV and JSON readers have been revamped.
Their performance has more than doubled, and they now support push-driven
parsing, and better streaming object storage integration.
+* **Faster Build Times and Reduced Codegen**: The `arrow` crate has been split
into multiple smaller crates, and large kernels have been moved behind optional
feature flags. These changes allow downstream projects to choose a smaller
dependency footprint and build times, if desired.
+* **Support for Copy-On-Write**: Arrow arrays now support copy-on-write, via
the
[`into_builder`](https://docs.rs/arrow/32.0.0/arrow/array/struct.ArrayData.html#method.into_builder)
methods
+* **Comparable Row Format**: [Much faster multi-column Sorting and
Grouping](https://arrow.apache.org/blog/2022/11/07/multi-column-sorts-in-arrow-rust-part-1/)
is now possible with the the new spillable, comparable
[row-format](https://docs.rs/arrow-row/32.0.0/arrow_row/index.html)
+* **FlightSQL Support**:
[FlightSQL](https://arrow.apache.org/docs/format/FlightSql.html)
[support](https://docs.rs/arrow-flight/32.0.0/arrow_flight/sql/index.html) has
been expanded
+* **Mid-Level Flight Client**: A new
[FlightClight](https://docs.rs/arrow-flight/32.0.0/arrow_flight/client/struct.FlightClient.html)
is available that handles lower level protocol details, and easier to use
[encoding](https://docs.rs/arrow-flight/32.0.0/arrow_flight/encode/struct.FlightDataEncoderBuilder.html)
and
[decoding](https://docs.rs/arrow-flight/32.0.0/arrow_flight/decode/struct.FlightDataDecoder.html)
APIs.
+* **IPC File Compression**: Arrow IPC file
[compression](https://docs.rs/arrow-ipc/32.0.0/arrow_ipc/gen/Message/struct.CompressionType.html)
with ZSTD and LZ4 is now fully supported.
+* **Full Decimal Support**: 256-bit decimals and negative scales can be
created and manipulated using many kernels, such as arithmetic.
+* **Improved Dictionary Support**: Dictionaries are now transparently
supported in most kernels.
+* **Improved Temporal Support**: Timestamps with Timezones and other temporal
types are supported in many more kernels.
+* **Improved Generics**: Improved generics allow writing code generic over all
arrays, or all arrays with the same layout
+* **Downcast Macros**: Various
[helper](https://docs.rs/arrow/32.0.0/arrow/macro.downcast_primitive_array.html)
[macros](https://docs.rs/arrow/32.0.0/arrow/macro.downcast_dictionary_array.html)
are now available to simplify dynamic dispatch to statically typed
implementations.
+
+# Parquet
+
+[Apache Parquet](https://parquet.apache.org/) is an open source,
column-oriented data file format designed for efficient data storage and
retrieval. It provides efficient data compression and encoding schemes with
enhanced performance to handle complex data in bulk. The Apache Parquet
implementation in Rust is one of the [fastest and most sophisticated
](https://arrow.apache.org/blog/2022/12/26/querying-parquet-with-millisecond-latency/)
open source implementations available.
+
+
+**Major Highlights**
+
+* **Arbitrarily Nested Schema**: Arbitrarily nested schemas can be read to and
written from arrow, as described in the series of [blog
posts](https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/)
on the topic.
+* **Predicate Pushdown**: The [arrow
reader](https://docs.rs/parquet/32.0.0/parquet/arrow/index.html) now supports
advanced predicate pushdown, including late materialization, as described in a
[the blog
posts](https://arrow.apache.org/blog/2022/12/26/querying-parquet-with-millisecond-latency/).
See
[RowSelection](https://docs.rs/parquet/32.0.0/parquet/arrow/arrow_reader/struct.RowSelection.html)
for the code.
+* **Bloom Filter Support**: Support for both [reading and writing bloom
filters](https://docs.rs/parquet/32.0.0/parquet/bloom_filter/index.html) has
been added.
+* **CLI Tools**: additional [CLI
tools](https://github.com/apache/arrow-rs/tree/master/parquet/src/bin) have
been added to introspect and manipulate parquet data.
+
+# Object Store
+
+Modern analytic workloads increasingly make use of blob storage facilities,
such as S3, to store large volumes of queryable data. A native Rust object
storage implementation that works well with the Rust Ecosystem and general and
the Arrow IO abstractions, is an important building block for many applications.
+The [object_store](https://docs.rs/object_store/latest/object_store/) crate
was [donated to the Apache Arrow project in July
2022](https://www.influxdata.com/blog/rust-object-store-donation/) to fill this
need, and while it follows a separate release schedule than the `arrow` and
`parquet` crates, it forms an integral part of the overarching Arrow IO story.
+
+Recent improvements include the following:
+
+**Major Highlights**
+
+* **Streaming Upload**: Multipart uploads are now supported.
+* **Minimised dependency footprint**: Upstream SDKs are no longer used,
improving consistency and reducing dependencies.
+* **HTTP / WebDAV Support**: Applications can access arbitrary HTTP servers
supporting `WebDAV`.
Review Comment:
```suggestion
* **HTTP / WebDAV Support**: Applications can read from arbitrary HTTP
servers, with mutation and listing supported on WebDAV-compatible endpoints.
```
To make it clearer that you can read from arbitrary HTTP servers, you only
need WebDAV if you want to mutate
##########
_posts/2023-01-26-rust-32.0.0.md:
##########
@@ -0,0 +1,193 @@
+---
+layout: post
+title: "January 2023 Rust Apache Arrow Highlights"
+date: "2023-01-26 00:00:00 +0000"
+author: pmc
+categories: [release]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+# Introduction
+
+With the recent release of [32.0.0](https://crates.io/crates/arrow/32.0.0) of
the Rust implementation of [Apache Arrow](https://arrow.apache.org/), it seemed
timely to highlight some of the communities work since the [last
update](https://arrow.apache.org/blog/2022/06/16/rust-16.0.0/).
+
+The most recent list of detailed changes can always be found in the
[CHANGELOG](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md), with
the full historical list available
[here](https://github.com/apache/arrow-rs/blob/master/CHANGELOG-old.md).
+
+# Arrow
+
+[arrow](https://crates.io/crates/arrow) and
[arrow-flight](https://crates.io/crates/arrow-flight) are native Rust
implementations of Apache Arrow. Apache Arrow defines a language-independent
columnar memory format for flat and hierarchical data, organized for efficient
analytic operations on modern hardware like CPUs and GPUs. The Arrow memory
format also supports zero-copy reads for lightning-fast data access without
serialization overhead.
+
+The [Rust language](https://www.rust-lang.org/) offers best in class
performance, memory safety, and the developer productivity of a modern
programming language. These features make Rust an excellent choice for building
modern high performance analytical systems. When combined, Rust and the Apache
Arrow Ecosystem are a compelling toolkit for building the next generation of
systems.
+
+The [repository](https://github.com/apache/arrow-rs) recently passed 1400
stars on github, and the community has been focused on performance and feature
completeness.
+
+
+**Major Highlights**
+
+* **New CSV and JSON Readers**: The CSV and JSON readers have been revamped.
Their performance has more than doubled, and they now support push-driven
parsing, and better streaming object storage integration.
+* **Faster Build Times and Reduced Codegen**: The `arrow` crate has been split
into multiple smaller crates, and large kernels have been moved behind optional
feature flags. These changes allow downstream projects to choose a smaller
dependency footprint and build times, if desired.
+* **Support for Copy-On-Write**: Arrow arrays now support copy-on-write, via
the
[`into_builder`](https://docs.rs/arrow/32.0.0/arrow/array/struct.ArrayData.html#method.into_builder)
methods
+* **Comparable Row Format**: [Much faster multi-column Sorting and
Grouping](https://arrow.apache.org/blog/2022/11/07/multi-column-sorts-in-arrow-rust-part-1/)
is now possible with the the new spillable, comparable
[row-format](https://docs.rs/arrow-row/32.0.0/arrow_row/index.html)
+* **FlightSQL Support**:
[FlightSQL](https://arrow.apache.org/docs/format/FlightSql.html)
[support](https://docs.rs/arrow-flight/32.0.0/arrow_flight/sql/index.html) has
been expanded
+* **Mid-Level Flight Client**: A new
[FlightClight](https://docs.rs/arrow-flight/32.0.0/arrow_flight/client/struct.FlightClient.html)
is available that handles lower level protocol details, and easier to use
[encoding](https://docs.rs/arrow-flight/32.0.0/arrow_flight/encode/struct.FlightDataEncoderBuilder.html)
and
[decoding](https://docs.rs/arrow-flight/32.0.0/arrow_flight/decode/struct.FlightDataDecoder.html)
APIs.
+* **IPC File Compression**: Arrow IPC file
[compression](https://docs.rs/arrow-ipc/32.0.0/arrow_ipc/gen/Message/struct.CompressionType.html)
with ZSTD and LZ4 is now fully supported.
+* **Full Decimal Support**: 256-bit decimals and negative scales can be
created and manipulated using many kernels, such as arithmetic.
+* **Improved Dictionary Support**: Dictionaries are now transparently
supported in most kernels.
+* **Improved Temporal Support**: Timestamps with Timezones and other temporal
types are supported in many more kernels.
+* **Improved Generics**: Improved generics allow writing code generic over all
arrays, or all arrays with the same layout
+* **Downcast Macros**: Various
[helper](https://docs.rs/arrow/32.0.0/arrow/macro.downcast_primitive_array.html)
[macros](https://docs.rs/arrow/32.0.0/arrow/macro.downcast_dictionary_array.html)
are now available to simplify dynamic dispatch to statically typed
implementations.
+
+# Parquet
+
+[Apache Parquet](https://parquet.apache.org/) is an open source,
column-oriented data file format designed for efficient data storage and
retrieval. It provides efficient data compression and encoding schemes with
enhanced performance to handle complex data in bulk. The Apache Parquet
implementation in Rust is one of the [fastest and most sophisticated
](https://arrow.apache.org/blog/2022/12/26/querying-parquet-with-millisecond-latency/)
open source implementations available.
+
+
+**Major Highlights**
+
+* **Arbitrarily Nested Schema**: Arbitrarily nested schemas can be read to and
written from arrow, as described in the series of [blog
posts](https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/)
on the topic.
+* **Predicate Pushdown**: The [arrow
reader](https://docs.rs/parquet/32.0.0/parquet/arrow/index.html) now supports
advanced predicate pushdown, including late materialization, as described in a
[the blog
posts](https://arrow.apache.org/blog/2022/12/26/querying-parquet-with-millisecond-latency/).
See
[RowSelection](https://docs.rs/parquet/32.0.0/parquet/arrow/arrow_reader/struct.RowSelection.html)
for the code.
Review Comment:
```suggestion
* **Predicate Pushdown**: The [arrow
reader](https://docs.rs/parquet/32.0.0/parquet/arrow/index.html) now supports
advanced predicate pushdown, including late materialization, as described
[here](https://arrow.apache.org/blog/2022/12/26/querying-parquet-with-millisecond-latency/).
See
[RowSelection](https://docs.rs/parquet/32.0.0/parquet/arrow/arrow_reader/struct.RowSelection.html)
and
[ArrowPredicate](https://docs.rs/parquet/32.0.0/parquet/arrow/arrow_reader/trait.ArrowPredicate.html).
```
##########
_posts/2023-01-26-rust-32.0.0.md:
##########
@@ -0,0 +1,193 @@
+---
+layout: post
+title: "January 2023 Rust Apache Arrow Highlights"
+date: "2023-01-26 00:00:00 +0000"
+author: pmc
+categories: [release]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+# Introduction
+
+With the recent release of [32.0.0](https://crates.io/crates/arrow/32.0.0) of
the Rust implementation of [Apache Arrow](https://arrow.apache.org/), it seemed
timely to highlight some of the communities work since the [last
update](https://arrow.apache.org/blog/2022/06/16/rust-16.0.0/).
+
+The most recent list of detailed changes can always be found in the
[CHANGELOG](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md), with
the full historical list available
[here](https://github.com/apache/arrow-rs/blob/master/CHANGELOG-old.md).
+
+# Arrow
+
+[arrow](https://crates.io/crates/arrow) and
[arrow-flight](https://crates.io/crates/arrow-flight) are native Rust
implementations of Apache Arrow. Apache Arrow defines a language-independent
columnar memory format for flat and hierarchical data, organized for efficient
analytic operations on modern hardware like CPUs and GPUs. The Arrow memory
format also supports zero-copy reads for lightning-fast data access without
serialization overhead.
+
+The [Rust language](https://www.rust-lang.org/) offers best in class
performance, memory safety, and the developer productivity of a modern
programming language. These features make Rust an excellent choice for building
modern high performance analytical systems. When combined, Rust and the Apache
Arrow Ecosystem are a compelling toolkit for building the next generation of
systems.
+
+The [repository](https://github.com/apache/arrow-rs) recently passed 1400
stars on github, and the community has been focused on performance and feature
completeness.
+
+
+**Major Highlights**
+
+* **New CSV and JSON Readers**: The CSV and JSON readers have been revamped.
Their performance has more than doubled, and they now support push-driven
parsing, and better streaming object storage integration.
+* **Faster Build Times and Reduced Codegen**: The `arrow` crate has been split
into multiple smaller crates, and large kernels have been moved behind optional
feature flags. These changes allow downstream projects to choose a smaller
dependency footprint and build times, if desired.
+* **Support for Copy-On-Write**: Arrow arrays now support copy-on-write, via
the
[`into_builder`](https://docs.rs/arrow/32.0.0/arrow/array/struct.ArrayData.html#method.into_builder)
methods
+* **Comparable Row Format**: [Much faster multi-column Sorting and
Grouping](https://arrow.apache.org/blog/2022/11/07/multi-column-sorts-in-arrow-rust-part-1/)
is now possible with the the new spillable, comparable
[row-format](https://docs.rs/arrow-row/32.0.0/arrow_row/index.html)
+* **FlightSQL Support**:
[FlightSQL](https://arrow.apache.org/docs/format/FlightSql.html)
[support](https://docs.rs/arrow-flight/32.0.0/arrow_flight/sql/index.html) has
been expanded
+* **Mid-Level Flight Client**: A new
[FlightClight](https://docs.rs/arrow-flight/32.0.0/arrow_flight/client/struct.FlightClient.html)
is available that handles lower level protocol details, and easier to use
[encoding](https://docs.rs/arrow-flight/32.0.0/arrow_flight/encode/struct.FlightDataEncoderBuilder.html)
and
[decoding](https://docs.rs/arrow-flight/32.0.0/arrow_flight/decode/struct.FlightDataDecoder.html)
APIs.
+* **IPC File Compression**: Arrow IPC file
[compression](https://docs.rs/arrow-ipc/32.0.0/arrow_ipc/gen/Message/struct.CompressionType.html)
with ZSTD and LZ4 is now fully supported.
+* **Full Decimal Support**: 256-bit decimals and negative scales can be
created and manipulated using many kernels, such as arithmetic.
+* **Improved Dictionary Support**: Dictionaries are now transparently
supported in most kernels.
+* **Improved Temporal Support**: Timestamps with Timezones and other temporal
types are supported in many more kernels.
+* **Improved Generics**: Improved generics allow writing code generic over all
arrays, or all arrays with the same layout
+* **Downcast Macros**: Various
[helper](https://docs.rs/arrow/32.0.0/arrow/macro.downcast_primitive_array.html)
[macros](https://docs.rs/arrow/32.0.0/arrow/macro.downcast_dictionary_array.html)
are now available to simplify dynamic dispatch to statically typed
implementations.
+
+# Parquet
+
+[Apache Parquet](https://parquet.apache.org/) is an open source,
column-oriented data file format designed for efficient data storage and
retrieval. It provides efficient data compression and encoding schemes with
enhanced performance to handle complex data in bulk. The Apache Parquet
implementation in Rust is one of the [fastest and most sophisticated
](https://arrow.apache.org/blog/2022/12/26/querying-parquet-with-millisecond-latency/)
open source implementations available.
+
+
+**Major Highlights**
+
+* **Arbitrarily Nested Schema**: Arbitrarily nested schemas can be read to and
written from arrow, as described in the series of [blog
posts](https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/)
on the topic.
+* **Predicate Pushdown**: The [arrow
reader](https://docs.rs/parquet/32.0.0/parquet/arrow/index.html) now supports
advanced predicate pushdown, including late materialization, as described in a
[the blog
posts](https://arrow.apache.org/blog/2022/12/26/querying-parquet-with-millisecond-latency/).
See
[RowSelection](https://docs.rs/parquet/32.0.0/parquet/arrow/arrow_reader/struct.RowSelection.html)
for the code.
+* **Bloom Filter Support**: Support for both [reading and writing bloom
filters](https://docs.rs/parquet/32.0.0/parquet/bloom_filter/index.html) has
been added.
+* **CLI Tools**: additional [CLI
tools](https://github.com/apache/arrow-rs/tree/master/parquet/src/bin) have
been added to introspect and manipulate parquet data.
+
+# Object Store
+
+Modern analytic workloads increasingly make use of blob storage facilities,
such as S3, to store large volumes of queryable data. A native Rust object
storage implementation that works well with the Rust Ecosystem and general and
the Arrow IO abstractions, is an important building block for many applications.
Review Comment:
```suggestion
Modern analytic workloads increasingly make use of blob storage facilities,
such as S3, to store large volumes of queryable data. A native Rust object
storage implementation that works well with the Rust Ecosystem in general, and
the Arrow IO abstractions, is an important building block for many applications.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
