Re: [C++][DISCUSS] FileSystem construction from URIs and secrets

ed, Apr 9, 2025 at 9:44 AM Raphael Taylor-Davies wrote: I'm not all that familiar with the C++ filesystem abstraction, but for ObjectStore, the closest equivalent abstraction in the Rust ecosystem, we follow what fsspec [1] and Hadoop [2] do and allow providing a set of key-value string pa

Re: [C++][DISCUSS] FileSystem construction from URIs and secrets

I'm not all that familiar with the C++ filesystem abstraction, but for ObjectStore, the closest equivalent abstraction in the Rust ecosystem, we follow what fsspec [1] and Hadoop [2] do and allow providing a set of key-value string pairs along with the URI [3]. This provides a great deal of fle

Re: Request for comments on adding new IPC option 'ensure_memory_alignment'

option 90% of the time, realignment >copies are cheap, but I do think the user should be forced to make the >choice. > >On Thu, Mar 27, 2025 at 10:21 AM Raphael Taylor-Davies > wrote: > >> If that is the eventual outcome of this discussion, I would be happy to, >> however,

Re: Request for comments on adding new IPC option 'ensure_memory_alignment'

If that is the eventual outcome of this discussion, I would be happy to, however, I'd like to avoid fragmenting the discussion just yet. On 27/03/2025 17:18, Antoine Pitrou wrote: Le 27/03/2025 à 18:14, Raphael Taylor-Davies a écrit : It's obviously preferrable to be zero-cop

Re: Request for comments on adding new IPC option 'ensure_memory_alignment'

On 27/03/2025 17:01, Antoine Pitrou wrote: Hello, Le 27/03/2025 à 17:53, Raphael Taylor-Davies a écrit : The current ambiguity, however, makes it hard to set reasonable defaults, as it isn't clear if FFI should be zero-copy and therefore have alignment restrictions or not. It's o

Re: Request for comments on adding new IPC option 'ensure_memory_alignment'

otocol is designed, but the design of flight in general is not amenable to zero-copy so perhaps this doesn't really matter. Kind Regards, Raphael Taylor-Davies [1]: https://github.com/apache/arrow-rs/pull/7137 [2]: https://github.com/apache/arrow-adbc/issues/2526 [3]: https://github.com/ap

Re: [VOTE][RUST] Release Apache Arrow Rust Object Store 0.12.0 RC2

+1 (binding) Verified on x86_64 GNU/Linux On 05/03/2025 17:02, Xuanwo wrote: +1 non-binding Tested on archlinux x86_64 On Wed, Mar 5, 2025, at 23:33, Jacob Wujciak wrote: +1 (non-binding) Ubuntu amd64 Am Mi., 5. März 2025 um 16:21 Uhr schrieb Andrew Lamb : Hi, (NOTE THIS IS RC2) I would

Re: [VOTE][RUST] Release Apache Arrow Rust Object Store 0.12.0 RC1

uff. Could ease the releases for arrow-rs, what do you think? Best Jacob Am Mi., 5. März 2025 um 12:24 Uhr schrieb Raphael Taylor-Davies : Aah yes, it appears my Apache GPG key has expired, and gpg "helpfully" used an unrelated one. pub rsa4096 2016-11-19 [SC]

[VOTE][RUST] Release Apache Arrow Rust Object Store 0.12.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.12.0. This release candidate is based on commit: 89a2ef8e06088c29433c41a8d8f6f2a46ba8f399 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Ple

Re: [VOTE][RUST] Release Apache Arrow Rust Object Store 0.12.0 RC1

this was signed with the wrong key? gpg: Signature made Wed 05 Mar 2025 11:43:17 AM CET gpg:using RSA key BF5A4[...] gpg: Can't check signature: No public key vs gpg: key 03F0D5EA3790810C: "Raphael Jaen Taylor-Davies " Am Mi., 5. März 2025 um 11:47 Uhr schrieb Raphael

Re: [VOTE][RUST] Release Apache Arrow Rust 54.2.1 RC1

+1 (binding) Verified on x86_64 GNU/Linux On 27/02/2025 12:23, Andrew Lamb wrote: Hi, I PLAN TO PUBLISH THIS TO CRATES.IO AS SOON AS THERE ARE 3 BINDING VOTES, NOT WAITING FOR 3 DAYS. See [5] As mentioned in [5], this release contains a small change to the version dependencies to fix downstre

Re: [VOTE][RUST] Release Apache Arrow Rust 54.1.0 RC1

+1 (binding) Verified on x86_64 GNU/Linux On 29/01/2025 13:50, Andrew Lamb wrote: Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 54.1.0. This release candidate is based on commit: 3bf29a2c7474e59722d885cd11fafd0dca13a28e [1] The proposed release tarball an

Re: [VOTE][RUST] Release Apache Arrow Rust Object Store 0.11.1 RC1

+1 (binding) Verified on x86_64 GNU/Linux On 15/10/2024 22:40, L. C. Hsieh wrote: +1 (binding) Verified on M3 Mac. Thanks Andrew. On Tue, Oct 15, 2024 at 2:06 PM Andrew Lamb wrote: Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.11.1. Thi

Re: [VOTE][RUST] Release Apache Arrow Rust Object Store 0.11.0 RC2

+1 (binding) Verified on x86_64 GNU/Linux On 13/08/2024 15:44, L. C. Hsieh wrote: +1 (binding) Verified on M3 Mac. Thanks Andrew. On Tue, Aug 13, 2024 at 3:35 AM Andrew Lamb wrote: Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.11.0. NOT

Re: [VOTE][RUST] Release Apache Arrow Rust Object Store 0.10.2 RC1

+1 (binding) Verified on x86_64 GNU/Linux Kind Regards, Raphael On 17/07/2024 18:36, Andrew Lamb wrote: Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.10.2. This release candidate is based on commit: b44497e1cdd84933b49b56dd00506411c040b46c

Re: [DISCUSS][C++] Empty directory marker creation in S3FileSystem

Many people are familiar with object stores these days. You could create a new abstraction `ObjectStore` which is very similar to `FileSystem` except the semantics are object store semantics and not filesystem semantics. FWIW in the Arrow Rust ecosystem we only provide an object store abstractio

RE: Re: [DISCUSS] Donation of a User-Defined Function Framework for Apache Arrow

I wonder if the DataFusion project might be a more natural home for this functionality? UDFs are more of a query engine concept, whereas arrow-rs is more focused on purely physical execution? On 28 June 2024 19:41:39 BST, Runji Wang wrote: >Hi Felipe, > >Vectorization will be applied whenever p

[RESULT][VOTE][RUST] Release Apache Arrow Rust 52.0.0 RC1

, Raphael Taylor-Davies wrote: Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 52.0.0. This release candidate is based on commit: f42218ae5d9c9f0b9ea3365f2b1e6025a43b8c71 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is

[VOTE][RUST] Release Apache Arrow Rust 52.0.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 52.0.0. This release candidate is based on commit: f42218ae5d9c9f0b9ea3365f2b1e6025a43b8c71 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please downloa

Re: [DISCUSS] Statistics through the C data interface

I'm likely missing something here, but why can't statistics be returned as arrow arrays encoded using the C data interface? My understanding of the C data interface is as a specification for exchanging arrow payloads, with it left to higher level protocols, such as ADBC, to assign semantic meani

Re: [DISCUSS] Statistics through the C data interface

Hi, One potential challenge with encoding statistics in the schema metadata is that some systems may consider this metadata as part of assessing schema equivalence. However, I think the bigger question is what the intended use-case for these statistics is? Often query engines want to collect

[RESULT][VOTE][RUST] Release Apache Arrow Rust Object Store 0.10.1 RC1

 release Raphael On 10/05/2024 18:29, Raphael Taylor-Davies wrote: Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.10.1. This is primarily motivated by a major bug introduced by 0.10.0 [1] This release candidate is based on commit

[VOTE][RUST] Release Apache Arrow Rust Object Store 0.10.1 RC1

Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.10.1. This is primarily motivated by a major bug introduced by 0.10.0 [1] This release candidate is based on commit: 3d3ddb2108502854da98654ada85364d5627ef21 [2] The proposed release tarball and

[RESULT][VOTE][RUST] Release Apache Arrow Rust Object Store 0.10.0 RC1

: +1(binding) Verified on m1 macbook Thanks Raphael Raphael Taylor-Davies 于 2024年4月18日周四 下午6:55写道： Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.10.0. This release candidate is based on commit: cd3331989d65f6d56830f9ffa758b4c96d10f4be [1

[VOTE][RUST] Release Apache Arrow Rust Object Store 0.10.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.10.0. This release candidate is based on commit: cd3331989d65f6d56830f9ffa758b4c96d10f4be [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Ple

[RESULT][VOTE][RUST] Release Apache Arrow Rust 51.0.0 RC1

With 5 +1 votes (4 binding) the release is approved The release is available here: https://dist.apache.org/repos/dist/release/arrow/arrow-rs-51.0.0 It has also been released to crates.io Thank you to everyone who helped verify this release On 3/15/24 20:40, Raphael Taylor-Davies wrote: Hi

[VOTE][RUST] Release Apache Arrow Rust 51.0.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 51.0.0. This release candidate is based on commit: ada986c7ec8f8fe4f94235c8aaeba4995392ee72 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please downloa

[RESULT][VOTE][RUST] Release Apache Arrow Rust Object Store 0.9.1 RC1

wrote: +1 (binding) Verified on M3 mac Thank you Raphael Andrew On Fri, Mar 1, 2024 at 2:45 AM L. C. Hsieh wrote: +1 (binding) Verified on M1 Mac. Thanks Raphael. On Thu, Feb 29, 2024 at 11:10 PM Raphael Taylor-Davies wrote: Hi, I would like to propose a release of Apache Arrow Rust

[VOTE][RUST] Release Apache Arrow Rust Object Store 0.9.1 RC1

Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.9.1. This release candidate is based on commit: 30151220c29fa5e01365c2a4e153de01d5d2c041 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Plea

[ANNOUNCE] New Arrow committer: Jeffrey Vo

On behalf of the Arrow PMC, I am happy to announce that Jeffrey Vo has accepted an invitation to become a committer on Apache Arrow. Welcome, and thank you for your contributions! Raphael Taylor-Davies

Re: [Python][Rust] Is Arrow Rust supposed to support S3-compatible storage with non-public certificates?

On a related note, version 0.9.0 switched to using the system CAs by default [1], and so if you've added your private CA chain into there it should work. [1]: https://github.com/apache/arrow-rs/pull/5056 On 25 January 2024 09:17:55 GMT, Raphael Taylor-Davies wrote: >The ticket for su

Re: [Python][Rust] Is Arrow Rust supposed to support S3-compatible storage with non-public certificates?

The ticket for supporting self-signed certificates can be found here [1]. If you switch the TLS backend to OpenSSL it may respect the SSL_CERT_FILE environment variable, but I'm not very familiar with the particulars of that library. This would require customising the Rust build, however, which

[RESULT][VOTE][RUST] Release Apache Arrow Rust 50.0.0 RC1

With 5 +1 votes (4 binding) the release is approved The release is available here: It has also been released to crates.io. Thank you to everyone who helped verify this release On 09/01/2024 10:24, Raphael Taylor-Davies wrote: Hi, I would like to propose a release of Apache Arrow Rust

[VOTE][RUST] Release Apache Arrow Rust 50.0.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 50.0.0. This release candidate is based on commit: db811083669df66992008c9409b743a2e365adb0 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please downloa

[RESULT][VOTE][RUST] Release Apache Arrow Rust Object Store 0.9.0 RC1

With 3 +1 votes (3 binding) the release is approved The release is available here: It has also been released to crates.io. Thank you to everyone who helped verify this release On 05/01/2024 13:29, Raphael Taylor-Davies wrote: Hi, I would like to propose a release of Apache Arrow Rust Object

[VOTE][RUST] Release Apache Arrow Rust Object Store 0.9.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.9.0. This release candidate is based on commit: cb16050ec732872d5995c7420cc6858749bbf743 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Plea

Re: [DISCUSS] [DATAFUSION] PMC for new DataFusion top level project

allowed by the process) >> >> From what I can tell, this means the PMC would be the following from the >> current Arrow PMC [3]: >> >> Andy Grove (NVidia) >> Andrew Lamb (InfuxData) >> Daniël Heres (Coralogix) >> Jie Wen (SelectDB) >> Kun Li

Re: [DISCUSS] Protocol for exchanging Arrow data over REST APIs

to TCP. Kind Regards, Raphael Taylor-Davies [1]: https://arrow.apache.org/docs/format/Columnar.html#ipc-streaming-format [2]: https://docs.rs/arrow-flight/latest/arrow_flight/sql/server/trait.FlightSqlService.html [3]: https://docs.rs/arrow-flight/latest/arrow_flight/encode/struct.FlightDataEnco

[RESULT][VOTE][RUST] Release Apache Arrow Rust 49.0.0 RC1

wrote: +1 (binding) Verified on Intel Mac. Thanks Raphael. On Tue, Nov 7, 2023 at 1:38 PM Andy Grove wrote: +1 (binding) Verified on Ubuntu 22.04.3 LTS. Thanks, Raphael. On Tue, Nov 7, 2023 at 2:22 PM Raphael Taylor-Davies wrote: Hi, I would like to propose a release of Apache Arrow

Re: [VOTE][RUST] Release Apache Arrow Rust 48.0.1 RC1

+1 (binding) Verified on x86_64 GNU/Linux On 09/11/2023 20:31, Andrew Lamb wrote: As discussed on [5], I would like to propose a patch release of Apache Arrow Rust Implementation, version 48.0.1 to include two bug fixes. This release candidate is based on commit: b60fc7bb09ada1385d3542b784fff2

Re: decimal64

Perhaps my maths is incorrect, but a decimal64 would have a maximum precision of 18, not 19? log(9223372036854775807) = 18.9? On 09/11/2023 16:01, Curt Hagenlocher wrote: Recently, someone opened an issue on GitHub ([C++] Decimal64/32 support? · Issue #38622 · apache/arrow (github.com)

[VOTE][RUST] Release Apache Arrow Rust 49.0.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 49.0.0. This release candidate is based on commit: 747dcbf0670aeab2ede474edb3c4f22028d6a7e6 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please downloa

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 33.0.0 RC1

Nov 7, 2023 at 6:41 AM Raphael Taylor-Davies wrote: I intend to cut a new arrow release later this week, I would prefer we wait for this. On 07/11/2023 11:39, Andrew Lamb wrote: Perhaps we can create an arrow 48.1.0 patch release to include the fix? On Tue, Nov 7, 2023 at 12:48 AM Will Jones

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 33.0.0 RC1

. That likely narrows the scope of who is affected. If this bug is present in DataFusion 33, then delta-rs will likely skip upgrading until 34. If we're the only downstream project this parsing issue affects, then I think it's fine to release. On Mon, Nov 6, 2023 at 8:22 PM Raphael Taylor-Dav

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 33.0.0 RC1

Hi, To further clarify the bug concerns the serde compatibility feature that allows converting a serde compatible data structure to arrow [1]. It will not impact workloads reading JSON. I am not sure this is a sufficiently fundamental bug to warrant special concern, but happy to defer to othe

[RESULT][VOTE][RUST] Release Apache Arrow Rust Object Store 0.8.0 RC1

good!' Release candidate looks good! So it looks good then. +1 if test issues can be ignored. On Thu, Nov 2, 2023 at 1:31 PM Raphael Taylor-Davies wrote: Aah, because the release script bails out on the first error? You could either: - Remove set -e from the top of the script to have it c

Re: [VOTE][RUST] Release Apache Arrow Rust Object Store 0.8.0 RC1

, I just felt there was prior precedent for not holding releases back on test issues. On 02/11/2023 20:07, L. C. Hsieh wrote: Hmm, I think we cannot run the verification script unless the issue is fixed? On Thu, Nov 2, 2023 at 6:40 AM Raphael Taylor-Davies wrote: Aah, that was a mistake

Re: [VOTE][RUST] Release Apache Arrow Rust Object Store 0.8.0 RC1

;💀" }, Path { raw: "directory/child.txt" }] note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace failures: local::tests::invalid_path ``` On Thu, Nov 2, 2023 at 7:48 AM Raphael Taylor-Davies wrote: Hi, I would like to propose a release of Apache Arrow

[VOTE][RUST] Release Apache Arrow Rust Object Store 0.8.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.8.0. This release candidate is based on commit: ad211fe324d259bf9fea1c43a3a82b3c833f6d7a [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Plea

[RESULT][VOTE][RUST] Release Apache Arrow Rust 48.0.0 RC2

 this release On 18/10/2023 14:59, Raphael Taylor-Davies wrote: Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 48.0.0 *RC2*. Please note that there were issues with the first release candidate that required cutting a second. This release candidate is

Re: Apache Arrow file format

To further what others have already mentioned, the IPC file format is primarily optimised for IPC use-cases, that is exchanging the entire contents between processes. It is relatively inexpensive to encode and decode, and supports all arrow datatypes, making it ideal for things like spill-to-di

[VOTE][RUST] Release Apache Arrow Rust 48.0.0 RC2

Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 48.0.0 *RC2*. Please note that there were issues with the first release candidate that required cutting a second. This release candidate is based on commit: 51ac6fec8755147cd6b1dfe7d76bfdcfacad0463 [1] The

Re: [DISCUSS][Rust][DataFusion][HiveMetaStore] Possible Metastore integration with Data Fusion

Hi, I think [1] might be a good place to start and handle coordination for this undertaking. I suspect it would probably want to live under the datafusion-contrib organisation, similar to HDFS [2] Kind Regards, Raphael [1]: https://github.com/apache/arrow-datafusion/issues/2209 [2]: https:/

Re: [DISCUSS][C++] Raw pointer string views

n to the format with StringView, I think it would help to get some input from those at DuckDB and Velox on their perspective Andrew On Mon, Oct 2, 2023 at 9:17 AM Raphael Taylor-Davies wrote: Oh I'm with you on it being a precedent we want to be very careful about setting, but if there isn

Re: [DISCUSS][C++] Raw pointer string views

quiring trusting and dereferencing raw pointers embedded in data buffers). There could even be an argument for making some of them canonical extension types if there's enough anteriority in favor. Regards Antoine. Le 02/10/2023 à 15:00, Raphael Taylor-Davies a écrit : I think what would

Re: [DISCUSS][C++] Raw pointer string views

I think what would really help would be some concrete numbers, do we have any numbers comparing the performance of the offset and pointer based representations? If there isn't a significant performance difference between them, would the systems that currently use a pointer-based approach be wil

Re: [VOTE][Format] Add ListView and LargeListView Arrays to Arrow Format

+1 On 02/10/2023 13:53, Antoine Pitrou wrote: Hello, +1 and thanks for working on this! There'll probably be some minor comments to the format PR, but those don't deter from accepting these new layouts into the standard. Regards Antoine. Le 29/09/2023 à 14:09, Felipe Oliveira Carvalho a

Re: [VOTE][RUST] Release Apache Arrow Rust Object Store 0.7.1 RC1

With 5 +1 votes (4 binding) the release is approved The release is available here: https://dist.apache.org/repos/dist/release/arrow/arrow-object-store-rs-0.7.1/ It has also been released to crates.io Thank you to everyone who helped verify this release On 26/09/2023 17:01, Raphael Taylor

Re: [VOTE][Format] Add ListView and LargeListView Arrays to Arrow Format

Hi Felipe, Can I confirm that DuckDB and Velox use the same encoding for these types, and so we aren't going to run into similar issues as [1]? Kind Regards, Raphael Taylor-Davies [1]: https://lists.apache.org/thread/l8t1vj5x1wdf75mdw3wfjvnxrfy5xomy On 29/09/2023 13:09, Felipe Oli

Re: [DISCUSS][C++] Raw pointer string views

FWIW Rust wouldn't have issues using raw pointers, I can't speak for other languages though. They would be more expensive to validate, but validation is not going to be cheap regardless. I could definitely see a world where view types use pointers and IPC coerces to/from the large non-view type

Re: [DISCUSS][C++] Raw pointer string views

Do you have any benchmarks comparing kernels with native pointer array support, compared to those that must first convert to the offset representation? I think this would help ground this discussion empirically. On 27 September 2023 22:25:02 BST, Benjamin Kietzman wrote: >Hello all, > >@Gang >

Re: [DISCUSS][C++] Raw pointer string views

r locations to contiguous buffers which could end up being >very significant depending on the shape and size of the data. The pointer >-based solution wouldn't be allowed in IPC though, only across the C Data >interface (correct me if I'm wrong). > >--Matt > >On Tue,

Re: [DISCUSS][C++] Raw pointer string views

Hi, Is the motivation here to avoid DuckDB and Velox having to duplicate the conversion logic from pointer-based to offset-based, or to allow arrow-cpp to operate directly on pointer-based arrays? If it is the former, I personally wouldn't have thought the conversion logic sufficiently compl

[VOTE][RUST] Release Apache Arrow Rust Object Store 0.7.1 RC1

Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.7.1. This release candidate is based on commit: 4ef7917bd57b701e30def8511b5fd8a7961f2fcf [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Plea

[RESULT][VOTE][RUST] Release Apache Arrow Rust 47.0.0 RC1

With 4 +1 votes (4 binding) the release is approved The release is available here: https://dist.apache.org/repos/dist/release/arrow/arrow-rs-47.0.0 It has also been released to crates.io Thank you to everyone who helped verify this release On 19/09/2023 14:20, Raphael Taylor-Davies wrote

Re: [VOTE][RUST] Release Apache Arrow Rust 47.0.0 RC1

://github.com/apache/arrow-rs/blob/master/dev/release/verify-release-candidate.sh On 19/09/2023 14:18, Raphael Taylor-Davies wrote: Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 47.0.0. This release candidate is based on commit

[VOTE][RUST] Release Apache Arrow Rust 47.0.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 47.0.0. This release candidate is based on commit: 1d6feeacebb8d0d659d493b783ba381940973745 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please downloa

Re: [FlightSQL] Supporting binding parameters to prepared statements with a stateless server

naged by the clients and sent with each request. This sort of maps to the ticket notion already present in many of the APIs, but could perhaps be formalized. Kind Regards, Raphael Taylor-Davies On 14/09/2023 11:52, Andrew Lamb wrote: Hello, As FlightSQL gets more widely adopted across the ecos

[RESULT][VOTE][RUST] Release Apache Arrow Rust 46.0.0 RC1

With 3 +1 votes (3 binding) the release is approved The release is available here: https://dist.apache.org/repos/dist/release/arrow/arrow-rs-46.0.0/ It has also been released to crates.io Thank you to everyone who helped verify this release On 21/08/2023 16:40, Raphael Taylor-Davies wrote

[VOTE][RUST] Release Apache Arrow Rust 46.0.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 46.0.0. This release candidate is based on commit: 90449ffb2ea6ceef43ce8fc97084b3373975f357 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please downloa

[RESULT][VOTE][RUST] Release Apache Arrow Rust Object Store 0.7.0 RC1

With 3 +1 votes (3 binding) the release is approved The release is available here: https://dist.apache.org/repos/dist/release/arrow/arrow-object-store-rs-0.7.0/ It has also been released to crates.io Thank you to everyone who helped verify this release On 15/08/2023 10:57, Raphael Taylor

Re: [VOTE][Format] Add Utf8View Arrays to Arrow Format

+1 (binding) Despite my earlier misgivings, I think this will be a valuable addition to the specification. To clarify I've interpreted this as a vote on both Utf8View and BinaryView as in the linked PR. On 28/06/2023 20:34, Benjamin Kietzman wrote: Hello, I'd like to propose adding Utf8Vi

Re: [Vote][Format] C Data Interface Format string for REE

+1 (binding) On 16/08/2023 16:58, Matt Topol wrote: It would be nice to get approval from authors of other implementations such as Rust, C#, Javascript... I'm hoping that some of them see this and participate in the vote. *crosses fingers* On Wed, Aug 16, 2023 at 11:10 AM Antoine Pitrou wrot

[VOTE][RUST] Release Apache Arrow Rust Object Store 0.7.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.7.0. This release candidate is based on commit: 77fe72ddd40c1d39068ad580b975504d57032060 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Plea

Re: Null buffer spec incompatibility between arrow2 and arrow-C++

bytes of the corresponding buffer would be 0. This is supported by arrow-rs [2], and I believe arrow2 is incorrect in not supporting this. I would recommend taking this up with the arrow2 maintainers [3]. Kind Regards, Raphael Taylor-Davies [1]: https://github.com/apache/arrow/pull/1480

[RESULT][VOTE][RUST] Release Apache Arrow Rust 45.0.0 RC1

) Verified on my M1 Mac. Thanks Raphael ! On Mon, Jul 31, 2023 at 12:32 AM Raphael Taylor-Davies wrote: Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 45.0.0. This release candidate is based on commit: 16744e5ac08d9ead6c51ff6e08d8b91e87460c52 [1] The

Re: [DISCUSS][Format] Draft implementation of string view array format

to be able to provide efficient safe APIs. Otherwise I don't see any major blockers towards getting these standardised, thank you for your continued efforts in this space. Kind Regards, Raphael Taylor-Davies [1]: https://github.com/apache/arrow-rs/pull/4585 On 12/07/2023 17:35, Pedro Eu

[VOTE][RUST] Release Apache Arrow Rust 45.0.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 45.0.0. This release candidate is based on commit: 16744e5ac08d9ead6c51ff6e08d8b91e87460c52 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please downloa

[RESULT][VOTE][RUST] Release Apache Arrow Rust 44.0.0 RC1

(binding) Verified on x86_64 mac Looks like another very nice release. Thanks for keeping the train moving Andrew On Fri, Jul 14, 2023 at 2:05 PM L. C. Hsieh wrote: +1 (binding) Verified on M1 Mac. Thanks Raphael. On Fri, Jul 14, 2023 at 10:44 AM Raphael Taylor-Davies wrote: Hi, I would like

[VOTE][RUST] Release Apache Arrow Rust 44.0.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 44.0.0. This release candidate is based on commit: 8f44472e5c773f0daec1965253143e94d14c55e5 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please downloa

Re: [DISCUSS] Canonical alternative layout proposal

ce in the C data interface, IPC, Flight, etc. Ian [5]https://lists.apache.org/thread/7x2714wookjqgkoykxpq9jtpyrgx2bx2 On Thu, Jul 13, 2023 at 11:00 AM Raphael Taylor-Davies r.taylordav...@googlemail.com.invalid wrote: I like this proposal, I think it strikes a pragmatic balance between prese

Re: [DISCUSS] Canonical alternative layout proposal

I like this proposal, I think it strikes a pragmatic balance between preserving interoperability whilst still allowing new ideas to be incorporated into the standard. Thank you for writing this up. On 13/07/2023 10:22, Matt Topol wrote: I don't have much to add but I do want to second Jacob's

Re: [DISCUSS][Format] Draft implementation of string view array format

7;s datasets to feed DuckDb or consuming a velox task as a record batch stream into a different system) and these sorts of challenges are in the forefront. On Fri, Jul 7, 2023 at 7:53 AM Raphael Taylor-Davies wrote: Thus the approach you describe for validating an entire character buffer as UTF-8 t

Re: [DISCUSS][Format] Draft implementation of string view array format

a kernel may decide to copy and densify dynamically if it detects that output is getting sparse or fragmented. It's also worth noting that unlike an explicit selection vector a Utf8View array (however sparse or fragmented) will still benefit from the prefix comparison fast path. Sincerely, Ben

[RESULT][VOTE][RUST] Release Apache Arrow Rust 43.0.0 RC1

With 5 +1 votes (5 binding) the release is approved The release is available here: https://dist.apache.org/repos/dist/release/arrow/arrow-rs-43.0.0 It has also been released to crates.io Thank you to everyone who helped verify this release Raphael On 30/06/2023 16:26, Raphael Taylor-Davies

Re: [DISCUSS] UTF-8 validation

us on this note, this interpretation doesn't appear to have caused issues so far. On 2 July 2023 13:07:20 BST, Antoine Pitrou wrote: > > >Le 02/07/2023 à 14:00, Raphael Taylor-Davies a écrit : >> >> More an observation than an issue, but UTF-8 validation for StringArray c

Re: [DISCUSS][Format] Draft implementation of string view array format

n selection performance can be improved in less intrusive ways using dictionaries or selection vectors. Kind Regards, Raphael Taylor-Davies On 02/07/2023 11:46, Andrew Lamb wrote: * This is the first layout where the number of buffers depends on the data and not the schema. I think this

[VOTE][RUST] Release Apache Arrow Rust 43.0.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 43.0.0. This release candidate is based on commit: 414235e7630d05cccf0b9f5032ebfc0858b8ae5b [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please downloa

Re: [DISCUSS][Format] Starting the draft implementation of the ArrayView array format

this is a reasonable price to pay for avoiding the ecosystem fragmentation that would potentially result from introducing alternative layouts for common types such as strings, whilst also avoiding the need to reimplement a load of kernels. What do people think? Kind Regards, Raphael Taylor-Dav

Scalars in Apache Arrow Rust

already grappled with, I would very much appreciate any feedback the community can give. To avoid bifurcating the discussion, please comment on the GitHub PR. I look forward to hearing your thoughts. Kind Regards, Raphael Taylor-Davies [1]: https://docs.rs/arrow-arith/latest/arrow_arith

[RESULT][VOTE][RUST] Release Apache Arrow Rust 41.0.0 RC1

) Verified on M1 Mac. Thanks Raphael. On Fri, Jun 2, 2023 at 11:55 AM Raphael Taylor-Davies wrote: Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 41.0.0. This release candidate is based on commit: e1badc0542ca82e2304cc3f51a9d25ea2dbb74eb [1] The proposed release

[RESULT][VOTE][RUST] Release Apache Arrow Rust Object Store 0.6.1 RC1

mac x86_64 Thank you Raphael On Fri, Jun 2, 2023 at 2:23 PM Raphael Taylor-Davies wrote: Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.6.1. This release candidate is based on commit: f323097584eaa8edb1193b4fb67bccadd39594f6 [1] The proposed

[VOTE][RUST] Release Apache Arrow Rust 41.0.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 41.0.0. This release candidate is based on commit: e1badc0542ca82e2304cc3f51a9d25ea2dbb74eb [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please downloa

[VOTE][RUST] Release Apache Arrow Rust Object Store 0.6.1 RC1

Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.6.1. This release candidate is based on commit: f323097584eaa8edb1193b4fb67bccadd39594f6 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Plea

[RESULT][VOTE][RUST] Release Apache Arrow Rust 40.0.0 RC1

8:49 AM Raphael Taylor-Davies wrote: Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 40.0.0. This release candidate is based on commit: 25bfccca58ff219d9f59ba9f4d75550493238a4f [1] The proposed release tarball and signatures are hosted at [2]. The changelog

[RESULT][VOTE][RUST] Release Apache Arrow Rust Object Store 0.6.0 RC1

, Raphael Taylor-Davies wrote: Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.6.0. This release candidate is based on commit: ec7706c1f2aeef5a289e46d1df7785e5c93e6bfb [1] The proposed release tarball and signatures are hosted at [2]. The changelog is

[VOTE][RUST] Release Apache Arrow Rust 40.0.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 40.0.0. This release candidate is based on commit: 25bfccca58ff219d9f59ba9f4d75550493238a4f [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please downloa

[VOTE][RUST] Release Apache Arrow Rust Object Store 0.6.0 RC1

Hi, I would like to propose a release of Apache Arrow Rust Object Store Implementation, version 0.6.0. This release candidate is based on commit: ec7706c1f2aeef5a289e46d1df7785e5c93e6bfb [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Plea

Re: [DISCUSS][Format] Starting the draft implementation of the ArrayView array format

ntation is probably a better way to represent variable length lists than what arrow settled upon, I'm not yet convinced it is sufficiently better to incentivise broad ecosystem adoption. Kind Regards, Raphael Taylor-Davies On 11/05/2023 21:20, Will Jones wrote: Hi Felipe, Thanks for the add

[RESULT][VOTE][RUST] Release Apache Arrow Rust 39.0.0 RC1

With 4 +1 votes (4 binding) the release is approved The release is available here: https://dist.apache.org/repos/dist/release/arrow/arrow-rs-39.0.0 It has also been released to crates.io Thank you to everyone who helped verify this release Raphael On 05/05/2023 15:45, Raphael Taylor-Davies

  1   2   >