Question: Arrow Flight handshake protocol version

Hi all; I am looking to Rust crate for arrow flight. One thing that I am not sure right now is that protocol version is dedicated to user-defined protocol or that's reserved in a sense for handshake for a future use (or current)? Best, Theo Mahmut

Re: Guidance on Very Conservative Default X86 ISA

Hi, I have added AVX512 kernels back in times but they got removed with a reason that I don't agree. (No objections, that was the maintainers decision.) I am totally in line with you and prefer having feature gated SIMD supersets for the compilation and have compile time selection. I was working

Re: [DISCUSS] Sponsor the upstream open source projects

> > This discussion is to discuss sponsoring the open source projects we used > as dependencies in Arrow. > > Remzi > > On Sat, 8 Oct 2022 at 19:12, vertexclique vertexclique < > vertexcli...@gmail.com> wrote: > > > I have developed projects uses Arrow in my

Re: [DISCUSS] Sponsor the upstream open source projects

I have developed projects uses Arrow in my current and previous roles. I also contributed to Arrow. Nevertheless, I didn't get a single dime or any support for Arrow related work. Never thought to ask for it too in this mail list (maybe that's my bad). If this conversation unfolds to positive side.

Re: [Rust] Is Rust Arrow's deserialization code zero-copy?

Hi Al; Yeah, that's true. The ownership rules of Rust prevents resource ownership outside the container that the resource lifetime taken. That's why there are buffer copies to take ownership and operate on it. Imo, it should be also removed from the docs if it's written like that (for Rust). B

Re: [Rust]: Architecture support

portant to > > actually test on machines of the target architectures (not just cross > > compile for them). It is not clear to me how much additional effort > > setting > > up such CI will be, or what resources we have available to test. > > > > Andrew >

Re: [DataFusion] Blocking async of async is not async

we are using internally. DataFusion can go in that direction instead of going fully async, you will see the performance improvement comes with it drastically. But this suggestion is TIOLI kind of... Best, Mahmut vertexclique vertexclique , 13 Kas 2020 Cum, 12:55 tarihinde şunu yazdı: > Hi, &

Re: [DataFusion] Blocking async of async is not async

ed in this code): https://gist.github.com/vertexclique/b9bcfe5c1596d4d087e71b5d608c279b Best, Mahmut Andrew Lamb , 13 Kas 2020 Cum, 12:35 tarihinde şunu yazdı: > My understanding of tokio is that there is exactly one global Runtime > <https://docs.rs/tokio/0.3.3/tokio/runtime/struct.Runt

Re: [Rust]: Architecture support

topic we might want to visit later. But I am %99 sure that it will work with the correct sysroot image and toolchain. Because it is working on Linux gnu build. Best, Mahmut vertexclique vertexclique , 12 Kas 2020 Per, 12:29 tarihinde şunu yazdı: > Hi Team; > > There are 3 topics fall u

[Rust]: Architecture support

Hi Team; There are 3 topics fall under this: * no_std compatibility * endianness compatibility * target datapath size (32-bit/64-bit, rust naming target_pointer_width) So after the sync call yesterday, Micah said that there were efforts on that for some time at Java, C++ side. That's nice. Curren

Re: Rust ParquetReader trait

ly benefit from going through FileSource? > > Before opening this PR, can you quickly look at how complex it would be to > change your custom sources to implement ChunkReader? I think it might be a > lot easier than you think ! :-) > > Remi > > Le mer. 11 nov. 2020 à 14:14, ver

Rust ParquetReader trait

. Best, Mahmut Bulut (vertexclique)

Rust CI failures

Hi All; I am seeing spurious connection drops at "Post Cache Docker Volumes" stage. Not sure if this is a direct issue or something we can trivially fix. Want to notify you. Best, Mahmut

Re: [Rust]: Exposed API

ou could use the >Rust >unsafe API for your advanced usecases rather than having to extend >arrow >itself. > >Andrew > >On Thu, Oct 8, 2020 at 9:10 AM vertexclique vertexclique < >vertexcli...@gmail.com> wrote: > >> Hi; >> >> Let me start

[Rust]: Exposed API

Hi; Let me start with my aim and how things are evolved in my mind. Through extensive usage of Arrow API, I've realized that we are doing so many unnecessary allocations and rebuilding for simple things like offset changes. (At least that's what I am doing). That said, it is tough to make the tra

Re: Help with memory alignment between Rust and C/Python

Hi; I have implemented Rust's specific memory alignments. If we want to enable fixed alignment on all platforms, we can do it by a feature gate. By default, it can come with the fixed alignment of 64; other users can enable platform-specific alignments with feature gate. Best, Mahmut On Sep 1

Re: [Rust] Async record batch reader?

Hi Max; There is an open issue in the tracker which needs to gather feedback to finalize how we will do overall async interface which spans to arrow crates. Please check that issue, it is mentioning sans IO and several design considerations. Imo we can carry async discussion under it. Best, Ma

Re: [RUST] Zero-copy Vec -> PrimitiveArray

Hi; If you have odd sized Vec it is not possible to get a zero-copy array with Arrow. Since you will access to non allocated memory region if you try to get a zero-copy view on that. What you will receive at the end of your buffer will be some garbage bytes (not mentioning UB and ownership of the

[Rust] Vectorized traits for using arrays outside Arrow

Hi; I wanted to discuss with Rust lib maintainers about how can we improve the current status of Rust's DictionaryArray and reading its' encoding array outside the Arrow. So a simple predicate filter needs to collect index over iterator and flat map over the optional values or map over the None