Re: sparse data array

Hi Bob, > I can observe that in a project like Arrow, there is always a tension > between compatibility and extensibility, and it makes me wonder if it would > be helpful to add capabilities without changing the spec. An extension type > can be defined in terms of one of the built-in layouts, but

Re: Re: [Format][RFC] Introduce COMPLEX type for IntervalUnit

To follow-up on this conversation I did some analysis on interval types: https://docs.google.com/document/d/1i1E_fdQ_xODZcAhsV11Pfq27O50k679OYHXFJpm9NS0/edit Please feel free to add more details/systems I missed. Given the disparate requirements of different systems I think the following might

Re: sparse data array

>From your response, I'm inferring that in order to introduce this kind of >compression, support in the spec is needed, similar to how compression types >and parameters are enumerated in >https://github.com/apache/arrow/blob/master/format/Message.fbs. Any change in >the spec is a Big Deal (and

Re: Status of Arrow Julia implementation?

Also, on the issue that there are no Julia-focused PMC members — note that I helped the JavaScript folks make their own independent releases for quite a while: called the votes (e.g. [1]), helped get people to verify and vote on the releases. After a time, it was decided to stop releasing independe

Re: Status of Arrow Julia implementation?

hi Jacob, On Tue, Mar 30, 2021 at 4:18 PM Jacob Quinn wrote: > > I can comment as the primary apache arrow liaison for the Arrow.jl > repository and original code donator. > > I apologize for the "surprise", but I commented a few times in various > places and put a snippet in the README >

Re: Status of Arrow Julia implementation?

I can comment as the primary apache arrow liaison for the Arrow.jl repository and original code donator. I apologize for the "surprise", but I commented a few times in various places and put a snippet in the README

Status of Arrow Julia implementation?

hi folks, I was very surprised today to learn that the Julia Arrow implementation has continued operating more or less like an independent open source project since the code donation last November: https://github.com/JuliaData/Arrow.jl/commits/main There may have been a misunderstanding about wh

RE: Re: sparse data array

> > > On a related note, such encoding would address DataFusion's issue of > > representing scalars / constant arrays: a constant array would be > > represented as a repetition. Currently we just unpack (i.e. allocate) a > > constant array when we want to transfer through a RecordBatch. > In the J

Arrow sync call March 31 at 12:00 US/Eastern, 16:00 UTC

Hi all, Our biweekly call is coming up tomorrow at https://meet.google.com/vtm-teks-phx. All are welcome to join. I won't be able to attend this week, but hopefully someone else will share notes with the mailing list afterward. Neal

Re: [R][Rust][IPC] Attempting to pass RecordBatch from R to Rust via C ABI

> I'd like to file an issue to add a > method to StreamWriter to output the StreamWriter.writer (basically how > .into_inner() is for BufWriter). I'd appreciate any feedback on that idea. This seems like a good idea to me On Mon, Mar 29, 2021 at 10:42 AM Eric Burden wrote: > (1) Zero-copy woul

[JIRA] Archived "CI" component

Hello, I've archived the "CI" component on JIRA since it was redundant with another component named "Continuous Integration". Regards Antoine.

[NIGHTLY] Arrow Build Report for Job nightly-2021-03-30-0

Arrow Build Report for Job nightly-2021-03-30-0 All tasks: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-03-30-0 Failed Tasks: - conda-linux-gcc-py37-aarch64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-03-30-0-drone-conda-linux