[jira] [Created] (ARROW-5276) [C#] NativeMemoryAllocator expose an option for clearing allocated memory

Eric Erhardt created ARROW-5276: --- Summary: [C#] NativeMemoryAllocator expose an option for clearing allocated memory Key: ARROW-5276 URL: https://issues.apache.org/jira/browse/ARROW-5276 Project:

[jira] [Created] (ARROW-5278) [C#] ArrowBuffer should either implement IEquatable correctly or not at all

Eric Erhardt created ARROW-5278: --- Summary: [C#] ArrowBuffer should either implement IEquatable correctly or not at all Key: ARROW-5278 URL: https://issues.apache.org/jira/browse/ARROW-5278 Project:

[jira] [Created] (ARROW-5275) [C++] Write generic filesystem tests

Antoine Pitrou created ARROW-5275: - Summary: [C++] Write generic filesystem tests Key: ARROW-5275 URL: https://issues.apache.org/jira/browse/ARROW-5275 Project: Apache Arrow Issue Type: Task

[jira] [Created] (ARROW-5277) [C#] MemoryAllocator.Allocate(length: 0) should not return null

Eric Erhardt created ARROW-5277: --- Summary: [C#] MemoryAllocator.Allocate(length: 0) should not return null Key: ARROW-5277 URL: https://issues.apache.org/jira/browse/ARROW-5277 Project: Apache Arrow

Re: Stored state of incremental writes to fixed size Arrow buffer?

hi John, On Tue, May 7, 2019 at 10:53 AM John Muehlhausen wrote: > > Wes et al, I completed a preliminary study of populating a Feather file > incrementally. Some notes and questions: > > I wrote the following dataframe to a feather file: > ab > 0 0123456789 0.0 > 1

Re: Stored state of incremental writes to fixed size Arrow buffer?

Wes et al, I completed a preliminary study of populating a Feather file incrementally. Some notes and questions: I wrote the following dataframe to a feather file: ab 0 0123456789 0.0 1 0123456789 NaN 2 0123456789 NaN 3 0123456789 NaN 4None NaN In re-writing the

Re: Stored state of incremental writes to fixed size Arrow buffer?

Thanks Wes: "the current Feather format is deprecated" ... yes, but there will be a future file format that replaces it, correct? And my discussion of immutable "portions" of Arrow buffers, rather than immutability of the entire buffer, applies to IPC as well, right? I am only championing the

Re: Stored state of incremental writes to fixed size Arrow buffer?

Wes, are we saying that `pa.ipc.open_file(...).read_pandas()` already reads the future Feather format? If not, how will the future format differ? I will work on my access pattern with this format instead of the current feather format. Sorry I was not clear on that earlier. Micah, thank you! On

RE: [DISCUSS][C++][Proposal] Threading engine for Arrow

> From: Jed Brown [mailto:j...@jedbrown.org] > Sent: Monday, May 6, 2019 16:35 > Nice paper, thanks! Did you investigate latency impact from the IPC counting > semaphore? Is your test code available? Not that deep. Basically I was looking only if its positive effect is enough to overcome the

Re: Stored state of incremental writes to fixed size Arrow buffer?

Hi John, To give a specific pointer [1] describes how the streaming protocol is stored to a file. [1] https://arrow.apache.org/docs/format/IPC.html#file-format On Tue, May 7, 2019 at 9:40 AM Wes McKinney wrote: > hi John, > > As soon as the R folks can install the Arrow R package consistently,

Re: Stored state of incremental writes to fixed size Arrow buffer?

On Tue, May 7, 2019 at 12:26 PM John Muehlhausen wrote: > > Wes, are we saying that `pa.ipc.open_file(...).read_pandas()` already reads > the future Feather format? If not, how will the future format differ? I > will work on my access pattern with this format instead of the current > feather

[jira] [Created] (ARROW-5279) [C++] Support reading delta dictionaries in IPC streams

Wes McKinney created ARROW-5279: --- Summary: [C++] Support reading delta dictionaries in IPC streams Key: ARROW-5279 URL: https://issues.apache.org/jira/browse/ARROW-5279 Project: Apache Arrow

[jira] [Created] (ARROW-5280) [C++] Find a better solution to the conda compilers macOS issue

Neal Richardson created ARROW-5280: -- Summary: [C++] Find a better solution to the conda compilers macOS issue Key: ARROW-5280 URL: https://issues.apache.org/jira/browse/ARROW-5280 Project: Apache

Re: [DISCUSS][C++] Static versus variable Arrow dictionary encoding

I have started working on this some to assess what is involved. My present plan is to have FixedDictionaryType and FixedDictionaryArray VariableDictionaryType and VariableDictionaryArray deprecate (?) current DictionaryType/DictionaryArray names, for clarity (thoughts about this would be welcome

[jira] [Created] (ARROW-5281) [Rust] [Parquet] Move DataPageBuilder to test_common

Renjie Liu created ARROW-5281: - Summary: [Rust] [Parquet] Move DataPageBuilder to test_common Key: ARROW-5281 URL: https://issues.apache.org/jira/browse/ARROW-5281 Project: Apache Arrow Issue

[jira] [Created] (ARROW-5282) Can't read data from parquet file in C++ library

worker24h created ARROW-5282: Summary: Can't read data from parquet file in C++ library Key: ARROW-5282 URL: https://issues.apache.org/jira/browse/ARROW-5282 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-5283) [C++][Plasma] Server crash when creating an aborted object 3 times

shengjun.li created ARROW-5283: -- Summary: [C++][Plasma] Server crash when creating an aborted object 3 times Key: ARROW-5283 URL: https://issues.apache.org/jira/browse/ARROW-5283 Project: Apache Arrow

[jira] [Created] (ARROW-5284) [Rust] Replace libc with std::alloc for memory allocation

Chao Sun created ARROW-5284: --- Summary: [Rust] Replace libc with std::alloc for memory allocation Key: ARROW-5284 URL: https://issues.apache.org/jira/browse/ARROW-5284 Project: Apache Arrow Issue

Re: [DISCUSS][JAVA]Support Fast/Unsafe Vector APIs for Arrow

Hi Jacques, Thanks a lot for your comments. I have evaluated the assembly code of original Arrow API, as well as the unsafe API in our PR Generally, the assembly code generated by JIT for both APIs are of high quality, and for most cases, the assembly