Great analysis Weston!
Looks SimpleRecordBatch::column() is not thread safe for gcc < 5.0 as we are
simulating shared_ptr atomic load/store with normal load/store.
https://github.com/apache/arrow/blob/master/cpp/src/arrow/record_batch.cc#L80-L87
On 5/21/21 8:15 AM, Weston Pace wrote:
I like Yib
The field is always Int64Array. Regarding the arrowBatch *error reading
variable* message, we believe this is an artifact of gdb/gcc optimizations.
I examined the variable in lower contexts with gdb and it looks fine.
We replaced:
std::static_pointer_cast(_arrowBatch->column(_nAtts +
dim))->raw_v
I like Yibo's stack overflow theory given the "error reading variable"
but I did confirm that I can cause a segmentation fault if
std::atomic_store / std::atomic_load are unavailable. I simulated
this by simply commenting out the specializations rather than actually
run against GCC 4.9.2 so it ma
Also, is it possible that the field is not an Int64Array?
On Wed, May 19, 2021 at 10:19 PM Yibo Cai wrote:
>
> On 5/20/21 4:15 AM, Rares Vernica wrote:
> > Hello,
> >
> > I'm using Arrow for accessing data outside the SciDB database engine. It
> > generally works fine but we are running into Segm
If you use a CompressedOutputStream, then you get a compressed (e.g.
gzip) file.
If you want to use the Arrow IPC buffer compression, you need to specify
in IpcWriteOptions.
Regards
Antoine.
Le 20/05/2021 à 18:45, Rares Vernica a écrit :
Hello,
Just a clarifying question, when a Compr
+1 (binding)
I ported the RC verification script to make it easy for a
non-Rustacean like me to check an RC. I ran it on Apple aarch64:
https://github.com/apache/arrow-rs/pull/331
Run like
dev/release/verify-release-candidate.sh 4.1.0 2
On Tue, May 18, 2021 at 11:24 PM Jorge Cardoso Leitão
wr
Hello,
Just a clarifying question, when a CompressedOutputStream is used with
RecordBatchStreamWriter, are the composing Arrow arrays compressed
independently or the entire output file is compressed at once?
For example if we use GZIP, is the resulting file a valid GZIP file that we
can uncompres
Arrow Build Report for Job nightly-2021-05-20-0
All tasks:
https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-05-20-0
Failed Tasks:
- conda-osx-clang-py38:
URL:
https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-05-20-0-azure-conda-osx-clang-py3
