Re: Efficient Pandas serialization for mixed object and numeric DataFrames

How are you serializing the dataframe? If you use *pyarrow.serialize(df)*, then each column should be serialized separately and numeric columns will be handled efficiently. On Thu, Oct 18, 2018 at 9:10 PM Mitar wrote: > Hi! > > It seems that if a DataFrame contains both numeric and object

Efficient Pandas serialization for mixed object and numeric DataFrames

Hi! It seems that if a DataFrame contains both numeric and object columns, the whole DataFrame is pickled and not that only object columns are pickled? Is this right? Are there any plans to improve this? Mitar -- http://mitar.tnode.com/ https://twitter.com/mitar_m

[jira] [Created] (ARROW-3559) Statically link libraries for plasma_store_server executable.

Robert Nishihara created ARROW-3559: --- Summary: Statically link libraries for plasma_store_server executable. Key: ARROW-3559 URL: https://issues.apache.org/jira/browse/ARROW-3559 Project: Apache

Re: [VOTE] Accept donation of Ruby bindings to Parquet GLib

+1 In "[VOTE] Accept donation of Ruby bindings to Parquet GLib" on Thu, 18 Oct 2018 16:59:41 -0400, Wes McKinney wrote: > hello, > > Kouhei Sutou is proposing to donate Ruby bindings to the Parquet GLib > library, which was received as a donation in September. This Ruby > library was

[jira] [Created] (ARROW-3558) Remove fatal error when plasma client calls get on an unsealed object that it created.

Robert Nishihara created ARROW-3558: --- Summary: Remove fatal error when plasma client calls get on an unsealed object that it created. Key: ARROW-3558 URL: https://issues.apache.org/jira/browse/ARROW-3558

Re: [VOTE] Accept donation of Ruby bindings to Parquet GLib

+1 > Am 18.10.2018 um 22:59 schrieb Wes McKinney : > > hello, > > Kouhei Sutou is proposing to donate Ruby bindings to the Parquet GLib > library, which was received as a donation in September. This Ruby > library was originally developed at > > https://github.com/red-data-tools/red-parquet/

[VOTE] Accept donation of Ruby bindings to Parquet GLib

hello, Kouhei Sutou is proposing to donate Ruby bindings to the Parquet GLib library, which was received as a donation in September. This Ruby library was originally developed at https://github.com/red-data-tools/red-parquet/ Kou has submitted the work as a pull request

[jira] [Created] (ARROW-3556) [CI] Disable optimizations on Windows

Antoine Pitrou created ARROW-3556: - Summary: [CI] Disable optimizations on Windows Key: ARROW-3556 URL: https://issues.apache.org/jira/browse/ARROW-3556 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-3555) [Plasma] Unify plasma client get function using metadata.

Yuhong Guo created ARROW-3555: - Summary: [Plasma] Unify plasma client get function using metadata. Key: ARROW-3555 URL: https://issues.apache.org/jira/browse/ARROW-3555 Project: Apache Arrow

Support for TIMESTAMP_NANOS in parquet-cpp

Hi everyone, in parquet-format, there is now support for TIMESTAMP_NANOS: https://github.com/apache/parquet-format/pull/102 For parquet-cpp, this is not yet supported. I have a few questions now: • is there an overview of what release of parquet-format is currently fully support in parquet-cpp

[jira] [Created] (ARROW-3554) [C++] Reverse traits for C++

Wolf Vollprecht created ARROW-3554: -- Summary: [C++] Reverse traits for C++ Key: ARROW-3554 URL: https://issues.apache.org/jira/browse/ARROW-3554 Project: Apache Arrow Issue Type:

Re: Making a bugfix 0.11.1 release

Hi Antoine, Thanks for the quick response! This helps to clear up my confusion. Best Regards, Kevin Gurney From: Antoine Pitrou Sent: Thursday, October 18, 2018 9:54:47 AM To: dev@arrow.apache.org Subject: Re: Making a bugfix 0.11.1 release Le

[jira] [Created] (ARROW-3553) [R] Error when losing data on int64, uint64 conversions to double

Wes McKinney created ARROW-3553: --- Summary: [R] Error when losing data on int64, uint64 conversions to double Key: ARROW-3553 URL: https://issues.apache.org/jira/browse/ARROW-3553 Project: Apache Arrow

Re: Making a bugfix 0.11.1 release

Le 18/10/2018 à 15:44, Kevin Gurney a écrit : > Hi All, > > We are working with the arrow version 0.9.0 C++ libraries in conjunction with > separate parquet-cpp version 1.4.0. > > Questions: > > 1. Does this zlib issue affect all clients of the arrow C++ libraries or > just the Python

[jira] [Created] (ARROW-3552) [Python] Implement pa.RecordBatch.serialize_to to write single message to an OutputStream

Wes McKinney created ARROW-3552: --- Summary: [Python] Implement pa.RecordBatch.serialize_to to write single message to an OutputStream Key: ARROW-3552 URL: https://issues.apache.org/jira/browse/ARROW-3552

Re: Making a bugfix 0.11.1 release

Hi All, We are working with the arrow version 0.9.0 C++ libraries in conjunction with separate parquet-cpp version 1.4.0. Questions: 1. Does this zlib issue affect all clients of the arrow C++ libraries or just the Python PyArrow code? 2. Does this zlib compression issue also affect the

[jira] [Created] (ARROW-3551) Change MapD to OmniSci on Powered By page

Todd Mostak created ARROW-3551: -- Summary: Change MapD to OmniSci on Powered By page Key: ARROW-3551 URL: https://issues.apache.org/jira/browse/ARROW-3551 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-3550) [C++] Use kUnknownNullCount in NumericArray constructor

Wolf Vollprecht created ARROW-3550: -- Summary: [C++] Use kUnknownNullCount in NumericArray constructor Key: ARROW-3550 URL: https://issues.apache.org/jira/browse/ARROW-3550 Project: Apache Arrow

Re: Making a bugfix 0.11.1 release

I've added the two zlib issues to 0.11.1 version: https://issues.apache.org/jira/projects/ARROW/versions/12344316 On Wed, Oct 17, 2018 at 10:51 PM Wes McKinney wrote: > Got it, thank you for clarifying. It wasn't clear whether the bug > would occur in the build environment (CentOS 5 +