Re: Pickle data from python

[Antoine Pitrou]

There's already https://issues.apache.org/jira/browse/ARROW-1715

As for pickling Buffers, it's a bit more contentious.  Perhaps we should
stick to pickling higher-level types (arrays, batches, etc.).

Regards

Antoine.


Le 13/04/2018 à 03:22, Wes McKinney a écrit :
> hi Alberto,
> 
> If you cannot find a JIRA about pickling RecordBatch objects, could
> you please create one? A patch would be welcome for this; it is
> certainly in scope for the project.
> 
> If you encounter any new problems, please open a bug report.
> 
> Thanks!
> Wes
> 
> On Thu, Apr 12, 2018 at 3:13 PM, ALBERTO Bocchinfuso
>  wrote:
>> Hello,
>>
>> I cannot pickle RecordBatches, Buffers etc.
>>
>> I found Issue 1654 in the issue tracker, that has been solved with pull 
>> request 1238. But this looks to apply only to the types listed (schemas, 
>> DataTypes, etc.).
>> When I try to Pickle Buffers etc. I get exactly the same error reported in 
>> the issue report.
>> Is the implementation of the possibility of pickling all the data types of 
>> pyarrow (with particular attention to RecordBatches etc.) on the agenda?
>>
>> Thank you,
>> Alberto

Re: Pickle data from python

[Wes McKinney]

hi Alberto,

If you cannot find a JIRA about pickling RecordBatch objects, could
you please create one? A patch would be welcome for this; it is
certainly in scope for the project.

If you encounter any new problems, please open a bug report.

Thanks!
Wes

On Thu, Apr 12, 2018 at 3:13 PM, ALBERTO Bocchinfuso
 wrote:
> Hello,
>
> I cannot pickle RecordBatches, Buffers etc.
>
> I found Issue 1654 in the issue tracker, that has been solved with pull 
> request 1238. But this looks to apply only to the types listed (schemas, 
> DataTypes, etc.).
> When I try to Pickle Buffers etc. I get exactly the same error reported in 
> the issue report.
> Is the implementation of the possibility of pickling all the data types of 
> pyarrow (with particular attention to RecordBatches etc.) on the agenda?
>
> Thank you,
> Alberto