[
https://issues.apache.org/jira/browse/ARROW-9039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126325#comment-17126325
]
Micah Kornfield edited comment on ARROW-9039 at 6/5/20, 2:58 AM:
-----------------------------------------------------------------
Thank you for the report. This is intended behavior, the documentation was
clarified I think as of 0.16 or 0.15
([https://arrow.apache.org/docs/python/generated/pyarrow.serialize.html#pyarrow.serialize]).
Serialize/Deserialize do not provide backward compatibility. You need to you
use IPC functionality for compatibility guarantees (0.11 is quite old but I
don't think anything should have been broken between versions).
was (Author: [email protected]):
This is intended behavior, the documentation was clarified I think as of 0.16
or 0.15
([https://arrow.apache.org/docs/python/generated/pyarrow.serialize.html#pyarrow.serialize]).
Serialize/Deserialize do not provide backward compatibility. You need to you
use IPC functionality for compatibility guarantees (0.11 is quite old but I
don't think anything should have been broken between versions).
> py_bytes created by pyarrow 0.11.1 cannot be deserialized by more recent
> versions
> ---------------------------------------------------------------------------------
>
> Key: ARROW-9039
> URL: https://issues.apache.org/jira/browse/ARROW-9039
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 0.11.1, 0.15.1
> Environment: python, windows
> Reporter: Yoav Git
> Priority: Minor
>
> I have been saving dataframes into mongodb using:
> {{import pandas as pd; import pyarrow as pa}}
> {{df = pd.DataFrame([[1,2,3],[2,3,4]], columns = ['x','y','z'])}}
> {{byte = pa.serialize(df).to_buffer().to_pybytes()}}
> and then reading back using:
> {{df = pa.deserialize(pa.py_buffer(memoryview(byte)))}}
> However, pyarrow is not back-compatible. i.e. both versions 0.11.1 and 0.15.1
> can read their own pybytes created by it. Alas, they cannot read each other.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
