Hi devs,
I am creating a pyarrow.Array from a pandas.Series and when specifying the
data type, if the value will be truncated an error is raised. For example:
import pyarrow as pa
import pandas as pd
s = pd.Series([pd.Timestamp(1)])
arr = pa.Array.from_pandas(s, type=pa.timestamp('us'))
/home/bryan/miniconda2/lib/python2.7/site-packages/pyarrow/error.pxi in
pyarrow.lib.check_status
(/arrow/python/build/temp.linux-x86_64-2.7/lib.cxx:8270)()
75 message = frombytes(status.message())
76 if status.IsInvalid():
---> 77 raise ArrowInvalid(message)
78 elif status.IsIOError():
79 raise ArrowIOError(message)
ArrowInvalid: Casting from timestamp[ns] to timestamp[us] would lose data: 1
If I do the casting after creating the pyarrow.Array like
pa.Array.from_pandas(s).cast(pa.timestamp('us'), safe=False)
then it is fine. My question is does pyarrow.Array.from_pandas only use
safe casting or is there a way to specify that truncation is ok?
Thanks,
Bryan