hi Simba, Yes -- Arrow list<T> types are converted to NumPy arrays when converting back to pandas with to_pandas(...). This conversion happens in C++ code in https://github.com/apache/arrow/blob/master/cpp/src/arrow/python/arrow_to_pandas.cc#L541
- Wes On Thu, Jan 18, 2018 at 1:26 PM, simba nyatsanga <simnyatsa...@gmail.com> wrote: > Good day everyone, > > I noticed what looks like type inference happening after persisting a > pandas DataFrame where one of the column values is a list. When I load up > the DataFrame again and do df.to_dict(), the value is no longer a list but > a numpy array. I dug through functions in the pandas_compat.py to try and > figure out at what point the dtype is being applied for that value. > > I'd like to verify if this is the intended behaviour. > > Here's an illustration of the behaviour: > > [image: Screen Shot 2018-01-18 at 15.54.59.png] > > Kind Regards > Simba >
