jorisvandenbossche commented on code in PR #14238:
URL: https://github.com/apache/arrow/pull/14238#discussion_r980019914
##########
python/pyarrow/src/arrow_to_pandas.cc:
##########
@@ -730,17 +731,33 @@ Status DecodeDictionaries(MemoryPool* pool, const
std::shared_ptr<DataType>& den
template <typename ListArrayT>
Status ConvertListsLike(PandasOptions options, const ChunkedArray& data,
PyObject** out_values) {
+ using ListArrayType = typename ListArrayT::TypeClass;
+ const auto& list_type = checked_cast<const ListArrayType&>(*data.type());
+ const auto& value_type = list_type.value_type();
+
+ const auto& val_type = checked_cast<const ExtensionType&>(*value_type);
+ const auto& storage_ty = val_type.storage_type();
+ const auto& lt = dynamic_cast<const FixedSizeListType*>(&list_type);
+
// Get column of underlying value arrays
ArrayVector value_arrays;
for (int c = 0; c < data.num_chunks(); c++) {
- const auto& arr = checked_cast<const ListArrayT&>(*data.chunk(c));
+
+ std::shared_ptr<DataType> out_ty;
+ if (lt != nullptr) {
+ out_ty = fixed_size_list(storage_ty, lt->list_size());
+ } else {
+ out_ty = list(storage_ty);
+ }
+ compute::CastOptions options;
+ options.to_type = out_ty;
+ ARROW_ASSIGN_OR_RAISE(std::shared_ptr<Array> casted,
compute::Cast(*data.chunk(c), out_ty, options));
Review Comment:
Possible alternative solution to casting, would be to enable the
`ConvertChunkedArrayToPandas` (which is called on the chunks that are being
gathered here, a few lines below) to fallback to the storage type.
This could be enabled (without necessarily changing this globably) by adding
an option that is set in `MakeInnerOptions`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
