jorisvandenbossche commented on a change in pull request #7623:
URL: https://github.com/apache/arrow/pull/7623#discussion_r451646062
##########
File path: python/pyarrow/_dataset.pyx
##########
@@ -881,10 +881,15 @@ cdef class RowGroupInfo:
name = frombytes(c_statistics.type.get().field(i).get().name())
c_minmax = <CStructScalar*> c_statistics.value[i].get()
- statistics[name] = {
- 'min': pyarrow_wrap_scalar(c_minmax.value[0]).as_py(),
- 'max': pyarrow_wrap_scalar(c_minmax.value[1]).as_py(),
- }
+ try:
+ statistics[name] = {
+ 'min': pyarrow_wrap_scalar(c_minmax.value[0]).as_py(),
+ 'max': pyarrow_wrap_scalar(c_minmax.value[1]).as_py(),
+ }
+ except ValueError:
+ # Don't treat failure to parse/convert a single Scalar as a
+ # failure. The min/max will simply be missing for this field.
Review comment:
See also related question above:
https://github.com/apache/arrow/pull/7623#discussion_r449647463
I am not sure we *can* test it right now, as all scalars are supported in
Python.
I suppose the rationale is that as a user experience it can be better to
still return the statistics of all other columns, instead of raising due to one
column not being recognized.
On the other hand, it indeed suppresses an error that would be useful to see
(for us developers).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
