madhushreeag commented on code in PR #40677:
URL: https://github.com/apache/superset/pull/40677#discussion_r3358122912
##########
superset/result_set.py:
##########
@@ -174,16 +174,23 @@ def __init__( # pylint: disable=too-many-locals # noqa:
C901
array = np.array(data, dtype=numpy_dtype)
for column in column_names:
- try:
- pa_data.append(pa.array(array[column].tolist()))
- except (
- pa.lib.ArrowInvalid,
- pa.lib.ArrowTypeError,
- pa.lib.ArrowNotImplementedError,
- ValueError,
- TypeError, # this is super hackey,
- # https://issues.apache.org/jira/browse/ARROW-7855
+ raw = array[column].tolist()
+ for col_values in (
+ raw,
+ db_engine_spec.normalize_column_values(raw),
Review Comment:
Good catch! The current code eagerly builds normalize_column_values(raw)
even when the first pa.array() succeeds. Adding a
requires_column_value_normalization flag to skip the second attempt entirely
for engines that don’t need it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
