danepitkin opened a new issue, #37235:
URL: https://github.com/apache/arrow/issues/37235
### Describe the bug, including details regarding any error messages,
version, and platform.
Refactor Substrait.run_query() in C++/PyArrow so that it does not suppress
user-defined Exceptions.
In PyArrow, Substrait can take in a `table_provider` object, which can map a
table name in the Substrait plan to a table object in memory. This
`table_provider` is user-defined python code, which can throw an exception. The
underlying C++ implementation currently suppresses user-defined expressions and
will instead report a more generic error[1]:
```
ARROW_ASSIGN_OR_RAISE(acero::Declaration source_decl,
named_table_provider(table_names,
*base_schema));
if (!source_decl.IsValid()) {
return Status::Invalid("Invalid NamedTable Source");
}
```
When upgrading to Cython 3, the user-defined exception is now raised by
default because cdef functions that are not extern now safely propagate Python
exceptions by default. To mask raising this exception, we have to explicitly
add `noexcept` to this Cython function[2]:
```
cdef CDeclaration _create_named_table_provider(
dict named_args, const std_vector[c_string]& names, const CSchema& schema
) noexcept:
```
This is the pytest output due to suppressing the python user-defined
Exception:
```
pyarrow/tests/test_substrait.py::test_named_table_invalid_table_name
/opt/homebrew/Caskroom/mambaforge/base/envs/arrow-dev/lib/python3.11/site-packages/_pytest/unraisableexception.py:78:
PytestUnraisableExceptionWarning: Exception ignored in:
'pyarrow._substrait._create_named_table_provider'
Traceback (most recent call last):
File "/Users/dane/code/arrow/python/pyarrow/tests/test_substrait.py",
line 250, in table_provider
raise Exception("Unrecognized table name")
Exception: Unrecognized table name
warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
```
[1]
https://github.com/apache/arrow/blob/main/cpp/src/arrow/engine/substrait/relation_internal.cc#L422-L427
[2]
https://github.com/apache/arrow/blob/main/python/pyarrow/_substrait.pyx#L29
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
