vyasr commented on issue #47337: URL: https://github.com/apache/arrow/issues/47337#issuecomment-3201924280
The bridge code (specifically https://github.com/apache/arrow/blob/cd8e6774dd02e1a32325c851c85a931276616cc2/python/pyarrow/includes/libarrow.pxd#L3172) can throw exceptions if certain invalid data is passed that is only validated in the C++ layer. Specifically, we observed this traceback: ``` #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x00007ffff7c6a859 in __GI_abort () at abort.c:79 #2 0x00007fffdd98a78a in google::LogMessage::Fail() () from /pyarrow/../../.././libglog.so.2 #3 0x00007fffdd98a977 in google::LogMessage::~LogMessage() () from /pyarrow/../../.././libglog.so.2 #4 0x00007fffe02eb371 in arrow::util::ArrowLog::~ArrowLog() () from /pyarrow/../../../libarrow.so.1900 #5 0x00007fffdf21f3f8 in arrow::Decimal128Type::Decimal128Type(int, int) () from /pyarrow/../../../libarrow.so.1900 #6 0x00007fffdf21f730 in arrow::decimal128(int, int) () from /pyarrow/../../../libarrow.so.1900 #7 0x00007fffdf238547 in arrow::(anonymous namespace)::SchemaImporter::ProcessDecimal() () from /cudf-dev/lib/python3.13/site-packages/pyarrow/../../../libarrow.so.1900 #8 0x00007fffdf23e923 in arrow::(anonymous namespace)::SchemaImporter::ProcessFormat() () from /cudf-dev/lib/python3.13/site-packages/pyarrow/../../../libarrow.so.1900 #9 0x00007fffdf240643 in arrow::(anonymous namespace)::SchemaImporter::DoImport() () from /pyarrow/../../../libarrow.so.1900 #10 0x00007fffdf24128e in arrow::ImportType(ArrowSchema*) () from /pyarrow/../../../libarrow.so.1900 #11 0x00007fffdf241674 in arrow::ImportArray(ArrowArray*, ArrowSchema*) () from /pyarrow/../../../libarrow.so.1900 #12 0x00007fffe0e93972 in __pyx_pw_7pyarrow_3lib_5Array_90_import_from_c_capsule(_object*, _object* const*, long, _object*) () ``` so at some point we hit [this line](https://github.com/apache/arrow/blob/066b2162206825f2d628f97f4113b0403da1f4ec/cpp/src/arrow/type.cc#L1521) validating that the decimal has valid precision (in this case it was because we intentionally produced a schema that had an out-of-range precision for decimal128 for testing). I'm sure there is other similar validation in the bridge code as well. Normally I expect pyarrow to do validation before handing off to libarrow so we don't encounter these error modes, but for the bridge code handling the C data interface that probably isn't worthwhile since all the parsing has to happen in C/C++ anyway, so rethrowing C++ exceptions as Python exceptions might be the easiest option for just the bridge APIs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
