Paul Horn created ARROW-16606:
---------------------------------
Summary: [FlightRPC][Python] Flight RPC crashes when a middleware
sends an authorization header written with an upper-case A as in
'Authorization'
Key: ARROW-16606
URL: https://issues.apache.org/jira/browse/ARROW-16606
Project: Apache Arrow
Issue Type: Bug
Components: FlightRPC, Python
Affects Versions: 8.0.0, 7.0.0
Environment: Python 3.9.12 on macOS 12.3.1
Reporter: Paul Horn
Sending a custom `Authorization` header leads to a crash of the client
Running this python code, for example
{code:java}
import pyarrow.flight as flight
class TestMiddlewareFactory(ClientMiddlewareFactory):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def start_call(self, info):
return TestMiddleware()
class TestMiddleware(ClientMiddleware):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def sending_headers(self):
return {"Authorization": "Basic dXNlcjpwYXNz"}
def test():
client = flight.FlightClient("grpc://localhost:8491",
middleware=[TestMiddlewareFactory()])
client.do_get(flight.Ticket(""))
{code}
Results in
{noformat}
tests/rpc_repro.py Fatal Python error: AbortedCurrent thread 0x0000000202ecc600
(most recent call first):
File "tests/rpc_repro.py", line 22 in test
File "venv/lib/python3.9/site-packages/_pytest/python.py", line 183 in
pytest_pyfunc_call
File "venv/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in
_multicall
File "venv/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in
_hookexec
File "venv/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
File "venv/lib/python3.9/site-packages/_pytest/python.py", line 1641 in
runtest
File "venv/lib/python3.9/site-packages/_pytest/runner.py", line 162 in
pytest_runtest_call
File "venv/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in
_multicall
File "venv/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in
_hookexec
File "venv/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
File "venv/lib/python3.9/site-packages/_pytest/runner.py", line 255 in
<lambda>
File "venv/lib/python3.9/site-packages/_pytest/runner.py", line 311 in
from_call
File "venv/lib/python3.9/site-packages/_pytest/runner.py", line 254 in
call_runtest_hook
File "venv/lib/python3.9/site-packages/_pytest/runner.py", line 215 in
call_and_report
File "venv/lib/python3.9/site-packages/_pytest/runner.py", line 126 in
runtestprotocol
File "venv/lib/python3.9/site-packages/_pytest/runner.py", line 109 in
pytest_runtest_protocol
File "venv/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in
_multicall
File "venv/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in
_hookexec
File "venv/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
File "venv/lib/python3.9/site-packages/_pytest/main.py", line 348 in
pytest_runtestloop
File "venv/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in
_multicall
File "venv/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in
_hookexec
File "venv/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
File "venv/lib/python3.9/site-packages/_pytest/main.py", line 323 in _main
File "venv/lib/python3.9/site-packages/_pytest/main.py", line 269 in
wrap_session
File "venv/lib/python3.9/site-packages/_pytest/main.py", line 316 in
pytest_cmdline_main
File "venv/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in
_multicall
File "venv/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in
_hookexec
File "venv/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
File "venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 162
in main
File "venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 185
in console_main
File "venv/bin/pytest", line 8 in <module>
Abort trap: 6 {noformat}
With an additional crash report from the OS
{noformat}
Process: Python [26728]Path:
/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/PythonIdentifier:
org.python.pythonVersion: 3.9.12 (3.9.12)Code Type:
X86-64 (Translated)Parent Process: bash [4683]Responsible:
iTerm2 [99236]User ID: 501
Date/Time: 2022-05-18 15:35:10.1978 +0200OS Version:
macOS 12.3.1 (21E258)Report Version: 12Anonymous UUID:
4A72633D-06AC-F2CE-0E3F-0AD87FA611CE
Sleep/Wake UUID: 3D7BD416-99A9-41B3-8163-5544AEF31FF5
Time Awake Since Boot: 1000000 secondsTime Since Wake: 22827 seconds
System Integrity Protection: enabled
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_CRASH (SIGABRT)Exception Codes:
0x0000000000000000, 0x0000000000000000Exception Note: EXC_CORPSE_NOTIFY
Application Specific Information:abort() called
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread0 ???
0x7ff8a597e940 ???1 libsystem_kernel.dylib
0x7ff81537300e __pthread_kill + 102 libsystem_pthread.dylib
0x7ff8153a91ff pthread_kill + 2633 libsystem_c.dylib
0x7ff8152b72c8 raise + 264 libsystem_platform.dylib 0x7ff8153bedfd
_sigtramp + 295 ??? 0x206e69207961 ???6
libsystem_c.dylib 0x7ff8152f4d24 abort + 1237
libarrow_flight.700.dylib 0x1333dac17
grpc::CoreCodegen::assert_fail(char const*, char const*, int) + 398
libarrow_flight.700.dylib 0x13337f4a2
grpc::internal::CallOpSet<grpc::internal::CallOpSendInitialMetadata,
grpc::internal::CallOpSendMessage, grpc::internal::CallOpClientSendClose,
grpc::internal::CallNoOp<4>, grpc::internal::CallNoOp<5>,
grpc::internal::CallNoOp<6> >::ContinueFillOpsAfterInterception() + 2589
libarrow_flight.700.dylib 0x13337aa07
arrow::flight::GrpcClientInterceptorAdapter::Intercept(grpc::experimental::InterceptorBatchMethods*)
+ 29510 libarrow_flight.700.dylib 0x13337dd76
grpc::internal::InterceptorBatchMethodsImpl::RunInterceptors() + 29411
libarrow_flight.700.dylib 0x13337f350
grpc::internal::CallOpSet<grpc::internal::CallOpSendInitialMetadata,
grpc::internal::CallOpSendMessage, grpc::internal::CallOpClientSendClose,
grpc::internal::CallNoOp<4>, grpc::internal::CallNoOp<5>,
grpc::internal::CallNoOp<6> >::FillOps(grpc::internal::Call*) + 25612
libarrow_flight.700.dylib 0x1333affb1
grpc::ClientReader<arrow::flight::protocol::FlightData>::ClientReader<arrow::flight::protocol::Ticket>(grpc::ChannelInterface*,
grpc::internal::RpcMethod const&, grpc::ClientContext*,
arrow::flight::protocol::Ticket const&) + 68913 libarrow_flight.700.dylib
0x13335b1cf
arrow::flight::FlightClient::FlightClientImpl::DoGet(arrow::flight::FlightCallOptions
const&, arrow::flight::Ticket const&,
std::__1::unique_ptr<arrow::flight::FlightStreamReader,
std::__1::default_delete<arrow::flight::FlightStreamReader> >*) + 25514
libarrow_flight.700.dylib 0x13335b0c1
arrow::flight::FlightClient::DoGet(arrow::flight::FlightCallOptions const&,
arrow::flight::Ticket const&,
std::__1::unique_ptr<arrow::flight::FlightStreamReader,
std::__1::default_delete<arrow::flight::FlightStreamReader> >*) + 1715
_flight.cpython-39-darwin.so 0x1328544b7
__pyx_pw_7pyarrow_7_flight_12FlightClient_22do_get(_object*, _object*,
_object*) + 23116 Python 0x108f05c5a
method_vectorcall_VARARGS_KEYWORDS + 17617 Python
0x108fe7ca4 _PyEval_EvalFrameDefault + 2226318 Python
0x108ef9fc0 _PyFunction_Vectorcall + 19219 Python
0x108fe7ca4 _PyEval_EvalFrameDefault + 2226320 Python
0x108fe142c _PyEval_EvalCode + 39821 Python
0x108efa078 _PyFunction_Vectorcall + 37622 Python
0x108fe7ca4 _PyEval_EvalFrameDefault +
2226323 Python 0x108ef9fc0
_PyFunction_Vectorcall + 19224 Python
0x108efcb53 method_vectorcall + 16025 Python
0x108fe8eff _PyEval_EvalFrameDefault + 2696226 Python
0x108fe142c _PyEval_EvalCode + 39827 Python
0x108efa078 _PyFunction_Vectorcall + 37628 Python
0x108efcb53 method_vectorcall + 16029 Python
0x108fe79bc _PyEval_EvalFrameDefault + 2151930 Python
0x108fe142c _PyEval_EvalCode + 39831 Python
0x108efa078 _PyFunction_Vectorcall + 37632 Python
0x108ef9c92 PyVectorcall_Call + 16433 Python
0x108fe3295 _PyEval_EvalFrameDefault + 330434
Python 0x108fe142c _PyEval_EvalCode + 39835
Python 0x108efa078 _PyFunction_Vectorcall +
37636 Python 0x108fe3295
_PyEval_EvalFrameDefault + 330437 Python
0x108fe142c _PyEval_EvalCode + 39838 Python
0x108efa078 _PyFunction_Vectorcall + 37639 Python
0x108fe8eff _PyEval_EvalFrameDefault + 2696240 Python
0x108ef9fc0 _PyFunction_Vectorcall + 19241 Python
0x108efcb53 method_vectorcall + 16042 Python
0x108fe8eff _PyEval_EvalFrameDefault + 2696243 Python
0x108fe142c _PyEval_EvalCode + 39844 Python
0x108efa078 _PyFunction_Vectorcall + 37645
Python 0x108ef984c
_PyObject_FastCallDictTstate + 21546 Python
0x108f67213 slot_tp_call + 18747 Python
0x108ef95ce _PyObject_MakeTpCall + 12748 Python
0x108fe90a7 _PyEval_EvalFrameDefault + 2738649 Python
0x108ef9fc0 _PyFunction_Vectorcall + 19250 Python
0x108fe7ca4 _PyEval_EvalFrameDefault + 2226351 Python
0x108ef9fc0 _PyFunction_Vectorcall + 19252 Python
0x108fe3295 _PyEval_EvalFrameDefault + 330453
Python 0x108fe142c _PyEval_EvalCode + 39854
Python 0x108efa078 _PyFunction_Vectorcall +
37655 Python 0x108fe8eff
_PyEval_EvalFrameDefault + 2696256 Python
0x108ef9fc0 _PyFunction_Vectorcall + 19257 Python
0x108efcb53 method_vectorcall + 16058 Python
0x108fe8eff _PyEval_EvalFrameDefault + 2696259 Python
0x108fe142c _PyEval_EvalCode + 39860 Python
0x108efa078 _PyFunction_Vectorcall + 37661 Python
0x108ef984c _PyObject_FastCallDictTstate + 21562
Python 0x108f67213 slot_tp_call + 18763 Python
0x108ef9e1a _PyObject_Call + 12164 Python
0x108fe3295 _PyEval_EvalFrameDefault + 330465
Python 0x108fe142c _PyEval_EvalCode + 39866
Python 0x108efa078 _PyFunction_Vectorcall +
37667 Python 0x108fe7957
_PyEval_EvalFrameDefault + 2141868 Python
0x108fe142c _PyEval_EvalCode + 39869 Python
0x108efa078 _PyFunction_Vectorcall + 37670 Python
0x108efcb53 method_vectorcall + 16071 Python
0x108fe79bc _PyEval_EvalFrameDefault + 2151972 Python
0x108fe142c _PyEval_EvalCode + 39873 Python
0x108efa078 _PyFunction_Vectorcall + 37674 Python
0x108fe3295 _PyEval_EvalFrameDefault + 330475 Python
0x108fe142c _PyEval_EvalCode + 39876 Python
0x108efa078 _PyFunction_Vectorcall + 37677
Python 0x108fe7957
_PyEval_EvalFrameDefault + 2141878 Python
0x108fe142c _PyEval_EvalCode + 39879 Python
0x108efa078 _PyFunction_Vectorcall + 37680 Python
0x108fe79bc _PyEval_EvalFrameDefault + 2151981 Python
0x108ef9fc0 _PyFunction_Vectorcall + 19282 Python
0x108fe3295 _PyEval_EvalFrameDefault + 330483 Python
0x108fe142c _PyEval_EvalCode + 39884 Python
0x108efa078 _PyFunction_Vectorcall + 37685
Python 0x108fe8eff
_PyEval_EvalFrameDefault + 2696286 Python
0x108ef9fc0 _PyFunction_Vectorcall + 19287 Python
0x108efcb53 method_vectorcall + 16088 Python
0x108fe8eff _PyEval_EvalFrameDefault + 2696289 Python
0x108fe142c _PyEval_EvalCode + 39890 Python
0x108efa078 _PyFunction_Vectorcall + 37691 Python
0x108ef984c _PyObject_FastCallDictTstate + 21592
Python 0x108f67213 slot_tp_call + 18793 Python
0x108ef95ce _PyObject_MakeTpCall + 12794 Python
0x108fe90a7 _PyEval_EvalFrameDefault + 2738695
Python 0x108ef9fc0 _PyFunction_Vectorcall +
19296 Python 0x108fe3295
_PyEval_EvalFrameDefault + 330497 Python
0x108fe142c _PyEval_EvalCode + 39898 Python
0x108efa078 _PyFunction_Vectorcall + 37699 Python
0x108fe8eff _PyEval_EvalFrameDefault + 26962100 Python
0x108ef9fc0 _PyFunction_Vectorcall + 192101 Python
0x108efcb53 method_vectorcall + 160102 Python
0x108fe8eff _PyEval_EvalFrameDefault + 26962103 Python
0x108fe142c _PyEval_EvalCode + 398104 Python
0x108efa078 _PyFunction_Vectorcall + 376105
Python 0x108ef984c
_PyObject_FastCallDictTstate + 215106 Python
0x108f67213 slot_tp_call + 187107 Python
0x108ef95ce _PyObject_MakeTpCall + 127108 Python
0x108fe90a7 _PyEval_EvalFrameDefault + 27386109 Python
0x108ef9fc0 _PyFunction_Vectorcall + 192110 Python
0x108fe7957 _PyEval_EvalFrameDefault + 21418111 Python
0x108ef9fc0 _PyFunction_Vectorcall + 192112 Python
0x108fe7957 _PyEval_EvalFrameDefault + 21418113
Python 0x108ef9fc0 _PyFunction_Vectorcall +
192114 Python 0x108fe3295
_PyEval_EvalFrameDefault + 3304115 Python
0x108fe142c _PyEval_EvalCode + 398116 Python
0x108efa078 _PyFunction_Vectorcall + 376117 Python
0x108fe8eff _PyEval_EvalFrameDefault + 26962118 Python
0x108ef9fc0 _PyFunction_Vectorcall + 192119 Python
0x108efcb53 method_vectorcall + 160120 Python
0x108fe8eff _PyEval_EvalFrameDefault + 26962121 Python
0x108fe142c _PyEval_EvalCode + 398122 Python
0x108efa078 _PyFunction_Vectorcall + 376123
Python 0x108ef984c
_PyObject_FastCallDictTstate + 215124 Python
0x108f67213 slot_tp_call + 187125 Python
0x108ef95ce _PyObject_MakeTpCall + 127126 Python
0x108fe90a7 _PyEval_EvalFrameDefault + 27386127 Python
0x108ef9fc0 _PyFunction_Vectorcall + 192128 Python
0x108fe7957 _PyEval_EvalFrameDefault + 21418129 Python
0x108ef9fc0 _PyFunction_Vectorcall + 192130 Python
0x108fe7957 _PyEval_EvalFrameDefault + 21418131
Python 0x108fe142c _PyEval_EvalCode + 398132
Python 0x10903c1de run_eval_code_obj +
128133 Python 0x10903c136 run_mod + 96134
Python 0x1090398f4 pyrun_file + 167135
Python 0x1090392e4 pyrun_simple_file + 271136
Python 0x1090391af PyRun_SimpleFileExFlags +
67137 Python 0x109058bfb pymain_run_file +
326138 Python 0x109058469 Py_RunMain +
993139 Python 0x1090597c4 Py_BytesMain + 42140
dyld 0x2006a551e start + 462{noformat}
----
Sending the header lower-case, {{{}authorization{}}}, works. There is no
indication that headers are case-sensitive and the error is most unhelpful at
pointing to the underlying issue. If headers are not case-insensitive, I would
at least hope for a better error diagnostic that would tell me so.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
