[
https://issues.apache.org/jira/browse/ARROW-16592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17538437#comment-17538437
]
Lubo Slivka edited comment on ARROW-16592 at 5/17/22 8:03 PM:
--------------------------------------------------------------
Hi [~lidavidm] , this was fun, I was trying many things left and right to
extend the barebone reproducer with extra embellishments of my impl. No luck.
So then I tried sending same data that I'm using for my tests to your code
above, and hit the internal error. Trying to narrow it down for your testcase,
I got the following reproducer (the only diff from your code is the loop to
write the batches):
{code:java}
import traceback
import pyarrow as pa
import pyarrow.flight as flight
class Server(flight.FlightServerBase):
def do_put(self, context, descriptor, reader, writer):
raise flight.FlightCancelledError("foo", extra_info=b"bar")
print("PyArrow version:", pa.__version__)
server = Server("grpc://localhost:0")
client = flight.connect(f"grpc://localhost:{server.port}")
schema = pa.schema([("a", pa.int64())])
writer, reader = client.do_put(flight.FlightDescriptor.for_command(b""), schema)
try:
# starts failing at this exact number; before that the error is still
cancelled
for i in range(27061):
# print(f"write {i}")
writer.write_batch(pa.record_batch([[1]], schema=schema))
writer.close()
except flight.FlightError as e:
traceback.print_exc()
print(e.extra_info)
except Exception:
traceback.print_exc() {code}
note: this may be related to timing or something, if the repro does not work,
try bumping the numbers :)
what strikes me (and smells like the actual root cause) is that the loop goes
on and does not fail after the first write. that looks awkward - could this be
the real problem?
also note, I originally reproduced this with the test data that I read from CSV
into a Table, and then tried doing a single writer.write_table() followed by
writer.close().
if needed, I can provide the test data, it's a sheet generated from tpch model.
was (Author: JIRAUSER286791):
Hi [~lidavidm] , this was fun, I was trying many things left and right to
extend the barebone reproducer with extra embellishments of my impl. No luck.
So then I tried sending same data that I'm using for my tests to your code
above, and hit the internal error. Trying to narrow it down for your testcase,
I got the following reproducer (the only diff from your code is the loop to
write the batches):
{code:java}
import traceback
import pyarrow as pa
import pyarrow.flight as flight
class Server(flight.FlightServerBase):
def do_put(self, context, descriptor, reader, writer):
raise flight.FlightCancelledError("foo", extra_info=b"bar")
print("PyArrow version:", pa.__version__)
server = Server("grpc://localhost:0")
client = flight.connect(f"grpc://localhost:{server.port}")
schema = pa.schema([("a", pa.int64())])
writer, reader = client.do_put(flight.FlightDescriptor.for_command(b""), schema)
try:
# starts failing at this exact number; before that the error is still
cancelled
for i in range(27061):
# print(f"write {i}")
writer.write_batch(pa.record_batch([[1]], schema=schema))
writer.close()
except flight.FlightError as e:
traceback.print_exc()
print(e.extra_info)
except Exception:
traceback.print_exc() {code}
note: this may be related to timing or something, if the repro does not work,
try bumping the numbers :)
what strikes me (and smells like the actual root cause) is that the loop goes
on and does not fail after the first write. that looks awkward - could this be
the real problem?
also note, I originally reproduced this with the test data that I read from CSV
into a Table, and then tried doing writer.write_table().
if needed, I can provide the test data, it's a sheet generated from tpch model.
> [FlightRPC][Python] Regression in DoPut error handling
> ------------------------------------------------------
>
> Key: ARROW-16592
> URL: https://issues.apache.org/jira/browse/ARROW-16592
> Project: Apache Arrow
> Issue Type: Bug
> Reporter: Lubo Slivka
> Assignee: David Li
> Priority: Major
>
> In PyArrow 8.0.0, any error raised while handling DoPut on the server results
> in FlightInternalError on the client.
> In PyArrow 7.0.0, errors raised while handling DoPut are propagated/converted
> to non-internal errors.
> —
> Example: on 7.0.0, raising FlightCancelledError while handling DoPut on the
> server would propagate that error including extra_info all the way to the
> FlightClient. This is not the case anymore on 8.0.0.
> The FlightInternalError contains extra detail that is derived from the
> cancelled error though:
> {code:java}
> /arrow/cpp/src/arrow/flight/client.cc:363: Close() failed: IOError: <error
> message from FlightError is here>. Detail: Cancelled. gRPC client debug
> context: {"created":"@1652777650.446052211","description":"Error received
> from peer
> ipv4:127.0.0.1:16001","file":"/opt/vcpkg/buildtrees/grpc/src/85a295989c-6cf7bf442d.clean/src/core/lib/surface/call.cc","file_line":903,"grpc_message":"<error
> message from FlightError is here>. Detail: Cancelled","grpc_status":1}.
> Client context: OK. Detail: Cancelled
> {code}
> Note: skimming through the code, it seems this problem is not unique to
> PyArrow.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
