Sampreeth Reddy B created CASSSIDECAR-509:
---------------------------------------------
Summary: Live-migration download hangs indefinitely when source
returns a non-200 for an invalid file path
Key: CASSSIDECAR-509
URL: https://issues.apache.org/jira/browse/CASSSIDECAR-509
Project: Sidecar for Apache Cassandra
Issue Type: Bug
Components: Rest API
Reporter: Sampreeth Reddy B
*Problem*
**
When a live-migration client issues a {{GET
/api/v1/cassandra/live-migration/\{dir-type}/\{dir-index}/files/...}} against
the sidecar and the server-side path-validation code throws an unexpected
exception, the HTTP response is never closed on the server side. The client's
underlying Vert.x {{HttpRequest.send()}} keeps the connection open and waits
for a body it will never receive, blocking the live-migration task's
thread-pool slot until the client-side {{HttpClientConfig.timeoutMillis()}}
fires (default: operation-wide timeout, often an hour).
The hang is caused by a missing {{.onFailure(...)}} handler on the
{{executorPools.service().executeBlocking(...)}} chain in
[{{LiveMigrationFileResolveHandler.handleInternal(...)}}|https://github.com/apache/cassandra-sidecar/blob/trunk/server/src/main/java/org/apache/cassandra/sidecar/handlers/livemigration/LiveMigrationFileResolveHandler.java].
*Steps to Reproduce*
# Start a sidecar (release 0.4.0 or main) and point {{data_dirs}} at any valid
Cassandra data directory.
# Issue a {{GET}} for an obviously-invalid path through either the Vert.x
client or {{{}curl{}}}:
{{curl -v
http://<sidecar-host>:9043/api/v1/cassandra/live-migration/instance/1/files/this_path_does_not_exist.db}}The
server correctly returns {{HTTP/1.1 404 Not Found}} .
# Repeat the request via the Vert.x client:
{{sidecarClient.liveMigrationStreamFileAsync(instance,
"this_path_does_not_exist.db", "/tmp/dest.db")
.get() this never completes}}
# Observed: the {{CompletableFuture}} returned by
{{liveMigrationStreamFileAsync}} never completes (success or failure). A thread
dump shows the calling thread parked on {{{}CompletableFuture.get{}}}.
{*}RCA{*}:
LiveMigrationFileResolveHandler does not have an {{.onFailure(...)}} branch
chained to the {{executeBlocking(...)}} future. Vert.x's documented contract is
that unhandled Future failures bubble up through {{UndeliverableException}} and
are logged, but the {{RoutingContext}} is not auto-failed by the framework the
handler is responsible for calling {{rc.fail(cause)}} or ending the response.
h4. Proposed Fix
chain {{.onFailure(...)}} that calls {{rc.fail(500, cause)}} (or
{{rc.response().setStatusCode(500).end(...)}} with a short JSON body, using the
project's existing error-wrapping convention
{{{}HttpExceptions.wrapHttpException(...){}}}):
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
