[
https://issues.apache.org/jira/browse/CASSANALYTICS-177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18110543#comment-18110543
]
Bianca Stanciu commented on CASSANALYTICS-177:
----------------------------------------------
[https://github.com/apache/cassandra-analytics/pull/222] is ready for another
look.
It attaches an optional instanceId query param to outbound Sidecar requests, so
Sidecar
can resolve the intended instance even when the request comes through the NLB
and the Host
header ends up being the NLB address instead of the node's (that's what causes
the 421).
The routing side doesn't need any Sidecar change: AbstractHandler already
prefers this param
over the Host header. What did need a Sidecar change is how the client learns
which id to
send per replica. That's CASSSIDECAR-495, which now reports each replica's
instance id in
the ring/token-range response. There's also a static fallback (host=id on a
contact point,
or the job-level sidecar.instance.id) for simpler setups and older Sidecar
versions.
Per-instance ids take precedence over the job-level one, so a multi-node
fan-out routes
correctly. And if a single job-level id would end up stamped onto multiple
instances (more
than one instance in the ring, at least one without its own id), it fails fast
instead of
misrouting. The new field is nullable and backward compatible both ways, and
there are unit
tests for all of this.
Scope here is the bulk writer path. One thing I'd like your take on: the read
path isn't
covered. CassandraDataLayer and SidecarTableSizeProvider still build their
Sidecar instances
without an id, so a read behind the NLB would hit the same 421. Does it make
sense to handle
that here, or would you rather a separate ticket for it?
cc: [~skoppu]
> Add instanceId query parameter to fix HTTP 421 errors when Sidecar is behind
> an AWS Load Balancer
> -------------------------------------------------------------------------------------------------
>
> Key: CASSANALYTICS-177
> URL: https://issues.apache.org/jira/browse/CASSANALYTICS-177
> Project: Apache Cassandra Analytics
> Issue Type: Improvement
> Components: Writer
> Reporter: Bianca Stanciu
> Assignee: Bianca Stanciu
> Priority: High
> Time Spent: 10m
> Remaining Estimate: 0h
>
> +_*Problem:*_+
> When Sidecar sits behind an AWS NLB, every HTTP request from the analytics
> client returns *421 Misdirected Request.*
> The NLB forwards TCP without modifying HTTP headers, so the Host header
> arriving at Sidecar contains the NLB's IP, not the Sidecar's own address.
> Sidecar uses Host header to identify which Cassandra instance the request is
> for.
> _*Solution:*_
> Add an optional instanceId query parameter appended to every outbound Sidecar
> request:
> {code:java}
> POST /api/v1/restore-job/slices?instanceId=1
> GET /api/v1/cassandra/timeSkew?instanceId=1{code}
> Sidecar already checks +_*?instanceId*_+ before the Host header, bypassing
> the NLB IP lookup entirely. No Sidecar-side changes needed.
> Enabled with one Spark conf flag:
> {code:java}
> --conf "spark.cassandra_analytics.sidecar.instance.id=1" {code}
> Default is null, no parameter added, zero behavior change for the existing
> flow.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
