rzo1 opened a new issue, #2098:
URL: https://github.com/apache/stormcrawler/issues/2098
## What happens
`ManagedChannelUtil.createChannel()` builds every gRPC channel with
`usePlaintext()`. It is the only channel factory in the module: `Spout`,
`StatusUpdaterBolt` and `QueueRegulatorBolt` all go through it. There is no
configuration key for TLS, mutual TLS or channel credentials anywhere in
`external/urlfrontier`, so an operator who wants the link encrypted cannot get
it, even deliberately.
## Where
`external/urlfrontier/src/main/java/org/apache/stormcrawler/urlfrontier/ManagedChannelUtil.java:46-53`.
Related config keys: `urlfrontier.address`, `urlfrontier.host`,
`urlfrontier.port`.
```java
static ManagedChannel createChannel(@NotNull String address) {
...
return ManagedChannelBuilder.forTarget(address).usePlaintext().build();
}
```
## Why it matters
URLFrontier is typically a separate service, and the module supports several
`urlfrontier.address` entries for exactly that layout, so the channel usually
crosses a host boundary. Everything the crawler knows about a URL travels on
it: the URL itself and its metadata, which can include cookies when cookie
support is enabled. Anyone who can read that segment reads the crawl state, and
anyone who can write to it can change what the workers fetch, because per-URL
metadata influences request behaviour. Operators who need the link protected
today have to tunnel it themselves.
## Reproduction
No automated test. Demonstrating the transport would need a running frontier
service and a second gRPC endpoint with TLS, which is not something to put in a
unit test. Manual steps:
1. `grep -rn "usePlaintext\|TlsChannelCredentials"
external/urlfrontier/src/main` returns only line 52 of
`ManagedChannelUtil.java` and no TLS builder.
2. `grep -rn "urlfrontier\."
external/urlfrontier/src/main/java/org/apache/stormcrawler/urlfrontier/Constants.java`
lists every configuration key the module reads. None of them concerns
transport security.
3. Start a URLFrontier service configured to require TLS and point a
topology at it. The channel fails to connect, and no setting changes that.
## Suggested fix
Add configuration to `ManagedChannelUtil.createChannel` for TLS channel
credentials, built with `Grpc.newChannelBuilder(address,
TlsChannelCredentials...)`, with optional client certificate and trust roots.
Keep plaintext as the default for now so existing deployments keep working, and
log a warning at channel creation when plaintext is used. Flipping the default
to TLS is a breaking change and belongs in a major release, not in a patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
