bvolpato opened a new pull request, #41006:
URL: https://github.com/apache/beam/pull/41006
This PR makes `ServerFactory` listen on the requested host. Today,
`--job-host` changes only the URL that a job server reports.
`InetSocketAddressServerFactory.createServer` calls
`NettyServerBuilder.forPort(socket.getPort())`, which discards the host and
listens on all interfaces. So `--job-host=localhost` listens on all interfaces,
and the class Javadoc says that the server listens on "localhost".
`EpollSocket` in the same file and `ExpansionServer` already use
`forAddress`. This PR makes `InetSocketAddressServerFactory` do the same:
- `create` listens on the host in the service descriptor. This is the path
for a job server with a fixed port.
- `allocateAddressAndCreate` listens on loopback for the default URL
factory. The URL that it reports is already `localhost`, so a client on another
host cannot use that URL.
- `allocateAddressAndCreate` with a custom `UrlFactory` still listens on all
interfaces. The only production user is `DockerEnvironmentFactory.DockerOnMac`.
Its clients are in containers, and it can use ports that are published from a
job server container, so loopback would break it.
**Behavior change:** A job server that accepts remote connections or
published container ports without `--job-host` will now refuse those
connections. For example, `docker run -p 8099:8099` and a Kubernetes `Service`
are affected. These deployments must set `--job-host=0.0.0.0`. The documented
`docker run --net=host` commands connect through `localhost`, so they are not
affected. A `--job-host` value that does not resolve to a local address now
fails at startup. I added a `CHANGES.md` entry under Breaking Changes, and
reviewers may prefer a different rollout.
**Validated with JDK 21 on macOS:**
- The new `ServerFactoryTest` cases check the address of the listen socket.
Before the fix, `createListensOnRequestedHost` and
`allocatedAddressListensOnLoopback` fail. After the fix, all of the cases pass,
except the two Epoll cases that macOS skips.
- `:runners:java-fn-execution:test`, `:runners:java-job-service:test`, and
`FlinkJobServerDriverTest` on Flink 1.20 pass.
- I ran the Flink 1.20 job server with `--job-host=localhost` and an
embedded Flink cluster. A Python 2.77.0 `wordcount` pipeline with
`--environment_type=LOOPBACK` reached `DONE` and wrote its output. `lsof`
showed the job service, the artifact service, and all of the Fn API servers on
`127.0.0.1`. The only other listeners were the Flink BLOB and REST servers,
which this PR does not change.
**Not validated:** Docker environments on Linux or macOS, the Spark job
server, Kubernetes, and published container ports. The Linux Docker path uses
`--network=host`, so loopback should work, but I confirmed this only by reading
the code. I did not run the listener test with `--job-host=0.0.0.0` end to end,
but `createListensOnAllInterfacesWhenRequested` covers the same path in
`ServerFactory`.
<!--
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
- [x] Replace these instructions with a description of your change and why
you have made it.
- [ ] Mention the appropriate issue in your description (for example:
`addresses #123` or `fixes #456`).
- [x] Update `CHANGES.md` with noteworthy changes.
See [CONTRIBUTING.md]() for more details.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
