[
https://issues.apache.org/jira/browse/BEAM-12814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17476347#comment-17476347
]
Kenneth Knowles commented on BEAM-12814:
----------------------------------------
If I understand the situation correctly, the {{EXTERNAL}} environment indicates
that some other party has spun up the SDK harness, and the worker should just
communicate with the provided address/port? Is that correct? Pinging
[~chamikara] who was on that thread and I think [~ibzib] knows something about
Flink on k8s (or am I getting mixed up?)
Is this issue specific to cross-language transforms, or is it something missing
in Java in general, but available for Python? I know that sometimes there is a
need for the runner to start/stop SDK harnesses and manage them directly using
the "container contract", which would seem unavailable in this configuration.
> Support External Environment For Java SDK Harness
> -------------------------------------------------
>
> Key: BEAM-12814
> URL: https://issues.apache.org/jira/browse/BEAM-12814
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-harness
> Reporter: Jeremy Lewi
> Priority: P2
> Labels: stale-P2
>
> As mentioned in this
> [thread|https://lists.apache.org/x/thread.html/r3cf56464c157da840f46e777c1b3f307f65ec2cf6b2278728f2cd1e1@%3Cuser.beam.apache.org%3E]
> the Java SDK harness only supports environments docker and embedded for the
> Java SDK harness.
> It would be nice if the Java SDK also supported external environments similar
> to what the Python SDK harness does.
> The use case would be running in Kubernetes. On Kubernetes I think its better
> to run the SDK harness containers as side cars of the taskmanager pods rather
> than relying on docker in docker. Docker in docker isn't great from a
> security standpoint because it requires privileged containers. It also makes
> it harder to debug and operate on Kubernetes.
>
> This was also mentioned in this thread on the developer mailing list;
> [thread|https://lists.apache.org/thread.html/r43fbff58c0d11b05a74617b175ed2d0565b9e65717e92f1dd908f4fa%40%3Cdev.beam.apache.org%3E]
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
