I was worried this might be the case.
The rest.port handling was simply copied from the legacy web-server,
which explicitly allowed shutting it down.
It may (I'm not entirely sure) also not be necessary for all deployment
modes; for example if the job is baked into the job/taskmanager images.
I'm not quite sure whether the rpc address is actually required for the
REST job submission, or only since we still rely partly on some legacy
code (ClusterClient). Maybe Till (cc) knows the answer to that.
Adding on to this point you made - " the rpc address is still *required *due
to some technical implementations; it may be that you can set this to some
arbitrary value however."
For job submission to happen successfully we should give specific rpc
address and not any arbitrary value. If any arbitrary value is given the
job submission fails with the following error -
org.apache.flink.client.deployment.ClusterRetrieveException: Couldn't
retrieve standalone cluster
at
org.apache.flink.client.deployment.StandaloneClusterDescriptor.retrieve(StandaloneClusterDescriptor.java:51)
at
org.apache.flink.client.deployment.StandaloneClusterDescriptor.retrieve(StandaloneClusterDescriptor.java:31)
at
org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:249)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:210)
at
org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1020)
at
org.apache.flink.client.cli.CliFrontend.lambda$main$9(CliFrontend.java:1096)
at
org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
at
org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1096)
Caused by: java.net.UnknownHostException: flinktest-flink-jobmanager1233445:
Name or service not known
(Random name flinktest-flink-jobmanager1233445)
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
at
java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
at java.net.InetAddress.getAllByName(InetAddress.java:1192)
at java.net.InetAddress.getAllByName(InetAddress.java:1126)
at java.net.InetAddress.getByName(InetAddress.java:1076)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils.getRpcUrl(AkkaRpcServiceUtils.java:171)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils.getRpcUrl(AkkaRpcServiceUtils.java:136)
at
org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:83)
at
org.apache.flink.client.program.ClusterClient.<init>(ClusterClient.java:158)
at
org.apache.flink.client.program.rest.RestClusterClient.<init>(RestClusterClient.java:184)
at
org.apache.flink.client.program.rest.RestClusterClient.<init>(RestClusterClient.java:157)
at
org.apache.flink.client.deployment.StandaloneClusterDescriptor.retrieve(StandaloneClusterDescriptor.java:49)
... 7 more
On Wed, Jun 20, 2018 at 11:18 AM, Sampath Bhat <sam414255p...@gmail.com>
wrote:
Hi Chesnay
If REST API (i.e. the web server) is mandatory for submitting jobs then
why is there an option to set rest.port to -1? I think it should be
mandatory to set some valid port for rest.port and make sure flink job
manager does not come up if valid port is not set for rest.port? Or else
there must be some way to submit jobs even if REST API (i.e. the web
server) is not instantiated.
If jobmanger.rpc.address is not required for flink client then why is it
still looking for that property in flink-conf.yaml? Isn't it not a bug?
Because if we comment out the jobmanger.rpc.address and jobmanger.rpc.port
then flink client will not be able to submit the job.
On Tue, Jun 19, 2018 at 5:49 PM, Chesnay Schepler <ches...@apache.org>
wrote:
In 1.5 we reworked the job-submission to go through the REST API instead
of akka.
I believe the jobmanager rpc port shouldn't be necessary anymore, the rpc
address is still *required *due to some technical implementations; it
may be that you can set this to some arbitrary value however.
As a result the REST API (i.e. the web server) must be running in order
to submit jobs.
On 19.06.2018 14:12, Sampath Bhat wrote:
Hello
I'm using Flink 1.5.0 version and Flink CLI to submit jar to flink
cluster.
In flink 1.4.2 only job manager rpc address and job manager rpc port were
sufficient for flink client to connect to job manager and submit the job.
But in flink 1.5.0 the flink client additionally requires the
rest.address and rest.port for submitting the job to job manager. What is
the advantage of this new method over the 1.4.2 method of submitting job?
Moreover if we make rest.port = -1 the web server will not be
instantiated then how should we submit the job?
Regards
Sampath
