[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718562#comment-16718562
]
ASF GitHub Bot commented on FLINK-11081:
yanghua commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240905696
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
##
@@ -181,14 +184,44 @@ protected void initChannel(SocketChannel ch) {
.channel(NioServerSocketChannel.class)
.childHandler(initializer);
- log.debug("Binding rest endpoint to {}:{}.",
restBindAddress, restBindPort);
- final ChannelFuture channel;
- if (restBindAddress == null) {
- channel = bootstrap.bind(restBindPort);
- } else {
- channel = bootstrap.bind(restBindAddress,
restBindPort);
+ ChannelFuture channel;
+
+ // parse port range definition and create port iterator
+ Iterator portsIterator;
+ try {
+ portsIterator =
NetUtils.getPortRangeFromString(restBindPort);
+ } catch (Exception e) {
Review comment:
OK, thank you for your clarification. It seems that I misunderstood what you
mean. I thought you meant to capture Exception and could not catch
IllegalConfigurationException. Now it seems that what you mean is that we
should break down the capture of a particular exception. Considering that
`NetUtils.getPortRangeFromString` will throw at least two possible exceptions,
one checked and one unchecked. I think we capture Exception directly.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718528#comment-16718528
]
ASF GitHub Bot commented on FLINK-11081:
walterddr commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240900279
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
##
@@ -181,14 +184,44 @@ protected void initChannel(SocketChannel ch) {
.channel(NioServerSocketChannel.class)
.childHandler(initializer);
- log.debug("Binding rest endpoint to {}:{}.",
restBindAddress, restBindPort);
- final ChannelFuture channel;
- if (restBindAddress == null) {
- channel = bootstrap.bind(restBindPort);
- } else {
- channel = bootstrap.bind(restBindAddress,
restBindPort);
+ ChannelFuture channel;
+
+ // parse port range definition and create port iterator
+ Iterator portsIterator;
+ try {
+ portsIterator =
NetUtils.getPortRangeFromString(restBindPort);
+ } catch (Exception e) {
Review comment:
What I meant is "should we catch `RuntimeExceptions`". function signature of
`startActorSystem` explicitly throws a generic `Exception` so there's no way to
distinguish in `BootstrapTool` use case. But here since we are directly using
`NetUtil.getPor...`, we might be able to do a better job.
Regarding whether we should catch it. I am fine with either decision as long
as they are properly documented.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718466#comment-16718466
]
ASF GitHub Bot commented on FLINK-11081:
yanghua commented on issue #7263: [FLINK-11081] Support binding port range for
REST server
URL: https://github.com/apache/flink/pull/7263#issuecomment-446460550
`ConfigOption.PORT` ? I can not find it.
I guess what you want to express is `RestOptions.PORT`. I think that it
can't be deprecated here. Initially its responsibilities are both the binding
port of the server and the connection port of the client. Now the server's
binding port is represented by `BIND_PORT`, then it can also be used to
represent the client's connection port. In some places it may be used:
```
Private void createClientConfiguration(URI restAddress) {
Configuration restClientConfig = new Configuration();
restClientConfig.setString(JobManagerOptions.ADDRESS, restAddress.getHost());
restClientConfig.setInteger(RestOptions.PORT, restAddress.getPort());
this.restClusterClientConfig = new
UnmodifiableConfiguration(restClientConfig);
}
```
There should be two pairs of configuration items `BIND_ADDRESS/BIND_PORT`
and `ADDRESS/PORT` here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718467#comment-16718467
]
ASF GitHub Bot commented on FLINK-11081:
yanghua edited a comment on issue #7263: [FLINK-11081] Support binding port
range for REST server
URL: https://github.com/apache/flink/pull/7263#issuecomment-446460550
`ConfigOption.PORT` ? I can not find it.
I guess what you want to express is `RestOptions.PORT`. I think that it
can't be deprecated here. Initially its responsibilities are both the binding
port of the server and the connection port of the client. Now the server's
binding port is represented by `BIND_PORT`, then it can also be used to
represent the client's connection port. In some places it may be used:
```java
Private void createClientConfiguration(URI restAddress) {
Configuration restClientConfig = new Configuration();
restClientConfig.setString(JobManagerOptions.ADDRESS,
restAddress.getHost());
restClientConfig.setInteger(RestOptions.PORT, restAddress.getPort());
this.restClusterClientConfig = new
UnmodifiableConfiguration(restClientConfig);
}
```
There should be two pairs of configuration items `BIND_ADDRESS/BIND_PORT`
and `ADDRESS/PORT` here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718464#comment-16718464
]
ASF GitHub Bot commented on FLINK-11081:
yanghua commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240880017
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
##
@@ -181,14 +184,44 @@ protected void initChannel(SocketChannel ch) {
.channel(NioServerSocketChannel.class)
.childHandler(initializer);
- log.debug("Binding rest endpoint to {}:{}.",
restBindAddress, restBindPort);
- final ChannelFuture channel;
- if (restBindAddress == null) {
- channel = bootstrap.bind(restBindPort);
- } else {
- channel = bootstrap.bind(restBindAddress,
restBindPort);
+ ChannelFuture channel;
+
+ // parse port range definition and create port iterator
+ Iterator portsIterator;
+ try {
+ portsIterator =
NetUtils.getPortRangeFromString(restBindPort);
+ } catch (Exception e) {
+ throw new IllegalArgumentException("Invalid
port range definition: " + restBindPort);
+ }
+
+ int chosenPort = 0;
+ while (portsIterator.hasNext()) {
+ try {
+ chosenPort = portsIterator.next();
+ if (restBindAddress == null) {
+ channel =
bootstrap.bind(chosenPort);
+ } else {
+ channel =
bootstrap.bind(restBindAddress, chosenPort);
+ }
+ serverChannel =
channel.syncUninterruptibly().channel();
+ break;
+ } catch (Exception e) {
+ // we can continue to try if this
contains a netty channel exception
+ Throwable cause = e.getCause();
+ if (!(cause instanceof
org.jboss.netty.channel.ChannelException ||
+ cause instanceof
java.net.BindException)) {
+ throw e;
Review comment:
Yes, this expression may be misunderstood, I will update it later. Regarding
the exception handling logic here, in the recent test, it is really a problem,
we should not call it :
```
Throwable cause = e.getCause();
```
It will get a `null` value. I will fix it later.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718463#comment-16718463
]
ASF GitHub Bot commented on FLINK-11081:
yanghua commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240880017
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
##
@@ -181,14 +184,44 @@ protected void initChannel(SocketChannel ch) {
.channel(NioServerSocketChannel.class)
.childHandler(initializer);
- log.debug("Binding rest endpoint to {}:{}.",
restBindAddress, restBindPort);
- final ChannelFuture channel;
- if (restBindAddress == null) {
- channel = bootstrap.bind(restBindPort);
- } else {
- channel = bootstrap.bind(restBindAddress,
restBindPort);
+ ChannelFuture channel;
+
+ // parse port range definition and create port iterator
+ Iterator portsIterator;
+ try {
+ portsIterator =
NetUtils.getPortRangeFromString(restBindPort);
+ } catch (Exception e) {
+ throw new IllegalArgumentException("Invalid
port range definition: " + restBindPort);
+ }
+
+ int chosenPort = 0;
+ while (portsIterator.hasNext()) {
+ try {
+ chosenPort = portsIterator.next();
+ if (restBindAddress == null) {
+ channel =
bootstrap.bind(chosenPort);
+ } else {
+ channel =
bootstrap.bind(restBindAddress, chosenPort);
+ }
+ serverChannel =
channel.syncUninterruptibly().channel();
+ break;
+ } catch (Exception e) {
+ // we can continue to try if this
contains a netty channel exception
+ Throwable cause = e.getCause();
+ if (!(cause instanceof
org.jboss.netty.channel.ChannelException ||
+ cause instanceof
java.net.BindException)) {
+ throw e;
Review comment:
Yes, this expression may be misunderstood, I will update it later. Regarding
the exception handling logic here, in the recent test, it is really a problem,
we should not call it again.
```
Throwable cause = e.getCause();
```
It will get a `null` value. I will fix it later.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718462#comment-16718462
]
ASF GitHub Bot commented on FLINK-11081:
yanghua commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240880004
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
##
@@ -181,14 +184,44 @@ protected void initChannel(SocketChannel ch) {
.channel(NioServerSocketChannel.class)
.childHandler(initializer);
- log.debug("Binding rest endpoint to {}:{}.",
restBindAddress, restBindPort);
- final ChannelFuture channel;
- if (restBindAddress == null) {
- channel = bootstrap.bind(restBindPort);
- } else {
- channel = bootstrap.bind(restBindAddress,
restBindPort);
+ ChannelFuture channel;
+
+ // parse port range definition and create port iterator
+ Iterator portsIterator;
+ try {
+ portsIterator =
NetUtils.getPortRangeFromString(restBindPort);
+ } catch (Exception e) {
Review comment:
Here, we catch `Exception` so that regardless of `NumberFormatException` or
`IllegalConfigurationException` will be caught, isn't it? Or did I
misunderstand what you mean? Do you want me to catch a more specific exception?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718456#comment-16718456
]
ASF GitHub Bot commented on FLINK-11081:
walterddr commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240877630
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
##
@@ -181,14 +184,44 @@ protected void initChannel(SocketChannel ch) {
.channel(NioServerSocketChannel.class)
.childHandler(initializer);
- log.debug("Binding rest endpoint to {}:{}.",
restBindAddress, restBindPort);
- final ChannelFuture channel;
- if (restBindAddress == null) {
- channel = bootstrap.bind(restBindPort);
- } else {
- channel = bootstrap.bind(restBindAddress,
restBindPort);
+ ChannelFuture channel;
+
+ // parse port range definition and create port iterator
+ Iterator portsIterator;
+ try {
+ portsIterator =
NetUtils.getPortRangeFromString(restBindPort);
+ } catch (Exception e) {
Review comment:
hmm. `IllegalConfigurationException` extends `RuntimeException`, shouldn't
we not catch that? I am wondering reason `BootstrapTool` handles it that way
was because the static `startActorSystem` method throws general exception in
method signature.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718455#comment-16718455
]
ASF GitHub Bot commented on FLINK-11081:
walterddr commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240877872
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
##
@@ -181,14 +184,44 @@ protected void initChannel(SocketChannel ch) {
.channel(NioServerSocketChannel.class)
.childHandler(initializer);
- log.debug("Binding rest endpoint to {}:{}.",
restBindAddress, restBindPort);
- final ChannelFuture channel;
- if (restBindAddress == null) {
- channel = bootstrap.bind(restBindPort);
- } else {
- channel = bootstrap.bind(restBindAddress,
restBindPort);
+ ChannelFuture channel;
+
+ // parse port range definition and create port iterator
+ Iterator portsIterator;
+ try {
+ portsIterator =
NetUtils.getPortRangeFromString(restBindPort);
+ } catch (Exception e) {
+ throw new IllegalArgumentException("Invalid
port range definition: " + restBindPort);
+ }
+
+ int chosenPort = 0;
+ while (portsIterator.hasNext()) {
+ try {
+ chosenPort = portsIterator.next();
+ if (restBindAddress == null) {
+ channel =
bootstrap.bind(chosenPort);
+ } else {
+ channel =
bootstrap.bind(restBindAddress, chosenPort);
+ }
+ serverChannel =
channel.syncUninterruptibly().channel();
+ break;
+ } catch (Exception e) {
+ // we can continue to try if this
contains a netty channel exception
+ Throwable cause = e.getCause();
+ if (!(cause instanceof
org.jboss.netty.channel.ChannelException ||
+ cause instanceof
java.net.BindException)) {
+ throw e;
Review comment:
you are right. yeah I think both place the comment is a bit misleading. My
take from the comment is: "we can continue to retry the same port within this
while loop" instead of the actual meaning: "we 'could allow' users to continue
try when we throw an exception to the user"
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718372#comment-16718372
]
ASF GitHub Bot commented on FLINK-11081:
yanghua commented on issue #7263: [FLINK-11081] Support binding port range for
REST server
URL: https://github.com/apache/flink/pull/7263#issuecomment-446434964
hi @walterddr Thank you for your review. I responded to them and I think
some suggestions make sense. In addition, rest.port will not be deprecated,
after which it will only be used by the rest client, which makes its
responsibilities clearer (it was originally used by both the server and the
client), see the description of this issue.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718363#comment-16718363
]
ASF GitHub Bot commented on FLINK-11081:
yanghua commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240860330
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/rest/RestServerEndpointConfigurationTest.java
##
@@ -49,17 +77,154 @@ public void testBasicMapping() throws
ConfigurationException {
Configuration originalConfig = new Configuration();
originalConfig.setString(RestOptions.ADDRESS, ADDRESS);
originalConfig.setString(RestOptions.BIND_ADDRESS,
BIND_ADDRESS);
- originalConfig.setInteger(RestOptions.PORT, PORT);
+ originalConfig.setString(RestOptions.BIND_PORT, BIND_PORT);
originalConfig.setInteger(RestOptions.SERVER_MAX_CONTENT_LENGTH,
CONTENT_LENGTH);
originalConfig.setString(WebOptions.TMP_DIR,
temporaryFolder.getRoot().getAbsolutePath());
final RestServerEndpointConfiguration result =
RestServerEndpointConfiguration.fromConfiguration(originalConfig);
Assert.assertEquals(ADDRESS, result.getRestAddress());
Assert.assertEquals(BIND_ADDRESS, result.getRestBindAddress());
- Assert.assertEquals(PORT, result.getRestBindPort());
+ Assert.assertEquals(BIND_PORT, result.getRestBindPort());
Assert.assertEquals(CONTENT_LENGTH,
result.getMaxContentLength());
Assert.assertThat(
result.getUploadDir().toAbsolutePath().toString(),
containsString(temporaryFolder.getRoot().getAbsolutePath()));
}
+
Review comment:
OK
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718361#comment-16718361
]
ASF GitHub Bot commented on FLINK-11081:
yanghua commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240860279
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
##
@@ -181,14 +184,44 @@ protected void initChannel(SocketChannel ch) {
.channel(NioServerSocketChannel.class)
.childHandler(initializer);
- log.debug("Binding rest endpoint to {}:{}.",
restBindAddress, restBindPort);
- final ChannelFuture channel;
- if (restBindAddress == null) {
- channel = bootstrap.bind(restBindPort);
- } else {
- channel = bootstrap.bind(restBindAddress,
restBindPort);
+ ChannelFuture channel;
+
+ // parse port range definition and create port iterator
+ Iterator portsIterator;
+ try {
+ portsIterator =
NetUtils.getPortRangeFromString(restBindPort);
+ } catch (Exception e) {
+ throw new IllegalArgumentException("Invalid
port range definition: " + restBindPort);
+ }
+
+ int chosenPort = 0;
+ while (portsIterator.hasNext()) {
+ try {
+ chosenPort = portsIterator.next();
+ if (restBindAddress == null) {
+ channel =
bootstrap.bind(chosenPort);
+ } else {
+ channel =
bootstrap.bind(restBindAddress, chosenPort);
+ }
+ serverChannel =
channel.syncUninterruptibly().channel();
+ break;
+ } catch (Exception e) {
Review comment:
Will add a test to let the first port conflict and choose the second.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718357#comment-16718357
]
ASF GitHub Bot commented on FLINK-11081:
yanghua commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240859949
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
##
@@ -181,14 +184,44 @@ protected void initChannel(SocketChannel ch) {
.channel(NioServerSocketChannel.class)
.childHandler(initializer);
- log.debug("Binding rest endpoint to {}:{}.",
restBindAddress, restBindPort);
- final ChannelFuture channel;
- if (restBindAddress == null) {
- channel = bootstrap.bind(restBindPort);
- } else {
- channel = bootstrap.bind(restBindAddress,
restBindPort);
+ ChannelFuture channel;
+
+ // parse port range definition and create port iterator
+ Iterator portsIterator;
+ try {
+ portsIterator =
NetUtils.getPortRangeFromString(restBindPort);
+ } catch (Exception e) {
+ throw new IllegalArgumentException("Invalid
port range definition: " + restBindPort);
+ }
+
+ int chosenPort = 0;
+ while (portsIterator.hasNext()) {
+ try {
+ chosenPort = portsIterator.next();
+ if (restBindAddress == null) {
+ channel =
bootstrap.bind(chosenPort);
+ } else {
+ channel =
bootstrap.bind(restBindAddress, chosenPort);
+ }
+ serverChannel =
channel.syncUninterruptibly().channel();
+ break;
+ } catch (Exception e) {
+ // we can continue to try if this
contains a netty channel exception
+ Throwable cause = e.getCause();
+ if (!(cause instanceof
org.jboss.netty.channel.ChannelException ||
+ cause instanceof
java.net.BindException)) {
+ throw e;
Review comment:
Here, we just retry when the exception is a port conflict specific
exception. If not, it seems we should throw it and let the caller know there is
another exception. Also, see `BootstrapTools#startActorSystem`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718350#comment-16718350
]
ASF GitHub Bot commented on FLINK-11081:
yanghua commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240857598
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
##
@@ -181,14 +184,44 @@ protected void initChannel(SocketChannel ch) {
.channel(NioServerSocketChannel.class)
.childHandler(initializer);
- log.debug("Binding rest endpoint to {}:{}.",
restBindAddress, restBindPort);
- final ChannelFuture channel;
- if (restBindAddress == null) {
- channel = bootstrap.bind(restBindPort);
- } else {
- channel = bootstrap.bind(restBindAddress,
restBindPort);
+ ChannelFuture channel;
+
+ // parse port range definition and create port iterator
+ Iterator portsIterator;
+ try {
+ portsIterator =
NetUtils.getPortRangeFromString(restBindPort);
+ } catch (Exception e) {
Review comment:
`getPortRangeFromString ` also throws `IllegalConfigurationException` which
is a unchecked exception defined in `flink-core`, so here we catch a general
`Exception` and in the catch block then rethrow a unified
`IllegalArgumentException `. This code style used in
`BootstrapTools#startActorSystem`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717605#comment-16717605
]
ASF GitHub Bot commented on FLINK-11081:
walterddr commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240704732
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
##
@@ -181,14 +184,44 @@ protected void initChannel(SocketChannel ch) {
.channel(NioServerSocketChannel.class)
.childHandler(initializer);
- log.debug("Binding rest endpoint to {}:{}.",
restBindAddress, restBindPort);
- final ChannelFuture channel;
- if (restBindAddress == null) {
- channel = bootstrap.bind(restBindPort);
- } else {
- channel = bootstrap.bind(restBindAddress,
restBindPort);
+ ChannelFuture channel;
+
+ // parse port range definition and create port iterator
+ Iterator portsIterator;
+ try {
+ portsIterator =
NetUtils.getPortRangeFromString(restBindPort);
+ } catch (Exception e) {
+ throw new IllegalArgumentException("Invalid
port range definition: " + restBindPort);
+ }
+
+ int chosenPort = 0;
+ while (portsIterator.hasNext()) {
+ try {
+ chosenPort = portsIterator.next();
+ if (restBindAddress == null) {
+ channel =
bootstrap.bind(chosenPort);
+ } else {
+ channel =
bootstrap.bind(restBindAddress, chosenPort);
+ }
+ serverChannel =
channel.syncUninterruptibly().channel();
+ break;
+ } catch (Exception e) {
Review comment:
I might be overlooking but is this exception code path tested?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717608#comment-16717608
]
ASF GitHub Bot commented on FLINK-11081:
walterddr commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240711303
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/rest/RestServerEndpointConfigurationTest.java
##
@@ -49,17 +77,154 @@ public void testBasicMapping() throws
ConfigurationException {
Configuration originalConfig = new Configuration();
originalConfig.setString(RestOptions.ADDRESS, ADDRESS);
originalConfig.setString(RestOptions.BIND_ADDRESS,
BIND_ADDRESS);
- originalConfig.setInteger(RestOptions.PORT, PORT);
+ originalConfig.setString(RestOptions.BIND_PORT, BIND_PORT);
originalConfig.setInteger(RestOptions.SERVER_MAX_CONTENT_LENGTH,
CONTENT_LENGTH);
originalConfig.setString(WebOptions.TMP_DIR,
temporaryFolder.getRoot().getAbsolutePath());
final RestServerEndpointConfiguration result =
RestServerEndpointConfiguration.fromConfiguration(originalConfig);
Assert.assertEquals(ADDRESS, result.getRestAddress());
Assert.assertEquals(BIND_ADDRESS, result.getRestBindAddress());
- Assert.assertEquals(PORT, result.getRestBindPort());
+ Assert.assertEquals(BIND_PORT, result.getRestBindPort());
Assert.assertEquals(CONTENT_LENGTH,
result.getMaxContentLength());
Assert.assertThat(
result.getUploadDir().toAbsolutePath().toString(),
containsString(temporaryFolder.getRoot().getAbsolutePath()));
}
+
+ @Test
+ public void testDefaultRestServerBindPort() throws Exception {
+ testRestServerBindPort(null, "8081");
+ }
+
+ @Test
+ public void testRestServerSpecificBindPort() throws Exception {
+ testRestServerBindPort("12345", "12345");
+ }
+
+ @Test
+ public void testRestServerSpecificBindPortRange() throws Exception {
+ testRestServerBindPort("12345,12346", "12345");
+ }
+
+ @Test
+ public void testRestServerBindPortRange() throws Exception {
+ testRestServerBindPort("12345-12350", "12345");
+ }
+
+ @Test
+ public void testRestServerBindPortAndConnectionPortSuccess() throws
Exception {
+ testRestServerBindPortAndConnectionPort("12345,12346", 12345);
+ }
+
+ @Test
+ public void testRestServerBindPortAndConnectionPortFailed() throws
Exception {
+ try {
+ testRestServerBindPortAndConnectionPort("12345-12346",
12347);
+ } catch (Exception e) {
+ assertTrue(e.getMessage().contains("Connection refused")
+ && e.getMessage().contains("12347"));
+ }
+ }
+
+ private void testRestServerBindPortAndConnectionPort(String bindPort,
int connectPort) throws Exception {
+ RestServerEndpoint serverEndpoint = null;
+ RestClient restClient = null;
+
+ try {
+ final Configuration config = new Configuration();
+ config.setString(RestOptions.ADDRESS, "localhost");
+ config.setString(RestOptions.BIND_PORT, bindPort);
+ config.setInteger(RestOptions.PORT, connectPort);
+
+ RestServerEndpointITCase.TestHandler testHandler;
+
+ RestServerEndpointConfiguration serverConfig =
RestServerEndpointConfiguration.fromConfiguration(config);
+ RestClientConfiguration clientConfig =
RestClientConfiguration.fromConfiguration(config);
+
+ final String restAddress = "http://localhost:1234;;
+ RestfulGateway mockRestfulGateway =
mock(RestfulGateway.class);
+
when(mockRestfulGateway.requestRestAddress(any(Time.class))).thenReturn(CompletableFuture.completedFuture(restAddress));
+
+ final GatewayRetriever
mockGatewayRetriever = () ->
+
CompletableFuture.completedFuture(mockRestfulGateway);
+
+ testHandler = new RestServerEndpointITCase.TestHandler(
+ CompletableFuture.completedFuture(restAddress),
+ mockGatewayRetriever,
+ RpcUtils.INF_TIMEOUT);
+
+ RestServerEndpointITCase.TestVersionHandler
testVersionHandler = new RestServerEndpointITCase.TestVersionHandler(
+
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717607#comment-16717607
]
ASF GitHub Bot commented on FLINK-11081:
walterddr commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240706051
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
##
@@ -181,14 +184,44 @@ protected void initChannel(SocketChannel ch) {
.channel(NioServerSocketChannel.class)
.childHandler(initializer);
- log.debug("Binding rest endpoint to {}:{}.",
restBindAddress, restBindPort);
- final ChannelFuture channel;
- if (restBindAddress == null) {
- channel = bootstrap.bind(restBindPort);
- } else {
- channel = bootstrap.bind(restBindAddress,
restBindPort);
+ ChannelFuture channel;
+
+ // parse port range definition and create port iterator
+ Iterator portsIterator;
+ try {
+ portsIterator =
NetUtils.getPortRangeFromString(restBindPort);
+ } catch (Exception e) {
+ throw new IllegalArgumentException("Invalid
port range definition: " + restBindPort);
+ }
+
+ int chosenPort = 0;
+ while (portsIterator.hasNext()) {
+ try {
+ chosenPort = portsIterator.next();
+ if (restBindAddress == null) {
+ channel =
bootstrap.bind(chosenPort);
+ } else {
+ channel =
bootstrap.bind(restBindAddress, chosenPort);
+ }
+ serverChannel =
channel.syncUninterruptibly().channel();
+ break;
+ } catch (Exception e) {
+ // we can continue to try if this
contains a netty channel exception
+ Throwable cause = e.getCause();
+ if (!(cause instanceof
org.jboss.netty.channel.ChannelException ||
+ cause instanceof
java.net.BindException)) {
+ throw e;
Review comment:
doesn't this actually throws the exception to top level instead of "continue
to try"? also since
`chosenPort = portsIterator.next();` always gets call, how can retry over
the same port here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717610#comment-16717610
]
ASF GitHub Bot commented on FLINK-11081:
walterddr commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240711143
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/rest/RestServerEndpointConfigurationTest.java
##
@@ -49,17 +77,154 @@ public void testBasicMapping() throws
ConfigurationException {
Configuration originalConfig = new Configuration();
originalConfig.setString(RestOptions.ADDRESS, ADDRESS);
originalConfig.setString(RestOptions.BIND_ADDRESS,
BIND_ADDRESS);
- originalConfig.setInteger(RestOptions.PORT, PORT);
+ originalConfig.setString(RestOptions.BIND_PORT, BIND_PORT);
originalConfig.setInteger(RestOptions.SERVER_MAX_CONTENT_LENGTH,
CONTENT_LENGTH);
originalConfig.setString(WebOptions.TMP_DIR,
temporaryFolder.getRoot().getAbsolutePath());
final RestServerEndpointConfiguration result =
RestServerEndpointConfiguration.fromConfiguration(originalConfig);
Assert.assertEquals(ADDRESS, result.getRestAddress());
Assert.assertEquals(BIND_ADDRESS, result.getRestBindAddress());
- Assert.assertEquals(PORT, result.getRestBindPort());
+ Assert.assertEquals(BIND_PORT, result.getRestBindPort());
Assert.assertEquals(CONTENT_LENGTH,
result.getMaxContentLength());
Assert.assertThat(
result.getUploadDir().toAbsolutePath().toString(),
containsString(temporaryFolder.getRoot().getAbsolutePath()));
}
+
+ @Test
+ public void testDefaultRestServerBindPort() throws Exception {
+ testRestServerBindPort(null, "8081");
+ }
+
+ @Test
+ public void testRestServerSpecificBindPort() throws Exception {
+ testRestServerBindPort("12345", "12345");
+ }
+
+ @Test
+ public void testRestServerSpecificBindPortRange() throws Exception {
+ testRestServerBindPort("12345,12346", "12345");
+ }
+
+ @Test
+ public void testRestServerBindPortRange() throws Exception {
+ testRestServerBindPort("12345-12350", "12345");
+ }
+
+ @Test
+ public void testRestServerBindPortAndConnectionPortSuccess() throws
Exception {
+ testRestServerBindPortAndConnectionPort("12345,12346", 12345);
+ }
+
+ @Test
+ public void testRestServerBindPortAndConnectionPortFailed() throws
Exception {
+ try {
+ testRestServerBindPortAndConnectionPort("12345-12346",
12347);
+ } catch (Exception e) {
+ assertTrue(e.getMessage().contains("Connection refused")
+ && e.getMessage().contains("12347"));
+ }
+ }
+
+ private void testRestServerBindPortAndConnectionPort(String bindPort,
int connectPort) throws Exception {
+ RestServerEndpoint serverEndpoint = null;
+ RestClient restClient = null;
+
+ try {
+ final Configuration config = new Configuration();
+ config.setString(RestOptions.ADDRESS, "localhost");
+ config.setString(RestOptions.BIND_PORT, bindPort);
+ config.setInteger(RestOptions.PORT, connectPort);
Review comment:
This is a bit confusing since this config should never be used in
RestServerEndpoint initialization (replaced by `BIND_PORT`, this should be
removed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717606#comment-16717606
]
ASF GitHub Bot commented on FLINK-11081:
walterddr commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240692650
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
##
@@ -181,14 +184,44 @@ protected void initChannel(SocketChannel ch) {
.channel(NioServerSocketChannel.class)
.childHandler(initializer);
- log.debug("Binding rest endpoint to {}:{}.",
restBindAddress, restBindPort);
- final ChannelFuture channel;
- if (restBindAddress == null) {
- channel = bootstrap.bind(restBindPort);
- } else {
- channel = bootstrap.bind(restBindAddress,
restBindPort);
+ ChannelFuture channel;
+
+ // parse port range definition and create port iterator
+ Iterator portsIterator;
+ try {
+ portsIterator =
NetUtils.getPortRangeFromString(restBindPort);
+ } catch (Exception e) {
Review comment:
`getPortRangeFromString` only throws `NumberFormatException` which extends
illegal argument exception already.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717609#comment-16717609
]
ASF GitHub Bot commented on FLINK-11081:
walterddr commented on a change in pull request #7263: [FLINK-11081] Support
binding port range for REST server
URL: https://github.com/apache/flink/pull/7263#discussion_r240711528
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/rest/RestServerEndpointConfigurationTest.java
##
@@ -49,17 +77,154 @@ public void testBasicMapping() throws
ConfigurationException {
Configuration originalConfig = new Configuration();
originalConfig.setString(RestOptions.ADDRESS, ADDRESS);
originalConfig.setString(RestOptions.BIND_ADDRESS,
BIND_ADDRESS);
- originalConfig.setInteger(RestOptions.PORT, PORT);
+ originalConfig.setString(RestOptions.BIND_PORT, BIND_PORT);
originalConfig.setInteger(RestOptions.SERVER_MAX_CONTENT_LENGTH,
CONTENT_LENGTH);
originalConfig.setString(WebOptions.TMP_DIR,
temporaryFolder.getRoot().getAbsolutePath());
final RestServerEndpointConfiguration result =
RestServerEndpointConfiguration.fromConfiguration(originalConfig);
Assert.assertEquals(ADDRESS, result.getRestAddress());
Assert.assertEquals(BIND_ADDRESS, result.getRestBindAddress());
- Assert.assertEquals(PORT, result.getRestBindPort());
+ Assert.assertEquals(BIND_PORT, result.getRestBindPort());
Assert.assertEquals(CONTENT_LENGTH,
result.getMaxContentLength());
Assert.assertThat(
result.getUploadDir().toAbsolutePath().toString(),
containsString(temporaryFolder.getRoot().getAbsolutePath()));
}
+
Review comment:
Can we add a test that tests initial port assignment failure (fall back to
next available port)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716331#comment-16716331
]
vinoyang commented on FLINK-11081:
--
[~walterddr] Thank you for recognizing the value of this issue. In fact, its
implementation emulates the traditional way Flink supports port ranges, such as
"taskmanager.rpc.port". It allows to specify:
* Single port (eg 8081, 0 means randomly picked by jm)
* The port list (12345, 1234) will be selected in order in this case.
* Port range (12345-12346), in this case will also be selected in order
The specific implementation verifies the validity of the port and detects
conflict exception.
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716173#comment-16716173
]
Rong Rong commented on FLINK-11081:
---
+1 to this feature. Very helpful actually for lots of our use cases. just quick
higher level questions, 1) what is the binding strategy of the port range to
the actual port number? (smallest? random?); and 2) should we introduce a
range limit to maximum range size?
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714866#comment-16714866
]
ASF GitHub Bot commented on FLINK-11081:
yanghua commented on issue #7263: [FLINK-11081] Support binding port range for
REST server
URL: https://github.com/apache/flink/pull/7263#issuecomment-445847433
@tillrohrmann Please review this PR when you have time.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713662#comment-16713662
]
ASF GitHub Bot commented on FLINK-11081:
yanghua opened a new pull request #7263: [FLINK-11081] Support binding port
range for REST server
URL: https://github.com/apache/flink/pull/7263
## What is the purpose of the change
*This pull request supports binding port range for REST server*
## Brief change log
- *Support binding port range for REST server*
## Verifying this change
*(Please pick either of the following options)*
This change added tests and can be verified as follows:
- *testDefaultRestServerBindPort*
- *testRestServerSpecificBindPort*
- *testRestServerSpecificBindPort*
- *testRestServerBindPortRange*
- *testRestServerBindPortAndConnectionPortSuccess*
- *testRestServerBindPortAndConnectionPortFailed*
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / **no**)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (yes / **no**)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes / **no**
/ don't know)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs
/ not documented)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
> Labels: pull-request-available
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-11081) Support binding port range for REST server
[
https://issues.apache.org/jira/browse/FLINK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712862#comment-16712862
]
vinoyang commented on FLINK-11081:
--
Will start to fix this issue.
> Support binding port range for REST server
> --
>
> Key: FLINK-11081
> URL: https://issues.apache.org/jira/browse/FLINK-11081
> Project: Flink
> Issue Type: Improvement
> Components: REST
>Affects Versions: 1.7.0, 1.8.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Major
>
> Currently the {{RestServerEndpoint}} binds to the port specified by
> {{RestOptions#PORT}}. {{PORT}} is of type integer. Sometimes, it would be
> useful to being able to specify not only a single port but a port range to
> pick a port from. Therefore, I propose to add similar to
> {{RestOptions#BIND_ADDRESS}} another option {{RestOptions#BIND_PORT}} which
> allows to specify a port range for the {{RestServerEndpoint}} to pick a port
> from. {{RestOptions#PORT}} would then only be used by the client to connect
> to the started {{RestServerEndpoint}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
