[jira] [Comment Edited] (FLINK-9231) Enable SO_REUSEADDR on listen sockets
[ https://issues.apache.org/jira/browse/FLINK-9231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16527672#comment-16527672 ] Andrey Zagrebin edited comment on FLINK-9231 at 6/29/18 2:16 PM: - [~triones], [~yuzhih...@gmail.com] I would suggest to rebase PR on the actual master and update `RestServerEndpoint` the same way as `WebFrontendBootstrap` in PR. Still I think, before we change socket options in web servers, it would be useful to have here in issue description/comments details about the use case where this is a problem. Is it some kind of frequent restart of web server locally? was (Author: azagrebin): [~triones], [~yuzhih...@gmail.com] I would suggest to rebase on actual master and update `RestServerEndpoint` the same as `WebFrontendBootstrap` in PR. Still I think, before we change socket options in web servers, it would be useful to have here in issue description/comments details about the use case where this is a problem. Is it some kind of frequent restart of web server locally? > Enable SO_REUSEADDR on listen sockets > - > > Key: FLINK-9231 > URL: https://issues.apache.org/jira/browse/FLINK-9231 > Project: Flink > Issue Type: Improvement > Components: Webfrontend >Reporter: Ted Yu >Assignee: Triones Deng >Priority: Major > Labels: pull-request-available > > This allows sockets to be bound even if there are sockets from a previous > application that are still pending closure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (FLINK-9231) Enable SO_REUSEADDR on listen sockets
[ https://issues.apache.org/jira/browse/FLINK-9231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450111#comment-16450111 ] mingleizhang edited comment on FLINK-9231 at 4/24/18 3:51 PM: -- Hi, [~triones] I think you can push a PR first. Would not suggest to push a bunch of code here. PR can show your idea in a easier way, if the PR not suitable for the issue, then people would give you some suggestions. then, After a few iterations with the PR. Patch then available to merge. Good luck ~ was (Author: mingleizhang): Hi, [~triones] I think you can push a PR first. Would not suggest to push a bunch of code here. > Enable SO_REUSEADDR on listen sockets > - > > Key: FLINK-9231 > URL: https://issues.apache.org/jira/browse/FLINK-9231 > Project: Flink > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Triones Deng >Priority: Major > > This allows sockets to be bound even if there are sockets > from a previous application that are still pending closure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (FLINK-9231) Enable SO_REUSEADDR on listen sockets
[ https://issues.apache.org/jira/browse/FLINK-9231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448142#comment-16448142 ] Ted Yu edited comment on FLINK-9231 at 4/24/18 3:07 PM: Plan for WebFrontendBootstrap sounds good to me. was (Author: yuzhih...@gmail.com): Sounds good to me. > Enable SO_REUSEADDR on listen sockets > - > > Key: FLINK-9231 > URL: https://issues.apache.org/jira/browse/FLINK-9231 > Project: Flink > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Triones Deng >Priority: Major > > This allows sockets to be bound even if there are sockets > from a previous application that are still pending closure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (FLINK-9231) Enable SO_REUSEADDR on listen sockets
[
https://issues.apache.org/jira/browse/FLINK-9231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448079#comment-16448079
]
Triones Deng edited comment on FLINK-9231 at 4/24/18 2:05 PM:
--
[~yuzhih...@gmail.com] I notice that there are four kind of server socket,
# JobManager and TaskManager create socket server by
NetUtils.createSocketFromPorts. here just looking for a available port to
create ActorSystem. the config in akka-remote.jar the config item
"tcp-reuse-addr = off-for-windows". and will close the ServerSocket at once
when find a available port. code as below. here i think we can direct call new
ServerSocker(port) without backlog will be ok, what do you think?
{code:java}
val result = AkkaUtils.retryOnBindException({
// Try all ports in the range until successful
val socket = NetUtils.createSocketFromPorts(
actorSystemPortRange,
new NetUtils.SocketFactory {
override def createSocket(port: Int): ServerSocket = new ServerSocket(
// Use the correct listening address, bound ports will only be
// detected later by Akka.
port, 0, InetAddress.getByName(NetUtils.getWildcardIPAddress))
})
val port =
if (socket == null) {
throw new BindException(s"Unable to allocate port for TaskManager.")
} else {
try {
socket.getLocalPort()
} finally {
socket.close()
}
}
..
}, { !actorSystemPortRange.hasNext }, 5000)
{code}
# BlobServer make use of ServerSocket or SSLContext to create ServerSocket
# make use of Netty for io,like NettyServer.
# WebFrontendBootstrap make use of netty to create ServerBootstrap.
I think if we plan to make use of SO_REUSEADDR, "which is suitable It is useful
if your server has been shut down, and then restarted right away while sockets
are still active on its port. You should be aware that if any unexpected data
comes in, it may confuse your server, but while this is possible, it is not
likely" (see
:[https://stackoverflow.com/questions/19960475/problems-related-to-so-reuseaddr?rq=1)]
, Here may be we can allow sockets to set SO_REUSEADDR when start
WebFrontendBootstrap. what's your idea? anything wrong please feel free to
correct me.
sample code for WebFrontendBootstrap.java like:
{code:java}
this.bootstrap = new ServerBootstrap();
this.bootstrap
.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(initializer).childOption(ChannelOption.SO_REUSEADDR,true);{code}
was (Author: triones):
[~yuzhih...@gmail.com] I notice that there are four kind of server socket,
# JobManager and TaskManager create socket server by
NetUtils.createSocketFromPorts. here just looking for a available port to
create ActorSystem. will close the ServerSocket at once when find a available
port. code as below. here i think we can direct call new ServerSocker(port)
without backlog will be ok, what do you think?
{code:java}
val result = AkkaUtils.retryOnBindException({
// Try all ports in the range until successful
val socket = NetUtils.createSocketFromPorts(
actorSystemPortRange,
new NetUtils.SocketFactory {
override def createSocket(port: Int): ServerSocket = new ServerSocket(
// Use the correct listening address, bound ports will only be
// detected later by Akka.
port, 0, InetAddress.getByName(NetUtils.getWildcardIPAddress))
})
val port =
if (socket == null) {
throw new BindException(s"Unable to allocate port for TaskManager.")
} else {
try {
socket.getLocalPort()
} finally {
socket.close()
}
}
..
}, { !actorSystemPortRange.hasNext }, 5000)
{code}
# BlobServer make use of ServerSocket or SSLContext to create ServerSocket
# make use of Netty for io,like NettyServer.
# WebFrontendBootstrap make use of netty to create ServerBootstrap.
I think if we plan to make use of SO_REUSEADDR, "which is suitable It is useful
if your server has been shut down, and then restarted right away while sockets
are still active on its port. You should be aware that if any unexpected data
comes in, it may confuse your server, but while this is possible, it is not
likely" (see
:[https://stackoverflow.com/questions/19960475/problems-related-to-so-reuseaddr?rq=1)]
, Here may be we can allow sockets to set SO_REUSEADDR when start
WebFrontendBootstrap. what's your idea? anything wrong please feel free to
correct me.
sample code for WebFrontendBootstrap.java like:
{code:java}
this.bootstrap = new ServerBootstrap();
this.bootstrap
.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(initializer).childOption(ChannelOption.SO_REUSEADDR,true);{code}
> Enable SO_REUSEADDR on listen sockets
> -
>
> Key: FLINK-9231
> URL: https://issues.apache.org/jira/browse/FLINK-9231
> Project: Flink
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Triones Deng
>Priority: Major
>
> This allows sockets to be bound ev
[jira] [Comment Edited] (FLINK-9231) Enable SO_REUSEADDR on listen sockets
[
https://issues.apache.org/jira/browse/FLINK-9231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448079#comment-16448079
]
Triones Deng edited comment on FLINK-9231 at 4/24/18 1:56 PM:
--
[~yuzhih...@gmail.com] I notice that there are four kind of server socket,
# JobManager and TaskManager create socket server by
NetUtils.createSocketFromPorts. here just looking for a available port to
create ActorSystem. will close the ServerSocket at once when find a available
port. code as below. here i think we can direct call new ServerSocker(port)
without backlog will be ok, what do you think?
{code:java}
val result = AkkaUtils.retryOnBindException({
// Try all ports in the range until successful
val socket = NetUtils.createSocketFromPorts(
actorSystemPortRange,
new NetUtils.SocketFactory {
override def createSocket(port: Int): ServerSocket = new ServerSocket(
// Use the correct listening address, bound ports will only be
// detected later by Akka.
port, 0, InetAddress.getByName(NetUtils.getWildcardIPAddress))
})
val port =
if (socket == null) {
throw new BindException(s"Unable to allocate port for TaskManager.")
} else {
try {
socket.getLocalPort()
} finally {
socket.close()
}
}
..
}, { !actorSystemPortRange.hasNext }, 5000)
{code}
# BlobServer make use of ServerSocket or SSLContext to create ServerSocket
# make use of Netty for io,like NettyServer.
# WebFrontendBootstrap make use of netty to create ServerBootstrap.
I think if we plan to make use of SO_REUSEADDR, "which is suitable It is useful
if your server has been shut down, and then restarted right away while sockets
are still active on its port. You should be aware that if any unexpected data
comes in, it may confuse your server, but while this is possible, it is not
likely" (see
:[https://stackoverflow.com/questions/19960475/problems-related-to-so-reuseaddr?rq=1)]
, Here may be we can allow sockets to set SO_REUSEADDR when start
WebFrontendBootstrap. what's your idea? anything wrong please feel free to
correct me.
sample code for WebFrontendBootstrap.java like:
{code:java}
this.bootstrap = new ServerBootstrap();
this.bootstrap
.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(initializer).childOption(ChannelOption.SO_REUSEADDR,true);{code}
was (Author: triones):
[~yuzhih...@gmail.com] I notice that there are three kind of server socket,
# JobManager and TaskManager create socket server by
NetUtils.createSocketFromPorts. here just looking for a available port to
create ActorSystem. will close the ServerSocket at once when find a available
port. code as below. here i think we can direct call new ServerSocker(port)
without backlog will be ok, what do you think?
{code:java}
val result = AkkaUtils.retryOnBindException({
// Try all ports in the range until successful
val socket = NetUtils.createSocketFromPorts(
actorSystemPortRange,
new NetUtils.SocketFactory {
override def createSocket(port: Int): ServerSocket = new ServerSocket(
// Use the correct listening address, bound ports will only be
// detected later by Akka.
port, 0, InetAddress.getByName(NetUtils.getWildcardIPAddress))
})
val port =
if (socket == null) {
throw new BindException(s"Unable to allocate port for TaskManager.")
} else {
try {
socket.getLocalPort()
} finally {
socket.close()
}
}
..
}, { !actorSystemPortRange.hasNext }, 5000)
{code}
# make use of Netty for io,like NettyServer.
# BlobServer make use of ServerSocket or SSLContext to create ServerSocket
# WebFrontendBootstrap make use of netty to create ServerBootstrap.
I think if we plan to make use of SO_REUSEADDR, "which is suitable It is useful
if your server has been shut down, and then restarted right away while sockets
are still active on its port. You should be aware that if any unexpected data
comes in, it may confuse your server, but while this is possible, it is not
likely" (see
:[https://stackoverflow.com/questions/19960475/problems-related-to-so-reuseaddr?rq=1)]
, Here may be we can allow sockets to set SO_REUSEADDR when start
WebFrontendBootstrap. what's your idea? anything wrong please feel free to
correct me.
sample code for WebFrontendBootstrap.java like:
{code:java}
this.bootstrap = new ServerBootstrap();
this.bootstrap
.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(initializer).childOption(ChannelOption.SO_REUSEADDR,true);{code}
> Enable SO_REUSEADDR on listen sockets
> -
>
> Key: FLINK-9231
> URL: https://issues.apache.org/jira/browse/FLINK-9231
> Project: Flink
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Triones Deng
>Priority: Major
>
> This allows sockets to be bound even if there are sockets
> from a previous application that are still pending closure.
[jira] [Comment Edited] (FLINK-9231) Enable SO_REUSEADDR on listen sockets
[
https://issues.apache.org/jira/browse/FLINK-9231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448079#comment-16448079
]
Triones Deng edited comment on FLINK-9231 at 4/24/18 1:54 PM:
--
[~yuzhih...@gmail.com] I notice that there are three kind of server socket,
# JobManager and TaskManager create socket server by
NetUtils.createSocketFromPorts. here just looking for a available port to
create ActorSystem. will close the ServerSocket at once when find a available
port. code as below. here i think we can direct call new ServerSocker(port)
without backlog will be ok, what do you think?
{code:java}
val result = AkkaUtils.retryOnBindException({
// Try all ports in the range until successful
val socket = NetUtils.createSocketFromPorts(
actorSystemPortRange,
new NetUtils.SocketFactory {
override def createSocket(port: Int): ServerSocket = new ServerSocket(
// Use the correct listening address, bound ports will only be
// detected later by Akka.
port, 0, InetAddress.getByName(NetUtils.getWildcardIPAddress))
})
val port =
if (socket == null) {
throw new BindException(s"Unable to allocate port for TaskManager.")
} else {
try {
socket.getLocalPort()
} finally {
socket.close()
}
}
..
}, { !actorSystemPortRange.hasNext }, 5000)
{code}
# make use of Netty for io,like NettyServer.
# BlobServer make use of ServerSocket or SSLContext to create ServerSocket
# WebFrontendBootstrap make use of netty to create ServerBootstrap.
I think if we plan to make use of SO_REUSEADDR, "which is suitable It is useful
if your server has been shut down, and then restarted right away while sockets
are still active on its port. You should be aware that if any unexpected data
comes in, it may confuse your server, but while this is possible, it is not
likely" (see
:[https://stackoverflow.com/questions/19960475/problems-related-to-so-reuseaddr?rq=1)]
, Here may be we can allow sockets to set SO_REUSEADDR when start
WebFrontendBootstrap. what's your idea? anything wrong please feel free to
correct me.
sample code for WebFrontendBootstrap.java like:
{code:java}
this.bootstrap = new ServerBootstrap();
this.bootstrap
.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(initializer).childOption(ChannelOption.SO_REUSEADDR,true);{code}
was (Author: triones):
[~yuzhih...@gmail.com] I notice that there are two kind of server socket,
# make use of SockerServer, like JobManager,TaskManager, TaskManagerRunner
and BlobServer, all of them create socket server by
NetUtils.createSocketFromPorts
# make use of Netty for io,like NettyServer.
I think if we plan to make use of SO_REUSEADDR, "which is suitable It is useful
if your server has been shut down, and then restarted right away while sockets
are still active on its port. You should be aware that if any unexpected data
comes in, it may confuse your server, but while this is possible, it is not
likely" (see
:[https://stackoverflow.com/questions/19960475/problems-related-to-so-reuseaddr?rq=1)]
, Here may be we can allow sockets to set SO_REUSEADDR when start
JobManager,TaskManager, TaskManagerRunner. anything wrong please correct me.
sample code for JobManager.scala like:
{code:java}
val socket = NetUtils.createSocketFromPorts(
listeningPortRange,
new NetUtils.SocketFactory {
override def createSocket(port: Int): ServerSocket = {
// Use the correct listening address, bound ports will only be
// detected later by Akka.
val serverSocket = new ServerSocket()
serverSocket.setReuseAddress(true)
serverSocket.bind(new
InetSocketAddress(InetAddress.getByName(NetUtils.getWildcardIPAddress), port),
0)
serverSocket
}
})
{code}
> Enable SO_REUSEADDR on listen sockets
> -
>
> Key: FLINK-9231
> URL: https://issues.apache.org/jira/browse/FLINK-9231
> Project: Flink
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Triones Deng
>Priority: Major
>
> This allows sockets to be bound even if there are sockets
> from a previous application that are still pending closure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
