Re: [Spark-Core]port opened by the SparkDriver is vulnerable to flooding attacks

2018-02-28 Thread Marcelo Vanzin
The mechanism to close idle connections already exists. It doesn't mean you can just use it as is in existing connections. So if you want to go and fix that, you're going to have to figure out that part. Or figure out a different solution. Either way, file a bug so that this is properly tracked.

Re: [Spark-Core]port opened by the SparkDriver is vulnerable to flooding attacks

2018-02-28 Thread Sandeep Katta
Yeh monitor is present but for some cases like long running job I found App master is idle.so it will end up closing the App master’s channel so job will not be completed. So needed a mechanism to close only invalid connections . On Wed, 28 Feb 2018 at 10:54 PM, Marcelo Vanzin

Re: [Spark-Core]port opened by the SparkDriver is vulnerable to flooding attacks

2018-02-28 Thread Marcelo Vanzin
Spark already has code to monitor idle connections and close them. That's in TransportChannelHandler.java. If there's anything to do here, it's to allow all users of the transport library to support the "close idle connections" feature of that class. On Wed, Feb 28, 2018 at 9:07 AM,