[
https://issues.apache.org/jira/browse/SPARK-23545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387307#comment-16387307
]
sandeep katta commented on SPARK-23545:
---------------------------------------
I will be working on this bug,
Solution is as follows
1.Send the HeartBeat(1 way message) from APP master to Driver ,so Driver treats
this channel as active
2.Driver can close all the inactive channels
If any questions regarding this solution,please be free to comment on this
> [Spark-Core] port opened by the SparkDriver is vulnerable for flooding attacks
> ------------------------------------------------------------------------------
>
> Key: SPARK-23545
> URL: https://issues.apache.org/jira/browse/SPARK-23545
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.2.1
> Reporter: sandeep katta
> Priority: Major
>
> port opened by the SparkDriver is vulnerable for flooding attacks
> *Steps*:
> set spark.network.timeout=60s //can be any value
> Start the thriftserver in client mode and you can see in below logs that the
> spark Driver opens the port for AM and executors to communicate.
> Logs:
> 018-03-01 16:11:16,497 | INFO | [main] | Successfully started service
> *'sparkDriver'* on port *22643*. |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
> 2018-03-01 16:11:17,265 | INFO | [main] | Successfully started service
> 'SparkUI' on port 22950. |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
> 2018-03-01 16:11:44,640 | INFO | [main] | Successfully started service
> 'org.apache.spark.network.netty.NettyBlockTransferService' on port 22663. |
> org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
> 2018-03-01 16:11:52,822 | INFO | [Thread-56] | Starting
> ThriftBinaryCLIService on port 22550 with 5...501 worker threads |
> org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.run(ThriftBinaryCLIService.java:111)
> Do telnet to this port using *telnet IP 22643* command and keep it idle,
> after 60 seconds check the status, connection is still established, it should
> be terminated
> *lsof command output along with the date*
>
> host1:/var/ # date
> Thu Mar 1 *16:12:55* CST 2018
> host1:/var/ # lsof | grep 22643
> java 66730 user1 292u IPv6 1482635919 0t0 TCP
> host1:22643->*10.18.152.191:59297* (ESTABLISHED)
> java 66730 user1 297u IPv6 1482374122 0t0 TCP
> host1:22643->BLR1000018529:43894 (ESTABLISHED)
> java 66730 user1 346u IPv6 1482314249 0t0 TCP host1:22643 (LISTEN)
> host1:/var/ # date
> Thu Mar 1 16:13:43 CST 2018
> host1:/var/ # date
> Thu Mar 1 *16:16:55* CST 2018
> host1:/var/ # lsof | grep 22643
> java 66730 user1 292u IPv6 1482635919 0t0 TCP
> host1:22643->*10.18.152.191:59297* (ESTABLISHED)
> java 66730 user1 297u IPv6 1482374122 0t0 TCP
> host1:22643->BLR1000018529:43894 (ESTABLISHED)
> java 66730 user1 346u IPv6 1482314249 0t0 TCP host1:22643 (LISTEN)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
