[ https://issues.apache.org/jira/browse/SPARK-19520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15858797#comment-15858797 ]
Apache Spark commented on SPARK-19520: -------------------------------------- User 'vanzin' has created a pull request for this issue: https://github.com/apache/spark/pull/16862 > WAL should not be encrypted > --------------------------- > > Key: SPARK-19520 > URL: https://issues.apache.org/jira/browse/SPARK-19520 > Project: Spark > Issue Type: Bug > Components: DStreams > Affects Versions: 2.1.0 > Reporter: Marcelo Vanzin > > If I/O encryption is enabled in Spark (SPARK-5682), the write ahead logs > written by streaming applications will also be encrypted. > That creates issues because only the driver that wrote those files knows the > key to decrypt them. So basically the WAL is unusuable in that scenario. > Instead, the WAL should not be encrypted by Spark; if the user has a need to > encrypt that data, other means (such as HDFS encryption zones) should be used > instead. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org