[jira] [Commented] (SPARK-9091) Add the codec interface to Text DStream.
[ https://issues.apache.org/jira/browse/SPARK-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629639#comment-14629639 ] SaintBacchus commented on SPARK-9091: - [~sowen] I agree user can design the output by DStream.foreachRDD, I purpose it for convenience to use. In my case, I had copy a bit code from Spark to adapt this function and I guess others may also have this scenario, so I open this Jira to push it into Spark. > Add the codec interface to Text DStream. > > > Key: SPARK-9091 > URL: https://issues.apache.org/jira/browse/SPARK-9091 > Project: Spark > Issue Type: Improvement > Components: Streaming >Reporter: SaintBacchus >Priority: Minor > > Since the RDD has the function *saveAsTextFile* which can use > *CompressionCodec* to compress the data, so it's better to add a similar > interface in DStream. > In some IO-bottleneck scenario, it's very useful for user to have this > interface in DStream. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9091) Add the codec interface to Text DStream.
[ https://issues.apache.org/jira/browse/SPARK-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629632#comment-14629632 ] Apache Spark commented on SPARK-9091: - User 'SaintBacchus' has created a pull request for this issue: https://github.com/apache/spark/pull/7442 > Add the codec interface to Text DStream. > > > Key: SPARK-9091 > URL: https://issues.apache.org/jira/browse/SPARK-9091 > Project: Spark > Issue Type: Improvement > Components: Streaming >Reporter: SaintBacchus >Priority: Minor > > Since the RDD has the function *saveAsTextFile* which can use > *CompressionCodec* to compress the data, so it's better to add a similar > interface in DStream. > In some IO-bottleneck scenario, it's very useful for user to have this > interface in DStream. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9091) Add the codec interface to Text DStream.
[ https://issues.apache.org/jira/browse/SPARK-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629483#comment-14629483 ] SaintBacchus commented on SPARK-9091: - [~srowen] Sorry for forgetting to change the type and add the description immediately, it's a small improvement in the DStream. I will reopen it after I had the PR. > Add the codec interface to Text DStream. > > > Key: SPARK-9091 > URL: https://issues.apache.org/jira/browse/SPARK-9091 > Project: Spark > Issue Type: Improvement > Components: Streaming >Affects Versions: 1.5.0 >Reporter: SaintBacchus >Priority: Minor > > Since the RDD has the function *saveAsTextFile* which can use > *CompressionCodec* to compress the data, so it's better to add a similar > interface in DStream -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
