[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-24 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16265167#comment-16265167 ] Steve Loughran commented on SPARK-22526: it says it in the javadocs for

[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-23 Thread mohamed imran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264931#comment-16264931 ] mohamed imran commented on SPARK-22526: --- [~ste...@apache.org] Thanks. But i have referred some of

[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-23 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264496#comment-16264496 ] Steve Loughran commented on SPARK-22526: I'm not giving a permanent fix. It's a bug in your code

[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-23 Thread mohamed imran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264208#comment-16264208 ] mohamed imran commented on SPARK-22526: --- [~ste...@apache.org] Thanks steve. I will modify my code

[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-23 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264095#comment-16264095 ] Steve Loughran commented on SPARK-22526: # Fix the code you invoke #. wrap the code you invoke

[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-22 Thread mohamed imran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16263097#comment-16263097 ] mohamed imran commented on SPARK-22526: --- [~ste...@apache.org] Yes. Thats sounds sense to me. Is

[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-22 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16263021#comment-16263021 ] Steve Loughran commented on SPARK-22526: If the input stream doesn't get closed, there probably

[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-22 Thread mohamed imran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16262489#comment-16262489 ] mohamed imran commented on SPARK-22526: --- [~ste...@apache.org] Thanks for your suggestions. I did

[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-21 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261292#comment-16261292 ] Steve Loughran commented on SPARK-22526: S3a uses the AWS S3 client, which uses httpclient

[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-17 Thread mohamed imran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257247#comment-16257247 ] mohamed imran commented on SPARK-22526: --- [~srowen] Yes I agree that it is an issue with the http

[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257230#comment-16257230 ] Sean Owen commented on SPARK-22526: --- It could be a problem with S3, or the S3 API. You haven't shown

[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-17 Thread mohamed imran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257224#comment-16257224 ] mohamed imran commented on SPARK-22526: --- [~srowen] I am processing inside the foreach loop. like

[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256986#comment-16256986 ] Sean Owen commented on SPARK-22526: --- You're only showing a read of one file, a .zip file. I don't see

[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-15 Thread mohamed imran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253819#comment-16253819 ] mohamed imran commented on SPARK-22526: --- Nope. Actually i am reading some zip files using the

[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-15 Thread mohamed imran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253820#comment-16253820 ] mohamed imran commented on SPARK-22526: --- Nope. Actually i am reading some zip files using the

[jira] [Commented] (SPARK-22526) Spark hangs while reading binary files from S3

2017-11-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253694#comment-16253694 ] Sean Owen commented on SPARK-22526: --- It's not clear what you're reporting. Are you saying it's an Avro