Hi I'm trying to read lzo compressed files from S3 using spark. The lzo files are not indexed. Spark job starts to read the files just fine but after a while it just hangs. No network throughput. I have to restart the worker process to get it back up. Any idea what could be causing this. We were using uncompressed files before and that worked just fine, went with the compression to reduce S3 storage.
Any help would be appreciated. Thanks! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/SPARK-S3-LZO-input-worker-stuck-tp9584.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
