Re: Spark, snappy and HDFS

2015-04-02 Thread Nick Travers
()Z* > >> > >> The following works for me when the file is uncompressed: > >> > >> import org.apache.hadoop.io._ > >> val hdfsPath = "hdfs://nost.name/path/to/folder" > >> val file = sc.sequenceFi

Re: Spark, snappy and HDFS

2015-04-02 Thread Sean Owen
file.count() >> >> but fails when the encoding is Snappy. >> >> I've seen some stuff floating around on the web about having to explicitly >> enable support for Snappy in spark, but it doesn't seem to work for me: >> http://www.ericlin.me/enablin

Re: Spark, snappy and HDFS

2015-04-01 Thread Nick Travers
esn't seem to work for me: > http://www.ericlin.me/enabling-snappy-support-for-sharkspark > <http://www.ericlin.me/enabling-snappy-support-for-sharkspark> > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/

Re: Spark, snappy and HDFS

2015-04-01 Thread Xianjin YE
f floating around on the web about having to explicitly > > > enable support for Snappy in spark, but it doesn't seem to work for me: > > > http://www.ericlin.me/enabling-snappy-support-for-sharkspark > > > <http://www.ericlin.me/enabling-sna

Re: Spark, snappy and HDFS

2015-04-01 Thread Nick Travers
Snappy in spark, but it doesn't seem to work for me: > http://www.ericlin.me/enabling-snappy-support-for-sharkspark > <http://www.ericlin.me/enabling-snappy-support-for-sharkspark> > > > > -- > View this message in context: > http://apache-spark-u

Re: Spark, snappy and HDFS

2015-04-01 Thread Xianjin YE
doesn't seem to work for me: > http://www.ericlin.me/enabling-snappy-support-for-sharkspark > <http://www.ericlin.me/enabling-snappy-support-for-sharkspark> > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spar

Spark, snappy and HDFS

2015-04-01 Thread Nick Travers
doesn't seem to work for me: http://www.ericlin.me/enabling-snappy-support-for-sharkspark <http://www.ericlin.me/enabling-snappy-support-for-sharkspark> -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-snappy-and-HDFS-tp22349.html Sent from