On Fri, Jul 4, 2014 at 3:33 PM, Gurvinder Singh <gurvinder.si...@uninett.no>
wrote:
csv =
> sc.newAPIHadoopFile(opts.input,"com.hadoop
> .mapreduce.LzoTextInputFormat","org.apache.hadoop
> .io.LongWritable","org.apache.hadoop.io.Text").count()
>
Does anyone know what the rough equivalent of this would be in the Scala
API?
I am trying the following, but the first import yields an error on my
spark-ec2 cluster:
import com.hadoop.mapreduce.LzoTextInputFormatimport
org.apache.hadoop.io.LongWritableimport org.apache.hadoop.io.Text
sc.newAPIHadoopFile("s3n://datasets.elasticmapreduce/ngrams/books/20090715/eng-us-all/1gram/data",
LzoTextInputFormat, LongWritable, Text)
scala> import com.hadoop.mapreduce.LzoTextInputFormat
<console>:12: error: object hadoop is not a member of package com
import com.hadoop.mapreduce.LzoTextInputFormat
Nick
