The correct import statement is "import
org.apache.hadoop.mapreduce.lib.input.TextInputFormat".
Best Regards,
Shixiong Zhu
2014-03-19 18:46 GMT+08:00 Pariksheet Barapatre <pbarapa...@gmail.com>:
> Seems like import issue, ran with HadoopFile and it worked. Not getting
> import statement for textInputFormat class location for new API.
>
> Can anybody help?
>
> Thanks
> Pariksheet
>
>
> On 19 March 2014 16:05, Bertrand Dechoux <decho...@gmail.com> wrote:
>
>> I don't know the Spark issue but the Hadoop context is clear.
>>
>> old api -> org.apache.hadoop.mapred
>> new api -> org.apache.hadoop.mapreduce
>>
>> You might only need to change your import.
>>
>> Regards
>>
>> Bertrand
>>
>>
>> On Wed, Mar 19, 2014 at 11:29 AM, Pariksheet Barapatre <
>> pbarapa...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Trying to read HDFS file with TextInputFormat.
>>>
>>> scala> import org.apache.hadoop.mapred.TextInputFormat
>>> scala> import org.apache.hadoop.io.{LongWritable, Text}
>>> scala> val file2 =
>>> sc.newAPIHadoopFile[LongWritable,Text,TextInputFormat]("hdfs://
>>> 192.168.100.130:8020/user/hue/pig/examples/data/sonnets.txt")
>>>
>>>
>>> This is giving me the error.
>>>
>>> <console>:14: error: type arguments
>>> [org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.TextInputFormat]
>>> conform to the bounds of none of the overloaded alternatives of
>>> value newAPIHadoopFile: [K, V, F <:
>>> org.apache.hadoop.mapreduce.InputFormat[K,V]](path: String, fClass:
>>> Class[F], kClass: Class[K], vClass: Class[V], conf:
>>> org.apache.hadoop.conf.Configuration)org.apache.spark.rdd.RDD[(K, V)] <and>
>>> [K, V, F <: org.apache.hadoop.mapreduce.InputFormat[K,V]](path:
>>> String)(implicit km: scala.reflect.ClassTag[K], implicit vm:
>>> scala.reflect.ClassTag[V], implicit fm:
>>> scala.reflect.ClassTag[F])org.apache.spark.rdd.RDD[(K, V)]
>>> val file2 =
>>> sc.newAPIHadoopFile[LongWritable,Text,TextInputFormat]("hdfs://
>>> 192.168.100.130:8020/user/hue/pig/examples/data/sonnets.txt")
>>>
>>>
>>> What is correct syntax if I want to use TextInputFormat.
>>>
>>> Also, how to use customInputFormat. Very silly question but I am not
>>> sure how and where to keep jar file containing customInputFormat class.
>>>
>>> Thanks
>>> Pariksheet
>>>
>>>
>>>
>>> --
>>> Cheers,
>>> Pari
>>>
>>
>>
>
>
> --
> Cheers,
> Pari
>
