Hey! Thank you for replying!
Please, confirm that I understand you correctly:
1. Use a class, which extends mapper
class MyMapper extends Mapper<LongWritable, Text, IntWritable, MyPage> {
public void map(LongWritable key, Text value, Context context) throws
IOException, InterruptedException {
//implement all logic here
}
}
2. Leave MultithreadMapper class empty:
public class MyMultithreadMapper extends MultithreadedMapper<LongWritable,
Text, IntWritable, MyPage> {
}
3. In main set
MultithreadedMapper.setMapperClass(job, MyMapper.class);
Should I set
job.setMapperClass(MyMapper.class);
or
job.setMapperClass(MyMultithreadMapper.class);
?
Arsen
On Wed, Sep 28, 2011 at 9:38 AM, Kamesh <[email protected]> wrote:
> **
> On Wednesday 28 September 2011 11:33 AM, Arsen Zahray wrote:
>
> MultithreadMapper extends MultithreadedMapper<LongWritable, Text,
> IntWritable, MyPage> {
>
> ConcurrentLinkedQueue<MyScraper> scrapers = new
> ConcurrentLinkedQueue<MyScraper>();
>
> public static final int nThreads = 5;
>
> public MyMultithreadMapper() {
> for (int i = 0; i < nT
>
> implement the map logic in a separate Mapper class and in the main method
> set the following property
> MultithreadedMapper.setMapperClass(job, MyMapper.class);
>
> --
> *Thanks&Regards,*
> *Bh.V.S.Kamesh*
>
