Hi Devaraj ,
Indeed, the previous email that I've sent you contained -ls output of
SequenceFileOutputFormat with signatures of the class in it. Hence it was
87 bytes. Hadoop was creating "empty" files(in fact, files containing only
the signature) before I started to use LazyOutputFormat.
Regards
Murat
On Tue, Jun 5, 2012 at 7:22 AM, Devaraj k <devara...@huawei.com> wrote:
> The output files should 0 kb size if you use
> FileOutputFormat/TextOutputFormat.
>
> I think your output format writer is writing some meta data in those
> files. Can you check what is the data present in those files.
>
> Can you tell me which output format are you using?
>
> Thanks
> Devaraj
>
> ________________________________________
> From: murat migdisoglu [murat.migdiso...@gmail.com]
> Sent: Monday, June 04, 2012 6:18 PM
> To: common-user@hadoop.apache.org
> Subject: Re: What happens when I do not output anything from my mapper
>
> Hi,
> Thanks for your answer. After I've read your emails, I decided to clear
> completely my mapper method to see If I can disable the output of the
> mapper class at all, but it seems it did not work
> So, here is my mapper method:
>
> @Override
> public void map(ByteBuffer key, SortedMap<ByteBuffer, IColumn> columns,
> Context context)
> throws IOException, InterruptedException
> {
>
> }
>
> when I execute hadoop fs -ls, I still see many small output files as
> following:
>
> -rw-r--r-- 3 mmigdiso supergroup 87 2012-06-04 12:44
> /user/mmigdiso/output/part-m-00034
> -rw-r--r-- 3 mmigdiso supergroup 87 2012-06-04 12:45
> /user/mmigdiso/output/part-m-00037
> -rw-r--r-- 3 mmigdiso supergroup 87 2012-06-04 12:45
> /user/mmigdiso/output/part-m-00039
> -rw-r--r-- 3 mmigdiso supergroup 87 2012-06-04 12:45
> /user/mmigdiso/output/part-m-00040
> -rw-r--r-- 3 mmigdiso supergroup 87 2012-06-04 12:45
> /user/mmigdiso/output/part-m-00042
>
> Do you know If I have to put something special to the context to specify
> the "empty" output?
>
> Regards
> Murat
>
>
>
> On Mon, Jun 4, 2012 at 2:38 PM, Devaraj k <devara...@huawei.com> wrote:
>
> > Hi Murat,
> >
> > As Praveenesh explained, you can control the map outputs as you want.
> >
> > map() function will be called for each input i.e map() function invokes
> > multiple times with different inputs in the same mapper. You can check by
> > having the logs in the map function what is happening in it.
> >
> >
> > Thanks
> > Devaraj
> >
> > ________________________________________
> > From: praveenesh kumar [praveen...@gmail.com]
> > Sent: Monday, June 04, 2012 5:57 PM
> > To: common-user@hadoop.apache.org
> > Subject: Re: What happens when I do not output anything from my mapper
> >
> > You can control your map outputs based on any condition you want. I have
> > done that - it worked for me.
> > It could be your code problem that its not working for you.
> > Can you please share your map code or cross-check whether your conditions
> > are correct ?
> >
> > Regards,
> > Praveenesh
> >
> > On Mon, Jun 4, 2012 at 5:52 PM, murat migdisoglu <
> > murat.migdiso...@gmail.com
> > > wrote:
> >
> > > Hi,
> > > I have a small application where I have only mapper class defined(no
> > > reducer, no combiner).
> > > Within the mapper class, I have an if condition according to which I
> > decide
> > > If I want to put something in the context or not.
> > > If my condition is not match, I want that mapper does not give any
> output
> > > to the hdfs.
> > > But apparently, this does not worj as I expected. Once I run my job, a
> > file
> > > per mapper in the hdfs with 87 kb of size.
> > >
> > > the if block that I'm using in the map method is as following:
> > > if (ip == null || ip.equals(cip)) {
> > > Text value = new Text(mwrapper.toJson());
> > > word.set(ip);
> > > context.write( word, value);
> > > } else {
> > > log.info("ip not match [" + ip + "]");
> > > }
> > > }
> > > }//end of mapper method
> > >
> > > How can I manage that? Does mapper always need to have an output?
> > >
> > > --
> > > "Find a job you enjoy, and you'll never work a day in your life."
> > > Confucius
> > >
> >
>
>
>
> --
> "Find a job you enjoy, and you'll never work a day in your life."
> Confucius
>
--
"Find a job you enjoy, and you'll never work a day in your life."
Confucius
