Ok Got it now. That is a good piece of information.
Thank You :)
Regards
Bejoy KS
Sent from handheld, please excuse typos.
-Original Message-
From: Harsh J
Date: Fri, 3 Aug 2012 16:28:27
To: ;
Cc: Mohammad Tariq
Subject: Re: Reading fields from a Text line
Bejoy,
In the new API
rsh J
> Date: Fri, 3 Aug 2012 13:02:32
> To:
> Reply-To: mapreduce-user@hadoop.apache.org
> Cc: Mohammad Tariq
> Subject: Re: Reading fields from a Text line
>
> That is not really a bug. Only if you use @Override will you be really
> asserting that you've overriden the right
12 13:02:32
To:
Reply-To: mapreduce-user@hadoop.apache.org
Cc: Mohammad Tariq
Subject: Re: Reading fields from a Text line
That is not really a bug. Only if you use @Override will you be really
asserting that you've overriden the right method (since new API uses
inheritance instead of
That is not really a bug. Only if you use @Override will you be really
asserting that you've overriden the right method (since new API uses
inheritance instead of interfaces). Without that kinda check, its easy
to make mistakes and add in methods that won't get considered by the
framework (and henc
Hi Tariq
On further analysis I noticed a odd behavior in this context.
If we use the default InputFormat (TextInputFormat) but specify the Key
type in mapper as IntWritable instead of Long Writable. The framework is
supposed throw a class cast exception.Such an exception is thrown only if
the key
Hi Tariq
Again I strongly suspect the IdentityMapper in play here. The reasoning why
I suspect so is
When you have the whole data in output file it should be the Identity
Mapper. Due to the mismatch in input key type at class level and method
level the framework is falling back to IdentityMapper.
ammad Tariq
> Date: Thu, 2 Aug 2012 15:48:42
> To:
> Reply-To: mapreduce-user@hadoop.apache.org
> Subject: Re: Reading fields from a Text line
>
> Thanks for the response Harsh n Sri. Actually, I was trying to prepare
> a template for my application using which I was trying to r
that as well.
Regards
Bejoy KS
Sent from handheld, please excuse typos.
-Original Message-
From: Mohammad Tariq
Date: Thu, 2 Aug 2012 15:48:42
To:
Reply-To: mapreduce-user@hadoop.apache.org
Subject: Re: Reading fields from a Text line
Thanks for the response Harsh n Sri. Actually, I
Hi Tariq,
Is your file splittable? If it's not, Mapper will process entire file in
one go!
http://hadoop.apache.org/common/docs/r0.20.1/api/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.html#isSplitable%28org.apache.hadoop.mapreduce.JobContext,%20org.apache.hadoop.fs.Path%29
How many mapp
Thanks for the response Harsh n Sri. Actually, I was trying to prepare
a template for my application using which I was trying to read one
line at a time, extract the first field from it and emit that
extracted value from the mapper. I have these few lines of code for
that :
public static class XPT
Wouldn't it be better if you could skip those unwanted lines
upfront(preprocess) and have a file which is ready to be processed by the
MR system? In any case, more details are needed.
On Thu, Aug 2, 2012 at 8:23 AM, Harsh J wrote:
> Mohammad,
>
> > But it seems I am not doing things in correct
Mohammad,
> But it seems I am not doing things in correct way. Need some guidance.
What do you mean by the above? What is your written code exactly
expected to do and what is it not doing? Perhaps since you ask for a
code question here, can you share it with us (pastebin or gists,
etc.)?
For sk
Hello list,
I have a flat file in which data is stored as lines of 107
bytes each. I need to skip the first 8 lines(as they don't contain any
valuable info). Thereafter, I have to read each line and extract the
information from them, but not the line as a whole. Each line is
composed of sev
13 matches
Mail list logo
