Wouldn't it be better if you could skip those unwanted lines upfront(preprocess) and have a file which is ready to be processed by the MR system? In any case, more details are needed.
On Thu, Aug 2, 2012 at 8:23 AM, Harsh J <ha...@cloudera.com> wrote: > Mohammad, > > > But it seems I am not doing things in correct way. Need some guidance. > > What do you mean by the above? What is your written code exactly > expected to do and what is it not doing? Perhaps since you ask for a > code question here, can you share it with us (pastebin or gists, > etc.)? > > For skipping 8 lines, if you are using splits, you need to detect > within the mapper or your record reader if the map task filesplit has > an offset of 0 and skip 8 line reads if so (Cause its the first split > of some file). > > On Thu, Aug 2, 2012 at 1:54 AM, Mohammad Tariq <donta...@gmail.com> wrote: > > Hello list, > > > > I have a flat file in which data is stored as lines of 107 > > bytes each. I need to skip the first 8 lines(as they don't contain any > > valuable info). Thereafter, I have to read each line and extract the > > information from them, but not the line as a whole. Each line is > > composed of several fields without any delimiter between them. For > > example, the first field is of 8 bytes, second of 2 bytes and so on. I > > was trying to reach each line as a Text value, convert it into string > > and using String.subring() method to extract the value of each field. > > But it seems I am not doing things in correct way. Need some > > guidance. Many thanks. > > > > Regards, > > Mohammad Tariq > > > > -- > Harsh J > -- It's just about how deep your longing is!
