Please ignore last one…
Thanks Deepak,
I tried using below but that also not helping in changing output file name…
In Job file
MultipleOutputs.addNamedOutput(job, "clean", AvroKeyOutputFormat.class,
Measurement.class , NullWritable.class);
In Reducer
Setup:
MultipleOutputs<Measurement, NullWritable> mos;
@Override
protected void setup(Context context) throws IOException,
InterruptedException {
mos = new MultipleOutputs<Measurement, NullWritable>(context);
}
And in reducer method
if (isClean) {
mos.write("clean", value.datum(), "clean");
} else {
mos.write("discarded", value.datum(), "discarded");
}
Can you please point on the what I am doing wrong?
Thank you
Ankur
From: Deepak Sharma [mailto:[email protected]]
Sent: Tuesday, January 13, 2015 11:38 AM
To: user; Ankur Jain
Subject: Re: AVRO MultipleOutput
Hi Ankur
Not sure on the avro part , but MR provides addNamedOutputs() that can be set
in Driver class to rename the output files from part-*** to something else.
HTH.
Thanks
Deepak
On Tue, Jan 13, 2015 at 11:29 AM, Ankur Jain
<[email protected]<mailto:[email protected]>> wrote:
Hi Team,
My usecase is to create files with different names.
I tried using AVRO mapreduce api for MultipleOutputs using
org.apache.avro.mapreduce.AvroMultipleOutputs.
I followed Javadoc mentioned for this class and found below issues…
1. We cannot assign AvroMultipleOutputs object to MultipleOutputs.
2. Even if I followed all steps mentioned in Javadoc I am still getting
file with name part-0000
Can someone please guide me on how I can create files with different name using
AVRO mapreduce api?
Any working example will be great help….
I am using HDP 2.2 , avro (1.7.4), JDK 1.7, reading and writing in avro files.
Thanks and Best Regards,
Ankur
Information transmitted by this e-mail is proprietary to YASH Technologies and/
or its Customers and is intended for use only by the individual or entity to
which it is addressed, and may contain information that is privileged,
confidential or exempt from disclosure under applicable law. If you are not the
intended recipient or it appears that this mail has been forwarded to you
without proper authority, you are notified that any use or dissemination of
this information in any manner is strictly prohibited. In such cases, please
notify us immediately at [email protected]<mailto:[email protected]> and delete this
mail from your records.
--
Thanks
Deepak
www.bigdatabig.com<http://www.bigdatabig.com>
www.keosha.net<http://www.keosha.net>
Information transmitted by this e-mail is proprietary to YASH Technologies and/
or its Customers and is intended for use only by the individual or entity to
which it is addressed, and may contain information that is privileged,
confidential or exempt from disclosure under applicable law. If you are not the
intended recipient or it appears that this mail has been forwarded to you
without proper authority, you are notified that any use or dissemination of
this information in any manner is strictly prohibited. In such cases, please
notify us immediately at [email protected] and delete this mail from your records.
