Hi,
WritableComparator, line# 73 (trunk version) is using Class' newInstance()
method which can not work for singletons like NullWritable.
/** Construct a new [EMAIL PROTECTED] WritableComparable} instance. */
public WritableComparable newKey() {
try {
return (WritableComparable)keyClass.newInstance();
} catch (InstantiationException e) {
throw new RuntimeException(e);
} catch (IllegalAccessException e) {
throw new RuntimeException(e);
}
}
Should this be changed to:
/** Construct a new [EMAIL PROTECTED] WritableComparable} instance. */
public WritableComparable newKey() {
try {
if (keyClass instanceof NullWritable) return NullWritable.get(); //
<--- this is new line
return (WritableComparable)keyClass.newInstance();
} catch (InstantiationException e) {
throw new RuntimeException(e);
} catch (IllegalAccessException e) {
throw new RuntimeException(e);
}
}
I didn't have a chance to check all WritableComparable classes whether they
have public constructor (or implicit one) but I am facing an issue with
NullWritable used as a key in reducer and I believe it is caused by the
newInstance() method being called on singleton.
Regards,
Lukas
On Feb 20, 2008 2:56 PM, Lukas Vlcek <[EMAIL PROTECTED]> wrote:
> Owen,
>
> This is still not clear to me. I see the following code in
> TextOutputFormat class:
>
> ...
> public synchronized void write(K key, V value)
> throws IOException {
>
> boolean nullKey = key == null || key instanceof NullWritable;
> boolean nullValue = value == null || value instanceof NullWritable;
> if (nullKey && nullValue) {
> return;
> }
> if (!nullKey) {
> writeObject(key);
> }
> if (!(nullKey || nullValue)) {
> out.write(tab);
> }
> if (!nullValue) {
> writeObject(value);
> }
> out.write(newline);
> }
>
> This seems to me as if the tab is always output if the value is not null
> regardless key being null or not.
> Am I missing something?
>
> Lukas
>
>
> On Feb 19, 2008 11:55 PM, Owen O'Malley <[EMAIL PROTECTED]> wrote:
>
> >
> > On Feb 19, 2008, at 1:52 PM, Lukas Vlcek wrote:
> >
> > > Hi,
> > >
> > > I don't care about key value in the output file. Is there any way
> > > how I can
> > > suppress key in the output?
> > > Is there a way how to tell (Text)OutputFormat not to write key but
> > > value
> > > only? Or can I pass my own implementation of RecordWriter into
> > > FileOutputFormat?
> >
> > The easiest way is to put either null or a NullWritable in for the
> > key coming out of the reduce. The TextOutputFormat will drop the tab
> > character. You can also define your own OutputFormat and encode them
> > as you wish.
> >
> > -- Owen
> >
>
>
>
> --
> http://blog.lukas-vlcek.com/
>
--
http://blog.lukas-vlcek.com/
