[
https://issues.apache.org/jira/browse/HADOOP-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12609404#action_12609404
]
Lukas Vlcek commented on HADOOP-3665:
-------------------------------------
The whole point is that I would like to understand how Reduce job can output a
file without any key values in it. The NullWritable seemed to be an ideal
candidate for this but unfortunately I ran into exceptions when trying it. So I
made a quick and dirty fix which is not meant to be a production ready
(obviously NullWritable should not be special-cased in any way!).
On the other hand there seemed to be some questions which need to be asked and
possible addressed. One of them is that ReflectionUtils is able to call any
constructor after setAccessible is set to true but is this what we really want
for singleton keys? And do we really need singleton keys at all? (I believe the
answer is positive). How about size (length) of key value? Is it allowed to be
zero? And why WritableComparato calls to newInstance method while this causes
issues with any class having non-public constructor?
These were the questions I was trying to acknowledge while learning Hadoop and
trying to experiment with some simple jobs having no Reduce output key...
> WritableComparator newKey() fails for NullWritable
> --------------------------------------------------
>
> Key: HADOOP-3665
> URL: https://issues.apache.org/jira/browse/HADOOP-3665
> Project: Hadoop Core
> Issue Type: Bug
> Components: io
> Affects Versions: 0.16.0, 0.16.1, 0.16.2, 0.16.3, 0.16.4, 0.17.0
> Environment: n/a
> Reporter: Lukas Vlcek
> Priority: Minor
> Fix For: 0.19.0
>
> Attachments: HADOOP-3665.path
>
>
> It is not possible to use NullWritable as a key in order to suppress key
> value in output.
> Syndrome exception:
> Caused by: java.lang.IllegalAccessException: Class
> org.apache.hadoop.io.WritableComparator can not access a member of class
> org.apache.hadoop.io.NullWritable with modifiers "private"
> The problem is that NullWritable is a singleton and does not provide public
> non-parametric constructor. The following code in WritableComparator causes
> the exception: return (WritableComparable)keyClass.newInstance();
> Proposed simple solution is to use ReflectionUtils instead (it requires
> modification as well).
> This issue is probably related to HADOOP-2922
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
