Two options. Depending on the size of the array, you could set your
floats as a comma separated list of strings that you send through the
configuration as a variable or in the configure method of the mapper you
can read in the values from the DFS and store them in a class variable.
This would be read once per map task, not once per map entry. A third
option is using a custom MapRunner but I think that is overkill for this.
Dennis Kubes
Ilya Vishnevsky wrote:
Well, as far as I understand the Properties object is packed into xml
file before to be sent to the Mapper, so it can contain only strings.
That's reasonable.
But I need an array (or collection) of floats to be passed to each
Mapper. Of course I can save it to DFS, so that Mapper could read it.
But won't it slow down work of the Hadoop if every Mapper reads the same
file from DFS? Maybe it's possible to attach this array to the job jar
in any way?
-----Original Message-----
From: Andrzej Bialecki [mailto:[EMAIL PROTECTED]
Sent: Wednesday, May 30, 2007 8:54 PM
To: [email protected]
Subject: Re: What objects can be passed to Mapper and Reducer classes
using Configuration?
Ilya Vishnevsky wrote:
There is method Configuration.setObject(String name, Object value). I
tried values of different classes and found that only Strings are
passed
normally. Other objects turn out to be null when Mapper or Reducer
tries
to get them using getConf().getObject(String name).
Is there possibility to pass a Set for example?
The short answer is that this method is deprecated and will be removed.
Don't use it.
Longer answer - see http://issues.apache.org/jira/browse/HADOOP-1343
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
