snowloong wrote: > Hi, > I want to share some data structures for the map tasks on a same node(not > through files), I mean, if one map task has already initialized some data > structures (e.g. an array or a list), can other map tasks share these memorys > and directly access them, for I don't want to reinitialize these datas and I > want to save some memory. Can hadoop help me do this?
You can enable jvm reuse across tasks. See mapred.job.reuse.jvm.num.tasks in mapred-default.xml for usage. Then you can cache the data in a static variable in your mapper. - Sharad
