limit map tasks for load function

2013-11-03 Thread John
Hi, is it possible to limit the number of map slots used for the load function? For example I have 5 nodes with 10 map slots (each node has 2 slots for every cpu) I want only one map task for every node. Is there a way to set this only for the load function? I know there is a option called

Re: limit map tasks for load function

2013-11-03 Thread Pradeep Gollakota
I think you’re misunderstanding how HBaseStorage works. HBaseStorage uses the HBaseInputFormat underneath the hood. The number of map tasks that are spawned is dependent on the number of regions you have. The map tasks are spawned such that the tasks are local to the regions they’re reading from.