[
https://issues.apache.org/jira/browse/HBASE-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell resolved HBASE-1477.
-----------------------------------
Resolution: Invalid
Resolving as invalid as the issue lacks focus
> Contention on META stalls mapreduce job execution
> -------------------------------------------------
>
> Key: HBASE-1477
> URL: https://issues.apache.org/jira/browse/HBASE-1477
> Project: HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
>
> From Jeremy Pinkham up on hbase-users@:
> bq. A typical mapper in the job takes several minutes, how many minutes
> depends on whether I use the the region partitioner and how many I let run
> concurrently... it's been anywhere from 2 minutes with no partitioner and
> small concurrency (5 mappers) to 8 minutes with the region partitioner and
> high concurrency (150 mappers). This seems to directly correlate with how
> long it takes to do a simple count of .META. while each job is running (2
> seconds to 1 minute)
> bq. I was able to get past this issue affecting my data load by reorganizing
> some of my workflow and data structures to force the ordering of keys without
> the region partitioner. Those changes appear to have side stepped the
> problem for me as I can now load from 100+ mappers without seeing the
> degradation that I was seeing with 40 when using the partitioner (and getting
> some sweet numbers in the requests column of the UI). It's still an
> interesting scaling situation with the region partitioner, but I'm good to go
> without it.
> I have seen this also in the form of freezing of master UI during high load,
> where the UI comes back as soon as load is reduced. When I thread dump it
> looks like all IPC handlers on the region server hosting .META. are busy.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
