[
https://issues.apache.org/jira/browse/HBASE-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phil Yang updated HBASE-16583:
------------------------------
Description:
Staged Event-Driven Architecture (SEDA) splits request-handling logic into
several stages, each stage is executed in a thread pool and they are connected
by queues.
Currently, in region server we use a thread pool to handle requests from
client. The number of handlers is configurable, reading and writing use
different pools. The current architecture has two limitations:
Performance:
Different part of the handling path has different bottleneck. For example,
accessing MemStore and cache mainly consumes CPU but accessing HDFS mainly
consumes network/disk IO. If we use SEDA and split them into two different
stages, we can use different numbers for two pools according to the
CPU/disk/network performance case by case.
Availability:
HBASE-16388 described a scene that if the client use a thread pool and use
blocking methods to access region servers, only one slow server may exhaust
most of threads of the client. For HBase, we are the client and HDFS datanodes
are the servers. A slow datanode may exhaust most of handlers. The best way to
resolve this issue is make HDFS requests non-blocking, which is exactly what
SEDA does.
was:
Staged Event-Driven Architecture (SEDA) splits request-handling logic into
several stages, each stage is executed in a thread pool and they are connected
by queues.
Currently, in region server we use a thread pool to handle requests from
client. The number of handlers is configurable, reading and writing use
different pools. The current architecture has two limitations:
Performance:
Different part of the handling path has different bottleneck. For example,
accessing MemStore and cache mainly consumes CPU but accessing HDFS mainly
consumes network/disk IO. If we use SEDA and split them into two different
stages, we can use different numbers for two pools according to the
CPU/disk/network performance case by case.
Availability:
HBASE-16388 described a scene that if the client use a thread pool and use
blocking methods to access region server, only one slow server may exhaust most
of threads of clients. For HBase, we are clients and HDFS datanodes are the
servers. A slow datanode may exhaust most of handlers. The best way to resolve
this issue is make HDFS requests non-blocking, which is exactly what SEDA does.
> Staged Event-Driven Architecture
> --------------------------------
>
> Key: HBASE-16583
> URL: https://issues.apache.org/jira/browse/HBASE-16583
> Project: HBase
> Issue Type: Umbrella
> Reporter: Phil Yang
>
> Staged Event-Driven Architecture (SEDA) splits request-handling logic into
> several stages, each stage is executed in a thread pool and they are
> connected by queues.
> Currently, in region server we use a thread pool to handle requests from
> client. The number of handlers is configurable, reading and writing use
> different pools. The current architecture has two limitations:
> Performance:
> Different part of the handling path has different bottleneck. For example,
> accessing MemStore and cache mainly consumes CPU but accessing HDFS mainly
> consumes network/disk IO. If we use SEDA and split them into two different
> stages, we can use different numbers for two pools according to the
> CPU/disk/network performance case by case.
> Availability:
> HBASE-16388 described a scene that if the client use a thread pool and use
> blocking methods to access region servers, only one slow server may exhaust
> most of threads of the client. For HBase, we are the client and HDFS
> datanodes are the servers. A slow datanode may exhaust most of handlers. The
> best way to resolve this issue is make HDFS requests non-blocking, which is
> exactly what SEDA does.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
