Andrew Schiefelbein created HDFS-12957:
------------------------------------------
Summary: Multi master name node
Key: HDFS-12957
URL: https://issues.apache.org/jira/browse/HDFS-12957
Project: Hadoop HDFS
Issue Type: Wish
Components: namenode
Affects Versions: 3.0.0
Environment: Multi node high availability with high throughput
Reporter: Andrew Schiefelbein
While I do appreciate that the HA and federated setups do take care of some of
the issues running in a multi node distributed fault tolerant way I believe
that a singe name node is a bit of a pain point for me.
My wish, and your mileage may vary, is that here could be a name node on each
data node system so that each system could act as a gateway and worker for the
cluster.
In my tests right now I can push nearly 2 million records / second through a
single node instance, but when I bring up 4 more nodes it doesn't go 4x faster
as it has to retrieve from the name node and go through it as a gateway to the
cluster. For speed and resiliency if this could be spread out among n number
of nodes and put behind a load balancer it would be the ideal solution for
distributed resilient and high throughput shared storage.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
