[
https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16720914#comment-16720914
]
Brahma Reddy Battula edited comment on HDFS-12943 at 12/14/18 4:33 AM:
-----------------------------------------------------------------------
Thanks all for great work here.
I think,write requests can be degraded..? As they also contains some read
requests like getFileinfo(),getServerDefaults() ...(getHAServiceState() is
newly added) .
Just I had checked for mkdir perf,it's like below.
* i) getHAServiceState() took 2+ sec ( 3 getHAServiceState() + 2 getFileInfo()
+ 1 mkdirs = 6 calls)
* ii) Every second request is getting timedout[1] and rpc call is getting
skipped from observer.( 7 getHAServiceState() + 4 getFileInfo() + 1 mkdirs =
12 calls).Here two getFileInfo() skipped from observer hence it's success with
Active.
{noformat}
time hdfs --loglevel debug dfs
-Ddfs.client.failover.proxy.provider.hacluster=org.apache.hadoop.hdfs.server.namenode.ha.ObserverReadProxyProvider
-mkdir /TestsORF1
real 0m4.314s
user 0m3.668s
sys 0m0.272s
time hdfs --loglevel debug dfs
-Ddfs.client.failover.proxy.provider.hacluster=org.apache.hadoop.hdfs.server.namenode.ha.ObserverReadProxyProvider
-mkdir /TestsORF2
real 0m22.238s
user 0m3.800s
sys 0m0.248s
{noformat}
*without ObserverReadProxyProvider ( 2 getFileInfo() + 1 mkdirs() = 3 Calls)*
{noformat}
time ./hdfs --loglevel debug dfs -mkdir /TestsCFP
real 0m2.105s
user 0m3.768s
sys 0m0.592s
{noformat}
*Please correct me if I am missing anyting.*
timedout[1],Every second write request I am getting following, did I miss
something here,these calls are skipped from observer.
{noformat}
2018-12-14 11:21:45,312 DEBUG ipc.Client: closing ipc connection to
vm1/10.*.*.*:65110: 10000 millis timeout while waiting for channel to be ready
for read. ch : java.nio.channels.SocketChannel[connected local=/10.*.*.*:58409
remote=vm1/10.*.*.*:65110]
java.net.SocketTimeoutException: 10000 millis timeout while waiting for channel
to be ready for read. ch : java.nio.channels.SocketChannel[connected
local=/10.*.*.*:58409 remote=vm1/10.*.*.*:65110]
at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164)
at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
at java.io.FilterInputStream.read(FilterInputStream.java:83)
at java.io.FilterInputStream.read(FilterInputStream.java:83)
at
org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:567)
at java.io.DataInputStream.readInt(DataInputStream.java:387)
at org.apache.hadoop.ipc.Client$IpcStreams.readResponse(Client.java:1849)
at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1183)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079)
2018-12-14 11:21:45,313 DEBUG ipc.Client: IPC Client (1006094903) connection to
vm1/10.*.*.*:65110 from brahma: closed{noformat}
was (Author: brahmareddy):
Thanks all for great work here.
I think,write requests can be degraded..? As they also contains some read
requests like getFileinfo(),getServerDefaults() ...(getHAServiceState() is
newly added) .
Just I had checked for mkdir perf,it's like below.
* i) getHAServiceState() took 2+ sec ( 3 getHAServiceState() + 2 getFileInfo()
+ 1 mkdirs = 6 calls)
* ii) Every second request is getting timedout[1] and rpc call is getting
skipped from observer.( 7 getHAServiceState() + 4 getFileInfo() + 1 mkdirs =
12 calls).Here two getFileInfo() skipped from observer hence it's success with
Active.
{noformat}
time hdfs --loglevel debug dfs
-Ddfs.client.failover.proxy.provider.hacluster=org.apache.hadoop.hdfs.server.namenode.ha.ObserverReadProxyProvider
-mkdir /TestsORF1
real 0m4.314s
user 0m3.668s
sys 0m0.272s
time hdfs --loglevel debug dfs
-Ddfs.client.failover.proxy.provider.hacluster=org.apache.hadoop.hdfs.server.namenode.ha.ObserverReadProxyProvider
-mkdir /TestsORF2
real 0m22.238s
user 0m3.800s
sys 0m0.248s
{noformat}
without ObserverReadProxyProvider ( 2 getFileInfo() + 1 mkdirs() = 3 Calls)
{noformat}
time ./hdfs --loglevel debug dfs -mkdir /TestsCFP
real 0m2.105s
user 0m3.768s
sys 0m0.592s
{noformat}
*Please correct me if I am missing anyting.*
timedout[1],Every second write request I am getting following, did I miss
something here,these calls are skipped from observer.
{noformat}
2018-12-14 11:21:45,312 DEBUG ipc.Client: closing ipc connection to
vm1/10.*.*.*:65110: 10000 millis timeout while waiting for channel to be ready
for read. ch : java.nio.channels.SocketChannel[connected local=/10.*.*.*:58409
remote=vm1/10.*.*.*:65110]
java.net.SocketTimeoutException: 10000 millis timeout while waiting for channel
to be ready for read. ch : java.nio.channels.SocketChannel[connected
local=/10.*.*.*:58409 remote=vm1/10.*.*.*:65110]
at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164)
at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
at java.io.FilterInputStream.read(FilterInputStream.java:83)
at java.io.FilterInputStream.read(FilterInputStream.java:83)
at
org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:567)
at java.io.DataInputStream.readInt(DataInputStream.java:387)
at org.apache.hadoop.ipc.Client$IpcStreams.readResponse(Client.java:1849)
at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1183)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079)
2018-12-14 11:21:45,313 DEBUG ipc.Client: IPC Client (1006094903) connection to
vm1/10.*.*.*:65110 from brahma: closed{noformat}
> Consistent Reads from Standby Node
> ----------------------------------
>
> Key: HDFS-12943
> URL: https://issues.apache.org/jira/browse/HDFS-12943
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: hdfs
> Reporter: Konstantin Shvachko
> Priority: Major
> Attachments: ConsistentReadsFromStandbyNode.pdf,
> ConsistentReadsFromStandbyNode.pdf, HDFS-12943-001.patch,
> TestPlan-ConsistentReadsFromStandbyNode.pdf
>
>
> StandbyNode in HDFS is a replica of the active NameNode. The states of the
> NameNodes are coordinated via the journal. It is natural to consider
> StandbyNode as a read-only replica. As with any replicated distributed system
> the problem of stale reads should be resolved. Our main goal is to provide
> reads from standby in a consistent way in order to enable a wide range of
> existing applications running on top of HDFS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
