Chao Sun created HDFS-13923:
-------------------------------
Summary: Add a configuration to turn on/off observer reads
Key: HDFS-13923
URL: https://issues.apache.org/jira/browse/HDFS-13923
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Chao Sun
In some situations having a config to turn on/off observer reads dynamically
may be useful. For instance, some applications may be sensitive for data
freshness and want to always reach directly to active NN. In a more complex
scenario, services such as Presto may want to apply observer reads for
different types of queries. In this case, simply change
{{dfs.client.failover.proxy.provider.<nameservice>}} may not be enough, since
with FileSystem cache (which is usually turned on) will ignore the change and
still use the same FileSystem object.
Here I'm proposing to add a flag in {{HdfsClientConfigKeys}}, such as
{{dfs.client.observer.reads.enabled}}, that can be used to dynamically turn
on/off observer reads. The FileSystem cache key should also take account of
this flag in its {{hashCode}} and {{equals}} impl, so that different FileSystem
objects will be used depending on the flag.
cc [~shv], [~xkrogen], [~vagarychen], [~zero45] for discussion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
