[ https://issues.apache.org/jira/browse/HDFS-13923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621294#comment-16621294 ]
Konstantin Shvachko commented on HDFS-13923: -------------------------------------------- Hey [~csun], some thoughts. * If we want to turn off observer reads globally we can simply kill ObserverNode (or transition it to SBN) and restart it (transition back) when we are ready to enable observer reads. * We can configure client with {{ConfiguredProxyProvider}} in order to make client not ever go to Observer. (Just as you mentioned). * Here you are talking, I believe, about a way of telling the client dynamically not to use ObserverNode. Do you? I don't see how configuration parameter can help with that. If the client is cached, then whatever value of a config variable stored in it during the initial instantiation will still be there, even if you updated the config file. So it looks more like you want a flag somewhere in ORPP, which tells the invoker to call ActiveNN on reads no matter what, and the flag should be updatable dynamically. Not sure I like the idea, because it sounds like it will require adding a new FileSystem method turning of observer reads, which exposes implementation details with Observer reads to the user space. I would rather use options like {{fs.hdfs.impl.disable.cache = true}} or spawn a different JVM. > Add a configuration to turn on/off observer reads > ------------------------------------------------- > > Key: HDFS-13923 > URL: https://issues.apache.org/jira/browse/HDFS-13923 > Project: Hadoop HDFS > Issue Type: Sub-task > Reporter: Chao Sun > Priority: Major > > In some situations having a config to turn on/off observer reads dynamically > may be useful. For instance, some applications may be sensitive for data > freshness and want to always reach directly to active NN. In a more complex > scenario, services such as Presto may want to apply observer reads for > different types of queries. In this case, simply change > {{dfs.client.failover.proxy.provider.<nameservice>}} may not be enough, since > with FileSystem cache (which is usually turned on) will ignore the change and > still use the same FileSystem object. > Here I'm proposing to add a flag in {{HdfsClientConfigKeys}}, such as > {{dfs.client.observer.reads.enabled}}, that can be used to dynamically turn > on/off observer reads. The FileSystem cache key should also take account of > this flag in its {{hashCode}} and {{equals}} impl, so that different > FileSystem objects will be used depending on the flag. > > cc [~shv], [~xkrogen], [~vagarychen], [~zero45] for discussion. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org