Ivan Veselovsky created IGNITE-345:
--------------------------------------
Summary: documentation: better describe secondary filesystem
configuration
Key: IGNITE-345
URL: https://issues.apache.org/jira/browse/IGNITE-345
Project: Ignite
Issue Type: Wish
Components: documentation
Affects Versions: sprint-2
Reporter: Ivan Veselovsky
Currently the documentation on how to configure the secondary HDFS
(http://hadoop.gridgain.org/latest/GGFS) is not quite clear.
1) The key point there is that this mode requires two different Hadoop
configurations (and, perhaps, installations): Hadoop server should have
ordinary hdfs filesystem configured, like hdfs://aaa:9000/ , while Hadoop
client should use igfs:// filesystem , that, in turn, runs upon the hdfs
filesystem configured in the Hadoop server. This way, the Hadoop client and
Hadoop server have its own core-site.xml each. In case of local deployment
(everything on one machine) this may be done as 2 different Hadoop
configurations or 2 different copies of Hadoop installation.
2) the second point that is quite unclear is the purpose of "cfgPath"
constructor attribute (see below) of the Ignite node configuration. Actually it
supposed to mean the path to server Hadoop configuration (core-site.xml), but
actually it is optional (currently it is impossible to omit this paramater, but
it is possible to use empty config file.) This config is used (1) to get the
secondary hdfs URI, if the one is not given directly as "uri" paramater , and
(2) to get additional parameters of that filesystem, if any. The difficulty
there is that this config file may actually reside on a different node (host),
so it may be tricky to provide local URL to this configuration. (We'll discuss
possibility to omit this paramater in configuration at all.)
<property name="igfsConfiguration">
<list>
<bean class="org.apache.ignite.configuration.IgfsConfiguration"
parent="igfsCfgBase">
<property name="name" value="igfs"/>
....
<property name="defaultMode" value="PROXY"/>
<property name="secondaryFileSystem">
<bean
class="org.apache.ignite.internal.igfs.hadoop.IgfsHadoopFileSystemWrapper">
<constructor-arg name="uri"
value="hdfs://172.17.0.4:9000"/>
<constructor-arg name="cfgPath"
value="....../etc/hadoop/core-site.xml"/>
</bean>
</property>
</bean>
</list>
</property>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
