[
https://issues.apache.org/jira/browse/OOZIE-913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425196#comment-13425196
]
Robert Kanter commented on OOZIE-913:
-------------------------------------
My understanding of OOZIE-477 is that you can check with a list of supported
filesystem types (i.e. schemes), so that you can use other filesystems besides
hdfs, as long as its on the list. But a FileSystem object isn't just a scheme
(e.g. hdfs://), its also an authority (e.g. hdfs://host1:port). So, in my
previous example, its true that they are both hdfs filesystems, but to access
both of them, you'd need two FileSystem objects. Correct me if I'm wrong, but
it doesn't have a list of actual existing filesystems; only the supported
types. We also don't necessarily want to initialize FS objects for each
supported fs (or actual existing filesystem) because the action might not be
using all of them.
If we wanted to minimize the number of times FsAE initializes FS objects, we
could have it read through the XML before doing any actual FS actions to build
a queue of FS actions and a list of all FS's used by these actions so we can
initialize each FS only once and then use those FS objects to execute the queue
of actions; this would require some fundamental changes in how the FsAE works,
and is maybe better off as its own JIRA?
> Add Name Node, job-xml, and configuration Elements to FS action
> ---------------------------------------------------------------
>
> Key: OOZIE-913
> URL: https://issues.apache.org/jira/browse/OOZIE-913
> Project: Oozie
> Issue Type: Improvement
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Priority: Minor
> Attachments: OOZIE-913-v2.patch, OOZIE-913.patch
>
>
> Adding a name node Element to the FS action will allow users to shorten FS
> actions and avoid specifying the name node (hdfs://HOST:PORT) multiple times.
> We can also add job-xml and configuration elements to allow users to set
> properties when the FS instance is created.
> e.g.
> {code}
> <action name="fs-node">
> <fs>
> <name-node>hdfs://host:port</name-node>
> <job-xml>fs-info.xml</job-xml>
> <configuration>
> <property>
> <name>some.property</name>
> <value>some.value</value>
> </property>
> </configuration>
> <mkdir path="/user/${wf:user()}/output-data/1"/>
> <mkdir path="/user/${wf:user()}/output-data/2"/>
> </fs>
> <ok to="end"/>
> <error to="fail"/>
> </action>
> {code}
> This can then also leverage the global section from OOZIE-874 automatically.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
