[
https://issues.apache.org/jira/browse/HDDS-672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16657006#comment-16657006
]
Anu Engineer edited comment on HDDS-672 at 10/19/18 4:12 PM:
-------------------------------------------------------------
[~elek] The issue is as follows:
# Ozone FS expects to find a configuration address called, *Ozone.om.address*.
# Spark, Hive, Yarn all these applications are familiar with HDCFs and they
will pass Hadoop Configuration to the file system.
# Hadoop Config will *not* have this key defined since we define
*ozone.om.address* in ozone-site.xml.
# One way to fix is this to add the ozone.om.address to hdfs--site.xml or
core–site.xml.
# Another way to fix this is to understand that the older applications are not
aware of ozone, and hence take the configuration and re-initialize
configuration to include ozone values also. That is what this patch does.
# Why this patch is not correct-- This does something that is surprising to
the user, the user passes a config and we are second-guessing the user, Which
IMHO is the wrong thing to do.
# We are making this modification so that applications like spark can work
against ozone without any modifications.
Please let me know if this makes sense.
Ps. --jars is a red herring, it has nothing to do with this issue.
was (Author: anu):
[~elek] The issue is as follows:
# Ozone FS expects to find a configuration address called, *Ozone.om.address*.
# Spark, Hive, Yarn all these applications are familiar with HDCFs and they
will pass Hadoop Configuration to the file system.
# Hadoop Config will *not* have this key defined since we define
*ozone.om.address* in ozone-site.xml.
# One way to fix is this to add the ozone.om.address to hdfs--site.xml or
core–site.xml.
# Another way to fix this is to understand that the older applications are not
aware of ozone, and hence take the configuration and re-initialize
configuration to include ozone values also. That is what this patch does.
# Why this patch is not correct-- This does something that is surprising to
the user, the user passes a config and we are second-guessing the user, Which
IMHO is the wrong thing to do.
# We are making this modification so that applications like spark can work
against ozone without any modifications.
Please let me know if this makes sense.
> Spark shell throws OzoneFileSystem not found
> --------------------------------------------
>
> Key: HDDS-672
> URL: https://issues.apache.org/jira/browse/HDDS-672
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Affects Versions: 0.2.1
> Reporter: Namit Maheshwari
> Assignee: Anu Engineer
> Priority: Major
> Labels: app-compat
> Attachments: HDDS-672.001.patch, HDDS-672.002.patch
>
>
> Spark shell throws OzoneFileSystem not found, if the ozone jars are not
> specified in the --jars options
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
