Marton Elek created HDDS-4289:
---------------------------------
Summary: Throw exception from hadoop2 filesystem jar in HA
environment
Key: HDDS-4289
URL: https://issues.apache.org/jira/browse/HDDS-4289
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: OM HA
Reporter: Marton Elek
Thanks for Tamas Pleszkan for reporting this problem.
ozone-filesystem-hadoop2 doesn't support OM-HA (today) as the used
Hadoop3OmTransport uses FailoverProxyProvider which is not available in hadoop2.
Long-term we need a custom failover mechanism, but this jira suggests to
improve the error handling. `Hadoop27OmTransportFactory` should throw an
exception if HA is used.
Used command:
{code}
spark-submit --master yarn --deploy-mode client --executor-memory 1g --conf
"spark.yarn.access.hadoopFileSystems=o3fs://bucket.hdfs.ozone1/" --jars
"/opt/cloudera/parcels/CDH-7.1.3-1.cdh7.1.3.p0.4992530/jars/hadoop-ozone-filesystem-hadoop2-0.5.0.7.1.3.0-100.jar"
SparkWordCount.py o3fs://bucket.hdfs.ozone1/words 2
{code}
Current exception:
{code}
Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ozone.om.exceptions.OMNotLeaderException):
OM:om2 is not the leader. Suggested leader is OM:om1.
{code}
Expected exception: Unsupported operation exception with meaningful hint to use
hadoop3 filesystem jar.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
