[
https://issues.apache.org/jira/browse/HDFS-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uma Maheswara Rao G updated HDFS-15533:
---------------------------------------
Description:
I have been working on a thought from last week is that, we wanted to provide
DFS compatible APIs with mount functionality. So, that existing DFS
applications can work with out class cast issues.
When we tested with other components like Hive and HBase, I noticed some
classcast issues.
{code:java}
HBase example:
java.lang.ClassCastException:
org.apache.hadoop.fs.viewfs.ViewFileSystemOverloadScheme cannot be cast to
org.apache.hadoop.hdfs.DistributedFileSystemjava.lang.ClassCastException:
org.apache.hadoop.fs.viewfs.ViewFileSystemOverloadScheme cannot be cast to
org.apache.hadoop.hdfs.DistributedFileSystem at
org.apache.hadoop.hbase.util.FSUtils.getDFSHedgedReadMetrics(FSUtils.java:1748)
at
org.apache.hadoop.hbase.regionserver.MetricsRegionServerWrapperImpl.<init>(MetricsRegionServerWrapperImpl.java:146)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1594)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1001)
at java.lang.Thread.run(Thread.java:748){code}
{code:java}
Hive:
|io.AcidUtils|: Failed to get files with ID; using regular API: Only supported
for DFS; got class
org.apache.hadoop.fs.viewfs.ViewFileSystemOverloadScheme{code}
SO, here the implementation details are like follows:
We extended DistributedFileSystem and created a class called "
ViewDistributedFileSystem"
This vfs=ViewFirstibutedFileSystem, try to initialize
ViewFileSystemOverloadScheme. If success call will delegate to vfs. If fails
to initialize due to no mount points, or other errors, it will just fallback to
regular DFS init. If users does not configure any mount, system will behave
exactly like today's DFS. If there are mount points, vfs functionality will
come under DFS.
I have a patch and will post it in some time.
was:
I have been working on a thought from last week is that, we wanted to provide
DFS compatible APIs with mount functionality. So, that existing DFS
applications can work with out class cast issues.
When we tested with other components like Hive and HBase, I noticed some
classcast issues.
{code:java}
HBase example:
java.lang.ClassCastException:
org.apache.hadoop.fs.viewfs.ViewFileSystemOverloadScheme cannot be cast to
org.apache.hadoop.hdfs.DistributedFileSystemjava.lang.ClassCastException:
org.apache.hadoop.fs.viewfs.ViewFileSystemOverloadScheme cannot be cast to
org.apache.hadoop.hdfs.DistributedFileSystem at
org.apache.hadoop.hbase.util.FSUtils.getDFSHedgedReadMetrics(FSUtils.java:1748)
at
org.apache.hadoop.hbase.regionserver.MetricsRegionServerWrapperImpl.<init>(MetricsRegionServerWrapperImpl.java:146)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1594)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1001)
at java.lang.Thread.run(Thread.java:748){code}
{code:java}
Hive:
|io.AcidUtils|: Failed to get files with ID; using regular API: Only supported
for DFS; got class
org.apache.hadoop.fs.viewfs.ViewFileSystemOverloadScheme{code}
SO, here the implementation details are like follows:
We extended DistributedFileSystem and created a class called "
ViewDistributedFileSystem"
This vfs=ViewFirstibutedFileSystem, try to initialize
ViewFileSystemOverloadScheme. If success call will delegate to vfs. If fails
to initialize due to no mount points, or other errors, it will just fallback to
regular DFS init. If users does not configure any mount, system will behave
exactly like today's DFS. If there are mount points, vfs functionality will
come under DFS.
I will a patch and will post it in some time.
> Provide DFS API compatible class, but use ViewFileSystemOverloadScheme inside
> -----------------------------------------------------------------------------
>
> Key: HDFS-15533
> URL: https://issues.apache.org/jira/browse/HDFS-15533
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: dfs, viewfs
> Affects Versions: 3.4.0
> Reporter: Uma Maheswara Rao G
> Assignee: Uma Maheswara Rao G
> Priority: Major
>
> I have been working on a thought from last week is that, we wanted to provide
> DFS compatible APIs with mount functionality. So, that existing DFS
> applications can work with out class cast issues.
> When we tested with other components like Hive and HBase, I noticed some
> classcast issues.
> {code:java}
> HBase example:
> java.lang.ClassCastException:
> org.apache.hadoop.fs.viewfs.ViewFileSystemOverloadScheme cannot be cast to
> org.apache.hadoop.hdfs.DistributedFileSystemjava.lang.ClassCastException:
> org.apache.hadoop.fs.viewfs.ViewFileSystemOverloadScheme cannot be cast to
> org.apache.hadoop.hdfs.DistributedFileSystem at
> org.apache.hadoop.hbase.util.FSUtils.getDFSHedgedReadMetrics(FSUtils.java:1748)
> at
> org.apache.hadoop.hbase.regionserver.MetricsRegionServerWrapperImpl.<init>(MetricsRegionServerWrapperImpl.java:146)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1594)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1001)
> at java.lang.Thread.run(Thread.java:748){code}
> {code:java}
> Hive:
> |io.AcidUtils|: Failed to get files with ID; using regular API: Only
> supported for DFS; got class
> org.apache.hadoop.fs.viewfs.ViewFileSystemOverloadScheme{code}
> SO, here the implementation details are like follows:
> We extended DistributedFileSystem and created a class called "
> ViewDistributedFileSystem"
> This vfs=ViewFirstibutedFileSystem, try to initialize
> ViewFileSystemOverloadScheme. If success call will delegate to vfs. If fails
> to initialize due to no mount points, or other errors, it will just fallback
> to regular DFS init. If users does not configure any mount, system will
> behave exactly like today's DFS. If there are mount points, vfs functionality
> will come under DFS.
> I have a patch and will post it in some time.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
