[
https://issues.apache.org/jira/browse/HDFS-15714?focusedWorklogId=559160&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-559160
]
ASF GitHub Bot logged work on HDFS-15714:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 01/Mar/21 02:54
Start Date: 01/Mar/21 02:54
Worklog Time Spent: 10m
Work Description: Zhangshunyu commented on pull request #2655:
URL: https://github.com/apache/hadoop/pull/2655#issuecomment-787600101
@PHILO-HE Thanks for the reply. Please find the error msg:
```
ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in
creating BlockSender: {}
org.apache.hadoop.hdfs.server.datanode.ReplicaNotFoundException: Replica not
found for BP-1079229821-192.168.0.156-1614366591560:blk_1073791203_50379. The
block may have been removed recently by the balancer or by intentionally
reducing the replication factor. This condition is usually harmless. To be
certain, please check the preceding datanode log messages for signs of a more
serious issue.
```
we use readOnly mount mode, the data is stored on s3 store, after mount we
read the data use readthrough mode, we find that some times the datanode will
throw this exception, it seems the read op request send to the wrong datanode?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 559160)
Time Spent: 50m (was: 40m)
> HDFS Provided Storage Read/Write Mount Support On-the-fly
> ---------------------------------------------------------
>
> Key: HDFS-15714
> URL: https://issues.apache.org/jira/browse/HDFS-15714
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode, namenode
> Affects Versions: 3.4.0
> Reporter: Feilong He
> Assignee: Feilong He
> Priority: Major
> Labels: pull-request-available
> Attachments: HDFS-15714-01.patch,
> HDFS_Provided_Storage_Design-V1.pdf, HDFS_Provided_Storage_Performance-V1.pdf
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> HDFS Provided Storage (PS) is a feature to tier HDFS over other file systems.
> In HDFS-9806, PROVIDED storage type was introduced to HDFS. Through
> configuring external storage with PROVIDED tag for DataNode, user can enable
> application to access data stored externally from HDFS side. However, there
> are two issues need to be addressed. Firstly, mounting external storage
> on-the-fly, namely dynamic mount, is lacking. It is necessary to get it
> supported to flexibly combine HDFS with an external storage at runtime.
> Secondly, PS write is not supported by current HDFS. But in real
> applications, it is common to transfer data bi-directionally for read/write
> between HDFS and external storage.
> Through this JIRA, we are presenting our work for PS write support and
> dynamic mount support for both read & write. Please note in the community
> several JIRAs have been filed for these topics. Our work is based on these
> previous community work, with new design & implementation to support called
> writeBack mount and enable admin to add any mount on-the-fly. We appreciate
> those folks in the community for their great contribution! See their pending
> JIRAs: HDFS-14805 & HDFS-12090.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
