[
https://issues.apache.org/jira/browse/HBASE-1296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-1296:
-------------------------
Comment: was deleted
(was: This happened to me about two weeks ago. Namenode was uncontactable -- so
it appeared -- and it was all deleting all the time.)
> Bulk upload can overwhelm a datanode filling its disk because HDFS deletes lag
> ------------------------------------------------------------------------------
>
> Key: HBASE-1296
> URL: https://issues.apache.org/jira/browse/HBASE-1296
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
>
> From a private mail with Jérôme Thièvre:
> "Two times I got the same kind of problems described by schubert zhang in the
> thread *HDFS unbalance issue. (HBase over HDFS). *The first time, I found one
> of my regionserver with a full disk, whereas the others were at 10% of their
> capacity. I saw a lot of move file command at hadoop level from the full
> regionserver to others, but after check on the hdfs, I can see that hadoop
> copies the file but fails to delete it. As the regionserver disk was full,
> it didn't work anymore, even after hadoop and hbase restart. I had to delete
> all the data.
> "The second time this problem occurs I stopped the row insertion process
> before the disk was full. After hadoop and hbase restart, hadoop has deleted
> the files and the system was operational.
> It seems that in some cases where the system is heavily loaded with
> continuous writes and compactions, hadoop can't remove files."
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
