Christopher Tubbs created ACCUMULO-4536:
-------------------------------------------
Summary: Infinite loop creating empty WAL files when disk space is
low
Key: ACCUMULO-4536
URL: https://issues.apache.org/jira/browse/ACCUMULO-4536
Project: Accumulo
Issue Type: Bug
Components: tserver
Affects Versions: 1.6.6
Reporter: Christopher Tubbs
Priority: Minor
Saw this on 1.6.6 with a small disk for testing (32GB disk). The default walog
size is around 1GB, and only 3.4GB were left available on each data node.
The namenode reported that no data nodes had space available when trying to
write the first block, so the tserver failed to write the file. It kept
retrying, resulting in the namenode filling up with thousands of zero-length
WAL files.
The fix was to lower the {{tserver.walog.max.size}} to {{100M}}. Another
solution would be to use a larger disk.
The infinite loop problem, constantly creating new empty WAL files is still a
problem, but it should only happen when low on disk space, which is likely
going to cause other, more serious problems... and could be avoided with good
system monitoring.
I have not tested on versions newer than 1.6.6, but I imagine it's still a
problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
