There are no piece of code in Cassandra that would remove this folder. You
should start looking elsewhere, like other people mentioned (chef, ansible
and so on), good luck
On 8 February 2018 at 22:54, test user <casst...@gmail.com> wrote:
> Does anyone have more inputs on the missing hints folder, rather why it
> gets deleted.
> Has anyone run into this scenario before?
> Cassandra User
> On Wed, Feb 7, 2018 at 9:21 PM, test user <casst...@gmail.com> wrote:
>> The problem is even though, I get that O_RDONLY WARN message, if I try to
>> navigate to the path where the hints folder is stored, the folder is not
>> I cannot check the permissions on that folder, its already missing, got
>> deleted somehow.
>> I believe everything runs as the root user.
>> I do see a lot of sstable activity performed by MemTableFlushWriter
>> (ColumnFamilyStore), CompactionExecutor, PerDiskFlushWriter (MemTable)
>> before and after this WARN message.
>> It is not a space issue, I checked that already.
>> On Wed, Feb 7, 2018 at 3:49 PM, Nate McCall <n...@thelastpickle.com>
>>> The environment is built using established images for Cassandra 3.10.
>>>> Unfortunately the debug log does not indicate any errors before I start
>>>> seeing the WARN for missing hints folder. I understand that hints file will
>>>> be deleted after replay is complete, but not sure of the root cause of why
>>>> the hints folder is getting deleted.
>>>> When I look at the nodetool status or nodetool ring - it indicates that
>>>> all nodes are up and running in normal state, no node went down. Also, I do
>>>> not see anything the debug logs indicating that a node went down. In such a
>>>> scenario, I am not sure why would HintsWriterExecutor would get triggered.
>>> That error code (O_RDONLY) in the log message indicates that the hints
>>> folder has had its permission bits set to read only.
>>> We've had several issues with some of the tools doing this type of thing
>>> when they are run as the root user. Is this specific node one on which you
>>> use any of the tools like sstableloader or similar? If so, are you running
>>> them as root?
>>> Another thought - if it is on a different partition than the data
>>> directory, is there free space left on the underlying device holding:
>>> Nate McCall
>>> Wellington, NZ
>>> Apache Cassandra Consulting