youssefbennour opened a new issue, #13076:
URL: https://github.com/apache/trafficserver/issues/13076
## Description
When deploying ATS using HashiCorp Nomad's Docker driver, the cache
consistently fails to initialize, rendering caching entirely non-functional.
The same setup works perfectly in a local Docker Compose environment, where
cache initialization succeeds and responses are cached as expected.
## Logs
```
[Apr 8 09:40:05.967] [ET_AIO 7:26] WARNING: cache disk operation failed
READ 0 1
[Apr 8 09:40:05.967] [ET_NET 2] WARNING: could not read disk header for
disk /mount/disk_1/cache.db: declaring disk bad
[Apr 8 09:40:05.967] [ET_NET 2] WARNING: failed operation: READ (opcode=1),
span: /mount/disk_1/cache.db (fd=26)
[Apr 8 09:40:05.967] [ET_NET 2] WARNING: failed to initialize the cache for
http: cache disabled
```
## Steps to Reproduce
1. Deploy ATS using Nomad's Docker driver with volumes enabled on the client
2. Configure a cache span pointing to a mounted volume (e.g.
`/mount/disk_1/cache.db`)
3. Start the Nomad job — ATS starts but cache fails to initialize
## What I've Tried
### 1. Mounted host volume via Nomad Docker driver
Configured a persistent cache volume by mounting a host path into the
container through
Nomad's `config.volumes`. Initially resulted in a permission denied error.
After
correcting the ownership and permissions on the host directory (chown/chmod
on the
host path to match the ATS process user inside the container), the
permission error
was resolved — but the original cache initialization error reappeared,
suggesting
the issue is deeper than filesystem permissions.
Notably, `cache.db` **is being created** on disk, so ATS does reach the
point of
creating the file, but fails when attempting to read the disk header from it.
### 2. Adjusted filesystem permissions on the mounted volume
Applied broad ownership and permission changes on the host mount path to
rule out any
access restriction:
```bash
chown -R nobody /mount/disk_1
chmod -R 777 /mount/disk_1
```
The same cache initialization error persisted regardless, which strongly
suggests that
**permissions are not the root cause** of this issue.
### 3. Removed volume mount entirely (ephemeral local storage)
To isolate the problem, I removed the volume mount altogether and pointed
the cache
to a path local to the container, with no host mount involved. The cache
initialization
error persisted identically, which suggests the issue is **not related to
volume
mounting or host-level permissions**, but rather something in how ATS
interacts with
the filesystem in a Nomad-managed container environment.
### 4. Ran the container in privileged mode
Enabled privileged mode in the Nomad Docker driver config to rule out any
capability-related restrictions. The error remained unchanged.
## Questions
- Is there a way to enable more verbose diagnostics around the `READ`
operation that
fails at cache startup?
- Are there any recommended `records.config` or storage configuration
adjustments
for containerized environments?
- Do you have any other suggestion to further debug the issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
