duncdrum opened a new issue, #16844:
URL: https://github.com/apache/lucene/issues/16844
### Summary
If `write.lock` exists in a lower overlayfs layer, such as a Docker image
layer, `ensureValid()` fails immediately after a successful `obtainLock()`. The
creation time recorded before the channel is opened no longer matches once the
open triggers a copy-up.
### Background
`NativeFSLock.close()` leaves `write.lock` on disk. An image pre-populated
by running an index-writing process at build time therefore contains an empty
`write.lock`. The first writer started from that image then hits this failure.
The creation-time check was introduced in LUCENE-6508 (#7567, commit
`fe6c3dc939`, 2015). The ordering is unchanged in 10.5.1 and `main`:
```java
// obtainFSLock
final FileTime creationTime =
Files.readAttributes(realPath,
BasicFileAttributes.class).creationTime(); // (1)
...
channel = FileChannel.open(realPath, StandardOpenOption.CREATE,
StandardOpenOption.WRITE); // (2)
lock = channel.tryLock();
```
`ensureValid()` later compares the current creation time to the one recorded
at (1). The code comment describes the check as a best-effort test that the
file was not deleted.
### Cause
On overlayfs, opening a lower-layer file for writing copies it up, and the
copy gets a new creation time. Birth time of such a file, via `java.nio`:
```
before open : 2026-10-15T06:05:34.204672547Z
after open(READ) : 2026-10-15T06:05:34.204672547Z
after open(WRITE) : 2026-10-15T06:05:35.216474881Z
```
So (2) changes the value recorded at (1).
### Reproduction
Docker Desktop, containerd overlayfs snapshotter, `lucene-core-10.5.1`,
Temurin 21.
```java
try (Directory d = FSDirectory.open(Path.of(args[0]));
Lock l = d.obtainLock("write.lock")) {
l.ensureValid();
}
```
```dockerfile
FROM <image with JDK 21 and lucene-core-10.5.1.jar>
COPY Probe.java /probe/
# leaves an empty /probe/stale/write.lock in this layer
RUN
["java","-cp","lucene-core-10.5.1.jar","/probe/Probe.java","/probe/stale"]
```
Running the same command in a container started from this image, against
`/probe/stale`:
```
org.apache.lucene.store.AlreadyClosedException: Underlying file changed by
an external force
at 2026-10-15T06:05:15.380809178Z,
(lock=NativeFSLock(path=/probe/stale/write.lock,...,
creationTime=2026-10-15T06:05:14.635552844Z))
at
...NativeFSLockFactory$NativeFSLock.ensureValid(NativeFSLockFactory.java:179)
```
A fresh directory with no existing `write.lock` succeeds. Once the file is
in the container's upper layer, subsequent runs also succeed.
### Possible change
Record the creation time after the channel is opened and the lock is held:
```java
channel = FileChannel.open(realPath, StandardOpenOption.CREATE,
StandardOpenOption.WRITE);
lock = channel.tryLock();
if (lock != null) {
FileTime creationTime =
Files.readAttributes(realPath,
BasicFileAttributes.class).creationTime();
return new NativeFSLock(lock, channel, realPath, creationTime);
}
```
Copy-up happens at `open(WRITE)`, so the value read afterwards is stable and
the check still detects a later delete or replace. A replacement between the
open and the attribute read would go unnoticed, which seems consistent with the
check being best-effort. I have not tested this change.
### Impact
eXist-db embeds Lucene and fails to start from images pre-populated this
way. The current workaround is to exclude `**/write.lock` when copying the data
directory into the final image.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
