lhotari opened a new pull request #11343:
URL: https://github.com/apache/pulsar/pull/11343
Fixes #11340
### Motivation
NarUnpacker used to extract Pulsar Functions jar files and Pulsar IO
connector nar files has concurrency and file corruption issues.
When multiple processes start at once, the files in the extracted directory
might get corrupted.
There are also cases where the same extraction directory is used for
different versions of the functions jar file therefore creating file corruption
issues.
### Modifications
- Calculate SHA-256 hash of the file content and use the Base64 encoded hash
as the directory name
- Prevent concurrent extraction of the file content across multiple JVMs on
the same host machine by using a OS level file lock. This solution is necessary
since Nar file extraction happens in separate JVMs, the Java function instance
processes.
- Prevent concurrent extraction of the file content in the same JVM with a
object monitor based on the file path.
- Java's file locking solution throws OverlappingFileLockException if the
same JVM acquires the file lock twice. Therefore it's necessary to have 2
solutions: one for preventing concurrent access within the JVM and the other to
prevent concurrent access across JVMs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
