jerryshao opened a new issue, #11976:
URL: https://github.com/apache/gravitino/issues/11976
### Version
main branch
### Describe what's wrong
After upgrading Gravitino across versions (e.g. via a rolling update on
Kubernetes), running a built-in job template (e.g.
`builtin-iceberg-rewrite-data-files`) fails because the server still tries to
load the old-version jar path (`gravitino-jobs-<old-version>.jar`) that no
longer exists in the container.
The root cause is in
`BuiltInJobTemplateEventListener.reconcileBuiltInJobTemplates()`
(`core/src/main/java/org/apache/gravitino/job/BuiltInJobTemplateEventListener.java`)
— it only compares the template's `version` custom field (e.g. `"v1"`) to
decide whether to refresh the persisted record. Since the built-in job's logic
didn't change, `version` stays at `"v1"` across a Gravitino release bump, so
the `executable` path — which is resolved dynamically at startup from the
on-disk jar location via `BuiltInJob.resolveExecutable()`
(`maintenance/jobs/src/main/java/org/apache/gravitino/maintenance/jobs/BuiltInJob.java`)
— is never refreshed in the database, even though the actual jar filename
changed with the release.
### Error message and/or stacktrace
```
{
"code": 1002,
"type": "RuntimeException",
"message": "Failed to operate object operation [RUN] under [test], reason
[Failed to fetch file from URI
/opt/gravitino/auxlib/gravitino-jobs-1.4.0-SNAPSHOT.jar]",
"stack": [
"java.lang.RuntimeException: Failed to fetch file from URI
/opt/gravitino/auxlib/gravitino-jobs-1.4.0-SNAPSHOT.jar",
"at
org.apache.gravitino.job.JobManager.fetchFileFromUri(JobManager.java:812)",
"at
org.apache.gravitino.job.JobManager.createRuntimeJobTemplate(JobManager.java:677)",
"at org.apache.gravitino.job.JobManager.runJob(JobManager.java:438)",
"...",
"Caused by: java.io.IOException: Source file does not exist:
/opt/gravitino/auxlib/gravitino-jobs-1.4.0-SNAPSHOT.jar",
"at
org.apache.gravitino.utils.FileFetcher.linkLocalFile(FileFetcher.java:129)",
"at
org.apache.gravitino.utils.FileFetcher.fetchFileFromUri(FileFetcher.java:107)",
"at
org.apache.gravitino.job.JobManager.fetchFileFromUri(JobManager.java:806)"
]
}
```
Container actual state after upgrade:
```
$ ls /opt/gravitino/auxlib/
gravitino-cli-2.0.0-SNAPSHOT.jar gravitino-jobs-2.0.0-SNAPSHOT.jar
```
### How to reproduce
1. Deploy a Gravitino build with built-in job templates enabled (e.g. via
Helm chart on Kubernetes)
2. Run any built-in job (e.g. `builtin-iceberg-rewrite-data-files`) — this
succeeds and persists the template to the database with `executable =
/opt/gravitino/auxlib/gravitino-jobs-<old-version>.jar`
3. Upgrade the Gravitino image to a new version (rolling update), which
ships `gravitino-jobs-<new-version>.jar`
4. Run the same built-in job again — fails with the error above
### Additional context
`reconcileBuiltInJobTemplates()` should also detect changes to the
`executable` path (and other mutable fields like arguments/configs), not rely
solely on the `version` string, when deciding whether to refresh the persisted
template. `JobTemplate` already implements a full `equals()`, so the fix can
compare `existing.toJobTemplate().equals(newTemplate)` in addition to the
version check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
