ravjot28 opened a new issue, #20585:
URL: https://github.com/apache/hudi/issues/20585
### Bug Description
**What happened:**
`CALL archive_commits(...)` with `hoodie.embed.timeline.server=true` in its
`options` fails while removing the leftover
marker directories of the archived instants, and no commit is archived:
```
org.apache.hudi.exception.HoodieCommitException: Failed to write commits
...
Caused by: org.apache.hudi.exception.HoodieRemoteException: Failed to delete
marker directory file:/.../.hoodie/.temp/<instant>
Connect to localhost:26754 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1]
failed: Connection refused
at
org.apache.hudi.table.marker.TimelineServerBasedWriteMarkers.deleteMarkerDir(...)
at
org.apache.hudi.client.timeline.TimelineArchiverV2.deleteAnyLeftOverMarkers(...)
at org.apache.hudi.cli.ArchiveExecutorUtils.archive(...)
at
org.apache.spark.sql.hudi.command.procedures.ArchiveCommitsProcedure.call(...)
```
On a table at table version 6 the same happens through `TimelineArchiverV1`
("Failed to archive commits").
**What you expected:**
The procedure archives the commits and removes the leftover marker
directories, whatever `hoodie.embed.timeline.server`
says in its options. The procedure never starts a timeline server, so with
the default local file system view
timeline-server-based markers should fall back to direct markers.
**Steps to reproduce:**
1. Create a copy-on-write table and write six commits.
2. Leave a marker directory under `.hoodie/.temp/` for one of the commits
that will be archived (or let a failed write
leave one).
3. Run:
```sql
call archive_commits(table => 't', min_commits => 2, max_commits => 3,
retain_commits => 1,
enable_metadata => false, options =>
'hoodie.embed.timeline.server=true');
```
`hoodie.write.markers.type` defaults to `TIMELINE_SERVER_BASED`, so
`hoodie.embed.timeline.server=true` alone is
enough. Any caller of `ArchiveExecutorUtils.archive` that passes properties
which keep the embedded timeline server on
(for example reused writer properties) hits the same failure.
**Root cause (master d5230b69e22):**
-
`hudi-spark-datasource/hudi-spark/src/main/java/org/apache/hudi/cli/ArchiveExecutorUtils.java:74-76`:
the builder
sets `.withEmbeddedTimelineServerEnabled(false)` and then applies
`.withProps(options)` last (added in #18437 so that
user options win over sub-config defaults).
`hoodie.embed.timeline.server=true` in `options` turns the server back on
in the write config.
-
`hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/WriteMarkersFactory.java:49`:
with the default local file system view, timeline-server-based markers
fall back to direct markers when the embedded
timeline server is off (explicit remote-view settings can still select
them). With it on,
`TimelineArchiverV2.java:393-394` (`TimelineArchiverV1.java:487-488` for
table version 6) creates
`TimelineServerBasedWriteMarkers`, which call `localhost:26754`.
`ArchiveExecutorUtils` creates the table and the
archiver without a write client, so no timeline server is running.
### Environment
**Hudi version:** master (1.3.0-SNAPSHOT at d5230b69e22), 1.2.1. 1.2.0 is
not affected (the procedure has no `options`
argument there).
**Query engine:** Spark 3.5
**Relevant configs:** `hoodie.embed.timeline.server=true` in the procedure's
`options`; `hoodie.write.markers.type` at its
default `TIMELINE_SERVER_BASED`.
### Logs and Stack Trace
See "What happened" above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]