danny0405 commented on PR #13399: URL: https://github.com/apache/hudi/pull/13399#issuecomment-2957501696
> In spark, the executors will ask the timeline server to create markers for the files being created which in turn will launch more spark tasks if the spark engine context is used I checked the code in `MarkerDirState` for `HoodieEngineContext` usages, one is in `MarkerDirState` constructor for markers sync and another is for delete all markers, both of these should happen only once because the `MarkerDirState` itself is a singleton on timeline server? And these 2 should not be called while creating markers for data files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
