LuciferYang commented on PR #58489: URL: https://github.com/apache/spark/pull/58489#issuecomment-5548580140
**1. startInformer() runs the blocking, unbounded run() on the SparkContext creation thread** `resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/InformerManager.scala:71` `InformerManager.startInformer()` calls `informer.run()` directly; in fabric8 7.x `run()` blocks until the initial LIST completes and the watch is established, and this runs on the SparkContext creation thread, whereas the legacy watch/polling sources start fully asynchronously. From the v7.8.0 Reflector source, the default exception handler declines to retry any error before the first successful sync, so an apiserver hiccup at startup (throttling, transient error) makes run() throw on the calling thread and SparkContext creation fail outright; a slow-but-progressing initial LIST (large namespace) blocks startup with no timeout and no log. A cold start lists ~0 matching pods, so manual verification wouldn't show it. Please switch to `start()` (which doesn't block the caller), but this must come with an exceptionHandler that forces retries (see my other comment) — otherwise you've only traded a startup crash for an informer that dies silently in the background. The lister poll also needs a `hasSynced()` check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
