xccui commented on a change in pull request #28444:
URL: https://github.com/apache/spark/pull/28444#discussion_r422200227
##########
File path: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala
##########
@@ -35,8 +36,24 @@ private[spark] class AppStatusStore(
val store: KVStore,
val listener: Option[AppStatusListener] = None) {
+ /**
+ * This method contains an automatic retry logic and tries to get a valid
[[v1.ApplicationInfo]].
+ * See [SPARK-31632] The ApplicationInfo in KVStore may be accessed before
it's prepared
+ */
def applicationInfo(): v1.ApplicationInfo = {
Review comment:
If blocking is supposed to be harmful, I'd prefer an optional result
instead of throwing an exception here. In fact, due to the previous design and
implementation, all the invokers of the method are expecting a valid
`ApplicationInfo`, which means they are not capable of dealing with a `None`
value or an exception now. If we change the returned type to `Option`, all the
logic of the invokers (and invokers of these invokers maybe) should be somehow
adjusted. There will be more design choices then.
Adding the retry logic solves the problem from the root. As I said, however,
I can definitely make another choice if we reach a consensus.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
