agrawaldevesh commented on a change in pull request #29014: URL: https://github.com/apache/spark/pull/29014#discussion_r456049148
########## File path: core/src/main/scala/org/apache/spark/scheduler/ExecutorDecommissionInfo.scala ########## @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.spark.scheduler + +/** + * Provides more detail when an executor is being decommissioned. + * @param message Human readable reason for why the decommissioning is happening. + * @param isHostDecommissioned Whether the host (aka the `node` or `worker` in other places) is + * being decommissioned too. Used to infer if the shuffle data might + * be lost if external shuffle service is enabled. + */ +private[spark] +case class ExecutorDecommissionInfo(message: String, isHostDecommissioned: Boolean) { Review comment: @Ngone51 ... as I have said before, this PR is indeed stacked on top of #29032, and so the only right way to review it is to really review the last commit in this PR (which is just this PR's changes). Reviewing the "Files Changed" is indeed confusing :-) I would be open to folding #29032 but I would not recommend it: It would complicate the revert process. Let's say we build some other feature on top of this plumbing (like say Spot kills). And then for some other reason, we need to revert this pr (#29014) -- it would be easiest to revert if #29014 is independent of the plumbing. Therefore its usually wise to keep a plumbing/refactoring PR (Which does not introduce much semantic changes) separate. Do you think its a blocker to fold #29032 into this PR ? @holdenk what do you think ? Should we just drag in the plumbing of the decommission info (#29032) in here, or should we keep it separate. I am not sure what the prior convention has been. I can be convinced both ways. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
