tgravescs commented on a change in pull request #25409: [SPARK-28414][WEBUI] UI
updates to show resource info in Standalone
URL: https://github.com/apache/spark/pull/25409#discussion_r317143311
##########
File path: core/src/main/scala/org/apache/spark/deploy/master/WorkerInfo.scala
##########
@@ -64,12 +61,54 @@ private[spark] class WorkerInfo(
def coresFree: Int = cores - coresUsed
def memoryFree: Int = memory - memoryUsed
- def resourcesFree: Map[String, Int] = {
+ def resourcesAmountFree: Map[String, Int] = {
resources.map { case (rName, rInfo) =>
rName -> rInfo.availableAddrs.length
}
}
+ def resourcesInfo[T: ClassTag]: Map[String, T] = {
+ resources.map { case (rName, rInfo) =>
+ rName -> createResourceInfo(rName, rInfo.addresses,
implicitly[ClassTag[T]])
+ }
+ }
+
+ def resourcesInfoFree: Map[String, ResourceInformation] = {
+ resources.map { case (rName, rInfo) =>
+ rName -> createResourceInfo(rName, rInfo.availableAddrs,
+ implicitly[ClassTag[ResourceInformation]])
+ }
+ }
+
+ def resourcesInfoUsed[T: ClassTag]: Map[String, T] = {
+ resources.map { case (rName, rInfo) =>
+ rName -> createResourceInfo(rName, rInfo.assignedAddrs,
implicitly[ClassTag[T]])
+ }
+ }
+
+ private def createResourceInfo[T](
+ name: String,
+ addresses: Seq[String],
+ ct: ClassTag[T]): T = {
+ val clazz = ct.runtimeClass
+ val rf = {
+ clazz match {
+ case _ if clazz.equals(classOf[MutableResourceInfo]) =>
Review comment:
I get why you are doing this, but this just seems more complex that what
should be needed. Usually when you see different return types they have some
same base class or trait so you don't actually have to look at the class,
although in this case you don't have any input to determine type by either. It
ends up being a runtime check instead of compile time check. So I think you
should just make 2 functions - 1 to get immutable, 1 to get mutable.
alternatively just get it immutable all the time and change the one place
its used as mutable to deal with it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
