wbo4958 commented on PR #44690: URL: https://github.com/apache/spark/pull/44690#issuecomment-1968184018
Hi @srowen, Sorry for my bad example in https://github.com/apache/spark/pull/44690#discussion_r1477679566, just like you said, the double has been converted to Long by multiplying `1E16`, so the calculation is based on Long instead of double. actually the sample should be like ``` scala scala> val ONE_ENTIRE_RESOURCE: Long = 10000000000000000L | val taskAmount = ((1.0/11.0) * ONE_ENTIRE_RESOURCE).toLong | var total = ONE_ENTIRE_RESOURCE | for (i <- 1 to 11 ) { | if (total >= taskAmount) { | total -= taskAmount | println(s"assign $taskAmount for task $i, total left: ${total.toDouble / ONE_ENTIRE_RESOURCE}") | } else { | println(s"ERROR Can't assign $taskAmount for task $i, total left: ${total.toDouble / ONE_ENTIRE_RESOURCE}") | } | } assign 909090909090909 for task 1, total left: 0.9090909090909092 assign 909090909090909 for task 2, total left: 0.8181818181818182 assign 909090909090909 for task 3, total left: 0.7272727272727273 assign 909090909090909 for task 4, total left: 0.6363636363636364 assign 909090909090909 for task 5, total left: 0.5454545454545455 assign 909090909090909 for task 6, total left: 0.4545454545454546 assign 909090909090909 for task 7, total left: 0.3636363636363637 assign 909090909090909 for task 8, total left: 0.2727272727272728 assign 909090909090909 for task 9, total left: 0.1818181818181819 assign 909090909090909 for task 10, total left: 0.090909090909091 assign 909090909090909 for task 11, total left: 1.0E-16 val ONE_ENTIRE_RESOURCE: Long = 10000000000000000 val taskAmount: Long = 909090909090909 var total: Long = 1 ``` This PR has already converted the double to Long when doing the GPU resource calculation, so the internal calcuation is totally based on Long instead of double anymore, please refer to [here](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/ExecutorResourcesAmounts.scala#L177) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
