wbo4958 commented on PR #44690: URL: https://github.com/apache/spark/pull/44690#issuecomment-1968190296
> But I think what you're doing is converting taskAmount * ONE_ENTIRE_RESOURCE to a long, thus rounding down and making the number a little smaller by throwing away a tiny bit. `(taskAmount * ONE_ENTIRE_RESOURCE).toLong` shouldn't be rounded down, but I just tested some numbers, the scala sometimes will store 16 digits after the decimal point, while sometimes keeping 17 digits. So the `ONE_ENTIRE_RESOURCE=1E16` will cause a round down in such a scenario if it is 17 digits, But we can change ONE_ENTIRE_RESOURCE to 1E17 or 1E18 to fix that. ## Round down scenario ``` scala scala> val ONE_ENTIRE_RESOURCE: Long = 100000000000000000L val ONE_ENTIRE_RESOURCE: Long = 100000000000000000 scala> val ONE_ENTIRE_RESOURCE: Long = (1E16).toLong val ONE_ENTIRE_RESOURCE: Long = 10000000000000000 scala> val taskAmount = 0.1000000000000123456789 val taskAmount: Double = 0.10000000000001234 scala> (taskAmount * ONE_ENTIRE_RESOURCE).toLong val res11: Long = 1000000000000123 ``` ## No round down ``` scala scala> val ONE_ENTIRE_RESOURCE: Long = (1E17).toLong val ONE_ENTIRE_RESOURCE: Long = 100000000000000000 scala> val taskAmount = 0.1000000000000123456789 val taskAmount: Double = 0.10000000000001234 scala> (taskAmount * ONE_ENTIRE_RESOURCE).toLong val res13: Long = 10000000000001234 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
