Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/5574#discussion_r28646409
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -1037,21 +1037,52 @@ private[spark] object Utils extends Logging {
}
/**
+ * Convert a passed byte string (e.g. 50b, 100kb, or 250mb) to bytes for
+ * internal use.
+ *
+ * If no suffix is provided, the passed number is assumed to be in bytes.
+ */
+ def byteStringAsBytes(str: String): Long = {
+ JavaUtils.byteStringAsBytes(str)
+ }
+
+ /**
+ * Convert a passed byte string (e.g. 50b, 100kb, or 250mb) to kilobytes
for
--- End diff --
It's pedantic, but kind of matters. I think we're all used to these memory
strings from the JVM `-Xmx` option and refer to `-Xmx4g` as "4 gigabytes" but
it does not give a heap of 4 x 10^9 bytes; it gives 4 x 1024^3 bytes which is
really "4 gibibytes".
I think we should stick to the same convention, but, at least in docs,
let's be specific that these are treated as kibibytes, mebibytes, and
gibibytes, technically. I don't think we have to fix other comments or talk
differently, but a bit of precision here would be useful.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
