dongjoon-hyun commented on a change in pull request #23412: [SPARK-26477][CORE]
Use ConfigEntry for hardcoded configs for unsafe category
URL: https://github.com/apache/spark/pull/23412#discussion_r248941983
##########
File path:
core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java
##########
@@ -59,28 +57,23 @@ public UnsafeSorterSpillReader(
File file,
BlockId blockId) throws IOException {
assert (file.length() > 0);
- long bufferSizeBytes =
+ final ConfigEntry<Object> bufferSizeConfigEntry =
+ package$.MODULE$.UNSAFE_SORTER_SPILL_READER_BUFFER_SIZE();
+ // This value must be less than or equal to MAX_BUFFER_SIZE_BYTES. Cast to
int is always safe.
+ final int DEFAULT_BUFFER_SIZE_BYTES =
(int)(long)bufferSizeConfigEntry.defaultValue().get();
Review comment:
My bad. Please change like this, @kiszk .
```scala
- final int DEFAULT_BUFFER_SIZE_BYTES =
(int)(long)bufferSizeConfigEntry.defaultValue().get();
+ final int DEFAULT_BUFFER_SIZE_BYTES =
((Long)bufferSizeConfigEntry.defaultValue().get()).intValue();
int bufferSizeBytes =
SparkEnv.get() == null ?
- DEFAULT_BUFFER_SIZE_BYTES :
(int)SparkEnv.get().conf().get(bufferSizeConfigEntry);
+ DEFAULT_BUFFER_SIZE_BYTES :
((Long)SparkEnv.get().conf().get(bufferSizeConfigEntry)).intValue();
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
