wonook commented on a change in pull request #279: [NEMO-420] OffHeapMemory
configuration only supports a single type of executor
URL: https://github.com/apache/incubator-nemo/pull/279#discussion_r366945015
##########
File path:
runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/MemoryPoolAssigner.java
##########
@@ -47,18 +47,18 @@
private final MemoryPool memoryPool;
@Inject
- public MemoryPoolAssigner(@Parameter(JobConf.MaxOffheapMb.class) final int
maxOffheapMb,
+ public MemoryPoolAssigner(@Parameter(JobConf.ExecutorMemoryMb.class) final
int memory,
+ @Parameter(JobConf.MaxOffheapRatio.class) final
double maxOffheapRatio,
@Parameter(JobConf.ChunkSizeKb.class) final int
chunkSizeKb) {
+
+ int maxOffheapMb = (int) (memory * maxOffheapRatio);
if (chunkSizeKb < MIN_CHUNK_SIZE_KB) {
throw new IllegalArgumentException("Chunk size too small. Minimum chunk
size is 4KB");
}
final long maxNumChunks = (long) maxOffheapMb * 1024 / chunkSizeKb;
if (maxNumChunks > Integer.MAX_VALUE) {
throw new IllegalArgumentException("Too many pages to allocate (exceeds
MAX_INT)");
}
- if (maxNumChunks < 1) {
- throw new IllegalArgumentException("The given amount of memory amounted
to less than one chunk.");
- }
Review comment:
Let's put this back in, as we can assume that the max off heap ratio is not
0 at all times.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
