wonook commented on a change in pull request #293:
URL: https://github.com/apache/incubator-nemo/pull/293#discussion_r529282981
##########
File path:
compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/runtime/DynamicTaskSizingRuntimePass.java
##########
@@ -59,17 +63,17 @@ public IRDAG apply(final IRDAG irdag, final
Message<Map<String, Long>> mapMessag
final IREdge representativeEdge = edgesToOptimize.iterator().next();
// double check
- if
(!representativeEdge.getDst().getPropertyValue(EnableDynamicTaskSizingProperty.class).get())
{
+ if
(!representativeEdge.getDst().getPropertyValue(EnableDynamicTaskSizingProperty.class).orElse(false))
{
return irdag;
}
final Map<String, Long> messageValue = mapMessage.getMessageValue();
LOG.info("messageValue {}", messageValue);
final int optimizedTaskSizeRatio = messageValue.get(mapKey).intValue();
final int partitionerProperty = getPartitionerProperty(irdag);
for (IREdge edge : edgesToOptimize) {
- if (edge.getPropertyValue(CommunicationPatternProperty.class).get()
- .equals(CommunicationPatternProperty.Value.SHUFFLE)
- &&
!edge.getPropertyValue(PartitionerProperty.class).get().right().equals(partitionerProperty))
{
+ if
(!edge.getPropertyValue(PartitionerProperty.class).get().right().equals(partitionerProperty)
+ && edge.getPropertyValue(CommunicationPatternProperty.class).get()
+ .equals(CommunicationPatternProperty.Value.SHUFFLE)) {
Review comment:
this I meant
```
CommunicationPatternProperty.Value.SHUFFLE.equals(edge.getPropertyValue(CommunicationPatternProperty.class).get())
&&
!partitionerProperty.equals(edge.getPropertyValue(PartitionerProperty.class).get().right())
```
##########
File path:
compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/runtime/DynamicTaskSizingRuntimePass.java
##########
@@ -105,7 +113,11 @@ private void setSubPartitionProperty(final IREdge edge,
final int growingFactor,
private void setDstVertexParallelismProperty(final IREdge edge,
final int partitionSize,
final int partitionerProperty) {
- final int start = (int)
edge.getPropertyValue(SubPartitionSetProperty.class).get().get(0).rangeBeginInclusive();
+ final List<KeyRange> keyRanges =
edge.getPropertyValue(SubPartitionSetProperty.class).orElse(new ArrayList<>());
+ if (keyRanges.isEmpty()) {
+ return;
+ }
Review comment:
Here too
##########
File path:
compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/runtime/DynamicTaskSizingRuntimePass.java
##########
@@ -92,7 +96,11 @@ private int getPartitionerProperty(final IRDAG dag) {
}
private void setSubPartitionProperty(final IREdge edge, final int
growingFactor, final int partitionerProperty) {
- final int start = (int)
edge.getPropertyValue(SubPartitionSetProperty.class).get().get(0).rangeBeginInclusive();
+ final List<KeyRange> keyRanges =
edge.getPropertyValue(SubPartitionSetProperty.class).orElse(new ArrayList<>());
+ if (keyRanges.isEmpty()) {
+ return;
+ }
Review comment:
orElseThrow ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
