Yingyi Bu has posted comments on this change. Change subject: Add flush() to IFrameWriter ......................................................................
Patch Set 6: (8 comments) 1. I missed a few places that needs to throw an exception in the flush() call because I looked a delta diff to write my previous comment. So there are just a few more places:-) 2. In MaterializingPipelinedPartition.java, it seems flush() could be done outside of the synchronization block. https://asterix-gerrit.ics.uci.edu/#/c/584/6/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/aggreg/AggregateRuntimeFactory.java File algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/aggreg/AggregateRuntimeFactory.java: Line 148: // This operator needs to consume all of its input before sending its output to the next frame throws an exception https://asterix-gerrit.ics.uci.edu/#/c/584/6/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/aggreg/NestedPlansAccumulatingAggregatorFactory.java File algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/aggreg/NestedPlansAccumulatingAggregatorFactory.java: Line 236: // This operator needs to consume all of its input before sending its output to the next frame throws an exception https://asterix-gerrit.ics.uci.edu/#/c/584/6/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/meta/SubplanRuntimeFactory.java File algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/meta/SubplanRuntimeFactory.java: Line 142: // This operator needs to consume all of its input before sending its output to the next frame throws an exception https://asterix-gerrit.ics.uci.edu/#/c/584/6/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/InMemorySortRuntimeFactory.java File algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/InMemorySortRuntimeFactory.java: Line 99: // This operator needs to consume all of its input before sending its output to the next frame throws an exception https://asterix-gerrit.ics.uci.edu/#/c/584/6/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/StringStreamingRuntimeFactory.java File algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/std/StringStreamingRuntimeFactory.java: Line 194: // This operator prints to a process output and flush() doesn't make sense here throws an exception? https://asterix-gerrit.ics.uci.edu/#/c/584/6/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/dataset/DatasetPartitionWriter.java File hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/dataset/DatasetPartitionWriter.java: Line 142: // This is a kind of a sink operator that writes its output to filesystem. flush() should do nothing here throws an exception? It looks flush should only be used in the feed ingestion path, but not in this query result distribution path. https://asterix-gerrit.ics.uci.edu/#/c/584/6/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/MaterializingPipelinedPartition.java File hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/MaterializingPipelinedPartition.java: Line 112: writer.flush(); similar to nextFrame(), it looks flush() doesn't need to be called in the synchronized block but can be called after line 137. You can use a local read to read the class member flush bit in the synchronized block. https://asterix-gerrit.ics.uci.edu/#/c/584/6/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/DeserializedPreclusteredGroupOperator.java File hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/DeserializedPreclusteredGroupOperator.java: Line 135: // operator should only send its output once all of its input has been consumed. hence, this is a no op throws an exception? -- To view, visit https://asterix-gerrit.ics.uci.edu/584 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I85424bab7965b71aac709280af066e1655457aa3 Gerrit-PatchSet: 6 Gerrit-Project: hyracks Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Murtadha Hubail <[email protected]> Gerrit-Reviewer: Till Westmann <[email protected]> Gerrit-Reviewer: Yingyi Bu <[email protected]> Gerrit-Reviewer: abdullah alamoudi <[email protected]> Gerrit-HasComments: Yes
