Yingyi Bu has posted comments on this change. Change subject: Implemented the memory-bounded HashGroupby and HashJoin for BigObject ......................................................................
Patch Set 10: (11 comments) https://asterix-gerrit.ics.uci.edu/#/c/398/10/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/ExternalGroupByPOperator.java File algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/ExternalGroupByPOperator.java: Line 252: //TODO(jianfeng) add the partial stage support What does "partial stage" support mean here? Line 256: true); why the group-by final stage bit is always true? https://asterix-gerrit.ics.uci.edu/#/c/398/10/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/context/IHyracksFrameMgrContext.java File hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/context/IHyracksFrameMgrContext.java: Line 30: //TODO tobedeleted Why is the default allocate() to be deleted? https://asterix-gerrit.ics.uci.edu/#/c/398/10/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/comm/io/FrameTupleAccessor.java File hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/comm/io/FrameTupleAccessor.java: Line 151: sb.append("AException!"); better error message. https://asterix-gerrit.ics.uci.edu/#/c/398/10/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/ISpillableTable.java File hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/ISpillableTable.java: Line 43: /** why sortFrames() is gone? The new implementation never sorts intermdiate data? https://asterix-gerrit.ics.uci.edu/#/c/398/10/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/external/ExternalGroupOperatorDescriptor.java File hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/external/ExternalGroupOperatorDescriptor.java: Line 66: ISpillableTableFactory spillableTableFactory, boolean isFinalStage) { the parameter isOutputSorted is removed? Why is isFinalState needed? We used to partialAggregatorFactory and mergeFactory together to dictate how to do aggregation. https://asterix-gerrit.ics.uci.edu/#/c/398/10/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/InMemoryHashJoin.java File hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/InMemoryHashJoin.java: Line 118: public void join(IFrameTupleAccessor accessorProbe, int tid, IFrameWriter writer) throws HyracksDataException { public-->private? https://asterix-gerrit.ics.uci.edu/#/c/398/10/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunGenerator.java File hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunGenerator.java: Line 60: RecordDescriptor recordDesc, Algorithm alg, EnumFreeSlotPolicy policy, int framesLimit, int outputLimit) What is outputLimit used for? Line 67: switch (policy) { have a FrameFreeSlotPolicyFactory to be a single place doing the dispatch? https://asterix-gerrit.ics.uci.edu/#/c/398/10/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractFrameSorter.java File hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractFrameSorter.java: Line 187: protected int compare(int tp1, int tp2) throws HyracksDataException { protected --> protected final (to prevent polymorphism for compare) https://asterix-gerrit.ics.uci.edu/#/c/398/10/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/ExternalSortRunGenerator.java File hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/ExternalSortRunGenerator.java: Line 21: import java.nio.ByteBuffer; unused import -- To view, visit https://asterix-gerrit.ics.uci.edu/398 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I248f3a374fdacad7d57e49cf18d8233745e55460 Gerrit-PatchSet: 10 Gerrit-Project: hyracks Gerrit-Branch: master Gerrit-Owner: Jianfeng Jia <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Jianfeng Jia <[email protected]> Gerrit-Reviewer: Pouria Pirzadeh <[email protected]> Gerrit-Reviewer: Preston Carman <[email protected]> Gerrit-Reviewer: Till Westmann <[email protected]> Gerrit-Reviewer: Yingyi Bu <[email protected]> Gerrit-HasComments: Yes
