Yingyi Bu has posted comments on this change. Change subject: [ASTERIXDB-1946][STO][IDX] Create BTreeIndex for Correlated Datasets ......................................................................
Patch Set 14: (19 comments) https://asterix-gerrit.ics.uci.edu/#/c/1813/14/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.1.ddl.sqlpp File asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.1.ddl.sqlpp: PS14, Line 21: WS https://asterix-gerrit.ics.uci.edu/#/c/1813/11/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.2.update.sqlpp File asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.2.update.sqlpp: PS11, Line 21: WS https://asterix-gerrit.ics.uci.edu/#/c/1813/12/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.2.update.sqlpp File asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.2.update.sqlpp: PS12, Line 21: WS https://asterix-gerrit.ics.uci.edu/#/c/1813/14/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.2.update.sqlpp File asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.2.update.sqlpp: PS14, Line 21: WS https://asterix-gerrit.ics.uci.edu/#/c/1813/12/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.4.query.sqlpp File asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.4.query.sqlpp: PS12, Line 21: WS https://asterix-gerrit.ics.uci.edu/#/c/1813/14/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.4.query.sqlpp File asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/dml/scan-delete-btree-correlated-secondary-index-open/scan-delete-btree-correlated-secondary-index-open.4.query.sqlpp: PS14, Line 21: WS https://asterix-gerrit.ics.uci.edu/#/c/1813/14/asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.2.ddl.aql File asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.2.ddl.aql: PS14, Line 21: WS https://asterix-gerrit.ics.uci.edu/#/c/1813/14/asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.3.update.aql File asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.3.update.aql: PS14, Line 21: WS https://asterix-gerrit.ics.uci.edu/#/c/1813/14/asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.4.txneu.aql File asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.4.txneu.aql: PS14, Line 21: WS https://asterix-gerrit.ics.uci.edu/#/c/1813/14/asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.5.ddl.aql File asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.5.ddl.aql: PS14, Line 21: WS https://asterix-gerrit.ics.uci.edu/#/c/1813/14/asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.6.txnqbc.aql File asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.6.txnqbc.aql: PS14, Line 21: WS https://asterix-gerrit.ics.uci.edu/#/c/1813/14/asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.9.txnqar.aql File asterixdb/asterix-installer/src/test/resources/transactionts/queries/recover_after_abort/primary_plus_default_correlated_secondary_index/primary_plus_default_correlated_secondary_index.9.txnqar.aql: PS14, Line 21: WS https://asterix-gerrit.ics.uci.edu/#/c/1813/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/LSMSecondaryCreationTupleProcessorNodePushable.java File asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/LSMSecondaryCreationTupleProcessorNodePushable.java: PS11, Line 71: Secondary SecondaryIndex https://asterix-gerrit.ics.uci.edu/#/c/1813/14/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/LSMSecondaryCreationTupleProcessorNodePushable.java File asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/LSMSecondaryCreationTupleProcessorNodePushable.java: PS14, Line 71: LSMSecondary LSMSecondary -> LSMSecondaryIndex PS14, Line 83: public static class DeletedTupleCounter extends AbstractStateObject { : private final Map<Integer, Integer> map = new HashMap<>(); : : public DeletedTupleCounter(JobId jobId) { : super(jobId, DeletedTupleCounter.class); : } : : public int get(int componentPos) { : Integer value = map.get(componentPos); : return value != null ? value : 0; : } : : public void inc(int componentPos) { : map.put(componentPos, get(componentPos) + 1); : } : } How is partition captured in this class? PS14, Line 119: ctx.setStateObject(deletedTupleCounter); The state is one per job per node, but /LSMSecondaryCreationTupleProcessorNodePushable is one per partition per job per node? PS14, Line 231: deletedTupleCounter.inc(componentPos); It looks that deletedTupleCounter is shared among different partitions on a node? https://asterix-gerrit.ics.uci.edu/#/c/1813/14/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/LSMSecondaryCreationTupleProcessorOperatorDescriptor.java File asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/LSMSecondaryCreationTupleProcessorOperatorDescriptor.java: PS14, Line 31: LSMSecondary LSMSecondaryIndex https://asterix-gerrit.ics.uci.edu/#/c/1813/14/asterixdb/asterix-runtime/src/test/java/org/apache/asterix/runtime/operators/LSMSecondaryCreationTupleProcessorTest.java File asterixdb/asterix-runtime/src/test/java/org/apache/asterix/runtime/operators/LSMSecondaryCreationTupleProcessorTest.java: PS14, Line 57: Secondary SecondaryIndex -- To view, visit https://asterix-gerrit.ics.uci.edu/1813 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I2a3435e6720f07bd6a5092d4d9ce42e8d4b7894c Gerrit-PatchSet: 14 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Luo Chen <[email protected]> Gerrit-Reviewer: Ian Maxon <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Jianfeng Jia <[email protected]> Gerrit-Reviewer: Luo Chen <[email protected]> Gerrit-Reviewer: Till Westmann <[email protected]> Gerrit-Reviewer: Yingyi Bu <[email protected]> Gerrit-Reviewer: abdullah alamoudi <[email protected]> Gerrit-HasComments: Yes
