Change in asterixdb[master]: Feed Connection Refactoring
abdullah alamoudi has submitted this change and it was merged. Change subject: Feed Connection Refactoring .. Feed Connection Refactoring 1. The feed subscription network using FeedJoint is removed. 2. FeedConnection metadata dataset is added (pkeys: dataverseName, feedName, datasetName). 3. Replaced the old intake job + collect job combination with one single job using SplitOperator. 4. Now one feed can connect to multiple datasets. 5. The disconnect feed job is replaced by ActiveManagerMessage. 6. The new feed life cycle is: - Create feed - Connect feed to dataset0, dataset1, dataset2, etc. - Start feed - Stop feed - Disconnect feed 7. New feedEventListner framework by Abdullah Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Reviewed-on: https://asterix-gerrit.ics.uci.edu/1259 Tested-by: JenkinsReviewed-by: Till Westmann Integration-Tests: Jenkins Reviewed-by: abdullah alamoudi --- M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveEvent.java D asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveJob.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveJobNotificationHandler.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveLifecycleListener.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActivityState.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEntityEventsListener.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEventSubscriber.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceRandomPartitioningFeedComputationRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedWorkCollection.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataset/metadata_dataset.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_index/metadata_index.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_selfjoin/metadata_selfjoin.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries/big-object/big_object_feed_20M/big_object_feed_20M.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-csv/change-feed-with-meta-csv.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.4.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-in-meta/change-feed-with-meta-pk-in-meta-index-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-in-value/change-feed-with-meta-pk-in-meta-open-index-in-value.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-with-missing/change-feed-with-meta-pk-in-meta-open-index-with-missing.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-with-mixed-index/change-feed-with-meta-with-mixed-index.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed/change-feed.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.0.ddl.aql C asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.1.update.aql R
Change in asterixdb[master]: Feed Connection Refactoring
abdullah alamoudi has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 16: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 16 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 16: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1923/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 16 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Till Westmann has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 16: Code-Review+1 Good for me. I think that either of the authors could +2 this. -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 16 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 16: BAD-1 BAD Compatibility Tests Failed https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/566/ : FAILURE -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 16 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 16: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1923/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 16 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 16: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/4329/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 16 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1259 to look at the new patch set (#16). Change subject: Feed Connection Refactoring .. Feed Connection Refactoring 1. The feed subscription network using FeedJoint is removed. 2. FeedConnection metadata dataset is added (pkeys: dataverseName, feedName, datasetName). 3. Replaced the old intake job + collect job combination with one single job using SplitOperator. 4. Now one feed can connect to multiple datasets. 5. The disconnect feed job is replaced by ActiveManagerMessage. 6. The new feed life cycle is: - Create feed - Connect feed to dataset0, dataset1, dataset2, etc. - Start feed - Stop feed - Disconnect feed 7. New feedEventListner framework by Abdullah Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 --- M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveEvent.java D asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveJob.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveJobNotificationHandler.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveLifecycleListener.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActivityState.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEntityEventsListener.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEventSubscriber.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceRandomPartitioningFeedComputationRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedWorkCollection.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataset/metadata_dataset.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_index/metadata_index.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_selfjoin/metadata_selfjoin.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries/big-object/big_object_feed_20M/big_object_feed_20M.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-csv/change-feed-with-meta-csv.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.4.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-in-meta/change-feed-with-meta-pk-in-meta-index-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-in-value/change-feed-with-meta-pk-in-meta-open-index-in-value.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-with-missing/change-feed-with-meta-pk-in-meta-open-index-with-missing.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-with-mixed-index/change-feed-with-meta-with-mixed-index.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed/change-feed.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.0.ddl.aql C asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.1.update.aql R asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.2.update.aql C asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.3.query.aql C
Change in asterixdb[master]: Feed Connection Refactoring
Till Westmann has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 15: (1 comment) Changes between 14 and 15 look good to me, just one long line in FeedOperations ... https://asterix-gerrit.ics.uci.edu/#/c/1259/15/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java File asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java: Line 148: org.apache.hyracks.algebricks.common.utils.PairsplitsAndConstraint = > MAJOR SonarQube violation: indeed. -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 15 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: Yes
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 15: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1922/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 15 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 15: Integration-Tests-1 Integration Tests Timed Out https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1921/ : ABORTED -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 15 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 15: BAD Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/565/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 15 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 15: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1921/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 15 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1259 to look at the new patch set (#15). Change subject: Feed Connection Refactoring .. Feed Connection Refactoring 1. The feed subscription network using FeedJoint is removed. 2. FeedConnection metadata dataset is added (pkeys: dataverseName, feedName, datasetName). 3. Replaced the old intake job + collect job combination with one single job using SplitOperator. 4. Now one feed can connect to multiple datasets. 5. The disconnect feed job is replaced by ActiveManagerMessage. 6. The new feed life cycle is: - Create feed - Connect feed to dataset0, dataset1, dataset2, etc. - Start feed - Stop feed - Disconnect feed 7. New feedEventListner framework by Abdullah Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 --- M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveEvent.java D asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveJob.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveJobNotificationHandler.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveLifecycleListener.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActivityState.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEntityEventsListener.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEventSubscriber.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceRandomPartitioningFeedComputationRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedWorkCollection.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataset/metadata_dataset.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_index/metadata_index.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_selfjoin/metadata_selfjoin.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries/big-object/big_object_feed_20M/big_object_feed_20M.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-csv/change-feed-with-meta-csv.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.4.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-in-meta/change-feed-with-meta-pk-in-meta-index-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-in-value/change-feed-with-meta-pk-in-meta-open-index-in-value.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-with-missing/change-feed-with-meta-pk-in-meta-open-index-with-missing.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-with-mixed-index/change-feed-with-meta-with-mixed-index.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed/change-feed.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.0.ddl.aql C asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.1.update.aql R asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.2.update.aql C asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.3.query.aql C
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 15: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/4328/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 15 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
abdullah alamoudi has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 14: (33 comments) https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActivityState.java File asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActivityState.java: PS14, Line 21: ActivityState > Could this remain an enum? Done https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEntityEventsListener.java File asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEntityEventsListener.java: PS14, Line 40: byte > This would be nicer, if the state were an enum. On the interface it's not c Done https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java File asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java: Line 158: PrintWriter writer = new PrintWriter(System.err, true); > Indeed. Why do we write to System.err here? Done https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_02/feeds_02.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_02/feeds_02.2.update.aql: PS14, Line 20: > Fix the WS? Done PS14, Line 21: > WS? Done https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_03/feeds_03.1.ddl.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_03/feeds_03.1.ddl.aql: PS14, Line 36: > WS? Done https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_07/feeds_07.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_07/feeds_07.2.update.aql: PS14, Line 20: > WS Done PS14, Line 29: > WS Done https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.2.update.aql: PS14, Line 31: > WS Done https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.2.update.aql: PS14, Line 20: > WS Done PS14, Line 29: > WS Done https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_10/feeds_10.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_10/feeds_10.2.update.aql: PS14, Line 28: > WS Done https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_11/feeds_11.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_11/feeds_11.2.update.aql: PS14, Line 20: > WS Done PS14, Line 21: > WS Done PS14, Line 29: > WS Done https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_12/feeds_12.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_12/feeds_12.2.update.aql: PS14, Line 20: > WS Done PS14, Line 21: > WS Done PS14, Line 29: > WS Done https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/issue_230_feeds/issue_230_feeds.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/issue_230_feeds/issue_230_feeds.2.update.aql: PS14, Line 20: > WS Done PS14, Line 21: > WS Done PS14, Line 29: > WS Done https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/hints/issue_251_dataset_hint_7/issue_251_dataset_hint_7.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/hints/issue_251_dataset_hint_7/issue_251_dataset_hint_7.2.update.aql: PS14, Line 20: > WS Done PS14, Line 22: > WS Done https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-installer/src/test/resources/integrationts/library/queries/library-adapters/typed_adapter/typed_adapter.2.update.aql File asterixdb/asterix-installer/src/test/resources/integrationts/library/queries/library-adapters/typed_adapter/typed_adapter.2.update.aql: PS14, Line 22: > WS Done PS14, Line 23: > WS Done https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-installer/src/test/resources/integrationts/library/queries/library-feeds/feed_ingest/feed_ingest.2.update.aql File
Change in asterixdb[master]: Feed Connection Refactoring
Till Westmann has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 14: (33 comments) https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActivityState.java File asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActivityState.java: PS14, Line 21: ActivityState Could this remain an enum? https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEntityEventsListener.java File asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEntityEventsListener.java: PS14, Line 40: byte This would be nicer, if the state were an enum. On the interface it's not clear what will be returned. https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java File asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java: Line 158: PrintWriter writer = new PrintWriter(System.err, true); > MAJOR SonarQube violation: Indeed. Why do we write to System.err here? https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_02/feeds_02.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_02/feeds_02.2.update.aql: PS14, Line 20: Fix the WS? PS14, Line 21: WS? https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_03/feeds_03.1.ddl.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_03/feeds_03.1.ddl.aql: PS14, Line 36: WS? https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_07/feeds_07.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_07/feeds_07.2.update.aql: PS14, Line 20: WS PS14, Line 29: WS https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_08/feeds_08.2.update.aql: PS14, Line 31: WS https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.2.update.aql: PS14, Line 20: WS PS14, Line 29: WS https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_10/feeds_10.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_10/feeds_10.2.update.aql: PS14, Line 28: WS https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_11/feeds_11.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_11/feeds_11.2.update.aql: PS14, Line 20: WS PS14, Line 21: WS PS14, Line 29: WS https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_12/feeds_12.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_12/feeds_12.2.update.aql: PS14, Line 20: WS PS14, Line 21: WS PS14, Line 29: WS https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/issue_230_feeds/issue_230_feeds.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/issue_230_feeds/issue_230_feeds.2.update.aql: PS14, Line 20: WS PS14, Line 21: WS PS14, Line 29: WS https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-app/src/test/resources/runtimets/queries/hints/issue_251_dataset_hint_7/issue_251_dataset_hint_7.2.update.aql File asterixdb/asterix-app/src/test/resources/runtimets/queries/hints/issue_251_dataset_hint_7/issue_251_dataset_hint_7.2.update.aql: PS14, Line 20: WS PS14, Line 22: WS https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-installer/src/test/resources/integrationts/library/queries/library-adapters/typed_adapter/typed_adapter.2.update.aql File asterixdb/asterix-installer/src/test/resources/integrationts/library/queries/library-adapters/typed_adapter/typed_adapter.2.update.aql: PS14, Line 22: WS PS14, Line 23: WS https://asterix-gerrit.ics.uci.edu/#/c/1259/14/asterixdb/asterix-installer/src/test/resources/integrationts/library/queries/library-feeds/feed_ingest/feed_ingest.2.update.aql File asterixdb/asterix-installer/src/test/resources/integrationts/library/queries/library-feeds/feed_ingest/feed_ingest.2.update.aql: PS14, Line 22: WS PS14, Line 23:
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 14: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1918/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 14 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 14: BAD-1 BAD Compatibility Tests Failed https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/562/ : FAILURE -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 14 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 14: BAD Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/562/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 14 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 14: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1918/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 14 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 14: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/4325/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 14 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1259 to look at the new patch set (#14). Change subject: Feed Connection Refactoring .. Feed Connection Refactoring 1. The feed subscription network using FeedJoint is removed. 2. FeedConnection metadata dataset is added (pkeys: dataverseName, feedName, datasetName). 3. Replaced the old intake job + collect job combination with one single job using SplitOperator. 4. Now one feed can connect to multiple datasets. 5. The disconnect feed job is replaced by ActiveManagerMessage. 6. The new feed life cycle is: - Create feed - Connect feed to dataset0, dataset1, dataset2, etc. - Start feed - Stop feed - Disconnect feed 7. New feedEventListner framework by Abdullah Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 --- M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveEvent.java D asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveJob.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveJobNotificationHandler.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveLifecycleListener.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActivityState.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEntityEventsListener.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEventSubscriber.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceRandomPartitioningFeedComputationRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedWorkCollection.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataset/metadata_dataset.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_index/metadata_index.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_selfjoin/metadata_selfjoin.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries/big-object/big_object_feed_20M/big_object_feed_20M.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-csv/change-feed-with-meta-csv.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.4.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-in-meta/change-feed-with-meta-pk-in-meta-index-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-in-value/change-feed-with-meta-pk-in-meta-open-index-in-value.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-with-missing/change-feed-with-meta-pk-in-meta-open-index-with-missing.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-with-mixed-index/change-feed-with-meta-with-mixed-index.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed/change-feed.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.0.ddl.aql C asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.1.update.aql R asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.2.update.aql C asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.3.query.aql C
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 13: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1912/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 13 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 13: BAD-1 BAD Compatibility Tests Failed https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/556/ : FAILURE -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 13 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 13: BAD Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/556/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 13 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 13: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1912/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 13 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 13: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/4310/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 13 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1259 to look at the new patch set (#13). Change subject: Feed Connection Refactoring .. Feed Connection Refactoring 1. The feed subscription network using FeedJoint is removed. 2. FeedConnection metadata dataset is added (pkeys: dataverseName, feedName, datasetName). 3. Replaced the old intake job + collect job combination with one single job using SplitOperator. 4. Now one feed can connect to multiple datasets. 5. The disconnect feed job is replaced by ActiveManagerMessage. 6. The new feed life cycle is: - Create feed - Connect feed to dataset0, dataset1, dataset2, etc. - Start feed - Stop feed - Disconnect feed 7. New feedEventListner framework by Abdullah Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 --- M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveEvent.java D asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveJob.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveJobNotificationHandler.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveLifecycleListener.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActivityState.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEntityEventsListener.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEventSubscriber.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceRandomPartitioningFeedComputationRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedWorkCollection.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataset/metadata_dataset.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_index/metadata_index.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_selfjoin/metadata_selfjoin.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries/big-object/big_object_feed_20M/big_object_feed_20M.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-csv/change-feed-with-meta-csv.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.4.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-in-meta/change-feed-with-meta-pk-in-meta-index-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-in-value/change-feed-with-meta-pk-in-meta-open-index-in-value.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-with-missing/change-feed-with-meta-pk-in-meta-open-index-with-missing.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-with-mixed-index/change-feed-with-meta-with-mixed-index.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed/change-feed.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.0.ddl.aql C asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.1.update.aql R asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.2.update.aql C asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.3.query.aql C
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 12: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/4307/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 12 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 12: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/4302/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 12 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 11: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1839/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 11 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 11: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1839/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 11 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 11: Integration-Tests-1 Integration Tests Timed Out https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1831/ : ABORTED -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 11 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 11: BAD-1 BAD Compatibility Tests Failed https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/497/ : FAILURE -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 11 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 11: BAD Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/497/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 11 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 11: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1831/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 11 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 11: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/4218/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 11 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1259 to look at the new patch set (#11). Change subject: Feed Connection Refactoring .. Feed Connection Refactoring 1. The feed subscription network using FeedJoint is removed. 2. FeedConnection metadata dataset is added (pkeys: dataverseName, feedName, datasetName). 3. Replaced the old intake job + collect job combination with one single job using SplitOperator. 4. Now one feed can connect to multiple datasets. 5. The disconnect feed job is replaced by ActiveManagerMessage. 6. The new feed life cycle is: - Create feed - Connect feed to dataset0, dataset1, dataset2, etc. - Start feed - Stop feed - Disconnect feed 7. New feedEventListner framework by Abdullah Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 --- M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveEvent.java D asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveJob.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveJobNotificationHandler.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveLifecycleListener.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActivityState.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEntityEventsListener.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEventSubscriber.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceRandomPartitioningFeedComputationRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AqlPlusExpressionToPlanTranslator.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedWorkCollection.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataset/metadata_dataset.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_index/metadata_index.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_selfjoin/metadata_selfjoin.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries/big-object/big_object_feed_20M/big_object_feed_20M.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.4.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-in-meta/change-feed-with-meta-pk-in-meta-index-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-in-value/change-feed-with-meta-pk-in-meta-open-index-in-value.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-with-missing/change-feed-with-meta-pk-in-meta-open-index-with-missing.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-with-mixed-index/change-feed-with-meta-with-mixed-index.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed/change-feed.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.0.ddl.aql C asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.1.update.aql R asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.2.update.aql C asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.3.query.aql C
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 10: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1823/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 10 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 10: BAD-1 BAD Compatibility Tests Failed https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/491/ : FAILURE -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 10 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 10: BAD Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/491/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 10 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1259 to look at the new patch set (#10). Change subject: Feed Connection Refactoring .. Feed Connection Refactoring 1. The feed subscription network using FeedJoint is removed. 2. FeedConnection metadata dataset is added (pkeys: dataverseName, feedName, datasetName). 3. Replaced the old intake job + collect job combination with one single job using SplitOperator. 4. Now one feed can connect to multiple datasets. 5. The disconnect feed job is replaced by ActiveManagerMessage. 6. The new feed life cycle is: - Create feed - Connect feed to dataset0, dataset1, dataset2, etc. - Start feed - Stop feed - Disconnect feed 7. New feedEventListner framework by Abdullah Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 --- M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveEvent.java D asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveJob.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveJobNotificationHandler.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveLifecycleListener.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActivityState.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEntityEventsListener.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEventSubscriber.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceRandomPartitioningFeedComputationRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AqlPlusExpressionToPlanTranslator.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedWorkCollection.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataset/metadata_dataset.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_index/metadata_index.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_selfjoin/metadata_selfjoin.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries/big-object/big_object_feed_20M/big_object_feed_20M.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.4.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-in-meta/change-feed-with-meta-pk-in-meta-index-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-in-value/change-feed-with-meta-pk-in-meta-open-index-in-value.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-with-missing/change-feed-with-meta-pk-in-meta-open-index-with-missing.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-with-mixed-index/change-feed-with-meta-with-mixed-index.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed/change-feed.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.0.ddl.aql C asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.1.update.aql R asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.2.update.aql C asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.3.query.aql C
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 10: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/4207/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 10 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Xikui Wang has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 5: (7 comments) I added several comments to yours. I will try to do some cleaning and merge the updates from master as well. Thanks! Sorry for the delay. I was preparing for the candidacy exam preparation. https://asterix-gerrit.ics.uci.edu/#/c/1259/5/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedOperations.java File asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedOperations.java: Line 116: private static final ILangCompilationProvider compilationProvider = new AqlCompilationProvider(); > get rid of static compilationProvider I added two local variables in QueryTranslator before calling getConnectJob. Is that ok? Line 162: PrintWriter writer = new PrintWriter(System.err, true); > MAJOR SonarQube violation: Any idea about this issue? Line 197: ingestionOp = new FeedIntakeOperatorDescriptor(jobSpec, feed, firstOp.getAdaptorLibraryName(), > when will the adapterFactory be null? If it's external adaptor from UDF, this will be null. It will be bind to class at execution time. Line 232: if (opDesc instanceof AsterixLSMTreeInsertDeleteOperatorDescriptor > I believe that we don't need the if or the else here!! The reason I didn't remove feed collect operator from the workflow is for feed policy. I couldn't find a better place to adopt feed policy in current setting, so I kept the feed collect operator https://asterix-gerrit.ics.uci.edu/#/c/1259/5/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java File asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java: Line 345: case Statement.Kind.CONNECT_FEED: > add test cases that queries the connection dataset after connect and after There is one test case connect-feed added for this case. I will change it to query for more detailed connection info. Line 2143: try { > what if feed has already been started? Fixed this, as well as for stopFeed. I met one tricky case in this though. If I execute 2 stop statement with small time gap, the 2nd stop will throw an exception as the 1st stop is not fully executed yet. Is that an issue? Line 2183: // TODO: check feed is actually running. > do the todo and add a test case? Done with the start fix. -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: Yes
Change in asterixdb[master]: Feed Connection Refactoring
abdullah alamoudi has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 5: (17 comments) Looks good overall and I can clearly see that you understand what is happening. Please go over the comments. I will look at the new patch. Cheers, Abdullah. https://asterix-gerrit.ics.uci.edu/#/c/1259/5/asterixdb/asterix-active/src/main/java/org/apache/asterix/active/message/ActiveManagerMessage.java File asterixdb/asterix-active/src/main/java/org/apache/asterix/active/message/ActiveManagerMessage.java: Line 31: revert? https://asterix-gerrit.ics.uci.edu/#/c/1259/5/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedOperations.java File asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedOperations.java: Line 116: private static final ILangCompilationProvider compilationProvider = new AqlCompilationProvider(); get rid of static compilationProvider and qtFactory and instead pass them as parameters! Line 197: ingestionOp = new FeedIntakeOperatorDescriptor(jobSpec, feed, firstOp.getAdaptorLibraryName(), when will the adapterFactory be null? Line 232: if (opDesc instanceof AsterixLSMTreeInsertDeleteOperatorDescriptor I believe that we don't need the if or the else here!! ideally, we should get rid of the feed collect (you changed it to take input) but it doesn't do anything essentially. we don't need to do this now. we can create an issue to get rid of it. Line 370: JobSpecification feedJob; remove the feedJob variable? move the declaration of ingestionLocations down to the assignment statement? Line 386: feedJob = combineIntakeCollectJobs(metadataProvider, feed, intakeJob, jobsList, feedConnections, return immediately rather than assign then return Line 391: private static PairbuildSendFeedMessageRuntime( remove unused Line 400: remove unused... and remove the feed message operator completely Line 405: } remove unused https://asterix-gerrit.ics.uci.edu/#/c/1259/5/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java File asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java: Line 345: case Statement.Kind.CONNECT_FEED: add test cases that queries the connection dataset after connect and after disconnect? Line 2143: try { what if feed has already been started? fix and add a test case please :) Line 2183: // TODO: check feed is actually running. do the todo and add a test case? https://asterix-gerrit.ics.uci.edu/#/c/1259/5/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/management/FeedConnectionRequest.java File asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/management/FeedConnectionRequest.java: Line 95: get rid of connection status as it is not needed anymore? https://asterix-gerrit.ics.uci.edu/#/c/1259/5/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/watch/FeedConnectJobInfo.java File asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/watch/FeedConnectJobInfo.java: Line 44: public FeedConnectJobInfo(EntityId entityId, JobId jobId, ActivityState state, JobSpecification spec) { rename this class? https://asterix-gerrit.ics.uci.edu/#/c/1259/5/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/FeedMessageOperatorDescriptor.java File asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/FeedMessageOperatorDescriptor.java: this class should go away https://asterix-gerrit.ics.uci.edu/#/c/1259/5/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj File asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj: do the same changes in the sqlpp.jj file? https://asterix-gerrit.ics.uci.edu/#/c/1259/5/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj File asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj: Line 1150: } stop is missing? -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Jenkins Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: Yes
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 5: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1020/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 5: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1020/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 5: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3145/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1259 to look at the new patch set (#5). Change subject: Feed Connection Refactoring .. Feed Connection Refactoring 1. The feed subscription network using FeedJoint is removed. 2. FeedConnection metadata dataset is added (pkeys: dataverseName, feedName, datasetName). 3. Replaced the old intake job + collect job combination with one single job using SplitOperator. 4. Now one feed can connect to multiple datasets. 5. The disconnect feed job is replaced by ActiveManagerMessage. 6. Twitter adaptor will run on all possible nodes now. 7. The new feed life cycle is: - Create feed - Connect feed to dataset0, dataset1, dataset2, etc. - Start feed - Stop feed - Disconnect feed 8. Fixed one sporadic error described in ASTERIXDB-1709. Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 --- M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/message/ActiveManagerMessage.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceRandomPartitioningFeedComputationRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AqlPlusExpressionToPlanTranslator.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataset/metadata_dataset.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_index/metadata_index.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_selfjoin/metadata_selfjoin.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries/big-object/big_object_feed_20M/big_object_feed_20M.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.4.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-in-meta/change-feed-with-meta-pk-in-meta-index-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-in-value/change-feed-with-meta-pk-in-meta-open-index-in-value.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-with-missing/change-feed-with-meta-pk-in-meta-open-index-with-missing.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-with-mixed-index/change-feed-with-meta-with-mixed-index.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed/change-feed.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.0.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.1.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.4.ddl.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.3.server.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.5.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-function/feed-with-external-function.1.ddl.aql M
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 4: Integration-Tests-1 Integration Tests Failed https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1019/ : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 4: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1019/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 4: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3141/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1259 to look at the new patch set (#4). Change subject: Feed Connection Refactoring .. Feed Connection Refactoring 1. The feed subscription network using FeedJoint is removed. 2. FeedConnection metadata dataset is added (pkeys: dataverseName, feedName, datasetName). 3. Replaced the old intake job + collect job combination with one single job using SplitOperator. 4. Now one feed can connect to multiple datasets. 5. The disconnect feed job is replaced by ActiveManagerMessage. 6. Twitter adaptor will run on all possible nodes now. 7. The new feed life cycle is: - Create feed - Connect feed to dataset0, dataset1, dataset2, etc. - Start feed - Stop feed - Disconnect feed 8. Fixed one sporadic error described in ASTERIXDB-1709. Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 --- M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/message/ActiveManagerMessage.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceRandomPartitioningFeedComputationRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AqlPlusExpressionToPlanTranslator.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataset/metadata_dataset.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_index/metadata_index.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_selfjoin/metadata_selfjoin.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries/big-object/big_object_feed_20M/big_object_feed_20M.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.4.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-in-meta/change-feed-with-meta-pk-in-meta-index-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-in-value/change-feed-with-meta-pk-in-meta-open-index-in-value.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-with-missing/change-feed-with-meta-pk-in-meta-open-index-with-missing.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-with-mixed-index/change-feed-with-meta-with-mixed-index.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed/change-feed.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.0.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.1.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.4.ddl.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.3.server.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.5.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-function/feed-with-external-function.1.ddl.aql M
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 3: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3139/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1259 to look at the new patch set (#3). Change subject: Feed Connection Refactoring .. Feed Connection Refactoring 1. The feed subscription network using FeedJoint is removed. 2. FeedConnection metadata dataset is added (pkeys: dataverseName, feedName, datasetName). 3. Replaced the old intake job + collect job combination with one single job using SplitOperator. 4. Now one feed can connect to multiple datasets. 5. The disconnect feed job is replaced by ActiveManagerMessage. 6. Twitter adaptor will run on all possible nodes now. 7. The new feed life cycle is: - Create feed - Connect feed to dataset0, dataset1, dataset2, etc. - Start feed - Stop feed - Disconnect feed 8. Fixed one sporadic error described in ASTERIXDB-1709. Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 --- M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/message/ActiveManagerMessage.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceRandomPartitioningFeedComputationRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AqlPlusExpressionToPlanTranslator.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_dataset/metadata_dataset.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_index/metadata_index.1.adm M asterixdb/asterix-app/src/test/resources/metadata/results/basic/metadata_selfjoin/metadata_selfjoin.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries/big-object/big_object_feed_20M/big_object_feed_20M.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.4.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-in-meta/change-feed-with-meta-pk-in-meta-index-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-in-value/change-feed-with-meta-pk-in-meta-open-index-in-value.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-with-missing/change-feed-with-meta-pk-in-meta-open-index-with-missing.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-with-mixed-index/change-feed-with-meta-with-mixed-index.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed/change-feed.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.0.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.1.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.4.ddl.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.3.server.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.5.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-function/feed-with-external-function.1.ddl.aql M
Change in asterixdb[master]: Feed Connection Refactoring
Jenkins has posted comments on this change. Change subject: Feed Connection Refactoring .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3104/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1259 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Feed Connection Refactoring
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1259 to look at the new patch set (#2). Change subject: Feed Connection Refactoring .. Feed Connection Refactoring 1. The feed subscription network using FeedJoint is removed. 2. FeedConnection metadata dataset is added (pkeys: dataverseName, feedName, datasetName). 3. Replaced the old intake job + collect job combination with one single job using SplitOperator. 4. Now one feed can connect to multiple datasets. 5. The disconnect feed job is replaced by ActiveManagerMessage. 6. Twitter adaptor will run on all possible nodes now. 7. The new feed life cycle is: - Create feed - Connect feed to dataset0, dataset1, dataset2, etc. - Start feed - Stop feed - Disconnect feed Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0 --- M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/message/ActiveManagerMessage.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceRandomPartitioningFeedComputationRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AqlPlusExpressionToPlanTranslator.java D asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedJoint.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java M asterixdb/asterix-app/src/test/resources/runtimets/queries/big-object/big_object_feed_20M/big_object_feed_20M.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.4.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-in-meta/change-feed-with-meta-pk-in-meta-index-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-in-value/change-feed-with-meta-pk-in-meta-open-index-in-value.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-open-index-with-missing/change-feed-with-meta-pk-in-meta-open-index-with-missing.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-with-mixed-index/change-feed-with-meta-with-mixed-index.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed/change-feed.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.0.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.1.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.4.ddl.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.2.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.3.server.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.5.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-function/feed-with-external-function.1.ddl.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-function/feed-with-external-function.3.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.4.update.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.4.update.aql M