Change in asterixdb[master]: Update the Feed Tutorial
Xikui Wang has submitted this change and it was merged. Change subject: Update the Feed Tutorial .. Update the Feed Tutorial 1. Updated the Twitter and RSS part to work with the current new feed connection model. 2. Reorganize the feed adapter part and add tutorial for `localfs` and `socket_adapter` feed. Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Reviewed-on: https://asterix-gerrit.ics.uci.edu/1533 Sonar-Qube: JenkinsTested-by: Jenkins Integration-Tests: Jenkins BAD: Jenkins Reviewed-by: abdullah alamoudi --- M asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md 1 file changed, 225 insertions(+), 78 deletions(-) Approvals: abdullah alamoudi: Looks good to me, approved Jenkins: Verified; No violations found; No violations found; Verified diff --git a/asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md b/asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md index 948c7aa..fa00648 100644 --- a/asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md +++ b/asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md @@ -22,55 +22,61 @@ ## Table of Contents ## * [Introduction](#Introduction) -* [Feed Adaptors](#FeedAdaptors) -* [Feed Policies](#FeedPolicies) +* [Feed Adapters](#FeedAdapters) + ## Introduction ## In this document, we describe the support for data ingestion in -AsterixDB. Data feeds are a new mechanism for having continuous data arrive into a BDMS from external sources and incrementally populate a persisted dataset and associated indexes. We add a new BDMS architectural component, called a data feed, that makes a Big Data system the caretaker for functionality that +AsterixDB. Data feeds are a new mechanism for having continuous +data arrive into a BDMS from external sources and incrementally +populate a persisted dataset and associated indexes. We add a new BDMS +architectural component, called a data feed, that makes a Big Data system the caretaker for functionality that used to live outside, and we show how it improves users' lives and system performance. -## Feed Adaptors ## +## Feed Adapters ## The functionality of establishing a connection with a data source and receiving, parsing and translating its data into ADM objects -(for storage inside AsterixDB) is contained in a feed adaptor. A -feed adaptor is an implementation of an interface and its details are -specific to a given data source. An adaptor may optionally be given +(for storage inside AsterixDB) is contained in a feed adapter. A +feed adapter is an implementation of an interface and its details are +specific to a given data source. An adapter may optionally be given parameters to configure its runtime behavior. Depending upon the -data transfer protocol/APIs offered by the data source, a feed adaptor +data transfer protocol/APIs offered by the data source, a feed adapter may operate in a push or a pull mode. Push mode involves just -one initial request by the adaptor to the data source for setting up +one initial request by the adapter to the data source for setting up the connection. Once a connection is authorized, the data source -"pushes" data to the adaptor without any subsequent requests by -the adaptor. In contrast, when operating in a pull mode, the adaptor +"pushes" data to the adapter without any subsequent requests by +the adapter. In contrast, when operating in a pull mode, the adapter makes a separate request each time to receive data. -AsterixDB currently provides built-in adaptors for several popular -data sources such as Twitter, CNN, and RSS feeds. AsterixDB additionally -provides a generic socket-based adaptor that can be used +AsterixDB currently provides built-in adapters for several popular +data sources such as Twitter and RSS feeds. AsterixDB additionally +provides a generic socket-based adapter that can be used to ingest data that is directed at a prescribed socket. -In this tutorial, we shall describe building two example data ingestion pipelines that cover the popular scenario of ingesting data from (a) Twitter and (b) RSS Feed source. +In this tutorial, we shall describe building two example data ingestion pipelines +that cover the popular scenarios of ingesting data from (a) Twitter (b) RSS (c) Socket Feed source. Ingesting Twitter Stream -We shall use the built-in push-based Twitter adaptor. -As a pre-requisite, we must define a Tweet using the AsterixDB Data Model (ADM) and the AsterixDB Query Language (AQL). Given below are the type definition in AQL that create a Tweet datatype which is representative of a real tweet as obtained from Twitter. +We shall use the built-in push-based Twitter adapter. +As a pre-requisite, we must define a Tweet using the AsterixDB Data Model
Change in asterixdb[master]: Update the Feed Tutorial
abdullah alamoudi has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 5: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 5: BAD+1 BAD Compatibility Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/991/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 5: BAD Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/991/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 5: BAD-1 BAD Compatibility Tests Failed https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/984/ : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 5: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/2455/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 5: BAD Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/984/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 5: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage/18/ (6/6) -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 5: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-sonar/3492/ (5/6) -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 5: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/4956/ (4/6) -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 5: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-asterix-app/96/ (1/6) -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1533 to look at the new patch set (#5). Change subject: Update the Feed Tutorial .. Update the Feed Tutorial 1. Updated the Twitter and RSS part to work with the current new feed connection model. 2. Reorganize the feed adapter part and add tutorial for `localfs` and `socket_adapter` feed. Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 --- M asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md 1 file changed, 225 insertions(+), 78 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/33/1533/5 -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 4: BAD-1 BAD Compatibility Tests Failed https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/978/ : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 4: BAD Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/978/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 4: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-sonar/3483/ (4/5) -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 4: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-no-installer-app/85/ (3/5) -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 4: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app/270/ (2/5) -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 4: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/2447/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 4: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-asterix-app/89/ (1/5) -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
abdullah alamoudi has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 3: (19 comments) Stopped at the a vs an comment. please, revisit the writing and ensure that it is grammatically correct https://asterix-gerrit.ics.uci.edu/#/c/1533/2/asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md File asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md: PS2, Line 248: disconnect feed my_feed from dataset RssDataset; add test case https://asterix-gerrit.ics.uci.edu/#/c/1533/3/asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md File asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md: PS3, Line 64: definition definitions PS3, Line 90: with Twitter remove" with Twitter" PS3, Line 91: an associated associated PS3, Line 92: credential credentials PS3, Line 92: includes include PS3, Line 99: adaptor let's unify all to be adapter rather than adaptor? PS3, Line 101: adaptor. tor -> ter PS3, Line 105: adaptor er PS3, Line 117: are provided valid values. are valid. remove the rest? PS3, Line 119: our the PS3, Line 120: AsterixDB the instance PS3, Line 122: (which we shall revisit later). remove PS3, Line 124: adaptor er PS3, Line 156: different multiple PS3, Line 190: Adaptors ers PS3, Line 192: adapters yes PS3, Line 224: an RSS endpoint or a RSS an then a?? The correct use IMO is an. PS3, Line 331: Ingesting tion -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: Yes
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 3: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/2421/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 3: BAD-1 BAD Compatibility Tests Failed https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/965/ : FAILURE -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 3: BAD Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/965/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 3: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app/253/ (5/5) -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 3: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/2421/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1533 to look at the new patch set (#3). Change subject: Update the Feed Tutorial .. Update the Feed Tutorial 1. Updated the Twitter and RSS part to work with the current new feed connection model. 2. Added one section about the Socket feed. 3. Commented the policy section. Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 --- M asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md 1 file changed, 204 insertions(+), 41 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/33/1533/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 3: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/4932/ (1/5) -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 3: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-no-installer-app/68/ (4/5) -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 3: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-sonar/3465/ (2/5) -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 3: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-asterix-app/70/ (3/5) -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Xikui Wang has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 2: Should we have the policy patch merged first and add that as part of the tutorial? -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
abdullah alamoudi has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 2: can we fix this. we should get it in easily -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
abdullah alamoudi has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 2: (2 comments) https://asterix-gerrit.ics.uci.edu/#/c/1533/2/asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md File asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md: PS2, Line 248: disconnect feed my_feed from dataset RssDataset; This should throw an error and disconnecting a running feed should not be allowed PS2, Line 294: : ip = '127.0.0.1' : port1 = 10001 : filePath = 'chu.adm' : : sock1 = socket() : sock1.connect((ip, port1)) : : with open(filePath) as inputData: : for line in inputData: : sock1.sendall(line) : sock1.close() : not sure if we should include this example. -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: Yes
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 2: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/2049/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 2: BAD+1 BAD Compatibility Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/680/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 2: BAD Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/680/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 2: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/2049/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/4496/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 1: BAD Compatibility Tests Failed https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/672/ : FAILURE -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 1: BAD Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/672/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 1: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/2024/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 1: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/2024/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Xikui Wang has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 1: (1 comment) I think the IT failure is a false alarm... Will have it test again in next revision. https://asterix-gerrit.ics.uci.edu/#/c/1533/1/asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md File asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md: Line 34: architectural component, called a data feed, that makes a Big Data system the caretaker for functionality that Do we need to limit 120 characters in md file? If so, I will fix this in next version. -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: Yes
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 1: Integration-Tests-1 Integration Tests Timed Out https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/2011/ : ABORTED -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 1: BAD-1 BAD Compatibility Tests Failed https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/642/ : FAILURE -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 1: BAD Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/642/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 1: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/2011/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Jenkins has posted comments on this change. Change subject: Update the Feed Tutorial .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/4457/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1533 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui WangGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Update the Feed Tutorial
Xikui Wang has uploaded a new change for review. https://asterix-gerrit.ics.uci.edu/1533 Change subject: Update the Feed Tutorial .. Update the Feed Tutorial 1. Updated the Twitter and RSS part to work with the current new feed connection model. 2. Added one section about the Socket feed. 3. Commented the policy section. Change-Id: Ia18d0b2e3f483332058d9739350643e7f8773433 --- M asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md 1 file changed, 139 insertions(+), 38 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/33/1533/1 diff --git a/asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md b/asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md index 948c7aa..9e53a68 100644 --- a/asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md +++ b/asterixdb/asterix-doc/src/site/markdown/feeds/tutorial.md @@ -23,12 +23,15 @@ * [Introduction](#Introduction) * [Feed Adaptors](#FeedAdaptors) -* [Feed Policies](#FeedPolicies) + ## Introduction ## In this document, we describe the support for data ingestion in -AsterixDB. Data feeds are a new mechanism for having continuous data arrive into a BDMS from external sources and incrementally populate a persisted dataset and associated indexes. We add a new BDMS architectural component, called a data feed, that makes a Big Data system the caretaker for functionality that +AsterixDB. Data feeds are a new mechanism for having continuous +data arrive into a BDMS from external sources and incrementally +populate a persisted dataset and associated indexes. We add a new BDMS +architectural component, called a data feed, that makes a Big Data system the caretaker for functionality that used to live outside, and we show how it improves users' lives and system performance. ## Feed Adaptors ## @@ -52,25 +55,28 @@ to ingest data that is directed at a prescribed socket. -In this tutorial, we shall describe building two example data ingestion pipelines that cover the popular scenario of ingesting data from (a) Twitter and (b) RSS Feed source. +In this tutorial, we shall describe building two example data ingestion pipelines +that cover the popular scenarios of ingesting data from (a) Twitter (b) RSS (c) Socket Feed source. Ingesting Twitter Stream We shall use the built-in push-based Twitter adaptor. -As a pre-requisite, we must define a Tweet using the AsterixDB Data Model (ADM) and the AsterixDB Query Language (AQL). Given below are the type definition in AQL that create a Tweet datatype which is representative of a real tweet as obtained from Twitter. +As a pre-requisite, we must define a Tweet using the AsterixDB Data Model (ADM) +and the AsterixDB Query Language (AQL). Given below are the type definition in AQL +that create a Tweet datatype which is representative of a real tweet as obtained from Twitter. create dataverse feeds; use dataverse feeds; create type TwitterUser as closed { -screen_name: string, -lang: string, -friends_count: int32, -statuses_count: int32 +screen_name: string, +lang: string, +friends_count: int32, +statuses_count: int32 }; create type Tweet as open { - id: int64, - user: TwitterUser +id: int64, +user: TwitterUser } create dataset Tweets (Tweet) @@ -81,7 +87,9 @@ #Using the "push_twitter" feed adapter# The "push_twitter" adaptor requires setting up an application account with Twitter. To retrieve -tweets, Twitter requires registering an application with Twitter. Registration involves providing a name and a brief description for the application. Each application has an associated OAuth authentication credential that includes OAuth keys and tokens. Accessing the +tweets, Twitter requires registering an application with Twitter. Registration involves providing +a name and a brief description for the application. Each application has an associated OAuth +authentication credential that includes OAuth keys and tokens. Accessing the Twitter API requires providing the following. 1. Consumer Key (API Key) 2. Consumer Secret (API Secret) @@ -89,7 +97,9 @@ 4. Access Token Secret The "push_twitter" adaptor takes as configuration the above mentioned -parameters. End users are required to obtain the above authentication credentials prior to using the "push_twitter" adaptor. For further information on obtaining OAuth keys and tokens and registering an application with Twitter, please visit http://apps.twitter.com +parameters. End users are required to obtain the above authentication credentials prior to +using the "push_twitter" adaptor. For further information on obtaining OAuth keys and tokens and +registering an application with Twitter,