Currently twitter streaming is implemented as ESB polling mechanism. the twitter connection is created only in first polling cycle not in every polling cycle and when the messages come to the listener, "onMessage" method is executed so we don't need the polling cycle.
[1] - https://github.com/wso2/esb-connectors/blob/master/twitter/twitter-inbound-endpoint/twitter-inbound-endpoint-1.0.0/src/main/java/org/wso2/carbon/inbound/twitter/poll/TwitterStreamData.java On Wed, Nov 11, 2015 at 10:40 AM, Malaka Silva <[email protected]> wrote: > May be the terminology used for this is not correct. > > Will be having an internal discussion with ESB Team on Friday to further > discuss this. > > On Wed, Nov 11, 2015 at 9:52 AM, Kasun Indrasiri <[email protected]> wrote: > >> Having a busy wait will always lead to spinning the CPU and we need to >> carefully design such inbound endpoints. This kind of changes needs to be >> done with a proper design/code review. ( I also see some basic formatting >> issues in the commited code) >> >> On Tue, Nov 10, 2015 at 8:01 PM, Chanaka Fernando <[email protected]> >> wrote: >> >>> Hi Malaka, >>> >>> By going through the links you have shared, what I understood was that >>> the most important requirement for a streaming api to work is having a >>> persisted connection. This needs to be done no matter how we implement our >>> logic. I cannot agree with you on the statement that polling cycle needs to >>> make all the connection, session creation each time. We don't need to do >>> that if we have the persisted connection. >>> >>> If you check the example mentioned in [1] salesforce streaming api >>> documentation, it has a while loop with a thread sleep for only receiving >>> data within that period. I could not find any place where they have >>> "recommended" busy-wait strategy for client implementations. In that >>> example, it has registered for a topic and implemented the "onMessage" >>> method to process the message when the messaging event is executed. This is >>> like an event based listening implementation. >>> >>> Am I missing something here? >>> >>> On Tue, Nov 10, 2015 at 5:54 PM, Malaka Silva <[email protected]> wrote: >>> >>>> Chanaka some apis only supports this behavior only and for some it is >>>> the recommended way. eg: twitter. salesforce >>>> >>>> The other concern is some apis have limited number of calls per 24 >>>> hours. eg: Salesforce has a daily limit. So if we keep on polling there are >>>> chances that we may reach there limit. >>>> >>>> Also with polling each cycle needs to make the connection, session, >>>> etc with api each time and may increase the latency. Users >>>> will see better performance with lower resource consumption >>>> overall with this approach >>>> . >>>> >>>> IMO optimum way depends on the api and since we have all 3 options now >>>> we can do the research and do it based on the best approach at the time of >>>> implementation. >>>> >>>> [1] https://dev.twitter.com/streaming/overview >>>> [2] >>>> https://developer.salesforce.com/page/Getting_Started_with_the_Force.com_Streaming_API >>>> >>>> >>>> >>>> On Tue, Nov 10, 2015 at 5:36 PM, Chanaka Fernando <[email protected]> >>>> wrote: >>>> >>>>> Hi Malaka, >>>>> >>>>> What I understood by a little research on the internet is that, busy >>>>> waiting will use the CPU cycles while waiting. But if we are going with >>>>> polling or event based listening approach, we can use those CPU cycles for >>>>> some other useful tasks. What are the advantages of using busy-waiting >>>>> instead of polling or listening for events? >>>>> >>>>> On Tue, Nov 10, 2015 at 12:13 PM, Malaka Silva <[email protected]> >>>>> wrote: >>>>> >>>>>> Added the new type and will be available with ESB 410. >>>>>> >>>>>> @Kathees and Hariprasath - We need to add a new version of twitter >>>>>> and salesforce streaming connectors after esb 410 release. >>>>>> >>>>>> [image: Inline image 1]2 >>>>>> >>>>>> On Mon, Nov 9, 2015 at 7:06 PM, Malaka Silva <[email protected]> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> With ESB 4.9.0 customized inbound can only be created for polling >>>>>>> and listening. >>>>>>> >>>>>>> But we need to have a different category to implement busy-waiting >>>>>>> types. >>>>>>> >>>>>>> This will not come under pure listening since coordination is part >>>>>>> of the requirement for this type. Currently these are implemented as >>>>>>> polling and this is not the best solution. [1] >>>>>>> >>>>>>> eg:- Twitter streaming inbound, Salesforce streaming inbound >>>>>>> >>>>>>> [1] https://docs.wso2.com/display/ESB490/Custom+Inbound+Endpoint >>>>>>> >>>>>>> Best Regards, >>>>>>> >>>>>>> Malaka Silva >>>>>>> Senior Tech Lead >>>>>>> M: +94 777 219 791 >>>>>>> Tel : 94 11 214 5345 >>>>>>> Fax :94 11 2145300 >>>>>>> Skype : malaka.sampath.silva >>>>>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >>>>>>> Blog : http://mrmalakasilva.blogspot.com/ >>>>>>> >>>>>>> WSO2, Inc. >>>>>>> lean . enterprise . middleware >>>>>>> http://www.wso2.com/ >>>>>>> http://www.wso2.com/about/team/malaka-silva/ >>>>>>> <http://wso2.com/about/team/malaka-silva/> >>>>>>> https://store.wso2.com/store/ >>>>>>> >>>>>>> Save a tree -Conserve nature & Save the world for your future. Print >>>>>>> this email only if it is absolutely necessary. >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Best Regards, >>>>>> >>>>>> Malaka Silva >>>>>> Senior Tech Lead >>>>>> M: +94 777 219 791 >>>>>> Tel : 94 11 214 5345 >>>>>> Fax :94 11 2145300 >>>>>> Skype : malaka.sampath.silva >>>>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >>>>>> Blog : http://mrmalakasilva.blogspot.com/ >>>>>> >>>>>> WSO2, Inc. >>>>>> lean . enterprise . middleware >>>>>> http://www.wso2.com/ >>>>>> http://www.wso2.com/about/team/malaka-silva/ >>>>>> <http://wso2.com/about/team/malaka-silva/> >>>>>> https://store.wso2.com/store/ >>>>>> >>>>>> Save a tree -Conserve nature & Save the world for your future. Print >>>>>> this email only if it is absolutely necessary. >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> -- >>>>> Chanaka Fernando >>>>> Senior Technical Lead >>>>> WSO2, Inc.; http://wso2.com >>>>> lean.enterprise.middleware >>>>> >>>>> mobile: +94 773337238 >>>>> Blog : http://soatutorials.blogspot.com >>>>> LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 >>>>> Twitter:https://twitter.com/chanakaudaya >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Best Regards, >>>> >>>> Malaka Silva >>>> Senior Tech Lead >>>> M: +94 777 219 791 >>>> Tel : 94 11 214 5345 >>>> Fax :94 11 2145300 >>>> Skype : malaka.sampath.silva >>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >>>> Blog : http://mrmalakasilva.blogspot.com/ >>>> >>>> WSO2, Inc. >>>> lean . enterprise . middleware >>>> http://www.wso2.com/ >>>> http://www.wso2.com/about/team/malaka-silva/ >>>> <http://wso2.com/about/team/malaka-silva/> >>>> https://store.wso2.com/store/ >>>> >>>> Save a tree -Conserve nature & Save the world for your future. Print >>>> this email only if it is absolutely necessary. >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Thank you and Best Regards, >>> Chanaka Fernando >>> Senior Technical Lead >>> WSO2, Inc.; http://wso2.com >>> lean.enterprise.middleware >>> >>> mobile: +94 773337238 >>> Blog : http://soatutorials.blogspot.com >>> LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 >>> Twitter:https://twitter.com/chanakaudaya >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Kasun Indrasiri >> Software Architect >> WSO2, Inc.; http://wso2.com >> lean.enterprise.middleware >> >> cell: +94 77 556 5206 >> Blog : http://kasunpanorama.blogspot.com/ >> > > > > -- > > Best Regards, > > Malaka Silva > Senior Tech Lead > M: +94 777 219 791 > Tel : 94 11 214 5345 > Fax :94 11 2145300 > Skype : malaka.sampath.silva > LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 > Blog : http://mrmalakasilva.blogspot.com/ > > WSO2, Inc. > lean . enterprise . middleware > http://www.wso2.com/ > http://www.wso2.com/about/team/malaka-silva/ > <http://wso2.com/about/team/malaka-silva/> > https://store.wso2.com/store/ > > Save a tree -Conserve nature & Save the world for your future. Print this > email only if it is absolutely necessary. > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Kathees Software Engineer, email: [email protected] mobile: +94772596173
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
