Hi, I have created two tables on DAS which collects fresh data daily(more than 1000,000) and run a script which is scheduled hourly to collect data into two tables called NodeTable(nose , retweets order by retweets desc) and EdgeTable(src,dest,retweets order by retweets desc).
Now I have to create a node graph. Node size give u from NodeTable and Width of edge gives u from retweet count in EdgeTable of I have looked up lot of java script libraries such as vis.js, arbor.js, D3.js,sigmajs. I couldn't figured out what is better (as most of them have specific to a single node and takes lot of time to study all) then i moved to d3.js as many have used and tried for different purposes. Can i get any other good suggestions for my graphing? If you have good suggestions, please feel free to post them so that it will help me .. Cheers! On Tue, Nov 17, 2015 at 3:17 PM, Yasara Dissanayake <[email protected]> wrote: > @yudhanjaya > Ok.Now it's working.Popular tweets are now saved to on database base on > that rank from CEP. Wrote custom window for siddi to calculate that rank > and sort them. Currently working with design web page using jaggery server. > > Regards > Yasara > > On Tue, Nov 17, 2015 at 3:11 PM, Yudhanjaya Wijeratne <[email protected] > > wrote: > >> We'll have to talk to Srinath about that, I'm afraid. Let's get the RT >> system rolling first. Whatever we come up with the replies will also have >> to be displayed as part of that graph, so let's get the simpler stuff out >> of the way. >> >> On Tue, Nov 17, 2015 at 2:54 PM, Yasara Dissanayake <[email protected]> >> wrote: >> >>> hi, >>> >>> @Nipuna, >>> Dinali and me both are collecting retweets in same manner which she >>> explained earlier. Since I also consider the details original tweets of >>> retweets. Thank you for comments and link it would be very useful. And >>> about favorite count, isn't it a separate option something like like to the >>> post which can do even non friends? Here I mean the publicity of tweet. >>> that is initially when tweet are published it will be on friends wall. But >>> it has not impact as much as retweet and favorite count. >>> >>> @yudhanjaya >>> And what about the replies? if we consider them there can be +1, 0 and >>> -1 support for that tweet. do we have to consider them ? if so do we have >>> to do some analysis? >>> >>> regards, >>> Yasara >>> >>> On Tue, Nov 17, 2015 at 2:38 PM, Yudhanjaya Wijeratne < >>> [email protected]> wrote: >>> >>>> Hi Madhawa, Nipuna, >>>> >>>> We'll be doing a bit of sentiment analysis later down the line using >>>> the stanford NLP implementation, but for now what we're trying to replicate >>>> in real-time is to >>>> a) rank tweets by freshness and popularity, which can give us some clue >>>> as to opinion influence and b) to basically do this >>>> https://iwringer.wordpress.com/2015/08/25/analysis-of-retweeting-patterns-in-sri-lankan-general-election/ >>>> >>>> Best, >>>> Yudha >>>> >>>> On Tue, Nov 17, 2015 at 2:27 PM, Nipuna Chandradasa <[email protected]> >>>> wrote: >>>> >>>>> HI Madhawa >>>>> >>>>> Context analysis in the sense we have to perform Natural language >>>>> processing right? I think this require additional libraries ... and this >>>>> is >>>>> not yet accurate most of the time if we don't get a right model... as we >>>>> dealing with natural languages .. >>>>> >>>>> Regards, >>>>> >>>>> On Tue, Nov 17, 2015 at 2:23 PM, Madhawa Gunasekara <[email protected] >>>>> > wrote: >>>>> >>>>>> Hi All, >>>>>> >>>>>> IMO It's better to do a context analysis on these tweet texts and >>>>>> draw separate two graphs, (positive / negative) >>>>>> >>>>>> Thanks, >>>>>> Madhawa >>>>>> >>>>>> On Tue, Nov 17, 2015 at 2:13 PM, Nipuna Chandradasa <[email protected] >>>>>> > wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I suggested above for if you are using tweeter API directly .... i >>>>>>> don't know whether these can be done using wso2 twitter connector. ... >>>>>>> i'll look in to it .. >>>>>>> Sorry for the misdirection if above replies are not helpful. >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> On Tue, Nov 17, 2015 at 2:10 PM, Nipuna Chandradasa < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Yasara, >>>>>>>> >>>>>>>> yeah you can check the retweets using retweeted_status as retweets >>>>>>>> contains slightly different json structure than tweets. But you have to >>>>>>>> keep track of whether your system already analyzed the original tweets >>>>>>>> of >>>>>>>> this retweets ... Because there can be a scenario where your system >>>>>>>> meet >>>>>>>> the retweet before the original tweet you are going to filter out that >>>>>>>> tweet and that data is loss.. >>>>>>>> >>>>>>>> [1] link has some answer to this.... that a way you can edit the >>>>>>>> search query to filter out retweets .... >>>>>>>> >>>>>>>> 1. >>>>>>>> http://stackoverflow.com/questions/21748880/exclude-retweets-in-twitter-search-api-1-1 >>>>>>>> >>>>>>>> But the i don't know whether this solve the above scenario i >>>>>>>> mentioned .... But the above mentioned scenario is less likely to >>>>>>>> happen >>>>>>>> if it is a live stream and there are no waiting periods included.... >>>>>>>> >>>>>>>> (You both replied while i'm typing the reply so ... following is >>>>>>>> the new suggestions .. but please go through above factors. those will >>>>>>>> help >>>>>>>> you understand how tweeter API's work ) >>>>>>>> >>>>>>>> @ Dinali ... >>>>>>>> >>>>>>>> +1 for your idea .. >>>>>>>> >>>>>>>> @ Yasara .. >>>>>>>> >>>>>>>> I think the favorites_count is calculated based on followers and >>>>>>>> retweets count ... >>>>>>>> >>>>>>>> https://dev.twitter.com/faq/finding-%26-counting-things >>>>>>>> >>>>>>>> Check before you do the same thing in your code. >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> On Tue, Nov 17, 2015 at 1:04 PM, Yasara Dissanayake < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Nipuna, >>>>>>>>> For our part we have to analyze tweets real time. For that >>>>>>>>> currently we are planed to filter retweets by considering presence of >>>>>>>>> retweeted_status attribute of a json massage which send by tweet >>>>>>>>> object, >>>>>>>>> within the CEP. Does it seems to be a proper method? Do you have any >>>>>>>>> better >>>>>>>>> idea ? >>>>>>>>> >>>>>>>>> regards, >>>>>>>>> Yasara >>>>>>>>> >>>>>>>>> On Tue, Nov 17, 2015 at 11:12 AM, Nipuna Chandradasa < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Is this connector the api provided by twitter?... Then there are >>>>>>>>>> few things u have to keep in mind... >>>>>>>>>> >>>>>>>>>> 1. Twitter api gives each app 15 min waiting period once app >>>>>>>>>> collect the data from api.... So basically you have to wait 15 min >>>>>>>>>> to get >>>>>>>>>> data again.... >>>>>>>>>> >>>>>>>>>> 2. There are no filter system to filter retweets.... What you can >>>>>>>>>> do is get data using search query with date periods.... And filter >>>>>>>>>> retweets >>>>>>>>>> in your app.... >>>>>>>>>> >>>>>>>>>> Hope you find this helpful.... >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> On 17 Nov 2015 10:11 a.m., "Dinali Dabarera" <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I collected tweets which has hash tag of Trump >>>>>>>>>>> #trump,#donaldtrump,#trump2016,#makeamericagreatagain >>>>>>>>>>> from ESB through a twitter connector to SPARK SQL tables in >>>>>>>>>>> DAS. >>>>>>>>>>> >>>>>>>>>>> The tweets are seems to be most probably retweets. Is there any >>>>>>>>>>> issue with the twitter connector,is it collecting only retweets? >>>>>>>>>>> >>>>>>>>>>> Cheers!! >>>>>>>>>>> >>>>>>>>>>> On Tue, Nov 17, 2015 at 8:49 AM, Yasara Dissanayake < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Dinali Rosemin >>>>>>>>>>> University of Peradeniya (Computer Engineering) >>>>>>>>>>> WSO2 Intern >>>>>>>>>>> 077-0198933 >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Dev mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Nipuna Marcus >>>>>>>> *Software Engineer* >>>>>>>> WSO2 Inc. >>>>>>>> http://wso2.com/ - "lean . enterprise . middleware" >>>>>>>> Mobile : +94 (0) 713 667906 >>>>>>>> [email protected] >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Nipuna Marcus >>>>>>> *Software Engineer* >>>>>>> WSO2 Inc. >>>>>>> http://wso2.com/ - "lean . enterprise . middleware" >>>>>>> Mobile : +94 (0) 713 667906 >>>>>>> [email protected] >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> [email protected] >>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Madhawa Gunasekara* >>>>>> Software Engineer >>>>>> WSO2 Inc.; http://wso2.com >>>>>> lean.enterprise.middleware >>>>>> >>>>>> mobile: +94 719411002 <+94+719411002> >>>>>> blog: *http://madhawa-gunasekara.blogspot.com >>>>>> <http://madhawa-gunasekara.blogspot.com>* >>>>>> linkedin: *http://lk.linkedin.com/in/mgunasekara >>>>>> <http://lk.linkedin.com/in/mgunasekara>* >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Nipuna Marcus >>>>> *Software Engineer* >>>>> WSO2 Inc. >>>>> http://wso2.com/ - "lean . enterprise . middleware" >>>>> Mobile : +94 (0) 713 667906 >>>>> [email protected] >>>>> >>>> >>>> >>>> >>>> -- >>>> Yudhanjaya Wijeratne >>>> Marketing Officer, WSO2 Inc <http://www.wso2.com> >>>> +94775496911 | @yudhanjaya >>>> >>> >>> >> >> >> -- >> Yudhanjaya Wijeratne >> Marketing Officer, WSO2 Inc <http://www.wso2.com> >> +94775496911 | @yudhanjaya >> > > > _______________________________________________ > Dev mailing list > [email protected] > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Dinali Rosemin University of Peradeniya (Computer Engineering) WSO2 Intern 077-0198933
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
