Hi Yudhanjaya, Yasara and Dinali have the basics for twitter graph and most important tweets in place. We need to design the story around this. ( I am ccing Dakshika so we can get UX feedback from him).
Dakshika, we are trying to build a website to analyze the US election data for twitter. IMO we have not figured out the story yet, although we have individual pieces. Following are my comments. 1. Looking at Twitter graph, I feel showing number of tweet each user did does not tell anything useful. 2. I feel twitter graph should include all tweeps, not tweeter graph for one candidate. ( we can do color coding to show what are tweeps for focus candidate) 3. I agree users want to know about one candidate. But I think we need to show the data in contrast. Shall we show each candidate's data in contrast to the first. ( For first we contrast with second) We also need to do sentimental analysis one and figure out where it fit in. When you will be in Trace? We should meet and discuss. Thanks Srinath On Fri, Nov 20, 2015 at 8:34 AM, Yudhanjaya Wijeratne <[email protected]> wrote: > Srinath, > > +1. RT's would show influence best. > > > > On Fri, Nov 20, 2015 at 8:32 AM, Srinath Perera <[email protected]> wrote: > >> Hi Yudhanjaya, >> >> On Thu, Nov 19, 2015 at 3:40 PM, Yudhanjaya Wijeratne < >> [email protected]> wrote: >> >>> Hi Srinath, >>> >>> Regarding Dinali's graph, we had a chat and realized that using the >>> width of the edge makes the graph harder to read as smaller connections are >>> hidden. What if we did it this way: >>> >>> *Distance between nodes = 1 / RTs between nodes* >>> >> >> Actually we do not need to do anything. Force base layouts we use will >> put connected nodes closer and not connected nodes further. >> >> >>> >>> This will bring together accounts that often retweet a node's content. >>> Less enthusiastic retweeters are further and further out. >>> >>> *Redness = no of RT's done by a node* >>> >>> High RT accounts, like bots, will show up in stages of red >>> >> +1 >> >>> >>> *Size of node = % of original tweets in sample space OR number of RTs >>> received by that node* >>> >>> Content creators and popular influencers are larger >>> >> >> I would say let's go with RT's received by that node. IMO that is the >> best measure of influence. >> >> >>> >>> Therefore we'll end up with a graph of large nodes (popular influencers) >>> surrounded closely by nodes that RT them a lot and at the edges of this >>> little community will be the nodes that don't RT them all that often. >>> >>> What do you think? >>> >>> Best, >>> Yudha >>> >>> On Thu, Nov 19, 2015 at 11:02 AM, Dinali Dabarera <[email protected]> >>> wrote: >>> >>>> Yes I will do sir. But I am doing some research on d3 by adding more >>>> features and directing getting data from DAS. I hope to do this as it will >>>> help in the future. >>>> >>>> On Thu, Nov 19, 2015 at 9:38 AM, Srinath Perera <[email protected]> >>>> wrote: >>>> >>>>> We must use d3 or a d3 based one, as that WSO2 platform uses. >>>>> >>>>> Sample I shared shows how to do it with d3. >>>>> >>>>> Thanks >>>>> Srinath >>>>> >>>>> On Wed, Nov 18, 2015 at 3:36 PM, Dinali Dabarera <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> I have created two tables on DAS which collects fresh data daily(more >>>>>> than 1000,000) and run a script which is scheduled hourly to collect data >>>>>> into two tables called NodeTable(nose , retweets order by retweets desc) >>>>>> and EdgeTable(src,dest,retweets order by retweets desc). >>>>>> >>>>>> Now I have to create a node graph. Node size give u from NodeTable >>>>>> and Width of edge gives u from retweet count in EdgeTable of I have >>>>>> looked >>>>>> up lot of java script libraries such as vis.js, arbor.js, D3.js,sigmajs. >>>>>> >>>>>> I couldn't figured out what is better (as most of them have specific >>>>>> to a single node and takes lot of time to study all) then i moved to >>>>>> d3.js >>>>>> as many have used and tried for different purposes. Can i get any other >>>>>> good suggestions for my graphing? >>>>>> >>>>>> If you have good suggestions, please feel free to post them so that >>>>>> it will help me .. >>>>>> >>>>>> >>>>>> Cheers! >>>>>> >>>>>> On Tue, Nov 17, 2015 at 3:17 PM, Yasara Dissanayake <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> @yudhanjaya >>>>>>> Ok.Now it's working.Popular tweets are now saved to on database base >>>>>>> on that rank from CEP. Wrote custom window for siddi to calculate that >>>>>>> rank >>>>>>> and sort them. Currently working with design web page using jaggery >>>>>>> server. >>>>>>> >>>>>>> Regards >>>>>>> Yasara >>>>>>> >>>>>>> On Tue, Nov 17, 2015 at 3:11 PM, Yudhanjaya Wijeratne < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> We'll have to talk to Srinath about that, I'm afraid. Let's get the >>>>>>>> RT system rolling first. Whatever we come up with the replies will also >>>>>>>> have to be displayed as part of that graph, so let's get the simpler >>>>>>>> stuff >>>>>>>> out of the way. >>>>>>>> >>>>>>>> On Tue, Nov 17, 2015 at 2:54 PM, Yasara Dissanayake < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> hi, >>>>>>>>> >>>>>>>>> @Nipuna, >>>>>>>>> Dinali and me both are collecting retweets in same manner which >>>>>>>>> she explained earlier. Since I also consider the details original >>>>>>>>> tweets of >>>>>>>>> retweets. Thank you for comments and link it would be very useful. And >>>>>>>>> about favorite count, isn't it a separate option something like like >>>>>>>>> to the >>>>>>>>> post which can do even non friends? Here I mean the publicity of >>>>>>>>> tweet. >>>>>>>>> that is initially when tweet are published it will be on friends >>>>>>>>> wall. But >>>>>>>>> it has not impact as much as retweet and favorite count. >>>>>>>>> >>>>>>>>> @yudhanjaya >>>>>>>>> And what about the replies? if we consider them there can be +1, 0 >>>>>>>>> and -1 support for that tweet. do we have to consider them ? if so do >>>>>>>>> we >>>>>>>>> have to do some analysis? >>>>>>>>> >>>>>>>>> regards, >>>>>>>>> Yasara >>>>>>>>> >>>>>>>>> On Tue, Nov 17, 2015 at 2:38 PM, Yudhanjaya Wijeratne < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Madhawa, Nipuna, >>>>>>>>>> >>>>>>>>>> We'll be doing a bit of sentiment analysis later down the line >>>>>>>>>> using the stanford NLP implementation, but for now what we're trying >>>>>>>>>> to >>>>>>>>>> replicate in real-time is to >>>>>>>>>> a) rank tweets by freshness and popularity, which can give us >>>>>>>>>> some clue as to opinion influence and b) to basically do this >>>>>>>>>> https://iwringer.wordpress.com/2015/08/25/analysis-of-retweeting-patterns-in-sri-lankan-general-election/ >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Yudha >>>>>>>>>> >>>>>>>>>> On Tue, Nov 17, 2015 at 2:27 PM, Nipuna Chandradasa < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> HI Madhawa >>>>>>>>>>> >>>>>>>>>>> Context analysis in the sense we have to perform Natural >>>>>>>>>>> language processing right? I think this require additional >>>>>>>>>>> libraries ... >>>>>>>>>>> and this is not yet accurate most of the time if we don't get a >>>>>>>>>>> right >>>>>>>>>>> model... as we dealing with natural languages .. >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> >>>>>>>>>>> On Tue, Nov 17, 2015 at 2:23 PM, Madhawa Gunasekara < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi All, >>>>>>>>>>>> >>>>>>>>>>>> IMO It's better to do a context analysis on these tweet texts >>>>>>>>>>>> and draw separate two graphs, (positive / negative) >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Madhawa >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Nov 17, 2015 at 2:13 PM, Nipuna Chandradasa < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> I suggested above for if you are using tweeter API directly >>>>>>>>>>>>> .... i don't know whether these can be done using wso2 twitter >>>>>>>>>>>>> connector. >>>>>>>>>>>>> ... i'll look in to it .. >>>>>>>>>>>>> Sorry for the misdirection if above replies are not helpful. >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Nov 17, 2015 at 2:10 PM, Nipuna Chandradasa < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Yasara, >>>>>>>>>>>>>> >>>>>>>>>>>>>> yeah you can check the retweets using retweeted_status as >>>>>>>>>>>>>> retweets contains slightly different json structure than tweets. >>>>>>>>>>>>>> But you >>>>>>>>>>>>>> have to keep track of whether your system already analyzed the >>>>>>>>>>>>>> original >>>>>>>>>>>>>> tweets of this retweets ... Because there can be a scenario >>>>>>>>>>>>>> where your >>>>>>>>>>>>>> system meet the retweet before the original tweet you are going >>>>>>>>>>>>>> to filter >>>>>>>>>>>>>> out that tweet and that data is loss.. >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1] link has some answer to this.... that a way you can edit >>>>>>>>>>>>>> the search query to filter out retweets .... >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1. >>>>>>>>>>>>>> http://stackoverflow.com/questions/21748880/exclude-retweets-in-twitter-search-api-1-1 >>>>>>>>>>>>>> >>>>>>>>>>>>>> But the i don't know whether this solve the above scenario i >>>>>>>>>>>>>> mentioned .... But the above mentioned scenario is less likely >>>>>>>>>>>>>> to happen >>>>>>>>>>>>>> if it is a live stream and there are no waiting periods >>>>>>>>>>>>>> included.... >>>>>>>>>>>>>> >>>>>>>>>>>>>> (You both replied while i'm typing the reply so ... following >>>>>>>>>>>>>> is the new suggestions .. but please go through above factors. >>>>>>>>>>>>>> those will >>>>>>>>>>>>>> help you understand how tweeter API's work ) >>>>>>>>>>>>>> >>>>>>>>>>>>>> @ Dinali ... >>>>>>>>>>>>>> >>>>>>>>>>>>>> +1 for your idea .. >>>>>>>>>>>>>> >>>>>>>>>>>>>> @ Yasara .. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think the favorites_count is calculated based on followers >>>>>>>>>>>>>> and retweets count ... >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://dev.twitter.com/faq/finding-%26-counting-things >>>>>>>>>>>>>> >>>>>>>>>>>>>> Check before you do the same thing in your code. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Nov 17, 2015 at 1:04 PM, Yasara Dissanayake < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Nipuna, >>>>>>>>>>>>>>> For our part we have to analyze tweets real time. For that >>>>>>>>>>>>>>> currently we are planed to filter retweets by considering >>>>>>>>>>>>>>> presence of >>>>>>>>>>>>>>> retweeted_status attribute of a json massage which send by >>>>>>>>>>>>>>> tweet object, >>>>>>>>>>>>>>> within the CEP. Does it seems to be a proper method? Do you >>>>>>>>>>>>>>> have any better >>>>>>>>>>>>>>> idea ? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>> Yasara >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Tue, Nov 17, 2015 at 11:12 AM, Nipuna Chandradasa < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Is this connector the api provided by twitter?... Then >>>>>>>>>>>>>>>> there are few things u have to keep in mind... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 1. Twitter api gives each app 15 min waiting period once >>>>>>>>>>>>>>>> app collect the data from api.... So basically you have to >>>>>>>>>>>>>>>> wait 15 min to >>>>>>>>>>>>>>>> get data again.... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 2. There are no filter system to filter retweets.... What >>>>>>>>>>>>>>>> you can do is get data using search query with date >>>>>>>>>>>>>>>> periods.... And filter >>>>>>>>>>>>>>>> retweets in your app.... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hope you find this helpful.... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>> On 17 Nov 2015 10:11 a.m., "Dinali Dabarera" < >>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I collected tweets which has hash tag of Trump >>>>>>>>>>>>>>>>> #trump,#donaldtrump,#trump2016,#makeamericagreatagain >>>>>>>>>>>>>>>>> from ESB through a twitter connector to SPARK SQL tables >>>>>>>>>>>>>>>>> in DAS. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The tweets are seems to be most probably retweets. Is >>>>>>>>>>>>>>>>> there any issue with the twitter connector,is it collecting >>>>>>>>>>>>>>>>> only retweets? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Cheers!! >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Tue, Nov 17, 2015 at 8:49 AM, Yasara Dissanayake < >>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>> Dinali Rosemin >>>>>>>>>>>>>>>>> University of Peradeniya (Computer Engineering) >>>>>>>>>>>>>>>>> WSO2 Intern >>>>>>>>>>>>>>>>> 077-0198933 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Nipuna Marcus >>>>>>>>>>>>>> *Software Engineer* >>>>>>>>>>>>>> WSO2 Inc. >>>>>>>>>>>>>> http://wso2.com/ - "lean . enterprise . middleware" >>>>>>>>>>>>>> Mobile : +94 (0) 713 667906 >>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Nipuna Marcus >>>>>>>>>>>>> *Software Engineer* >>>>>>>>>>>>> WSO2 Inc. >>>>>>>>>>>>> http://wso2.com/ - "lean . enterprise . middleware" >>>>>>>>>>>>> Mobile : +94 (0) 713 667906 >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>> >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *Madhawa Gunasekara* >>>>>>>>>>>> Software Engineer >>>>>>>>>>>> WSO2 Inc.; http://wso2.com >>>>>>>>>>>> lean.enterprise.middleware >>>>>>>>>>>> >>>>>>>>>>>> mobile: +94 719411002 <+94+719411002> >>>>>>>>>>>> blog: *http://madhawa-gunasekara.blogspot.com >>>>>>>>>>>> <http://madhawa-gunasekara.blogspot.com>* >>>>>>>>>>>> linkedin: *http://lk.linkedin.com/in/mgunasekara >>>>>>>>>>>> <http://lk.linkedin.com/in/mgunasekara>* >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Nipuna Marcus >>>>>>>>>>> *Software Engineer* >>>>>>>>>>> WSO2 Inc. >>>>>>>>>>> http://wso2.com/ - "lean . enterprise . middleware" >>>>>>>>>>> Mobile : +94 (0) 713 667906 >>>>>>>>>>> [email protected] >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Yudhanjaya Wijeratne >>>>>>>>>> Marketing Officer, WSO2 Inc <http://www.wso2.com> >>>>>>>>>> +94775496911 | @yudhanjaya >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Yudhanjaya Wijeratne >>>>>>>> Marketing Officer, WSO2 Inc <http://www.wso2.com> >>>>>>>> +94775496911 | @yudhanjaya >>>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> [email protected] >>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Dinali Rosemin >>>>>> University of Peradeniya (Computer Engineering) >>>>>> WSO2 Intern >>>>>> 077-0198933 >>>>>> >>>>>> _______________________________________________ >>>>>> Dev mailing list >>>>>> [email protected] >>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> ============================ >>>>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>>>> Site: http://people.apache.org/~hemapani/ >>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>> Phone: 0772360902 >>>>> >>>> >>>> >>>> >>>> -- >>>> Dinali Rosemin >>>> University of Peradeniya (Computer Engineering) >>>> WSO2 Intern >>>> 077-0198933 >>>> >>> >>> >>> >>> -- >>> Yudhanjaya Wijeratne >>> Marketing Officer, WSO2 Inc <http://www.wso2.com> >>> +94775496911 | @yudhanjaya >>> >> >> >> >> -- >> ============================ >> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >> Site: http://people.apache.org/~hemapani/ >> Photos: http://www.flickr.com/photos/hemapani/ >> Phone: 0772360902 >> > > > > -- > Yudhanjaya Wijeratne > Marketing Officer, WSO2 Inc <http://www.wso2.com> > +94775496911 | @yudhanjaya > -- ============================ Blog: http://srinathsview.blogspot.com twitter:@srinath_perera Site: http://people.apache.org/~hemapani/ Photos: http://www.flickr.com/photos/hemapani/ Phone: 0772360902
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
