I'm creating a new thread for this because a few others have mentioned it, and we haven't gotten a response yet. My hunch is that changing those APIs involve other teams within Twitter, so figuring out a solution could be challenging.
Here is the issue. We need to be able to get matches on the original URL through the streaming and search APIs. For me, I'm tracking "act" so I can match tweets that link to 'http://act.ly'. This is not a link shortener service, the actual pages live at act.ly, and it was all designed specifically for Twitter so there would be no need for url shorteners. As far as I'm concerned, it's fine if that link changes to t.co, as long as I can still get matches on act.ly (or act) through the streaming API (the search API is going to be important for people too, but less of an issue for me personally). The most elegant way to fix this would be to allow tracking of the original URL. So I can put in a domain name, or URL substring, and match everything that way. Same with search. This would be useful to a lot of people, and virtually all link oriented web apps with APIs provide a way to get all the matches for a particular domain. (digg, google, yahoo, etc) I'm sure there are other workaround ways of doing this, and I'm all ears. It would be SUPER NICE (wink wink) to hear some kind of assurance that there will be a way for us to query this type of information before the t.cochanges go live. Thanks guys... Jim Gilliam http://act.ly/ http://twitter.com/jgilliam On Tue, Jun 8, 2010 at 4:43 PM, Jim Gilliam <j...@gilliam.com> wrote: > Will we be able to get matches on the original URL through the streaming > API? > > For example, I'm tracking "act" so I can match tweets that link to ' > http://act.ly'. Will I still be able to do that? > > Jim Gilliam > http://act.ly/ > http://twitter.com/jgilliam > > > On Tue, Jun 8, 2010 at 4:33 PM, Dewald Pretorius <dpr...@gmail.com> wrote: > >> Raffi, >> >> I'm fine with everything up to the new 140 character count. >> >> If you count the characters *after* link wrapping, you are seriously >> going to mess up my system. My short URLs are currently 18 characters >> long, and they will be 18 long for quite some time to come. After that >> they will be 19 for a very long time to come. >> >> If you implement this change, a ton, and I mean a *huge* number of my >> system's updates are going to be rejected for being over 140 >> characters. >> >> On Jun 8, 7:57 pm, Raffi Krikorian <ra...@twitter.com> wrote: >> > hi all. >> > >> > twitter has been wrapping links in e-mailed DMs for a couple months >> > now<http://bit.ly/twttldmemail>. >> > with that feature, we're trying to protect users against phishing and >> other >> > malicious attacks. the way that we're doing this is that any URL that >> comes >> > through in a DM gets currently wrapped with a twt.tl URL -- if the URL >> turns >> > out to be malicious, Twitter can simply shut it down, and whoever >> follows >> > that link will be presented with a page that warns them of potentially >> > malicious content. in a few weeks, we're going to start slowly enabling >> this >> > throughout the API for all statuses as well, but instead of twt.tl, we >> will >> > be using t.co. >> > >> > practically, any tweet that is sent through statuses/update that has a >> link >> > on it will have the link automatically converted to a t.co link on its >> way >> > through the Twitter platform. if you fetch any tweet created after this >> > change goes live, then its text field will have all its links >> automatically >> > wrapped with t.co links. when a user clicks on that link, Twitter will >> > redirect them to the original URL after first confirming with our >> database >> > that that URL is not malicious. on top of the end-user benefit, we hope >> to >> > eventually provide all developers with aggregate usage data around your >> > applications such as the number of clicks people make on URLs you >> display >> > (it will, of course, be in aggregate and not identifiable manner). >> > additionally, we want to be able to build services and APIs that can >> make >> > algorithmic recommendations to users based on the content they are >> > consuming. gathering the data from t.co will help make these possible. >> > >> > our current plan is that no user will see a t.co URL on twitter.com but >> we >> > still have some details to work through. the links will still be >> displayed >> > as they were sent in, but the target of the link will be the t.co link >> > instead. and, we want to provide the same ability to display original >> links >> > to developers. we're going to use the entities attribute to make this >> > possible. >> > >> > let's say i send out the following tweet: "you have to check outhttp:// >> dev.twitter.com!" >> > >> > a returned (and truncated) status object may look like: >> > >> > { >> > "text" : "you have to check outhttp://t.co/s9gfk2d4!";, >> > ... >> > "user" : { >> > "screen_name" : "raffi", >> > ... >> > }, >> > ... >> > "entities" : { >> > "urls" : [ >> > { >> > "url" : "http://t.co/s9gfk2d4";, >> > "display_url" : "http://dev.twitter.com";, >> > "indices" : [23, 43] >> > } >> > ], >> > ... >> > }, >> > ... >> > >> > } >> > >> > two things to note: the text of the returned status object doesn't have >> the >> > original URL and instead it has a t.co URL, and the entities block now >> has a >> > display_url attribute associated with it. what we're hoping is that with >> > this data, it should be relatively easy to create a UI where you replace >> thehttp://t.co/s9gfk2d4in the text with the equivalent of >> > >> > <a href="http://t.co/s9gfk2d4";>http://dev.twitter.com</a> >> > >> > this means the user would not see the t.co link, but we all can still >> > provide the protection and gather data from the wrapped link. for the >> > applications that don't choose to update, the t.co link will be shown >> (and >> > the goal to protect users will be met). i just want to emphasize -- we >> > really do hope that you all render the original URL, but please send the >> > user through the t.co link. if you do choose to prefetch all the URLs >> on a >> > timeline, then, when a user actually clicks on one of the links, please >> > still send him or her through t.co. We will be updating the TOS to >> require >> > you to check t.co and register the click. >> > >> > related to this: the way the Twitter API counts characters is going to >> > change ever so slightly. our 140 characters is now going to be defined >> as >> > 140 characters after link wrapping. t.co links are of a predictable >> length >> > -- they will always be 20 characters. after we make this live, it will >> be >> > feasible to send in the text for a status that is greater than 140 >> > characters. the rule is after the link wrapping, the text transforms to >> 140 >> > characters or fewer. we'll be using the same logic that is in >> > twitter-text-rb to figure out what is a URL. >> > >> > look for an update to dev.twitter.com where we'll have a best practices >> > document on how to use these t.co links. >> > >> > what's the timeline? "soon" we'll enable this on @twitterapi, @rsarver, >> > @raffi, and a few other test accounts so you all have live data to play >> > with. on the timescale of weeks (to potentially a month or two), we'll >> roll >> > this out to everybody. >> > >> > of course, if there are any questions, just feel free to direct them to >> > @twitterapi! >> > >> > -- >> > Raffi Krikorian >> > Twitter Platform Teamhttp://twitter.com/raffi >> > >
