[twitter-dev] Re: API search, json results, 404 pages, and docs
Reposting. I should have fixed the horrendous spelling my phone dished out. Sorry for the double post, but this one should be clearer… Thanks. I have a feeling I'm going to go over the 20,000 a hour hit rate limit. Are those hard limits or does Twitter make concessions to busy sites? I can not have the user login, too much of a barrier. I figure each visit is going to be at best 2 API hits, and at worst, maybe 4. That's about 5000 unique visits per hour. I'll cache the page entirely so reload happy people to dot mess things up. After the 4 hits, there will be Ajax enabled options that allow them to drill in and hit Twitter more. 5000 a hour will soon become 2500 an hour. Respectible, but if I have my way, this site will get more traffic than that. Suggestions? -- Scott Iphone says hello.
[twitter-dev] Re: Twitter User List
I dint think this is a function of a workaround. This is a function of Twitter having a good policy in place to prevent abuse. You can do what you want by incrementally querrying the API. The API limits will make it take too long. Even with multiple accounts it will be months before you get a final list. Even then, I'm not sure you could keep on top of new user registrations. Having acces to this data could only be used for nefarious efforts. What you want would be a spammers dream. I think you would be better and faster to build a crawl farm and crawl all links on Twitter.com and parse the users out, bypassing the API. Even with the API, as you add new records, those records you just added will expire, delete, get banned, blocked etc. There is no way you could ever have a reconciled system. Consider if each username is an average 10 bytes. You have 520,000,000 bytes to download of just username data. Let's double that for http overhead and other misc data that will come over the wire. 1 billion bytes. That's a strongly conservative terrabyte of data that you would have to download once a day and reconcile against the previous day. A terrabyte of just usernames. Then you have all the CPU that you will need, network lag, time to insert into your data source. This is not something that can be worked around. This is simply a limitation of scale, one that can not be overcome. You need a direct link to twitters data sources, ideally from within their data center to reduce network lag. This probably will not be approved :) -- Scott Iphone says hello. On Jun 29, 2009, at 9:06 AM, Arunachalam arunachala...@gmail.com wrote: Even if i have my account whitelisted, which have 20,000 request per hour, i need to run for many days which is not feasible. Any other workaround. Any other way to get rid of these request limit. Cheers, Arunachalam On Mon, Jun 29, 2009 at 7:01 PM, Abraham Williams 4bra...@gmail.com wrote: There has been over 5200 profiles created. You could just start at 1 and count up. Might take you a while though. Abraham On Mon, Jun 29, 2009 at 07:55, Arunachalamarunachala...@gmail.com wrote: Any idea how to implement the same using php / any other language. Im confused abt the implementation. Cheers, Arunachalam On Mon, Jun 29, 2009 at 5:57 PM, Cameron Kaiser spec...@floodgap.com wrote: I am looking to find the entire twitter user list ids. Social graph method provides the way to fetch the friends and followers id, thorough which we can access the profile of the person using user method - show. But this requires a code to be written to recursively crawl the list from any starting id and appending the followers and friends id of the person without duplicating. Do we have any other API to get entire list. If not, any other ways apart from crawling to get the entire list. No, and no, there are no other ways. -- personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- Careful with that Axe, Eugene. -- Pink Floyd --- -- Abraham Williams | Community Evangelist | http://web608.org Hacker | http://abrah.am | http://twitter.com/abraham Project | http://fireeagle.labs.poseurtech.com This email is: [ ] blogable [x] ask first [ ] private.
[twitter-dev] Re: API search, json results, 404 pages, and docs
Hi Mark, I am just not there yet, 80% into the dev. If I can take this off list, I would be more than happy to explain to you what it is all about. Thanks for the reply and working with me on this. I plan on being public and getting a normal API key soon, as soon as I manage to come up with a name, which is being more elusive than ever for some reason. On Jun 29, 2009, at 3:02 PM, Doug Williams wrote: Can you include the details of what you are attempting to do and how you are going about doing it? 2500 uniques an hour is pretty strong, what site will this application be running on? -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Security Best Practices
I heard the other day that in the wake of the MJ stuff, a few high profile celebs accounts where hacked. Is this media hacking and there were just weak passwords, or their email accounts were compromised, or were these real live hacks where someone brute forced, or did otherwise nefarious acts to get in. Some clarification on these events would help to let us know where and how people are getting in, so we can tighten things up on our end. If the hacks are just email accounts being gotten into, there is nothing twitter apps need to do. If it is something else, there may be other things we can do to keep the accounts safe. Thanks. On Jun 29, 2009, at 3:34 PM, Alex Payne wrote: I wanted to point out a blog post (http://apiblog.twitter.com/security-best-practices-for-twitter-apps ) that addresses the coming Month of Twitter Bugs. Long story short: Twitter is in the loop, we've got security at the forefront of our daily work right now, and we're available to help if your application is identified as vulnerable or compromised. Please check out the new wiki page (http://apiwiki.twitter.com/Security-Best-Practices ) and let us know what's missing. Thanks! -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Security Best Practices
Got to love these headlines: http://mashable.com/2009/06/28/britney-spears-dead/ They clearly point the finger at twitter in the headline, but reading on, and it is clearly a twit pic issue. I see these all over the place. Have you considered some sort of vetting system for sites that are asking for twitter credentials on a 3rd party site? I can see that twitpic may not be able to use o-auth, as they want to be able to stand alone and a image host. If there was some sort of communication where you worked with the large sites like twit pic, it may help. As it is now, I fell for it, I read the headline, and thought ti was a twitter issue. Just some food for thought. On Jun 29, 2009, at 3:54 PM, Alex Payne wrote: Any recent celebrity-related compromises I'm aware of having been, as you said, media 'hacking'. The last issue I'm aware of that resulted from actually taking advantage of a security flaw in our system was the Mikeyy worm that was going around for a weekend several months ago. We've done a lot of security work since then, and there's more in progress. On Mon, Jun 29, 2009 at 15:40, Scott Haneda talkli...@newgeo.com wrote: I heard the other day that in the wake of the MJ stuff, a few high profile celebs accounts where hacked. Is this media hacking and there were just weak passwords, or their email accounts were compromised, or were these real live hacks where someone brute forced, or did otherwise nefarious acts to get in. Some clarification on these events would help to let us know where and how people are getting in, so we can tighten things up on our end. If the hacks are just email accounts being gotten into, there is nothing twitter apps need to do. If it is something else, there may be other things we can do to keep the accounts safe. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] How are people getting to the first tweet
Found a site today that gives you your first tweet. Looking over the API, I can not figure out how to do this. Would doing a date search, and going way back in time, to get the first tweet work? I thought the search API only afforded you 7 days. If I use REST, then I am not sure how I would do this other than paging through the users timeline, which would take a long time. I find that myfirsttweet.com is able to give very fast results. Are they pulling data from an archive? They mention whitelisting on the site, implying they are polling the API, but I can not seem to figure out how this is being pulled off. Thanks for any help. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Timeline excessive data
Looking at the timeline API calls, all seem to return much redundant data. Everything in the user section is repeated. If I return 200 items, I will get 200 copies of the user data. Is this correct? Seems like a lot of extra data to send across the wire. I was looking at the search API, which return much less, but also does not allow me to go back in time far enough. Suggestions? -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Counting retweets
Is RT an official Twitter funtion or a community invented convention? If I have a timeline loaded what is the best way to determine the number of retweets? There seem to be many formats in which RT is done. -- Scott Iphone says hello.
[twitter-dev] @username matches and hash tags
Has Twitter ever shared their logic for locating @usernames and hash tags? @([A-Za-z0-9_]+) The above regex seems logical but I can see faults. It will pick up the trailing domain in an email address. I could look for a whitespace in front, or nothing in front and do better. Then there is the (via:@username) which luckily has a colon, but there are cases where it does not, making a very special case. Hash tags can be strange like #hash/foo Any suggestions on the most reliable patterns? Twitter does a very good job, catching string,@username as just @username but not messing up on email addresses. -- Scott Iphone says hello.
[twitter-dev] Re: finishing application
I will look it over when I have a larger screen than a phone. I am guessing English is not your native language? I would start with getting someone to help you with the grammer on the first page and a few other pages. If you would like help in that area, more than happy to lend a hand. Drop me an email to scott@(the domain I sent this email from). -- Scott Iphone says hello. On Jun 30, 2009, at 8:10 AM, M1Sh0u m1shu2...@yahoo.com wrote: Thanks. I'll tweet a lot :)) I hope that users will use it :) What do you think about the website? Do you like the idea? Do you think that this idea will have success? Answer me if this is not an off topic subject.. Thanks again :)
[twitter-dev] Tweet threading
I am finding near all apps I use with twitter in some way or another fail at threading a conversation. Anyone have pointers for how to do this, based on the current twitter API, and whatever bugs have been uncovered, perhaps with workarounds? Each tweet has a 'in_reply_to_status_id', if I understand, the existence of in_reply_to_status_id, means that the current tweet is a child of some parent. tweet: id = 1234 in_reply_to_status_id = 5678 In the above example, tweet #5678 would be the start of the conversation, and tweet #1234 would be the reply? What has me stumped: http://twitter.com/criticalmassey/status/2383870573 Clearly a reply to something @ahem said, which started here: http://twitter.com/Ahem/status/2382725245 However, if I search.twitter.com for @ahem, I can get this conversation: http://dl.getdropbox.com/u/340087/Drops/06.30.09/twitter-b06e01bd-154810.png But it is missing the parent, the start of the thread. I can see the master parent here: http://search.twitter.com/search?max_id=2410761989page=2q=+from%3Aahemrpp=10 But threading is not an option. Having a hard time wrapping my head around what the limitations of threading are. Any suggestions on how to better understand this would be most appreciated. Ideally, what I am looking for, is to take any tweet, determine what other replies there are to it, and get back to the parent, displaying all children. I would like to avoid any ambiguous tweet searches that are not based on a message-id, and could pollute the results with inaccurate threading. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Timeline excessive data
Thank you, makes sense. On Jun 30, 2009, at 7:35 AM, Abraham Williams wrote: The bandwidth is cheaper for Twitter then the cycles to drop duplicate user objects. Abraham On Tue, Jun 30, 2009 at 02:25, Scott Hanedatalkli...@newgeo.com wrote: Looking at the timeline API calls, all seem to return much redundant data. Everything in the user section is repeated. If I return 200 items, I will get 200 copies of the user data. Is this correct? Seems like a lot of extra data to send across the wire. I was looking at the search API, which return much less, but also does not allow me to go back in time far enough. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Why are there pagination limits
Been pondering this today. There seem to be 7 day limits, or around 3000 tweet limits to the API. At first, my gut told me that was for load reasons, and it made sense. I started thinking about paging results in development projects I have worked on. Looking at this from a database perspective SELECT foo, bar from something where name = 'test' ORDER BY id limit 1, 200; Start at id 1, get me 200, may take x seconds. Next page: SELECT foo, bar from something where name = 'test' ORDER BY id limit 200, 200; Start at id 200, get me 200, may also take x seconds Arbitrary page: SELECT foo, bar from something where name = 'test' ORDER BY id limit 5000, 200; Start at id 5000, get me 200, will also take x seconds In each case, x as time, does not change. Now, this assumes all the data is in a single database, or is normal in a way that facilitates this. This question is just one of curiosity. I am betting, the tweets table has been distributed across many tables, and there is no simple way to get to the pages results as shown above? If it is not, I am not seeing any performance hit to getting the first 100 records, or a subset that is 20,000 tweets into the record set. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Counting retweets
Good to know, thank you very much. Looks like, RT, Retweet, and Via are the most standard, and anything outside of that would be rather obscure. On Jun 30, 2009, at 10:11 AM, Doug Williams wrote: Retweet is not a top level feature for Twitter. People have suggestions for formats though [1]. A cleaver regex should help you parse if a tweet is a retweet. http://www.google.com/webhp? complete=1hl=en#complete=1hl=enq=retweet +formataq=foq=aqi=fp=LH9toxtiWpk Thanks, Doug On Tue, Jun 30, 2009 at 9:56 AM, Scott Haneda talkli...@newgeo.com wrote: Is RT an official Twitter funtion or a community invented convention? If I have a timeline loaded what is the best way to deter -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: @username matches and hash tags
Awesome, thanks. I found I had to use a B, so this works form me: \B@([A-Za-z0-9_]+) Word boundary is indeed very handy, works perfect, and the hash tag one works close enough. I know there are limits on length, what are they for both hash tags and @usersnames? On Jun 30, 2009, at 10:57 AM, Chad Etzel wrote: For usernames, you could add the word boundary special character (sometimes dependent on your programming language) \b@([A-Za-z0-9_]+) which should avoid email addresses. For hashtags I use a similar one: \b#([A-Za-z0-9_-]+) I don't think #foo/bar is a valid hashtag, so I don't account for those. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Tweet threading
Hope this is not out of line, but this list has been pretty busy lately in traffic, and I am looking for a little hand holding on tweet threading... so bump :) On Jun 30, 2009, at 3:53 PM, Scott Haneda wrote: I am finding near all apps I use with twitter in some way or another fail at threading a conversation. Anyone have pointers for how to do this, based on the current twitter API, and whatever bugs have been uncovered, perhaps with workarounds? Each tweet has a 'in_reply_to_status_id', if I understand, the existence of in_reply_to_status_id, means that the current tweet is a child of some parent. tweet: id = 1234 in_reply_to_status_id = 5678 In the above example, tweet #5678 would be the start of the conversation, and tweet #1234 would be the reply? What has me stumped: http://twitter.com/criticalmassey/status/2383870573 Clearly a reply to something @ahem said, which started here: http://twitter.com/Ahem/status/2382725245 However, if I search.twitter.com for @ahem, I can get this conversation: http://dl.getdropbox.com/u/340087/Drops/06.30.09/twitter-b06e01bd-154810.png But it is missing the parent, the start of the thread. I can see the master parent here: http://search.twitter.com/search?max_id=2410761989page=2q=+from%3Aahemrpp=10 But threading is not an option. Having a hard time wrapping my head around what the limitations of threading are. Any suggestions on how to better understand this would be most appreciated. Ideally, what I am looking for, is to take any tweet, determine what other replies there are to it, and get back to the parent, displaying all children. I would like to avoid any ambiguous tweet searches that are not based on a message-id, and could pollute the results with inaccurate threading. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Tweet threading
What is the getMentions method? I did not see that in the twitter API. I suspect this is a framework you are referring to? Or are you talking about this: http://apiwiki.twitter.com/Twitter-REST-API-Method%3A-statuses-mentions?SearchFor=mentionssp=1 That seems inaccurate though, would it not be better to start with the in_reply_to_status_id fields? Thanks for any help. On Jul 1, 2009, at 8:35 PM, Arnaud wrote: Take a look on the app I'm workig on, Twitoaster: http:// twitoaster.com The threading part is not that hard. Recursive function jumping from parents to parents. You should use the getMentions method, instead of hiting the search API. You'll get the full object that way, so you won't have to use the show/statuses method. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Pull email address after authentication?
On Jul 8, 2009, at 4:22 PM, Doug Williams wrote: It is not possible to view a user's email address. Additionally, it is not possible to perform a user lookup based on an email address. This is not entirely true, though can not be done through the API. I had friends find me on twitter, and for month, never knew why. It is quote simple to load an email into gmail, and add that to twitter and it will look up any user with that email. I wish this feature was able to be disabled. I have had to create a junk email account in twitter, just so I have some privacy. I get that the posts are not private, but I had an account that was not something I wanted people to connect with me. That is too late now. No where on the twitter Web site is it explained that your email address can be used as a way for users to locate you. I personally believe, this should be explained in detail on the join page where you enter in your email address. And I should be able to opt out of this feature. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: How to insure that all tweets are retrieved in a search?
You are correct, you have to do 15 requests. However, you can cache the results in your end, so when you come back, you are only getting the new stuff. Twitter has pretty good date handling, so you specify your last date, and pull forward from there. You may even be able to get the last id of the last tweet you pulled, and just tell it to get you all the new ones. On Jul 9, 2009, at 2:45 PM, owkaye wrote: I'm building an app that uses the atom search API to retrieve recent posts which contain a specific keyword. The API docs say: Clients may request up to 1,500 statuses via the page and rpp parameters for the search method. But this 1500 hits per search cannot be done in a single request because of the rpp limit. Instead I have to perform 15 sequential requests in order to get only 100 items returned on each page ... for a total of 1500 items. This is certainly a good way to increase the server load, since 15 connections at 100 results each takes far more server resources than 1 connection returring all 1500 results. Therefore I'm wondering if I'm misunderstanding something here, or if this is really the only way I can get the maximum of 1500 items via atom search? -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: How to insure that all tweets are retrieved in a search?
You are correct, you have to do 15 requests. However, you can cache the results in your end, so when you come back, you are only getting the new stuff. Thanks Scott. I'm storing the results in a database on my server but that doesn't stop the search from retrieving the same results repetitively, because the search string/terms are still the same. My problem is going to occur when thousands of people start tweeting my promo codes every minute and I'm not able to retrieve all those tweets because of the search API limitations. If I'm limited to retrieving 1500 tweets every 6 minutes and people post 1000 tweets every minute I need some way of retrieving the missing 4500 tweets -- but apparently Twitter doesn't offer anything even remotely close to this capability -- so I can see where it has a long way to go before it's ready to support the kind of search capabilities I need. Have you read this: http://apiwiki.twitter.com/Rate-limiting Section on search rate limiting. I do not believe there is a rate limit on search. I am sure there is a very high limit, but you should be ok. I would also suggest you get whitelisted, that bumps you up to 20,000 request per hour I believe. Further, you could make the users o-auth, in which case, the hit counts to them, not you. You will still get stuck with the fact that search is a pretty broad thing, and not exacting, but you should be able to complete your searches in the amount you want. I have seen apps that I know are using search on their back end, and they are not requiring auth by the user, and they are further pretty heavily hit. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Getting tweets from Twitter API
From what I understand, it is UTC time. The +/- is the offset depending on what zone you are in. This allows for a time value that is the same across the world, but can be offset for any particular locale. I think http://en.m.wikipedia.org/wiki/Coordinated_Universal_Time will explain it adequitly. You can also read in the time value from the API and compare it to the time shown on Twitter.com. Then change your preferences to alter your time zone. This is a good way to see how the offsets work as you go. -- Scott Iphone says hello. On Jul 10, 2009, at 10:30 PM, praveen kumar praveen.neteli...@gmail.com wrote: The time field returned contains the offset (usually +) What is the meaning of this is it GMT+0 or System time. can u please explain it. On Fri, Jul 10, 2009 at 12:39 PM, Kevin Mesiab ke...@mesiablabs.com wrote: The time field returned contains the offset (usually +) Tue Apr 07 22:52:51 + 2009 On Thu, Jul 9, 2009 at 7:13 PM, praveen kumar praveen.neteli...@gmail.com wrote: Hi, If we are getting tweets from Twitter API , User's tweet dates are in which timezone. Is it in GMT or else different timezones. -- Regards, Praveen Kumar.N -- Kevin Mesiab CEO, Mesiab Labs L.L.C. 208-447-6016 http://www.mesiablabs.com http://www.plsadvise.com -- Regards, Praveen Kumar .N Software Engineer Netelixir e-Marketing Solutions Hyderabad www.netelixir.com
[twitter-dev] Re: FYI - Digg hijacking links
I think that you should look at tr.im. They make it rather clear that they have no intention of ever doing something like this. Their API is really nice to work with. I had one issue that I exceeded their limits on a non API key account, and my IP was blocked. It took them way too long to deal with solving that, and support was not of any help. I finally just waited the few days to get my IP block released. That being said, any url hijacking is counter to the entire way tr.im wants to make their system work. I also find their site, and the integration with twitter, to be very nice. They have some pretty good stats and charts, as well as solid preferences to customize how it all works for you. You can even tell tr.im to go get all your previous tr.im url's and bring them into your account. It is fast, and has had the least downtime for me aside from tinyurl. TinyUrl is nice, but I feel I need the extra characters when I only have 140 - username - @ - url. I do not have any affiliation with tr.im, I just like how they operate and how their system works. On Jul 20, 2009, at 1:45 AM, Vision Jinx wrote: I just hate the way some services have to quickly follow trends and does this mean other short url services are going to port all traffic to their sites now and force people to sign up and create accounts there? (and increase the number of ads they are serving) -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: following twitter conversations by topics
Can you tell me more about this auto topic discovery feature? I am not seeing anything of that nature on the twitter Web site at all. On Aug 5, 2009, at 2:29 AM, AJ Chen wrote: After playing around with auto-discovery of topics in twitter conversations for a while, it seems to me that following topics is another effective way to communicate on twitter. So, I've added a new set of features on http://web2express.org website to make it easy for people to follow and tweet about topics. The auto- discovered topics give you a fairly good starting point to read about current new hot topics. In addition, one can create any topic to follow. By adding of a set of keywords or phrases to the topic, you will find that topic following pulls out more complete list of conversations from twitter with much less work. When you tweet or retweet about a topic, the topic's hashtag is automatically appended to your message so that you don't need to remember what hashtag to use. I notice several websites including twitter.com homepage have started to provide similar functions recently. Anyone has any early experience or comment to share? -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: The silence is deafening....
I agree. I also think it is very important to recognize Twitter made a strong move with such an open API. As a result, it is just as important to recognize, Twitter very well may not be where it is today were it not for third party apps. I may go as far as to say the API should be a higher priority than the Twitter.com site. I can say with zero doubt, I myself would not have learned to love Twitter were it not for tweetie on the iPhone. I know every single one of my friends can say the same, though it may be a different app and a different phone. Stats from Twitter on this would be interesting. I bet majority use is Twitter.com . However, I bet majority use where there is real interaction is all third party apps. Web user traffic is the casual login once a week user. Third party apps are what made Twitter I household name. To me, the API should be equal or more a priority, especially since a down API gets the third party app blamed, not Twitter. -- Scott Iphone says hello. On Aug 8, 2009, at 5:03 PM, chinaski007 chinaski...@gmail.com wrote: If Twitter.com itself were down, you know that they would stay there until it was back up. But since it is just a large number of third party apps that are down... well, hey, it's a weekend in August!
[twitter-dev] Re: 302s are NOT the solution
Can someone point me to the details on the attack? I am a little out of the loop. I've heard Twitter only uses around 200Mbit/s of data. From a net ops perspective, why is this challenging to detect and block? I'm not trying to degrade the efforts of the engineers, this is a genuine question of curiosity. I would imagine a detection system is in place, so why not block off at the upstream the offending attack? As far as the API is concerned, I'm not sure I see why this can't be prevented in the future. If every Twitter app had to get an API key, which I believe is the case, those get whitelisted, all else are blocked. Create a test sandbox for easy non key based testing of new developers who want to play. There are a few thousand third party apps, whitelist their secret keys and how is this not solved for API reliability? -- Scott Iphone says hello. On Aug 8, 2009, at 5:09 PM, Howard Siegel hsie...@gmail.com wrote: I support them wholeheartedly and appreciate everything they've done to thwart the DDOS attack. While it is true that many of the tools used in the attack do not appear to follow the 302s right now, you can be your bottom dollar that they will very quickly be updated to do just that, perhaps even quicker than Twitter can finish recovering from the attack and put in to place measures to better survive future attacks. At best it is a stopgap to get over the current attack.
[twitter-dev] Re: OK Seriously People
In all honesty, I think you are stuck. The responsible thing to do is follow this list and wait until the API is reported 100% functional. Writing a new app is bound to have testing issues, currently you have zero way of knowing if it is your app, the API, routing blocks, or even congestion. You will waste time, sanity, and money by trying to develop in this environment. I think all you can do, if wanting to do the right thing by your employer, is tell them no future work can be done at this time. Of course, there may be other areas you can work on, like UI. There is enough data in the API wiki you can mimic some of the results and hobble along. I still think the only smart thing to do is wait. This may be a good lead in to a conversation about relying entirely on another service for a source of a business model. -- Scott Iphone says hello. On Aug 9, 2009, at 10:42 AM, Ryan dunn.r...@gmail.com wrote: I have just started writing a Twitter App for a small company (great timing lol). Part of it works and part of it is down. Is there any where else I can check to see which parts of the API are down? I have continually checked status.twitter.com, however I want to make sure that any errors I'm seeing currently are caused by the twitter outtage and not my own (possibly errant) code. Basically I'm under a deadline and if I need to contact my superiors I want to make sure that I'm right in assuming sections of the API are truly down.
[twitter-dev] Re: FW: Twitter is Suing me!!!
It may get even harder and open the door to an already hot topic with T.W.I.T. (The Week In Tech) which is a show by Leo Laporte. I believe this show pre-dates the use of twit, and nay pre-date Twitter. I seem to recall at some point Leo Laporte would not even use Twitter as a result of feeling they were infinging on his name. I don't know the trademark status with regard to this, but he has been vocal about it to say the least. If anything, Laporte has prior art in this one. -- Scott Iphone says hello. On Aug 12, 2009, at 4:55 AM, Goblin stu...@abovetheinternet.org wrote: The question is, are they going to be going after Twitteriffic, Twitterholic, Twitpic, Twitvid, Twittelator, Twitterena, Twitterfon, iTwitter etc? I admit that I was fair game having the blue birds in the backdrop (as I say, it was a stupid project that got traction and the new version is live now anyway), but if Twitter is deciding to take down everyone with Twit in their name then there are going to be some serious issues. I know they have to show they are attempting to protect trademark or risk losing it, but this seems a little heavy handed :(
[twitter-dev] Re: FW: Twitter is Suing me!!!
I don't know how we get to the point of meaningful auto following. That seems hard to define. If I post a tweet mentioning photography I get 5-10 new followers in a few minutes. Use the word cock or pussy and the auto follow rate is higher. Hash tags are vulnerable here as well. In most cases, I've yet to see them come back and solicit me in any way, so I will never notice them aside from my wrongly inflated follow metrics. Under what conditions could any automated system follow someone in a meaningful way? There is no way to tell context. My suggestion is that the apps should bring in a list of suggested followers. The user of the app must review the list, and the corresponding tweets and decide to follow or not. Further, after a follow has been established, you should not communicate with that person in an unsolicited way. Treat it like email/spam. If I am bashing a brand and they want to make good on my issues, they can tweet me. They need not even follow me to start this relationship. If they want to DM me they can then start that relationship. For Starbucks to auto follow because I said starbucks sucks is not meaningful. To me, the nature of auto follow shows a wanting to use Twitter as a way to market, brand, and build relationships. That much is fine. But they want to do so in as easy a way possible. That is where I draw the line. If you want to market to me, you must put in the time to learn about me and my value to your brand. Having a robot do it for you is dirty, and in the end, will probably hurt more than help. If someone can tell me how automatic auto follow is not analogous to the end result of a spam network, my opinion may change. In the end, there is no easy way to build and establish relationships with your end users. Either do the work, or stop abusing Twitter. * The above statements are a generality, the OP may have learned a meaningful way to auto follow that I am not seeing. I do not intend to call out the OP in any way. -- Scott Iphone says hello. On Aug 12, 2009, at 5:17 AM, Dale Merritt mogul...@gmail.com wrote: What is Twitters real stance on auto following? In there API they prohibit mass following so what does that mean exactly. More than 1, 100? In my app, I had planned on integrating some meaniful auto following
[twitter-dev] Re: FW: Twitter is Suing me!!!
The second you can play drinking games based on how many times a company is mentioned on local news; I think that companies ability to be clear and unambiguous becomes as hard as not getting hammered in 5 minutes of watching the news. -- Scott Iphone says hello. On Aug 12, 2009, at 6:04 AM, Dewald Pretorius dpr...@gmail.com wrote: Logically, isn't it necessary that a clear and unambiguous definition of aggressive following to be publicly available before any legal action can be based on it?
[twitter-dev] Re: FW: Twitter is Suing me!!!
I was always under the impression trademarks came down to a reasonable expectation that users could be confused as to which was the original name. In the example of the Edge iPhone game, he was called out against using Edge as a name since there was a shady software development house that owned the mark in the gaming industry. There very well could have been confusion. Edge can safely be used in other industry. If I started Edge Clothing I suspect there would be no confusion and no issue. Twitter makes sense, their only presence being the web. A user who hears Twitter, downloads a Twitter app by searching for one in their mobile store, could easily walk away thinking that mobile app is Twitter. They could easily think they are at the source. I don't see why they care, they care about service users. Alienate the 3rd part apps and Twitter is going back in time, not forward. It is the existance of this API that I believe made Twitter what it is. But tweet, I don't know the history. Is it like hash tags, a user invented term for an action within the service? To me, this is as if eBay tries to trademark selling because that is the verb describing what you do within the service. Tweet is the verb within the service, and likely was user crafted anyway. All technical and legal issues aside, this boils down to just doing the right thing. Trademark Twitter, that seems fair and smart. There will be some fallout while protecting the name, but that is just part of the process. Trademark tweet, and you are doing the wrong thing. From podcasting, syncing, downloading, IM'ing, DM'ing, texting, etc. Just leave it alone. They will do far better to allow the word to become more a household term, than to aggravate everyone. -- Scott Iphone says hello. On Aug 12, 2009, at 5:24 AM, Vision Jinx vjn...@gmail.com wrote: FYI - mashable.com just posted a story on this here http://mashable.com/2009/08/12/twitter-not-suing-developer/ Interesting to know that if Twitter gets the trademark for Tweet also what about the apps and businesses that have been using it before the claim like TweetDeck etc etc? Seems they would have a justifiable claim to the name also? How does that work? Not sure about the US but where I am if someone has made a stable reputation from a name they are also entitled to a trademark clause for it regardless if they officially trademarked it. (in my own words) The owner of a common law trademark may also file suit, but an unregistered mark may be protectable only within the geographical area within which it has been used or in geographical areas into which it may be reasonably expected to expand. http://en.wikipedia.org/wiki/Trademark Disclaimer: I'm not a lawyer and don't play one on TV either so seek your own legal council regarding it.
[twitter-dev] Re: FW: Twitter is Suing me!!!
I am not a lawyer, but everything I have read about this makes the below impossible. If you have a trademark on a name, you MUST protect it. Failure to protect it, results in loss of the mark. The quote below, clearly states that Twitter is not going to protect this mark. That being the case, they will lose the mark, making the filing and costs associated with it a complete waste. The statement below very well may invalidate the mark entirely. On Aug 12, 2009, at 2:52 AM, Rich wrote: I'm not aware of this but this link http://blog.twitter.com/2009/07/may-tweets-be-with-you.html , published only last month says We have applied to trademark Tweet because it is clearly attached to Twitter from a brand perspective but we have no intention of going after the wonderful applications and services that use the word in their name when associated with Twitter. In fact, we encourage the use of the word Tweet. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: FW: Twitter is Suing me!!!
Others use your following and follower list as a way to bridge new connections to interesting people. If I am interested in Person X, I can look at who is following him, and know that others are probably of like minds. I can dig into their list of followers and following, and build deeper relationships that are only separated by small degrees. If I go to someone's account and they have 500 followers, 90% of them are adult porn and social media experts, not only do I get the wrong impression as to what that person is about, but I also have a completely off kilter signal to noise ratio. On Aug 12, 2009, at 10:07 AM, Dewald Pretorius wrote: In other words, what does it matter if 50,000 undesirable accounts follow you, except for perhaps a minor personal irritation? Do you want to be selective about who is allowed and who is not allowed to follow you? -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] In case you have not heard, Ashton Kutcher made twitter famous
This is a little OT, and while somewhat tongue in cheek, I thought it amusing, just for the utter inaccuracy of it, as well as how some peoples minds works: Suffer through the ad, and go to 20:00 in the video http://www.hulu.com/watch/89201/late-night-with-jimmy-fallon-tue-aug-11-2009 Twitter is talked about, Ashton is under the impression the current rise to fame was his doing. I suggest he set up a paypal account, and we can all donate to him for his service to this community. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: FW: Twitter is Suing me!!!
I used to ass well, this does not work well when number grow. On Aug 12, 2009, at 11:54 AM, JDG wrote: sure they do. it's called blocking. every time a pain in the ass porn bot or social media expert following 100x more people than follow them follows me, i block them. then they can't follow me. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: FW: Twitter is Suing me!!!
On Aug 12, 2009, at 11:42 AM, Dewald Pretorius wrote: Personally I think that is a mutilation of the use and purpose of Twitter. I surely hope people would not judge me based on who is following me. I would not judge you. No, I never take it that far. However, consider if a politician was on Twitter, and a mass follow of spam porn bots hit that person up. People unlike you and I, and others on the dev list here, very well will get the wrong impression, simply for lack of understanding how this all works. Whether it is a mutilation of the use and purpose, I can certainly see your point. Though to be honest, I do not know a person who has not looked up one person, to find who they follow and who follows them. It very much is the entire premise of myspace, facebook, and linked in. I personally use it for that purpose all the time, though I have learned based on username and profile picture how to discern a real user and a bot pretty easily with relatively good accuracy. It does muddy the pool, and make me have more pages of stuff to wade through. The only way one can maintain a clean list of people who follow you is to block those whom you don't want as followers. Also, you can privatize your account, which to be honest, I wish was not even an option, as that indeed is a mutilation of the use and purpose of twitter in my humble opinion. But tell me how feasible it is to remove followers once you get past a few thousand? It is going to cause the suspension of many innocent Twitter accounts if people block others simply because they do not agree with their political views, the way they comb their hair, or their profession. Can you elaborate on this? How does a block of a twitter account end up as a account suspension? Do multiple blocks tally up and end up being used as a way to suspend an account? Followers do no, zero, nada harm. Just let them be. I think I have shown how they do in fact harm. Even at a very basic level, if a army of bots were to follow you, say, a million of them, what would you do? You yourself would look like part of the army, you would not be able to maintain that account easily, if you put aside your ability to programatically remove those bots. Follower and following as also an at a glance metric I use to see how interesting someone may be. If they have 5000 followers, but only follow back 100, I can gauge immediately the type of twitter use that person is. If they have 5000 followers and around the same following counts, I know there is no way they are engaging all 5000 of those people, and that account has less value to me. These are pretty rough generalizations, but I certainly do not agree with the no harm statement. It just depends on your use and how you define harm, which to me is defined as inconvenience. On Aug 12, 3:28 pm, Scott Haneda talkli...@newgeo.com wrote: If I go to someone's account and they have 500 followers, 90% of them are adult porn and social media experts, not only do I get the wrong impression as to what that person is about, but I also have a completely off kilter signal to noise ratio. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: My Issue with the ReTweet API and my solutions
Good point. There are a lot more broken things that have higher priority. Top of the list, notify all third party clients that do not attach a message ID to a reply. Threading of tweets is so fundamentally broken it should be way up on the list. Sure, not twitters fault, but the third party apps make Twitter, and in this case, they break Twitter. The first time user experience is degraded if they have to go digging to find why a tweet seems so out of context. I think Twitter needs an RFC style set of guidelines. Using words like SHOULD, MUST, MUST NOT, and MAY WOULD help a great deal. I'm not saying to have an app store approval process, but definately a way to officially define a behavior as broken. Back to retweets, there are and were too many ways to do it. Do you inject your commentary to the end or beginning of the retweets? Some use RT, others use (via @username). Most lean on RT for shortness, but the via is much more clear to a new user. Either way it is not immediately clear to a new user what's going on. For something so simple, there is way to large a barrier of learning to a new Twitter user. Of all the people who say I don't get it, it boils down to problems with this barrier. I always say, give it a week, you will learn the value and utility and fun. In most cases that is true. That's a core problem that needs to be solved. Above and beyond work put into a retweeting feature. There is just too much I don't get it for something so fundamentally simple as instant messenger makes happpy fun time with RSS limited to 140 characters. My .02. -- Scott Iphone says hello. On Aug 17, 2009, at 5:25 AM, Vision Jinx vjn...@gmail.com wrote: Long story short... I just keep thinking, If it ain't broken, don't fix it ;) That's my $0.02 on it.
Re: Should your favorites be public information? (was RE: [twitter-dev] Re: My Issue with the ReTweet API and my solutions)
On Aug 17, 2009, at 11:40 AM, Brian Smith br...@briansmith.org wrote: Paul Kinlan wrote: Favorites are open to be read, it is just that not many people use it and I can't actually find who favorited my tweets - (probably no one in my case ;) - if I had that information I could do a lot of things (with out resorting to the RT stream). I tried it and you are right, I can read anybody's favorites. But, is that intentional? I had always thought my favorites were private and I think that other users have that same expectation of privacy. - Brian Am I the only one who thinks favorites is fundamentally wrong? With so much noise, is there any value in marking one tweet as the signal? I bastardize it and use it as a way to make a list of my favorite users. This allows me to easily get to a small handful of users. I don't care what their tweet was, I just need an easy way to get to them. This is what I suspect most average users use favorites for. Marking a user. I asked a few friends just now and they all thought it was a favorite user feature. One thought it was broken because he could mark the same user more than once. -- Scott Iphone says hello.
[twitter-dev] Re: My Issue with the ReTweet API and my solutions
On Aug 17, 2009, at 1:38 PM, iphone.noob wrote: Will wrote: My mindset was that why hold back a feature that solves one problem just because it doesn't solve two. What I didn't take into consideration is that it creates another problem - disparate methods of retweeting because people will revert back to the old method of retweeting when they DO want to comment on the original tweet. That makes any method of aggregating retweets posted via the API method incomplete. That is a BIG minus. Almost big enough for me to switch sides on this issue. You're right on the mark with your comments. Nearly everyone in my Twitter social circle uses RT with added commentary. In most cases they truncate or abbreviate the original message to make room for their addition. A Retweet function without a text attribute is really surprising. That is the core issue, there is going to be fragmentation. Developers are not going to even want to implement, if some use it, and others do not. The users are going to decide this one, and I suspect the users are not going to use it, since it offers less than the old way. This adds easier and more perfect search results, something I suspect users are not using much anyway. Twitter has a lifespan of a few days, after that, those tweets are no longer in any users head, and they have moved on. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Twitter API docs 404 URL
Hello. The Twitter API has a mobile version. I have never seen it work. http://apiwiki.twitter.com/Getting-Started As you can see, if you are detected as mobile all API doc pages 404. I click the desktop version link at the bottom and everything works as advertised. I thought I would bring this to the attention of Twitter. I find if I look at the API on my iPhone to answer questions that are mostly small curious inquries. Not a huge deal, but it would be nice if it worked, and I suspect it will be a simple fix. -- Scott Iphone says hello.
[twitter-dev] Re: Do My Customers Have a Twitter Account?
Good point. Why not just send them an email, and offer to let them follow you? This puts it as a opt in on their part. You can then follow them back if you desire, which I assume you do. Probably not a good idea to do anything of this nature when you are talking about 400K. On Aug 19, 2009, at 1:14 PM, David Fisher wrote: Sounds like something you should be able to do in an email to them or with a message on your website. On Aug 19, 1:29 pm, arawajy araw...@gmail.com wrote: I want to invite them to follow the company on Twitter. On Aug 19, 8:25 pm, Andrew Badera and...@badera.us wrote: On Wed, Aug 19, 2009 at 11:07 AM, arawajyaraw...@gmail.com wrote: Dear Developers, I have a list of 400,000 e-mail addresses of my clients. I want to know Is it possible to develop a script to check if they have a twitter account or not?. I will then want to generate 2 separate lists based upon the result; one for the twitter users and one for the non-twitter users. I want to only invite the users and create a custom invitation message. Is it possible to check if the e-mail address's owner is a twitter user or not? provide details please. Thanks and Regards, Mahmoud If they're already your clients then what are you inviting them to? -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] User agent
Playng with the http://apiwiki.twitter.com/Twitter-REST-API-Method%3A-statuses%C2%A0update part of the API today. I can post easily with curl: curl -u user:password -d status=foo bar baz The Tweet says it is from API. How do I set the user agent as I have seen other developers do? It is not listed on the above API page. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Do My Customers Have a Twitter Account?
My only statements regarding ethics or morality is that with a list of 400K people, there will be many ways in which those 400K people interpret what you are doing. So you were able to do 1700 emails so far, can you figure out what the limit is? At that point, I would probably use a carefully made screen playback tool to help automate what you are doing. It is not pretty, but it will get you the result you are after. On Aug 19, 2009, at 1:17 PM, Mahmoud Abdur-Rahman wrote: This is already the approach I've followed and it worked. I created a contact (CSV) file and imported it into a gmail account. It was around 1700+e-mail addresses which correspond to one segment of the customers. I got the list and I copied them and started to omit the information I don't need. But when I tried to work with the second group in order it didn't work as there're 30,000+ e-mail addresses. I tried to split them into 2 groups, it didn't work. Simply the gmail and Yahoo! are not able to import them due to the large number. I didn't get your concern about the ethics and privacy. I didn't crawl those e-mails or buy them. They're our customers and we want to invite them to our page. They gave us their e-mail addresses so willingly and I guess we're not misusing them. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Do My Customers Have a Twitter Account?
From the google docs on importing: • You can import 3,000 contacts at a time. • Non-ASCII or non-Latin characters may not be accepted. • Any information formatted as a group or distribution list won't transfer into your Gmail Contacts list. • Importing information that matches the email address of an existing entry will replace the information in your Contacts list with the most recently uploaded information. Looks like you have about 133 files to split and script into it :) On Aug 19, 2009, at 1:17 PM, Mahmoud Abdur-Rahman wrote: This is already the approach I've followed and it worked. I created a contact (CSV) file and imported it into a gmail account. It was around 1700+e-mail addresses which correspond to one segment of the customers. I got the list and I copied them and started to omit the information I don't need. But when I tried to work with the second group in order it didn't work as there're 30,000+ e-mail addresses. I tried to split them into 2 groups, it didn't work. Simply the gmail and Yahoo! are not able to import them due to the large number. I didn't get your concern about the ethics and privacy. I didn't crawl those e-mails or buy them. They're our customers and we want to invite them to our page. They gave us their e-mail addresses so willingly and I guess we're not misusing them. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Stop playing around with Source parameters
Ha ha. Php does have a regex parser. This should not matter if coded correctly. Twitter did announce they would be messing around last week. -- Scott Iphone says hello. On Aug 22, 2009, at 3:07 AM, Andrew Badera and...@badera.us wrote: Crazily enough, not everyone writes in PHP. ∞ Andy Badera ∞ This email is: [ ] bloggable [x] ask first [ ] private ∞ Google me: http://www.google.com/search?q=(andrew+badera)+OR+(andy+badera ) On Sat, Aug 22, 2009 at 3:28 AM, Joel Strellnerj...@twitturly.com wrote: Ummm... strip_tags()'s? On Fri, Aug 21, 2009 at 9:17 PM, TCI ticoconid...@gmail.com wrote: Recently you added nofollow's, and now you moved the nofollow after the href. Some of us filter these out and you changing them is only making it more complicated. Please make up your mind and stop changing these... a href=http://fun140.com/;Fun140/a a rel=nofollow href=http://fun140.com/;Fun140/a a href=http://fun140.com/; rel=nofollowFun140/a
[twitter-dev] Bug: negation search
Building my ever growing exclusion list for Twitter to get more signal and less noise... #php -lance -phpge3k -stackalert -RT -imoracle -#freelance -phplinks - web2feed -freelance -webtechman -tuvinh -freelancer -job -jobs That works. #php -lance -phpge3k -stackalert -RT -imoracle -#freelance -phplinks - web2feed -freelance -webtechman -tuvinh -freelancer -job -jobs -wefollow That does not work. Any combination of follow cussed an error. I'm only testing on a mobile, in tweetie, but on mobile safari, in the search box, results are the same. search.twitter.com seems immune. It is not length bound. Just -follow, will also not work. This appears API limited for the most part. -- Scott Iphone says hello.
[twitter-dev] Re: using Twitter API with PHP
If you don't want a public community API, and want to write it in your own, then you are sort stuck. The docs show you sample curl commands or you can http request in a browser to test simulate. You have sample curl examples, if you are objectionable to libraries, then warp your php around those curl commands. If that is not what you want, it sounds like you should look for an introduction to php tutorial, or a general introduction to web based scripting languages. -- Scott Iphone says hello. On Aug 23, 2009, at 1:45 PM, Termanater13 termana...@gmail.com wrote: Ive been looking to use the Twitter API with my php Site and I cant find much just a bunch of URLs used wuth the Twitter API and the curl commands used with the API. Can someone point me in the direction of where to start, and please nothing done by the comunity unless its a tutorial on how to write my own if at al possible.
[twitter-dev] Re: get the id's of twitter users who have authenticated with OAuth?
I agree with your disagreement. The other day I was playing with a service that made a background. When I clicked done, I thought it would prompt me to save the image and I would be on my own to upload it into my account. That is not what happened. It auto replaced my background. I also did not have a backup of my original background, and now had a background I thought I was only testing, in live use. It also had a ridiculously large logo in the upper left corner. This took me by surprise, I was not aware until then that the API allowed this. It makes sense now, but the developer should warn users. That was a destructive change. If you want to say welcome @user I would go for it. I may be inclined to limit that to public accounts. If the account is blocked then they desire privacy. Holding an @username is holding something that is public outside the API. Search google for site:Twitter.com and they have a database of all @usernames as well, and they certainly did not oauth those. -- Scott Iphone says hello. On Aug 26, 2009, at 8:08 AM, JDG ghil...@gmail.com wrote: I disagree. By granting the application access to my account, I tacitly accept the fact that they can access any information that the API provides. The API returns the user's screen name every time you fetch their posts. For crying out loud, a malicious app could go through and delete your last 3200 posts without your even realizing it. You're concerned about using a piece of readily available information -- one that does not actually accurately identify you on Twitter since you can change your username at will -- for something that could probably be relatively benign (Welcome, @user!). Yes, I know there are malicious ways to use it, but there are malicious ways to use any read/write API.
[twitter-dev] Best way to get tweets from me, to me, mentioning me
Hello, what is the best way to get tweets that are from me OR to me OR mentioning me? I have been playing with with search API: feed://search.twitter.com/search.atom?q=from:some_user+...@some_user I believe that does what I want, but was not sure if the search API is the best place to be making these calls. In some ways, this is similar to the widgets that people shove into their blogs. My goal will be to not hassle the user with entering in their login/pass, so oauth is out for this one. All these requests will come from one server, maybe more if the service grows :) I assume I will need to apply for whitelisting? Or is the search API pretty lenient on requests? If there is a specific API call, or combination of them I should be using, and then request whitelisting, I am not objectionable, as I personally have found non search API based calls to return more reliable results. Thank you for any suggestions and guidance. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Best way to get tweets from me, to me, mentioning me
Thanks. These users will be mobile, largely, and asking them to log in to see what will amount to comments is asking too much. This is more a add on feature that some may find value in. Looks like search it is. On Aug 26, 2009, at 9:41 PM, Raffi Krikorian wrote: if you were to allow people to log in, then there are a few great API methods /statuses/friends_timeline /statuses/mentions those would do exactly what you want. unfortunately, you're right, without logging in, you're only left with the search API. your query seems like the best one, and the one that i would use. Hello, what is the best way to get tweets that are from me OR to me OR mentioning me? I have been playing with with search API: feed://search.twitter.com/search.atom?q=from:some_user+...@some_user I believe that does what I want, but was not sure if the search API is the best place to be making these calls. In some ways, this is similar to the widgets that people shove into their blogs. My goal will be to not hassle the user with entering in their login/pass, so oauth is out for this one. All these requests will come from one server, maybe more if the service grows :) I assume I will need to apply for whitelisting? Or is the search API pretty lenient on requests? If there is a specific API call, or combination of them I should be using, and then request whitelisting, I am not objectionable, as I personally have found non search API based calls to return more reliable results. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Followers count
I hope you find out. I long ago gave up. If I really needed the feature, I would scrape that one out of the html, which I know is frowned upon, however, as your data shows, this is pretty all over the map. On Sep 2, 2009, at 2:44 PM, Jason Tan wrote: Anyways, to get back to my original question. How do I get an accurate followers count for a user? Also, why are there still XML/ JSON discrepancies (I came across a few reported issues that said they had been resolved). -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Is twitter a fad or worth development efforts?
Yes. On Sep 3, 2009, at 9:41 AM, ka...@sbcglobal.net wrote: Is twitter a fad or worth development efforts? -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Default profile pics
I have not looked at this so this is mostly curiosity. Why use md5 on a moving target? Who knows when someone may resave an image to compress it more. I bet 1% compression savings translates to thousands of dollars over short time. Isn't the path relatively static? /images/default... Maybe you can use the file path? Do user profile images mantain the same trailing path as the default image? Maybe then a 2x daily script that uses curl with regex support to find if any images up to a point 404. Looks like you have 5-6 known cases. If you hit a 404 alert yourself, investigate, and make adjustments. It sounds to me like those images are in control of designers not developers so I would consider those images hostile at all times. I agree, has_updated_profile_image would be good for the API. As it is now, someone could upload a profile pic of the default, and it would appear as not being updated, when in fact it has. -- Scott Iphone says hello. On Sep 15, 2009, at 4:38 AM, timwhitlock tim.whitl...@publicreative.com wrote: I notice today that Twitter has created a new default profile pic; e.g: http://s.twimg.com/a/1252980779/images/default_profile_1_normal.png Great. That's broken some of my algorithms on Twitblock.org. (identifying re-used images) I can fix that. I'll just add the new MD5 to my app config. But, wait. Did I spot some different colours? Yes, that example is only one; e.g.2: http://s.twimg.com/a/1252980779/images/default_profile_2_normal.png a. Can Twitter tell use how many there are of these? b. How about a user object property profile_image_default (true| false) ? c. How about Twitter start notifying the developer community of changes?
[twitter-dev] Re: Comments for the group and Twitter staff
Then maybe mark it in the docs as highly experimental, this way, people do not build their business plans around something. Make it clear, this feature could go away at any time. On Sep 15, 2009, at 11:04 AM, Alex Payne wrote: Please understand that the denormalized lists are currently provided to developers on a best-effort basis. For the vast majority of Twitter applications, this data isn't necessary. A specialized class of applications need this data, and we're doing our best to provide it. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Comments for the group and Twitter staff
Probably too late for this, but perhaps moving forward, it could be done... Twitter.com should move to using their own API. The tools they use to power their own site should be the same tools we use and rely on. In all reality, this seems a simpler approach, rather than pushing out code for their stuff, and then essentially backporting that to an API, just work on making the API, and then integrate that into the twitter.com site. As far as I can tell, this would solve pretty much every problem the API has, as there can not be a case where twitter is down, but the API is up, or the API is down, and twitter is up. Twitter should be eating their own dog food :) -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Comments for the group and Twitter staff
I think the important part here is in some places. The problem is, twitter.com probably has 75% or more of the exposure. The lowly app developer hits a bug in the API, and people say wtf, works on twitter.com, this app sucks. Good to know that facebook and the mobile site are using the API, but I suspect that when you say some places, while great news, it needs to be all places. If twitter.com is up, and super app for iphone is not, twitter.com gets no heat from that, the developer does. Thank you for the explanation though. -- Scott * If you contact me off list replace talklists@ with scott@ * On Sep 15, 2009, at 2:16 PM, Alex Payne wrote: The main twitter.com site already uses the API in some places. Our revised mobile site is built entirely on the API, and our Facebook application has been built off our API for some time.
[twitter-dev] Re: SERIOUS Problem With Cursors In JSON Followers/Friends Ids
Can someone explain this issue to me in more detail? How do I know my php can handle it? I'm running on an older PPC machine, which is 64 bit, but I have no idea if I built it as 64, I just supplied standard configure arguments. If php sees a 64 bit integer come in as JSON what happens on 32 bit? Does it just truncate it or completly ruin it's representation? Why can't it just be treated as a string?don't these ID's end up in a database, or maybe just passed as a URL argument? As a URL arg you pass to Twitter, so it will work fine. Any database can store a 64 bit int as a string, which gives your ability to get the string back to post to a URL. Can a 32 bit build of a database store 64 bit ints? That to me seems the bigger issue. I'm not seeing where there is a need for math on a 64 bit int ditectly in php. You get the value from JSON, treat it as a string, store that string, and use it for your lookups and sorts. Inneficient on lookups and storage, is that the core of this? What key aspect of this problem am I missing? -- Scott Iphone says hello. On Sep 24, 2009, at 6:07 AM, Dewald Pretorius dpr...@gmail.com wrote: Chad, Shouldn't Twitter be providing an API that works for everyone? From what you said it sounds as if you're saying, Tough. If you want to consume the API with PHP, either run your stuff on a 64-bit machine, or scrape the raw JSON output and make it so that it works for you. That doesn't sound right. Dewald On Sep 24, 1:02 am, Chad Etzel c...@twitter.com wrote: Hello, As Joseph points out, PHP on a 64-bit system can handle these numbers. If you really want this data as a string, you could write a regex in PHP to alter the json string to wrap the digits in quotes before sending it through json_decode(), but that would be a pretty gnarly kludge. -Chad On Wed, Sep 23, 2009 at 10:28 PM, Dewald Pretorius dpr...@gmail.com wrote: All that Twitter needs to do to solve this problem is to build the JSON out with next_cursor and previous_cursor as string values. I.e., the JSON data should contain: next_cursor :12398712981212987,previous_cursor:-12398712981212987 I don't know what it will do to Java apps, but for PHP apps it will solve the problem. Dewald
[twitter-dev] Re: SERIOUS Problem With Cursors In JSON Followers/Friends Ids
If you are on 32 bit, what php.ini changes are you referring to that would be beneficial? -- Scott Iphone says hello. On Sep 25, 2009, at 10:35 AM, jmathai jmat...@gmail.com wrote: I like Abraham's idea: Twitter could add: next_cursor_string:1314614526448841129 Or a general return_as_string parameter which would return all values as strings. This would solve the problem for future values that might exceed the max int. There are going to be a lot of devs using PHP that don't have the option to switch to 64 bit or make any php.ini changes. On Sep 24, 12:29 pm, Dewald Pretorius dpr...@gmail.com wrote: That magical maximum number appears to be 1 (1.0E+12). So, for tweet ids we still have a bit of breathing space. Dewald On Sep 24, 4:18 pm, Dewald Pretorius dpr...@gmail.com wrote: Clearly PHP_INT_MAX plays no role in json_decode. There must be some other mystical maximum number above which it represents the number as float in the decoded data. Dewald
[twitter-dev] Re: Brain Wave Help required with Twitter - rss- map
Ok, I'll bite :) I though you meant phishing at first. Do you have programmin experience? If so, what languages are you familiar with? If not, do you have access to a programer and are just looking for confirmation that your idea could be realized? This mailing list is primarily for developers/programmers so you may be in the wrong area. Though you may find a receptive developer to hire here, this is probably not the best place to be asking. Let us know and we will see if we can not get you going in the right direction. -- Scott Iphone says hello. On Sep 25, 2009, at 8:58 AM, billy2...@googlemail.com billy2...@googlemail.com wrote: Thank you for reading this as I am looking for some help to sort out a brain wave. You know what it is like do not know enough and do not know if it can be done. Is it possible to question twitter for Fishing Matches? What I would like to do is take all the information on Fishing Matches from Twitter into a RSS feed and then onto a map keeping every one informed of fishing matches in there area. Am I dreaming to much or is this possible. Any help in the right direction much appreciated and would also appreciate being told that it is too much work and not very good. Thanking you in anticipation. Billy World Fishing News UK Yorkshire
[twitter-dev] Re: SERIOUS Problem With Cursors In JSON Followers/Friends Ids
I tend to agree, a MacMini, around 600.00, would build php out as 64 bit no issue. If you are a baby startup, that should really cover you and be able to handle a lot more load than you would think. You may have to pop your database on something more robust, but it works quite well as a php front end. Take it to 1000.00 or so, and you can built out your own 1U server that is going to hold it's own very very well. Or, rent a 1U, they are a few hundred bucks a month, you get to pick the OS, pick the bit'ness, etc. If you are a startup, on a shoestring, part of the fun is being creative, and making due with what you have. And don't laugh at that MacMini, you can easily get to the SATA port, and plug it right into a SATA array elsewhere, getting SATA II on full size dives. I just pegged the drives to a cork board, its not pretty, but it works. I have a second on standby in case that one craps. I just had them laying around, and made them up from parts. While I am not going to mention the site, I can tell you, it is php on the front, apache2 with a good deal of mod_rewriting going on, and substantial database work, all happening on the MacMini, handling 500,000+ average unique visitors, which is a lot more requests when you add in images and the rest. (this site is not twitter related) If I was sloppy, that database would fall on it's face under that load. I have added in some caching where needed, so pages are rendered to disk at the expense of a php include of the data. I am in this thread to learn about the issue, but so far, getting to 64 bits should not be a barrier. Isn't this really a php issue anyway? Why can't php handle a 64 bit int in a 32 bit system? Maybe they need a bug report or a patch. MySql seems to handle 64 bit on my 32 bit test system here just fine. Maybe someone can write a wrapper for the json encode/decode that fixes this in that part of php? -- Scott * If you contact me off list replace talklists@ with scott@ * On Sep 25, 2009, at 12:23 PM, David Fisher wrote: Saying that startups can't afford 64 bit processors in systems is crazy. Most startups I know are running on EC2 or have fairly new hardware. I bought a killer 64-bit quad xeon server for less than $1,500 for our startup and its rocking. If your startup doesn't have $1,500 for a primary capital computing expense that's another problem you have there.
[twitter-dev] Re: SERIOUS Problem With Cursors In JSON Followers/Friends Ids
I would not change either. But there are those here that are stating they need new hardware to work around this issue, and that they can not afford that. I was trying to be that voice of reason if that is the road/excuse they are choosing to go. There seem to be acceptable workarounds, solid proposed workarounds, etc. I guess I am not getting it, JSON is just a string returned, yes, it can represent type of data, but it is still just a string. I can not see it being that huge a performance hit to massage that string a bit once you get ahold of it. -- Scott * If you contact me off list replace talklists@ with scott@ * On Sep 25, 2009, at 2:02 PM, jmathai wrote: It's ridiculous to suggest a change in hardware (64 bit) or software (switch from PHP) to use Twitter's API. It's not like either of these are archaic. It sucks, sure, but it's silly to suggest such a solution. BTW, I don't have this problem. I'm just trying to be the voice of reason.
[twitter-dev] Re: SERIOUS Problem With Cursors In JSON Followers/Friends Ids
Why is the API not versioned then? api.twitter.com/?v=1, api.twitter.com/?v=1.1, api.twitter.com/?v=1.2 etc Or, if that is too much maintenance, how about api.twitter.com/?bitfix=32 or whatever. -- Scott * If you contact me off list replace talklists@ with scott@ * On Sep 25, 2009, at 6:40 PM, JDG wrote: and it would also break everyone who CAN handle 64 bit ints and expects results in decimal numeric format. On Fri, Sep 25, 2009 at 16:01, Richard ryt...@gmail.com wrote: Can this not be returned as hex or base64? It would save bandwidth for Twitter (and us) and make it a string people could convert it to 64bit int if they still want to. On Sep 25, 10:16 pm, Scott Haneda talkli...@newgeo.com wrote: I would not change either. But there are those here that are stating they need new hardware to work around this issue, and that they can not afford that. I was trying to be that voice of reason if that is the road/excuse they are choosing to go. There seem to be acceptable workarounds, solid proposed workarounds, etc. I guess I am not getting it, JSON is just a string returned, yes, it can represent type of data, but it is still just a string. I can not see it being that huge a performance hit to massage that string a bit once you get ahold of it. -- Scott * If you contact me off list replace talklists@ with scott@ * On Sep 25, 2009, at 2:02 PM, jmathai wrote: It's ridiculous to suggest a change in hardware (64 bit) or software (switch from PHP) to use Twitter's API. It's not like either of these are archaic. It sucks, sure, but it's silly to suggest such a solution. BTW, I don't have this problem. I'm just trying to be the voice of reason.
[twitter-dev] Re: Please Make 401 Singular In Meaning
I think it would be more appropriate to create a GUI interface in Visual Basic, see if I can track an IP address. :) -- Scott * If you contact me off list replace talklists@ with scott@ * On Sep 25, 2009, at 6:05 PM, Dewald Pretorius wrote: I am going to remotely install a VB program on your laptop that is going to repeatedly pick up the mouse and whack you over the head with it.
[twitter-dev] Where in API docs to get first tweet only
I'm not sure this is possible, I'm trying to avoid a local data store to make it possible. I would like to get a small bit of data from a tweet, but only the first tweet, ignoring that user from that point forward. I can of course grab their username and disregard, but my list will grow quickly. If my goal is to get stats on what time of day most people are joining Twitter and posting their first tweet, what would be the best place to query for that data? I just need the time, and perhaps location fields. Would the search API be good? I hear about this firehose thing, but don't know what that is. I'm thinking the public timeline may be best, but there may be way too much data for me to deal with. Suggestions on methods appreciated. -- Scott Iphone says hello.
[twitter-dev] Re: Where in API docs to get first tweet only
Thanks Peter. Any pointers on general docs on what the heck spritzer and garden hose is? The public timeline api says this: statuses/public_timeline Returns the 20 most recent statuses from non-protected users who have set a custom user icon. The public timeline is cached for 60 seconds so requesting it more often than that is a waste of resources. If I pull this into an RSS feed: feed://twitter.com/statuses/public_timeline.rss I refresh it a few seconds later, I get new tweets. Is that 60 second cache an out of date note in the docs? The API also states this is rate limited. To get the data I am after, I am going to be hitting this think pretty hard. If there is no cache, it will be more than 60 seconds in frequency, more like as soon as the script is done working, I will request it again. Pretty much perpetual requesting. Or is this treated more like the search API, and is rate limited very liberally? Thank you for your help this Saturday. -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 10, 2009, at 2:06 PM, Peter Denton wrote: Hi Scott, Since it seems you are looking for a sampling situation, you might want to poll the public timeline and check for 1st tweet, (created at and 1st update timeframe are same/near day). Also, you could expand your sample size and look into accessing the spritzer or garden hose and again running some best guess scenario of signup and first tweet. Cheers Peter On Sat, Oct 10, 2009 at 1:19 PM, Scott Haneda talkli...@newgeo.com wrote: I'm not sure this is possible, I'm trying to avoid a local data store to make it possible. I would like to get a small bit of data from a tweet, but only the first tweet, ignoring that user from that point forward. I can of course grab their username and disregard, but my list will grow quickly. If my goal is to get stats on what time of day most people are joining Twitter and posting their first tweet, what would be the best place to query for that data? I just need the time, and perhaps location fields. Would the search API be good? I hear about this firehose thing, but don't know what that is. I'm thinking the public timeline may be best, but there may be way too much data for me to deal with. Suggestions on methods appreciated. -- Scott Iphone says hello.
[twitter-dev] Re: Where in API docs to get first tweet only
Thank you. I think I just got booted from hitting the public timeline too much. I requested whitelisting via the whitelist form. Since I am not authenticating, and am just curl'ing the json resource for the public timeline, is there a way for me to tell what is really happening? I currently get an http 400 bad request, which I am betting is a throttle/block. Is it possible to determine the remaining number of queries I am allowed via some command? Thanks. -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 10, 2009, at 3:42 PM, John Kalucki wrote: http://apiwiki.twitter.com/Streaming-API-Documentation
[twitter-dev] Re: Where in API docs to get first tweet only
One last question I think. Is there a way to control how many items come back in a request to the public timeline? I currently can not tell how many there are in the result set, I will have to wait an hour. However, if I could get a larger batch, I could query it much less often. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Regex to get user agent aka 'source'
It appears Twitter returns the 'source' field which has a user agent in it. Most of the time this is an HTML HREF string. I have found one case where that is not true. web will exist as only that string. Is web the only non HTML string I can expect at this time? I'm logging the event if I get another edge case, but wanted to check. Regarding regex's to get just the user agent name, and strip the HTML bits away... I've heard some noise on this list about the format changing. I see some have nofollow, others do not, many variations. I don't want to chase this around, though I also log non matches here. My regex looks for '\.+\i' I believe this to be pretty solid, but it feels a little too easy. I'm also left with the returned match having and in the string. Of course, those are easy enough to replace/trim off. What do you think of this approach? Any cases others have seen that would lead this to failure? -- Scott Iphone says hello.
[twitter-dev] Re: OAuth wed desktop feedback
Isn't part of the point of oauth to teach the user they are entering in credentials for another website into that other website? By rebranding the twitter oauth page it gets to a point where you may as well just ask their user/pass on your own site, and never have them leave your site/app. I don't look at oauth as real time over the wire security, though it helps in that area. I look at it as user education with the goal to get users to understand not to cross polinate user/pass data to other sites. It would come down to users understanding to look at the URL, something many just do not understand enough to do. I do not understand all the details of oauth at this point. What I do know is I've learned when I auth an app I should be looking for a distinct look and feel at a specific URL. If that look and feel changes, red flags are raised on my part. I'd look to the URL, but wonder if I was not being tricked from TWITTER.COM TO TWlTTER.COM. (those look identical to me, lowercase L looks a lot like an uppercase I. From my perspective, oauth needs to go after the OS vendors. Integration with their underlying systems and API's is the only way this will gain widespread use that is as secure as it can be for an end user. Right now, integration is not tight enough for a user to even understand what they are gaining from this procedure. I can find 5 Twitter sites right now that ask for me to store a login and pass into my account, they are not using oauth, and could do what they want with my account. This just proves users are not understanding this. Heck, twitters own account settings ask for raw login and password data to gmail, yahoo, and I believe aol, last I looked. -- Scott Iphone says hello. On Oct 12, 2009, at 1:41 PM, Amitab hiamita...@gmail.com wrote: 2) Please integrate the OAuth authntication with my branding. At the moment it is just the logo. I would like to have the whole background be of my branding.
[twitter-dev] Re: Regex to get user agent aka 'source'
Well holy smokes, add another one to the kitchen sink list of php functions I was not aware of. Thanks! I worry about ' a hreffoo/a, which a browser happily takes, but I get this feeling strip_tags is going to only chop off the /a closing tag. Have you had this in use and confirmed twitters data is mostly sane and without such HTML formatting issues? -- Scott Iphone says hello. On Oct 12, 2009, at 2:24 PM, shiplu shiplu@gmail.com wrote: Do you use php?? If yes, use strip_tags() function. otherwise
[twitter-dev] Re: Regex to get user agent aka 'source'
Agreed, just for data savings alone, there is no reason why *we* need to presentation data that Twitter uses. If they want to use rel=nofollow, they they can, but that has no bearing on how I may or may not want to use the data. If there was just url and agent, that alone should shave of GB's of data a day I on API calls I would bet. Thanks for the link. -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 12, 2009, at 5:08 PM, Dave Briccetti wrote: I don’t know the background, but I do wish they would split that into, say, sourceName and sourceUrl. http://github.com/dcbriccetti/talking-puffin/blob/master/twitter-api/src/main/scala/org/talkingpuffin/twitter/TwitterStatus.scala
[twitter-dev] Ummmm, how in the heck does someone get support around here
I brought this up on this list before, let's look at this: http://help.twitter.com/home 1) Where do I go to open a ticket? I read the entire page, to find a little link, that says ask us. That takes me to: http://twitter.zendesk.com/requests/new That redirects me around a few times, and takes me right back to help.twitter.com/home. 2) How do I open a new ticket? Why do I need to? Because I just got this email from twitter support: http://dl.getdropbox.com/u/340087/Drops/10.13.09/what-6855eebe-132402.png Thanks -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: /users/show.xml? doesn't return xml string
I just ran your exact code, and was able to pull an xml string just fine, so I do not believe it is your code. The only thing I can think is you may want to curl_close() the connection, perhaps there is some caching or other similar thing in effect, though that is a big long shot. Maybe try curl on the command line, on the same host, and see what your results are, if that works, you can start looking at your code, otherwise, it may just be an temporary glitch in the API. You may want to look into doing some more testing in your command though, return the headers, and look at the http responses, so you know when you have a 200 ok one, versus some of the others, in which case you will want to be abel to gracefully degrade your application in some way. Sorry I could not be of more help. -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 13, 2009, at 9:37 PM, ArnieLapinig wrote: Hello, I hope someone can help my query with the REST API /users/show.xml doesn't seem to be returning an xml string... I'm getting strange data back from my query. Here's the code: $twitterHost = 'http://twitter.com/users/show.xml? screen_name=' . urlencode($lookforname); $curl = curl_init(); curl_setopt($curl, CURLOPT_CONNECTTIMEOUT, 2); curl_setopt($curl, CURLOPT_HEADER, false); curl_setopt($curl, CURLOPT_HTTPAUTH, CURLAUTH_BASIC); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); curl_setopt($curl, CURLOPT_USERPWD, $username: $password); curl_setopt($curl, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1); curl_setopt($curl, CURLOPT_URL, $twitterHost); $query = curl_exec($curl); and here's what i'm getting: $twitterHoststring(59) http://twitter.com/users/show.xml? screen_name=mandarine_one $query string(2023) 32432559 anton mandarine_one Neuburg an der Donau Mediengestalter, Nerd, Comic- und Musikliebhaber, nörgelt gerne über die Arbeit. http://a3.twimg.com/profile_images/435301043/twitter_normal.jpg http://www.grizzlyfear.de false 159 EBEBEB 33 99 F3F3F3 DFDFDF 238 Fri Apr 17 14:49:15 + 2009 23 3600 Berlin http://a1.twimg.com/profile_background_images/41257908/8edd7_014-sergeypoluse.jpg true 2129 false false false false Tue Oct 13 17:19:41 + 2009 4839657896 @_Schnucki_ madame sie gruseln mir die scheisse aus dem leid. du schreibst von stromschlag und bei mir geht das licht aus #scary a target='_blank' href=http://www.atebits.com/; rel=nofollowTweetie/a false 4839540863 50962463 false _Schnucki_ when i paste http://twitter.com/users/show.xml? screen_name=mandarine_one into the browser, i get an xml string: user id32432559/id nameanton/name screen_namemandarine_one/screen_name locationNeuburg an der Donau/location description Mediengestalter, Nerd, Comic- und Musikliebhaber, nörgelt gerne über die Arbeit. /description profile_image_url http://a3.twimg.com/profile_images/435301043/twitter_normal.jpg /profile_image_url urlhttp://www.grizzlyfear.de/url protectedfalse/protected etc etc etc etc... Am I making some sort of basic mistake? Thanks for helping...
[twitter-dev] Re: Help estimating tweets per day...
On Oct 14, 2009, at 8:38 AM, Kyle B wrote: I am creating a mathematical model based on some results from Twitter's API, but I am missing one critical number in the model. I need to estimate the number of total tweets in the USA each day. The better an estimate I get and the less assumptions I make, the more useful the model will be (it will be published for the public to use). I have been told that this type of information is important and usually kept secret by internet start ups. Understanding this, I have come up with a work around that is not yet accurate enough so I am looking for your advice. I am very interested in this data, as it is a metric I will need with a service I am working on. I had no idea how to get this accurately, so I was going to look for services that are making educated guesses. Idea: I gather data from Twitter's search API at least once an hour. I do not think this is your best approach. The search API seems to have a lot of flux to it. The public timeline would be better, but still not perfect. I think you want the streaming API, which I will elaborate on below. My idea is to store the first tweet ID I see each day, and subtract it from the ID of the previous day to estimate the number of tweets per day. I have three problems here: 1. How are tweet IDs incremented? Do they increase by a factor of 1, 2, 5, 10...? It does not matter. I would guess they are incremented by one. Just given the 32/64 bit counter issue, it would appear it has to be 1, or they would have breached the 32 bit limits a long time ago. This does not take into account any number of technical things Twitter may do on their end such as distributed databases, completely de- normalized data in order to deal with the massive volume they have, and most importantly, users deleting tweets. 2. I need an estimate for the number of private/protected users assuming each private user's tweet gets an ID number. This is required because I am sampling the public tweets. I am not sure you need this in your calculation, if you only read public tweets, and have a way to count them accurately, there should not be a need to subtract out any private ones. I do not believe you can ever arrive at an accurate count by subtracting first and last tweet. You have time zones to content with as well. 3. I need to estimate the number of tweets coming from overseas. I am modeling the USA. This is less of a problem than the previous two. This will be hard, as there is no mandate stating you must set your location, let alone set it to something accurate. It has been suggested that changing your location to a false one is a good way to trick some politically charged countries from preventing conversational discourse. If you look at the streaming API, you can define a set of parameters that will allow a stream of data to come in. It is a lot of data, in my opinion, a boatload of data. It also sounds just like what you need. Hit the streaming API, open a socket, and start reading in the data. Of course, you will not want to read it all, but determine some batch you want to grab, and some schedule you want to grab it on. You can then extrapolate your numbers from there. Perhaps a 1 minute read of the data every 5 minutes would get you where you need to go. Then you could determine patterns in usage and adjust accordingly. From what I understand about the streaming API, is that it is in fact a full stream, always on. Not only do you have to make sure that stream stays open, and reconnect it if it closes, consider it to be a youtube video playing all day long. Even if you only want a chunk of the data, you are still moving all that data across your wire. If you are in any way bandwidth constrained, be sure to be careful, your bills and resources could go through the roof. This of course is just a small technical hurdle, you could open and close the stream as needed, but you may sacrifice some accuracy by doing so. If there is any chance you could contact me off list, address below, and keep me posted on your data when it goes public, I would be very appreciative. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Help estimating tweets per day...
And you don't think the streaming API will answer that for you? -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 14, 2009, at 3:27 PM, Kyle B wrote: Thanks for the info. It helps a lot. Figuring out an accurate number is essential to my model, so much so that I am determined to find some method of estimating it to acceptable margins of error!
[twitter-dev] http return 0
I have a pretty simple function I made to curl a url against twitter.
I am whitelisted. I call a url once every 15 seconds, about once an
hour, I get 'http_response' of 0, the rest I get 200 OK.
When I do not see a 200, I log the 'http_response', is there anything
else I can log or enable in the curl code below to figure out what is
going on here? I am going to add curl_error to the result, and see
what I get from that, but thought I would as for any pointers.
function curl_url($url) {
$ch = curl_init(); // create a new curl resource
curl_setopt($ch, CURLOPT_URL, $url); // set URL to download
curl_setopt($ch, CURLOPT_REFERER, 'http://
example.com/'); // set referer:
curl_setopt($ch, CURLOPT_USERAGENT, 'example'); // user
agent:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); // return,
not print
curl_setopt($ch, CURLOPT_TIMEOUT, 10); // timeout in seconds
// download the given URL, and return output
$output= curl_exec($ch);
$http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch); // close the curl resource, and free
system resources
$return_data['http_response'] = $http_code;
$return_data['page'] = $output;
return $return_data;
}
--
Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: http return 0
I think so too, I am running some more tests. I have confirmed with a
parallel test to a nearby host to twitter, that I am able, at the
same time, to make a http request to their resource. While I can not
time the curl actions to happen at the exact same time, one much
happen before the other, the difference in time is:
2009-10-15 15:01:05.5321030
2009-10-15 15:01:05.5327020
So something like 5990 microseconds. I have set the order to be
random, so one will happen first some of the time, and the other first
some of the time, if random() is relatively even, and it was my
connection, I should see errors on the other resource a few times,
which I am not.
If curl_error shows me anything else, I will let the list know.
--
Scott * If you contact me off list replace talklists@ with scott@ *
On Oct 15, 2009, at 1:45 PM, JDG wrote:
i think http 0 in curl just means the connection failed. should
probably
just retry.
On Thu, Oct 15, 2009 at 14:44, Scott Haneda talkli...@newgeo.com
wrote:
I have a pretty simple function I made to curl a url against
twitter. I am
whitelisted. I call a url once every 15 seconds, about once an
hour, I get
'http_response' of 0, the rest I get 200 OK.
When I do not see a 200, I log the 'http_response', is there
anything else
I can log or enable in the curl code below to figure out what is
going on
here? I am going to add curl_error to the result, and see what I
get from
that, but thought I would as for any pointers.
function curl_url($url) {
$ch = curl_init(); // create a new curl resource
curl_setopt($ch, CURLOPT_URL, $url); // set URL to download
curl_setopt($ch, CURLOPT_REFERER, 'http://example.com/'http://example.com/%27
);
// set referer:
curl_setopt($ch, CURLOPT_USERAGENT, 'example'); // user
agent:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); // return,
not
print
curl_setopt($ch, CURLOPT_TIMEOUT, 10); // timeout in seconds
// download the given URL, and return output
$output= curl_exec($ch);
$http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch); // close the curl resource, and free system
resources
$return_data['http_response'] = $http_code;
$return_data['page'] = $output;
return $return_data;
}
[twitter-dev] Re: Approx time for White listing?
I had a glitch, and was denied, and had whitelisting completed, from start to finish, in about 30 hours. -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 15, 2009, at 4:56 PM, Atul Kulkarni wrote: Hi All, What is the approximate expected time for white listing from the day of application?
[twitter-dev] Re: http return 0
Correct, error 0 means no data was returned, and that the timeout in curl was hit, which I have set to 10 seconds. I am seeing about 5 of these per Admin block 24 hours, out of around 300 calls per 24 hours. I see a lot more of them when I see more of the 502 errors, which seems to tell me that when twitter is over capacity there is a tendency for them to completely close down the port for communication all together. What is annoying about this, is that twitter.com is fine during all this, their public timeline works 100%, yet my API driven calls to the public timeline are getting error 0 and 502 many times per day. -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 15, 2009, at 1:45 PM, JDG wrote: i think http 0 in curl just means the connection failed. should probably just retry.
[twitter-dev] Re: Why have you removed the HTML/CSS badge/widget?
Specifically, what are you referring to? -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 17, 2009, at 10:25 AM, Jonathan Timar wrote: Now Twitter is only offering complex, overly stylized widgets that I cannot integrate cleanly with my website. Why would you remove this feature?
[twitter-dev] Re: getting only old messages when using API
What constitutes sketchy behavior? -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 17, 2009, at 7:28 PM, John Kalucki wrote: Your user should begin to flow into search again. But, if there is any further sketchy behavior on this account, it will be permanently banned. On Oct 17, 2:42 pm, Vlad vshala...@gmail.com wrote: Hi All I'm building a page (www.cmt.ca/unstable) that needs to use twitter feed from this person -http://twitter.com/UnstableTammy So I'm sending request tohttp://search.twitter.com/search.atom?q=from%3AUnstableTammy If you compare both feeds you'll see that last message on API one is from Oct 12th and there were more messages posted by Tammy but I'm not getting them when using API by some reson. I'm also getting feeds from 2 other persons (for ex.http://twitter.com/UnstableAllie) and there is no problem when using API.
[twitter-dev] Re: What is included In the Queries are limited 140 URL encoded characters. restriction?
I am not sure I understand the confusion, the API is 140 characters
exactly in the query string. You can run this, don't run it often, or
put a sleep 1; in the loop if you do.
#!/bin/bash
# This script assumes you pre url encode your data.
# Request url
URL=http://search.twitter.com/search.json?q=;;
for (( i = 1; i 200; i++ )); do
# Current position/length
echo Query Length: $i;
# Shove on another character
j=$j'A'
# The url we are about to poke
echo $URL$j
RESULT=$( curl -o /dev/null -s -w %{http_code} %
{size_request} {$URL$j} )
# Show http_code
http_code=`echo $RESULT | awk '{print $1}'`
echo http_code: $http_code
# Show size_request
size_request=`echo $RESULT | awk '{print $2}'`
echo size_request: $size_request \(Sent bytes\)
echo
done
Here is a snip of the results, from the first, to the last bits around
140, right when you hit 141, it goes from http 200, to http 403.
Though this script should make it rather easy to test what happens
with url encoded characters. I do not think it matters, each
character is a character, so if it is a space, that will get url
encoded into %20 taking up three characters.
Query Length: 1
http://search.twitter.com/search.json?q=A
http_code: 200
size_request: 163 (Sent bytes)
..
Query Length: 138
http://search.twitter.com/search.json?q=AA
http_code: 200
size_request: 300 (Sent bytes)
Query Length: 139
http://search.twitter.com/search.json?q=AAA
http_code: 200
size_request: 301 (Sent bytes)
Query Length: 140
http://search.twitter.com/search.json?q=
http_code: 200
size_request: 302 (Sent bytes)
Query Length: 141
http://search.twitter.com/search.json?q=A
http_code: 403
size_request: 303 (Sent bytes)
Query Length: 142
http://search.twitter.com/search.json?q=AA
http_code: 403
size_request: 304 (Sent bytes)
--
Scott * If you contact me off list replace talklists@ with scott@ *
On Oct 17, 2009, at 7:39 PM, leonspencer wrote:
And still waiting on a response. More information from a associate:
Subject: Re: Do you know what is being counted toward query length
Yeah, because your using twitter search and not api!
An Api String would be
For Geo Locations
http://search.twitter.com/search.atom?geocode=40.757929%2C-73.985506%2C25km
Or
For Since...
http://search.twitter.com/search.atom?q=twittersince_id=1520639490
On Oct 17, 2:29 pm, leonspencer spencer_l...@yahoo.com wrote:
Still waiting for a response here. I tried a query with the Twitter
Advanced Search
tool:http://search.twitter.com/search?q=ands=Allofthesewordsphrase=Thise...=enfrom=leonspencerto=leonspencerref=leonspencernear=within=15units=misince=2009-10-07until=2009-10-18rpp=15
When I strip away the parameter names and operators, this is the
values stringed together:
AllofthesewordsThisexactphraseAnyofthesewordsNoneofthesewordsThishashtagenleonspencerleonspencerleonspencer15mi2009
-10-072009-10-1815
Length is at 133 but still getting error from the advanced search:
http://search.twitter.com/search?q=ands=Allofthesewordsphrase=Thise...=enfrom=leonspencerto=leonspencerref=leonspencernear=within=15units=misince=2009-10-07until=2009-10-18rpp=15
Sorry, your query cannot be more than 140 characters long (it is 161
characters).
[twitter-dev] Re: What is included In the Queries are limited 140 URL encoded characters. restriction?
I believe Query string is everything after the q=, not including the
= of course. So no matter how it is encoded or not, you just count up
the chars, and that should be all you get. It is somewhat limited,
but I do not think entirely designed to do massively complex queries.
I used to try to negate certain twitter users in the #php area, by
taking out things like freelance via query string. I did not get
very far before I breached the 140 limit.
Technically speaking, I am not sure why there needs to be a limitation
based on length, it should be based on number of name/value pairs.
Performance wise, the hit is going to come from multiple name value
pairs, and not length.
--
Scott * If you contact me off list replace talklists@ with scott@ *
On Oct 17, 2009, at 9:31 PM, Leon Spencer wrote:
Thank you for your response. I'll run the lines to look at this
further. The confusion is as whether Query string refers to HTTP
query string (? until end) or the Twitter API query (?q=value)
value w/operators and parameters?
Usage Notes:
* Query strings should be URL encoded.
* Queries are limited 140 URL encoded characters.
When I check this against the Advanced Search Tool, it wasn't clear
what was being included. And the tools seems to be somewhat buggy.
Leon
From: Scott Haneda talkli...@newgeo.com
To: twitter-development-talk@googlegroups.com
Sent: Sat, October 17, 2009 9:22:44 PM
Subject: [twitter-dev] Re: What is included In the Queries are
limited 140 URL encoded characters. restriction?
I am not sure I understand the confusion, the API is 140 characters
exactly in the query string. You can run this, don't run it often,
or put a sleep 1; in the loop if you do.
#!/bin/bash
# This script assumes you pre url encode your data.
# Request url
URL=http://search.twitter.com/search.json?q=;;
for (( i = 1; i 200; i++ )); do
# Current position/length
echo Query Length: $i;
# Shove on another character
j=$j'A'
# The url we are about to poke
echo $URL$j
RESULT=$( curl -o /dev/null -s -w %{http_code} %
{size_request} {$URL$j} )
# Show http_code
http_code=`echo $RESULT | awk '{print $1}'`
echo http_code: $http_code
# Show size_request
size_request=`echo $RESULT | awk '{print $2}'`
echo size_request: $size_request \(Sent bytes\)
echo
done
Here is a snip of the results, from the first, to the last bits
around 140, right when you hit 141, it goes from http 200, to http
403. Though this script should make it rather easy to test what
happens with url encoded characters. I do not think it matters,
each character is a character, so if it is a space, that will get
url encoded into %20 taking up three characters.
Query Length: 1
http://search.twitter.com/search.json?q=A
http_code: 200
size_request: 163 (Sent bytes)
..
Query Length: 138
http://search.twitter.com/search.json?q=AA
http_code: 200
size_request: 300 (Sent bytes)
Query Length: 139
http://search.twitter.com/search.json?q=AAA
http_code: 200
size_request: 301 (Sent bytes)
Query Length: 140
http://search.twitter.com/search.json?q=
http_code: 200
size_request: 302 (Sent bytes)
Query Length: 141
http://search.twitter.com/search.json?q=A
http_code: 403
size_request: 303 (Sent bytes)
Query Length: 142
http://search.twitter.com/search.json?q=AA
http_code: 403
size_request: 304 (Sent bytes)
--Scott * If you contact me off list replace talklists@ with scott@ *
On Oct 17, 2009, at 7:39 PM, leonspencer wrote:
And still waiting on a response. More information from a associate:
Subject: Re: Do you know what is being counted toward query length
Yeah, because your using twitter search and not api!
An Api String would be
For Geo Locations
http://search.twitter.com/search.atom?geocode=40.757929%2C-73.985506%2C25km
Or
For Since...
http://search.twitter.com/search.atom?q=twittersince_id=1520639490
On Oct 17, 2:29 pm, leonspencer spencer_l...@yahoo.com wrote:
Still waiting for a response here. I tried a query with the Twitter
Advanced Search
tool:http://search.twitter.com/search?q=ands=Allofthesewordsphrase=Thise...=enfrom=leonspencerto=leonspencerref=leonspencernear=within=15units=misince=2009-10-07until
[twitter-dev] Re: What is included In the Queries are limited 140 URL encoded characters. restriction?
Yes, it seems a bit of a mess, but rely on the API, and the responses it gives you. The web based advanced search is a nice way to do a quick visual test, but as you have seen, can spit out some bizarre things. I think the bash script will help you definitively answer anything you need to though, you can stuff pretty much anything into it, and at least run some solid tests against it. Good luck -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 17, 2009, at 9:36 PM, Leon Spencer wrote: Ahhh. I see what you are doing. My confusion was based off the TWitter Advanced Search Tool. As you add more parameters and search operators, these seem to be counted toward the max 140 character length. But I was seeking definitive confirmation was someone.
[twitter-dev] Re: What is included In the Queries are limited 140 URL encoded characters. restriction?
I brought that up the other day, twitter eating their own dog food, to which I was told they do, but only in some parts. It would be nice, so that when the API is down, twitter is down, and we as developers did not look like our apps suck, but that may not be a goal for twitter, or it may be, I just do not know. I hope it is though. -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 17, 2009, at 9:41 PM, Leon Spencer wrote: Thanks Scott. Thats what another associate was suggesting last night. I figured the Twitter Advance Search Tool was wrapping around the Api and I could plug its queries into curl. I guess that is not the case.
[twitter-dev] Re: Why have you removed the HTML/CSS badge/widget?
Not sure I understand, you mean like this one: http://twitter.com/widgets/html_widget Either way, text only widget, you being on a developer mailing list, I am sure in all honestly, a few lines, maybe 10 lines or so, and you could have a widget to your liking. Whatever old widget your refer to, has to be in use by others, they could not just take it away, as that would break everyones sites who included it in their website. I suspect they may have moved it elsewhere, but I am certain the old widget still works. There are also 10's of websites that have twitter widgets, from simple to complex. Did your widgets stop working? If not, just copy the source from those and keep using them, they will continue to work. -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 17, 2009, at 10:13 PM, Jonathan Timar wrote: I am referring to the text only version of this: http://twitter.com/widgets/which_widget , which is oddly still present on the Twitter website (I found it after some in depth Googling) but no longer linked to or referenced in any way, in favour of this page: http://twitter.com/goodies which features some far less useful and customizable widgets. What is the reasoning behind this? Does Twitter really think that there is no demand for a simply, fully CSS styleable widget?
[twitter-dev] 502 error for public timeline
I am calling the public timeline once every 60 seconds. Today has been a bad day, 26 total 502 errors since 12:21 PST to now. http public timeline is fine on the website. Any ideas when this is going to be resolved, or if it is even something that Twitter is aware of? If not, heads up :) -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] Re: Nero 9 - FULL Version - [Precracked] 51MB ONLY!
I do not really understand their motivation, 99% of the groups out there are not going to be susceptible to spam. Most groups are tech, or at least, highly niche, and the people on it are going to know it is spam. Most groups are filtered into a folder, there are just so many red flags. Spammers are a strange group. How come this list is such a target? I am on some other google groups, larger than this by a fair degree, and this does not happen. -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 19, 2009, at 6:37 PM, Jeffrey Greenberg wrote: This looks just great... can't wait to try itj On Mon, Oct 19, 2009 at 2:01 PM, Peter Denton petermden...@gmail.comwrote: I would say, considering I can only recall a few spam posts getting through, you guys [sic] do a great job. On Mon, Oct 19, 2009 at 1:34 PM, Chad Etzel jazzyc...@gmail.com wrote: Why yes we can, and we do... loads of it. The problem is that these spammers are spoofing the from address of list owners who usually get automatically posted and skip the moderation step. This is a flaw of the way Google Groups handles incoming posts, and not of the group admins. -Chad On Mon, Oct 19, 2009 at 4:28 PM, Dave Briccetti da...@davebsoft.com wrote: Google group admins can actually DELETE spam, too, which would be nice.
[twitter-dev] Re: 502 error for public timeline
This is getting out of hand, I am getting 10 a minute. I guess it is time to write that back off code that I figured would not be needed on the public timeline. -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 19, 2009, at 3:06 PM, Michael Steuer wrote: I've been seeing fail whales on the site as well.. Something must be going on... On 10/19/09 3:03 PM, Scott Haneda talkli...@newgeo.com wrote: I am calling the public timeline once every 60 seconds. Today has been a bad day, 26 total 502 errors since 12:21 PST to now. http public timeline is fine on the website. Any ideas when this is going to be resolved, or if it is even something that Twitter is aware of? If not, heads up :)
[twitter-dev] Re: 502 error for public timeline
They were piling up, it is not my routine to call them every minute, but what happens is I have a 10 second timeout on the curl call, another request comes in, but it is chewing away on the 10 seconds, and they just build up. I have resolved this with some backoff code. I am calling the public timeline 4 times a minute. I read about the cached for 30 seconds, which is just not accurate, the public timeline is updated often. I see it updated on every call, you can call it 60 times in a minute, and will get 60 different result sets. Unless there is a way to pull more than 20 at a time, and I am just getting batches outside the 20 I last called. I did not see the number of results returned as something I can control in the API. I mentioned the frequency in which I was calling the public timeline in my whitelist request, and there was no issue with it. I can not use the streaming API as it does not give me the same type of data I am after. I am going to move down to once every 30 seconds, as it seems four times every 60 seconds does not provide me any more value in the data I am parsing, and just fills the database up with 2x as much data to get the same result. -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 20, 2009, at 6:09 AM, John Kalucki wrote: Slightly off topic, but why are you calling the public timeline 10 times a minute? It's only updated once every 30 seconds. If you want more public statuses, you should be using the Streaming API. -John Kalucki http://twitter.com/jkalucki Services, Twitter Inc. On Oct 20, 12:44 am, Scott Haneda talkli...@newgeo.com wrote: This is getting out of hand, I am getting 10 a minute. I guess it is time to write that back off code that I figured would not be needed on the public timeline. -- Scott * If you contact me off list replace talklists@ with scott@ * On Oct 19, 2009, at 3:06 PM, Michael Steuer wrote: I've been seeing fail whales on the site as well.. Something must be going on... On 10/19/09 3:03 PM, Scott Haneda talkli...@newgeo.com wrote: I am calling the public timeline once every 60 seconds. Today has been a bad day, 26 total 502 errors since 12:21 PST to now. http public timeline is fine on the website. Any ideas when this is going to be resolved, or if it is even something that Twitter is aware of? If not, heads up :)
[twitter-dev] Re: Im a newbie, eager to learn, teach me how to setup API
The Twitter Dev/API list is the one list that there is a very often posted thread of this nature. I call it the Twitter Think Tank BandWagon™ :) Twitter hits the news, the news reports that application x, y, and z are all getting millions of users, millions of users translates to some form of money, popularity, or general enticement. And off to the races we are. However, programming is no different than any other industry. And this reply goes out to North Carolina developer and well as any other Developer that makes his way here. Reporters work hard to become reporters, veterinarians work hard to become vets, and programmers work hard to become programmers. Starting your own business is the American Dream. They are all hard work. Sure there are some overnight success stories, but those people all had past experience in a development field. If you want to get in on any form of new development, you would do yourself good to think about it as any other idea you have ever had. If you were sitting at home, and decided you were going to start a new company, say, one that makes drills for the construction business. Most logical people would start looking at other drills, taking them apart, calling motor manufacturers, researching all aspect of drills and the parts that make them up. If after all that, you still think you have an edge, you may be able to move forward. Development, be it Twitter or any other computer programming related field, is no different. It very well may be harder. Quite frankly, the lack of respect some developers get compared to other fields is a little disconcerting. It takes many years to become proficient in any programming language. That does not include the ramp up time to learn the basics of what ssh, ftp, sftp, tcp/ip ack/ syn, post, get, json, rest, ajax, html, css, oop, precedural, I could go on. There is just so much to learn, and so many parts, I would say it is one of the harder things to embark on. Add into all this, as a startup, not only are you learning the technical sides of things, but you are also learning how to run a business, marketing, etc, and each of those sub aspects of your end goal, has just as large a laundry list of acronyms to define it as well. I do not want to discourage anyone, as anyone can learn anything, if they put their mind to it. But please, of the many people who come to this list, it is borderline insulting to say I have idea x, how do I do it. No one walks into NASA and says, I want to build a rocket, where do I get started. It is that very analogy that non developers need to understand. In all honestly, every passionate developer is a rocket scientist in their own special way. At the very least, they are a scientist. To specifically answer the original posters questions: 1) If you are looking at a free host, you need to start your research learning and understanding about hardware and software in general. You need to learn that the backbone of your entire business is going to sit on a server, or servers somewhere. Do not run your business on free, it is not possible. Learn why this is not possible. 2) How do you display feeds? Start learning about RSS and any push/ pull driven protocol. In the end, it is all just a stream of data, you read it in, and parse it, and display it how you want. It will be up to you to determine the logic of how you do that. There are at least 20 variables I could ask, such as, how many feeds, when do you want them to expire, what if they contain profanity, do you want links on the feeds, etc, the list goes on. These are decisions you need to make, and then learn how to programatically implement them. 3) If you have been through 20+ tutorials, and still are not getting it, then you are looking at twitter tutorials, and you should be looking at ftp, php, perl, apache, server, and other more general tutorials. Can you build a rock, paper scissors game in php that runs in a web browser? Until you can take any idea you have and write pseudo code as to how you would deploy it, asking on the Twitter list for specifics is too far outside the scope of what you currently understand. 4) Curl I would lump into #3, it is just a tool, that can be used within any language, to do some work, what you do with the data it returns, comes back to general programming logic and understanding. 5) In regards to how to interact with the API. This is again, more detail that is not important at this time. 6) I am taking some guesses here. My suggestions.. a) Find a patient web designer friend, have them show you how to design web sites, so you learn ftp, and basic development of non programming aspect of the web. b) Start making simple apps in php, 50 lines or less. Rock, paper, scissors, blackjack, or towers of hanoi, any of the
[twitter-dev] Re: Im a newbie, eager to learn, teach me how to setup API
On Oct 27, 2009, at 4:05 AM, Dwi Sasongko Supriyadi wrote: Twitter is not a point and click API, none are; a Twitter programmer could build any web app they want. With that in mind, I would look to forums and mailing lists for beginner introductions to programming. A good programmer could read the twitter API, and make an app in a day, as they could with any API, be it one from Twitter, google, Amazon, ebay, or a private one. Hope that was helpful. [snip... Big huge rant about how Twitter dev is a strange mailing list :) ] May I, cat this /my/blog ? Certainly, I was going to work it over as a blog post, with some links to what exactly development is. But please, be my guest, anything in the mail list is public as far as i am concerned. -- Scott * If you contact me off list replace talklists@ with scott@ *
[twitter-dev] For what it is worth, twitter and facebook are both in spamcop
Whether you like or dislike DNS Bl's they are part of email severs in wide use today. I had both twitter.com and facebook.com whitelisted, so I was ok, however, http://www.spamcop.net/w3m?action=checkblockip=128.121.146.141 http://www.spamcop.net/w3m?action=checkblockip=69.63.178.175 Still listed. I posted the facebook one because I am sure people are friends with each other. What is very concerning, is that twitter has sent mail to spamtraps, that means only one thing, which is that an address was harvested. Assuming no malicious spamcop user put the address into their system, this is pretty grim behavior. twitter has been in spamcop Listing History In the past 20.4 days, it has been listed 8 times for a total of 7.4 days At the end of the day, a lot of twitter users are not getting email alerts, or any emails. -- Scott * If you contact me off list replace talklists@ with scott@ *
Re: [twitter-dev] Re: Twitter PHP + CentOS
Apparently you have issues pasting image urls as well :)
I would try to see what the errors are, there are a few ways you can
do this, depending on how centOS is set up.
http://php.net/manual/en/function.error-reporting.php
You can add the error handling enabler to your script, this works most
of the time. With certain errors, the script will not have run,
therefore, the enabling of the error can not be run to report an error.
In those cases, you need to set the error enabling in .htaccess or
php.ini, neither of which you want to leave on, once going to
production state. I usually wrap them up in a way that only loads
an .htaccess file for my IP, and a strict one for everyone else.
In .htaccess it may be something like this:
php_flag display_errors On
- or -
php_flag display_errors 1
* I get different results depending on different php's if it is string
or boolean.
Now, when you load your page, you will get the exact error as the page
generated it. If you have full control over the server, it may also
be just as simple to set the logging to a file, usually the error_log
for Apache. Then you can `tail -f error_log` in your shell. This is
generally default behavior, check phpInfo() for the path.
I have a feeling you are missing curl. It may be a simpler test to
just make one call to check for it:
if (function_exists('curl')) {
echo 'good';
} else {
echo 'bad';
}
Maybe php was installed with curl, but the OS itself is missing curl?
Can you `curl --head http://google.com` in a shell on this machine?
--
Scott * If you contact me off list replace talklists@ with scott@ *
On Nov 27, 2009, at 7:50 PM, aztroboy wrote:
It seems that I can't send imgs, anyways, here are the screenshots:
http:// img16. imageshack . us /img16/7901/curl.png
http:// img38. imageshack. us/img38/4926/curl2.png
On 25 nov, 20:55, Michael Steuer mste...@gmail.com wrote:
Create a page that prints out phpinfo(). See if the curl module is
active in your php
On Nov 25, 2009, at 5:19 PM, aztroboy jbasur...@gmail.com wrote:
Thank you for your reply,
[r...@twirelezz ~]# php -version
PHP 5.1.6 (cli) (built: Apr 7 2009 08:00:04)
Copyright (c) 1997-2006 The PHP Group
Zend Engine v2.1.0, Copyright (c) 1998-2006 Zend Technologies
Not bad, now I updated PHP to its lastest version:
[r...@twirelezz ~]# php -v
PHP 5.3.1 (cli) (built: Nov 20 2009 17:51:14)
Copyright (c) 1997-2009 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2009 Zend Technologies
However, I'm getting the same response: twitter won't give me the
auth_token :( (or at least, it's giving me a null response)
On 25 nov, 18:36, jmathai jmat...@gmail.com wrote:
Check the versions of software you're using. CentOS likes to make
you
jump through hoops in order to get newer software.
Namely, check your PHP version.
On Nov 25, 9:08 am, aztroboy jbasur...@gmail.com wrote:
Hello there! I'm using a PHP Twitter OAuth library
(twitter.abrah.am),
and it works perfect on Windows: the php script looks asks twitter
for
an auth token. However, as soon as I put the same script on CentOS
(with its default httpd service), it won't give any auth tokens at
all.
The script uses cURL to get the token from twitter. I've used cURL
(curlwww.google.com) and it works fine in console.
What could be going wrong here? Is there something I should add of
change first in order to receive the tokens? By the way, I got the
CentOS default FW set allowing traffic on port 80 and SSH only.
I would like to ask for a hint about what should I do, I'm kinda
newbie on CentOS.
thank you in advance
Re: [twitter-dev] Spurious SMS notifications
My G/F is getting the same thing. She has DM's set to go via SMS, but she also gets tweets as SMS on a daly basis, at some pretty unruly hours. These are just normal tweets, by people she is following. Some, if not most, are not following her back. I do not have this problem, and follow much larger base of people than she does. However, I also follow the same people she is following, which are somehow able to get a normal tweet to go over SMS to her. Is this a known issue, is there some way to make it stop? It is important to get DM's via SMS, but it is also important to not get woken up at 4AM from @someone_i_just_follow. -- Scott * If you contact me off list replace talklists@ with scott@ * On Dec 9, 2009, at 3:02 PM, Kee Hinckley wrote: I am receiving spurious SMS notifications for tweets sent by people I am following, but do not have notifications turned on for. These are *not* retweets, I've even asked a couple of the original senders to check and see if the item had been retweeted, and the answer was no. Furthermore, I only have my account (@nazgul) set to receive text messages for about four users, and none of them retweeted these messages. Possibly related, I have a friend who turned off *all* phone notifications over a day ago, and is still getting random new text messages from people she follows but didn't set to send her notificationqs. Examples: I confirmed that these were not retweeted by anyone for whom I get notifications (or in one case, by anyone at all). http://twitter.com/stevegarfield/status/6463021810 http://twitter.com/daddyclaxton/status/6463935419 Others I haven't had a chance to investigate, but which it seems highly unlikely were retweeted by anyone who sends me notifications. http://twitter.com/kevinmarks/status/6500306267 http://twitter.com/stoweboyd/status/6497224194
[twitter-dev] The Twitter DNS fiasco
Am I the only one who thinks this is somewhat disingenous, or at least lacking in details? http://blog.twitter.com/2009/12/update-on-last-nights-dns-disruption.html I dontv even know what happened. I'm guessing, somehow, Twitter.com had their DNS records pointed to some arbitrary host. I believe bluehost, that's what dig told me at the time. How bluehost handled that traffic is a testament to the 6.00 account it must have been sitting on. Is that what happened? Why did bluehost not immediately close that ip? But the statement that no accounts are believed to be compromised... How many have remember me enabled? Doesn't this mean all those users had their login cookie sent along for capture? If the hackers were more nefarious, they could have easily cloned the login/pass box and captured the credentials and redirected to fail whale. Smarter still, round robin the ip's to only 1 being false, most would get in, but those who did not just gave up login and pass details. They will try again later and all would work fine. This would have taken much longer to rven discover. How did someone get control of DNS? With twitters size, could a call not been made to netsol, openDNS, 8.8.8.8, and the rest of the large 3rd party dns providers to shunt in records with the correct IP's for a shirt time, until the real TTL's refreshed? Netsol could have solved it in one swoop. I think a lot more detail about this need to be disclosed. This does not seem like a Twitter security issue, it seems like a DNS issue, largely outside of twitters control. Why not explain that? Right now it appears twitter got hacked, again, but I dont think that to be the case, though this blog posts lack of detail makes the public feel Twitter was hacked. Where did all the forgot password emails go, were MX records also put in place. Where did email in general go, can we see the hacked zone copy put in place? Twitter did little wrong here, the blog post is so vague, it makes the general public think It's twitters Machines, which if I understand this, it's not. Pretty sure I could self fix this with a few entries to /etc/hosts or in my case, I would have just added the zone to my RR, had i known what to add in. Comments appreciated. -- Scott (Sent from a mobile device)
Re: [twitter-dev] 168.143.162.36... Connection timed out
On Dec 20, 2009, at 5:53 AM, shiplu shiplu@gmail.com wrote: Why is this? $ curl -iv http://twitter.com/shiplu * About to connect() to twitter.com port 80 (#0) * Trying 168.143.162.36... Connection timed out * couldn't connect to host * Closing connection #0 curl: (7) couldn't connect to host I am facing this for almost 3-4 days. Telnet to the ip with correct port, run traceroute or mtr. Report your results back here. -- Scott (Sent from a mobile device)
