Re: [twitter-dev] Re: Need help with the streaming API syntax....specifically how to point to the track text file without using curl
There is indeed a hard limit to the length of URLs. POST parameters, however, can be quite large. We have many clients that send parameters with hundreds of thousands to millions of terms, so this is broadly possible., Your HTTP client may or many not support this scale. -John Kalucki http://twitter.com/jkalucki Infrastructure, Twitter Inc. On Sun, Mar 7, 2010 at 5:57 PM, Mad Euchre mad.ukrain...@gmail.com wrote: It turns out you were right with this and John was right with the POST. Thanks to all who replied. As soon as I changed (request.readuntilend()) to (request.readline()) the data started flowing in. I don't want to muddy the water on this thread but I will by asking the next predictable problem. If I have query string that is ? track=peter,paul,mary, etc for 1000 terms..won't that exceed some http limit on length? How does one track about 1000 terms on a single stream connection? even if there is a method called request.query(track=peter,paul,mary,etc) isn't that just a substitute for putting it on the actual URL thus still exceding some length limit? Thanks, peter On Mar 5, 4:38 pm, Mark McBride mmcbr...@twitter.com wrote: I think this is slightly backwards. You want to use the GET method, but set up the URI you have (with the track=Microsoft parameter). You will also need to authenticate. Note that this is a streaming API. I don't know VB all that well, but there's a reasonable chance that this call only returns data when the HTTP call has finished. The streaming API will *never* finish, so you'll need to parse data as it's available. Without looking at VB doc I have no idea how you would set that up. ---Mark http://twitter.com/mccv On Fri, Mar 5, 2010 at 9:54 AM, Mad Euchre mad.ukrain...@gmail.com wrote: Thanks. Now I'm using the post method. How should I use the track parameter? Something like this? address = New Uri(http://stream.twitter.com/1/statuses/filter.json? track=Microsoft) I'm getting connected but no data that matches Microsoft is streaming over.No data for that matter. I'm passing my name and pw in the request.credentials method. The server returned a 200 OK when I added the credentials but not when it was in the URL alone. ie; address = New Uri(http://stream.twitter.com/1/statuses/ filter.json?track=Microsoft - name:pw Thanks, Peter On Mar 2, 5:19 pm, John Kalucki j...@twitter.com wrote: The text file approach only applies to POST parameters set from the curl command, and in no other case. When creating an HTTP client from within a program, you should be able to configure the POST parameters via method calls. If you can't, it's a pretty worthless HTTP library. Each client library is different, check your docs. -John Kaluckihttp://twitter.com/jkalucki Infrastructure, Twitter Inc. On Tue, Mar 2, 2010 at 12:42 PM, Mad Euchre mad.ukrain...@gmail.com wrote: This is the VB code I would use to start any http stream request = DirectCast(WebRequest.Create(http:// stream.twitter.com/1/statuses/filter.json - name:pw), HttpWebRequest) request.Credentials = New NetworkCredential(name, pw) ' Get response response = DirectCast(request.GetResponse(), HttpWebResponse) ' Get the response stream into a reader reader = New StreamReader(response.GetResponseStream()) The streaming api documentation says to create a file called track.txt and add text similar to this without the quotes. track=peter, paul, mary Then use curl @track.txthttp:// stream.twitter.com/1/statuses/filter.json - name:pw I can't believe I have to shell out to DOS and run the curl command line. My direct question is how do others incorportate the @track.txt in the VB.Net web request? Maybe something like this? http://stream.twitter.com/1/statuses/filter.json-name:pw? track.txt Thanks- Hide quoted text - - Show quoted text -- Hide quoted text - - Show quoted text -
Re: [twitter-dev] Re: Need help with the streaming API syntax....specifically how to point to the track text file without using curl
I think this is slightly backwards. You want to use the GET method, but set up the URI you have (with the track=Microsoft parameter). You will also need to authenticate. Note that this is a streaming API. I don't know VB all that well, but there's a reasonable chance that this call only returns data when the HTTP call has finished. The streaming API will *never* finish, so you'll need to parse data as it's available. Without looking at VB doc I have no idea how you would set that up. ---Mark http://twitter.com/mccv On Fri, Mar 5, 2010 at 9:54 AM, Mad Euchre mad.ukrain...@gmail.com wrote: Thanks. Now I'm using the post method. How should I use the track parameter? Something like this? address = New Uri(http://stream.twitter.com/1/statuses/filter.json? track=Microsoft) I'm getting connected but no data that matches Microsoft is streaming over.No data for that matter. I'm passing my name and pw in the request.credentials method. The server returned a 200 OK when I added the credentials but not when it was in the URL alone. ie; address = New Uri(http://stream.twitter.com/1/statuses/ filter.json?track=Microsoft - name:pw Thanks, Peter On Mar 2, 5:19 pm, John Kalucki j...@twitter.com wrote: The text file approach only applies to POST parameters set from the curl command, and in no other case. When creating an HTTP client from within a program, you should be able to configure the POST parameters via method calls. If you can't, it's a pretty worthless HTTP library. Each client library is different, check your docs. -John Kaluckihttp://twitter.com/jkalucki Infrastructure, Twitter Inc. On Tue, Mar 2, 2010 at 12:42 PM, Mad Euchre mad.ukrain...@gmail.com wrote: This is the VB code I would use to start any http stream request = DirectCast(WebRequest.Create(http:// stream.twitter.com/1/statuses/filter.json - name:pw), HttpWebRequest) request.Credentials = New NetworkCredential(name, pw) ' Get response response = DirectCast(request.GetResponse(), HttpWebResponse) ' Get the response stream into a reader reader = New StreamReader(response.GetResponseStream()) The streaming api documentation says to create a file called track.txt and add text similar to this without the quotes. track=peter, paul, mary Then use curl @track.txthttp:// stream.twitter.com/1/statuses/filter.json - name:pw I can't believe I have to shell out to DOS and run the curl command line. My direct question is how do others incorportate the @track.txt in the VB.Net web request? Maybe something like this? http://stream.twitter.com/1/statuses/filter.json- name:pw? track.txt Thanks- Hide quoted text - - Show quoted text -
Re: [twitter-dev] Re: Need help with the streaming API syntax....specifically how to point to the track text file without using curl
You need a client returning incremental HTTP responses. I don't think WebResponse does that. TcpClient definitely does, that's what I'm using in C#. ∞ Andy Badera ∞ +1 518-641-1280 Google Voice ∞ This email is: [ ] bloggable [x] ask first [ ] private ∞ Google me: http://www.google.com/search?q=andrew%20badera On Fri, Mar 5, 2010 at 4:38 PM, Mark McBride mmcbr...@twitter.com wrote: I think this is slightly backwards. You want to use the GET method, but set up the URI you have (with the track=Microsoft parameter). You will also need to authenticate. Note that this is a streaming API. I don't know VB all that well, but there's a reasonable chance that this call only returns data when the HTTP call has finished. The streaming API will *never* finish, so you'll need to parse data as it's available. Without looking at VB doc I have no idea how you would set that up. ---Mark http://twitter.com/mccv On Fri, Mar 5, 2010 at 9:54 AM, Mad Euchre mad.ukrain...@gmail.com wrote: Thanks. Now I'm using the post method. How should I use the track parameter? Something like this? address = New Uri(http://stream.twitter.com/1/statuses/filter.json? track=Microsoft) I'm getting connected but no data that matches Microsoft is streaming over.No data for that matter. I'm passing my name and pw in the request.credentials method. The server returned a 200 OK when I added the credentials but not when it was in the URL alone. ie; address = New Uri(http://stream.twitter.com/1/statuses/ filter.json?track=Microsoft - name:pw Thanks, Peter On Mar 2, 5:19 pm, John Kalucki j...@twitter.com wrote: The text file approach only applies to POST parameters set from the curl command, and in no other case. When creating an HTTP client from within a program, you should be able to configure the POST parameters via method calls. If you can't, it's a pretty worthless HTTP library. Each client library is different, check your docs. -John Kaluckihttp://twitter.com/jkalucki Infrastructure, Twitter Inc. On Tue, Mar 2, 2010 at 12:42 PM, Mad Euchre mad.ukrain...@gmail.com wrote: This is the VB code I would use to start any http stream request = DirectCast(WebRequest.Create(http:// stream.twitter.com/1/statuses/filter.json - name:pw), HttpWebRequest) request.Credentials = New NetworkCredential(name, pw) ' Get response response = DirectCast(request.GetResponse(), HttpWebResponse) ' Get the response stream into a reader reader = New StreamReader(response.GetResponseStream()) The streaming api documentation says to create a file called track.txt and add text similar to this without the quotes. track=peter, paul, mary Then use curl @track.txthttp://stream.twitter.com/1/statuses/filter.json - name:pw I can't believe I have to shell out to DOS and run the curl command line. My direct question is how do others incorportate the @track.txt in the VB.Net web request? Maybe something like this? http://stream.twitter.com/1/statuses/filter.json- name:pw? track.txt Thanks- Hide quoted text - - Show quoted text -
Re: [twitter-dev] Re: Need help with the streaming API syntax....specifically how to point to the track text file without using curl
Let me clarify... You can do one of 1) Use GET, and specify the parameters as part of the URL query string (e.g. http://stream.twitter.com/1/statuses/filter.json?track=Microsofthttp://stream.twitter.com/1/statuses/filter.json? track=Microsoft ) 2) Use POST, and pass your arguments in through some VB method. In this case the URL will be http://stream.twitter.com/1/statuses/filter.jsonhttp://stream.twitter.com/1/statuses/filter.json? track=Microsoft ---Mark http://twitter.com/mccv On Fri, Mar 5, 2010 at 2:55 PM, Mad Euchre mad.ukrain...@gmail.com wrote: I switch to post after reading John's response. When creating an HTTP client from within a program, you should be able to configure the POST parameters via method calls. If you can't, it's a pretty worthless HTTP library. Each client library is different, check your docs. -John Kalucki On Mar 5, 4:38 pm, Mark McBride mmcbr...@twitter.com wrote: I think this is slightly backwards. You want to use the GET method, but set up the URI you have (with the track=Microsoft parameter). You will also need to authenticate. Note that this is a streaming API. I don't know VB all that well, but there's a reasonable chance that this call only returns data when the HTTP call has finished. The streaming API will *never* finish, so you'll need to parse data as it's available. Without looking at VB doc I have no idea how you would set that up. ---Mark http://twitter.com/mccv On Fri, Mar 5, 2010 at 9:54 AM, Mad Euchre mad.ukrain...@gmail.com wrote: Thanks. Now I'm using the post method. How should I use the track parameter? Something like this? address = New Uri(http://stream.twitter.com/1/statuses/filter.json? track=Microsoft) I'm getting connected but no data that matches Microsoft is streaming over.No data for that matter. I'm passing my name and pw in the request.credentials method. The server returned a 200 OK when I added the credentials but not when it was in the URL alone. ie; address = New Uri(http://stream.twitter.com/1/statuses/ filter.json?track=Microsoft - name:pw Thanks, Peter On Mar 2, 5:19 pm, John Kalucki j...@twitter.com wrote: The text file approach only applies to POST parameters set from the curl command, and in no other case. When creating an HTTP client from within a program, you should be able to configure the POST parameters via method calls. If you can't, it's a pretty worthless HTTP library. Each client library is different, check your docs. -John Kaluckihttp://twitter.com/jkalucki Infrastructure, Twitter Inc. On Tue, Mar 2, 2010 at 12:42 PM, Mad Euchre mad.ukrain...@gmail.com wrote: This is the VB code I would use to start any http stream request = DirectCast(WebRequest.Create(http:// stream.twitter.com/1/statuses/filter.json - name:pw), HttpWebRequest) request.Credentials = New NetworkCredential(name, pw) ' Get response response = DirectCast(request.GetResponse(), HttpWebResponse) ' Get the response stream into a reader reader = New StreamReader(response.GetResponseStream()) The streaming api documentation says to create a file called track.txt and add text similar to this without the quotes. track=peter, paul, mary Then use curl @track.txthttp:// stream.twitter.com/1/statuses/filter.json - name:pw I can't believe I have to shell out to DOS and run the curl command line. My direct question is how do others incorportate the @track.txt in the VB.Net web request? Maybe something like this? http://stream.twitter.com/1/statuses/filter.json-name:pw? track.txt Thanks- Hide quoted text - - Show quoted text -- Hide quoted text - - Show quoted text -
