Ok, I can replicate your results with curl
$ curl -v "http://search.twitter.com/search.json?q=\"exeter%20city\"";
...returns the wrong results, as you say.
$ curl -v "http://search.twitter.com/search.json?q=%22exeter%20city
%22"
...returns the correct results.
I think double quotes are not actually valid URL characters (tho some
browsers try to treat them as such), so you should really turn " into
%22 before the requests go out.
That said, I'm starting to agree with Paul that twitter is doing some
sort of encoding trick on their end when literal quotes are sent in
the request:
$ curl -v http://search.twitter.com/search.json?q=\"exeter%20city\";
* About to connect() to search.twitter.com port 80 (#0)
* Trying 128.121.146.107... connected
* Connected to search.twitter.com (128.121.146.107) port 80 (#0)
GET /search.json?q="exeter%20city" HTTP/1.1
User-Agent: curl/7.16.4 (i486-pc-linux-gnu) libcurl/7.16.4 OpenSSL/
0.9.8e zlib/1.2.3.3 libidn/1.0
Host: search.twitter.com
Accept: */*
< HTTP/1.1 200 OK
< Date: Tue, 03 Mar 2009 17:09:34 GMT
< Server: hi
< Status: 200 OK
< Cache-Control: max-age=20, must-revalidate, max-age=300
< Content-Type: application/json; charset=utf-8
< X-Served-By: searchweb003.twitter.com
< Expires: Tue, 03 Mar 2009 17:14:34 GMT
< Content-Length: 195
< Vary: Accept-Encoding
< X-Varnish: 1733084231
< Age: 0
< Via: 1.1 varnish
< X-Cache-Svr: searchweb003.twitter.com
< X-Cache: MISS
< Connection: close
<
* Closing connection #0
{"results":[],"since_id":0,"max_id":1274236746,"refresh_url":"?
since_id=1274236746&q=%22exeter%2520city%22","results_per_page":
15,"completed_in":0.112164,"page":1,"query":"%22exeter%2520city%22"}
Now, one could argue that the request itself is invalid or malformed,
and so the result may be undefined, but I do agree that something is
happening on twitter's end.
Moral of the story: encode " as %22 in URLs.
-Chad
On Tue, Mar 3, 2009 at 11:33 AM, Paul Kinlan <[email protected]>
wrote:
Hi,
It works with the "+", but I knew that :)
With a space (in IE) it encodes it as %20 when it makes the request
and I
can see it through fiddler (as below) and it comes back.
GET /search.json?q="exeter%20city" HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
application/x-ms-application, application/vnd.ms-xpsdocument,
application/xaml+xml, application/x-ms-xbap, application/x-
shockwave-flash,
application/vnd.ms-excel, application/vnd.ms-powerpoint,
application/msword,
*/*
Accept-Language: en-gb
UA-CPU: x86
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0;
SLCC1; .NET
CLR 2.0.50727; .NET CLR 3.5.21022; .NET CLR 3.5.30729; .NET CLR
3.0.30618;
InfoPath.2; OfficeLiveConnector.1.3; OfficeLivePatch.0.0)
Host: search.twitter.com
Connection: Keep-Alive
Cookie:
__utma
=43838368.379476752167577530.1234449205.1234449205.1234449205.1;
__utmz=43838368.1234449205.1.1.utmcsr=blog.twe2.com|
utmccn=(referral)|utmcmd=referral|utmcct=/;
__utmv=43838368.lang%3A%20en_GB
I fully accept it is probably our client software that is not
encoding
correcly, but I also tried this from the command line curl
"http://search.twitter.com/search.json?q=\"exeter%20city\""; the
response
comes back as %22exeter%2520city%22 in the json object. From my
point of
view I know the quotes are not correct, but it looks like twitter is
encoding them when it recieves them. I belive our client API is
sending
double quotes rather %22.
Kind Regards,
Paul
2009/3/3 Chad Etzel <[email protected]>
I also just tested searching "exeter city" in TweetGrid with IE,
FireFox, and Chrome. All came back with the same results.
fwiw,
-Chad
On Tue, Mar 3, 2009 at 11:14 AM, Matt Sanford <[email protected]>
wrote:
Hi Paul,
I just tested form the command line and everything seems fine
with: curl
'http://search.twitter.com/search.json?q=%22exeter%20city%22'
If you are typing %20 into the IE address bar it is likely try
to
correct your % (which is not a valid URL character) and making
it %25
in
the request but displaying it correctly to you. Try replacing it
with a
+ or
a space and see what you get.
Thanks;
— Matt
- Show quoted text -
On Mar 3, 2009, at 08:06 AM, Paul Kinlan wrote:
Forgot to add, I am checking our client library now too.
Paul.
2009/3/3 Paul Kinlan <[email protected]>
Hi Matt,
I was typing the search term through IE (to test it after
reports that
""
enclosed searches aren't working) as
http://search.twitter.com/search.json?q="exeter city" which it
then
converts
to http://search.twitter.com/search.json?q="exeter%20city"; but the
result
came back as "%22exeter%2520city%22" (see json object below) in
the
search
API json object. It works in firefox so I am presuming firefox is
correctly
encoding the url.
{"results":[],"since_id":0,"max_id":1273765306,"refresh_url":"?
since_id=1273765306&q=%22exeter%2520city%22","results_per_page":
15,"completed_in":1.313905,"page":1,"query":"%22exeter%2520city
%22"}
it is highly likely that if IE is having the issue, the client API
would
probably have it, however the query that is going out over the
wire (I
checked with fiddler as "exeter%20city" and the result comes
back as
above,
so I don't think it is us for the entire problem).
Kind Regards,
Paul.
2009/3/3 Matt Sanford <[email protected]>
Hi Paul,
It sounds like whatever is generating your API requests is
double
URL
encoding. So the space becomes %20, and then on the second url
encoding the
% becomes a %25.
Thanks;
— Matt Sanford / @mzsanford
On Mar 3, 2009, at 07:34 AM, Paul Kinlan wrote:
Hi,
I am noticing something that I think is odd at the moment.
Some of our users are not getting searches that are enclosed in
quotes
via the API, yet they work directly from the website.
For example there is a difference between the following query
on the
API
and Website:
http://search.twitter.com/search?q=%22exeter%20city%22 which
has the
same
results as http://search.twitter.com/search?q=%22exeter+city%22
but returns a different result via the API using the following
query
http://search.twitter.com/search.json?q="exeter%20city";
Looking at what is returned by the API the query looks like it
has
been
transformed in to "%22exeter%2520city%22". To me the %2520
looks odd
when I
would expect %20
Kind Regards,
Paul Kinlan
