Thanks for the quick reply!

I went back to testing with command line with a curl call, it worked
so I realized the PHP code was buffering and I did not realize it on
large amount of results...

The problem was with an internal buffer between cURL and the PHP
Stream. I went another way instead and posting a PHP base class if it
may help anyone else.

I typically extend it to override handleTweet($tweet).


<?php

class snow_twitter_bot {

    protected $buffer;


    /**
     *  Read a stream looking for track keywords on behalf of a user
     *
     *
     * @param string $track                             Comma separated values 
keywords
     * @param string $twitter_user              Twitter username
     * @param string $twitter_pass              Twitter password
     */
    public function readStreamFilter( $track, $twitter_user,
$twitter_pass )
    {

                $wait = 10;
                while( true )
                {

                        echo "\n" . date("c") . " - Connecting to Twitter 
Stream\n";

                        // Configuration of curl
                        $ch = curl_init();
                        curl_setopt($ch, CURLOPT_URL,                           
"http://stream.twitter.com/1/
statuses/filter.json");
                        curl_setopt($ch, CURLOPT_HEADER,                        
0);
                        curl_setopt($ch, CURLOPT_RETURNTRANSFER,        
"default");
                        curl_setopt($ch, CURLOPT_POST,                          
true);
                        curl_setopt($ch, CURLOPT_POSTFIELDS,            
"track=$track");
                        curl_setopt($ch, CURLOPT_HTTPAUTH,                      
CURLAUTH_BASIC ) ;
                        curl_setopt($ch, CURLOPT_USERPWD,                       
"$twitter_user:
$twitter_pass");

                        curl_setopt($ch, CURLOPT_WRITEFUNCTION,         
array($this,
"receiveResponse"));

                        curl_exec($ch);

                        $info = curl_getinfo($ch);

                        if( $info['http_code'] == 200 )
                                $wait = 10;
                        else
                        {
                                echo "\n" . date("c") . " - ERROR 
($info[http_code]) - Waiting
$wait sec \n";
                                sleep ( $wait );
                                $wait *= ($wait >= 240 ) ? 1 : 2;
                        }
                }


                curl_close($ch);
    }


    /**
     *  Call back function for cURL
     *
     *
     * @param curl handler $curlHandle
     * @param string $data
     */
    private function receiveResponse( $curlHandle, $data )
    {
        $lines = explode("\n", $data);

        // The buffer contains the end of the last line from previous
time
        // => Is goes at the beginning of the first line we are
getting this time
        $lines[0] = $this->buffer . $lines[0];

        // And the last line os only partial
        // => save it for next time, and remove it from the list this
time
        $nb_lines = count($lines);
        $this->buffer = $lines[$nb_lines-1];
        unset($lines[$nb_lines-1]);

        // Here, do your work with the lines you have in the buffer
        foreach( $lines as $one )
        {
                $tweet = @json_decode($one,1);
                if(isset($tweet['user']['screen_name']))
                {
                        $this->handleTweet( $tweet );
                }
        }

                return strlen( $data );
    }



    /**
     *  Called for each tweet.
     *
     *  Override me with you logic /twss
     *
     *
     * @param array $tweet      A single tweet parsed from json, see online
doc.
     */
    public function handleTweet( $tweet )
    {
        echo '@'.$tweet['user']['screen_name']." at " .
$tweet['created_at']. " says:\n";
        echo $tweet['text']."\n";
        echo "\n------ Memory: " . round(memory_get_usage(true)/
1024) ."kb\n";
    }


}


?>


On Sep 21, 12:57 am, John Kalucki <j...@twitter.com> wrote:
> You say you are backing off on HTTP connections. Are you connecting
> constantly, or just once, and staying connected for very long periods? Are
> you tracking the number of limit messages you are receiving? Perhaps your
> track search was rate limited?
>
> -John Kaluckihttp://twitter.com/jkalucki
> Twitter, Inc.
>
> On Mon, Sep 20, 2010 at 2:46 PM, Nicolas Grasset
> <nicolas.gras...@gmail.com>wrote:
>
>
>
> > Hi there,
>
> > I have been reading many comments here, and while it seems at least
> > another person had the same issue, I could not find an answer.
>
> > I have a test account (@hemtexdream) using the statuses/filter stream
> > api which works on words like twitter or trending hashtags like
> > #ireallythink.
>
> > However, I would like it to work with track=#dreamproject and it gets
> > none of the tweets from my main account (@fellowshipofone). Tried over
> > long period of time and with a few settings. I am fairly sure I am not
> > rate limited since I implement the logic to report http error
> > responses and back off exponentially.
>
> > Also, if of any help, both accounts can find at least 2 tweets in the
> > website search page.
>
> > Is it an index question? Is it a "sample" question?
>
> > Thanks for your help!
>
> > --
> > Twitter developer documentation and resources:http://dev.twitter.com/doc
> > API updates via Twitter:http://twitter.com/twitterapi
> > Issues/Enhancements Tracker:
> >http://code.google.com/p/twitter-api/issues/list
> > Change your membership to this group:
> >http://groups.google.com/group/twitter-development-talk?hl=en

-- 
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group: 
http://groups.google.com/group/twitter-development-talk?hl=en

Reply via email to