On Wed, Dec 3, 2008 at 9:49 PM, avip <[EMAIL PROTECTED]> wrote:
>
> Hi Tim,
>
> I was wondering if you could post some example consumer/worker code
> using your class?
>
> I've managed to daemonize a php worker script but inevitably there's a
> polling while(1) loop somewhere attempting to reserve() a job. This
> works fine when there are actually jobs on the queue, but for an empty
> queue cpu-usage shoots up to 99%.
>
> To test out whether or not this is a task or invocation issue, I tried
> replacing the job reservation/worker code with a simple print
> statement. The result was a well-behaved background worker. When I
> switched back to trying to reserve and delete a job, cpu-usage shot
> back up on an empty queue.
>
> Which obviously leads me to suspect there's some other loop somewhere
> in the class responsible for reading or blocking or...not sure
> what...that's causing this.
Hi,
I ran into same problems at the beginning. Then I switched to
sequential peek method:
$beanstalk = BeanStalk::open(
array(
'servers' => array(
'192.168.1.40:11300',
),
'select' => 'sequential peek',
'peek_usleep' => 2500, // 2.5 ms
)
);
and no 100% CPU problem now.
Hope it helps,
Ludo
>
> If you could shed some light on this?
>
> Thanks very much!
> --Avi
>
> On Nov 9, 3:56 pm, avip <[EMAIL PROTECTED]> wrote:
>> Hi yet again!
>>
>> Scratch the last post. I spent more time experimenting with a telnet
>> session and reading the protocol itself more carefully.
>>
>> For any other people who are completely new to beanstalkd and are like
>> me - wanting to go from installing server to writing code - take note:
>> the "use <tube>" command is *just* for producers. Attempting to
>> "switch tubes" using "use <tube>" and then reserve will not work and
>> actually block if the default tube is empty, not to mention confuse
>> you like it did me. The command you're actually looking for when you
>> wish to *consume* jobs is "watch <tube>"
>>
>> Very important-and much farther down the protocol doc-is the
>> following:
>>
>> The "watch" command adds the named tube to the watch list for the
>> current
>> connection. A reserve command will take a job from any of the tubes in
>> the
>> watch list. For each new connection, the watch list initially consists
>> of one
>> tube, named "default".
>>
>> Take my code from above as example, seems like it should work because
>> it's almost straight from the example. My first mistake was assuming
>> that. I ran the example first, it worked fine without any error, but
>> running it several times in a row after my own code failed, I noticed
>> it was erratically timing out. When it wasn't timing out, it behaves
>> the way I described in the post prior to this one, just silently
>> returns with no actual action taking place.
>>
>> Both the example or the example code above will timeout not because of
>> anyPHP-specific error but because doing "use_tube('test2')" doesn't
>> actually *switch* tubes in order for you to consume from that job-
>> queue - that is managed by the daemon for you provided you tell it you
>> wish to *watch* that tube. Not the most obvious thing to realise upon
>> initial inspection!
>>
>> So, in the above piece of code I pasted, reserve() is actually trying
>> to ask for a job from the tube "default" (created automatically),
>> since it is empty (because in a previous piece of code I, correctly,
>> used "use_tube('test2')" to add jobs), the function just blocks...in
>> other words, the would-be consumer waits for a job; however,PHPtimes
>> out the script based on its own timer.
>>
>> If you wish to consume jobs usingPHPscripts in the background
>> elsewhere without this time limit simply add "set_time_limit(0)" to
>> the top of the relevantPHPscript to allow for infinite time - this
>> will allow your consumer scripts to run in the background indefinitely
>> until there is a job to be done.
>>
>> Anyway, using "watch" fixes the problem for me, I hope this post saves
>> someone a few hours of confusion.
>> Tim, if you could correct me on anything I got wrong... :)
>>
>> Thanks!
>> --Avi
>> On Nov 8, 12:30 pm, avip <[EMAIL PROTECTED]> wrote:
>>
>> > Hi again Tim,
>>
>> > Running into some basic problems: I have a script that loads the queue
>> > with some jobs using put(), this works fine and is verified manually
>> > via telnet. In a different script, I attempt to reserve a job using:
>>
>> > <code>
>> > require('bin/Others/BeanStalk.class.php');
>>
>> > $beanstalk = BeanStalk::open(array('servers' =>
>> > array('127.0.0.1:11300'),'select' => 'random wait'));
>>
>> > $beanstalk->use_tube('test2');
>>
>> > $job = $beanstalk->reserve();
>> > echo "job=".print_r($job,true);
>>
>> > BeanStalk::delete($job);
>> > </code>
>>
>> > Almost straight out of the example, only difference is that put() and
>> > reserve() are spread out across two files. I've tried every select
>> > mode as well with similar results.
>>
>> > BeanStalk::__call(use_tube)
>> > BeanStalk::_do_my_server
>> > BeanStalk::reserve
>> > BeanQueue::reserve
>> > Fatal error: Maximum execution time of 30 seconds exceeded in /bin/
>> > Others/BeanStalk.class.phpon line 989
>>
>> > Since I added some echos to original, my line 989 is: $data =
>> > @fread($this->socket,$in_buf_size);
>>
>> > Is this related to the problems from yesterday? Also, you mentioned
>> > v1.1 will be released soon, could you be more specific? (weeks?
>> > months?) I'm just trying to figure out whether or not I should try the
>> > python or other libs for now.
>>
>> > Thanks!
>> > --Avi
>> > On Nov 7, 4:33 pm, avip <[EMAIL PROTECTED]> wrote:
>>
>> > > Ah! Of all the functions, I choose this one to test first with :)
>>
>> > > Will avoid the peek modes until next version, as suggested.
>>
>> > > Thanks again!
>> > > --Avi
>>
>> > > On Nov 7, 4:15 pm, "Tim Gunter" <[EMAIL PROTECTED]> wrote:
>>
>> > > > Hey,
>>
>> > > > Yeah this is an issue with beanstalk clients like mine, and beanstalkd
>> > > > 1.0. I was trying to implement a polling check that could pull the
>> > > > next ready job from a pool, but peek() didn't turn out to work as I
>> > > > initially expected.
>>
>> > > > Avoid the peek modes for now. Stick to 'sequential wait' and 'random
>> > > > wait' select values.
>>
>> > > > I will release v1.1 of my client shortly, and then the peek modes will
>> > > > work again, as expected.
>>
>> > > > On Fri, Nov 7, 2008 at 4:11 PM, avip <[EMAIL PROTECTED]> wrote:
>>
>> > > > > Hi,
>>
>> > > > > Wow, that was a fast turn-around! I thought it would take days :)
>> > > > > Thanks for both explanations, now that I read them (and after poking
>> > > > > in the code for a while), it seems pretty "obviously necessary" to
>> > > > > tell the function which server to perform the peek on given there's a
>> > > > > pool.
>>
>> > > > > I've been trying some very basicPHPscripts to learn the internals, I
>> > > > > have one script that puts a job onto the server (I manually verify
>> > > > > it's there with a telnet session). Then the following:
>> > > > > <code>
>> > > > > require('bin/Others/BeanStalk.class.php');
>> > > > > $beanstalk = BeanStalk::open(array('servers' =>
>> > > > > array('127.0.0.1:11300'),'select' => 'random peek'));
>>
>> > > > > $beanstalk->use_tube('test');
>>
>> > > > > $nextJob = '';
>> > > > > $ret = $beanstalk->peek_ready('127.0.0.1:11300',$nextJob);
>>
>> > > > > if ($ret === 1)
>> > > > > {
>> > > > > echo 'PEEK OPERATION OK<br/>';
>> > > > > echo 'Next ready job='.$nextJob.'<br/>';
>> > > > > }
>> > > > > else {
>> > > > > echo 'PEEK OPERATION FAILED<br/>';
>> > > > > }
>> > > > > </code>
>>
>> > > > > I also turned debugging on and some simple debugging echos to
>> > > > > peek_ready(). Result:
>>
>> > > > > Normal output from "if (BeanStalk::DEBUG) echo __METHOD__."\n";":
>> > > > > ----------------------------------------------------------------------------------------------------
>> > > > > BeanStalk::__call(use_tube)
>> > > > > BeanStalk::_do_my_server
>> > > > > BeanQueue::use_tube
>> > > > > BeanStalk::__call(peek_ready)
>> > > > > BeanStalk::_do_my_server
>>
>> > > > > Output from extra echo statements I added:
>> > > > > ----------------------------------------------------------------------------------------------------
>> > > > > peek_ready(): in_writeback=127.0.0.1:11300 <--first line in function
>> > > > > JOB=I was here <--output right after "$job = $this-
>> > > > >>read_message($data[1]);"
>> > > > > Writeback variable provided <--from inside of "if (!
>> > > > > is_null($in_writeback))"
>> > > > > Output from code pasted above:
>> > > > > ----------------------------------------------------------------------------------------------------
>> > > > > PEEK OPERATION OK
>> > > > > Next ready job=
>>
>> > > > > Not sure why this is happening, could be some type of
>> > > > > misunderstanding
>> > > > > on my end again. Any ideas as to what I may be doing wrong?
>>
>> > > > > Thanks for a great lib and the feedback by the way! Much
>> > > > > appreciated :)
>> > > > > --Avi
>> > > > > On Nov 7, 3:06 pm, "Tim Gunter" <[EMAIL PROTECTED]> wrote:
>> > > > >> Hi,
>>
>> > > > >> Thanks for pointing that out. I need to roll a new release soon,
>> > > > >> just
>> > > > >> havent had the spare time.
>>
>> > > > >> Regarding your questions, I think it's just a misunderstanding.
>>
>> > > > >> $beanstalk refers to a pool of servers, and is represented by the
>> > > > >> BeanStalk class. Calling peek_ready() on $beanstalk expects a server
>> > > > >> name and optional writeback.
>>
>> > > > >> $beanstalk then routes the peek_ready() call internally to an actual
>> > > > >> server instance, represented by a BeanQueue object. THIS
>> > > > >> peek_ready()
>> > > > >> does not need a servername parameter, and therefore only accepts the
>> > > > >> writeback.
>>
>> > > > >> Since array_shift modified the passed array, the top level
>> > > > >> peek_ready() only sends the writeback to the internal peek_ready, as
>> > > > >> the servername as already been shifted off.
>>
>> > > > >> Does that make sense?
>>
>> > > > >> - Tim
>>
>> > > > >> On Fri, Nov 7, 2008 at 2:53 PM, avip <[EMAIL PROTECTED]> wrote:
>>
>> > > > >> > Hi Tim,
>>
>> > > > >> > Not sure if you've already spotted this typo or not but in
>> > > > >> > function
>> > > > >> > __call(..) you have
>> > > > >> > "do_my_server" instead of "_do_my_server" (underscore in front).
>>
>> > > > >> > Also, I'm not sure if this is my misunderstanding ofPHP, the way
>> > > > >> > you
>> > > > >> > planned it, or an actual error but: peek_ready() accepts only a
>> > > > >> > write
>> > > > >> > back variable, not a server name; in the code you do an
>> > > > >> > array_shift()
>> > > > >> > in hopes of getting the server name. So, elsewhere, if someone
>> > > > >> > tried
>> > > > >> > doing just $beanstalk->peek_ready($nextJob); it would fail...
>>
>> > > > >> > On Oct 14, 5:53 pm, "FaceySpacey.com" <[EMAIL PROTECTED]> wrote:
>> > > > >> >> Hey everyone, I'm looking to hire someone to help me use
>> > > > >> >> beanstalkd
>> > > > >> >> (phplibrary) to speed up some heavy numerical calculations in a
>> > > > >> >> Mortgage Loan search application/site I've built...I'm still
>> > > > >> >> trying to
>> > > > >> >> find the right solution and was thinking map reduce type stuff
>> > > > >> >> (maybe
>> > > > >> >> Pig) might be the right way to go. Currently, we're all setup on
>> > > > >> >> Amazon EC2, and was also thinking Amazon SQS, but heard that this
>> > > > >> >> might be faster...So I still need to find the solution that
>> > > > >> >> offers the
>> > > > >> >> most value to implement in terms of cost and speed.
>>
>> > > > >> >> Anybody want to check out the scenario with me to
>>
>> ...
>>
>> read more ยป
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"beanstalk-talk" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/beanstalk-talk?hl=en
-~----------~----~----~----~------~----~------~--~---
