Hi Tim,
I was wondering if you could post some example consumer/worker code
using your class?
I've managed to daemonize a php worker script but inevitably there's a
polling while(1) loop somewhere attempting to reserve() a job. This
works fine when there are actually jobs on the queue, but for an empty
queue cpu-usage shoots up to 99%.
To test out whether or not this is a task or invocation issue, I tried
replacing the job reservation/worker code with a simple print
statement. The result was a well-behaved background worker. When I
switched back to trying to reserve and delete a job, cpu-usage shot
back up on an empty queue.
Which obviously leads me to suspect there's some other loop somewhere
in the class responsible for reading or blocking or...not sure
what...that's causing this.
If you could shed some light on this?
Thanks very much!
--Avi
On Nov 9, 3:56 pm, avip <[EMAIL PROTECTED]> wrote:
> Hi yet again!
>
> Scratch the last post. I spent more time experimenting with a telnet
> session and reading the protocol itself more carefully.
>
> For any other people who are completely new to beanstalkd and are like
> me - wanting to go from installing server to writing code - take note:
> the "use <tube>" command is *just* for producers. Attempting to
> "switch tubes" using "use <tube>" and then reserve will not work and
> actually block if the default tube is empty, not to mention confuse
> you like it did me. The command you're actually looking for when you
> wish to *consume* jobs is "watch <tube>"
>
> Very important-and much farther down the protocol doc-is the
> following:
>
> The "watch" command adds the named tube to the watch list for the
> current
> connection. A reserve command will take a job from any of the tubes in
> the
> watch list. For each new connection, the watch list initially consists
> of one
> tube, named "default".
>
> Take my code from above as example, seems like it should work because
> it's almost straight from the example. My first mistake was assuming
> that. I ran the example first, it worked fine without any error, but
> running it several times in a row after my own code failed, I noticed
> it was erratically timing out. When it wasn't timing out, it behaves
> the way I described in the post prior to this one, just silently
> returns with no actual action taking place.
>
> Both the example or the example code above will timeout not because of
> anyPHP-specific error but because doing "use_tube('test2')" doesn't
> actually *switch* tubes in order for you to consume from that job-
> queue - that is managed by the daemon for you provided you tell it you
> wish to *watch* that tube. Not the most obvious thing to realise upon
> initial inspection!
>
> So, in the above piece of code I pasted, reserve() is actually trying
> to ask for a job from the tube "default" (created automatically),
> since it is empty (because in a previous piece of code I, correctly,
> used "use_tube('test2')" to add jobs), the function just blocks...in
> other words, the would-be consumer waits for a job; however,PHPtimes
> out the script based on its own timer.
>
> If you wish to consume jobs usingPHPscripts in the background
> elsewhere without this time limit simply add "set_time_limit(0)" to
> the top of the relevantPHPscript to allow for infinite time - this
> will allow your consumer scripts to run in the background indefinitely
> until there is a job to be done.
>
> Anyway, using "watch" fixes the problem for me, I hope this post saves
> someone a few hours of confusion.
> Tim, if you could correct me on anything I got wrong... :)
>
> Thanks!
> --Avi
> On Nov 8, 12:30 pm, avip <[EMAIL PROTECTED]> wrote:
>
> > Hi again Tim,
>
> > Running into some basic problems: I have a script that loads the queue
> > with some jobs using put(), this works fine and is verified manually
> > via telnet. In a different script, I attempt to reserve a job using:
>
> > <code>
> > require('bin/Others/BeanStalk.class.php');
>
> > $beanstalk = BeanStalk::open(array('servers' =>
> > array('127.0.0.1:11300'),'select' => 'random wait'));
>
> > $beanstalk->use_tube('test2');
>
> > $job = $beanstalk->reserve();
> > echo "job=".print_r($job,true);
>
> > BeanStalk::delete($job);
> > </code>
>
> > Almost straight out of the example, only difference is that put() and
> > reserve() are spread out across two files. I've tried every select
> > mode as well with similar results.
>
> > BeanStalk::__call(use_tube)
> > BeanStalk::_do_my_server
> > BeanStalk::reserve
> > BeanQueue::reserve
> > Fatal error: Maximum execution time of 30 seconds exceeded in /bin/
> > Others/BeanStalk.class.phpon line 989
>
> > Since I added some echos to original, my line 989 is: $data =
> > @fread($this->socket,$in_buf_size);
>
> > Is this related to the problems from yesterday? Also, you mentioned
> > v1.1 will be released soon, could you be more specific? (weeks?
> > months?) I'm just trying to figure out whether or not I should try the
> > python or other libs for now.
>
> > Thanks!
> > --Avi
> > On Nov 7, 4:33 pm, avip <[EMAIL PROTECTED]> wrote:
>
> > > Ah! Of all the functions, I choose this one to test first with :)
>
> > > Will avoid the peek modes until next version, as suggested.
>
> > > Thanks again!
> > > --Avi
>
> > > On Nov 7, 4:15 pm, "Tim Gunter" <[EMAIL PROTECTED]> wrote:
>
> > > > Hey,
>
> > > > Yeah this is an issue with beanstalk clients like mine, and beanstalkd
> > > > 1.0. I was trying to implement a polling check that could pull the
> > > > next ready job from a pool, but peek() didn't turn out to work as I
> > > > initially expected.
>
> > > > Avoid the peek modes for now. Stick to 'sequential wait' and 'random
> > > > wait' select values.
>
> > > > I will release v1.1 of my client shortly, and then the peek modes will
> > > > work again, as expected.
>
> > > > On Fri, Nov 7, 2008 at 4:11 PM, avip <[EMAIL PROTECTED]> wrote:
>
> > > > > Hi,
>
> > > > > Wow, that was a fast turn-around! I thought it would take days :)
> > > > > Thanks for both explanations, now that I read them (and after poking
> > > > > in the code for a while), it seems pretty "obviously necessary" to
> > > > > tell the function which server to perform the peek on given there's a
> > > > > pool.
>
> > > > > I've been trying some very basicPHPscripts to learn the internals, I
> > > > > have one script that puts a job onto the server (I manually verify
> > > > > it's there with a telnet session). Then the following:
> > > > > <code>
> > > > > require('bin/Others/BeanStalk.class.php');
> > > > > $beanstalk = BeanStalk::open(array('servers' =>
> > > > > array('127.0.0.1:11300'),'select' => 'random peek'));
>
> > > > > $beanstalk->use_tube('test');
>
> > > > > $nextJob = '';
> > > > > $ret = $beanstalk->peek_ready('127.0.0.1:11300',$nextJob);
>
> > > > > if ($ret === 1)
> > > > > {
> > > > > echo 'PEEK OPERATION OK<br/>';
> > > > > echo 'Next ready job='.$nextJob.'<br/>';
> > > > > }
> > > > > else {
> > > > > echo 'PEEK OPERATION FAILED<br/>';
> > > > > }
> > > > > </code>
>
> > > > > I also turned debugging on and some simple debugging echos to
> > > > > peek_ready(). Result:
>
> > > > > Normal output from "if (BeanStalk::DEBUG) echo __METHOD__."\n";":
> > > > > ----------------------------------------------------------------------------------------------------
> > > > > BeanStalk::__call(use_tube)
> > > > > BeanStalk::_do_my_server
> > > > > BeanQueue::use_tube
> > > > > BeanStalk::__call(peek_ready)
> > > > > BeanStalk::_do_my_server
>
> > > > > Output from extra echo statements I added:
> > > > > ----------------------------------------------------------------------------------------------------
> > > > > peek_ready(): in_writeback=127.0.0.1:11300 <--first line in function
> > > > > JOB=I was here <--output right after "$job = $this-
> > > > >>read_message($data[1]);"
> > > > > Writeback variable provided <--from inside of "if (!
> > > > > is_null($in_writeback))"
> > > > > Output from code pasted above:
> > > > > ----------------------------------------------------------------------------------------------------
> > > > > PEEK OPERATION OK
> > > > > Next ready job=
>
> > > > > Not sure why this is happening, could be some type of misunderstanding
> > > > > on my end again. Any ideas as to what I may be doing wrong?
>
> > > > > Thanks for a great lib and the feedback by the way! Much
> > > > > appreciated :)
> > > > > --Avi
> > > > > On Nov 7, 3:06 pm, "Tim Gunter" <[EMAIL PROTECTED]> wrote:
> > > > >> Hi,
>
> > > > >> Thanks for pointing that out. I need to roll a new release soon, just
> > > > >> havent had the spare time.
>
> > > > >> Regarding your questions, I think it's just a misunderstanding.
>
> > > > >> $beanstalk refers to a pool of servers, and is represented by the
> > > > >> BeanStalk class. Calling peek_ready() on $beanstalk expects a server
> > > > >> name and optional writeback.
>
> > > > >> $beanstalk then routes the peek_ready() call internally to an actual
> > > > >> server instance, represented by a BeanQueue object. THIS peek_ready()
> > > > >> does not need a servername parameter, and therefore only accepts the
> > > > >> writeback.
>
> > > > >> Since array_shift modified the passed array, the top level
> > > > >> peek_ready() only sends the writeback to the internal peek_ready, as
> > > > >> the servername as already been shifted off.
>
> > > > >> Does that make sense?
>
> > > > >> - Tim
>
> > > > >> On Fri, Nov 7, 2008 at 2:53 PM, avip <[EMAIL PROTECTED]> wrote:
>
> > > > >> > Hi Tim,
>
> > > > >> > Not sure if you've already spotted this typo or not but in function
> > > > >> > __call(..) you have
> > > > >> > "do_my_server" instead of "_do_my_server" (underscore in front).
>
> > > > >> > Also, I'm not sure if this is my misunderstanding ofPHP, the way
> > > > >> > you
> > > > >> > planned it, or an actual error but: peek_ready() accepts only a
> > > > >> > write
> > > > >> > back variable, not a server name; in the code you do an
> > > > >> > array_shift()
> > > > >> > in hopes of getting the server name. So, elsewhere, if someone
> > > > >> > tried
> > > > >> > doing just $beanstalk->peek_ready($nextJob); it would fail...
>
> > > > >> > On Oct 14, 5:53 pm, "FaceySpacey.com" <[EMAIL PROTECTED]> wrote:
> > > > >> >> Hey everyone, I'm looking to hire someone to help me use
> > > > >> >> beanstalkd
> > > > >> >> (phplibrary) to speed up some heavy numerical calculations in a
> > > > >> >> Mortgage Loan search application/site I've built...I'm still
> > > > >> >> trying to
> > > > >> >> find the right solution and was thinking map reduce type stuff
> > > > >> >> (maybe
> > > > >> >> Pig) might be the right way to go. Currently, we're all setup on
> > > > >> >> Amazon EC2, and was also thinking Amazon SQS, but heard that this
> > > > >> >> might be faster...So I still need to find the solution that
> > > > >> >> offers the
> > > > >> >> most value to implement in terms of cost and speed.
>
> > > > >> >> Anybody want to check out the scenario with me to
>
> ...
>
> read more »
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"beanstalk-talk" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/beanstalk-talk?hl=en
-~----------~----~----~----~------~----~------~--~---
