RE: [PHP] Forking a search - pcntl
On Wed, April 19, 2006 3:17 am, James Nunnerley wrote: > Thanks for everyone's replies - but I'm a little uncertain as to what > the > reasons for not running the pcntl functions under the web browser are > - is > it down to security? I could be wrong, but I believe the correct response is: No, it's down to various threads tromping all over each other corrupting RAM and crashing your machine in about 5 seconds under any kind of load. Other than that, though, you're all set. :-) -- Like Music? http://l-i-e.com/artists.htm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Forking a search - pcntl
James Nunnerley wrote:
Thanks for everyone's replies - but I'm a little uncertain as to what the
reasons for not running the pcntl functions under the web browser are - is
it down to security?
http://php.net/pcntl
Process Control should not be enabled within a webserver environment and
unexpected
results may happen if any Process Control functions are used within a webserver
environment.
in short you don't want to be forking your apache processes from within
php. you don't need to understand why so much as trust that there is a
good reason for not doing it - the more you delve in the underbelly of
process control/forking/etc the more you'll understand why the above advice
is sound :-)
cheers
-Original Message-
From: Richard Lynch [mailto:[EMAIL PROTECTED]
Sent: 18 April 2006 22:52
To: James Nunnerley
Cc: php-general@lists.php.net
Subject: Re: [PHP] Forking a search - pcntl
On Tue, April 18, 2006 5:21 am, James Nunnerley wrote:
What we'd like to do is run the actual search query in the background
(i.e.
fork) while the viewable page shows a nice scrollie banner etc!
fork is not the only solution for this, thank [insert deity here]
Due to various problems with the server we are using (Zeus) we can't
just
run an exec(). I've had a look around, and it would seem that pcntl
maybe
the way forward.
NO WAY!
pcntl should NOT be run in a web-hosted environment, according to the
docs.
I don't think Zeus' threading model is going to make it "okay" to use.
Does anyone have an example working script, or indeed a decent
tutorial on
how to use the functionality? The php manual has limited information
on
using the functions, and I just can't get my head around how it's
meant to
work?!!!
pctnl should only be used from CLI/CGI
The examples should be sufficient for that, but you don't really care,
as it's not useful for what you want.
Here is what *I* would recommend:
Create a new table of "searches":
create table search(
search_id int(11) auto_increment unique not null primary key,
status enum{0, 1, 2} default 0,
search_pid int(11) default null,
inputs text,
results text
);
0 = new search
1 = in progress
2 = complete
Now, when somebody wants you to do a search, just insert a record:
$query = "insert into search(inputs) values('$CLEAN[inputs]')";
where $CLEAN is your sanitized validated $_REQUEST input, of course.
$id = mysql_insert_id();
Your scrolling banner can than have a META Refresh of a few
seconds/minutes/whatever, and embed the $id into the URL for that.
See: http://php.net/mysql_insert_id
Then, of course, you need something to actually PERFORM the searches,
which is where a nice cron job comes in.
That cron job can start a new search task which will do this:
[in psuedo-code]
$pid = getmypid(); // or something like that:
UPDATE search set status = 1, search_pid = $pid where status = 0 LIMIT 1
SELECT id, inputs from search where search_pid = $pid
$id = $row['id'];
update search set status = 1 where id = $id
//do the search
//when done:
update search set status = 2, results = '$search_results' where id = $id
Doing it this way means you could even run several processes at once,
each working on a different search.
Note that the UPDATE marks the record as "in progress" and ties it to
the process running, so that there is NO race condition.
If MySQL does not support LIMIT 1 in an UPDATE, which I'm pretty sure
it does, but not 100% certain, then you'd have to just update all the
inputs available, and have the thread handle each search that it
"took" in the UPDATE.
You could still have an army of search processes, though, as new
search inputs are coming in all the time, and each search process
would handle however many were on the "To Do" (status 0) stack when
they started.
This is very scalable, and you could even buy more computers and throw
them at it, with just one database, provided you tagged the process_id
with a machine_id of some kind as well, to avoid duplciate process IDs
on 2 computers. I'll leave that part as an exercise.
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] Forking a search - pcntl
Thanks for everyone's replies - but I'm a little uncertain as to what the
reasons for not running the pcntl functions under the web browser are - is
it down to security?
cheers
-Original Message-
From: Richard Lynch [mailto:[EMAIL PROTECTED]
Sent: 18 April 2006 22:52
To: James Nunnerley
Cc: php-general@lists.php.net
Subject: Re: [PHP] Forking a search - pcntl
On Tue, April 18, 2006 5:21 am, James Nunnerley wrote:
> What we'd like to do is run the actual search query in the background
> (i.e.
> fork) while the viewable page shows a nice scrollie banner etc!
fork is not the only solution for this, thank [insert deity here]
> Due to various problems with the server we are using (Zeus) we can't
> just
> run an exec(). I've had a look around, and it would seem that pcntl
> maybe
> the way forward.
NO WAY!
pcntl should NOT be run in a web-hosted environment, according to the
docs.
I don't think Zeus' threading model is going to make it "okay" to use.
> Does anyone have an example working script, or indeed a decent
> tutorial on
> how to use the functionality? The php manual has limited information
> on
> using the functions, and I just can't get my head around how it's
> meant to
> work?!!!
pctnl should only be used from CLI/CGI
The examples should be sufficient for that, but you don't really care,
as it's not useful for what you want.
Here is what *I* would recommend:
Create a new table of "searches":
create table search(
search_id int(11) auto_increment unique not null primary key,
status enum{0, 1, 2} default 0,
search_pid int(11) default null,
inputs text,
results text
);
0 = new search
1 = in progress
2 = complete
Now, when somebody wants you to do a search, just insert a record:
$query = "insert into search(inputs) values('$CLEAN[inputs]')";
where $CLEAN is your sanitized validated $_REQUEST input, of course.
$id = mysql_insert_id();
Your scrolling banner can than have a META Refresh of a few
seconds/minutes/whatever, and embed the $id into the URL for that.
See: http://php.net/mysql_insert_id
Then, of course, you need something to actually PERFORM the searches,
which is where a nice cron job comes in.
That cron job can start a new search task which will do this:
[in psuedo-code]
$pid = getmypid(); // or something like that:
UPDATE search set status = 1, search_pid = $pid where status = 0 LIMIT 1
SELECT id, inputs from search where search_pid = $pid
$id = $row['id'];
update search set status = 1 where id = $id
//do the search
//when done:
update search set status = 2, results = '$search_results' where id = $id
Doing it this way means you could even run several processes at once,
each working on a different search.
Note that the UPDATE marks the record as "in progress" and ties it to
the process running, so that there is NO race condition.
If MySQL does not support LIMIT 1 in an UPDATE, which I'm pretty sure
it does, but not 100% certain, then you'd have to just update all the
inputs available, and have the thread handle each search that it
"took" in the UPDATE.
You could still have an army of search processes, though, as new
search inputs are coming in all the time, and each search process
would handle however many were on the "To Do" (status 0) stack when
they started.
This is very scalable, and you could even buy more computers and throw
them at it, with just one database, provided you tagged the process_id
with a machine_id of some kind as well, to avoid duplciate process IDs
on 2 computers. I'll leave that part as an exercise.
--
Like Music?
http://l-i-e.com/artists.htm
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Forking a search - pcntl
On Tue, April 18, 2006 5:21 am, James Nunnerley wrote:
> What we'd like to do is run the actual search query in the background
> (i.e.
> fork) while the viewable page shows a nice scrollie banner etc!
fork is not the only solution for this, thank [insert deity here]
> Due to various problems with the server we are using (Zeus) we can't
> just
> run an exec(). I've had a look around, and it would seem that pcntl
> maybe
> the way forward.
NO WAY!
pcntl should NOT be run in a web-hosted environment, according to the
docs.
I don't think Zeus' threading model is going to make it "okay" to use.
> Does anyone have an example working script, or indeed a decent
> tutorial on
> how to use the functionality? The php manual has limited information
> on
> using the functions, and I just can't get my head around how it's
> meant to
> work?!!!
pctnl should only be used from CLI/CGI
The examples should be sufficient for that, but you don't really care,
as it's not useful for what you want.
Here is what *I* would recommend:
Create a new table of "searches":
create table search(
search_id int(11) auto_increment unique not null primary key,
status enum{0, 1, 2} default 0,
search_pid int(11) default null,
inputs text,
results text
);
0 = new search
1 = in progress
2 = complete
Now, when somebody wants you to do a search, just insert a record:
$query = "insert into search(inputs) values('$CLEAN[inputs]')";
where $CLEAN is your sanitized validated $_REQUEST input, of course.
$id = mysql_insert_id();
Your scrolling banner can than have a META Refresh of a few
seconds/minutes/whatever, and embed the $id into the URL for that.
See: http://php.net/mysql_insert_id
Then, of course, you need something to actually PERFORM the searches,
which is where a nice cron job comes in.
That cron job can start a new search task which will do this:
[in psuedo-code]
$pid = getmypid(); // or something like that:
UPDATE search set status = 1, search_pid = $pid where status = 0 LIMIT 1
SELECT id, inputs from search where search_pid = $pid
$id = $row['id'];
update search set status = 1 where id = $id
//do the search
//when done:
update search set status = 2, results = '$search_results' where id = $id
Doing it this way means you could even run several processes at once,
each working on a different search.
Note that the UPDATE marks the record as "in progress" and ties it to
the process running, so that there is NO race condition.
If MySQL does not support LIMIT 1 in an UPDATE, which I'm pretty sure
it does, but not 100% certain, then you'd have to just update all the
inputs available, and have the thread handle each search that it
"took" in the UPDATE.
You could still have an army of search processes, though, as new
search inputs are coming in all the time, and each search process
would handle however many were on the "To Do" (status 0) stack when
they started.
This is very scalable, and you could even buy more computers and throw
them at it, with just one database, provided you tagged the process_id
with a machine_id of some kind as well, to avoid duplciate process IDs
on 2 computers. I'll leave that part as an exercise.
--
Like Music?
http://l-i-e.com/artists.htm
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Forking a search - pcntl
On 4/18/06, Stut <[EMAIL PROTECTED]> wrote: > James Nunnerley wrote: > > >I'm creating a search function as part of our webmail frontend, however I'm > >having big problems with timeout etc. > > > >What we'd like to do is run the actual search query in the background (i.e. > >fork) while the viewable page shows a nice scrollie banner etc! > > > >Due to various problems with the server we are using (Zeus) we can't just > >run an exec(). I've had a look around, and it would seem that pcntl maybe > >the way forward. > > > >Does anyone have an example working script, or indeed a decent tutorial on > >how to use the functionality? The php manual has limited information on > >using the functions, and I just can't get my head around how it's meant to > >work?!!! > > > > > IMHO forking in response to a web request is a very very very bad idea(tm). On top of that advice, see the big notice on the php manual page: http://www.php.net/pcntl Process Control should not be enabled within a webserver environment and unexpected results may happen if any Process Control functions are used within a webserver environment. -- Postgresql & php tutorials http://www.designmagick.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Forking a search - pcntl
James Nunnerley wrote: I'm creating a search function as part of our webmail frontend, however I'm having big problems with timeout etc. What we'd like to do is run the actual search query in the background (i.e. fork) while the viewable page shows a nice scrollie banner etc! Due to various problems with the server we are using (Zeus) we can't just run an exec(). I've had a look around, and it would seem that pcntl maybe the way forward. Does anyone have an example working script, or indeed a decent tutorial on how to use the functionality? The php manual has limited information on using the functions, and I just can't get my head around how it's meant to work?!!! IMHO forking in response to a web request is a very very very bad idea(tm). I had a similar problem a while back. I solved it by having a CLI process that watches a search queue table in the DB. When it sees an entry it kicks off another script to do the actual search. Meanwhile the frontend is using AJAX to display a progress bar using a percentage written to the search queue record by the searching process. Works very well for that site, although it has to be said it is not a particularly busy site so scalability has never been tested. Using this method it's possible to restrict each session to a single search if needed since you can set a cancel flag in the search queue record which gets checked by the searching process. This particular site ties each search to the php session id. HTH. -Stut -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Forking a search - pcntl
I'm creating a search function as part of our webmail frontend, however I'm having big problems with timeout etc. What we'd like to do is run the actual search query in the background (i.e. fork) while the viewable page shows a nice scrollie banner etc! Due to various problems with the server we are using (Zeus) we can't just run an exec(). I've had a look around, and it would seem that pcntl maybe the way forward. Does anyone have an example working script, or indeed a decent tutorial on how to use the functionality? The php manual has limited information on using the functions, and I just can't get my head around how it's meant to work?!!! Cheers Nunners
