Re: [PHP] Command line PHP
On 7 January 2011 16:55, la...@garfieldtech.com la...@garfieldtech.com wrote: Hi folks. I have a project coming up that will involve writing a non-trivial command line PHP application. Most of it will be nice and abstracted and standalone and all of that jazz, but it will need to do command line interation. I'm not sure yet if it will be interactive or if I just need to parse lots of command line switches. Has anyone used a CLI-handling library they like? I recall briefly using the PEAR CLI library many many years ago and disliking it because it was only barely a step above the raw PHP-CLI SAPI, and still required lots of if-else branching in my code. I don't know if there's anything better since then, however. I prefer clean OO to procedural, but can work with procedural if needs be. The fewer dependencies it has the better as well. Any recommendations? (Open source, GPLv2-compatible required.) --Larry Garfield Hello Larry, Sorry for being late to the game. I use PHP on Windows and I use PHP CLI for nearly everything I do that isn't web based. Occasionally I'll use a .BAT file, but invariably, I need logging and reporting, etc. I get all of that from pre-existing PHP classes, so a no brainer in that regard. I co-maintain the PEAR Console_CommandLine package [1]. This allows quite complex command line argument setups. At the bottom of this post, I've included the help screen from an app. Long/short options all catered for. commands can have their own options/arguments too (so script command --help would show that command's help). If you are on windows, take a look at PHP command line usage on windows [2] to make things a lot easier for yourself. The app shows me differences in source files between different servers (live, dev, test, etc). To get values during run time from the console, I use (in the most basic of example), fgets(STDIN); As PHP on windows requires an [ENTER] key to be pressed to pass the typed string to the code (even for fgetc(STDIN) ), then this may not be what you want. Try this ... php -n -r echo fgetc(STDIN); If you can then press 1 letter and see 2 copies of it and the program quits, then that's it - in a nutshell. On windows, I have to press [ENTER]. If I type abc[ENTER], then only the a is returned and the bc is buffered somewhere. php -n -r echo fgetc(STDIN), fgetc(STDIN); shows the buffering. I also use Jason Hood's ANSICON [3] (full source available), which allows me to output coloured text in PHP using a simple home-brew class (ANSI::Write($text, $foreground = null, $background = null)). If you build phpdoc using PhD and enable the coloured output, all looks great with ANSICON. ANSICON supports quite a few of the non colour related ANSI code sequences (cursor positing, inserting/deleting lines/characters/etc.), so maybe doing things like drawing windows, boxes, menus, etc. could all be done using just ANSICON. It would be a LOT of work for probably not a lot of users. As far as I know, there are no libraries for PHP on windows allowing you to do this sort of thing from the command line. I know of curses (ncurses ?), so if you are on something other than windows, these are an option for colour processing. As for pushing the envelope ... I run multiple PHP scripts as Windows Services using the pecl extension win32service [4], implementing multiple threads using ... $WshShell = new COM(WScript.Shell); $WshShell-Run(...); and have shared memory via WinCache [5]. All command line scripts in essence. Why? Well. I know PHP quite well. I understand the 2 extensions very well. I can re-use all my existing code/frameworks. And I can fit these service into the environment in such a way that the system admins/sysops only see these tasks as normal windows services. If they need to reboot a server bank, the scripts shutdown automatically, appropriately and correctly and restart after the reboot. I'm just working on adding dependency to the services so that if (for example) they take down the mail server for some reason, any of the PHP services requiring mail will also close down correctly. Just as if you had a real service. And, once I'd found the solution and documented it, using PHP scripts as command line filters, allows you to do some pretty complex argument chaining. collect --source internet | filter email | updateDB --server x --user y --password z | generateReports --smtp random for example, where collect, filter, updateDB and generateReports are all PHP scripts which acccept command line parameters as well as use stdin to read data in realtime. Just like | more would. Richard. [1] http://pear.php.net/package/Console_CommandLine [2] http://docs.php.net/manual/en/install.windows.commandline.php [3] http://adoxa.110mb.com/ansicon/index.html [4] http://pecl.php.net/package/win32service [5] http://pecl.php.net/package/wincache VD3 : Source code synchronisation. View and synchronize source code modules. Usage: VD3 [options] VD3
Re: [PHP] Command line PHP
On 11-01-11 11:27 AM, Richard Quadling wrote: As PHP on windows requires an [ENTER] key to be pressed to pass the typed string to the code (even for fgetc(STDIN) ), then this may not be what you want. Have you tried setting the stream to non-blocking to prevent the need for the return key? Cheers, Rob. -- E-Mail Disclaimer: Information contained in this message and any attached documents is considered confidential and legally protected. This message is intended solely for the addressee(s). Disclosure, copying, and distribution are prohibited unless authorized. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On 11 January 2011 16:43, Robert Cummings rob...@interjinn.com wrote: On 11-01-11 11:27 AM, Richard Quadling wrote: As PHP on windows requires an [ENTER] key to be pressed to pass the typed string to the code (even for fgetc(STDIN) ), then this may not be what you want. Have you tried setting the stream to non-blocking to prevent the need for the return key? Cheers, Rob. -- E-Mail Disclaimer: Information contained in this message and any attached documents is considered confidential and legally protected. This message is intended solely for the addressee(s). Disclosure, copying, and distribution are prohibited unless authorized. PHP on windows doesn't support non-blocking streams. It's not so much PHP's fault, but the underlying code from MS that PHP uses (AFAICT). Setting a stream to non blocking and then immediately asking if it is blocking shows that it is still blocking. http://bugs.php.net/bug.php?id=34972 is the best example of this issue. Especially the last comment ... what makes you think that anything has been done about this feature request?. I'm not holding my breath for a fix. -- Richard Quadling Twitter : EE : Zend @RQuadling : e-e.com/M_248814.html : bit.ly/9O8vFY -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On 11 January 2011 16:43, Robert Cummings rob...@interjinn.com wrote: On 11-01-11 11:27 AM, Richard Quadling wrote: As PHP on windows requires an [ENTER] key to be pressed to pass the typed string to the code (even for fgetc(STDIN) ), then this may not be what you want. Have you tried setting the stream to non-blocking to prevent the need for the return key? Cheers, Rob. -- E-Mail Disclaimer: Information contained in this message and any attached documents is considered confidential and legally protected. This message is intended solely for the addressee(s). Disclosure, copying, and distribution are prohibited unless authorized. And this was my bug report [1] nearly 2 years ago on this. Richard. [1] http://bugs.php.net/bug.php?id=47918 -- Richard Quadling Twitter : EE : Zend @RQuadling : e-e.com/M_248814.html : bit.ly/9O8vFY -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On 11-01-11 11:55 AM, Richard Quadling wrote: On 11 January 2011 16:43, Robert Cummingsrob...@interjinn.com wrote: On 11-01-11 11:27 AM, Richard Quadling wrote: As PHP on windows requires an [ENTER] key to be pressed to pass the typed string to the code (even for fgetc(STDIN) ), then this may not be what you want. Have you tried setting the stream to non-blocking to prevent the need for the return key? Cheers, Rob. -- E-Mail Disclaimer: Information contained in this message and any attached documents is considered confidential and legally protected. This message is intended solely for the addressee(s). Disclosure, copying, and distribution are prohibited unless authorized. PHP on windows doesn't support non-blocking streams. It's not so much PHP's fault, but the underlying code from MS that PHP uses (AFAICT). Setting a stream to non blocking and then immediately asking if it is blocking shows that it is still blocking. http://bugs.php.net/bug.php?id=34972 is the best example of this issue. Especially the last comment ... what makes you think that anything has been done about this feature request?. I'm not holding my breath for a fix. Doh, that sucks! Cheers, Rob. -- E-Mail Disclaimer: Information contained in this message and any attached documents is considered confidential and legally protected. This message is intended solely for the addressee(s). Disclosure, copying, and distribution are prohibited unless authorized. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
tedd wrote: At 1:54 PM -0500 1/7/11, Joshua Kehn wrote: Why should someone stop learning ever? Because my head fills up. I have to wait until I forget something before I can learn something new. The up-side is that I'm learning something new almost every day now. Cheers, tedd lol.. I just play too many video games.. it frees up all sorts of space. ;-) Donovan -- D Brooke -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On 11-01-11 12:14 PM, Donovan Brooke wrote: tedd wrote: At 1:54 PM -0500 1/7/11, Joshua Kehn wrote: Why should someone stop learning ever? Because my head fills up. I have to wait until I forget something before I can learn something new. The up-side is that I'm learning something new almost every day now. Cheers, tedd lol.. I just play too many video games.. it frees up all sorts of space. ;-) Does anyone remember the episode of Married With Children where they discover that they can feed facts into Kelly's brain but there's a limit after which if something goes in something comes out? Then there's the big tv trivia challenge at the end and she's stuffed full. The host tells her something and you hear a *bing* that something has gone out... which happens to be the answer to the final question :) After remembering this, it's obvious I've got some junk banging around up top that doesn't need keeping :) Cheers, Rob. -- E-Mail Disclaimer: Information contained in this message and any attached documents is considered confidential and legally protected. This message is intended solely for the addressee(s). Disclosure, copying, and distribution are prohibited unless authorized. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
At 11:14 AM -0600 1/11/11, Donovan Brooke wrote: tedd wrote: At 1:54 PM -0500 1/7/11, Joshua Kehn wrote: Why should someone stop learning ever? Because my head fills up. I have to wait until I forget something before I can learn something new. The up-side is that I'm learning something new almost every day now. Cheers, tedd lol.. I just play too many video games.. it frees up all sorts of space. ;-) Donovan My down time is playing XBOX Black Ops. It allows my mind to focus on things that don't matter, much like a vacation, that's frees space My gamer tag is special tedd -- for which many will understand. Cheers, tedd -- --- http://sperling.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On 11 January 2011 17:53, Robert Cummings rob...@interjinn.com wrote: On 11-01-11 12:14 PM, Donovan Brooke wrote: tedd wrote: At 1:54 PM -0500 1/7/11, Joshua Kehn wrote: Why should someone stop learning ever? Because my head fills up. I have to wait until I forget something before I can learn something new. The up-side is that I'm learning something new almost every day now. Cheers, tedd lol.. I just play too many video games.. it frees up all sorts of space. ;-) Does anyone remember the episode of Married With Children where they discover that they can feed facts into Kelly's brain but there's a limit after which if something goes in something comes out? Then there's the big tv trivia challenge at the end and she's stuffed full. The host tells her something and you hear a *bing* that something has gone out... which happens to be the answer to the final question :) After remembering this, it's obvious I've got some junk banging around up top that doesn't need keeping :) Cheers, Rob. -- E-Mail Disclaimer: Information contained in this message and any attached documents is considered confidential and legally protected. This message is intended solely for the addressee(s). Disclosure, copying, and distribution are prohibited unless authorized. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php For some unknown reason, my pc just went bing bong. I'm guessing it has forgotten (deleted) something useful for me. Thanks! -- Richard Quadling Twitter : EE : Zend @RQuadling : e-e.com/M_248814.html : bit.ly/9O8vFY -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Tue, Jan 11, 2011 at 10:12 AM, tedd tedd.sperl...@gmail.com wrote: My down time is playing XBOX Black Ops. It allows my mind to focus on things that don't matter, much like a vacation, that's frees space For me that's Left 4 Dead 2 as Captain Cujo. I think it's beneficial to cultivate skills in something that totally doesn't matter. Of course, in the event of an *actual* zombie apocalypse I'll be well prepared. :) David
Re: [PHP] Command line PHP
On Jan 7, 2011, at 8:50 PM, David Hutto wrote: On Fri, Jan 7, 2011 at 8:44 PM, TR Shaw ts...@oitc.com wrote: On Jan 7, 2011, at 8:36 PM, David Hutto wrote: I'm with some of the others above on using Python. Writing a command line app is about as simple as: import subprocess word = 'hello' self.espeak = subprocess.Popen(['espeak', word], stdout = subprocess.PIPE).communicate()[0] I think of PHP as more browser, than desktop app/webapp. And in php the above is just: echo hello; Which is simpler? The answer is the one that you are most comfortable with. I expect this might have been meant to go to the list, but I'll reply directly back. With the espeak installed. This calls a command line from your python script, and speaks the word, not just display. I understand this might be direct with certain abilities to display and speak the info. In python2.x it would be print 'hello', or print hello, notice no ';'. How would you call a command line(from terminal) script from within a php desktop app? That was what the above was meant to demonstrate in python, not print it out to page, or terminal. But I'm guessing it might not be that different. Sorry, David I should not reply quickly when I have a cold but again its all pretty similar. Here it is on unix/linux. PHP: $word = hello; exec(say \$word\); Bash: word=hello say $word In all the languages its hust: 1) set a variable 2) call an external program with the variable as an argument after that its all about language specific syntax. Tom -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
Like i said, my introduction to php is browser,and desktop app is python, but I will try php in the command line out. 1) set a variable 2) call an external program with the variable as an argument this is something I recognize very well:) -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
David Hutto wrote: I'm with some of the others above on using Python. Writing a command line app is about as simple as: snip I think of PHP as more browser, than desktop app/webapp. The point I was trying to make was one where there are two paths to doing the same job ... Example Thumbnail generation ... These are either generated as part of the upload page, or processed as a background task. There is no sense at all in writing the same function in two different languages. We need the PHP version for the on-line processing, so why would you then NOT use it also from the command line for the background processing as part of a cron job? Processing email traffic is another area where the same code is used both on and off line so using the same language again makes sense. OK new code could be done in a new language, but where php is already supported it just makes sense to retain it? Unless of cause php is just wraping some other system that is actually doing the work, such as browsing an hg repository, when things start getting even more unmanageable :( -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Sat, Jan 8, 2011 at 07:25, TR Shaw ts...@oitc.com wrote:
Sorry, David I should not reply quickly when I have a cold but again its all
pretty similar. Here it is on unix/linux.
PHP:
$word = hello;
exec(say \$word\);
Bash:
word=hello
say $word
?php txt2wav('Hello','hello.wav'); ?
--
/Daniel P. Brown
Network Infrastructure Manager
Documentation, Webmaster Teams
http://www.php.net/
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Sat, Jan 8, 2011 at 00:23, Larry Garfield la...@garfieldtech.com wrote: On Friday, January 07, 2011 9:34:42 pm David Hutto wrote: Which yielded this as the first result: http://php.net/manual/en/features.commandline.php As noted in my original email, I find the native SAPI clunky and difficult to work with. Hence I was looking for something more usable and robust built on top of it that I could leverage rather than rolling my own one-off. Of course, I got lost somewhere in the language holy wars (dear god, people...) so I'll probably just take the roll my own approach. Larry, I have a malicious process detector I wrote a while back. Haven't maintained or even looked at the code in a while, and it was only intended for my own usage (so it's not going to be in the best of shape), but you're welcome to it if you want something to jump-start your work. Just let me know. -- /Daniel P. Brown Network Infrastructure Manager Documentation, Webmaster Teams http://www.php.net/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Command line PHP
Hi folks. I have a project coming up that will involve writing a non-trivial command line PHP application. Most of it will be nice and abstracted and standalone and all of that jazz, but it will need to do command line interation. I'm not sure yet if it will be interactive or if I just need to parse lots of command line switches. Has anyone used a CLI-handling library they like? I recall briefly using the PEAR CLI library many many years ago and disliking it because it was only barely a step above the raw PHP-CLI SAPI, and still required lots of if-else branching in my code. I don't know if there's anything better since then, however. I prefer clean OO to procedural, but can work with procedural if needs be. The fewer dependencies it has the better as well. Any recommendations? (Open source, GPLv2-compatible required.) --Larry Garfield -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Jan 7, 2011, at 11:55 AM, la...@garfieldtech.com wrote: Hi folks. I have a project coming up that will involve writing a non-trivial command line PHP application. Most of it will be nice and abstracted and standalone and all of that jazz, but it will need to do command line interation. I'm not sure yet if it will be interactive or if I just need to parse lots of command line switches. Has anyone used a CLI-handling library they like? I recall briefly using the PEAR CLI library many many years ago and disliking it because it was only barely a step above the raw PHP-CLI SAPI, and still required lots of if-else branching in my code. I don't know if there's anything better since then, however. I prefer clean OO to procedural, but can work with procedural if needs be. The fewer dependencies it has the better as well. Any recommendations? (Open source, GPLv2-compatible required.) --Larry Garfield Larry- Why are you writing a command line application in PHP? I would think that is starting off on a very wrong foot. Can you explain the requirements / use cases a bit more? Regards, -Josh Joshua Kehn | josh.k...@gmail.com http://joshuakehn.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Fri, Jan 7, 2011 at 11:55, la...@garfieldtech.com la...@garfieldtech.com wrote: Hi folks. I have a project coming up that will involve writing a non-trivial command line PHP application. Most of it will be nice and abstracted and standalone and all of that jazz, but it will need to do command line interation. I'm not sure yet if it will be interactive or if I just need to parse lots of command line switches. Has anyone used a CLI-handling library they like? I recall briefly using the PEAR CLI library many many years ago and disliking it because it was only barely a step above the raw PHP-CLI SAPI, and still required lots of if-else branching in my code. I don't know if there's anything better since then, however. I prefer clean OO to procedural, but can work with procedural if needs be. The fewer dependencies it has the better as well. Any recommendations? I've never used any external libraries, I always built my interface to suit the project at hand. Are you looking for it to run as its own pseudo shell, or just a basic batch script or daemon-style application? -- /Daniel P. Brown Network Infrastructure Manager Documentation, Webmaster Teams http://www.php.net/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
(sorry for top post, still not worked out how not to on phone) Can you not just code it like you normally would any app that doesn't use a framework? I've been writing some cli data importers at work. Basic really, with only classes used that I need, no framework needed (its very simple) and all the output is just echo statements with new lines rather than br tags. getopt() is your friend for command line arguments, but not all windows systems will support it, although Linux is fine. Thanks, Ash http://www.ashleysheridan.co.uk - Reply message - From: la...@garfieldtech.com la...@garfieldtech.com Date: Fri, Jan 7, 2011 16:55 Subject: [PHP] Command line PHP To: php-general@lists.php.net Hi folks. I have a project coming up that will involve writing a non-trivial command line PHP application. Most of it will be nice and abstracted and standalone and all of that jazz, but it will need to do command line interation. I'm not sure yet if it will be interactive or if I just need to parse lots of command line switches. Has anyone used a CLI-handling library they like? I recall briefly using the PEAR CLI library many many years ago and disliking it because it was only barely a step above the raw PHP-CLI SAPI, and still required lots of if-else branching in my code. I don't know if there's anything better since then, however. I prefer clean OO to procedural, but can work with procedural if needs be. The fewer dependencies it has the better as well. Any recommendations? (Open source, GPLv2-compatible required.) --Larry Garfield -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Jan 7, 2011, at 12:12 PM, Daniel Brown wrote: On Fri, Jan 7, 2011 at 12:01, Joshua Kehn josh.k...@gmail.com wrote: Why are you writing a command line application in PHP? I would think that is starting off on a very wrong foot. I would not be exaggerating to say that I've written over a thousand command line applications in PHP since about 2003. For a variety of reasons and uses (more and more because my wife needs something done or I'm adding more home automation stuff) I write no less than ten each week. For the majority, it's because it's easier than writing it in C and more portable than Bash --- I can write a PHP script and put it on a dozen machines at home and in the offices and it'll work the same (when written properly). No need to worry about OS or architecture. -- /Daniel P. Brown Network Infrastructure Manager Documentation, Webmaster Teams Using another language more suited towards CLI / standalone (non-web) development would be easier. PHP at it's core is a templating language. I don't think it is as suited as say Python for developing standalone applications. Regards, -Josh Joshua Kehn | josh.k...@gmail.com http://joshuakehn.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Fri, Jan 7, 2011 at 12:18, Joshua Kehn josh.k...@gmail.com wrote: Using another language more suited towards CLI / standalone (non-web) development would be easier. PHP at it's core is a templating language. I don't think it is as suited as say Python for developing standalone applications. One might argue that it depends on your mastery of and comfort with the language. That in mind, the same is true of nearly any programming language. And thanks for reminding me of what PHP is at its core. ;-P -- /Daniel P. Brown Network Infrastructure Manager Documentation, Webmaster Teams http://www.php.net/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Fri, Jan 7, 2011 at 11:55 AM, la...@garfieldtech.com la...@garfieldtech.com wrote: Hi folks. I have a project coming up that will involve writing a non-trivial command line PHP application. Most of it will be nice and abstracted and standalone and all of that jazz, but it will need to do command line interation. I'm not sure yet if it will be interactive or if I just need to parse lots of command line switches. Has anyone used a CLI-handling library they like? I recall briefly using the PEAR CLI library many many years ago and disliking it because it was only barely a step above the raw PHP-CLI SAPI, and still required lots of if-else branching in my code. I don't know if there's anything better since then, however. I prefer clean OO to procedural, but can work with procedural if needs be. The fewer dependencies it has the better as well. Any recommendations? (Open source, GPLv2-compatible required.) --Larry Garfield -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php If all you want is a way to handle command line arguments, I built a library to do that a few years ago. I got sick of needing to switch languages to write a quick command-line script, and I hated the PEAR CLI tools. I haven't really looked at it since, and since it was just about the first thing I wrote that was object oriented I suspect it's got some problems, but you're welcome to give it a try. http://people.chem.umass.edu/amckenzie/code/php/php_cli_io/ It handles flags both with and without short forms, and flags with either zero, one or two arguments. The demo file in the package has the docs written into it -- read through it to see how the library works. If anyone tries it, let me know what you think, and what I've done wrong. It does everything I need, but I don't think anyone else has ever used it. -Alex -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Jan 7, 2011, at 12:34 PM, Daniel Brown wrote: On Fri, Jan 7, 2011 at 12:18, Joshua Kehn josh.k...@gmail.com wrote: Using another language more suited towards CLI / standalone (non-web) development would be easier. PHP at it's core is a templating language. I don't think it is as suited as say Python for developing standalone applications. One might argue that it depends on your mastery of and comfort with the language. That in mind, the same is true of nearly any programming language. And thanks for reminding me of what PHP is at its core. ;-P -- /Daniel P. Brown Network Infrastructure Manager Documentation, Webmaster Teams http://www.php.net/ My apologies. I just view PHP as a perfected web language, due to it's templating nature, while using it for other things (scripts, utilities, cron) is a misuse in my opinion. It does completely depend on your mastery of the language. If you're very good with PHP and you don't often do non-web things it might not make sense to learn another language as well. Regards, -Josh Joshua Kehn | josh.k...@gmail.com http://joshuakehn.com
Re: [PHP] Command line PHP
On Fri, Jan 7, 2011 at 11:31 AM, Joshua Kehn josh.k...@gmail.com wrote: On Jan 7, 2011, at 12:34 PM, Daniel Brown wrote: On Fri, Jan 7, 2011 at 12:18, Joshua Kehn josh.k...@gmail.com wrote: Using another language more suited towards CLI / standalone (non-web) development would be easier. PHP at it's core is a templating language. I don't think it is as suited as say Python for developing standalone applications. One might argue that it depends on your mastery of and comfort with the language. That in mind, the same is true of nearly any programming language. And thanks for reminding me of what PHP is at its core. ;-P -- /Daniel P. Brown Network Infrastructure Manager Documentation, Webmaster Teams http://www.php.net/ My apologies. I just view PHP as a perfected web language, due to it's templating nature, while using it for other things (scripts, utilities, cron) is a misuse in my opinion. shrug, you must not be too familiar with php then. 9 times out of 10 it's the natural, perfect choice for a cli program. there are situations where you get past what php is ideal for on the cli, typically when you get into heavy forking or require threading. It does completely depend on your mastery of the language. If you're very good with PHP and you don't often do non-web things it might not make sense to learn another language as well. why bother learning 2 languages when 1 will suit most needs perfectly? for most folks who work with the web and a typical deployment environment like a linux server, the second language of choice most likely would be a client side one like javascript. -nathan
Re: [PHP] Command line PHP
On Fri, 2011-01-07 at 13:31 -0500, Joshua Kehn wrote: On Jan 7, 2011, at 12:34 PM, Daniel Brown wrote: On Fri, Jan 7, 2011 at 12:18, Joshua Kehn josh.k...@gmail.com wrote: Using another language more suited towards CLI / standalone (non-web) development would be easier. PHP at it's core is a templating language. I don't think it is as suited as say Python for developing standalone applications. One might argue that it depends on your mastery of and comfort with the language. That in mind, the same is true of nearly any programming language. And thanks for reminding me of what PHP is at its core. ;-P -- /Daniel P. Brown Network Infrastructure Manager Documentation, Webmaster Teams http://www.php.net/ My apologies. I just view PHP as a perfected web language, due to it's templating nature, while using it for other things (scripts, utilities, cron) is a misuse in my opinion. It does completely depend on your mastery of the language. If you're very good with PHP and you don't often do non-web things it might not make sense to learn another language as well. Regards, -Josh Joshua Kehn | josh.k...@gmail.com http://joshuakehn.com There's no real reason why you shouldn't use PHP for cli apps, it has a lot of features specifically intended for it even. I've found it to be fast enough for my needs, it's familiar, and I don't need to compile an app every time I make a small code change. Yes PHP is primarily a web-based language, but there's no reason it can't be used other places too. In fact, I've even got PHP installed on my Android phone, although I've yet to find a practical use for that! Thanks, Ash http://www.ashleysheridan.co.uk
Re: [PHP] Command line PHP
On Jan 7, 2011, at 1:48 PM, Nathan Nobbe wrote: shrug, you must not be too familiar with php then. 9 times out of 10 it's the natural, perfect choice for a cli program. there are situations where you get past what php is ideal for on the cli, typically when you get into heavy forking or require threading. It does completely depend on your mastery of the language. If you're very good with PHP and you don't often do non-web things it might not make sense to learn another language as well. why bother learning 2 languages when 1 will suit most needs perfectly? for most folks who work with the web and a typical deployment environment like a linux server, the second language of choice most likely would be a client side one like javascript. -nathan You can't say that PHP is a more natural CLI choice then bash or Python. Maybe I'm using PHP too much for web development. Perhaps I am unfamiliar with it. Why bother learning other languages? Is this a joke? Why should someone stop learning ever? Having a mastery of multiple languages can only enhance you. Regards, -Josh Joshua Kehn | josh.k...@gmail.com http://joshuakehn.com
Re: [PHP] Command line PHP
On Jan 7, 2011, at 1:53 PM, Ashley Sheridan wrote: There's no real reason why you shouldn't use PHP for cli apps, it has a lot of features specifically intended for it even. I've found it to be fast enough for my needs, it's familiar, and I don't need to compile an app every time I make a small code change. Yes PHP is primarily a web-based language, but there's no reason it can't be used other places too. In fact, I've even got PHP installed on my Android phone, although I've yet to find a practical use for that! Thanks, Ash http://www.ashleysheridan.co.uk I don't see a reason other then I find other languages more nimble for small scripts. If you are using lots of shared PHP code then of course it would make sense to leverage that existing code rather then re-implement it. The OP was asking (I assume) about a standalone application built from scratch. I would kill for a PHP interpreter (+ Haskell and Python) on iOS. Unfortunately I haven't found one. Regards, -Josh Joshua Kehn | josh.k...@gmail.com http://joshuakehn.com
Re: [PHP] Command line PHP
On Fri, Jan 7, 2011 at 13:31, Joshua Kehn josh.k...@gmail.com wrote: My apologies. I just view PHP as a perfected web language, due to it's templating nature, while using it for other things (scripts, utilities, cron) is a misuse in my opinion. No one ever needs to apologize for their opinion, Josh, no worries. -- /Daniel P. Brown Network Infrastructure Manager Documentation, Webmaster Teams http://www.php.net/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On 1/7/11 11:08 AM, Nicholas Kell wrote: On Jan 7, 2011, at 11:01 AM, Joshua Kehn wrote: On Jan 7, 2011, at 11:55 AM, la...@garfieldtech.com wrote: Hi folks. I have a project coming up that will involve writing a non-trivial command line PHP application. Most of it will be nice and abstracted and standalone and all of that jazz, but it will need to do command line interation. I'm not sure yet if it will be interactive or if I just need to parse lots of command line switches. Has anyone used a CLI-handling library they like? I recall briefly using the PEAR CLI library many many years ago and disliking it because it was only barely a step above the raw PHP-CLI SAPI, and still required lots of if-else branching in my code. I don't know if there's anything better since then, however. I prefer clean OO to procedural, but can work with procedural if needs be. The fewer dependencies it has the better as well. Any recommendations? (Open source, GPLv2-compatible required.) --Larry Garfield Larry- Why are you writing a command line application in PHP? I would think that is starting off on a very wrong foot. Can you explain the requirements / use cases a bit more? I agree. Not saying that it isn't doable, because it certainly is, but there may be other languages that are available and easier to implement a CLI app, such as Python, or some other QnD scripting language. Application is perhaps a misnomer. I'm not looking at rewriting Emacs or anything. Just some batch processing that would get run as: php myscript.php --config=foo.xml --setting-1=stuff And then it will run off and move a few million records around between different data stores, a process that will probably take an hour or so. (The backend will be cycling through a queue server.) I just need something to make handling of the args and environment easier, because I find the native SAPI calls to be ugly/cumbersome. I'm sure it could be written in Perl or Python or Java. But I know extremely little Perl, no Python, and my Java is quite rusty, plus there are mature PHP libraries that talk to the 3rd party systems I'm tying together. My PHP-fu is much stronger than my Perl, Python, and Java combined. OT Yeah, PHP was intended as a template language only; that fell by the wayside a decade ago or more when people started building real web apps in it, which are a lot more than templates. That boat has long since sailed and is irrelevant to this discussion. /OT --Larry Garfield -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Fri, Jan 7, 2011 at 10:31 AM, Joshua Kehn josh.k...@gmail.com wrote: My apologies. I just view PHP as a perfected web language, due to it's templating nature, while using it for other things (scripts, utilities, cron) is a misuse in my opinion. Even if you are proficient in more CLI-appropriate languages, there are times where you'd still choose to use PHP in this role. For example, our data access layer is written in PHP. It was an easy choice to write our sitemap-generating tool in PHP even though I am more comfortable in Java and Python. The right tool for the job rarely depends on only one or two factors. :) David
Re: [PHP] Command line PHP
On 11-01-07 01:31 PM, Joshua Kehn wrote: On Jan 7, 2011, at 12:34 PM, Daniel Brown wrote: On Fri, Jan 7, 2011 at 12:18, Joshua Kehnjosh.k...@gmail.com wrote: Using another language more suited towards CLI / standalone (non-web) development would be easier. PHP at it's core is a templating language. I don't think it is as suited as say Python for developing standalone applications. One might argue that it depends on your mastery of and comfort with the language. That in mind, the same is true of nearly any programming language. And thanks for reminding me of what PHP is at its core. ;-P -- /Daniel P. Brown Network Infrastructure Manager Documentation, Webmaster Teams http://www.php.net/ My apologies. I just view PHP as a perfected web language, due to it's templating nature, while using it for other things (scripts, utilities, cron) is a misuse in my opinion. It does completely depend on your mastery of the language. If you're very good with PHP and you don't often do non-web things it might not make sense to learn another language as well. I disagree... many of my cron/shell scripts leverage the very codebase that the site works on to do things for the site from the command-line. Why would I use another language that would require re-implementation of existing functionality? The right tool for the job depends on the job in question regardless of someone else's perception of the intended purpose of the tool. Cheers, Rob. -- E-Mail Disclaimer: Information contained in this message and any attached documents is considered confidential and legally protected. This message is intended solely for the addressee(s). Disclosure, copying, and distribution are prohibited unless authorized. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Fri, Jan 7, 2011 at 11:54 AM, Joshua Kehn josh.k...@gmail.com wrote: On Jan 7, 2011, at 1:48 PM, Nathan Nobbe wrote: shrug, you must not be too familiar with php then. 9 times out of 10 it's the natural, perfect choice for a cli program. there are situations where you get past what php is ideal for on the cli, typically when you get into heavy forking or require threading. It does completely depend on your mastery of the language. If you're very good with PHP and you don't often do non-web things it might not make sense to learn another language as well. why bother learning 2 languages when 1 will suit most needs perfectly? for most folks who work with the web and a typical deployment environment like a linux server, the second language of choice most likely would be a client side one like javascript. -nathan You can't say that PHP is a more natural CLI choice then bash or Python. Maybe I'm using PHP too much for web development. Perhaps I am unfamiliar with it. Um, I just did say it :P It's pretty simple, I already know PHP, I don't know Python or BASH well enough, obviously PHP is the best choice for a CLI program in this case. Yeah.., try writing a few scripts in PHP on the CLI, you'll find it solid for many applications. Why bother learning other languages? Is this a joke? Why should someone stop learning *ever?** *Having a mastery of multiple languages can only enhance you. No, it's not a joke. The idea is why bother learning an language that will score you little milage, in the context of the one you already know well. For example, if I'm a Web developer coding what CLI requirements I have in PHP, then I might venture to learn something like Javascript before ever touching Python. So yes, having mastery of multiple languages can help you, but it's best when these multiple languages don't overlap as much as PHP and Python. -nathan
Re: [PHP] Command line PHP
On Jan 7, 2011, at 1:41 PM, la...@garfieldtech.com wrote: Application is perhaps a misnomer. I'm not looking at rewriting Emacs or anything. Just some batch processing that would get run as: php myscript.php --config=foo.xml --setting-1=stuff And then it will run off and move a few million records around between different data stores, a process that will probably take an hour or so. (The backend will be cycling through a queue server.) I just need something to make handling of the args and environment easier, because I find the native SAPI calls to be ugly/cumbersome. I'm sure it could be written in Perl or Python or Java. But I know extremely little Perl, no Python, and my Java is quite rusty, plus there are mature PHP libraries that talk to the 3rd party systems I'm tying together. My PHP-fu is much stronger than my Perl, Python, and Java combined. OT Yeah, PHP was intended as a template language only; that fell by the wayside a decade ago or more when people started building real web apps in it, which are a lot more than templates. That boat has long since sailed and is irrelevant to this discussion. /OT --Larry Garfield In that case I can't offer any good CLI libs, but it sounds like a few others here could offer some. Regards, -Josh Joshua Kehn | josh.k...@gmail.com http://joshuakehn.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Fri, Jan 7, 2011 at 10:41 AM, la...@garfieldtech.com la...@garfieldtech.com wrote: Application is perhaps a misnomer. I'm not looking at rewriting Emacs or anything. Just some batch processing that would get run as: php myscript.php --config=foo.xml --setting-1=stuff For this I used getopt() which worked well enough. Long options don't work on Windows IIRC (check the docs), but you can easily get arguments from short options. An example for the tool I wrote is src/sitemap.php -s 28372 -d mydomain.com -l debug -p -z 9 -b sitemaps -o xml The code that parses the options is very straightforward. If you already use Zend Framework, it has its own CLI library I think. David
Re: [PHP] Command line PHP
On 11-01-07 11:55 AM, la...@garfieldtech.com wrote: Hi folks. I have a project coming up that will involve writing a non-trivial command line PHP application. Most of it will be nice and abstracted and standalone and all of that jazz, but it will need to do command line interation. I'm not sure yet if it will be interactive or if I just need to parse lots of command line switches. Has anyone used a CLI-handling library they like? I recall briefly using the PEAR CLI library many many years ago and disliking it because it was only barely a step above the raw PHP-CLI SAPI, and still required lots of if-else branching in my code. I don't know if there's anything better since then, however. I prefer clean OO to procedural, but can work with procedural if needs be. The fewer dependencies it has the better as well. Any recommendations? (Open source, GPLv2-compatible required.) Hi Larry, I wrote the following some time ago: http://www.interjinn.com/jinnDoc/interJinn.class.JinnCoreServicesCliArguments.php It is tied to my framework, but could easily be lifted and embedded in a standalone class. License is GPL V2 so feel free to adapt as befits said license. Cheers, Rob. -- E-Mail Disclaimer: Information contained in this message and any attached documents is considered confidential and legally protected. This message is intended solely for the addressee(s). Disclosure, copying, and distribution are prohibited unless authorized. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Jan 7, 2011, at 12:08 PM, Nicholas Kell wrote:
On Jan 7, 2011, at 11:01 AM, Joshua Kehn wrote:
On Jan 7, 2011, at 11:55 AM, la...@garfieldtech.com wrote:
Hi folks. I have a project coming up that will involve writing a
non-trivial command line PHP application. Most of it will be nice and
abstracted and standalone and all of that jazz, but it will need to do
command line interation. I'm not sure yet if it will be interactive or if
I just need to parse lots of command line switches.
Has anyone used a CLI-handling library they like? I recall briefly using
the PEAR CLI library many many years ago and disliking it because it was
only barely a step above the raw PHP-CLI SAPI, and still required lots of
if-else branching in my code. I don't know if there's anything better
since then, however. I prefer clean OO to procedural, but can work with
procedural if needs be. The fewer dependencies it has the better as well.
Any recommendations?
(Open source, GPLv2-compatible required.)
--Larry Garfield
Larry-
Why are you writing a command line application in PHP? I would think that is
starting off on a very wrong foot.
Can you explain the requirements / use cases a bit more?
I agree. Not saying that it isn't doable, because it certainly is, but there
may be other languages that are available and easier to implement a CLI app,
such as Python, or some other QnD scripting language.
Pardon me but what's the big deal?
Put a shebang up top and then add:
//
// --
// Get CLI args from argv
// --
//
function getargs($argv)
{
// ./cli.php file1 file2 --path=/foo/bar -t -B quux
// getargs($GLOBALS['argv'])
$args['self'] = array_shift( $argv ) ;
while( 0 sizeof( $argv ) ) {
$arg = array_shift( $argv ) ;
if( '--' == substr( $arg , 0 , 2 ) ) {
$eq = strpos( $arg , '=' ) ;
$name = substr( $arg , 2 , $eq - 2 ) ;
$val = substr( $arg , $eq + 1 ) ;
$args[$name][] = $val ;
} else if( '-' == $arg[0] ) {
$name = substr( $arg , 1 ) ;
if (isset($argv[0][0]) ( '-' != $argv[0][0] )) {
$val = array_shift( $argv ) ;
} else {
$val = '' ;
}
$args[$name][] = $val ;
} else {
$args['file'][] = substr( $arg , 0 ) ;
}
}
return $args ;
}
// --
// Main
// --
//Get CLI options
$args = getargs($GLOBALS['argv']);
//Display Basic Help
if (isset($args['h']) || isset($args['help'])) {
echo ./your_php_cli_app.php -h \n;
echo -h - displays this help\n;
echo ... is more of your command line args\n;
echo \n;
exit;
}
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Fri, 2011-01-07 at 11:31 -0800, David Harkness wrote: On Fri, Jan 7, 2011 at 10:41 AM, la...@garfieldtech.com la...@garfieldtech.com wrote: Application is perhaps a misnomer. I'm not looking at rewriting Emacs or anything. Just some batch processing that would get run as: php myscript.php --config=foo.xml --setting-1=stuff For this I used getopt() which worked well enough. Long options don't work on Windows IIRC (check the docs), but you can easily get arguments from short options. An example for the tool I wrote is src/sitemap.php -s 28372 -d mydomain.com -l debug -p -z 9 -b sitemaps -o xml The code that parses the options is very straightforward. If you already use Zend Framework, it has its own CLI library I think. David getopt() doesn't work well on Windows on early versions of PHP 5.2 at all, as I found to my dismay at work. As an alternative, I used the $GLOBALS['argv'] array, which contains the command line arguments, although not in as nice a format as you get them from getopt() Thanks, Ash http://www.ashleysheridan.co.uk
Re: [PHP] Command line PHP
At 11:48 AM -0700 1/7/11, Nathan Nobbe wrote: why bother learning 2 languages when 1 will suit most needs perfectly? for most folks who work with the web and a typical deployment environment like a linux server, the second language of choice most likely would be a client side one like javascript. -nathan And don't forget MySQL, HTML and CSS (if you want to call them languages). We are a ways from having one unified language, but closer than we were a few years ago. Cheers, tedd -- --- http://sperling.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On 11-01-07 02:33 PM, Robert Cummings wrote: On 11-01-07 11:55 AM, la...@garfieldtech.com wrote: Hi folks. I have a project coming up that will involve writing a non-trivial command line PHP application. Most of it will be nice and abstracted and standalone and all of that jazz, but it will need to do command line interation. I'm not sure yet if it will be interactive or if I just need to parse lots of command line switches. Has anyone used a CLI-handling library they like? I recall briefly using the PEAR CLI library many many years ago and disliking it because it was only barely a step above the raw PHP-CLI SAPI, and still required lots of if-else branching in my code. I don't know if there's anything better since then, however. I prefer clean OO to procedural, but can work with procedural if needs be. The fewer dependencies it has the better as well. Any recommendations? (Open source, GPLv2-compatible required.) Hi Larry, I wrote the following some time ago: http://www.interjinn.com/jinnDoc/interJinn.class.JinnCoreServicesCliArguments.php It is tied to my framework, but could easily be lifted and embedded in a standalone class. License is GPL V2 so feel free to adapt as befits said license. I just realized it's one of my lazy classes and so it's not fully documented :) It would be as simple as (if converted to a standalone class): ?php $args = new CliArgs(); $param1 = $args-getArg( 'param1' ); $param2 = $args-getArg( 'param2' ); $files = $args-getSequence(); ? For any of the following command-lines: ./myscript.php --param1=foo --param2=fee file1 file2 file3 ./myscript.php --param1 foo --param2 fee file1 file2 file3 ./myscript.php -param1=foo -param2=fee file1 file2 file3 ./myscript.php -param1 foo -param2 fee file1 file2 file3 Cheers, Rob. -- E-Mail Disclaimer: Information contained in this message and any attached documents is considered confidential and legally protected. This message is intended solely for the addressee(s). Disclosure, copying, and distribution are prohibited unless authorized. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
At 1:54 PM -0500 1/7/11, Joshua Kehn wrote: Why should someone stop learning ever? Because my head fills up. I have to wait until I forget something before I can learn something new. The up-side is that I'm learning something new almost every day now. Cheers, tedd -- --- http://sperling.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
At 12:16 PM -0700 1/7/11, Nathan Nobbe wrote: On Fri, Jan 7, 2011 at 11:54 AM, Joshua Kehn josh.k...@gmail.com wrote: Why bother learning other languages? Is this a joke? Why should someone stop learning *ever?** *Having a mastery of multiple languages can only enhance you. No, it's not a joke. The idea is why bother learning an language that will score you little milage, in the context of the one you already know well. For example, if I'm a Web developer coding what CLI requirements I have in PHP, then I might venture to learn something like Javascript before ever touching Python. So yes, having mastery of multiple languages can help you, but it's best when these multiple languages don't overlap as much as PHP and Python. -nathan nathan: I have to disagree with you a little-bit about this. If one knows the languages involved and realize their overlap, then they should have a better understanding of which language to use for what purpose. For example, which is better for styling CSS or Javascript? Both can do it, but one is the clear winner. Cheers, tedd -- --- http://sperling.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On 11-01-07 02:53 PM, tedd wrote: At 12:16 PM -0700 1/7/11, Nathan Nobbe wrote: On Fri, Jan 7, 2011 at 11:54 AM, Joshua Kehnjosh.k...@gmail.com wrote: Why bother learning other languages? Is this a joke? Why should someone stop learning *ever?** *Having a mastery of multiple languages can only enhance you. No, it's not a joke. The idea is why bother learning an language that will score you little milage, in the context of the one you already know well. For example, if I'm a Web developer coding what CLI requirements I have in PHP, then I might venture to learn something like Javascript before ever touching Python. So yes, having mastery of multiple languages can help you, but it's best when these multiple languages don't overlap as much as PHP and Python. -nathan nathan: I have to disagree with you a little-bit about this. If one knows the languages involved and realize their overlap, then they should have a better understanding of which language to use for what purpose. For example, which is better for styling CSS or Javascript? Both can do it, but one is the clear winner. Is the winner JavaScript? *ducks and runs* I love Fridays! Cheers, Rob. -- E-Mail Disclaimer: Information contained in this message and any attached documents is considered confidential and legally protected. This message is intended solely for the addressee(s). Disclosure, copying, and distribution are prohibited unless authorized. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Fri, Jan 7, 2011 at 12:53 PM, tedd tedd.sperl...@gmail.com wrote: At 12:16 PM -0700 1/7/11, Nathan Nobbe wrote: On Fri, Jan 7, 2011 at 11:54 AM, Joshua Kehn josh.k...@gmail.com wrote: Why bother learning other languages? Is this a joke? Why should someone stop learning *ever?** *Having a mastery of multiple languages can only enhance you. No, it's not a joke. The idea is why bother learning an language that will score you little milage, in the context of the one you already know well. For example, if I'm a Web developer coding what CLI requirements I have in PHP, then I might venture to learn something like Javascript before ever touching Python. So yes, having mastery of multiple languages can help you, but it's best when these multiple languages don't overlap as much as PHP and Python. -nathan nathan: I have to disagree with you a little-bit about this. If one knows the languages involved and realize their overlap, then they should have a better understanding of which language to use for what purpose. For example, which is better for styling CSS or Javascript? Both can do it, but one is the clear winner. agreed, however, if im a skilled js programmer and need some rounded corners, id probly just grab a library that leverages css to accomplish the job before sitting down and really learning css. or perhaps if my sql skills aren't strong enough, i know i can get the job done w/ some extra manipulation in php. one should consider how much time will be spent solving the current problem, how much time is available before the solution must ship, and if it's beneficial to learn something entirely new just to solve it when your current knowledge may provide a sufficient solution. we're comparing php to bash/python here and frankly id like to know what the main advantages of those languages over php for cli processing would be .. marginal at best. id hazard a guess that python supports threading tho :O moreover, if i already know php, python is probly one of the last languages i want to learn unless im considering abandoning php... the thought has occurred to me ducks id rather spend time learning something that runs fast like brushing up on java or maybe C or C++, that way i'll get more mileage for my time as i'll be more versatile with less overlap. it's like i could know 2 scripting languages w/ heavy overlap or 1 scripting language, and one compiled language, that seems better from my perspective. there are always merits to knowing a shell language tho, and i've wanted to get stronger with bash, but python .. i see it more as python OR php. i worked on an asterisk app one time w/ a php front-end and many of the asterisk hook scripts were written in bash. this was a poor choice imo. #1 we weren't leveraging code from the main app #2 now i'm looking at running mysql client through awk on the cli to get args into bash .. ouch. give me mysql_query(), lol. much easier for anyone coming into the biz who knows php to keep things consistent imo. much of the gripe comparing php to python over the years is slowly fading with the advent of new features of php, most notably closures. once traits are available im sure the multiple inheritance that python offers will be less of an advantage over php's single inheritance paradigm. -nathan
Re: [PHP] Command line PHP
On 11-01-07 03:24 PM, Nathan Nobbe wrote: much of the gripe comparing php to python over the years is slowly fading with the advent of new features of php, most notably closures. once traits are available im sure the multiple inheritance that python offers will be less of an advantage over php's single inheritance paradigm. One other thing that improves PHP on the CLI front is the addition of garbage collection for circular references. Solves the occasional hard to find leak :) Cheers, Rob. -- E-Mail Disclaimer: Information contained in this message and any attached documents is considered confidential and legally protected. This message is intended solely for the addressee(s). Disclosure, copying, and distribution are prohibited unless authorized. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Jan 7, 2011, at 1:34 PM, TR Shaw wrote: On Jan 7, 2011, at 12:08 PM, Nicholas Kell wrote: On Jan 7, 2011, at 11:01 AM, Joshua Kehn wrote: On Jan 7, 2011, at 11:55 AM, la...@garfieldtech.com wrote: Hi folks. I have a project coming up that will involve writing a non-trivial command line PHP application. Most of it will be nice and abstracted and standalone and all of that jazz, but it will need to do command line interation. I'm not sure yet if it will be interactive or if I just need to parse lots of command line switches. Has anyone used a CLI-handling library they like? I recall briefly using the PEAR CLI library many many years ago and disliking it because it was only barely a step above the raw PHP-CLI SAPI, and still required lots of if-else branching in my code. I don't know if there's anything better since then, however. I prefer clean OO to procedural, but can work with procedural if needs be. The fewer dependencies it has the better as well. Any recommendations? (Open source, GPLv2-compatible required.) --Larry Garfield Larry- Why are you writing a command line application in PHP? I would think that is starting off on a very wrong foot. Can you explain the requirements / use cases a bit more? I agree. Not saying that it isn't doable, because it certainly is, but there may be other languages that are available and easier to implement a CLI app, such as Python, or some other QnD scripting language. Pardon me but what's the big deal? Put a shebang up top and then add: [code] [/snip] There isn't a big deal. If PHP is the only suitable tool you have, go for it. There are plenty of ways to do this. My natural first reaction is to write it in Python, Ruby or something else. But thats me - my personal toolbox is chock full of languages. PHP only being one of them. That being said, have I ever written a CLI app in PHP - sure whenever I need a quick and dirty to move a few thousand records, or whatever, why not. As the OP has said (After my comment) his PHP-fu is strong, whereas in other languages are not. This in my humble opinion is the perfect time to use PHP. Do I ever see one language to be the end all be all for every application? No, absolutely not. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
At 1:24 PM -0700 1/7/11, Nathan Nobbe wrote: On Fri, Jan 7, 2011 at 12:53 PM, tedd mailto:tedd.sperl...@gmail.comtedd.sperl...@gmail.com wrote: much of the gripe comparing php to python -nathan I try to stay away from snakes. Cheers, tedd -- --- http://sperling.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
At 3:05 PM -0500 1/7/11, Robert Cummings wrote: Is the winner JavaScript? *ducks and runs* Rob. Careful or I'll have to fog you as well. :-) Cheers, tedd -- --- http://sperling.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Fri, Jan 7, 2011 at 2:52 PM, tedd tedd.sperl...@gmail.com wrote: At 1:24 PM -0700 1/7/11, Nathan Nobbe wrote: On Fri, Jan 7, 2011 at 12:53 PM, tedd mailto:tedd.sperl...@gmail.com tedd.sperl...@gmail.com wrote: much of the gripe comparing php to python -nathan I try to stay away from snakes. I have one tattooed on my back ;) -nathan
Re: [PHP] Command line PHP
Joshua Kehn wrote: why bother learning 2 languages when 1 will suit most needs perfectly? for most folks who work with the web and a typical deployment environment like a linux server, the second language of choice most likely would be a client side one like javascript. You can't say that PHP is a more natural CLI choice then bash or Python. Maybe I'm using PHP too much for web development. Perhaps I am unfamiliar with it. Why bother learning other languages? Is this a joke? Why should someone stop learning ever? Having a mastery of multiple languages can only enhance you. Having got a basket of several dialects of machine code, cobol, fortran, algol, C, C++, pascal, javascript, and probably a few more ... it WOULD be nice not to NOW have to learn java to fix Eclipse, python to fix Hg, ruby for some of my mapping requirements AND bash scripts to sort out keeping everything running. If a website is running background tasks to maintain the site, then using the SAME language for the command line functions makes perfect sense. The SAME library is doing the processing so any bugs should be the same which ever route is used. Trying to debug a fault which is also going cross language just adds to the complexity? I think this is one of the reasons that all this push to reinvent PHP into something it does not need to be is so annoying. How about a RealPHP which is frozen at some point where those of us who can't be bothered with Traits and all this other mumbo jumbo can simply carry on designing sites. And the people who are more interested in development for development sake can please themselves? -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
I'm with some of the others above on using Python. Writing a command line app is about as simple as: import subprocess word = 'hello' self.espeak = subprocess.Popen(['espeak', word], stdout = subprocess.PIPE).communicate()[0] I think of PHP as more browser, than desktop app/webapp. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
There have been a lot of responses, but this might be the best place to start: http://www.google.com/search?client=ubuntuchannel=fsq=command+line+PHP+application.ie=utf-8oe=utf-8 Which yielded this as the first result: http://php.net/manual/en/features.commandline.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
On Friday, January 07, 2011 9:34:42 pm David Hutto wrote: Which yielded this as the first result: http://php.net/manual/en/features.commandline.php As noted in my original email, I find the native SAPI clunky and difficult to work with. Hence I was looking for something more usable and robust built on top of it that I could leverage rather than rolling my own one-off. Of course, I got lost somewhere in the language holy wars (dear god, people...) so I'll probably just take the roll my own approach. --Larry Garfield -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line PHP
I could go yoda, but suffice it to say, From The Language Speaks The Soul Of The Man's Design. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP memory limit
Per Jessen wrote: Rene Fournier wrote: Is it possible to set a unique memory limit for PHP scripts that are run from the command line? (That is, different from what's specified in php.ini.) This might specific to openSUSE, but the typical installation comes with separate php.inis for apache and cli. /etc/php5/cli/php.ini /etc/php5/apache2/php.ini /Per Jessen, Zürich K/Ubuntu, I would assume Debian as well. -Shawn -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] Command-line PHP memory limit
-Original Message- From: Shawn McKenzie [mailto:[EMAIL PROTECTED] Sent: Thursday, June 12, 2008 11:06 AM To: php-general@lists.php.net Subject: Re: [PHP] Command-line PHP memory limit Per Jessen wrote: Rene Fournier wrote: Is it possible to set a unique memory limit for PHP scripts that are run from the command line? (That is, different from what's specified in php.ini.) This might specific to openSUSE, but the typical installation comes with separate php.inis for apache and cli. /etc/php5/cli/php.ini /etc/php5/apache2/php.ini /Per Jessen, Zürich K/Ubuntu, I would assume Debian as well. Correct. I am running Debian etch, and I have two separate php.ini files (automatically configured as such)--one for Apache, one for command line. Todd Boyd Web Programmer
[PHP] Command-line PHP memory limit
Is it possible to set a unique memory limit for PHP scripts that are run from the command line? (That is, different from what's specified in php.ini.) ...Rene
Re: [PHP] Command-line PHP memory limit
On Wed, 2008-06-11 at 23:48 +0200, Rene Fournier wrote: Is it possible to set a unique memory limit for PHP scripts that are run from the command line? (That is, different from what's specified in php.ini.) In your script: ini_set( 'memory_limit', -1 ); Cheers, Rob. -- http://www.interjinn.com Application and Templating Framework for PHP -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP memory limit
Rene Fournier wrote: Is it possible to set a unique memory limit for PHP scripts that are run from the command line? (That is, different from what's specified in php.ini.) This might specific to openSUSE, but the typical installation comes with separate php.inis for apache and cli. /etc/php5/cli/php.ini /etc/php5/apache2/php.ini /Per Jessen, Zürich -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] MySQL to blame? (was Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?)
M5 wrote: On 20-Dec-07, at 1:17 AM, Per Jessen wrote: René Fournier wrote: I'm really not sure what to try next. ps -aux shows MySQL as hogging the CPU, not PHP or Terminal: When this happens, do a 'SHOW PROCESSLIST' in mysql to see what it's doing. I have, and I can't see anything unusual. There are a few scripts that loop with very slow overhead (with sufficient sleep(), etc.) plus a few outside HTTP requests. Nothing unusual. Incidentally, yesterday, when MySQL went to 80-90% again after a week, I let it stay there while I poked around MySQL (doing the above) and the OS to see if there are some magical limit that I might be breaking. So it the server ran with MySQL at 80-90% CPU for about eight hours. Everything still worked fine, scripts ran, the database was available. That's the thing about this problem--it's not a show-stopper, it's just really strange. And I can't figure its source. Check your mysql logs for just before this time and see if there are any queries there that need attention. Do you have the slow-queries log enabled? Also make sure you have the option to log queries that don't use an index turned on. See if anything there gives you some clues. Are you committing a big transaction (thousands of records or something)? Or do you have a transaction idling and not committing or rolling back? Are you replicating data to another server and this is triggering the problem? Or a backup is running? It doesn't shutdown. Finally--and I really hate doing this, because it sees dangerous to data (is it?)--I issue a kill -9 command to its process. I'd say it's very dangerous to do that to a database but ask the mysql list, they will have better insight into what this does. -- Postgresql php tutorials http://www.designmagick.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
René Fournier wrote: I'm really not sure what to try next. ps -aux shows MySQL as hogging the CPU, not PHP or Terminal: When this happens, do a 'SHOW PROCESSLIST' in mysql to see what it's doing. /Per Jessen, Zürich -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] MySQL to blame? (was Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?)
On 20-Dec-07, at 1:17 AM, Per Jessen wrote: René Fournier wrote: I'm really not sure what to try next. ps -aux shows MySQL as hogging the CPU, not PHP or Terminal: When this happens, do a 'SHOW PROCESSLIST' in mysql to see what it's doing. I have, and I can't see anything unusual. There are a few scripts that loop with very slow overhead (with sufficient sleep(), etc.) plus a few outside HTTP requests. Nothing unusual. Incidentally, yesterday, when MySQL went to 80-90% again after a week, I let it stay there while I poked around MySQL (doing the above) and the OS to see if there are some magical limit that I might be breaking. So it the server ran with MySQL at 80-90% CPU for about eight hours. Everything still worked fine, scripts ran, the database was available. That's the thing about this problem--it's not a show- stopper, it's just really strange. And I can't figure its source. After not finding anything, I decided restart the script. So I stop the [seemingly offending] script and wait for CPU load to return to normal. It doesn't. MySQL remains at 80-90%. Even with all the other processes turned off that call MySQL and Web Server off, MySQL remains at 80-90%. Yet SHOW PROCESSES lists no processes, just the show processlist command I issue. With the load still high, I attempted to Stop MySQL via the Adminstrator control panel. I waited a few minutes. It doesn't shutdown. Finally--and I really hate doing this, because it sees dangerous to data (is it?)--I issue a kill -9 command to its process. Then it starts fine, I start the script in question, and everything is back to normal. ...Rene -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
On 11-Dec-07, at 2:13 PM, Per Jessen wrote: René Fournier wrote: However, the number of socket clients connecting in the past 3-4 months has steadily increased, and this seems to have exposed (if not created) a strange performance issue with PHP 5.2.4, MySQL 5.0.45 and/or Mac OS X Server 10.4.11. (I say and/or because I am unsure where the problem's cause really lies.) Basically, after the script has been running for a day or so (processing essentially the amount data that used to take two weeks), the CPU usage of the machine goes from 30% (normal) to 80-90%. Have you tried stracing it to see what's really happening when the load goes that high? Good advice, since I think there's nothing left for me to do but inspect the MySQL process. Incidentally, I've made some changes to the script a week ago, which has seemed to improve the situation somewhat. Now, the script has run for nearly 7 days without interruption or high CPU load. Problem solved? Again this morning, I noticed CPU went up to 90% and is staying there. (Previously, this would happen after 1-2 days.) The number of distinct MySQL connections remains low, since the script (which runs in a loop, with a sleep(1) and a timeout on the stream_socket_select()), only creates one MySQL connection in the beginning. All MySQL queries run through that. The script has run for 6-7 days, during which time it's executed 2.7 million queries (mostly SELECTs) and created 105,198 external, short- lived child processes (each lasts about a second or two, then closes after mysql_close())--I don't think this is an issue. Memory usage seems okay. By the end of 7 days, the script's memory usage has peaked at 4MB (out of 16MB max). Typically it's around 3MB. MySQL memory usage doesn't seem to be a constraint. I'm using my- huge.cnf, and the nature of the queries is fairly regular. I would say that the database structure is not an issue (though I could be wrong!)--everything is pretty well normalized and indexed. I'm logging slow queries, non-indexed SELECTs, etc. I'm really not sure what to try next. ps -aux shows MySQL as hogging the CPU, not PHP or Terminal: PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE 342 mysqld 83.3% 16:13:19 33 125 139 435M 4.75M 439M 539M 385 Terminal 4.7% 5:36:35 22 184 251 20.1M 33.5M 28.8M+ 256M 1190 php 4.3% 3:13:33 1 15 148 6.51M 8.15M 12.1M 89.0M 0 kernel_tas 1.3% 2:02:40 47 2 619 5.00M- 0B 219M- 1.26G- It's really strange and strangely consistent. The script will run for a few million cycles, whereupon MySQL suddenly uses 50% more CPU. But for what? I'm looking at tutorials on ktrace and kdump to see what I can learn from MySQL. I wonder if I would have this problem under Linux... ...Rene
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
Jochem Maas wrote: Have you tried stracing it to see what's really happening when the load goes that high? am I correct that that would be done like so?: strace -p process id of php deamon Yep, that's it. You'll probably want to record the output for analysis, but sometimes it's very obvious what's happening. Which doesn't mean it's also easy to fix, but it could give you a clue. /Per Jessen, Zürich -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
M5 wrote: Thanks Jim. No problem. The processing is pretty quick. I don't think that's a bottleneck. It basically just inserts the data into MySQL, not much processing actually. What is the likely hood that two connections would come in at the same time, or at least within close enough time that the second would have to wait for the first to finish its job? --PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- Jim Lucas Perseverance is not a long race; it is many short races one after the other Walter Elliot Some men are born to greatness, some achieve greatness, and some have greatness thrust upon them. Twelfth Night, Act II, Scene V by William Shakespeare -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
stream_socket_server simply listens, stream_socket_accept handles the connection, stream_set_write_buffer and stream_set_blocking help you keep up, especially when combined with stream_get_line, no need to shile forever when you can just: while (is_resource($conn = stream_socket_accept($socket, -1))) while (is_resource($conn) $pkt = stream_get_line($conn, 100, \n)) Key I find though is multithreading, listener thread with stream_socket_server, 2 or 3 stream_socket_accept threads and a pair of new thread spawned to handle each connection (one to read, one to write) (not needed for stateless http style request processing). Nathan M5 wrote: Curiously, would you agree with this guy's comments concerning low-level PHP socket functions vs stream_socket_server() ? If you want a high speed socket server, use the low-level sockets instead (socket_create/bind/listen). The stream_socket_server version appears to have internal fixed 8k buffers that will overflow if you don't keep up by reading. This is a serious problem if you an application that reads the socket for messages and then, say, saves the result in a database. The delay while it is busy processing means you can't read the data in time unless you get involved in muti-threading. With the the low-level functions, the OS quietly buffers TCP/IP packets so there is no problem (tested on Windows XP Professional). (http://www.php.net/manual/en/function.stream-socket-server.php#67837) On 10-Dec-07, at 9:46 PM, Tom Rogers wrote: Hi, Tuesday, December 11, 2007, 10:01:38 AM, you wrote: RF On 10-Dec-07, at 4:42 PM, Tom Rogers wrote: Put a usleep(1000) in the listen while() loop and give the cpu a break. RF Good advice, but I've already been doing that. The thing is, when the RF script first starts up, the CPU rarely exceeds 30%, even when many RF clients (200+) are simultaneously connected and sending data. When a RF few clients are connected, CPU is typically below 10%. Again, it's RF only after 24-48 hours that, all of a sudden, CPU usage increases by RF 40-50%. And it stays high until I stop the script and restart it. RF One question I have though is, is there actually any benefit to using RF mysql_pconnect(), since the script simply loops? My understanding is RF that pconnect only benefits if a script would otherwise be using RF mysql_connect repeatedly--and this script doesn't, since it calls RF mysql_[p]connect() just once, in the start tof execution. RF ...Rene I have found pconnect to be a problem (several years ago) and have never tried it since, it may well be ok now. The most likely cause is memory consumption on long running php scripts, what does top say? I have a script which runs from cron and was hammering the system when it ran and i have had to put the usleep() in the while($result = ..) loop as there are a few thousand rows. Probably bad design but it works and I'm loath to touch it :) One way to solve the memory issue is to have the script started by inetd, slower but more memory friendly. Also have a look at memcached to reduce the load a bit. -- regards, Tom -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
hi Nathan, any chance of a 'full blown' example for all the muppets who want to try and grok this stuff? (bork bork, say I :-)) Nathan Rixham wrote: stream_socket_server simply listens, stream_socket_accept handles the connection, stream_set_write_buffer and stream_set_blocking help you keep up, especially when combined with stream_get_line, no need to shile forever when you can just: while (is_resource($conn = stream_socket_accept($socket, -1))) while (is_resource($conn) $pkt = stream_get_line($conn, 100, \n)) Key I find though is multithreading, listener thread with stream_socket_server, 2 or 3 stream_socket_accept threads and a pair of new thread spawned to handle each connection (one to read, one to write) (not needed for stateless http style request processing). Nathan M5 wrote: Curiously, would you agree with this guy's comments concerning low-level PHP socket functions vs stream_socket_server() ? If you want a high speed socket server, use the low-level sockets instead (socket_create/bind/listen). The stream_socket_server version appears to have internal fixed 8k buffers that will overflow if you don't keep up by reading. This is a serious problem if you an application that reads the socket for messages and then, say, saves the result in a database. The delay while it is busy processing means you can't read the data in time unless you get involved in muti-threading. With the the low-level functions, the OS quietly buffers TCP/IP packets so there is no problem (tested on Windows XP Professional). (http://www.php.net/manual/en/function.stream-socket-server.php#67837) On 10-Dec-07, at 9:46 PM, Tom Rogers wrote: Hi, Tuesday, December 11, 2007, 10:01:38 AM, you wrote: RF On 10-Dec-07, at 4:42 PM, Tom Rogers wrote: Put a usleep(1000) in the listen while() loop and give the cpu a break. RF Good advice, but I've already been doing that. The thing is, when the RF script first starts up, the CPU rarely exceeds 30%, even when many RF clients (200+) are simultaneously connected and sending data. When a RF few clients are connected, CPU is typically below 10%. Again, it's RF only after 24-48 hours that, all of a sudden, CPU usage increases by RF 40-50%. And it stays high until I stop the script and restart it. RF One question I have though is, is there actually any benefit to using RF mysql_pconnect(), since the script simply loops? My understanding is RF that pconnect only benefits if a script would otherwise be using RF mysql_connect repeatedly--and this script doesn't, since it calls RF mysql_[p]connect() just once, in the start tof execution. RF ...Rene I have found pconnect to be a problem (several years ago) and have never tried it since, it may well be ok now. The most likely cause is memory consumption on long running php scripts, what does top say? I have a script which runs from cron and was hammering the system when it ran and i have had to put the usleep() in the while($result = ..) loop as there are a few thousand rows. Probably bad design but it works and I'm loath to touch it :) One way to solve the memory issue is to have the script started by inetd, slower but more memory friendly. Also have a look at memcached to reduce the load a bit. -- regards, Tom -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
Jochem Maas wrote: Nathan Rixham wrote: Key I find though is multithreading, listener thread with stream_socket_server, 2 or 3 stream_socket_accept threads and a pair of new thread spawned to handle each connection (one to read, one to write) (not needed for stateless http style request processing). Nathan hi Nathan, any chance of a 'full blown' example for all the muppets who want to try and grok this stuff? (bork bork, say I :-)) I'd be interested to see how he does the multi-threading in php. Personally I'd always opt for C to write this type of thing, except for perhaps the most simple cases. /Per Jessen, Zürich -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
Per Jessen wrote: Jochem Maas wrote: Nathan Rixham wrote: Key I find though is multithreading, listener thread with stream_socket_server, 2 or 3 stream_socket_accept threads and a pair of new thread spawned to handle each connection (one to read, one to write) (not needed for stateless http style request processing). Nathan hi Nathan, any chance of a 'full blown' example for all the muppets who want to try and grok this stuff? (bork bork, say I :-)) I'd be interested to see how he does the multi-threading in php. Personally I'd always opt for C to write this type of thing, except for perhaps the most simple cases. any chance of an example from you too? /Per Jessen, Zürich -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
Jochem Maas wrote: I'd be interested to see how he does the multi-threading in php. Personally I'd always opt for C to write this type of thing, except for perhaps the most simple cases. any chance of an example from you too? Sure - http://jessen.ch/files/distripg_main.c It can't be compiled, but the pseudo-code goes like this: initialize bind() to address(es) to listen to start a number of threads (=workers) do until terminated poll() for new work if new_work(), accept(), queue it, then wake up the workers. done A worker thread: initialize do until terminated wait for work accept() process work done It wouldn't be too difficult to have threads dynamically started and stopped depending on the amount of work queued up. /Per Jessen, Zürich -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
That makes sense, but I'm not sure I really want to do this, since it's fairly important that Listener continue listening without interruption. I also don't think it's probably necessary, since from what I read, I'm not really pushing the envelope in terms of real load. Right now, I might have max ~250 clients connected, each sending 5-20 kb / day of data. It's not much data, nor many concurrent connections. If Jim's Listener handles 80-85k connections per day, then mine should be able to do 250 concurrently easily, and 2000 cumulative per day without a hitch. Did I mention, I'm on Mac OS X Server 10.4.11? Shouldn't matter, but anyway. On 10-Dec-07, at 5:48 PM, Jochem Maas wrote: Jim Lucas wrote: Tom Rogers wrote: Hi, ... Also, make sure you are not using an array that you are not re- initializing through each iteration of the loop. If the array keeps getting bigger, PHP might $*% on itself. Always re-initialize arrays to clean them up. even then he may still have creeping memory ... in which it might be possible to have a mother process that spawns and watchs a child process .. the child process is the actual deamon, the child could then keep a track of it's own memory usage and then kill itself when it gets too big ... the mother in turn would automatically spawn a new child deamon process upon seeing it's child has committed suicide. does that make sense? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
René Fournier wrote: However, the number of socket clients connecting in the past 3-4 months has steadily increased, and this seems to have exposed (if not created) a strange performance issue with PHP 5.2.4, MySQL 5.0.45 and/or Mac OS X Server 10.4.11. (I say and/or because I am unsure where the problem's cause really lies.) Basically, after the script has been running for a day or so (processing essentially the amount data that used to take two weeks), the CPU usage of the machine goes from 30% (normal) to 80-90%. Have you tried stracing it to see what's really happening when the load goes that high? /Per Jessen, Zürich -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
Per Jessen wrote: René Fournier wrote: However, the number of socket clients connecting in the past 3-4 months has steadily increased, and this seems to have exposed (if not created) a strange performance issue with PHP 5.2.4, MySQL 5.0.45 and/or Mac OS X Server 10.4.11. (I say and/or because I am unsure where the problem's cause really lies.) Basically, after the script has been running for a day or so (processing essentially the amount data that used to take two weeks), the CPU usage of the machine goes from 30% (normal) to 80-90%. Have you tried stracing it to see what's really happening when the load goes that high? am I correct that that would be done like so?: strace -p process id of php deamon Im a little new to this kind of coolness, I found this page which is very helpful: http://www.ibm.com/developerworks/aix/library/au-unix-strace.html I was going to suggest using gdb to figure what the process is doing but figured it wouldn't be handy to have the deamon running via gdb for 48 hours ... silly me didn't grok that you can attach to an existing process (something that ibm article made clear to me) can't wait till my macbook arrives so that I can start to play with this kind of stuff locally ... aka another excuse to justify the aluminium beast ;-) /Per Jessen, Zürich -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
Hello, I have a command-line PHP script--called Listener--that is designed to run indefinitely with a predictable CPU usage and memory footprint. In a nutshell, it's a multi-client socket server that waits for incoming connections, processes incoming data, stores results in a MySQL database, and basically gets on with its life. And it works. No errors or serious problems to speak of. And I've been running it for a couple years on an Xserve dual-G5 2GHz w/ OS X Server 10.4.11). Six months ago, the program would run for days, even a couple weeks, without a hitch. The only reason I would stop the script is for some other purpose, like a software update. However, the number of socket clients connecting in the past 3-4 months has steadily increased, and this seems to have exposed (if not created) a strange performance issue with PHP 5.2.4, MySQL 5.0.45 and/or Mac OS X Server 10.4.11. (I say and/or because I am unsure where the problem's cause really lies.) Basically, after the script has been running for a day or so (processing essentially the amount data that used to take two weeks), the CPU usage of the machine goes from 30% (normal) to 80-90%. This appears to be irrespective of the number of clients connected to the server at the time, but rather the amount of time the script has been running (and therefore cumulative cycles it's operated, data processed, MySQL queries executed, etc.). And the CPU usage stays high, even when the actual load (number of clients) decreases. At this time, if I run top, I get the following info: 22512 mysqld 91.6% 8:22:12 31 106 125 305M+ 3.20M 260M 475M 17104 php 7.2% 81:14.01 115 145 5.08M 7.80M 10.9M- 87.5M 22537 Terminal 6.6% 2:59:59 22 176 240 12.3M 21.2M 18.3M- 236M [...] At first I thought, okay, it's MySQL's fault. Maybe a lot of slow- running queries. But the slow query log is pretty clean. So maybe it's a combination of Mac OS X and MySQL and PHP? I Googled for a similar problem, and finally ran across this article: http://www.shawnhogan.com/2005/10/mysql-problems-on-mac-os-x-server.html ...where the author describes a very similar CPU usage pattern. I tried his suggested fixes and they seemed to have helped a little (or maybe it's my wishful thinking--hard to tell), since the high CPU load issue doesn't appear to happen as soon... But it still happens. Anyway, I'm really stumped as to what to do next, where to look, etc. If I stop the script, and restart it (but not the MySQL itself), CPU usage goes back to normal--for about a day or two. The only thing I thought might be connected is how many short-lived PHP child processes Listener creates--around 20-30,000 per day. Sounds higher, but on average it's just one every 2-3 seconds. Anyway, although the number of child processes isn't concurrent, would there be a problem with the number of historical child processes in view of ulimits or kern.maxfilesperproc? Anyway suggestions, tips, or links are much appreciated. Thanks. ...Rene -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
René Fournier wrote: Hello, I have a command-line PHP script--called Listener--that is designed to run indefinitely with a predictable CPU usage and memory footprint. In a nutshell, it's a multi-client socket server that waits for incoming connections, processes incoming data, stores results in a MySQL database, and basically gets on with its life. And it works. No errors or serious problems to speak of. And I've been running it for a couple years on an Xserve dual-G5 2GHz w/ OS X Server 10.4.11). Six months ago, the program would run for days, even a couple weeks, without a hitch. The only reason I would stop the script is for some other purpose, like a software update. However, the number of socket clients connecting in the past 3-4 months has steadily increased, and this seems to have exposed (if not created) a strange performance issue with PHP 5.2.4, MySQL 5.0.45 and/or Mac OS X Server 10.4.11. (I say and/or because I am unsure where the problem's cause really lies.) Basically, after the script has been running for a day or so (processing essentially the amount data that used to take two weeks), the CPU usage of the machine goes from 30% (normal) to 80-90%. This appears to be irrespective of the number of clients connected to the server at the time, but rather the amount of time the script has been running (and therefore cumulative cycles it's operated, data processed, MySQL queries executed, etc.). And the CPU usage stays high, even when the actual load (number of clients) decreases. At this time, if I run top, I get the following info: 22512 mysqld 91.6% 8:22:12 31 106 125 305M+ 3.20M 260M 475M 17104 php 7.2% 81:14.01 115 145 5.08M 7.80M 10.9M- 87.5M 22537 Terminal 6.6% 2:59:59 22 176 240 12.3M 21.2M 18.3M- 236M [...] At first I thought, okay, it's MySQL's fault. Maybe a lot of slow-running queries. But the slow query log is pretty clean. So maybe it's a combination of Mac OS X and MySQL and PHP? I Googled for a similar problem, and finally ran across this article: http://www.shawnhogan.com/2005/10/mysql-problems-on-mac-os-x-server.html ...where the author describes a very similar CPU usage pattern. I tried his suggested fixes and they seemed to have helped a little (or maybe it's my wishful thinking--hard to tell), since the high CPU load issue doesn't appear to happen as soon... But it still happens. Anyway, I'm really stumped as to what to do next, where to look, etc. If I stop the script, and restart it (but not the MySQL itself), CPU usage goes back to normal--for about a day or two. The only thing I thought might be connected is how many short-lived PHP child processes Listener creates--around 20-30,000 per day. Sounds higher, but on average it's just one every 2-3 seconds. Anyway, although the number of child processes isn't concurrent, would there be a problem with the number of historical child processes in view of ulimits or kern.maxfilesperproc? Anyway suggestions, tips, or links are much appreciated. Thanks. ...Rene I have a server that listens like yours does. I get 80k - 85k connections a day to it. When I first started it, I was only getting about 3k of connections aday. Then I upped the listening pattern and it tanked. I noticed that all my mail/web/db connections just sat there. When I investigated, I found that the number of connections to the server was being overloaded. So I increased the kern.maxfilesperproc setting to 32000. All the problems went away. I have about have the horse power you do, running OpenBSD 4.1, and it runs great now as my listener / web / ftp / mail / named / database / spam filter / etc... One question about the listener program, does it maintain a connection to the DB or does it open/close a connection upon each socket connection? If it does the latter, you might look into using a constant connection rather then opening/closing on a per connection basis. -- Jim Lucas Some men are born to greatness, some achieve greatness, and some have greatness thrust upon them. Twelfth Night, Act II, Scene V by William Shakespeare -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
Hi Jim, I have a server that listens like yours does. I get 80k - 85k connections a day to it. When I first started it, I was only getting about 3k of connections aday. Then I upped the listening pattern and it tanked. I noticed that all my mail/web/db connections just sat there. When I investigated, I found that the number of connections to the server was being overloaded. So I increased the kern.maxfilesperproc setting to 32000. All the problems went away. I have about have the horse power you do, running OpenBSD 4.1, and it runs great now as my listener / web / ftp / mail / named / database / spam filter / etc... One question about the listener program, does it maintain a connection to the DB or does it open/close a connection upon each socket connection? If it does the latter, you might look into using a constant connection rather then opening/closing on a per connection basis. Thanks for the reply. (Not many people seem to be doing what I'm doing in the way I'm doing it... so I really appreciate the feedback.) I don't think the kern.maxfilesperproc setting is a problem. I'm currently set to 102400 per Shawn Hogan's advice. And my Listener program only receives ~1000 connections per day. However, each connection involves multiple MySQL queries, often as many as 50 or so per connection each day. So perhaps your second observation applies. MySQL is showing ~ 1,500 connections per hour. And I'm using mysql_connect() in Listener. I will see if using mysql_pconnect(), and reducing the number of connections helps. My.cnf's max connections, presently at 100, may have been too low in view using mysql_connect(). If mysql_pconnect() doesn't improve things, maybe I should bump up max_connections to 500? ..Rene -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
Hi, Tuesday, December 11, 2007, 6:42:18 AM, you wrote: RF Hello, RF I have a command-line PHP script--called Listener--that is designed RF to run indefinitely with a predictable CPU usage and memory RF footprint. In a nutshell, it's a multi-client socket server that RF waits for incoming connections, processes incoming data, stores RF results in a MySQL database, and basically gets on with its life. And RF it works. No errors or serious problems to speak of. And I've been RF running it for a couple years on an Xserve dual-G5 2GHz w/ OS X RF Server 10.4.11). Six months ago, the program would run for days, even RF a couple weeks, without a hitch. The only reason I would stop the RF script is for some other purpose, like a software update. RF However, the number of socket clients connecting in the past 3-4 RF months has steadily increased, and this seems to have exposed (if not RF created) a strange performance issue with PHP 5.2.4, MySQL 5.0.45 RF and/or Mac OS X Server 10.4.11. (I say and/or because I am unsure RF where the problem's cause really lies.) Basically, after the script RF has been running for a day or so (processing essentially the amount RF data that used to take two weeks), the CPU usage of the machine goes RF from 30% (normal) to 80-90%. This appears to be irrespective of the RF number of clients connected to the server at the time, but rather the RF amount of time the script has been running (and therefore cumulative RF cycles it's operated, data processed, MySQL queries executed, etc.). RF And the CPU usage stays high, even when the actual load (number of RF clients) decreases. At this time, if I run top, I get the following RF info: RF 22512 mysqld 91.6% 8:22:12 31 106 125 305M+ 3.20M RF 260M 475M RF 17104 php 7.2% 81:14.01 115 145 5.08M 7.80M RF 10.9M- 87.5M RF 22537 Terminal 6.6% 2:59:59 22 176 240 12.3M 21.2M RF 18.3M- 236M RF [...] RF At first I thought, okay, it's MySQL's fault. Maybe a lot of slow- RF running queries. But the slow query log is pretty clean. So maybe RF it's a combination of Mac OS X and MySQL and PHP? I Googled for a RF similar problem, and finally ran across this article: RF http://www.shawnhogan.com/2005/10/mysql-problems-on-mac-os-x-server.html RF ...where the author describes a very similar CPU usage pattern. I RF tried his suggested fixes and they seemed to have helped a little (or RF maybe it's my wishful thinking--hard to tell), since the high CPU RF load issue doesn't appear to happen as soon... But it still happens. RF Anyway, I'm really stumped as to what to do next, where to look, etc. RF If I stop the script, and restart it (but not the MySQL itself), CPU RF usage goes back to normal--for about a day or two. RF The only thing I thought might be connected is how many short-lived RF PHP child processes Listener creates--around 20-30,000 per day. RF Sounds higher, but on average it's just one every 2-3 seconds. RF Anyway, although the number of child processes isn't concurrent, RF would there be a problem with the number of historical child RF processes in view of ulimits or kern.maxfilesperproc? RF Anyway suggestions, tips, or links are much appreciated. Thanks. RF ...Rene Put a usleep(1000) in the listen while() loop and give the cpu a break. -- regards, Tom -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
On 10-Dec-07, at 4:42 PM, Tom Rogers wrote: Put a usleep(1000) in the listen while() loop and give the cpu a break. Good advice, but I've already been doing that. The thing is, when the script first starts up, the CPU rarely exceeds 30%, even when many clients (200+) are simultaneously connected and sending data. When a few clients are connected, CPU is typically below 10%. Again, it's only after 24-48 hours that, all of a sudden, CPU usage increases by 40-50%. And it stays high until I stop the script and restart it. One question I have though is, is there actually any benefit to using mysql_pconnect(), since the script simply loops? My understanding is that pconnect only benefits if a script would otherwise be using mysql_connect repeatedly--and this script doesn't, since it calls mysql_[p]connect() just once, in the start tof execution. ...Rene -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
Tom Rogers wrote:
Hi,
Tuesday, December 11, 2007, 6:42:18 AM, you wrote:
RF Hello,
Put a usleep(1000) in the listen while() loop and give the cpu a
break.
This makes me think about asking if you have to short of a timeout on your
receiving connection?
What are you using to setup your connection? fsockopen() or
stream_socket_server() ?
here is a snippet of what I have
?php
$conn = mysql_connect(...);
if ( $socket = @stream_socket_server('udp://'.LISTEN_IP.':'.LISTEN_PORT,
$errno, $errstr, STREAM_SERVER_BIND) ) {
while ( true ) {
/* Get the exact same packet again, but remove it from the
buffer this time. */
$buff = stream_socket_recvfrom($socket, 1024, 0, $remote_ip);
# do stuff with your incoming data and mysql connection
}
fclose($socket);
}
mysql_close($conn);
?
I don't have a timeout set on the *_recvfrom() call. I just wait until the
next connection comes in.
You don't need to use mysql_pconnect(), especially if you are using, what in
essence is, a daemon.
Just don't open and close the connection constantly, leave it open.
Also, make sure you are not using an array that you are not re-initializing
through each iteration
of the loop. If the array keeps getting bigger, PHP might $*% on itself.
Always re-initialize
arrays to clean them up.
Hope some of this helps!
--
Jim Lucas
Some men are born to greatness, some achieve greatness,
and some have greatness thrust upon them.
Twelfth Night, Act II, Scene V
by William Shakespeare
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
Jim Lucas wrote: Tom Rogers wrote: Hi, ... Also, make sure you are not using an array that you are not re-initializing through each iteration of the loop. If the array keeps getting bigger, PHP might $*% on itself. Always re-initialize arrays to clean them up. even then he may still have creeping memory ... in which it might be possible to have a mother process that spawns and watchs a child process .. the child process is the actual deamon, the child could then keep a track of it's own memory usage and then kill itself when it gets too big ... the mother in turn would automatically spawn a new child deamon process upon seeing it's child has committed suicide. does that make sense? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
On 10-Dec-07, at 5:20 PM, Jim Lucas wrote:
Tom Rogers wrote:
Hi,
Tuesday, December 11, 2007, 6:42:18 AM, you wrote:
RF Hello,
Put a usleep(1000) in the listen while() loop and give the cpu a
break.
This makes me think about asking if you have to short of a timeout
on your receiving connection?
One second on stream_socket_server(), with a 900 second timeout
stream_socket_accept().
What are you using to setup your connection? fsockopen() or
stream_socket_server() ?
here is a snippet of what I have
[...]
Thanks for sharing your code. Seems pretty similar to mine at first
glance.
I don't have a timeout set on the *_recvfrom() call. I just wait
until the next connection comes in.
You don't need to use mysql_pconnect(), especially if you are
using, what in essence is, a daemon.
Yeah, after thinking about it, that's what I figured. Thanks for
confirming though.
Just don't open and close the connection constantly, leave it open.
Yes, I just open it once at the top of the script. And that's it.
Also, make sure you are not using an array that you are not re-
initializing through each iteration
of the loop. If the array keeps getting bigger, PHP might $*% on
itself. Always re-initialize
arrays to clean them up.
Hope some of this helps!
All good advice. I will check my arrays, although I don't think this
is a problem since I monitor the scripts memory usage with
memory_get_usage() and memory_get_peak_usage(), and it never tops 3MB
(max allocated is 16MB). There are a few little parts to the daemon.
One thing I'm doing that could be problematic is running an include
(); on a couple files each time a socket has new data (this allows me
to adjust the processing logic on the fly without having to start the
script and wait for clients to reconnect)--but I can see this being
expensive in terms of performance and resources. Actually, I wonder
if THAT is not what's starving the script of resources over time--
each fread() involves several includes(); I'll have to look into that...
FWIW, here's the stripped-down skeleton of the server:
As always, constructive criticism is very welcome.
?php
$socket = stream_socket_server(tcp://127.0.0.1:9876, $errno, $errstr);
if ($socket) {
$master[] = $socket;
$read = $master;
$write = $master;
while (1) {
$read = $master;
$write = $master;
$mod_fd = stream_select($read, $_w = NULL, $_e = NULL, 1);
if ($mod_fd === FALSE) {
break;
}
for ($i = 0; $i $mod_fd; ++$i) {
if ($read[$i] === $socket) {// NEW SOCKET
$conn = stream_socket_accept($socket, 900);
$master[] = $conn;
$key_num = array_search($conn, $master, TRUE);
} else {
$sock_data = fread($read[$i], 32768);
if (strlen($sock_data) === 0) { //
CONNECTION GONE
$key_to_del = array_search($read[$i],
$master, TRUE);
fclose($read[$i]);
unset($master[$key_to_del]);
} elseif ($sock_data === FALSE) {
// CONNECTION BROKEN
$key_to_del = array_search($read[$i],
$master, TRUE);
fclose($read[$i]);
unset($master[$key_to_del]);
} else {
// READ INCOMING DATA
// include (somefiles);
// include (somefiles);
// include (somefiles);
// [ ... ]
}
}
}
}
}
?
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
Jochem Maas wrote: Jim Lucas wrote: Tom Rogers wrote: Hi, ... Also, make sure you are not using an array that you are not re-initializing through each iteration of the loop. If the array keeps getting bigger, PHP might $*% on itself. Always re-initialize arrays to clean them up. even then he may still have creeping memory ... in which it might be possible to have a mother process that spawns and watchs a child process .. the child process is the actual deamon, the child could then keep a track of it's own memory usage and then kill itself when it gets too big ... the mother in turn would automatically spawn a new child deamon process upon seeing it's child has committed suicide. does that make sense? it may ... but it's besides the point ... apparently I'm having difficulty even reading the subject of [EMAIL PROTECTED] :-P -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
René Fournier wrote:
FWIW, here's the stripped-down skeleton of the server:
As always, constructive criticism is very welcome.
?php
$socket = stream_socket_server(tcp://127.0.0.1:9876, $errno, $errstr);
if ($socket) {
$master[] = $socket;
$read = $master;
$write = $master;
while (1) {
$read = $master;
$write = $master;
The follow part, I think, is where your problem is.
This line tells the system to wait 1 second and then continue, whether
you have an inbound connection or not.
$mod_fd = stream_select($read, $_w = NULL, $_e = NULL, 1);
Then here you are testing for success or failure of the last call.
if ($mod_fd === FALSE) {
break;
}
Problem, if you don't have a connection, then the will fail constantly.
once every second X (times) the number of connections you have...
for ($i = 0; $i $mod_fd; ++$i) {
if ($read[$i] === $socket) {// NEW SOCKET
$conn = stream_socket_accept($socket, 900);
$master[] = $conn;
$key_num = array_search($conn, $master, TRUE);
} else {
$sock_data = fread($read[$i], 32768);
if (strlen($sock_data) === 0) { // CONNECTION GONE
$key_to_del = array_search($read[$i], $master, TRUE);
fclose($read[$i]);
unset($master[$key_to_del]);
} elseif ($sock_data === FALSE) {// CONNECTION
BROKEN
$key_to_del = array_search($read[$i], $master, TRUE);
fclose($read[$i]);
unset($master[$key_to_del]);
} else {// READ INCOMING
DATA
Here, you are not removing the successful connections from $master, so
it keeps growing on and on...
As for the includes, well, I would turn them into function calls and
then have a different function for the different ways you want the
program to react.
// include (somefiles);
// include (somefiles);
// include (somefiles);
// [ ... ]
}
}
}
}
}
?
I didn't see anything about your DB connections. Are those located in
the includes?
How long, on average, does your processing of the incoming data take?
Because, to me, it looks like you might have a blocking problem with the
in-coming connections. If the initial connection takes too long, then
the following connections might be getting blocked. You might want to
look into pcntl_* functions to do forking if you need it.
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re[2]: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
Hi, Tuesday, December 11, 2007, 10:01:38 AM, you wrote: RF On 10-Dec-07, at 4:42 PM, Tom Rogers wrote: Put a usleep(1000) in the listen while() loop and give the cpu a break. RF Good advice, but I've already been doing that. The thing is, when the RF script first starts up, the CPU rarely exceeds 30%, even when many RF clients (200+) are simultaneously connected and sending data. When a RF few clients are connected, CPU is typically below 10%. Again, it's RF only after 24-48 hours that, all of a sudden, CPU usage increases by RF 40-50%. And it stays high until I stop the script and restart it. RF One question I have though is, is there actually any benefit to using RF mysql_pconnect(), since the script simply loops? My understanding is RF that pconnect only benefits if a script would otherwise be using RF mysql_connect repeatedly--and this script doesn't, since it calls RF mysql_[p]connect() just once, in the start tof execution. RF ...Rene I have found pconnect to be a problem (several years ago) and have never tried it since, it may well be ok now. The most likely cause is memory consumption on long running php scripts, what does top say? I have a script which runs from cron and was hammering the system when it ran and i have had to put the usleep() in the while($result = ..) loop as there are a few thousand rows. Probably bad design but it works and I'm loath to touch it :) One way to solve the memory issue is to have the script started by inetd, slower but more memory friendly. Also have a look at memcached to reduce the load a bit. -- regards, Tom -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
Thanks Jim. Several good points here that I will look into. I've
already moved the include() bits into function calls. (That's simple
thing I should have corrected long ago.) The socket areas though I'm
less sure about how to adjust, since networking programming isn't
something I grok naturally.
On 10-Dec-07, at 9:57 PM, Jim Lucas wrote:
René Fournier wrote:
FWIW, here's the stripped-down skeleton of the server:
As always, constructive criticism is very welcome.
?php
$socket = stream_socket_server(tcp://127.0.0.1:9876, $errno,
$errstr);
if ($socket) {
$master[] = $socket;
$read = $master;
$write = $master; while (1) {
$read = $master;
$write = $master;
The follow part, I think, is where your problem is.
This line tells the system to wait 1 second and then continue,
whether you have an inbound connection or not.
$mod_fd = stream_select($read, $_w = NULL, $_e = NULL, 1);
Then here you are testing for success or failure of the last call.
if ($mod_fd === FALSE) {
break;
}
Problem, if you don't have a connection, then the will fail
constantly.
once every second X (times) the number of connections you have...
But my understanding from the docs (where I used one of the examples
as a template for the script) socket is that it could/would only fail
on startup, that is if it can't perform stream_select() because it's
unable to bind to with stream_socket_server(), or am I wrong?:
If the call fails, it will return FALSE and if the optional errno
and errstr arguments are present they will be set to indicate the
actual system level error that occurred in the system-level socket(),
bind(), and listen() calls. If the value returned in errno is 0 and
the function returned FALSE, it is an indication that the error
occurred before the bind() call. This is most likely due to a problem
initializing the socket. Note that the errno and errstr arguments
will always be passed by reference. (http://www.php.net/manual/en/
function.stream-socket-server.php)
In other wrongs, mod_fd can't return FALSE once the socket_server has
been created and bound to the specified IP... right?
for ($i = 0; $i $mod_fd; ++$i) {
if ($read[$i] === $socket) {// NEW SOCKET
$conn = stream_socket_accept($socket, 900);
$master[] = $conn;
$key_num = array_search($conn, $master, TRUE);
} else {
$sock_data = fread($read[$i], 32768);
if (strlen($sock_data) === 0)
{ // CONNECTION GONE
$key_to_del = array_search($read
[$i], $master, TRUE);
fclose($read[$i]); unset
($master[$key_to_del]);
} elseif ($sock_data ===
FALSE) {// CONNECTION BROKEN
$key_to_del = array_search
($read[$i], $master, TRUE);
fclose($read[$i]);
unset($master[$key_to_del]);
} else {// READ
INCOMING DATA
Here, you are not removing the successful connections from $master,
so it keeps growing on and on...
But only until the connection closes, or no longer blocks (goes
away), in which case the program fcloses that socket and removes it
from master[].
As for the includes, well, I would turn them into function calls
and then have a different function for the different ways you want
the program to react.
Yes, I did that. It helps a bit with CPU.
// include (somefiles);
// include (somefiles);
// include (somefiles);
// [ ... ]
}
}
}
}
}
?
I didn't see anything about your DB connections. Are those located
in the includes?
Just at header.inc, once.
How long, on average, does your processing of the incoming data
take? Because, to me, it looks like you might have a blocking
problem with the in-coming connections. If the initial connection
takes too long, then the following connections might be getting
blocked. You might want to look into pcntl_* functions to do
forking if you need it.
The processing is pretty quick. I don't think that's a bottleneck. It
basically just inserts the data into MySQL, not much processing
actually.
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: Re[2]: [PHP] Command-line PHP script CPU usage goes sky-high, stays there--why?
Curiously, would you agree with this guy's comments concerning low- level PHP socket functions vs stream_socket_server() ? If you want a high speed socket server, use the low-level sockets instead (socket_create/bind/listen). The stream_socket_server version appears to have internal fixed 8k buffers that will overflow if you don't keep up by reading. This is a serious problem if you an application that reads the socket for messages and then, say, saves the result in a database. The delay while it is busy processing means you can't read the data in time unless you get involved in muti-threading. With the the low-level functions, the OS quietly buffers TCP/IP packets so there is no problem (tested on Windows XP Professional). (http://www.php.net/manual/en/function.stream-socket- server.php#67837) On 10-Dec-07, at 9:46 PM, Tom Rogers wrote: Hi, Tuesday, December 11, 2007, 10:01:38 AM, you wrote: RF On 10-Dec-07, at 4:42 PM, Tom Rogers wrote: Put a usleep(1000) in the listen while() loop and give the cpu a break. RF Good advice, but I've already been doing that. The thing is, when the RF script first starts up, the CPU rarely exceeds 30%, even when many RF clients (200+) are simultaneously connected and sending data. When a RF few clients are connected, CPU is typically below 10%. Again, it's RF only after 24-48 hours that, all of a sudden, CPU usage increases by RF 40-50%. And it stays high until I stop the script and restart it. RF One question I have though is, is there actually any benefit to using RF mysql_pconnect(), since the script simply loops? My understanding is RF that pconnect only benefits if a script would otherwise be using RF mysql_connect repeatedly--and this script doesn't, since it calls RF mysql_[p]connect() just once, in the start tof execution. RF ...Rene I have found pconnect to be a problem (several years ago) and have never tried it since, it may well be ok now. The most likely cause is memory consumption on long running php scripts, what does top say? I have a script which runs from cron and was hammering the system when it ran and i have had to put the usleep() in the while($result = ..) loop as there are a few thousand rows. Probably bad design but it works and I'm loath to touch it :) One way to solve the memory issue is to have the script started by inetd, slower but more memory friendly. Also have a look at memcached to reduce the load a bit. -- regards, Tom -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command Line PHP Advice
Hi, you can do this by using unix command 'for'. Please apply the command written below reply me your experiment result. for filename in `ls *.txt`;do ./edit.php $filename var1 var2;done --Nirmalya Angelo Christou [EMAIL PROTECTED] wrote: Hello List I would like some advice from PHP users regarding PHP and the command line. I have a PHP script that does a whole bunch of stuff to a file on my intranet. ./edit.php filename var1 var2 Everything works fine but I need to run it on a list of files - ./edit.php invoice00212.txt var1 var2 ./edit.php invoice00213.txt var1 var2 ./edit.php invoice00214.txt var1 var2 I have a list of several thousand files that will change every month. The list has the filenames and variables, like this so I am half way there I think - invoice00212.txt var1 var2 invoice00213.txt var1 var2 invoice00214.txt var1 var2 My question is how should I incorporate this with my PHP script? I read that it's better to split scripts up into small reusable parts so my plan is to keep the logic out of the edit.php script and simply pass the variables to it using another script. Am I on the right path doing this? I am only a beginner with PHP so I admit I dont really know what Im doing, that is why I am asking for pointers from the PHP Mail List :) Below is my model, however I am unsure how to achieve this or even if it's the best way to do it? open filelist.txt for each line create $filename $var1 $var2 then run ./edit.php $filename $var1 $var2 loop back to the next line end Many thanks in advance, Ang. - Do you Yahoo!? With a free 1 GB, there's more in store with Yahoo! Mail. - Bring words and photos together (easily) with PhotoMail - it's free and works with your Yahoo! Mail.
Re: [PHP] Command Line PHP Advice
Angelo, I am very happy after knowing that it is working. :) --Nirmalya Angelo Christou [EMAIL PROTECTED] wrote: Hello Nirmalya, Thank you for your response. With the help of your reply, I've now got it working! :) Ang. Nirmalya Lahiri [EMAIL PROTECTED] wrote: Hi, you can do this by using unix command 'for'. Please apply the command written below.. reply me your experiment result. for filename in `ls *.txt`;do ./edit.php $filename var1 var2;done --Nirmalya --- Angelo Christou wrote: Hello List I would like some advice from PHP users regarding PHP and the command line. I have a PHP script that does a whole bunch of stuff to a file on my intranet. ./edit.php filename var1 var2 Everything works fine but I need to run it on a list of files - ./edit.php invoice00212.txt var1 var2 ./edit.php invoice00213.txt var1 var2 ./edit.php invoice00214.txt var1 var2 - Bring words and photos together (easily) with PhotoMail - it's free and works with your Yahoo! Mail. - Bring words and photos together (easily) with PhotoMail - it's free and works with your Yahoo! Mail.
[PHP] Command Line PHP Advice
Hello List I would like some advice from PHP users regarding PHP and the command line. I have a PHP script that does a whole bunch of stuff to a file on my intranet. ./edit.php filename var1 var2 Everything works fine but I need to run it on a list of files - ./edit.php invoice00212.txt var1 var2 ./edit.php invoice00213.txt var1 var2 ./edit.php invoice00214.txt var1 var2 I have a list of several thousand files that will change every month. The list has the filenames and variables, like this so I am half way there I think - invoice00212.txt var1 var2 invoice00213.txt var1 var2 invoice00214.txt var1 var2 My question is how should I incorporate this with my PHP script? I read that it's better to split scripts up into small reusable parts so my plan is to keep the logic out of the edit.php script and simply pass the variables to it using another script. Am I on the right path doing this? I am only a beginner with PHP so I admit I dont really know what Im doing, that is why I am asking for pointers from the PHP Mail List :) Below is my model, however I am unsure how to achieve this or even if it's the best way to do it? open filelist.txt for each line create $filename $var1 $var2 then run ./edit.php $filename $var1 $var2 loop back to the next line end Many thanks in advance, Ang. - Do you Yahoo!? With a free 1 GB, there's more in store with Yahoo! Mail.
Re: [PHP] Command Line PHP Advice
Angelo Christou wrote: I read that it's better to split scripts up into small reusable parts so my plan is to keep the logic out of the edit.php script and simply pass the variables to it using another script. Am I on the right path doing this? That is a good ideology, but whether it's best for your script depends on the data. I, personally, would probably split it up, just to keep things clear and separated so when I go back to it later, I can more easily modify it.. I am only a beginner with PHP so I admit I don’t really know what I’m doing, that is why I am asking for pointers from the PHP Mail List :) Below is my model, however I am unsure how to achieve this or even if it's the best way to do it? open filelist.txt for each line create $filename $var1 $var2 then run ./edit.php $filename $var1 $var2 loop back to the next line end That looks good to me. It's not too difficult to jsut throw something together with the file() function. If your file gets really big you may want to look into reading the file line by line with fopen() / fgets(). But if you're going to be running it once a month only, file() should work fine. Many thanks in advance, Ang. - Do you Yahoo!? With a free 1 GB, there's more in store with Yahoo! Mail. Chris -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Command-line php in debian/woody
I am running a Woody server. I'd like to run php scripts from the command line, but I note that the php/php4 executable is not in my PATH. I have php4, version 4.1.2-7.0.1 installed. Is the command-line option available with this version, or do I need to install another package? I understand that the command-line option was not available with earlier versions of php4 I don't want to mix my system up with Sarge - it usually screws everything up when I have tried to do that. I note that php isn't available in the backports. Is there any sort of workaround (short of upgrading the whole system to sarge, which I don't want to do)? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line php in debian/woody
Robert S wrote: I am running a Woody server. I'd like to run php scripts from the command line, but I note that the php/php4 executable is not in my PATH. Should you ask this at a debian list? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line php in debian/woody
Should you ask this at a debian list? I tried . . .no luck. I thought that you php folks might know a bit more about specific versions. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line php in debian/woody
Hi You might want to download the php source and compile the cli binary yourself (Command Line Interface) It takes only little time compared to compiling it as an apache module Once you got thing setup, you can drop the php extension and just put a shebang like this #! /usr/bin/php At the start of the file. Andy On Wednesday 06 April 2005 12:07, Robert S wrote: I am running a Woody server. I'd like to run php scripts from the command line, but I note that the php/php4 executable is not in my PATH. I have php4, version 4.1.2-7.0.1 installed. Is the command-line option available with this version, or do I need to install another package? I understand that the command-line option was not available with earlier versions of php4 I don't want to mix my system up with Sarge - it usually screws everything up when I have tried to do that. I note that php isn't available in the backports. Is there any sort of workaround (short of upgrading the whole system to sarge, which I don't want to do)? -- Registered Linux User Number 379093 -- Feel free to check out these few php utilities that I released under the GPL2 and that are meant for use with a php cli binary: http://www.vlaamse-kern.com/sas/ -- -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line php in debian/woody
Robert S a écrit : I am running a Woody server. I'd like to run php scripts from the command line, but I note that the php/php4 executable is not in my PATH. just install php4-cgi package (apt-get install php4-cgi) and the PHP 4 CLI will be install : /usr/bin/php4 Next time, try to use 'apt-cache search foo' or packages.debian.org to find it yourself have tried to do that. I note that php isn't available in the backports. If you need more recent LAMP stuff on Woody (ex php5) add these lines in your /etc/apt/sources.list: deb http://packages.dotdeb.org ./ deb-src http://sources.dotdeb.org ./ This will use recent Debian packages (backported for Woody) from http://dotdeb.org/ They got a server crash without backups, but the packages are there, and it seems the server will be up again soon Is there any sort of workaround (short of upgrading the whole system to sarge, which I don't want to do)? Debian testing (currently Sarge) dont have security support. So either stick with woody (perhaps with backported packages) -- esp if you're on a production server or choose the DIY (do it yourself) way... -- ie 'manual' compilation, trouble and upgrades :p Ch. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line php in debian/woody
just download the php4-cli deb package and install it, i think this should work, and correct me if it's a stupid idea. and Sarge is ok for me, i'm running a production e-mail server for last 6 months with a heavy load on it, i'm using php4-cli on it for my 'home made' exim4 administration utility, no problem yet!! vk. On Apr 6, 2005 6:40 PM, Robert S [EMAIL PROTECTED] wrote: Should you ask this at a debian list? I tried . . .no luck. I thought that you php folks might know a bit more about specific versions. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- vk. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command-line php in debian/woody
If you need more recent LAMP stuff on Woody (ex php5) add these lines in your /etc/apt/sources.list: deb http://packages.dotdeb.org ./ deb-src http://sources.dotdeb.org ./ This will use recent Debian packages (backported for Woody) from http://dotdeb.org/ That looks like what I'm after They got a server crash without backups, but the packages are there, and it seems the server will be up again soon This does not give one great confidence!! Are they reputable? Debian testing (currently Sarge) dont have security support. So either stick with woody (perhaps with backported packages) -- esp if you're on a production server I am or choose the DIY (do it yourself) way... -- ie 'manual' compilation, trouble and upgrades :p No thanks - willing to do this at home but not at my office!! -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line php....
4.3.2 Ray Hunter wrote: Technically, yes it should however, I think this is a bug...are you running php 5b? -- BigDog - Original Message - From: John Nichel [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, August 05, 2003 9:07 PM Subject: [PHP] Command line php Can someone tell me why php waits for me to input something before it prints out the first line of the code below? Shouldn't it echo out the first line then wait for my input? #!/usr/local/bin/php ?php echo ( Enter MySQL admin username : [root] ); $username = rtrim ( fgets ( STDIN ) ); echo ( \n\n\nThe username is . $username . \n\n ); ? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- By-Tor.com It's all about the Rush http://www.by-tor.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Command line php....
Can someone tell me why php waits for me to input something before it prints out the first line of the code below? Shouldn't it echo out the first line then wait for my input? #!/usr/local/bin/php ?php echo ( Enter MySQL admin username : [root] ); $username = rtrim ( fgets ( STDIN ) ); echo ( \n\n\nThe username is . $username . \n\n ); ? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line php....
Technically, yes it should however, I think this is a bug...are you running php 5b? -- BigDog - Original Message - From: John Nichel [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, August 05, 2003 9:07 PM Subject: [PHP] Command line php Can someone tell me why php waits for me to input something before it prints out the first line of the code below? Shouldn't it echo out the first line then wait for my input? #!/usr/local/bin/php ?php echo ( Enter MySQL admin username : [root] ); $username = rtrim ( fgets ( STDIN ) ); echo ( \n\n\nThe username is . $username . \n\n ); ? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Command line php....
That did it. Thanks. Jason Wong wrote: On Wednesday 06 August 2003 11:11, John Nichel wrote: 4.3.2 Try disabling output buffer in php.ini. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
