Re: [PHP] Queuing servers
On Thu, Dec 2, 2010 at 10:10 PM, la...@garfieldtech.com la...@garfieldtech.com wrote: Hi folks. I have a project coming up where I will need to process a bazillion (OK, a few million) records, possibly with multiple steps. (In this case I'm reading data from one data archive into an Apache Solr server.) This is a natural use case for a queue server, I believe, and while the requirements of the project do not dictate a language it makes sense to me to use PHP for the processing code since 1) Other parts of the project will be using it for web-facing logic and 2) It's the language I know best. I'm trying to select a queue server to use. The two I'm investigating in particular are Beanstalkd (http://kr.github.com/beanstalkd/) and Gearman (http://gearman.org/). In this case I do need a reliable queue, even if that means a record gets processed multiple times by accident (which in this use case is fine). Has anyone worked with either of these systems? Any war stories to share, good or bad? Any guidelines on the number of resources we need? I did a presentation at the PHP London meetup in May - my slides with some good Beanstalk points are at: http://abulman.co.uk/2010/05/queues-and-beanstalkd/ Gearman does have the advantage of being pretty much drop-in and work, calling remote PHP functions (or other languages that are setup as workers). With beanstalkd, there is no solid framework in place yet (though there are a couple of things around, I've seen one for CakePHP, and something for Drupal I think). That said, it's not hard to write one, depending on your personal choice of development framework, or not. Both have solid C-based daemons backing them up (which are so much easier to run than the original Perl daemon). I see Beanstalk as more flexible though, the Priorities, TTR limits, tubes and optional delays make for a powerful set of tools within the queue itself. Alister -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Array difficulty
On 31/07/07, Carlton Whitehead [EMAIL PROTECTED] wrote: Couldn't you just do arsort($chance); $lastItem = chance[( count( $chance ) - 1 )]; I tried that earlier, but the problem is: count( $chance ) - 1 ); returns an integer, so I would be asking for something like $chance[1] or $chance[0], neither of which exist in the array. Keep in mind $chance only has keys with string names: http://uk3.php.net/current $chance = array(lowercase = 27, uppercase = 62, integer = 46); arsort($chance); $lastItem = current($chance); echo $lastItem\n\n; // 62 Alister -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Question
On 13/05/07, Robert Cummings [EMAIL PROTECTED] wrote: On Sat, 2007-05-12 at 23:03 -0500, Richard Lynch wrote: You may find this entertaining, and even useful: http://richardlynch.blogspoot.com I'm sure Richard meant http://richardlynch.blogspot.com Unless he's trying to promote bondage and SM :) Well, I found this entertaining. As for useful? I'll say no. Alister -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Selecting a special row from the database
On 04/05/07, Marcelo Wolfgang [EMAIL PROTECTED] wrote:
I'm building a news display page for a website, and since the user has 2
ways to arrive there, I want to know if this is possible:
1) the user arrive at news.php
I will run a query at the db, and get the latest news to be the main one
(full display) and display the others news in a list
2) the user arrived from a link to a specific news to news.php?id=10
It should display the news with id = 10 as the main news and get the
latest ones to display in a list of other news
Here is the code I have so far, I hope it serve as a better explanation
than mine!
?
$newsId = $_GET['id'];
if (isset($newsID)){
$whereClause = 'WHERE auto_id ='.$newsId;
} else {
$whereClause = '';
}
mysql_connect(localhost,$user,$pass) or die (mysql_error());
mysql_select_db ($db_table);
$SQL = SELECT * FROM tb_noticias $whereClause ORDER BY auto_id DESC;
Yep, thats pretty classic.
One thing I would do - assuming $newsId should always be an integer
$whereClause = '';
if (isset($_GET['id']) {
$newsId = intval($_GET['id']);
if ($newsId) // if not 0
$whereClause = 'WHERE auto_id ='.$newsId;
}
.
Alister
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Appending into associative arrays
On 15/04/07, Robert Cummings [EMAIL PROTECTED] wrote: On Sun, 2007-04-15 at 20:36 +0100, Alister Bulman wrote: On 15/04/07, Zoltán Németh [EMAIL PROTECTED] wrote: 2007. 04. 15, vasárnap keltezéssel 21.20-kor Otto Wyss ezt írta: I want to sort directories according there modification time and thought accociative arrays would be perfect. But when I add an element like Better, I think, to put the unique thing - the name, as the index. Two directories may have the same modification time. $dirs[$d] = filemtime($d); Has he even retrieved the directories in sorted order by modification time? If not he still needs to sort. Then he'll need an asort($dirs); They would not have come in any particular order, so you have to sort them for whatever you need anyway. Alister -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] File extension for PHP's serialization format?
On 27/10/06, Stut [EMAIL PROTECTED] wrote: Hamish Lawson wrote: I have a web application (not written in PHP) that can return data in various formats, including JSON and PHP's serialization format. At the moment my URL scheme looks like this: staff/engineering?format=json but I'd like to switch to using a file extension to denote the format: staff/engineering.json Given that file extensions are purely for association purposes and no application I am aware of knows what to do with PHP serialised data, personally I would use .dat. Or, if I wanted it to be more descriptive, something like .phpdata. But to be perfectly frank it doesn't matter. I know of one website (Livejournal) that has /url.../data/rss or /data/atom etc. /data/php, /data/wddx or /data/json are pretty obvious extensions to that. That said, since all are them are designed for a script to consume, and not a person, I'd call them entirely 'clean' and obvious enough for the task, though ultimately, it's about having a pretty URL, as the program that reads it won't care what the extension is, if any, and people won't be typing it in regularly anyway. Alister -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] PHP and mySQL getting smashed...
On 17/05/06, Dave Goodchild [EMAIL PROTECTED] wrote: I have a site that is getting 30K+ traffic daily and it is smashing mySQL - any ideas on what to do to make the mysql connections more efficient, or anything in general. No bandwidth issue here, just the server getting killed. ...I may be wrong but using persistent connections (mysql_pconnect) may help. Also, optimise your tables and use the explain command to see how efficient your queries are. And caching things that don't need to looked up form the database right now. Even the fastest server in the world would wilt under a slashdotting, if it had to do a dozen big queries for every page display. Meanwhile a Pentium3 with a decent network can serve hundreds of requests a second of a few static pages. Alister -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Multi-threaded port listener
On 28/04/06, René Fournier [EMAIL PROTECTED] wrote: Anyone find any good tutorials, code samples, etc. on such a thing? Basically, I want to write server (in PHP) that listeners on a particular port, and spins off a thread/process (essentially, execute a separate script) for each incoming connection. There won't be a lot of data to process, but there will be many simultaneous connections— upwards of 1000s of connections (each spun off as seperate threads). it's possible in PHP - but you'll unlikely want to do it as seperate processes, even as threads (besides, PHP doesn't have threads?) Check out http://php.net/stream_select and http://netevil.org/node.php?uuid=427d6d12-c107-9816-7428-27d6d128f5e5 With stream_select and the other stream handling software, there are some guys writing a mail server in PHP. I've got a script opening a dozen outbound connections to an external website to push it data myself, and that saves a lot of time rather than waiting for a connection to come back from fopen/fsockopen Alister -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Common Files
On 16/04/06, Jochem Maas [EMAIL PROTECTED] wrote: Shaun wrote: Hi, We have a dedicated UNIX server running FreeBSD, is it possible to set a directory where we can place a set of common files that can be used by all of our web sites? also check out PEAR (pear.php.net) which does exactly what you asked about, the pear system may give you more/some ideas about how to structure and access you common files. I tend to collect all my common code into a single directory heirarchy, and put a copy into the systems' PEAR directory. Then I can just go require_once 'MyCompany/filename.php'; from anywhere on the system. Until PEAR produces a module called 'MyCompany' (unlikely), it will live there entirely happily. Alister -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Why Should I Use Zend Optimizer?
On 27/03/06, Jonathan Duncan [EMAIL PROTECTED] wrote: I posted this on the Zend.com forums but have not been able to get a response yet. So I decided to ask the people that know. I do not understand the need for an optimizer. What exactly is Zend Optimizer optimizing? If it is changing my code, then how about if I just learn how to code better? Is that all that Zend Optimizer is doing? Making my code better? There is no good reason to use ZO unless you have bought code that requires it, and the code has been encrypted to require it. If you run your own server, you would be far better served with something to actually speed your code, like APC (http://pecl.php.net/apc) or Eaccellerator (http://eaccelerator.net/) that is a compiled-code cache. And then learn how to write good code as well. Alister -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Optimizing Tips
On Thu, 5 Aug 2004 16:17:34 +0530, Vinayakam Murugan [EMAIL PROTECTED] wrote: I am a fresh PHP programmer and am part of a team developing a LAMP based website. The problem we are facing is that when the site was hosted on a local server, it was quite fast. Now that we have to use the internet to get to it, it has become very slow even keeping in mind considerations like connection speed . Any pointers on what how optimizations can be done? How big are the pages you are sending around? How much data are you getting from external sources (like a mysql database or draged in from other sites). How fast is the ilnk from the webserver? There's a lot of good questions you can ask to figure it out. For the first, you can trim the html and images you are sending, and gzip the output (see mod_gzip or http://php.net/ob_gzhandler Alister -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Multipart Email Problem
On Thu, 24 Jun 2004 12:27:17 +0100, Matt MacLeod [EMAIL PROTECTED] wrote: I have a script that sends a multipart email from a php page. The script appears to work fine on my mail client (Mail on Mac OSX) and on an online email reader (mail2web.com). However a colleague using Outlook on Windows 2003 views the whole email (ie the raw code - both the text only and the HTML) and when I sent an email to his hotmail account, hotmail simply displayed a blank page. Sorting out the MIME headers is not easy. So user http://pear.php.net/package/Mail_Mime to do the hard work. Alister -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
