Re: waiting for new files in a directory

2000-12-28 Thread Peter Pentchev
On Thu, Dec 28, 2000 at 12:23:12PM +1300, Dan Langille wrote: On 27 Dec 2000, at 19:56, Peter Pentchev wrote: On Wed, Dec 27, 2000 at 09:16:34AM -0800, Alfred Perlstein wrote: * Dan Langille [EMAIL PROTECTED] [001226 23:50] wrote: My idea is to have a daemon, or something

Re: waiting for new files in a directory

2000-12-28 Thread Dan Langille
On 28 Dec 2000, at 10:50, Peter Pentchev wrote: Hmm. On second thoughts, I wonder if the sleep/opendir method might not work better under temporarily high load - even better than the cron-based one. If a bunch of mails arrive at the same time.. hmm I should play around with kevent to see

Re: waiting for new files in a directory

2000-12-28 Thread Volker Stolz
Am 28. Dec 2000 um 10:33 MET schrieb Dan Langille: What about a daemon signalling a waiting perl script? Is it an issue if the daemon signals the perl script when it's already processing? Could a signal be missed? How about using a FIFO (maybe in /tmp) and let the daemon printf,echo,cat,...

Re: waiting for new files in a directory

2000-12-28 Thread Peter Wemm
Volker Stolz wrote: Am 28. Dec 2000 um 10:33 MET schrieb Dan Langille: What about a daemon signalling a waiting perl script? Is it an issue if the daemon signals the perl script when it's already processing? Could a signal be missed? How about using a FIFO (maybe in /tmp) and let the

Re: waiting for new files in a directory

2000-12-28 Thread Dan Langille
On 28 Dec 2000, at 11:29, Volker Stolz wrote: Am 28. Dec 2000 um 10:33 MET schrieb Dan Langille: What about a daemon signalling a waiting perl script? Is it an issue if the daemon signals the perl script when it's already processing? Could a signal be missed? How about using a FIFO

Re: waiting for new files in a directory

2000-12-28 Thread Peter Pentchev
On Thu, Dec 28, 2000 at 11:36:50PM +1300, Dan Langille wrote: On 28 Dec 2000, at 11:29, Volker Stolz wrote: Am 28. Dec 2000 um 10:33 MET schrieb Dan Langille: What about a daemon signalling a waiting perl script? Is it an issue if the daemon signals the perl script when it's already

Re: waiting for new files in a directory

2000-12-28 Thread Kris Kennaway
On Thu, Dec 28, 2000 at 02:35:19AM -0800, Peter Wemm wrote: This sort of thing is why we added poll(2) and later kqueue(2) support for getting notifications on directory changes.. eg: you can get an event to tell you that a new file "appeared" in your directory. See how the l0pht-watch port

Re: waiting for new files in a directory

2000-12-28 Thread Dag-Erling Smorgrav
What are you guys smoking? Use cron to run a spool scanning job every minute or so, and use a lock file to make sure one doesn't start until the previous one is done. Note that reliable locking is non-trivial in Perl; a quick workaround is to use a lock directory instead (mkdir() will fail if the

Re: waiting for new files in a directory

2000-12-28 Thread Volker Stolz
On Thu, Dec 28, 2000 at 01:35:08PM +0100, Dag-Erling Smorgrav wrote: What are you guys smoking? *shrug* Can you spell "event-driven"? There are ways to do things much more elegantly today (see all the references to kevent()). -- \usepackage[latin1]{inputenc}! Volker Stolz * [EMAIL PROTECTED] *

Re: waiting for new files in a directory

2000-12-28 Thread Peter Pentchev
On Thu, Dec 28, 2000 at 01:35:08PM +0100, Dag-Erling Smorgrav wrote: What are you guys smoking? Use cron to run a spool scanning job every minute or so, and use a lock file to make sure one doesn't start until the previous one is done. Note that reliable locking is non-trivial in Perl; a

Re: waiting for new files in a directory

2000-12-28 Thread Dag-Erling Smorgrav
Volker Stolz [EMAIL PROTECTED] writes: On Thu, Dec 28, 2000 at 01:35:08PM +0100, Dag-Erling Smorgrav wrote: What are you guys smoking? *shrug* Can you spell "event-driven"? There are ways to do things much more elegantly today (see all the references to kevent()). I choose simple and

Re: waiting for new files in a directory

2000-12-28 Thread Peter Pentchev
On Thu, Dec 28, 2000 at 01:44:34PM +0100, Dag-Erling Smorgrav wrote: Volker Stolz [EMAIL PROTECTED] writes: On Thu, Dec 28, 2000 at 01:35:08PM +0100, Dag-Erling Smorgrav wrote: What are you guys smoking? *shrug* Can you spell "event-driven"? There are ways to do things much more

Re: waiting for new files in a directory

2000-12-28 Thread Dag-Erling Smorgrav
Peter Pentchev [EMAIL PROTECTED] writes: On Thu, Dec 28, 2000 at 01:44:34PM +0100, Dag-Erling Smorgrav wrote: Volker Stolz [EMAIL PROTECTED] writes: On Thu, Dec 28, 2000 at 01:35:08PM +0100, Dag-Erling Smorgrav wrote: What are you guys smoking? *shrug* Can you spell "event-driven"?

Re: waiting for new files in a directory

2000-12-27 Thread Mark Murray
Any ideas on how to do this? Any suggestions on the process? Simple lock (like flock(3)) in the perl script. Lock some ${FILE}, and if you can't get the lock, die. The file should contain the PID of the process that holds the lock, so that a cleanerd can kill stuck processes, or so that the

Re: waiting for new files in a directory

2000-12-27 Thread Dan Langille
On 27 Dec 2000, at 10:11, Mark Murray wrote: Any ideas on how to do this? Any suggestions on the process? Simple lock (like flock(3)) in the perl script. Lock some ${FILE}, and if you can't get the lock, die. The file should contain the PID of the process that holds the lock, so that a

Re: waiting for new files in a directory

2000-12-27 Thread Peter Pentchev
On Wed, Dec 27, 2000 at 08:49:51PM +1300, Dan Langille wrote: FreshPorts2 will have a new processing strategy for incoming messages. Each message will be in a separate file in a predetermined directory. As each file arrives, it is processed by a perl script. I want only one instance of

Re: waiting for new files in a directory

2000-12-27 Thread Dan Langille
On 27 Dec 2000, at 12:11, Peter Pentchev wrote: I would do that (and have done it in several projects) using opendir() and readdir(). Open the directory, read entry by entry, when you find a file you want, process it and unlink() it. Get to the end of the dir, sleep, repeat. Thanks for

Re: waiting for new files in a directory

2000-12-27 Thread Dima Dorfman
On 27 Dec 2000, at 10:11, Mark Murray wrote: [use flock(2)] But what part of the solution does flock solve? It solves the problem of finding out whether the Perl script is already running, but as I understood the original posting, this isn't what you were asking. See below. I'm not

Re: waiting for new files in a directory

2000-12-27 Thread Peter Pentchev
On Wed, Dec 27, 2000 at 11:17:47PM +1300, Dan Langille wrote: On 27 Dec 2000, at 12:11, Peter Pentchev wrote: I would do that (and have done it in several projects) using opendir() and readdir(). Open the directory, read entry by entry, when you find a file you want, process it and

Re: waiting for new files in a directory

2000-12-27 Thread Dan Langille
On 27 Dec 2000, at 10:11, Mark Murray wrote: Any ideas on how to do this? Any suggestions on the process? Simple lock (like flock(3)) in the perl script. Lock some ${FILE}, and if you can't get the lock, die. The file should contain the PID of the process that holds the lock, so that a

Re: waiting for new files in a directory

2000-12-27 Thread Mike Bristow
On Wed, Dec 27, 2000 at 12:53:37PM +0200, Peter Pentchev wrote: Btw, anybody reading this discussion - I tried the attached script with #!/usr/bin/perl -wT, and Perl died on the unlink() - "unsafe dependency". What gives? $ man perldiag [snip] Insecure dependency in %s (F)

Re: waiting for new files in a directory

2000-12-27 Thread Peter Pentchev
On Wed, Dec 27, 2000 at 11:09:40AM +, Mike Bristow wrote: On Wed, Dec 27, 2000 at 12:53:37PM +0200, Peter Pentchev wrote: Btw, anybody reading this discussion - I tried the attached script with #!/usr/bin/perl -wT, and Perl died on the unlink() - "unsafe dependency". What gives? $

Re: waiting for new files in a directory

2000-12-27 Thread Peter Pentchev
On Wed, Dec 27, 2000 at 01:18:28PM +0200, Peter Pentchev wrote: [snip..] closedir(D); foreach $fname (@files) { next if (($fname eq ".") || ($fname eq "..")); # more filename vailidity checks go here ^ validity..

Re: waiting for new files in a directory

2000-12-27 Thread Mark Murray
unlock the file The cleaner you mentioned: run it every 15 minutes, compare the date/time on the lockfile, if more than 15 minutes old, grab the PID, and kill the job, remove the lock. Correct. Actually, you can make it a lot better: If the lockfile exists, then kill -0

RE: waiting for new files in a directory

2000-12-27 Thread Koster, K.J.
Dear All, What you'd really want is some kind of message queueing system for this kind of work. What message queueing systems are (non-commercially) available on UNIX systems? Kees Jan You are only young once, but you can stay

Re: waiting for new files in a directory

2000-12-27 Thread Alfred Perlstein
* Dan Langille [EMAIL PROTECTED] [001226 23:50] wrote: My idea is to have a daemon, or something resembling one, sitting on the box watching the directory. When a new file appears, it starts a perl script. This perl script is beyound the scope of my question, but it processes all the

Re: waiting for new files in a directory

2000-12-27 Thread Peter Pentchev
On Wed, Dec 27, 2000 at 09:16:34AM -0800, Alfred Perlstein wrote: * Dan Langille [EMAIL PROTECTED] [001226 23:50] wrote: My idea is to have a daemon, or something resembling one, sitting on the box watching the directory. When a new file appears, it starts a perl script. This perl

Re: waiting for new files in a directory

2000-12-27 Thread Jack Rusher
I was about to write up a group of suggestions that include the notion that you could use kqueue to watch the directory's vnode, you could use Erez's stackable file system code to pass all file creates through a filter, use lpd's spooling mechanism to treat the incoming directory like a print

Re: waiting for new files in a directory

2000-12-27 Thread Dan Langille
On 27 Dec 2000, at 12:53, Peter Pentchev wrote: Something like.. | /usr/bin/perl $HOME/process.pl ~/msgs/$FILE.tmp \ mv ~/msgs/$FILE.tmp ~/msgs/$FILE.cvs Thanks for that. It's helped me solve a procmail problem I was having. The files were 600 instead of 640, so I did this:

Re: waiting for new files in a directory

2000-12-27 Thread Dan Langille
On 27 Dec 2000, at 19:56, Peter Pentchev wrote: On Wed, Dec 27, 2000 at 09:16:34AM -0800, Alfred Perlstein wrote: * Dan Langille [EMAIL PROTECTED] [001226 23:50] wrote: My idea is to have a daemon, or something resembling one, sitting on the box watching the directory. When a new

Re: waiting for new files in a directory

2000-12-27 Thread Dan Langille
On 27 Dec 2000, at 11:25, Jack Rusher wrote: At present the files are created through procmail like this: |/usr/bin/perl $HOME/process_cvs_mail.pl ~/msgs/$FILE ...this fragment tells me that you are in control of the process of creating these files. That is correct. This makes

waiting for new files in a directory

2000-12-26 Thread Dan Langille
FreshPorts2 will have a new processing strategy for incoming messages. Each message will be in a separate file in a predetermined directory. As each file arrives, it is processed by a perl script. I want only one instance of that perl script running at a given time. This is primarily for