RE: shell scripting: help appreciated
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Atom Powers Sent: Monday, July 03, 2006 5:40 PM Cc: freebsd-questions@freebsd.org Subject: Re: shell scripting: help appreciated On 7/3/06, [EMAIL PROTECTED]@mgedv.net [EMAIL PROTECTED] wrote: if i change the line /bin/sh -c subprogram.sh $h_val1 $h_val2 $h_opt1; to echo /bin/sh -c subprogram.sh $h_val1 $h_val2 $h_opt1; the program loops for all records in the foo.conf correctly. if i remove the echo keyword, the sub-script get's executed, but the shell terminates as if there were lesser records in the foo.conf file! try: eval /bin/sh -c subprogram.sh $h_val1 $h_val2 $h_opt1; well, even changing to /bin/sh -c subprogram.sh without passing any parameters (everything is taken now from the environment, the program still stops after reading lesser records from the input file as there are :-( the shell returnvalue is being checked after the subprogram returns and i get all the messages after the subprogram inside the loop, too. so it's not an error-exit or something like that. it's just as if there were lesser records for the input to the loop. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: shell scripting: help appreciated
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Parv Sent: Monday, July 03, 2006 7:50 PM To: [EMAIL PROTECTED]@mgedv.net Cc: freebsd-questions@freebsd.org Subject: Re: shell scripting: help appreciated in message [EMAIL PROTECTED], wrote [EMAIL PROTECTED]@mgedv.net thusly... /bin/sh stops working correctly with a content-controlled do-loop. the shell-script layout is as follows: (it's not that trivial, just to show the meaning) --- /test/foo.conf (originally ~60 lines): test1 testval1optional_testval1 test2 testval2optional_testval2 /test/foo.sh (this is the original loop code): cat $g_dir_etc/compile.lst|\ ... ( some pipeline ) ... while read gh_name gh_src gh_srcdir do set some vars here /bin/sh -c subprogram.sh $h_val1 $h_val2 $h_opt1; do some stuff here done; /test/foo_sub.sh: we do very much stuff in here, like compiling programs etc... --- if i change the line /bin/sh -c subprogram.sh $h_val1 $h_val2 $h_opt1; to echo /bin/sh -c subprogram.sh $h_val1 $h_val2 $h_opt1; the program loops for all records in the foo.conf correctly. if i remove the echo keyword, the sub-script get's executed, but the shell terminates as if there were lesser records in the foo.conf file! i scripted as many debug messages as possible, for every loop they get executed and there are no errors/etc... happening. and interestingly, the execution stops always on the same record! if i comment out some records of my foo.conf, the sub-shell gets executed for more records. So, what is it exactly in the records (and/or values given to subprogram.sh) where the execution stops? it's not depending on the record. the layout of each record is 2 fixed an one variable field of data which is then put together to directory/file-names inside the subprogram. the subprogram just simply should loop for each record inside the controlfile, which is done correctly if i change the subprogram to an just say hello script. if i run the really program, the loop stops (not immediately after calling the subprogram, it just normal finishes the loop) after working on lesser records as there are inside the .conf file. no idea of what's going on. btw., for each record - after the subprogram has successfully worked on it a done flag is set and the subprogram is not called again for this record. looping the main program again and again renders the subprogram to get executed for the other records as well. it looks like, if there is heavy execution inside the subprogram, the main program forgets about some data inside the loop. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
shell scripting: help appreciated
short and weird problem description (from my point of view): /bin/sh stops working correctly with a content-controlled do-loop. the shell-script layout is as follows: (it's not that trivial, just to show the meaning) --- /test/foo.conf (originally ~60 lines): test1 testval1optional_testval1 test2 testval2optional_testval2 /test/foo.sh (this is the original loop code): cat $g_dir_etc/compile.lst|\ sed -e 's-^ *--g'|\ sed -e 's-^ *--g'|\ grep -v '^#'|\ sed -e 's-#.*$--g'|\ grep -v '^$'|\ while read gh_name gh_src gh_srcdir do set some vars here /bin/sh -c subprogram.sh $h_val1 $h_val2 $h_opt1; do some stuff here done; /test/foo_sub.sh: we do very much stuff in here, like compiling programs etc... --- if i change the line /bin/sh -c subprogram.sh $h_val1 $h_val2 $h_opt1; to echo /bin/sh -c subprogram.sh $h_val1 $h_val2 $h_opt1; the program loops for all records in the foo.conf correctly. if i remove the echo keyword, the sub-script get's executed, but the shell terminates as if there were lesser records in the foo.conf file! i scripted as many debug messages as possible, for every loop they get executed and there are no errors/etc... happening. and interestingly, the execution stops always on the same record! if i comment out some records of my foo.conf, the sub-shell gets executed for more records. it seems like something with a buffer or somewhat, but i'm not a debugger/c-guru, so i don't know where to start here... maybe i'm using a wrong technique for looping? some facts: conf-file is readable and doesn't change during execution the scripts also don't change during exec we're running on 6.1-RELEASE with 2GB RAM and a XEON 2.8 no errors/warnings during exec-time from kernel or daemons fs is fine, fsck is happy! everything runs /bin/sh. the whole scripts runs currently ~45min. if i change the subshell to an empty script with just hi i'm here and exit 1; the loop works for all records in the .conf file. any ideas, what the hell is going on here? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: shell scripting: help appreciated
On 7/3/06, [EMAIL PROTECTED]@mgedv.net [EMAIL PROTECTED] wrote: if i change the line /bin/sh -c subprogram.sh $h_val1 $h_val2 $h_opt1; to echo /bin/sh -c subprogram.sh $h_val1 $h_val2 $h_opt1; the program loops for all records in the foo.conf correctly. if i remove the echo keyword, the sub-script get's executed, but the shell terminates as if there were lesser records in the foo.conf file! try: eval /bin/sh -c subprogram.sh $h_val1 $h_val2 $h_opt1; -- -- Perfection is just a word I use occasionally with mustard. --Atom Powers-- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: shell scripting: help appreciated
in message [EMAIL PROTECTED], wrote [EMAIL PROTECTED]@mgedv.net thusly... /bin/sh stops working correctly with a content-controlled do-loop. the shell-script layout is as follows: (it's not that trivial, just to show the meaning) --- /test/foo.conf (originally ~60 lines): test1 testval1optional_testval1 test2 testval2optional_testval2 /test/foo.sh (this is the original loop code): cat $g_dir_etc/compile.lst|\ ... ( some pipeline ) ... while read gh_name gh_src gh_srcdir do set some vars here /bin/sh -c subprogram.sh $h_val1 $h_val2 $h_opt1; do some stuff here done; /test/foo_sub.sh: we do very much stuff in here, like compiling programs etc... --- if i change the line /bin/sh -c subprogram.sh $h_val1 $h_val2 $h_opt1; to echo /bin/sh -c subprogram.sh $h_val1 $h_val2 $h_opt1; the program loops for all records in the foo.conf correctly. if i remove the echo keyword, the sub-script get's executed, but the shell terminates as if there were lesser records in the foo.conf file! i scripted as many debug messages as possible, for every loop they get executed and there are no errors/etc... happening. and interestingly, the execution stops always on the same record! if i comment out some records of my foo.conf, the sub-shell gets executed for more records. So, what is it exactly in the records (and/or values given to subprogram.sh) where the execution stops? - Parv -- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: shell scripting: help appreciated
in message [EMAIL PROTECTED], wrote Parv thusly... in message [EMAIL PROTECTED], wrote [EMAIL PROTECTED]@mgedv.net thusly... By the way [EMAIL PROTECTED]@mgedv.net do something about ... Reporting-MTA: dns; mta9.adelphia.net Arrival-Date: Mon, 3 Jul 2006 13:49:48 -0400 Received-From-MTA: dns; default.chvlva.adelphia.net (69.160.66.115) Final-Recipient: RFC822; [EMAIL PROTECTED] Action: failed Status: 5.2.0 Remote-MTA: dns; mail.mgedv.net (81.223.168.230) Diagnostic-Code: smtp; 521 mta9.adelphia.net[68.168.78.199]: Client host rejected: 550 service denied (20003) - Parv -- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Scripting help
I would like some advice on how to script something that will search directories below a named root for all files ending with a certain file extension. Then, mv or cp them to another location. -- Best regards, Chris The first 90 percent of the task takes 90 percent of the time, the last 10 percent takes the other 90 percent. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Scripting help
install the 'mmv' port. From the description... This is mmv, a program to move/copy/append/link multiple files according to a set of wildcard patterns. The wildcard matches can be reused in forming the target names. You can i.e. move all *.c.or? files to or?.new.*.c by saying 'mmv *.c.or? or=2.new.=1.c' The multiple action is performed safely, i.e. without any unexpected deletion of files due to collisions of target names with existing filenames or with other target names. Furthermore, before doing anything, mmv attempts to detect any errors that would result from the entire set of actions specified and gives the user the choice of either aborting before beginning, or proceeding by avoiding the offending parts. On Thu, 12 May 2005, Chris wrote: I would like some advice on how to script something that will search directories below a named root for all files ending with a certain file extension. Then, mv or cp them to another location. -- Best regards, Chris The first 90 percent of the task takes 90 percent of the time, the last 10 percent takes the other 90 percent. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Scripting help
On Thu, May 12, 2005 at 11:44:49AM -0500, Chris wrote: I would like some advice on how to script something that will search directories below a named root for all files ending with a certain file extension. Finding files is done (unsurprisingly) with find. E.g. to find all .c files under the current directory do: find . -name *.c Find can select files based on many other criteria. See the find manual page for more detail. You can then pipe the output through xargs to do things with the files. Roland -- R.F.Smith (http://www.xs4all.nl/~rsmith/) Please send e-mail as plain text. public key: http://www.xs4all.nl/~rsmith/pubkey.txt pgp2XZjow53it.pgp Description: PGP signature
Re: Scripting help
On Thu, 12 May 2005 11:44:49 -0500
Chris [EMAIL PROTECTED] wrote:
I would like some advice on how to script something that will search
directories below a named root for all files ending with a certain
file extension.
Then, mv or cp them to another location.
--
Best regards,
Chris
Hello,
Try this:
find /your/path -type f -name *.tar -exec cp {} /destination/dir \;
/your/path - put here the root path to operate on
-type f - type f means to search for files
-name *.tar - search for anything (*) ending in .tar (shell pattern)
-exec cp {} /destination/dir \; - execute the command cp file
/destination/dir replacing file with each file found (one at time).
The '\' is to escape the ';' (so it is not interpreted by the shell as a
command separator).
It is also posible to do much more complex functions with 'find'. For
more information see man find.
Hope that helps.
Best Regards,
Ale
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Scripting help
Roland Smith writes: Finding files is done (unsurprisingly) with find. E.g. to find all .c files under the current directory do: find . -name *.c I believe find . -name *.c is prefered, so the wildcard doesn't get mangled by the shell. Robert Huff ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Scripting help
Dear list: I hope think this is not off-topic, and is just about running a script in FBSD to delete specific lines in a flat file based on info in another flat file. My scripting skills are very limited and need help on how to purge my mail lists. We maintain several majordomo mail lists on a Tech magazine site (www.antennex.com) with 60,000 readers. Needless to say, it is real tedious to try and keep the lists up to date (groan). I have scripted a way to capture the bounces (by list) into a plain text file (1 address per line), but haven't figured out how to purge those from the lists. In other words, if the address is found in the bounce list, then delete from the main lists (also one address per line). I'd appreciate suggestions on how to do this. Thanks for any responses. :-) Best regards, Jack L. Stone, Administrator Sage American http://www.sage-american.com [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Scripting help
On 09/01/04 09:19 -0600, Jack L. Stone wrote: Dear list: I hope think this is not off-topic, and is just about running a script in FBSD to delete specific lines in a flat file based on info in another flat file. My scripting skills are very limited and need help on how to purge my mail lists. We maintain several majordomo mail lists on a Tech magazine site (www.antennex.com) with 60,000 readers. Needless to say, it is real tedious to try and keep the lists up to date (groan). I have scripted a way to capture the bounces (by list) into a plain text file (1 address per line), but haven't figured out how to purge those from the lists. In other words, if the address is found in the bounce list, then delete from the main lists (also one address per line). I'd appreciate suggestions on how to do this. One way to to it would be to use grep(1). grep -i -v -f file of deletes master list outfile The resulting outfile will have no instances of the delete list Thats one way to do it. Good Luck, Jason ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Scripting help
On Fri, Jan 09, 2004 at 09:19:37AM -0600, Jack L. Stone wrote: Dear list: I hope think this is not off-topic, and is just about running a script in FBSD to delete specific lines in a flat file based on info in another flat file. My scripting skills are very limited and need help on how to purge my mail lists. We maintain several majordomo mail lists on a Tech magazine site (www.antennex.com) with 60,000 readers. Needless to say, it is real tedious to try and keep the lists up to date (groan). I have scripted a way to capture the bounces (by list) into a plain text file (1 address per line), but haven't figured out how to purge those from the lists. In other words, if the address is found in the bounce list, then delete from the main lists (also one address per line). I'd appreciate suggestions on how to do this. comm(1) is what you need. Try: % sort mailing-list mailing-list.sorted % sort bounce-addresses bounce-addresses.sorted % comm -23 mailing-list.sorted bounce-addresses.sorted mailing-list Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 26 The Paddocks Savill Way PGP: http://www.infracaninophile.co.uk/pgpkey Marlow Tel: +44 1628 476614 Bucks., SL7 1TH UK pgp0.pgp Description: PGP signature
Re: Scripting help
At 10:19 AM 1/9/2004, Jack L. Stone wrote: My scripting skills are very limited and need help on how to purge my mail lists. We maintain several majordomo mail lists on a Tech magazine site (www.antennex.com) with 60,000 readers. Needless to say, it is real tedious to try and keep the lists up to date (groan). I have scripted a way to capture the bounces (by list) into a plain text file (1 address per line), but haven't figured out how to purge those from the lists. In other words, if the address is found in the bounce list, then delete from the main lists (also one address per line). Jack, with that volume of subscribers I think it's a good example of where a database is appropriate. MySQL, postGreSQL etc.. with an application written in Perl or PHP. Oh, that happens to be just the sort of work I do. :) But is a very small effort and would be cleaner imho; maybe triggered via a cron to keep things updated. You'd need an initial load then a simple program to attempt key matches by email from bounces which then delete/write to a report file and email to you so you have an idea of what's going on day to day. I don't know if overhead doing a flat file approach would cause your system noticeable degradation or not. 60K subscribers, assume 100 bounces/day, linear search averages 30K comparisons per search (because on average a key is found 1/2 way through the file, if all the bounces are there which they should be) if you're going line by line so 3M comparisons to run the daily bounce processor w/ my assumption of 100 bounces a day. TBH 3M comparisons, i/o's with a 2ish GH processor, 1GB of ram or whatever... you may not even feel it. Just thinking out loud, sorry. Marty Landman Face 2 Interface Inc 845-679-9387 Sign On Required: Web membership software for your site Make a Website: http://face2interface.com/Home/Demo.shtml ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Scripting help
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Friday 09 January 2004 15:19, Jack L. Stone wrote: My scripting skills are very limited and need help on how to purge my mail lists. We maintain several majordomo mail lists on a Tech magazine site (www.antennex.com) with 60,000 readers. Needless to say, it is real tedious to try and keep the lists up to date (groan). If you migrate to mailman, mailman can do stuff like catch bounces automatically. - -- Cheers, Chris Howells -- [EMAIL PROTECTED], [EMAIL PROTECTED] Web: http://chrishowells.co.uk, PGP ID: 0x33795A2C KDE/Qt/C++/PHP Developer: http://www.kde.org -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQE//s94F8Iu1zN5WiwRAjVAAJ4hbO1C8Bm6iBdvioeibQzOLCfZCACfQA12 knUifRdRCPMt/QtFuyq00KE= =E0IQ -END PGP SIGNATURE- ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Scripting help
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Friday 09 January 2004 15:19, Jack L. Stone wrote: My scripting skills are very limited and need help on how to purge my mail lists. We maintain several majordomo mail lists on a Tech magazine site (www.antennex.com) with 60,000 readers. Needless to say, it is real tedious to try and keep the lists up to date (groan). If you migrate to mailman, mailman can do stuff like catch bounces automatically. - -- Cheers, Chris Howells -- [EMAIL PROTECTED], [EMAIL PROTECTED] Web: http://chrishowells.co.uk, PGP ID: 0x33795A2C KDE/Qt/C++/PHP Developer: http://www.kde.org -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQE//0eOF8Iu1zN5WiwRAoKlAJ9h3klNyNxsCKW+jjMFgueL1UpfuQCeOTE5 wrLKwLnySmhozmrmkaRl8NA= =BHr6 -END PGP SIGNATURE- ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
