Re: Book recommendations (slightly OT)
On 26/07/2011 20:57, Mark Moellering wrote:
I want to automate some tasks, creating directories, file editing, etc.
I was going to pick up a book on shell scripting but wanted to ask the
list if;
A) I am barking up the wrong tree and should use something else.
No -- automating routine tasks is exactly what shell scripting is for.
B) If I am headed in the right direction, what is the best book /
resource to use?
Personally, I wouldn't spend any money on textbooks trying to teach you
shell programming. Not because there aren't any good books available,
but because the free on-line resources are more than adequate to get you
going.
First of all, choose your shell. On FreeBSD I'd say that it's got to be
/bin/sh for programming. This is the POSIX compatible Bourne Shell. If
you write your scripts to the POSIX standard then you'll be able to run
them just about anywhere eg. using bash on a Linux box. The converse is
not true.
You could learn bash -- it is pretty much a de-facto standard nowadays
-- but bash is pretty bloated with lots of interactive usage stuff, and
there's nothing you can't do in POSIX shell that you can in bash. Also,
bash has to be installed from ports, which might not seem like a big
deal (usually it isn't), but it tends to become really quite important
when you're dealing with systems in extremis.
Don't bother trying to use tcsh for programming -- that's not what it is
for. tcsh is great interactively (it's what I use for my login shell),
but a pain in the bum for scripting.
Now, resources for learning how to program in /bin/sh --
* The sh(1) man page is invaluable. It's a really nicely written and
concise description of what sh can do. I'm constantly referring to
this man page when shell scripting.
* Code examples. Copying from what someone else wrote really is the
best way to get ahead. There are many good examples that come with
FreeBSD -- look at the periodic scripts, rc scripts (including from
ports) and things like mergemaster(1). For instance, if you want to
see how to deal with command line arguments, the standard idiom is
very clearly demonstrated in mergemaster.
* On-line resources like http://tldp.org/LDP/abs/html/ (Yes -- this is
all about bash, but there's a lot of overlap with sh)
* Learn about all of those Unixy commands. /bin/sh in many ways is
designed as a means to glue together compiled C programs to achieve a
desired effect. You should be familiar with programs like test(1),
jot(1), comm(1), xargs(1), printf(1), comm(1), sort(1). Not to
mention those stalwarts of shell programming sed(1) and awk(1) --
although each of those is in itself is a programming language about
as complex as pure shell.
Counterintuitively, given the above, the best shell scripts use built-in
shell capabilities rather than calling out to external programmes
wherever possible. eg. Using the variable prefix / suffix selection
operators: ${progname%%*/} has much the same effect as basename(1).
All the usual programming best-practices apply in shell scripting: write
clean, well structured code divided into relatively short functions each
of which has a single specific purpose. Avoid overuse of global
variables and magic side-effects. Prefer clarity over cleverness.
Comment liberally, but make sure your comments add value. Choose
conventions (eg. on variable naming and code formatting) and stick to them.
One other piece of advice -- as a matter of style, try and avoid
interactive behaviour in scripts. If you prompt a user to type in some
value, then it makes it very hard to call your script from another
script. Instead, pass in any values you need using the command line, or
by using environment variables.
Cheers,
Matthew
--
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk Kent, CT11 9PW
signature.asc
Description: OpenPGP digital signature
Re: Book recommendations (slightly OT)
On Tue, Jul 26, 2011 at 9:57 PM, Mark Moellering m...@msen.com wrote: I want to automate some tasks, creating directories, file editing, etc. I was going to pick up a book on shell scripting but wanted to ask the list if; A) I am barking up the wrong tree and should use something else. B) If I am headed in the right direction, what is the best book / resource to use? Most automation can be done with shell scripting, but there are situations where shell won't cut it. Then, you may want to give Expect a try (hint: combine it with netcat a.k.a. nc and other tools). If you don't like its TCL syntax, there's a port to Python in misc/py-pexpect: http://pexpect.sourceforge.net/ Good luck. Thanks in advance Mark Moellering -cpghost. -- Cordula's Web. http://www.cordula.ws/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Book recommendations (slightly OT)
On Tue, 26 Jul 2011, Mark Moellering wrote: I want to automate some tasks, creating directories, file editing, etc. I was going to pick up a book on shell scripting but wanted to ask the list if; A) I am barking up the wrong tree and should use something else. B) If I am headed in the right direction, what is the best book / resource to use? Thanks in advance Mark Moellering I second Matthew's sh recommendation. Doing admin stuff is much much easier if you learn the basics of regular expressions, awk, sed and xargs. Also find. The daily jobs and the scripts in /etc have lots of coding examples. Of course mergermaster and portmaster are the king and queen of sh scripts. Google will yield thousands of simple examples of all the above and more. If you are doing lexical stuff perl is hard to beat. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Book recommendations (slightly OT)
In the following, I cut out anything not needed as context for my response. Where I cut something out, you should assume that I agree with what Matthew Seaman wrote, and have nothing in particular to add to it at this time. The only possible exception is the specific list of resources he suggested for learning shell scripting, but only because I am not personally familiar with all the recommendations and thus am not in a position to comment on them. On Wed, Jul 27, 2011 at 08:45:30AM +0100, Matthew Seaman wrote: On 26/07/2011 20:57, Mark Moellering wrote: I want to automate some tasks, creating directories, file editing, etc. I was going to pick up a book on shell scripting but wanted to ask the list if; A) I am barking up the wrong tree and should use something else. No -- automating routine tasks is exactly what shell scripting is for. Actually, he said automating tasks with no specific reference to *routine* tasks, and I'd say that pretty much anything involving computers is about automating tasks -- especially scripting/programming. As you suggest, though, the more routine these tasks are (particularly as system administration tasks), the more likely they are to be exactly the right time to use shell scripting. First of all, choose your shell. On FreeBSD I'd say that it's got to be /bin/sh for programming. This is the POSIX compatible Bourne Shell. If you write your scripts to the POSIX standard then you'll be able to run them just about anywhere eg. using bash on a Linux box. The converse is not true. You could learn bash -- it is pretty much a de-facto standard nowadays -- but bash is pretty bloated with lots of interactive usage stuff, and there's nothing you can't do in POSIX shell that you can in bash. Also, bash has to be installed from ports, which might not seem like a big deal (usually it isn't), but it tends to become really quite important when you're dealing with systems in extremis. Don't bother trying to use tcsh for programming -- that's not what it is for. tcsh is great interactively (it's what I use for my login shell), but a pain in the bum for scripting. I would say that the Bourne shell (that is, /bin/sh) is the right choice for pretty much *all* shell scripting. If you need more than the Bourne shell, or its POSIX compatible equivalent, you should be using a high level programming language such as Perl or Ruby instead of an interactive shell syntax. More sophisticated shells are fine for interactive use, but should not be relied upon for shell scripting in the vast majority of cases for reasons of portability and consistency. -- Chad Perrin [ original content licensed OWL: http://owl.apotheon.org ] pgpODY3kSLiNG.pgp Description: PGP signature
Re: Book recommendations (slightly OT)
On Tue, Jul 26, 2011 at 2:57 PM, Mark Moellering m...@msen.com wrote: I want to automate some tasks, creating directories, file editing, etc. I was going to pick up a book on shell scripting but wanted to ask the list if; A) I am barking up the wrong tree and should use something else. B) If I am headed in the right direction, what is the best book / resource to use? Thanks in advance Mark Moellering You should check this out, from our friends at Apple: http://developer.apple.com/library/mac/#documentation/OpenSource/Conceptual/ShellScripting/ I haven't gone through it, but I've perused it, and it looks like a good place to start learning. -Brandon ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Book recommendations (slightly OT)
On Tue, Jul 26, 2011 at 7:57 PM, Mark Moellering m...@msen.com wrote: I want to automate some tasks, creating directories, file editing, etc. I was going to pick up a book on shell scripting but wanted to ask the list if; A) I am barking up the wrong tree and should use something else. B) If I am headed in the right direction, what is the best book / resource to use? Thanks in advance Mark, There are many utilities out there and programs, I would recommend visiting http://www.freebsd.org/projects/newbies.html In that page, several references are given, including but not limited to the powerful handbook http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/basics.html A nice short introduction: http://8help.osu.edu/wks/unix_course/ Chris several posts ago recommended me visit a nice page: http://steve-parker.org/sh/eg/directories/ You can look over several pages, there are many out there that you can get without paying $$, and also there's another document by William Shotts that could be of help: http://linuxcommand.org/tlcl.php Don't worry that it says that it is for linux, it can be used on freebsd too!, just be careful if you don't have bash shell, /bin/bash, change to /bin/sh and the script should work, just make sure that commands are not linux specific :) Regards, Antonio ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Book recommendations (slightly OT)
2011/7/26 Mark Moellering m...@msen.com: I want to automate some tasks, creating directories, file editing, etc. I was going to pick up a book on shell scripting but wanted to ask the list if; A) I am barking up the wrong tree and should use something else. B) If I am headed in the right direction, what is the best book / resource to use? Hi, I learn a lot from this http://tldp.org/LDP/abs/html/ it's free :-) Best regards -- Matteo Filippetto http://op83.blogspot.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Book recommendations (slightly OT)
On 26/07/2011 21:57, Mark Moellering wrote: I want to automate some tasks, creating directories, file editing, etc. I was going to pick up a book on shell scripting but wanted to ask the list if; A) I am barking up the wrong tree and should use something else. B) If I am headed in the right direction, what is the best book / resource to use? Thanks in advance Mark Moellering ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org The first thing to do is to define what you want to do with scripting. Most users have now turned to bash, very easy and quite powerful, though it has some specifics you won't find in any other shell. Such as replacing certain simple commands on the shell line by its own internal version, which can be very frustrating. This said it is probably the easiest shell to learn given there are lots and lots of examples, tutorial and users around here. For pure Unix/BSD/Solaris... professional administration, you have to learn tcsh/csh (basically the same thing, tcsh being an improved version). Basically it is a bit like vi. Even if you do not like vi, but want to professionally maintain Unix/BSD/Solaris..., you have to learn it, because one day you will have to log on an old server and vi will be the only modern editor available. Csh/Tcsh will basically be installed on pretty much every computer you might find. And csh can be tricky at time if you only know Bash. On the other hand if you are a user/dev just wanting to automate some of his daily routine, then you can go for pretty much any shell you want. I personally prefer zsh. One shell that is great but you need to be aware of is ksh. The problem of ksh is that it is so different from every other shell that learning it is a bit of a trouble. It is hard to find good example, and it is hard to transcribe ksh scripts and logic unto an other shell. I can only advise you to browse around, look at what every shell has to offer and pick one. Do not hesitate to change if you are not happy. As far as learning a shell goes, well it is more about going for net tutorials and reading man pages over and over again. At first you will be using cat, | and a lot. That is normal, but the only way to progress is to try to use them all as little as possible. (Which generally translates into reading the man page again). Last thing, though it is considered to be a welcome ritual among admins, do backups, lots of backups, and test your scripts with another account that cannot destroy all your files at once. When learning to script you will one day make a stupid mistake, it will be a very simple script and a very stupid mistake. But you will be very happy you have a backup when the worst happens. Classical mistakes involves making a find with exec, but forgetting to target real files only (such as removing all 0 bytes files from a system = say goodbye to /dev, links, sockets etc.) and running a script with a badly set var (like export deluser=FOO; rm -rf /hom/$delusr). Good luck on your learning. Jerome Herman ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
