Re: Does compression on zfs pool affect performance? .......WAS, Re: Silicon Image SiI 3124 and 3132 RAID controllers
- Original Message From: Dan Nelson dnel...@allantgroup.com In the last episode (Mar 11), Leslie Jensen said: The solution for me was to create a raidz which gave me the same amount of space. Now I wonder, should I enable compression? Will it affect performance? The default lzjb compression is very fast and won't consume much CPU. If you have lots of easily-compressabe data it should improve performance. In informal testing I have found it depends on a lot of factors but is often a net gain. For instance, writes during testing in a VM running FBSD8 w/ ZFS writing to an attached single test drive on my not-terribly-fast XP laptop showed a significant improvement when I enabled compression on an FS, jumping from about 5 MB/s to easily 8+ MB/s. In testing at work on a FBSD8 guest on an ESX3.5 host running on a blade backed by a lightly loaded set of 24 15krpm drives, I found enabling compression didn't change much unless I tossed more than a single processor at it, at which point the gains were more tangible though not spectacular. I would suspect that it depends on the ratio of the speed of your CPU to the speed of your hard disk subsystem. Faster CPUs with slower disks will benefit more, slow CPUs with fast disks may even slow down. Obviously, what constitutes Fast and Slow is the big sticking point. :) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: sftp from home wireless box to work - get is much faster that put
From: Anton Shterenlikht me...@bristol.ac.uk To: Vincent Hoffman vi...@unsane.co.uk Cc: freebsd-questions@freebsd.org Sent: Tue, February 9, 2010 5:38:25 PM Subject: Re: sftp from home wireless box to work - get is much faster that put On Tue, Feb 09, 2010 at 11:24:48PM +, Vincent Hoffman wrote: On 09/02/2010 23:16, Anton Shterenlikht wrote: I was trying to measure the file transfer rates between my home and my office boxes. Both are 9.0-current. [snip] At home I've wireless, TL-WN851N, using ath(4) driver. [snip] I used sftp(1), which I launch from the home box. So putting (sending) a file is about 5-17 times faster than getting (receiving) it. What is the reason behind this? Just a thought, Since you are in the uk, do you have ADSL at home? If so the upload on ADSL is much lower than the download. yes, probably. It's a Virgin broadband. I guess it's ADSL. Anyway, that's just what I wanted to hear. many thanks anton Isn't that the wrong way around? Put some numbers to it and you'll see. Pretend a 5Mb download and 1 Mb upload at home. To a faster-than-you location, you would download at 5Mb, upload at 1Mb. NOT the other way around as the OP mentions. Now, if your upload was slower, as is mine, a 5 to 1 speed ratio the other way isn't a stretch at all. Here's mine (inexpensive 1 Mb/384Kb ADSL at home to 45 Mb symmetric fiber at work) sftp put output.txt Uploading output.txt to /output.txt output.txt100% 12MB 38.8KB/s 05:15 sftp get output.txt output.txt2 Fetching /output.txt to output.txt2 /output.txt 100% 12MB 102.7KB/s 01:59 sftp 38KB/s up, 102 KB/s down. 3 to 1. Some cable modems do larger asymmetries, like 5 Mb/256 Kb, and that could give you a ratio like that, but only if the work was on a connection like that and you were on something more symmetric. 17:1 is a bit hard to fathom. That's some serious asymmetry on the work end. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Backup and FreeBSD/ZFS
- Original Message From: Svein Skogen (Listmail Account) svein-listm...@stillbilde.net To: freebsd-questions@freebsd.org Sent: Thu, February 4, 2010 12:14:18 PM Subject: Re: Backup and FreeBSD/ZFS On 04.02.2010 17:57, Matthew Seaman wrote: On 04/02/2010 15:35, Svein Skogen (Listmail Account) wrote: Alas, a full backup of the current disk setup takes 4 tapes and ... I really don't feel like staying up one entire night per week to swap tapes (both for the backup and the verify). The autoloader I've got now (8 slot, 1 drive, LTO-3, SAS) works fairly well with the currently installed OS (Windows Storage Server 2008), giving about 60MB/Sec sustained transfer rate. LTO4 tapes are rated at 800--1600GB depending on achievable compression, so they might be big enough on their own. As image formats are already internally compressed, I'd expect them to come in at the low end of that, which might be tight. Worth trying out if you can get a drive on evaluation. A standalone LTO-4 might be a good alternative, if I didn't already have the tapeloader. ;) Some (certainly not all) autoloaders can be upgraded/converted from LTO-3 to LTO-4 for about the same price as a standalone LTO-4. We use a windows based server for backups at work (nothing but a maintenance nightmare, let me tell you), and at home I have only a single-drive tape backup on my FreeBSD box (never a hiccup!) so I haven't been able to test the following, but would dump be able to understand the EOT and just be able to ask for a new one in the autoloader, which should be able to be set up to automatically move a new tape into the drive until it ran out? For what it's worth, I found Amanda unnecessarily complicated for my simple needs at home. I tried Bacula as well and it seemed easier, but not enough to make it worth it. I just dump the stuff I need to back up externally straight to tape on a weekly cron job. They still fit on one tape. :) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: xclip
- Original Message From: Thomas Adam thomas.ada...@gmail.com To: Charles Howse cho...@charter.net Cc: Thomas Adam thomas.ada...@gmail.com; FreeBSD-Questions freebsd-questions@freebsd.org Sent: Fri, January 1, 2010 10:51:25 AM Subject: Re: xclip On Fri, Jan 01, 2010 at 09:48:28AM -0600, Charles Howse wrote: Hi Thomas, thanks for the reply. This is kinda gnarly. I'm using VMware Player on Windows 7, FreeBSD is the guest OS. I have a script that outputs some text that I would like in the clipboard that I can paste into an email in Windows Outlook. Ah right. You'll find that won't work at all. -- Thomas Adam With VMware Workstation, I do something similar by launching the VM but ignoring the console of it. Use PuTTY to connect to the virtual machine via its IP address. From PuTTY, anything on screen is trivially copied to the host's windows clipboard by selecting it with your mouse. How may that work for you? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Last login message
Subject: Last login message When I ssh to my FreeBSD machine, I get something like this: Last login: Thu Dec 3 15:12:40 2009 from 11.22.33.44 Copyright (c) 1980, 1983, 1986, 1988, 1990, 1991, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 7.1-RELEASE-p9 (DAFFY) #0: Thu Dec 3 11:33:28 PST 2009 ..where 11.22.33.44 is an IP address. However, sometimes, in place of an IP address I get a truncated hostname, for example daffy.nerius.co (note the last 'm' missing). I was wondering what controls this, meaning if I get an IP or a hostname, and why it's being truncated. Just a thought - could the truncation be to the length of a full-length IPV4 address... 011.022.033.044 daffy.nerius.co They seem to match in length. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Apache22 + Subversion 1.6.6 = No go.
--- On Thu, 11/19/09, Glen Johnson nel...@verizon.net wrote: 1 I tried chmod -R 777 /home/svn/repos. normal operations [Thu Nov 19 09:36:10 2009] [error] [client 192.168.2.12] (20014)Internal error: Can't open file '/usr/home/svn/repos/default/format': No such file or directory [Thu Nov 19 09:36:10 2009] [error] [client 192.168.2.12] Could not fetch resource information. [500, #0] [Thu Nov 19 09:36:10 2009] [error] [client 192.168.2.12] Could not open the requested SVN filesystem [500, #2] Please post the relevant bits from your httpd.conf where you set up the SVNPath, etc. It looks like maybe the SVNPath directive is pointing to the wrong place. Thanks for your reply. I currently have this info in /usr/local/etc/apache22/Includes/svn.conf. Apache22 loads all the conf files in this directory when httpd.conf is loaded. Location /svn DAV svn SVNParentPath /usr/home/svn/repos SVNListParentPath on SVNPathAuthz off SVNIndexXSLT /data-dist/svnindex.xsl # anonymous first Satisfy Any Require valid-user # authenticating them valid ones AuthType Basic AuthName Subversion Repositories AuthUserFile /usr/home/svn/access/users /Location Here's one of mine that I think I've minimally modded to fit your paths. Could you try dropping that into place and see if it fails? That would at least cut down the places it may go wrong (e.g. SVNParentPath...). You'll need to htpasswd a new user into the new AuthUserFile. Then try the /test and see what happens. Location /test DAV svn SVNPath /usr/home/svn/repos/FIXME_TO_A_SVN_DIR AuthType Basic AuthName Documentation Repository AuthUserFile /usr/home/svn/access/users.new LimitExcept GET PROPFIND OPTIONS REPORT Require valid-user /LimitExcept /Location -Rich ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: conky calendar
From: Roland Smith rsm...@xs4all.nl
Subject: Re: conky calendar
To: PJ af.gour...@videotron.ca
Cc: freebsd-questions@freebsd.org
Date: Saturday, October 10, 2009, 9:27 AM
On Sat, Oct 10, 2009 at 05:01:34AM -0400, PJ wrote:
I'm having a bit of a time with the calendar.sh script I
found on the Net; it doesn't display quite correctly.
It should have brackets around the current date, but I
can't figure out what is not functioning correctly:
#!/bin/sh
cal | awk 'NR2' | sed -e 's/ / /g' -e 's/[^ ] / /g' -e 's/..*/ /'
-e 's/ \('`date | awk '{print $2}'`'\) /\['`date | awk '{print $2}'`'\]/'
Look at the output of the date command:
Sat Oct 10 15:12:39 CEST 2009
Change 'print $2' to 'print $3' to get the numercal date.
Or even simpler: use date +%d instead of date | awk '{print $3}'.
Roland
I could not get it to work until I changed the single quotes in the last -e
expression to double quotes. (This either interactively under csh or as a
script under sh). BTW, using `date +%s` and with an additional minor change to
make the numbers continue to line up ... Oh! This will not fix mis-alignments
on days when it is not the end of the week, I don't think ... anyway.
cal | awk 'NR1' | sed -e 's/ //g' -e 's/[^ ] / /g' -e 's/..*/ /' -e
s/\ `date +%d`/\[`date +%d`\]/
Gives
$ sh newcal.sh
Su Mo Tu We Th Fr Sa
1 2 3
4 5 6 7 8 9 [10]
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31
Now, if you had a space character at the end of each line, you could do
something like ...
cal | awk 'NR1' | sed -e 's/ //g' -e 's/[^ ] / /g' -e 's/..*/ /' -e
s/\ `date +%d`\ /\[`date +%d`\]/
And then it would replace (underscore is space) _8_ with [8] so it would
always line up. You can't do that without the space at the end of the line
because the trailing numbers look like this _17 not _17_. But, fix that,
and you can use the above. That is left as an exercise for the reader.
-Rich
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: / almost out of space just after installation
--- On Sat, 10/10/09, Polytropon free...@edvax.de wrote: From: Polytropon free...@edvax.de Subject: Re: / almost out of space just after installation To: Chad Perrin per...@apotheon.com Cc: freebsd-questions@freebsd.org Date: Saturday, October 10, 2009, 2:04 PM On Sat, 10 Oct 2009 11:36:08 -0600, Chad Perrin per...@apotheon.com wrote: Someone mentioned giving the `home` directory its own partition. I think a separate partition for /usr/home, mounted within /usr, is a great idea. It would help substantially with system rebuilds, backups, and using separate drives for `home`, because that's where the majority of the stuff you want to keep between installs will reside. Basically everything else within /usr (with the possible exception of /usr/local/etc) is just what happens when you install and configure your system in the first place. If you can estimate disk requirements good enough, or simply have huge hard disks that can compensate any requirements, there's no problem giving /home a separate partition. There's no need to put the mountpoint into /usr, because /home could physically exist; in the home in usr setting, /home is just a symlink to /usr/home. Personally, I often put /home on a separate partition, simply because of comfortability. If I can't say enough about how /usr and /home will grow, I go with the default approach. I sometimes even use the one big / setting. One advantage of /home as a separate partition is that you can easily use dump to create a backup - you simply backup the whole partition. You could have a directory, let's say /home/settings, where you keep duplicates of /etc, /usr/local/etc and other files that contain settings you consider worth being backed up. -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... I agree completely. I also go a step farther and put most other things that I consider user data in there. Like Subversion repositories and non-user-specific Samba shares (E.g. public type shares). I do not generally want /tmp on memory, though. While it can be fun and quite a festive thing, I have far too many systems too limited in RAM to want to do this (my current production system at home is 512 MB of RAM, my play box is 256 MB). The only time I can really think I'd want /tmp to be in RAM is if I already had too much RAM for the needs of the box - otherwise, just give me the RAM... While I'm reasonably happy rolling my own FS sizes, I would be even happier if I didn't have to. As long as we're doing the wish list, I'd guess for this (all numbers significantly flexible): Drive 16 GB = keep current layout? Drive 16 and 40 GB = / = 1 GB swap = 1.5x RAM /tmp = 2 GB /var = 2 GB /usr = remaining space Drive 40 GB = / = 1 GB swap = 1.5x RAM /tmp = 2 GB /var = 2 GB /usr = 1/2 of remaining space, min 20 GB, max 35 GB /home = everything else. And, as long as this is a wish list, how about... 1) When I create, I would love to not to *always* have to backspace over like 17 digits every time to type something short like 16G. Can we just make it operate in MB or something instead of blocks? Does anyone need smaller than 1 MB divisions now? 1.1) If it would take a decimal point, I'd be fine with GB, for that matter. (For compatibility, allow either , or . as decimal.) 1.2) Or if there was just a quick key to delete all 14 digits of number of blocks left at once. 2) When I 'auto' size, I end up deleting most except / and swap partition and remaking (it is just habit I 'a'uto before I think, and no harm in it) except the last few times I've done it, as I deleted all the other partitions, / kept expanding from the default (512 MB?) until it was 1.5 GB. So I had to deleted them ALL and start over. Bug or Feature? 3) Ability to resize any partition directly, if there's empty space left. So if I have 30 GB of my 400 GB drive already decided upon, and I decide that I want /var to be 5 GB instead of 2 GB, I would love to be able to just highlight it and press some key to Resize and it would just move the rest of them up to fit. Of course, Just because this is a bike shed doesn't mean I will get upset if any or even all of this is too much to implement and doesn't make it in any revision of sysinstall. It's just a wish list. In fact, I may pull open the code myself... though I've heard it's pretty nasty... -Rich ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: / almost out of space just after installation
From: Polytropon free...@edvax.de Subject: Re: / almost out of space just after installation Date: Saturday, October 10, 2009, 4:00 PM On Sat, 10 Oct 2009 12:28:08 -0700 (PDT), Richard Mahlerwein mahle...@yahoo.com wrote: According to your suggestion: Drive 16 and 40 GB = / = 1 GB swap = 1.5x RAM I know that there was the idea of saying swap = 2 x the maximum of RAM you could put into the box, but is this approach still valid today? Unknown, but since most servers support more RAM than you are likely to put in them*, I think it would make more sense to set swap to 2x the largest _likely_ amount of RAM (assuming the 2x rule IS good in a general sense). I seem to recall the reason for the 2x was a combination of reasons, but it seemed the most important internally was because the memory management routines in place when the rule was created were built to be most effective at that particular ratio. This was many, many years ago, and heaven knows I could be totally wrong ... so some research may be warranted. *The HP DL380 G6s we've been buying now support something like 128 GB. Drive 40 GB = /var = 2 GB There could be a different requirement, especially when someone wants to run a) an anonymous FTP server (/var/ftp subtree) b) database operations (/var/db subtree) and have the /var sizes grow very fast. Of course, there's no problem putting databases and FTP stuff somewhere under /home (which is in /usr in your example). Excellent point. I was trying to stay away from usage patterns, though, and just stick with predetermined items like how much space do I have available?. Once you get past that, you have an order of magnitude more things to consider, IMO. I think the most commonly increased partition would be /var. Again, I think something reasonably simple like being able to delete the last partition (we'll assume /home at the moment), then just resize /var to be bigger, let all the intermediary partitions slide up and then recreating /home to be whats left now would be simple and may work to handle these cases more cleanly. And, as long as this is a wish list, how about... 1) When I create, I would love to not to *always* have to backspace over like 17 digits every time to type something short like 16G. Can we just make it operate in MB or something instead of blocks? There is an easier approach, I'd call it overwrite with first keystroke. This is common for many dialog libraries, such as in Midnight Commander. That would be stellar. I hadn't even realized it but so many things (in all *sorts* of places!) use that method. A quick glance at the code that seems to be responsible for the keystroke handling (/usr/src/gnu/lib/libdialog/lineedit.c) seems to indicate it's fairly stateless - it doesn't seem to know things like if the dialog still has the oroginal input values in it or if you've already typed something. Also, changes here may affect all sorts of things (since it's as far from /usr/src/usr.sbin/sysinstall/ as you can get, tree-wise). Maybe Meta-Backspace (Esc, then Backspace) would be available to erase the whole content of the input field as you suggested in 1.2. This would be fairly easy to implement, I think. Unfortunately, I would feel horrible for implementing something like this when there's so many serious bugs in sysinstall. http://www.freebsd.org/cgi/query-pr-summary.cgi?text=sysinstall I wonder if the delete key, when pressed at the end of the input, would do? Seems like a magic key, but on the other hand, it also seems pretty innocuous. I'm still thinking that using MB or GB as the default might be easier. Maybe this is a nice item for a dialog wishlist for sysinstall. :-) I couldn't agree more. Does anyone really know what the plans for either sysinstall or a replacement is? There's a ton of bugs in it... -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: / almost out of space just after installation
--- On Sat, 10/10/09, RW rwmailli...@googlemail.com wrote: From: RW rwmailli...@googlemail.com Subject: Re: / almost out of space just after installation To: freebsd-questions@freebsd.org Date: Saturday, October 10, 2009, 8:43 PM On Sat, 10 Oct 2009 12:28:08 -0700 (PDT) Richard Mahlerwein mahle...@yahoo.com wrote: The only time I can really think I'd want /tmp to be in RAM is if I already had too much RAM for the needs of the box - otherwise, just give me the RAM... But it wouldn't actually be a ram disk, that's just just a misnomer that people, who ought to know better, are throwing around. It would probably be tmpfs. Correction (or at least correction to precision) noted. I'd still rather use it as RAM the regular way. :) While I'm reasonably happy rolling my own FS sizes, I would be even happier if I didn't have to. As long as we're doing the wish list, I'd guess for this (all numbers significantly flexible): Drive 16 GB = keep current layout? Drive 16 and 40 GB = / = 1 GB swap = 1.5x RAM /tmp = 2 GB /var = 2 GB /usr = remaining space 2 GB each for /var and /tmp is far too high for such small disks, I wouldn't want to squander 4GB like that much below a TB. It's a figure that's hardly ever going to be about right either for /tmp or /var, when it isn't far too big, it's likely to be too small. So, your opinion is that if 768 MB (or 512 MB, or 1G, whatever) isn't enough, then it's likely that 2 GB also isn't enough? That those who need more than the default /var and /tmp often (or usually) need a LOT more? Reasonable, and I am not sure I could disagree with that completely. I was approaching it from perhaps a slightly different tack, though. What I was thinking of was of defaults for people who will use the defaults. Someone running a mail server is unlikely to use the defaults, and you are completely correct that they'd need a lot more space in /var. But, average Joe may just use it for fiddling around with. Maybe one day he'll start fiddling with MySQL or perhaps even trying to partially or completely host his own email. I'd like him, with his 250 GB drive, to have enough space to at least play with that for a while without worrying overly much about running out of room or having to move DB files or something. For that matter, I wonder if the solution for those sorts is to make a 'simple' mode that does swap and one big partition for everything else? Or make 'auto' do that, and let everyone else use their own sizes? Thinking out loud here: What if 'auto' did one big /, and 'advanced' only laid in the partitions without sizes at all, then for each you'd have to just tell it how big to make it. A special option would be on the /home one, which would be to symlink it to /usr/home. Not that this would happen any time soon - that code doesn't look to be easily convertable to somethign like this. Drive 40 GB = / = 1 GB swap = 1.5x RAM /tmp = 2 GB /var = 2 GB /usr = 1/2 of remaining space, min 20 GB, max 35 GB /home = everything else. Having a home directory separate from /usr is often a good idea, but making it part of the default install is a really bad idea IMO. A desktop user with a largish disk may want 98% of it under /home, a server may need next to nothing under /home. The amount needed for /usr also varies enormously. I had been assuming that someone setting up a server was unlikely to accept the default 'a'uto sizes and would have rolled their own. Under the scheme I had above, the desktop user with a large disk - say 1 TB - would have ended up with 1TB - (1 GB / + ~4 GB swap + 2 GB /var + 2 GB /tmp + 35 GB /usr) = about 950 GB in /home. (Or, well, that'd be what, 870MB out of 925MB or something?) A server with that same drive would likely never have had the 'a' key pressed inside disklabel. It's so hard to come-up with sensible values that the only sensible thing to do is leave them on the same partition by default. It's not exactly rocket science to add your own /home partition. I do agree to some extent. On the other hand, what's the 'a'uto key do now? / seems a bit small, notice the OP's subject? I've never had this problem, though... Hmm. All food for thought. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: / almost out of space just after installation
From: Randi Harper ra...@freebsd.org I was thinking that a more acceptable default layout (leaving swap at it's current default size) would be: / = 1GB /var = 2GB /tmp = 2GB Similar enough to what I use for general systems that I vote YES. I'd love to add one more - on a drive bigger than, say, 40 GB, adding a separate /home would be wonderful. Maybe allow up to 20 GB for user, all remaining space allocated to /home? Regardless of the second point, the first point is fine, though. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Automatic chmod
From: Victor Subervi victorsube...@gmail.com Subject: Automatic chmod To: freebsd-questions@freebsd.org Date: Friday, October 9, 2009, 10:19 AM Hi; I have a python script that automatically writes another script. I need to be able to automatically chmod the script so that it will execute. Also, it appears that's not enough, because when I manually chmod the script (775), it throws this error: fopen: Permission denied TIA, V What user are you running this under? Without seeing code, my first guess is that you are trying to open a file you don't have permission to open. The chmod you are doing only affects the script's permissions, not the permissions of the files it may touch. For more, I suggest posting the code itself. -Rich ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Automatic chmod
From: Victor Subervi victorsube...@gmail.com Subject: Automatic chmod To: freebsd-questions@freebsd.org Date: Friday, October 9, 2009, 10:19 AM Hi; I have a python script that automatically writes another script. I need to be able to automatically chmod the script so that it will execute. Also, it appears that's not enough, because when I manually chmod the script (775), it throws this error: fopen: Permission denied TIA, V What user are you running this under? Without seeing code, my first guess is that you are trying to open a file you don't have permission to open. The chmod you are doing only affects the script's permissions, not the permissions of the files it may touch. For more, I suggest posting the code itself. Sorry, missed the 'script that writes a script that won't run' piece. First solution isn't likely to be the solution (though still could be), but I still suggest posting the code. -Rich ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Automatic chmod
From: Victor Subervi victorsube...@gmail.com
Subject: Re: Automatic chmod
To: mahle...@yahoo.com, freebsd-questions@freebsd.org
Date: Friday, October 9, 2009, 11:20 AM
User? I only have one user on this shared server. Here's the code:
#!/usr/local/bin/python
import cgitb; cgitb.enable()
import MySQLdb
import cgi
import sys,os
sys.path.append(os.getcwd())
from login import login
user, passwd, db, host = login()
form = cgi.FieldStorage()
picid = int(form['id'].value)
x = int(form['x'].value)
pics = {1:'pic1',2:'pic2',3:'pic3',4:'pic4',5:'pic5',6:'pic6'}
pic = pics[x]
db = MySQLdb.connect(host=host, user=user, passwd=passwd, db=db)
cursor= db.cursor()
sql = select + pic + from productsX where id=' + str(picid) + ';
cursor.execute(sql)
content = cursor.fetchall()[0][0].tostring()
cursor.close()
print '''Content-Type: text/plain
Content-Encoding: base64
'''
print
print content.encode('base64')
I finally got to where I could test this. I'm no Python expert (in fact, this
was the first time I've touched it), but your code, with heavy modifications to
slim it to something that can run on my system, seems to be mostly OK. Here's
the code I ended up with:
**
#!/usr/local/bin/python
import cgitb; cgitb.enable()
import MySQLdb
import cgi
import sys,os
sys.path.append(os.getcwd())
user=root
passwd=
db=mysql
host=localhost
form = cgi.FieldStorage()
db = MySQLdb.connect(host=host, user=user, passwd=passwd, db=db)
cursor= db.cursor()
sql = select User from user;
cursor.execute(sql)
content = cursor.fetchall()
cursor.close()
print '''Content-Type: text/plain
Content-Encoding: base64
'''
print
print content
**
That all seems to work as I would expect and gives not unreasonable output.
Not that I know it's correct or what's needed, but it seems to print what you'd
think it would.
Can you try running a test script that does, Oh, say, something like the below
to see if it works? (AGAIN, I don't know python AND I'm not testing this, just
hand-writing it so excuse my code!)
#!/usr/local/bin/python
print '''Content-Type: text/plain
'''
print Hopefully this works
At this point, I really haven't much more to go on. The above may pinpoint
what sort of permissions issue it is. Besides, if it works, you could slowly
add in lines from your previous example until you find the offending line...
Also, If you haven't already done so, you may want to try posting in some
python help forums or something. This doesn't have the feel of a FreeBSD
specific problem, so there's bound to be other Python folks who've hit this and
solved it before.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Dump/Restore?
--- On Sun, 9/13/09, Chris Maness ch...@chrismaness.com wrote: From: Chris Maness ch...@chrismaness.com Subject: Re: Dump/Restore? To: freebsd-questions@freebsd.org Date: Sunday, September 13, 2009, 9:50 PM On Sun, Sep 13, 2009 at 6:15 PM, Chris Maness ch...@chrismaness.com wrote: I level 0 dump of my server. I lost a file that I need back. Is it possible to use restore like tar and explode it into a directory instead of a pristine partition/mount? Or even better, is it possible to just extract a single file without exploding the whole tape dump? Sorry if the question seems stupid. Chris KQ6UP Sorry, I was reading the restore man from my mac, and it was not as clear. The restore does not seem to work from my mac (this is where my backup dumps reside as I have two massive HDs). I guess the mac restore would only work with HFS+ and not UFS. I guess the only way would be to move the massive dump file back over to the FreeBSD server. If the dump was made on the mac, it's highly likely restore will need to be run from the mac. If it was made on freebsd, you'll likely need to run restore from freebsd. Assuming you run it from the appropriate place.. I don't have my Mac handy to check it's man pages, but in FreeBSD I believe in it that it would be #restore -i -f file or #restore -i device Then use 'ls' and 'cd' to find the file you want. In the restore : prompt you can add filename to add it to the restore list. Works with folders, too. extract to finally pull those out. YMMV, so read the docs. I would suspect the Mac has similar options, though can't confirm that at the moment. -Rich ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Dump/Restore?
--- On Mon, 9/14/09, Polytropon free...@edvax.de wrote: From: Polytropon free...@edvax.de Subject: Re: Dump/Restore? To: mahle...@yahoo.com Cc: freebsd-questions@freebsd.org, Chris Maness ch...@chrismaness.com Date: Monday, September 14, 2009, 4:37 PM On Mon, 14 Sep 2009 05:45:01 -0700 (PDT), Richard Mahlerwein mahle...@yahoo.com wrote: In the restore : prompt you can add filename to add it to the restore list. Works with folders, too. Excuse me, just a little terminology note: FreeBSD has directories, not folders. It doesn't have sheets of papers instead of files, too. :-) Pie on my face. I work too much with multiple operating systems. *sigh* BTW, I also work and develop heavily with a (non BSD, non-open source) document imaging and workflow management software, so you probably will, at some point, see me confuse files and sheets of paper. I will not mind a gentle reminder just like the above when I do that . :) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Inconsistency in root partition size
--- On Mon, 9/7/09, jaymax jayma...@gmail.com wrote: From: jaymax jayma...@gmail.com Subject: Re: Inconsistency in root partition size To: freebsd-questions@freebsd.org Date: Monday, September 7, 2009, 7:06 PM Mel Flynn-2 wrote: On Monday 07 September 2009 20:54:51 jaymax wrote: Thanks, will do a new dump, one question - how can one determine that the dumpfile produced is good? -- I've always found a test restore works wonders for the peace of my mind. Two types: cd dest dir restore -Nrf backupfilename To test that it all appears it ought to restore. Then, cd dest dir restore -if backupfilename To confirm a few files restore properly completely. YMMV, and obviously check man restore! -Rich ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Don't let mergemaster beat you down [was Re: Failed update]
--- On Tue, 8/11/09, Wayne Sierke w...@au.dyndns.ws wrote: From: Wayne Sierke w...@au.dyndns.ws Subject: Don't let mergemaster beat you down [was Re: Failed update] To: mahle...@yahoo.com Cc: FreeBSD-Questions freebsd-questions@freebsd.org Date: Tuesday, August 11, 2009, 2:18 AM On Sun, 2009-08-09 at 08:34 -0700, Richard Mahlerwein wrote: I'm redoing the whole process in single user mode. My guess is I goofed something during mergemaster and devd.conf is messed up. (Mergemaster is, undeniably, my least favorite utility). I lost practically all of my 'mergemaster pain' when I adopted the habit of using it with -iUP options: -i Automatically install any files that do not exist in the des- tination directory. -P Preserve files that you replace in /var/tmp/mergemaster/preserved-files-date, or another directory you specify in your mergemaster rc file. -U Attempt to auto upgrade files that have not been user modi- fied. Try it - you'll like it! Wayne Thanks for the tips! I'll have more info in a couple of weeks on this - I'm in training and not at home until thursday, and Friday we leave to visit relatives for another week. I did do the rebuild all in single-user mode before I left Sunday, and now I get another signal 12 core crash, at the *same* virtual address, but in a *different* utility... don't remember which one right now, but it seemed completely unrelated. When I get home in a few weeks, I'm going to run some memory tests and yanking extraneous hardware to try to see if I can make it stop. Anything else you all can think of to try? Thanks, Rich ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Failed update
In upgrading 7.1-PRELEASE to -stable, all seemed fine until I rebooted out of single user mode after doing make installworld and mergemaster. Now I get to devd and it dies. I've copied down what's on screen and typed it here. [snip] starting devd. Fatal trap 12: page fault while in kernel mode. cpu id = 0; apic id = 00 fault virtual address = 0x3030313a fault code = supervisor write, page not present [snip] current process = 355 (devd) What critical step did I miss? Single user mode seems OK, and I can mount the drives (though right now it'll tell me to fsck, since I just hard crashed). I have not tried to cycle this thing much, for fear of trashing something further, but I did at least try one reboot. Same issue. (and yes, I'll be researching this myself, but I thought I'd get this message out there sooner rather than later...) Rich Mahlerwein Mobile: 715-891-7420 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Failed update
--- On Sun, 8/9/09, Richard Mahlerwein mahle...@yahoo.com wrote: From: Richard Mahlerwein mahle...@yahoo.com Subject: Failed update To: FreeBSD-Questions freebsd-questions@freebsd.org Date: Sunday, August 9, 2009, 10:23 AM In upgrading 7.1-PRELEASE to -stable, all seemed fine until I rebooted out of single user mode after doing make installworld and mergemaster. Now I get to devd and it dies. I've copied down what's on screen and typed it here. [snip] starting devd. Fatal trap 12: page fault while in kernel mode. cpu id = 0; apic id = 00 fault virtual address = 0x3030313a fault code = supervisor write, page not present [snip] current process = 355 (devd) What critical step did I miss? Single user mode seems OK, and I can mount the drives (though right now it'll tell me to fsck, since I just hard crashed). I have not tried to cycle this thing much, for fear of trashing something further, but I did at least try one reboot. Same issue. (and yes, I'll be researching this myself, but I thought I'd get this message out there sooner rather than later...) Rich Mahlerwein I'm redoing the whole process in single user mode. My guess is I goofed something during mergemaster and devd.conf is messed up. (Mergemaster is, undeniably, my least favorite utility). I can suffer the system being down for a while, as long as my wife has access to a handful of files. How would I go about mounting a USB stick, if such can be done in single-user mode? I have several sitting around that I could copy stuff to (I'm sure that's easier than pulling off the backup I made last night, since the backup is in dump format on tape, and since my only bsd box is currently not working...) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Failed update
--- On Sun, 8/9/09, Richard Mahlerwein mahle...@yahoo.com wrote: From: Richard Mahlerwein mahle...@yahoo.com Subject: Re: Failed update To: FreeBSD-Questions freebsd-questions@freebsd.org Date: Sunday, August 9, 2009, 11:34 AM --- On Sun, 8/9/09, Richard Mahlerwein mahle...@yahoo.com wrote: From: Richard Mahlerwein mahle...@yahoo.com Subject: Failed update To: FreeBSD-Questions freebsd-questions@freebsd.org Date: Sunday, August 9, 2009, 10:23 AM In upgrading 7.1-PRELEASE to -stable, all seemed fine until I rebooted out of single user mode after doing make installworld and mergemaster. Now I get to devd and it dies. I've copied down what's on screen and typed it here. [snip] starting devd. Fatal trap 12: page fault while in kernel mode. cpu id = 0; apic id = 00 fault virtual address = 0x3030313a fault code = supervisor write, page not present [snip] current process = 355 (devd) What critical step did I miss? Single user mode seems OK, and I can mount the drives (though right now it'll tell me to fsck, since I just hard crashed). I have not tried to cycle this thing much, for fear of trashing something further, but I did at least try one reboot. Same issue. (and yes, I'll be researching this myself, but I thought I'd get this message out there sooner rather than later...) Rich Mahlerwein I'm redoing the whole process in single user mode. My guess is I goofed something during mergemaster and devd.conf is messed up. (Mergemaster is, undeniably, my least favorite utility). I can suffer the system being down for a while, as long as my wife has access to a handful of files. How would I go about mounting a USB stick, if such can be done in single-user mode? I have several sitting around that I could copy stuff to (I'm sure that's easier than pulling off the backup I made last night, since the backup is in dump format on tape, and since my only bsd box is currently not working...) I'll answer my own question: mount -t msdosfs /dev/da0s1 /mnt then copy away at will. I suppose my biggest issue had been wondering if that works in single user mode. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Freebsd-update question
I thought I'd give freebsd-update a try since I run a GENERIC kernel. mobius# freebsd-update -s update.freebsd.org fetch Looking up update.freebsd.org mirrors... none found. Fetching public key from update.freebsd.org... failed. No mirrors remaining, giving up. Thinking perhaps a networking issue, I checked the machine is accessible... mobius# ping update.freebsd.org PING update1.FreeBSD.org (72.21.59.252): 56 data bytes 64 bytes from 72.21.59.252: icmp_seq=0 ttl=51 time=64.557 ms 64 bytes from 72.21.59.252: icmp_seq=1 ttl=51 time=64.580 ms ^C --- update1.FreeBSD.org ping statistics --- 2 packets transmitted, 2 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 64.557/64.569/64.580/0.012 ms It responds with update1, so I tried again using update1.freebsd.org (and several others that I could ping) but it always gives me the same response. A quick check of the handbook and the man pages for both freebsd-update(5) and freebsd-update.conf(8) didn't tell me much about this. I'm sure it's something stupidly simple. Does anyone have some ideas? Rich Mahlerwein ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Freebsd-update question
--- On Sat, 8/8/09, Glen Barber glen.j.bar...@gmail.com wrote: From: Glen Barber glen.j.bar...@gmail.com Subject: Re: Freebsd-update question To: mahle...@yahoo.com Cc: freebsd-questions@freebsd.org Date: Saturday, August 8, 2009, 10:20 AM Hi Richard, On Sat, Aug 8, 2009 at 10:16 AM, Richard Mahlerweinmahle...@yahoo.com wrote: I thought I'd give freebsd-update a try since I run a GENERIC kernel. mobius# freebsd-update -s update.freebsd.org fetch Looking up update.freebsd.org mirrors... none found. Fetching public key from update.freebsd.org... failed. No mirrors remaining, giving up. Thinking perhaps a networking issue, I checked the machine is accessible... mobius# ping update.freebsd.org PING update1.FreeBSD.org (72.21.59.252): 56 data bytes 64 bytes from 72.21.59.252: icmp_seq=0 ttl=51 time=64.557 ms 64 bytes from 72.21.59.252: icmp_seq=1 ttl=51 time=64.580 ms ^C --- update1.FreeBSD.org ping statistics --- 2 packets transmitted, 2 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 64.557/64.569/64.580/0.012 ms It responds with update1, so I tried again using update1.freebsd.org (and several others that I could ping) but it always gives me the same response. A quick check of the handbook and the man pages for both freebsd-update(5) and freebsd-update.conf(8) didn't tell me much about this. I'm sure it's something stupidly simple. Does anyone have some ideas? There's quite a bit of useful information missing. For starters, what is the output of 'uname -a'? -- Glen Barber Sorry, forgot to paste that. mobius# uname -a FreeBSD mobius.mahlerwein.homeip.net 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #0: Fri Sep 5 02:34:20 CDT 2008 r...@mobius.mahlerwein.homeip.net:/usr/obj/usr/src/sys/GENERIC i386 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Freebsd-update question
[random snippage all over] From: Glen Barber glen.j.bar...@gmail.com Subject: Re: Freebsd-update question On Sat, Aug 8, 2009 at 10:25 AM, Richard Mahlerweinmahle...@yahoo.com wrote: mobius# freebsd-update -s update.freebsd.org fetch Looking up update.freebsd.org mirrors... none found. Fetching public key from update.freebsd.org... failed. No mirrors remaining, giving up. mobius# uname -a FreeBSD mobius 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #0: Fri Sep 5 02:34:20 CDT 2008 r...@mobius.mahlerwein.homeip.net:/usr/obj/usr/src/sys/GENERIC i386 I was able to get a mirror using: freebsd-update -r 7.1-PRERELEASE fetch -- Glen Barber mobius# freebsd-update -r 7.1-PRERELEASE fetch Looking up update1.FreeBSD.org mirrors... none found. Fetching public key from update1.FreeBSD.org... failed. No mirrors remaining, giving up. I'm puzzled. It seems like this shouldn't be hard, and google seems to agree. There must be something stupidly simple messed up with my system, or configured incorrectly. What protocols/ports does freebsd-update use? Watching tcpdump while running freebsd-update shows no traffic whatsoever relating to this that I can tell. Makes me wish for a verbose mode on freebsd-update. I fiddled with truss, but that seems harder to interpret than strace on linux (which I'm installing from the ports as I write this). I mean, I'd just update it the old fashioned way, but now I'm curious (and, let's face it, I've not updated it in quite a while, so I suspect another day or two won't hurt any more). I'm about to add a verbose option to freebsd-update and see if I can get it to print out in more detail what it's actually trying to do... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Freebsd-update question
From: RW rwmailli...@googlemail.com Subject: Re: Freebsd-update question To: freebsd-questions@freebsd.org Date: Saturday, August 8, 2009, 11:46 AM On Sat, 8 Aug 2009 07:16:15 -0700 (PDT) Richard Mahlerwein mahle...@yahoo.com wrote: I thought I'd give freebsd-update a try since I run a GENERIC kernel. mobius# freebsd-update -s update.freebsd.org fetch Looking up update.freebsd.org mirrors... none found. Fetching public key from update.freebsd.org... failed. No mirrors remaining, giving up. Can you access the svr record? $ dig +short _http._tcp.update.freebsd.org srv 1 50 80 update5.FreeBSD.org. 2 10 80 update1.FreeBSD.org. 1 35 80 update4.FreeBSD.org. If not try running freebsd-update with servers 4 and 5. mobius# dig +short _http._tcp.update.freebsd.org srv (returns nothing) I tried various of the update servers both via 'dig' and via 'freebsd-update', all returned the same. I attempted using a -s 216.14.97.73 seeing if pointing it to IP would work, but no go - same failure. For what it's worth, making up a -s responds identically (like '-s bleh.a.oorg'). How is freebsd-update resolving addresses? I'm sure this is all user error somewhere along the line. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Freebsd-update question
Thanks for the help, I figured out the [likely] answer and included it at the bottom. --- On Sat, 8/8/09, Richard Mahlerwein mahle...@yahoo.com wrote: From: Richard Mahlerwein mahle...@yahoo.com Subject: Re: Freebsd-update question To: freebsd-questions@freebsd.org Date: Saturday, August 8, 2009, 2:06 PM [random snippage all over] From: Glen Barber glen.j.bar...@gmail.com Subject: Re: Freebsd-update question On Sat, Aug 8, 2009 at 10:25 AM, Richard Mahlerweinmahle...@yahoo.com wrote: mobius# freebsd-update -s update.freebsd.org fetch Looking up update.freebsd.org mirrors... none found. Fetching public key from update.freebsd.org... failed. No mirrors remaining, giving up. mobius# uname -a FreeBSD mobius 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #0: Fri Sep 5 02:34:20 CDT 2008 r...@mobius.mahlerwein.homeip.net:/usr/obj/usr/src/sys/GENERIC i386 I was able to get a mirror using: freebsd-update -r 7.1-PRERELEASE fetch -- Glen Barber mobius# freebsd-update -r 7.1-PRERELEASE fetch Looking up update1.FreeBSD.org mirrors... none found. Fetching public key from update1.FreeBSD.org... failed. No mirrors remaining, giving up. I mean, I'd just update it the old fashioned way, but now I'm curious (and, let's face it, I've not updated it in quite a while, so I suspect another day or two won't hurt any more). I'm about to add a verbose option to freebsd-update and see if I can get it to print out in more detail what it's actually trying to do... Found what I believe to be the answer, and yes it's mostly user error. :) Your test above notwithstanding, it seem the PRERELEASE isn't supported for freebsd-update, or at least it's not signed. I found the source to freebsd-update (it's a shell script) and found that there was a way to specify --debug. So, when I ran... mobius# freebsd-update --debug -s update1.freebsd.org fetch Looking up update1.freebsd.org mirrors... none found. Fetching public key from update1.freebsd.org... fetch: http://update1.freebsd.org/7.1-PRERELEASE/i386/pub.ssl: Not Found failed. That gave me a good lead to follow. Browsing around update1.freebsd.org for a bit leads me to find things under, say, http://update.freebsd.org/7.1-RELEASE/i386/pub.ssl Just not under 7.1-PRERELEASE. I'll update the old fashioned way. NP. I think once I'm off PRERELEASE I'll be able to use freebsd-update. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Freebsd-update question
--- On Sat, 8/8/09, RW rwmailli...@googlemail.com wrote: From: RW rwmailli...@googlemail.com Subject: Re: Freebsd-update question To: freebsd-questions@freebsd.org Date: Saturday, August 8, 2009, 4:59 PM On Sat, 8 Aug 2009 11:14:10 -0700 (PDT) Richard Mahlerwein mahle...@yahoo.com wrote: mobius# dig +short _http._tcp.update.freebsd.org srv (returns nothing) This is typically either due either to broken SRV support in DNS, or the absence of full dns on a private network behind proxies. Perhaps you need to set HTTP_PROXY. I currently have my little westell DSL router set to be my DNS for all my boxes behind it. While a neat little box, it has its issues from time to time. Should I at least point my DNS to the DNS it uses to save an extra relay? Sort of off topic, but it has begun to annoy me that Verizon has decided to redirect requests to domains that don't exist to their search pages. I haven't noticed they are proxying, but they could be if they did so reasonably transparently. And, with hijacking nonexistent domains, they've led me to believe they COULD be doing something goofy like that. Is there any easy way to actually confirm or deny they're doing more goofy stuff? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: VMWare ESX and FBSD 7.2 AMD64 guest
From: John Nielsen li...@jnielsen.net Subject: Re: VMWare ESX and FBSD 7.2 AMD64 guest To: freebsd-questions@freebsd.org Cc: Steve Bertrand st...@ibctech.ca Date: Friday, July 24, 2009, 10:22 AM On Thursday 23 July 2009 19:44:15 Steve Bertrand wrote: This message has a foot that has nearly touched down over the OT borderline. We received an HP Proliant DL360G5 collocation box yesterday that has two processors, and 8GB of memory. All the client wants to use this box for is a single instance of Windows web hosting. Knowing the sites the client wants to aggregate into IIS, I know that the box is far over-rated. Making a long story short, they have agreed to allow us to put their Windows server inside of a virtual-ized container, so we can use the unused horsepower for other vm's (test servers etc). My problem is performance. I'm only willing to make this box virtual if I can keep the abstraction performance loss to 25% (my ultimate goal would be 15%). The following is what I have, followed by my benchmark findings: # 7.2-RELEASE AMD64 FreeBSD 7.2-RELEASE #0: Fri May 1 07:18:07 UTC 2009 r...@driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC Timecounter i8254 frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(R) CPU 5150 @ 2.66GHz (2666.78-MHz K8-class CPU) Origin = GenuineIntel Id = 0x6f6 Stepping = 6 usable memory = 8575160320 (8177 MB) avail memory = 8273620992 (7890 MB) FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 6 cpu3 (AP): APIC ID: 7: Did you give the VM 4 virtual processors as well? How much RAM did it have? What type of storage does the server have? Did the VM just get a .vmdk on VMFS? What version of ESX? Benchmarks: # time make -j4 buildworld (under vmware) 5503.038u 3049.500s 1:15:46.25 188.1% 5877+1961k 3298+586716io 2407pf+0w # time make -j4 buildworld (native) 4777.568u 992.422s 33:02.12 291.1% 6533+2099k 25722+586485io 3487pf+0w Note that the user time is within your 15% margin (if you round to the nearest percent). The system time is what's running away. My guess is that that is largely due to disk I/O and virtualization of same. What you can do to address this depends on what hardware you have. Giving the VM a raw slice/LUN/disk instead of a .vmdk file may improve matters somewhat. If you do use a disk file be sure that it lives on a stripe (or whatever unit is relevant) boundary of the underlying storage. Ways to do that (if any) depend on the storage. Improving the RAID performance, etc. of the storage will improve your benchmark overall, and may or may not narrow the divide. The (virtual) storage driver (mpt IIRC) might have some parameters you could tweak, but I don't know about that off the top of my head. ...both builds were from the exact same sources, and both runs were running with the exact same environment. I was extremely careful to ensure that the environments were exactly the same. I'd appreciate any feedback on tweaks that I can make (either to VMWare, or FreeBSD itself) to make the virtualized environment much more efficient. See above about storage. Similar questions come up periodically; searching the archives if you haven't already may prove fruitful. You may want to try running with different kernel HZ settings for instance. I would also try to isolate the performance of different components and evaluate their importance for your actual intended load. CPU and RAM probably perform like you expect out of the box. Disk and network I/O won't be as close to native speed, but the difference and the impact are variable depending on your hardware and load. A lightly-loaded Windows server is the poster child of virtualization candidates. If your decision is to dedicate the box to Winders or to virtualize and use the excess capacity for something else I would say it's a no-brainer if the cost of ESX isn't a factor (or if ESXi gives you similar performance). If that's already a given and your decision is between running a specific FreeBSD instance on the ESX host or on its own hardware then you're wise to spec out the performance differences. HTH, JN If I recall correctly from ESX (well, VI) training*, there may be a minor scheduling issue affecting things here. If you set up the VM with 4 processors, ESX schedules time on the CPU only when there's 4 things to execute (well, there's another time period it also uses, so even a single thread will get run eventually, but anyway...). The physical instance will run one thread immediately even if there's nothing else waiting, whereas the VM will NOT execute a single thread necessarily immediately. I would retry using perhaps -j8 or even -j12 to make sure the 4 CPUs see plenty of work to do and
RE: VMWare ESX and FBSD 7.2 AMD64 guest
From: Dean Weimer dwei...@orscheln.com Subject: RE: VMWare ESX and FBSD 7.2 AMD64 guest To: freebsd-questions@freebsd.org Cc: st...@ibctech.ca Date: Friday, July 24, 2009, 10:49 AM [snip] servers while running between datacenters. Also keep in mind that as of vSphere 4 (We will be upgrading to this once the new data center is complete, just waiting on the shipment of the racks at this point), VMware does officially support FreeBSD 7.1, so you might want to go with that instead of 7.2, as there may be a performance issue with Awesome news! That's teach me to keep shuttling the nearly-spam I get from VMware into the trash can right away. I'd love to hear about your experience with the upgrade and how things go later. We're looking to do something very similar sometime in the next 6 to 9 months. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: First Traffic not graphing, Now nothing graphs anymore.
--- On Tue, 7/21/09, Leandro Quibem Magnabosco leandr...@gmail.com wrote: From: Leandro Quibem Magnabosco leandr...@gmail.com Subject: First Traffic not graphing, Now nothing graphs anymore. To: freebsd-questions@freebsd.org Date: Tuesday, July 21, 2009, 7:56 AM Hello guys, I have a running cacti on a mid to large environment running on a FreeBSD 7.1. Cacti's version is 0.8.7e and rrdtool is 1.2.23. First I was using 0.8.7d version of cacti but traffic was not graphing and I read somewhere on the net that this was corrected on 0.7.8e. Then I decided to upgrade to 0.8.7e. But since I upgraded, Cacti stopped graphing. You know when you look too much at the same thing and it makes you incapable of coming with new solutions? That is how I feel right now. I've been trying to figure this out for a while now, but I'm probably making a huge noob mistake and I feel blinded for some reason. That is why I need your help. The DEBUG log is available for those who think they can help: http://www.pastebin.org/3373 Thank you in advance, -- Leandro Quibem Magnabosco. leandr...@gmail.com Well, it *seems* your recording data OK so it seems it's only a cosmetic problem with Cacti (e.g. your data is still being collected). Confirm this by checking an rrd: # cd /usr/local/share/cacti/rra/ # /usr/local/bin/rrdtool dump lan_server_2_hdd_free_74.rrd |grep 2009-07-21 You should see a bunch of non-zero and non-NaN numbers in there covering the data it has collected today. Feel free to check a few others, as well, like svn-scsc21_hdd_free_587.rrd. Usually, my biggest problem with upgrading cacti is losing permissions on some or another directory. Often it's that the user apache runs under php can't access the rra folder. What *specific* problem are you having from cacti? Do you see where the graphs should be but they're broken images? Do you see graphs with titles but the data is all zero? -Rich ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Odd behavior after installing a tape drive
From: Polytropon free...@edvax.de Subject: Re: Odd behavior after installing a tape drive To: Tim Judd taj...@gmail.com Cc: mahle...@yahoo.com, freebsd-questions@freebsd.org Date: Monday, July 20, 2009, 12:22 AM On Sun, 19 Jul 2009 21:43:29 -0600, Tim Judd taj...@gmail.com wrote: I'm no expert on tape drives either, but I was sure that losing a SCSI device is a bad thing for SCSI -- think of it as an IDE drive. you don't just go pulling power or data from a running, booted computer. With SCSI, hot plug is usually not that problematic as with modern ATA and SATA on the PC. Anyway, using # camcontrol stop unit before switching off or detaching a SCSI component is often a good idea. All the devices in a computer are on, stays on, until the system shuts down. SCSI allows you to have internal devices outside the computer, connected with a cable. In principle, it doesn't even matter if a hard disk is inside the computer or outside, same for optical disc drives, tape drives, and even scanners. Hot plug has always been a nice feature of SCSI, even 10 or more years ago, where you couldn't imagine something similar in the PC world. The PTY/SCSI subject of your email should be unrelated, but a abruptly missing device is never a positive outcome for an OS. Think about the old removing a mounted USB drive = panic issue we've dealt with for years. Or /dev/mem: device disappeared. :-) I am questioning your reasoning behind turning off a tape drive on a live system. I would never recommend that. As I said, if you do it the SCSI way, it's completely unproblematic. -- Polytropon From Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... Thanks! I'll do that in the future - I was getting the idea that since camcontrol *includes* a stop command I should have done that before pulling the power anyway. :) I still don't know if the two items were related in any way, but I'm not really that worried about it unless it happens again - or at least more than once in a blue moon. I'll have a bit of time this week to test taking it down and back up a few times the correct way and see if it exhibits any of the same behavior. Thanks again! -Rich ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ZFS or UFS for 4TB hardware RAID6?
--- On Tue, 7/14/09, Matthew Seaman m.sea...@infracaninophile.co.uk wrote: From: Matthew Seaman m.sea...@infracaninophile.co.uk Subject: Re: ZFS or UFS for 4TB hardware RAID6? To: mahle...@yahoo.com Cc: Free BSD Questions list freebsd-questions@freebsd.org Date: Tuesday, July 14, 2009, 4:23 AM Richard Mahlerwein wrote: With 4 drives, you could get much, much higher performance out of RAID10 (which is alternatively called RAID0+1 or RAID1+0 depending on the manufacturer Uh -- no. RAID10 and RAID0+1 are superficially similar but quite different things. The main differentiator is resilience to disk failure. RAID10 takes the raw disks in pairs, creates a mirror across each pair, and then stripes across all the sets of mirrors. RAID0+1 divides the raw disks into two equal sets, constructs stripes across each set of disks, and then mirrors the two stripes. Read/Write performance is similar in either case: both perform well for the sort of small randomly distributed IO operations you'ld get when eg. running a RDBMS. However, consider what happens if you get a disk failure. In the RAID10 case *one* of your N/2 mirrors is degraded but the other N-1 drives in the array operate as normal. In the RAID0+1 case, one of the 2 stripes is immediately out of action and the whole IO load is carried by the N/2 drives in the other stripe. Now consider what happens if a second drive should fail. In the RAID10 case, you're still up and running so long as the failed drive is one of the N-2 disks that aren't the mirror pair of the 1st failed drive. In the RAID0+1 case, you're out of action if the 2nd disk to fail is one of the N/2 drives from the working stripe. Or in other words, if two random disks fail in a RAID10, chances are the RAID will still work. If two arbitrarily selected disks fail in a RAID0+1 chances are basically even that the whole RAID is out of action[*]. I don't think I've ever seen a manufacturer say RAID1+0 instead of RAID10, but I suppose all things are possible. My impression was that the 0+1 terminology was specifically invented to make it more visually distinctive -- ie to prevent confusion between '01' and '10'. Cheers, Matthew [*] Astute students of probability will point out that this really only makes a difference for N 4, and for N=4 chances are evens either way that failure of two drives would take out the RAID. -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW --- On Tue, 7/14/09, Matthew Seaman m.sea...@infracaninophile.co.uk wrote: From: Matthew Seaman m.sea...@infracaninophile.co.uk Subject: Re: ZFS or UFS for 4TB hardware RAID6? To: mahle...@yahoo.com Cc: Free BSD Questions list freebsd-questions@freebsd.org Date: Tuesday, July 14, 2009, 4:23 AM Richard Mahlerwein wrote: With 4 drives, you could get much, much higher performance out of RAID10 (which is alternatively called RAID0+1 or RAID1+0 depending on the manufacturer Uh -- no. RAID10 and RAID0+1 are superficially similar but quite different things. The main differentiator is resilience to disk failure. RAID10 takes the raw disks in pairs, creates a mirror across each pair, and then stripes across all the sets of mirrors. RAID0+1 divides the raw disks into two equal sets, constructs stripes across each set of disks, and then mirrors the two stripes. Read/Write performance is similar in either case: both perform well for the sort of small randomly distributed IO operations you'ld get when eg. running a RDBMS. However, consider what happens if you get a disk failure. In the RAID10 case *one* of your N/2 mirrors is degraded but the other N-1 drives in the array operate as normal. In the RAID0+1 case, one of the 2 stripes is immediately out of action and the whole IO load is carried by the N/2 drives in the other stripe. Now consider what happens if a second drive should fail. In the RAID10 case, you're still up and running so long as the failed drive is one of the N-2 disks that aren't the mirror pair of the 1st failed drive. In the RAID0+1 case, you're out of action if the 2nd disk to fail is one of the N/2 drives from the working stripe. Or in other words, if two random disks fail in a RAID10, chances are the RAID will still work. If two arbitrarily selected disks fail in a RAID0+1 chances are basically even that the whole RAID is out of action[*]. I don't think I've ever seen a manufacturer say RAID1+0 instead of RAID10, but I suppose all things are possible. My impression was that the 0+1 terminology was specifically invented to make it more visually distinctive -- ie to prevent confusion between '01' and '10'. Cheers, Matthew
Re: ZFS or UFS for 4TB hardware RAID6?
--- On Sun, 7/12/09, Maxim Khitrov mkhit...@gmail.com wrote: From: Maxim Khitrov mkhit...@gmail.com Subject: ZFS or UFS for 4TB hardware RAID6? To: Free BSD Questions list freebsd-questions@freebsd.org Date: Sunday, July 12, 2009, 11:47 PM Hello all, I'm about to build a new file server using 3ware 9690SA-8E controller and 4x Western Digital RE4-GP 2TB drives in RAID6. It is likely to grow in the future up to 10TB. I may use FreeBSD 8 on this one, since the release will likely be made by the time this server goes into production. The question is a simple one - I have no experience with ZFS and so wanted to ask for recommendations of that versus UFS2. How stable is the implementation and does it offer any benefits in my setup (described below)? All of the RAID6 space will only be used for file storage, accessible by network using NFS and SMB. It may be split into separate partitions, but most likely the entire array will be one giant storage area that is expanded every time another hard drive is added. The OS and all installed apps will be on a separate software RAID1 array. Given that security is more important than performance, what would be your recommended setup and why? - Max Your mileage may vary, but... I would investigate either using more spindles if you want to stick to RAID6, or perhaps using another RAID level if you will be with 4 drives for a while. The reasoning is that there's an overhead with RAID 6 - parity blocks are written to 2 disks, so in a 4 drive combination you have 2 drives with data and 2 with parity. With 4 drives, you could get much, much higher performance out of RAID10 (which is alternatively called RAID0+1 or RAID1+0 depending on the manufacturer and on how accurate they wish to be, and on how they actually implemented it, too). This would also mean 2 usable drives, as well, so you'd have the same space available in RAID10 as your proposed RAID6. I would confirm you can, on the fly, convert from RAID10 to RAID6 after you add more drives. If you can not, then by all means stick with RAID6 now! With 4 1 TB drives (for simpler examples) RAID5 = 3 TB available, 1 TB worth used in parity. Fast reads, slow writes. RAID6 = 2 TB available, 2 TB worth used in parity. Moderately fast reads, slow writes. RAID10 = 2 TB available, 2TB in duplicate copies (easier work than parity calculations). Very fast reads, moderately fast writes. When you switch to, say, 8 drives, the numbers start to change a bit. RAID5 = 7TB available, 1 lost. RAID6 = 6TB available, 2 lost. RAID10 = 4TB available, 4 lost. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ZFS or UFS for 4TB hardware RAID6?
--- On Mon, 7/13/09, Richard Mahlerwein mahle...@yahoo.com wrote: From: Richard Mahlerwein mahle...@yahoo.com Subject: Re: ZFS or UFS for 4TB hardware RAID6? To: Free BSD Questions list freebsd-questions@freebsd.org Date: Monday, July 13, 2009, 1:29 PM --- On Sun, 7/12/09, Maxim Khitrov mkhit...@gmail.com wrote: From: Maxim Khitrov mkhit...@gmail.com Subject: ZFS or UFS for 4TB hardware RAID6? To: Free BSD Questions list freebsd-questions@freebsd.org Date: Sunday, July 12, 2009, 11:47 PM Hello all, I'm about to build a new file server using 3ware 9690SA-8E controller and 4x Western Digital RE4-GP 2TB drives in RAID6. It is likely to grow in the future up to 10TB. I may use FreeBSD 8 on this one, since the release will likely be made by the time this server goes into production. The question is a simple one - I have no experience with ZFS and so wanted to ask for recommendations of that versus UFS2. How stable is the implementation and does it offer any benefits in my setup (described below)? All of the RAID6 space will only be used for file storage, accessible by network using NFS and SMB. It may be split into separate partitions, but most likely the entire array will be one giant storage area that is expanded every time another hard drive is added. The OS and all installed apps will be on a separate software RAID1 array. Given that security is more important than performance, what would be your recommended setup and why? - Max Your mileage may vary, but... I would investigate either using more spindles if you want to stick to RAID6, or perhaps using another RAID level if you will be with 4 drives for a while. The reasoning is that there's an overhead with RAID 6 - parity blocks are written to 2 disks, so in a 4 drive combination you have 2 drives with data and 2 with parity. With 4 drives, you could get much, much higher performance out of RAID10 (which is alternatively called RAID0+1 or RAID1+0 depending on the manufacturer and on how accurate they wish to be, and on how they actually implemented it, too). This would also mean 2 usable drives, as well, so you'd have the same space available in RAID10 as your proposed RAID6. I would confirm you can, on the fly, convert from RAID10 to RAID6 after you add more drives. If you can not, then by all means stick with RAID6 now! With 4 1 TB drives (for simpler examples) RAID5 = 3 TB available, 1 TB worth used in parity. Fast reads, slow writes. RAID6 = 2 TB available, 2 TB worth used in parity. Moderately fast reads, slow writes. RAID10 = 2 TB available, 2TB in duplicate copies (easier work than parity calculations). Very fast reads, moderately fast writes. When you switch to, say, 8 drives, the numbers start to change a bit. RAID5 = 7TB available, 1 lost. RAID6 = 6TB available, 2 lost. RAID10 = 4TB available, 4 lost. Sorry, consider myself chastised for having missed the Security is more important than performance bit. I tend toward solutions that show the most value, and with 4 drives, it seems that I'd stick with the same data security only pick up the free speed of RAID10. Change when you get to 6 or more drives, if necessary. For data security, I can't answer for the UFS2 vs. ZFS. For hardware setup, let me amend everything I said above with the following: Since you are seriously focusing on data integrity, ignore everything I said but make sure you have good backups! :) Sorry, -Rich ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ZFS or UFS for 4TB hardware RAID6?
--- On Mon, 7/13/09, Maxim Khitrov mkhit...@gmail.com wrote: From: Maxim Khitrov mkhit...@gmail.com Subject: Re: ZFS or UFS for 4TB hardware RAID6? To: mahle...@yahoo.com Cc: Free BSD Questions list freebsd-questions@freebsd.org Date: Monday, July 13, 2009, 2:02 PM On Mon, Jul 13, 2009 at 1:46 PM, Richard Mahlerweinmahle...@yahoo.com wrote: Your mileage may vary, but... I would investigate either using more spindles if you want to stick to RAID6, or perhaps using another RAID level if you will be with 4 drives for a while. The reasoning is that there's an overhead with RAID 6 - parity blocks are written to 2 disks, so in a 4 drive combination you have 2 drives with data and 2 with parity. With 4 drives, you could get much, much higher performance out of RAID10 (which is alternatively called RAID0+1 or RAID1+0 depending on the manufacturer and on how accurate they wish to be, and on how they actually implemented it, too). This would also mean 2 usable drives, as well, so you'd have the same space available in RAID10 as your proposed RAID6. I would confirm you can, on the fly, convert from RAID10 to RAID6 after you add more drives. If you can not, then by all means stick with RAID6 now! With 4 1 TB drives (for simpler examples) RAID5 = 3 TB available, 1 TB worth used in parity. Fast reads, slow writes. RAID6 = 2 TB available, 2 TB worth used in parity. Moderately fast reads, slow writes. RAID10 = 2 TB available, 2TB in duplicate copies (easier work than parity calculations). Very fast reads, moderately fast writes. When you switch to, say, 8 drives, the numbers start to change a bit. RAID5 = 7TB available, 1 lost. RAID6 = 6TB available, 2 lost. RAID10 = 4TB available, 4 lost. Sorry, consider myself chastised for having missed the Security is more important than performance bit. I tend toward solutions that show the most value, and with 4 drives, it seems that I'd stick with the same data security only pick up the free speed of RAID10. Change when you get to 6 or more drives, if necessary. For data security, I can't answer for the UFS2 vs. ZFS. For hardware setup, let me amend everything I said above with the following: Since you are seriously focusing on data integrity, ignore everything I said but make sure you have good backups! :) Sorry, -Rich No problem :) I've been doing some reading since I posted this question and it turns out that the controller will actually not allow me to create a RAID6 array using only 4 drives. 3ware followed the same reasoning as you; with 4 drives use RAID10. I know that you can migrate from one to the other when a 5th disk is added, but RAID10 can only handle 2 failed drives if they are from separate RAID1 groups. In this way, it is just slightly less resilient to failure than RAID6. With this new information, I think I may as well get one more 2TB drive and start with 6TB of RAID6 space. This will be less of a headache later on. - Max Just as a question: how ARE you planning on backing this beast up? While I don't want to sound like a worry-wort, I have had odd things happen at the worst of times. RAID cards fail, power supplies let out the magic smoke, users delete items they really want back... *sigh* A bit of reading shows that ZFS, if it's stable enough, has some really great features that would be nice on such a large pile o' drives. See http://wiki.freebsd.org/ZFSQuickStartGuide I guess the last question I'll ask (as any more may uncover my ignorance) is if you need to use hardware RAID at all? It seems both UFS2 and ZFS can do software RAID which seems to be quite reasonable with respect to performance and in many ways seems to be more robust since it is a bit more portable (no specialized hardware). There are others who may respond with better information on that front. I've been a strong proponent of hardware RAID, but have recently begun to realize many of the reasons for that are only of limited validity now. -Rich ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ZFS or UFS for 4TB hardware RAID6?
--- On Mon, 7/13/09, Maxim Khitrov mkhit...@gmail.com wrote: From: Maxim Khitrov mkhit...@gmail.com Subject: Re: ZFS or UFS for 4TB hardware RAID6? To: mahle...@yahoo.com Cc: Free BSD Questions list freebsd-questions@freebsd.org Date: Monday, July 13, 2009, 3:23 PM On Mon, Jul 13, 2009 at 2:13 PM, Richard Mahlerweinmahle...@yahoo.com wrote: --- On Mon, 7/13/09, Maxim Khitrov mkhit...@gmail.com wrote: From: Maxim Khitrov mkhit...@gmail.com Subject: Re: ZFS or UFS for 4TB hardware RAID6? To: mahle...@yahoo.com Cc: Free BSD Questions list freebsd-questions@freebsd.org Date: Monday, July 13, 2009, 2:02 PM On Mon, Jul 13, 2009 at 1:46 PM, Richard Mahlerweinmahle...@yahoo.com wrote: Your mileage may vary, but... I would investigate either using more spindles if you want to stick to RAID6, or perhaps using another RAID level if you will be with 4 drives for a while. The reasoning is that there's an overhead with RAID 6 - parity blocks are written to 2 disks, so in a 4 drive combination you have 2 drives with data and 2 with parity. With 4 drives, you could get much, much higher performance out of RAID10 (which is alternatively called RAID0+1 or RAID1+0 depending on the manufacturer and on how accurate they wish to be, and on how they actually implemented it, too). This would also mean 2 usable drives, as well, so you'd have the same space available in RAID10 as your proposed RAID6. I would confirm you can, on the fly, convert from RAID10 to RAID6 after you add more drives. If you can not, then by all means stick with RAID6 now! With 4 1 TB drives (for simpler examples) RAID5 = 3 TB available, 1 TB worth used in parity. Fast reads, slow writes. RAID6 = 2 TB available, 2 TB worth used in parity. Moderately fast reads, slow writes. RAID10 = 2 TB available, 2TB in duplicate copies (easier work than parity calculations). Very fast reads, moderately fast writes. When you switch to, say, 8 drives, the numbers start to change a bit. RAID5 = 7TB available, 1 lost. RAID6 = 6TB available, 2 lost. RAID10 = 4TB available, 4 lost. Sorry, consider myself chastised for having missed the Security is more important than performance bit. I tend toward solutions that show the most value, and with 4 drives, it seems that I'd stick with the same data security only pick up the free speed of RAID10. Change when you get to 6 or more drives, if necessary. For data security, I can't answer for the UFS2 vs. ZFS. For hardware setup, let me amend everything I said above with the following: Since you are seriously focusing on data integrity, ignore everything I said but make sure you have good backups! :) Sorry, -Rich No problem :) I've been doing some reading since I posted this question and it turns out that the controller will actually not allow me to create a RAID6 array using only 4 drives. 3ware followed the same reasoning as you; with 4 drives use RAID10. I know that you can migrate from one to the other when a 5th disk is added, but RAID10 can only handle 2 failed drives if they are from separate RAID1 groups. In this way, it is just slightly less resilient to failure than RAID6. With this new information, I think I may as well get one more 2TB drive and start with 6TB of RAID6 space. This will be less of a headache later on. - Max Just as a question: how ARE you planning on backing this beast up? While I don't want to sound like a worry-wort, I have had odd things happen at the worst of times. RAID cards fail, power supplies let out the magic smoke, users delete items they really want back... *sigh* Rsync over ssh to another server. Most of the data stored will never change after the first upload. A daily rsync run will transfer one or two gigs at the most. History is not required for the same reason; this is an append-only storage for the most part. A backup for the previous day is all that is required, but I will keep a weekly backup as well until I start running out of space. A bit of reading shows that ZFS, if it's stable enough, has some really great features that would be nice on such a large pile o' drives. See http://wiki.freebsd.org/ZFSQuickStartGuide I guess the last question I'll ask (as any more may uncover my ignorance) is if you need to use hardware RAID at all? It seems both UFS2 and ZFS can do software RAID which seems to be quite reasonable with respect to performance and in many ways seems to be more robust since it is a bit more portable (no specialized hardware). I've thought about this one a lot. In my case, the hard drives are in a separate enclosure from the server and the two had to be connected via SAS cables. The 9690SA-8E card was the best choice I
Re: POLL: Linux preferences from FreeBSD users
My preferences for Linux: I have used FreeBSD fairly regularly since 2.x and various flavors of Linux since around that time as well. As I was writing the first pass at this, I realized that many or most of the problems I have with Linuxes are endemic to Linux (whatever that is) and not to particular distributions. My main problems with most of the them are that they are just so inconsistent. Directory structures, documentation, even just where they install packages to by default - the standardization inside FreeBSD and that which is supplied by the ports system just makes for so much more of a sane and predictable experience. Secondarily, apart from *some* of the source ones and the debian-based ones, I always end up with broken dependencies or some weird circular inconsistencies. I'm sure I could fix them if I were a rpm guru, but I am not. FreeBSD just [generally] makes it so much easier and makes me not want to become an rpm guru. Servers: As you have probably guessed by this point, the only Linux that I feel suits my needs well enough to have used it long term (on my own, that is, not when I've been required to use it) is Debian and some of its progeny (including, in fact, Progeny itself! :). I usually end up with Ubuntu server. And it's OK. Desktops: Now, on desktops I flit around like a jack rabbit on crack. My desktop needs are completely different from my server needs. I'm usually XP (for games at home, work at work), so it's always the second and third OS on my boxes, so I try 'em all. PCLinux is actually been very good to me recently (Surprise! It's rpm based, too! How weird is that!). Kbuntu and some variants are decent enough. None of these last long enough to need more than a few patches, so I don't have the problem of dependency issues. Rich Mahlerwein Mobile: 715-891-7420 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Disappearing Hard Drive?
I have a puzzling problem. I'm sure I just missed something simple, but I can't figure out what. I added a drive to my system as Master on controller 2. After turning back on, I found I couldn't see that 80 GB drive on my system. I turned it off, checked all settings, then turned it back on and verified that the BIOS can see it - it's properly listed as an 80 GB Samsung as Master on 2. It sounds normal, looks normal and I can feel it spin up. But I still don't see it on my system to mount it. I had just done this process with a 40 GB drive and it worked just fine, so I knew that I knew how to do it. That one showed up in dmesg and was easily mounted. For what it's worth, my Linux desktop, when I hook this drive up to an oddball USB-ATA adapter, acts exactly like it does with every OTHER drive I've pulled out of a FreeBSD box - it sees it but hasn't a clue how to mount it properly. So I don't think the drive is a dud or anything. It was in use until a few weeks ago. I will be verifying the drive is seen OK on another system shortly, but I'd thought I'd throw this out there before then to see if anyone has any ideas,. If someone would be so kind as to point me to some relevant help, I would greatly appreciate it. Please reply to me in addition to the list: I receive the list in digest only. dmesg follows: Copyright (c) 1992-2008 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.1-PRERELEASE #0: Fri Sep 5 02:34:20 CDT 2008 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC Timecounter i8254 frequency 1193182 Hz quality 0 CPU: Intel(R) Pentium(R) 4 CPU 1.60GHz (1595.16-MHz 686-class CPU) Origin = GenuineIntel Id = 0xf12 Stepping = 2 Features=0x3febfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM real memory = 536608768 (511 MB) avail memory = 511090688 (487 MB) ACPI APIC Table: D850MV MV85010A ioapic0 Version 2.0 irqs 0-23 on motherboard kbd1 at kbdmux0 ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) acpi0: D850MV MV85010A on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) acpi0: reservation of 0, a (3) failed acpi0: reservation of 10, 1ff0 (3) failed Timecounter ACPI-fast frequency 3579545 Hz quality 1000 acpi_timer0: 24-bit timer at 3.579545MHz port 0x408-0x40b on acpi0 pcib0: ACPI Host-PCI bridge port 0xcf8-0xcff on acpi0 pci0: ACPI PCI bus on pcib0 agp0: Intel 82850 host to AGP bridge on hostb0 pcib1: PCI-PCI bridge at device 1.0 on pci0 pci1: PCI bus on pcib1 vgapci0: VGA-compatible display mem 0xff4e-0xff4f,0xf000-0xf3ff,0xec00-0xefff irq 16 at device 0.0 on pci1 pcib2: ACPI PCI-PCI bridge at device 30.0 on pci0 pci2: ACPI PCI bus on pcib2 fxp0: Intel 82558 Pro/100 Ethernet port 0xdf80-0xdf9f mem 0xf6aff000-0xf6af,0xff90-0xff9f irq 23 at device 11.0 on pci2 miibus0: MII bus on fxp0 inphy0: i82555 10/100 media interface PHY 1 on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fxp0: Ethernet address: 00:90:27:1b:5f:f7 fxp0: [ITHREAD] isab0: PCI-ISA bridge at device 31.0 on pci0 isa0: ISA bus on isab0 atapci0: Intel ICH2 UDMA100 controller port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 31.1 on pci0 ata0: ATA channel 0 on atapci0 ata0: [ITHREAD] ata1: ATA channel 1 on atapci0 ata1: [ITHREAD] uhci0: Intel 82801BA/BAM (ICH2) USB controller USB-A port 0xef40-0xef5f irq 19 at device 31.2 on pci0 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: Intel 82801BA/BAM (ICH2) USB controller USB-A on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 on usb0 uhub0: 2 ports with 2 removable, self powered pci0: serial bus, SMBus at device 31.3 (no driver attached) uhci1: Intel 82801BA/BAM (ICH2) USB controller USB-B port 0xef80-0xef9f irq 23 at device 31.4 on pci0 uhci1: [GIANT-LOCKED] uhci1: [ITHREAD] usb1: Intel 82801BA/BAM (ICH2) USB controller USB-B on uhci1 usb1: USB revision 1.0 uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 on usb1 uhub1: 2 ports with 2 removable, self powered pci0: multimedia, audio at device 31.5 (no driver attached) cpu0: ACPI CPU on acpi0 p4tcc0: CPU Frequency Thermal Control on cpu0 acpi_button0: Power Button on acpi0 atkbdc0: Keyboard controller (i8042) port 0x60,0x64 irq 1 on acpi0 atkbd0: AT Keyboard irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] fdc0: floppy drive controller port 0x3f0-0x3f1,0x3f2-0x3f3,0x3f4-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FILTER] fd0: 1440-KB 3.5 drive on fdc0 drive 0 sio0: 16550A-compatible COM port port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio0: [FILTER] sio1: 16550A-compatible COM port port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A sio1:
