Re: Optimising NFS for system files
I am trying a memory disk on server to see the effect of hard drive performances, and also discovering the function :-) The conclusion is that memory disk is faster that this drive ;-) 45MB/s vs 10Mb/s But the NFS access to the memory drive is still 5MB/s :-( As there is no more hard drive involved, we know that there is a bottleneck at 5MB in NFS layer on this system... Where ? Thanks a lot for any help on the method to find/diagnose this ! Details are below : nfsserver# uname -a FreeBSD nfsserver 7.1-RC1 FreeBSD 7.1-RC1 #0: Sun Dec 7 00:38:13 UTC 2008 r...@driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 nfsserver# mdconfig -a -t swap -s 600m -o reserve -u 7 nfsserver# ls /dev/md* /dev/md7/dev/mdctl nfsserver# newfs -i 2048 /dev/md7 Reduced frags per cylinder group from 60864 to 59928 to enlarge last cyl group /dev/md7: 600.0MB (1228800 sectors) block size 16384, fragment size 2048 using 6 cylinder groups of 117.05MB, 7491 blks, 59968 inodes. super-block backups (for fsck -b #) at: 160, 239872, 479584, 719296, 959008, 1198720 nfsserver# mkdir /tstnfs nfsserver# mount /dev/md7 /tstnfs nfsserver# date ; time tar -cf - /nfsro/commun/clientusr-amd64-7.2-RC2-20081230/ports /dev/null ; date ; Wed Dec 31 09:11:08 CET 2008 tar: Removing leading '/' from member names 3.794u 8.766s 0:46.40 27.0% 71+1406k 123375+0io 0pf+0w Wed Dec 31 09:11:54 CET 2008 That makes 498MB / 46s = 10.8MB/s for disk drive. nfsserver# date ; cp -r -p /nfsro/commun/clientusr-amd64-7.2-RC2-20081230/ports /tstnfs/ports ; date Wed Dec 31 09:33:09 CET 2008 Wed Dec 31 09:34:46 CET 2008 df -h /dev/md7 512M498M-27M 106%/tstnfs nfsserver# date ; time tar -cf - /tstnfs/ports /dev/null ; date ; Wed Dec 31 09:36:59 CET 2008 tar: Removing leading '/' from member names 2.947u 6.218s 0:10.61 86.2% 74+1463k 104885+0io 0pf+0w Wed Dec 31 09:37:10 CET 2008 nfsserver# date ; time tar -cf - /tstnfs/ports /dev/null ; date ; Wed Dec 31 09:37:12 CET 2008 tar: Removing leading '/' from member names 2.895u 6.487s 0:11.01 85.1% 74+1466k 112838+0io 0pf+0w Wed Dec 31 09:37:23 CET 2008 nfsserver# date ; time tar -cf - /tstnfs/ports /dev/null ; date ; Wed Dec 31 09:40:22 CET 2008 tar: Removing leading '/' from member names 2.902u 6.610s 0:11.10 85.6% 75+1483k 113393+0io 0pf+0w Wed Dec 31 09:40:33 CET 2008 That makes 498MB / 11s = 45MB/s : better that 10MB/s for disk, but not exceptional. Now on the client : client9# uname -a FreeBSD client9 7.1-RC2 FreeBSD 7.1-RC2 #0: Tue Dec 23 11:42:13 UTC 2008 r...@driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 client9# mount -o ro nfsserver:/tstnfs /tstnfs client9# df -h nfsserver:/tstnfs 512M498M 0B 100%/tstnfs client9# date ; time tar -cf - /tstnfs/ports /dev/null ; date ; Wed Dec 31 09:50:39 CET 2008 tar: Removing leading '/' from member names 2.896u 13.020s 1:35.22 16.7%75+1483k 0+0io 2pf+0w Wed Dec 31 09:52:14 CET 2008 client9# date ; time tar -cf - /tstnfs/ports /dev/null ; date ; Wed Dec 31 09:52:22 CET 2008 tar: Removing leading '/' from member names 2.700u 12.755s 1:27.78 17.6%76+1498k 0+0io 0pf+0w Wed Dec 31 09:53:50 CET 2008 client9# date ; time tar -cf - /tstnfs/ports /dev/null ; date ; Wed Dec 31 09:55:02 CET 2008 tar: Removing leading '/' from member names 2.681u 12.688s 1:28.15 17.4%74+1464k 0+0io 0pf+0w Wed Dec 31 09:56:30 CET 2008 That makes between 95s and 87s, then 498MB / 95s = 5,2MB/s and 5.7MB/s, like previous test from hard drive NFS export. Top is showing around 100MB of free memory while taring on client9, so i don't think tar is paging on network : last pid: 3318; load averages: 0.17, 0.11, 0.04up 0+11:14:27 10:08:10 30 processes: 1 running, 29 sleeping CPU: 0.8% user, 0.0% nice, 9.0% system, 0.0% interrupt, 90.2% idle Mem: 19M Active, 720M Inact, 136M Wired, 240K Cache, 110M Buf, 98M Free Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Optimising NFS for system files
Peter Boosten wrote: On server, it means : 1440MB / 84s = 17MB/s On client, that becomes : 1440MB / 266s = 5.4MB/s I know the disk is not very fast, but i would like the NFS layer not to add too much... I don't want my users to wait between 3 or 4 times more because computer is using NFS. In my opinion there are more considerations than only nfs: the data is pulled twice over the network, and the tar process might initiate paging which is done over the network as well. The tar comparison is not a good one. I would welcome any way to check that idea on the system. But : - tar is directed to /dev/null so that should avoid any physical writing ; - there is still memory FREE on both server and client while taring ; - the effect of tar is the same on server and client, so the induced error should be the same time on both. Thanks a lot, Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Optimising NFS for system files
Manolis Kiagias wrote: First thing that may be wrong is the understanding of the time figures. The documentation is not clear about them and the -h option is not working : client6# time -h tar -cf - /usr /dev/null -h: Command not found. 0.000u 0.000s 0:00.00 0.0% 0+0k 0+0io 0pf+0w Just a sidenote, you are probably getting a version of time integrated to your shell. The -h option works fine in /usr/bin/time, so run like this: client6# /usr/bin/time -h tar -cf - /usr /dev/null Very true, this is it :-) Thanks a lot for your help ! Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Optimising NFS for system files
usleep wrote: - Second installation - FreeNAS, RAID0 - Tested throughput ( to local RAID0 ): - ftp: 82MB/s - nfs: 75MB/s - cifs/samba: 42MB/s Thanks a lot for these clear references ! Test issues ( things that get you confused ) - if you expect to be able to copy a file at Gigabit speeds, you need to be able to write as fast to your local disk as well. So to reliable test SAN/NAS performance at Gigabit speeds you need RAID at the server and at the client. Or write to /dev/null This is what i did with the tar. What function do you use for testing ? - if you repeatedly test with the same file, it will get cached into memory of the NAS. so you won't be testing troughput disk-network-disk anymore: you are testing NAS-memory-network-disk. I was testing with ubuntu-iso's, but with 1GB of memory, ISO's get cached as well. i was fearing that, but it seems that does not happen in my testing, as repeating same test give same result. I don't know why it does not happens... - if you repeatedly test with the same file, and you have enough local memory, and you test with nfs or cifs/samba, the file will get cached locally as well. this results into transfer-speeds to /dev/null exceeding 100MB/s ( Gigabit speeds ). i have observed transfer speeds to /dev/null of 400MB/s! I would love to see that behaviour. But it didn't happen neither in my testing :-( We now have a NAS that performs faster than local disk. Is it the FreeNAS you describe in your testing or a new one ? We plan to use it run development-virtual-machines on. This is also my target :-) I will have some high density servers (6 independent servers in 1U) and trying to master the freebsd diskless process before... Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Optimising pxeboot disk size
Wojciech Puchar wrote: I couldn't find the hostname sent by dhcp, but found a way to find the ip address. just use hostname command. I did, but the answer is empty (remember this is before /etc/rc is run). Where is kept the data received by dhcp_client during pxe boot ? The only problem i have is that the echo done and other standard outputs are not visible in /var/log/messages. How can i keep them either in dmesg or /var/log/messages ? add second echo to /var/log/messages ;) Hope not to forget one ;-) But is there a system call or configuration to do that automatically ? I can see some echo in /etc/rc without any , and their result seems to go to /var/messages. I wish to learn to do clean scripting :-) Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Optimising NFS for system files
Wojciech Puchar wrote: i can see a reading speed difference 4 time slower on client than on server (time tar -cf - /usr /dev/null). I will play with jumbo MTU for network performance, but would anybody know if i can ask system files NFS exports to stay in server memory ? I have less than 2Go to share and 2GO DDR2 is affordable. you don't have to. So you din't think that if all files are already in RAM on server, i will save the drive access time ? Or do you think the NFS network access is so much slow that the disk access time is just marginal ? Do you think i should use something more efficient than NFS ? Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Optimising NFS for system files
Wojciech Puchar wrote: So you din't think that if all files are already in RAM on server, i will save the drive access time ? FreeBSD automatically use all free memory as cache. OK there is slowdown because network introduces slight delay, but few ms at most if network is made properly This is a Gbps network with only 1 switch between nfs server and client, with less than 0.2ms ping. So bandwidth should not be a problem, seems that NFSV3 is the limitation... Trying to change mtu, but don't look easy, where can i find the possible range for ports ? Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Optimising NFS for system files
Vince wrote: Trying to change mtu, but don't look easy, where can i find the possible range for ports ? MTU can be a pain, check what your switch supports, and the manpage for your network driver should say what MTU the nic supports. Thank you for the method ! It seems that em and re are not behaving like they should : re(4) says : The 8169, 8169S and 8110S also support jumbo frames, which can be configured via the interface MTU setting. The MTU is limited to 7422, since the chip cannot transmit larger frames. But : nfsserver# ifconfig re0 -mtu 7422 ifconfig: -mtu: bad value nfsserver# ifconfig re0 -mtu 7421 ifconfig: -mtu: bad value It should be a Realtek RTL 8111c but i don't know where to find the relationship to what pciconf -l gives me : r...@pci0:2:0:0: class=0x02 card=0xe0001458 chip=0x816810ec rev=0x02 hdr=0x00 em(4) says : The maximum MTU size for Jumbo Frames is 16114. But : client9# ifconfig em1 -mtu 8192 ifconfig: -mtu: bad value with : Dec 30 16:02:36 client9 kernel: em1: Intel(R) PRO/1000 Network Connection 6.9.6 port 0x7f00-0x7f1f mem 0xfd4e-0xfd4f irq 17 at device 0.0 on pci7 client1# ifconfig em0 -mtu 8192 ifconfig: -mtu: bad value with : Dec 30 18:10:38 client1 kernel: em0: Intel(R) PRO/1000 Network Connection 6.9.6 port 0xfe00-0xfe1f mem 0xf dfc-0xfdfd,0xfdffe000-0xfdffefff irq 20 at device 25.0 on pci0 Now i understand better MTU can be a pain ;-) Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Optimising NFS for system files
Wojciech Puchar wrote: This is a Gbps network with only 1 switch between nfs server and client, with less than 0.2ms ping. So bandwidth should not be a it should work with near-wire speed on 100Mbit clients. Server and clients are 1Gbps. But i have a 4 factor of performance for reading only ... nfsserver# time tar -cf - clientusr-amd64 /dev/null 5.001u 12.147s 1:23.92 20.4%69+1369k 163345+0io 0pf+0w client9# time tar -cf - /usr /dev/null tar: Removing leading '/' from member names 3.985u 19.779s 4:32.47 8.7% 74+1457k 0+0io 0pf+0w Note : clientusr-amd64 is around 1.3GB and is the same directory exported to client9 /usr with nfs. I have tried on 7.1-RC1 and 7.1-RC2, with amd64 architecture. CPU don't seem to be the limiting factor, more than 80% idle on server, they are either Core2duo on nfsserver : Dec 23 04:52:18 nfsserver kernel: CPU: Intel(R) Celeron(R) CPU E1200 @ 1.60GHz (1600.01-MHz K8-class CPU) Dec 23 04:52:18 nfsserver kernel: Origin = GenuineIntel Id = 0x6fd Stepping = 13 or on client9 : /var/log/messages.3:Dec 29 12:21:20 client9 kernel: CPU: Intel(R) Core(TM)2 Duo CPU E4500 @ 2.20GHz (2200.01-MHz K8-class CPU) If anybody can help to look at right places... ? How may i divide the problem ? Or is my simple test wrong ? I use a tar directed to /dev/null to avoid any writing. Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Optimising NFS for system files
Matthew Seaman wrote: It's 'mtu ' not '-mtu ' I'm confused, thanks so much ! There was no option without - in my old unix time ;-) Thanks to you, it seems that my max mtu is 9216 on em : client9# ifconfig em1 mtu 9216 client9# ifconfig em1 mtu 9217 ifconfig: ioctl (set mtu): Invalid argument Max mtu is changing on re : nfsserver# ifconfig re0 mtu 1504 nfsserver# ifconfig re0 mtu 1505 ifconfig: ioctl (set mtu): Invalid argument But another re accept 7422 : client6# ifconfig re0 mtu 7422 client6# ifconfig re0 mtu 7423 ifconfig: ioctl (set mtu): Invalid argument It seems that only testing can give the limit, this is not documented. Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Optimising NFS for system files
Wojciech Puchar wrote: nfsserver# time tar -cf - clientusr-amd64 /dev/null 5.001u 12.147s 1:23.92 20.4%69+1369k 163345+0io 0pf+0w client9# time tar -cf - /usr /dev/null tar: Removing leading '/' from member names 3.985u 19.779s 4:32.47 8.7% 74+1457k 0+0io 0pf+0w Note : clientusr-amd64 is around 1.3GB and is the same directory exported to client9 /usr with nfs. it's FAST. what's wrong? First thing that may be wrong is the understanding of the time figures. The documentation is not clear about them and the -h option is not working : client6# time -h tar -cf - /usr /dev/null -h: Command not found. 0.000u 0.000s 0:00.00 0.0% 0+0k 0+0io 0pf+0w The main thing is that the 3rd figures 1:23.92 and 4:32.47 seems to be the time i wait in front of the computer while it works (ok, i know, i should enjoy a beer, or hot coffee with this nice snow ;-) : client9# date ; time tar -cf - /usr /dev/null ; date ; Wed Dec 31 08:23:59 CET 2008 tar: Removing leading '/' from member names 4.103u 19.651s 4:25.80 8.9% 74+1453k 0+0io 2pf+0w Wed Dec 31 08:28:25 CET 2008 and 08:28:25 - 08:23:59 = 00:04:26 is very close to 4:25.80. On server, it means : 1440MB / 84s = 17MB/s On client, that becomes : 1440MB / 266s = 5.4MB/s I know the disk is not very fast, but i would like the NFS layer not to add too much... I don't want my users to wait between 3 or 4 times more because computer is using NFS. I have plenty of cpu and bandwidth available : something is slowing the process that should not... But what ? How to diagnose NFS ? Where should i look in a logical diagnosis process ? Best regards -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Optimising pxeboot disk size
Hello, Wojciech Puchar wrote: i already did such things but with NetBSD 1.5 for my Xterminal distro. it's simple: mount / readonly, put everything here. in /etc put /etc/rc consisting only #!/bin/sh exec /systemrc in /systemrc put something like that: #!/bin/sh echo -n Mounting workstation config directory... #get your IP or MAC address using ifconfig,grep,cut,awk,how you like #put to to say MYIP /sbin/mount_nfs yournfsserver:/clients/etc/MYIP /etc echo done exec /etc/rc Thank you very much, it is now running well after some hardware adventure. I couldn't find the hostname sent by dhcp, but found a way to find the ip address. The only problem i have is that the echo done and other standard outputs are not visible in /var/log/messages. How can i keep them either in dmesg or /var/log/messages ? Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Optimising NFS for system files
Hi, I'm working on a project to have many diskless clients PXEbooting on 1 nfs server. With some help :-) i could manage to share almost all system files (/, /usr,..) through NFS. i can see a reading speed difference 4 time slower on client than on server (time tar -cf - /usr /dev/null). I will play with jumbo MTU for network performance, but would anybody know if i can ask system files NFS exports to stay in server memory ? I have less than 2Go to share and 2GO DDR2 is affordable. Thanks a lot, Best regards, ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Extracting changed files list from snapshot
Mel wrote: I'm still wondering if you're not better off with ZFS, but this does seem like a useful app in it's own right. ZFS is not handling replication on remote servers very well currently, and they don't seem to care about this from the reactions i had on some forums. And i would not use ZFS for production system on FreeBSD now. FFS snapshots are stable in production, and adding a reading function on snapshot structures will not change stability :-) The TAILQ_HEAD statement means it's creating a tail queue(3) (double linked fifo/stack) of inodes. The snapdata structure contains the start of the list, the size and a lock. The sn_blklist pointer, I will have to look up. I think i'm gonna have fun with this for a bit ;) This is christmas gift. But for who :-? Thanks a lot ! -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Extracting changed files list from snapshot
Mel wrote:
On Tuesday 16 December 2008 15:18:19 Bernard Dugas wrote:
I want to extract the list of files changed between 2 snapshots, to be
able to do efficient backups.
Just use dump(8)?
Yes in first step, may be. But for fast replication of changes ?
Imagine i have a 1TB drive with 1 partition on serverA, and i want it
duplicated on another 1TB drive serverB, with 1Gbps network between
them, so network speed is not a problem.
I begin with dump or any copy mean to have the 2 drives having the
partitions in the same state, from a first snapshot image 0 which is
the reference on serverA, made with mksnap_ffs.
A user change only 1 small file on serverA.
After 1mn, i create snapshot0+1 file on serverA.
To find the files changed on serverA between snapshot0 and snapshot0+1,
any current dump or equivalent will have to read all file records of
1TB partition, inside snapshot0+1 complete view of partition, to find 1
changed file. Even on nice day, it may take more than 1mn.
But the beauty and efficiency of snapshot concept is that snapshot0 file
(internal structure) contains a reference to all data blocks changed
between snapshot0 and snapshot0+1 : this is why a snapshot file is so
small and efficient.
It is certainly possible to find in snapshot0 file inside structure the
list of blocks changed until snapshot0+1 was made, and then find which
file is the owner of the changed blocks.
Then it will appear that only 1 file has been changed on whole partition
and we just have to send this changed file from snapshot0+1 view to
serverB.
So using the internal structure of snapshots file is far more efficient
than reading all directory records for last change time...
Please note that i try to use the expressions snapshot file and
snapshot view very cautiosly, because they don't mean the same at all :-)
And what snapshots do you mean? As in mksnap_ffs? Cause that's described
in /usr/src/sys/ufs/ffs/ffs_snapshot.c:
127 TAILQ_HEAD(snaphead, inode);
128
129 struct snapdata {
130 struct snaphead sn_head;
131 daddr_t sn_listsize;
132 daddr_t *sn_blklist;
133 struct lock sn_lock;
134 };
135
and not exposed to userland.
Thanks, this is a good hint ! Just needing some doc and help to
understand how to use it : i'm more in design now, programmation skills
are far away :-(
Best regards,
--
Bernard DUGAS Mobile +33 615 333 770
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Any doc reference on /entropy file ?
Hi, When i look at / in a standard installation like : FreeBSD XXX 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Sun Feb 24 19:59:52 UTC 2008 r...@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386 There is only 1 file, named entropy : -rw--- 1 root wheel 4096 Dec 11 17:36 entropy I can't find any reference to that file in FreeBSD doc. Any reference to it ? What is it used for ? I have look in RANDOM area, but no reference... Do i have to keep it read/write ? Can i put it in /var instead of / to be able to keep / read-only ? Thanks a lot, Best regards, ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Any doc reference on /entropy file ?
Wojciech Puchar wrote: -rw--- 1 root wheel 4096 Dec 11 17:36 entropy I can't find any reference to that file in FreeBSD doc. Any reference to it ? What is it used for ? I have look in RANDOM area, but no reference... at startup it seeds random generator, every 11 minutes and at shutdown it's saved from random generator. it's here to make random generator more random - not starting clean at boot. Thanks a lot ! 1- How may i suggest/learn to add this info in the random generator man page, so that a search in doc gives the right result ? 2- As / will be mounted read-only, how can i tell the ramdom generator to put this /entropy file somewhere in /var, where i think it should be ? Or is there any tricky hidden problem ? Thanks a lot, Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Optimising pxeboot disk size
Wojciech Puchar wrote: mount_unionfs but i don't know how stable it is. Thanks to fill my dreams :-) The dream is close, but still a dream :-( http://www.freebsd.org/cgi/man.cgi?query=mount_unionfssektion=8apropos=0manpath=FreeBSD+7.0-RELEASE BUGS THIS FILE SYSTEM TYPE IS NOT YET FULLY SUPPORTED (READ: IT DOESN'T WORK)AND USING IT MAY, IN FACT, DESTROY DATA ON YOUR SYSTEM. USE AT YOUR OWN RISK. BEWARE OF DOG. SLIPPERY WHEN WET. Sad not to be a coder... Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Optimising pxeboot disk size
Hi, I'm working on a project to have many diskless clients PXEbooting on 1 nfs server. This works now, but i would like to have a more efficient use of disk space on NFS server. Lot of root directories can be shared and mounted in fstab, after booting process : /usr, /home... In current solution, the minimum root directories i had to keep for managing to boot are : bin libexec sbin boot etc lib But it appears that only /etc is really specific to each diskless clientX, having basically fstab and rc.conf inside. All other directories could be shared, and specifically /boot which is more that 100MO : i would prefer not to duplicate it, as 10 servers would mean 1GO for nothing. Please would anybody help find a way to share all what we can during boot process ? Another way to ask the question is : how can we mount / and then /etc at the begining of boot process instead of mounting only / with etc inside ? I have tried to do links, but no way to make the boot process follow them... Thanks a lot, Best regards, ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Extracting changed files list from snapshot
Hi, I want to extract the list of files changed between 2 snapshots, to be able to do efficient backups. Any idea where to find a document about snapshots internal structure ? Or if you know any port that already do this :-? Thanks a lot, Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Optimising pxeboot disk size
Wojciech Puchar wrote: i already did such things but with NetBSD 1.5 for my Xterminal distro. it's simple: More simple when you tell it ;-) Thanks a lot, i will try it tonight ! I wish it's helpful, doing this doesn't just save space but saves time - you have to upgrade software once. So preserving consistency, which is the most important when you have lot of diskless stations ! you may like to make /etc-common directory and put most of files there, and symlinks in each station's /etc In fact, it makes me think that we miss a concept in mount, or at least i don't know it currently : imagine a -tl (TransparentLayer) option for mount, allowing to mount multiple source to the same directory, for instance /etc : mount -r yournfsserver:/basic/etc /etc mount -tl -r yournfsserver:/TypeX/etc /etc mount -tl -r yournfsserver:/StationY/etc /etc A file is first look for in yournfsserver:/StationY/etc, then in yournfsserver:/TypeX/etc and finally in yournfsserver:/basic/etc. This means that StationX will see in its /etc firts its specific files, then the files dedicated to TypeX station (webserver, dns server, workstation,...) and then all basic files unchanged from standard distribution. When you want to change something, you add a rw TransparentLayer : mount -tl yournfsserver:/StationYchanges/etc /etc So that changed or added files are only stored in this rw partition, thus very small and easy to manage. This would be a kind of partition inheritance, like in object languages... Dreams are allowed :-) Thanks a lot, Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Any doc reference on /entropy file ?
Wojciech Puchar wrote: 2- As / will be mounted read-only, how can i tell the ramdom generator to put this /entropy file somewhere in /var, where i think it should be ? Or is there any tricky hidden problem ? man rc.conf seek entropy_file and entropy_dir Thanks, detail is available in /etc/default/rc.conf : entropy_file=/entropy # Set to NO to disable caching entropy through reboots. # /var/db/entropy-file is preferred if / is not avail. It means that i can put it in /var when L is read-only. There should be no problem as the file is choosen in rc.conf, so after /var is mounted. As this is for diskless station, it would be fun to change the entropy file through the nfs server RANDOM, even better thant at each boot :-) Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Optimising pxeboot disk size
Paul B. Mahol wrote: On 12/16/08, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: When you want to change something, you add a rw TransparentLayer : mount -tl yournfsserver:/StationYchanges/etc /etc So that changed or added files are only stored in this rw partition, thus very small and easy to manage. This would be a kind of partition inheritance, like in object languages... Dreams are allowed :-) try mount_unionfs and mount_nullfs Thanks Paul, FreeBSD has always hidden treasures :-) MOUNT_NULLFS(8) doc is very detailled, but i can't find how to use it for my problem : being able to go from common shared configuration to specific private configuration. unionfs looks very close to my dream, but it is currently not available for production :-( Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Copying system/ports configuration?
On Tue, 16 Dec 2008 09:26:05 -0800 (PST), Dr. Jennifer Nussbaum bg271828 at yahoo.com wrote: I dont quickly see a free tool to do this. Maybe its easier to do this from scratch I just found last week ports-mngnt/portmaster that can help you to do that, and i have suggested portmaster author to add an -export option readable to built new ports on another computer. May be an -export-packages option would be nice also ? Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Any doc reference on /entropy file ?
Thank you RW, i am new on the list, i didn't answer you because i didn't received your answer while i received others, don't know why yet. I would be very interested to find a large but right view in the man documentation of what is happening exactly to this poor lost file :-) The etc/default/rc.conf detailled file is not searchable in the man page, nor the default /entropy file. Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Extracting changed files list from snapshot
Lowell Gilbert wrote: Bernard Dugas bern...@dugas-family.org writes: I want to extract the list of files changed between 2 snapshots, to be able to do efficient backups. Any idea where to find a document about snapshots internal structure ? Or if you know any port that already do this :-? An easier way would be to generate an mtree(1) description for each snapshot, and using that to compare against other snapshots. You have to mount the snapshot for that, though... -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Extracting changed files list from snapshot
Lowell Gilbert wrote: Bernard Dugas bern...@dugas-family.org writes: I want to extract the list of files changed between 2 snapshots, to be able to do efficient backups. Any idea where to find a document about snapshots internal structure ? Or if you know any port that already do this :-? An easier way would be to generate an mtree(1) description for each snapshot, and using that to compare against other snapshots. You have to mount the snapshot for that, though... Thanks for your answer, sorry not to be clear on the goal : find the changed files without having to read the whole LARGE disk. By definition, snapshot files have internally the list of block changed. So i would like to have a description of the internal structure of a snapshot file and a way to find the file by which any changed block is owned. Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Optimising pxeboot disk size
Paul B. Mahol wrote: Well, I tested unionfs on CURRENT and it did not crashed on me. Feel free to test it on 7 STABLE and report results to developers. Thanks for the feed-back, i will no more be afraid to test :-) I will tell you later, Best regards, -- Bernard DUGAS Mobile +33 615 333 770 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Any 4.10 installation on asus pundit ?
Hi, Robert Downes a écrit : Bernard Dugas wrote: Thanks very much, Robert, it was the udma option : but this is quite inefficient if I can't use UDMA with FreeBSD ? FreeBSD will drop down to PIO mode, probably mode 4. According to Scott Mueller's book, PIO mode 4 offers up to 16.67 MB/sec, whereas UDMA can offer up to 100 or 133 MB/sec. So, yes, less efficient (but also a bit quieter in my experience). And the cpu load should increase, no ? Have you been able to switch the data cable for a know good cable? That solved the problem for me. I've tried a brand new one, but no difference... Best regards, -- __ Bernard DUGAS | | | Technoparc Pays de Gex mailto:[EMAIL PROTECTED] | | 30 Rue Auguste Piccard Tel.: +33 615 333 770 | | FR 01630 St Genis Pouilly Fax : +33 450 205 106 | |_| ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Any 4.10 installation on asus pundit ?
Hi, Phil Schulz a écrit : I didn't see the beginning of the thread. http://lists.freebsd.org/mailman/htdig/freebsd-questions/2004-June/049482.html But I had a similar problem on my desktop machine using (I think) 4.9-Release. It started when I had to change the motherboard. The new one came w/ an SiS chipset that wasn't supported by FreeBSD. I solved the problem by adding a line to /sys/dev/ata/ata-dma.c. It worked for a few weeks w/o any problems until I switched to RELENG_5_2 for various other reasons. You might want to post what chipset your mainboard uses. http://www.asus.com/prog/spec.asp?m=Punditlangs=01 - Chipset SIS 651/962 - Lan Broadcom BCM4401 The freeBSD 4.10 ata0 seems not to work properly with the native udma 6 of the pundit. Forcing dma disabled in the bios for the disk prevents the ad0 : read command timeout ata0 resetting devices blocking. But no more DMA is not ideal for a server... I've tried a new ide cable, and a new drive, without effect. In addition, I can't find the bfe0 network driver for the Broadcom BCM4401 in the kernel configuration, but the 4.10 release notes told the bfe should be in. Any hints ? Thanks a lot, best regards, -- __ Bernard DUGAS | | | Technoparc Pays de Gex mailto:[EMAIL PROTECTED] | | 30 Rue Auguste Piccard Tel.: +33 615 333 770 | | FR 01630 St Genis Pouilly Fax : +33 450 205 106 | |_| ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Any 4.10 installation on asus pundit ?
Phil Schulz a écrit : http://www.asus.com/prog/spec.asp?m=Punditlangs=01 - Chipset SIS 651/962 - Lan Broadcom BCM4401 Try to apply the attached patch to /sys/dev/ata/ata-dma.c Neither do I have a mainboard w/ the same chipset as yours nor do I have any PC running RELENG_4 atm. But I had to apply sth similar to make my SiS-746(?)-based mainboard work under RELENG_4. I'm not a developer so I don't know if it's a good and clean solution but it worked for me back then. Thanks a lot, the content of this patch seems well targeted. The problem is I'm not a system developper either, and I only had the iso images for installation, so I will have to find a way to apply the patch. In addition, I can't find the bfe0 network driver for the Broadcom BCM4401 in the kernel configuration, but the 4.10 release notes told the bfe should be in. From bfe(4): The bfe device driver first appeared in FreeBSD 5.1. Yes, but in http://www.freebsd.org/releases/4.10R/errata.html : (27 May 2004) The bfe(4) driver for Broadcom BCM4401 based Fast Ethernet adapters has been added but the release notes did not mention that. I've reported a bug on that problems : http://www.freebsd.org/cgi/query-pr.cgi?pr=68149 Thanks a lot, -- __ Bernard DUGAS | | | Technoparc Pays de Gex mailto:[EMAIL PROTECTED] | | 30 Rue Auguste Piccard Tel.: +33 615 333 770 | | FR 01630 St Genis Pouilly Fax : +33 450 205 106 | |_| ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Any 4.10 installation on asus pundit ?
Hi, Robert Downes a écrit : [EMAIL PROTECTED] wrote: I've tried to install the last freebds4.10 on an ASUS PUNDIT, from the iso images downloaded from the freeBSD website. But it can't install. It stops on a : ata0 : resetting devices I had that problem. I disabled UDMA (Ultra DMA) in the BIOS, and the problem was overcome. However, it ultimately turned out to be a bad data cable to the hard drive, so see if you've got a good spare lying around to give that a try. (I got some serious errors in 5.2.1 trying to use the bad cable). Thanks very much, Robert, it was the udma option : but this is quite inefficient if I can't use UDMA with FreeBSD ? Best regards, -- __ Bernard DUGAS | | | Technoparc Pays de Gex mailto:[EMAIL PROTECTED] | | 30 Rue Auguste Piccard Tel.: +33 615 333 770 | | FR 01630 St Genis Pouilly Fax : +33 450 205 106 | |_| ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Any 4.10 installation on asus pundit ?
Hi, I've tried to install the last freebds4.10 on an ASUS PUNDIT, from the iso images downloaded from the freeBSD website. But it can't install. It stops on a : ata0 : resetting devices And I couldn't manage to have the Broadcom BCM4401 bfe0 recognised even if the 4.10 release notes tell bfe is included in 4.10 : http://www.freebsd.org/releases/4.10R/errata.html, 27 May 2004 addition. How can I do ? It seems that the problem was identified in 4.9 : http://research.lumeta.com/ches/cheap/asus.html and even earlier : http://lists.freebsd.org/pipermail/freebsd-hardware/2003-August/000451.html but there is no answer on the list. I should have look at that before, but I was confident in freeBSD ;-) Thanks a lot, -- __ Bernard DUGAS | | | Technoparc Pays de Gex mailto:[EMAIL PROTECTED] | | 30 Rue Auguste Piccard Tel.: +33 615 333 770 | | FR 01630 St Genis Pouilly Fax : +33 450 205 106 | |_| ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: how to talk to the serial and parallel ports through a C
Hi, I also did some programming on serial port to control pins with a c program under linux, but I didn't found any simple way to port it under freebsd. Any hint ? For instance, I need the equivalent of : inb(adr) outb(val,adr) iopl(n) ioperm(adr) from sys/io.h and stdio.h in linux. Best regard, Bernard Dugas -- Message: 32 Date: Mon, 09 Jun 2003 09:44:28 -0700 From: Dan Malaby [EMAIL PROTECTED] Subject: how to talk to the serial and parallel ports through a C program To: [EMAIL PROTECTED] Message-ID: [EMAIL PROTECTED] Content-Type: text/plain; charset=us-ascii; format=flowed I was wondering if there was a good place to go to get programing examples on how to talk to the serial and parallel ports. I have looked in the developers handbook but have not any luck finding what I want. Any pointers would be appreciated Thanks Daniel Malaby voice:(510) 531-6500 Peritek Corp. fax: (510) 530-8563 5550 Redwood Road email: [EMAIL PROTECTED] Oakland, CA 94619 -- -- __ Bernard DUGAS | | | Technoparc Pays de Gex mailto:[EMAIL PROTECTED] | | 30 Rue Auguste Piccard Tel.: +33 450 205 105 | | FR 01630 St Genis Pouilly Fax : +33 450 205 106 | |_| ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
