Re: [9fans] file server speed
perhaps high-efficiency wall warts could make up much of the difference. picked at random (first link) ... http://www.amazon.com/Nokia-AC-10U-Micro-USB-Efficiency-Charger/dp/B00DP0TQLG Given that the rpi has some weird power issues and without specification of the amperage of the charger this might be bad advice.
Re: [9fans] file server speed
On Fri Jul 18 12:51:49 EDT 2014, 23h...@gmail.com wrote: perhaps high-efficiency wall warts could make up much of the difference. picked at random (first link) ... http://www.amazon.com/Nokia-AC-10U-Micro-USB-Efficiency-Charger/dp/B00DP0TQLG Given that the rpi has some weird power issues and without specification of the amperage of the charger this might be bad advice. good point. thanks for pointing that out. - erik
Re: [9fans] file server speed
On Fri, 18 Jul 2014 15:36:09 EDT erik quanstrom quans...@quanstro.net wrote: On Fri Jul 18 12:51:49 EDT 2014, 23h...@gmail.com wrote: perhaps high-efficiency wall warts could make up much of the difference. picked at random (first link) ... http://www.amazon.com/Nokia-AC-10U-Micro-USB-Efficiency-Charger/dp/B00DP0 TQLG Given that the rpi has some weird power issues and without specification of the amperage of the charger this might be bad advice. good point. thanks for pointing that out. The new B+ has a switching regulator. It uses less power and it is more tolerant of voltage changes -- below 5V USB will stop working but there are reports of booting Linux to login screen on as low as 2.6V. So power issues should be resolved.
Re: [9fans] file server speed
I've used ReadyNAS appliances at home for almost 10 years. The current product line is made up of low-power Atoms. I'm running a RAID5 across 4 500G enterprise SATA drives (that should indicate how old this unit is pretty well...) I have a wired network primarily in the rack in the office at home - I absolutely would not use wireless to connect fossil to venti (fossil does *not* cope well with the connection to venti dropping). so this is absolutely a potential problem with ken's file server using aoe storage. the way i delt with the problem is to wait forever for aoe to come back. since there's no connection, there is no procedure for reestablishing a connection. there is also no overhead like a proxy connection. i gave a little paper at iwp9 about the diskless file server and a few weeks after we got back a bug in the storage caused it to stop responding. we were able to fix the storage, load a new image, and restart the storage kernel. the file server didn't miss a beat and none of the cpu servers had any issues. i would think the same approach would work with fossil. of course one would need a more sophisticated solution than just wait forever, due to the tcp connection. - erik
Re: [9fans] file server speed
My desire is to have one file server with auth server and any numbers of terminals which can also be used as cpu server (for drawterm). In this case the smallest config is a file server and a terminal/cpu server. Ken's file server is standallone and has special user space. Then can we add auth functinality on this file server machine? sure one could. but there is no user space, so this would need to be bespoke code. i'm surprised that running one extra raspberry pi only when working would make much of a difference. the total cost couldn't be much more than 10w, even with the power supply losses. the monitor likely uses much more power. perhaps high-efficiency wall warts could make up much of the difference. picked at random (first link) ... http://www.amazon.com/Nokia-AC-10U-Micro-USB-Efficiency-Charger/dp/B00DP0TQLG - erik
Re: [9fans] file server speed
On Thu, Jul 17, 2014 at 6:43 AM, Steven Stallion sstall...@gmail.com wrote: On Wed, Jul 16, 2014 at 9:53 AM, Ramakrishnan Muthukrishnan vu3...@gmail.com wrote: I am very interested to use such a setup. Could you please add more about the setup? What hardware do you use for the NAS? Any scripts etc? Sure thing - I've copied everything you should need under sources/contrib/stallion/venti (http://plan9.bell-labs.com/sources/contrib/stallion/venti/) Thanks, Steve. -- Ramakrishnan
Re: [9fans] file server speed
On Jul 17, 2014, at 9:56 AM, erik quanstrom quans...@quanstro.net wrote: i would think the same approach would work with fossil. of course one would need a more sophisticated solution than just wait forever, due to the tcp connection. There is no particular reason for it to be tcp given lack of any authentication. The beauty of CAS is that it need not even talk to the same server!
Re: [9fans] file server speed
There is no particular reason for it to be tcp given lack of any authentication. The beauty of CAS is that it need not even talk to the same server but it is! even when venti and fossil are on the same machine. - erik
Re: [9fans] file server speed
On Thu, 17 Jul 2014 11:14:01 PDT Bakul Shah ba...@bitblocks.com wrote: On Jul 17, 2014, at 9:56 AM, erik quanstrom quans...@quanstro.net wrote: i would think the same approach would work with fossil. of course one would need a more sophisticated solution than just wait forever, due to the tcp connection. There is no particular reason for it to be tcp given lack of any authentication. The beauty of CAS is that it need not even talk to the same server! To expand this a bit: So long as a server returns a block corresponding to its SHA1 score, you (the client) don't care whether it is the same server you wrote the original block to or another (and you can always verify the returned block). This opens up some interesting choices. For instance, you can multicast each read or write or you can set up a hierarchy of servers and switch to another one (or even get different blocks from different instances). How the writes are handled/replicated.stored is entirely upto the server (cluster). Or you can implement a bittorrent like facility on top of venti (I call it bitventi or b20). I have been playing with Lucho's 'govt' package to explore stuff like this (though staying with the current venti wire protocol for now).
Re: [9fans] file server speed
So long as a server returns a block corresponding to its SHA1 score, you (the client) don't care whether it is the same server you wrote the original block to or another (and you can always verify the returned block). This opens up some but this isn't unique to content-addressed storage. as long as the block contents are the same, normal block storage can be served from anywhere. interesting choices. For instance, you can multicast each read or write or you can set up a hierarchy of servers and switch to another one (or even get different blocks from different instances). How the writes are handled/replicated.stored is entirely upto the server (cluster). Or you can implement a bittorrent like facility on top of venti (I call it bitventi or b20). why not distributed hashing? distributed hashes have existing fast implementations. venti does not. - erik
Re: [9fans] file server speed
On Thu, 17 Jul 2014 15:10:34 EDT erik quanstrom quans...@quanstro.net wrote: So long as a server returns a block corresponding to its SHA1 score, you (the client) don't care whether it is the same server you wrote the original block to or another (and you can always verify the returned block). This opens up some but this isn't unique to content-addressed storage. as long as the block contents are the same, normal block storage can be served from anywhere. But this is a much stricter requirement for normal block storage (every replica has to have the same content for a given block number N and if you have disks of different sizes, you either waste storage or have to add another mapping layer). With CAS you have much more freedom. interesting choices. For instance, you can multicast each read or write or you can set up a hierarchy of servers and switch to another one (or even get different blocks from different instances). How the writes are handled/replicated.stored is entirely upto the server (cluster). Or you can implement a bittorrent like facility on top of venti (I call it bitventi or b20). why not distributed hashing? distributed hashes have existing fast implementations. venti does not. You can change implementations. Russ's ventino and Lucho's govt's vtmm server use different storage schemes from plan9's venti server. I am just playing with this stuff but if you want clustered, distributed storage you'd probably need some server side protocol to manage storage sanely. And add more complications if you want to improve performance.
Re: [9fans] file server speed
That was in an office environment. At home I use fossil+(plan9port)venti running on linux-based NAS. This ends up working very well for me since I have resources to spare on that machine. This also lets me backup my arenas via CrashPlan. I use a I am very interested to use such a setup. Could you please add more about the setup? What hardware do you use for the NAS? Any scripts etc? (disclaimer: i am fairly new to the plan9 universe, so my apologies for anything i missed. most of this is from memory.) my setup is slightly different, but similar in concept. my only machine is a mac, so i run plan9 under qemu.␣ i have a p9p venti running on the mac with a few qemu vm's using it. qemu also kind of solves the problem of the availability of compatible hardware for me. when i first installed plan9, i installed using the fossil+venti option. when i wanted to expand beyond a simple terminal, i decided to get venti out of the vm and run it off the mac using p9p. i also wanted to keep the data from the original venti. the old venti was ~8G, but used the default arena size of 512M. only 4 of the previous arenas were sealed with the the fifth active. i also added another 10G worth of arenas at 1G each, so this is roughly what i did. on plan9: for (i in 0 1 2 3 4) rdarena /dev/sdC0/arenas arenas0$i /tmp/arenas0$i on the mac: dd if=/dev/zero of=arenas1.img bs=1024 count=2622212 dd if=/dev/zero of=arenas2.img bs=1024 count=10486532 dd if=/dev/zero of=isect.img bs=1024k count=1024 dd if=/dev/zero of=bloom.img bs=1024k count=128 fmtarenas arenas1 arenas1.img fmtarenas -a 1024m arenas2 arenas2.img fmtisect isect isect.img fmtbloom -N 24 bloom.img fmtindex venti.conf buildindex -b venti.conf $PLAN9/bin/venti/venti -c venti.conf back to plan9: for (i in /tmp/arenas*) venti/wrarena -h tcp!$ventiaddr!17034 $i -- i keep the files on an apple raid mirror for venti. this is my venti.conf: index main arenas /Volumes/venti/arenas1.img arenas /Volumes/venti/arenas2.img isect /Volumes/venti/isect.img bloom /Volumes/venti/bloom.img mem 48m bcmem 96m icmem 128m addr tcp!*!17034 httpaddr tcp!*!8000 webroot /Volumes/venti/webroot -- the end result is that i had a 12.5G venti with a 1G isect and all my previous data content. the venti memory footprint is approximately 650M. it works quite well for me. in all instances below, i have adjusted my configuration files to what one would use if using separate machines. we have static ip addressing here, so my config will reflect that. substitute $ventiaddr for the ip address of your p9p venti server, $gateway for your router, $ipmask for your netmask, and $clientip for the machine(s) using the p9p venti. my auth+fs vm (the main p9p venti client) has the usual 9fat, nvram, swap, and fossil partitions with plan9.ini as follows: bootfile=sdC0!9fat!9pccpu console=0 b115200 l8 pn s1 *noe820scan=1 bootargs=local!#S/sdC0/fossi bootdisk=local!#S/sdC0/fossil nobootprompt=local!#S/sdC0/fossil venti=tcp!$ventiaddr!17034 -g $gateway ether /net/ether0 $clientip $ipmask sysname=p9auth -- the other cpu vm's pxe boot off the auth+fs vm using pxe config files like this: bootfile=ether0!/386/9pccpu console=0 b115200 l8 pn s1 readparts= nvram=/dev/sdC0/nvram *noe820scan=1 nobootprompt=tcp sysname=whatever -- since two of my cpu vm's have local fossils that i would like to keep archived, i added the following to /bin/cpurc.local: venti=tcp!$ventiaddr!17034 and then i start the fossils from /cfg/$sysname/cpurc. hope this is helpful in some way.
Re: [9fans] file server speed
On Tue, Jul 15, 2014 at 7:04 PM, kokam...@hera.eonet.ne.jp wrote: not a fair comparsion. Yes, I'd have been more specific. my intension was cwfs fossil+venti of 9atom fossil+venti labs. I did not consider kenfs itself, because I consider it should be file+auth+cpu server. The last is not important, but for drawterm from others. Recent kenfs can be such a machine? Please remember I plan it for my private home machine, not any sofisticated office use. kenfs works well, but you have to be well prepared to maintain it. Invest in a decent UPS - preferably one that is supported by the auto-shutdown (ISTR support was added for that a while back). You need to be careful when sizing your cache - I would invest in a pair of decent SSDs for cache, and two or more drives for housing the WORM. Be prepared for failure. The last large kensfs I maintained (around 16TB usable, 48TB raw) worked very well but would still crash one every year or two. Make sure you keep hard copies of your fsconfig and get comfortable with scripting as erik mentioned. That was in an office environment. At home I use fossil+(plan9port)venti running on linux-based NAS. This ends up working very well for me since I have resources to spare on that machine. This also lets me backup my arenas via CrashPlan. I use a cheap SSD for fossil in my file server and a small SATA DOM for booting (the idea is I can throw away the SSD at any time and still recover). Speed has been on par with kenfs, but it takes a little work to get there. I hope this is useful to you - maintaining an fs shouldn't be taken on lightly, but it is one of the best ways to understand what it means to maintain a plan9 installation. Cheers, Steve
Re: [9fans] file server speed
Recent kenfs can be such a machine? Please remember I plan it for my private home machine, not any sofisticated office use. i use ken's file server for personal use. i enjoy the fact that a cpu kernel panic does not impact the file server. - erik
Re: [9fans] file server speed
On Wed Jul 16 13:06:16 EDT 2014, kokam...@hera.eonet.ne.jp wrote: kenfs(of course 64 bit)+auth server +++9pi terminal/cpu server may be best for home use... i would go ahead and use to raspberry pi machines. having a dedicated cpu server is quite nice, and of course ken's file server is not an auth server. - erik
Re: [9fans] file server speed
kenfs works well, but you have to be well prepared to maintain it. Invest in a decent UPS - preferably one that is supported by the auto-shutdown (ISTR support was added for that a while back). You need to be careful when sizing your cache - I would invest in a pair of decent SSDs for cache, and two or more drives for housing the WORM. Be prepared for failure. The last large kensfs I maintained (around 16TB usable, 48TB raw) worked very well but would still crash one every year or two. Make sure you keep hard copies of your fsconfig and get comfortable with scripting as erik mentioned. the cache comment is spot on. i've stuck with 20GB caches. this is because a very large cache will use too many buckets. buckets are stored on disk in the cache, but a bucket needs to be in memory to use that part of the cache. with too many buckets, most of the i/o will be thrashing through different buckets. i'm very familiar with doing this wrong, as i set up one machine (kibbiee, for those who remember) with a 750G cache. that was a bad idea. i'd have to add that after running ken's file server at home since 2005, and at least half a dozen in work environments, the file system has been pretty good with my data. from 2006 to 2008 i ran with local scsi disks on an old va linux box. in that location i had no ups, and lightning took out power about once a week. suprisingly, this didn't cause any trouble. the only time i did have trouble was not the file server's fault entirely. the storage was disconnected during a dump, and then the file server was rebooted. in that case, only the corrupt dump was lost. clearly any file server is ideally on a ups, with generator backup and automatic shutdown. :-) in my experience, using ssds for cache doesn't speed up the file server much, since one is usually rtt limited, and by craftiness with the cache, files are typically written sequentially to the disk. i have ssds in my current file server, and am a bit disappointed in the performance. - erik
Re: [9fans] file server speed
i use ken's file server for personal use. i enjoy the fact that a cpu kernel panic does not impact the file server. My desire is to have one file server with auth server and any numbers of terminals which can also be used as cpu server (for drawterm). In this case the smallest config is a file server and a terminal/cpu server. Ken's file server is standallone and has special user space. Then can we add auth functinality on this file server machine? Kenji
Re: [9fans] file server speed
That was in an office environment. At home I use fossil+(plan9port)venti running on linux-based NAS. Do you use wireless LAN? If so you also need wireless bridge? The combination of NAS and venti sounds like charm, because the snmallest config is two machines. How about the power-eating of that machine? Recent low-power machine can do that task? Kenji
Re: [9fans] file server speed
On Wed, Jul 16, 2014 at 6:15 PM, kokam...@hera.eonet.ne.jp wrote: That was in an office environment. At home I use fossil+(plan9port)venti running on linux-based NAS. Do you use wireless LAN? If so you also need wireless bridge? The combination of NAS and venti sounds like charm, because the snmallest config is two machines. How about the power-eating of that machine? Recent low-power machine can do that task? I've used ReadyNAS appliances at home for almost 10 years. The current product line is made up of low-power Atoms. I'm running a RAID5 across 4 500G enterprise SATA drives (that should indicate how old this unit is pretty well...) I have a wired network primarily in the rack in the office at home - I absolutely would not use wireless to connect fossil to venti (fossil does *not* cope well with the connection to venti dropping). I switched over to fossil on the ReadyNAS a little over a year ago and have had really good luck; not a single crash. Performance has also been very good. It just so happens I wrote a README at the time since it was non-obvious how to set it up correctly: https://dl.dropboxusercontent.com/u/102312978/FOSSIL Cheers, Steve
Re: [9fans] file server speed
On Wed, 16 Jul 2014 19:29:43 CDT Steven Stallion sstall...@gmail.com wrote: I absolutely would not use wireless to connect fossil to venti (fossil does *not* cope well with the connection to venti dropping). To deal with this you can use a local venti proxy like contrib/vsrinvas/vtrc.c. One change I would like is to see is to cache all the blocks for the last (or last N) snapshots so that disconnected operation is possible. Not sure if there is a way to do this within fossil.
Re: [9fans] file server speed
On Wed, Jul 16, 2014 at 9:53 AM, Ramakrishnan Muthukrishnan vu3...@gmail.com wrote: I am very interested to use such a setup. Could you please add more about the setup? What hardware do you use for the NAS? Any scripts etc? Sure thing - I've copied everything you should need under sources/contrib/stallion/venti (http://plan9.bell-labs.com/sources/contrib/stallion/venti/) Steve
Re: [9fans] file server speed
Sorry, found it now. On 07/17/2014 07:31 AM, Steven Stallion wrote: On Wed, Jul 16, 2014 at 7:29 PM, Steven Stallion sstall...@gmail.com wrote: On Wed, Jul 16, 2014 at 6:15 PM, kokam...@hera.eonet.ne.jp wrote: It just so happens I wrote a README at the time since it was non-obvious how to set it up correctly: Corrected link: https://dl.dropboxusercontent.com/u/102312978/FOSSIL%2BVENTI -- John Francis Lee Thanon Sanam Gila, Ban Fa Sai 79/151 Moo 22 T. Ropwieng Mueang Chiangrai 57000 Thailand
Re: [9fans] file server speed
Not Found The resource could not be found. WSGI Server On 07/17/2014 07:29 AM, Steven Stallion wrote: On Wed, Jul 16, 2014 at 6:15 PM, kokam...@hera.eonet.ne.jp wrote: That was in an office environment. At home I use fossil+(plan9port)venti running on linux-based NAS. Do you use wireless LAN? If so you also need wireless bridge? The combination of NAS and venti sounds like charm, because the snmallest config is two machines. How about the power-eating of that machine? Recent low-power machine can do that task? I've used ReadyNAS appliances at home for almost 10 years. The current product line is made up of low-power Atoms. I'm running a RAID5 across 4 500G enterprise SATA drives (that should indicate how old this unit is pretty well...) I have a wired network primarily in the rack in the office at home - I absolutely would not use wireless to connect fossil to venti (fossil does *not* cope well with the connection to venti dropping). I switched over to fossil on the ReadyNAS a little over a year ago and have had really good luck; not a single crash. Performance has also been very good. It just so happens I wrote a README at the time since it was non-obvious how to set it up correctly: https://dl.dropboxusercontent.com/u/102312978/FOSSIL Cheers, Steve -- John Francis Lee Thanon Sanam Gila, Ban Fa Sai 79/151 Moo 22 T. Ropwieng Mueang Chiangrai 57000 Thailand
Re: [9fans] file server speed
On Jul 15, 2014, at 2:13 , kokam...@hera.eonet.ne.jp wrote: I've experienced three kinds of Plan9 file servers, Lab's one, 9atom and plan9front. Can you clarify which file server, specifically, you're comparing for each of these? The Labs doesn't distribute kenfs any more, and venti+fossil is a very different model. The old kenfs should reliably trounce it on performance. Then there's newer options in both 9atom and 9front. Anthony signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [9fans] file server speed
not a fair comparsion. i'd look into kenfs for a fileserver only machine. might require some time to get it to work with your hardware tho. -- cinap
Re: [9fans] file server speed
On Tue Jul 15 12:31:57 EDT 2014, cinap_len...@felloff.net wrote: not a fair comparsion. i'd look into kenfs for a fileserver only machine. might require some time to get it to work with your hardware tho. if you have a recent 64-bit intel machine, the hardware support should be nearly the same as plan 9 for networking, and all ahci and aoe are supported for disks. getting kenfs going seems to trip a lot of people up. but 9atom's mkfsboot (see mkfsconf(8)) should make life easier. perhaps an image of the current system bootable from the first ahci device might be better - erik
Re: [9fans] file server speed
not a fair comparsion. Yes, I'd have been more specific. my intension was cwfs fossil+venti of 9atom fossil+venti labs. I did not consider kenfs itself, because I consider it should be file+auth+cpu server. The last is not important, but for drawterm from others. Recent kenfs can be such a machine? Please remember I plan it for my private home machine, not any sofisticated office use. Kenji
Re: [9fans] file server speed
kenfs(of course 64 bit)+auth server +++9pi terminal/cpu server may be best for home use... Kenji
Re: [9fans] file server uptime
On Sun, Dec 15, 2013 at 5:27 PM, erik quanstrom quans...@labs.coraid.com wrote: i believe this is a personal record for any system at any time: plano: version 63-bit plano as of Thu Jan 6 13:20:07 EDT 2011 last boot Fri Jan 21 14:57:19 EDT 2011 Pish. Plano is barely used!
Re: [9fans] file server design documentation
I don't think either of those is a special rule, or undocumented. Indeed, I can't think of any behaviour that isn't covered in section 5. Following an attach, you're at the root, and a walk with no qids will leave you there, so stat will just work, although I don't know anything that relies on that. Walk of .. in the root of a file system leaves you there (see walk(5)). On 21 December 2012 18:23, steve st...@quintile.net wrote: the initial stat of a zero length name should return the Dir of the root dir. and a walk up to the root directory tells mount driver that you want to pass control back to the parent filesystem.
Re: [9fans] file server design documentation
Following an attach, you're at the root, and a walk with no qids will leave you there, so stat will just work, although I don't know anything that relies on that. cclone(). - erik
Re: [9fans] file server design documentation
Sorry, I was unclear, I meant stat( ...) On 22 December 2012 15:59, erik quanstrom quans...@quanstro.net wrote: , although I don't know anything that relies on that.
Re: [9fans] file server design documentation
I normally use a combination of running iostats and ramfs with debugging and reading again and again intro(5). HTH. G. On Dec 21, 2012, at 7:23 PM, steve st...@quintile.net wrote: hi, I writing another non-disc file server after a gap of a few years and am making mistakes. is there a written spec of how its supposed to work? For example: the initial stat of a zero length name should return the Dir of the root dir. and a walk up to the root directory tells mount driver that you want to pass control back to the parent filesystem. -Steve
Re: [9fans] file server design documentation
IIRC, I think I wrote something about that in the 9.intro book. But it's likely you already know all that's written there and you want more details… On Dec 21, 2012, at 7:23 PM, steve st...@quintile.net wrote: hi, I writing another non-disc file server after a gap of a few years and am making mistakes. is there a written spec of how its supposed to work? For example: the initial stat of a zero length name should return the Dir of the root dir. and a walk up to the root directory tells mount driver that you want to pass control back to the parent filesystem. -Steve
Re: [9fans] file server design documentation
I like to 1+ this comment :) On Fri, Dec 21, 2012 at 10:44 AM, Gorka Guardiola pau...@gmail.com wrote: I normally use a combination of running iostats and ramfs with debugging and reading again and again intro(5). HTH. G. On Dec 21, 2012, at 7:23 PM, steve st...@quintile.net wrote: hi, I writing another non-disc file server after a gap of a few years and am making mistakes. is there a written spec of how its supposed to work? For example: the initial stat of a zero length name should return the Dir of the root dir. and a walk up to the root directory tells mount driver that you want to pass control back to the parent filesystem. -Steve
Re: [9fans] file server?
2009/8/14 Lyndon Nerenberg lyn...@orthanc.ca: This is what we do at Sandia. We have one machine which serves cpu/auth/file, but the actual Venti disks are in a Coraid connected via GigE. The fossil disk is in the server, but if it dies we can just build a new one. Which reminds me of an often overlooked but important point: Save your fossil vac scores on another machine! Amen, a dozen times! Having lost the lot once or twice due to stupidity or hardware failures I've now got this under control, still not entirely happy with my venti arena archiving and seeking inspiration there. I'll add the description of my trivial home system to the mix. A ubuntu linux PC running p9p's venti, a plan9 cpu/fossil-fileserver running under QEMU and drawterm. A few times I've tried running 9vx but for unknown reasons I can crash it as soon as I look at it sideways - I suspect either operator error or oddities of my setup. All running on a P4 shuttle system tucked into the corner of my desk. Without them, your seperate venti server is JBOD :-P Well, not quite. You can eventually find the right vac score, but you have to manually mount each and every score in the venti until you find the right one. See /sys/src/cmd/venti/words/dumpvacroots. You could probably semi-automate the process by writing a script that mounted each of the scores in turn, checking the mtime of something like /sys/log/timesync in each, and sorting the vac scores accordingly. On my setup I aux/clog the fileserver console to a u9fs mounted directory on a UNIX server. You could also cobble something up that scans the fossil console for vac scores and emails them to an offsite address. For me, cron runs fossil/last and mails it to gmail --lyndon Adrain -- Adrian
Re: [9fans] file server?
This is what we do at Sandia. We have one machine which serves cpu/auth/file, but the actual Venti disks are in a Coraid connected via GigE. The fossil disk is in the server, but if it dies we can just build a new one. Which reminds me of an often overlooked but important point: Save your fossil vac scores on another machine! Without them, your seperate venti server is JBOD :-P Well, not quite. You can eventually find the right vac score, but you have to manually mount each and every score in the venti until you find the right one. See /sys/src/cmd/venti/words/dumpvacroots. You could probably semi-automate the process by writing a script that mounted each of the scores in turn, checking the mtime of something like /sys/log/timesync in each, and sorting the vac scores accordingly. On my setup I aux/clog the fileserver console to a u9fs mounted directory on a UNIX server. You could also cobble something up that scans the fossil console for vac scores and emails them to an offsite address. --lyndon
Re: [9fans] file server?
On Fri, Aug 14, 2009 at 7:14 AM, Lyndon Nerenberg lyn...@orthanc.ca wrote:
This is what we do at Sandia. We have one machine which serves
cpu/auth/file, but the actual Venti disks are in a Coraid connected
via GigE. The fossil disk is in the server, but if it dies we can just
build a new one.
Which reminds me of an often overlooked but important point:
Save your fossil vac scores on another machine!
I have a nightly cron job that sends the latest output of the fossil console
as an email.
In cpurc of the fileserver I setup a
aux/clog /srv/fscmd /sys/log/fossil
and my nightly batch just does this:
who = (myself fos...@some.other.place)
today = `{date}
tail -20 /sys/log/fossil |mail -s 'Fossil output of ' ^ $today $who
The last 20 lines is probably overkill, but who cares.
The venti archive starts at 2AM, and my cron job is at 4AM. So far, I've not
yet had an archive take longer than 2 hours. But that's partly due to
triggering one explicitly after a pull that's just replaced all my
executables ;-)
Robby
Re: [9fans] file server?
The venti archive starts at 2AM, and my cron job is at 4AM. So far, I've not yet had an archive take longer than 2 hours. But that's partly due to triggering one explicitly after a pull that's just replaced all my executables ;-) that's surprising to me that it would take that long. is that typical? just on the back of the proverbial envelope ... a completely seek-bound disk with 8k blocks gets ~3mb/s even when diabolicly seek bound, lets take something way more pessimistic like 0.5mb/s for 1 hr = 1800mb. 180mb dumps would still be large, so you must be getting on the order of 50kb/s dumps? i'm not a venti expert, but i find it hard to believe that venti would be that slow normally. do you need to resize memory or your bloom filter? hope my display of ignorance here doesn't seem like criticism. - erik
Re: [9fans] file server?
Without them, your seperate venti server is JBOD :-P Well, not quite. You can eventually find the right vac score, but you have to manually mount each and every score in the venti until you find the right one. See /sys/src/cmd/venti/words/dumpvacroots. You could probably semi-automate the process by writing a script that mounted each of the scores in turn, checking the mtime of something like /sys/log/timesync in each, and sorting the vac scores accordingly. any reason fossil can't haul its own water? - erik
Re: [9fans] file server?
On Fri, Aug 14, 2009 at 1:07 PM, erik quanstrom quans...@quanstro.netwrote: The venti archive starts at 2AM, and my cron job is at 4AM. So far, I've not yet had an archive take longer than 2 hours. But that's partly due to triggering one explicitly after a pull that's just replaced all my executables ;-) that's surprising to me that it would take that long. is that typical? just on the back of the proverbial envelope ... a completely seek-bound disk with 8k blocks gets ~3mb/s even when diabolicly seek bound, lets take something way more pessimistic like 0.5mb/s for 1 hr = 1800mb. 180mb dumps would still be large, so you must be getting on the order of 50kb/s dumps? i'm not a venti expert, but i find it hard to believe that venti would be that slow normally. do you need to resize memory or your bloom filter? hope my display of ignorance here doesn't seem like criticism. - erik I am running an old (pre bloom) venti on a 500MHz VIA with some crappy no-name 30GB PATA drive. But, yes, I would be surprised if an archival dump took longer than a few minutes (from looking at my logs anything from 45sec to 4.5min is typical, but I haven't looked at the amounts of data). Saying that, those pulls from a while ago, where pretty much the whole file tree got replaced did lead to a dump taking almost an hour. Robby
Re: [9fans] file server?
I drawterm all the time. Lately I have started using 9vx for quick hacks and then cpu from it to wherever I want to go. It has its quirks (not as stable as drawterm) but I'm not complaining. With drawterm being so solid and well integrated with X11/OSX I haven't had the need to dedicate a terminal to Plan 9 since 2005. If I need to spend a relaxing day at work I simply fire dt full-screen and look at the colors :) So, I guess that means venti+fossil+cpu on one headless machine in some forgotten corner of the datacentre. andrey
Re: [9fans] file server?
So, I guess that means venti+fossil+cpu on one headless machine in some forgotten corner of the datacentre. regardless of one's terminal accomidations, i still think it makes a lot of sense to have a stand-alone fileserver. it really does stink if your fs goes down for no reason at all. this is especially true if you're doing a lot of experimenting or don't have a proper terminal. in fact, i think it makes sense to devote two machines to one's fileserver. one diskless fileserver and an coraid storage appliance. :-) shameless, aren't i? - erik
Re: [9fans] file server?
no that data center here, but hopefully at a corner of the room On Thu, Aug 13, 2009 at 9:55 PM, andrey mirtchovskimirtchov...@gmail.com wrote: I drawterm all the time. Lately I have started using 9vx for quick hacks and then cpu from it to wherever I want to go. It has its quirks (not as stable as drawterm) but I'm not complaining. With drawterm being so solid and well integrated with X11/OSX I haven't had the need to dedicate a terminal to Plan 9 since 2005. If I need to spend a relaxing day at work I simply fire dt full-screen and look at the colors :) So, I guess that means venti+fossil+cpu on one headless machine in some forgotten corner of the datacentre. andrey -- Federico G. Benavento
Re: [9fans] file server?
Once, it used to be the standard configuration to have one machine as a CPU/auth server, one machine as a file server, and one machine as a terminal, for a total of three systems, if one had the available hardware. The power in that model comes primarily when you have a number of terminals being supported by the file and cpu/auth servers. Throwing bigger disks on the file server gives everyone more space. Upgrading the big honkin' cpu server gives everyone a speed boost. Of course, these days, a $400 laptop is more honkin' (in many ways) than big systems of just a few years ago. Still the separation of functionality does have advantages. But I digress... What's the recommended setup now? Are most people using a combined cpu/auth/file server running Fossil+Venti, or is the recommendation to use a seperate fossil+venti server dedicated to file serving, and another to serve as CPU/auth? Currently, I'm running a combined file/cpu/auth server, and I run 9vx in FreeBSD as a terminal connecting to my server. While not mentioned often, 9vx has a -b option that allows you to point it to a file server the same way a terminal does. I can also cpu into the cpu server just like with a real terminal. And when I'm away from home with my laptop, I can still run 9vx with a local file system. As soon as my supply of round tuits is replinished, I intend to put a CPU server in place. BLS
Re: [9fans] file server?
On Thu, Aug 13, 2009 at 6:01 PM, erik quanstromquans...@quanstro.net wrote: So, I guess that means venti+fossil+cpu on one headless machine in some forgotten corner of the datacentre. regardless of one's terminal accomidations, i still think it makes a lot of sense to have a stand-alone fileserver. it really does stink if your fs goes down for no reason at all. this is especially true if you're doing a lot of experimenting or don't have a proper terminal. in fact, i think it makes sense to devote two machines to one's fileserver. one diskless fileserver and an coraid storage appliance. :-) shameless, aren't i? - erik This is what we do at Sandia. We have one machine which serves cpu/auth/file, but the actual Venti disks are in a Coraid connected via GigE. The fossil disk is in the server, but if it dies we can just build a new one. John -- Object-oriented design is the roman numerals of computing -- Rob Pike
Re: [9fans] file server?
This is what we do at Sandia. We have one machine which serves cpu/auth/file, but the actual Venti disks are in a Coraid connected via GigE. The fossil disk is in the server, but if it dies we can just build a new one. coraid's configuration using ken's fs is outlined here http://www.quanstro.net/plan9/disklessfs.pdf the one change since the paper has been that the wireless connection has been replaced by 100Mbit fiber. my own setup is here http://www.quanstro.net/plan9/fs.html the advantage to such a setup is than when disks do stupid things, you can take care of the problem without disturbing the fileserver. - erik
Re: [9fans] file server?
regardless of one's terminal accomidations, i still think it makes a lot of sense to have a stand-alone fileserver. it really does stink if your fs goes down for no reason at all. this is especially true if you're doing a lot of experimenting or don't have a proper terminal. Amen! Three times now I've lost everything due to a combined CPU/fileserver crashing. The joy of a diskless CPU server is it cannot corrupt your filesystem no matter what manner of stupid things you do. My current fileserver runs fossil, venti, keyfs, secstored, dhcpd, and tftpd. I.e. only the bare minimum necessary to provide fileservice and bootstrap the diskless machines. (I will soon be moving the venti to a different machine to eliminate the current single point of disk failure.) It doesn't even have a keyboard hooked up. And it's rock solid. Real work is done on the diskless CPU server. Currently I drawterm to the CPU box, but that's because the terminal machine is in pieces waiting for some hardware upgrades. --lyndon
