Re: [9fans] file server speed
perhaps high-efficiency wall warts could make up much of the difference. picked at random (first link) ... http://www.amazon.com/Nokia-AC-10U-Micro-USB-Efficiency-Charger/dp/B00DP0TQLG Given that the rpi has some weird power issues and without specification of the amperage of the charger this might be bad advice.
Re: [9fans] file server speed
On Fri Jul 18 12:51:49 EDT 2014, 23h...@gmail.com wrote: perhaps high-efficiency wall warts could make up much of the difference. picked at random (first link) ... http://www.amazon.com/Nokia-AC-10U-Micro-USB-Efficiency-Charger/dp/B00DP0TQLG Given that the rpi has some weird power issues and without specification of the amperage of the charger this might be bad advice. good point. thanks for pointing that out. - erik
Re: [9fans] file server speed
On Fri, 18 Jul 2014 15:36:09 EDT erik quanstrom quans...@quanstro.net wrote: On Fri Jul 18 12:51:49 EDT 2014, 23h...@gmail.com wrote: perhaps high-efficiency wall warts could make up much of the difference. picked at random (first link) ... http://www.amazon.com/Nokia-AC-10U-Micro-USB-Efficiency-Charger/dp/B00DP0 TQLG Given that the rpi has some weird power issues and without specification of the amperage of the charger this might be bad advice. good point. thanks for pointing that out. The new B+ has a switching regulator. It uses less power and it is more tolerant of voltage changes -- below 5V USB will stop working but there are reports of booting Linux to login screen on as low as 2.6V. So power issues should be resolved.
Re: [9fans] file server speed
I've used ReadyNAS appliances at home for almost 10 years. The current product line is made up of low-power Atoms. I'm running a RAID5 across 4 500G enterprise SATA drives (that should indicate how old this unit is pretty well...) I have a wired network primarily in the rack in the office at home - I absolutely would not use wireless to connect fossil to venti (fossil does *not* cope well with the connection to venti dropping). so this is absolutely a potential problem with ken's file server using aoe storage. the way i delt with the problem is to wait forever for aoe to come back. since there's no connection, there is no procedure for reestablishing a connection. there is also no overhead like a proxy connection. i gave a little paper at iwp9 about the diskless file server and a few weeks after we got back a bug in the storage caused it to stop responding. we were able to fix the storage, load a new image, and restart the storage kernel. the file server didn't miss a beat and none of the cpu servers had any issues. i would think the same approach would work with fossil. of course one would need a more sophisticated solution than just wait forever, due to the tcp connection. - erik
Re: [9fans] file server speed
My desire is to have one file server with auth server and any numbers of terminals which can also be used as cpu server (for drawterm). In this case the smallest config is a file server and a terminal/cpu server. Ken's file server is standallone and has special user space. Then can we add auth functinality on this file server machine? sure one could. but there is no user space, so this would need to be bespoke code. i'm surprised that running one extra raspberry pi only when working would make much of a difference. the total cost couldn't be much more than 10w, even with the power supply losses. the monitor likely uses much more power. perhaps high-efficiency wall warts could make up much of the difference. picked at random (first link) ... http://www.amazon.com/Nokia-AC-10U-Micro-USB-Efficiency-Charger/dp/B00DP0TQLG - erik
Re: [9fans] file server speed
On Thu, Jul 17, 2014 at 6:43 AM, Steven Stallion sstall...@gmail.com wrote: On Wed, Jul 16, 2014 at 9:53 AM, Ramakrishnan Muthukrishnan vu3...@gmail.com wrote: I am very interested to use such a setup. Could you please add more about the setup? What hardware do you use for the NAS? Any scripts etc? Sure thing - I've copied everything you should need under sources/contrib/stallion/venti (http://plan9.bell-labs.com/sources/contrib/stallion/venti/) Thanks, Steve. -- Ramakrishnan
Re: [9fans] file server speed
On Jul 17, 2014, at 9:56 AM, erik quanstrom quans...@quanstro.net wrote: i would think the same approach would work with fossil. of course one would need a more sophisticated solution than just wait forever, due to the tcp connection. There is no particular reason for it to be tcp given lack of any authentication. The beauty of CAS is that it need not even talk to the same server!
Re: [9fans] file server speed
There is no particular reason for it to be tcp given lack of any authentication. The beauty of CAS is that it need not even talk to the same server but it is! even when venti and fossil are on the same machine. - erik
Re: [9fans] file server speed
On Thu, 17 Jul 2014 11:14:01 PDT Bakul Shah ba...@bitblocks.com wrote: On Jul 17, 2014, at 9:56 AM, erik quanstrom quans...@quanstro.net wrote: i would think the same approach would work with fossil. of course one would need a more sophisticated solution than just wait forever, due to the tcp connection. There is no particular reason for it to be tcp given lack of any authentication. The beauty of CAS is that it need not even talk to the same server! To expand this a bit: So long as a server returns a block corresponding to its SHA1 score, you (the client) don't care whether it is the same server you wrote the original block to or another (and you can always verify the returned block). This opens up some interesting choices. For instance, you can multicast each read or write or you can set up a hierarchy of servers and switch to another one (or even get different blocks from different instances). How the writes are handled/replicated.stored is entirely upto the server (cluster). Or you can implement a bittorrent like facility on top of venti (I call it bitventi or b20). I have been playing with Lucho's 'govt' package to explore stuff like this (though staying with the current venti wire protocol for now).
Re: [9fans] file server speed
So long as a server returns a block corresponding to its SHA1 score, you (the client) don't care whether it is the same server you wrote the original block to or another (and you can always verify the returned block). This opens up some but this isn't unique to content-addressed storage. as long as the block contents are the same, normal block storage can be served from anywhere. interesting choices. For instance, you can multicast each read or write or you can set up a hierarchy of servers and switch to another one (or even get different blocks from different instances). How the writes are handled/replicated.stored is entirely upto the server (cluster). Or you can implement a bittorrent like facility on top of venti (I call it bitventi or b20). why not distributed hashing? distributed hashes have existing fast implementations. venti does not. - erik
Re: [9fans] file server speed
On Thu, 17 Jul 2014 15:10:34 EDT erik quanstrom quans...@quanstro.net wrote: So long as a server returns a block corresponding to its SHA1 score, you (the client) don't care whether it is the same server you wrote the original block to or another (and you can always verify the returned block). This opens up some but this isn't unique to content-addressed storage. as long as the block contents are the same, normal block storage can be served from anywhere. But this is a much stricter requirement for normal block storage (every replica has to have the same content for a given block number N and if you have disks of different sizes, you either waste storage or have to add another mapping layer). With CAS you have much more freedom. interesting choices. For instance, you can multicast each read or write or you can set up a hierarchy of servers and switch to another one (or even get different blocks from different instances). How the writes are handled/replicated.stored is entirely upto the server (cluster). Or you can implement a bittorrent like facility on top of venti (I call it bitventi or b20). why not distributed hashing? distributed hashes have existing fast implementations. venti does not. You can change implementations. Russ's ventino and Lucho's govt's vtmm server use different storage schemes from plan9's venti server. I am just playing with this stuff but if you want clustered, distributed storage you'd probably need some server side protocol to manage storage sanely. And add more complications if you want to improve performance.
Re: [9fans] file server speed
That was in an office environment. At home I use fossil+(plan9port)venti running on linux-based NAS. This ends up working very well for me since I have resources to spare on that machine. This also lets me backup my arenas via CrashPlan. I use a I am very interested to use such a setup. Could you please add more about the setup? What hardware do you use for the NAS? Any scripts etc? (disclaimer: i am fairly new to the plan9 universe, so my apologies for anything i missed. most of this is from memory.) my setup is slightly different, but similar in concept. my only machine is a mac, so i run plan9 under qemu.␣ i have a p9p venti running on the mac with a few qemu vm's using it. qemu also kind of solves the problem of the availability of compatible hardware for me. when i first installed plan9, i installed using the fossil+venti option. when i wanted to expand beyond a simple terminal, i decided to get venti out of the vm and run it off the mac using p9p. i also wanted to keep the data from the original venti. the old venti was ~8G, but used the default arena size of 512M. only 4 of the previous arenas were sealed with the the fifth active. i also added another 10G worth of arenas at 1G each, so this is roughly what i did. on plan9: for (i in 0 1 2 3 4) rdarena /dev/sdC0/arenas arenas0$i /tmp/arenas0$i on the mac: dd if=/dev/zero of=arenas1.img bs=1024 count=2622212 dd if=/dev/zero of=arenas2.img bs=1024 count=10486532 dd if=/dev/zero of=isect.img bs=1024k count=1024 dd if=/dev/zero of=bloom.img bs=1024k count=128 fmtarenas arenas1 arenas1.img fmtarenas -a 1024m arenas2 arenas2.img fmtisect isect isect.img fmtbloom -N 24 bloom.img fmtindex venti.conf buildindex -b venti.conf $PLAN9/bin/venti/venti -c venti.conf back to plan9: for (i in /tmp/arenas*) venti/wrarena -h tcp!$ventiaddr!17034 $i -- i keep the files on an apple raid mirror for venti. this is my venti.conf: index main arenas /Volumes/venti/arenas1.img arenas /Volumes/venti/arenas2.img isect /Volumes/venti/isect.img bloom /Volumes/venti/bloom.img mem 48m bcmem 96m icmem 128m addr tcp!*!17034 httpaddr tcp!*!8000 webroot /Volumes/venti/webroot -- the end result is that i had a 12.5G venti with a 1G isect and all my previous data content. the venti memory footprint is approximately 650M. it works quite well for me. in all instances below, i have adjusted my configuration files to what one would use if using separate machines. we have static ip addressing here, so my config will reflect that. substitute $ventiaddr for the ip address of your p9p venti server, $gateway for your router, $ipmask for your netmask, and $clientip for the machine(s) using the p9p venti. my auth+fs vm (the main p9p venti client) has the usual 9fat, nvram, swap, and fossil partitions with plan9.ini as follows: bootfile=sdC0!9fat!9pccpu console=0 b115200 l8 pn s1 *noe820scan=1 bootargs=local!#S/sdC0/fossi bootdisk=local!#S/sdC0/fossil nobootprompt=local!#S/sdC0/fossil venti=tcp!$ventiaddr!17034 -g $gateway ether /net/ether0 $clientip $ipmask sysname=p9auth -- the other cpu vm's pxe boot off the auth+fs vm using pxe config files like this: bootfile=ether0!/386/9pccpu console=0 b115200 l8 pn s1 readparts= nvram=/dev/sdC0/nvram *noe820scan=1 nobootprompt=tcp sysname=whatever -- since two of my cpu vm's have local fossils that i would like to keep archived, i added the following to /bin/cpurc.local: venti=tcp!$ventiaddr!17034 and then i start the fossils from /cfg/$sysname/cpurc. hope this is helpful in some way.
Re: [9fans] file server speed
On Tue, Jul 15, 2014 at 7:04 PM, kokam...@hera.eonet.ne.jp wrote: not a fair comparsion. Yes, I'd have been more specific. my intension was cwfs fossil+venti of 9atom fossil+venti labs. I did not consider kenfs itself, because I consider it should be file+auth+cpu server. The last is not important, but for drawterm from others. Recent kenfs can be such a machine? Please remember I plan it for my private home machine, not any sofisticated office use. kenfs works well, but you have to be well prepared to maintain it. Invest in a decent UPS - preferably one that is supported by the auto-shutdown (ISTR support was added for that a while back). You need to be careful when sizing your cache - I would invest in a pair of decent SSDs for cache, and two or more drives for housing the WORM. Be prepared for failure. The last large kensfs I maintained (around 16TB usable, 48TB raw) worked very well but would still crash one every year or two. Make sure you keep hard copies of your fsconfig and get comfortable with scripting as erik mentioned. That was in an office environment. At home I use fossil+(plan9port)venti running on linux-based NAS. This ends up working very well for me since I have resources to spare on that machine. This also lets me backup my arenas via CrashPlan. I use a cheap SSD for fossil in my file server and a small SATA DOM for booting (the idea is I can throw away the SSD at any time and still recover). Speed has been on par with kenfs, but it takes a little work to get there. I hope this is useful to you - maintaining an fs shouldn't be taken on lightly, but it is one of the best ways to understand what it means to maintain a plan9 installation. Cheers, Steve
Re: [9fans] file server speed
Recent kenfs can be such a machine? Please remember I plan it for my private home machine, not any sofisticated office use. i use ken's file server for personal use. i enjoy the fact that a cpu kernel panic does not impact the file server. - erik
Re: [9fans] file server speed
On Wed Jul 16 13:06:16 EDT 2014, kokam...@hera.eonet.ne.jp wrote: kenfs(of course 64 bit)+auth server +++9pi terminal/cpu server may be best for home use... i would go ahead and use to raspberry pi machines. having a dedicated cpu server is quite nice, and of course ken's file server is not an auth server. - erik
Re: [9fans] file server speed
kenfs works well, but you have to be well prepared to maintain it. Invest in a decent UPS - preferably one that is supported by the auto-shutdown (ISTR support was added for that a while back). You need to be careful when sizing your cache - I would invest in a pair of decent SSDs for cache, and two or more drives for housing the WORM. Be prepared for failure. The last large kensfs I maintained (around 16TB usable, 48TB raw) worked very well but would still crash one every year or two. Make sure you keep hard copies of your fsconfig and get comfortable with scripting as erik mentioned. the cache comment is spot on. i've stuck with 20GB caches. this is because a very large cache will use too many buckets. buckets are stored on disk in the cache, but a bucket needs to be in memory to use that part of the cache. with too many buckets, most of the i/o will be thrashing through different buckets. i'm very familiar with doing this wrong, as i set up one machine (kibbiee, for those who remember) with a 750G cache. that was a bad idea. i'd have to add that after running ken's file server at home since 2005, and at least half a dozen in work environments, the file system has been pretty good with my data. from 2006 to 2008 i ran with local scsi disks on an old va linux box. in that location i had no ups, and lightning took out power about once a week. suprisingly, this didn't cause any trouble. the only time i did have trouble was not the file server's fault entirely. the storage was disconnected during a dump, and then the file server was rebooted. in that case, only the corrupt dump was lost. clearly any file server is ideally on a ups, with generator backup and automatic shutdown. :-) in my experience, using ssds for cache doesn't speed up the file server much, since one is usually rtt limited, and by craftiness with the cache, files are typically written sequentially to the disk. i have ssds in my current file server, and am a bit disappointed in the performance. - erik
Re: [9fans] file server speed
i use ken's file server for personal use. i enjoy the fact that a cpu kernel panic does not impact the file server. My desire is to have one file server with auth server and any numbers of terminals which can also be used as cpu server (for drawterm). In this case the smallest config is a file server and a terminal/cpu server. Ken's file server is standallone and has special user space. Then can we add auth functinality on this file server machine? Kenji
Re: [9fans] file server speed
That was in an office environment. At home I use fossil+(plan9port)venti running on linux-based NAS. Do you use wireless LAN? If so you also need wireless bridge? The combination of NAS and venti sounds like charm, because the snmallest config is two machines. How about the power-eating of that machine? Recent low-power machine can do that task? Kenji
Re: [9fans] file server speed
On Wed, Jul 16, 2014 at 6:15 PM, kokam...@hera.eonet.ne.jp wrote: That was in an office environment. At home I use fossil+(plan9port)venti running on linux-based NAS. Do you use wireless LAN? If so you also need wireless bridge? The combination of NAS and venti sounds like charm, because the snmallest config is two machines. How about the power-eating of that machine? Recent low-power machine can do that task? I've used ReadyNAS appliances at home for almost 10 years. The current product line is made up of low-power Atoms. I'm running a RAID5 across 4 500G enterprise SATA drives (that should indicate how old this unit is pretty well...) I have a wired network primarily in the rack in the office at home - I absolutely would not use wireless to connect fossil to venti (fossil does *not* cope well with the connection to venti dropping). I switched over to fossil on the ReadyNAS a little over a year ago and have had really good luck; not a single crash. Performance has also been very good. It just so happens I wrote a README at the time since it was non-obvious how to set it up correctly: https://dl.dropboxusercontent.com/u/102312978/FOSSIL Cheers, Steve
Re: [9fans] file server speed
On Wed, 16 Jul 2014 19:29:43 CDT Steven Stallion sstall...@gmail.com wrote: I absolutely would not use wireless to connect fossil to venti (fossil does *not* cope well with the connection to venti dropping). To deal with this you can use a local venti proxy like contrib/vsrinvas/vtrc.c. One change I would like is to see is to cache all the blocks for the last (or last N) snapshots so that disconnected operation is possible. Not sure if there is a way to do this within fossil.
Re: [9fans] file server speed
On Wed, Jul 16, 2014 at 9:53 AM, Ramakrishnan Muthukrishnan vu3...@gmail.com wrote: I am very interested to use such a setup. Could you please add more about the setup? What hardware do you use for the NAS? Any scripts etc? Sure thing - I've copied everything you should need under sources/contrib/stallion/venti (http://plan9.bell-labs.com/sources/contrib/stallion/venti/) Steve
Re: [9fans] file server speed
Sorry, found it now. On 07/17/2014 07:31 AM, Steven Stallion wrote: On Wed, Jul 16, 2014 at 7:29 PM, Steven Stallion sstall...@gmail.com wrote: On Wed, Jul 16, 2014 at 6:15 PM, kokam...@hera.eonet.ne.jp wrote: It just so happens I wrote a README at the time since it was non-obvious how to set it up correctly: Corrected link: https://dl.dropboxusercontent.com/u/102312978/FOSSIL%2BVENTI -- John Francis Lee Thanon Sanam Gila, Ban Fa Sai 79/151 Moo 22 T. Ropwieng Mueang Chiangrai 57000 Thailand
Re: [9fans] file server speed
Not Found The resource could not be found. WSGI Server On 07/17/2014 07:29 AM, Steven Stallion wrote: On Wed, Jul 16, 2014 at 6:15 PM, kokam...@hera.eonet.ne.jp wrote: That was in an office environment. At home I use fossil+(plan9port)venti running on linux-based NAS. Do you use wireless LAN? If so you also need wireless bridge? The combination of NAS and venti sounds like charm, because the snmallest config is two machines. How about the power-eating of that machine? Recent low-power machine can do that task? I've used ReadyNAS appliances at home for almost 10 years. The current product line is made up of low-power Atoms. I'm running a RAID5 across 4 500G enterprise SATA drives (that should indicate how old this unit is pretty well...) I have a wired network primarily in the rack in the office at home - I absolutely would not use wireless to connect fossil to venti (fossil does *not* cope well with the connection to venti dropping). I switched over to fossil on the ReadyNAS a little over a year ago and have had really good luck; not a single crash. Performance has also been very good. It just so happens I wrote a README at the time since it was non-obvious how to set it up correctly: https://dl.dropboxusercontent.com/u/102312978/FOSSIL Cheers, Steve -- John Francis Lee Thanon Sanam Gila, Ban Fa Sai 79/151 Moo 22 T. Ropwieng Mueang Chiangrai 57000 Thailand
[9fans] file server speed
I've experienced three kinds of Plan9 file servers, Lab's one, 9atom and plan9front. I felt that the file server speed is 9front 9atom lab's. This is based on verious different machines for each. Is my feeling wrong, or hasve some base facts? This is to prepare the fastest file server on a planed relatively weak energy-saving machine. Kenji
Re: [9fans] file server speed
On Jul 15, 2014, at 2:13 , kokam...@hera.eonet.ne.jp wrote: I've experienced three kinds of Plan9 file servers, Lab's one, 9atom and plan9front. Can you clarify which file server, specifically, you're comparing for each of these? The Labs doesn't distribute kenfs any more, and venti+fossil is a very different model. The old kenfs should reliably trounce it on performance. Then there's newer options in both 9atom and 9front. Anthony signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [9fans] file server speed
not a fair comparsion. i'd look into kenfs for a fileserver only machine. might require some time to get it to work with your hardware tho. -- cinap
Re: [9fans] file server speed
On Tue Jul 15 12:31:57 EDT 2014, cinap_len...@felloff.net wrote: not a fair comparsion. i'd look into kenfs for a fileserver only machine. might require some time to get it to work with your hardware tho. if you have a recent 64-bit intel machine, the hardware support should be nearly the same as plan 9 for networking, and all ahci and aoe are supported for disks. getting kenfs going seems to trip a lot of people up. but 9atom's mkfsboot (see mkfsconf(8)) should make life easier. perhaps an image of the current system bootable from the first ahci device might be better - erik
Re: [9fans] file server speed
not a fair comparsion. Yes, I'd have been more specific. my intension was cwfs fossil+venti of 9atom fossil+venti labs. I did not consider kenfs itself, because I consider it should be file+auth+cpu server. The last is not important, but for drawterm from others. Recent kenfs can be such a machine? Please remember I plan it for my private home machine, not any sofisticated office use. Kenji
Re: [9fans] file server speed
kenfs(of course 64 bit)+auth server +++9pi terminal/cpu server may be best for home use... Kenji
