Re: [Samba] Ideas for distributed Samba servers
On 12/04/2010 06:21, Robert LeBlanc wrote: Ever heard of glusterfs? The exact thing I was going to ask :) Yes, I don't think it works well in a geography diverse clusters though. I'll test it soon in a 2-node scenario, where nodes are on separate networks and max speed is limited by an optical link to about 100Mbps. Currently NFS shares accessed throught the link are OK (just a bit slower than local ones -- but we're speaking of two small labs: 10 PCs in one and 25 in the other). And it adds full replication (instead of just a cache) for free, making only the diffs travel throught the slow link. Maybe it could take a long time after a split brain to resync. Their appliance supports exporting as samba share, but I don't know the granularity and how well it could be integrated in a domain. -- Diego Zuccato Servizi Informatici Dip. di Astronomia - Università di Bologna Via Ranzani, 1 - 40126 Bologna - Italy tel.: +39 051 20 95786 mail: diego.zucc...@unibo.it -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] Ideas for distributed Samba servers
On Mon, Apr 12, 2010 at 7:17 AM, Stan Hoeppner s...@hardwarefreak.com wrote: Robert LeBlanc put forth on 4/11/2010 8:19 PM: On Sun, Apr 11, 2010 at 9:03 AM, ravi channavajhala ravi.channavajh...@dciera.com wrote: WAFS (Wide Area File System) appliances can be very well deployed for this sort of thing precisely. Unfortunately, I don't know of any opensource project for WAFS. However, commercial solutions such as Riverbed, Expand Networks, CISCO/WAFS, Juniper/Peribit do exist. So far, this is the direction that we may go. We have looked at a Riverbed product, it's good to know alternatives. This may not be as much of an issue as it was in the past as I believe we my get a network upgrade that will negate the need for this. I would think it would be cheaper and more straight forward to replace the GbE port on each end of the fiber link with a 10GbE port than to deal with the complexity of caching and replication, or other such options, especially for two buildings on the same campus. The fiber link is on campus and thus you control any right-of-way issues, correct? I'd like to know if anyone else thinks this can work as well as a method with write back caching etc... Regards, /rkc -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] Ideas for distributed Samba servers
On Sat, 2010-04-10 at 10:14 -0700, Eric Shubert wrote: Robert LeBlanc wrote: I'm trying to think about how to setup a Samba system and would like to pick the brains of some experts. We are looking up put a large amount of storage ~75TB in a central data center. We have some remote (ok, not remote, but across slower links, ok if you consider several hundred clients over 1Gb to be slow) locations that we would like to set up samba servers that 'cache' the file system and serve it up to the clients in the building and sync with the main data center storage. a.) I don't think you can really do that with a 'file server' b.) I believe what you describe is almost exactly how AFS works. http://www.openafs.org/ OpenAFS is the world's foremost location independent file system. c.) Most SAN vendors provide a block-level replication solution for their products. The idea is have a couple of TB that are located in the building that serve up the Samba share. When a client requests a file, if it's in the local cache it is served up from there, if not then the Samba server grabs the file from the main data center and serves it to the client. When a file is written, something like rsync is used to transfer only difference back to the main data center. The problem is that I'm not sure of a file system that does this. We are using Lustre on our HPC, but this won't do what we want. With all the fun of file locking, concurrent access, etc... I think what you describe just won't work, or at least will never work well. Why not just you a groupware server that supports document check-out and check-in; that seems like the correct solution to me. Or possibly something like iFolder http://ifolder.com/ifolder -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] Ideas for distributed Samba servers
WAFS (Wide Area File System) appliances can be very well deployed for this sort of thing precisely. Unfortunately, I don't know of any opensource project for WAFS. However, commercial solutions such as Riverbed, Expand Networks, CISCO/WAFS, Juniper/Peribit do exist. Regards, /rkc CTO DCiEra (P) Ltd -Original Message- From: samba-boun...@lists.samba.org [mailto:samba-boun...@lists.samba.org] On Behalf Of Adam Tauno Williams Sent: Sunday, April 11, 2010 8:15 PM To: samba@lists.samba.org Subject: Re: [Samba] Ideas for distributed Samba servers On Sat, 2010-04-10 at 10:14 -0700, Eric Shubert wrote: Robert LeBlanc wrote: I'm trying to think about how to setup a Samba system and would like to pick the brains of some experts. We are looking up put a large amount of storage ~75TB in a central data center. We have some remote (ok, not remote, but across slower links, ok if you consider several hundred clients over 1Gb to be slow) locations that we would like to set up samba servers that 'cache' the file system and serve it up to the clients in the building and sync with the main data center storage. a.) I don't think you can really do that with a 'file server' b.) I believe what you describe is almost exactly how AFS works. http://www.openafs.org/ OpenAFS is the world's foremost location independent file system. c.) Most SAN vendors provide a block-level replication solution for their products. The idea is have a couple of TB that are located in the building that serve up the Samba share. When a client requests a file, if it's in the local cache it is served up from there, if not then the Samba server grabs the file from the main data center and serves it to the client. When a file is written, something like rsync is used to transfer only difference back to the main data center. The problem is that I'm not sure of a file system that does this. We are using Lustre on our HPC, but this won't do what we want. With all the fun of file locking, concurrent access, etc... I think what you describe just won't work, or at least will never work well. Why not just you a groupware server that supports document check-out and check-in; that seems like the correct solution to me. Or possibly something like iFolder http://ifolder.com/ifolder -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] Ideas for distributed Samba servers
On Sun, Apr 11, 2010 at 8:14 PM, Adam Tauno Williams awill...@whitemice.org wrote: On Sat, 2010-04-10 at 10:14 -0700, Eric Shubert wrote: Robert LeBlanc wrote: I'm trying to think about how to setup a Samba system and would like to pick the brains of some experts. We are looking up put a large amount of storage ~75TB in a central data center. We have some remote (ok, not remote, but across slower links, ok if you consider several hundred clients over 1Gb to be slow) locations that we would like to set up samba servers that 'cache' the file system and serve it up to the clients in the building and sync with the main data center storage. a.) I don't think you can really do that with a 'file server' b.) I believe what you describe is almost exactly how AFS works. http://www.openafs.org/ OpenAFS is the world's foremost location independent file system. c.) Most SAN vendors provide a block-level replication solution for their products. The idea is have a couple of TB that are located in the building that serve up the Samba share. When a client requests a file, if it's in the local cache it is served up from there, if not then the Samba server grabs the file from the main data center and serves it to the client. When a file is written, something like rsync is used to transfer only difference back to the main data center. The problem is that I'm not sure of a file system that does this. We are using Lustre on our HPC, but this won't do what we want. With all the fun of file locking, concurrent access, etc... I think what you describe just won't work, or at least will never work well. Why not just you a groupware server that supports document check-out and check-in; that seems like the correct solution to me. Or possibly something like iFolder http://ifolder.com/ifolder WAFS (Wide Area File System) appliances can be very well deployed for this sort of thing precisely. Unfortunately, I don't know of any opensource project for WAFS. However, commercial solutions such as Riverbed, Expand Networks, CISCO/WAFS, Juniper/Peribit do exist. Regards, /rkc CTO DCiEra (P) Ltd -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] Ideas for distributed Samba servers
On Sun, Apr 11, 2010 at 9:03 AM, ravi channavajhala ravi.channavajh...@dciera.com wrote: WAFS (Wide Area File System) appliances can be very well deployed for this sort of thing precisely. Unfortunately, I don't know of any opensource project for WAFS. However, commercial solutions such as Riverbed, Expand Networks, CISCO/WAFS, Juniper/Peribit do exist. So far, this is the direction that we may go. We have looked at a Riverbed product, it's good to know alternatives. This may not be as much of an issue as it was in the past as I believe we my get a network upgrade that will negate the need for this. Thanks, Robert LeBlanc Life Sciences Undergraduate Education Computer Support Brigham Young University -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] Ideas for distributed Samba servers
Robert LeBlanc put forth on 4/11/2010 8:19 PM: On Sun, Apr 11, 2010 at 9:03 AM, ravi channavajhala ravi.channavajh...@dciera.com wrote: WAFS (Wide Area File System) appliances can be very well deployed for this sort of thing precisely. Unfortunately, I don't know of any opensource project for WAFS. However, commercial solutions such as Riverbed, Expand Networks, CISCO/WAFS, Juniper/Peribit do exist. So far, this is the direction that we may go. We have looked at a Riverbed product, it's good to know alternatives. This may not be as much of an issue as it was in the past as I believe we my get a network upgrade that will negate the need for this. I would think it would be cheaper and more straight forward to replace the GbE port on each end of the fiber link with a 10GbE port than to deal with the complexity of caching and replication, or other such options, especially for two buildings on the same campus. The fiber link is on campus and thus you control any right-of-way issues, correct? If this is the case, upgrading the link speed on the fiber is definitely the way to go. If multiple pairs were run when the line was originally trenched, as is customary, setup ISL bonding of two 10GbE links between the two buildings' switches. Problem solved. Make sure you have at least one 10GbE NIC (preferably two NICs bonded) in the Samba server that exports the data on the disk array or the fat pipe between the buildings won't matter much. It will be interesting to see what Samba bottlenecks you run into after you get the big phat pipes setup. -- Stan -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] Ideas for distributed Samba servers
On Sun, Apr 11, 2010 at 7:47 PM, Stan Hoeppner s...@hardwarefreak.comwrote: I would think it would be cheaper and more straight forward to replace the GbE port on each end of the fiber link with a 10GbE port than to deal with the complexity of caching and replication, or other such options, especially for two buildings on the same campus. The fiber link is on campus and thus you control any right-of-way issues, correct? If this is the case, upgrading the link speed on the fiber is definitely the way to go. If multiple pairs were run when the line was originally trenched, as is customary, setup ISL bonding of two 10GbE links between the two buildings' switches. Problem solved. Make sure you have at least one 10GbE NIC (preferably two NICs bonded) in the Samba server that exports the data on the disk array or the fat pipe between the buildings won't matter much. It will be interesting to see what Samba bottlenecks you run into after you get the big phat pipes setup. Although the buildings are on the same campus (multiple buildings about 8 total that we occupy and only parts of building for most of the buildings) we don't have control over the network. That is in the hands of the campus IT organization and they like things done a certain way. We can light some fibre, but it's only point to point and we don't have that much fibre running to our building to connect all the buildings, plus the expense would be astronomical as we can't tie into their network and so connection in the other buildings would be limited. Since they are finally deciding to upgrade the core switching to 10GbE, they are possibly putting our building on the list to get a 10GbE link first. I think that would alleviate the biggest part of the problem, as we suspect that most of the storage will sit idle and not really accessed. Since all the desktops are only running 100 Mb connections, it gives us enough concurrent connections that we feel comfortable with. Robert LeBlanc Life Sciences Undergraduate Education Computer Support Brigham Young University -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] Ideas for distributed Samba servers
Ever heard of glusterfs? -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] Ideas for distributed Samba servers
On Sun, Apr 11, 2010 at 10:18 PM, Adam squeeze...@gmail.com wrote: Ever heard of glusterfs? Yes, I don't think it works well in a geography diverse clusters though. Lustre has this same problem. I could be wrong. Robert LeBlanc Life Sciences Undergraduate Education Computer Support Brigham Young University -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] Ideas for distributed Samba servers
Robert LeBlanc wrote: I'm trying to think about how to setup a Samba system and would like to pick the brains of some experts. We are looking up put a large amount of storage ~75TB in a central data center. We have some remote (ok, not remote, but across slower links, ok if you consider several hundred clients over 1Gb to be slow) locations that we would like to set up samba servers that 'cache' the file system and serve it up to the clients in the building and sync with the main data center storage. The idea is have a couple of TB that are located in the building that serve up the Samba share. When a client requests a file, if it's in the local cache it is served up from there, if not then the Samba server grabs the file from the main data center and serves it to the client. When a file is written, something like rsync is used to transfer only difference back to the main data center. The problem is that I'm not sure of a file system that does this. We are using Lustre on our HPC, but this won't do what we want. Any suggestions are welcome. Robert LeBlanc Life Sciences Undergraduate Education Computer Support Brigham Young University I'm curious to know what you came up with for this. Care you share? TIA. -- -Eric 'shubes' -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] Ideas for distributed Samba servers
On Sat, Apr 10, 2010 at 1:14 PM, Eric Shubert e...@shubes.net wrote: Robert LeBlanc wrote: I'm trying to think about how to setup a Samba system and would like to pick the brains of some experts. We are looking up put a large amount of storage ~75TB in a central data center. We have some remote (ok, not remote, but across slower links, ok if you consider several hundred clients over 1Gb to be slow) locations that we would like to set up samba servers that 'cache' the file system and serve it up to the clients in the building and sync with the main data center storage. The idea is have a couple of TB that are located in the building that serve up the Samba share. When a client requests a file, if it's in the local cache it is served up from there, if not then the Samba server grabs the file from the main data center and serves it to the client. When a file is written, something like rsync is used to transfer only difference back to the main data center. The problem is that I'm not sure of a file system that does this. We are using Lustre on our HPC, but this won't do what we want. Any suggestions are welcome. Robert LeBlanc Life Sciences Undergraduate Education Computer Support Brigham Young University I'm curious to know what you came up with for this. Care you share? TIA. It's a historically tricky problem, and takes serious thought about who is allowed to make changes, and how they propagate to remote locations, and about how synchronization is scheduled. One of my favorites has become git. Use local repositories, much as git is used for the Linux kernel, and submit changes to a central repository for users who are so authorized. The merging capabilities are pretty good, and changes can be made locally without committing them to the central repository. And by the way, it run on Samba one heck of a lot more efficiently and effectively than Subversion does. -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] Ideas for distributed Samba servers
On Sat, Apr 10, 2010 at 11:14 AM, Eric Shubert e...@shubes.net wrote: Robert LeBlanc wrote: I'm trying to think about how to setup a Samba system and would like to pick the brains of some experts. We are looking up put a large amount of storage ~75TB in a central data center. We have some remote (ok, not remote, but across slower links, ok if you consider several hundred clients over 1Gb to be slow) locations that we would like to set up samba servers that 'cache' the file system and serve it up to the clients in the building and sync with the main data center storage. The idea is have a couple of TB that are located in the building that serve up the Samba share. When a client requests a file, if it's in the local cache it is served up from there, if not then the Samba server grabs the file from the main data center and serves it to the client. When a file is written, something like rsync is used to transfer only difference back to the main data center. The problem is that I'm not sure of a file system that does this. We are using Lustre on our HPC, but this won't do what we want. Any suggestions are welcome. Robert LeBlanc Life Sciences Undergraduate Education Computer Support Brigham Young University I'm curious to know what you came up with for this. Care you share? TIA. We haven't come up with anything yet. We are still thinking this over. It's not pressing yet as we don't have the storage yet. Robert LeBlanc Life Sciences Undergraduate Education Computer Support Brigham Young University -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
[Samba] Ideas for distributed Samba servers
I'm trying to think about how to setup a Samba system and would like to pick the brains of some experts. We are looking up put a large amount of storage ~75TB in a central data center. We have some remote (ok, not remote, but across slower links, ok if you consider several hundred clients over 1Gb to be slow) locations that we would like to set up samba servers that 'cache' the file system and serve it up to the clients in the building and sync with the main data center storage. The idea is have a couple of TB that are located in the building that serve up the Samba share. When a client requests a file, if it's in the local cache it is served up from there, if not then the Samba server grabs the file from the main data center and serves it to the client. When a file is written, something like rsync is used to transfer only difference back to the main data center. The problem is that I'm not sure of a file system that does this. We are using Lustre on our HPC, but this won't do what we want. Any suggestions are welcome. Robert LeBlanc Life Sciences Undergraduate Education Computer Support Brigham Young University -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
