Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-11 Thread Joseba Torre
El 09/01/12 14:50, Phil Turmel escribió: I've been following this thread with great interest, but no advice to offer. The content is entirely appropriate, and appreciated. Don't be embarrassed by your enthusiasm, Stan. +1

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-11 Thread Sven Hartge
Sven Hartge s...@svenhartge.de wrote: I am currently in the planning stage for a new and improved mail system at my university. OK, executive summary of the design ideas so far: - deployment of X (starting with 4, but easily scalable) virtual servers on VMware ESX - storage will be backed

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Stan Hoeppner
On 1/8/2012 9:39 AM, Sven Hartge wrote: Memory size. I am a bit hesistant to deploy a VM with 16GB of RAM. My cluster nodes each have 48GB, so no problem on this side though. Shouldn't be a problem if you're going to spread the load over 2 to 4 cluster nodes. 16/2 = 8GB per VM, 16/4 = 4GB per

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Stan Hoeppner
On 1/8/2012 2:15 PM, Sven Hartge wrote: Wouldn't such a setup be the Best of Both Worlds? Having the main traffic going to local disks (being RDMs) and also being able to provide shared folders to every user who needs them without the need to move those users onto one server? The only

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Stan Hoeppner
On 1/8/2012 3:07 PM, Sven Hartge wrote: Ah, I forgot: I _already_ have the mechanisms in place to statically redirect/route accesses for users to different backends, since some of the users are already redirected to a different mailsystem at another location of my university. I assume you

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Sven Hartge
Stan Hoeppner s...@hardwarefreak.com wrote: On 1/8/2012 9:39 AM, Sven Hartge wrote: Memory size. I am a bit hesistant to deploy a VM with 16GB of RAM. My cluster nodes each have 48GB, so no problem on this side though. Shouldn't be a problem if you're going to spread the load over 2 to 4

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Phil Turmel
On 01/09/2012 08:38 AM, Stan Hoeppner wrote: On 1/8/2012 3:07 PM, Sven Hartge wrote: [...] (Are my words making any sense? I got the feeling I'm writing German with English words and nobody is really understanding anything ...) You're making perfect sense, and frankly, if not for the .de

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Sven Hartge
Stan Hoeppner s...@hardwarefreak.com wrote: On 1/8/2012 3:07 PM, Sven Hartge wrote: Ah, I forgot: I _already_ have the mechanisms in place to statically redirect/route accesses for users to different backends, since some of the users are already redirected to a different mailsystem at

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Sven Hartge
Stan Hoeppner s...@hardwarefreak.com wrote: On 1/8/2012 2:15 PM, Sven Hartge wrote: Wouldn't such a setup be the Best of Both Worlds? Having the main traffic going to local disks (being RDMs) and also being able to provide shared folders to every user who needs them without the need to move

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Timo Sirainen
Too much text in the rest of this thread so I haven't read it, but: On 8.1.2012, at 0.20, Sven Hartge wrote: Right now, I am pondering with using an additional server with just the shared folders on it and using NFS (or a cluster FS) to mount the shared folder filesystem to each backend

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Sven Hartge
Timo Sirainen t...@iki.fi wrote: On 8.1.2012, at 0.20, Sven Hartge wrote: Right now, I am pondering with using an additional server with just the shared folders on it and using NFS (or a cluster FS) to mount the shared folder filesystem to each backend storage server, so each user has

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Charles Marcus
On 2012-01-09 9:51 AM, Timo Sirainen t...@iki.fi wrote: The proper solution for this that I've been thinking about would be to use v2.1's imapc backend with master users. So that when user A wants to access user B's shared folder, Dovecot connects to B's IMAP server using master user login, and

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Stan Hoeppner
On 1/9/2012 8:08 AM, Sven Hartge wrote: Stan Hoeppner s...@hardwarefreak.com wrote: The quota for students is 1GiB here. If I provide each of my 4 nodes with 500GiB of storage space, this gives me 2TiB now, which should be sufficient. If a nodes fills, I increase its storage space. Only if it

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Sven Hartge
Stan Hoeppner s...@hardwarefreak.com wrote: The more I think about your planned architecture the more it reminds me of a shared nothing database cluster--even a relatively small one can outrun a well tuned mainframe, especially doing decision support/data mining workloads (TPC-H). As long

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Timo Sirainen
On 9.1.2012, at 17.14, Charles Marcus wrote: On 2012-01-09 9:51 AM, Timo Sirainen t...@iki.fi wrote: The proper solution for this that I've been thinking about would be to use v2.1's imapc backend with master users. So that when user A wants to access user B's shared folder, Dovecot connects

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Sven Hartge
Timo Sirainen t...@iki.fi wrote: On 8.1.2012, at 0.20, Sven Hartge wrote: Right now, I am pondering with using an additional server with just the shared folders on it and using NFS (or a cluster FS) to mount the shared folder filesystem to each backend storage server, so each user has

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Timo Sirainen
On 9.1.2012, at 20.25, Sven Hartge wrote: Timo Sirainen t...@iki.fi wrote: On 8.1.2012, at 0.20, Sven Hartge wrote: Right now, I am pondering with using an additional server with just the shared folders on it and using NFS (or a cluster FS) to mount the shared folder filesystem to each

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Sven Hartge
Timo Sirainen t...@iki.fi wrote: On 9.1.2012, at 20.25, Sven Hartge wrote: Timo Sirainen t...@iki.fi wrote: On 8.1.2012, at 0.20, Sven Hartge wrote: Right now, I am pondering with using an additional server with just the shared folders on it and using NFS (or a cluster FS) to mount the

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Timo Sirainen
On 9.1.2012, at 20.47, Sven Hartge wrote: Can mmap_disable = yes and the other NFS options be set per namespace or only globally? Currently only globally. Ah, too bad. Back to the drawing board then. mmap_disable=yes works pretty well even if you're only using it for local

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Timo Sirainen
On 9.1.2012, at 21.16, Timo Sirainen wrote: passdb { type = static args = user=shareduser Of course you should also require a password: args = user=shareduser pass=master-user-password

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Sven Hartge
Timo Sirainen t...@iki.fi wrote: On 9.1.2012, at 20.47, Sven Hartge wrote: Can mmap_disable = yes and the other NFS options be set per namespace or only globally? Currently only globally. Ah, too bad. Back to the drawing board then. mmap_disable=yes works pretty well even if you're

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Timo Sirainen
On 9.1.2012, at 21.31, Sven Hartge wrote: , | # User's private mail location | mail_location = mdbox:~/mdbox | | # When creating any namespaces, you must also have a private namespace: | namespace { | type = private | separator = . | prefix = INBOX. | #location defaults to

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Sven Hartge
Timo Sirainen t...@iki.fi wrote: On 9.1.2012, at 21.31, Sven Hartge wrote: , | # User's private mail location | mail_location = mdbox:~/mdbox | | # When creating any namespaces, you must also have a private namespace: | namespace { | type = private | separator = . | prefix =

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Timo Sirainen
On 9.1.2012, at 21.45, Sven Hartge wrote: | location = imapc:~/imapc-shared What is the syntax of this location? What does imapc-shared do in this case? It's the directory for index files. The backend IMAP server is used as a rather dummy storage, so if for example you do a FETCH 1:*

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Sven Hartge
Timo Sirainen t...@iki.fi wrote: On 9.1.2012, at 21.45, Sven Hartge wrote: | location = imapc:~/imapc-shared What is the syntax of this location? What does imapc-shared do in this case? It's the directory for index files. The backend IMAP server is used as a rather dummy storage, so if

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Timo Sirainen
On 9.1.2012, at 22.13, Sven Hartge wrote: Timo Sirainen t...@iki.fi wrote: On 9.1.2012, at 21.45, Sven Hartge wrote: | location = imapc:~/imapc-shared What is the syntax of this location? What does imapc-shared do in this case? It's the directory for index files. The backend IMAP

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Sven Hartge
Timo Sirainen t...@iki.fi wrote: On 9.1.2012, at 22.13, Sven Hartge wrote: Timo Sirainen t...@iki.fi wrote: On 9.1.2012, at 21.45, Sven Hartge wrote: | location = imapc:~/imapc-shared What is the syntax of this location? What does imapc-shared do in this case? It's the directory for

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-09 Thread Stan Hoeppner
On 1/9/2012 7:48 AM, Sven Hartge wrote: It seems my initial idea was not so bad after all ;) Yeah, but you didn't know how not so bad it really was until you had me analyze it, flesh it out, and confirm it. ;) Now I just need o built a little test setup, put some dummy users on it and see,

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-08 Thread Stan Hoeppner
On 1/7/2012 7:55 PM, Sven Hartge wrote: Stan Hoeppner s...@hardwarefreak.com wrote: It's highly likely your problems can be solved without the drastic architecture change, and new problems it will introduce, that you describe below. The main reason is I need to replace the hardware as its

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-08 Thread Sven Hartge
Stan Hoeppner s...@hardwarefreak.com wrote: On 1/7/2012 7:55 PM, Sven Hartge wrote: Stan Hoeppner s...@hardwarefreak.com wrote: It's highly likely your problems can be solved without the drastic architecture change, and new problems it will introduce, that you describe below. The main

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-08 Thread Sven Hartge
Sven Hartge s...@svenhartge.de wrote: Stan Hoeppner s...@hardwarefreak.com wrote: If an individual VMware node don't have sufficient RAM you could build a VM based Dovecot cluster, run these two VMs on separate nodes, and thin out the other VMs allowed to run on these nodes. Since you can't

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-08 Thread Sven Hartge
Sven Hartge s...@svenhartge.de wrote: Sven Hartge s...@svenhartge.de wrote: Stan Hoeppner s...@hardwarefreak.com wrote: If an individual VMware node don't have sufficient RAM you could build a VM based Dovecot cluster, run these two VMs on separate nodes, and thin out the other VMs allowed

[Dovecot] Providing shared folders with multiple backend servers

2012-01-07 Thread Sven Hartge
Hi *, I am currently in the planning stage for a new and improved mail system at my university. Right now, everything is on one big backend server but this is causing me increasing amounts of pain, beginning with the time a full backup takes. So naturally, I want to split this big server into

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-07 Thread Stan Hoeppner
On 1/7/2012 4:20 PM, Sven Hartge wrote: Hi *, I am currently in the planning stage for a new and improved mail system at my university. Right now, everything is on one big backend server but this is causing me increasing amounts of pain, beginning with the time a full backup takes. You

Re: [Dovecot] Providing shared folders with multiple backend servers

2012-01-07 Thread Sven Hartge
Stan Hoeppner s...@hardwarefreak.com wrote: It's highly likely your problems can be solved without the drastic architecture change, and new problems it will introduce, that you describe below. The main reason is I need to replace the hardware as its service contract ends this year and I am