On Oct 30, 2006 10:20 -0800, Lin Shen (lshen) wrote: > Thanks for getting back to me. The CFS would be on a series of > platforms, so the RAM number varies. The low end could be as little as > 512MG and that's for the whole system including routing, VoIP etc.
The lustre clients (IO nodes) on the BG/L system (#1 supercomputer) "only" have 1MB of RAM and they do IO for up to 128 compute clients at a time. Lustre as a whole (MDS + 5 OSTs + 2 client mounts) can run inside a 64MB UML. The lustre code already has internal tunables that adjust the default buffer size, thread count, etc based on the amount of RAM on the node. The only issue is that by restricting the amount of memory that lustre uses it might negatively impact the performance because the client cannot keep IO in flight to saturate the network. > What's the footprint of Lustre and how much lower could it be reduced to > with the code optimization you mentioned? Maybe we can cut it even more > w/o medata server cluster etc. The clustered metadata isn't in the released Lustre code, and even if it was available there wouldn't be a requirement to use it. With debugging symbols the lustre modules are a whopping 3-8MB a piece, and you need about 10 of them loaded to have a functioning client. If you strip out the debugging symbols you lose about 70% of that. If you _really_ wanted to slim down the modules you could make the CDEBUG() macro a no-op for anything except error messages, at the expense of making debugging much harder. That could be mitigated (and I'd be thrilled to see it) by having systemtap scripts to replace the CDEBUG trace/debug functionality. In terms of Lustre buffers and such, on a small clients this can be limited to a few MB in total. On larger clients it grows into the 10s of MB, and of course whatever the VM gives the filesystem for data cache. Lustre has its own tunable limits on how much dirty data a client can have, as older kernels didn't do a good job with that. > And I don't think the number of nodes will ever exceed a dozen. The number of clients doesn't impact the amount of memory on other clients. Lustre is strictly a client-server filesystem in that regard. It does impact memory usage on the server, which would be more important if you also want the server to live inside this environment. > -----Original Message----- > From: Andreas Dilger [mailto:[EMAIL PROTECTED] > Sent: Saturday, October 28, 2006 7:38 PM > To: Lin Shen (lshen) > Cc: [email protected] > Subject: Re: [Lustre-discuss] Lustre Lite > > On Oct 27, 2006 11:37 -0700, Lin Shen (lshen) wrote: > > I wonder what's the current status of Lustre Lite, is it still > > supported? We're looking for a CFS for an embedded system without a > > lot of nodes. Lustre sounds a bit too heavyweight for that. > > How much RAM is in the embedded system? With tuning if various buffer > sizes and thread counts, the amount of RAM used on a client could be > reduced substantially (at some cost in performance), depending on just > how tight the RAM is. Also, it would likely be possible to shrink the > code in a number of ways by adding #ifdefs for different functionality > you might not need: > > - disable flock > - disable quotas > - disable O_DIRECT and associated IO path > - make no-op debugging macros > > We'd likely accept patches to make these configure options if it is done > cleanly. > > Cheers, Andreas > -- > Andreas Dilger > Principal Software Engineer > Cluster File Systems, Inc. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc. _______________________________________________ Lustre-discuss mailing list [email protected] https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
