Kyle,
Lets step back a second. What's the actual bug that motivated this?
My impression from your email was that a default value of num_groups=2
didn't work in a single server environment, only because when it tried
to divide num_servers (1) by num_groups (2) it got 0.
That's completely separate from _setting_ the num_groups parameter and
providing the checks to make sure it doesn't get set to something
invalid. Instead I'm arguing for checks inside the distribution code
itself, where we do have knowledge of the number of servers (the
filesystem layout) and have to choose which server to send data. We
need checks there as well, and a default behavior when we get invalid
values. Does that make sense?
-sam
On Jul 1, 2008, at 8:26 PM, Kyle Schochenmaier wrote:
The place where this gets checked does not have knowledge of the
layout of the filesystem,j it just sets parameters for future files, I
believe. The checks are in place to make sure we dont supply invalid
values - values <= 0 inside set_param(). The problem I see with this
on first glance at the code is that set_param() gets called 3 times
whenever you set a fattr() and it parses the string you give it, once
for each type of dist_param. A particular call into set_param that
would set the num_groups would be out of scope of the call that sets
the strip_factor so I dont see an easy way inside dist-twod-stripe.c
to accomplish this?
I'd be willing to look into it further if anyone has any fancy tricks
to accomplish this.
Kyle
On Tue, Jul 1, 2008 at 5:16 PM, Sam Lang <[EMAIL PROTECTED]> wrote:
On Jul 1, 2008, at 4:15 PM, Kyle Schochenmaier wrote:
We should use simple stripe in that case, which is why i have it
defaulted to it now, someone was playing with twod-dist today and
noticed that it defaults to having 2 groups, which causes errors
when
you run it on only a single server.
That seems like kind of a bug. Can we just add the appropriate
check that
fixes the error?
-sam
I would much rather have the
defaults allow the distribution to work on *any* setup, rather than
just a n>1 servers setup.
Its not particularly useful, but I belive it is the right thing to
do,
maybe I'm wrong?
(the original default settings are worthless for my applications
so I
have to change them anyhow)
Kyle
On Tue, Jul 1, 2008 at 4:04 PM, Sam Lang <[EMAIL PROTECTED]> wrote:
Kyle,
How is this patch useful for you guys? With a single server (and
only
one
stripe), it doesn't matter what the factor and num_groups are,
does it?
It
still behaves the same as simple-stripe in that case. With
multiple
servers
setting the factor and num_groups to 1 does make the 2d
distribution act
like simple-stripe, but why not just use simple stripe in that
case?
-sam
On Jul 1, 2008, at 3:19 PM, Kyle Schochenmaier wrote:
Patch is against cvs head, and modifies the default num_groups
to be 1
and the group_strip_factor = 1 so twod-stripe's defaults still
work
when you use it on a single-server, it will now by default
behave the
same as simple-stripe until the user modifies the num_groups and
strip_factor xattrs.
Also added a brief description of the definitions for the
dist_params
in twod-stripe.
Tested fully functional on my servers.
Kyle
Index: src/io/description/dist-twod-stripe.c
=
==================================================================
RCS file: /anoncvs/pvfs2/src/io/description/dist-twod-stripe.c,v
retrieving revision 1.5
diff -r1.5 dist-twod-stripe.c
6a7,15
/* twod-stripe will take all of the servers in the filesystem and
* partition them into num_groups groups. Data will then be
striped to
* each group before we move onto the next group. The
strip_factor will
* determine how many chunks of strip_size are written to each
server
* in each group before we transition to the next group.
* The striping on the group level is done round-robin in the same
* fashion as simple-stripe
*/
Index: include/pvfs2-dist-twod-stripe.h
=
==================================================================
RCS file: /anoncvs/pvfs2/include/pvfs2-dist-twod-stripe.h,v
retrieving revision 1.4
diff -r1.4 pvfs2-dist-twod-stripe.h
17c17
< #define PVFS_DIST_TWOD_STRIPE_DEFAULT_GROUPS 2
---
#define PVFS_DIST_TWOD_STRIPE_DEFAULT_GROUPS 1
19c19
< #define PVFS_DIST_TWOD_STRIPE_DEFAULT_FACTOR 256
---
#define PVFS_DIST_TWOD_STRIPE_DEFAULT_FACTOR 1
-
Kyle Schochenmaier
<twod-dist.patch>_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
--
Kyle Schochenmaier
--
Kyle Schochenmaier
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
