I'm not sure I'm following, but are you suggesting this be part of a
distribution? Generally the selection of servers is orthogonal to the
distribution, thus that really should be its own thing.
Walt
slang wrote:
This was all sounding awefully familiar to me...see:
http://www.beowulf-underground.org/pipermail/pvfs2-developers/2007-March/003269.html
So I guess this is turning into a monthly discussion. :-) It sounds
like people prefer that scheme over the indices array idea I proposed at
the beginning of this thread. I agree that wasn't terribly well
thought-out.
The issue I have with the current implementation (and I think the
proposed solution in that thread linked above) is that the initial
server is still chosen randomly in the
PINT_cached_config_get_next_[io|meta] call. Maybe there's a good reason
for this? It seems like that code should be part of the distribution.
Pete had a good use-case for this: debugging IO or metadata can be hard
to pin-down if the first server is always chosen randomly. We discussed
this yesterday a bit, and came up with yet another param (optionally
exists over all distros?), that would specify an ordering to how the
servers are chosen:
none: 1...N
random-start: choose the first randomly and the rest in order
random: chose all randomly
One could imagine interesting algorithms here that do things like
randomize within subsets of the range.
Anyway, the distribution callbacks would need to be augmented to support
this. Somehow the distribution would have to specify the indices, or
order the server aliases differently. So here's a proposed solution:
The distribution struct has a new get_ordering callout that takes a list
of server aliases, and returns a separate list based on the algorithm
specified (default would be random-start), or if the server map were
specified directly as part of the distribution, just return that. It
seems like the num_dfiles param could also be used to figure out the
result server list.
int (*get_ordering) (int incount, char *inaliases, int *outcount, char
*outaliases);
Seem reasonable? Does this fit in well with everyone's requirements?
-sam
On Apr 27, 2007, at 1:03 PM, Rob Ross wrote:
I like the idea of specifying aliases rather than handle ranges, etc.,
if that is feasible. I also agree that we want this interface to allow
this specification at create time, but then we want to use our
existing scheme for long-term metadata storage (e.g. just keep the
handles).
Regards,
Rob
Julian Martin Kunkel wrote:
Hi,
In general like the proposed solution better than the current, but an
extension which allows the distribution to see the aliases of the
servers would be neat as well. This certainly would solve the issue
of placing datafiles.
Julian, how would this stuff fit in with your migration/hints stuff?
I use the actual server aliases as targets to ensure proper placing
of the datafiles in an environment with miscellaneous sets of
dataservers.
In my opinion a decoupled system for file creation and the
distribution parameters seems important to allow later relocation of
datafiles to another data servers without conflicting with the
distribution parameters. Therefore, the location of the datafiles
should be transparent for the distribution and never be set in the
distribution parameters itself. Another reason not to store this info
in the distribution parameters is that once the target dataservers
are determined this information simply becomes obsolete. Julian
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
begin:vcard
fn:Walt Ligon
n:Ligon;Walt
org:Clemson University;ECE Department
adr;dom:;;;Clemson;SC;29634
email;internet:[EMAIL PROTECTED]
title:Associate Professor
tel;work:864-656-1224
x-mozilla-html:FALSE
version:2.1
end:vcard
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
