On Oct 6, 2006, at 2:01 AM, Julian Martin Kunkel wrote:
Hi,
This way callers wouldn't be able to muck with the internals of the
hint struct.
Ok, I will definitly do this.
As I said, I prefer letting the hint struct be defined
externally and requiring an array of them to the system interfaces.
It seems to match what we have throughout the rest of PVFS.
If the pvfs2-team thinks that is the way to go I will do so, even
if I prefer
the list for the system interface. So I think the array wins ?
Setting specific IO servers for a file is interesting for research
purposes, but kind of breaks the encapsulation and abstraction that
pvfs tries to provide. We've already got a distribution parameter
that we pass in to PVFS_sys_create when appropriate.
I don't know that we want to go around telling users to set an
environment variable (a separate interface, remember) for specific
behavior that they've requested. I'd rather figure out why the dist
parameter isn't providing the bits of functionality that they need,
and solve their problem from that angle.
I vaguely remember having
this discussion about specifying datafile handles through the dist
parameter, and there being some concerns, but I can't remember the
details. Can you remind me why that doesn't work?
The dist parameter is opaque for the client interface and should
only be
interpreted from the distribution. However, on the layer of the
distribution
the hostnames are not available, also it would be available only to
distributions which interprete the given hostnames.
The specific dist implementation is certainly opaque from the client
interface, but we do provide functions to set parameters on a
particular distribution. What I would envision is something along
these lines:
struct PINT_dist_server_settable_indices {
int count;
int * servers;
};
...
PVFS_sys_dist mydist;
struct PINT_dist_server_settable_indices indices =
{
4,
{2, 4, 6, 8}
}
...
mydist.name = "server-settable-dist";
PVFS_sys_dist_setparam(&mydist, "servers", &indices);
....
PVFS_sys_create(..., &mydist, ...);
The server-settable-dist would be implemented to store the indices
for the IO servers in the params field of the PVFS_sys_dist
structure. I used server indices, because the PINT_dist_* interfaces
allow for that (through the PINT_request_file_data struct), but its a
bit ugly and probably confusing to the user. We could change that
though, and use something like server aliases or hostnames, passing
them through to the distribution parameters instead of indices. This
would require a change to the distribution method calls, and the
Also the create sm has to
be modified a lot to use the new distribution facilities.
It certainly would. The client create state machine is a little
broken in this respect anyway though. It normally just gets the list
of IO servers from the cached server config file and does a create
request to all of them. There's special case handling right now for
the directory hints on the parent to see if the number of datafiles
handles should be fewer than available.
It seems like that could be generalized to get a list of server
indices from the distribution with a distribution method that returns
that info. We already store a distribution in the directory hints
anyway, so we could probably just throw away that dfile_count
parameter. In any case, I would be in favor of a change to the
create sm that allows the distribution to optionally specify the
server indices, and fallback to the cached config lookup otherwise.
Of course by adding
another array to the distribution struct which is common to all
distributions
would solve the issue but I would not prefer this solution.
No that's gross.
With the hint it is a common infrastructure given for all available
distributions and it also is a uncommon case to set the distribution..
Right, I don't like the idea of 'overriding' a distribution's
behavior with a hint. Conceptually it seems like the distribution
should take care of this, and if it doesn't, we should fix it so it
does.
Also, the server list hint gets a little ugly across sysint calls,
since the list doesn't actually get stored anywhere for that file
(does it?). With the distribution, we only have to specify it once
at creation time, instead of passing it along to all future IO calls
on that file.
-sam
Wasn't
the hint be intented to be useful for research purposes ?
Thanks for the discussion and the time you spend,
Julian
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
