Murali,
Thank you very much for your patch. It seems to work well. For example,
with 2 I/O servers, and a varstrip configuration "0:512;1:1024", when I
copy a 6 MB file, 2MB of it are stored in one server, and the remaining
4 MB in the other server. The command
setfattr -n "user.pvfs2.dist_params" -v "strips:0:512;1:1024"
also works nicely.
The pvfs2_viewdist command, however, does not work. I get the following
error:
dist_name = varstrip_dist
strips = 0:512;1:1024
Number of datafiles/servers = 2
Server 0 - (null), handle: 4294967293 (fffffffd.bstream)
Segmentation fault
Anyway, this error does not worry me. I will take a look at Julian's branch.
Thanks again,
Juan.
Murali Vilayannur wrote:
> Hi Juan,
> We had never tested varstrip placement with xattrs... Clearly you just
> hit upon a bug that I knew was waiting to happen all this while but
> did not get to fixing it... :(
> Can you try the attached patch against pvfs-2.6.* and see if it works?
> You still cannot control which specific I/O node a file will be
> created since the
> varstrip placement is with respect to some ordinal numbering of servers
> which can and is typically random :)
>
> Sample invocation of setfattr could be like
> % setfattr -n "user.pvfs2.dist_params" -v "strips:0:512;1:1024"
> /mnt/pvfs2/testdir/
>
> will take bytes 0-511 on node 0, 511-1535 on node 1 and so on...
>
> Note how fragile the parsing logic and requirement on how the distrib
> params are to be constructed. Very sorry about that.. We really should
> find a better way for expressing parameters I think...
>
> If you really need very specific server location placement, you have
> to checkout Julian's branch from HEAD (kunkel-hint-branch) and figure
> out how to invoke PVFS_sys_create() with the hint parameters. NOTE:
> You cannot use xattrs (either system or vfs) since noone has done that
> yet with Julian's server-placement hint branch yet...should not be too
> hard though.
>
> let me know if this patch still does not get things working again..
> thanks,
> Murali
>
> On 1/12/07, Juan Piernas Canovas <[EMAIL PROTECTED]> wrote:
>> Hi folks,
>>
>> I'm testing PVFS2, and one of the things I want to do is to control in
>> which I/O node a specific file will be stored. With the "simple stripe"
>> distribution, I can create a directory and specify for it 1 as the
>> number of datafiles to be used. The problem is that every file created
>> in that directory is stored in a different I/O node in a round-robin
>> fashion.
>>
>> With the varstrip distribution, you can, in theory at least, indicate
>> which I/O nodes, and which chunk size in every node, you want to use for
>> a file. But this distribution does not work for me. I have tried the
>> following:
>>
>> $ mkdir testdir
>> $ setfattr -n "user.pvfs2.dist_name" -v "varstrip_dist" testdir
>> $ setfattr -n "user.pvfs2.dist_params" -v "strips:1:512" testdir
>> $ cp somefile testdir
>> cp: writing `testdir/somefile': Unknown error 132
>>
>> I have also tried to run:
>>
>> $ setfattr -n "user.pvfs2.dist_params" -v "1:512" testdir
>>
>> but the cp command gets the same error.
>>
>> Any clue? Is there any way to specific where I can store a file in
>> PVFS2?
>>
>> Thanks in advance,
>>
>> Juan.
>> _______________________________________________
>> Pvfs2-users mailing list
>> [email protected]
>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>>
> ------------------------------------------------------------------------
>
> --- src/client/sysint/sys-create.sm.old 2007-01-12 19:32:00.776908000
-0800
> +++ src/client/sysint/sys-create.sm 2007-01-12 19:44:49.307825000 -0800
> @@ -15,7 +15,9 @@
>
> #include "client-state-machine.h"
> #include "pvfs2-debug.h"
> +#include "pvfs2-dist-basic.h"
> #include "pvfs2-dist-simple-stripe.h"
> +#include "pvfs2-dist-varstrip.h"
> #include "job.h"
> #include "gossip.h"
> #include "str-utils.h"
> @@ -1134,34 +1136,50 @@
> {
> gossip_debug(GOSSIP_CLIENT_DEBUG, "distribution parameter %s,
value %s\n",
> key[i], val[i]);
> - /* NOTE: just as in server-config.c when parsing "Param" and
> - * "Value" fields, we will assume that all values are 64 bit
> - * integers. The only difference here is that we scan
> - * directly into a 64 bit integer, rather than converting
> - * from the int format that dotconf supports.
> - */
> - ret = sscanf(val[i], SCANF_lld, &tmp_val);
> - if(ret != 1)
> - {
> - gossip_err("Error: unsupported type for distribution
parameter %s, value %s found in directory hints.\n",
> - key[i], val[i]);
> - gossip_err("Error: continuing anyway.\n");
> - }
> - else
> - {
> + if (strncmp(current_dist->dist_name,
> + PVFS_DIST_SIMPLE_STRIPE_NAME,
> + PVFS_DIST_SIMPLE_STRIPE_NAME_SIZE) == 0) {
> + /* NOTE: just as in server-config.c when parsing "Param"
and
> + * "Value" fields, we will assume that all values are 64
bit
> + * integers. The only difference here is that we scan
> + * directly into a 64 bit integer, rather than converting
> + * from the int format that dotconf supports.
> + */
> + ret = sscanf(val[i], SCANF_lld, &tmp_val);
> + if(ret != 1)
> + {
> + gossip_err("Error: unsupported type for distribution
parameter %s, value %s found in directory hints.\n",
> + key[i], val[i]);
> + gossip_err("Error: continuing anyway.\n");
> + }
> + else
> + {
> +
if(current_dist->methods->set_param(current_dist->dist_name,
> + current_dist->params,
> + key[i],
> + &tmp_val))
> + {
> +
> + gossip_err("Error: could not override hinted
distribution parameter %s, value %s found in directory hints\n",
> + key[i],
> + val[i]);
> + }
> + }
> + } else if (strncmp(current_dist->dist_name,
> + PVFS_DIST_VARSTRIP_NAME,
> + PVFS_DIST_VARSTRIP_NAME_SIZE) == 0) {
>
if(current_dist->methods->set_param(current_dist->dist_name,
> - current_dist->params,
> - key[i],
> - &tmp_val))
> + current_dist->params,
> + key[i],
> + val[i]))
> {
>
> gossip_err("Error: could not override hinted distribution
parameter %s, value %s found in directory hints\n",
> - key[i],
> - val[i]);
> + key[i], val[i]);
> }
> - }
> - free(key[i]);
> - free(val[i]);
> + }
> + free(key[i]);
> + free(val[i]);
> }
> free(key);
> free(val);
> --- src/common/misc/str-utils.c.old 2007-01-12 19:26:49.129773000 -0800
> +++ src/common/misc/str-utils.c 2007-01-12 19:27:49.448030000 -0800
> @@ -688,6 +688,7 @@
> * The given string must be comma separated, and each
> * segment within the comma regions must be of of
> * the form key:val.
> + * val itself can contain colons.
> * Return the number of such keyval pairs and a
> * pointer to a double dimensional array of keys and values.
> * In case of errors, a -ve PVFS error is returned.
> @@ -697,6 +698,7 @@
> * NULL - return -PVFS_EINVAL
> * ab:23 - return nkey as 1, pkey <"ab">, pval <"23">
> * ab:23,bc:34 - returns nkey as 2, pkey <"ab", "bc">, pval<"23", "34">
> + * ab:2:3 - returns nkey as 1, pkey <"ab">, pval <"2:3">
> *
> */
> int PINT_split_keyvals(char *string, int *nkey,
> @@ -780,8 +782,10 @@
> ptr2 = strdup(val[i]);
> if (ptr1 == NULL || ptr2 == NULL)
> break;
> +#if 0
> if (strchr(ptr1, ':') || strchr(ptr2, ':'))
> break;
> +#endif
> key[i] = ptr1;
> val[i] = ptr2;
> }
>
