Hi Tony,
Thanks for the bug report. I have not looked at the varstrip issue yet,
but the attached patch (also available in cvs head) should at least fix
the basic_dist problem.
The problem was that basic_dist only works for files with 1 datafile,
but the logic that lets it override the number of datafiles was
happening before the directory hints were processed.
Someone correct me if I'm wrong, but I reorganized the code a little
chose the number of datafiles in this priority (after making sure the
final distribution for the file is set first, via hints or otherwise):
1) count specified in attr's passed into sys_create
- example: -n option to pvfs2-cp
2) directory hints
- example: "user.pvfs2.num_dfiles" extended attribute
3) mount options
- example: num_dfiles option in /etc/pvfs2tab
4) system default
- example: DefaultNumDfiles in server conf, or number of servers
The distribution itself gets a chance to override the value at the end
if (as in the case of basic_dist) it doesn't support an arbitrary number
of datafiles.
I'll look into the varstrip issue soon.
-Phil
Tony Kew wrote:
Dear PVFS users list,
Mounting a pvfs 7.2.1, creating directories and setting the distribution
type with setfattr for each directory, only simple_stipe and twod_stripe
would allow me to write a file in the given directory.
Adding the varstrip_dist xattrs patch from Murali Vilayannur attached to
this message thread:
http://osdir.com/ml/file-systems.pvfs2.users/2007-01/msg00050.html
fixes the variable strip distibution, but the basic distribution still
doesn't work for me.
The patch adds
#include "pvfs2-dist-basic.h
to src/client/sysint/sys-create.sm so I wonder if no one has tried
this...???
Perhaps there is a little more code needed to make basic_dist work with
xargs?
The patch above is against 2.6, but works (and is needed for varstrip_dist
with xargs) with 7.2.1 & the latest cvs (best I can tell) - perhaps
this patch
could be added to both?
Let me know if there is useful debug info I can provide...
Thanks Much
Tony
Tony Kew
SAN Administrator
The Center for Computational Research
New York State Center of Excellence
in Bioinformatics & Life Sciences
701 Ellicott Street, Buffalo, NY 14203
CoE Office: (716) 881-8930 Fax: (716) 849-6656
CSE Office: (716) 645-3797 x2174
Cell: (716) 560-0910 Home: (716) 874-2126
"I love deadlines, I love the whooshing noise they make as they go by."
Douglas Adams
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
? log.txt
? src/common/misc/.pint-cached-config.c.swp
? test/automated/vfs-tests.d/fsx-bin
Index: src/client/sysint/sys-create.sm
===================================================================
RCS file: /projects/cvsroot/pvfs2-1/src/client/sysint/sys-create.sm,v
retrieving revision 1.118
diff -a -u -p -r1.118 sys-create.sm
--- src/client/sysint/sys-create.sm 16 Jun 2008 21:36:13 -0000 1.118
+++ src/client/sysint/sys-create.sm 30 Jun 2008 16:09:29 -0000
@@ -186,7 +186,6 @@ PVFS_error PVFS_isys_create(
PVFS_error ret = -PVFS_EINVAL;
PINT_smcb *smcb = NULL;
PINT_client_sm *sm_p = NULL;
- int num_dfiles_req;
gossip_debug(GOSSIP_CLIENT_DEBUG, "PVFS_isys_create entered\n");
@@ -295,41 +294,6 @@ PVFS_error PVFS_isys_create(
}
}
- /* If an application hint has been provided, use that to request dfile
- else, if a tabfile hint has been provided, use that instead*/
- num_dfiles_req = 0;
- if (attr.mask & PVFS_ATTR_SYS_DFILE_COUNT)
- {
- num_dfiles_req = attr.dfile_count;
- }
- else
- {
- /* Check the mount options */
- int rc;
- struct PVFS_sys_mntent mntent;
-
- rc = PVFS_util_get_mntent_copy(sm_p->object_ref.fs_id, &mntent);
- if (0 == rc)
- {
- num_dfiles_req = mntent.default_num_dfiles;
- PVFS_util_free_mntent(&mntent);
- }
- }
-
- /* Determine the number of dfiles, passing in client hints to
- override any server hints */
- ret = PINT_cached_config_get_num_dfiles(sm_p->object_ref.fs_id,
- sm_p->u.create.dist,
- num_dfiles_req,
- &sm_p->u.create.num_data_files);
-
- if (ret < 0)
- {
- gossip_err("Failed to get number of data servers\n");
- PINT_smcb_free(smcb);
- return ret;
- }
-
gossip_debug(
GOSSIP_CLIENT_DEBUG, "Creating file %s under %llu, %d\n",
object_name, llu(parent_ref.handle), parent_ref.fs_id);
@@ -1047,9 +1011,9 @@ static PINT_sm_action create_parent_geta
{
struct PINT_client_sm *sm_p = PINT_sm_frame(smcb, PINT_FRAME_CURRENT);
PVFS_object_attr *attr = NULL;
- int num_dfiles_requested_override = 0;
PINT_dist *current_dist;
int ret = 0;
+ int num_dfiles_requested = 0;
gossip_debug(GOSSIP_CLIENT_DEBUG, "create state: parent_getattr_inspect\n");
@@ -1077,18 +1041,6 @@ static PINT_sm_action create_parent_geta
attr->u.dir.hint.dist_name_len,
attr->u.dir.hint.dist_params_len);
- num_dfiles_requested_override = attr->u.dir.hint.dfile_count;
- /* override the # of data files for this create */
- if (num_dfiles_requested_override > 0)
- {
- /* Determine the number of dfiles */
- PINT_cached_config_get_num_dfiles(sm_p->object_ref.fs_id,
- sm_p->u.create.dist,
- num_dfiles_requested_override,
- &sm_p->u.create.num_data_files);
- }
- gossip_debug(GOSSIP_CLIENT_DEBUG, "Setting number of datafiles to %d [requested %d]\n",
- sm_p->u.create.num_data_files, num_dfiles_requested_override);
current_dist = sm_p->u.create.dist;
/* We have an overriding distribution name for this directory.. honor that */
if (attr->u.dir.hint.dist_name_len > 0)
@@ -1119,6 +1071,55 @@ static PINT_sm_action create_parent_geta
current_dist->dist_name);
}
}
+
+ /* priority for determining user's preference for number of data files:
+ * 1) count specified in attr's passed into sys_create
+ * 2) directory hints
+ * 3) mount options
+ * 4) system default
+ * All of the above can be overridden by the distribution itself.
+ */
+
+ if(sm_p->u.create.sys_attr.mask & PVFS_ATTR_SYS_DFILE_COUNT)
+ {
+ /* specified by sys_create caller */
+ num_dfiles_requested = sm_p->u.create.sys_attr.dfile_count;
+ }
+ else if(attr->u.dir.hint.dfile_count > 0)
+ {
+ num_dfiles_requested = attr->u.dir.hint.dfile_count;
+ }
+ else
+ {
+ /* Check the mount options */
+ int rc;
+ struct PVFS_sys_mntent mntent;
+
+ rc = PVFS_util_get_mntent_copy(sm_p->object_ref.fs_id, &mntent);
+ if (0 == rc)
+ {
+ num_dfiles_requested = mntent.default_num_dfiles;
+ PVFS_util_free_mntent(&mntent);
+ }
+ }
+
+ /* Determine the number of dfiles. Pass in the number requested by the
+ * client, but will be overridden by default configuration and/or
+ * distribution if necessary
+ */
+ ret = PINT_cached_config_get_num_dfiles(sm_p->object_ref.fs_id,
+ sm_p->u.create.dist,
+ num_dfiles_requested,
+ &sm_p->u.create.num_data_files);
+ if(ret < 0)
+ {
+ gossip_err("Error: failed to get number of data servers\n");
+ js_p->error_code = ret;
+ return SM_ACTION_COMPLETE;
+ }
+
+ gossip_debug(GOSSIP_CLIENT_DEBUG, "Setting number of datafiles to %d [requested %d]\n",
+ sm_p->u.create.num_data_files, num_dfiles_requested);
/* okay, we might need to override some dist params as well */
if (attr->u.dir.hint.dist_params_len > 0)
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
