create-round-robin.patch:
--------------------------
We found after doing some testing that we weren't getting a very even
distribution of directories across servers when running benchmarks with
large numbers of files and directories. The existing logic is supposed
to choose a random server to begin with each time that it needs to pick
somewhere to place a metafile or datafile, but we have seen some meta
servers end up with as much as 40% more directory workload than others.
This patch changes the logic in pint-cached-config.c so that a random
server is chosen only on the first time that the selection function is
called. From that point on, all servers are selected in round-robin
fashion when creating new objects. This improved balance significantly
for us.
io-release.patch:
--------------------------
This doesn't fix an observed bug, but it cleans up a little bit of
cruft. The io.sm state machine was making an unnecessary check to see
if its scheduler id existed before calling the scheduler release
function. This patch also updates the job req sched release function to
error out if it loses track of an id rather than failing silently. This
might be useful in catching state machine programming errors.
makefile-mpi-objects.patch:
---------------------------
Small cleanup to the makefile for the test subdirectory. It previously
linked all MPI_MISC objects into every MPI test program, whether needed
or not. This patch adjust things so that the misc. objects are only
linked into relevant executables.
pvfs2-stat-objtype.patch:
---------------------------
This fixes a bug in how pvfs2-stat checks the type of the object before
printing out information about it.
sys-getattr-mask.patch:
---------------------------
This fixes a few problems in the handling of the datafile_count and
dirent_count fields in sys_getattr. They were being left out of some of
the object attribute <--> system attribute conversion functions. The
dirent_count was also not being copied out to the caller upon completion.
-Phil
---------------------
PatchSet 456
Date: 2006/02/27 20:26:35
Author: pcarns
Branch: HEAD
Tag: (none)
Log:
Adjusted policy for picking next server to create meta or io handles on: only
randomize the first time that the function is called. Simply round robin
afterwards.
[artf32506]
Members:
src/common/misc/pint-cached-config.c:1.1->1.2
Index: src/common/misc/pint-cached-config.c
diff -u src/common/misc/pint-cached-config.c:1.1 src/common/misc/pint-cached-config.c:1.2
--- src/common/misc/pint-cached-config.c:1.1 Mon Feb 7 12:57:42 2005
+++ src/common/misc/pint-cached-config.c Mon Feb 27 13:26:35 2006
@@ -34,6 +33,8 @@
static int cache_server_array(
struct server_configuration_s *config, PVFS_fs_id fsid);
+static int meta_randomized = 0;
+static int io_randomized = 0;
/* PINT_cached_config_initialize()
*
@@ -268,7 +269,16 @@
num_meta_servers = PINT_llist_count(
cur_config_cache->fs->meta_handle_ranges);
- jitter = (rand() % num_meta_servers);
+ /* pick random starting point, then round robin */
+ if(!meta_randomized)
+ {
+ jitter = (rand() % num_meta_servers);
+ meta_randomized = 1;
+ }
+ else
+ {
+ jitter = 0;
+ }
while(jitter-- > -1)
{
cur_mapping = PINT_llist_head(
@@ -343,8 +353,16 @@
num_io_servers = PINT_llist_count(
cur_config_cache->fs->data_handle_ranges);
- /* pick random starting point */
- jitter = (rand() % num_io_servers);
+ /* pick random starting point, then round robin */
+ if(!io_randomized)
+ {
+ jitter = (rand() % num_io_servers);
+ io_randomized = 1;
+ }
+ else
+ {
+ jitter = 0;
+ }
while(jitter-- > -1)
{
cur_mapping = PINT_llist_head(
---------------------
PatchSet 383
Date: 2006/01/18 18:43:28
Author: pcarns
Branch: HEAD
Tag: (none)
Log:
removed unecessary check in io_release() function and made sure that
job_req_sched_release() reports an error if the job descriptor cannot be
found.
[artf26076]
Members:
src/io/job/job.c:1.9->1.10
src/server/io.sm:1.6->1.7
Index: src/io/job/job.c
diff -u src/io/job/job.c:1.9 src/io/job/job.c:1.10
--- src/io/job/job.c:1.9 Wed Jan 4 08:51:01 2006
+++ src/io/job/job.c Wed Jan 18 11:43:28 2006
@@ -1116,7 +1116,12 @@
if (!match_jd)
{
/* id has been released or was not registered */
- return 0;
+ gossip_err("Error: job_req_sched_release() failed to locate descriptor.\n");
+ out_status_p->error_code = -PVFS_EINVAL;
+ out_status_p->status_user_tag = status_user_tag;
+ dealloc_job_desc(jd);
+ jd = NULL;
+ return 1;
}
ret = PINT_req_sched_release(match_jd->u.req_sched.id, jd,
Index: src/server/io.sm
diff -u src/server/io.sm:1.6 src/server/io.sm:1.7
--- src/server/io.sm:1.6 Wed Jan 4 08:51:02 2006
+++ src/server/io.sm Wed Jan 18 11:43:28 2006
@@ -273,11 +273,8 @@
tell the scheduler that we are done with this operation (if it
was scheduled in the first place)
*/
- if (s_op->scheduled_id)
- {
- ret = job_req_sched_release(
- s_op->scheduled_id, s_op, 0, js_p, &i, server_job_context);
- }
+ ret = job_req_sched_release(
+ s_op->scheduled_id, s_op, 0, js_p, &i, server_job_context);
return ret;
}
diff -Naur pvfs2A/test/Makefile.in pvfs2B/test/Makefile.in
--- pvfs2A/test/Makefile.in 2005-12-05 22:49:16.000000000 +0100
+++ pvfs2B/test/Makefile.in 2005-12-13 15:26:36.000000000 +0100
@@ -281,12 +281,11 @@
endif
# rule for building MPI executables from object files
-$(MPITESTS): %: %.o $(MPIMISCOBJS) $(LIBRARIES)
+$(MPITESTS): %: %.o $(LIBRARIES)
$(Q) " MPILD $@"
$(E)$(MPICC) $^ $(LDFLAGS) $(LIBS) -lm -o $@
# rule for building MPI-IO executables from object files
-# note: add a $(MPIMISCOBJS) prereq if we build multi-object tests someday
$(MPIIOTESTS): %: %.o $(LIBRARIES)
$(Q) " MPILD $@"
$(E)$(MPICC) $^ $(LDFLAGS) $(LIBS) -lm -o $@
diff -Naur pvfs2A/test/correctness/pts/module.mk.in pvfs2B/test/correctness/pts/module.mk.in
--- pvfs2A/test/correctness/pts/module.mk.in 2005-03-02 20:29:39.000000000 +0100
+++ pvfs2B/test/correctness/pts/module.mk.in 2005-12-13 15:26:36.000000000 +0100
@@ -37,8 +37,12 @@
$(DIR)/test-romio-noncontig-pattern2.c
MPIMISCSRC += $(TEST_PTS_MISC)
-
MPITESTSRC += $(TEST_PTS_SRC)
+TEST_PTS_MISCOBJS := $(patsubst %.c,%.o, $(TEST_PTS_MISC))
+TEST_PTS_TESTS := $(patsubst %.c,%, $(TEST_PTS_SRC))
+
+# note: the MPITESTS rule automatically links dependencies
+$(TEST_PTS_TESTS): %: $(TEST_PTS_MISCOBJS)
# need client interface for all tests
MODCFLAGS_$(DIR) = -I$(srcdir)/client/sysint
diff -Naur pvfs2A/test/io/bmi/module.mk.in pvfs2B/test/io/bmi/module.mk.in
--- pvfs2A/test/io/bmi/module.mk.in 2005-03-02 20:29:53.000000000 +0100
+++ pvfs2B/test/io/bmi/module.mk.in 2005-12-13 15:26:36.000000000 +0100
@@ -26,8 +26,12 @@
$(DIR)/driver-fs-read.c
MPIMISCSRC += $(LOCAL_MPIMISCSRC)
-
MPITESTSRC += $(LOCAL_MPITESTSRC)
+LOCAL_MPIMISCOBJS := $(patsubst %.c,%.o, $(LOCAL_MPIMISCSRC))
+LOCAL_MPITESTS := $(patsubst %.c,%, $(LOCAL_MPITESTSRC))
+
+# note: the MPITESTS rule automatically links dependencies
+$(LOCAL_MPITESTS): %: $(LOCAL_MPIMISCOBJS)
#$LOCALMPITESTS := $(patsubst %.c,%, $(LOCAL_MPITESTSRC))
#LOCALMPIOBJS := $(patsubst %.c,%.o, $(LOCAL_MPIMISCSRC))
---------------------
PatchSet 466
Date: 2006/02/28 20:21:45
Author: mjamis
Branch: HEAD
Tag: (none)
Log:
Fixed bugs with object type checking for dfile and dirent count display
Members:
src/apps/admin/pvfs2-stat.c:1.3->1.4
Index: src/apps/admin/pvfs2-stat.c
diff -u src/apps/admin/pvfs2-stat.c:1.3 src/apps/admin/pvfs2-stat.c:1.4
--- src/apps/admin/pvfs2-stat.c:1.3 Tue Dec 6 13:46:30 2005
+++ src/apps/admin/pvfs2-stat.c Tue Feb 28 13:21:45 2006
@@ -465,13 +465,13 @@
* datafiles there are
*/
if( (attr->mask & PVFS_ATTR_SYS_DFILE_COUNT) &&
- (attr->mask & PVFS_TYPE_METAFILE))
+ (attr->objtype == PVFS_TYPE_METAFILE))
{
fprintf(stdout, " datafiles : %d\n", attr->dfile_count);
}
/* dirent_count is only valid on directories */
if( (attr->mask & PVFS_ATTR_SYS_DIRENT_COUNT) &&
- (attr->mask & PVFS_TYPE_DIRECTORY))
+ (attr->objtype == PVFS_TYPE_DIRECTORY))
{
fprintf(stdout, " dir entries : %llu\n", llu(attr->dirent_count));
}
---------------------
PatchSet 459
Date: 2006/02/27 23:33:04
Author: pcarns
Branch: HEAD
Tag: (none)
Log:
fixed bugs in conversion of masks (and dirent_count value) in sys-getattr
path so that dfile_count and dirent_count are passed out properly to the
caller.
[artf32531]
Members:
src/client/sysint/sys-getattr.sm:1.12->1.13
src/common/misc/pvfs2-util.c:1.14->1.15
Index: src/client/sysint/sys-getattr.sm
diff -u src/client/sysint/sys-getattr.sm:1.12 src/client/sysint/sys-getattr.sm:1.13
--- src/client/sysint/sys-getattr.sm:1.12 Mon Jan 16 11:06:44 2006
+++ src/client/sysint/sys-getattr.sm Mon Feb 27 16:33:04 2006
@@ -874,6 +874,12 @@
{
sysresp->attr.mask |= PVFS_ATTR_SYS_LNK_TARGET;
}
+
+ if(sm_p->getattr.req_attrmask & PVFS_ATTR_DIR_DIRENT_COUNT)
+ {
+ sysresp->attr.dirent_count = attr->u.dir.dirent_count;
+ sysresp->attr.mask |= PVFS_ATTR_SYS_DIRENT_COUNT;
+ }
}
else
{
Index: src/common/misc/pvfs2-util.c
diff -u src/common/misc/pvfs2-util.c:1.14 src/common/misc/pvfs2-util.c:1.15
--- src/common/misc/pvfs2-util.c:1.14 Mon Feb 13 11:36:04 2006
+++ src/common/misc/pvfs2-util.c Mon Feb 27 16:33:05 2006
@@ -1479,6 +1479,11 @@
attrmask |= PVFS_ATTR_META_DFILES;
}
+ if (sys_attrmask & PVFS_ATTR_SYS_DIRENT_COUNT)
+ {
+ attrmask |= PVFS_ATTR_DIR_DIRENT_COUNT;
+ }
+
if (sys_attrmask & PVFS_ATTR_SYS_LNK_TARGET)
{
attrmask |= PVFS_ATTR_SYMLNK_TARGET;
@@ -1532,6 +1537,14 @@
{
sys_mask |= PVFS_ATTR_SYS_LNK_TARGET;
}
+ if (obj_mask & PVFS_ATTR_DIR_DIRENT_COUNT)
+ {
+ sys_mask |= PVFS_ATTR_SYS_DIRENT_COUNT;
+ }
+ if (obj_mask & PVFS_ATTR_META_DFILES)
+ {
+ sys_mask |= PVFS_ATTR_SYS_DFILE_COUNT;
+ }
return sys_mask;
}
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
