Dear Kyle,
Thanks, that was the fix - which follows what is says in 6.6.2 of
the FAQ (so my bad!)
I got confused with the syntax for the varstrip_dist where the
node_number:strip_size pairs are separated by semicolons ';'
As I mentioned before it would be useful if the setfattr command
would fail (ideally with a useful error message) if the parameters
are incorrect, though I don't know how hard this would be to achieve?
Many thanks to you & Phil for helping me with this,
Tony
Tony Kew
SAN Administrator
The Center for Computational Research
New York State Center of Excellence
in Bioinformatics & Life Sciences
701 Ellicott Street, Buffalo, NY 14203
CoE Office: (716) 881-8930 Fax: (716) 849-6656
CSE Office: (716) 645-3797 x2174
Cell: (716) 560-0910 Home: (716) 874-2126
"I love deadlines, I love the whooshing noise they make as they go by."
Douglas Adams
Kyle Schochenmaier wrote:
It looks like you might have the setfattr syntax wrong?
Here is an example that works on my systems with the previous patching:
setfattr -n "user.pvfs2.dist_params" -v
"strip_size:4194304,num_groups:3,group_strip_factor:32" /scratch/2d
You need to specify num_groups >= num_servers, the groups are the 2D
part, so no you cant really do a 2D stripe on a single server, but it
should still work, the code is there to make it work exactly like
simple_stripe if there is only one group.
If there is an issue with the setfattr, the distribution will set the
attributes to default (2), I will roll out a patch this afternoon to
make the default num_groups = 1 so this dist will not break on a
single-server.
For now, try to set the num_groups = 1, and see if you are still
having a problem with setfattr.
Kyle
On Tue, Jul 1, 2008 at 12:09 PM, Tony Kew <[EMAIL PROTECTED]> wrote:
Dear Phil,
I applied the patch, but it doesn't work for me when testing with a single
node (if this is a valid thing to do with a 2D stripe...)
Could you check that I got the setfattr syntax correct, I'd hate for the
problem to be that! (...user error...)
Would it be possible for the fsetattr call to fail if the parameters are
invalid?
e.g. setting
$ setfattr -n "user.pvfs2.dist_params" -v "num_groups:3" _some_dir
would fail if you only have 2 or less server nodes, strip_size should always
be greater than zero etc...
$ mkdir twod_stripe/
$ setfattr -n "user.pvfs2.dist_name" -v "twod_stripe" twod_stripe
$ setfattr -n "user.pvfs2.dist_params" \
-v "strip_size:16384;num_groups:1;group_strip_factor:512" twod_stripe
$ cd twod_stripe/
$ dd if=/dev/urandom of=testfile bs=16k count=16
16+0 records in
16+0 records out
262144 bytes (262 kB) copied, 0.148667 seconds, 1.8 MB/s
$ ls -l testfile
-rw-r--r-- 1 tonykew ccrstaff 262144 Jul 1 12:48 testfile
$ pvfs2-viewdist -f testfile
dist_name = twod_stripe
dist_params:
num_groups:2,strip_size:65536,factor:256
Number of datafiles/servers = 1
Server 0 - tcp://ramones:3334, handle: 9223372036854775801
(7ffffffffffffff9.bstream)
$
The server logfile contains (as before)
[...]
[E 07/01 12:45] logical_to_physical_offset: Invalid num_groups/server_ct
options: gr:2 server:1
[E 07/01 12:45] next_mapped_offset: Invalid num_groups/server_ct options:
gr:2 server:1
[E 07/01 12:45] logical_to_physical_offset: Invalid num_groups/server_ct
options: gr:2 server:1
[E 07/01 12:45] next_mapped_offset: Invalid num_groups/server_ct options:
gr:2 server:1
[...]
If I set num_groups to 3 (just to test):
$ rm testfile
$ cd ..
$ setfattr -n "user.pvfs2.dist_params" -v
"strip_size:16384;num_groups:3;group_strip_factor:512" twod_stripe
$ echo $?
0
$ cd twod_stripe/
$ dd if=/dev/urandom of=testfile bs=16k count=16
16+0 records in
16+0 records out
262144 bytes (262 kB) copied, 0.144843 seconds, 1.8 MB/s
$ ls -l testfile
-rw-r--r-- 1 tonykew ccrstaff 262144 Jul 1 12:48 testfile
$ pvfs2-viewdist -f testfile
dist_name = twod_stripe
dist_params:
num_groups:2,strip_size:65536,factor:256
Number of datafiles/servers = 1
Server 0 - tcp://ramones:3334, handle: 9223372036854775800
(7ffffffffffffff8.bstream)
$
server log file also (still) shows 2 groups:
[E 07/01 12:48] logical_to_physical_offset: Invalid num_groups/server_ct
options: gr:2 server:1
[E 07/01 12:48] next_mapped_offset: Invalid num_groups/server_ct options:
gr:2 server:1
[E 07/01 12:48] logical_to_physical_offset: Invalid num_groups/server_ct
options: gr:2 server:1
[E 07/01 12:48] next_mapped_offset: Invalid num_groups/server_ct options:
gr:2 server:1
Thanks,
Tony
Tony Kew
SAN Administrator
The Center for Computational Research
New York State Center of Excellence
in Bioinformatics & Life Sciences
701 Ellicott Street, Buffalo, NY 14203
CoE Office: (716) 881-8930 Fax: (716) 849-6656
CSE Office: (716) 645-3797 x2174
Cell: (716) 560-0910 Home: (716) 874-2126
"I love deadlines, I love the whooshing noise they make as they go by."
Douglas Adams
Phil Carns wrote:
The twod_stripe problem that you describe sounds like the one that Kyle
Schochenmaier found and fixed in this mailing list thread:
http://www.beowulf-underground.org/pipermail/pvfs2-developers/2008-June/004069.html
You can check by running pvfs2-viewdist -f <filename> on the file that you
create with these parameters. If the num_groups shows up as 2 then you are
hitting a bug that prevents the distribution from honoring that setting.
The attached patch should fix it if that's the case.
-Phil
[...]
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-user
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
