>
> There are no Lustre ioctl's of any kind implemented in liblustre. The
> Lustre client driver takes them but just returns an error. Maybe my info
> dated here? I admit it's been a coupe of months since I last looked at
> this.
Actually, it is possible for a catamount client to use llapi_file_create()
to specify striping on a newly-created file, and that internally calls the
ioctl that the Lustre ADIO driver as presented here calls explicitly.
At ORNL, I have been utilizing some code of llapi_file_create to create
files with increasing number of file stripes based on file sizes, on a linux
platform though. I did notice its availability on Cray XT login nodes (maybe
available on compute nodes, but I am not that positive). However, for this
to be fully useful to MPI-IO, there needs to be a complementary API for
MPI-IO layer on compute nodes to query and validate the striping pattern,
for which ioctl is the only mechanism right now.
So for the Cray XT users, this is yet some time to pass to use additional
feature. It does not seem to be a big setback because it would take also
some time for Cray to adopt Lustre-1.6.2 to allow the use of
ADIO_FCNTL_SET_DISKSPACE
fcnt on Cray XT. By then, maybe we (ORNL) are already running CNL (compute
node linux), then presumably ioctl will no longer be an issue under CNL.
Correct me here as I am not aware of changes on CNL.
The data sieve can be engaged via an MPI hint and executed at an extant,
> higher, layer if I recall correctly.
I agree - having small IO handled via sieving or collective IO is much
better than trying to make small IOs marginally more efficient.
Not to sound all negative... As I said, those comments are only meant
for us goofy XT folks. Excluding the XT, this looks like wonderful, very
worthwhile work. Kudos and much thanks to Mr. Yu.
Being a Cray XT user for some, I understand your comments completely! I
mentioned that this patch was just the base for Linux platform.
But I do have an ADIO-UFS derived code that works for both Linux and Cray
XT, which seems to provide comparative (sometimes better) performance than
the default ADIO over libsysio over Cray XT. It also contains further
optimizations that (a) increase communication and IO overlap and (b) reduce
collective communication cost in collective IO. The benefits can be from 40%
to 200% depends what benchmarks you are running.
BTW, the work mentioned is in part for a SciDAC SDM project for which Dr.
Rob Ross is the IO lead. He is aware of the effort here too. Anyhow, good
that we have channels to the upstream MPI-IO code base.
Cheers,
--
Weikuan Yu, Ph.D.
Future Technologies Group and Technology Integration Group
http://ft.ornl.gov/~wyu/
_______________________________________________
Lustre-discuss mailing list
[email protected]
https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
