Andeas and others,
Thanks again for all the info. I guess its about time for me to attempt
to contribute to the Lustre code base. A question about
llapi_layout_dom_size(). I have code that is building up a multi
component layout. After switching to the second component with
llapi_layout_comp_use( layout, LLAPI_COMP_USE_NEXT ) I call
llapi_layout_dom_size() so I know where to set the extent start for
second component. Internal to ...dom_size() the current component gets
set back to the first component, see below. I then make calls thinking
I am modifying the second component, but I am actually modifying the
first. Is this behavior documented somewhere, that calls to
llapi_layout_... functions can change the current component without notice?
Is there a llapi function that returns the lustre file system's max dom
size? I mistakenly thought this was the purpose of
llapi_layout_dom_size(), but found out I made a bad assumption. Trying
to find the max dom_size is what sent me down this path.
John
int llapi_layout_dom_size(struct llapi_layout *layout,uint64_t *size)
{
uint64_t pattern, start;
int rc;
if (!layout || !llapi_layout_is_composite(layout)) {
*size =0;
return 0;
}
rc = llapi_layout_comp_use(layout, LLAPI_LAYOUT_COMP_USE_FIRST);
if (rc)
return -errno;
rc = llapi_layout_pattern_get(layout, &pattern);
if (rc)
return -errno;
if (pattern != LOV_PATTERN_MDT && pattern != LLAPI_LAYOUT_MDT) {
*size =0;
return 0;
}
rc = llapi_layout_comp_extent_get(layout, &start, size);
if (rc)
return -errno;
if (start)
return -ERANGE;
return 0;
}
On 7/28/22 19:42, Andreas Dilger wrote:
John, you are probably right that allowing a passed fd would also be
useful, but nobody has done it this way before because of the need to
use O_LOV_DELAY_CREATE within the application code... and to be
honest very few applications tune their IO to this extent, especially
with PFL layouts being available to handle 99% of the usage.
Please feel free to submit a patch that splits
llapi_layout_file_open() into two functions:
- llapi_layout_open_fd() (or ...fd_open()? not sure) that only opens
the file with O_LOV_DELAY_CREATE and returns the fd
- llapi_layout_set_by_fd() that sets the layout on the provided fd
(whatever the source)
Then llapi_layout_file_open() would just call those two functions
internally, and you could use only the llapi_layout_set_by_fd()
function, if available (you could add:
#defeine HAVE_LLAPI_LAYOUT_SET_BY_FD
in the lustreapi.h header to avoid the need for a separate configure
check. Otherwise, your code would call your own internal wrapper of
the same name that calls llapi_layout_file_open() and immediately
closes the returned fd. That would be slightly less efficient than
the new API (two opens and closes), but would allow you to migrate to
the new (more efficient) implementation easily in the future.
Cheers, Andreas
On Jul 28, 2022, at 14:03, John Bauer <bau...@iodoctors.com> wrote:
Andreas,
Thanks for the info. A related question: I am using the
O_LOV_DELAY_CREATE open flag mechanism to open a file and then set
the composite layout with llapi_layout_file_open(). I was kind of
surprised this worked. This ends up opening the file twice and I
simply close the fd returned from llapi_layout_file_open(). It would
seem there should be an llapi function such as
llapi_layout_set_by_fd() to match the llapi_layout_get_by_fd(). I
need to use this mechanism to set striping for files where the
pathname is not necessarily known before the open, such as the
mkstemps() family of opens. It also makes it easier to handle
setting striping for files opened with openat(). It seems it would
be more straight forward for llapi to work with an fd than a pathname
if a valid fd already exists. Am I missing an easier way to do this?
Thanks,
John
On 7/27/22 16:25, Andreas Dilger wrote:
The HLD document was written before the feature was implemented, and
is outdated. The lustreapi.h and llapi_layout_file_comp_del.3 man
page are correct. Feel free to update the wiki to use the correct
argument list.
I believe that it is possible to delete multiple components that
match the <flags> argument (e.g. LCME_FL_NEG|LCME_FL_INIT), but I
haven't tested that.
On Jul 26, 2022, at 14:35, John Bauer <bau...@iodoctors.com> wrote:
Hi all,
I would like to use the llapi_layout_file_comp_del() function. I
have found 2 prototypes in different places. One has the 3rd
argument, uint32_t flags, and the other doesn't. I suspect the
High Level Design document is incorrect. The one line of
documentation in lustreapi.h indicates I could delete multiple
components with one call. How does one do that? What are the
applicable flags?
From version 2.12.8 lustreapi.h
/**
* Delete component(s) by the specified component id or flags.
*/
int llapi_layout_file_comp_del(const char *path, uint32_t id,
uint32_t flags);
From https://wiki.lustre.org/PFL2_High_Level_Design
A new interface llapi_layout_file_comp_del(3) to delete
component(s) by the specified component id (accepting LCME_ID_*
wildcards also) from an existing file:
int llapi_layout_file_comp_del(const char *path, uint32_t id);
John
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud
Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
