Re: [f2fs-dev] [PATCH v10 3/8] libfs: Introduce case-insensitive string comparison helper
Eugen Hristev writes: > Okay, I am changing it. > > By the way, is this supposed to work like this on case-insensitive > directories ? > > user@debian-rockchip-rock5b-rk3588:~$ ls -la /media/CI_dir/*cuc > ls: cannot access '/media/CI_dir/*cuc': No such file or directory > user@debian-rockchip-rock5b-rk3588:~$ ls -la /media/CI_dir/*CUC > -rw-r--r-- 1 root root 0 Feb 12 17:47 /media/CI_dir/CUC > user@debian-rockchip-rock5b-rk3588:~$ ls -la /media/CI_dir/cuc > -rw-r--r-- 1 root root 0 Feb 12 17:47 /media/CI_dir/cuc > user@debian-rockchip-rock5b-rk3588:~$ > > > basically wildcards don't work. Yes, at least from a kernel point of view. Your shell does wildcards in userspace, probably by doing getdents and then comparing with possible matches. Since the shell itself is not case-insensitive aware, its comparison is case-sensitive, and you get these apparent weird semantics. Not ideal from a user point of view. But not a kernel bug. If it pushes people away from using case-insensitive directories in their day-to-day work and leave it to only be used by Windows compatibility layers, maybe that's a win? :) -- Gabriel Krisman Bertazi ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH v10 3/8] libfs: Introduce case-insensitive string comparison helper
On 2/19/24 16:55, Gabriel Krisman Bertazi wrote:
> Eugen Hristev writes:
>
>> On 2/16/24 18:12, Gabriel Krisman Bertazi wrote:
>>> Eugen Hristev writes:
>>>
From: Gabriel Krisman Bertazi
generic_ci_match can be used by case-insensitive filesystems to compare
strings under lookup with dirents in a case-insensitive way. This
function is currently reimplemented by each filesystem supporting
casefolding, so this reduces code duplication in filesystem-specific
code.
Signed-off-by: Gabriel Krisman Bertazi
[eugen.hris...@collabora.com: rework to first test the exact match]
Signed-off-by: Eugen Hristev
---
fs/libfs.c | 80 ++
include/linux/fs.h | 4 +++
2 files changed, 84 insertions(+)
diff --git a/fs/libfs.c b/fs/libfs.c
index bb18884ff20e..82871fa1b066 100644
--- a/fs/libfs.c
+++ b/fs/libfs.c
@@ -1773,6 +1773,86 @@ static const struct dentry_operations
generic_ci_dentry_ops = {
.d_hash = generic_ci_d_hash,
.d_compare = generic_ci_d_compare,
};
+
+/**
+ * generic_ci_match() - Match a name (case-insensitively) with a dirent.
+ * This is a filesystem helper for comparison with directory entries.
+ * generic_ci_d_compare should be used in VFS' ->d_compare instead.
+ *
+ * @parent: Inode of the parent of the dirent under comparison
+ * @name: name under lookup.
+ * @folded_name: Optional pre-folded name under lookup
+ * @de_name: Dirent name.
+ * @de_name_len: dirent name length.
+ *
+ *
>>>
>>> Since this need a respin, mind dropping the extra empty line here?
>>>
+ * Test whether a case-insensitive directory entry matches the filename
+ * being searched. If @folded_name is provided, it is used instead of
+ * recalculating the casefold of @name.
+ *
+ * Return: > 0 if the directory entry matches, 0 if it doesn't match, or
+ * < 0 on error.
+ */
+int generic_ci_match(const struct inode *parent,
+ const struct qstr *name,
+ const struct qstr *folded_name,
+ const u8 *de_name, u32 de_name_len)
+{
+ const struct super_block *sb = parent->i_sb;
+ const struct unicode_map *um = sb->s_encoding;
+ struct fscrypt_str decrypted_name = FSTR_INIT(NULL, de_name_len);
+ struct qstr dirent = QSTR_INIT(de_name, de_name_len);
+ int res;
+
+ if (IS_ENCRYPTED(parent)) {
+ const struct fscrypt_str encrypted_name =
+ FSTR_INIT((u8 *) de_name, de_name_len);
+
+ if (WARN_ON_ONCE(!fscrypt_has_encryption_key(parent)))
+ return -EINVAL;
+
+ decrypted_name.name = kmalloc(de_name_len, GFP_KERNEL);
+ if (!decrypted_name.name)
+ return -ENOMEM;
+ res = fscrypt_fname_disk_to_usr(parent, 0, 0, _name,
+ _name);
+ if (res < 0)
+ goto out;
+ dirent.name = decrypted_name.name;
+ dirent.len = decrypted_name.len;
+ }
+
+ /*
+ * Attempt a case-sensitive match first. It is cheaper and
+ * should cover most lookups, including all the sane
+ * applications that expect a case-sensitive filesystem.
+ *
>>>
>>>
+ * This comparison is safe under RCU because the caller
+ * guarantees the consistency between str and len. See
+ * __d_lookup_rcu_op_compare() for details.
+ */
>>>
>>> This paragraph doesn't really make sense here. It is originally from
>>> the d_compare hook, which can be called under RCU, but there is no RCU
>>> here. Also, here we are comparing the dirent with the
>>> name-under-lookup, name which is already safe.
>>>
>>>
+ if (folded_name->name) {
+ if (dirent.len == folded_name->len &&
+ !memcmp(folded_name->name, dirent.name, dirent.len)) {
+ res = 1;
+ goto out;
+ }
+ res = !utf8_strncasecmp_folded(um, folded_name, );
>>>
>>> Hmm, second thought on this. This will ignore errors from
>>> utf8_strncasecmp*,
>>> which CAN happen for the first time here, if the dirent itself is
>>> corrupted on disk (exactly why we have patch 6). Yes, ext4_match will drop
>>> the
>>> error, but we want to propagate it from here, such that the warning on
>>> patch 6 can trigger.
>>>
>>> This is why I did that match dance on the original submission. Sorry
>>> for suggesting it. We really want to get the error from utf8 and
>>> propagate it if it is negative. basically:
>>>
>>> res > 0: match
>>> res == 0: no match.
>>> res < 0: propagate error and let the caller handle it
>>
>> In that case I will revert to
Re: [f2fs-dev] [PATCH v10 3/8] libfs: Introduce case-insensitive string comparison helper
Eugen Hristev writes:
> On 2/16/24 18:12, Gabriel Krisman Bertazi wrote:
>> Eugen Hristev writes:
>>
>>> From: Gabriel Krisman Bertazi
>>>
>>> generic_ci_match can be used by case-insensitive filesystems to compare
>>> strings under lookup with dirents in a case-insensitive way. This
>>> function is currently reimplemented by each filesystem supporting
>>> casefolding, so this reduces code duplication in filesystem-specific
>>> code.
>>>
>>> Signed-off-by: Gabriel Krisman Bertazi
>>> [eugen.hris...@collabora.com: rework to first test the exact match]
>>> Signed-off-by: Eugen Hristev
>>> ---
>>> fs/libfs.c | 80 ++
>>> include/linux/fs.h | 4 +++
>>> 2 files changed, 84 insertions(+)
>>>
>>> diff --git a/fs/libfs.c b/fs/libfs.c
>>> index bb18884ff20e..82871fa1b066 100644
>>> --- a/fs/libfs.c
>>> +++ b/fs/libfs.c
>>> @@ -1773,6 +1773,86 @@ static const struct dentry_operations
>>> generic_ci_dentry_ops = {
>>> .d_hash = generic_ci_d_hash,
>>> .d_compare = generic_ci_d_compare,
>>> };
>>> +
>>> +/**
>>> + * generic_ci_match() - Match a name (case-insensitively) with a dirent.
>>> + * This is a filesystem helper for comparison with directory entries.
>>> + * generic_ci_d_compare should be used in VFS' ->d_compare instead.
>>> + *
>>> + * @parent: Inode of the parent of the dirent under comparison
>>> + * @name: name under lookup.
>>> + * @folded_name: Optional pre-folded name under lookup
>>> + * @de_name: Dirent name.
>>> + * @de_name_len: dirent name length.
>>> + *
>>> + *
>>
>> Since this need a respin, mind dropping the extra empty line here?
>>
>>> + * Test whether a case-insensitive directory entry matches the filename
>>> + * being searched. If @folded_name is provided, it is used instead of
>>> + * recalculating the casefold of @name.
>>> + *
>>> + * Return: > 0 if the directory entry matches, 0 if it doesn't match, or
>>> + * < 0 on error.
>>> + */
>>> +int generic_ci_match(const struct inode *parent,
>>> +const struct qstr *name,
>>> +const struct qstr *folded_name,
>>> +const u8 *de_name, u32 de_name_len)
>>> +{
>>> + const struct super_block *sb = parent->i_sb;
>>> + const struct unicode_map *um = sb->s_encoding;
>>> + struct fscrypt_str decrypted_name = FSTR_INIT(NULL, de_name_len);
>>> + struct qstr dirent = QSTR_INIT(de_name, de_name_len);
>>> + int res;
>>> +
>>> + if (IS_ENCRYPTED(parent)) {
>>> + const struct fscrypt_str encrypted_name =
>>> + FSTR_INIT((u8 *) de_name, de_name_len);
>>> +
>>> + if (WARN_ON_ONCE(!fscrypt_has_encryption_key(parent)))
>>> + return -EINVAL;
>>> +
>>> + decrypted_name.name = kmalloc(de_name_len, GFP_KERNEL);
>>> + if (!decrypted_name.name)
>>> + return -ENOMEM;
>>> + res = fscrypt_fname_disk_to_usr(parent, 0, 0, _name,
>>> + _name);
>>> + if (res < 0)
>>> + goto out;
>>> + dirent.name = decrypted_name.name;
>>> + dirent.len = decrypted_name.len;
>>> + }
>>> +
>>> + /*
>>> +* Attempt a case-sensitive match first. It is cheaper and
>>> +* should cover most lookups, including all the sane
>>> +* applications that expect a case-sensitive filesystem.
>>> +*
>>
>>
>>> +* This comparison is safe under RCU because the caller
>>> +* guarantees the consistency between str and len. See
>>> +* __d_lookup_rcu_op_compare() for details.
>>> +*/
>>
>> This paragraph doesn't really make sense here. It is originally from
>> the d_compare hook, which can be called under RCU, but there is no RCU
>> here. Also, here we are comparing the dirent with the
>> name-under-lookup, name which is already safe.
>>
>>
>>> + if (folded_name->name) {
>>> + if (dirent.len == folded_name->len &&
>>> + !memcmp(folded_name->name, dirent.name, dirent.len)) {
>>> + res = 1;
>>> + goto out;
>>> + }
>>> + res = !utf8_strncasecmp_folded(um, folded_name, );
>>
>> Hmm, second thought on this. This will ignore errors from utf8_strncasecmp*,
>> which CAN happen for the first time here, if the dirent itself is
>> corrupted on disk (exactly why we have patch 6). Yes, ext4_match will drop
>> the
>> error, but we want to propagate it from here, such that the warning on
>> patch 6 can trigger.
>>
>> This is why I did that match dance on the original submission. Sorry
>> for suggesting it. We really want to get the error from utf8 and
>> propagate it if it is negative. basically:
>>
>> res > 0: match
>> res == 0: no match.
>> res < 0: propagate error and let the caller handle it
>
> In that case I will revert to the original v9 implementation and send a v11 to
> handle that.
Please, note that the memcmp optimization is still valid. On
Re: [f2fs-dev] [PATCH v10 3/8] libfs: Introduce case-insensitive string comparison helper
On 2/16/24 18:12, Gabriel Krisman Bertazi wrote:
> Eugen Hristev writes:
>
>> From: Gabriel Krisman Bertazi
>>
>> generic_ci_match can be used by case-insensitive filesystems to compare
>> strings under lookup with dirents in a case-insensitive way. This
>> function is currently reimplemented by each filesystem supporting
>> casefolding, so this reduces code duplication in filesystem-specific
>> code.
>>
>> Signed-off-by: Gabriel Krisman Bertazi
>> [eugen.hris...@collabora.com: rework to first test the exact match]
>> Signed-off-by: Eugen Hristev
>> ---
>> fs/libfs.c | 80 ++
>> include/linux/fs.h | 4 +++
>> 2 files changed, 84 insertions(+)
>>
>> diff --git a/fs/libfs.c b/fs/libfs.c
>> index bb18884ff20e..82871fa1b066 100644
>> --- a/fs/libfs.c
>> +++ b/fs/libfs.c
>> @@ -1773,6 +1773,86 @@ static const struct dentry_operations
>> generic_ci_dentry_ops = {
>> .d_hash = generic_ci_d_hash,
>> .d_compare = generic_ci_d_compare,
>> };
>> +
>> +/**
>> + * generic_ci_match() - Match a name (case-insensitively) with a dirent.
>> + * This is a filesystem helper for comparison with directory entries.
>> + * generic_ci_d_compare should be used in VFS' ->d_compare instead.
>> + *
>> + * @parent: Inode of the parent of the dirent under comparison
>> + * @name: name under lookup.
>> + * @folded_name: Optional pre-folded name under lookup
>> + * @de_name: Dirent name.
>> + * @de_name_len: dirent name length.
>> + *
>> + *
>
> Since this need a respin, mind dropping the extra empty line here?
>
>> + * Test whether a case-insensitive directory entry matches the filename
>> + * being searched. If @folded_name is provided, it is used instead of
>> + * recalculating the casefold of @name.
>> + *
>> + * Return: > 0 if the directory entry matches, 0 if it doesn't match, or
>> + * < 0 on error.
>> + */
>> +int generic_ci_match(const struct inode *parent,
>> + const struct qstr *name,
>> + const struct qstr *folded_name,
>> + const u8 *de_name, u32 de_name_len)
>> +{
>> +const struct super_block *sb = parent->i_sb;
>> +const struct unicode_map *um = sb->s_encoding;
>> +struct fscrypt_str decrypted_name = FSTR_INIT(NULL, de_name_len);
>> +struct qstr dirent = QSTR_INIT(de_name, de_name_len);
>> +int res;
>> +
>> +if (IS_ENCRYPTED(parent)) {
>> +const struct fscrypt_str encrypted_name =
>> +FSTR_INIT((u8 *) de_name, de_name_len);
>> +
>> +if (WARN_ON_ONCE(!fscrypt_has_encryption_key(parent)))
>> +return -EINVAL;
>> +
>> +decrypted_name.name = kmalloc(de_name_len, GFP_KERNEL);
>> +if (!decrypted_name.name)
>> +return -ENOMEM;
>> +res = fscrypt_fname_disk_to_usr(parent, 0, 0, _name,
>> +_name);
>> +if (res < 0)
>> +goto out;
>> +dirent.name = decrypted_name.name;
>> +dirent.len = decrypted_name.len;
>> +}
>> +
>> +/*
>> + * Attempt a case-sensitive match first. It is cheaper and
>> + * should cover most lookups, including all the sane
>> + * applications that expect a case-sensitive filesystem.
>> + *
>
>
>> + * This comparison is safe under RCU because the caller
>> + * guarantees the consistency between str and len. See
>> + * __d_lookup_rcu_op_compare() for details.
>> + */
>
> This paragraph doesn't really make sense here. It is originally from
> the d_compare hook, which can be called under RCU, but there is no RCU
> here. Also, here we are comparing the dirent with the
> name-under-lookup, name which is already safe.
>
>
>> +if (folded_name->name) {
>> +if (dirent.len == folded_name->len &&
>> +!memcmp(folded_name->name, dirent.name, dirent.len)) {
>> +res = 1;
>> +goto out;
>> +}
>> +res = !utf8_strncasecmp_folded(um, folded_name, );
>
> Hmm, second thought on this. This will ignore errors from utf8_strncasecmp*,
> which CAN happen for the first time here, if the dirent itself is
> corrupted on disk (exactly why we have patch 6). Yes, ext4_match will drop
> the
> error, but we want to propagate it from here, such that the warning on
> patch 6 can trigger.
>
> This is why I did that match dance on the original submission. Sorry
> for suggesting it. We really want to get the error from utf8 and
> propagate it if it is negative. basically:
>
> res > 0: match
> res == 0: no match.
> res < 0: propagate error and let the caller handle it
In that case I will revert to the original v9 implementation and send a v11 to
handle that.
Eugen
>
>
>> +} else {
>> +if (dirent.len == name->len &&
>> +!memcmp(name->name, dirent.name, dirent.len) &&
>> +
Re: [f2fs-dev] [PATCH v10 3/8] libfs: Introduce case-insensitive string comparison helper
Eugen Hristev writes:
> From: Gabriel Krisman Bertazi
>
> generic_ci_match can be used by case-insensitive filesystems to compare
> strings under lookup with dirents in a case-insensitive way. This
> function is currently reimplemented by each filesystem supporting
> casefolding, so this reduces code duplication in filesystem-specific
> code.
>
> Signed-off-by: Gabriel Krisman Bertazi
> [eugen.hris...@collabora.com: rework to first test the exact match]
> Signed-off-by: Eugen Hristev
> ---
> fs/libfs.c | 80 ++
> include/linux/fs.h | 4 +++
> 2 files changed, 84 insertions(+)
>
> diff --git a/fs/libfs.c b/fs/libfs.c
> index bb18884ff20e..82871fa1b066 100644
> --- a/fs/libfs.c
> +++ b/fs/libfs.c
> @@ -1773,6 +1773,86 @@ static const struct dentry_operations
> generic_ci_dentry_ops = {
> .d_hash = generic_ci_d_hash,
> .d_compare = generic_ci_d_compare,
> };
> +
> +/**
> + * generic_ci_match() - Match a name (case-insensitively) with a dirent.
> + * This is a filesystem helper for comparison with directory entries.
> + * generic_ci_d_compare should be used in VFS' ->d_compare instead.
> + *
> + * @parent: Inode of the parent of the dirent under comparison
> + * @name: name under lookup.
> + * @folded_name: Optional pre-folded name under lookup
> + * @de_name: Dirent name.
> + * @de_name_len: dirent name length.
> + *
> + *
Since this need a respin, mind dropping the extra empty line here?
> + * Test whether a case-insensitive directory entry matches the filename
> + * being searched. If @folded_name is provided, it is used instead of
> + * recalculating the casefold of @name.
> + *
> + * Return: > 0 if the directory entry matches, 0 if it doesn't match, or
> + * < 0 on error.
> + */
> +int generic_ci_match(const struct inode *parent,
> + const struct qstr *name,
> + const struct qstr *folded_name,
> + const u8 *de_name, u32 de_name_len)
> +{
> + const struct super_block *sb = parent->i_sb;
> + const struct unicode_map *um = sb->s_encoding;
> + struct fscrypt_str decrypted_name = FSTR_INIT(NULL, de_name_len);
> + struct qstr dirent = QSTR_INIT(de_name, de_name_len);
> + int res;
> +
> + if (IS_ENCRYPTED(parent)) {
> + const struct fscrypt_str encrypted_name =
> + FSTR_INIT((u8 *) de_name, de_name_len);
> +
> + if (WARN_ON_ONCE(!fscrypt_has_encryption_key(parent)))
> + return -EINVAL;
> +
> + decrypted_name.name = kmalloc(de_name_len, GFP_KERNEL);
> + if (!decrypted_name.name)
> + return -ENOMEM;
> + res = fscrypt_fname_disk_to_usr(parent, 0, 0, _name,
> + _name);
> + if (res < 0)
> + goto out;
> + dirent.name = decrypted_name.name;
> + dirent.len = decrypted_name.len;
> + }
> +
> + /*
> + * Attempt a case-sensitive match first. It is cheaper and
> + * should cover most lookups, including all the sane
> + * applications that expect a case-sensitive filesystem.
> + *
> + * This comparison is safe under RCU because the caller
> + * guarantees the consistency between str and len. See
> + * __d_lookup_rcu_op_compare() for details.
> + */
This paragraph doesn't really make sense here. It is originally from
the d_compare hook, which can be called under RCU, but there is no RCU
here. Also, here we are comparing the dirent with the
name-under-lookup, name which is already safe.
> + if (folded_name->name) {
> + if (dirent.len == folded_name->len &&
> + !memcmp(folded_name->name, dirent.name, dirent.len)) {
> + res = 1;
> + goto out;
> + }
> + res = !utf8_strncasecmp_folded(um, folded_name, );
Hmm, second thought on this. This will ignore errors from utf8_strncasecmp*,
which CAN happen for the first time here, if the dirent itself is
corrupted on disk (exactly why we have patch 6). Yes, ext4_match will drop the
error, but we want to propagate it from here, such that the warning on
patch 6 can trigger.
This is why I did that match dance on the original submission. Sorry
for suggesting it. We really want to get the error from utf8 and
propagate it if it is negative. basically:
res > 0: match
res == 0: no match.
res < 0: propagate error and let the caller handle it
> + } else {
> + if (dirent.len == name->len &&
> + !memcmp(name->name, dirent.name, dirent.len) &&
> + (!sb_has_strict_encoding(sb) || !utf8_validate(um, name))) {
> + res = 1;
> + goto out;
> + }
> + res = !utf8_strncasecmp(um, name, );
> + }
> +
> +out:
> + kfree(decrypted_name.name);
> +
