date:20230918

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available) [block_cloning and zilsaxattr missing from loader's features_for_read]

2023-09-18 Thread Mark Millard

On Sep 18, 2023, at 17:05, Alexander Motin  wrote:

> On 18.09.2023 19:21, Mark Millard wrote:
>> On Sep 18, 2023, at 15:51, Mark Millard  wrote:
>>> Alexander Motin  wrote on
>>> Date: Mon, 18 Sep 2023 13:26:56 UTC :
 block_cloning feature is marked as READONLY_COMPAT. It should not
 require any special handling from the boot code.
>>> 
>>> From stand/libsa/zfs/zfsimpl.c but adding a comment about the
>>> read-only compatibility status of each entry:
>>> 
>>> /*
>>> * List of ZFS features supported for read
>>> */
>> static const char *features_for_read[] = {
>>>"com.datto:bookmark_v2", // READ-ONLY COMPATIBLE no
>>>"com.datto:encryption", // READ-ONLY COMPATIBLE no
>>>"com.datto:resilver_defer", // READ-ONLY COMPATIBLE yes
>>>"com.delphix:bookmark_written", // READ-ONLY COMPATIBLE no
>>>"com.delphix:device_removal", // READ-ONLY COMPATIBLE no
>>>"com.delphix:embedded_data", // READ-ONLY COMPATIBLE no
>>>"com.delphix:extensible_dataset", // READ-ONLY COMPATIBLE no
>>>"com.delphix:head_errlog", // READ-ONLY COMPATIBLE no
>>>"com.delphix:hole_birth", // READ-ONLY COMPATIBLE no
>>>"com.delphix:obsolete_counts", // READ-ONLY COMPATIBLE yes
>>>"com.delphix:spacemap_histogram", // READ-ONLY COMPATIBLE yes
>>>"com.delphix:spacemap_v2", // READ-ONLY COMPATIBLE yes
>>>"com.delphix:zpool_checkpoint", // READ-ONLY COMPATIBLE yes
>>>"com.intel:allocation_classes", // READ-ONLY COMPATIBLE yes
>>>"com.joyent:multi_vdev_crash_dump", // READ-ONLY COMPATIBLE no
>>>"com.klarasystems:vdev_zaps_v2", // READ-ONLY COMPATIBLE no
>>>"org.freebsd:zstd_compress", // READ-ONLY COMPATIBLE no
>>>"org.illumos:lz4_compress", // READ-ONLY COMPATIBLE no
>>>"org.illumos:sha512", // READ-ONLY COMPATIBLE no
>>>"org.illumos:skein", // READ-ONLY COMPATIBLE no
>>>"org.open-zfs:large_blocks", // READ-ONLY COMPATIBLE no
>>>"org.openzfs:blake3", // READ-ONLY COMPATIBLE no
>>>"org.zfsonlinux:allocation_classes", // READ-ONLY COMPATIBLE yes
>>>"org.zfsonlinux:large_dnode", // READ-ONLY COMPATIBLE no
>>>NULL
>>> };
>>> 
>>> So it appears that the design is that both "no" and "yes" ones
>>> that are known to be supported are listed and anything else is
>>> supposed to lead to rejection until explicitly added as
>>> known-compatibile.
> 
> I don't think so.  I think somebody by mistake added first featured that 
> should not be here, and then others continued this irrelevant routine. My own 
> development server/builder is happily running latest main with ZFS root 
> without any patches and with block cloning not only enabled, but even active. 
>  So as I have told, it is not needed:
> 
> mav@srv:/root# zpool get all | grep clon
> mavlab  bcloneused 20.5M  -
> mavlab  bclonesaved20.9M  -
> mavlab  bcloneratio2.02x  -
> mavlab  feature@block_cloning  active local
> 
> Somebody should go through the list and clean in up from read-compatible 
> features and document it, unless there are some features that were 
> re-qualified at some point, I haven't checked if it could be.
> 
>>> This matches up with stand/libsa/zfs/zfsimpl.c 's:
>>> 
>>> static int
>>> nvlist_check_features_for_read(nvlist_t *nvl)
>>> {
> ...
>>>rc = nvlist_find(nvl, ZPOOL_CONFIG_FEATURES_FOR_READ,
>>>DATA_TYPE_NVLIST, NULL, , NULL);
> 
> Take a note it reads ZPOOL_CONFIG_FEATURES_FOR_READ.  Same time features 
> declared as READONLY_COMPAT are stored in FEATURES_FOR_WRITE, that boot 
> loader does not even care.
> 
>>> I do not know if vfs.zfs.bclone_enabled=0 leads the loader
>>> to see vs. not-see a "com.fudosecurity:block_cloning".
> 
> bclone_enabled=0 block copy_file_range() usage, that should keep the feature 
> enabled, but not active.  It could be related if the feature would be in 
> FEATURES_FOR_WRITE, but here and now it is not.
> 
>> It appears that 2 additions afeter opebzfas-2.1-freebsd are
>> missing from the above list:
>> com.fudosecurity:block_cloning
>> org.openzfs:zilsaxattr
> 
> Nothing of ZIL is required for read-only import.  So no, it is also not 
> needed.

Thanks for the details in your notes, including that bclone_enabled=0
is not relevant.

As a result I found out about zhack feature stat POOLNAME that shows
the ZPOOL_CONFIG_FEATURES_FOR_READ separately from then ones for
write.

Looking at the pool that I used for testing block_cloning
via poudriere bulk build activity (and sorting the for_*_obj
lists produced) . . .

The for_read_obj list (with matching line added as a suffix):

com.datto:bookmark_v2 = 0   "com.datto:bookmark_v2", // 
READ-ONLY COMPATIBLE no
com.datto:encryption = 0"com.datto:encryption", // 
READ-ONLY

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available) [block_cloning and zilsaxattr missing from loader's features_for_read]

2023-09-18 Thread Alexander Motin


On 18.09.2023 19:21, Mark Millard wrote:

On Sep 18, 2023, at 15:51, Mark Millard  wrote:

Alexander Motin  wrote on
Date: Mon, 18 Sep 2023 13:26:56 UTC :

block_cloning feature is marked as READONLY_COMPAT. It should not
require any special handling from the boot code.


 From stand/libsa/zfs/zfsimpl.c but adding a comment about the
read-only compatibility status of each entry:

/*
* List of ZFS features supported for read
*/

static const char *features_for_read[] = {

"com.datto:bookmark_v2", // READ-ONLY COMPATIBLE no
"com.datto:encryption", // READ-ONLY COMPATIBLE no
"com.datto:resilver_defer", // READ-ONLY COMPATIBLE yes
"com.delphix:bookmark_written", // READ-ONLY COMPATIBLE no
"com.delphix:device_removal", // READ-ONLY COMPATIBLE no
"com.delphix:embedded_data", // READ-ONLY COMPATIBLE no
"com.delphix:extensible_dataset", // READ-ONLY COMPATIBLE no
"com.delphix:head_errlog", // READ-ONLY COMPATIBLE no
"com.delphix:hole_birth", // READ-ONLY COMPATIBLE no
"com.delphix:obsolete_counts", // READ-ONLY COMPATIBLE yes
"com.delphix:spacemap_histogram", // READ-ONLY COMPATIBLE yes
"com.delphix:spacemap_v2", // READ-ONLY COMPATIBLE yes
"com.delphix:zpool_checkpoint", // READ-ONLY COMPATIBLE yes
"com.intel:allocation_classes", // READ-ONLY COMPATIBLE yes
"com.joyent:multi_vdev_crash_dump", // READ-ONLY COMPATIBLE no
"com.klarasystems:vdev_zaps_v2", // READ-ONLY COMPATIBLE no
"org.freebsd:zstd_compress", // READ-ONLY COMPATIBLE no
"org.illumos:lz4_compress", // READ-ONLY COMPATIBLE no
"org.illumos:sha512", // READ-ONLY COMPATIBLE no
"org.illumos:skein", // READ-ONLY COMPATIBLE no
"org.open-zfs:large_blocks", // READ-ONLY COMPATIBLE no
"org.openzfs:blake3", // READ-ONLY COMPATIBLE no
"org.zfsonlinux:allocation_classes", // READ-ONLY COMPATIBLE yes
"org.zfsonlinux:large_dnode", // READ-ONLY COMPATIBLE no
NULL
};

So it appears that the design is that both "no" and "yes" ones
that are known to be supported are listed and anything else is
supposed to lead to rejection until explicitly added as
known-compatibile.


I don't think so.  I think somebody by mistake added first featured that 
should not be here, and then others continued this irrelevant routine. 
My own development server/builder is happily running latest main with 
ZFS root without any patches and with block cloning not only enabled, 
but even active.  So as I have told, it is not needed:


mav@srv:/root# zpool get all | grep clon
mavlab  bcloneused 20.5M  -
mavlab  bclonesaved20.9M  -
mavlab  bcloneratio2.02x  -
mavlab  feature@block_cloning  active local

Somebody should go through the list and clean in up from read-compatible 
features and document it, unless there are some features that were 
re-qualified at some point, I haven't checked if it could be.



This matches up with stand/libsa/zfs/zfsimpl.c 's:

static int
nvlist_check_features_for_read(nvlist_t *nvl)
{

...

rc = nvlist_find(nvl, ZPOOL_CONFIG_FEATURES_FOR_READ,
DATA_TYPE_NVLIST, NULL, , NULL);


Take a note it reads ZPOOL_CONFIG_FEATURES_FOR_READ.  Same time features 
declared as READONLY_COMPAT are stored in FEATURES_FOR_WRITE, that boot 
loader does not even care.



I do not know if vfs.zfs.bclone_enabled=0 leads the loader
to see vs. not-see a "com.fudosecurity:block_cloning".


bclone_enabled=0 block copy_file_range() usage, that should keep the 
feature enabled, but not active.  It could be related if the feature 
would be in FEATURES_FOR_WRITE, but here and now it is not.



It appears that 2 additions afeter opebzfas-2.1-freebsd are
missing from the above list:

com.fudosecurity:block_cloning
org.openzfs:zilsaxattr


Nothing of ZIL is required for read-only import.  So no, it is also not 
needed.


--
Alexander Motin

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available) [block_cloning and zilsaxattr missing from loader's features_for_read]

2023-09-18 Thread Mark Millard




On Sep 18, 2023, at 15:51, Mark Millard  wrote:

> Alexander Motin  wrote on
> Date: Mon, 18 Sep 2023 13:26:56 UTC :
> 
>> block_cloning feature is marked as READONLY_COMPAT. It should not 
>> require any special handling from the boot code.
> 
> From stand/libsa/zfs/zfsimpl.c but adding a comment about the
> read-only compatibility status of each entry:
> 
> /*
> * List of ZFS features supported for read
> */
> static const char *features_e: vfs.zfs.bclone_enabled

Sorry, a stupid error of mine messed up the above line.
It should have been:

static const char *features_for_read[] = {

>"com.datto:bookmark_v2", // READ-ONLY COMPATIBLE no
>"com.datto:encryption", // READ-ONLY COMPATIBLE no
>"com.datto:resilver_defer", // READ-ONLY COMPATIBLE yes
>"com.delphix:bookmark_written", // READ-ONLY COMPATIBLE no
>"com.delphix:device_removal", // READ-ONLY COMPATIBLE no
>"com.delphix:embedded_data", // READ-ONLY COMPATIBLE no
>"com.delphix:extensible_dataset", // READ-ONLY COMPATIBLE no
>"com.delphix:head_errlog", // READ-ONLY COMPATIBLE no
>"com.delphix:hole_birth", // READ-ONLY COMPATIBLE no
>"com.delphix:obsolete_counts", // READ-ONLY COMPATIBLE yes
>"com.delphix:spacemap_histogram", // READ-ONLY COMPATIBLE yes
>"com.delphix:spacemap_v2", // READ-ONLY COMPATIBLE yes
>"com.delphix:zpool_checkpoint", // READ-ONLY COMPATIBLE yes
>"com.intel:allocation_classes", // READ-ONLY COMPATIBLE yes
>"com.joyent:multi_vdev_crash_dump", // READ-ONLY COMPATIBLE no
>"com.klarasystems:vdev_zaps_v2", // READ-ONLY COMPATIBLE no
>"org.freebsd:zstd_compress", // READ-ONLY COMPATIBLE no
>"org.illumos:lz4_compress", // READ-ONLY COMPATIBLE no
>"org.illumos:sha512", // READ-ONLY COMPATIBLE no
>"org.illumos:skein", // READ-ONLY COMPATIBLE no
>"org.open-zfs:large_blocks", // READ-ONLY COMPATIBLE no
>"org.openzfs:blake3", // READ-ONLY COMPATIBLE no
>"org.zfsonlinux:allocation_classes", // READ-ONLY COMPATIBLE yes
>"org.zfsonlinux:large_dnode", // READ-ONLY COMPATIBLE no
>NULL
> };
> 
> So it appears that the design is that both "no" and "yes" ones
> that are known to be supported are listed and anything else is
> supposed to lead to rejection until explicitly added as
> known-compatibile.
> 
> This matches up with stand/libsa/zfs/zfsimpl.c 's:
> 
> static int
> nvlist_check_features_for_read(nvlist_t *nvl)
> {
>nvlist_t *features = NULL;
>nvs_data_t *data;
>nvp_header_t *nvp;
>nv_string_t *nvp_name;
>int rc;
> 
>rc = nvlist_find(nvl, ZPOOL_CONFIG_FEATURES_FOR_READ,
>DATA_TYPE_NVLIST, NULL, , NULL);
>switch (rc) {
>case 0:
>break;  /* Continue with checks */
> 
>case ENOENT:
>return (0); /* All features are disabled */
> 
>default:
>return (rc);/* Error while reading nvlist */
>}
> 
>data = (nvs_data_t *)features->nv_data;
>nvp = >nvl_pair;  /* first pair in nvlist */
> 
>while (nvp->encoded_size != 0 && nvp->decoded_size != 0) {
>int i, found;
> 
>nvp_name = (nv_string_t *)((uintptr_t)nvp + sizeof(*nvp));
>found = 0;
> 
>for (i = 0; features_for_read[i] != NULL; i++) {
>if (memcmp(nvp_name->nv_data, features_for_read[i],
>nvp_name->nv_size) == 0) {
>found = 1;
>break;
>}
>}
> 
>if (!found) {
>printf("ZFS: unsupported feature: %.*s\n",
>nvp_name->nv_size, nvp_name->nv_data);
>rc = EIO;
>}
>nvp = (nvp_header_t *)((uint8_t *)nvp + nvp->encoded_size);
>}
>nvlist_destroy(features);
> 
>return (rc);
> }
> 
> I do not know if vfs.zfs.bclone_enabled=0 leads the loader
> to see vs. not-see a "com.fudosecurity:block_cloning".

It appears that 2 additions afeter opebzfas-2.1-freebsd are
missing from the above list:

com.fudosecurity:block_cloning
org.openzfs:zilsaxattr

See, for reference (but shorter naming):

# diff -u99 
/usr/main-src/sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.1-freebsd 
/usr/main-src/sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.2
--- 
/usr/main-src/sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.1-freebsd 
2021-06-24 20:08:57.206621000 -0700
+++ /usr/main-src/sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.2 
2023-06-10 15:51:13.220607000 -0700
@@ -1,34 +1,40 @@
-# Features supported by OpenZFS 2.1 on FreeBSD
+# Features supported by OpenZFS 2.2 on Linux and FreeBSD
 allocation_classes
 async_destroy
+blake3
+block_cloning
 bookmark_v2

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available)

2023-09-18 Thread Mark Millard

Alexander Motin  wrote on
Date: Mon, 18 Sep 2023 13:26:56 UTC :

> block_cloning feature is marked as READONLY_COMPAT. It should not 
> require any special handling from the boot code.

From stand/libsa/zfs/zfsimpl.c but adding a comment about the
read-only compatibility status of each entry:

/*
 * List of ZFS features supported for read
 */
static const char *features_e: vfs.zfs.bclone_enabled
"com.datto:bookmark_v2",// READ-ONLY COMPATIBLE no
"com.datto:encryption", // READ-ONLY COMPATIBLE no
"com.datto:resilver_defer", // READ-ONLY COMPATIBLE yes
"com.delphix:bookmark_written", // READ-ONLY COMPATIBLE no
"com.delphix:device_removal",   // READ-ONLY COMPATIBLE no
"com.delphix:embedded_data",// READ-ONLY COMPATIBLE no
"com.delphix:extensible_dataset",   // READ-ONLY COMPATIBLE no
"com.delphix:head_errlog",  // READ-ONLY COMPATIBLE no
"com.delphix:hole_birth",   // READ-ONLY COMPATIBLE no
"com.delphix:obsolete_counts",  // READ-ONLY COMPATIBLE yes
"com.delphix:spacemap_histogram",   // READ-ONLY COMPATIBLE yes
"com.delphix:spacemap_v2",  // READ-ONLY COMPATIBLE yes
"com.delphix:zpool_checkpoint", // READ-ONLY COMPATIBLE yes
"com.intel:allocation_classes", // READ-ONLY COMPATIBLE yes
"com.joyent:multi_vdev_crash_dump", // READ-ONLY COMPATIBLE no
"com.klarasystems:vdev_zaps_v2",// READ-ONLY COMPATIBLE no
"org.freebsd:zstd_compress",// READ-ONLY COMPATIBLE no
"org.illumos:lz4_compress", // READ-ONLY COMPATIBLE no
"org.illumos:sha512",   // READ-ONLY COMPATIBLE no
"org.illumos:skein",// READ-ONLY COMPATIBLE no
"org.open-zfs:large_blocks",// READ-ONLY COMPATIBLE no
"org.openzfs:blake3",   // READ-ONLY COMPATIBLE no
"org.zfsonlinux:allocation_classes",// READ-ONLY COMPATIBLE yes
"org.zfsonlinux:large_dnode",   // READ-ONLY COMPATIBLE no
NULL
};

So it appears that the design is that both "no" and "yes" ones
that are known to be supported are listed and anything else is
supposed to lead to rejection until explicitly added as
known-compatibile.

This matches up with stand/libsa/zfs/zfsimpl.c 's:

static int
nvlist_check_features_for_read(nvlist_t *nvl)
{
nvlist_t *features = NULL;
nvs_data_t *data;
nvp_header_t *nvp;
nv_string_t *nvp_name;
int rc;

rc = nvlist_find(nvl, ZPOOL_CONFIG_FEATURES_FOR_READ,
DATA_TYPE_NVLIST, NULL, , NULL);
switch (rc) {
case 0:
break;  /* Continue with checks */

case ENOENT:
return (0); /* All features are disabled */

default:
return (rc);/* Error while reading nvlist */
}

data = (nvs_data_t *)features->nv_data;
nvp = >nvl_pair;  /* first pair in nvlist */

while (nvp->encoded_size != 0 && nvp->decoded_size != 0) {
int i, found;

nvp_name = (nv_string_t *)((uintptr_t)nvp + sizeof(*nvp));
found = 0;

for (i = 0; features_for_read[i] != NULL; i++) {
if (memcmp(nvp_name->nv_data, features_for_read[i],
nvp_name->nv_size) == 0) {
found = 1;
break;
}
}

if (!found) {
printf("ZFS: unsupported feature: %.*s\n",
nvp_name->nv_size, nvp_name->nv_data);
rc = EIO;
}
nvp = (nvp_header_t *)((uint8_t *)nvp + nvp->encoded_size);
}
nvlist_destroy(features);

return (rc);
}

I do not know if vfs.zfs.bclone_enabled=0 leads the loader
to see vs. not-see a "com.fudosecurity:block_cloning".

===
Mark Millard
marklmi at yahoo.com

Re: [Intel AlderLake] Read files to FAT32 or UFS partition cause data corrupt due to P-Core-Core

2023-09-18 Thread Konstantin Belousov

On Mon, Sep 18, 2023 at 01:27:55PM -0500, Mike Karels wrote:
> On 18 Sep 2023, at 10:38, Michael Butler wrote:
> 
> > On 8/8/23 13:50, Michael Butler wrote:
> >> On 8/8/23 10:56, Tomoaki AOKI wrote:
> >>> On Tue, 8 Aug 2023 17:02:32 +0300
> >>> Konstantin Belousov  wrote:
> >>
> >>   [ .. snip .. ]
> >>
>  The workaround is switched on automatically, when kernel detects 'small 
>  cores'
>  reported by CPUID.
> >>>
> >>> If I read the code correctly, vm.pmap.pcid_invlpg_workaround
> >>> (precicely, the corresponding variable) is set to non-zero when the
> >>> workaround is enabled. Not sure it was detected correctly at the
> >>> original reporter's environment, but forcibly setting the tunable to 1
> >>> didn't reported to help sufficiently.
> >>> Currently, only setting tunable vm.pmap.pcid_enabled to 0 could help.
> >>
> >> I'm seeing similar stability problems on an N95-based device. This too is 
> >> an Alderlake-N device with only E-cores although I'm running it with a 
> >> compilation with CPUTYPE=tremont .. from an older, verbose start-up ..
> >>
> >> PPIM 0: PA=0x40, VA=0x8271, size=0x1d5000, mode=0x1
> >> pmap: large map 8 PML4 slots (4096 GB)
> >> VT(efifb): resolution 800x600
> >> Preloaded elf kernel "/boot/kernel.new/kernel" at 0x8234e000.
> >> Preloaded boot_entropy_cache "/boot/entropy" at 0x82357d08.
> >> Preloaded cpu_microcode "/boot/firmware/intel-ucode.bin" at 
> >> 0x82357d60.
> >> Preloaded hostuuid "/etc/hostid" at 0x82357dc0.
> >> Preloaded TSLOG data "TSLOG" at 0x82357e10.
> >> CPU: Intel(R) N95 (1689.60-MHz K8-class CPU)
> >>    Origin="GenuineIntel"  Id=0xb06e0  Family=0x6  Model=0xbe  Stepping=0
> >>
> >> Features=0xbfebfbff
> >>
> >> Features2=0x7ffafbbf
> >>    AMD Features=0x2c100800
> >>    AMD Features2=0x121
> >>    Structured Extended 
> >> Features=0x239ca7eb
> >>    Structured Extended 
> >> Features2=0x98c007bc
> >>    Structured Extended 
> >> Features3=0xfc184410
> >>    XSAVE Features=0xf
> >>    IA32_ARCH_CAPS=0x180fd6b
> >>    VT-x: Basic Features=0x3da0500
> >>      Pin-Based Controls=0xff
> >>      Primary Processor 
> >> Controls=0xfffbfffe
> >>      Secondary Processor 
> >> Controls=0x75d7fff
> >>      Exit Controls=0x3da0500
> >>      Entry Controls=0x3da0500
> >>      EPT Features=0x6f34141
> >>      VPID Features=0xf01
> >>    TSC: P-state invariant, performance statistics
> >> 64-Byte prefetching
> >> L2 cache: 2048 kbytes, 16-way associative, 64 bytes/line
> >> real memory  = 17179869184 (16384 MB)
> >> Physical memory chunk(s):
> >> 0x0001 - 0x0009dfff, 581632 bytes (142 pages)
> >> 0x0009f000 - 0x0009, 4096 bytes (1 pages)
> >> 0x0010 - 0x5fff, 1609564160 bytes (392960 pages)
> >> 0x62401000 - 0x7264dfff, 270848000 bytes (66125 pages)
> >> 0x75fff000 - 0x75ff, 4096 bytes (1 pages)
> >> 0x00011000 - 0x000462497fff, 14533881856 bytes (3548311 pages)
> >> 0x00047fa0 - 0x00047fb68fff, 1478656 bytes (361 pages)
> >> avail memory = 16363008000 (15604 MB)
> >> CPU microcode: updated from 0xc to 0x10
> >
> > With the most recent microcode update, this device reports ..
> >
> > CPU microcode: updated from 0xc to 0x11
> >
> >  .. and is now stable with vm.pmap.pcid_enabled=0, 
> > vm.pmap.pcid_invlpg_workaround=1, and CPUTYPE?=alderlake set in 
> > /etc/make.conf over multiple full system builds.
> >
> > I have not tested with vm.pmap.pcid_invlpg_workaround=0.
> 
> I believe that vm.pmap.pcid_invlpg_workaround does not matter if
> vm.pmap.pcid_enabled=0.  Enabling the workaround or disabling pcid should
> be basically the same for this CPU, so I don't understand why that isn't
> true.
No, pcid != workaround.  The big is that INVPG instruction leaves some
entries in TLB which it should not, when PCID is enabled.  Workaround uses
a different instruction to ensure that no ghost entries left.

So it is possible that there is some other issue which makes similar
visible effect, with PCID enabled.

The problem is most likely because it is first Atom micro-arch where Intel
added PCID.

> It might be interesting to test with pcid enabled with the new
> microcode, although I don't see why that would affect the results (pcid
> should still not be used on any CPU).
Yes, testing a microcode with the fix for the original issue would be
interesting.  It is not nop, see above.

> 
> The CPUTYPE for the compiler should not affect the pcid vm issues, just
> change the optimization by the compiler.

True as well.

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available)

2023-09-18 Thread Tomoaki AOKI

At least, if I read the code correctly,
"com.fudosecurity:block_cloning",
should be added to array
*features_for_read[]
of stand/libsa/zfs/zfsimpl.c.

There are check codes like below, so without it, boot codes would
reject to boot from any pool having block_cloning feature enabled.
Am I missing something?

>   for (i = 0; features_for_read[i] != NULL; i++) {
>   if (memcmp(nvp_name->nv_data,
features_for_read[i], nvp_name->nv_size) == 0) {
>   found = 1;
>   break;
>   }
>   }
> 
>   if (!found) {
>   printf("ZFS: unsupported feature: %.*s\n",
>   nvp_name->nv_size, nvp_name->nv_data);
>   rc = EIO;
>   }

Regards.


On Mon, 18 Sep 2023 09:26:56 -0400
Alexander Motin  wrote:

> block_cloning feature is marked as READONLY_COMPAT.  It should not 
> require any special handling from the boot code.
> 
> On 18.09.2023 07:22, Tomoaki AOKI wrote:
> > Really OK?
> > 
> > I cannot find block_cloning in array *features_for_read[] of
> > stand/libsa/zfs/zfsimpl.c, which possibly mean boot codes (including
> > loader) cannot boot from Root-on-ZFS pool having block_cloning active.
> > 
> > Not sure adding '"com.fudosecurity:block_cloning",' here is sufficient
> > or not. Possibly more works are needed.
> > 
> > IMHO, all default-enabled features should be safe for booting.
> > Implement features with disalded, impement boot codes to support them,
> > then finally enable them by default should be the only valid route.
> > 
> > 
> > [1] https://cgit.freebsd.org/src/tree/stand/libsa/zfs/zfsimpl.c
> > 
> > 
> > On Mon, 18 Sep 2023 07:31:46 +0200
> > Martin Matuska  wrote:
> > 
> >> I vote for enabling block cloning on main :-)
> >>
> >> mm
> >>
> >> On 16. 9. 2023 19:14, Alexander Motin wrote:
> >>> On 16.09.2023 01:25, Graham Perrin wrote:
>  On 16/09/2023 01:28, Glen Barber wrote:
> > o A fix for the ZFS block_cloning feature has been implemented.
> 
>  Thanks
> 
>  I see
>  ,
>  with
>  
>  in stable/14.
> 
>  As vfs.zfs.bclone_enabled is still 0 (at least, with 15.0-CURRENT
>  n265350-72d97e1dd9cc): should we assume that additional fixes, not
>  necessarily in time for 14.0-RELEASE, will be required before
>  vfs.zfs.bclone_enabled can default to 1?
> >>>
> >>> I am not aware of any block cloning issues now.  All this thread about
> >>> bclone_enabled actually started after I asked why it is still
> >>> disabled. Thanks to Mark Millard for spotting this issue I could fix,
> >>> but now we are back at the point of re-enabling it again.  Since the
> >>> tunable does not even exist anywhere outside of FreeBSD base tree, I'd
> >>> propose to give this code another try here too.  I see no point to
> >>> have it disabled at least in main unless somebody needs time to run
> >>> some specific tests first.
> > 
> 
> -- 
> Alexander Motin

-- 
Tomoaki AOKI

Re: [Intel AlderLake] Read files to FAT32 or UFS partition cause data corrupt due to P-Core-Core

2023-09-18 Thread Michael Butler


On 9/18/23 14:27, Mike Karels wrote:

 [ .. snip .. ]


avail memory = 16363008000 (15604 MB)
CPU microcode: updated from 0xc to 0x10


With the most recent microcode update, this device reports ..

CPU microcode: updated from 0xc to 0x11

  .. and is now stable with vm.pmap.pcid_enabled=0, 
vm.pmap.pcid_invlpg_workaround=1, and CPUTYPE?=alderlake set in /etc/make.conf 
over multiple full system builds.

I have not tested with vm.pmap.pcid_invlpg_workaround=0.


 .. sorry that was a typo .. I'm actually using ..

vm.pmap.pcid_invlpg_workaround: 1
vm.pmap.invpcid_works: 1
vm.pmap.pcid_enabled: 1


I believe that vm.pmap.pcid_invlpg_workaround does not matter if
vm.pmap.pcid_enabled=0.  Enabling the workaround or disabling pcid should
be basically the same for this CPU, so I don't understand why that isn't
true.  It might be interesting to test with pcid enabled with the new
microcode, although I don't see why that would affect the results (pcid
should still not be used on any CPU).

The CPUTYPE for the compiler should not affect the pcid vm issues, just
change the optimization by the compiler.


Agreed. However, I was previously using CPUTYPE?=tremont so I just 
wanted to note that two things had changed in my testing, microcode and 
CPUTYPE, not just one,


Michael

Re: [Intel AlderLake] Read files to FAT32 or UFS partition cause data corrupt due to P-Core-Core

2023-09-18 Thread Mike Karels

On 18 Sep 2023, at 10:38, Michael Butler wrote:

> On 8/8/23 13:50, Michael Butler wrote:
>> On 8/8/23 10:56, Tomoaki AOKI wrote:
>>> On Tue, 8 Aug 2023 17:02:32 +0300
>>> Konstantin Belousov  wrote:
>>
>>   [ .. snip .. ]
>>
 The workaround is switched on automatically, when kernel detects 'small 
 cores'
 reported by CPUID.
>>>
>>> If I read the code correctly, vm.pmap.pcid_invlpg_workaround
>>> (precicely, the corresponding variable) is set to non-zero when the
>>> workaround is enabled. Not sure it was detected correctly at the
>>> original reporter's environment, but forcibly setting the tunable to 1
>>> didn't reported to help sufficiently.
>>> Currently, only setting tunable vm.pmap.pcid_enabled to 0 could help.
>>
>> I'm seeing similar stability problems on an N95-based device. This too is an 
>> Alderlake-N device with only E-cores although I'm running it with a 
>> compilation with CPUTYPE=tremont .. from an older, verbose start-up ..
>>
>> PPIM 0: PA=0x40, VA=0x8271, size=0x1d5000, mode=0x1
>> pmap: large map 8 PML4 slots (4096 GB)
>> VT(efifb): resolution 800x600
>> Preloaded elf kernel "/boot/kernel.new/kernel" at 0x8234e000.
>> Preloaded boot_entropy_cache "/boot/entropy" at 0x82357d08.
>> Preloaded cpu_microcode "/boot/firmware/intel-ucode.bin" at 
>> 0x82357d60.
>> Preloaded hostuuid "/etc/hostid" at 0x82357dc0.
>> Preloaded TSLOG data "TSLOG" at 0x82357e10.
>> CPU: Intel(R) N95 (1689.60-MHz K8-class CPU)
>>    Origin="GenuineIntel"  Id=0xb06e0  Family=0x6  Model=0xbe  Stepping=0
>>
>> Features=0xbfebfbff
>>
>> Features2=0x7ffafbbf
>>    AMD Features=0x2c100800
>>    AMD Features2=0x121
>>    Structured Extended 
>> Features=0x239ca7eb
>>    Structured Extended 
>> Features2=0x98c007bc
>>    Structured Extended 
>> Features3=0xfc184410
>>    XSAVE Features=0xf
>>    IA32_ARCH_CAPS=0x180fd6b
>>    VT-x: Basic Features=0x3da0500
>>      Pin-Based Controls=0xff
>>      Primary Processor 
>> Controls=0xfffbfffe
>>      Secondary Processor 
>> Controls=0x75d7fff
>>      Exit Controls=0x3da0500
>>      Entry Controls=0x3da0500
>>      EPT Features=0x6f34141
>>      VPID Features=0xf01
>>    TSC: P-state invariant, performance statistics
>> 64-Byte prefetching
>> L2 cache: 2048 kbytes, 16-way associative, 64 bytes/line
>> real memory  = 17179869184 (16384 MB)
>> Physical memory chunk(s):
>> 0x0001 - 0x0009dfff, 581632 bytes (142 pages)
>> 0x0009f000 - 0x0009, 4096 bytes (1 pages)
>> 0x0010 - 0x5fff, 1609564160 bytes (392960 pages)
>> 0x62401000 - 0x7264dfff, 270848000 bytes (66125 pages)
>> 0x75fff000 - 0x75ff, 4096 bytes (1 pages)
>> 0x00011000 - 0x000462497fff, 14533881856 bytes (3548311 pages)
>> 0x00047fa0 - 0x00047fb68fff, 1478656 bytes (361 pages)
>> avail memory = 16363008000 (15604 MB)
>> CPU microcode: updated from 0xc to 0x10
>
> With the most recent microcode update, this device reports ..
>
> CPU microcode: updated from 0xc to 0x11
>
>  .. and is now stable with vm.pmap.pcid_enabled=0, 
> vm.pmap.pcid_invlpg_workaround=1, and CPUTYPE?=alderlake set in 
> /etc/make.conf over multiple full system builds.
>
> I have not tested with vm.pmap.pcid_invlpg_workaround=0.

I believe that vm.pmap.pcid_invlpg_workaround does not matter if
vm.pmap.pcid_enabled=0.  Enabling the workaround or disabling pcid should
be basically the same for this CPU, so I don't understand why that isn't
true.  It might be interesting to test with pcid enabled with the new
microcode, although I don't see why that would affect the results (pcid
should still not be used on any CPU).

The CPUTYPE for the compiler should not affect the pcid vm issues, just
change the optimization by the compiler.

Mike

>> On start-up, vm.pmap.pcid_invlpg_workaround=1 but seemingly random faults 
>> still occurred under load, for example, 'make buildworld'. Apparent misreads 
>> of source-files resulting in syntax errors were the most common symptom. 
>> Compilation reattempts (mostly) succeed.
>>
>> Initially, I put this down to an inadequate power-supply but setting 
>> vm.pmap.pcid_enabled=0 seems to have stabilised it.
>>
>> I guess there's another dragon in there .. :-(
>>
>>  Michael

Re: [Intel AlderLake] Read files to FAT32 or UFS partition cause data corrupt due to P-Core-Core

2023-09-18 Thread Michael Butler


On 8/8/23 13:50, Michael Butler wrote:

On 8/8/23 10:56, Tomoaki AOKI wrote:

On Tue, 8 Aug 2023 17:02:32 +0300
Konstantin Belousov  wrote:


  [ .. snip .. ]

The workaround is switched on automatically, when kernel detects 
'small cores'

reported by CPUID.


If I read the code correctly, vm.pmap.pcid_invlpg_workaround
(precicely, the corresponding variable) is set to non-zero when the
workaround is enabled. Not sure it was detected correctly at the
original reporter's environment, but forcibly setting the tunable to 1
didn't reported to help sufficiently.
Currently, only setting tunable vm.pmap.pcid_enabled to 0 could help.


I'm seeing similar stability problems on an N95-based device. This too 
is an Alderlake-N device with only E-cores although I'm running it with 
a compilation with CPUTYPE=tremont .. from an older, verbose start-up ..


PPIM 0: PA=0x40, VA=0x8271, size=0x1d5000, mode=0x1
pmap: large map 8 PML4 slots (4096 GB)
VT(efifb): resolution 800x600
Preloaded elf kernel "/boot/kernel.new/kernel" at 0x8234e000.
Preloaded boot_entropy_cache "/boot/entropy" at 0x82357d08.
Preloaded cpu_microcode "/boot/firmware/intel-ucode.bin" at 
0x82357d60.

Preloaded hostuuid "/etc/hostid" at 0x82357dc0.
Preloaded TSLOG data "TSLOG" at 0x82357e10.
CPU: Intel(R) N95 (1689.60-MHz K8-class CPU)
   Origin="GenuineIntel"  Id=0xb06e0  Family=0x6  Model=0xbe  Stepping=0

Features=0xbfebfbff

Features2=0x7ffafbbf
   AMD Features=0x2c100800
   AMD Features2=0x121
   Structured Extended 
Features=0x239ca7eb
   Structured Extended 
Features2=0x98c007bc
   Structured Extended 
Features3=0xfc184410

   XSAVE Features=0xf
   IA32_ARCH_CAPS=0x180fd6b
   VT-x: Basic Features=0x3da0500
     Pin-Based Controls=0xff
     Primary Processor 
Controls=0xfffbfffe
     Secondary Processor 
Controls=0x75d7fff

     Exit Controls=0x3da0500
     Entry Controls=0x3da0500
     EPT Features=0x6f34141
     VPID Features=0xf01
   TSC: P-state invariant, performance statistics
64-Byte prefetching
L2 cache: 2048 kbytes, 16-way associative, 64 bytes/line
real memory  = 17179869184 (16384 MB)
Physical memory chunk(s):
0x0001 - 0x0009dfff, 581632 bytes (142 pages)
0x0009f000 - 0x0009, 4096 bytes (1 pages)
0x0010 - 0x5fff, 1609564160 bytes (392960 pages)
0x62401000 - 0x7264dfff, 270848000 bytes (66125 pages)
0x75fff000 - 0x75ff, 4096 bytes (1 pages)
0x00011000 - 0x000462497fff, 14533881856 bytes (3548311 pages)
0x00047fa0 - 0x00047fb68fff, 1478656 bytes (361 pages)
avail memory = 16363008000 (15604 MB)
CPU microcode: updated from 0xc to 0x10


With the most recent microcode update, this device reports ..

CPU microcode: updated from 0xc to 0x11

 .. and is now stable with vm.pmap.pcid_enabled=0, 
vm.pmap.pcid_invlpg_workaround=1, and CPUTYPE?=alderlake set in 
/etc/make.conf over multiple full system builds.


I have not tested with vm.pmap.pcid_invlpg_workaround=0.

On start-up, vm.pmap.pcid_invlpg_workaround=1 but seemingly random 
faults still occurred under load, for example, 'make buildworld'. 
Apparent misreads of source-files resulting in syntax errors were the 
most common symptom. Compilation reattempts (mostly) succeed.


Initially, I put this down to an inadequate power-supply but setting 
vm.pmap.pcid_enabled=0 seems to have stabilised it.


I guess there's another dragon in there .. :-(

 Michael

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available)

2023-09-18 Thread Alexander Motin

block_cloning feature is marked as READONLY_COMPAT.  It should not 
require any special handling from the boot code.


On 18.09.2023 07:22, Tomoaki AOKI wrote:

Really OK?

I cannot find block_cloning in array *features_for_read[] of
stand/libsa/zfs/zfsimpl.c, which possibly mean boot codes (including
loader) cannot boot from Root-on-ZFS pool having block_cloning active.

Not sure adding '"com.fudosecurity:block_cloning",' here is sufficient
or not. Possibly more works are needed.

IMHO, all default-enabled features should be safe for booting.
Implement features with disalded, impement boot codes to support them,
then finally enable them by default should be the only valid route.


[1] https://cgit.freebsd.org/src/tree/stand/libsa/zfs/zfsimpl.c


On Mon, 18 Sep 2023 07:31:46 +0200
Martin Matuska  wrote:


I vote for enabling block cloning on main :-)

mm

On 16. 9. 2023 19:14, Alexander Motin wrote:

On 16.09.2023 01:25, Graham Perrin wrote:

On 16/09/2023 01:28, Glen Barber wrote:

o A fix for the ZFS block_cloning feature has been implemented.


Thanks

I see
,
with

in stable/14.

As vfs.zfs.bclone_enabled is still 0 (at least, with 15.0-CURRENT
n265350-72d97e1dd9cc): should we assume that additional fixes, not
necessarily in time for 14.0-RELEASE, will be required before
vfs.zfs.bclone_enabled can default to 1?


I am not aware of any block cloning issues now.  All this thread about
bclone_enabled actually started after I asked why it is still
disabled. Thanks to Mark Millard for spotting this issue I could fix,
but now we are back at the point of re-enabling it again.  Since the
tunable does not even exist anywhere outside of FreeBSD base tree, I'd
propose to give this code another try here too.  I see no point to
have it disabled at least in main unless somebody needs time to run
some specific tests first.




--
Alexander Motin

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available)

2023-09-18 Thread Tomoaki AOKI

(Intentionally dropped @gmail.com recipients, as gmail refuses to
accept emaif from my email domain, unfortunately.)

Really OK?

I cannot find block_cloning in array *features_for_read[] of
stand/libsa/zfs/zfsimpl.c, which possibly mean boot codes (including
loader) cannot boot from Root-on-ZFS pool having block_cloning active.

Not sure adding '"com.fudosecurity:block_cloning",' here is sufficient
or not. Possibly more works are needed.

IMHO, all default-enabled features should be safe for booting.
Implement features with disalded, impement boot codes to support them,
then finally enable them by default should be the only valid route.


[1] https://cgit.freebsd.org/src/tree/stand/libsa/zfs/zfsimpl.c


On Mon, 18 Sep 2023 07:31:46 +0200
Martin Matuska  wrote:

> I vote for enabling block cloning on main :-)
> 
> mm
> 
> On 16. 9. 2023 19:14, Alexander Motin wrote:
> > On 16.09.2023 01:25, Graham Perrin wrote:
> >> On 16/09/2023 01:28, Glen Barber wrote:
> >>> o A fix for the ZFS block_cloning feature has been implemented.
> >>
> >> Thanks
> >>
> >> I see 
> >> ,
> >>  
> >> with 
> >> 
> >>  
> >> in stable/14.
> >>
> >> As vfs.zfs.bclone_enabled is still 0 (at least, with 15.0-CURRENT 
> >> n265350-72d97e1dd9cc): should we assume that additional fixes, not 
> >> necessarily in time for 14.0-RELEASE, will be required before 
> >> vfs.zfs.bclone_enabled can default to 1?
> >
> > I am not aware of any block cloning issues now.  All this thread about 
> > bclone_enabled actually started after I asked why it is still 
> > disabled. Thanks to Mark Millard for spotting this issue I could fix, 
> > but now we are back at the point of re-enabling it again.  Since the 
> > tunable does not even exist anywhere outside of FreeBSD base tree, I'd 
> > propose to give this code another try here too.  I see no point to 
> > have it disabled at least in main unless somebody needs time to run 
> > some specific tests first.

-- 
Tomoaki AOKI

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available)

2023-09-18 Thread Mark Johnston

On Mon, Sep 18, 2023 at 07:31:46AM +0200, Martin Matuska wrote:
> I vote for enabling block cloning on main :-)

Have you or anyone else run through the test suite with block cloning
enabled?

> On 16. 9. 2023 19:14, Alexander Motin wrote:
> > On 16.09.2023 01:25, Graham Perrin wrote:
> > > On 16/09/2023 01:28, Glen Barber wrote:
> > > > o A fix for the ZFS block_cloning feature has been implemented.
> > > 
> > > Thanks
> > > 
> > > I see 
> > > ,
> > > with 
> > > 
> > > in stable/14.
> > > 
> > > As vfs.zfs.bclone_enabled is still 0 (at least, with 15.0-CURRENT
> > > n265350-72d97e1dd9cc): should we assume that additional fixes, not
> > > necessarily in time for 14.0-RELEASE, will be required before
> > > vfs.zfs.bclone_enabled can default to 1?
> > 
> > I am not aware of any block cloning issues now.  All this thread about
> > bclone_enabled actually started after I asked why it is still disabled.
> > Thanks to Mark Millard for spotting this issue I could fix, but now we
> > are back at the point of re-enabling it again.  Since the tunable does
> > not even exist anywhere outside of FreeBSD base tree, I'd propose to
> > give this code another try here too.  I see no point to have it disabled
> > at least in main unless somebody needs time to run some specific tests
> > first.
> > 
>

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available) [block_cloning and zilsaxattr missing from loader's features_for_read]

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available) [block_cloning and zilsaxattr missing from loader's features_for_read]

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available) [block_cloning and zilsaxattr missing from loader's features_for_read]

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available)

Re: [Intel AlderLake] Read files to FAT32 or UFS partition cause data corrupt due to P-Core-Core

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available)

Re: [Intel AlderLake] Read files to FAT32 or UFS partition cause data corrupt due to P-Core-Core

Re: [Intel AlderLake] Read files to FAT32 or UFS partition cause data corrupt due to P-Core-Core

Re: [Intel AlderLake] Read files to FAT32 or UFS partition cause data corrupt due to P-Core-Core

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available)

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available)

Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available)

12 matches

Site Navigation

Mail list logo

Footer information