[developer] [openzfs/openzfs] zfs holds: In scripted mode, do not pad columns with spaces (#488)

2017-10-31 Thread Allan Jude
'zfs holds -H' does not properly output content in scripted mode. It uses a tab instead of two spaces, but it still pads column widths with spaces when it should not. Before: $ /sbin/zfs holds -Hp zstore/svn/base@r325191 zstore/svn/base@r325191 testing 1509498917 zstore/

[developer] Re: [openzfs/openzfs] zfs holds: In scripted mode, do not pad columns with spaces (#488)

2017-11-01 Thread Allan Jude
@allanjude pushed 1 commit. 168a284 Address feedback from Matt Ahrens -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/openzfs/openzfs/pull/488/files/4ffe832d4cdd55fd3d045472c88adb96b4edd8c8..168a2840808a557a5a59dd7209b44924eb65b9c9 --

[developer] Re: [openzfs/openzfs] zfs holds: In scripted mode, do not pad columns with spaces (#488)

2017-11-01 Thread Allan Jude
@allanjude pushed 1 commit. df0c780 print_holds(): remove sep and sepnum entirely -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/openzfs/openzfs/pull/488/files/3c7b9b717a8b9b074045fbd7b972f32438b8d370..df0c7802b85821f6509b2827ef63d1e0

[developer] Re: [openzfs/openzfs] 8835 Speculative prefetch in ZFS not working for misaligned reads (#497)

2017-11-20 Thread Allan Jude
allanjude approved this pull request. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/497#pullrequestreview-77975929 -- openzfs-developer Arc

[developer] Re: [openzfs/openzfs] zfs holds: In scripted mode, do not pad columns with spaces (#488)

2018-01-10 Thread Allan Jude
That does not seem to have gone well at all... -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/488#issuecomment-356839345 -- openzfs-developer Ar

[developer] Re: [openzfs/openzfs] zfs holds: In scripted mode, do not pad columns with spaces (#488)

2018-01-10 Thread Allan Jude
Ha. My first rebase, only took like 6 tries. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/488#issuecomment-356841770 -- openzfs-developer Arch

[developer] Re: [openzfs/openzfs] arc_loan_compressed_buf() can increment arc_loaned_bytes by the wrong value (#558)

2018-02-19 Thread Allan Jude
Sorry, missing step to the reproduction case: compressed arc must be disabled. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/558#issuecomment-366872279 ---

[developer] [openzfs/openzfs] arc_loan_compressed_buf() can increment arc_loaned_bytes by the wrong value (#558)

2018-02-19 Thread Allan Jude
arc_loan_compressed_buf() increments arc_loaned_bytes by psize unconditionally. In the case of zfs_compressed_arc_enabled=0, when the buf is returned via arc_return_buf(), if ARC_BUF_COMPRESSED(buf) is false, then arc_loaned_bytes is decremented by lsize, not psize. Switch to using arc_buf_size

[developer] Re: [openzfs/openzfs] Reads from indirect mirrors only find one copy (#561)

2018-02-21 Thread Allan Jude
allanjude approved this pull request. > +out: + /* +* Note, indirect vdevs don't handle a few important cases for split +* blocks when one side of a mirror is silently corrupt: +* +* When we read bad data and then retry the other side of the mirror, +

[developer] Re: [openzfs/openzfs] arc_loan_compressed_buf() can increment arc_loaned_bytes by the wrong value (#558)

2018-02-23 Thread Allan Jude
@allanjude pushed 1 commit. 73ea8bd fixup! arc_loan_compressed_buf() can increment arc_loaned_bytes by the wrong value -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/openzfs/openzfs/pull/558/files/ca78e6e459f4d99efb0fefbbc27b6087c238

[developer] Re: [openzfs/openzfs] 9188 increase size of dbuf cache to reduce indirect block decompression (#564)

2018-02-23 Thread Allan Jude
allanjude approved this pull request. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/564#pullrequestreview-99086879 -- openzfs-developer Arc

[developer] Re: [openzfs/openzfs] 9194 mechanism to override ashift at pool creation time (#570)

2018-02-23 Thread Allan Jude
allanjude approved this pull request. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/570#pullrequestreview-9909 -- openzfs-developer Arc

[developer] Re: [openzfs/openzfs] 9194 mechanism to override ashift at pool creation time (#570)

2018-02-23 Thread Allan Jude
Would be nice to have this. On FreeBSD we currently use the global tunable zfs.min_auto_ashift to accomplish this. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/570#issuecomment-3681

[developer] Re: [openzfs/openzfs] arc_loan_compressed_buf() can increment arc_loaned_bytes by the wrong value (#558)

2018-02-27 Thread Allan Jude
Looks like the Jenkins test failed before it even started the build, can we get a rerun on this? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/558#issuecomment-369118860

[developer] Re: [openzfs/openzfs] 9286 want refreservation=auto (#592)

2018-03-18 Thread Allan Jude
It is a bit unclear what 'dense' actually means. If I understand correctly, it sets the reservation to what it would have been by default if the volume had not been created with a lower-than-default reservation? Basically sets it to volsize+overhead? -- You are receiving this because you are s

[developer] [openzfs/openzfs] Refuse compressed replication streams with unknown compression types (#630)

2018-04-22 Thread Allan Jude
Return ENOTSUP if the data is compressed with an unknown/unsupported compression scheme You can view, comment on, or merge this pull request online at: https://github.com/openzfs/openzfs/pull/630 -- Commit Summary -- * Refuse compressed replication streams with unknown compression types --

[developer] [openzfs/openzfs] Reserve DMU_BACKUP_FEATURE for ZSTD (#652)

2018-06-12 Thread Allan Jude
Reserve #25 for the ZSTD compression feature from FreeBSD. Also note that #24 is reserved for 'raw send', part of the zfs encryption patch from ZoL. You can view, comment on, or merge this pull request online at: https://github.com/openzfs/openzfs/pull/652 -- Commit Summary -- * Reserve DMU

[developer] Re: [openzfs/openzfs] 9647 Introduce ZFS Read/Write kstats (#664)

2018-07-14 Thread Allan Jude
allanjude commented on this pull request. > @@ -653,43 +654,28 @@ zfs_read(vnode_t *vp, uio_t *uio, int ioflag, cred_t > *cr, caller_context_t *ct) static int zfs_write(vnode_t *vp, uio_t *uio, int ioflag, cred_t *cr, caller_context_t *ct) { - znode_t *zp = VTOZ(vp); Why are

[developer] Re: [openzfs/openzfs] Reserve DMU_BACKUP_FEATURE for ZSTD (#652)

2018-07-23 Thread Allan Jude
https://www.illumos.org/issues/9672 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/652#issuecomment-407278708 -- openzfs: openzfs-developer Perm

[developer] [openzfs/openzfs] fix typo in vdev_impl.h (#684)

2018-08-06 Thread Allan Jude
You can view, comment on, or merge this pull request online at: https://github.com/openzfs/openzfs/pull/684 -- Commit Summary -- * fix typo in vdev_impl.h -- File Changes -- M usr/src/uts/common/fs/zfs/sys/vdev_impl.h (2) -- Patch Links -- https://github.com/openzfs/openzfs/pull/684

[developer] Re: [openzfs/openzfs] 8589 zfs send | recv crashes receiving system (#685)

2018-09-12 Thread Allan Jude
allanjude commented on this pull request. > @@ -392,6 +392,22 @@ static int dump_freeobjects(dmu_sendarg_t *dsp, uint64_t firstobj, uint64_t numobjs) { struct drr_freeobjects *drrfo = &(dsp->dsa_drr->drr_u.drr_freeobjects); + uint64_t maxobj = DNODES_PER_BLOCK * + (DMU_

[developer] Re: [openzfs/openzfs] 9861 Harden ZFS receive against out-of-bounds property values (#630)

2018-10-01 Thread Allan Jude
> @allanjude can you open an illumos bug for this? https://www.illumos.org/issues/9861 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/630#issuecomment-426119431 --

[developer] Re: [openzfs/openzfs] 9689 zfs range lock code should not be zpl-specific (#680)

2018-10-01 Thread Allan Jude
allanjude approved this pull request. > @@ -92,124 +93,146 @@ * growth all other writers and readers must be excluded. This comment should be updated to reflect that 128kb is no longer the maximum block size > - */ - if (new->r_type == RL_APPEND) -

[developer] Re: [openzfs/openzfs] 9861 Harden ZFS receive against out-of-bounds property values (#630)

2018-10-01 Thread Allan Jude
allanjude commented on this pull request. > @@ -2644,6 +2644,17 @@ receive_read_record(struct receive_arg *ra) uint32_t size = P2ROUNDUP(drro->drr_bonuslen, 8); void *buf = kmem_zalloc(size, KM_SLEEP); dmu_object_info_t doi; + + if (d

[developer] Re: [openzfs/openzfs] 9861 Harden ZFS receive against out-of-bounds property values (#630)

2018-10-01 Thread Allan Jude
Update -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/630#issuecomment-426144587 -- openzfs: openzfs-developer Permalink: https://openzfs.topic

[developer] Re: [openzfs/openzfs] 9861 Harden ZFS receive against out-of-bounds property values (#630)

2018-10-02 Thread Allan Jude
allanjude commented on this pull request. > @@ -2669,6 +2678,18 @@ receive_read_record(struct receive_arg *ra) case DRR_WRITE: { struct drr_write *drrw = &ra->rrd->header.drr_u.drr_write; + if (drrw->drr_compressiontype >= ZIO_COMPRESS_FUNCTIONS) +

[developer] Re: [openzfs/openzfs] 9861 Harden ZFS receive against out-of-bounds property values (#630)

2018-10-02 Thread Allan Jude
Sorry for the build error, it seemed to have crept in when I refactored this -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/630#issuecomment-426450196 -

[developer] Re: [openzfs/openzfs] 9861 Harden ZFS receive against out-of-bounds property values (#630)

2018-10-08 Thread Allan Jude
@allanjude pushed 1 commit. ccc957a fixup typo -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/openzfs/openzfs/pull/630/files/1131831132e8efaaf1a0d455f9c22c3dc1007bb2..ccc957a58bc83f199baa1eb828bf966023f2f018 -

Re: [developer] Re: [zfs-discuss] OpenZFS Code of Conduct

2019-04-24 Thread Allan Jude
ct committee. We need to give them as as broad a mandate as possible to do the right thing, and trust in them to do that. -- Allan Jude -- openzfs: openzfs-developer Permalink: https://openzfs.topicbox.com/groups/developer/T5dc11696c7b4bddb-M4aba381bdcc5e4a722

[developer] Call For Testing of FreeBSD + ZoL

2019-04-24 Thread Allan Jude
OpenZFS Developers, The FreeBSD project is attempting to switch its upstream source of ZFS from the illumos repo to ZFS-on-Linux. Since there is not really a shared point in history to try to make this switch commit-by-commit, it was decided to attempt to just port ZFS-on-Linux over to FreeBSD

Re: [developer] Who wants to implement compression on by default?

2019-05-18 Thread Allan Jude
pression, since if compress != off, we check for the 'all zeros' case and write a hole, rather than actually allocating space. If you write something else compressible, you should be able to use zdb to see if it ended up using lzjb or lz4. -- Allan Jude

Re: [developer] Who wants to implement compression on by default?

2019-05-19 Thread Allan Jude
On 5/18/19 2:02 PM, Sean Fagan wrote: On May 18, 2019, at 1:47 AM, Allan Jude wrote: Writing a file of all zeros doesn't test the compression, since if compress != off, we check for the 'all zeros' case and write a hole, rather than actually allocating space. That

Re: [developer] Re: Using many zpools

2019-07-04 Thread Allan Jude
int total, "TOTAL", used, cache } ' | sort -n | perl -a -p -e 'while (($j, $_) = each(@F)) { 1 while s/^(-?\d+)(\d{3})/$1,$2/; print $_, " "} print "\n"' | column -t And now which values are growing. This breaks down each UMA cache and how much slack it contains. -- Allan Jude signature.asc Description: OpenPGP digital signature This is a multi-part message in MIME format... =_1562279682-930255-1--

Re: [developer] Re: Using many zpools

2019-07-05 Thread Allan Jude
On 2019-07-05 04:21, nagy.att...@gmail.com wrote: > On Friday, July 05, 2019, at 12:34 AM, Allan Jude wrote: >> And now which values are growing. This breaks down each UMA cache and >> how much slack it contains. > I had one, but lost it with machine reboots (my fault). >

Re: [developer] Pathway to better DDT, and value-for-effort assessment of mitigations in the meantime.

2019-07-07 Thread Allan Jude
>>> >> > *openzfs <https://openzfs.topicbox.com/latest>* / openzfs-developer / > see discussions <https://openzfs.topicbox.com/groups/developer> + > participants <https://openzfs.topicbox.com/groups/developer/members> + > delivery options > <https://openzfs.topicbox.com/groups/developer/subscription> Permalink > <https://openzfs.topicbox.com/groups/developer/Td9c7189186fd24f2-Mf90f2c92931dd028277934cd> > -- Allan Jude signature.asc Description: OpenPGP digital signature This is a multi-part message in MIME format... =_1562513049-17796-1--

[developer] Dedup review requests

2020-04-28 Thread Allan Jude
make sure the naming of these features, commands, and properties will still make sense if their implementations are replaced with different ones in an eventual enhancement (ddt log) or replacement (panzura dedup) of the dedup feature. Thank you. -- Allan Jude -- VP Engineering ** Klara Inc. | al

Re: [developer] Manual dedup, aka --reflink support.

2020-06-15 Thread Allan Jude
Permalink: > > https://openzfs.topicbox.com/groups/developer/Te62797341aee0806-M77d44ed51187031460b91160 > Delivery options: > https://openzfs.topicbox.com/groups/developer/subscription > > *openzfs <https://openzfs.topicbox.com/latest>* / openzfs-developer / > see discussions <https://openzfs.topicbox.com/groups/developer> + > participants <https://openzfs.topicbox.com/groups/developer/members> + > delivery options > <https://openzfs.topicbox.com/groups/developer/subscription> Permalink > <https://openzfs.topicbox.com/groups/developer/Te62797341aee0806-Ma15bb4984382ab368f7e13f0> > -- Allan Jude signature.asc Description: OpenPGP digital signature This is a multi-part message in MIME format... =_1592246248-330386-1--

[developer] ZSTD Project Weekly Status Update

2020-06-22 Thread Allan Jude
d the 'zfs send' when using ZSTD compression. I would also like to thank the various members of the ZFS-on-Linux community to have helped with the integration and testing on Linux, a platform I am not very familiar with. -- Allan Jude signature.asc Description: OpenPGP digital sign

[developer] Re: ZSTD Project Weekly Status Update

2020-06-29 Thread Allan Jude
7;s compression setting is set to lz4 (or any other type that doesn't use a level). This project was sponsored by the FreeBSD Foundation. -- Allan Jude signature.asc Description: OpenPGP digital signature This is a multi-part message in MIME format... =_1593480745-111408-1--

[developer] Re: ZSTD Project Weekly Status Update

2020-07-06 Thread Allan Jude
this. I have thought about only sending ZSTD compressed blocks if the user specifies the -Z (--zstd) flag, but this can lead to confusion where using -c without -Z would have to either error out, or send the ZSTD compressed blocks uncompressed. This project is sponsored by the FreeBSD Foundation.

[developer] Re: ZSTD Project Weekly Status Update

2020-07-14 Thread Allan Jude
feature flag to 'active' as soon as it is set, rather than only once a block is born. As well as fixing up compatibility around zfs send/recv with the embedded block points flag. This project is sponsored by the FreeBSD Foundation. On 2020-07-06 20:07, Allan Jude wrote: > In my cont

[developer] Re: ZSTD Project Weekly Status Update

2020-07-20 Thread Allan Jude
older versions of ZFS. I will be discussing this on the OpenZFS Leadership call tomorrow, and am open to suggestions on how to best handle this. On 2020-07-14 22:26, Allan Jude wrote: > In my continuing effort to complete the integration of ZSTD into > OpenZFS, here is my fourth weekly status

[developer] Re: ZSTD Project Weekly Status Update

2020-07-29 Thread Allan Jude
#x27;zfs recv' that does not know about ZSTD (where the values are outside of the valid range for the enum). If anyone has any ideas here, I'd be very interested. On 2020-07-20 23:40, Allan Jude wrote: > This is the fifth weekly status report on the project to integrate ZSTD

[developer] Re: ZSTD Project Weekly Status Update

2020-08-05 Thread Allan Jude
ion. On 2020-07-29 21:10, Allan Jude wrote: > This is the sixth weekly status report on the project to integrate ZSTD > into OpenZFS. > > https://github.com/openzfs/zfs/pull/10631 - Improved the `zfs recv` > error handling when it receives an out-of-bounds property value. > Spec

Re: [developer] Re: ZSTD Project Weekly Status Update

2020-08-10 Thread Allan Jude
On 2020-08-06 16:09, Richard Laager wrote: > On 8/5/20 9:49 PM, Allan Jude wrote: >> I've also been discussing ideas with Brian about future-proofing, to >> handle the case where a newer version of ZSTD might compression the same >> input differently (better ratio), and h

[developer] Re: ZSTD Project Weekly Status Update

2020-08-10 Thread Allan Jude
ject is sponsored by the FreeBSD Foundation. On 2020-08-05 22:49, Allan Jude wrote: > This is the seventh weekly status report on the project to integrate > ZSTD into OpenZFS. > > The compatibility related changes I created last week were refined and > marged into the mainline bran

[developer] Re: ZSTD Project Weekly Status Update

2020-08-18 Thread Allan Jude
FreeBSD Foundation. On 2020-08-10 23:45, Allan Jude wrote: > This is the eighth weekly status report on the project to complete the > integration of ZSTD compression into OpenZFS. > > https://github.com/openzfs/zfs/pull/10692 - I created some new tests > around the L2ARC to facili

[developer] Re: ZSTD Project Weekly Status Update

2020-08-25 Thread Allan Jude
most consistent and repeatable results when benchmarking like this? On 2020-08-18 18:51, Allan Jude wrote: > This is the ninth weekly status report on my FreeBSD Foundation > sponsored project to complete the integration of ZSTD compression into > OpenZFS. > > https://github.com/openz

[developer] Re: ZSTD Project Weekly Status Update

2020-08-31 Thread Allan Jude
ing this work. On 2020-08-25 22:22, Allan Jude wrote: > This is the tenth weekly status report on my FreeBSD Foundation > sponsored project to complete the integration of ZSTD compression into > OpenZFS. > > Late last week the main pull request was merged, and ZSTD support is no

[developer] Re: ZSTD Project Weekly Status Update

2020-09-15 Thread Allan Jude
ng before I finish wearing these SSDs out. Thanks again to the FreeBSD Foundation for sponsoring this work. On 2020-08-31 23:21, Allan Jude wrote: > This is the eleventh weekly status report on my FreeBSD Foundation > sponsored project to complete the integration of ZSTD compression i

[developer] Re: ZSTD Project Weekly Status Update

2020-09-23 Thread Allan Jude
b.com/openzfs/zfs/pull/10975 I am working on some benchmarks of this change as well. Thank you. On 2020-09-15 22:11, Allan Jude wrote: > This is another weekly status report on my FreeBSD Foundation sponsored > project to complete the integration of ZSTD compression into OpenZFS. > >