svn commit: r352025 - head/sbin/mount_fusefs
Author: asomers Date: Sat Sep 7 21:49:01 2019 New Revision: 352025 URL: https://svnweb.freebsd.org/changeset/base/352025 Log: mount_fusefs: fix a segfault on memory allocation failure Reported by: Coverity Coverity CID: 1354188 MFC after:4 days Sponsored by: The FreeBSD Foundation Modified: head/sbin/mount_fusefs/mount_fusefs.c Modified: head/sbin/mount_fusefs/mount_fusefs.c == --- head/sbin/mount_fusefs/mount_fusefs.c Sat Sep 7 21:26:56 2019 (r352024) +++ head/sbin/mount_fusefs/mount_fusefs.c Sat Sep 7 21:49:01 2019 (r352025) @@ -212,6 +212,8 @@ main(int argc, char *argv[]) q++; mv->mv_len = q - p + 1; mv->mv_value = malloc(mv->mv_len); + if (mv->mv_value == NULL) + err(1, "malloc"); memcpy(mv->mv_value, p, mv->mv_len - 1); ((char *)mv->mv_value)[mv->mv_len - 1] = '\0'; break; ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r352021 - head/tests/sys/fs/fusefs
Author: asomers
Date: Sat Sep 7 19:25:45 2019
New Revision: 352021
URL: https://svnweb.freebsd.org/changeset/base/352021
Log:
fusefs: suppress some Coverity resource leak CIDs in the tests
The fusefs tests deliberately leak file descriptors. To do otherwise would
add extra complications to the tests' mock FUSE server. This annotation
should hopefully convince Coverity to shut up about the leaks.
Reviewed by: uqs
MFC after:4 days
Sponsored by: The FreeBSD Foundation
Modified:
head/tests/sys/fs/fusefs/utils.hh
Modified: head/tests/sys/fs/fusefs/utils.hh
==
--- head/tests/sys/fs/fusefs/utils.hh Sat Sep 7 17:22:25 2019
(r352020)
+++ head/tests/sys/fs/fusefs/utils.hh Sat Sep 7 19:25:45 2019
(r352021)
@@ -231,6 +231,7 @@ class FuseTest : public ::testing::Test {
* to document the leakage, and provide a single point of suppression
* for static analyzers.
*/
+ /* coverity[+close: arg-0] */
static void leak(int fd __unused) {}
/*
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r352018 - head/sbin/camcontrol
Author: mav
Date: Sat Sep 7 16:52:40 2019
New Revision: 352018
URL: https://svnweb.freebsd.org/changeset/base/352018
Log:
Unify cam_send_ccb() error reporting.
Error there mean that command was not even executed, and all information
we have about it is errno, and cam_error_print() call is not very useful.
Plus it is most likely a programmatic error, that shoud not happen.
MFC after:1 week
Sponsored by: iXsystems, Inc.
Modified:
head/sbin/camcontrol/attrib.c
head/sbin/camcontrol/camcontrol.c
head/sbin/camcontrol/persist.c
Modified: head/sbin/camcontrol/attrib.c
==
--- head/sbin/camcontrol/attrib.c Sat Sep 7 16:10:57 2019
(r352017)
+++ head/sbin/camcontrol/attrib.c Sat Sep 7 16:52:40 2019
(r352018)
@@ -344,12 +344,6 @@ scsiattrib(struct cam_device *device, int argc, char *
if (cam_send_ccb(device, ccb) < 0) {
warn("error sending %s ATTRIBUTE", (read_attr != 0) ?
"READ" : "WRITE");
-
- if (verbosemode != 0) {
- cam_error_print(device, ccb, CAM_ESF_ALL,
- CAM_EPF_ALL, stderr);
- }
-
error = 1;
goto bailout;
}
Modified: head/sbin/camcontrol/camcontrol.c
==
--- head/sbin/camcontrol/camcontrol.c Sat Sep 7 16:10:57 2019
(r352017)
+++ head/sbin/camcontrol/camcontrol.c Sat Sep 7 16:52:40 2019
(r352018)
@@ -420,7 +420,7 @@ getdevlist(struct cam_device *device)
ccb->cgdl.status = CAM_GDEVLIST_MORE_DEVS;
while (ccb->cgdl.status == CAM_GDEVLIST_MORE_DEVS) {
if (cam_send_ccb(device, ccb) < 0) {
- perror("error getting device list");
+ warn("error getting device list");
cam_freeccb(ccb);
return (1);
}
@@ -769,7 +769,7 @@ print_dev_mmcsd(struct device_match_result *dev_result
advi->buf = (uint8_t *)&mmc_ident_data;
if (cam_send_ccb(dev, ccb) < 0) {
- warn("error sending CAMIOCOMMAND ioctl");
+ warn("error sending XPT_DEV_ADVINFO CCB");
cam_freeccb(ccb);
cam_close_device(dev);
return (1);
@@ -811,7 +811,7 @@ nvme_get_cdata(struct cam_device *dev, struct nvme_con
advi->buf = (uint8_t *)cdata;
if (cam_send_ccb(dev, ccb) < 0) {
- warn("error sending CAMIOCOMMAND ioctl");
+ warn("error sending XPT_DEV_ADVINFO CCB");
cam_freeccb(ccb);
cam_close_device(dev);
return(1);
@@ -876,13 +876,7 @@ testunitready(struct cam_device *device, int task_attr
if (cam_send_ccb(device, ccb) < 0) {
if (quiet == 0)
- perror("error sending test unit ready");
-
- if (arglist & CAM_ARG_VERBOSE) {
- cam_error_print(device, ccb, CAM_ESF_ALL,
- CAM_EPF_ALL, stderr);
- }
-
+ warn("error sending TEST UNIT READY command");
cam_freeccb(ccb);
return (1);
}
@@ -945,13 +939,7 @@ scsistart(struct cam_device *device, int startstop, in
ccb->ccb_h.flags |= CAM_PASS_ERR_RECOVER;
if (cam_send_ccb(device, ccb) < 0) {
- perror("error sending start unit");
-
- if (arglist & CAM_ARG_VERBOSE) {
- cam_error_print(device, ccb, CAM_ESF_ALL,
- CAM_EPF_ALL, stderr);
- }
-
+ warn("error sending START STOP UNIT command");
cam_freeccb(ccb);
return (1);
}
@@ -1114,13 +1102,7 @@ scsiinquiry(struct cam_device *device, int task_attr,
ccb->ccb_h.flags |= CAM_PASS_ERR_RECOVER;
if (cam_send_ccb(device, ccb) < 0) {
- perror("error sending SCSI inquiry");
-
- if (arglist & CAM_ARG_VERBOSE) {
- cam_error_print(device, ccb, CAM_ESF_ALL,
- CAM_EPF_ALL, stderr);
- }
-
+ warn("error sending INQUIRY command");
cam_freeccb(ccb);
return (1);
}
@@ -1196,13 +1178,7 @@ scsiserial(struct cam_device *device, int task_attr, i
ccb->ccb_h.flags |= CAM_PASS_ERR_RECOVER;
if (cam_send_ccb(device, ccb) < 0) {
- warn("error getting serial number");
-
- if (arglist & CAM_ARG_VERBOSE) {
- cam_error_print(device, ccb, CAM_ESF_ALL,
- CAM_EPF_ALL, stderr);
- }
-
+ warn("error sending INQUIRY command")
svn commit: r352017 - head/sys/kern
Author: kib
Date: Sat Sep 7 16:10:57 2019
New Revision: 352017
URL: https://svnweb.freebsd.org/changeset/base/352017
Log:
In do_execve(), use shared text vnode lock consistently.
Reviewed by: markj
MFC after:1 week
Differential revision:https://reviews.freebsd.org/D21560
Modified:
head/sys/kern/kern_exec.c
Modified: head/sys/kern/kern_exec.c
==
--- head/sys/kern/kern_exec.c Sat Sep 7 16:05:17 2019(r352016)
+++ head/sys/kern/kern_exec.c Sat Sep 7 16:10:57 2019(r352017)
@@ -526,7 +526,7 @@ interpret:
euip = uifind(attr.va_uid);
change_euid(imgp->newcred, euip);
}
- vn_lock(imgp->vp, LK_EXCLUSIVE | LK_RETRY);
+ vn_lock(imgp->vp, LK_SHARED | LK_RETRY);
if (attr.va_mode & S_ISGID)
change_egid(imgp->newcred, attr.va_gid);
/*
@@ -555,7 +555,7 @@ interpret:
oldcred->cr_svgid != oldcred->cr_gid) {
VOP_UNLOCK(imgp->vp, 0);
imgp->newcred = crdup(oldcred);
- vn_lock(imgp->vp, LK_EXCLUSIVE | LK_RETRY);
+ vn_lock(imgp->vp, LK_SHARED | LK_RETRY);
change_svuid(imgp->newcred, imgp->newcred->cr_uid);
change_svgid(imgp->newcred, imgp->newcred->cr_gid);
}
@@ -572,7 +572,7 @@ interpret:
if (vn_fullpath(td, imgp->vp, &imgp->execpath,
&imgp->freepath) != 0)
imgp->execpath = args->fname;
- vn_lock(imgp->vp, LK_EXCLUSIVE | LK_RETRY);
+ vn_lock(imgp->vp, LK_SHARED | LK_RETRY);
}
/*
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r352016 - head/sys/kern
Author: kib Date: Sat Sep 7 16:05:17 2019 New Revision: 352016 URL: https://svnweb.freebsd.org/changeset/base/352016 Log: In do_execve(), clear imgp->textset when restarting for interpreter. Otherwise, we might left the boolean set, which would affect cleanup after an error on interpreter activation. Reviewed by: markj MFC after:1 week Differential revision:https://reviews.freebsd.org/D21560 Modified: head/sys/kern/kern_exec.c Modified: head/sys/kern/kern_exec.c == --- head/sys/kern/kern_exec.c Sat Sep 7 16:03:26 2019(r352015) +++ head/sys/kern/kern_exec.c Sat Sep 7 16:05:17 2019(r352016) @@ -616,7 +616,9 @@ interpret: * The vnode lock is held over this entire period * so nothing should illegitimately be blocked. */ - VOP_UNSET_TEXT_CHECKED(imgp->vp); + MPASS(imgp->textset); + VOP_UNSET_TEXT_CHECKED(newtextvp); + imgp->textset = false; /* free name buffer and old vnode */ if (args->fname != NULL) NDFREE(&nd, NDF_ONLY_PNBUF); ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r352015 - head/sys/kern
Author: kib Date: Sat Sep 7 16:03:26 2019 New Revision: 352015 URL: https://svnweb.freebsd.org/changeset/base/352015 Log: When loading ELF interpreter, initialize whole nested image_params with zero. Otherwise we could mishandle imgp->textset. Reviewed by: markj MFC after:1 week Differential revision:https://reviews.freebsd.org/D21560 Modified: head/sys/kern/imgact_elf.c Modified: head/sys/kern/imgact_elf.c == --- head/sys/kern/imgact_elf.c Sat Sep 7 16:01:45 2019(r352014) +++ head/sys/kern/imgact_elf.c Sat Sep 7 16:03:26 2019(r352015) @@ -733,7 +733,7 @@ __elfN(load_file)(struct proc *p, const char *file, u_ return (ECAPMODE); #endif - tempdata = malloc(sizeof(*tempdata), M_TEMP, M_WAITOK); + tempdata = malloc(sizeof(*tempdata), M_TEMP, M_WAITOK | M_ZERO); nd = &tempdata->nd; attr = &tempdata->attr; imgp = &tempdata->image_params; @@ -743,10 +743,6 @@ __elfN(load_file)(struct proc *p, const char *file, u_ */ imgp->proc = p; imgp->attr = attr; - imgp->firstpage = NULL; - imgp->image_header = NULL; - imgp->object = NULL; - imgp->execlabel = NULL; NDINIT(nd, LOOKUP, ISOPEN | FOLLOW | LOCKSHARED | LOCKLEAF, UIO_SYSSPACE, file, curthread); ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r352014 - head/sys/vm
Author: kib
Date: Sat Sep 7 16:01:45 2019
New Revision: 352014
URL: https://svnweb.freebsd.org/changeset/base/352014
Log:
vm_object_deallocate(): Remove no longer needed code.
We track text mappings explicitly, there is no removal of the text
refs on the object deallocate any more, so tmpfs objects should not be
treated specially. Doing so causes excess deref.
Reported and tested by: gallatin
Reviewed by: markj
MFC after:1 week
Differential revision:https://reviews.freebsd.org/D21560
Modified:
head/sys/vm/vm_object.c
Modified: head/sys/vm/vm_object.c
==
--- head/sys/vm/vm_object.c Sat Sep 7 15:58:48 2019(r352013)
+++ head/sys/vm/vm_object.c Sat Sep 7 16:01:45 2019(r352014)
@@ -508,7 +508,6 @@ void
vm_object_deallocate(vm_object_t object)
{
vm_object_t temp;
- struct vnode *vp;
while (object != NULL) {
VM_OBJECT_WLOCK(object);
@@ -531,25 +530,6 @@ vm_object_deallocate(vm_object_t object)
VM_OBJECT_WUNLOCK(object);
return;
} else if (object->ref_count == 1) {
- if (object->type == OBJT_SWAP &&
- (object->flags & OBJ_TMPFS) != 0) {
- vp = object->un_pager.swp.swp_tmpfs;
- vhold(vp);
- VM_OBJECT_WUNLOCK(object);
- vn_lock(vp, LK_SHARED | LK_RETRY);
- VM_OBJECT_WLOCK(object);
- if (object->type == OBJT_DEAD ||
- object->ref_count != 1) {
- VM_OBJECT_WUNLOCK(object);
- VOP_UNLOCK(vp, 0);
- vdrop(vp);
- return;
- }
- if ((object->flags & OBJ_TMPFS) != 0)
- VOP_UNSET_TEXT(vp);
- VOP_UNLOCK(vp, 0);
- vdrop(vp);
- }
if (object->shadow_count == 0 &&
object->handle == NULL &&
(object->type == OBJT_DEFAULT ||
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r352013 - head/sys/vm
Author: kib
Date: Sat Sep 7 15:58:48 2019
New Revision: 352013
URL: https://svnweb.freebsd.org/changeset/base/352013
Log:
vm_object_coalesce(): avoid extending any nosplit objects, not only
that which back tmpfs nodes.
Reviewed by: markj
MFC after:1 week
Differential revision:https://reviews.freebsd.org/D21560
Modified:
head/sys/vm/vm_object.c
Modified: head/sys/vm/vm_object.c
==
--- head/sys/vm/vm_object.c Sat Sep 7 15:57:23 2019(r352012)
+++ head/sys/vm/vm_object.c Sat Sep 7 15:58:48 2019(r352013)
@@ -2062,7 +2062,7 @@ vm_object_coalesce(vm_object_t prev_object, vm_ooffset
VM_OBJECT_WLOCK(prev_object);
if ((prev_object->type != OBJT_DEFAULT &&
prev_object->type != OBJT_SWAP) ||
- (prev_object->flags & OBJ_TMPFS_NODE) != 0) {
+ (prev_object->flags & OBJ_NOSPLIT) != 0) {
VM_OBJECT_WUNLOCK(prev_object);
return (FALSE);
}
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r352012 - head/sys/ufs/ufs
Author: kib
Date: Sat Sep 7 15:57:23 2019
New Revision: 352012
URL: https://svnweb.freebsd.org/changeset/base/352012
Log:
Properly check for writers when fetching quotas for writeable vnodes
in UFS quotaon().
Reviewed by: markj
MFC after:1 week
Differential revision:https://reviews.freebsd.org/D21560
Modified:
head/sys/ufs/ufs/ufs_quota.c
Modified: head/sys/ufs/ufs/ufs_quota.c
==
--- head/sys/ufs/ufs/ufs_quota.cSat Sep 7 15:56:00 2019
(r352011)
+++ head/sys/ufs/ufs/ufs_quota.cSat Sep 7 15:57:23 2019
(r352012)
@@ -617,7 +617,7 @@ again:
MNT_VNODE_FOREACH_ALL_ABORT(mp, mvp);
goto again;
}
- if (vp->v_type == VNON || vp->v_writecount == 0) {
+ if (vp->v_type == VNON || vp->v_writecount <= 0) {
VOP_UNLOCK(vp, 0);
vrele(vp);
continue;
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r352011 - in head: sbin/camcontrol sys/cam/scsi
Author: mav
Date: Sat Sep 7 15:56:00 2019
New Revision: 352011
URL: https://svnweb.freebsd.org/changeset/base/352011
Log:
Supply SAT layer with valid transfer sizes.
This is a rework of r344701, that noticed that number of bytes passes to
8 bit sector count field gets truncated. First decision was to not pass
anything, since ATA specs define the field as N/A. But it appeared to be a
problem for some SAT devices, that require information about data transfer
to operate properly. Some additional investigation shown that it is quite
a common practice to set unused fields of ATA commands (fortunately ATA
specs formally allow it) to supply the information to SAT layer. I have
found SAS-SATA interposer that does not allow pass-through without it.
As side effect, reduce code duplication by removing ata_do_28bit_cmd()
function, replacing it with more universal ata_do_cmd().
MFC after:1 week
Sponsored by: iXsystems, Inc.
Modified:
head/sbin/camcontrol/camcontrol.c
head/sbin/camcontrol/fwdownload.c
head/sys/cam/scsi/scsi_all.c
Modified: head/sbin/camcontrol/camcontrol.c
==
--- head/sbin/camcontrol/camcontrol.c Sat Sep 7 12:31:36 2019
(r352010)
+++ head/sbin/camcontrol/camcontrol.c Sat Sep 7 15:56:00 2019
(r352011)
@@ -1878,13 +1878,11 @@ ata_cam_send(struct cam_device *device, union ccb *ccb
static int
ata_do_pass_16(struct cam_device *device, union ccb *ccb, int retries,
u_int32_t flags, u_int8_t protocol, u_int8_t ata_flags,
- u_int8_t tag_action, u_int8_t command, u_int8_t features,
- u_int64_t lba, u_int8_t sector_count, u_int8_t *data_ptr,
+ u_int8_t tag_action, u_int8_t command, u_int16_t features,
+ u_int64_t lba, u_int16_t sector_count, u_int8_t *data_ptr,
u_int16_t dxfer_len, int timeout, int quiet)
{
if (data_ptr != NULL) {
- ata_flags |= AP_FLAG_BYT_BLOK_BYTES |
- AP_FLAG_TLEN_SECT_CNT;
if (flags & CAM_DIR_OUT)
ata_flags |= AP_FLAG_TDIR_TO_DEV;
else
@@ -1935,44 +1933,10 @@ ata_try_pass_16(struct cam_device *device)
}
static int
-ata_do_28bit_cmd(struct cam_device *device, union ccb *ccb, int retries,
-u_int32_t flags, u_int8_t protocol, u_int8_t tag_action,
-u_int8_t command, u_int8_t features, u_int32_t lba,
-u_int8_t sector_count, u_int8_t *data_ptr, u_int16_t dxfer_len,
-int timeout, int quiet)
-{
-
-
- switch (ata_try_pass_16(device)) {
- case -1:
- return (1);
- case 1:
- /* Try using SCSI Passthrough */
- return ata_do_pass_16(device, ccb, retries, flags, protocol,
- 0, tag_action, command, features, lba,
- sector_count, data_ptr, dxfer_len,
- timeout, quiet);
- }
-
- CCB_CLEAR_ALL_EXCEPT_HDR(&ccb->ataio);
- cam_fill_ataio(&ccb->ataio,
- retries,
- NULL,
- flags,
- tag_action,
- data_ptr,
- dxfer_len,
- timeout);
-
- ata_28bit_cmd(&ccb->ataio, command, features, lba, sector_count);
- return ata_cam_send(device, ccb, quiet);
-}
-
-static int
ata_do_cmd(struct cam_device *device, union ccb *ccb, int retries,
u_int32_t flags, u_int8_t protocol, u_int8_t ata_flags,
- u_int8_t tag_action, u_int8_t command, u_int8_t features,
- u_int64_t lba, u_int8_t sector_count, u_int8_t *data_ptr,
+ u_int8_t tag_action, u_int8_t command, u_int16_t features,
+ u_int64_t lba, u_int16_t sector_count, u_int8_t *data_ptr,
u_int16_t dxfer_len, int timeout, int force48bit)
{
int retval;
@@ -2219,14 +2183,15 @@ atahpa_password(struct cam_device *device, int retry_c
retry_count,
/*flags*/CAM_DIR_OUT,
/*protocol*/protocol,
- /*ata_flags*/AP_FLAG_CHK_COND,
+ /*ata_flags*/AP_FLAG_BYT_BLOK_BLOCKS |
+ AP_FLAG_TLEN_SECT_CNT | AP_FLAG_CHK_COND,
/*tag_action*/MSG_SIMPLE_Q_TAG,
/*command*/cmd,
/*features*/ATA_HPA_FEAT_SET_PWD,
/*lba*/0,
- /*sector_count*/0,
+ /*sector_count*/sizeof(*pwd) / 512,
/*data_ptr*/(u_int8_t*)pwd,
- /*dxfer_len*/sizeof(struct ata_set_max_pwd),
+ /*dxfer_len*/sizeof(*pwd),
timeout ? timeout : 1000
Re: svn commit: r351673 - in head: lib/libmemstat share/man/man9 sys/cddl/compat/opensolaris/kern sys/kern sys/vm
On Sat, Sep 07, 2019 at 10:50:34AM -0400, Mark Johnston wrote:
> On Wed, Sep 04, 2019 at 05:45:24PM +0300, Slawa Olhovchenkov wrote:
> > On Tue, Sep 03, 2019 at 06:01:06PM -0400, Mark Johnston wrote:
> > > > Mostly problem I am see at this
> > > > work -- very slowly vm_page_free(). May be currenly this is more
> > > > speedy...
> > >
> > > How did you determine this?
> >
> > This is you guess:
>
> So was the guess correct?
I am just trust to you.
How to check this guess?
> If so, IMO the real solution is to avoid kmem_*
> for data buffers and use ABD instead.
What problem resolve this?
ABD any way is slowly vs kmem_*.
> > ==
> > > while ((slab = SLIST_FIRST(&freeslabs)) != NULL) {
> > > SLIST_REMOVE(&freeslabs, slab, uma_slab, us_hlink);
> > > keg_free_slab(keg, slab, keg->uk_ipers);
> > > }
> > > 2019 Feb 2 19:49:54.800524364 zio_data_buf_1048576 1032605
> > > cache_reclaim limit 100 dom 0 nitems 1672 imin 298
> > > 2019 Feb 2 19:49:54.800524364 zio_data_buf_1048576 1033736
> > > cache_reclaim recla 149 dom 0 nitems 1672 imin 298
> > > 2019 Feb 2 19:49:54.802524468 zio_data_buf_1048576 3119710
> > > cache_reclaim limit 100 dom 1 nitems1 imin0
> > > 2019 Feb 2 19:49:54.802524468 zio_data_buf_1048576 3127550
> > > keg_drain2
> > > 2019 Feb 2 19:49:54.803524487 zio_data_buf_1048576 219
> > > keg_drain3
> > > 2019 Feb 2 19:49:54.838524634 zio_data_buf_1048576 39553705
> > > keg_drain4
> > > 2019 Feb 2 19:49:54.838524634 zio_data_buf_1048576 39565323
> > > zone_reclaim:return
> > >
> > > 35109.486 ms for last loop, 149 items to freed.
> >
> > 35ms to free 149MB (38144 4KB pages), so roughly 1us per page. That
> > does seem like a lot, but freeing a page (vm_page_free(m)) is much
> > more expensive than freeing an item to UMA (i.e., uma_zfree()).
> > Most of that time will be spent in _kmem_unback().
> > ==
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r351673 - in head: lib/libmemstat share/man/man9 sys/cddl/compat/opensolaris/kern sys/kern sys/vm
On Wed, Sep 04, 2019 at 05:45:24PM +0300, Slawa Olhovchenkov wrote:
> On Tue, Sep 03, 2019 at 06:01:06PM -0400, Mark Johnston wrote:
> > > Mostly problem I am see at this
> > > work -- very slowly vm_page_free(). May be currenly this is more
> > > speedy...
> >
> > How did you determine this?
>
> This is you guess:
So was the guess correct? If so, IMO the real solution is to avoid kmem_*
for data buffers and use ABD instead.
> ==
> > while ((slab = SLIST_FIRST(&freeslabs)) != NULL) {
> > SLIST_REMOVE(&freeslabs, slab, uma_slab, us_hlink);
> > keg_free_slab(keg, slab, keg->uk_ipers);
> > }
> > 2019 Feb 2 19:49:54.800524364 zio_data_buf_1048576 1032605
> > cache_reclaim limit 100 dom 0 nitems 1672 imin 298
> > 2019 Feb 2 19:49:54.800524364 zio_data_buf_1048576 1033736
> > cache_reclaim recla 149 dom 0 nitems 1672 imin 298
> > 2019 Feb 2 19:49:54.802524468 zio_data_buf_1048576 3119710
> > cache_reclaim limit 100 dom 1 nitems1 imin0
> > 2019 Feb 2 19:49:54.802524468 zio_data_buf_1048576 3127550
> > keg_drain2
> > 2019 Feb 2 19:49:54.803524487 zio_data_buf_1048576 219
> > keg_drain3
> > 2019 Feb 2 19:49:54.838524634 zio_data_buf_1048576 39553705
> > keg_drain4
> > 2019 Feb 2 19:49:54.838524634 zio_data_buf_1048576 39565323
> > zone_reclaim:return
> >
> > 35109.486 ms for last loop, 149 items to freed.
>
> 35ms to free 149MB (38144 4KB pages), so roughly 1us per page. That
> does seem like a lot, but freeing a page (vm_page_free(m)) is much
> more expensive than freeing an item to UMA (i.e., uma_zfree()).
> Most of that time will be spent in _kmem_unback().
> ==
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r351673 - in head: lib/libmemstat share/man/man9 sys/cddl/compat/opensolaris/kern sys/kern sys/vm
On Wed, Sep 04, 2019 at 09:27:00AM +0300, Andriy Gapon wrote: > On 04/09/2019 01:01, Mark Johnston wrote: > > Some of the vm_lowmem eventhandlers probably shouldn't be called each > > time the page daemon scans the inactive queue (every 0.1s under memory > > pressure). ufsdirhash_lowmem and mb_reclaim in particular don't seem > > like they need to be invoked very frequently. We could easily define > > multiple eventhandlers to differentiate between these cases, though. > > My proposal is to run uma_reclaim(UMA_RECLAIM_TRIM) before invoking vm_lowmem. > If that reclaims "enough" memory, then do not call vm_lowmem at all. > > Oh, and I have a related question. > Say, a zone has X items in use and Y items in cache. > After running vm_lowmem the composition changes to X - C and Y + C > respectively. > Then we run uma_reclaim(UMA_RECLAIM_TRIM). > Question: is that guaranteed to free at least C items? No. The estimation cannot distinguish between items that were freed as a result of a lowmem request vs. those that were freed as a regular operation. As a result it considers the extra C items as part of the cache's working size. > I am thinking about a possibility that the zone's workset size is estimated to > be at least X (and maybe even more than X + Y). In that case UMA_RECLAIM_TRIM > won't free those C items, they will stay in the cache. So, it seems like > releasing them was in vain, at least for the immediate reclaim. This is true. Of course, the items may eventually become reclaimable as the estimate decays. ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r351673 - in head: lib/libmemstat share/man/man9 sys/cddl/compat/opensolaris/kern sys/kern sys/vm
On Wed, Sep 04, 2019 at 09:00:03AM +0300, Andriy Gapon wrote: > On 04/09/2019 01:01, Mark Johnston wrote: > > Slawa and I talked about this in the past. His complaint is that a > > large cache can take a significant amount of time to trim, and it > > manifests as a spike of CPU usage and contention on the zone lock. In > > particular, keg_drain() iterates over the list of free slabs with the > > keg lock held, and if the many items were freed to the keg while > > trimming/draining, the list can be quite long. This can have effects > > outside the zone, for example if we are reclaiming items from zones used > > by other UMA zones, like the bucket or slab zones. > > My concern is different, though. > I feel that having oversized caches for long periods of time produces a skewed > picture of memory usage. Particularly, some ZFS caches are sometimes > extremely > oversized. I don't care much about details of consequences of such oversized > caches. I just think that that is not right on a more general level. > > > Reclaiming cached items when there is no demand for free pages seems > > wrong to me. > > It certainly was wrong before. > Now that we have a capability to trim a cache size to a workset size it > doesn't > feel as wrong to me. One partial problem is that some UMA items are expensive to allocate and free. On amd64 slabs larger than the page size must be mapped into KVA, and this is not a scalable operation: pages must be inserted into and removed from kernel_object, and when removed we must issue a TLB shootdown to all CPUs. Proactively freeing such items from the cache might also exacerbate fragmentation over time. For direct-mapped items I think the tradeoff makes more sense and we could indeed start regularly freeing items based on the current WSS estimation. There has been a lot of work in the past year or two to make the page allocator cheaper and more scalable. > > We historically had similar problems with the page daemon, > > which last year was changed to perform smaller reclamations at a greater > > frequency. I suspect a better approach for UMA would be to similarly > > increase reclaim frequency and reduce the number of items freed in one > > go. ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
