Re: svn commit: r365249 - head

[Slawa Olhovchenkov]

On Wed, Sep 02, 2020 at 11:53:57AM -0400, Ryan Moeller wrote:

> 
> On 9/2/20 11:43 AM, Andriy Gapon wrote:
> > On 02/09/2020 18:23, Ryan Moeller wrote:
> >> On 9/2/20 10:28 AM, Warner Losh wrote:
> >>> Author: imp
> >>> Date: Wed Sep  2 14:28:54 2020
> >>> New Revision: 365249
> >>> URL: https://svnweb.freebsd.org/changeset/base/365249
> >>>
> >>> Log:
> >>>     Add note about needing to manually import the zfs pools or update
> >>>     /etc/rc.d due to the cache file moving to /etc.
> >>>
> >>> Modified:
> >>>     head/UPDATING
> >>>
> >>> Modified: head/UPDATING
> >>> ==
> >>> --- head/UPDATING    Wed Sep  2 12:57:34 2020    (r365248)
> >>> +++ head/UPDATING    Wed Sep  2 14:28:54 2020    (r365249)
> >>> @@ -36,6 +36,10 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 13.x IS SLOW:
> >>>    scenarios have been tested and fixed, but rebuilding kernels 
> >>> without
> >>>    rebuilding world may fail.
> >>>    +    The ZFS cache file has moved from /boot to /etc to match the 
> >>> OpenZFS
> >>> +    upstream default. This means your zpool won't auto import until you
> >>> +    upgrade your /etc/rc.d files or you import them manually.
> >>> +
> >>>    20200824:
> >>>    The resume code now notifies devd with the 'kernel' system
> >>>    rather than the old 'kern' subsystem to be consistent with
> >> Thanks, enough people seemed to be getting tripped up by this.
> > I think that this is a very useful note.
> >
> > But I do not see a direct connection between the change of zpool.cache 
> > location
> > and the new ZFS's not automatically importing zpool.cache pools on boot.
> >
> 
> True, the real reason is that the kernel module in OpenZFS does not 
> autoimport pools.
> Instead we explicitly "zpool import -a" in one of the ZFS rc scripts. 

Is posible to restore old behavior?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r358995 - head/sys/net

[Slawa Olhovchenkov]

On Sat, Mar 14, 2020 at 07:43:44PM +, Patrick Kelsey wrote:

> Author: pkelsey
> Date: Sat Mar 14 19:43:44 2020
> New Revision: 358995
> URL: https://svnweb.freebsd.org/changeset/base/358995
> 
> Log:
>   Fix iflib freelist state corruption
>   
>   This fixes a bug in iflib freelist management that breaks the required
>   correspondence between freelist indexes and driver ring slots.
>   
>   PR: 243126, 243392, 240628

Are you sure?
Bug 240628 - sysutils/zfsnap2: Completion script link for zsh has
wrong path 

>   Reported by:avg, alexandr.oleyni...@gmail.com, Harald Schmalzbauer
>   Reviewed by:avg, gallatin
>   MFC after:  1 week
>   Differential Revision:  https://reviews.freebsd.org/D23943

Very interesting.
Is this related to my problem?
I am have trouble w/ netmap:

1. netmap reported RX slots filled all ring (tail == head-1)
2. application processed all RX slots and reported to nemap (head == tail)
3. RX stuck.

workaround: report on step 2 head = tail-1 (one RX slots left
unprocessed).

Also I am have trouble w/ TX, similar.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r358655 - head/sbin/mount_nfs

[Slawa Olhovchenkov]

On Thu, Mar 05, 2020 at 07:19:59PM +0100, Dimitry Andric wrote:

> On 5 Mar 2020, at 18:44, Gleb Smirnoff  wrote:
> > 
> > On Thu, Mar 05, 2020 at 08:35:15PM +0300, Slawa Olhovchenkov wrote:
> > S> > > D> sbin/mount_nfs/mount_nfs.c:549:10: error: cast from 'char *' to 
> > 'struct
> > S> > > D> if_msghdr *' increases required alignment from 1 to 4
> > S> > > D> [-Werror,-Wcast-align]
> > S> > > D> ifm = (struct if_msghdr *)buf;
> > S> > > D>   ^~~
> > S> > > D> 1 error generated.
> > S> > > D>
> > S> > > D> In practice I don't think the buffer can ever get misaligned, so 
> > can you
> > S> > > D> please add a NO_WCAST_ALIGN= to the Makefile?
> > S> > >
> > S> > > route(8) handles the same problem via intermediate (void *) cast. 
> > What is
> > S> > > preferred way to solve the problem? Change compiler flags file wide, 
> > or
> > S> > > just through (void *) cast?
> > S> >
> > S> > Copy to aligned buffer or got SIGBUS on some architectures?
> > S>
> > S> char buf[2048] __aligned(__alignof(struct if_msghdr));
> > S>
> > S> resolve this watning.
> > 
> > Thanks, Slawa! I think this is the most elegant solution.
> 
> Why don't just declare the buffer as:
> 
>   struct if_msghdr buf;
> 
> and then do:
> 
>   nread = read(s, , sizeof buf);
> 
> ?  You are never reading more than one if_msghdr anyway, and then there
> is no need to cast anything.

My inspiration: route socket can return other messages (man 4 route)
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r358655 - head/sbin/mount_nfs

[Slawa Olhovchenkov]

On Thu, Mar 05, 2020 at 08:24:54PM +0300, Slawa Olhovchenkov wrote:

> On Thu, Mar 05, 2020 at 08:33:50AM -0800, Gleb Smirnoff wrote:
> 
> > On Thu, Mar 05, 2020 at 03:29:23PM +0100, Dimitry Andric wrote:
> > D> On 2020-03-04 23:27, Gleb Smirnoff wrote:
> > D> > Author: glebius
> > D> > Date: Wed Mar  4 22:27:16 2020
> > D> > New Revision: 358655
> > D> > URL: https://svnweb.freebsd.org/changeset/base/358655
> > D> > 
> > D> > Log:
> > D> >When a machine boots the NFS mounting script is executed after
> > D> >interfaces are configured, but for many interfaces (e.g. all Intel)
> > D> >ifconfig causes link renegotiation, so the first attempt to mount
> > D> >NFS always fails. After that mount_nfs sleeps for 30 seconds, while
> > D> >only a couple seconds are actually required for interface to get up.
> > D> >Instead of sleeping, do select(2) on routing socket and check if
> > D> >some interface became UP and in this case retry immediately.
> > D> 
> > D> At least on i386, this causes a -Werror warning:
> > D> 
> > D> sbin/mount_nfs/mount_nfs.c:549:10: error: cast from 'char *' to 'struct
> > D> if_msghdr *' increases required alignment from 1 to 4
> > D> [-Werror,-Wcast-align]
> > D> ifm = (struct if_msghdr *)buf;
> > D>   ^~~
> > D> 1 error generated.
> > D> 
> > D> In practice I don't think the buffer can ever get misaligned, so can you
> > D> please add a NO_WCAST_ALIGN= to the Makefile?
> > 
> > route(8) handles the same problem via intermediate (void *) cast. What is
> > preferred way to solve the problem? Change compiler flags file wide, or
> > just through (void *) cast?
> 
> Copy to aligned buffer or got SIGBUS on some architectures?

char buf[2048] __aligned(__alignof(struct if_msghdr));

resolve this watning.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r358655 - head/sbin/mount_nfs

[Slawa Olhovchenkov]

On Thu, Mar 05, 2020 at 08:33:50AM -0800, Gleb Smirnoff wrote:

> On Thu, Mar 05, 2020 at 03:29:23PM +0100, Dimitry Andric wrote:
> D> On 2020-03-04 23:27, Gleb Smirnoff wrote:
> D> > Author: glebius
> D> > Date: Wed Mar  4 22:27:16 2020
> D> > New Revision: 358655
> D> > URL: https://svnweb.freebsd.org/changeset/base/358655
> D> > 
> D> > Log:
> D> >When a machine boots the NFS mounting script is executed after
> D> >interfaces are configured, but for many interfaces (e.g. all Intel)
> D> >ifconfig causes link renegotiation, so the first attempt to mount
> D> >NFS always fails. After that mount_nfs sleeps for 30 seconds, while
> D> >only a couple seconds are actually required for interface to get up.
> D> >Instead of sleeping, do select(2) on routing socket and check if
> D> >some interface became UP and in this case retry immediately.
> D> 
> D> At least on i386, this causes a -Werror warning:
> D> 
> D> sbin/mount_nfs/mount_nfs.c:549:10: error: cast from 'char *' to 'struct
> D> if_msghdr *' increases required alignment from 1 to 4
> D> [-Werror,-Wcast-align]
> D> ifm = (struct if_msghdr *)buf;
> D>   ^~~
> D> 1 error generated.
> D> 
> D> In practice I don't think the buffer can ever get misaligned, so can you
> D> please add a NO_WCAST_ALIGN= to the Makefile?
> 
> route(8) handles the same problem via intermediate (void *) cast. What is
> preferred way to solve the problem? Change compiler flags file wide, or
> just through (void *) cast?

Copy to aligned buffer or got SIGBUS on some architectures?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r357849 - head/sys/cam/scsi

[Slawa Olhovchenkov]

On Thu, Feb 13, 2020 at 08:36:08AM -0700, Warner Losh wrote:

> On Thu, Feb 13, 2020 at 7:05 AM Slawa Olhovchenkov  wrote:
> 
> > On Thu, Feb 13, 2020 at 01:23:45AM +, Warner Losh wrote:
> >
> > > Author: imp
> > > Date: Thu Feb 13 01:23:44 2020
> > > New Revision: 357849
> > > URL: https://svnweb.freebsd.org/changeset/base/357849
> > >
> > > Log:
> > >   Convert rotating and unmapped_io to a DA flag
> > >
> > >   Rotating and unmapped_io are really da flags. Convert them to a flag
> > so it will
> > >   be reported with the other flags for the device. Deprecate the
> > .rotating and
> > >   .unmapped_io sysctls in FreeBSD 14 and remove the softc ints.
> >
> > Can you clarification this deprecate?
> >
> 
> The functionality has moved into the new .flags sysctl. I plan on keeping

Thanks!
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r357849 - head/sys/cam/scsi

[Slawa Olhovchenkov]

On Thu, Feb 13, 2020 at 01:23:45AM +, Warner Losh wrote:

> Author: imp
> Date: Thu Feb 13 01:23:44 2020
> New Revision: 357849
> URL: https://svnweb.freebsd.org/changeset/base/357849
> 
> Log:
>   Convert rotating and unmapped_io to a DA flag
>   
>   Rotating and unmapped_io are really da flags. Convert them to a flag so it 
> will
>   be reported with the other flags for the device. Deprecate the .rotating and
>   .unmapped_io sysctls in FreeBSD 14 and remove the softc ints.

Can you clarification this deprecate?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r357553 - head/sys/dev/cxgbe

[Slawa Olhovchenkov]

On Wed, Feb 05, 2020 at 12:13:15AM +, Navdeep Parhar wrote:

> Author: np
> Date: Wed Feb  5 00:13:15 2020
> New Revision: 357553
> URL: https://svnweb.freebsd.org/changeset/base/357553
> 
> Log:
>   cxgbe(4): Add a knob to allow netmap tx traffic to be checksummed by
>   the hardware.
>   
>   hw.cxgbe.nm_txcsum=1

Very interesting.
Please, describe some more detail about using this feture (for
example, set this before driver loading? first netmap open? any netmap
open? on the fly?)
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r356758 - in head/usr.sbin/bsdinstall: . scripts

[Slawa Olhovchenkov]

On Thu, Jan 16, 2020 at 02:43:37PM +0700, Eugene Grosbein wrote:

> 16.01.2020 4:41, Ed Maste wrote:
> 
> > On Wed, 15 Jan 2020 at 16:10, Eugene Grosbein  wrote:
> >>
> >> There are multiple scenarios there ZFS may be sub-optimal at least: small 
> >> i386 virtual guests
> >> or 32-bit only hardware like AMD Geode, or big amd64 SSD-only systems with 
> >> bhyve and multiple guests
> >> that need lots of memory and should not fight with ZFS for RAM etc.
> > 
> > That may well be the case, but our defaults should represent the
> > configuration that's desirable to the largest set of users, and IMO
> > that's ZFS in most cases today.
> > 
> > It might be that we should default to UFS on i386 and ZFS on amd64?
> 
> UFS may be better for any virtual guest having RAM less or equal to 4GB.

Why?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r352868 - in head/sys/netinet: . tcp_stacks

[Slawa Olhovchenkov]

On Sun, Sep 29, 2019 at 10:45:13AM +, Michael Tuexen wrote:

> Author: tuexen
> Date: Sun Sep 29 10:45:13 2019
> New Revision: 352868
> URL: https://svnweb.freebsd.org/changeset/base/352868
> 
> Log:
>   RFC 7112 requires a host to put the complete IP header chain
>   including the TCP header in the first IP packet.
>   Enforce this in tcp_output(). In addition make sure that at least
>   one byte payload fits in the TCP segement to allow making progress.
>   Without this check, a kernel with INVARIANTS will panic.
>   This issue was found by running an instance of syzkaller.

How to posible this?
Host required to handle packets up to 576 bytes, how to IP and TCP
options can exhaust this size?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r351673 - in head: lib/libmemstat share/man/man9 sys/cddl/compat/opensolaris/kern sys/kern sys/vm

[Slawa Olhovchenkov]

On Sun, Sep 08, 2019 at 04:18:19PM -0400, Mark Johnston wrote:

> On Sat, Sep 07, 2019 at 06:31:10PM +0300, Slawa Olhovchenkov wrote:
> > On Sat, Sep 07, 2019 at 10:50:34AM -0400, Mark Johnston wrote:
> > 
> > > On Wed, Sep 04, 2019 at 05:45:24PM +0300, Slawa Olhovchenkov wrote:
> > > > On Tue, Sep 03, 2019 at 06:01:06PM -0400, Mark Johnston wrote:
> > > > > > Mostly problem I am see at this
> > > > > > work -- very slowly vm_page_free(). May be currenly this is more
> > > > > > speedy...
> > > > > 
> > > > > How did you determine this?
> > > > 
> > > > This is you guess:
> > > 
> > > So was the guess correct?
> > 
> > I am just trust to you.
> > How to check this guess?
> 
> You can try to measure time spent in pmap_remove() relative to the rest
> of _kmem_unback().

===
fbt:kernel:_kmem_unback:entry {   self->kts = timestamp; }
fbt:kernel:_kmem_unback:return /self->kts/{  @ts["kts"] = sum(timestamp - 
self->kts);  self->kts = 0; }
fbt:kernel:pmap_remove:entry   /self->kts/{  self->pts = timestamp; }
fbt:kernel:pmap_remove:entry  /self->kts == 0/ { self->ats = timestamp; }
fbt:kernel:pmap_remove:return /self->pts/ {  @ts["pts"] = sum(timestamp - 
self->pts); self->pts = 0; }
fbt:kernel:pmap_remove:return /self->ats/ {  @ts["ats"] = sum(timestamp - 
self->ats); self->ats = 0; }

tick-1s
{
  printa("%-8s %20@d\n", @ts);
  printf("\n");
  clear(@ts);
}
===

pts   7166680
ats   8143193
kts  16721822

pts   3458647
ats   5393452
kts  10504523

kts 0
pts 0
ats   2758752

pts   3387748
ats   4322653
kts   8100282

pts   4002444
ats   5422748
kts  11761955

pts   1151713
kts   2742176
ats   8242958

pts 34441
kts145683
ats   4822328

kts 0
pts 0
ats   5357808

pts 16008
kts148203
ats   4947368

pts 26945
kts156011
ats   8368502

pts323620
kts   1154981
ats  10093137

pts 79504
kts228135
ats   7131059

pts734062
kts   1747364
ats   4619796

pts401453
kts   1036605
ats   8751919

> > > If so, IMO the real solution is to avoid kmem_*
> > > for data buffers and use ABD instead.
> > 
> > What problem resolve this?
> 
> To allocate buffers larger than PAGE_SIZE the kernel must allocate a
> number of physical pages and map them using the page tables.  The step
> of creating and destroying mappings is expensive and doesn't scale well
> to many CPUs.  With ABD, ZFS avoids this expense when its caches are
> shrunk.
> 
> > ABD any way is slowly vs kmem_*.
> 
> Can we solve this problem instead?

slowly ABD? or expensive of creating and destroying mappings?
slowly ABD -- don't use ABD and scatter gather.
expensive of creating and destroying mappings:

- do kernel more NUMA-aware and go mostly alloc and free to zone cache
  (curenly may be allocation can do from dom1 and freeing in dom0. I am see 
this.)
- calculate sensible working set size of zone cache and aim trim to do nothing
- my be do batch removed/unmaped and do TLB shootdown only once at end of batch?

> > > > ==
> > > > > while ((slab = SLIST_FIRST()) != NULL) {
> > > > > SLIST_REMOVE(, slab, uma_slab, us_hlink);
> > > > > keg_free_slab(keg, slab, keg->uk_ipers);
> > > > > }
> > > > > 2019 Feb  2 19:49:54.800524364   zio_data_buf_1048576  1032605 
> > > > > cache_reclaim limit  100 dom 0 nitems 1672 imin  298
> > > > > 2019 Feb  2 19:49:54.800524364   zio_data_buf_1048576  1033736 
> > > > > cache_reclaim recla  149 dom 0 nitems 1672 imin  298
> > > > > 2019 Feb  2 19:49:54.802524468   zio_data_buf_1048576  3119710 
> > > > > cache_reclaim limit  100 dom 1 nitems1 imin0
> > > > > 2019 Feb  2 19:49:54.802524468   zio_data_buf_1048576  3127550 
> > > > > keg_dr

Re: svn commit: r351673 - in head: lib/libmemstat share/man/man9 sys/cddl/compat/opensolaris/kern sys/kern sys/vm

[Slawa Olhovchenkov]

On Sat, Sep 07, 2019 at 10:50:34AM -0400, Mark Johnston wrote:

> On Wed, Sep 04, 2019 at 05:45:24PM +0300, Slawa Olhovchenkov wrote:
> > On Tue, Sep 03, 2019 at 06:01:06PM -0400, Mark Johnston wrote:
> > > > Mostly problem I am see at this
> > > > work -- very slowly vm_page_free(). May be currenly this is more
> > > > speedy...
> > > 
> > > How did you determine this?
> > 
> > This is you guess:
> 
> So was the guess correct?

I am just trust to you.
How to check this guess?

> If so, IMO the real solution is to avoid kmem_*
> for data buffers and use ABD instead.

What problem resolve this?
ABD any way is slowly vs kmem_*.

> > ==
> > > while ((slab = SLIST_FIRST()) != NULL) {
> > > SLIST_REMOVE(, slab, uma_slab, us_hlink);
> > > keg_free_slab(keg, slab, keg->uk_ipers);
> > > }
> > > 2019 Feb  2 19:49:54.800524364   zio_data_buf_1048576  1032605 
> > > cache_reclaim limit  100 dom 0 nitems 1672 imin  298
> > > 2019 Feb  2 19:49:54.800524364   zio_data_buf_1048576  1033736 
> > > cache_reclaim recla  149 dom 0 nitems 1672 imin  298
> > > 2019 Feb  2 19:49:54.802524468   zio_data_buf_1048576  3119710 
> > > cache_reclaim limit  100 dom 1 nitems1 imin0
> > > 2019 Feb  2 19:49:54.802524468   zio_data_buf_1048576  3127550 
> > > keg_drain2
> > > 2019 Feb  2 19:49:54.803524487   zio_data_buf_1048576  219 
> > > keg_drain3
> > > 2019 Feb  2 19:49:54.838524634   zio_data_buf_1048576 39553705 
> > > keg_drain4
> > > 2019 Feb  2 19:49:54.838524634   zio_data_buf_1048576 39565323 
> > > zone_reclaim:return
> > >
> > > 35109.486 ms for last loop, 149 items to freed.
> > 
> > 35ms to free 149MB (38144 4KB pages), so roughly 1us per page.  That
> > does seem like a lot, but freeing a page (vm_page_free(m)) is much
> > more expensive than freeing an item to UMA (i.e., uma_zfree()).
> > Most of that time will be spent in _kmem_unback().
> > ==
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r351673 - in head: lib/libmemstat share/man/man9 sys/cddl/compat/opensolaris/kern sys/kern sys/vm

[Slawa Olhovchenkov]

On Wed, Sep 04, 2019 at 09:27:00AM +0300, Andriy Gapon wrote:

> On 04/09/2019 01:01, Mark Johnston wrote:
> > Some of the vm_lowmem eventhandlers probably shouldn't be called each
> > time the page daemon scans the inactive queue (every 0.1s under memory
> > pressure).  ufsdirhash_lowmem and mb_reclaim in particular don't seem
> > like they need to be invoked very frequently.  We could easily define
> > multiple eventhandlers to differentiate between these cases, though.
> 
> My proposal is to run uma_reclaim(UMA_RECLAIM_TRIM) before invoking vm_lowmem.
> If that reclaims "enough" memory, then do not call vm_lowmem at all.
> 
> Oh, and I have a related question.
> Say, a zone has X items in use and Y items in cache.
> After running vm_lowmem the composition changes to X - C and Y + C 
> respectively.
> Then we run uma_reclaim(UMA_RECLAIM_TRIM).
> Question: is that guaranteed to free at least C items?
> I am thinking about a possibility that the zone's workset size is estimated to
> be at least X (and maybe even more than X + Y).  In that case UMA_RECLAIM_TRIM
> won't free those C items, they will stay in the cache.  So, it seems like
> releasing them was in vain, at least for the immediate reclaim.

My goal is uma_reclaim(UMA_RECLAIM_TRIM) reclaim from zone cache up to zone 
cache working set.
Idealy in stable run uma_reclaim(UMA_RECLAIM_TRIM) do nothing and all
allocation/free from zone go from/to zone cache. As result
uma_reclaim(UMA_RECLAIM_TRIM) work only for rebalance memory consume at
working profile change.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r351673 - in head: lib/libmemstat share/man/man9 sys/cddl/compat/opensolaris/kern sys/kern sys/vm

[Slawa Olhovchenkov]

On Tue, Sep 03, 2019 at 06:01:06PM -0400, Mark Johnston wrote:

> > > have you considered running UMA_RECLAIM_TRIM periodically, even without a 
> > > memory
> > > pressure?
> > > I think that with such a periodic trimming there will be less need to 
> > > invoke
> > > vm_lowmem().
> 
> Slawa and I talked about this in the past.  His complaint is that a
> large cache can take a significant amount of time to trim, and it

Not only large cache. Also I am see large mbuf cache (10GB+ after pike network 
activites)
and many other zones. For example, live server w/ stable/11, cache sizes of 
zones in MB:

54 RADIX NODE
55 zio_data_buf_131072
73 zio_data_buf_12288
77 zio_data_buf_98304
93 socket
99 tcpcb
  1072 mbuf
  1136 zio_buf_131072
  1443 zio_data_buf_1048576
 17242 mbuf_jumbo_page

> manifests as a spike of CPU usage and contention on the zone lock.  In
> particular, keg_drain() iterates over the list of free slabs with the
> keg lock held, and if the many items were freed to the keg while
> trimming/draining, the list can be quite long.  This can have effects
> outside the zone, for example if we are reclaiming items from zones used
> by other UMA zones, like the bucket or slab zones.
> 
> Reclaiming cached items when there is no demand for free pages seems
> wrong to me.  We historically had similar problems with the page daemon,
> which last year was changed to perform smaller reclamations at a greater
> frequency.  I suspect a better approach for UMA would be to similarly
> increase reclaim frequency and reduce the number of items freed in one
> go.

My goal is next:

1. Today memory size is quite big. 64GB is minimal, 256GB is not extraordinary.
   As result memory processed at lowmem event can be very big compared to 
historicaly.
2. Memory reclaiming is very expensetive at last stage. As result reclaiming 
some 10GB
   can take 10s and more
3. Memory depletion may be very fast at current speed (40Gbit network 
connectivity is 5GB/s).
   As result (2+3) memory reclaiming at lowmem event may be slow for depletion 
compensation.
4. Many susbsystem now try do't cause lowmem by automatic memory depletion. 
Large unused in
   zone cache memory cause inefficent memory use (see above -- about 18GB 
memory may be used
   as cache or other way but currently just in zone cache. lowmem don't caused 
because all consumer
   try to rest sufficient free memory)
5. NUMA dramatize situation because (as I see) memory allocation can give from 
dom1 and free to dom0.
   As result zone cache in dom0 rise and don't used. Currently kernel not fully 
NUMA-avare and need many work.
6. All of this can cause exause memory below vmd_free_reserved and slowly many 
operation in kernel.

I am see all of this.

> > > Also, I think that we would be able to retire (or re-purpose) 
> > > lowmem_period.
> > > E.g., the trimming would be done every lowmem_period, but vm_lowmem() 
> > > would not
> > > be throttled.
> 
> Some of the vm_lowmem eventhandlers probably shouldn't be called each
> time the page daemon scans the inactive queue (every 0.1s under memory
> pressure).  ufsdirhash_lowmem and mb_reclaim in particular don't seem
> like they need to be invoked very frequently.  We could easily define
> multiple eventhandlers to differentiate between these cases, though.
> 
> > > One example of the throttling of vm_lowmem being bad is its interaction 
> > > with the
> > > ZFS ARC.  When there is a spike in memory usage we want the ARC to adapt 
> > > as
> > > quickly as possible.  But at present the lowmem_period logic interferes 
> > > with that.
> >
> > Some time ago, I sent Mark a patch that implements this logic,
> > specialy for ARC and mbuf cooperate.
> >
> > Mostly problem I am see at this
> > work -- very slowly vm_page_free(). May be currenly this is more
> > speedy...
> 
> How did you determine this?

This is you guess:

==
> while ((slab = SLIST_FIRST()) != NULL) {
> SLIST_REMOVE(, slab, uma_slab, us_hlink);
> keg_free_slab(keg, slab, keg->uk_ipers);
> }
> 2019 Feb  2 19:49:54.800524364   zio_data_buf_1048576  1032605 
> cache_reclaim limit  100 dom 0 nitems 1672 imin  298
> 2019 Feb  2 19:49:54.800524364   zio_data_buf_1048576  1033736 
> cache_reclaim recla  149 dom 0 nitems 1672 imin  298
> 2019 Feb  2 19:49:54.802524468   zio_data_buf_1048576  3119710 
> cache_reclaim limit  100 dom 1 nitems1 imin0
> 2019 Feb  2 19:49:54.802524468   zio_data_buf_1048576  3127550 keg_drain2
> 2019 Feb  2 19:49:54.803524487   zio_data_buf_1048576  219 keg_drain3
> 2019 Feb  2 19:49:54.838524634   zio_data_buf_1048576 39553705 keg_drain4
> 2019 Feb  2 19:49:54.838524634   zio_data_buf_1048576 39565323 
> zone_reclaim:return
>
> 35109.486 ms for last loop, 149 items to freed.

35ms to free 149MB (38144 4KB pages), so roughly 1us per page.  That
does seem like a lot, but freeing a page 

Re: svn commit: r351673 - in head: lib/libmemstat share/man/man9 sys/cddl/compat/opensolaris/kern sys/kern sys/vm

[Slawa Olhovchenkov]

On Tue, Sep 03, 2019 at 10:02:59AM +0300, Andriy Gapon wrote:

> On 02/09/2019 01:22, Mark Johnston wrote:
> > Author: markj
> > Date: Sun Sep  1 22:22:43 2019
> > New Revision: 351673
> > URL: https://svnweb.freebsd.org/changeset/base/351673
> > 
> > Log:
> >   Extend uma_reclaim() to permit different reclamation targets.
> >   
> >   The page daemon periodically invokes uma_reclaim() to reclaim cached
> >   items from each zone when the system is under memory pressure.  This
> >   is important since the size of these caches is unbounded by default.
> >   However it also results in bursts of high latency when allocating from
> >   heavily used zones as threads miss in the per-CPU caches and must
> >   access the keg in order to allocate new items.
> >   
> >   With r340405 we maintain an estimate of each zone's usage of its
> >   (per-NUMA domain) cache of full buckets.  Start making use of this
> >   estimate to avoid reclaiming the entire cache when under memory
> >   pressure.  In particular, introduce TRIM, DRAIN and DRAIN_CPU
> >   verbs for uma_reclaim() and uma_zone_reclaim().  When trimming, only
> >   items in excess of the estimate are reclaimed.  Draining a zone
> >   reclaims all of the cached full buckets (the previous behaviour of
> >   uma_reclaim()), and may further drain the per-CPU caches in extreme
> >   cases.
> >   
> >   Now, when under memory pressure, the page daemon will trim zones
> >   rather than draining them.  As a result, heavily used zones do not incur
> >   bursts of bucket cache misses following reclamation, but large, unused
> >   caches will be reclaimed as before.
> 
> Mark,
> 
> have you considered running UMA_RECLAIM_TRIM periodically, even without a 
> memory
> pressure?
> I think that with such a periodic trimming there will be less need to invoke
> vm_lowmem().
> 
> Also, I think that we would be able to retire (or re-purpose) lowmem_period.
> E.g., the trimming would be done every lowmem_period, but vm_lowmem() would 
> not
> be throttled.
> 
> One example of the throttling of vm_lowmem being bad is its interaction with 
> the
> ZFS ARC.  When there is a spike in memory usage we want the ARC to adapt as
> quickly as possible.  But at present the lowmem_period logic interferes with 
> that.

Some time ago, I sent Mark a patch that implements this logic,
specialy for ARC and mbuf cooperate. Mostly problem I am see at this
work -- very slowly vm_page_free(). May be currenly this is more
speedy...
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r348772 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs

[Slawa Olhovchenkov]

On Fri, Jun 07, 2019 at 06:35:42AM +, Andriy Gapon wrote:

> Author: avg
> Date: Fri Jun  7 06:35:42 2019
> New Revision: 348772
> URL: https://svnweb.freebsd.org/changeset/base/348772
> 
> Log:
>   Restore ARC MFU/MRU pressure
>   
>   Before r305323 (MFV r302991: 6950 ARC should cache compressed data)
>   arc_read() code did this for access to a ghost buffer:
>arc_adapt() (from arc_get_data_buf())
>arc_access(hdr, hash_lock)
>   I.e., we first checked access to the MFU ghost/MRU ghost buffer and
>   adapt MFU/MRU sizes (in arc_adapt()) and next move buffer from the ghost
>   state to regular.
>   
>   After r305323 the sequence is different:
>arc_access(hdr, hash_lock);
>arc_hdr_alloc_pabd(hdr);
>   I.e., we first move the buffer from the ghost state in arc_access() and
>   then we check access to buffer in ghost state (in arc_hdr_alloc_pabd()
>   -> arc_get_data_abd() -> arc_get_data_impl() -> arc_adapt()).  This is
>   incorrect: arc_adapt() never see access to the ghost buffer because
>   arc_access() already migrated the buffer from the ghost state to
>   regular.
>   
>   So, the fix is to restore a call to arc_adapt() before arc_access() and
>   to suppress the call to arc_adapt() after arc_access().
>   
>   Submitted by:   Slawa Olhovchenkov 
>   MFC after:  2 weeks
>   Sponsored by:   Integros [integros.com]
>   Differential Revision: https://reviews.freebsd.org/D19094

Include in 11.3-RELEASE planed?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r348117 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs

[Slawa Olhovchenkov]

On Wed, May 22, 2019 at 04:00:58PM -0400, Alexander Motin wrote:

> On 22.05.2019 15:49, Slawa Olhovchenkov wrote:
> > On Wed, May 22, 2019 at 06:43:48PM +, Alexander Motin wrote:
> >> Author: mav
> >> Date: Wed May 22 18:43:48 2019
> >> New Revision: 348117
> >> URL: https://svnweb.freebsd.org/changeset/base/348117
> >>
> >> Log:
> >>   Allocate buffers smaller then ABD chunk size as linear.
> >>   
> >>   This allows to reduce memory waste by letting UMA to put multiple small
> >>   buffers into one memory page slab.  The page sharing means that UMA
> >>   may not be able to free memory page when some of buffers are freed, but
> >>   alternatively memory used by that buffer would just be wasted from the
> >>   beginning.
> >>   
> >>   This change follows alike change in ZoL, but unlike Linux (according to
> >>   my understanding of it from comments) FreeBSD never shares slabs bigger
> >>   then one memory page, so this should be even less invasive then there.
> >>   
> >>   MFC after:   2 weeks
> >>   Sponsored by:iXsystems, Inc.
> >>
> >> Modified:
> >>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/abd.c
> >>
> >> Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/abd.c
> >> ==
> >> --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/abd.c  Wed May 
> >> 22 17:42:22 2019(r348116)
> >> +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/abd.c  Wed May 
> >> 22 18:43:48 2019(r348117)
> >> @@ -290,7 +290,7 @@ abd_free_struct(abd_t *abd)
> >>  abd_t *
> >>  abd_alloc(size_t size, boolean_t is_metadata)
> >>  {
> >> -  if (!zfs_abd_scatter_enabled)
> >> +  if (!zfs_abd_scatter_enabled || size <= zfs_abd_chunk_size)
> >>return (abd_alloc_linear(size, is_metadata));
> > 
> > may be `size < zfs_abd_chunk_size`?
> 
> Why?  It should be the same from memory usage, but I see theoretical
> benefits from having linear buffer with one page rather then scatter
> list with one page.  Plus ZoL also had there `size <= PAGESIZE`.

May be malloc() for page size more effictive? Not sure.
Less overhead for kegs and etc.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r348117 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs

[Slawa Olhovchenkov]

On Wed, May 22, 2019 at 06:43:48PM +, Alexander Motin wrote:

> Author: mav
> Date: Wed May 22 18:43:48 2019
> New Revision: 348117
> URL: https://svnweb.freebsd.org/changeset/base/348117
> 
> Log:
>   Allocate buffers smaller then ABD chunk size as linear.
>   
>   This allows to reduce memory waste by letting UMA to put multiple small
>   buffers into one memory page slab.  The page sharing means that UMA
>   may not be able to free memory page when some of buffers are freed, but
>   alternatively memory used by that buffer would just be wasted from the
>   beginning.
>   
>   This change follows alike change in ZoL, but unlike Linux (according to
>   my understanding of it from comments) FreeBSD never shares slabs bigger
>   then one memory page, so this should be even less invasive then there.
>   
>   MFC after:  2 weeks
>   Sponsored by:   iXsystems, Inc.
> 
> Modified:
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/abd.c
> 
> Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/abd.c
> ==
> --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/abd.c Wed May 22 
> 17:42:22 2019(r348116)
> +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/abd.c Wed May 22 
> 18:43:48 2019(r348117)
> @@ -290,7 +290,7 @@ abd_free_struct(abd_t *abd)
>  abd_t *
>  abd_alloc(size_t size, boolean_t is_metadata)
>  {
> - if (!zfs_abd_scatter_enabled)
> + if (!zfs_abd_scatter_enabled || size <= zfs_abd_chunk_size)
>   return (abd_alloc_linear(size, is_metadata));

may be `size < zfs_abd_chunk_size`?

>   VERIFY3U(size, <=, SPA_MAXBLOCKSIZE);
> ___
> svn-src-...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/svn-src-all
> To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r347410 - in head: . sys/amd64/conf sys/arm/conf sys/arm64/conf sys/i386/conf sys/powerpc/conf sys/riscv/conf sys/sparc64/conf

[Slawa Olhovchenkov]

On Fri, May 10, 2019 at 11:31:21AM -0400, Andrew Gallatin wrote:

> On 2019-05-10 08:44, Slawa Olhovchenkov wrote:
> 
> > pf have ifdef for IPSEC, but don't have support IPSEC_SUPPORT
> > (netpfil/pf/if_pfsync.c).
> > 
> 
> Thanks for pointing this out.  It seems like IPSEC_SUPPORT would work 
> for this.  I've made a patch, and it compiles and the pf module loads.
> However, I have no knowledge of how to test it.  Is this something
> that you use, and which you can test?

I am don't use pf, I am just explore kernel for ame reasson (IPSEC
performance penalty) and see mostly IPSEC code ifdef to

#if defined(IPSEC) || defined(IPSEC_SUPPORT)

and only netpfil/pf/if_pfsync.c ifdef to

#if defined(IPSEC)


> Thanks,
> 
> Drew
> 

> diff --git a/sys/netpfil/pf/if_pfsync.c b/sys/netpfil/pf/if_pfsync.c
> index 45b1e090f95c..cc06637b862e 100644
> --- a/sys/netpfil/pf/if_pfsync.c
> +++ b/sys/netpfil/pf/if_pfsync.c
> @@ -308,7 +308,7 @@ static void   pfsync_bulk_update(void *);
>  static void  pfsync_bulk_fail(void *);
>  
>  static void  pfsync_detach_ifnet(struct ifnet *);
> -#ifdef IPSEC
> +#ifdef IPSEC_SUPPORT
>  static void  pfsync_update_net_tdb(struct pfsync_tdb *);
>  #endif
>  static struct pfsync_bucket  *pfsync_get_bucket(struct pfsync_softc *,
> @@ -1228,7 +1228,7 @@ pfsync_in_tdb(struct pfsync_pkt *pkt, struct mbuf *m, 
> int offset, int count)
>  {
>   int len = count * sizeof(struct pfsync_tdb);
>  
> -#if defined(IPSEC)
> +#if defined(IPSEC_SUPPORT)
>   struct pfsync_tdb *tp;
>   struct mbuf *mp;
>   int offp;
> @@ -1249,7 +1249,7 @@ pfsync_in_tdb(struct pfsync_pkt *pkt, struct mbuf *m, 
> int offset, int count)
>   return (len);
>  }
>  
> -#if defined(IPSEC)
> +#if defined(IPSEC_SUPPORT)
>  /* Update an in-kernel tdb. Silently fail if no tdb is found. */
>  static void
>  pfsync_update_net_tdb(struct pfsync_tdb *pt)

> ___
> svn-src-...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/svn-src-all
> To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r347410 - in head: . sys/amd64/conf sys/arm/conf sys/arm64/conf sys/i386/conf sys/powerpc/conf sys/riscv/conf sys/sparc64/conf

[Slawa Olhovchenkov]

On Thu, May 09, 2019 at 10:38:15PM +, Andrew Gallatin wrote:

> Author: gallatin
> Date: Thu May  9 22:38:15 2019
> New Revision: 347410
> URL: https://svnweb.freebsd.org/changeset/base/347410
> 
> Log:
>   Remove IPSEC from GENERIC due to performance issues
>   
>   Having IPSEC compiled into the kernel imposes a non-trivial
>   performance penalty on multi-threaded workloads due to IPSEC
>   refcounting. In my benchmarks of multi-threaded UDP
>   transmit (connected sockets), I've seen a roughly 20% performance
>   penalty when the IPSEC option is included in the kernel (16.8Mpps
>   vs 13.8Mpps with 32 senders on a 14 core / 28 HTT Xeon
>   2697v3)). This is largely due to key_addref() incrementing and
>   decrementing an atomic reference count on the default
>   policy. This cause all CPUs to stall on the same cacheline, as it
>   bounces between different CPUs.
>   
>   Given that relatively few users use ipsec, and that it can be
>   loaded as a module, it seems reasonable to ask those users to
>   load the ipsec module so as to avoid imposing this penalty on the
>   GENERIC kernel. Its my hope that this will make FreeBSD look
>   better in "out of the box" benchmark comparisons with other
>   operating systems.
>   
>   Many thanks to ae for fixing auto-loading of ipsec.ko when
>   ifconfig tries to configure ipsec, and to cy for volunteering
>   to ensure the the racoon ports will load the ipsec.ko module
>   
>   Reviewed by:cem, cy, delphij, gnn, jhb, jpaetzel
>   Differential Revision:  https://reviews.freebsd.org/D20163

pf have ifdef for IPSEC, but don't have support IPSEC_SUPPORT
(netpfil/pf/if_pfsync.c).
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r335579 - head/sys/vm

[Slawa Olhovchenkov]

On Sat, Jun 23, 2018 at 08:10:10AM +, Jeff Roberson wrote:

> Author: jeff
> Date: Sat Jun 23 08:10:09 2018
> New Revision: 335579
> URL: https://svnweb.freebsd.org/changeset/base/335579
> 
> Log:
>   Sort uma_zone fields according to 64 byte cache line with adjacent line
>   prefetch on 64bit architectures.  Prior to this, two lines were needed
>   for the fast path and each line may fetch an unused adjacent neighbor.
>- Move fields used by the fast path into a single line.
>- Move constants into the adjacent line which is mostly used for
>  the spare bucket alloc 'medium path'.
>- Unpad the mtx which is only used by the fast path and place it in
>  a line with rarely used data.  This aligns the cachelines better and
>  eliminates 128 bytes of wasted space.
>   
>   This gives a 45% improvement on a will-it-scale test on a 24 core machine.

Can this commint mergered to stable/11? Or this commit depends on other commits?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r333133 - head/usr.sbin/pwd_mkdb

[Slawa Olhovchenkov]

On Tue, May 01, 2018 at 02:18:52PM -0400, Ed Maste wrote:

> On 1 May 2018 at 09:01, Slawa Olhovchenkov <s...@zxy.spb.ru> wrote:
> > On Tue, May 01, 2018 at 12:53:46AM +, Ed Maste wrote:
> >
> >> Author: emaste
> >> Date: Tue May  1 00:53:46 2018
> >> New Revision: 333133
> >> URL: https://svnweb.freebsd.org/changeset/base/333133
> >>
> >> Log:
> >>   pwd_mkdb: retire legacy v3 db support (-l option)
> >>
> > May be good idea to check db version before install new binary?
> 
> Perhaps "v3 db" is too brief shorthand: it's not that the database as
> a whole is v3/v4, but that it contains records of one or both types.
> For 15 years we've been emitting v4 and consuming v4 records.

15 years is not aged, I am not sure about some of my databases.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r333133 - head/usr.sbin/pwd_mkdb

[Slawa Olhovchenkov]

On Tue, May 01, 2018 at 12:53:46AM +, Ed Maste wrote:

> Author: emaste
> Date: Tue May  1 00:53:46 2018
> New Revision: 333133
> URL: https://svnweb.freebsd.org/changeset/base/333133
> 
> Log:
>   pwd_mkdb: retire legacy v3 db support (-l option)
>   
>   pwd_mkdb has emitted v4 password database records since 2003 (r113596)
>   in addition to v3, and as of r283981 by default it emitted only v4.
>   As described in r283981, retire the -l legacy option.
>   
>   The -B and -L options were originally added to set the endianness of v3
>   records emitted by pwd_mkdb, but they also set the db hash endiannes and
>   so have been retained temporarily.
>   
>   Announced on the FreeBSD-Current and FreeBSD-Stable lists.  In stable/11
>   the man page contains a deprecation notice, and pwd_mkdb will emit a
>   deprecation notice if the -l option is specified.
>   
>   Reviewed by:delphij, lidl, rgrimes
>   Relnotes:   Yes
>   Sponsored by:   The FreeBSD Foundation
>   Differential Revision:  https://reviews.freebsd.org/D15144

May be good idea to check db version before install new binary?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r332365 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs

[Slawa Olhovchenkov]

On Tue, Apr 10, 2018 at 11:17:33AM -0400, Mark Johnston wrote:

> On Tue, Apr 10, 2018 at 05:09:57PM +0300, Slawa Olhovchenkov wrote:
> > On Tue, Apr 10, 2018 at 01:56:06PM +, Mark Johnston wrote:
> > 
> > > Author: markj
> > > Date: Tue Apr 10 13:56:06 2018
> > > New Revision: 332365
> > > URL: https://svnweb.freebsd.org/changeset/base/332365
> > > 
> > > Log:
> > >   Set zfs_arc_free_target to v_free_target.
> > >   
> > >   Page daemon output is now regulated by a PID controller with a setpoint
> > >   of v_free_target. Moreover, the page daemon now wakes up regularly
> > >   rather than waiting for a wakeup from another thread. This means that
> > >   the free page count is unlikely to drop below the old
> > >   zfs_arc_free_target value, and as a result the ARC was not readily
> > >   freeing pages under memory pressure. Address the immediate problem by
> > >   updating zfs_arc_free_target to match the page daemon's new behaviour.
> > 
> > Can you explain some more about new page daemon algo (and reclaim zone
> > free memory)?
> 
> The old algorithm was pretty simple: there was a free page target and
> below that, a wakeup threshold. Any time a thread allocated a page and
> in so doing caused the free page count to drop below the wakeup
> threshold, that thread would wake up the page daemon, which would scan
> the inactive queue and free pages until the free target is reached, or
> the end of the inactive queue was reached.
> 
> This is simple and easy to reason about, but has some drawbacks. When
> memory pressure is constant, it leads to bursts of CPU usage and lock
> contention. The static watermarks may also be insufficient for some
> demanding workloads. In particular, the wakeup threshold might be too
> low, thus allowing the free page count to drop to dangerous levels and
> triggering expensive memory shortage handling (i.e., VM_WAIT).
> 
> The new algorithm uses a control loop to dynamically compute a target
> for each scan of the inactive queue. The loop takes as input the
> magnitude of the page shortage (v_free_target - v_free_count) and keeps
> track of the rate of change of this difference (i.e., the rate at which
> free pages are being consumed) and the sum of this difference over time
> (i.e., a cumulative value for the magnitude of recent page shortages).
> These factors are used to compute "shortage", the number of pages to
> reclaim with the goal of maintaining a free page count of v_free_target.
> 
> The effect of the new algorithm is that the page daemon runs more
> frequently but for shorter durations, so its CPU usage is more even. It
> responds dynamically to the demands of the workload, so the shortcomings
> of a pair of static watermarks are gone.
> 
> r329882 doesn't really change anything with respect to reclamation of
> pages from UMA zones. There are some plans to address shortcomings there
> in the near future though.

Thank, very nice explain.
IMHO, in this case for ZFS best is old zfs_arc_free_target: too close
zfs_arc_free_target to vm_cnt.v_free_min can cause too often run
arc_target correction and cause CPU consumption and memory subsystem
overuse.
ZFS need more correct pressure, ZFS-specific, and I am try it in D7538.

> > PS: zfs need some more time for free pages from ARC. Also, vanila zfs
> > have broken logic for count used and free ARC's memory. For most
> > correctly count system-wide used and free memory need accounting
> > in-zone free memory.
> 
> Yes, there is a number of problems in this area predating r329882. This
> commit is really just a bandaid.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r332365 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs

[Slawa Olhovchenkov]

On Tue, Apr 10, 2018 at 01:56:06PM +, Mark Johnston wrote:

> Author: markj
> Date: Tue Apr 10 13:56:06 2018
> New Revision: 332365
> URL: https://svnweb.freebsd.org/changeset/base/332365
> 
> Log:
>   Set zfs_arc_free_target to v_free_target.
>   
>   Page daemon output is now regulated by a PID controller with a setpoint
>   of v_free_target. Moreover, the page daemon now wakes up regularly
>   rather than waiting for a wakeup from another thread. This means that
>   the free page count is unlikely to drop below the old
>   zfs_arc_free_target value, and as a result the ARC was not readily
>   freeing pages under memory pressure. Address the immediate problem by
>   updating zfs_arc_free_target to match the page daemon's new behaviour.

Can you explain some more about new page daemon algo (and reclaim zone
free memory)?

PS: zfs need some more time for free pages from ARC. Also, vanila zfs
have broken logic for count used and free ARC's memory. For most
correctly count system-wide used and free memory need accounting
in-zone free memory.

>  arc_free_target_init(void *unused __unused)
>  {
>  
> - zfs_arc_free_target = (vm_cnt.v_free_min / 10) * 11;
> + zfs_arc_free_target = vm_cnt.v_free_target;
>  }
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r327559 - in head: . sys/net

[Slawa Olhovchenkov]

On Fri, Jan 05, 2018 at 11:59:30PM +0700, Eugene Grosbein wrote:

> 05.01.2018 21:38, Slawa Olhovchenkov wrote:
> 
> >>> Irrelevant to RSS and etc. flowid distribution in lacp case work very
> >>> bad. This is good and must be MFC (IMHO).
> >>
> >> It may work bad depending on NIC and/or traffic type.
> >> It works just fine in common case of IP forwarding for packets with 
> >> TCP/UDP inside.
> >>
> >> It can be easily disabled locally for specific cases when it does not work.
> > 
> > Packet distrubuting on network equipment (lacp case) w/ enabled flowid cause
> > uneven queue distributing.
> 
> Once again: this heavily depends on local environment and this is not true 
> for many cases.

For any case resolving cavears need uncommon skills. It's true.
Don't using flowid from hardware need negligible CPU power.
Safe is disable flowid bDefault disable flowid is safe way w/o
performace penalty.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r327559 - in head: . sys/net

[Slawa Olhovchenkov]

On Fri, Jan 05, 2018 at 08:36:48PM +0700, Eugene Grosbein wrote:

> 05.01.2018 20:11, Slawa Olhovchenkov wrote:
> 
> > Irrelevant to RSS and etc. flowid distribution in lacp case work very
> > bad. This is good and must be MFC (IMHO).
> 
> It may work bad depending on NIC and/or traffic type.
> It works just fine in common case of IP forwarding for packets with TCP/UDP 
> inside.
> 
> It can be easily disabled locally for specific cases when it does not work.
> 

Packet distrubuting on network equipment (lacp case) w/ enabled flowid cause
uneven queue distributing. Yes, this is may be disabled locally, but
diagnostic this root cause need uncommon skills.

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r327559 - in head: . sys/net

[Slawa Olhovchenkov]

On Fri, Jan 05, 2018 at 03:50:31AM +0700, Eugene Grosbein wrote:

> 05.01.2018 3:05, Steven Hartland wrote:
> 
> > Author: smh
> > Date: Thu Jan  4 20:05:47 2018
> > New Revision: 327559
> > URL: https://svnweb.freebsd.org/changeset/base/327559
> > 
> > Log:
> >   Disabled the use of flowid for lagg by default
> >   
> >   Disabled the use of RSS hash from the network card aka flowid for
> >   lagg(4) interfaces by default as it's currently incompatible with
> >   the lacp and loadbalance protocols.
> >   
> >   The incompatibility is due to the fact that the flowid isn't know
> >   for the first packet of a new outbound stream which can result in
> >   the hash calculation method changing and hence a stream being
> >   incorrectly split across multiple interfaces during normal
> >   operation.
> >   
> >   This can be re-enabled by setting the following in loader.conf:
> >   net.link.lagg.default_use_flowid="1"
> >   
> >   Discussed with: kmacy
> >   Sponsored by: Multiplay
> 
> RSS by definition has meaning to received stream. What is "outbound" stream
> in this context, why can the hash calculatiom method change and what exactly
> does it mean "a stream being incorrectly split"?
> 
> Defaults should not be changed so easily just because they are not optimal
> for some specific case. Each lagg has its own setting for flowid usage
> and why one cannot just use "ifconfig lagg0 -use_flowid" for such cases?

Irrelevant to RSS and etc. flowid distribution in lacp case work very
bad. This is good and must be MFC (IMHO).
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r322863 - head/sys/cam

[Slawa Olhovchenkov]

On Fri, Aug 25, 2017 at 09:33:35AM -0600, Warner Losh wrote:

> On Fri, Aug 25, 2017 at 7:35 AM, Slawa Olhovchenkov <s...@zxy.spb.ru> wrote:
> 
> > On Thu, Aug 24, 2017 at 10:11:10PM +, Warner Losh wrote:
> >
> > > Author: imp
> > > Date: Thu Aug 24 22:11:10 2017
> > > New Revision: 322863
> > > URL: https://svnweb.freebsd.org/changeset/base/322863
> > >
> > > Log:
> > >   Expand the latency tracking array from 1.024s to 8.192s to help track
> > >   extreme outliers from dodgy drives. Adjust comments to reflect this,
> > >   and make sure that the number of latency buckets match in the two
> > >   places where it matters.
> >
> > May be up to 1min?
> >
> 
> I'm not sure what additional data you'll get between "totally sucks, at
> least 8s latency" and "totally sucks, at least 32s." or "totally sucks, at
> least 64s" though the latter isn't possible with the default timeouts...
> 
> I'm planning on adding a 'max' latency that's self-resetting instead of
> expanding the bucket upwards. I'm also contemplating expanding it down to
> 100us or even 10us since nda serves nvme drives which easily can be sub
> 100us.

This is just stat data?
Don't produce error messages about timeout commands and etc?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r322863 - head/sys/cam

[Slawa Olhovchenkov]

On Thu, Aug 24, 2017 at 10:11:10PM +, Warner Losh wrote:

> Author: imp
> Date: Thu Aug 24 22:11:10 2017
> New Revision: 322863
> URL: https://svnweb.freebsd.org/changeset/base/322863
> 
> Log:
>   Expand the latency tracking array from 1.024s to 8.192s to help track
>   extreme outliers from dodgy drives. Adjust comments to reflect this,
>   and make sure that the number of latency buckets match in the two
>   places where it matters.

May be up to 1min?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r317269 - in head/sys: fs/nfs sys

[Slawa Olhovchenkov]

On Fri, Apr 21, 2017 at 08:08:10PM +, Rick Macklem wrote:

> Author: rmacklem
> Date: Fri Apr 21 20:08:10 2017
> New Revision: 317269
> URL: https://svnweb.freebsd.org/changeset/base/317269
> 
> Log:
>   Set default uid/gid to nobody/nogroup for NFSv4 mapping.
>   
>   The default uid/gid for NFSv4 are set by the nfsuserd(8) daemon.
>   However, they were 0 until the nfsuserd(8) was run. Since it is
>   possible to use NFSv4 without running the nfsuserd(8) daemon, set them
>   to nobody/nogroup initially.
>   Without this patch, the values would be set by the nfsuserd(8) daemon
>   and left changed even if the nfsuserd(8) daemon was killed. The default
>   values of 0 meant that setting a group to "wheel" would fail even when
>   done by root.
>   It also adds a definition of GID_NOGROUP to sys/conf.h.
>   
>   Discussed on:   freebsd-current@
>   MFC after:  2 weeks

Can you also look to this problem
https://lists.freebsd.org/pipermail/freebsd-stable/2012-March/066868.html
?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r317094 - head/share/mk

[Slawa Olhovchenkov]

On Thu, Apr 20, 2017 at 09:58:10AM -0700, John Baldwin wrote:

> > Well it seems it can do some rather important trivial things, like post
> > mortem process kernel dumps during savecore processing helping users
> > submit a kernel panic bugzilla with more than just traceback info.
> 
> Only somewhat.  You should actually try to use kgdb and compare it to the
> kgdb in ports.

I am don't see many difference: both do strange results, both crashed.
For kernel and userland too.

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r317143 - in head/sys/cam: . ata scsi

[Slawa Olhovchenkov]

On Wed, Apr 19, 2017 at 03:04:53PM +, Scott Long wrote:

> Author: scottl
> Date: Wed Apr 19 15:04:52 2017
> New Revision: 317143
> URL: https://svnweb.freebsd.org/changeset/base/317143
> 
> Log:
>   Add infrastructure to the ATA and SCSI transports that supports
>   using a driver-supplied sbuf for printing device discovery
>   announcements. This helps ensure that messages to the console
>   will be properly serialized (through sbuf_putbuf) and not be
>   truncated and interleaved with other messages. The
>   infrastructure mirrors the existing xpt_announce_periph()
>   entry point and is opt-in for now. No content or formatting
>   changes are visible to the operator other than the new coherency.
>   
>   While here, eliminate the stack usage of the temporary
>   announcement buffer in some of the drivers. It's moved to the
>   softc for now, but future work will eliminate it entirely by
>   making the code flow more linear. Future work will also address
>   locking so that the sbufs can be dynamically sized.
>   
>   The scsi_da, scs_cd, scsi_ses, and ata_da drivers are converted
>   at this point, other drivers can be converted at a later date.
>   A tunable+sysctl, kern.cam.announce_nosbuf, exists for testing
>   purposes but will be removed later.
>   
>   TODO:
>   Eliminate all of the code duplication and temporary buffers.  The
>   old printf-based methods will be retired, and xpt_announce_periph()
>   will just be a wrapper that uses a dynamically sized sbuf.  This
>   requires that the register and deregister paths be made malloc-safe,
>   which they aren't currently.

Thanks!
MFC planed?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r317094 - head/share/mk

[Slawa Olhovchenkov]

On Tue, Apr 18, 2017 at 12:45:25PM -0700, John Baldwin wrote:

> On Tuesday, April 18, 2017 07:30:13 PM Slawa Olhovchenkov wrote:
> > On Tue, Apr 18, 2017 at 04:27:48PM +, John Baldwin wrote:
> > 
> > > Author: jhb
> > > Date: Tue Apr 18 16:27:48 2017
> > > New Revision: 317094
> > > URL: https://svnweb.freebsd.org/changeset/base/317094
> > > 
> > > Log:
> > >   Disable in-tree GDB by default on x86, mips, and powerpc.
> > >   
> > >   GDB in ports contains all of the functionality as GDB in base
> > >   (including kgdb) for these platforms along with additional
> > >   functionality.  In-tree GDB remains enabled on ARM and sparc64.
> > >   GDB in ports does not currently support kernel debugging on arm,
> > >   and ports GDB for sparc64 has not been tested (though it does
> > >   include sparc64 support).
> > >   
> > >   Reviewed by:bdrewery, emaste, imp
> > >   Relnotes:   yes
> > >   Sponsored by:   DARPA / AFRL
> > >   Differential Revision:  https://reviews.freebsd.org/D10399
> > 
> > Generating core.txt now complety broken?
> 
> No.  crashinfo has supported gdb from ports for quite a while now.
> If you 'pkg install gdb' crashinfo defaults to using the ports gdb over
> the base one already.

I am about clean install, w/o ports.
Also, how to generate core.txt after crash, reboot and install gdb
from ports? (port instaled after crash)
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r317094 - head/share/mk

[Slawa Olhovchenkov]

On Tue, Apr 18, 2017 at 04:27:48PM +, John Baldwin wrote:

> Author: jhb
> Date: Tue Apr 18 16:27:48 2017
> New Revision: 317094
> URL: https://svnweb.freebsd.org/changeset/base/317094
> 
> Log:
>   Disable in-tree GDB by default on x86, mips, and powerpc.
>   
>   GDB in ports contains all of the functionality as GDB in base
>   (including kgdb) for these platforms along with additional
>   functionality.  In-tree GDB remains enabled on ARM and sparc64.
>   GDB in ports does not currently support kernel debugging on arm,
>   and ports GDB for sparc64 has not been tested (though it does
>   include sparc64 support).
>   
>   Reviewed by:bdrewery, emaste, imp
>   Relnotes:   yes
>   Sponsored by:   DARPA / AFRL
>   Differential Revision:  https://reviews.freebsd.org/D10399

Generating core.txt now complety broken?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Slawa Olhovchenkov]

On Sat, Apr 15, 2017 at 07:00:57AM -0700, Rodney W. Grimes wrote:

> > On Fri, Apr 14, 2017 at 03:05:25PM -0700, Mark Johnston wrote:
> > 
> > > > And with textdumps available, the benefit
> > > > of having compression is limited because we can request for minidump
> > > > or full dumps only when the textdumps are not good enough for
> > > > diagnosing the kernel bug.
> > > 
> > > Sure, but in this case the compression may be vital.
> > > 
> > > > 
> > > > I don't think security (e.g. leaking information because of the use of
> > > > compression) is a very big concern in this context because in order
> > > > for the potential attacker to read the raw material needs a
> > > > compromised system (unlike an attack from the network, where someone
> > > > who controls the network would have access to the raw material); the
> > > > dump is usually quite large, and measuring downtime would be hard at
> > > > that scale.
> > > 
> > > Ok.
> > > 
> > > > 
> > > > By the way (not meant to bikeshed) if I was to do this I'd prefer
> > > > using lz4 or something that compresses faster than zlib.
> > > 
> > > I agree, but I think the existing lz4 implementation in the kernel is
> > > not so well suited to running after a panic. It seems fixable, but
> > > compression speed also isn't hugely important here IMO.
> > 
> > On production system this is downtime.
> > For may case, dumped about 32GB (from 256GB RAM). This is take several
> > minutes. Can compression increase this to hour?
> 
> On productions systems the compression layer of dump may very well
> be a win situation depending on choosen algorith (you want something
> fairly fast, but still effective).  If your rate to compress bytes
> is close to the disk write bandwith you have an over all win caused
> by writting less to disk.
> 
> Someone who enjoys math should write an equation for given
> compression bandwidth cb and given disk bandwidth db and
> compression ratio cr what do the curves look like? 
> 
> IIRC we measure cpu/memory bandwidth in the 10'sG bytes/sec

wrong. single thread cpu/memory bandwidth very different on i7 and e5
cpu (e5 less, about 6 GB/s).

> range, compression should be some place under that, and
> our disk bandwidth even on SSD is in the 500MB range,
> even the fastest 15k rpm spinning rust is in the <200MB
> range, we should be able to compress at a higher rate than
> this.

yes.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Slawa Olhovchenkov]

On Fri, Apr 14, 2017 at 03:05:25PM -0700, Mark Johnston wrote:

> > And with textdumps available, the benefit
> > of having compression is limited because we can request for minidump
> > or full dumps only when the textdumps are not good enough for
> > diagnosing the kernel bug.
> 
> Sure, but in this case the compression may be vital.
> 
> > 
> > I don't think security (e.g. leaking information because of the use of
> > compression) is a very big concern in this context because in order
> > for the potential attacker to read the raw material needs a
> > compromised system (unlike an attack from the network, where someone
> > who controls the network would have access to the raw material); the
> > dump is usually quite large, and measuring downtime would be hard at
> > that scale.
> 
> Ok.
> 
> > 
> > By the way (not meant to bikeshed) if I was to do this I'd prefer
> > using lz4 or something that compresses faster than zlib.
> 
> I agree, but I think the existing lz4 implementation in the kernel is
> not so well suited to running after a panic. It seems fixable, but
> compression speed also isn't hugely important here IMO.

On production system this is downtime.
For may case, dumped about 32GB (from 256GB RAM). This is take several
minutes. Can compression increase this to hour?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Slawa Olhovchenkov]

On Fri, Apr 14, 2017 at 01:45:22PM -0700, Ngie Cooper (yaneurabeya) wrote:

> 
> > On Apr 14, 2017, at 13:42, Slawa Olhovchenkov <s...@zxy.spb.ru> wrote:
> 
> …
> 
> > No, something like:
> > % swapinfo
> > Device  1K-blocks UsedAvail Capacity
> > /dev/ada0p2  335544320 33554432 0%
> > /dev/ada1p2  335544320 33554432 0%
> > Total671088640 67108864 0%
> 
> Striping dumps across multiple swap devices isn’t supported. Sorry.

What about support in the future?

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Slawa Olhovchenkov]

On Fri, Apr 14, 2017 at 01:33:36PM -0700, Ngie Cooper (yaneurabeya) wrote:

> 
> > On Apr 14, 2017, at 13:26, Larry Rosenman <l...@lerctr.org> wrote:
> > 
> > On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
> > <owner-svn-src-...@freebsd.org on behalf of yaneurab...@gmail.com> wrote:
> > 
> >> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov <s...@zxy.spb.ru> wrote:
> >> 
> >> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
> >> 
> >>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper <n...@freebsd.org> wrote:
> >>>> Author: ngie
> >>>> Date: Fri Apr 14 19:41:48 2017
> >>>> New Revision: 316938
> >>>> URL: https://svnweb.freebsd.org/changeset/base/316938
> >>>> 
> >>>> Log:
> >>>> savecore: fix space calculation with respect to `minfree` in 
> >>>> check_space(..)
> >>>> 
> >>>> - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> >>>>   representable data to INT_MAX. Check the values received from
> >>>>   strtoll(3), trimming trailing whitespace off the end to maintain
> >>>>   POLA.
> >>>> - Use `KiB` instead of `kB` when describing free space, total space,
> >>>>   etc. I am now fully aware of `KiB` being the IEC standard for 1024
> >>>>   bytes and `kB` being the IEC standard for 1000 bytes.
> >>>> - Store available number of KiB in `available` so it can be more
> >>>>   easily queried and compared to ensure that there are enough KiB to
> >>>>   store the dump image on disk.
> >>>> - Print out the reserved space on disk, per `minfree`, so end-users
> >>>>   can troubleshoot why check_space(..) is reporting that there isn't
> >>>>   enough free space.
> >>>> 
> >>>> MFC after:7 weeks
> >>>> Reviewed by:  Anton Rang <r...@acm.com> (earlier diff), cem (earlier 
> >>>> diff)
> >>>> Tested with:  positive/negative cases (see review); make tinderbox
> >>>> Sponsored by: Dell EMC Isilon
> >>>> Differential Revision:D10379
> >>> 
> >>> The free space calculation is still uselessly conservative, because it
> >>> doesn't account for the fact that core dumps will always be either
> >>> spare or compressed.  The result is that savecore will frequently
> >>> refuse to save corefiles even when there's plenty of space.  I
> >>> proposed removing the space check altogether in
> >>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> >>> after the compressed core dump feature was merged, because then mostly
> >>> accurate space checks will be possible.  AFAIK the compressed core
> >>> dump feature still hasn't been finished.
> >> 
> >> Is posible (in the future) to use multiple swaps (on multiple disks)
> >> for save core dumps?
> > 
> >Multiple swap devices is already handled by savecore(8), if one uses 
> > fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
> > devices.
> > 
> >As far as saving to multiple disks is concerned, I would hope that one 
> > is using a redundancy capable filesystem (zfs) or RAID-like technology 
> > (gmirror, graid, LSI Fusion’s RAID product line) to stripe and/or mirror 
> > the data across multiple disks.
> 
> …
> 
> > How do I use multiple devices to have the system dump on all of my swap?  I 
> > got a message about not enough space, but there (I think) was enough 
> > between multiple drives….
> 
> Something like:
> 
> - Create a zpool
> - Mount zpool to /crashdumps
> - Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
> ‘dumpdir=/crashdumps’
> 
> ?

No, something like:
% swapinfo
Device  1K-blocks UsedAvail Capacity
/dev/ada0p2  335544320 33554432 0%
/dev/ada1p2  335544320 33554432 0%
Total671088640 67108864 0%

> 
> PS The issue with lack of space might be the issue that Alan brought up 
> earlier with compressed dumps and overly conservative free space checks, or 
> it might be the fact that dumpdir (default: /var/crash) is full.


___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Slawa Olhovchenkov]

On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:

> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
> > Author: ngie
> > Date: Fri Apr 14 19:41:48 2017
> > New Revision: 316938
> > URL: https://svnweb.freebsd.org/changeset/base/316938
> >
> > Log:
> >   savecore: fix space calculation with respect to `minfree` in 
> > check_space(..)
> >
> >   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> > representable data to INT_MAX. Check the values received from
> > strtoll(3), trimming trailing whitespace off the end to maintain
> > POLA.
> >   - Use `KiB` instead of `kB` when describing free space, total space,
> > etc. I am now fully aware of `KiB` being the IEC standard for 1024
> > bytes and `kB` being the IEC standard for 1000 bytes.
> >   - Store available number of KiB in `available` so it can be more
> > easily queried and compared to ensure that there are enough KiB to
> > store the dump image on disk.
> >   - Print out the reserved space on disk, per `minfree`, so end-users
> > can troubleshoot why check_space(..) is reporting that there isn't
> > enough free space.
> >
> >   MFC after:7 weeks
> >   Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
> >   Tested with:  positive/negative cases (see review); make tinderbox
> >   Sponsored by: Dell EMC Isilon
> >   Differential Revision:D10379
> 
> The free space calculation is still uselessly conservative, because it
> doesn't account for the fact that core dumps will always be either
> spare or compressed.  The result is that savecore will frequently
> refuse to save corefiles even when there's plenty of space.  I
> proposed removing the space check altogether in
> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> after the compressed core dump feature was merged, because then mostly
> accurate space checks will be possible.  AFAIK the compressed core
> dump feature still hasn't been finished.

Is posible (in the future) to use multiple swaps (on multiple disks)
for save core dumps?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316699 - head/sys/net

[Slawa Olhovchenkov]

On Tue, Apr 11, 2017 at 11:49:56AM -0700, Rodney W. Grimes wrote:

> [ Charset UTF-8 unsupported, converting... ]
> > On 11.04.2017 19:51, Rodney W. Grimes wrote:
> > >> Modified: head/sys/net/if_gre.c
> > >> ==
> > >> --- head/sys/net/if_gre.cTue Apr 11 08:29:12 2017
> > >> (r316698)
> > >> +++ head/sys/net/if_gre.cTue Apr 11 08:56:18 2017
> > >> (r316699)
> > >> @@ -88,7 +88,7 @@ __FBSDID("$FreeBSD$");
> > >>  #include 
> > >>  #include 
> > >>  
> > >> -#define GREMTU  1500
> > >> +#define GREMTU  1476
> > > 
> > > I would of thought that this was
> > > #define   GREMTU  ETHERMTU
> > > and now should probably be:
> > > #define   GREMTU  ETHERMTU - gre_hlen; 
> > > Or what ever the approprite sizeof(foo) is;
> > 
> > I just returned back constants that were here :)
> 
> Can we try to improve on this, or maybe just ignore it?
> 
> > The man page says:
> >  "The MTU of gre interfaces is set to 1476 by default, to match the
> >   value used by Cisco routers."
> 
> That might need some clean up too?
> 
> > > Isn't this arguably wrong in the face of JumboFrames?
> > >
> > 
> > I don't see why it may be wrong for jumbo frames. The default MTU value
> > used by any interfaces is ETHERMTU, if you know that your NIC supports
> > jumbo frames, you will configure specific MTU value.
> > If you want 9k MTU on gre(4), you should configure it.
> 
> I may some day want to #define ETHERMTU 9000 and just have all the right
> magic happen, wouldnt that be nice?

Not before 2038.

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r315662 - in head: contrib/bsnmp/snmp_mibII contrib/ipfilter/ipsend lib/libprocstat sys/netinet sys/sys usr.bin/netstat usr.bin/sockstat usr.bin/systat usr.sbin/tcpdrop usr.sbin/trpt

[Slawa Olhovchenkov]

On Tue, Mar 21, 2017 at 01:40:22PM -0700, Gleb Smirnoff wrote:

> On Tue, Mar 21, 2017 at 10:52:02PM +0300, Slawa Olhovchenkov wrote:
> S> > On Tue, Mar 21, 2017 at 10:40:34AM -0700, John Baldwin wrote:
> S> > J> First, this is a very good change and long overdue in divorcing the
> S> > J> user-facing structure for live system reporting vs the kernel 
> structure.
> S> > J> 
> S> > J> However, I realize you don't use info from netstat when debugging 
> kernel
> S> > J> crash dumps, but other people _do_.  It's ok if the kvm bits of 
> netstat
> S> > J> require a matching kernel and thus require recompiling everytime the 
> ABI
> S> > J> changes, but it is useful to have them.  Please restore those.
> S> > 
> S> > I have very much anticipated this comment from you, John.
> S> > 
> S> > I would like to remind you, that we have had this very exact conversation
> S> > back when I removed kvm support from netstat/route.c. Let me search the
> S> > archives:
> S> > 
> S> > https://lists.freebsd.org/pipermail/svn-src-head/2015-April/070480.html
> S> > 
> S> > This conversation has had a continuation on IRC, which I don't archive.
> S> > 
> S> > AFAIR, first I told that with all my involvement into networking stack,
> S> > I never ever had experienced a need to run route stats on a core. The
> S> > debugger were the only useful tool. And that opinion was seconded by
> S> > other network hackers. Then we discussed that a proper tool chould use
> S> > dynamic type parsing and not kvm(3). You said that future gdb has python
> S> > scripting and that would work fine. Meanwhile, you insisted that I 
> restore
> S> > the functionality. I resisted to put kvm(3) back into netstat/route.c, 
> and
> S> > instead I created a gdb script that prints exactly what 'nestat -anr -M 
> core'
> S> > prints. And I committed the script just to satisfy your demand:
> S> > 
> S> > tools/debugscripts/netstat-anr.gdb
> S> > 
> S> > Can you please fairly answer, have you (or anyone else) ever used the
> S> > script during these 2 years?
> S> > 
> S> > I believe, the inpcb/tcpcb printing from a core functionality has the
> S> > same level of real usefulness. I could create the same script for pcbs,
> S> > and I am afraid it is going to share fate of netstat-anr.gdb.
> S> 
> S> How long run this script for core file w/ 40K TCP connections?
> 
> You tell me. When I analyze cores, I don't read through 40K TCP connections.

Real, frech core:

# grep netstat /var/crash/core.txt.3
netstat -s
netstat -m
netstat: memstat_kvm_all: invalid address (0x0)
netstat -anA
netstat -aL
# grep -c tcp4 /var/crash/core.txt.3
38172

This created automaticly at reboot and boot paused until core.txt
created.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r315662 - in head: contrib/bsnmp/snmp_mibII contrib/ipfilter/ipsend lib/libprocstat sys/netinet sys/sys usr.bin/netstat usr.bin/sockstat usr.bin/systat usr.sbin/tcpdrop usr.sbin/trpt

[Slawa Olhovchenkov]

On Tue, Mar 21, 2017 at 01:07:42PM -0700, Cy Schubert wrote:

> In message <20170321195202.gf86...@zxy.spb.ru>, Slawa Olhovchenkov writes:
> > On Tue, Mar 21, 2017 at 11:50:04AM -0700, Gleb Smirnoff wrote:
> > 
> > >   John,
> > > 
> > > On Tue, Mar 21, 2017 at 10:40:34AM -0700, John Baldwin wrote:
> > > J> First, this is a very good change and long overdue in divorcing the
> > > J> user-facing structure for live system reporting vs the kernel 
> > > structure.
> > > J> 
> > > J> However, I realize you don't use info from netstat when debugging 
> > > kernel
> > > J> crash dumps, but other people _do_.  It's ok if the kvm bits of netstat
> > > J> require a matching kernel and thus require recompiling everytime the 
> > > ABI
> > > J> changes, but it is useful to have them.  Please restore those.
> > > 
> > > I have very much anticipated this comment from you, John.
> > > 
> > > I would like to remind you, that we have had this very exact conversation
> > > back when I removed kvm support from netstat/route.c. Let me search the
> > > archives:
> > > 
> > > https://lists.freebsd.org/pipermail/svn-src-head/2015-April/070480.html
> > > 
> > > This conversation has had a continuation on IRC, which I don't archive.
> > > 
> > > AFAIR, first I told that with all my involvement into networking stack,
> > > I never ever had experienced a need to run route stats on a core. The
> > > debugger were the only useful tool. And that opinion was seconded by
> > > other network hackers. Then we discussed that a proper tool chould use
> > > dynamic type parsing and not kvm(3). You said that future gdb has python
> > > scripting and that would work fine. Meanwhile, you insisted that I restore
> > > the functionality. I resisted to put kvm(3) back into netstat/route.c, and
> > > instead I created a gdb script that prints exactly what 'nestat -anr -M 
> > > cor
> > e'
> > > prints. And I committed the script just to satisfy your demand:
> > > 
> > > tools/debugscripts/netstat-anr.gdb
> > > 
> > > Can you please fairly answer, have you (or anyone else) ever used the
> > > script during these 2 years?
> > > 
> > > I believe, the inpcb/tcpcb printing from a core functionality has the
> > > same level of real usefulness. I could create the same script for pcbs,
> > > and I am afraid it is going to share fate of netstat-anr.gdb.
> > 
> > How long run this script for core file w/ 40K TCP connections?
> 
> Would it be possible to replace this script with new DTrace probes and a 
> DTrace script?

DTrace probes in /etc/rc.d/savecore? Realy?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r315662 - in head: contrib/bsnmp/snmp_mibII contrib/ipfilter/ipsend lib/libprocstat sys/netinet sys/sys usr.bin/netstat usr.bin/sockstat usr.bin/systat usr.sbin/tcpdrop usr.sbin/trpt

[Slawa Olhovchenkov]

On Tue, Mar 21, 2017 at 11:50:04AM -0700, Gleb Smirnoff wrote:

>   John,
> 
> On Tue, Mar 21, 2017 at 10:40:34AM -0700, John Baldwin wrote:
> J> First, this is a very good change and long overdue in divorcing the
> J> user-facing structure for live system reporting vs the kernel structure.
> J> 
> J> However, I realize you don't use info from netstat when debugging kernel
> J> crash dumps, but other people _do_.  It's ok if the kvm bits of netstat
> J> require a matching kernel and thus require recompiling everytime the ABI
> J> changes, but it is useful to have them.  Please restore those.
> 
> I have very much anticipated this comment from you, John.
> 
> I would like to remind you, that we have had this very exact conversation
> back when I removed kvm support from netstat/route.c. Let me search the
> archives:
> 
> https://lists.freebsd.org/pipermail/svn-src-head/2015-April/070480.html
> 
> This conversation has had a continuation on IRC, which I don't archive.
> 
> AFAIR, first I told that with all my involvement into networking stack,
> I never ever had experienced a need to run route stats on a core. The
> debugger were the only useful tool. And that opinion was seconded by
> other network hackers. Then we discussed that a proper tool chould use
> dynamic type parsing and not kvm(3). You said that future gdb has python
> scripting and that would work fine. Meanwhile, you insisted that I restore
> the functionality. I resisted to put kvm(3) back into netstat/route.c, and
> instead I created a gdb script that prints exactly what 'nestat -anr -M core'
> prints. And I committed the script just to satisfy your demand:
> 
> tools/debugscripts/netstat-anr.gdb
> 
> Can you please fairly answer, have you (or anyone else) ever used the
> script during these 2 years?
> 
> I believe, the inpcb/tcpcb printing from a core functionality has the
> same level of real usefulness. I could create the same script for pcbs,
> and I am afraid it is going to share fate of netstat-anr.gdb.

How long run this script for core file w/ 40K TCP connections?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r315653 - in head: lib/libstand sys/boot/common sys/boot/i386/libi386

[Slawa Olhovchenkov]

On Tue, Mar 21, 2017 at 05:53:12PM +0200, Toomas Soome wrote:

> 
> > On 21. märts 2017, at 17:15, Ian Lepore <i...@freebsd.org> wrote:
> > 
> > On Tue, 2017-03-21 at 15:35 +0300, Slawa Olhovchenkov wrote:
> >> On Mon, Mar 20, 2017 at 10:20:17PM +, Toomas Soome wrote:
> >> 
> >>> 
> >>> Author: tsoome
> >>> Date: Mon Mar 20 22:20:17 2017
> >>> New Revision: 315653
> >>> URL: https://svnweb.freebsd.org/changeset/base/315653
> >>> 
> >>> Log:
> >>>   loader: verify the value from dhcp.interface-mtu and use snprintf
> >>> to set mtu
> >>>   
> >>>   Since the uset can set dhcp.interface-mtu, we need to try to
> >>> validate the
> >>>   value. So we verify if the conversion to int is successful and we
> >>> will not
> >>>   allow to set value greater than max IPv4 packet size.
> >>>   
> >>> + tmp > USHRT_MAX) {
> >>> + printf("%s: bad value:
> >>> \"%s\", "
> >>> + "ignoring\n",
> >>> + "dhcp.interface-mtu",
> >>> val);
> >> ===
> >> USHRT_MAX  Maximum value for an object of type unsigned short
> >> int
> >> 65535 (216-1) or greater*
> >> 
> >> * the actual value depends on the particular system and library
> >> implementation, but shall reflect the limits of these types in the
> >> target platform.
> >> ===
> >> 
> >> I mean IF_MAXMTU more correct.
> >> 
> > 
> > The context here is libstand; because it is standalone by design, the
> > code doesn't have access to IF_MAXMTU or other kernel/userland
> > constants.  There is also no question that a short is 16 bits or that
> > USHRT_MAX will be anything other than 65535 in that environment.  If
> > some platform did appear that had a different-sized short by default,
> > we would add whatever flags are necessary to force it back to 16 bits
> > in src/share/mk/bsd.stand.mk.
> > 
> 
> 
> Also note the “upper” value is entirely fictional - we felt we need to pick 
> some “sane” default, with current (common 1Gb/s) ethernet hardware you will 
> be in trouble long before reaching that value;)

18000?
IMHO, max found in different spec is about 16K.
Mostly jumbo is 9000-9198.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r315653 - in head: lib/libstand sys/boot/common sys/boot/i386/libi386

[Slawa Olhovchenkov]

On Mon, Mar 20, 2017 at 10:20:17PM +, Toomas Soome wrote:

> Author: tsoome
> Date: Mon Mar 20 22:20:17 2017
> New Revision: 315653
> URL: https://svnweb.freebsd.org/changeset/base/315653
> 
> Log:
>   loader: verify the value from dhcp.interface-mtu and use snprintf to set mtu
>   
>   Since the uset can set dhcp.interface-mtu, we need to try to validate the
>   value. So we verify if the conversion to int is successful and we will not
>   allow to set value greater than max IPv4 packet size.
>   
> + tmp > USHRT_MAX) {
> + printf("%s: bad value: \"%s\", "
> + "ignoring\n",
> + "dhcp.interface-mtu", val);

===
USHRT_MAX   Maximum value for an object of type unsigned short int
65535 (216-1) or greater*

* the actual value depends on the particular system and library
implementation, but shall reflect the limits of these types in the
target platform.
===

I mean IF_MAXMTU more correct.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r314274 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs

[Slawa Olhovchenkov]

On Tue, Mar 07, 2017 at 11:36:34AM +0200, Andriy Gapon wrote:

> On 07/03/2017 11:29, Slawa Olhovchenkov wrote:
> > You are kidding?! How to work this commit w/o r307265?
> > r307265 is l2arc compression suport (buggly).
> > this commit fix l2arc part.
> > arc part still buggly.
> > 
> > hmm, may be I am not clean, I am not rever ALL to r307265. I am rever
> > exact r307265 and 314274. This is like bisect by one try.
> 
> I am not sure what you are saying...
> Do you see the problem with just r307265?
> Or with r307265 + r314274 ?

r307265 + r314274
Don't try r307265 (by bad feedbacks).
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r314274 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs

[Slawa Olhovchenkov]

On Tue, Mar 07, 2017 at 11:02:46AM +0200, Andriy Gapon wrote:

> On 07/03/2017 05:53, Slawa Olhovchenkov wrote:
> > On Sat, Feb 25, 2017 at 05:03:49PM +, Andriy Gapon wrote:
> > 
> >> Author: avg
> >> Date: Sat Feb 25 17:03:48 2017
> >> New Revision: 314274
> >> URL: https://svnweb.freebsd.org/changeset/base/314274
> >>
> >> Log:
> >>   l2arc: try to fix write size calculation broken by Compressed ARC commit
> >>   
> >>   While there, make a change to not evict a first buffer outside the
> >>   requested eviciton range.
> >>   
> >>   To do:
> >>   - give more consistent names to the size variables
> >>   - upstream to OpenZFS
> > 
> > Mem: 6647M Active, 8399M Inact, 87G Wired, 49K Buf, 23G Free
> > ^
> > ARC: 130G Total, 255M MFU, 80G MRU, 267M Anon, 72M Header, 41M Other
> >  ^
> > 
> > 128GB phys mem
> > 
> 
> Why this commit?

revert this commit and r307265 resolve this.
i.e. may be r307265, but r307265 related to this.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r314274 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs

[Slawa Olhovchenkov]

On Sat, Feb 25, 2017 at 05:03:49PM +, Andriy Gapon wrote:

> Author: avg
> Date: Sat Feb 25 17:03:48 2017
> New Revision: 314274
> URL: https://svnweb.freebsd.org/changeset/base/314274
> 
> Log:
>   l2arc: try to fix write size calculation broken by Compressed ARC commit
>   
>   While there, make a change to not evict a first buffer outside the
>   requested eviciton range.
>   
>   To do:
>   - give more consistent names to the size variables
>   - upstream to OpenZFS

Mem: 6647M Active, 8399M Inact, 87G Wired, 49K Buf, 23G Free
^
ARC: 130G Total, 255M MFU, 80G MRU, 267M Anon, 72M Header, 41M Other
 ^

128GB phys mem
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r314669 - head/sys/i386/conf

[Slawa Olhovchenkov]

On Sat, Mar 04, 2017 at 03:04:17PM +, Pedro F. Giffuni wrote:

> Author: pfg
> Date: Sat Mar  4 15:04:17 2017
> New Revision: 314669
> URL: https://svnweb.freebsd.org/changeset/base/314669
> 
> Log:
>   Drop i486 from the default i386 GENERIC kernel configuration.
>   
>   80486 production was stopped by Intel on September 2007. Dropping the 486
>   configuration option from the GENERIC kernel improves performance
>   slightly.
>   
>   Removing I486_CPU is consistent at this time: we don't support any
>   processor without a FPU and the PC-98 arch, which frequently involved i486
>   CPUs, is also gone so we don't test such platforms anymore.

What is realy mean?
Some Via CPU is like i486 (by instruction set).

CPU: VIA Ezra (800.04-MHz 686-class CPU)
  Origin="CentaurHauls"  Id=0x678  Family=0x6  Model=0x7  Stepping=8
  Features=0x803035
  AMD Features=0x8000<3DNow!>

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r314036 - head/usr.sbin/bsdinstall/scripts

[Slawa Olhovchenkov]

On Wed, Feb 22, 2017 at 10:13:41AM -0800, Conrad Meyer wrote:

> On Wed, Feb 22, 2017 at 10:05 AM, Slawa Olhovchenkov <s...@zxy.spb.ru> wrote:
> > On Wed, Feb 22, 2017 at 08:11:14AM -0800, Conrad Meyer wrote:
> >
> >> On Wed, Feb 22, 2017 at 3:23 AM, Joel Dahl <j...@vnode.se> wrote:
> >> > On Wed, Feb 22, 2017 at 07:56:52AM +, Bartłomiej Rutkowski wrote:
> >> >> I strongly believe we should, by default, ship as secured and hardened 
> >> >> as
> >> >> possible in order to improve overall security of new users 
> >> >> installations.
> >> >> Power users will and do change the OS as they please, they most likely
> >> >> don't use bsdinstall in first place, so they're not affected in any way.
> >> >
> >> > Sorry, I strongly disagree with that. I'm most likely a "power user" and 
> >> > I use
> >> > bsdinstall.
> >>
> >> Ditto.  I'm also unfamiliar enough with the installer to trip on this
> >> kind of thing.  Slawa's proposed "disable all" option would be fine.
> >
> > My english not enought fluent for more explicate proposal, from my
> > point most of this options do hardened in only limited cases, for
> > other cases same options do system more un-hardened by force working
> > as root. Some have unevident effects (/tmp cleaning, for example).
> 
> Yep.  I am not concerned about disabling sendmail or remote syslog by
> default, though.

Also, what mean by 'disabling remote syslog'?
As I know syslogd by default don't collect remote messages and need -a
options. May be this is about -s options? How many -s? Not clean.

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r314036 - head/usr.sbin/bsdinstall/scripts

[Slawa Olhovchenkov]

On Wed, Feb 22, 2017 at 08:11:14AM -0800, Conrad Meyer wrote:

> On Wed, Feb 22, 2017 at 3:23 AM, Joel Dahl  wrote:
> > On Wed, Feb 22, 2017 at 07:56:52AM +, Bartłomiej Rutkowski wrote:
> >> I strongly believe we should, by default, ship as secured and hardened as
> >> possible in order to improve overall security of new users installations.
> >> Power users will and do change the OS as they please, they most likely
> >> don't use bsdinstall in first place, so they're not affected in any way.
> >
> > Sorry, I strongly disagree with that. I'm most likely a "power user" and I 
> > use
> > bsdinstall.
> 
> Ditto.  I'm also unfamiliar enough with the installer to trip on this
> kind of thing.  Slawa's proposed "disable all" option would be fine.

My english not enought fluent for more explicate proposal, from my
point most of this options do hardened in only limited cases, for
other cases same options do system more un-hardened by force working
as root. Some have unevident effects (/tmp cleaning, for example).

For many users this options will be source of weird issuses (gdb don't
work? fucking ugly freebsd! migrate to linux).

This is evil trend of enforcing weird solutions under the auspices of
'my safety': airport security check, backgound check on every point,
lawfull intercept, block access to hardware management in safety
enviroment by 'leak ecnription'. I am enoght smart for self-sufficient
security risk assessment!

Industry already have at some "hardened" BSD: OpenBSD and HardenedBSD.
Waht about market share?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r314036 - head/usr.sbin/bsdinstall/scripts

[Slawa Olhovchenkov]

On Tue, Feb 21, 2017 at 09:37:34AM +, Bartek Rutkowski wrote:

> Author: robak (ports committer)
> Date: Tue Feb 21 09:37:33 2017
> New Revision: 314036
> URL: https://svnweb.freebsd.org/changeset/base/314036
> 
> Log:
>   Enable bsdinstall hardening options by default.
>   
>   As discussed previously, in order to introduce new OS hardening
>   defaults, we've added them to bsdinstall in 'off by default' mode.
>   It has been there for a while, so the next step is to change them
>   to 'on by defaul' mode, so that in future we could simply enable
>   them in base OS.

Please include option "disable all" for simple disable all.

>   Reviewed by:brd
>   Approved by:adrian
>   Differential Revision:  https://reviews.freebsd.org/D9641
> 
> Modified:
>   head/usr.sbin/bsdinstall/scripts/hardening
> 
> Modified: head/usr.sbin/bsdinstall/scripts/hardening
> ==
> --- head/usr.sbin/bsdinstall/scripts/hardeningTue Feb 21 09:33:21 
> 2017(r314035)
> +++ head/usr.sbin/bsdinstall/scripts/hardeningTue Feb 21 09:37:33 
> 2017(r314036)
> @@ -36,15 +36,15 @@ FEATURES=$( dialog --backtitle "FreeBSD 
>  --title "System Hardening" --nocancel --separate-output \
>  --checklist "Choose system security hardening options:" \
>  0 0 0 \
> - "0 hide_uids" "Hide processes running as other users" ${hide_uids:-off} 
> \
> - "1 hide_gids" "Hide processes running as other groups" 
> ${hide_gids:-off} \
> - "2 read_msgbuf" "Disable reading kernel message buffer for unprivileged 
> users" ${read_msgbuf:-off} \
> - "3 proc_debug" "Disable process debugging facilities for unprivileged 
> users" ${proc_debug:-off} \
> - "4 random_pid" "Randomize the PID of newly created processes" 
> ${random_pid:-off} \
> - "5 stack_guard" "Insert stack guard page ahead of the growable 
> segments" ${stack_guard:-off} \
> - "6 clear_tmp" "Clean the /tmp filesystem on system startup" 
> ${clear_tmp:-off} \
> - "7 disable_syslogd" "Disable opening Syslogd network socket (disables 
> remote logging)" ${disable_syslogd:-off} \
> - "8 disable_sendmail" "Disable Sendmail service" 
> ${disable_sendmail:-off} \
> + "0 hide_uids" "Hide processes running as other users" ${hide_uids:-on} \
> + "1 hide_gids" "Hide processes running as other groups" ${hide_gids:-on} 
> \
> + "2 read_msgbuf" "Disable reading kernel message buffer for unprivileged 
> users" ${read_msgbuf:-on} \
> + "3 proc_debug" "Disable process debugging facilities for unprivileged 
> users" ${proc_debug:-on} \
> + "4 random_pid" "Randomize the PID of newly created processes" 
> ${random_pid:-on} \
> + "5 stack_guard" "Insert stack guard page ahead of the growable 
> segments" ${stack_guard:-on} \
> + "6 clear_tmp" "Clean the /tmp filesystem on system startup" 
> ${clear_tmp:-on} \
> + "7 disable_syslogd" "Disable opening Syslogd network socket (disables 
> remote logging)" ${disable_syslogd:-on} \
> + "8 disable_sendmail" "Disable Sendmail service" ${disable_sendmail:-on} 
> \
>  2>&1 1>&3 )
>  exec 3>&-
>  
> ___
> svn-src-...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/svn-src-all
> To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r313330 - in head: contrib/netcat lib/libipsec sbin/ifconfig sbin/setkey share/man/man4 sys/conf sys/modules sys/modules/ipsec sys/modules/tcp/tcpmd5 sys/net sys/netinet sys/netinet/tc

[Slawa Olhovchenkov]

On Tue, Feb 07, 2017 at 03:53:05AM +0300, Andrey V. Elsukov wrote:

> On 06.02.2017 17:31, Dmitry Morozovsky wrote:
> >> Date: Mon Feb  6 08:49:57 2017
> >> New Revision: 313330
> >> URL: https://svnweb.freebsd.org/changeset/base/313330
> >>
> >> Log:
> >>   Merge projects/ipsec into head/.
> >
> > [snip]
> >
> > Great, thanks!
> >
> > Have you any plans to merge this into stable/11 to reduce diffs in network
> > stack code?
> 
> It depends from the further users feedback.
> I wanted to do MFC after one or two months. But there are two things 
> that are questionable. The date of stable/11 feature freeze is not 
> known. And there is also some changes that can be considered as POLA 
> violations. E.g. now SPIs are unique, and if user had manually 
> configured SAs with the same SPI, the MFC will break this.

What about IKE? I am don't know, do IKE SPI number negotiation?
Or remote side just assign implicit SPI? In last case posible race on
local system.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r313318 - in head: share/man/man4 sys/dev/cxgbe

[Slawa Olhovchenkov]

On Mon, Feb 06, 2017 at 05:19:30AM +, Navdeep Parhar wrote:

> Author: np
> Date: Mon Feb  6 05:19:29 2017
> New Revision: 313318
> URL: https://svnweb.freebsd.org/changeset/base/313318
> 
> Log:
>   cxgbe(4):  Allow tunables that control the number of queues to be set to
>   '-n' to tell the driver to create _up to_ 'n' queues if enough cores are
>   available.  For example, setting hw.cxgbe.nrxq10g="-32" will result in
>   16 queues if the system has 16 cores, 32 if it has 32.
>   
>   There is no change in the default number of queues of any type.

Just for my info: how many queues supported by different hardware (T4/T5/T6)?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r313275 - in head/sys: kern sys

[Slawa Olhovchenkov]

On Sun, Feb 05, 2017 at 11:01:28AM +0100, Mateusz Guzik wrote:

> On Sun, Feb 05, 2017 at 01:00:11PM +0300, Slawa Olhovchenkov wrote:
> > On Sun, Feb 05, 2017 at 08:04:12AM +, Mateusz Guzik wrote:
> > 
> > > Author: mjg
> > > Date: Sun Feb  5 08:04:11 2017
> > > New Revision: 313275
> > > URL: https://svnweb.freebsd.org/changeset/base/313275
> > > 
> > > Log:
> > >   mtx: move lockstat handling out of inline primitives
> > >   
> > >   Lockstat requires checking if it is enabled and if so, calling a 6 
> > > argument
> > >   function. Further, determining whether to call it on unlock requires
> > >   pre-reading the lock value.
> > >   
> > >   This is problematic in at least 3 ways:
> > >   - more branches in the hot path than necessary
> > >   - additional cacheline ping pong under contention
> > >   - bigger code
> > >   
> > >   Instead, check first if lockstat handling is necessary and if so, just 
> > > fall
> > >   back to regular locking routines. For this purpose a new macro is 
> > > introduced
> > >   (LOCKSTAT_PROFILE_ENABLED).
> > >   
> > >   LOCK_PROFILING uninlines all primitives. Fold in the current inline lock
> > >   variant into the _mtx_lock_flags to retain the support. With this change
> > >   the inline variants are not used when LOCK_PROFILING is defined and thus
> > >   can ignore its existence.
> > >   
> > >   This results in:
> > >  textdata bss dec hex filename
> > >   222596671303208 4994976 285578511b3c21b kernel.orig
> > >   217973151303208 4994976 280954991acb40b kernel.patched
> > >   
> > >   i.e. about 3% reduction in text size.
> > >   
> > >   A remaining action is to remove spurious arguments for internal kernel
> > >   consumers.
> > 
> > Do you planed MFC all of this?
> 
> Yes, around march.

Thanks!
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r313275 - in head/sys: kern sys

[Slawa Olhovchenkov]

On Sun, Feb 05, 2017 at 08:04:12AM +, Mateusz Guzik wrote:

> Author: mjg
> Date: Sun Feb  5 08:04:11 2017
> New Revision: 313275
> URL: https://svnweb.freebsd.org/changeset/base/313275
> 
> Log:
>   mtx: move lockstat handling out of inline primitives
>   
>   Lockstat requires checking if it is enabled and if so, calling a 6 argument
>   function. Further, determining whether to call it on unlock requires
>   pre-reading the lock value.
>   
>   This is problematic in at least 3 ways:
>   - more branches in the hot path than necessary
>   - additional cacheline ping pong under contention
>   - bigger code
>   
>   Instead, check first if lockstat handling is necessary and if so, just fall
>   back to regular locking routines. For this purpose a new macro is introduced
>   (LOCKSTAT_PROFILE_ENABLED).
>   
>   LOCK_PROFILING uninlines all primitives. Fold in the current inline lock
>   variant into the _mtx_lock_flags to retain the support. With this change
>   the inline variants are not used when LOCK_PROFILING is defined and thus
>   can ignore its existence.
>   
>   This results in:
>  textdata bss dec hex filename
>   222596671303208 4994976 285578511b3c21b kernel.orig
>   217973151303208 4994976 280954991acb40b kernel.patched
>   
>   i.e. about 3% reduction in text size.
>   
>   A remaining action is to remove spurious arguments for internal kernel
>   consumers.

Do you planed MFC all of this?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r312910 - in head: . etc/etc.pc98 etc/rc.d lib/libsysdecode libexec release release/doc release/doc/en_US.ISO8859-1/hardware release/doc/en_US.ISO8859-1/readme release/doc/share/exampl

[Slawa Olhovchenkov]

On Wed, Feb 01, 2017 at 10:56:24AM -0700, Warner Losh wrote:

> >> > > Fair enough.  I haven't fully put away my 12 axe and am toying with
> >> > > dropping any ISA-only storage and NIC drivers (and perhaps pccard-only
> >> > > as well in that case).  Hardware that wants to use ISA/pccard for
> >> > > storage is probably happier running 4.x anyway.  One question is if we
> >> > > should drop ISA attachments in that case for drivers that support PCI
> >> > > and ISA.  However, there's a fair list of ISA-only adapters that would
> >> > > be a good place to start anyway.  One concern is to not drop any 
> >> > > drivers
> >> >
> >> > ps/2 keyboard, mouse, touchpads, lm sensors, etc?...
> >>
> >> Those aren't storage or NICs.  There are many "legacy" device still present
> >> in modern systems that we obviously have to support.  However, if you have
> >> an ISA Adapter SCSI HBA in an ISA slot, the machine running that is 
> >> probably
> >> happier running 4.x than 12.0.
> >
> > My be I am missread, I think removed ISA bus and all devices attached
> > to ISA bus.
> 
> We will never ever ever remove the ISA bus on the i386 or amd64 ports.
> That's simply not possible due to the legacy busses and the fact that
> many devices are logically attached to the address space used by the
> ISA bus even if there aren't physical plastic ISA slots.

Ah, sorry, this is my missread.
I am don't have any ISA cards in use many years.
Only some ISA COM/LPT/sound cards in far-away box.

PS: again, may be time to move all drivers to modules and load all
GENERIC's devices from loader.conf? /boot/loader have good performance
now.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r312910 - in head: . etc/etc.pc98 etc/rc.d lib/libsysdecode libexec release release/doc release/doc/en_US.ISO8859-1/hardware release/doc/en_US.ISO8859-1/readme release/doc/share/exampl

[Slawa Olhovchenkov]

On Wed, Feb 01, 2017 at 01:45:14PM -0800, John Baldwin wrote:

> On Wednesday, February 01, 2017 03:39:57 PM Mark Linimon wrote:
> > On Wed, Feb 01, 2017 at 08:09:50PM +0300, Slawa Olhovchenkov wrote:
> > > Also, I am think current ports don't build on 4.x.
> > 
> > I will personally guarantee, in writing, that current ports do not build
> > on 4.x, nor have they done so for years.  I personally removed the legacy
> > cruft when 4.11 finally went EOL.
> > 
> > > I am got complains about using ports on 8.x.
> > 
> > I am 99% certain that ports will not work on either 8.x or 9.x.  Legacy
> > cruft was removed at the EOL in each of those cases.
> > 
> > Anyone who think that we can support ports on 4.x, 5.x, 6.x, 7.x, 8.x,
> > 9.x, 10.x, 11.x, and -current, all at the same time, needs to seek medical
> > attention at once.  At the absolute least, that era spans 3 major versions
> > of make(1) and two completely different package implementations.
> 
> I think Slawa's point was that my suggestion that people with old hardware
> should run old releases means that they cannot use a modern ports checkout

Yes.

> with those older systems.  That point is correct, but you probably don't
> want to run newer, more bloated 3rd party software on such gear either.

Not all software bloated, some software need to updated (for example, not sure
about sshd in 4.x system. Or openssl).
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r312910 - in head: . etc/etc.pc98 etc/rc.d lib/libsysdecode libexec release release/doc release/doc/en_US.ISO8859-1/hardware release/doc/en_US.ISO8859-1/readme release/doc/share/exampl

[Slawa Olhovchenkov]

On Wed, Feb 01, 2017 at 03:39:57PM -0600, Mark Linimon wrote:

> On Wed, Feb 01, 2017 at 08:09:50PM +0300, Slawa Olhovchenkov wrote:
> > Also, I am think current ports don't build on 4.x.
> 
> I will personally guarantee, in writing, that current ports do not build
> on 4.x, nor have they done so for years.  I personally removed the legacy
> cruft when 4.11 finally went EOL.
> 
> > I am got complains about using ports on 8.x.
> 
> I am 99% certain that ports will not work on either 8.x or 9.x.  Legacy
> cruft was removed at the EOL in each of those cases.
> 
> Anyone who think that we can support ports on 4.x, 5.x, 6.x, 7.x, 8.x,
> 9.x, 10.x, 11.x, and -current, all at the same time, needs to seek medical
> attention at once.  At the absolute least, that era spans 3 major versions
> of make(1) and two completely different package implementations.

I am assume no problem to use new make on old system.
May be posible use pkg too (not sure about pkg requirements to
syscalls).
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r312910 - in head: . etc/etc.pc98 etc/rc.d lib/libsysdecode libexec release release/doc release/doc/en_US.ISO8859-1/hardware release/doc/en_US.ISO8859-1/readme release/doc/share/exampl

[Slawa Olhovchenkov]

On Wed, Feb 01, 2017 at 08:59:40AM -0800, John Baldwin wrote:

> > > Fair enough.  I haven't fully put away my 12 axe and am toying with
> > > dropping any ISA-only storage and NIC drivers (and perhaps pccard-only
> > > as well in that case).  Hardware that wants to use ISA/pccard for
> > > storage is probably happier running 4.x anyway.  One question is if we
> > > should drop ISA attachments in that case for drivers that support PCI
> > > and ISA.  However, there's a fair list of ISA-only adapters that would
> > > be a good place to start anyway.  One concern is to not drop any drivers
> > 
> > ps/2 keyboard, mouse, touchpads, lm sensors, etc?...
> 
> Those aren't storage or NICs.  There are many "legacy" device still present
> in modern systems that we obviously have to support.  However, if you have
> an ISA Adapter SCSI HBA in an ISA slot, the machine running that is probably
> happier running 4.x than 12.0.

Also, I am think current ports don't build on 4.x.
I am got complains about using ports on 8.x.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r312910 - in head: . etc/etc.pc98 etc/rc.d lib/libsysdecode libexec release release/doc release/doc/en_US.ISO8859-1/hardware release/doc/en_US.ISO8859-1/readme release/doc/share/exampl

[Slawa Olhovchenkov]

On Wed, Feb 01, 2017 at 08:59:40AM -0800, John Baldwin wrote:

> On Wednesday, February 01, 2017 01:35:56 PM Slawa Olhovchenkov wrote:
> > On Tue, Jan 31, 2017 at 02:46:23PM -0800, John Baldwin wrote:
> > 
> > > On Tuesday, January 31, 2017 03:33:55 PM Warner Losh wrote:
> > > > On Tue, Jan 31, 2017 at 3:20 PM, John Baldwin <j...@freebsd.org> wrote:
> > > > > On Saturday, January 28, 2017 02:22:15 AM Takahashi Yoshihiro wrote:
> > > > >> Author: nyan
> > > > >> Date: Sat Jan 28 02:22:15 2017
> > > > >> New Revision: 312910
> > > > >> URL: https://svnweb.freebsd.org/changeset/base/312910
> > > > >>
> > > > >> Log:
> > > > >>   Remove pc98 support completely.
> > > > >>   I thank all developers and contributors for pc98.
> > > > >>
> > > > >>   Relnotes:   yes
> > > > >
> > > > > BTW, my impression was that there are some other device drivers
> > > > > that are effectively PC-98 only (e.g. everything that uses scsi_low.c)
> > > > > but they might have pccard attachments for use with PC-98 laptops?
> > > > >
> > > > > Perhaps Warner might know?
> > > > >
> > > > > It seems stg(4) had PCI variants, but nsp(4), ncv(4), and stg(4)
> > > > > all came from NetBSD/pc98 via PAO.
> > > > 
> > > > These all work correctly on any PC Card machine. The only reason they
> > > > came in this way was because these devices were original marketed only
> > > > in Japan. I've used all these cards with external SCSI drives in the
> > > > past.
> > > > 
> > > > As far as I know, only the if_snc driver, which was removed, is truly
> > > > pc98 specific. It is wired in such a way that cannot be used in ibm-at
> > > > compatible laptops.  IIRC, it had hard-wired memory decode lines that
> > > > landed in the middle of the VGA graphics pages or BIOS low memory
> > > > areas. I have one of these cards still, and it will be detected on my
> > > > laptops, but can't work due to the required mappings.
> > > > 
> > > > Now, there's an different question about whether it is time to retire
> > > > some of the now-ancient SCSI cards from the system, but that's a
> > > > different kettle of fish that's larger than just nsp, ncv and stg.
> > > 
> > > Fair enough.  I haven't fully put away my 12 axe and am toying with
> > > dropping any ISA-only storage and NIC drivers (and perhaps pccard-only
> > > as well in that case).  Hardware that wants to use ISA/pccard for
> > > storage is probably happier running 4.x anyway.  One question is if we
> > > should drop ISA attachments in that case for drivers that support PCI
> > > and ISA.  However, there's a fair list of ISA-only adapters that would
> > > be a good place to start anyway.  One concern is to not drop any drivers
> > 
> > ps/2 keyboard, mouse, touchpads, lm sensors, etc?...
> 
> Those aren't storage or NICs.  There are many "legacy" device still present
> in modern systems that we obviously have to support.  However, if you have
> an ISA Adapter SCSI HBA in an ISA slot, the machine running that is probably
> happier running 4.x than 12.0.

My be I am missread, I think removed ISA bus and all devices attached
to ISA bus.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r312910 - in head: . etc/etc.pc98 etc/rc.d lib/libsysdecode libexec release release/doc release/doc/en_US.ISO8859-1/hardware release/doc/en_US.ISO8859-1/readme release/doc/share/exampl

[Slawa Olhovchenkov]

On Tue, Jan 31, 2017 at 02:46:23PM -0800, John Baldwin wrote:

> On Tuesday, January 31, 2017 03:33:55 PM Warner Losh wrote:
> > On Tue, Jan 31, 2017 at 3:20 PM, John Baldwin  wrote:
> > > On Saturday, January 28, 2017 02:22:15 AM Takahashi Yoshihiro wrote:
> > >> Author: nyan
> > >> Date: Sat Jan 28 02:22:15 2017
> > >> New Revision: 312910
> > >> URL: https://svnweb.freebsd.org/changeset/base/312910
> > >>
> > >> Log:
> > >>   Remove pc98 support completely.
> > >>   I thank all developers and contributors for pc98.
> > >>
> > >>   Relnotes:   yes
> > >
> > > BTW, my impression was that there are some other device drivers
> > > that are effectively PC-98 only (e.g. everything that uses scsi_low.c)
> > > but they might have pccard attachments for use with PC-98 laptops?
> > >
> > > Perhaps Warner might know?
> > >
> > > It seems stg(4) had PCI variants, but nsp(4), ncv(4), and stg(4)
> > > all came from NetBSD/pc98 via PAO.
> > 
> > These all work correctly on any PC Card machine. The only reason they
> > came in this way was because these devices were original marketed only
> > in Japan. I've used all these cards with external SCSI drives in the
> > past.
> > 
> > As far as I know, only the if_snc driver, which was removed, is truly
> > pc98 specific. It is wired in such a way that cannot be used in ibm-at
> > compatible laptops.  IIRC, it had hard-wired memory decode lines that
> > landed in the middle of the VGA graphics pages or BIOS low memory
> > areas. I have one of these cards still, and it will be detected on my
> > laptops, but can't work due to the required mappings.
> > 
> > Now, there's an different question about whether it is time to retire
> > some of the now-ancient SCSI cards from the system, but that's a
> > different kettle of fish that's larger than just nsp, ncv and stg.
> 
> Fair enough.  I haven't fully put away my 12 axe and am toying with
> dropping any ISA-only storage and NIC drivers (and perhaps pccard-only
> as well in that case).  Hardware that wants to use ISA/pccard for
> storage is probably happier running 4.x anyway.  One question is if we
> should drop ISA attachments in that case for drivers that support PCI
> and ISA.  However, there's a fair list of ISA-only adapters that would
> be a good place to start anyway.  One concern is to not drop any drivers

ps/2 keyboard, mouse, touchpads, lm sensors, etc?...

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r312437 - in head/sys/dev: mpr mps

[Slawa Olhovchenkov]

On Thu, Jan 19, 2017 at 09:47:51PM +, Scott Long wrote:

> Author: scottl
> Date: Thu Jan 19 21:47:50 2017
> New Revision: 312437
> URL: https://svnweb.freebsd.org/changeset/base/312437
> 
> Log:
>   Rework the debug print API. Event printing no longer gets special handling.
>   All of the printing from the tables file now has wrappers so that the
>   handling is cleaner and it's possible to print something out (say, during
>   development) without having to fight the global debug flags. This re-org
>   will also make it easier to have the tables be compiled out at build time
>   if desired.
>   
>   Other than fixing some minor bugs, there are no user-visible changes from
>   this change

Can you also do "atomics" print?
Now debugs interleave w/ other outputs and line can split multiple
times.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r307319 - in head/sys/netinet: . tcp_stacks

[Slawa Olhovchenkov]

On Fri, Oct 14, 2016 at 02:57:43PM +, Jonathan T. Looney wrote:

> Author: jtl
> Date: Fri Oct 14 14:57:43 2016
> New Revision: 307319
> URL: https://svnweb.freebsd.org/changeset/base/307319
> 
> Log:
>   The code currently resets the keepalive timer each time a packet is
>   received on a TCP session that has entered the ESTABLISHED state. This
>   results in a lot of calls to reset the keepalive timer.
>   
>   This patch changes the behavior so we set the keepalive timer for the
>   keepalive idle time (TP_KEEPIDLE). When the keepalive timer fires, it will
>   first check to see if the session has been idle for TP_KEEPIDLE ticks. If
>   not, it will reschedule the keepalive timer for the time the session will
>   have been idle for TP_KEEPIDLE ticks.
>   
>   For a session with regular communication, the keepalive timer should fire
>   approximately once every TP_KEEPIDLE ticks. For sessions with irregular
>   communication, the keepalive timer might fire more often. But, the
>   disruption from a periodic keepalive timer should be less than the regular
>   cost of resetting the keepalive timer on every packet.
>   
>   (FWIW, this change saved approximately 1.73% of the busy CPU cycles on a
>   particular test system with a heavy TCP output load. Of course, the
>   actual impact is very specific to the particular hardware and workload.)
>   
>   Reviewed by:gallatin, rrs
>   MFC after:  2 weeks
>   Sponsored by:   Netflix
>   Differential Revision:  https://reviews.freebsd.org/D8243

Some issus exist?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r311952 - head/sys/ddb

[Slawa Olhovchenkov]

On Sun, Jan 15, 2017 at 10:46:35AM -0800, Adrian Chadd wrote:

> url? :)

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=184987
Assignee:   Adrian Chadd

> On 15 January 2017 at 06:45, Slawa Olhovchenkov <s...@zxy.spb.ru> wrote:
> > On Sun, Jan 15, 2017 at 01:07:58PM +0800, Julian Elischer wrote:
> >
> >>
> >> the old DEC machines had (from memory) ^O  which dropped all pending
> >> output.
> >> Not so useful when there is 2MB of network buffers queued up to you,
> >> but very useful on
> >> and asr-33 teletype.
> >
> > Now it useful too, on serial console.
> > ^O processing be present years ago in FreeBSD, but removed at time of
> > tty code rewrite. As I know patch for restore exist, nobody
> > review/commit.
> >
> >
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r311952 - head/sys/ddb

[Slawa Olhovchenkov]

On Sun, Jan 15, 2017 at 01:07:58PM +0800, Julian Elischer wrote:

> 
> the old DEC machines had (from memory) ^O  which dropped all pending 
> output.
> Not so useful when there is 2MB of network buffers queued up to you, 
> but very useful on
> and asr-33 teletype.

Now it useful too, on serial console.
^O processing be present years ago in FreeBSD, but removed at time of
tty code rewrite. As I know patch for restore exist, nobody
review/commit.

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r310023 - in head/sys/cddl: compat/opensolaris/sys contrib/opensolaris/uts/common/fs/zfs

[Slawa Olhovchenkov]

On Wed, Dec 14, 2016 at 02:38:11PM +0200, Andriy Gapon wrote:

> Alexander,
> 
> I tried to see how the code looks after expanding illumos variables to FreeBSD
> variables before and after your change.
> 
> Before:
> n = PAGESIZE * ((int64_t)freemem - zfs_arc_free_target);
> ==>
> n = PAGESIZE * ((int64_t)vm_cnt.v_free_count - zfs_arc_free_target);
> 
> After:
> n = PAGESIZE * (freemem - lotsfree - needfree - desfree);
> ==>
> n = PAGESIZE * ((long)vm_cnt.v_free_count - zfs_arc_free_target -
> (long)vm_pageout_deficit - (long)vm_cnt.v_free_target);
> 
> Default value of zfs_arc_free_target is vm_pageout_wakeup_thresh and its 
> default
> value is (vm_cnt.v_free_min / 10) * 11.
> 
> vm_pageout_deficit is probably just a noise most of the time.

vm_pageout_deficit zeroed at read by vm_pageout_scan().

In illumos case target is "target_free" + "pageout_high_water" + "extra
pages here to make sure the scanner doesn't start up while we're
freeing memory."

As zero-order approximation we can let

target_free as v_free_target (desfree)
pageout_high_water as zfs_arc_free_target (lotsfree)
extra pages as vm_pageout_deficit (needfree)

In that case noise in vm_pageout_deficit is not problem.

> But v_free_target is a substantial value (even greater than the default
> zfs_arc_free_target).
> 
> It seems that now we subtract much more than we did before.

Code was totaly broken in FreeBSD case: needfree don't updated in process of
reclaim. Mostly reclaim stops only after ARC touch arc_min.

> So, this change does not merely reduce diff, it also changes ARC sizing 
> behavior.

And this is right way.

> I wonder how much testing have you done for this change and if you can qualify
> ARC size behavior in various scenarios.  I expect that with your change the 
> ARC
> would more easily give in to the memory pressure.  That may delight some, but 
> it
> could be an issue for others.  Especially if it forces ARC to its minimum size
> for no good reason.
> 
> P.S.
> My impression is that the page daemon in illumos has a different algorithm 
> from
> our page daemon, so some similarities could be misleading rather than helpful.
> 
> -- 
> Andriy Gapon
> ___
> svn-src-...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/svn-src-all
> To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r309703 - in head/sys: amd64/amd64 arm64/arm64 i386/i386 vm

[Slawa Olhovchenkov]

On Thu, Dec 08, 2016 at 04:29:29AM +, Alan Cox wrote:

> Author: alc
> Date: Thu Dec  8 04:29:29 2016
> New Revision: 309703
> URL: https://svnweb.freebsd.org/changeset/base/309703
> 
> Log:
>   Previously, vm_radix_remove() would panic if the radix trie didn't
>   contain a vm_page_t at the specified index.  However, with this
>   change, vm_radix_remove() no longer panics.  Instead, it returns NULL
>   if there is no vm_page_t at the specified index.  Otherwise, it
>   returns the vm_page_t.  The motivation for this change is that it
>   simplifies the use of radix tries in the amd64, arm64, and i386 pmap
>   implementations.  Instead of performing a lookup before every remove,
>   the pmap can simply perform the remove.

Is this performance improvement?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r305852 - in head/sys: dev/cxgbe dev/cxgbe/firmware modules/cxgbe modules/cxgbe/if_cc modules/cxgbe/if_ccv

[Slawa Olhovchenkov]

On Fri, Sep 16, 2016 at 12:08:37AM +, Navdeep Parhar wrote:

> Author: np
> Date: Fri Sep 16 00:08:37 2016
> New Revision: 305852
> URL: https://svnweb.freebsd.org/changeset/base/305852
> 
> Log:
>   cxgbe(4): Attach to cards with the Terminator 6 ASIC.  T6 cards will
>   come up as 't6nex' nexus devices with 'cc' ports hanging off them.
>   
>   The T6 firmware and configuration files will be added as soon as they
>   are released.  For now the driver will try to work with whatever
>   firmware and configuration is on the card's flash.
>   
>   Sponsored by:   Chelsio Communications

No MFC planed?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r307326 - head/sys/boot/efi/loader

[Slawa Olhovchenkov]

On Wed, Oct 19, 2016 at 04:33:39PM -0600, Warner Losh wrote:

> On Wed, Oct 19, 2016 at 11:34 AM, John Baldwin  wrote:
> > On Wednesday, October 19, 2016 02:01:18 AM Warner Losh wrote:
> >> One of my systems would export these as sysctls:
> >>
> >> smbios.bios.reldate="07/05/2013"
> >> smbios.bios.vendor="American Megatrends Inc."
> >> smbios.bios.version="3.00"
> >> smbios.chassis.maker="Supermicro"
> >> smbios.chassis.serial="0123456789"
> >> smbios.chassis.tag="To Be Filled By O.E.M."
> >> smbios.chassis.version="0123456789"
> >> smbios.memory.enabled="268435456"
> >> smbios.planar.location="To be filled by O.E.M."
> >> smbios.planar.maker="Supermicro"
> >> smbios.planar.product="X9SRH-7F/7TF"
> >> smbios.planar.serial="VM13CS028237"
> >> smbios.planar.tag="To be filled by O.E.M."
> >> smbios.planar.version="0123456789"
> >> smbios.socket.enabled="1"
> >> smbios.socket.populated="1"
> >> smbios.system.family="To be filled by O.E.M."
> >> smbios.system.maker="Supermicro"
> >> smbios.system.product="X9SRH-7F/7TF"
> >> smbios.system.serial="0123456789"
> >> smbios.system.sku="To be filled by O.E.M."
> >> smbios.system.uuid="----002590e4d0ec"
> >> smbios.system.version="0123456789"
> >> smbios.version="2.7"
> >>
> >> though some of them are silly due to the BIOS writer being silly...
> >
> > So are you planning to just duplicate the existing kenv vars as sysctl
> > nodes?  Can't you parse the output of kenv instead?
> 
> I often get sysctl -a from ailing systems. I never or rarely get kenv.
> This adds to the information available when I'm looking into problems
> since it gives me more data about the system where the problem occurs.

Look like we need show-tech in base system
:)
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r306680 - in head/sys: amd64/amd64 amd64/include i386/include x86/include x86/x86

[Slawa Olhovchenkov]

On Wed, Oct 12, 2016 at 06:30:09PM +0300, Andriy Gapon wrote:

> On 12/10/2016 16:45, Konstantin Belousov wrote:
> > On Wed, Oct 12, 2016 at 04:25:00PM +0300, Andriy Gapon wrote:
> >> On 04/10/2016 20:01, Konstantin Belousov wrote:
> >>> Author: kib
> >>> Date: Tue Oct  4 17:01:24 2016
> >>> New Revision: 306680
> >>> URL: https://svnweb.freebsd.org/changeset/base/306680
> >>>
> >>> Log:
> >>>   Re-apply r306516 (by cem):
> >>>   
> >>>   Reduce the cost of TLB invalidation on x86 by using per-CPU completion 
> >>> flags
> >>>   
> >>>   Reduce contention during TLB invalidation operations by using a per-CPU
> >>>   completion flag, rather than a single atomically-updated variable.
> >>
> >> Kostik,
> >>
> >> could this commit cause a problem reported in the below links?
> >> https://bz-attachments.freebsd.org/attachment.cgi?id=175614
> >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=213371
> > 
> > If I am reading the report right, the problem appears on the
> > 11.0-RELEASE system. The patch you reference was only applied to HEAD a
> > week ago and was not merged even to stable/11.
> 
> Sorry for the noise, then.  Somehow I thought that this went into the release
> branch, but obviously there was too little time for that.
> 
> > The examination must start with backtracing the thread which owns the
> > smp_ipi_mtx (shown on the screenshot).
> 
> It looks like DDB is not in GENERIC in 11.0?

Yes, DDB is absent in GENERIC 11.0

> Not sure if the reporter would be able to configure a dump device and then 
> save
> the dump given that the panic happens in the installer.
> If anyone could provide them with instructions that would be great.
> 
> -- 
> Andriy Gapon
> ___
> svn-src-...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/svn-src-all
> To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r307082 - in head: . sys/amd64/conf sys/arm/conf sys/arm64/conf sys/conf sys/i386/conf sys/mips/conf sys/modules/cc sys/modules/khelp sys/netinet sys/netinet/tcp_stacks sys/pc98/conf s

[Slawa Olhovchenkov]

On Wed, Oct 12, 2016 at 02:16:42AM +, Jonathan T. Looney wrote:

> Author: jtl
> Date: Wed Oct 12 02:16:42 2016
> New Revision: 307082
> URL: https://svnweb.freebsd.org/changeset/base/307082
> 
> Log:
>   In the TCP stack, the hhook(9) framework provides hooks for kernel modules
>   to add actions that run when a TCP frame is sent or received on a TCP
>   session in the ESTABLISHED state. In the base tree, this functionality is
>   only used for the h_ertt module, which is used by the cc_cdg, cc_chd, cc_hd,
>   and cc_vegas congestion control modules.
>   
>   Presently, we incur overhead to check for hooks each time a TCP frame is
>   sent or received on an ESTABLISHED TCP session.

Do you perform estimate of performane impact of this overhead?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r306346 - head/sys/kern

[Slawa Olhovchenkov]

On Thu, Oct 06, 2016 at 02:08:46PM +1100, Bruce Evans wrote:

> On Wed, 5 Oct 2016, Slawa Olhovchenkov wrote:
> 
> > On Wed, Oct 05, 2016 at 11:19:10AM +1100, Bruce Evans wrote:
> >
> >> On Tue, 4 Oct 2016, Gleb Smirnoff wrote:
> >>
> >>> On Mon, Sep 26, 2016 at 03:30:30PM +, Eric van Gyzen wrote:
> >>> E> ...
> >>> E> Modified: head/sys/kern/kern_mutex.c
> >>> E> 
> >>> ==
> >>> E> --- head/sys/kern/kern_mutex.c Mon Sep 26 15:03:31 2016
> >>> (r306345)
> >>> E> +++ head/sys/kern/kern_mutex.c Mon Sep 26 15:30:30 2016
> >>> (r306346)
> >>> E> @@ -924,7 +924,7 @@ __mtx_assert(const volatile uintptr_t *c
> >>> E>  {
> >>> E>const struct mtx *m;
> >>> E>
> >>> E> -  if (panicstr != NULL || dumping)
> >>> E> +  if (panicstr != NULL || dumping || SCHEDULER_STOPPED())
> >>> E>return;
> >>>
> >>> I wonder if all this disjunct can be reduced just to SCHEDULER_STOPPED()?
> >>> Positive panicstr and dumping imply scheduler stopped.
> >>
> >> 'dumping' doesn't imply SCHEDULER_STOPPED().
> >>
> >> Checking 'dumping' here seems to be just an old bug.  It just breaks
> >> __mtx_assert(), while all other mutex operations work normally for dumping
> >> without panicing.
> >
> > [...]
> >
> > Is this related to halted (not reboted) 11.0 after ~^B and `panic`?
> 
> There might be related problems, but I don't see any here.
> 
> > What I see on serial console:
> > =
> > db> panic
> > panic: from debugger
> 
> I wouldn't trust panic from the debugger, but it is safer than dump
> from the debugger (both are ddb commands, but this is another bug).
> 
> > cpuid = 1
> > KDB: stack backtrace:
> > db_trace_self_wrapper() at 0x8031fadb = 
> > db_trace_self_wrapper+0x2b/frame 0xfe1f9e198120
> > vpanic() at 0x804a0302 = vpanic+0x182/frame 0xfe1f9e1981a0
> > panic() at 0x804a0383 = panic+0x43/frame 0xfe1f9e198200
> > db_panic() at 0x8031d987 = db_panic+0x17/frame 0xfe1f9e198210
> > db_command() at 0x8031d019 = db_command+0x299/frame 
> > 0xfe1f9e1982e0
> > db_command_loop() at 0x8031cd74 = db_command_loop+0x64/frame 
> > 0xfe1f9e1982f0
> > db_trap() at 0x8031fc1b = db_trap+0xdb/frame 0xfe1f9e198380
> > kdb_trap() at 0x804dd8c3 = kdb_trap+0x193/frame 0xfe1f9e198410
> > trap() at 0x806e3065 = trap+0x255/frame 0xfe1f9e198620
> > calltrap() at 0x806cafd1 = calltrap+0x8/frame 0xfe1f9e198620
> > --- trap 0x3, rip = 0x804dd11e, rsp = 0xfe1f9e1986f0, rbp = 
> > 0xfe1f9e198710 ---
> > kdb_alt_break_internal() at 0x804dd11e = 
> > kdb_alt_break_internal+0x18e/frame 0xfe1f9e198710
> > kdb_alt_break() at 0x804dcf8b = kdb_alt_break+0xb/frame 
> > 0xfe1f9e198720
> > uart_intr_rxready() at 0x803e38a8 = uart_intr_rxready+0x98/frame 
> > 0xfe1f9e198750
> > uart_intr() at 0x803e4621 = uart_intr+0x121/frame 0xfe1f9e198790
> > intr_event_handle() at 0x8046c74b = intr_event_handle+0x9b/frame 
> > 0xfe1f9e1987e0
> > intr_execute_handlers() at 0x8076d2d8 = 
> > intr_execute_handlers+0x48/frame 0xfe1f9e198810
> > lapic_handle_intr() at 0x8077163f = lapic_handle_intr+0x3f/frame 
> > 0xfe1f9e198830
> > Xapic_isr1() at 0x806cb6b7 = Xapic_isr1+0xb7/frame 
> > 0xfe1f9e198830
> > --- interrupt, rip = 0x8032fedf, rsp = 0xfe1f9e198900, rbp = 
> > 0xfe1f9e198940 ---
> > acpi_cpu_idle() at 0x8032fedf = acpi_cpu_idle+0x2af/frame 
> > 0xfe1f9e198940
> > cpu_idle_acpi() at 0x8076ad1f = cpu_idle_acpi+0x3f/frame 
> > 0xfe1f9e198960
> > cpu_idle() at 0x8076adc5 = cpu_idle+0x95/frame 0xfe1f9e198980
> > sched_idletd() at 0x804cbbe5 = sched_idletd+0x495/frame 
> > 0xfe1f9e198a70
> > fork_exit() at 0x8046a211 = fork_exit+0x71/frame 0xfe1f9e198ab0
> > fork_trampoline() at 0x806cb50e = fork_trampoline+0xe/frame 
> > 0xfe1f9e198ab0
> > --- trap 0, rip = 0, rsp = 0, rbp = 0 ---
> 
> This looks like a normal kdb entry then a not so normal panic from ddb,
> but no problems.

Yes, I am just capture all output from console after command (`panic

Re: svn commit: r306346 - head/sys/kern

[Slawa Olhovchenkov]

On Wed, Oct 05, 2016 at 11:19:10AM +1100, Bruce Evans wrote:

> On Tue, 4 Oct 2016, Gleb Smirnoff wrote:
> 
> > On Mon, Sep 26, 2016 at 03:30:30PM +, Eric van Gyzen wrote:
> > E> ...
> > E> Modified: head/sys/kern/kern_mutex.c
> > E> 
> > ==
> > E> --- head/sys/kern/kern_mutex.c   Mon Sep 26 15:03:31 2016
> > (r306345)
> > E> +++ head/sys/kern/kern_mutex.c   Mon Sep 26 15:30:30 2016
> > (r306346)
> > E> @@ -924,7 +924,7 @@ __mtx_assert(const volatile uintptr_t *c
> > E>  {
> > E>  const struct mtx *m;
> > E>
> > E> -if (panicstr != NULL || dumping)
> > E> +if (panicstr != NULL || dumping || SCHEDULER_STOPPED())
> > E>  return;
> >
> > I wonder if all this disjunct can be reduced just to SCHEDULER_STOPPED()?
> > Positive panicstr and dumping imply scheduler stopped.
> 
> 'dumping' doesn't imply SCHEDULER_STOPPED().
> 
> Checking 'dumping' here seems to be just an old bug.  It just breaks
> __mtx_assert(), while all other mutex operations work normally for dumping
> without panicing.

[...]

Is this related to halted (not reboted) 11.0 after ~^B and `panic`?
What I see on serial console:
=
db> panic
panic: from debugger
cpuid = 1
KDB: stack backtrace:
db_trace_self_wrapper() at 0x8031fadb = 
db_trace_self_wrapper+0x2b/frame 0xfe1f9e198120
vpanic() at 0x804a0302 = vpanic+0x182/frame 0xfe1f9e1981a0
panic() at 0x804a0383 = panic+0x43/frame 0xfe1f9e198200
db_panic() at 0x8031d987 = db_panic+0x17/frame 0xfe1f9e198210
db_command() at 0x8031d019 = db_command+0x299/frame 0xfe1f9e1982e0
db_command_loop() at 0x8031cd74 = db_command_loop+0x64/frame 
0xfe1f9e1982f0
db_trap() at 0x8031fc1b = db_trap+0xdb/frame 0xfe1f9e198380
kdb_trap() at 0x804dd8c3 = kdb_trap+0x193/frame 0xfe1f9e198410
trap() at 0x806e3065 = trap+0x255/frame 0xfe1f9e198620
calltrap() at 0x806cafd1 = calltrap+0x8/frame 0xfe1f9e198620
--- trap 0x3, rip = 0x804dd11e, rsp = 0xfe1f9e1986f0, rbp = 
0xfe1f9e198710 ---
kdb_alt_break_internal() at 0x804dd11e = 
kdb_alt_break_internal+0x18e/frame 0xfe1f9e198710
kdb_alt_break() at 0x804dcf8b = kdb_alt_break+0xb/frame 
0xfe1f9e198720
uart_intr_rxready() at 0x803e38a8 = uart_intr_rxready+0x98/frame 
0xfe1f9e198750
uart_intr() at 0x803e4621 = uart_intr+0x121/frame 0xfe1f9e198790
intr_event_handle() at 0x8046c74b = intr_event_handle+0x9b/frame 
0xfe1f9e1987e0
intr_execute_handlers() at 0x8076d2d8 = 
intr_execute_handlers+0x48/frame 0xfe1f9e198810
lapic_handle_intr() at 0x8077163f = lapic_handle_intr+0x3f/frame 
0xfe1f9e198830
Xapic_isr1() at 0x806cb6b7 = Xapic_isr1+0xb7/frame 0xfe1f9e198830
--- interrupt, rip = 0x8032fedf, rsp = 0xfe1f9e198900, rbp = 
0xfe1f9e198940 ---
acpi_cpu_idle() at 0x8032fedf = acpi_cpu_idle+0x2af/frame 
0xfe1f9e198940
cpu_idle_acpi() at 0x8076ad1f = cpu_idle_acpi+0x3f/frame 
0xfe1f9e198960
cpu_idle() at 0x8076adc5 = cpu_idle+0x95/frame 0xfe1f9e198980
sched_idletd() at 0x804cbbe5 = sched_idletd+0x495/frame 
0xfe1f9e198a70
fork_exit() at 0x8046a211 = fork_exit+0x71/frame 0xfe1f9e198ab0
fork_trampoline() at 0x806cb50e = fork_trampoline+0xe/frame 
0xfe1f9e198ab0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
Uptime: 1d4h53m19s
Dumping 12148 out of 131020 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
Dump complete
mps2: Sending StopUnit: path (xpt0:mps2:0:14:):  handle 12
mps2: Incrementing SSU count
mps2: Sending StopUnit: path (xpt0:mps2:0:18:):  handle 9
mps2: Incrementing SSU count
=

This is normal reboot (by /sbin/reboot):

===
Sending StopUnit: path (xpt0:mps2:0:14:):  handle 13
mps2: Incrementing SSU count
mps2: Sending StopUnit: path (xpt0:mps2:0:18:):  handle 9
mps2: Incrementing SSU count
mps2: Decrementing SSU count.
mps2: Completing stop unit for (xpt0:mps2:0:18:): 
mps2: Decrementing SSU count.
mps2: Completing stop unit for (xpt0:mps2:0:14:): 
===


mps2: lagg0: link state changed to DOWN
Sending StopUnit: path (xpt0:mps2:0:14:):  handle 12
mps2: Incrementing SSU count
mps2: Sending StopUnit: path (xpt0:mps2:0:18:):  handle 9
mps2: Incrementing SSU count
mps2: Decrementing SSU count.
mps2: Completing stop unit for (xpt0:mps2:0:18:): 
mps2: Decrementing SSU count.
mps2: Completing stop unit for (xpt0:mps2:0:14:): 

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r306350 - head/sys/amd64/amd64

[Slawa Olhovchenkov]

On Tue, Sep 27, 2016 at 02:56:29PM +0300, Konstantin Belousov wrote:

> On Tue, Sep 27, 2016 at 12:28:28AM +0300, Slawa Olhovchenkov wrote:
> > On Mon, Sep 26, 2016 at 05:22:44PM +, Konstantin Belousov wrote:
> > 
> > > Author: kib
> > > Date: Mon Sep 26 17:22:44 2016
> > > New Revision: 306350
> > > URL: https://svnweb.freebsd.org/changeset/base/306350
> > > 
> > > Log:
> > >   For machines which support PCID but not have INVPCID instruction,
> > >   i.e. SandyBridge and IvyBridge, correct a race between pmap_activate()
> > >   and invltlb_pcid_handler().
> > >   
> > >   Reported by and tested by:  Slawa Olhovchenkov <s...@zxy.spb.ru>
> > >   MFC after:  1 week
> > 
> > Thanks!
> > Planed commit to releng? Or SA?
> 
> A notice on the errata page is planned.

EN, not SA?
I think this is exploitable.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r306350 - head/sys/amd64/amd64

[Slawa Olhovchenkov]

On Mon, Sep 26, 2016 at 05:22:44PM +, Konstantin Belousov wrote:

> Author: kib
> Date: Mon Sep 26 17:22:44 2016
> New Revision: 306350
> URL: https://svnweb.freebsd.org/changeset/base/306350
> 
> Log:
>   For machines which support PCID but not have INVPCID instruction,
>   i.e. SandyBridge and IvyBridge, correct a race between pmap_activate()
>   and invltlb_pcid_handler().
>   
>   Reported by and tested by:  Slawa Olhovchenkov <s...@zxy.spb.ru>
>   MFC after:  1 week

Thanks!
Planed commit to releng? Or SA?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r305968 - head/etc/autofs

[Slawa Olhovchenkov]

On Mon, Sep 19, 2016 at 10:19:04PM -0600, Warner Losh wrote:

> For MSDOS it's one thing and likely helps. But for ufs it doesn't
> help. Soft updates already does an excellent job at reducing wear and
> all async does is give added danger of dataloss. Linux distros don't
> matter for UFS because different design decisions have been made for
> ext[234] and UFS.

SU mount of UFS and unexpected media loast can caused kernel panic at
next mount this media. async mount don't have such issuse.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r297225 - in head/sys: net netinet netinet6

[Slawa Olhovchenkov]

On Thu, Mar 24, 2016 at 07:54:56AM +, George V. Neville-Neil wrote:

> Author: gnn
> Date: Thu Mar 24 07:54:56 2016
> New Revision: 297225
> URL: https://svnweb.freebsd.org/changeset/base/297225
> 
> Log:
>   FreeBSD previously provided route caching for TCP (and UDP). Re-add
>   route caching for TCP, with some improvements. In particular, invalidate
>   the route cache if a new route is added, which might be a better match.
>   The cache is automatically invalidated if the old route is deleted.
>   
>   Submitted by:   Mike Karels
>   Reviewed by:gnn
>   Differential Revision:  https://reviews.freebsd.org/D4306

Can this mergered to stable/10 (in private branch, I mean ABI breakage)?
Have any dependecis of this commit?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304436 - in head: . sys/netinet

[Slawa Olhovchenkov]

On Sun, Aug 28, 2016 at 10:20:08AM -0700, Adrian Chadd wrote:

> Hi,
> 
> There are some no brainers here so far(tm):
> 
> working from the bottom up:
> 
> * yeah, the ixgbe locking is a bit silly. Kip's work with iflib and
> converting ixgbe to use that instead of its own locking for managing
> things should remove the bottom two locks
> * the rtalloc1_fib thing - that's odd, because it shouldn't be
> contending there unless there's some temporary redirect that's been
> learnt. What's the routing table look like on your machine? I Remember

As I understund this code:

===
static struct radix_node *
in_matroute(void *v_arg, struct radix_node_head *head)
{
struct radix_node *rn = rn_match(v_arg, head);
struct rtentry *rt = (struct rtentry *)rn;
  
if (rt) {
RT_LOCK(rt);
if (rt->rt_flags & RTPRF_OURS) {
rt->rt_flags &= ~RTPRF_OURS;
rt->rt_expire = 0;
}
RT_UNLOCK(rt);
}
return rn;
}
===

lock congestion will be created for any route: for any route captured
captured exclusive lock (for test).
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304436 - in head: . sys/netinet

[Slawa Olhovchenkov]

On Sun, Aug 28, 2016 at 10:20:08AM -0700, Adrian Chadd wrote:

> Then the rest of the big entries are just a combination of rtentry
> locking, tcp timer locking, zfs locking and madvise locking. There's
> some sowakeup locking there as well, from the socket producer/consumer
> locking.

On some high network speed I am see like kevent's congestion:

22.79%  [89522]__mtx_lock_sleep @ /boot/kernel.VSTREAM/kernel
 82.99%  [74295] knote
  99.97%  [74269]  sowakeup
   99.07%  [73581]   tcp_do_segment
100.0%  [73581]tcp_input
 100.0%  [73581] ip_input
  100.0%  [73581]  netisr_dispatch_src
   100.0%  [73581]   ether_demux
100.0%  [73581]ether_nh_input
 100.0%  [73581] netisr_dispatch_src
  68.88%  [50683]  t4_eth_rx @ 
/boot/kernel.VSTREAM/if_cxgbe.ko
   100.0%  [50683]   service_iq
100.0%  [50683]t4_intr
 100.0%  [50683] intr_event_execute_handlers @ 
/boot/kernel.VSTREAM/kernel
  100.0%  [50683]  ithread_loop
   100.0%  [50683]   fork_exit
  31.12%  [22898]  tcp_lro_flush
   97.30%  [22280]   service_iq @ 
/boot/kernel.VSTREAM/if_cxgbe.ko
100.0%  [22280]t4_intr
 100.0%  [22280] intr_event_execute_handlers @ 
/boot/kernel.VSTREAM/kernel
  100.0%  [22280]  ithread_loop
   100.0%  [22280]   fork_exit
   01.55%  [354] tcp_lro_flush_inactive
100.0%  [354]  service_iq @ 
/boot/kernel.VSTREAM/if_cxgbe.ko
 100.0%  [354]   t4_intr
  100.0%  [354]intr_event_execute_handlers @ 
/boot/kernel.VSTREAM/kernel
   100.0%  [354] ithread_loop
   01.15%  [264] tcp_lro_rx
100.0%  [264]  t4_eth_rx @ 
/boot/kernel.VSTREAM/if_cxgbe.ko
 100.0%  [264]   service_iq
  100.0%  [264]t4_intr
   100.0%  [264] intr_event_execute_handlers @ 
/boot/kernel.VSTREAM/kernel

Can I do some recomendation?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r305004 - in head/sys: amd64/amd64 amd64/include i386/i386

[Slawa Olhovchenkov]

On Tue, Aug 30, 2016 at 03:58:54AM +1000, Bruce Evans wrote:

> On Mon, 29 Aug 2016, Slawa Olhovchenkov wrote:
> 
> > On Mon, Aug 29, 2016 at 01:07:21PM +, Bruce Evans wrote:
> >> ...
> >> Log:
> >>   On amd64, declare sse2_pagezero() and start using it again, but only
> >>   for zeroing pages in idle where nontemporal writes are clearly best.
> >>   This is almost a no-op since zeroing in idle works does nothing good
> >>   and is off by default.  Fix END() statement forgotten in previous
> >>   commit.
> >> ...
> >
> > Do you think about using nontemporal writes for copying from user
> > space to kernel space? In much cases this copy result don't need any parsing
> > anf just go to DMA, eliminate cache use.
> 
> I once tried that (for copyin/out, without DMA).  It was just a small
> pessimization.  For small data, it increases setup overheads, and
> large data isn't common enough to matter.  DMA can be even slower
> since it increases latency.  First there is latency to set it up,
> and

May be I am not very clear: not using DMA for tranfer. Using
nontemporal for data passed to devices DMA buffers.

> then you often don't want it to touch the cache since that gives more
> latency for cache misses.

No, I am talk not about total replace such copy, only on per-case
bases.
I.e. introduce new function and use it in special cases (for example:
tcp output).
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r305004 - in head/sys: amd64/amd64 amd64/include i386/i386

[Slawa Olhovchenkov]

On Mon, Aug 29, 2016 at 01:07:21PM +, Bruce Evans wrote:

> Author: bde
> Date: Mon Aug 29 13:07:21 2016
> New Revision: 305004
> URL: https://svnweb.freebsd.org/changeset/base/305004
> 
> Log:
>   On amd64, declare sse2_pagezero() and start using it again, but only
>   for zeroing pages in idle where nontemporal writes are clearly best.
>   This is almost a no-op since zeroing in idle works does nothing good
>   and is off by default.  Fix END() statement forgotten in previous
>   commit.
>   
>   Align the loop in sse2_pagezero().  Since it writes to main memory,
>   the loop doesn't have to be very carefully written to keep up.
>   Unrolling it was considered useless or harmful and was not done on
>   i386, but that was too careless.
>   
>   Timing for i386: the loop was not unrolled at all, and moved only 4
>   bytes/iteration.  So on a 2GHz CPU, it needed to run at 2 cycles/
>   iteration to keep up with a memory speed of just 4GB/sec.  But when
>   it crossed a 16-byte boundary, on old CPUs it ran at 3 cycles/
>   iteration so it gave a maximum speed of 2.67GB/sec and couldn't even
>   keep up with PC3200 memory.  Fix the alignment so that it keep up with
>   4GB/sec memory, and unroll once to get nearer to 8GB/sec.  Further
>   unrolling might be useless or harmful since it would prevent the loop
>   fitting in 16-bytes.  My test system with an old CPU and old DDR1 only
>   needed 5+ GB/sec.  My test system with a new CPU and DDR3 doesn't need
>   any changes to keep up ~16GB/sec.
>   
>   Timing for amd64: with 8-byte accesses and newer faster CPUs it is
>   easy to reach 16GB/sec but not so easy to go much faster.  The
>   alignment doesn't matter much if the CPU is not very old.  The loop
>   was already unrolled 4 times, but needs 32 bytes and uses a fancy
>   method that doesn't work for 2-way unrolling in 16 bytes.  Just
>   align it to 32-bytes.

Do you think about using nontemporal writes for copying from user
space to kernel space? In much cases this copy result don't need any parsing
anf just go to DMA, eliminate cache use.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304436 - in head: . sys/netinet

[Slawa Olhovchenkov]

On Sun, Aug 28, 2016 at 10:20:08AM -0700, Adrian Chadd wrote:

> * the rtalloc1_fib thing - that's odd, because it shouldn't be
> contending there unless there's some temporary redirect that's been
> learnt. What's the routing table look like on your machine? I Remember
> investigating the rtentry reference counting a while ago and concluded
> that .. it's terrible, and one specific corner case was checking for
> routes from redirects. I'll look at my notes again and see what I
> find.

JFYI:

  kernel`rtalloc1_fib+0x6d
  kernel`rtalloc_ign_fib+0xcc
  kernel`ip_output+0x3a5
  kernel`tcp_output+0x1852
  kernel`tcp_timer_rexmt+0x60e
  kernel`softclock_call_cc+0x17b
  kernel`softclock+0x94
  kernel`intr_event_execute_handlers+0xab
  kernel`ithread_loop+0x96
  kernel`fork_exit+0x9a
  kernel`0x806c8e0e
176868370


kernel`ip_output+0x3a5 is sys/netinet/ip_output.c:282

/*
 * We want to do any cloning requested by the link layer,
 * as this is probably required in all cases for correct
 * operation (as it is for ARP).
 */
if (rte == NULL) {
#ifdef RADIX_MPATH
rtalloc_mpath_fib(ro,
ntohl(ip->ip_src.s_addr ^ ip->ip_dst.s_addr),
inp ? inp->inp_inc.inc_fibnum : M_GETFIB(m));
#else
in_rtalloc_ign(ro, 0,
inp ? inp->inp_inc.inc_fibnum : M_GETFIB(m));
#endif
rte = ro->ro_rt;
}
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304436 - in head: . sys/netinet

[Slawa Olhovchenkov]

On Sat, Aug 27, 2016 at 07:05:01PM -0700, Adrian Chadd wrote:

> On 26 August 2016 at 17:46, Slawa Olhovchenkov <s...@zxy.spb.ru> wrote:
> > On Fri, Aug 26, 2016 at 04:55:34PM -0700, Adrian Chadd wrote:
> >
> >> Hi,
> >>
> >> I use the kernel lock profiling debugging,
> >
> > I am already have 100% utilise all CPU cores, I think this is drop
> > performance?
> >
> >> but you can use dtrace to
> >> get an idea:
> >>
> >> dtrace -n 'lockstat:::adaptive-block { @[stack()] = sum(arg1); }'
> >
> > How to interpret results (how to distinct lock contention from lock
> > cost/overhead (LOCK CMPXCGQ is very expensive))?
> 
> well, paste the results? :)

OK, thanks!
http://m.uploadedit.com/ba3s/1472390519813.txt

> the lock is expensive because it's contended. :)

As I am understund lock contended:

1. Multiple thread wait same lock.
2. Performance not scaled by adding CPU power

As I understund lock expensive: Intel cache coherence protocol is
expensive and take lock 800K per second take too much cpu/memory
bandwidth w/o any contention. Memory latency at cache miss too.

Adding mory CPU power (more cores, more GHz) take me performane boost.
I am think my case is not lock contended.

> >> (https://wiki.freebsd.org/DTrace/One-Liners)
> >>
> >>
> >>
> >> -adrian
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304436 - in head: . sys/netinet

[Slawa Olhovchenkov]

On Fri, Aug 26, 2016 at 04:55:34PM -0700, Adrian Chadd wrote:

> Hi,
> 
> I use the kernel lock profiling debugging,

I am already have 100% utilise all CPU cores, I think this is drop
performance?

> but you can use dtrace to
> get an idea:
> 
> dtrace -n 'lockstat:::adaptive-block { @[stack()] = sum(arg1); }'

How to interpret results (how to distinct lock contention from lock
cost/overhead (LOCK CMPXCGQ is very expensive))?

> (https://wiki.freebsd.org/DTrace/One-Liners)
> 
> 
> 
> -adrian
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304436 - in head: . sys/netinet

[Slawa Olhovchenkov]

On Fri, Aug 26, 2016 at 02:42:27PM -0700, Adrian Chadd wrote:

> On 26 August 2016 at 14:36, Slawa Olhovchenkov <s...@zxy.spb.ru> wrote:
> > On Fri, Aug 26, 2016 at 02:32:00PM -0700, Adrian Chadd wrote:
> >
> >> Hi,
> >>
> >> It's pcb lock contention.
> >
> > Not sure: only 5% of all time.
> > And same 5% for tcbhashsize = 65K and 256K.
> > Or you talk about some more thin effect?
> 
> You're in the inpcb lock from multiple places.

For tcp case I am found only 3 places: in_pcbdrop in_pcbremlists 
in_pcblookup_hash

> the tcbhashsize doesnt influence the pcb lock contention - it just
> affects how long you take doing lookups. iF your hash table is too
> small then you end up doing lots of O(n) walks of a hash bucket to
> find a pcb entry. :)

Hmm. I am not clearly understund you.
How originate pcb lock contention?
How I can see this (in pmc profile, in dtarce probese)?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304436 - in head: . sys/netinet

[Slawa Olhovchenkov]

On Fri, Aug 26, 2016 at 02:32:00PM -0700, Adrian Chadd wrote:

> Hi,
> 
> It's pcb lock contention.

Not sure: only 5% of all time.
And same 5% for tcbhashsize = 65K and 256K.
Or you talk about some more thin effect?

> 
> On 26 August 2016 at 08:13, Slawa Olhovchenkov <s...@zxy.spb.ru> wrote:
> > On Fri, Aug 26, 2016 at 04:01:14PM +0100, Bruce Simpson wrote:
> >
> >> Slawa,
> >>
> >> I'm afraid this may be a bit of a non-sequitur. Sorry.. I seem to be
> >> missing something. As I understand it this thread is about Ryan's change
> >> to netinet for broadcast.
> >>
> >> On 26/08/16 15:49, Slawa Olhovchenkov wrote:
> >> > On Sun, Aug 21, 2016 at 03:04:00AM +0300, Slawa Olhovchenkov wrote:
> >> >> On Sun, Aug 21, 2016 at 12:25:46AM +0100, Bruce Simpson wrote:
> >> >>> Whilst I agree with your concerns about multipoint, I support the
> >> >>> motivation behind Ryan's original change: optimize the common case.
> >> >>
> >> >> Oh, common case...
> >> >> I am have pmc profiling for TCP output and see on this SVG picture and
> >> >> don't find any simple way.
> >> >> You want to watch too?
> >> >
> >> > At time peak network traffic (more then 25K connections, about 20Gbit
> >> > total traffic) half of cores fully utilised by network stack.
> >> >
> >> > This is flamegraph from one core: http://zxy.spb.ru/cpu10.svg
> >> > This is same, but stack cut of at ixgbe_rxeof for more unified
> >> > tcp/ip stack view http://zxy.spb.ru/cpu10u.svg
> >> ...
> >>
> >> I appreciate that you've taken the time to post a flamegraph (a
> >> fashionable visualization) of relative performance in the FreeBSD
> >> networking stack.
> >>
> >> Sadly, I am mostly out of my depth for looking at stack wide performance
> >> for the moment; for the things I look at involving FreeBSD at work just
> >> at the moment, I would not generally go down there except for specific
> >> performance issues (e.g. with IEEE 1588).
> >>
> >> It sounds as though perhaps you should raise a wider discussion about
> >> your results on -net. I would caution you however that the Function
> >> Boundary Trace (FBT) provider for DTrace can introduce a fair amount of
> >> noise to the raw performance data because of the trap mechanism it uses.
> >> This ruled it out for one of my own studies requiring packet-level 
> >> accuracy.
> >>
> >> Whilst raw pmc(4) profiles may require more post-processing, they will
> >> provide less equivocal data (and a better fix) on the hot path, due also
> >> to being sampled effectively on a PMC interrupt (a gather stage- poll
> >> core+uncore MSRs), not purely a software timer interrupt.
> >
> > Thanks for answer, I am now try to start discussion on -net.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304436 - in head: . sys/netinet

[Slawa Olhovchenkov]

On Fri, Aug 26, 2016 at 04:01:14PM +0100, Bruce Simpson wrote:

> Slawa,
> 
> I'm afraid this may be a bit of a non-sequitur. Sorry.. I seem to be 
> missing something. As I understand it this thread is about Ryan's change 
> to netinet for broadcast.
> 
> On 26/08/16 15:49, Slawa Olhovchenkov wrote:
> > On Sun, Aug 21, 2016 at 03:04:00AM +0300, Slawa Olhovchenkov wrote:
> >> On Sun, Aug 21, 2016 at 12:25:46AM +0100, Bruce Simpson wrote:
> >>> Whilst I agree with your concerns about multipoint, I support the
> >>> motivation behind Ryan's original change: optimize the common case.
> >>
> >> Oh, common case...
> >> I am have pmc profiling for TCP output and see on this SVG picture and
> >> don't find any simple way.
> >> You want to watch too?
> >
> > At time peak network traffic (more then 25K connections, about 20Gbit
> > total traffic) half of cores fully utilised by network stack.
> >
> > This is flamegraph from one core: http://zxy.spb.ru/cpu10.svg
> > This is same, but stack cut of at ixgbe_rxeof for more unified
> > tcp/ip stack view http://zxy.spb.ru/cpu10u.svg
> ...
> 
> I appreciate that you've taken the time to post a flamegraph (a 
> fashionable visualization) of relative performance in the FreeBSD 
> networking stack.
> 
> Sadly, I am mostly out of my depth for looking at stack wide performance 
> for the moment; for the things I look at involving FreeBSD at work just 
> at the moment, I would not generally go down there except for specific 
> performance issues (e.g. with IEEE 1588).
> 
> It sounds as though perhaps you should raise a wider discussion about 
> your results on -net. I would caution you however that the Function 
> Boundary Trace (FBT) provider for DTrace can introduce a fair amount of 
> noise to the raw performance data because of the trap mechanism it uses. 
> This ruled it out for one of my own studies requiring packet-level accuracy.
> 
> Whilst raw pmc(4) profiles may require more post-processing, they will 
> provide less equivocal data (and a better fix) on the hot path, due also 
> to being sampled effectively on a PMC interrupt (a gather stage- poll 
> core+uncore MSRs), not purely a software timer interrupt.

Thanks for answer, I am now try to start discussion on -net.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304436 - in head: . sys/netinet

[Slawa Olhovchenkov]

On Sun, Aug 21, 2016 at 03:04:00AM +0300, Slawa Olhovchenkov wrote:

> On Sun, Aug 21, 2016 at 12:25:46AM +0100, Bruce Simpson wrote:
> 
> > On 20/08/16 23:05, Slawa Olhovchenkov wrote:
> > > I am think this substitution is very bad idea (by design).
> > > Also, on transmit side this is must be irrelevant on received L2
> > > header (and this in many cases this is will be L2 unicast packet). For
> > > other cases packet will be created on host and don't have any received
> > > information.
> > >
> > 
> > Whilst I agree with your concerns about multipoint, I support the 
> > motivation behind Ryan's original change: optimize the common case.
> 
> Oh, common case...
> I am have pmc profiling for TCP output and see on this SVG picture and
> don't find any simple way.
> You want to watch too?

At time peak network traffic (more then 25K connections, about 20Gbit
total traffic) half of cores fully utilised by network stack.

This is flamegraph from one core: http://zxy.spb.ru/cpu10.svg
This is same, but stack cut of at ixgbe_rxeof for more unified
tcp/ip stack view http://zxy.spb.ru/cpu10u.svg

Top 3 used lines is:

7036 0x804bf02d atomic_cmpset_long 
/usr/obj/usr/src/sys/VSTREAM/./machine/atomic.h:163

static __inline int
atomic_cmpset_long(volatile u_long *dst, u_long expect, u_long src)
{
u_char res;
 
__asm __volatile(
"   " MPLOCKED ""
>   "   cmpxchgq %3,%1 ;"
"   sete%0 ;"
"# atomic_cmpset_long"
: "=q" (res),   /* 0 */
  "+m" (*dst),  /* 1 */
  "+a" (expect) /* 2 */
: "r" (src) /* 3 */
: "memory", "cc");
return (res);
}
 
6099 0x81171963 ?? ??:0

0x81171940 <ixgbe_rxeof+1168>:  mov0x10(%r15),%rax
0x81171944 <ixgbe_rxeof+1172>:  add$0x8,%rax
0x81171948 <ixgbe_rxeof+1176>:  mov-0x4c(%rbp),%ecx
0x8117194b <ixgbe_rxeof+1179>:  test   %cx,%cx
0x8117194e <ixgbe_rxeof+1182>:  mov%rax,0x10(%r15)
0x81171952 <ixgbe_rxeof+1186>:  je 0x8117198d 
<ixgbe_rxeof+1245>
0x81171954 <ixgbe_rxeof+1188>:  mov0x10(%rdi),%rcx
0x81171958 <ixgbe_rxeof+1192>:  mov-0x4c(%rbp),%edx
0x8117195b <ixgbe_rxeof+1195>:  nopl   0x0(%rax,%rax,1)
0x81171960 <ixgbe_rxeof+1200>:  mov(%rcx),%rsi
0x81171963 <ixgbe_rxeof+1203>:  mov%rsi,(%rax)
0x81171966 <ixgbe_rxeof+1206>:  mov0x8(%rcx),%rsi
0x8117196a <ixgbe_rxeof+1210>:  mov%rsi,0x8(%rax)
0x8117196e <ixgbe_rxeof+1214>:  mov0x10(%rcx),%rsi
0x81171972 <ixgbe_rxeof+1218>:  mov%rsi,0x10(%rax)
0x81171976 <ixgbe_rxeof+1222>:  mov0x18(%rcx),%rsi
0x8117197a <ixgbe_rxeof+1226>:  mov%rsi,0x18(%rax)
0x8117197e <ixgbe_rxeof+1230>:  add$0xffe0,%edx
0x81171981 <ixgbe_rxeof+1233>:  add$0x20,%rcx
0x81171985 <ixgbe_rxeof+1237>:  add$0x20,%rax
0x81171989 <ixgbe_rxeof+1241>:  test   %edx,%edx

5594 0x8053395a mb_free_ext /usr/src/sys/kern/uipc_mbuf.c:301

if (*(m->m_ext.ref_cnt) == 1 ||


I am able collect and process more measure for help
to improve FreeBSD network stack.

Do you have some idea about this?
I am don't see evident and simple points of optimisation :(

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z

[Slawa Olhovchenkov]

On Tue, Aug 23, 2016 at 03:26:04PM +0300, Toomas Soome wrote:

> > Main trouble (by kib@) is 640KB real mode limit.
> > Separated heap don't soled this.
> > May be solution is early switch to protected mode, boot2?
> 
> hm, but boot2 is already btx client and btx is setting up the
> protected mode. Only zfsboot/gpt[zfs]boot memory copy (boot2
> relocator) is working in segmented real mode, so the gptldr.S and
> zfsldr.S has to deal with memory segments to set boot2 up. Or did
> got you wrong?

Mat be I am wrong.
May be I am not very clean.
As I understund kib@ restriction caused by 640KB realmode limit.
In case of boot2 start in real mode and read enterly by boot1 -- we
also touch this trouble.

How to resolve this:

1) start entirely; boot2 in protected mode and read all boot2 to
extended memory

2) read only part of boot2 (boot2 relocator) by boot1 and switch to
protected mode before reading rest of boot2 by boot2 code.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z

[Slawa Olhovchenkov]

On Tue, Aug 23, 2016 at 03:00:32PM +0300, Toomas Soome wrote:

> 
> > On 23. aug 2016, at 14:29, Slawa Olhovchenkov <s...@zxy.spb.ru> wrote:
> > 
> > On Tue, Aug 23, 2016 at 11:05:47AM +0300, Toomas Soome wrote:
> > 
> >> 
> >>> On 22. aug 2016, at 17:56, Toomas Soome <tso...@me.com> wrote:
> >>> 
> >>> 
> >>>> On 22. aug 2016, at 17:19, Warner Losh <i...@bsdimp.com> wrote:
> >>>> 
> >>>> On Mon, Aug 22, 2016 at 3:44 AM, Toomas Soome <tso...@me.com> wrote:
> >>>>> I do suspect the size difference there is partially due to ficl, in 
> >>>>> illumos (ficl 4):
> >>>>> 
> >>>>> -rw-r--r--   1 tsoome   staff 132508 aug 22 09:18 libficl.a
> >>>>> 
> >>>>> and freebsd (ficl 3):
> >>>>> 
> >>>>> -rw-r--r--  1 root  wheel  213748 Aug 19 01:57 libficl.a
> >>>>> 
> >>>>> so, there definitely is some space…
> >>>> 
> >>>> Same compiler? Clang bloats the boot code rather substantially, even 
> >>>> after
> >>>> all the flags to tell it to generate smaller code are used. gcc 4.2.x
> >>>> built stuff
> >>>> was substantially smaller.
> >>>> 
> >>>> There's a 520kb limit enforced in the boot1 for similar reasons. Looks 
> >>>> like
> >>>> the combination of options makes us use just enough extra memory to
> >>>> sink the battleship...
> >>>> 
> >>>> Warner
> >>>> 
> >>> 
> >>> 
> >>> Actually I only now realized I was comparing apples with oranges… I 
> >>> forgot the fbsd builds 32bit version in ficl32, this one is 64bit. and 
> >>> yes the 32bit version is not that big at all:D
> >>> 
> >>> Also, after done some digging, I have found few instances of duplicated 
> >>> code (we can share sha2 with geli and so if sha512 is already needed, it 
> >>> will become another “free lunch”). Also, unless I’m mistaken, for some 
> >>> reason the bzip *compression* is brought in - correct me if I’m wrong, 
> >>> but afaik only decompression is needed…
> >>> 
> >>> So before going after “useless features”, there are some “hidden” 
> >>> resources to remove extra fat.
> >>> 
> >> 
> >> I did some more digging. while ld has —gc-sections to clean up unused 
> >> bits, to make it effective, the code build does also need -Os 
> >> -fdata-sections -ffunction-sections.
> >> So I did just very simple test by adding those flags to bsd.stand.mk and:
> >> 
> >> first the “default” binaries from /boot:
> >> -r-xr-xr-x  1 root  wheel  446464 Aug 19 08:46 /boot/zfsloader
> >> -rw-r--r--  1 root  wheel  438272 Aug 23 00:30 /boot/zfsloader.b
> >> -r-xr-xr-x  1 root  wheel  446464 Aug  5 08:37 /boot/zfsloader.old
> >> -r--r--r--  1 root  wheel  406568 Aug 19 08:46 /boot/userboot.so
> >> 
> >> (note, zfsloader.b here is built with https://reviews.freebsd.org/D7600)
> >> 
> >> now after adding compile flags  -Os -fdata-sections -ffunction-sections:
> >> 
> >> -rw-r--r--  1 root  wheel  389120 Aug 23 10:12 zfsloader
> >> -rwxr-xr-x  1 root  wheel  378156 Aug 23 10:12 zfsloader.bin
> >> -rwxr-xr-x  1 root  wheel  437514 Aug 23 10:12 zfsloader.sym
> >> -rwxr-xr-x  1 root  wheel  375912 Aug 23 10:03 userboot.so
> >> 
> >> and finally test for Andriy with:
> >> LOADER_BZIP2_SUPPORT=yes
> >> LOADER_FIREWIRE_SUPPORT=yes
> >> 
> >> -rw-r--r--  1 root  wheel  421888 Aug 23 10:22 zfsloader
> >> -rwxr-xr-x  1 root  wheel  409932 Aug 23 10:22 zfsloader.bin
> >> -rwxr-xr-x  1 root  wheel  472021 Aug 23 10:22 zfsloader.sym
> >> -rwxr-xr-x  1 root  wheel  375912 Aug 23 10:22 userboot.so
> >> 
> >> note the userboot.so did not change from those flags.
> >> 
> >> This is just an result from compile, and by adding 3 options to 
> >> bsd.stand.mk; however, not all Makefiles in loader tree seem to include 
> >> it, and most importantly, haven’t tested real boot yet;)
> >> 
> >> To conclude, some more work is needed to review the Makefiles, build 
> >> options used etc, also I don’t know all the background why the compiler 
> >> options are set as they currently are - were there any related 
> >> compiler/linker bugs, or any other reasons, also how/if other platforms 
> >> are affected - for example bsd.stand.mk does set -Os for pc98, but not for 
> >> others…
> > 
> > This is only size on disk, memory consuming still same, IMHO.
> 
> Actually this reduction above is entirely due to -Os, the —gc-sections is not 
> passed to linker (at least for zfsloader case). I think we will need linker 
> script to preserve set_Xcommand_set linker set to use —gc-sections with ld.
> 
> Loader heap is separate already and does not contribute to this issue. Also, 
> I already did test the boot with thinned zfsloader, both with and without 
> bzip2 are appearing to work, at least for this quick test anyhow.

Main trouble (by kib@) is 640KB real mode limit.
Separated heap don't soled this.
May be solution is early switch to protected mode, boot2?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/z

[Slawa Olhovchenkov]

On Tue, Aug 23, 2016 at 11:05:47AM +0300, Toomas Soome wrote:

> 
> > On 22. aug 2016, at 17:56, Toomas Soome  wrote:
> > 
> > 
> >> On 22. aug 2016, at 17:19, Warner Losh  wrote:
> >> 
> >> On Mon, Aug 22, 2016 at 3:44 AM, Toomas Soome  wrote:
> >>> I do suspect the size difference there is partially due to ficl, in 
> >>> illumos (ficl 4):
> >>> 
> >>> -rw-r--r--   1 tsoome   staff 132508 aug 22 09:18 libficl.a
> >>> 
> >>> and freebsd (ficl 3):
> >>> 
> >>> -rw-r--r--  1 root  wheel  213748 Aug 19 01:57 libficl.a
> >>> 
> >>> so, there definitely is some space…
> >> 
> >> Same compiler? Clang bloats the boot code rather substantially, even after
> >> all the flags to tell it to generate smaller code are used. gcc 4.2.x
> >> built stuff
> >> was substantially smaller.
> >> 
> >> There's a 520kb limit enforced in the boot1 for similar reasons. Looks like
> >> the combination of options makes us use just enough extra memory to
> >> sink the battleship...
> >> 
> >> Warner
> >> 
> > 
> > 
> > Actually I only now realized I was comparing apples with oranges… I forgot 
> > the fbsd builds 32bit version in ficl32, this one is 64bit. and yes the 
> > 32bit version is not that big at all:D
> > 
> > Also, after done some digging, I have found few instances of duplicated 
> > code (we can share sha2 with geli and so if sha512 is already needed, it 
> > will become another “free lunch”). Also, unless I’m mistaken, for some 
> > reason the bzip *compression* is brought in - correct me if I’m wrong, but 
> > afaik only decompression is needed…
> > 
> > So before going after “useless features”, there are some “hidden” resources 
> > to remove extra fat.
> > 
> 
> I did some more digging. while ld has —gc-sections to clean up unused bits, 
> to make it effective, the code build does also need -Os -fdata-sections 
> -ffunction-sections.
> So I did just very simple test by adding those flags to bsd.stand.mk and:
> 
> first the “default” binaries from /boot:
> -r-xr-xr-x  1 root  wheel  446464 Aug 19 08:46 /boot/zfsloader
> -rw-r--r--  1 root  wheel  438272 Aug 23 00:30 /boot/zfsloader.b
> -r-xr-xr-x  1 root  wheel  446464 Aug  5 08:37 /boot/zfsloader.old
> -r--r--r--  1 root  wheel  406568 Aug 19 08:46 /boot/userboot.so
> 
> (note, zfsloader.b here is built with https://reviews.freebsd.org/D7600)
> 
> now after adding compile flags  -Os -fdata-sections -ffunction-sections:
> 
> -rw-r--r--  1 root  wheel  389120 Aug 23 10:12 zfsloader
> -rwxr-xr-x  1 root  wheel  378156 Aug 23 10:12 zfsloader.bin
> -rwxr-xr-x  1 root  wheel  437514 Aug 23 10:12 zfsloader.sym
> -rwxr-xr-x  1 root  wheel  375912 Aug 23 10:03 userboot.so
> 
> and finally test for Andriy with:
> LOADER_BZIP2_SUPPORT=yes
> LOADER_FIREWIRE_SUPPORT=yes
> 
> -rw-r--r--  1 root  wheel  421888 Aug 23 10:22 zfsloader
> -rwxr-xr-x  1 root  wheel  409932 Aug 23 10:22 zfsloader.bin
> -rwxr-xr-x  1 root  wheel  472021 Aug 23 10:22 zfsloader.sym
> -rwxr-xr-x  1 root  wheel  375912 Aug 23 10:22 userboot.so
> 
> note the userboot.so did not change from those flags.
> 
> This is just an result from compile, and by adding 3 options to bsd.stand.mk; 
> however, not all Makefiles in loader tree seem to include it, and most 
> importantly, haven’t tested real boot yet;)
> 
> To conclude, some more work is needed to review the Makefiles, build options 
> used etc, also I don’t know all the background why the compiler options are 
> set as they currently are - were there any related compiler/linker bugs, or 
> any other reasons, also how/if other platforms are affected - for example 
> bsd.stand.mk does set -Os for pc98, but not for others…

This is only size on disk, memory consuming still same, IMHO.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304555 - head/sys/compat/cloudabi

[Slawa Olhovchenkov]

On Mon, Aug 22, 2016 at 06:55:58PM +1000, Bruce Evans wrote:

> On Sun, 21 Aug 2016, Slawa Olhovchenkov wrote:
> 
> > On Mon, Aug 22, 2016 at 02:49:07AM +1000, Bruce Evans wrote:
> >
> >> On Sun, 21 Aug 2016, Slawa Olhovchenkov wrote:
> >>
> >>> On Sun, Aug 21, 2016 at 11:39:02PM +1000, Bruce Evans wrote:
> >>>
> >>>> On Sun, 21 Aug 2016, Slawa Olhovchenkov wrote:
> >>>>> I am remeber about platforms with missaligment trap when
> >>>>> accessing int16 by odd address. Now platforms like this do not exist
> >>>>> anymore?
> >>>>
> >>>> i386 still exists, and it supports trapping on misalignement for at least
> >>>> CPL 3 (not kernel CPL 0).  IIRC, amd64 drops support for this.
> >>>
> >>> Someone enable and support this? I am don't see.
> >>> May be PPC trap on this?
> >>> Alpha trap on this, but support of Alpha is droped.
> >>
> >> It is a 1-line change in asm (or a little more in C with #includes) to
> >> enable the trap:
> >
> > OK, we can turn amd64 in this mode.
> > And cat do request to kernel with unalligned access, this cause trap
> > and panic, yes?
> 
> No.  PSL_AC is ignored in kernel mode.

OK. I.e. i386 and amd64 is not target.
cloudabi work in kernel mode, yes?

> >> It is a trillion-line change to fix the compilers and applications to not
> >> do misaligned accesses :-).  I only tried to use this ~25 years ago.  Then
> >> the most obvious compiler bug was generating 32-bit acccesses to assign
> >> large but misaligned structs.  If the compiler just generated calls to
> >> memcpy(), that might work, but in practice libraries also assume alignment.
> >
> > This issuse can be trigerred and by two-bytes assigmen, yes?
> 
> Not quite that short.  i386 has the 1-byte cli instruction for conveniently
> setting the interrupt enable flag, but setting PSL_AC seems to take at
> least 3 instructions and 6-7 bytes (pushf; orb $N,$M(%[re][bs]p); popf).

I am miss something. Why you talk about bytes per instruction?
I think this is about returning value to applications unaligning
buffer? 8 bytes in this commit or 2 bytes in my example.

(for this commit, as I see, td_retval always aligned)

> >>>>>> There are also endianness problems.  The old version was even more 
> >>>>>> broken
> >>>>>> on big endian systems.  The current version needs some magic to reverse
> >>>>>> the memcpy() of the bits.  We already depend on this for some 64-bit
> >>>>>> syscalls like lseek().
> >>>>>
> >>>>> Can you explain some more?
> >>>>> This is not transfer over network and don't read from external media.
> >>>>> Where is problem?
> >>>>
> >>>> It is similar to a network transfer.  It needs a protocol to pass values
> >>>> to applications.  Type puns are fragile even within a single compilation
> >>>> unit.
> >>>
> >>> Application ad kernel run with same byte order, not?
> >>
> >> The application can do anything it wants, but has to translate if it uses
> >> the kernel or a library written in another language.
> >
> > You talk about different byte order in differenr languages?
> 
> Could be, or the same language with a different ABI.

ABI enforced by `server`, ex: kernel, or cloudabi in this case.
If language need communicate -- language need adopted, not kernel or
cloadabi. No simple way from kernel/cloudabi/etc detect caller abi.
Or I am missunderstund you.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304572 - in head: sbin/ipfw sys/conf sys/netinet sys/netinet6

[Slawa Olhovchenkov]

On Sun, Aug 21, 2016 at 07:20:12PM +, Bjoern A. Zeeb wrote:

> On 21 Aug 2016, at 19:08, Slawa Olhovchenkov wrote:
> 
> > On Sun, Aug 21, 2016 at 06:55:30PM +, Bjoern A. Zeeb wrote:
> >
> >> Author: bz
> >> Date: Sun Aug 21 18:55:30 2016
> >> New Revision: 304572
> >> URL: https://svnweb.freebsd.org/changeset/base/304572
> >>
> >> Log:
> >>   Remove the kernel optoion for IPSEC_FILTERTUNNEL, which was 
> >> deprecated
> >>   more than 7 years ago in favour of a sysctl in r192648.
> >
> > Need note to UPDAING.
> 
> Why?  The default behaviour hasn’t changed and a new custom kernel 
> with the option will not compile anymore.  Is there a compelling reason 
> to warn users anyway?

Old config don't compiling and need updating? This is break
compatibility and need record in UPDATING (because this place where
find information about like events).
After config updating behaivor changed? Yes. This is need also record
in UPDATING.
For restoring old beheaivor need updating /boot/loader.conf? Yes. This
is also need be documented.

Good documentation is very positive for krama :)
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304572 - in head: sbin/ipfw sys/conf sys/netinet sys/netinet6

[Slawa Olhovchenkov]

On Sun, Aug 21, 2016 at 06:55:30PM +, Bjoern A. Zeeb wrote:

> Author: bz
> Date: Sun Aug 21 18:55:30 2016
> New Revision: 304572
> URL: https://svnweb.freebsd.org/changeset/base/304572
> 
> Log:
>   Remove the kernel optoion for IPSEC_FILTERTUNNEL, which was deprecated
>   more than 7 years ago in favour of a sysctl in r192648.

Need note to UPDAING.

> Modified:
>   head/sbin/ipfw/ipfw.8
>   head/sys/conf/NOTES
>   head/sys/conf/options
>   head/sys/netinet/ip_ipsec.c
>   head/sys/netinet6/ip6_ipsec.c
> 
> Modified: head/sbin/ipfw/ipfw.8
> ==
> --- head/sbin/ipfw/ipfw.8 Sun Aug 21 18:37:21 2016(r304571)
> +++ head/sbin/ipfw/ipfw.8 Sun Aug 21 18:55:30 2016(r304572)
> @@ -1,7 +1,7 @@
>  .\"
>  .\" $FreeBSD$
>  .\"
> -.Dd August 13, 2016
> +.Dd August 21, 2016
>  .Dt IPFW 8
>  .Os
>  .Sh NAME
> @@ -1588,8 +1588,7 @@ Matches IPv4 packets whose precedence fi
>  .It Cm ipsec
>  Matches packets that have IPSEC history associated with them
>  (i.e., the packet comes encapsulated in IPSEC, the kernel
> -has IPSEC support and IPSEC_FILTERTUNNEL option, and can correctly
> -decapsulate it).
> +has IPSEC support, and can correctly decapsulate it).
>  .Pp
>  Note that specifying
>  .Cm ipsec
> 
> Modified: head/sys/conf/NOTES
> ==
> --- head/sys/conf/NOTES   Sun Aug 21 18:37:21 2016(r304571)
> +++ head/sys/conf/NOTES   Sun Aug 21 18:55:30 2016(r304572)
> @@ -626,17 +626,6 @@ options  TCP_OFFLOAD # TCP offload supp
>  options  IPSEC   #IP security (requires device crypto)
>  #options IPSEC_DEBUG #debug for IP security
>  #
> -# #DEPRECATED#
> -# Set IPSEC_FILTERTUNNEL to change the default of the sysctl to force packets
> -# coming through a tunnel to be processed by any configured packet filtering
> -# twice. The default is that packets coming out of a tunnel are _not_ 
> processed;
> -# they are assumed trusted.
> -#
> -# IPSEC history is preserved for such packets, and can be filtered
> -# using ipfw(8)'s 'ipsec' keyword, when this option is enabled.
> -#
> -#options IPSEC_FILTERTUNNEL  #filter ipsec packets from a tunnel
> -#
>  # Set IPSEC_NAT_T to enable NAT-Traversal support.  This enables
>  # optional UDP encapsulation of ESP packets.
>  #
> 
> Modified: head/sys/conf/options
> ==
> --- head/sys/conf/options Sun Aug 21 18:37:21 2016(r304571)
> +++ head/sys/conf/options Sun Aug 21 18:55:30 2016(r304572)
> @@ -424,7 +424,6 @@ IPFIREWALL_VERBOSEopt_ipfw.h
>  IPFIREWALL_VERBOSE_LIMIT opt_ipfw.h
>  IPSECopt_ipsec.h
>  IPSEC_DEBUG  opt_ipsec.h
> -IPSEC_FILTERTUNNEL   opt_ipsec.h
>  IPSEC_NAT_T  opt_ipsec.h
>  IPSTEALTH
>  KRPC
> 
> Modified: head/sys/netinet/ip_ipsec.c
> ==
> --- head/sys/netinet/ip_ipsec.c   Sun Aug 21 18:37:21 2016
> (r304571)
> +++ head/sys/netinet/ip_ipsec.c   Sun Aug 21 18:55:30 2016
> (r304572)
> @@ -68,11 +68,7 @@ __FBSDID("$FreeBSD$");
>  
>  extern   struct protosw inetsw[];
>  
> -#ifdef IPSEC_FILTERTUNNEL
> -static VNET_DEFINE(int, ip4_ipsec_filtertunnel) = 1;
> -#else
>  static VNET_DEFINE(int, ip4_ipsec_filtertunnel) = 0;
> -#endif
>  #define  V_ip4_ipsec_filtertunnel VNET(ip4_ipsec_filtertunnel)
>  
>  SYSCTL_DECL(_net_inet_ipsec);
> 
> Modified: head/sys/netinet6/ip6_ipsec.c
> ==
> --- head/sys/netinet6/ip6_ipsec.c Sun Aug 21 18:37:21 2016
> (r304571)
> +++ head/sys/netinet6/ip6_ipsec.c Sun Aug 21 18:55:30 2016
> (r304572)
> @@ -79,11 +79,7 @@ __FBSDID("$FreeBSD$");
>  
>  extern   struct protosw inet6sw[];
>  
> -#ifdef IPSEC_FILTERTUNNEL
> -static VNET_DEFINE(int, ip6_ipsec6_filtertunnel) = 1;
> -#else
>  static VNET_DEFINE(int, ip6_ipsec6_filtertunnel) = 0;
> -#endif
>  #define  V_ip6_ipsec6_filtertunnel   VNET(ip6_ipsec6_filtertunnel)
>  
>  SYSCTL_DECL(_net_inet6_ipsec6);
> ___
> svn-src-...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/svn-src-all
> To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304555 - head/sys/compat/cloudabi

[Slawa Olhovchenkov]

On Mon, Aug 22, 2016 at 02:49:07AM +1000, Bruce Evans wrote:

> On Sun, 21 Aug 2016, Slawa Olhovchenkov wrote:
> 
> > On Sun, Aug 21, 2016 at 11:39:02PM +1000, Bruce Evans wrote:
> >
> >> On Sun, 21 Aug 2016, Slawa Olhovchenkov wrote:
> >>> I am remeber about platforms with missaligment trap when
> >>> accessing int16 by odd address. Now platforms like this do not exist
> >>> anymore?
> >>
> >> i386 still exists, and it supports trapping on misalignement for at least
> >> CPL 3 (not kernel CPL 0).  IIRC, amd64 drops support for this.
> >
> > Someone enable and support this? I am don't see.
> > May be PPC trap on this?
> > Alpha trap on this, but support of Alpha is droped.
> 
> It is a 1-line change in asm (or a little more in C with #includes) to
> enable the trap:

OK, we can turn amd64 in this mode.
And cat do request to kernel with unalligned access, this cause trap
and panic, yes?

> It is a trillion-line change to fix the compilers and applications to not
> do misaligned accesses :-).  I only tried to use this ~25 years ago.  Then
> the most obvious compiler bug was generating 32-bit acccesses to assign
> large but misaligned structs.  If the compiler just generated calls to
> memcpy(), that might work, but in practice libraries also assume alignment.

This issuse can be trigerred and by two-bytes assigmen, yes?

> >>>> There are also endianness problems.  The old version was even more broken
> >>>> on big endian systems.  The current version needs some magic to reverse
> >>>> the memcpy() of the bits.  We already depend on this for some 64-bit
> >>>> syscalls like lseek().
> >>>
> >>> Can you explain some more?
> >>> This is not transfer over network and don't read from external media.
> >>> Where is problem?
> >>
> >> It is similar to a network transfer.  It needs a protocol to pass values
> >> to applications.  Type puns are fragile even within a single compilation
> >> unit.
> >
> > Application ad kernel run with same byte order, not?
> 
> The application can do anything it wants, but has to translate if it uses
> the kernel or a library written in another language.

You talk about different byte order in differenr languages?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304555 - head/sys/compat/cloudabi

[Slawa Olhovchenkov]

On Sun, Aug 21, 2016 at 11:39:02PM +1000, Bruce Evans wrote:

> On Sun, 21 Aug 2016, Slawa Olhovchenkov wrote:
> 
> > On Sun, Aug 21, 2016 at 11:00:24PM +1000, Bruce Evans wrote:
> >
> >> On Sun, 21 Aug 2016, Slawa Olhovchenkov wrote:
> >>
> >>> On Sun, Aug 21, 2016 at 09:32:35PM +1000, Bruce Evans wrote:
> >>> ...
> >>>> *(foo_t *)asks for alignment bugs.  We have already fixed lots of these
> >>>> bugs for copying struct timevals in places like ping.c.  Compilers warn
> >>>> about misalignment when certain warnings are enabled, but only on arches
> >>>> where misalignment is more than a pessimization.  There is no reason why
> >>>> td_retval would be always aligned on these arches.  Alignment of 64-bit
> >>>> types on 32-bit arches is usually so unimportant that even int32_t is
> >>>> not required to be aligned by the ABI, and there is no point in
> >>>> aligning td_retval specially unless you also do it for a large fraction
> >>>> of 64-bit integers in the kernel, and there are negative points for
> >>>> doing that.
> >>>
> >>> For eliminate aligment bugs need to replace all assigment more then 1
> >>> bytes to *td_retval by memcpy?
> >>
> >> The copying must be of size 1 or 2 ints unless you are making even larger
> >> type puns than now.  1 int is obviously safe to just assign, and 2 ints
> >> should use memcpy().
> >
> > Why?
> 
> If it has size not 1 * sizeof(int) or 2 * sizeof(int) or is not an integer,
> than it is had to assign to a 2-byte array and might need more careful
> packing just to memcpy() it.

I am miss you point.

> > I am remeber about platforms with missaligment trap when
> > accessing int16 by odd address. Now platforms like this do not exist
> > anymore?
> 
> i386 still exists, and it supports trapping on misalignement for at least
> CPL 3 (not kernel CPL 0).  IIRC, amd64 drops support for this.

Someone enable and support this? I am don't see.
May be PPC trap on this?
Alpha trap on this, but support of Alpha is droped.


> >> There are also endianness problems.  The old version was even more broken
> >> on big endian systems.  The current version needs some magic to reverse
> >> the memcpy() of the bits.  We already depend on this for some 64-bit
> >> syscalls like lseek().
> >
> > Can you explain some more?
> > This is not transfer over network and don't read from external media.
> > Where is problem?
> 
> It is similar to a network transfer.  It needs a protocol to pass values
> to applications.  Type puns are fragile even within a single compilation
> unit.

Application ad kernel run with same byte order, not?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304555 - head/sys/compat/cloudabi

[Slawa Olhovchenkov]

On Sun, Aug 21, 2016 at 11:00:24PM +1000, Bruce Evans wrote:

> On Sun, 21 Aug 2016, Slawa Olhovchenkov wrote:
> 
> > On Sun, Aug 21, 2016 at 09:32:35PM +1000, Bruce Evans wrote:
> >...
> >> *(foo_t *)asks for alignment bugs.  We have already fixed lots of these
> >> bugs for copying struct timevals in places like ping.c.  Compilers warn
> >> about misalignment when certain warnings are enabled, but only on arches
> >> where misalignment is more than a pessimization.  There is no reason why
> >> td_retval would be always aligned on these arches.  Alignment of 64-bit
> >> types on 32-bit arches is usually so unimportant that even int32_t is
> >> not required to be aligned by the ABI, and there is no point in
> >> aligning td_retval specially unless you also do it for a large fraction
> >> of 64-bit integers in the kernel, and there are negative points for
> >> doing that.
> >
> > For eliminate aligment bugs need to replace all assigment more then 1
> > bytes to *td_retval by memcpy?
> 
> The copying must be of size 1 or 2 ints unless you are making even larger
> type puns than now.  1 int is obviously safe to just assign, and 2 ints
> should use memcpy().

Why? I am remeber about platforms with missaligment trap when
accessing int16 by odd address. Now platforms like this do not exist
anymore?

> There are also endianness problems.  The old version was even more broken
> on big endian systems.  The current version needs some magic to reverse
> the memcpy() of the bits.  We already depend on this for some 64-bit
> syscalls like lseek().

Can you explain some more?
This is not transfer over network and don't read from external media.
Where is problem?

> Hmm, lseek() uses different magic.  Instead of the memcpy(), we assign
> to tdu_off in td_uretoff.  td_retval is really td_uretoff.tdu_retval
> obfuscated like this for compatibilty.  This is slightly better than
> the memcpy() since it makes tdu_off and also tdu_retval 64-bit aligned
> if that is required for off_t.  The same magic still occurs in userland.
> On normal 32-bit arches, the 2 integers arrive in 2 registers that have
> to be combined in the right order to give a 64-bit value.  The magic is
> just that things are arranged so that no code is needed to rearrange the
> registers in the usual case where the application uses a similar ABI to
> the kernel.  Not the same ABI, since the application might be 32 bits
> and the kernel 64 bits.  This requires lots of conversions, but none for
> register order for at least x86.
> 
> Bruce
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r304555 - head/sys/compat/cloudabi

[Slawa Olhovchenkov]

On Sun, Aug 21, 2016 at 09:32:35PM +1000, Bruce Evans wrote:

> On Sun, 21 Aug 2016, Slawa Olhovchenkov wrote:
> 
> > On Sun, Aug 21, 2016 at 07:41:11AM +, Ed Schouten wrote:
> >> ...
> >> Log:
> >>   Use memcpy() to copy 64-bit timestamps into the syscall return values.
> >>
> >>   On 32-bit platforms, our 64-bit timestamps need to be split up across
> >>   two registers. A simple assignment to td_retval[0] will cause the top 32
> >>   bits to get lost. By using memcpy(), we will automatically either use 1
> >>   or 2 registers depending on the size of register_t.
> >> ..
> >> Modified: head/sys/compat/cloudabi/cloudabi_clock.c
> >> ==
> >> --- head/sys/compat/cloudabi/cloudabi_clock.c  Sun Aug 21 07:28:38 
> >> 2016(r304554)
> >> +++ head/sys/compat/cloudabi/cloudabi_clock.c  Sun Aug 21 07:41:11 
> >> 2016(r304555)
> >> @@ -117,7 +117,7 @@ cloudabi_sys_clock_res_get(struct thread
> >>error = cloudabi_convert_timespec(, );
> >>if (error != 0)
> >>return (error);
> >> -  td->td_retval[0] = cts;
> >> +  memcpy(td->td_retval, , sizeof(cts));
> >>return (0);
> >>  }
> >>
> >> @@ -129,6 +129,6 @@ cloudabi_sys_clock_time_get(struct threa
> >>int error;
> >>
> >>error = cloudabi_clock_time_get(td, uap->clock_id, );
> >> -  td->td_retval[0] = ts;
> >> +  memcpy(td->td_retval, , sizeof(ts));
> >>return (error);
> >
> > Why do not use more simple solution:
> 
> Because it doesn't work.
> 
> > *(cloudabi_timestamp_t *)td->td_retval = cts;
> >
> > This is eliminate call to memcpy and allow compiler to use most
> > effeicient way.
> 
> memcpy() is the most efficient way (except for the kernel pessimization
> of losing builtin memcpy with -ffreestanding 15 years ago and not bringing
> it back).

> *(foo_t *)asks for alignment bugs.  We have already fixed lots of these
> bugs for copying struct timevals in places like ping.c.  Compilers warn
> about misalignment when certain warnings are enabled, but only on arches
> where misalignment is more than a pessimization.  There is no reason why
> td_retval would be always aligned on these arches.  Alignment of 64-bit
> types on 32-bit arches is usually so unimportant that even int32_t is
> not required to be aligned by the ABI, and there is no point in
> aligning td_retval specially unless you also do it for a large fraction
> of 64-bit integers in the kernel, and there are negative points for
> doing that.

For eliminate aligment bugs need to replace all assigment more then 1
bytes to *td_retval by memcpy?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"