Jenkins build is unstable: FreeBSD_HEAD #162

[jenkins-admin]

See 

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

freebsd-update

[Alexis Megas]

Hello.

Please consider a new clean command in the freebsd-update script. The modified 
manual and script are located at https://github.com/textbrowser/freebsd-update. 
Included are two diffs. Sorry for the long e-mail.

--- /usr/src/usr.sbin/freebsd-update/freebsd-update.8    2015-08-12 
10:21:35.0 -0400
+++ ./freebsd-update.8    2016-04-02 15:16:47.780095000 -0400
@@ -119,6 +119,12 @@
 .Cm command
 can be any one of the following:
 .Bl -tag -width "rollback"
+.It Cm clean
+Remove the contents of
+.Ar workdir .
+(default:
+.Ar /var/db/freebsd-update/
+).
 .It Cm fetch
 Based on the currently installed world and the configuration
 options set, fetch all available binary updates.

--- /usr/src/usr.sbin/freebsd-update/freebsd-update.sh    2015-08-12 
10:21:35.0 -0400
+++ ./freebsd-update.sh    2016-04-02 15:26:57.990003000 -0400
@@ -53,6 +53,7 @@
   --not-running-from-cron
    -- Run without a tty, for use by automated tools
 Commands:
+  clean    -- Clean workdir
   fetch    -- Fetch updates from server
   cron -- Sleep rand(3600) seconds, fetch updates, and send an
   email if updates were found
@@ -474,7 +475,7 @@
         ;;
 
     # Commands
-        cron | fetch | upgrade | install | rollback | IDS)
+        clean | cron | fetch | upgrade | install | rollback | IDS)
         COMMANDS="${COMMANDS} $1"
         ;;
 
@@ -559,6 +560,25 @@
 mergeconfig
 }
 
+# Perform sanity checks in preparation of cleaning workdir.
+clean_check_params () {
+    # Check that we are root.  All sorts of things won't work otherwise.
+    if [ `id -u` != 0 ]; then
+        echo "You must be root to run this."
+        exit 1
+    fi
+
+    # Check that we have a working directory.
+    _WORKDIR_bad="Directory does not exist or is not writable: "
+    if ! [ -d "${WORKDIR}" -a -w "${WORKDIR}" ]; then
+        echo -n "`basename $0`: "
+        echo -n "${_WORKDIR_bad}"
+        echo ${WORKDIR}
+        exit 1
+    fi
+    cd ${WORKDIR} || exit 1
+}
+
 # Set utility output filtering options, based on ${VERBOSELEVEL}
 fetch_setup_verboselevel () {
 case ${VERBOSELEVEL} in
@@ -2047,6 +2067,11 @@
 echo ${NOWTIME} > lasteolwarn
 }
 
+# Clean workdir.
+clean_run() {
+    rm -fr *
+}
+
 # Do the actual work involved in "fetch" / "cron".
 fetch_run () {
 workdir_init || return 1
@@ -3225,6 +3250,12 @@
 default_params
 }
 
+# Clean command. Allow non-interactive use.
+cmd_clean () {
+    clean_check_params
+    clean_run || exit 1
+}
+
 # Fetch command.  Make sure that we're being called
 # interactively, then run fetch_check_params and fetch_run
 cmd_fetch () {

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Build failed in Jenkins: FreeBSD_HEAD #161

[jenkins-admin]

See 

--
Started by user rodrigc
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/freebsd/freebsd-ci.git # 
 > timeout=10
Fetching upstream changes from https://github.com/freebsd/freebsd-ci.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://github.com/freebsd/freebsd-ci.git +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 926429031e0241da821577c12b4b8f7db789e7e1 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 926429031e0241da821577c12b4b8f7db789e7e1
 > git rev-list 926429031e0241da821577c12b4b8f7db789e7e1 # timeout=10
[Pipeline] Allocate node : Start
Still waiting to schedule task
Waiting for next available executor on jenkins-10.freebsd.org
Running on jenkins-10.freebsd.org in /builds/workspace/FreeBSD_HEAD
[Pipeline] node {
[Pipeline] pwd
[Pipeline] sh
[FreeBSD_HEAD] Running shell script
+ sudo chown -R jenkins .
[Pipeline] deleteDir
[Pipeline] } //node
[Pipeline] Allocate node : End
[Pipeline] Allocate node : Start
Still waiting to schedule task
havoc.ysv.freebsd.org is offline
Running on havoc.ysv.freebsd.org in /builds/workspace/FreeBSD_HEAD
[Pipeline] node {
[Pipeline] Change current directory : Start
Running in /builds/workspace/FreeBSD_HEAD/freebsd-ci
[Pipeline] dir {
[Pipeline] git
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/freebsd/freebsd-ci.git # 
 > timeout=10
Fetching upstream changes from https://github.com/freebsd/freebsd-ci.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://github.com/freebsd/freebsd-ci.git +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 926429031e0241da821577c12b4b8f7db789e7e1 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 926429031e0241da821577c12b4b8f7db789e7e1
[Pipeline] } //dir
[Pipeline] Change current directory : End
[Pipeline] writeFile
[Pipeline] } //node
[Pipeline] Allocate node : End
[Pipeline] Allocate node : Start
Running on master in /usr/local/jenkins/workspace/FreeBSD_HEAD
[Pipeline] node {

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: FreeBSD_HEAD_amd64_gcc - Build #1147 - Still Failing

[Li-Wen Hsu]

On Tue, Apr 05, 2016 at 09:22:11 -0700, Bryan Drewery wrote:
> On 4/5/16 3:54 AM, jenkins-ad...@freebsd.org wrote:
> > + sudo pkg install -y devel/amd64-xtoolchain-gcc
> > Updating FreeBSD repository catalogue...
> > FreeBSD repository is up-to-date.
> > All repositories are up-to-date.
> > Build step 'Execute shell' marked build as failure
> 
> pkg install -y is returning an error now here where it didn't before.

Yeah, Baptiste is working on a fix and I think it will be completed
soon.

I'll disable this step in the build process temporarily until pkg is
fixed.

Li-Wen

-- 
Li-Wen Hsu 
https://lwhsu.org


pgp3qOLaJqSHL.pgp
Description: PGP signature

Re: FreeBSD_HEAD_amd64_gcc - Build #1147 - Still Failing

[Bryan Drewery]

On 4/5/16 3:54 AM, jenkins-ad...@freebsd.org wrote:
> + sudo pkg install -y devel/amd64-xtoolchain-gcc
> Updating FreeBSD repository catalogue...
> FreeBSD repository is up-to-date.
> All repositories are up-to-date.
> Build step 'Execute shell' marked build as failure

pkg install -y is returning an error now here where it didn't before.

-- 
Regards,
Bryan Drewery
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: accessing a PCIe register from userspace through kmem or other ways ?

[Ryan Stone]

On Tue, Apr 5, 2016 at 2:10 AM, Konstantin Belousov 
wrote:

> The mmap(2) interface to /dev/mem did not have the issue ever.  The problem
> was only with the read(2)/write(2) accesses.
>

I mis-remembered my situation.  I was performing a read on /dev/mem rather
than reading through a mmap.  Thanks for clearing this up.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

FreeBSD_HEAD_amd64_gcc - Build #1148 - Fixed

[jenkins-admin]

FreeBSD_HEAD_amd64_gcc - Build #1148 - Fixed:

Build information: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc/1148/
Full change log: 
https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc/1148/changes
Full build log: 
https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc/1148/console

Change summaries:

297584 by skra:
Fix typo. No functional change.

297583 by ian:
Add more DPRINTF() to the ftdi driver.  Now everything that can change the
chip's state has a DPRINTF, with things that happen repeatedly at debug=2
level and things that happen frequently (like per-transfer IO) at debug=3.

297582 by skra:
Rework BCM283x gpio interrupt controller for INTRNG. It's used on RPI-B
and RPI2 where INTRNG is already enabled by default.

Differential Revision:  https://reviews.freebsd.org/D5810

297581 by skra:
Implement bcm2836 interrupt controller for INTRNG and enable it
on RPI2 by default.

Differential Revision:  https://reviews.freebsd.org/D5822

297580 by skra:
Rework bcm283x interrupt controller for INTRNG and enable it
on RPI-B by default.

Reviewed by:gonzo
Differential Revision:  https://reviews.freebsd.org/D5809

297579 by mmel:
ehci_interrupt is MPSAFE code. Most drivers in tree calls bus_setup_intr
with MPSAFE, some are not. Fix those.

Submitted by: Howard Su 
Differential Revision: https://reviews.freebsd.org/D5755

297578 by trasz:
Use proper locking macros in RACCT in RCTL.

MFC after:  1 month
Sponsored by:   The FreeBSD Foundation

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Packaging the FreeBSD base system with pkg(8)

[Allan Jude]

On 2016-04-05 13:42, Benjamin Kaduk wrote:

On Tue, 5 Apr 2016, Gary Jennejohn wrote:


Will there be an option not to merge?  I never update /etc when
I do installworld because what I have works for me and I see no
need to make any changes to a working system.


And you expect your system to continue working after a new system user is
added?

-Ben
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



At a minimum, you need to run 'mergemaster -p', which does things 
required for 'installworld' to actually succeed.


--
Allan Jude
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Packaging the FreeBSD base system with pkg(8)

[Benjamin Kaduk]

On Tue, 5 Apr 2016, Gary Jennejohn wrote:

> Will there be an option not to merge?  I never update /etc when
> I do installworld because what I have works for me and I see no
> need to make any changes to a working system.

And you expect your system to continue working after a new system user is
added?

-Ben
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: accessing a PCIe register from userspace through kmem or other ways ?

[Konstantin Belousov]

On Tue, Apr 05, 2016 at 08:27:32PM +0300, Konstantin Belousov wrote:
> On Tue, Apr 05, 2016 at 10:02:07AM -0700, John Baldwin wrote:
> > For the ioctl I planned to either 1) call vm_mmap_object() or the like 
> > directly
> > and return the virtual address to the user, or 2) return the mmap offset to 
> > use
> > from the ioctl that the user would then supply to mmap() and d_mmap_single 
> > would
> > use to find the object created by the ioctl.  1) is probably simpler and is 
> > more
> > what I was leaning towards.  Still, I want to be able to handle 
> > invalidations
> > either by pinning the BAR while the object is mapped, or being able to 
> > invalidate
> > the object.  Given that you can eject a hotplug PCI device, I think explicit
> > invalidation is the better route in this case.  I would create a VM object 
> > for
> > each BAR on the first mmap request and save a reference to it in the PCI bus
> > ivars.  If the BAR is ever cleared I would be able to find the object and
> > invalidate it ensuring any programs that then tried to access it would get a
> > page fault instead of accessing some other random thing.
> 
> Option 2) is what I discussed, and what has been used for GEM and TTM.
> It allows to create an object per buffer (per BAR for /dev/pci case),
> and you indeed can easily iterate over managed pages belonging to the
> given buffer/BAR because they belong to the object' queue.
> 
> This scheme utilizes d_mmap_single() on the 'global' object (/dev/pci,
> /dev/card/dri etc), which takes offset and decodes it into the
> buffer/BAR.
> 
> In my opinion, it is prettier than explicit call to vm_mmap_object()
> since it leaves all high-level stuff to the VM subsystem proper. Driver
> only needs to create the suitable object (and manage offsets).
> 
> On the other hand, GEM has to emulate another Linux interface, where
> ioctl() really performs mapping. But again, there it is simpler
> to ensure that vm_object for buffer/BAR is created, and then call
> vm_map_find((), not even touching middle-level of vm_mmap_object(). See
> i915_gem_mmap_ioctl().
> 

Important thing I forgot about, is that managed fictitious pages,
which are created by MGTDEVICE pagers, must be supported by the
pmap.  It is not hard, the issue is that typical pmap does not
make a distinction between unmanaged and fictitious pages.

For x86, this was implemented by r224746 + r233168.  For other pmaps,
r224746 alone might be enough, but it was never tested for clear reasons.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: accessing a PCIe register from userspace through kmem or other ways ?

[Konstantin Belousov]

On Tue, Apr 05, 2016 at 10:02:07AM -0700, John Baldwin wrote:
> For the ioctl I planned to either 1) call vm_mmap_object() or the like 
> directly
> and return the virtual address to the user, or 2) return the mmap offset to 
> use
> from the ioctl that the user would then supply to mmap() and d_mmap_single 
> would
> use to find the object created by the ioctl.  1) is probably simpler and is 
> more
> what I was leaning towards.  Still, I want to be able to handle invalidations
> either by pinning the BAR while the object is mapped, or being able to 
> invalidate
> the object.  Given that you can eject a hotplug PCI device, I think explicit
> invalidation is the better route in this case.  I would create a VM object for
> each BAR on the first mmap request and save a reference to it in the PCI bus
> ivars.  If the BAR is ever cleared I would be able to find the object and
> invalidate it ensuring any programs that then tried to access it would get a
> page fault instead of accessing some other random thing.

Option 2) is what I discussed, and what has been used for GEM and TTM.
It allows to create an object per buffer (per BAR for /dev/pci case),
and you indeed can easily iterate over managed pages belonging to the
given buffer/BAR because they belong to the object' queue.

This scheme utilizes d_mmap_single() on the 'global' object (/dev/pci,
/dev/card/dri etc), which takes offset and decodes it into the
buffer/BAR.

In my opinion, it is prettier than explicit call to vm_mmap_object()
since it leaves all high-level stuff to the VM subsystem proper. Driver
only needs to create the suitable object (and manage offsets).

On the other hand, GEM has to emulate another Linux interface, where
ioctl() really performs mapping. But again, there it is simpler
to ensure that vm_object for buffer/BAR is created, and then call
vm_map_find((), not even touching middle-level of vm_mmap_object(). See
i915_gem_mmap_ioctl().

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: accessing a PCIe register from userspace through kmem or other ways ?

[John Baldwin]

On Tuesday, April 05, 2016 07:21:29 PM Konstantin Belousov wrote:
> On Tue, Apr 05, 2016 at 08:55:54AM -0700, John Baldwin wrote:
> > Mostly I do not have experience with MGTDEVICE, though I was planning to
> > look at it as a way to implement this.  Two things though: 1) there may
> > not be a cdev to associate with, and 2) I know of at least one device driver
> > that would use this in addition to using this for a general "map this BAR"
> > ioctl on /dev/pci.
> So /dev/pci is the natural cdev to place the functionality.
> An ioctl on /dev/pci may mmap BAR and return the base address.
> 
> > There are other cases in the past where I used OBJT_SG
> > but would have preferred to use a variant that used managed pages so that
> > I could invidate any existing mappings.  In particular what I want to do
> > is invalidate an object so that any future uses fail.
> > 
> > Alternatively, it might be nice to hook a destructor call into a VM object
> > so that I could know when the object is no longer in use (knowing that all
> > its mappings have been destroyed).  When using OBJT_SG objects as aliases
> > for other things (memory allocated via contigmalloc or bus_dma or for
> > resources like PCI BARs), I could keep a reference count on the original
> > "thing" that I increment when creating an OBJT_SG object to return from
> > something like d_mmap_single() or the /dev/pci ioctl and drop the reference
> > count in the destructor hook for that object.
> This is in essence how GEM objects + MGTDEVICE mappings work for i915.
> 
> The only bottleneck in the API arrangement is that d_mmap_single() only
> gets the offset as the identifying data to construct the mapping.
> For /dev/pci, the offset parameter would need to encode d:b:s:f and BAR
> index.

For the ioctl I planned to either 1) call vm_mmap_object() or the like directly
and return the virtual address to the user, or 2) return the mmap offset to use
from the ioctl that the user would then supply to mmap() and d_mmap_single would
use to find the object created by the ioctl.  1) is probably simpler and is more
what I was leaning towards.  Still, I want to be able to handle invalidations
either by pinning the BAR while the object is mapped, or being able to 
invalidate
the object.  Given that you can eject a hotplug PCI device, I think explicit
invalidation is the better route in this case.  I would create a VM object for
each BAR on the first mmap request and save a reference to it in the PCI bus
ivars.  If the BAR is ever cleared I would be able to find the object and
invalidate it ensuring any programs that then tried to access it would get a
page fault instead of accessing some other random thing.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: accessing a PCIe register from userspace through kmem or other ways ?

[Konstantin Belousov]

On Tue, Apr 05, 2016 at 08:55:54AM -0700, John Baldwin wrote:
> Mostly I do not have experience with MGTDEVICE, though I was planning to
> look at it as a way to implement this.  Two things though: 1) there may
> not be a cdev to associate with, and 2) I know of at least one device driver
> that would use this in addition to using this for a general "map this BAR"
> ioctl on /dev/pci.
So /dev/pci is the natural cdev to place the functionality.
An ioctl on /dev/pci may mmap BAR and return the base address.

> There are other cases in the past where I used OBJT_SG
> but would have preferred to use a variant that used managed pages so that
> I could invidate any existing mappings.  In particular what I want to do
> is invalidate an object so that any future uses fail.
> 
> Alternatively, it might be nice to hook a destructor call into a VM object
> so that I could know when the object is no longer in use (knowing that all
> its mappings have been destroyed).  When using OBJT_SG objects as aliases
> for other things (memory allocated via contigmalloc or bus_dma or for
> resources like PCI BARs), I could keep a reference count on the original
> "thing" that I increment when creating an OBJT_SG object to return from
> something like d_mmap_single() or the /dev/pci ioctl and drop the reference
> count in the destructor hook for that object.
This is in essence how GEM objects + MGTDEVICE mappings work for i915.

The only bottleneck in the API arrangement is that d_mmap_single() only
gets the offset as the identifying data to construct the mapping.
For /dev/pci, the offset parameter would need to encode d:b:s:f and BAR
index.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: accessing a PCIe register from userspace through kmem or other ways ?

[John Baldwin]

On Tuesday, April 05, 2016 09:14:31 AM Konstantin Belousov wrote:
> On Mon, Apr 04, 2016 at 03:45:07PM -0700, John Baldwin wrote:
> > However, another question is how to deal with systems that do bus address
> > translation (like the arm64 ThunderX boxes) where the values in the PCI
> > BAR are not CPU physical addresses.  To do this properly we may need some
> > sort of bus method akin to my bus_map_resource() WIP but one that instead
> > returns a suitable 'struct sglist' for a given 'struct resource *' that
> > can be used with OBJT_SG to build a VM object to use for the mapping.
> Is there any documentation on the ThunderX PCI access mechanisms ?

The Host-PCI (or really some other bus -> PCI) bridges apply a fixed offset
to PCI addresses, so if your BAR has address X written to it, the CPU has
to use an address of X + Y (where Y is the PCI window base address) to access
it.  The bridge device decodes the access to address X + Y and generates a
PCI transaction to address X.  It's not clear to me how DMA works for these
devices (e.g. if all DMA requests from PCI devices have to use an IOMMU built
into the bridge).

> > (At some point I do think I would like a variant of OBJT_SG that used
> > managed pages so that mappings could be revoked when whatever is backing
> > the sglist is disabled (e.g. the device is ejected or the BAR is
> > relocated, etc.).)
> Why cannot you use MGTDEVICE pager already ? Driver would need to
> maintain its private list of pages, and from this PoV, a convenience
> wrapper around MGTDEVICE which unifies operations on sg lists could be
> useful. Still, it could be that the wrapper appear to be single-purpose.

Mostly I do not have experience with MGTDEVICE, though I was planning to
look at it as a way to implement this.  Two things though: 1) there may
not be a cdev to associate with, and 2) I know of at least one device driver
that would use this in addition to using this for a general "map this BAR"
ioctl on /dev/pci.  There are other cases in the past where I used OBJT_SG
but would have preferred to use a variant that used managed pages so that
I could invidate any existing mappings.  In particular what I want to do
is invalidate an object so that any future uses fail.

Alternatively, it might be nice to hook a destructor call into a VM object
so that I could know when the object is no longer in use (knowing that all
its mappings have been destroyed).  When using OBJT_SG objects as aliases
for other things (memory allocated via contigmalloc or bus_dma or for
resources like PCI BARs), I could keep a reference count on the original
"thing" that I increment when creating an OBJT_SG object to return from
something like d_mmap_single() or the /dev/pci ioctl and drop the reference
count in the destructor hook for that object.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Packaging the FreeBSD base system with pkg(8)

[Baptiste Daroussin]

On Tue, Apr 05, 2016 at 12:22:49PM +0200, Gary Jennejohn wrote:
> On Tue, 5 Apr 2016 10:22:04 +0100
> David Chisnall  wrote:
> 
> > On 5 Apr 2016, at 10:07, Gergely Czuczy  wrote:
> > > 
> > > Also, quite often entries from the base system are changed
> > > manually, think of root's/toor's password.  Are such cases
> > > going to be dealt with properly between upgrades, including
> > > self-built-and-packaged base systems?  Currently it can be a
> > > PITA with mergemaster to handle things like master.passwd
> > > properly between upgrades, automation so far wasn't famous on
> > > doing it properly. 
> > 
> > Mergemaster uses a 2-way merge.  It has the version that you
> > have installed and the version that's being proposed for
> > installation.  Etcupdate and pkg perform a 3-way merge.  It has
> > the pristine version, the version that you have made changes
> > to, and the new version.  If you have changed an entry and so
> > has the package, then you will get a conflict that you have to
> > resolve manually.  If you have added lines and so has the
> > upstream version, then that should cleanly apply.  Similarly,
> > if you and upstream have both modified different lines, then
> > there should be no problem.
> > 
> 
> Will there be an option not to merge?  I never update /etc when
> I do installworld because what I have works for me and I see no
> need to make any changes to a working system.

Yes pkg has an option to not merge and will give you some .pkgnew files

Bapt


signature.asc
Description: PGP signature

FreeBSD_HEAD_amd64_gcc - Build #1147 - Still Failing

[jenkins-admin]

FreeBSD_HEAD_amd64_gcc - Build #1147 - Still Failing:

Build information: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc/1147/
Full change log: 
https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc/1147/changes
Full build log: 
https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc/1147/console

Change summaries:

297577 by avg:
x86 topo: add some comments, descriptions and references to documentation

Plus a minor cosmetic change.

MFC after:  1 month

297576 by mmel:
TEGRA: Fix CPU frequency switching.
The PLL_X, base CPU frequency source, doesn't have a bypass switch and thus
we must use another frequency source for CPU while changing its frequency.
PLL_P is ideal for this, it runs at 480MHz and CPU can be clocked at this
frequency at any CPU voltage.

297573 by jhibbits:
Add support for the Microchip mcp7941x.

This is compatible with the ds1307, but comparing the mcp7941x datasheet vs the
ds1307 code, appears there is one bit placement difference, so that is now
accounted for.

Relnotes:   yes

297572 by jhibbits:
Make i2c device child auto-probe work for MPC85xx and QorIQ SoCs.

OFW i2c probing requires a new method ofw_bus_get_node(), and the bus device is
assumed iichb.  With these changes, i2c devices attached in fdt are probed and
attached automagically.

297571 by wblock:
Add another real-life example of setting a quirk for a USB gaming
keyboard.  From forum thread: https://forums.freebsd.org/threads/55717/

MFC after:  1 week

297570 by jhb:
Remove a redundant check.

cpu_suspend_map is always empty if smp_started is false.

Sponsored by:   Netflix

297569 by jhb:
Remove an unneeded check.

CPUs with valid per-CPU data are not absent.

Sponsored by:   Netflix

297568 by jhb:
Don't wakeup the fdc worker thread once a second when idle.

The fdc worker thread was using a one second timeout while waiting for
a new bio to arrive or for the device to detach.  However, the driver
already does a wakeup when queueing a new bio or asking the thread to
detach, so the timeout only served to waste CPU time waking up the
thread once a second just so it could go right back to sleep.  Use an
infinite timeout instead.

Discussed with: phk
Sponsored by:   Netflix

297566 by bdrewery:
DIRDEPS_BUILD: Use 1 parameter for defining -rpath-link.

Sponsored by:   EMC / Isilon Storage Division

297565 by adrian:
[net80211] Add an A-MSDU debug output shortcut.

297564 by glebius:
Add early_customize_cmd() that allows to register custom functions run
before the build stage.

Reviewed by:imp
Obtained from:  Netflix

297563 by adrian:
[net80211] teach wlanstats about the ff_encapfail field.

Without this it just displays a blank, short column which is just
plainly not useful.

297562 by adrian:
[net80211] add amsdu and fast frames encap failure counters in the ioctl
definition.

The code to set these will come in a subsequent commit (when I start
fleshing out A-MSDU support.)

297561 by andrew:
Add a table to map from the FreeBSD CPUID space to the GIC CPUID space. On
many SoCs these two are the same, however there is no requirement for this
to be the case, e.g. on the ARM Juno we boot on what the GIC thinks of as
CPU 2, but FreeBSD numbers it CPU 0.

Obtained from:  ABT Systems Ltd
Sponsored by:   The FreeBSD Foundation

297558 by avg:
new x86 smp topology detection code

Previously, the code determined a topology of processing units
(hardware threads, cores, packages) and then deduced a cache topology
using certain assumptions.  The new code builds a topology that
includes both processing units and caches using the information
provided by the hardware.

At the moment, the discovered full topology is used only to creeate
a scheduling topology for SCHED_ULE.
There is no KPI for other kernel uses.

Summary:
- based on APIC ID derivation rules for Intel and AMD CPUs
- can handle non-uniform topologies
- requires homogeneous APIC ID assignment (same bit widths for ID
  components)
- topology for dual-node AMD CPUs may not be optimal
- topology for latest AMD CPU models may not be optimal as the code is
  several years old
- supports only thread/package/core/cache nodes

Todo:
  - AMD dual-node processors
  - latest AMD processors
  - NUMA nodes
  - checking for homogeneity of the APIC ID assignment across packages
  - more flexible cache placement within topology
  - expose topology to userland, e.g., via sysctl nodes

Long term todo:
  - KPI for CPU sharing and affinity with respect to various resources
(e.g., two logical processors may share the same FPU, etc)

Reviewed by:mav
Tested by:  mav
MFC after:  1 month
Differential Revision:  https://reviews.freebsd.org/D2728

297557 by ache:
SJIS encoding don't have single byte characters >= 224

MFC after:  1 week

297556 by andrew:
Reduce the diff for when we switch to intrng. The IPI interrupts will be
split out to multiple handlers.

Obtained from:  ABT Systems Ltd
Sponsored by:   The FreeBSD Foundation



The end of the build 

Re: Packaging the FreeBSD base system with pkg(8)

[Gary Jennejohn]

On Tue, 5 Apr 2016 10:22:04 +0100
David Chisnall  wrote:

> On 5 Apr 2016, at 10:07, Gergely Czuczy  wrote:
> > 
> > Also, quite often entries from the base system are changed
> > manually, think of root's/toor's password.  Are such cases
> > going to be dealt with properly between upgrades, including
> > self-built-and-packaged base systems?  Currently it can be a
> > PITA with mergemaster to handle things like master.passwd
> > properly between upgrades, automation so far wasn't famous on
> > doing it properly. 
> 
> Mergemaster uses a 2-way merge.  It has the version that you
> have installed and the version that's being proposed for
> installation.  Etcupdate and pkg perform a 3-way merge.  It has
> the pristine version, the version that you have made changes
> to, and the new version.  If you have changed an entry and so
> has the package, then you will get a conflict that you have to
> resolve manually.  If you have added lines and so has the
> upstream version, then that should cleanly apply.  Similarly,
> if you and upstream have both modified different lines, then
> there should be no problem.
> 

Will there be an option not to merge?  I never update /etc when
I do installworld because what I have works for me and I see no
need to make any changes to a working system.

-- 
Gary Jennejohn
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Packaging the FreeBSD base system with pkg(8)

[David Chisnall]

On 5 Apr 2016, at 10:07, Gergely Czuczy  wrote:
> 
> Also, quite often entries from the base system are changed manually, think of 
> root's/toor's password. Are such cases going to be dealt with properly 
> between upgrades, including self-built-and-packaged base systems? Currently 
> it can be a PITA with mergemaster to handle things like master.passwd 
> properly between upgrades, automation so far wasn't famous on doing it 
> properly.

Mergemaster uses a 2-way merge.  It has the version that you have installed and 
the version that’s being proposed for installation.  Etcupdate and pkg perform 
a 3-way merge.  It has the pristine version, the version that you have made 
changes to, and the new version.  If you have changed an entry and so has the 
package, then you will get a conflict that you have to resolve manually.  If 
you have added lines and so has the upstream version, then that should cleanly 
apply.  Similarly, if you and upstream have both modified different lines, then 
there should be no problem.

David

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Packaging the FreeBSD base system with pkg(8)

[Gergely Czuczy]

On 2016-01-27 23:33, Glen Barber wrote:

As many know, work has been in progress for quite some time to provide
the ability to package and upgrade the FreeBSD base system using pkg(8).
The majority of the initial implementation has provided much of the core
functionality to make this possible, however much work still needs to be
done.

Over the past few weeks, there have been several inquiries on if this
work is still targeted for the 11.0-RELEASE, as well as the status of
the project branch (base/projects/release-pkg).

The answer to the first question is: Yes.  This is still targeted for
11.0-RELEASE, which was one of the requirements during discussion of the
new support model announced early last year [1].

The status of the in progress work is a bit more complex to answer in
a short email, but work on packaging the FreeBSD base system is indeed
ongoing, and has been my primary focus over the past several weeks.

I am finishing an initial list of outstanding items that need to be
resolved before the project branch can feasibly merged back to head,
which I will send to the new freebsd-pkgbase@ mailing list.  People
interested in discussion surrounding this topic are urged to subscribe:

 https://lists.freebsd.org/mailman/listinfo/freebsd-pkgbase

Finally, I want to personally thank Baptiste Daroussin for all of his
tireless efforts to get us to the point we are at now.  Without his
ideas and insights, as well as ensuring pkg(8) contained the necessary
functionality, we would not be anywhere close to completing this work
for the 11.0-RELEASE.
May I ask how are you going to handle the tricky merging part, like 
/etc/master.passwd?

Usually that file has entries from 3 sources:
 - From the Base system, which might change between releases
 - From installed ports
 - Manually created entries.

Also, quite often entries from the base system are changed manually, 
think of root's/toor's password. Are such cases going to be dealt with 
properly between upgrades, including self-built-and-packaged base 
systems? Currently it can be a PITA with mergemaster to handle things 
like master.passwd properly between upgrades, automation so far wasn't 
famous on doing it properly.


Another thing is, there are a couple of parts of the base system where 
we add or remove features using knobs, and those take effect at multiple 
places. Like if I want to have wireless support, there's a bunch of 
userland utilities being built, and (the important part) some utilities 
are going to be built differently, like ifconfig. Is handling such 
features implemented properly by packaging base? We still have to be 
able to switch between different builds using the new tools.


Another thing is, sometimes when upgrading systems, to make things 
easier, I deploy the new major version of base, leave old libs/stuff in 
there till I rebuild and upgrade the packages installed, and after that 
remove the old libs (rm-old-libs target IIRC). The reason for this, for 
smaller systems there's usually a build jail which produces packages, 
and it needs to be upgraded to the new release to make the packages for 
it, so it's a bit of catch-22, and running rm-old-libs late just solved 
the issue. Is such a functionality still supported during upgrades? That 
is, upgrading base systems first in a way that old packages are still 
functional?


It's a very big projects, with lots of corner cases and difficult issues 
to tackle, I really appreciate your effort on this.


Best regards,
Gergely




[1]
https://lists.freebsd.org/pipermail/freebsd-announce/2015-February/001624.html

Thanks.

Glen



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: D3702: add support for Bluetooth Magic Mouse

[Lars Engels]

On Mon, Apr 04, 2016 at 10:22:30PM -0600, Warren Block wrote:
> Is anyone working on Bluetooth stuff?  https://reviews.freebsd.org/D3702 
> adds support for the Apple Magic Mouse, and has been tested and reported 
> working: 
> https://lists.freebsd.org/pipermail/freebsd-bluetooth/2016-April/002053.html
> 
> Could someone please review and commit this?  Thanks!

Sobomax@ (cc'ed) is the most likely person to review this.


pgpjtTcxHUOeI.pgp
Description: PGP signature

Re: CURRENT slow and shaky network stability

[Cy Schubert]

In message <20160405092712.131ee...@freyja.zeit4.iv.bundesimmobilien.de>, 
"O. H
artmann" writes:
> On Mon, 04 Apr 2016 23:46:08 -0700
> Cy Schubert  wrote:
> 
> > In message <20160405082047.670d7...@freyja.zeit4.iv.bundesimmobilien.de>, 
> > "O. H
> > artmann" writes:
> > > On Sat, 02 Apr 2016 16:14:57 -0700
> > > Cy Schubert  wrote:
> > >   
> > > > In message <20160402231955.41b05526.ohart...@zedat.fu-berlin.de>, "O. 
> > > > Hartmann"
> > > >  writes:  
> > > > > --Sig_/eJJPtbrEuK1nN2zIpc7BmVr
> > > > > Content-Type: text/plain; charset=US-ASCII
> > > > > Content-Transfer-Encoding: quoted-printable
> > > > > 
> > > > > Am Sat, 2 Apr 2016 11:39:10 +0200
> > > > > "O. Hartmann"  schrieb:
> > > > > 
> > > > > > Am Sat, 2 Apr 2016 10:55:03 +0200
> > > > > > "O. Hartmann"  schrieb:
> > > > > >=20
> > > > > > > Am Sat, 02 Apr 2016 01:07:55 -0700
> > > > > > > Cy Schubert  schrieb:
> > > > > > >  =20
> > > > > > > > In message <56f6c6b0.6010...@protected-networks.net>, Michael
> > > > > > > > Butle  
> > > r  
> > > > > > > > =
> > > > > writes:   =20
> > > > > > > > > -current is not great for interactive use at all. The strateg
> y
> > > > > > > > > of pre-emptively dropping idle processes to swap is hurting .
> .
> > > > > > > > > big tim=
> > > > > e. =20
> > > > > > > >=20
> > > > > > > > FreeBSD doesn't "preemptively" or arbitrarily push pages out to
> > > > > > > > disk.=
> > > > >  LRU=20
> > > > > > > > doesn't do this.
> > > > > > > >=20
> > > > > > > > >=20
> > > > > > > > > Compare inactive memory to swap in this example ..
> > > > > > > > >=20
> > > > > > > > > 110 processes: 1 running, 108 sleeping, 1 zombie
> > > > > > > > > CPU:  1.2% user,  0.0% nice,  4.3% system,  0.0% interrupt,
> > > > > > > > > 94.5% i=
> > > > > dle
> > > > > > > > > Mem: 474M Active, 1609M Inact, 764M Wired, 281M Buf, 119M Fre
> e
> > > > > > > > > Swap: 4096M Total, 917M Used, 3178M Free, 22% Inuse =20  
>   
> > > > > > > >=20
> > > > > > > > To analyze this you need to capture vmstat output. You'll see t
> he
> > > > > > > > fre=
> > > > > e pool=20
> > > > > > > > dip below a threshold and pages go out to disk in response. If 
> you
> > > > > > > > ha=
> > > > > ve=20
> > > > > > > > daemons with small working sets, pages that are not part of the
> > > > > > > > worki=
> > > > > ng=20
> > > > > > > > sets for daemons or applications will eventually be paged out.
> > > > > > > > This i=
> > > > > s not=20
> > > > > > > > a bad thing. In your example above, the 281 MB of UFS buffers a
> re
> > > > > > > > mor=
> > > > > e=20
> > > > > > > > active than the 917 MB paged out. If it's paged out and never u
> sed
> > > > > > > > ag=
> > > > > ain,=20
> > > > > > > > then it doesn't hurt. However the 281 MB of buffers saves you I
> /O.
> > > > > > > > Th=
> > > > > e=20
> > > > > > > > inactive pages are part of your free pool that were active at o
> ne
> > > > > > > > tim=
> > > > > e but=20
> > > > > > > > now are not. They may be reclaimed and if they are, you've just
> > > > > > > > saved=
> > > > >  more=20
> > > > > > > > I/O.
> > > > > > > >=20
> > > > > > > > Top is a poor tool to analyze memory use. Vmstat is the better
> > > > > > > > tool t=
> > > > > o help=20
> > > > > > > > understand memory use. Inactive memory isn't a bad thing per se
> .
> > > > > > > > Moni=
> > > > > tor=20
> > > > > > > > page outs, scan rate and page reclaims.
> > > > > > > >=20
> > > > > > > >=20
> > > > > > >=20
> > > > > > > I give up! Tried to check via ssh/vmstat what is going on. Last
> > > > > > > lines b=
> > > > > efore broken
> > > > > > > pipe:
> > > > > > >=20
> > > > > > > [...]
> > > > > > > procs  memory   pagedisks
> > > > > > > faults   
> > > cpu  
> > > > > > > r b w  avm   fre   flt  re  pi  pofr   sr ad0 ad1   insy
> > > > > > > c  
> > > s  
> > > > > > > =
> > > > > us sy id
> > > > > > > 22 0 22 5.8G  1.0G 46319   0   0   0 55721 1297   0   4  219 2390
> 7
> > > > > > > 540=
> > > > > 0 95  5  0
> > > > > > > 22 0 22 5.4G  1.3G 51733   0   0   0 72436 1162   0   0  108 4086
> 9
> > > > > > > 345=
> > > > > 9 93  7  0
> > > > > > > 15 0 22  12G  1.2G 54400   0  27   0 52188 1160   0  42  148 5219
> 2
> > > > > > > 436=
> > > > > 6 91  9  0
> > > > > > > 14 0 22  12G  1.0G 44954   0  37   0 37550 1179   0  39  141 8620
> 9
> > > > > > > 436=
> > > > > 8 88 12  0
> > > > > > > 26 0 22  12G  1.1G 60258   0  81   0 69459 1119   0  27  123 7795
> 69
> > > > > > > 704=
> > > > > 359 87 13  0
> > > > > > > 29 3 22  13G  774M 50576   0  68   0 32204 1304   0   2  102 5073
> 37
> > > > > > > 484=
> > > > > 861 93  7  0
> > > > > > 

Re: CURRENT slow and shaky network stability

[O. Hartmann]

On Mon, 04 Apr 2016 23:46:08 -0700
Cy Schubert  wrote:

> In message <20160405082047.670d7...@freyja.zeit4.iv.bundesimmobilien.de>, 
> "O. H
> artmann" writes:
> > On Sat, 02 Apr 2016 16:14:57 -0700
> > Cy Schubert  wrote:
> >   
> > > In message <20160402231955.41b05526.ohart...@zedat.fu-berlin.de>, "O. 
> > > Hartmann"
> > >  writes:  
> > > > --Sig_/eJJPtbrEuK1nN2zIpc7BmVr
> > > > Content-Type: text/plain; charset=US-ASCII
> > > > Content-Transfer-Encoding: quoted-printable
> > > > 
> > > > Am Sat, 2 Apr 2016 11:39:10 +0200
> > > > "O. Hartmann"  schrieb:
> > > > 
> > > > > Am Sat, 2 Apr 2016 10:55:03 +0200
> > > > > "O. Hartmann"  schrieb:
> > > > >=20
> > > > > > Am Sat, 02 Apr 2016 01:07:55 -0700
> > > > > > Cy Schubert  schrieb:
> > > > > >  =20
> > > > > > > In message <56f6c6b0.6010...@protected-networks.net>, Michael
> > > > > > > Butle  
> > r  
> > > > > > > =
> > > > writes:   =20
> > > > > > > > -current is not great for interactive use at all. The strategy
> > > > > > > > of pre-emptively dropping idle processes to swap is hurting ..
> > > > > > > > big tim=
> > > > e. =20
> > > > > > >=20
> > > > > > > FreeBSD doesn't "preemptively" or arbitrarily push pages out to
> > > > > > > disk.=
> > > >  LRU=20
> > > > > > > doesn't do this.
> > > > > > >=20
> > > > > > > >=20
> > > > > > > > Compare inactive memory to swap in this example ..
> > > > > > > >=20
> > > > > > > > 110 processes: 1 running, 108 sleeping, 1 zombie
> > > > > > > > CPU:  1.2% user,  0.0% nice,  4.3% system,  0.0% interrupt,
> > > > > > > > 94.5% i=
> > > > dle
> > > > > > > > Mem: 474M Active, 1609M Inact, 764M Wired, 281M Buf, 119M Free
> > > > > > > > Swap: 4096M Total, 917M Used, 3178M Free, 22% Inuse =20
> > > > > > >=20
> > > > > > > To analyze this you need to capture vmstat output. You'll see the
> > > > > > > fre=
> > > > e pool=20
> > > > > > > dip below a threshold and pages go out to disk in response. If you
> > > > > > > ha=
> > > > ve=20
> > > > > > > daemons with small working sets, pages that are not part of the
> > > > > > > worki=
> > > > ng=20
> > > > > > > sets for daemons or applications will eventually be paged out.
> > > > > > > This i=
> > > > s not=20
> > > > > > > a bad thing. In your example above, the 281 MB of UFS buffers are
> > > > > > > mor=
> > > > e=20
> > > > > > > active than the 917 MB paged out. If it's paged out and never used
> > > > > > > ag=
> > > > ain,=20
> > > > > > > then it doesn't hurt. However the 281 MB of buffers saves you I/O.
> > > > > > > Th=
> > > > e=20
> > > > > > > inactive pages are part of your free pool that were active at one
> > > > > > > tim=
> > > > e but=20
> > > > > > > now are not. They may be reclaimed and if they are, you've just
> > > > > > > saved=
> > > >  more=20
> > > > > > > I/O.
> > > > > > >=20
> > > > > > > Top is a poor tool to analyze memory use. Vmstat is the better
> > > > > > > tool t=
> > > > o help=20
> > > > > > > understand memory use. Inactive memory isn't a bad thing per se.
> > > > > > > Moni=
> > > > tor=20
> > > > > > > page outs, scan rate and page reclaims.
> > > > > > >=20
> > > > > > >=20
> > > > > >=20
> > > > > > I give up! Tried to check via ssh/vmstat what is going on. Last
> > > > > > lines b=
> > > > efore broken
> > > > > > pipe:
> > > > > >=20
> > > > > > [...]
> > > > > > procs  memory   pagedisks
> > > > > > faults   
> > cpu  
> > > > > > r b w  avm   fre   flt  re  pi  pofr   sr ad0 ad1   insy
> > > > > > c  
> > s  
> > > > > > =
> > > > us sy id
> > > > > > 22 0 22 5.8G  1.0G 46319   0   0   0 55721 1297   0   4  219 23907
> > > > > > 540=
> > > > 0 95  5  0
> > > > > > 22 0 22 5.4G  1.3G 51733   0   0   0 72436 1162   0   0  108 40869
> > > > > > 345=
> > > > 9 93  7  0
> > > > > > 15 0 22  12G  1.2G 54400   0  27   0 52188 1160   0  42  148 52192
> > > > > > 436=
> > > > 6 91  9  0
> > > > > > 14 0 22  12G  1.0G 44954   0  37   0 37550 1179   0  39  141 86209
> > > > > > 436=
> > > > 8 88 12  0
> > > > > > 26 0 22  12G  1.1G 60258   0  81   0 69459 1119   0  27  123 779569
> > > > > > 704=
> > > > 359 87 13  0
> > > > > > 29 3 22  13G  774M 50576   0  68   0 32204 1304   0   2  102 507337
> > > > > > 484=
> > > > 861 93  7  0
> > > > > > 27 0 22  13G  937M 47477   0  48   0 59458 1264   3   2  112 68131
> > > > > > 4440=
> > > > 7 95  5  0
> > > > > > 36 0 22  13G  829M 83164   0   2   0 82575 1225   1   0  126 99366
> > > > > > 3806=
> > > > 0 89 11  0
> > > > > > 35 0 22 6.2G  1.1G 98803   0  13   0 121375 1217   2   8  112 99371
> > > > > > 49=
> > > > 99 85 15  0
> > > > > > 34 0 22  13G  723M 

Re: CURRENT slow and shaky network stability

[Cy Schubert]

In message <20160405082047.670d7...@freyja.zeit4.iv.bundesimmobilien.de>, 
"O. H
artmann" writes:
> On Sat, 02 Apr 2016 16:14:57 -0700
> Cy Schubert  wrote:
> 
> > In message <20160402231955.41b05526.ohart...@zedat.fu-berlin.de>, "O. 
> > Hartmann"
> >  writes:
> > > --Sig_/eJJPtbrEuK1nN2zIpc7BmVr
> > > Content-Type: text/plain; charset=US-ASCII
> > > Content-Transfer-Encoding: quoted-printable
> > > 
> > > Am Sat, 2 Apr 2016 11:39:10 +0200
> > > "O. Hartmann"  schrieb:
> > >   
> > > > Am Sat, 2 Apr 2016 10:55:03 +0200
> > > > "O. Hartmann"  schrieb:
> > > >=20  
> > > > > Am Sat, 02 Apr 2016 01:07:55 -0700
> > > > > Cy Schubert  schrieb:
> > > > >  =20  
> > > > > > In message <56f6c6b0.6010...@protected-networks.net>, Michael Butle
> r
> > > > > > =  
> > > writes:   =20  
> > > > > > > -current is not great for interactive use at all. The strategy of
> > > > > > > pre-emptively dropping idle processes to swap is hurting .. big
> > > > > > > tim=  
> > > e. =20  
> > > > > >=20
> > > > > > FreeBSD doesn't "preemptively" or arbitrarily push pages out to
> > > > > > disk.=  
> > >  LRU=20  
> > > > > > doesn't do this.
> > > > > >=20  
> > > > > > >=20
> > > > > > > Compare inactive memory to swap in this example ..
> > > > > > >=20
> > > > > > > 110 processes: 1 running, 108 sleeping, 1 zombie
> > > > > > > CPU:  1.2% user,  0.0% nice,  4.3% system,  0.0% interrupt, 94.5%
> > > > > > > i=  
> > > dle  
> > > > > > > Mem: 474M Active, 1609M Inact, 764M Wired, 281M Buf, 119M Free
> > > > > > > Swap: 4096M Total, 917M Used, 3178M Free, 22% Inuse =20  
> > > > > >=20
> > > > > > To analyze this you need to capture vmstat output. You'll see the
> > > > > > fre=  
> > > e pool=20  
> > > > > > dip below a threshold and pages go out to disk in response. If you
> > > > > > ha=  
> > > ve=20  
> > > > > > daemons with small working sets, pages that are not part of the
> > > > > > worki=  
> > > ng=20  
> > > > > > sets for daemons or applications will eventually be paged out. This
> > > > > > i=  
> > > s not=20  
> > > > > > a bad thing. In your example above, the 281 MB of UFS buffers are
> > > > > > mor=  
> > > e=20  
> > > > > > active than the 917 MB paged out. If it's paged out and never used
> > > > > > ag=  
> > > ain,=20  
> > > > > > then it doesn't hurt. However the 281 MB of buffers saves you I/O.
> > > > > > Th=  
> > > e=20  
> > > > > > inactive pages are part of your free pool that were active at one
> > > > > > tim=  
> > > e but=20  
> > > > > > now are not. They may be reclaimed and if they are, you've just
> > > > > > saved=  
> > >  more=20  
> > > > > > I/O.
> > > > > >=20
> > > > > > Top is a poor tool to analyze memory use. Vmstat is the better tool
> > > > > > t=  
> > > o help=20  
> > > > > > understand memory use. Inactive memory isn't a bad thing per se.
> > > > > > Moni=  
> > > tor=20  
> > > > > > page outs, scan rate and page reclaims.
> > > > > >=20
> > > > > >=20  
> > > > >=20
> > > > > I give up! Tried to check via ssh/vmstat what is going on. Last lines
> > > > > b=  
> > > efore broken  
> > > > > pipe:
> > > > >=20
> > > > > [...]
> > > > > procs  memory   pagedisks faults 
> cpu
> > > > > r b w  avm   fre   flt  re  pi  pofr   sr ad0 ad1   insyc
> s
> > > > > =  
> > > us sy id  
> > > > > 22 0 22 5.8G  1.0G 46319   0   0   0 55721 1297   0   4  219 23907
> > > > > 540=  
> > > 0 95  5  0  
> > > > > 22 0 22 5.4G  1.3G 51733   0   0   0 72436 1162   0   0  108 40869
> > > > > 345=  
> > > 9 93  7  0  
> > > > > 15 0 22  12G  1.2G 54400   0  27   0 52188 1160   0  42  148 52192
> > > > > 436=  
> > > 6 91  9  0  
> > > > > 14 0 22  12G  1.0G 44954   0  37   0 37550 1179   0  39  141 86209
> > > > > 436=  
> > > 8 88 12  0  
> > > > > 26 0 22  12G  1.1G 60258   0  81   0 69459 1119   0  27  123 779569
> > > > > 704=  
> > > 359 87 13  0  
> > > > > 29 3 22  13G  774M 50576   0  68   0 32204 1304   0   2  102 507337
> > > > > 484=  
> > > 861 93  7  0  
> > > > > 27 0 22  13G  937M 47477   0  48   0 59458 1264   3   2  112 68131
> > > > > 4440=  
> > > 7 95  5  0  
> > > > > 36 0 22  13G  829M 83164   0   2   0 82575 1225   1   0  126 99366
> > > > > 3806=  
> > > 0 89 11  0  
> > > > > 35 0 22 6.2G  1.1G 98803   0  13   0 121375 1217   2   8  112 99371
> > > > > 49=  
> > > 99 85 15  0  
> > > > > 34 0 22  13G  723M 54436   0  20   0 36952 1276   0  17  153 29142
> > > > > 443=  
> > > 1 95  5  0  
> > > > > Fssh_packet_write_wait: Connection to 192.168.0.1 port 22: Broken pip
> e
> > > > >=20
> > > > >=20
> > > > > This makes this crap system completely unusable. The server (FreeBSD
> > > > > 11=  
> > > .0-CURRENT #20  
> > > > > r297503: Sat Apr  2 09:02:41 CEST 2016 amd64) in question did
> > > > > poudriere=  
> > >  bulk job. I  
> > > > > can not even determine what terminal goes down first - 

Re: CURRENT slow and shaky network stability

[O. Hartmann]

On Sat, 02 Apr 2016 16:14:57 -0700
Cy Schubert  wrote:

> In message <20160402231955.41b05526.ohart...@zedat.fu-berlin.de>, "O. 
> Hartmann"
>  writes:
> > --Sig_/eJJPtbrEuK1nN2zIpc7BmVr
> > Content-Type: text/plain; charset=US-ASCII
> > Content-Transfer-Encoding: quoted-printable
> > 
> > Am Sat, 2 Apr 2016 11:39:10 +0200
> > "O. Hartmann"  schrieb:
> >   
> > > Am Sat, 2 Apr 2016 10:55:03 +0200
> > > "O. Hartmann"  schrieb:
> > >=20  
> > > > Am Sat, 02 Apr 2016 01:07:55 -0700
> > > > Cy Schubert  schrieb:
> > > >  =20  
> > > > > In message <56f6c6b0.6010...@protected-networks.net>, Michael Butler
> > > > > =  
> > writes:   =20  
> > > > > > -current is not great for interactive use at all. The strategy of
> > > > > > pre-emptively dropping idle processes to swap is hurting .. big
> > > > > > tim=  
> > e. =20  
> > > > >=20
> > > > > FreeBSD doesn't "preemptively" or arbitrarily push pages out to
> > > > > disk.=  
> >  LRU=20  
> > > > > doesn't do this.
> > > > >=20  
> > > > > >=20
> > > > > > Compare inactive memory to swap in this example ..
> > > > > >=20
> > > > > > 110 processes: 1 running, 108 sleeping, 1 zombie
> > > > > > CPU:  1.2% user,  0.0% nice,  4.3% system,  0.0% interrupt, 94.5%
> > > > > > i=  
> > dle  
> > > > > > Mem: 474M Active, 1609M Inact, 764M Wired, 281M Buf, 119M Free
> > > > > > Swap: 4096M Total, 917M Used, 3178M Free, 22% Inuse =20  
> > > > >=20
> > > > > To analyze this you need to capture vmstat output. You'll see the
> > > > > fre=  
> > e pool=20  
> > > > > dip below a threshold and pages go out to disk in response. If you
> > > > > ha=  
> > ve=20  
> > > > > daemons with small working sets, pages that are not part of the
> > > > > worki=  
> > ng=20  
> > > > > sets for daemons or applications will eventually be paged out. This
> > > > > i=  
> > s not=20  
> > > > > a bad thing. In your example above, the 281 MB of UFS buffers are
> > > > > mor=  
> > e=20  
> > > > > active than the 917 MB paged out. If it's paged out and never used
> > > > > ag=  
> > ain,=20  
> > > > > then it doesn't hurt. However the 281 MB of buffers saves you I/O.
> > > > > Th=  
> > e=20  
> > > > > inactive pages are part of your free pool that were active at one
> > > > > tim=  
> > e but=20  
> > > > > now are not. They may be reclaimed and if they are, you've just
> > > > > saved=  
> >  more=20  
> > > > > I/O.
> > > > >=20
> > > > > Top is a poor tool to analyze memory use. Vmstat is the better tool
> > > > > t=  
> > o help=20  
> > > > > understand memory use. Inactive memory isn't a bad thing per se.
> > > > > Moni=  
> > tor=20  
> > > > > page outs, scan rate and page reclaims.
> > > > >=20
> > > > >=20  
> > > >=20
> > > > I give up! Tried to check via ssh/vmstat what is going on. Last lines
> > > > b=  
> > efore broken  
> > > > pipe:
> > > >=20
> > > > [...]
> > > > procs  memory   pagedisks faults cpu
> > > > r b w  avm   fre   flt  re  pi  pofr   sr ad0 ad1   insycs
> > > > =  
> > us sy id  
> > > > 22 0 22 5.8G  1.0G 46319   0   0   0 55721 1297   0   4  219 23907
> > > > 540=  
> > 0 95  5  0  
> > > > 22 0 22 5.4G  1.3G 51733   0   0   0 72436 1162   0   0  108 40869
> > > > 345=  
> > 9 93  7  0  
> > > > 15 0 22  12G  1.2G 54400   0  27   0 52188 1160   0  42  148 52192
> > > > 436=  
> > 6 91  9  0  
> > > > 14 0 22  12G  1.0G 44954   0  37   0 37550 1179   0  39  141 86209
> > > > 436=  
> > 8 88 12  0  
> > > > 26 0 22  12G  1.1G 60258   0  81   0 69459 1119   0  27  123 779569
> > > > 704=  
> > 359 87 13  0  
> > > > 29 3 22  13G  774M 50576   0  68   0 32204 1304   0   2  102 507337
> > > > 484=  
> > 861 93  7  0  
> > > > 27 0 22  13G  937M 47477   0  48   0 59458 1264   3   2  112 68131
> > > > 4440=  
> > 7 95  5  0  
> > > > 36 0 22  13G  829M 83164   0   2   0 82575 1225   1   0  126 99366
> > > > 3806=  
> > 0 89 11  0  
> > > > 35 0 22 6.2G  1.1G 98803   0  13   0 121375 1217   2   8  112 99371
> > > > 49=  
> > 99 85 15  0  
> > > > 34 0 22  13G  723M 54436   0  20   0 36952 1276   0  17  153 29142
> > > > 443=  
> > 1 95  5  0  
> > > > Fssh_packet_write_wait: Connection to 192.168.0.1 port 22: Broken pipe
> > > >=20
> > > >=20
> > > > This makes this crap system completely unusable. The server (FreeBSD
> > > > 11=  
> > .0-CURRENT #20  
> > > > r297503: Sat Apr  2 09:02:41 CEST 2016 amd64) in question did
> > > > poudriere=  
> >  bulk job. I  
> > > > can not even determine what terminal goes down first - another one,
> > > > muc=  
> > h more time  
> > > > idle than the one shwoing the "vmstat 5" output, is still alive!=20
> > > >=20
> > > > i consider this a serious bug and it is no benefit what happened since
> > > > =  
> > this "fancy"  
> > > > update. :-( =20  
> > >=20
> > > By the way - it might be of interest and some hint.
> > >=20
> > > One of my boxes is 

Re: accessing a PCIe register from userspace through kmem or other ways ?

[Konstantin Belousov]

On Mon, Apr 04, 2016 at 03:45:07PM -0700, John Baldwin wrote:
> However, another question is how to deal with systems that do bus address
> translation (like the arm64 ThunderX boxes) where the values in the PCI
> BAR are not CPU physical addresses.  To do this properly we may need some
> sort of bus method akin to my bus_map_resource() WIP but one that instead
> returns a suitable 'struct sglist' for a given 'struct resource *' that
> can be used with OBJT_SG to build a VM object to use for the mapping.
Is there any documentation on the ThunderX PCI access mechanisms ?

> 
> (At some point I do think I would like a variant of OBJT_SG that used
> managed pages so that mappings could be revoked when whatever is backing
> the sglist is disabled (e.g. the device is ejected or the BAR is
> relocated, etc.).)
Why cannot you use MGTDEVICE pager already ? Driver would need to
maintain its private list of pages, and from this PoV, a convenience
wrapper around MGTDEVICE which unifies operations on sg lists could be
useful. Still, it could be that the wrapper appear to be single-purpose.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: accessing a PCIe register from userspace through kmem or other ways ?

[Konstantin Belousov]

On Mon, Apr 04, 2016 at 09:02:49PM -0700, John Baldwin wrote:
> kib@ fixed /dev/mem to handle addresses beyond the direct map limit to use
> temporary mappings instead of failing with EFAULT in 277051 which was only
> committed to HEAD last January, so well after 8.2.

The mmap(2) interface to /dev/mem did not have the issue ever.  The problem
was only with the read(2)/write(2) accesses.

>From what I understand, since the goal of the OP was to measure BAR
access latencies, read(2) (or write) is unsuitable for him for obvious
reasons.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"