Build failed in Jenkins: Build-UFS-image #2720

[jenkins-admin]

See 

--
Started by upstream project "Build_Image_and_Run_Tests_in_Bhyve_HEAD" build 
number 1759
originally caused by:
 Started by upstream project "FreeBSD_HEAD" build number 3517
 originally caused by:
  Started by an SCM change
Building remotely on jenkins-10.freebsd.org (FreeBSD-10) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/freebsd/freebsd-ci # 
 > timeout=10
Fetching upstream changes from https://github.com/freebsd/freebsd-ci
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://github.com/freebsd/freebsd-ci +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 65a323ece2701c4a650ed241aea8bdecfaf56ea9 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 65a323ece2701c4a650ed241aea8bdecfaf56ea9
 > git rev-list 65a323ece2701c4a650ed241aea8bdecfaf56ea9 # timeout=10
[Build-UFS-image] $ /bin/sh -xe /tmp/hudson6510864903989029590.sh
+ freebsd-ci/scripts/build/build-ufs-image.sh
+ [ -z  ]
+ [ -z /builds/FreeBSD_HEAD ]
+ [ -z '' ]
+ export MAKEOBJDIRPREFIX=/builds/FreeBSD_HEAD/obj
+ [ -z '' ]
+ basename /builds/FreeBSD_HEAD
+ 
PACKAGE_ROOT=
+ [ -z '' ]
+ basename /builds/FreeBSD_HEAD
+ 
IMAGE_ROOT=
+ [ -n '' ]
+ [ -z /builds/FreeBSD_HEAD/obj ]
+ cd /builds/FreeBSD_HEAD
+ [ -z '' ]
+ [ -f /builds/FreeBSD_HEAD/make.conf ]
+ __MAKE_CONF=/builds/FreeBSD_HEAD/make.conf
+ sudo rm -fr 

rm: 
:
 Operation not permitted
rm: 
: 
Directory not empty
rm: : 
Directory not empty
Build step 'Execute shell' marked build as failure
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: OpenSSH HPN

[Dag-Erling Smørgrav]

Bryan Drewery  writes:
> It's harder to maintain the port version due to how the patches are
> applied and generated.

I have the diametrically opposite experience.  The workflow for ports is
significantly simpler than for base.  Perhaps I should set up a paralell
workflow for OpenSSH and apply the output of that workflow to the source
tree instead of working entirely within the source tree.

DES
-- 
Dag-Erling Smørgrav - d...@des.no
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: OpenSSH HPN

[Allan Jude]

On 2015-11-12 12:56, John-Mark Gurney wrote:
> Allan Jude wrote this message on Thu, Nov 12, 2015 at 12:15 -0500:
>> On 2015-11-11 19:06, Slawa Olhovchenkov wrote:
>>> On Wed, Nov 11, 2015 at 01:32:27PM -0800, Bryan Drewery wrote:
>>>
 On 11/10/2015 1:42 AM, Dag-Erling Smørgrav wrote:
>  I would also like to remove the NONE cipher
> patch, which is also available in the port (off by default, just like in
> base).

 Fun fact, it's been broken in the port for several months with no
 complaints. It was just reported and fixed upstream in the last day and
 I wrote in a similar fix in the port. That speaks a lot about its usage
 in the port currently.
>>>
>>> I am try using NPH/NONE with base ssh and confused: don't see
>>> performance rise, too complex to enable and too complex for use.
>>
>> I did a few quick (and dirty) benchmarks and it shows that the NONE
>> cipher definitely makes a difference. Version of OpenSSL also seems to
>> make a difference, as one might expect.
>>
>> Note: openssh from ports seems to link against both base and ports
>> libcrypto, I am still trying to make sure this isn't corrupting my
>> benchmark results.
> 
> You don't need the aesni.ko module loaded for OpenSSL (which is how
> OpenSSH uses most crypto algos) to use AES-NI..
> 
> Also, do you set any sysctl's to play w/ the buffer sizes or anything?
> 
>> I am still debugging my dummynet setup to be able to prove that HPN
>> makes a difference (but it does).
> 
> Does my example on the page not work for you?
> 
>> https://wiki.freebsd.org/SSHPerf
> 

I found that when I set even 5ms of delay with dummynet, bandwidth over
the LAN drops more than it should. Dummynet is limiting the rate rather
than just adding the delay. I am investigating.

I found this document:
http://www.cs.unc.edu/~jeffay/dirt/FAQ/hstcp-howto.pdf

Which is from the 6.x era, but suggests:

"One subtle bug exists in the stock Dummynet implementation that should
be corrected for experiments.  When a packet arrives in dummynet it is
shoved into a queue which limits the bandwidth a TCP flow may use. Upon
exit from the queue, the packet is transferred to a pipe where it sits
for any configured amount of delay time and might possibly be dropped
depending on the loss probability. Once the delay time has passed, the
packet is released to ip output."

May be the cause of my problem

-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: Wake on LAN broken (probably between r290542 - r290606)?

[Yuri]

On 11/11/2015 06:33, David Wolfskill wrote:

Can anyone else confirm or refute my observations?  Or suggest a
hint?  I'll try narrowing it down myself, but I need to do it during
times I'm at home (so I can manually power the machine back up when
it fails to respond to WoL), so it may be a few days before I can
accomplish much that way.


Did you confirm (for ex. with Wireshark) that wol actually sends the packet?
If you are trying to wake the machine that is not booted, this is 
primarily dependent on the BIOS of the target machine to wake it up.
In my experience, BIOSes are prone to losing settings, due to low 
battery, or other (sometimes mysterious) reasons.

Please confirm that BIOS setting "Wake On LAN" is on on target.

Yuri

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Wake on LAN broken (probably between r290542 - r290606)?

[David Wolfskill]

On Thu, Nov 12, 2015 at 05:02:27PM -0800, Yuri wrote:
> ...
> Did you confirm (for ex. with Wireshark) that wol actually sends the packet?
> If you are trying to wake the machine that is not booted, this is 
> primarily dependent on the BIOS of the target machine to wake it up.
> In my experience, BIOSes are prone to losing settings, due to low 
> battery, or other (sometimes mysterious) reasons.
> Please confirm that BIOS setting "Wake On LAN" is on on target.
> 

I confirmed it by successfully using the /usr/local/bin/wol program on
the target machine when the latter had last been booted to run FreeBSD
stable/10.

Previously, this worked whether the target machine had been running
stable/10 or head; now it only works if it had been running stable/10 at
the time the "shutdown -p now" command was issued.

Peace,
david
-- 
David H. Wolfskill  da...@catwhisker.org
Those who would murder in the name of God or prophet are blasphemous cowards.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.


signature.asc
Description: PGP signature

Build failed in Jenkins: Build-UFS-image #2718

[jenkins-admin]

See 

--
Started by upstream project "Build_Image_and_Run_Tests_in_Bhyve_HEAD" build 
number 1758
originally caused by:
 Started by upstream project "FreeBSD_HEAD" build number 3516
 originally caused by:
  Started by an SCM change
Building remotely on jenkins-10.freebsd.org (FreeBSD-10)No JDK named ?null? 
found
 in workspace 
No JDK named ?null? found
No JDK named ?null? found
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/freebsd/freebsd-ci # 
 > timeout=10
Fetching upstream changes from https://github.com/freebsd/freebsd-ci
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://github.com/freebsd/freebsd-ci +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 65a323ece2701c4a650ed241aea8bdecfaf56ea9 
(refs/remotes/origin/master)
No JDK named ?null? found
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 65a323ece2701c4a650ed241aea8bdecfaf56ea9
 > git rev-list 65a323ece2701c4a650ed241aea8bdecfaf56ea9 # timeout=10
No JDK named ?null? found
[Build-UFS-image] $ /bin/sh -xe /tmp/hudson887574467276929336.sh
+ freebsd-ci/scripts/build/build-ufs-image.sh
+ [ -z  ]
+ [ -z /builds/FreeBSD_HEAD ]
+ [ -z '' ]
+ export MAKEOBJDIRPREFIX=/builds/FreeBSD_HEAD/obj
+ [ -z '' ]
+ basename /builds/FreeBSD_HEAD
+ 
PACKAGE_ROOT=
+ [ -z '' ]
+ basename /builds/FreeBSD_HEAD
+ 
IMAGE_ROOT=
+ [ -n '' ]
+ [ -z /builds/FreeBSD_HEAD/obj ]
+ cd /builds/FreeBSD_HEAD
+ [ -z '' ]
+ [ -f /builds/FreeBSD_HEAD/make.conf ]
+ __MAKE_CONF=/builds/FreeBSD_HEAD/make.conf
+ sudo rm -fr 

rm: 
:
 Operation not permitted
rm: 
: 
Directory not empty
rm: : 
Directory not empty
Build step 'Execute shell' marked build as failure
No JDK named ?null? found
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: OpenSSH HPN

[Benjamin Kaduk]

On Thu, 12 Nov 2015, Dewayne Geraghty wrote:

> Heimdal is (and has been for some time) undergoing constant development.
> For reasons unknown, they do not perform releases.  I am aware of updates
> from heimdal that are being applied to the samba project (in fact some of
> the samba developers are also feeding into heimdal).  The latest discussion
> was that the heimdal project are going to release a 1.7 "sometime",
> skipping 1.6 completely.

Things seem to have slowed down a lot since the lead Heimdal developer got
hired for Apple.  They have Apple-internal changes that don't necessarily
make their way back to the public project right away, and he is quite
busy.  There is no one who is employed to be a Heimdal release manager,
and the main developers all have other projects -- putting out a release
takes a fair bit of energy.  MIT employs developers whose job descriptions
include being the krb5 release manager, so there is financial support for
putting out regular releases.

Heimdal has changed plans to a 1.7 release because certain Linux
distributions packaged a snapshot of the 1.6 tree (to support Samba, as I
understand it), but then Heimdal development continued so that what would
be in the next release would not really reflect what was already deployed
using the 1.6 label.  As I understand it, there are still a couple
bugfixes/features that are considered to be blockers for the 1.7 release
that have not been implemented yet, and since the developers in question
are being paid to work on other things, there is no real timeline for the
release.


-Ben Kaduk

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: OpenSSH HPN

[Julian Elischer]

On 11/12/15 3:28 AM, Brooks Davis wrote:

On Tue, Nov 10, 2015 at 04:40:42PM -0800, Bryan Drewery wrote:

On 11/10/15 1:42 AM, Dag-Erling Sm??rgrav wrote:

Some of you may have noticed that OpenSSH in base is lagging far behind
the upstream code.

The main reason for this is the burden of maintaining the HPN patches.
They are extensive, very intrusive, and touch parts of the OpenSSH code
that change significantly in every release.  Since they are not
regularly updated, I have to choose between trying to resolve the
conflicts myself (hoping I don't break anything) or waiting for them to
catch up and then figuring out how to apply the new version.

Therefore, I would like to remove the HPN patches from base and refer
anyone who really needs them to the openssh-portable port, which has
them as a default option.  I would also like to remove the NONE cipher
patch, which is also available in the port (off by default, just like in
base).

I had this same problem as well, but have since reworked the HPN patch
for ports to be more easily maintained.  I've considered offering or
just updating the base SSH, but have not since we have random changes in
the HPN functionality in base that would be lost.  We for some reason
decided we were going to maintain our own version and not even upstream
the changes to the HPN authors which has contributed to this situation.

We had ever intention of upstreaming our cleaned up HPN patches and some
interest from OpenSSH devs to take the window scaling portion of the
patch upstream, but other things intruded and we never found time to
complete that work.  I think both the window scaling and NONE cipher
changes are useful, but do not have time to do anything with them.  I'm
fine with them being removed from base and replaced or just dropped if
they are in the way of progress.


it would be nice if the outcome of this thread was that HPN patches 
(or something equivalent) were available by default in OpenSSH.

WE also have our own patch we add to give a NODELAY option.
It made a huge difference with tunnels where lots of small RPC packets 
were being sent.

I'll look at getting it into upstream.




-- Brooks


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: OpenSSH HPN

[Dag-Erling Smørgrav]

Benjamin Kaduk  writes:
> Things seem to have slowed down a lot since the lead Heimdal developer
> got hired for Apple.  [...]  MIT employs developers whose job
> descriptions include being the krb5 release manager [...]  Heimdal has
> changed plans to a 1.7 release [...] and since the developers in
> question are being paid to work on other things, there is no real
> timeline for the release.

Given this state of affairs, it might not be unreasonable to consider
switching back for 11.  There should be enough time, provided our
Kerberos maintainers have some spare cycles.

DES
-- 
Dag-Erling Smørgrav - d...@des.no
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: OpenSSH HPN

[Dewayne Geraghty]

Slawa,
Heimdal is (and has been for some time) undergoing constant development.
For reasons unknown, they do not perform releases.  I am aware of updates
from heimdal that are being applied to the samba project (in fact some of
the samba developers are also feeding into heimdal).  The latest discussion
was that the heimdal project are going to release a 1.7 "sometime",
skipping 1.6 completely.

Des - good to make your intentions public.  I've enjoyed your youtube
presentations and recognise that your time will be better spent.  ( better
authentication perhaps ;) )

Bryan - is doing a good job of looking after the openssh port.  And if
folks really need those additional features, then that is the place to
enhance the "standard" offering; which can be upgraded in a pretty
straightforward manner.

Thought-provoking use of inetd perhaps its time to revisit as (an
additional) DOS measure(?)

Regards, Dewayne.
PS My apologies for the repetition Slawa, I meant to reply all earlier.
I'm recently becoming familiar with the gmail interface.

On Thu, Nov 12, 2015 at 5:44 AM, Slawa Olhovchenkov  wrote:

> On Wed, Nov 11, 2015 at 07:18:31PM +0100, Dag-Erling Smørgrav wrote:
>
> > Slawa Olhovchenkov  writes:
> > > Can you explain what is problem?
> >
> > Radical suggestion: read the first email in the thread.
>
> I am read and don't understund (you talk about trouble of maintaining
> the HPN patches).
> I see patched version in ports. This version maintaining.
> What is problem? Differnt openssh? Quality of patches?
> Different branches?
> ports branch is worse (by some reaason) base branch?
>
> > > PS: As I today know, kerberos heimdal is practicaly dead as opensource
> > > project. Have FreeBSD planed switch to MIT Kerberos?  I am know about
> > > security/krb5.
> >
> > We switched from MIT to Heimdal at some point in the past for some
> > reason I don't remember.  MIT and Heimdal are *not* interchangeable at
>
> I think because MIT stop development in the past.
>
> > the source or binary level, so switching back is not trivial.
>
> I am know about this.
> ___
> freebsd-secur...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-security
> To unsubscribe, send any mail to "freebsd-security-unsubscr...@freebsd.org
> "
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

FreeBSD_HEAD-tests - Build #1683 - Still Unstable

[jenkins-admin]

FreeBSD_HEAD-tests - Build #1683 - Still Unstable:

Build information: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD-tests/1683/
Full change log: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD-tests/1683/changes
Full build log: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD-tests/1683/console

Change summaries:

No changes


The failed test cases:

5 tests failed.
FAILED:  bin.sh.builtins.functional_test.case7

Error Message:
atf-check failed; see the output of the test for details

FAILED:  bin.sh.builtins.functional_test.locale1

Error Message:
atf-check failed; see the output of the test for details

FAILED:  lib.libc.locale.c16rtomb_test.c16rtomb_test

Error Message:
Premature exit; test case received signal 11 (core dumped)

FAILED:  lib.libc.locale.mbrtoc16_test.mbrtoc16_test

Error Message:
Premature exit; test case received signal 11 (core dumped)

FAILED:  
lib.libc.stdio.printfloat_test.thousands_separator_and_other_locale_tests

Error Message:
printf("%'.4f", 12345678.0625) ==> [1,23,45,678.0625], expected 
[123,456,78.0625]<>

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

FreeBSD_HEAD_amd64_gcc4.9 - Build #787 - Fixed

[jenkins-admin]

FreeBSD_HEAD_amd64_gcc4.9 - Build #787 - Fixed:

Build information: 
https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc4.9/787/
Full change log: 
https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc4.9/787/changes
Full build log: 
https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc4.9/787/console

Change summaries:

290729 by ache:
1) Remove my overcomplicated error fallback and just return error
immediatelly as old code does, now for append modes too.
Real use case for such fallback is impossible (unless specially crafted).

2) Remove now unneded include I forgot to remove in prev. commits.

MFC after:  1 week

290728 by jhb:
Export various helper variables describing the layout and size of
certain kernel structures for use by debuggers. This mostly aids
in examining cores from a kernel without debug symbols as a debugger
can infer these values if debug symbols are available.

One set of variables describes the layout of 'struct linker_file' to
walk the list of loaded kernel modules.

A second set of variables describes the layout of 'struct proc' and
'struct thread' to walk the list of processes in the kernel and the
threads in each process.

The 'pcb_size' variable is used to index into the stoppcbs[] array.

The 'vm_maxuser_address' is used to distinguish kernel virtual addresses
from user addresses. This doesn't have to be perfect, and
'vm_maxuser_address' is a cheap and simple way to differentiate kernel
pointers from simple values like TIDs and PIDs.

While here, annotate the fields in struct pcb used by kgdb on amd64
and i386 to note that their ABI should be preserved.  Annotations for
other platforms will be added in the future.

Reviewed by:kib
MFC after:  2 weeks
Differential Revision:  https://reviews.freebsd.org/D3773

290727 by gonzo:
Enable cloks for all USDHC interfaces, previous value was USDHC 1-3 + USBOH3

290726 by gavin:
Consistently capitalize "TI" as an abbreviation for Texas Instruments.

290725 by cem:
NTB: MFV 8b782fab: unify translation addresses

There is no need for the upstream and downstream addresses to be
different for the NTB configs.  Go to using a single set of address. It
is still possible to configure them differently using module parameter
override however (CEM: tunable).

Authored by:Dave Jiang 
Reviewed by:Allen Hubbe 
Reviewed by:Jon Mason 
Obtained from:  Linux (Dual BSD/GPL driver)
Sponsored by:   EMC / Isilon Storage Division

290724 by cem:
kern_testfrwk.9:  Clean up manual page style

Igor has many less complaints now.  I think the two remaining are bogus, but I
am also not sure why Igor is producing them.

The page still needs more work.

Sponsored by:   EMC / Isilon Storage Division

290723 by bdrewery:
Add note about usr.sbin/pkg -- Please discuss behavior/flag changes with pkg 
team.

290722 by imp:
Move the root to s3a given this kernel will be used with a nanobsd
build that defaults there.

290721 by imp:
NO_CPU_OPTFLAGS doesn't exist, so don't document it.

290720 by sbruno:
spelling is important.

Submitted by:   vangy...@freebsd.org

290719 by bdrewery:
META MODE: Don't set schg flags in the stagedir for distrib-dirs.

This recently started setting /var/empty to schg after r290628.

Sponsored by:   EMC / Isilon Storage Division

290718 by bdrewery:
Respect NO_FSCHG for extracting mtrees.

PR: 194189
X-MFC-With: r290628
MFC after:  2 weeks
Sponsored by:   EMC / Isilon Storage Division

290717 by sbruno:
Intel has requested that changes to these driver get a phabricator review
before commits to keep conflicts down to a minimum.

290716 by rrs:
Some basic documentation (a man page) on kern_testfrwk

290715 by rrs:
Bump version number since callout_stop() macro now has new NULL arg.
Thanks Hans for spotting this!

290714 by rrs:
Style 9 changes.
MFC after:  1 Month

290711 by hselasky:
Fix integer to pointer of different size conversion warnings when
using GCC for 32-bit platforms. The integer size in this case is
hardcoded 64-bit while the pointer size is 32-bit.

Sponsored by:   Mellanox Technologies
MFC after:  2 weeks

290710 by hselasky:
Fix print formatting compile warnings for Sparc64 and PowerPC platforms.

Sponsored by:   Mellanox Technologies
MFC after:  2 weeks

290709 by hselasky:
Compile fix for 32-bit platforms:
- The Linux timers data field is "unsigned long".

Sponsored by:   Mellanox Technologies
MFC after:  2 weeks

290708 by smh:
Fix ixl debug sysctls panic

Remove the use of sbuf_data on drained sbufs from the debug sysctls:
* ixl_sysctl_hw_res_alloc
* ixl_sysctl_switch_config

This prevents a kernel panic when accessing these values under a kernel
compiled with INVARIANTS.

Sponsored by:   Multiplay

290707 by kib:
Fix UP build.

290706 by hselasky:
Build fixes:
- Add some missing I/O functions for non-i386 and amd64 platforms.
- Stub ioremap() to NULL using a macro to ensure non-existing memory
  

Re: [CFT] build: WITH_FAST_DEPEND and WITH_CCACHE_BUILD

[Craig Rodrigues]

On Thu, Nov 12, 2015 at 9:40 AM, Bryan Drewery  wrote:

> - The first is WITH_FAST_DEPEND. Please see the commit for its full
>

I have enabled WITH_FAST_DEPEND in make.conf for several builds.

(1) Job: https://jenkins.freebsd.org/job/FreeBSD_HEAD
 Before WITH_FAST_DEPEND:
https://jenkins.freebsd.org/job/FreeBSD_HEAD/3488/1 hour, 23 minutes
 After WITH_FAST_DEPEND:
https://jenkins.freebsd.org/job/FreeBSD_HEAD/3497/ ,  1 hour, 17 minutes

 Compiler used: clang

(2)   Job: https://jenkins.freebsd.org/job/FreeBSD_HEAD_sparc64/
Before WITH_FAST_DEPEND:
https://jenkins.freebsd.org/job/FreeBSD_HEAD_sparc64/1309/ , 23 minutes
After WITH_FAST_DEPEND:
https://jenkins.freebsd.org/job/FreeBSD_HEAD_sparc64/1329/ , 22 minutes

Compiler used: gcc 4.2.1

(3)  Job: https://jenkins.freebsd.org/job/FreeBSD_HEAD_amd64_gcc4.9/
   Before WITH_FAST_DEPEND:
https://jenkins.freebsd.org/job/FreeBSD_HEAD_amd64_gcc4.9/761/ , 1 hour, 7
minutes
   After WITH_FAST_DEPEND:
https://jenkins.freebsd.org/job/FreeBSD_HEAD_amd64_gcc4.9/762/ , 1 hour 2
minutes

   Compiler:  gcc 4.9.2

The timings are not very scientific, because the VM's in which the builds
are occurring are
very loaded, since each build host is running multiple VM's, which are
building different things.

This gives you a basic idea though, that WITH_FAST_DEPEND works on a few
compilers (clang, gcc 4.2, gcc 4.9),
and there is some build time reduction, and no increase in build time.

Congratulations for tackling this stuff in FreeBSD.  As you mentioned, this
area has not been touched
in 14 years.
--
Craig
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] build: WITH_FAST_DEPEND and WITH_CCACHE_BUILD

[Sergey V. Dyatko]

On Thu, 12 Nov 2015 21:04:53 -0800
Craig Rodrigues  wrote: 

> On Thu, Nov 12, 2015 at 9:40 AM, Bryan Drewery  wrote:
> 
> > - The first is WITH_FAST_DEPEND. Please see the commit for its full
> >  
> 
> I have enabled WITH_FAST_DEPEND in make.conf for several builds.

I have enabled it in src.conf ( head@r290570, GENERIC-NODEBUG kernel)

On hw.model: Intel(R) Xeon(R) CPU   E5504  @ 2.00GHz 
time make -j8 buildworld:

3685.33 real 25176.56 user  1918.05 sys WITH_FAST_DEPEND
3904.77 real 25833.18 user  2327.22 sys  without

> (1) Job: https://jenkins.freebsd.org/job/FreeBSD_HEAD
>  Before WITH_FAST_DEPEND:
> https://jenkins.freebsd.org/job/FreeBSD_HEAD/3488/1 hour, 23 minutes
>  After WITH_FAST_DEPEND:
> https://jenkins.freebsd.org/job/FreeBSD_HEAD/3497/ ,  1 hour, 17 minutes
> 
>  Compiler used: clang
> 
> (2)   Job: https://jenkins.freebsd.org/job/FreeBSD_HEAD_sparc64/
> Before WITH_FAST_DEPEND:
> https://jenkins.freebsd.org/job/FreeBSD_HEAD_sparc64/1309/ , 23 minutes
> After WITH_FAST_DEPEND:
> https://jenkins.freebsd.org/job/FreeBSD_HEAD_sparc64/1329/ , 22 minutes
> 
> Compiler used: gcc 4.2.1
> 
> (3)  Job: https://jenkins.freebsd.org/job/FreeBSD_HEAD_amd64_gcc4.9/
>Before WITH_FAST_DEPEND:
> https://jenkins.freebsd.org/job/FreeBSD_HEAD_amd64_gcc4.9/761/ , 1 hour, 7
> minutes
>After WITH_FAST_DEPEND:
> https://jenkins.freebsd.org/job/FreeBSD_HEAD_amd64_gcc4.9/762/ , 1 hour 2
> minutes
> 
>Compiler:  gcc 4.9.2
> 
> The timings are not very scientific, because the VM's in which the builds
> are occurring are
> very loaded, since each build host is running multiple VM's, which are
> building different things.
> 
> This gives you a basic idea though, that WITH_FAST_DEPEND works on a few
> compilers (clang, gcc 4.2, gcc 4.9),
> and there is some build time reduction, and no increase in build time.
> 
> Congratulations for tackling this stuff in FreeBSD.  As you mentioned, this
> area has not been touched
> in 14 years.
> --
> Craig

--
wbr, tiger

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: OpenSSH HPN

[Julian Elischer]

On 11/12/15 5:32 AM, Bryan Drewery wrote:

On 11/10/2015 1:42 AM, Dag-Erling Smørgrav wrote:

  I would also like to remove the NONE cipher
patch, which is also available in the port (off by default, just like in
base).

Fun fact, it's been broken in the port for several months with no
complaints. It was just reported and fixed upstream in the last day and
I wrote in a similar fix in the port. That speaks a lot about its usage
in the port currently.


we use it all the time, we just don't update all that often.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: OpenSSH HPN

[Slawa Olhovchenkov]

On Thu, Nov 12, 2015 at 12:15:35PM -0500, Allan Jude wrote:

> On 2015-11-11 19:06, Slawa Olhovchenkov wrote:
> > On Wed, Nov 11, 2015 at 01:32:27PM -0800, Bryan Drewery wrote:
> > 
> >> On 11/10/2015 1:42 AM, Dag-Erling Smørgrav wrote:
> >>>  I would also like to remove the NONE cipher
> >>> patch, which is also available in the port (off by default, just like in
> >>> base).
> >>
> >> Fun fact, it's been broken in the port for several months with no
> >> complaints. It was just reported and fixed upstream in the last day and
> >> I wrote in a similar fix in the port. That speaks a lot about its usage
> >> in the port currently.
> > 
> > I am try using NPH/NONE with base ssh and confused: don't see
> > performance rise, too complex to enable and too complex for use.
> > 
> > ___
> > freebsd-current@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
> > 
> 
> I did a few quick (and dirty) benchmarks and it shows that the NONE
> cipher definitely makes a difference. Version of OpenSSL also seems to
> make a difference, as one might expect.
> 
> Note: openssh from ports seems to link against both base and ports
> libcrypto, I am still trying to make sure this isn't corrupting my
> benchmark results.
> 
> I am still debugging my dummynet setup to be able to prove that HPN
> makes a difference (but it does).
> 
> https://wiki.freebsd.org/SSHPerf

I see you test NONE only on OpenSSH_7.1p1/1.0.2d.
I am try OpenSSH_6.6.1p1./1.0.1p (both side)
I am got about 500Mbit/s.
For OpenSSH_6.6.1p1/NONE I am got abot same.

I am don't see this combination in you table (OpenSSH_6.6.1p1./1.0.1p ix0 
OpenSSH_6.6.1p1./1.0.1p)
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

LOR in mpr(4)

[Pete Wright]

Hi All,
Just wanted a sanity check before filing a PR.  I am running r290688 and
am seeing a LOR being triggered in the mpr(4) device:

$ uname -ar
FreeBSD srd0013 11.0-CURRENT FreeBSD 11.0-CURRENT #1 r290688: Wed Nov 11
21:28:26 PST 2015 root@srd0013:/usr/obj/usr/src/sys/GENERIC  amd64


lock order reversal:
 1st 0xf8000d26bc60 CAM device lock (CAM device lock) @
/usr/src/sys/cam/cam_xpt.c:784
 2nd 0xfe00012811c0 MPR lock (MPR lock) @
/usr/src/sys/cam/cam_xpt.c:2620
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
0xfe04608ee890
witness_checkorder() at witness_checkorder+0xe79/frame 0xfe04608ee910
__mtx_lock_flags() at __mtx_lock_flags+0xa4/frame 0xfe04608ee960
xpt_action_default() at xpt_action_default+0xb6c/frame 0xfe04608ee9b0
scsi_scan_bus() at scsi_scan_bus+0x1d5/frame 0xfe04608eea20
xpt_scanner_thread() at xpt_scanner_thread+0x15c/frame 0xfe04608eea70
fork_exit() at fork_exit+0x84/frame 0xfe04608eeab0
fork_trampoline() at fork_trampoline+0xe/frame 0xfe04608eeab0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---


I can file a PR for this as a quick search on my end didn't return any
bugs related to this.  Checking here first to make sure this isn't
already known/expected behavior.

Thanks!
-pete

-- 
Pete Wright
p...@nomadlogic.org
twitter => @nomadlogicLA

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: OpenSSH HPN

[John-Mark Gurney]

Allan Jude wrote this message on Thu, Nov 12, 2015 at 12:15 -0500:
> On 2015-11-11 19:06, Slawa Olhovchenkov wrote:
> > On Wed, Nov 11, 2015 at 01:32:27PM -0800, Bryan Drewery wrote:
> > 
> >> On 11/10/2015 1:42 AM, Dag-Erling Smørgrav wrote:
> >>>  I would also like to remove the NONE cipher
> >>> patch, which is also available in the port (off by default, just like in
> >>> base).
> >>
> >> Fun fact, it's been broken in the port for several months with no
> >> complaints. It was just reported and fixed upstream in the last day and
> >> I wrote in a similar fix in the port. That speaks a lot about its usage
> >> in the port currently.
> > 
> > I am try using NPH/NONE with base ssh and confused: don't see
> > performance rise, too complex to enable and too complex for use.
> 
> I did a few quick (and dirty) benchmarks and it shows that the NONE
> cipher definitely makes a difference. Version of OpenSSL also seems to
> make a difference, as one might expect.
> 
> Note: openssh from ports seems to link against both base and ports
> libcrypto, I am still trying to make sure this isn't corrupting my
> benchmark results.

You don't need the aesni.ko module loaded for OpenSSL (which is how
OpenSSH uses most crypto algos) to use AES-NI..

Also, do you set any sysctl's to play w/ the buffer sizes or anything?

> I am still debugging my dummynet setup to be able to prove that HPN
> makes a difference (but it does).

Does my example on the page not work for you?

> https://wiki.freebsd.org/SSHPerf

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: OpenSSH HPN

[Slawa Olhovchenkov]

On Thu, Nov 12, 2015 at 12:51:30PM -0500, Allan Jude wrote:

> On 2015-11-12 12:44, Slawa Olhovchenkov wrote:
> > On Thu, Nov 12, 2015 at 12:15:35PM -0500, Allan Jude wrote:
> > 
> >> On 2015-11-11 19:06, Slawa Olhovchenkov wrote:
> >>> On Wed, Nov 11, 2015 at 01:32:27PM -0800, Bryan Drewery wrote:
> >>>
>  On 11/10/2015 1:42 AM, Dag-Erling Smørgrav wrote:
> >  I would also like to remove the NONE cipher
> > patch, which is also available in the port (off by default, just like in
> > base).
> 
>  Fun fact, it's been broken in the port for several months with no
>  complaints. It was just reported and fixed upstream in the last day and
>  I wrote in a similar fix in the port. That speaks a lot about its usage
>  in the port currently.
> >>>
> >>> I am try using NPH/NONE with base ssh and confused: don't see
> >>> performance rise, too complex to enable and too complex for use.
> >>>
> >>> ___
> >>> freebsd-current@freebsd.org mailing list
> >>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> >>> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
> >>>
> >>
> >> I did a few quick (and dirty) benchmarks and it shows that the NONE
> >> cipher definitely makes a difference. Version of OpenSSL also seems to
> >> make a difference, as one might expect.
> >>
> >> Note: openssh from ports seems to link against both base and ports
> >> libcrypto, I am still trying to make sure this isn't corrupting my
> >> benchmark results.
> >>
> >> I am still debugging my dummynet setup to be able to prove that HPN
> >> makes a difference (but it does).
> >>
> >> https://wiki.freebsd.org/SSHPerf
> > 
> > I see you test NONE only on OpenSSH_7.1p1/1.0.2d.
> > I am try OpenSSH_6.6.1p1./1.0.1p (both side)
> > I am got about 500Mbit/s.
> > For OpenSSH_6.6.1p1/NONE I am got abot same.
> > 
> > I am don't see this combination in you table (OpenSSH_6.6.1p1./1.0.1p ix0 
> > OpenSSH_6.6.1p1./1.0.1p)
> > 
> 
> If NONE is actually being used, big warnings will be printed to your screen:
> 
> WARNING: ENABLED NONE CIPHER
> WARNING: ENABLED NONE CIPHER
> 
> If you don't see this, NONE is not being used.

I am see this, of couse.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] build: WITH_FAST_DEPEND and WITH_CCACHE_BUILD

[Bryan Drewery]

On 11/12/2015 9:40 AM, Bryan Drewery wrote:
> It is only useful for frequent builders who do not use -DNO_CLEAN and
> want a reliable incremental build. 

I forgot to mention that I am also working on making buildworld reliably
incremental without ccache. This will not be ready until the Spring
probably though.

-- 
Regards,
Bryan Drewery



signature.asc
Description: OpenPGP digital signature

Re: OpenSSH HPN

[Allan Jude]

On 2015-11-12 12:44, Slawa Olhovchenkov wrote:
> On Thu, Nov 12, 2015 at 12:15:35PM -0500, Allan Jude wrote:
> 
>> On 2015-11-11 19:06, Slawa Olhovchenkov wrote:
>>> On Wed, Nov 11, 2015 at 01:32:27PM -0800, Bryan Drewery wrote:
>>>
 On 11/10/2015 1:42 AM, Dag-Erling Smørgrav wrote:
>  I would also like to remove the NONE cipher
> patch, which is also available in the port (off by default, just like in
> base).

 Fun fact, it's been broken in the port for several months with no
 complaints. It was just reported and fixed upstream in the last day and
 I wrote in a similar fix in the port. That speaks a lot about its usage
 in the port currently.
>>>
>>> I am try using NPH/NONE with base ssh and confused: don't see
>>> performance rise, too complex to enable and too complex for use.
>>>
>>> ___
>>> freebsd-current@freebsd.org mailing list
>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>>> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
>>>
>>
>> I did a few quick (and dirty) benchmarks and it shows that the NONE
>> cipher definitely makes a difference. Version of OpenSSL also seems to
>> make a difference, as one might expect.
>>
>> Note: openssh from ports seems to link against both base and ports
>> libcrypto, I am still trying to make sure this isn't corrupting my
>> benchmark results.
>>
>> I am still debugging my dummynet setup to be able to prove that HPN
>> makes a difference (but it does).
>>
>> https://wiki.freebsd.org/SSHPerf
> 
> I see you test NONE only on OpenSSH_7.1p1/1.0.2d.
> I am try OpenSSH_6.6.1p1./1.0.1p (both side)
> I am got about 500Mbit/s.
> For OpenSSH_6.6.1p1/NONE I am got abot same.
> 
> I am don't see this combination in you table (OpenSSH_6.6.1p1./1.0.1p ix0 
> OpenSSH_6.6.1p1./1.0.1p)
> 

If NONE is actually being used, big warnings will be printed to your screen:

WARNING: ENABLED NONE CIPHER
WARNING: ENABLED NONE CIPHER

If you don't see this, NONE is not being used.

-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: OpenSSH HPN

[Allan Jude]

On 2015-11-11 19:06, Slawa Olhovchenkov wrote:
> On Wed, Nov 11, 2015 at 01:32:27PM -0800, Bryan Drewery wrote:
> 
>> On 11/10/2015 1:42 AM, Dag-Erling Smørgrav wrote:
>>>  I would also like to remove the NONE cipher
>>> patch, which is also available in the port (off by default, just like in
>>> base).
>>
>> Fun fact, it's been broken in the port for several months with no
>> complaints. It was just reported and fixed upstream in the last day and
>> I wrote in a similar fix in the port. That speaks a lot about its usage
>> in the port currently.
> 
> I am try using NPH/NONE with base ssh and confused: don't see
> performance rise, too complex to enable and too complex for use.
> 
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
> 

I did a few quick (and dirty) benchmarks and it shows that the NONE
cipher definitely makes a difference. Version of OpenSSL also seems to
make a difference, as one might expect.

Note: openssh from ports seems to link against both base and ports
libcrypto, I am still trying to make sure this isn't corrupting my
benchmark results.

I am still debugging my dummynet setup to be able to prove that HPN
makes a difference (but it does).

https://wiki.freebsd.org/SSHPerf

-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

[CFT] build: WITH_FAST_DEPEND and WITH_CCACHE_BUILD

[Bryan Drewery]

Hi,

Recently I have introduced two new features into the build. These apply
to anything using /usr/share/mk including buildworld, buildkernel,
universe, etc.

- The first is WITH_FAST_DEPEND. Please see the commit for its full
description, benefits and discussion. It saves 16-30% in build times
without gimmicks or risk. It avoids running the preprocessor twice in
the build and only runs it during compilation. This is a feature that's
14 years overdue.

https://svnweb.freebsd.org/changeset/base/290433

There was a problem with the GCC build, but that has been fixed. I
intend to enable this *by default* for the src buildworld/etc... build
in a few weeks.

I do need to schedule a ports exp-run with this as well to see if
anything outside of the src tree is broken by it, which I very much
doubt. If you have a downstream fork of FreeBSD, please import r290433
and r290629 and give me feedback on any issues you encounter.

There is 1 gotcha that I realized. People running 'make depend' manually
may actually want to see a .depend file generated without having to
compile first. I may add support for that somehow but am not sure yet.
It may be a 'make mkdep' target.

- The second is WITH_CCACHE_BUILD. This replaces the previous suggestion
of modifying CC and CXX in /etc/make.conf.  This is purposely chosen to
match the ports name. It can save up to 65% build times when combined
with WITH_FAST_DEPEND. This fixes all known issues with
buildworld+ccache. There is a rare problem that can occur with header
detection that is documented in the ccache manpage in the DIRECT MODE
section.

**It is only useful for frequent builders who do not use -DNO_CLEAN and
want a reliable incremental build. It is not useful for people who build
infrequently.**

See commit for further details and stats. I do not intend to ever
support enabling this by default. I just intend to update the
devel/ccache pkg-message to suggest using it once it is known to work
for all.

https://svnweb.freebsd.org/changeset/base/290526


Thanks!
Bryan Drewery




signature.asc
Description: OpenPGP digital signature