Re: r228700 can't dhclient em0

2011-12-20 Thread Doug Barton
On 12/19/2011 05:24, Dimitry Andric wrote:
 On 2011-12-19 10:17, Doug Barton wrote:
 I updated to r228700 from 228122 and dhclient exits immediately saying
 that em0 doesn't exist. However ifconfig seems to disagree:


 em0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST  metric 0 mtu
 1500

 options=4219bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO

  ether 00:24:e8:30:10:9b
  nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL
  media: Ethernet autoselect (100baseTXfull-duplex)
  status: active
 lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST  metric 0 mtu 16384
  options=3RXCSUM,TXCSUM
  nd6 options=21PERFORMNUD,AUTO_LINKLOCAL


 Interestingly, some of the options are different in that version, vs.
 the working version:

 em0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST  metric 0 mtu
 1500
 
 options=219bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC

 ether 00:24:e8:30:10:9b
 inet 172.17.198.245 netmask 0x broadcast 172.17.255.255
 nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL
 media: Ethernet autoselect (100baseTXfull-duplex)
 status: active
 
 I saw this too, when my kernel and userland were out of sync (e.g. just
 after installing a new kernel, and before installworld).  I suspect it
 is caused by the changes in r228571, which cause old ifconfig and
 dhclient to not recognize any interfaces.  I'm not 100% sure though...

I tried replacing both ifconfig and dhclient with the versions that were
built along with the new kernel, and that didn't work.

The traditional (and documented) upgrade process for many years has been
to boot the new kernel, make sure it's Ok, then update world. Obviously
something different is needed this time, so it needs an UPDATING entry
(assuming that all this is not just a bug).


Doug

-- 

[^L]

Breadth of IT experience, and depth of knowledge in the DNS.
Yours for the right price.  :)  http://SupersetSolutions.com/

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Failure to compile world

2011-12-20 Thread Garrett Cooper
On Mon, Dec 19, 2011 at 9:36 PM, Alex Kuster vertexsymph...@zoho.com wrote:
 On 12/20/2011 01:52, Garrett Cooper wrote:

 On Mon, Dec 19, 2011 at 7:31 PM, Alex Kustervertexsymph...@zoho.com
  wrote:

 A follow-up on this is libc not building because of missing
 SCTP_REMOTE_UDP_ENCAPS_PORT
 apparently the Makefile doesn't include /sys/ into the includes of the
 libc.

 My current version (/usr/include/netinet/sctp.h) lacks that definition,
 it
 should look in the headers of the source, not the current system headers
 ...
 so I just added that to the Makefile ( lib/libc/net/Makefile.inc ).

 I'm leaving note if anyone else experiences the same problem.

     Please file a PR for this and other similar build issues. The
 mantra I've gotten in the past is that builds aren't guaranteed to
 work in a subdirs, but I would really like for this to become a
 reality because I really wouldn't want to have to installworld (or
 installincludes) a whole system just to get some headers installed for
 a trivial program in the base system :).
     Just to make sure though, did you do make depend all , or just make
 all?
 Thanks!
 -Garrett


 Hi Garett ... Well, those issues were raised by a simple make buildworld
 in the traditional /usr/src
 When I found the first issue with libc i just went to /usr/src/lib/libc,
 fixed and ran a make in there, so the second issue appeared and libc was
 built with no problems.

Hmm.. yeah, that's a bug then.

 Now I'm facing another one which I'll find out and see how to fix to get a
 compiling/working system.

Ok.

 Thanks for your time!

Np!

 P.S → I didn't know about installincludes, I'll read about that

make distribution does more than that too (and mergemaster runs that).
man build has some other interesting tricks that may or may not be of
value to you.

 P.S 2 → I never-ever-ever filed a PR

http://www.freebsd.org/send-pr.html

Cheers!
-Garrett
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: WITHOUT_PROFILE=yes by default

2011-12-20 Thread Christer Solskogen
On Mon, Dec 19, 2011 at 8:46 PM, Warner Losh i...@bsdimp.com wrote:

 On Dec 2, 2011, at 3:37 PM, Steve Kargl wrote:

 On Fri, Dec 02, 2011 at 04:21:14PM +0700, Max Khon wrote:

 The most important thing is to have reasonable defaults.
 Having WITH_PROFILE by default does not seem to be a reasonable default to 
 me.


 Now all users that want to profile anything need to build their own custom 
 FreeBSD?  That seems even more nuts to me.

So that all users that do not want to profile anything need to build
their own custom FreeBSD?

-- 
chs,
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: WITHOUT_PROFILE=yes by default

2011-12-20 Thread sthaug
  Now all users that want to profile anything need to build their own custom 
  FreeBSD?  That seems even more nuts to me.
 
 So that all users that do not want to profile anything need to build
 their own custom FreeBSD?

No. It simply means these users will have profiled libraries available
that they don't use.

FWIW, I support keeping the build of profiled libraries.

Steinar Haug, Nethelp consulting, sth...@nethelp.no
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: SCHED_ULE should not be the default

2011-12-20 Thread Gary Jennejohn
On Mon, 19 Dec 2011 23:22:40 +0200
Andriy Gapon a...@freebsd.org wrote:

 on 19/12/2011 17:50 Nathan Whitehorn said the following:
  The thing I've seen is that ULE is substantially more enthusiastic about
  migrating processes between cores than 4BSD.
 
 Hmm, this seems to be contrary to my theoretical expectations.  I thought that
 with 4BSD all threads that were not in one of the following categories:
 - temporary pinned
 - bound to cpu in kernel via sched_bind
 - belong to a cpu set which a strict subset of a total set
 were placed onto a common queue that was shared by all cpus.  And as such I
 expected them to get picked up by the cpus semi-randomly.
 
 In other words, I thought that it was ULE that took into account cpu/cache
 affinities while 4BSD was deliberately entirely ignorant of those details.
 

I have a 6-core AMD CPU running FreeeBSD 10.0 and SCHED_4BSD.

I've noticed with large ports builds which are not MAKE_JOBS_SAFE that
the compile load migrates between the cores pretty quickly, but I haven't
compared it to ULE.

-- 
Gary Jennejohn
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Christer Solskogen
On Mon, Dec 19, 2011 at 2:16 PM, Alexander Yerenkow yeren...@gmail.com wrote:
 FreeBSD currently have very obscure, closed community. To get in touch, you
 need to subscribe to several mail lists, constantly read them, I've just
 found recently (my shame of course) in mail list that there is service (
 pub.allbsd.org) which constantly building current versions. This is great,
 but at homepage of freebsd.org there is no word about it :)

That's because it's not official. Do you take the risk? Would a
multi-milion-dollar company do that?
For your private server, sure it's probably fine. But how do you know
that those files are not contaminated?
(That being said, the purpose of that service is good. And the files
there a most probably 100% fine. But if it's not official... then..)

-- 
chs, if there is only one candiate, there is one one choice!
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Garrett Cooper
On Dec 20, 2011, at 1:01 AM, Christer Solskogen wrote:

 On Mon, Dec 19, 2011 at 2:16 PM, Alexander Yerenkow yeren...@gmail.com 
 wrote:
 FreeBSD currently have very obscure, closed community. To get in touch, you
 need to subscribe to several mail lists, constantly read them, I've just
 found recently (my shame of course) in mail list that there is service (
 pub.allbsd.org) which constantly building current versions. This is great,
 but at homepage of freebsd.org there is no word about it :)
 
 That's because it's not official. Do you take the risk? Would a
 multi-milion-dollar company do that?
 For your private server, sure it's probably fine. But how do you know
 that those files are not contaminated?
 (That being said, the purpose of that service is good. And the files
 there a most probably 100% fine. But if it's not official... then..)

As long as I have reliable checksums that match the what the upstream source 
says is the real thing, it doesn't practically matter where I get my images 
from.
-Garrett___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Christer Solskogen
On Tue, Dec 20, 2011 at 10:42 AM, Garrett Cooper yaneg...@gmail.com wrote:

 As long as I have reliable checksums that match the what the upstream source 
 says is the real thing, it doesn't practically matter where I get my images 
 from.

Checksums compared to what? How would you know what the correct
checksums for OpenBSD-current is, if it's not built by Theo?


-- 
chs,
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Garrett Cooper
On Dec 20, 2011, at 1:51 AM, Christer Solskogen wrote:

 On Tue, Dec 20, 2011 at 10:42 AM, Garrett Cooper yaneg...@gmail.com wrote:
 
 As long as I have reliable checksums that match the what the upstream source 
 says is the real thing, it doesn't practically matter where I get my images 
 from.
 
 Checksums compared to what? How would you know what the correct
 checksums for OpenBSD-current is, if it's not built by Theo?

Release engineering for FreeBSD produces SHA256 checksums for all 
official releases. AFAIK though they're only in the announcement emails and not 
stored anywhere else.
I can't speak for OpenBSD's release process.
Thanks,
-Garrett___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Is the svn2cvs gateway down ?

2011-12-20 Thread Claude Buisson

Hi,

It seems (from my own csup's and cvswe.cgi) that the src commits are lost,
starting with r228697 Sun Dec 18 22:04:55 2011)

What is going on (or off) ?

Claude Buisson
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Uneven load on drives in ZFS RAIDZ1

2011-12-20 Thread Stefan Esser
Am 19.12.2011 17:36, schrieb Michael Reifenberger:
 Hi,
 a quick test using `dd if=/dev/zero of=/test ...` shows:
 
 dT: 10.004s  w: 10.000s  filter: ^a?da?.$
  L(q)  ops/sr/s   kBps   ms/rw/s   kBps   ms/w   %busy Name
 0378  0  0   12.5376  36414   11.9   60.6| ada0
 0380  0  0   12.2378  36501   11.8   60.0| ada1
 0382  0  07.7380  36847   11.6   59.2| ada2
 0375  0  07.4374  361649.6   51.3| ada3
 0377  0  1   10.2375  36325   10.1   53.3| ada4
10391  0  0   39.3389  38064   15.7   80.2| ada5
 
 Seems to be sufficiently equally distributed for a life system...

Hi Michael,

in an earlier reply I mentioned the suspicious queue length and %busy of
ada5, which may be the result of other load (not caused by the dd
command) or of a hardware problem (I'd check drive health ...).

(Hmmm, the numbers look strange: ops/s is not the sum of r/s and w/s,
but misses that value by 2. I could understand a rounding difference of
1, but not 2 counts per second. But this is a different issue ...)

Anyway: The imbalance that I observe on my system is specific to reads,
not writes. Could you please check, whether sending a large (multi-GB)
file to /dev/null shows identical read load over all drives?

I suspect that 2 of the drives will see slightly (some 20%, perhaps)
less read requests than the rest.

Regards, STefan
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Daniel Kalchev



On 20.12.11 11:42, Garrett Cooper wrote:
As long as I have reliable checksums that match the what the upstream 
source says is the real thing, it doesn't practically matter where I 
get my images from.


Relying on checksums that are published on the same web site where you 
download the files from and given that most of these sites do not even 
use SSL so much about 'security'.


Daniel
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: cross-arch building picobsd/nanobsd images ?

2011-12-20 Thread Olivier Cochard-Labbé
On Mon, Dec 19, 2011 at 11:45 PM, Luigi Rizzo ri...@iet.unipi.it wrote:

 On a related topic, does anyone have experience on cross-building
 nanobsd images ?

Hi Luigi,

I using little cross-building nanobsd images (i386 on amd64 and vice versa).
All my patchs for nanobsd are available on BSD Router Project
(http://bsdrp.net) including a patch for compiling ports from nanobsd
too.

Right now I'm working on adding cross-build mips (RouterStation Pro)
nanobsd patch but without the compiling ports feature, because I can
only cross-compile word/kernel and I didn't know how to cross-compile
ports.

Regards,

Olivier
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Uneven load on drives in ZFS RAIDZ1

2011-12-20 Thread Stefan Esser
Am 19.12.2011 22:53, schrieb Dan Nelson:
 In the last episode (Dec 19), Stefan Esser said:
 poolalloc   free   read  write   read  write
 --  -  -  -  -  -  -
 raid1   4.41T  2.21T139 72  12.3M   818K
   raidz14.41T  2.21T139 72  12.3M   818K
 ada0p2  -  -114 17  4.24M   332K
 ada1p2  -  -106 15  3.82M   305K
 ada2p2  -  - 65 20  2.09M   337K
 ada3p2  -  - 58 18  2.18M   329K

 The same difference of read operations per second as shown by gstat ...
 
 I was under the impression that the parity blocks were scattered evenly
 across all disks, but from reading vdev_raidz.c, it looks like that isn't
 always the case.  See the comment at the bottom of the
 vdev_raidz_map_alloc() function; it looks like it will toggle parity between
 the first two disks in a stripe every 1MB.  It's not necessarily the first

Thanks, this is very interesting information, indeed. I observed the
problem when minidlna rebuild its index database, which scans all media
files, many of them GBytes in length and sequentially written. This is a
typical scenario that should trigger the code you point at.

The comment explains that an attempt has been made to spread the (read)
load more evenly, if large files are sequentially written:

 * If all data stored spans all columns, there's a danger that parity
 * will always be on the same device and, since parity isn't read
 * during normal operation, that that device's I/O bandwidth won't be
 * used effectively. We therefore switch the parity every 1MB.

But they later found, that they failed to implement a good solution:

 * ... at least that was, ostensibly, the theory. As a practical
 * matter unless we juggle the parity between all devices evenly, we
 * won't see any benefit. Further, occasional writes that aren't a
 * multiple of the LCM of the number of children and the minimum
 * stripe width are sufficient to avoid pessimal behavior.

But I do not understand the reasoning behind:

 * Unfortunately, this decision created an implicit on-disk format
 * requirement that we need to support for all eternity, but only
 * for single-parity RAID-Z.

I see how the devidx and offset are swapped between col[0] and col[1],
and it appears that this swapping is not explicitly reflected in the
meta data. But there is no reason, that the algorithm could not be
modified to cover all drives, if some flag is set (which effectively
would lead to a 2nd generation raidz1 with incompatible block layout).

Anyway, I do not think that the current behavior is so bad, that it
needs immediate fixing.

 two disks assigned to the zvol, since stripes don't have to span all disks
 as long as there's one parity block (a small sync write may just hit two
 disks, essentially being written mirrored).  The imbalance is only visible
 if you're writing full-width stripes in sequence, so if you write a 1TB file
 in one long stream, chances are that that file's parity blocks will be
 concentrated on just two disks, so those two disks will get less I/O on
 later reads.  I don't know why the code toggles parity between just the
 first two columns; rotating it between all columns would give you an even
 balance.

Yes, but as the comment indicates, this would require introduction of a
different raidz1 (a higher ZFS revision or a flag could trigger that).

 Is it always the last two disks that have less load, or does it slowly
 rotate to different disks depending on the data that you are reading?  An
 interesting test would be to idle the system, run a tar cvf /dev/null
 /raidz1 in one window, and watch iostat output on another window.  If the
 load moves from disk to disk as tar reads different files, then my parity
 guess is probably right.  If ada0 and ada1 are always busier, than you can
 ignore me :)

Yes, you are perfectly right! I tested the tar on a spool directory
holding DVB-C recordings (typical files length 2GB to 8GB). The

dT: 10.001s  w: 10.000s  filter: ^a?da?.$
 L(q)  ops/sr/s   kBps   ms/rw/s   kBps   ms/w   %busy Name
0935921  402160.4 131390.5   32.8| ada0
0927913  365300.3 131081.5   31.8| ada1
0474460  201100.7 141410.9   32.4| ada2
0474461  201020.7 131410.7   31.6| ada3

 L(q)  ops/sr/s   kBps   ms/rw/s   kBps   ms/w   %busy Name
0   1046   1041  455030.3  5 350.9   31.5| ada0
0   1039   1035  413530.3  4 230.4   31.6| ada1
0531526  228270.6  5 380.4   33.4| ada2
1523518  227720.6  5 380.6   30.8| ada3

 L(q)  ops/sr/s   kBps   ms/rw/s   kBps   ms/w   %busy Name
0384377  164140.8  7 463.3   30.2| ada0
0380373  158570.8  6 420.4   30.5| ada1
0553547  239370.5  6 

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Chiron IO
Guys,

I have a question about these benchmarks.

Why worry about that if the CURRENT comes with debug enabled by default?

http://joaobarros.blogspot.com/2005/07/freebsd-how-to-turn-off-debug-options.html




On 19/12/2011, at 22:28, Petro Rossini wrote:

 Hi all,
 
 just a thought here:
 
 On Tue, Dec 20, 2011 at 12:45 AM, Daniel Kalchev dan...@digsys.bg wrote:
 As were told, Phoronix used default setup, not tuned.
 Not really. They created some weird test environment, at least for FreeBSD
 -- who knows, possibly for Linux as well.
 
 For example, ZFS is by no means a default file system in FreeBSD. You need
 to go trough manual steps, to enable it, to build the pool, filesystems etc.
 
 ..
 
 Of course the benchmark setup and procedure is strange but..
 
 it could be improved, I think.
 
 Have a good collection of tuning parameters for popular cases,
 advertised properly so it gets hard to miss them.
 
 I am a sysadmin and, over the years, I had to run file servers,
 database servers, web servers, tomcats...
 
 Well, most of the time I set it up and it just works because the
 system in question is not maxed out, not even close to it.
 
 But if I want to squeeze the last 20% out of it googling starts, and
 here and there I find hints how to tune the OS, the file system, what
 scheduler to use etc.
 
 It would be great to have a set of case studies at hand, e.g. under
 the /usr/share/examples directory, that describes tweaks to have a
 performing postgresql server, or mysql, or apache or a desktop or..
 
 Things I find, for example, in the BSD Magazine.
 
 Maybe benchmarks become more meaningful then..
 
 A general remark for people doing benchmarks for comparison: you need
 a well-informed system engineer for the systems you compare. So, if
 you compare a Linux system with  FreeBSD, have two experienced admins
 that know their OS well.
 
 Regards
 Peter
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: a few usb issues related to edge cases

2011-12-20 Thread Andriy Gapon

Now the juicy stuff :)

on 19/12/2011 16:30 Hans Petter Selasky said the following:
 3.  Looking at usbd_transfer_poll I see that it touches a lot of locks,
 including taking the bus lock.  As we've discussed before, this is not safe
 in a particular context where the polling is supposed to be used - in the
 kdb/ddb context.  If the lock is already taken by another thread, then
 instead of being able to use a USB keyboard a user would get even less
 debug-able crash.  Also, it seems that usbd_transfer_poll calls into the
 usual state machine with various callbacks and dynamically made decisions
 about whether to execute some actions directly or defer their execution to
 a different thread. 
 
 This is an optimisation. If the current thread can do the job without a LOR, 
 then we do it right away. Else we let another thread do it. It is possible to 
 have a more simple model, but then you will also get more task switches.

I think that you are speaking here about the code behavior in the general 
context.
And I want to limit this part of discussion to the contexts where
usbd_transfer_poll is actually used - kdb and panic.  In those contexts there is
one and only one thread that must do all the work.  Other threads are stopped
and frozen in the middle of whatever they were doing before kdb was entered or
panic called (provided SCHEDULER_STOPPED() == true).

 That code also touches locks in various places.  I
 think that it would be more preferable to have a method that does the job
 in a more straight-forward way, without touching any locks, ignoring the
 usual code paths and assuming that no other treads are running in
 parallel.  Ditto for the method to submit a request.
 
 The current USB code can be run fine without real locks, if you do a few 
 tricks. I have a single-threaded BSD-kernel replacement for this which works 
 like a charm for non-FreeBSD projects.

That's very cool.  I believe that various implementations of USB stack for BIOS
and similar are also non-threaded.

 I'm going to paste a few lines FYI:
 
 Why not extend struct mtx to have two fields which are only used in case of 
 system polling (no scheduler running):
 
 struct mtx {
   xxx;
   int owned_polling = 0;
   struct mtx *parent_polling;
 };
 
 void
 mtx_init(struct mtx *mtx, const char *name, const char *type, int opt)
 {
 mtx-owned = 0;
 mtx-parent = mtx;
 }
 
 void
 mtx_lock(struct mtx *mtx)
 {
 mtx = mtx-parent;
 mtx-owned++;
 }
 
 void
 mtx_unlock(struct mtx *mtx)
 {
 mtx = mtx-parent;
 mtx-owned--;
 }
 
 int
 mtx_owned(struct mtx *mtx)
 {
 mtx = mtx-parent;
 return (mtx-owned != 0);
 }
 
 void
 mtx_destroy(struct mtx *mtx)
 {
 /* NOP */
 }
 
 Maybe mtx_init, mtx_lock, mtx_unlock mtx_owned, mtx_destroy, etc, could be 
 function pointers, which are swapped at panic.

I am not sure if I got your idea right based on the pseudo-code above.
Right now in the head we already skip all locks when SCHEDULER_STOPPED() is
true.  But, but, that doesn't cover all polling cases as the automatic lock
skipping is _not_ done for kdb.  There are various reasons for that.  That's why
the kdb_active flag should be checked by the code that can be executed in the
kdb context when dealing with locking or shared resources in general.


 USB is SMP! 

Right and that's good, but there are cases where SMP is effectively stopped.
Those are panic and kdb.

 To run SMP code from a single thread, you need to create a 
 hiherachy of the threads:
 
 1) Callbacks (Giant)
 2) Callbacks (non-Giant)
 3) Control EP (non-Giant)
 4) Explore thread (non-Giant)
 
 When the explore thread is busy, we look for work in the level above and so 
 on. The USB stack implements this principle, which is maybe not documented 
 anywhere btw. If you want more than code, you can hire me to do that.
 
 The mtx-code above I believe is far less work than to make new code which 
 handles the polling case only.
 
 The reason for the parent mutex field, is to allow easy grouping of mutexes, 
 so that USB easily can figure out what can run or not.

This all is really above my level of knowledge.

Basically my current intention is the following: make ukbd/usb code work in
panic+SCHEDULER_STOPPED case in the same way (or not worse, at least) as it
currently works for the kdb case.  I don't have enough knowledge (and time) to
go beyond that.
I just wanted to draw your attention to the fact that obtaining any locks in the
kdb context (or USB polling code in general, even) is not a good idea.
Chances of getting into trouble on those locks are probably quite moderate or
even low, but they do exist.  I am not sure if you are getting any bug reports
about such troubles :-)  Regular users probably do not use kdb too often and a
panic for them is just a crash, so they likely do not expect anything
usable/debuggable after that :-)

P.S.  Since most (but not all) locking operations in the USB code are already
wrapped under 

x11/sessreg: build fails with CLANG in FreeBSD 9.0-PRERELEASE

2011-12-20 Thread O. Hartmann
On a freshly updated box the installation of x11/sessreg fails with the
shown message below.
On all boxes I run with FBSD 9 or 10 (all amd64, CLANG build) the build
and installation works fine.

Since I update the box from 8.2-STABLE to 9.0-PRE last night, cleaning
up all ports and having them rebuilt from scratch, I guess I have a
problem with remnants (or missing compatibility?) in the system.

Where to start looking? ttyslot looks like a base system function.


Regards, thanks in advance,
Oliver

===  Building for sessreg-1.0.7
/usr/bin/make  all-recursive
Making all in man
  GENfilenames.sed
  GENsessreg.1
  CC sessreg.o
sessreg.c:281:27: warning: implicit declaration of function 'ttyslot' is
invalid in C99 [-Wimplicit-function-declaration]
sysnerr (slot_number = ttyslot (), ttyslot);
   ^
1 warning generated.
  CCLD   sessreg
sessreg.o: In function `main':
sessreg.c:(.text+0x7bf): undefined reference to `ttyslot'
clang: error: linker command failed with exit code 1 (use -v to see
invocation)
*** Error code 1

Stop in /usr/ports/x11/sessreg/work/sessreg-1.0.7.
*** Error code 1

Stop in /usr/ports/x11/sessreg/work/sessreg-1.0.7.
*** Error code 1

Stop in /usr/ports/x11/sessreg/work/sessreg-1.0.7.
*** Error code 1

Stop in /usr/ports/x11/sessreg.



signature.asc
Description: OpenPGP digital signature


Re: x11/sessreg: build fails with CLANG in FreeBSD 9.0-PRERELEASE

2011-12-20 Thread O. Hartmann
On 12/20/11 11:13, O. Hartmann wrote:
 On a freshly updated box the installation of x11/sessreg fails with the
 shown message below.
 On all boxes I run with FBSD 9 or 10 (all amd64, CLANG build) the build
 and installation works fine.
 
 Since I update the box from 8.2-STABLE to 9.0-PRE last night, cleaning
 up all ports and having them rebuilt from scratch, I guess I have a
 problem with remnants (or missing compatibility?) in the system.
 
 Where to start looking? ttyslot looks like a base system function.
 
 
 Regards, thanks in advance,
 Oliver
 
 ===  Building for sessreg-1.0.7
 /usr/bin/make  all-recursive
 Making all in man
   GENfilenames.sed
   GENsessreg.1
   CC sessreg.o
 sessreg.c:281:27: warning: implicit declaration of function 'ttyslot' is
 invalid in C99 [-Wimplicit-function-declaration]
 sysnerr (slot_number = ttyslot (), ttyslot);
^
 1 warning generated.
   CCLD   sessreg
 sessreg.o: In function `main':
 sessreg.c:(.text+0x7bf): undefined reference to `ttyslot'
 clang: error: linker command failed with exit code 1 (use -v to see
 invocation)
 *** Error code 1
 
 Stop in /usr/ports/x11/sessreg/work/sessreg-1.0.7.
 *** Error code 1
 
 Stop in /usr/ports/x11/sessreg/work/sessreg-1.0.7.
 *** Error code 1
 
 Stop in /usr/ports/x11/sessreg/work/sessreg-1.0.7.
 *** Error code 1
 
 Stop in /usr/ports/x11/sessreg.


Sorry for the noise.

As I gave myself the answer, some remnants polluted the system and I got
rid by simply call the propper make delete-old-libs/files in /usr/src.

Oliver



signature.asc
Description: OpenPGP digital signature


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Vincent Hoffman
On 20/12/2011 10:39, Daniel Kalchev wrote:


 On 20.12.11 11:42, Garrett Cooper wrote:
 As long as I have reliable checksums that match the what the upstream
 source says is the real thing, it doesn't practically matter where I
 get my images from.

 Relying on checksums that are published on the same web site where you
 download the files from and given that most of these sites do not even
 use SSL so much about 'security'.

This does remind me of one issue that while a little off topic for this
thread
If i wanted to get, for example the SHA265 checksums from a verified
source, how would i verify this currently? There doesnt seem to be an
SSL site for www.freebsd.org and its not too hard to redirect someone to
a fake website.
What would be a more reasonable list to request this on?

Vince


 Daniel
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to
 freebsd-current-unsubscr...@freebsd.org

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Sleeping thread (tid 100033, pid 16): panic in FreeBSD 10.0-CURRENT/amd64 r228662

2011-12-20 Thread John Baldwin
On Saturday, December 17, 2011 10:41:15 pm m...@freebsd.org wrote:
 On Sat, Dec 17, 2011 at 5:45 PM, Alexander Kabaev kab...@gmail.com wrote:
  On Sun, 18 Dec 2011 01:09:00 +0100
  O. Hartmann ohart...@zedat.fu-berlin.de wrote:
 
  Sleeping thread (tid 100033, pid 16) owns a non sleepable lock
  panic: sleeping thread
  cpuid = 0
 
  PID 16 is always USB on my box.
 
  You really need to give us a backtrace when you quote panics. It is
  impossible to make any sense of the above panic message without more
  context.
 
 In the case of this panic, the stack of the thread which panics is
 useless; it's someone trying to propagate priority that discovered it.
  A backtrace on tid 100033 would be useful.
 
 With WITNESS enabled, it's possible to have this panic display the
 stack of the incorrectly sleeping thread at the time it acquired the
 lock, as well, but this code isn't in CURRENT or any release.  I have
 a patch at $WORK I can dig up on Monday.

Huh?  The stock kernel dumps a stack trace of the offending thread if you have 
DDB enabled:

/*
 * If the thread is asleep, then we are probably about
 * to deadlock.  To make debugging this easier, just
 * panic and tell the user which thread misbehaved so
 * they can hopefully get a stack trace from the truly
 * misbehaving thread.
 */
if (TD_IS_SLEEPING(td)) {
printf(
Sleeping thread (tid %d, pid %d) owns a non-sleepable lock\n,
td-td_tid, td-td_proc-p_pid);
#ifdef DDB
db_trace_thread(td, -1);
#endif
panic(sleeping thread);
}

It may be that we can make use of the STACK API here instead to output this
trace even when DDB isn't enabled.  The patch below tries to do that 
(untested).  It does some odd thigns though since it is effectively running
from a panic context already, so it uses a statically allocated 'struct stack'
rather than using stack_create() and uses stack_print_ddb() since it is 
holding spin locks and can't possibly grab an sx lock:

Index: subr_turnstile.c
===
--- subr_turnstile.c(revision 228534)
+++ subr_turnstile.c(working copy)
@@ -72,6 +72,7 @@ __FBSDID($FreeBSD$);
 #include sys/proc.h
 #include sys/queue.h
 #include sys/sched.h
+#include sys/stack.h
 #include sys/sysctl.h
 #include sys/turnstile.h
 
@@ -175,6 +176,7 @@ static void turnstile_fini(void *mem, int size);
 static void
 propagate_priority(struct thread *td)
 {
+   static struct stack st;
struct turnstile *ts;
int pri;
 
@@ -217,8 +219,10 @@ propagate_priority(struct thread *td)
printf(
Sleeping thread (tid %d, pid %d) owns a non-sleepable lock\n,
td-td_tid, td-td_proc-p_pid);
-#ifdef DDB
-   db_trace_thread(td, -1);
+#ifdef STACK
+   stack_zero(st);
+   stack_save_td(st, td);
+   stack_print_ddb(st);
 #endif
panic(sleeping thread);
}

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Sleeping thread (tid 100033, pid 16): panic in FreeBSD 10.0-CURRENT/amd64 r228662

2011-12-20 Thread Andriy Gapon
on 20/12/2011 15:52 John Baldwin said the following:
 -#ifdef DDB
 - db_trace_thread(td, -1);
 +#ifdef STACK
 + stack_zero(st);
 + stack_save_td(st, td);
 + stack_print_ddb(st);
  #endif

This leads to an idea - what about an umbrella stack_print() that would call the
best stack printing routine, if any, based on their availabilities (STACK, DDB,
etc?).

P.S. Sorry to hijack the thread.

-- 
Andriy Gapon
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: making crdup()/crcopy() safe??

2011-12-20 Thread John Baldwin
On Monday, December 19, 2011 8:21:45 pm Rick Macklem wrote:
 Hi,
 
 A recent NFS client crash:
   http://glebius.int.ru/tmp/nfs_panic.jpg
 appears to have happened because some field is
 bogus when crfree() is called. I've asked Gleb
 to disassemble crfree() for me, so I can try and
 see exactly which field causes the crash, however...
 
 Basically, the code:
newcred = crdup(cred);
- does read with newcred
crfree(newcred);  -- which crashes at 0x65 into
  crfree()
 
 Looking at crdup(), it calls crcopy(), which copies
 4 pointers and then ref. counts them:
   cr_uidinfo, cr_ruidinfo, cr_prison and cr_loginclass
 
 It seems some lock should be held while crcopy() does this,
 so that the pointers don't get deref'd during the copy/ref. count?
 (Or is there some rule that guarantees these won't change. ie. No
  no calls to change_euid() or similar.)
 
 Is there such a lock and should crdup() use it?

In general the caller of crdup is expected to hold a reference on cred or some 
other lock to ensure that cred remains valid and cannot be free'd while it is 
being duplicated.  There is no global lock that crdup can hold for that, 
instead the caller is required to guarantee that.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: svn commit: r228576 - in head: . sys/boot/forth sys/modules sys/modules/carp sys/modules/if_carp

2011-12-20 Thread John Baldwin
On Sunday, December 18, 2011 11:41:54 am Gleb Smirnoff wrote:
   Alexander,
 
 On Sat, Dec 17, 2011 at 03:08:43PM +0100, Alexander Leidinger wrote:
 A we never had a kernel part in the list. Reinstallkernel is not a valid 
target after updating the sources. The renaming will only take effekt after 
updating. And we already hat issues because the list was too long.
 A Your entry for the carp module is completely out of question for this 
list. Please remove it.
 
 The file /boot/kernel/if_carp.ko had been installed on older installations. 
It
 is not overwritten now. Thus, it may happen in a some weird case, that it is
 left intact. 'make installkernel' is not the only way to upgrade FreeBSD.
 To cover these potential cases I have added an entry. This entry doesn't
 hurt anybody or anything.

'make installkernel' is the only supported way of upgrading. :)

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Sleeping thread (tid 100033, pid 16): panic in FreeBSD 10.0-CURRENT/amd64 r228662

2011-12-20 Thread Attilio Rao
2011/12/20 John Baldwin j...@freebsd.org:
 On Saturday, December 17, 2011 10:41:15 pm m...@freebsd.org wrote:
 On Sat, Dec 17, 2011 at 5:45 PM, Alexander Kabaev kab...@gmail.com wrote:
  On Sun, 18 Dec 2011 01:09:00 +0100
  O. Hartmann ohart...@zedat.fu-berlin.de wrote:
 
  Sleeping thread (tid 100033, pid 16) owns a non sleepable lock
  panic: sleeping thread
  cpuid = 0
 
  PID 16 is always USB on my box.
 
  You really need to give us a backtrace when you quote panics. It is
  impossible to make any sense of the above panic message without more
  context.

 In the case of this panic, the stack of the thread which panics is
 useless; it's someone trying to propagate priority that discovered it.
  A backtrace on tid 100033 would be useful.

 With WITNESS enabled, it's possible to have this panic display the
 stack of the incorrectly sleeping thread at the time it acquired the
 lock, as well, but this code isn't in CURRENT or any release.  I have
 a patch at $WORK I can dig up on Monday.

 Huh?  The stock kernel dumps a stack trace of the offending thread if you have
 DDB enabled:

                /*
                 * If the thread is asleep, then we are probably about
                 * to deadlock.  To make debugging this easier, just
                 * panic and tell the user which thread misbehaved so
                 * they can hopefully get a stack trace from the truly
                 * misbehaving thread.
                 */
                if (TD_IS_SLEEPING(td)) {
                        printf(
                Sleeping thread (tid %d, pid %d) owns a non-sleepable lock\n,
                            td-td_tid, td-td_proc-p_pid);
 #ifdef DDB
                        db_trace_thread(td, -1);
 #endif
                        panic(sleeping thread);
                }

 It may be that we can make use of the STACK API here instead to output this
 trace even when DDB isn't enabled.  The patch below tries to do that
 (untested).  It does some odd thigns though since it is effectively running
 from a panic context already, so it uses a statically allocated 'struct stack'
 rather than using stack_create() and uses stack_print_ddb() since it is
 holding spin locks and can't possibly grab an sx lock:

 Index: subr_turnstile.c
 ===
 --- subr_turnstile.c    (revision 228534)
 +++ subr_turnstile.c    (working copy)
 @@ -72,6 +72,7 @@ __FBSDID($FreeBSD$);
  #include sys/proc.h
  #include sys/queue.h
  #include sys/sched.h
 +#include sys/stack.h
  #include sys/sysctl.h
  #include sys/turnstile.h

 @@ -175,6 +176,7 @@ static void turnstile_fini(void *mem, int size);
  static void
  propagate_priority(struct thread *td)
  {
 +       static struct stack st;
        struct turnstile *ts;
        int pri;

 @@ -217,8 +219,10 @@ propagate_priority(struct thread *td)
                        printf(
                Sleeping thread (tid %d, pid %d) owns a non-sleepable lock\n,
                            td-td_tid, td-td_proc-p_pid);
 -#ifdef DDB
 -                       db_trace_thread(td, -1);
 +#ifdef STACK
 +                       stack_zero(st);
 +                       stack_save_td(st, td);
 +                       stack_print_ddb(st);
  #endif
                        panic(sleeping thread);
                }

 --

I'm not sure it is a wise idea to trimm the DDB part, because it is
much more common than STACK enabled. Note that stack(9) is working if
you define DDB too, so I'd say to do that for both.
Also, I don't think you need the stack_zero() prior to set it.

As we are here, however, I have a question for Robert here: do you
think we should support the _ddb() variant of options even in the case
DDB is not enabled in the kernel?
Probabilly the way it is nowadays is easier to have stack(9) both
defined for DDB and STACK, but in general I wouldn't advertise that.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Sleeping thread (tid 100033, pid 16): panic in FreeBSD 10.0-CURRENT/amd64 r228662

2011-12-20 Thread mdf
On Tue, Dec 20, 2011 at 5:52 AM, John Baldwin j...@freebsd.org wrote:
 On Saturday, December 17, 2011 10:41:15 pm m...@freebsd.org wrote:
 On Sat, Dec 17, 2011 at 5:45 PM, Alexander Kabaev kab...@gmail.com wrote:
  On Sun, 18 Dec 2011 01:09:00 +0100
  O. Hartmann ohart...@zedat.fu-berlin.de wrote:
 
  Sleeping thread (tid 100033, pid 16) owns a non sleepable lock
  panic: sleeping thread
  cpuid = 0
 
  PID 16 is always USB on my box.
 
  You really need to give us a backtrace when you quote panics. It is
  impossible to make any sense of the above panic message without more
  context.

 In the case of this panic, the stack of the thread which panics is
 useless; it's someone trying to propagate priority that discovered it.
  A backtrace on tid 100033 would be useful.

 With WITNESS enabled, it's possible to have this panic display the
 stack of the incorrectly sleeping thread at the time it acquired the
 lock, as well, but this code isn't in CURRENT or any release.  I have
 a patch at $WORK I can dig up on Monday.

 Huh?  The stock kernel dumps a stack trace of the offending thread if you have
 DDB enabled:

                /*
                 * If the thread is asleep, then we are probably about
                 * to deadlock.  To make debugging this easier, just
                 * panic and tell the user which thread misbehaved so
                 * they can hopefully get a stack trace from the truly
                 * misbehaving thread.
                 */
                if (TD_IS_SLEEPING(td)) {
                        printf(
                Sleeping thread (tid %d, pid %d) owns a non-sleepable lock\n,
                            td-td_tid, td-td_proc-p_pid);
 #ifdef DDB
                        db_trace_thread(td, -1);
 #endif
                        panic(sleeping thread);
                }

Hmm, maybe this wasn't in 7, or maybe I'm just remembering that we
added code to print *which* lock it holds (using WITNESS data).  I do
recall that this panic alone was often not sufficient to debug the
problem.

Thanks,
matthew


 It may be that we can make use of the STACK API here instead to output this
 trace even when DDB isn't enabled.  The patch below tries to do that
 (untested).  It does some odd thigns though since it is effectively running
 from a panic context already, so it uses a statically allocated 'struct stack'
 rather than using stack_create() and uses stack_print_ddb() since it is
 holding spin locks and can't possibly grab an sx lock:

 Index: subr_turnstile.c
 ===
 --- subr_turnstile.c    (revision 228534)
 +++ subr_turnstile.c    (working copy)
 @@ -72,6 +72,7 @@ __FBSDID($FreeBSD$);
  #include sys/proc.h
  #include sys/queue.h
  #include sys/sched.h
 +#include sys/stack.h
  #include sys/sysctl.h
  #include sys/turnstile.h

 @@ -175,6 +176,7 @@ static void turnstile_fini(void *mem, int size);
  static void
  propagate_priority(struct thread *td)
  {
 +       static struct stack st;
        struct turnstile *ts;
        int pri;

 @@ -217,8 +219,10 @@ propagate_priority(struct thread *td)
                        printf(
                Sleeping thread (tid %d, pid %d) owns a non-sleepable lock\n,
                            td-td_tid, td-td_proc-p_pid);
 -#ifdef DDB
 -                       db_trace_thread(td, -1);
 +#ifdef STACK
 +                       stack_zero(st);
 +                       stack_save_td(st, td);
 +                       stack_print_ddb(st);
  #endif
                        panic(sleeping thread);
                }

 --
 John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: regression: Xorg get 100% cpu and freeze

2011-12-20 Thread Alexander Yerenkow
20 декабря 2011 г. 15:37 пользователь Alex Keda ad...@lissyara.su написал:

 I use CURRENT from 2011-11-18 - all OK
 After update to today - I have problem - on start, Xorg get 100% CPU and
 freeze (monitor go to turn off)
 I recompile Xorg, all modules - no happy.
 I try update x11-drivers/xf86-video-ati to 6.14.3 - no happy

 If I delete /boot/kernel/drm.ko - all work OK, but very slow...


BTW, mine test images built from sources with KMS do the same thing - start
xorg, and it just totally hangs pc.
I think it's about a month I have such situation.


 =
 FreeBSD lissyara.moskb.local 10.0-CURRENT FreeBSD 10.0-CURRENT #0 r228726:
 Tue Dec 20 08:24:56 MSK 2011 
 root@lissyara.moskb.local:/usr/obj/usr/src/sys/GENERIC
  amd64
 =
 pciconf, Xorg.log with and without drm.ko - in attached files

 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org




-- 
Regards,
Alexander Yerenkow
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: 9.0-RC1 panic in tcp_input: negative winow.

2011-12-20 Thread John Baldwin
On Saturday, December 17, 2011 6:21:27 pm Pawel Jakub Dawidek wrote:
 On Mon, Dec 12, 2011 at 11:00:23AM -0500, John Baldwin wrote:
  An update.  I've sent Pawel a testing patch to see if my hypothesis is 
  correct
  (www.freebsd.org/~jhb/patches/tcp_negwin_test.patch).  If it is then I 
  intend
  to commit www.freebsd.org/~jhb/patches/tcp_negwin2.patch as the fix.
 
 Unfortunately it paniced today. Take a look at:
 
   http://people.freebsd.org/~pjd/misc/tcp_panic.jpg

Ok, the one use case I was worried about is happening regularly before your
panic, so that is good.  Can you use gdb to figure out which call to
tcp_output() is actually panic'ing?  I wonder if it is this case:

/*
 * Return any desired output.
 */
if (needoutput || (tp-t_flags  TF_ACKNOW)) {
(void) tcp_output(tp);
/* XXX: Debug */
KASSERT(SEQ_GEQ(tp-rcv_adv, tp-rcv_nxt),
(tcp_input: negative window after ACK));

And if 'needoutput' is true, but TF_ACKNOW is not set, and tcp_output() decides
to not do anything.  I've updated tcp_negwin_test.patch to not panic if that 
call
to tcp_output() doesn't actually send a packet.  Please re-test.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Sleeping thread (tid 100033, pid 16): panic in FreeBSD 10.0-CURRENT/amd64 r228662

2011-12-20 Thread John Baldwin
On Tuesday, December 20, 2011 9:22:48 am m...@freebsd.org wrote:
 On Tue, Dec 20, 2011 at 5:52 AM, John Baldwin j...@freebsd.org wrote:
  On Saturday, December 17, 2011 10:41:15 pm m...@freebsd.org wrote:
  On Sat, Dec 17, 2011 at 5:45 PM, Alexander Kabaev kab...@gmail.com wrote:
   On Sun, 18 Dec 2011 01:09:00 +0100
   O. Hartmann ohart...@zedat.fu-berlin.de wrote:
  
   Sleeping thread (tid 100033, pid 16) owns a non sleepable lock
   panic: sleeping thread
   cpuid = 0
  
   PID 16 is always USB on my box.
  
   You really need to give us a backtrace when you quote panics. It is
   impossible to make any sense of the above panic message without more
   context.
 
  In the case of this panic, the stack of the thread which panics is
  useless; it's someone trying to propagate priority that discovered it.
   A backtrace on tid 100033 would be useful.
 
  With WITNESS enabled, it's possible to have this panic display the
  stack of the incorrectly sleeping thread at the time it acquired the
  lock, as well, but this code isn't in CURRENT or any release.  I have
  a patch at $WORK I can dig up on Monday.
 
  Huh?  The stock kernel dumps a stack trace of the offending thread if you 
  have
  DDB enabled:
 
 /*
  * If the thread is asleep, then we are probably about
  * to deadlock.  To make debugging this easier, just
  * panic and tell the user which thread misbehaved so
  * they can hopefully get a stack trace from the truly
  * misbehaving thread.
  */
 if (TD_IS_SLEEPING(td)) {
 printf(
 Sleeping thread (tid %d, pid %d) owns a non-sleepable 
  lock\n,
 td-td_tid, td-td_proc-p_pid);
  #ifdef DDB
 db_trace_thread(td, -1);
  #endif
 panic(sleeping thread);
 }
 
 Hmm, maybe this wasn't in 7, or maybe I'm just remembering that we
 added code to print *which* lock it holds (using WITNESS data).  I do
 recall that this panic alone was often not sufficient to debug the
 problem.

I think the db_trace_thread() has been around for a while (since 5 or 6),
but it is true that we don't tell you which lock is held even with this.
That might be a useful thing to output before the panic.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Sleeping thread (tid 100033, pid 16): panic in FreeBSD 10.0-CURRENT/amd64 r228662

2011-12-20 Thread John Baldwin
On Tuesday, December 20, 2011 9:20:09 am Attilio Rao wrote:
 2011/12/20 John Baldwin j...@freebsd.org:
  On Saturday, December 17, 2011 10:41:15 pm m...@freebsd.org wrote:
  On Sat, Dec 17, 2011 at 5:45 PM, Alexander Kabaev kab...@gmail.com wrote:
   On Sun, 18 Dec 2011 01:09:00 +0100
   O. Hartmann ohart...@zedat.fu-berlin.de wrote:
  
   Sleeping thread (tid 100033, pid 16) owns a non sleepable lock
   panic: sleeping thread
   cpuid = 0
  
   PID 16 is always USB on my box.
  
   You really need to give us a backtrace when you quote panics. It is
   impossible to make any sense of the above panic message without more
   context.
 
  In the case of this panic, the stack of the thread which panics is
  useless; it's someone trying to propagate priority that discovered it.
   A backtrace on tid 100033 would be useful.
 
  With WITNESS enabled, it's possible to have this panic display the
  stack of the incorrectly sleeping thread at the time it acquired the
  lock, as well, but this code isn't in CURRENT or any release.  I have
  a patch at $WORK I can dig up on Monday.
 
  Huh?  The stock kernel dumps a stack trace of the offending thread if you 
  have
  DDB enabled:
 
 /*
  * If the thread is asleep, then we are probably about
  * to deadlock.  To make debugging this easier, just
  * panic and tell the user which thread misbehaved so
  * they can hopefully get a stack trace from the truly
  * misbehaving thread.
  */
 if (TD_IS_SLEEPING(td)) {
 printf(
 Sleeping thread (tid %d, pid %d) owns a non-sleepable 
  lock\n,
 td-td_tid, td-td_proc-p_pid);
  #ifdef DDB
 db_trace_thread(td, -1);
  #endif
 panic(sleeping thread);
 }
 
  It may be that we can make use of the STACK API here instead to output this
  trace even when DDB isn't enabled.  The patch below tries to do that
  (untested).  It does some odd thigns though since it is effectively running
  from a panic context already, so it uses a statically allocated 'struct 
  stack'
  rather than using stack_create() and uses stack_print_ddb() since it is
  holding spin locks and can't possibly grab an sx lock:
 
  Index: subr_turnstile.c
  ===
  --- subr_turnstile.c(revision 228534)
  +++ subr_turnstile.c(working copy)
  @@ -72,6 +72,7 @@ __FBSDID($FreeBSD$);
   #include sys/proc.h
   #include sys/queue.h
   #include sys/sched.h
  +#include sys/stack.h
   #include sys/sysctl.h
   #include sys/turnstile.h
 
  @@ -175,6 +176,7 @@ static void turnstile_fini(void *mem, int size);
   static void
   propagate_priority(struct thread *td)
   {
  +   static struct stack st;
 struct turnstile *ts;
 int pri;
 
  @@ -217,8 +219,10 @@ propagate_priority(struct thread *td)
 printf(
 Sleeping thread (tid %d, pid %d) owns a non-sleepable 
  lock\n,
 td-td_tid, td-td_proc-p_pid);
  -#ifdef DDB
  -   db_trace_thread(td, -1);
  +#ifdef STACK
  +   stack_zero(st);
  +   stack_save_td(st, td);
  +   stack_print_ddb(st);
   #endif
 panic(sleeping thread);
 }
 
  --
 
 I'm not sure it is a wise idea to trimm the DDB part, because it is
 much more common than STACK enabled. Note that stack(9) is working if
 you define DDB too, so I'd say to do that for both.
 Also, I don't think you need the stack_zero() prior to set it.

Err, STACK is enabled in GENERIC in release kernels but DDB is not, so I think
STACK is the more common one.  As far as stack_zero(), I was just being 
paranoid.

 As we are here, however, I have a question for Robert here: do you
 think we should support the _ddb() variant of options even in the case
 DDB is not enabled in the kernel?
 Probabilly the way it is nowadays is easier to have stack(9) both
 defined for DDB and STACK, but in general I wouldn't advertise that.

The _ddb variants are always enabled by my reading.  They just use different
entry points into the linker that don't use locking.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Is the svn2cvs gateway down ?

2011-12-20 Thread Bjoern A. Zeeb
On 20. Dec 2011, at 10:01 , Claude Buisson wrote:

 It seems (from my own csup's and cvswe.cgi) that the src commits are lost,
 starting with r228697 Sun Dec 18 22:04:55 2011)
 
 What is going on (or off) ?

Re $subject -- yes.  It will be worked on.

-- 
Bjoern A. Zeeb You have to have visions!
 Stop bit received. Insert coin for new address family.

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ukbd locking update

2011-12-20 Thread Andriy Gapon

And the new patch:
http://people.freebsd.org/~avg/usb.new.diff

-- 
Andriy Gapon
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Sleeping thread (tid 100033, pid 16): panic in FreeBSD 10.0-CURRENT/amd64 r228662

2011-12-20 Thread Attilio Rao
2011/12/20 John Baldwin j...@freebsd.org:
 On Tuesday, December 20, 2011 9:20:09 am Attilio Rao wrote:
 2011/12/20 John Baldwin j...@freebsd.org:
  On Saturday, December 17, 2011 10:41:15 pm m...@freebsd.org wrote:
  On Sat, Dec 17, 2011 at 5:45 PM, Alexander Kabaev kab...@gmail.com 
  wrote:
   On Sun, 18 Dec 2011 01:09:00 +0100
   O. Hartmann ohart...@zedat.fu-berlin.de wrote:
  
   Sleeping thread (tid 100033, pid 16) owns a non sleepable lock
   panic: sleeping thread
   cpuid = 0
  
   PID 16 is always USB on my box.
  
   You really need to give us a backtrace when you quote panics. It is
   impossible to make any sense of the above panic message without more
   context.
 
  In the case of this panic, the stack of the thread which panics is
  useless; it's someone trying to propagate priority that discovered it.
   A backtrace on tid 100033 would be useful.
 
  With WITNESS enabled, it's possible to have this panic display the
  stack of the incorrectly sleeping thread at the time it acquired the
  lock, as well, but this code isn't in CURRENT or any release.  I have
  a patch at $WORK I can dig up on Monday.
 
  Huh?  The stock kernel dumps a stack trace of the offending thread if you 
  have
  DDB enabled:
 
                 /*
                  * If the thread is asleep, then we are probably about
                  * to deadlock.  To make debugging this easier, just
                  * panic and tell the user which thread misbehaved so
                  * they can hopefully get a stack trace from the truly
                  * misbehaving thread.
                  */
                 if (TD_IS_SLEEPING(td)) {
                         printf(
                 Sleeping thread (tid %d, pid %d) owns a non-sleepable 
  lock\n,
                             td-td_tid, td-td_proc-p_pid);
  #ifdef DDB
                         db_trace_thread(td, -1);
  #endif
                         panic(sleeping thread);
                 }
 
  It may be that we can make use of the STACK API here instead to output this
  trace even when DDB isn't enabled.  The patch below tries to do that
  (untested).  It does some odd thigns though since it is effectively running
  from a panic context already, so it uses a statically allocated 'struct 
  stack'
  rather than using stack_create() and uses stack_print_ddb() since it is
  holding spin locks and can't possibly grab an sx lock:
 
  Index: subr_turnstile.c
  ===
  --- subr_turnstile.c    (revision 228534)
  +++ subr_turnstile.c    (working copy)
  @@ -72,6 +72,7 @@ __FBSDID($FreeBSD$);
   #include sys/proc.h
   #include sys/queue.h
   #include sys/sched.h
  +#include sys/stack.h
   #include sys/sysctl.h
   #include sys/turnstile.h
 
  @@ -175,6 +176,7 @@ static void turnstile_fini(void *mem, int size);
   static void
   propagate_priority(struct thread *td)
   {
  +       static struct stack st;
         struct turnstile *ts;
         int pri;
 
  @@ -217,8 +219,10 @@ propagate_priority(struct thread *td)
                         printf(
                 Sleeping thread (tid %d, pid %d) owns a non-sleepable 
  lock\n,
                             td-td_tid, td-td_proc-p_pid);
  -#ifdef DDB
  -                       db_trace_thread(td, -1);
  +#ifdef STACK
  +                       stack_zero(st);
  +                       stack_save_td(st, td);
  +                       stack_print_ddb(st);
   #endif
                         panic(sleeping thread);
                 }
 
  --

 I'm not sure it is a wise idea to trimm the DDB part, because it is
 much more common than STACK enabled. Note that stack(9) is working if
 you define DDB too, so I'd say to do that for both.
 Also, I don't think you need the stack_zero() prior to set it.

 Err, STACK is enabled in GENERIC in release kernels but DDB is not, so I think
 STACK is the more common one.  As far as stack_zero(), I was just being 
 paranoid.

And what is the point for not having
#ifdef STACK
as
#if defined(STACK) || defined(DDB)

?

 As we are here, however, I have a question for Robert here: do you
 think we should support the _ddb() variant of options even in the case
 DDB is not enabled in the kernel?
 Probabilly the way it is nowadays is easier to have stack(9) both
 defined for DDB and STACK, but in general I wouldn't advertise that.

 The _ddb variants are always enabled by my reading.  They just use different
 entry points into the linker that don't use locking.

My question is different: why we define them anyway even when DDB is
not enabled?

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: making crdup()/crcopy() safe??

2011-12-20 Thread Rick Macklem
John Baldwin wrote:
 On Monday, December 19, 2011 8:21:45 pm Rick Macklem wrote:
  Hi,
 
  A recent NFS client crash:
http://glebius.int.ru/tmp/nfs_panic.jpg
  appears to have happened because some field is
  bogus when crfree() is called. I've asked Gleb
  to disassemble crfree() for me, so I can try and
  see exactly which field causes the crash, however...
 
  Basically, the code:
 newcred = crdup(cred);
 - does read with newcred
 crfree(newcred); -- which crashes at 0x65 into
   crfree()
 
  Looking at crdup(), it calls crcopy(), which copies
  4 pointers and then ref. counts them:
cr_uidinfo, cr_ruidinfo, cr_prison and cr_loginclass
 
  It seems some lock should be held while crcopy() does this,
  so that the pointers don't get deref'd during the copy/ref. count?
  (Or is there some rule that guarantees these won't change. ie. No
   no calls to change_euid() or similar.)
 
  Is there such a lock and should crdup() use it?
 
 In general the caller of crdup is expected to hold a reference on cred
 or some
 other lock to ensure that cred remains valid and cannot be free'd
 while it is
 being duplicated. There is no global lock that crdup can hold for
 that,
 instead the caller is required to guarantee that.
 
Well, I think it does hold a reference on cred. I think the problem is
that this doesn't stop another process from changing the pointer fields:
 cr_uidinfo, cr_ruidinfo, cr_loginclass and cr_prison

For example, change_euid() replaces cr_uidinfo. There is something called
cr_copysafe(), which assumes PROC_LOCK(p) is held. However, for the case
that crashed, it is an iod read-ahead thread, so I don't think I know
what the correct p argument is? It also appears that PROC_LOCK(p) is
used to lock change_euid(), when it replaces cr_uidinfo with a different
pointer. (Basically, it appears that the cr_uidinfo, cr_ruidinfo,
cr_loginclass and cr_prison are protected by PROC_LOCK(p), but it isn't
obvious to me that the NFS iod thread can know what the correct p is.
In fact, that process may have already exited, since the cred is refenenced
by b_rcred for the buffer in the buffer cache that is being read-ahead.

For my NFS client case, I need a new cred, but it has to have cr_uidinfo
etc filled in, since the kernel rpc does a crdup() and the cr_uidinfo
field is used in socket calls further down. Basically, the NFS client
fills in uid, gid-list for the new cred and doesn't care about other
fields, except whatever the kernel rpc and socket ops care about.

Would it be ok if, instead of using crdup(), I create the new cred via:
  cr = crget();
  - do the same as crcopy(), except for the pointer fields, which would be
set as follows
  cr-cr_uidinfo = uifind(0);  /* This means that chgsbsize() will record
* the change for uid 0, but since this is
* an iod thread for the NFS client, that
* seems ok?
*/
  cr-cr_ruidinfo = uifind(0);
  cr-cr_loginclass = loginclass_find(default);
  /* For cr_prison, I think what crcopy() does is safe, since cr_prison
   * doesn't normally get changed after a process does I/O, I think?
   * Alternately, it could be set to prison0. Does setting it to prison0
   * break anything?
   */
  prison_hold(cr-cr_prison);

crfree() does check for these fields being non-NULL before freeing them,
but crdup() does not check for the NULL case before incrementing ref cnts
on them. If crdup() were changed to check for non-NULL, then I think the
only one of the above that would need to be set is cr_uidinfo, since that
appears to be the only one used by socket ops.

Comments? Am I missing something? Thanks, rick

 --
 John Baldwin
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to
 freebsd-current-unsubscr...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: r228700 can't dhclient em0

2011-12-20 Thread Dimitry Andric

On 2011-12-20 09:38, Doug Barton wrote:
...

I tried replacing both ifconfig and dhclient with the versions that were
built along with the new kernel, and that didn't work.


Looking at the changes in r228571, it seems they also affect libc, so
installing that is probably also needed.  Since all my test systems are
already upgraded to post-r228571, can somebody please confirm it works
after doing:

make -C $srcdir/lib/libc install
make -C $srcdir/sbin/dhclient install
make -C $srcdir/sbin/ifconfig install



The traditional (and documented) upgrade process for many years has been
to boot the new kernel, make sure it's Ok, then update world. Obviously
something different is needed this time, so it needs an UPDATING entry
(assuming that all this is not just a bug).


Well, if there *is* any method of getting your network working before
installworld, it should be added to UPDATING... :)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Sleeping thread (tid 100033, pid 16): panic in FreeBSD 10.0-CURRENT/amd64 r228662

2011-12-20 Thread Robert Watson

On Tue, 20 Dec 2011, Attilio Rao wrote:

As we are here, however, I have a question for Robert here: do you think we 
should support the _ddb() variant of options even in the case DDB is not 
enabled in the kernel?


It's possible that _ddb() should be spelled _unlocked(), or perhaps _debug(), 
but neither really suggests what the name should actually imply: using it is 
safe only in a marginal (debugging) sense, and not in a production code sense. 
One might also reasonable call them stack_foo_dontusethis().


The _ddb() variants are used in at least two not strictly DDB cases: redzone 
support, and Solaris memory allocation.  And, I guess, the current lock 
debugging case that we're talking about now, but I'm not sure if those 
debugging features specifically require DDB in the kernel themselves?


Robert
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: cross-arch building picobsd/nanobsd images ?

2011-12-20 Thread Garrett Cooper
2011/12/20 Olivier Cochard-Labbé oliv...@cochard.me:
 On Mon, Dec 19, 2011 at 11:45 PM, Luigi Rizzo ri...@iet.unipi.it wrote:

 On a related topic, does anyone have experience on cross-building
 nanobsd images ?

Hello Mr. Olivier!

 I using little cross-building nanobsd images (i386 on amd64 and vice versa).
 All my patchs for nanobsd are available on BSD Router Project
 (http://bsdrp.net) including a patch for compiling ports from nanobsd
 too.

Yeah, FreeNAS 8.x employs a similar semi-hacky way of doing a
full-blown chroot with a clean environment setup [that you might want
to steal ;)..[1]]

 Right now I'm working on adding cross-build mips (RouterStation Pro)
 nanobsd patch but without the compiling ports feature, because I can
 only cross-compile word/kernel and I didn't know how to cross-compile
 ports.

Let's work together on this. It's a non-trivial project that I'd like
to see come true for FreeNAS to build an ARM platform on x86 hardware
(someday..).

Also, I'd pick up some of the recent changes we made to nanobsd [2] --
it might help your cause.

Cheers,
-Garrett

1. http://freenas.svn.sourceforge.net/viewvc/freenas/trunk/nanobsd/common
(look for the CR function; follow the history back for credits to the
original inspiration).
2. http://freenas.svn.sourceforge.net/viewvc/freenas/trunk/build/nanobsd/
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: regression: Xorg get 100% cpu and freeze

2011-12-20 Thread Garrett Cooper
On Dec 20, 2011, at 6:29 AM, Alexander Yerenkow wrote:

 20 декабря 2011 г. 15:37 пользователь Alex Keda ad...@lissyara.su написал:
 
 I use CURRENT from 2011-11-18 - all OK
 After update to today - I have problem - on start, Xorg get 100% CPU and
 freeze (monitor go to turn off)
 I recompile Xorg, all modules - no happy.
 I try update x11-drivers/xf86-video-ati to 6.14.3 - no happy
 
 If I delete /boot/kernel/drm.ko - all work OK, but very slow...
 
 
 BTW, mine test images built from sources with KMS do the same thing - start
 xorg, and it just totally hangs pc.
 I think it's about a month I have such situation.

+1 for my machine with the nvidia-driver. It ran out of swap over the 
weekend after the drive disconnected from the bus after 2 months or so of 
uptime. /var/log/messages for everyone might help isolate the issue... Previous 
version was r226140; new version is r227801.
Thanks,
-Garrett

Dec 19 10:28:12 streetfighter kernel: 
(ada0:g_vfs_done():ahcich0:0:0:ada0p2[READ(offset=836907008, length=4096)]0): 
error = 6
Dec 19 10:28:12 streetfighter kernel: lost device
Dec 19 10:28:12 streetfighter kernel: swap_pager: I/O error - pagein failed; 
blkno 17914,size 12288, error 6
Dec 19 10:28:12 streetfighter kernel: vnode_pager_getpages: I/O read error
Dec 19 10:28:12 streetfighter kernel: vm_fault: pager read error, pid 1291 
(syslogd)
Dec 19 10:28:12 streetfighter kernel: vm_fault: pager read error, pid 79074 
(smartd)
Dec 19 10:28:12 streetfighter kernel: g_vfs_done():ada0p2[READ(offset=131072, 
length=32768)]error = 6
Dec 19 10:28:12 streetfighter kernel: g_vfs_done():ada0p2[READ(offset=131072, 
length=32768)]error = 6
Dec 19 10:28:12 streetfighter kernel: 
g_vfs_done():ada0p2[WRITE(offset=797257728, length=4096)]error = 6
Dec 19 10:28:12 streetfighter kernel: /: got error 6 while accessing filesystem
Dec 19 10:28:12 streetfighter kernel: panic: softdep_deallocate_dependencies: 
unrecovered I/O error
Dec 19 10:28:12 streetfighter kernel: cpuid = 0
Dec 19 10:28:12 streetfighter kernel: KDB: enter: panic
Dec 19 10:28:12 streetfighter kernel:
Dec 19 10:28:12 streetfighter kernel:
Dec 19 10:28:12 streetfighter kernel: Fatal trap 3: breakpoint instruction 
fault while in kernel mode
Dec 19 10:28:12 streetfighter kernel: cpuid = 0; apic id = 00
Dec 19 10:28:12 streetfighter kernel: instruction pointer   = 
0x20:0x804b482b
Dec 19 10:28:12 streetfighter kernel: stack pointer = 
0x28:0xff80002759f0
Dec 19 10:28:12 streetfighter kernel: frame pointer = 
0x28:0xff8000275a10
Dec 19 10:28:12 streetfighter kernel: code segment  = base 0x0, 
limit 0xf, type 0x1b
Dec 19 10:28:12 streetfighter kernel: = DPL 0, pres 1, long 1, def32 0, gran 1
Dec 19 10:28:12 streetfighter kernel: processor eflags  = interrupt enabled, 
IOPL = 0
Dec 19 10:28:12 streetfighter kernel: current process   = 13 (g_up)
Dec 19 10:28:12 streetfighter kernel: trap number   = 3
Dec 19 10:28:12 streetfighter kernel: panic: breakpoint instruction fault
Dec 19 10:28:12 streetfighter kernel: cpuid = 0
Dec 19 10:28:12 streetfighter kernel: KDB: enter: 
panic___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Garrett Cooper
On Tue, Dec 20, 2011 at 5:46 AM, Vincent Hoffman vi...@unsane.co.uk wrote:
 On 20/12/2011 10:39, Daniel Kalchev wrote:


 On 20.12.11 11:42, Garrett Cooper wrote:
 As long as I have reliable checksums that match the what the upstream
 source says is the real thing, it doesn't practically matter where I
 get my images from.

 Relying on checksums that are published on the same web site where you
 download the files from and given that most of these sites do not even
 use SSL so much about 'security'.

 This does remind me of one issue that while a little off topic for this
 thread
 If i wanted to get, for example the SHA265 checksums from a verified
 source, how would i verify this currently? There doesnt seem to be an
 SSL site for www.freebsd.org and its not too hard to redirect someone to
 a fake website.
 What would be a more reasonable list to request this on?

And so the masses go off on a quest to answer how to obtain releases
instead of staying focused on the original problem at hand..
-Garrett
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Sleeping thread (tid 100033, pid 16): panic in FreeBSD 10.0-CURRENT/amd64 r228662

2011-12-20 Thread mdf
On Tue, Dec 20, 2011 at 6:32 AM, John Baldwin j...@freebsd.org wrote:
 On Tuesday, December 20, 2011 9:22:48 am m...@freebsd.org wrote:
 On Tue, Dec 20, 2011 at 5:52 AM, John Baldwin j...@freebsd.org wrote:
  On Saturday, December 17, 2011 10:41:15 pm m...@freebsd.org wrote:
  On Sat, Dec 17, 2011 at 5:45 PM, Alexander Kabaev kab...@gmail.com 
  wrote:
   On Sun, 18 Dec 2011 01:09:00 +0100
   O. Hartmann ohart...@zedat.fu-berlin.de wrote:
  
   Sleeping thread (tid 100033, pid 16) owns a non sleepable lock
   panic: sleeping thread
   cpuid = 0
  
   PID 16 is always USB on my box.
  
   You really need to give us a backtrace when you quote panics. It is
   impossible to make any sense of the above panic message without more
   context.
 
  In the case of this panic, the stack of the thread which panics is
  useless; it's someone trying to propagate priority that discovered it.
   A backtrace on tid 100033 would be useful.
 
  With WITNESS enabled, it's possible to have this panic display the
  stack of the incorrectly sleeping thread at the time it acquired the
  lock, as well, but this code isn't in CURRENT or any release.  I have
  a patch at $WORK I can dig up on Monday.
 
  Huh?  The stock kernel dumps a stack trace of the offending thread if you 
  have
  DDB enabled:
 
                 /*
                  * If the thread is asleep, then we are probably about
                  * to deadlock.  To make debugging this easier, just
                  * panic and tell the user which thread misbehaved so
                  * they can hopefully get a stack trace from the truly
                  * misbehaving thread.
                  */
                 if (TD_IS_SLEEPING(td)) {
                         printf(
                 Sleeping thread (tid %d, pid %d) owns a non-sleepable 
  lock\n,
                             td-td_tid, td-td_proc-p_pid);
  #ifdef DDB
                         db_trace_thread(td, -1);
  #endif
                         panic(sleeping thread);
                 }

 Hmm, maybe this wasn't in 7, or maybe I'm just remembering that we
 added code to print *which* lock it holds (using WITNESS data).  I do
 recall that this panic alone was often not sufficient to debug the
 problem.

 I think the db_trace_thread() has been around for a while (since 5 or 6),
 but it is true that we don't tell you which lock is held even with this.
 That might be a useful thing to output before the panic.


This patch isn't quite right since I had to hand-edit it.  There's a
small chance I can commit this in the near future, but of someone else
wants to take it, feel free.  Style isn't yet fixed up to be FreeBSD
standard either.


--- /data/sb/bsd.git/sys/kern/subr_turnstile.c  2011-12-12
10:23:12.542196632 -0800
+++ kern/subr_turnstile.c   2011-12-09 10:59:29.882643558 -0800
@@ -165,10 +165,43 @@
 static voidturnstile_dtor(void *mem, int size, void *arg);
 #endif
 static int turnstile_init(void *mem, int size, int flags);
 static voidturnstile_fini(void *mem, int size);

+#ifdef INVARIANTS
+static void
+sleeping_thread_owns_a_nonsleepable_lock(struct thread *td)
+{
+   printf(Sleeping thread (tid %d, pid %d) owns a non-sleepable lock\n,
+   td-td_tid, td-td_proc-p_pid);
+#ifdef DDB
+   db_trace_thread(td, -1);
+#endif
+#ifdef WITNESS
+   struct lock_list_entry *lock_list, *lle;
+   int i;
+
+   lock_list = td-td_sleeplocks;
+   if (lock_list == NULL || lock_list-ll_count == 0) {
+   printf(Thread does not appear to hold any mutexes!\n);
+   return;
+   }
+
+   for (lle = lock_list; lle != NULL; lle = lle-ll_next) {
+   for (i = lle-ll_count - 1; i = 0; i--) {
+   struct lock_instance *li = lle-ll_children[i];
+
+   printf(Lock %s acquired at %s:%d\n,
+   li-li_lock-lo_name, li-li_file, li-li_line);
+   }
+   }
+#endif /* WITNESS */
+}
+#else
+#define sleeping_thread_owns_a_nonsleepable_lock(td) do { } while (0)
+#endif /* INVARIANTS */
+
 /*
  * Walks the chain of turnstiles and their owners to propagate the priority
  * of the thread being blocked to all the threads holding locks that have to
  * release their locks before this thread can run again.
  */
@@ -210,19 +243,31 @@
 * If the thread is asleep, then we are probably about
 * to deadlock.  To make debugging this easier, just
 * panic and tell the user which thread misbehaved so
 * they can hopefully get a stack trace from the truly
 * misbehaving thread.
 */
if (TD_IS_SLEEPING(td)) {
-   printf(
-   Sleeping thread (tid %d, pid %d) owns a non-sleepable lock\n,
-   td-td_tid, td-td_proc-p_pid);
-#ifdef DDB
-   db_trace_thread(td, -1);
-#endif
-   panic(sleeping thread);
+   

Re: ukbd locking update

2011-12-20 Thread Hans Petter Selasky
On Tuesday 20 December 2011 16:16:36 Andriy Gapon wrote:
 And the new patch:
 http://people.freebsd.org/~avg/usb.new.diff

Patch looks OK. I will give it a spin later today. Feel free to commit if 
you've tested the three use-cases I listed in a previous e-mail.

--HPS
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: r228700 can't dhclient em0

2011-12-20 Thread Brooks Davis
On Tue, Dec 20, 2011 at 04:43:21PM +0100, Dimitry Andric wrote:
 On 2011-12-20 09:38, Doug Barton wrote:
 ...
  I tried replacing both ifconfig and dhclient with the versions that were
  built along with the new kernel, and that didn't work.
 
 Looking at the changes in r228571, it seems they also affect libc, so
 installing that is probably also needed.  Since all my test systems are
 already upgraded to post-r228571, can somebody please confirm it works
 after doing:
 
 make -C $srcdir/lib/libc install
 make -C $srcdir/sbin/dhclient install
 make -C $srcdir/sbin/ifconfig install

We almost certainly need to back r228571 out.  This is not an acceptable
upgrade path that would be acceptable historically.  Specially, we have
effectively promised users that an X.Y world will work on an X+1.0
kernel for most of history.  There are obvious exceptions to this, but
we have never allowed ifconfig to be one of them (I broke it many years
ago with my first attempt to add if_epoch to if_data and had to back
that out).

-- Brooks


pgpHMjvu8huYI.pgp
Description: PGP signature


Re: Uneven load on drives in ZFS RAIDZ1

2011-12-20 Thread Michael Reifenberger

On Mon, 19 Dec 2011, Peter Maloney wrote:
...

Thanks for the info. But I am confused by it, because when my disks
moved around randomly on reboot, it really did mess things up. The first
few times it happened, there was no issue, but when a spare took the
place of a pool disk, it messed things up. I can see the UUIDs when I
look at zdb output, so I really have no idea why it messed things up.
... but it did, so I will always caution people anyway. I can't point
you to any relevant lines of code that cause the problem, but I know it
can happen... and it will when you least expect it. ;)



Normaly spare disks are not part of the pool so it can't replace
a pools disk automatically (except the property 'autoreplace' is set to 'on').

But as allways no software is error free and your issue could be an uncaught
edge case of something.

Theoretically it could be an timing issue during boot too due to the async
GEOM/CAM nature...

Bye/2
---
Michael Reifenberger
mich...@reifenberger.com
http://www.Reifenberger.com

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


status of ports and clang

2011-12-20 Thread Mark Linimon
I have recently been able to get the new build cluster on pointyhat-west
set up to run full builds of ports with clang on amd64-9.  I have documented
the latest results on the wiki:

  http://wiki.freebsd.org/PortsAndClang

If you are interested in working on ports being built via clang, this
is your place to start.

Please also note that now that we have up-to-date builds going, it is
not as useful to us to report individual clang build failures.  Patches
to fix problems are, of course, highly welcome.

mcl
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: cross-arch building picobsd/nanobsd images ?

2011-12-20 Thread Michael Reifenberger

Hi,
I used to build a few ARM images (on amd64 host) using
/usr/src/tools/tools/nanobsd/gateworks/
and regularly i386 images (on amd64 host too) using
/usr/src/tools/tools/nanobsd/pcengines/

Bye/2
---
Michael Reifenberger
mich...@reifenberger.com
http://www.Reifenberger.com

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: making crdup()/crcopy() safe??

2011-12-20 Thread John Baldwin
On Tuesday, December 20, 2011 10:38:34 am Rick Macklem wrote:
 John Baldwin wrote:
  On Monday, December 19, 2011 8:21:45 pm Rick Macklem wrote:
   Hi,
  
   A recent NFS client crash:
 http://glebius.int.ru/tmp/nfs_panic.jpg
   appears to have happened because some field is
   bogus when crfree() is called. I've asked Gleb
   to disassemble crfree() for me, so I can try and
   see exactly which field causes the crash, however...
  
   Basically, the code:
  newcred = crdup(cred);
  - does read with newcred
  crfree(newcred); -- which crashes at 0x65 into
crfree()
  
   Looking at crdup(), it calls crcopy(), which copies
   4 pointers and then ref. counts them:
 cr_uidinfo, cr_ruidinfo, cr_prison and cr_loginclass
  
   It seems some lock should be held while crcopy() does this,
   so that the pointers don't get deref'd during the copy/ref. count?
   (Or is there some rule that guarantees these won't change. ie. No
no calls to change_euid() or similar.)
  
   Is there such a lock and should crdup() use it?
  
  In general the caller of crdup is expected to hold a reference on cred
  or some
  other lock to ensure that cred remains valid and cannot be free'd
  while it is
  being duplicated. There is no global lock that crdup can hold for
  that,
  instead the caller is required to guarantee that.
  
 Well, I think it does hold a reference on cred. I think the problem is
 that this doesn't stop another process from changing the pointer fields:
  cr_uidinfo, cr_ruidinfo, cr_loginclass and cr_prison

No, a credential with more than one reference is immutable and can not be 
changed.

 For example, change_euid() replaces cr_uidinfo. There is something called
 cr_copysafe(), which assumes PROC_LOCK(p) is held. However, for the case
 that crashed, it is an iod read-ahead thread, so I don't think I know
 what the correct p argument is? It also appears that PROC_LOCK(p) is
 used to lock change_euid(), when it replaces cr_uidinfo with a different
 pointer. (Basically, it appears that the cr_uidinfo, cr_ruidinfo,
 cr_loginclass and cr_prison are protected by PROC_LOCK(p), but it isn't
 obvious to me that the NFS iod thread can know what the correct p is.
 In fact, that process may have already exited, since the cred is 
refenenced
 by b_rcred for the buffer in the buffer cache that is being read-ahead.

No, change_euid() only operates on a private credential where the caller knows 
that it holds the only reference to the credential.  The various system calls 
like setuid(), etc. all allocate a new credential, grab the PROC_LOCK to 
protect what p_ucred refers to (and to serialize updates to p_ucred for a 
given process), copy the existing credential from p_ucred into the new 
credential, update the new credential as appropriate, then change p_ucred to 
point to the new credential before dropping the PROC_LOCK.  The old credential 
then has its reference count dropped (since p_ucred no longer references it) 
via crfree().  However, that old credential is not changed and will remain 
immutable until the last reference is dropped and it is freed.

 For my NFS client case, I need a new cred, but it has to have cr_uidinfo
 etc filled in, since the kernel rpc does a crdup() and the cr_uidinfo
 field is used in socket calls further down. Basically, the NFS client
 fills in uid, gid-list for the new cred and doesn't care about other
 fields, except whatever the kernel rpc and socket ops care about.
 
 Would it be ok if, instead of using crdup(), I create the new cred via:
   cr = crget();
   - do the same as crcopy(), except for the pointer fields, which would be
 set as follows
   cr-cr_uidinfo = uifind(0);  /* This means that chgsbsize() will record
 * the change for uid 0, but since this is
 * an iod thread for the NFS client, that
 * seems ok?
 */
   cr-cr_ruidinfo = uifind(0);
   cr-cr_loginclass = loginclass_find(default);
   /* For cr_prison, I think what crcopy() does is safe, since cr_prison
* doesn't normally get changed after a process does I/O, I think?
* Alternately, it could be set to prison0. Does setting it to prison0
* break anything?
*/
   prison_hold(cr-cr_prison);
 
 crfree() does check for these fields being non-NULL before freeing them,
 but crdup() does not check for the NULL case before incrementing ref cnts
 on them. If crdup() were changed to check for non-NULL, then I think the
 only one of the above that would need to be set is cr_uidinfo, since that
 appears to be the only one used by socket ops.
 
 Comments? Am I missing something? Thanks, rick

Using crdup() should be fine.  Your old credential should not be changed out 
from under you since you hold a reference to it.  I think there is some other
bug that trashed your temporary credential before it was free'd.

-- 
John Baldwin

clang (__builtin_ffs) vs /usr/include/string.h

2011-12-20 Thread Larry Rosenman
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Is anyone going to fix the following in the clang or FreeBSD system?

In file included from /usr/include/string.h:45:
/usr/include/strings.h:47:6: error: conflicting types for '__builtin_ffs'
int  ffs(int) __pure2;
 ^
/usr/include/machine/cpufunc.h:140:24: note: expanded from:
#defineffs(x)  __builtin_ffs(x)
   ^
/usr/include/strings.h:47:6: note: '__builtin_ffs' is a builtin with
type 'int (unsigned int)'
7 warnings and 1 error generated.
*** Error code 1


This looks like something that needs to change in clang/FreeBSD headers.

- -- 
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 512-248-2683 E-Mail: l...@lerctr.org
US Mail: 430 Valona Loop, Round Rock, TX 78681-3893
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJO8MzGAAoJENC8dtAvA1zmaJAH/iQFOp2253yJgCCz8vbFotyY
zV04olNDrLTYlKLZ8XNJ01qyiDnOXm/gL6TMZMvRiSNQg3e78ZYYPl2ufn0a1VNg
lnaQ7SO4qCT1o/66HJKr9YGyjZ12IYQwCkHnzGc+OURlQ/ZLYfOrOVvuE1tFsNUH
5PEuZ2ywOGWdBf9BlzN8PaBp6fJ+zBMvTOamy5jjYhD5pd3E2io/DwGUGJogsACa
3ETUrZdnoxjdfuC3VxEy0kSuW7iaODbbI1AMVAR3eWivjayiAo26HnuJ6rW7GSiN
Amn02uueCEQhl4d+Qvj4fDhMBg4PI+0blklheNJ9bjcxwtJDLk/KbevSU7SSCYg=
=Nvdn
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: r228700 can't dhclient em0

2011-12-20 Thread Bjoern A. Zeeb
On 20. Dec 2011, at 16:51 , Brooks Davis wrote:

 On Tue, Dec 20, 2011 at 04:43:21PM +0100, Dimitry Andric wrote:
 On 2011-12-20 09:38, Doug Barton wrote:
 ...
 I tried replacing both ifconfig and dhclient with the versions that were
 built along with the new kernel, and that didn't work.
 
 Looking at the changes in r228571, it seems they also affect libc, so
 installing that is probably also needed.  Since all my test systems are
 already upgraded to post-r228571, can somebody please confirm it works
 after doing:
 
 make -C $srcdir/lib/libc install
 make -C $srcdir/sbin/dhclient install
 make -C $srcdir/sbin/ifconfig install
 

Depends on what you are trying to achieve.  There is more software affected.

 We almost certainly need to back r228571 out.  This is not an acceptable
 upgrade path that would be acceptable historically.  Specially, we have
 effectively promised users that an X.Y world will work on an X+1.0

I am not sure we supported that but X.latest to X+1.Y maybe?

 kernel for most of history.  There are obvious exceptions to this, but
 we have never allowed ifconfig to be one of them (I broke it many years
 ago with my first attempt to add if_epoch to if_data and had to back
 that out).

There is a couple of issues here.

The one that's on me is struct ifa_msghdr / getifaddrs() and the problem here
is that software incl. getifaddrs() doesn't use the in structs embedded lengths
in first place.  In if_data only a spare was reused, so no changes there.
That's certainly something we can and should fix in 9 before 10.0 will happen.
If that was the problem back then, it's certainly a pity it wasn't fixed as
we wouldn't have that problem these days then.  It would allow appending more
stuff.

For the appended int to ifaliasreq, in_aliasreq, in6_aliasreq I haven't
looked at the actual problem in the upgrade paths.  I guess it's not much
outside of ifconfig and probably ppp at least for the in_ in6_ variants.

Backing r228571 out completely will be harder with every change glebius commits
on top.  So if we think it'll be needed we should stop those commits now and
think first.

Just breaking carp and backing out the user space API parts is only a usable
path with a plan B on how to fix carp as well in the same go really.

Ingoring 9.0 - 10-CURRENT or an update on HEAD (neither of which is not a
normal user upgrade path an supported) I am still wondering how much of that
can be mitigated and if it might make sense to ponder that direction (also
for further future changes for sure to happen) to not be stuck forever?
I fear we'll see/want to do more of these kinds as more people look at the
if/ll layer...

/bz

-- 
Bjoern A. Zeeb You have to have visions!
 Stop bit received. Insert coin for new address 
family.___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Samuel J. Greear
On Tue, Dec 20, 2011 at 4:28 AM, Chiron IO io.chi...@gmail.com wrote:

 Guys,

 I have a question about these benchmarks.

 Why worry about that if the CURRENT comes with debug enabled by default?


 http://joaobarros.blogspot.com/2005/07/freebsd-how-to-turn-off-debug-options.html



In the real world problems happen and someone has to be able to, in some
fashion, identify and resolve those problems. As such, shipping FreeBSD
releases with INVARIANTS disabled is a mistake and any benchmarks done
without INVARIANTS enabled will fail to reflect most reasonable real world
use-cases.

Although these benchmarks cannot stand on their own merits for many
reasons, I do not see how any benchmark is automatically invalidated by
using the default development configuration.

Sam
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: r228700 can't dhclient em0

2011-12-20 Thread Gleb Smirnoff
  Doug,

On Tue, Dec 20, 2011 at 12:38:53AM -0800, Doug Barton wrote:
D  I saw this too, when my kernel and userland were out of sync (e.g. just
D  after installing a new kernel, and before installworld).  I suspect it
D  is caused by the changes in r228571, which cause old ifconfig and
D  dhclient to not recognize any interfaces.  I'm not 100% sure though...
D 
D I tried replacing both ifconfig and dhclient with the versions that were
D built along with the new kernel, and that didn't work.

This shouldn't happen. If you did 'make buildworld buildkernel', then
your world in objdir would have binaries compiled with includes from
source tree, not from /usr/include, thus compatible with new kernel.

'make buildworld buildkernel' always produces compatible kernel and
worlds.

However, if you did 'cd /usr/src/sbin/ifconfig  make all install' then
that didn't work, since used headers from /usr/include.

D The traditional (and documented) upgrade process for many years has been
D to boot the new kernel, make sure it's Ok, then update world. Obviously
D something different is needed this time, so it needs an UPDATING entry
D (assuming that all this is not just a bug).

The documented one says 'Reboot into single user mode' and then install
new world. This path was not broken, since single user mode doesn't
imply network support.

The undocumented brave way 'make installkernel installworld  reboot'
works also, without any problems.

-- 
Totus tuus, Glebius.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: r228700 can't dhclient em0

2011-12-20 Thread Gleb Smirnoff
  Brooks,

On Tue, Dec 20, 2011 at 10:51:34AM -0600, Brooks Davis wrote:
B We almost certainly need to back r228571 out.  This is not an acceptable
B upgrade path that would be acceptable historically.  Specially, we have
B effectively promised users that an X.Y world will work on an X+1.0
B kernel for most of history.  There are obvious exceptions to this, but
B we have never allowed ifconfig to be one of them (I broke it many years
B ago with my first attempt to add if_epoch to if_data and had to back
B that out).

Pardon, where did we promise that? The applications in jail should work,
but not kernel configuration tools. The network facilities like ipfw
and pf has changed their ABI numerous times, making a new kernel
with older world inaccessible via network after boot.

Considering r228571: we need to specify vhid as additional address
attribute in atomic manner, via one ioctl(). Address can't be first
configured, and then made redundant, that would lead it to being
static for a short period, sending gratutious ARP announcement, etc.

An assumption that we are not allowed to change ABI for our own tools
strongly discourages bringing in new features :(

-- 
Totus tuus, Glebius.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: r228700 can't dhclient em0

2011-12-20 Thread Gleb Smirnoff
T An assumption that we are not allowed to change ABI for our own tools
T strongly discourages bringing in new features :(

P.S. Quoting documentation: The old world might not run correctly on the new 
kernel, so you must install the new world immediately upon installing the new 
kernel.

http://www.freebsd.org/doc/en/books/handbook/makeworld.html

-- 
Totus tuus, Glebius.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: making crdup()/crcopy() safe??

2011-12-20 Thread Gleb Smirnoff
  John,

On Tue, Dec 20, 2011 at 09:00:39AM -0500, John Baldwin wrote:
J In general the caller of crdup is expected to hold a reference on cred or 
some 
J other lock to ensure that cred remains valid and cannot be free'd while it 
is 
J being duplicated.  There is no global lock that crdup can hold for that, 
J instead the caller is required to guarantee that.

I already noticed to Rick in a private email, that there is suspisious
place in new NFS, where newly allocated (via crdup) cred is temporarily
placed on td_ucred, and then removed at the end of function. The function
may sleep in malloc() and also block on mutexes.

I suspect, that it may happen, that some other kernel facility performs
crfree(td-td_ucred), and later on we are using already freed cred.

However, I can't imagine which facility may do this. What if process gets
SIGKILL while its thread is blocked on mutex or sleeping? Would it
be reaped after it yields or before?

Attached patch demonstrates what I am trying to address, but I'm not sure
that this temporary placing on td_ucred is really unsafe. Can you please
look at this?

-- 
Totus tuus, Glebius.
Index: nfs_commonkrpc.c
===
--- nfs_commonkrpc.c	(revision 228700)
+++ nfs_commonkrpc.c	(working copy)
@@ -186,9 +186,9 @@
 	 * Use the credential in nr_cred, if not NULL.
 	 */
 	if (nrp-nr_cred != NULL)
-		td-td_ucred = nrp-nr_cred;
+		td-td_ucred = NFSHOLDCRED(nrp-nr_cred);
 	else
-		td-td_ucred = cred;
+		td-td_ucred = NFSHOLDCRED(cred);
 	saddr = nrp-nr_nam;
 
 	if (saddr-sa_family == AF_INET)
@@ -220,10 +220,8 @@
 	saddr = NFSSOCKADDR(nrp-nr_nam, struct sockaddr *);
 	error = socreate(saddr-sa_family, so, nrp-nr_sotype, 
 	nrp-nr_soproto, td-td_ucred, td);
-	if (error) {
-		td-td_ucred = origcred;
+	if (error)
 		goto out;
-	}
 	do {
 	if (error != 0  pktscale  2)
 		pktscale--;
@@ -251,10 +249,8 @@
 	error = soreserve(so, sndreserve, rcvreserve);
 	} while (error != 0  pktscale  2);
 	soclose(so);
-	if (error) {
-		td-td_ucred = origcred;
+	if (error)
 		goto out;
-	}
 
 	client = clnt_reconnect_create(nconf, saddr, nrp-nr_prog,
 	nrp-nr_vers, sndreserve, rcvreserve);
@@ -305,10 +301,11 @@
 		mtx_unlock(nrp-nr_mtx);
 	}
 
+out:
 	/* Restore current thread's credentials. */
+	NFSFREECRED(td-td_ucred);
 	td-td_ucred = origcred;
 
-out:
 	NFSEXITCODE(error);
 	return (error);
 }
Index: nfsport.h
===
--- nfsport.h	(revision 228700)
+++ nfsport.h	(working copy)
@@ -515,6 +515,7 @@
 #define	NFSNEWCRED(c)		(crdup(c))
 #define	NFSPROCCRED(p)		((p)-td_ucred)
 #define	NFSFREECRED(c)		(crfree(c))
+#define	NFSHOLDCRED(c)		(crhold(c))
 #define	NFSUIOPROC(u, p)	((u)-uio_td = NULL)
 #define	NFSPROCP(p)		((p)-td_proc)
 
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: r228700 can't dhclient em0

2011-12-20 Thread Brooks Davis
On Tue, Dec 20, 2011 at 06:48:40PM +, Bjoern A. Zeeb wrote:
 On 20. Dec 2011, at 16:51 , Brooks Davis wrote:
 
  On Tue, Dec 20, 2011 at 04:43:21PM +0100, Dimitry Andric wrote:
  On 2011-12-20 09:38, Doug Barton wrote:
  ...
  I tried replacing both ifconfig and dhclient with the versions that were
  built along with the new kernel, and that didn't work.
  
  Looking at the changes in r228571, it seems they also affect libc, so
  installing that is probably also needed.  Since all my test systems are
  already upgraded to post-r228571, can somebody please confirm it works
  after doing:
  
  make -C $srcdir/lib/libc install
  make -C $srcdir/sbin/dhclient install
  make -C $srcdir/sbin/ifconfig install
  
 
 Depends on what you are trying to achieve.  There is more software affected.
 
  We almost certainly need to back r228571 out.  This is not an acceptable
  upgrade path that would be acceptable historically.  Specially, we have
  effectively promised users that an X.Y world will work on an X+1.0
 
 I am not sure we supported that but X.latest to X+1.Y maybe?

That's the guarantee we've advertised, but the breakage we've supported
has mostly been for more narrowly focused tools that grovel around in
KVM space.  We've usually allowed an old world to work with only a few
shims (ps for instance).  Adding a new ifconfig to the list would be an
expansion there.

I also worry about the problem that once you've installed world there is
no easy way back.

  kernel for most of history.  There are obvious exceptions to this, but
  we have never allowed ifconfig to be one of them (I broke it many years
  ago with my first attempt to add if_epoch to if_data and had to back
  that out).
 
 There is a couple of issues here.
 
 The one that's on me is struct ifa_msghdr / getifaddrs() and the problem here
 is that software incl. getifaddrs() doesn't use the in structs embedded 
 lengths
 in first place.  In if_data only a spare was reused, so no changes there.
 That's certainly something we can and should fix in 9 before 10.0 will happen.
 If that was the problem back then, it's certainly a pity it wasn't fixed as
 we wouldn't have that problem these days then.  It would allow appending more
 stuff.

It would nice to fix, but allowing append to if_data means reving the
routing socket API.  I looked at it a bit and then gave up because there
were too many consumers (many out of tree), pretty much all of which
ignore the version number.  I didn't have time to rev the whole API
which is probably where the bar is for breaking the API.

 For the appended int to ifaliasreq, in_aliasreq, in6_aliasreq I haven't
 looked at the actual problem in the upgrade paths.  I guess it's not much
 outside of ifconfig and probably ppp at least for the in_ in6_ variants.

Those many have few enough consumers that we can come up with a
reasonable path forward.

 Backing r228571 out completely will be harder with every change glebius 
 commits
 on top.  So if we think it'll be needed we should stop those commits now and
 think first.
 
 Just breaking carp and backing out the user space API parts is only a usable
 path with a plan B on how to fix carp as well in the same go really.
 
 Ingoring 9.0 - 10-CURRENT or an update on HEAD (neither of which is not a
 normal user upgrade path an supported) I am still wondering how much of that
 can be mitigated and if it might make sense to ponder that direction (also
 for further future changes for sure to happen) to not be stuck forever?
 I fear we'll see/want to do more of these kinds as more people look at the
 if/ll layer...

If we can identify all the changed interfaces and we can teach 9-STABLE
to deal with them then this change is probably OK and we just need to
do the appropriate libc and ifconfig spackling and merge it.  We can't
let this slide too long though because the current state requires an
installkernel+installworld for all intents and purposes and that means
that if the kernel is broken you are dead in the water.

I agree we could really use some sort of way forward that increases our
abilty to make these sorts of changes.

- Brooks

 /bz
 
 -- 
 Bjoern A. Zeeb You have to have visions!
  Stop bit received. Insert coin for new address family.
 


pgp3SB4aWm9NV.pgp
Description: PGP signature


Re: r228700 can't dhclient em0

2011-12-20 Thread Gleb Smirnoff
On Tue, Dec 20, 2011 at 01:30:34PM -0600, Brooks Davis wrote:
B I also worry about the problem that once you've installed world there is
B no easy way back.

When working on the patch for last months I did numerous 'make installkernel
installworld' switching between a patched sources and unpatched, and everything
goes fine.

B If we can identify all the changed interfaces and we can teach 9-STABLE
B to deal with them then this change is probably OK and we just need to
B do the appropriate libc and ifconfig spackling and merge it.  We can't
B let this slide too long though because the current state requires an
B installkernel+installworld for all intents and purposes and that means
B that if the kernel is broken you are dead in the water.
B 
B I agree we could really use some sort of way forward that increases our
B abilty to make these sorts of changes.

Why 9-STABLE? This change isn't going to be merged.

-- 
Totus tuus, Glebius.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Chiron IO
http://wiki.freebsd.org/DefaultDebuggingKnobs

I am not aware of any linux distribution that comes with debug enabled by 
default, even on RC releases.

It seems that this approach (debug by default) is welcome to help solve 
problems that might appear,  but I would be happy if these benchmarks were made 
without such config (debug) enabled.

On 20/12/2011, at 17:09, Samuel J. Greear wrote:

 On Tue, Dec 20, 2011 at 4:28 AM, Chiron IO io.chi...@gmail.com wrote:
 Guys,
 
 I have a question about these benchmarks.
 
 Why worry about that if the CURRENT comes with debug enabled by default?
 
 http://joaobarros.blogspot.com/2005/07/freebsd-how-to-turn-off-debug-options.html
 
 
 
 In the real world problems happen and someone has to be able to, in some 
 fashion, identify and resolve those problems. As such, shipping FreeBSD 
 releases with INVARIANTS disabled is a mistake and any benchmarks done 
 without INVARIANTS enabled will fail to reflect most reasonable real world 
 use-cases.
 
 Although these benchmarks cannot stand on their own merits for many reasons, 
 I do not see how any benchmark is automatically invalidated by using the 
 default development configuration.
 
 Sam

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: r228700 can't dhclient em0

2011-12-20 Thread Bjoern A. Zeeb

On 20. Dec 2011, at 19:35 , Gleb Smirnoff wrote:

 On Tue, Dec 20, 2011 at 01:30:34PM -0600, Brooks Davis wrote:
 B I also worry about the problem that once you've installed world there is
 B no easy way back.
 
 When working on the patch for last months I did numerous 'make installkernel
 installworld' switching between a patched sources and unpatched, and 
 everything
 goes fine.

Yeah for the time being but we are talking about a 4-5 year timeframe.


 B If we can identify all the changed interfaces and we can teach 9-STABLE
 B to deal with them then this change is probably OK and we just need to
 B do the appropriate libc and ifconfig spackling and merge it.  We can't
 B let this slide too long though because the current state requires an
 B installkernel+installworld for all intents and purposes and that means
 B that if the kernel is broken you are dead in the water.
 B 
 B I agree we could really use some sort of way forward that increases our
 B abilty to make these sorts of changes.
 
 Why 9-STABLE? This change isn't going to be merged.

Because we'd need to make sure that fixes go backward to work with the
new stuff for the update path - the fixes, not your change.

/bz

-- 
Bjoern A. Zeeb You have to have visions!
 Stop bit received. Insert coin for new address family.

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ukbd locking update

2011-12-20 Thread Andriy Gapon
on 20/12/2011 18:41 Hans Petter Selasky said the following:
 On Tuesday 20 December 2011 16:16:36 Andriy Gapon wrote:
 And the new patch:
 http://people.freebsd.org/~avg/usb.new.diff
 
 Patch looks OK. I will give it a spin later today. Feel free to commit if 
 you've tested the three use-cases I listed in a previous e-mail.

Thank you for the quick review.
My tests were all successful, including those 3 scenarios.
Please let me know if you have any questions and how your testing goes.
I will try to commit this tomorrow morning.

-- 
Andriy Gapon
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Igor Mozolevsky
Interestingly, while people seem to be (arguably rightly) focused on
criticising Phoronix's benchmarking, nobody has offered an alternative
benchmark; and while (again, arguably rightly) it is important to
benchmark real world performance, equally, nobody has offered any
numbers in relation to, for example, HTTP or SMTP, or any other real
world-application torture tests done on the aforementioned two
platforms... IMO, this just goes to show that doing is hard and
criticising is much easier (yes, I am aware of the irony involved in
making this statement, but someone has to!)


Cheers,
Igor M :-)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: making crdup()/crcopy() safe??

2011-12-20 Thread John Baldwin
On Tuesday, December 20, 2011 2:31:40 pm Gleb Smirnoff wrote:
   John,
 
 On Tue, Dec 20, 2011 at 09:00:39AM -0500, John Baldwin wrote:
 J In general the caller of crdup is expected to hold a reference on cred or 
some 
 J other lock to ensure that cred remains valid and cannot be free'd while 
it is 
 J being duplicated.  There is no global lock that crdup can hold for that, 
 J instead the caller is required to guarantee that.
 
 I already noticed to Rick in a private email, that there is suspisious
 place in new NFS, where newly allocated (via crdup) cred is temporarily
 placed on td_ucred, and then removed at the end of function. The function
 may sleep in malloc() and also block on mutexes.

None of that matters.  Only curthread touches td_ucred.  It isn't going to 
free its own credential while it is asleep. :)

 I suspect, that it may happen, that some other kernel facility performs
 crfree(td-td_ucred), and later on we are using already freed cred.
 
 However, I can't imagine which facility may do this. What if process gets
 SIGKILL while its thread is blocked on mutex or sleeping? Would it
 be reaped after it yields or before?

No, a signal is merely marked as pending.  It isn't delivered to a thread in 
the kernel until it exits back out of the kernel and prepares to return to 
userland (e.g. in userret()).  Only at that time will the thread actually be 
killed.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Sleeping thread (tid 100033, pid 16): panic in FreeBSD 10.0-CURRENT/amd64 r228662

2011-12-20 Thread O. Hartmann
On 12/20/11 16:55, Robert Watson wrote:
 On Tue, 20 Dec 2011, Attilio Rao wrote:
 
 As we are here, however, I have a question for Robert here: do you
 think we should support the _ddb() variant of options even in the case
 DDB is not enabled in the kernel?
 
 It's possible that _ddb() should be spelled _unlocked(), or perhaps
 _debug(), but neither really suggests what the name should actually
 imply: using it is safe only in a marginal (debugging) sense, and not in
 a production code sense. One might also reasonable call them
 stack_foo_dontusethis().
 
 The _ddb() variants are used in at least two not strictly DDB cases:
 redzone support, and Solaris memory allocation.  And, I guess, the
 current lock debugging case that we're talking about now, but I'm not
 sure if those debugging features specifically require DDB in the kernel
 themselves?
 
 Robert


Sorry, I have to come back to the topic of my hijacked thread ;-)

I realized that the problem occured when I enabled, just for fun, some
features in the kernel config file, in particular were these

device  ipmi
device  smbios
device  vdp
device  tpm

My box in question does not have any IPMI facility, nor does it have a
TPM chip or vdp. So I commented out vdp and tpm and the problem seems to
be gone.

I didn't find any manpage for smbios, the NOTES in amd64 says, it is for
DMI informations, but dmidecide works also without explicitely enabling
smbios in the kernel.

If someone wish that I have to perform another try with those modules
enabled, let me know.

oh



signature.asc
Description: OpenPGP digital signature


extattr_set_*() return type

2011-12-20 Thread John Baldwin
Hmm, if these functions are expected to operate like 'write(2)' and are 
supposed to return the number of bytes written, shouldn't their return value 
be 'ssize_t' instead of 'int'?  It looks like the system calls themselves 
already do the right thing in setting td_retval[] (they assign a ssize_t to it 
and td_retval[0] can hold a ssize_t on all of our current platforms).  It 
would seem that the only change would be to the header and probably 
syscalls.master.  I guess this would require a symver bump to fix though.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Samuel J. Greear
http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved

PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux
and Solaris. Steps to reproduce these benchmarks provided.

Sam

On Tue, Dec 20, 2011 at 1:20 PM, Igor Mozolevsky i...@hybrid-lab.co.ukwrote:

 Interestingly, while people seem to be (arguably rightly) focused on
 criticising Phoronix's benchmarking, nobody has offered an alternative
 benchmark; and while (again, arguably rightly) it is important to
 benchmark real world performance, equally, nobody has offered any
 numbers in relation to, for example, HTTP or SMTP, or any other real
 world-application torture tests done on the aforementioned two
 platforms... IMO, this just goes to show that doing is hard and
 criticising is much easier (yes, I am aware of the irony involved in
 making this statement, but someone has to!)


 Cheers,
 Igor M :-)
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: VM images for FreeBSD

2011-12-20 Thread Baptiste Daroussin
On Mon, Dec 19, 2011 at 10:31:51PM +0200, Alexander Yerenkow wrote:
 2011/12/19 Adrian Chadd adr...@freebsd.org
 
  Hi,
 
  Hm, so this lets us create a virtualbox image from what, a set of
  install tarballs? Or /usr/src build?
 
  I'm using cross-build and installation from sources dir (which is after
 that got svn-up'ed and all goes again).
 It shouldn't be complex to install to image from installation media and/or
 tarballs, but mine main idea is to have rolling image for making some
 automated tests.
 Currently I'm establishing building and providing images scheme, will do
 images with KMS+small graphical programs, with qt+unstable KDE, and
 probably with BHyVe. I think that's most useful setups currently. And maybe
 some image for benchmarking :)
 
 

FYI I have been working on a ova file generator for the release, I manage to
create ova images that do work on VirtualBox without problems, there are still
some problems with vmware for now, the goal is to have a standard vm ready image
(ova is standard) that would work on every system that do support it.

the image can be easily created from the release.

regards,
Bapt


pgpVx97YSdHJw.pgp
Description: PGP signature


Re: extattr_set_*() return type

2011-12-20 Thread mdf
On Tue, Dec 20, 2011 at 1:49 PM, John Baldwin j...@freebsd.org wrote:
 Hmm, if these functions are expected to operate like 'write(2)' and are
 supposed to return the number of bytes written, shouldn't their return value
 be 'ssize_t' instead of 'int'?  It looks like the system calls themselves
 already do the right thing in setting td_retval[] (they assign a ssize_t to it
 and td_retval[0] can hold a ssize_t on all of our current platforms).  It
 would seem that the only change would be to the header and probably
 syscalls.master.  I guess this would require a symver bump to fix though.

An extended attribute larger than 2GB is a programming abuse, though.
Technically int may not be 32 bits but it is on all supported
platforms now.

Cheers,
matthew
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread O. Hartmann
On 12/20/11 22:45, Samuel J. Greear wrote:
 http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved
 
 PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux
 and Solaris. Steps to reproduce these benchmarks provided.
 
 Sam
 
 On Tue, Dec 20, 2011 at 1:20 PM, Igor Mozolevsky i...@hybrid-lab.co.ukwrote:
 
 Interestingly, while people seem to be (arguably rightly) focused on
 criticising Phoronix's benchmarking, nobody has offered an alternative
 benchmark; and while (again, arguably rightly) it is important to
 benchmark real world performance, equally, nobody has offered any
 numbers in relation to, for example, HTTP or SMTP, or any other real
 world-application torture tests done on the aforementioned two
 platforms... IMO, this just goes to show that doing is hard and
 criticising is much easier (yes, I am aware of the irony involved in
 making this statement, but someone has to!)


 Cheers,
 Igor M :-)
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Thanks for those numbers.
Impressive how Matthew Dillon's project jumps forward now. And it is
still impressive to see that the picture is still in the right place
when it comes to a comparison to Linux.
Also, OpenIndiana shows an impressive performance.
But this is only one suite of testing. Scientific Linux is supposed to
give the best performance for scientifi purposes, i.e. for longhaul
calculations, much numerical stuff. It outperforms in a typical server
application FreeBSd, were FreeBSD shoulkd have the power to serve.

Is the postgresql benchmark the only way to benchmark?

Well, this inspires me to gather together all the benchmarks someone
could find. There were lots of compalins about FreeBSD's poor
performance with BIND - once a domain of FreeBSD. Network performance
seems also to be an issue if it comes to scalability.
It would be nice to see what portion of the raw CPU/GPU power the OS
(FreeBSD, Linux ...)  delivers to scientific applications.

I only know some kind of benchmarks, BYTE UNIX benchmark, LINPACK test
... Does someone know a site to look for a couple of benchmarks to test

a) memory system
b) scalability (apart from pgbench)
c) network performance/throughput/network scalability
d) portion of CPU performance the system delivers for numerical
applications to the user apart from the system's own consumption
e) disk I/O performance and scalability

it would also be nice to discuss some nice settings and performance
tunings for FreeBSD for several scenarios. I guess, starting developing
benchmarking test scenarios for several purposes would lead faster to
real numbers and non polemic than weird discussions ...



signature.asc
Description: OpenPGP digital signature


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Jeremy Chadwick
On Tue, Dec 20, 2011 at 11:54:23PM +0100, O. Hartmann wrote:
 On 12/20/11 22:45, Samuel J. Greear wrote:
  http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved
  
  PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux
  and Solaris. Steps to reproduce these benchmarks provided.
  
  Sam
  
  On Tue, Dec 20, 2011 at 1:20 PM, Igor Mozolevsky 
  i...@hybrid-lab.co.ukwrote:
  
  Interestingly, while people seem to be (arguably rightly) focused on
  criticising Phoronix's benchmarking, nobody has offered an alternative
  benchmark; and while (again, arguably rightly) it is important to
  benchmark real world performance, equally, nobody has offered any
  numbers in relation to, for example, HTTP or SMTP, or any other real
  world-application torture tests done on the aforementioned two
  platforms... IMO, this just goes to show that doing is hard and
  criticising is much easier (yes, I am aware of the irony involved in
  making this statement, but someone has to!)
 
 
  Cheers,
  Igor M :-)
  ___
  freebsd-current@freebsd.org mailing list
  http://lists.freebsd.org/mailman/listinfo/freebsd-current
  To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
 
 
 Thanks for those numbers.
 Impressive how Matthew Dillon's project jumps forward now. And it is
 still impressive to see that the picture is still in the right place
 when it comes to a comparison to Linux.
 Also, OpenIndiana shows an impressive performance.

Preface to my long post below:

The things being discussed here are benchmarks, as in how much work
can you get out of Thing.  This is VERY DIFFERENT from testing
interactivity in a scheduler, which is more of a test that says when
Thing X is executed while heavier-Thing Y is also being executed, how
much interaction is lost in Thing X.

The reason people notice this when using Xorg is because it's visual,
in an environment where responsiveness is absolutely mandatory above all
else.  Nobody is going to put up with a system where during a buildworld
they go to move a window or click a mouse button or type a key and find
that the window doesn't move, the mouse click is lost, or the key typed
has gone into the bit bucket -- or, that those things are SEVERELY
delayed, to the point where interactivity is crap.

I just want to make that clear to folks.  This immense thread has been
with regards to the latter -- bad interactivity/responsiveness on a
system which was undergoing load that SHOULD be distributed more
evenly across the system *while* keeping interactivity/responsiveness
high.  Historically nice/renice has been used for this task, but that
was when kernels were a little less complex and I/O subsystems were less
complex.  Remember: we've now got schedulers for each type of thing,
and who gets what priority?  You get my point I'm sure.

So remember: this was to discuss that aspect, with regards to ULE vs.
4BSD schedulers.

Now, back to the benchmarks:

This also interested me:

* Linux system crashed
  http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg8.html

* OpenIndiana system crashed same way as Linux system
  http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg00017.html

I cannot help but wonder if the Linux and OpenIndiana installations were
more stressful on the hardware -- getting more out of the system, maybe
resulting in increased power/load, which in turn resulted in the systems
locking up (shoddy PSU, unstable mainboard, MCH problems, etc.).

My point is that Francois states these things in such a way to imply
that DragonflyBSD was more stable, when in fact I happen to wonder the
opposite point -- that is to say, Linux and OpenIndiana were trying to
use the hardware more-so than DragonflyBSD, thus tickled what may be a
hardware-level problem.

 But this is only one suite of testing. Scientific Linux is supposed to
 give the best performance for scientifi purposes, i.e. for longhaul
 calculations, much numerical stuff. It outperforms in a typical server
 application FreeBSd, were FreeBSD shoulkd have the power to serve.
 
 Is the postgresql benchmark the only way to benchmark?

I sure hope not.  But you know what's equally as interesting?  This:

http://people.freebsd.org/~kris/scaling/

Specifically circa 2008:

http://people.freebsd.org/~kris/scaling/4cpu-pgsql.png
http://people.freebsd.org/~kris/scaling/pgsql-16cpu-2.png
http://people.freebsd.org/~kris/scaling/pgsql-16cpu.png

Now, I don't know if what was used in those (pgsql sysbench) was the
same thing as pg_bench in the DragonflyBSD tests, but if so, the
numbers are different to a point that is preposterous.

There's also this:

http://people.freebsd.org/~kris/scaling/pgsql-ncpu.png

Now, compare those numbers to the TPS numbers shown here:

http://dl.wolfpond.org/Pg-benchmarks.pdf

So um... yeah.  Now, if someone here is going to say well, what
was tested by Kris was FreeBSD 7.0, while what was 

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Matthew Tippett

Bottom post this time to follow Oliver :).

On 12/20/2011 02:54 PM, O. Hartmann wrote:

On 12/20/11 22:45, Samuel J. Greear wrote:

http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved

PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux
and Solaris. Steps to reproduce these benchmarks provided.

Sam
There are still possible issues with those benchmarks.  The Xeon has 
known problems scaling from 6 to 12 cores (well enabling the 
hyperthreading), so you may find that some platforms are penalized in 
performance if HT is turned on.  See the scaling that Phoronix has done in


http://openbenchmarking.org/result/1112166-AR-1112153AR03

Most systems are good with scaling on real cores, the hyperthreading 
(and for that matter the Bulldozer thread affinity) can really break 
performance.   Different platforms have different behaviours.  
Benchmarking is a mucky business..


Note that the benchmarks with Phoronix test suite are repeatable, once 
installed, you can just run ./phoronix-test-suite benchmark 
1112113-AR-ORACLELIN37 to repeat (as close as the system allows) the 
benchmarks that started this thread.

Is the postgresql benchmark the only way to benchmark?
pgbench is already included in the Phoronix Test Suite (at least 9.0.1 
TPC-B benchmark.




Well, this inspires me to gather together all the benchmarks someone
could find. There were lots of compalins about FreeBSD's poor
performance with BIND - once a domain of FreeBSD. Network performance
seems also to be an issue if it comes to scalability.
It would be nice to see what portion of the raw CPU/GPU power the OS
(FreeBSD, Linux ...)  delivers to scientific applications.

I only know some kind of benchmarks, BYTE UNIX benchmark, LINPACK test
... Does someone know a site to look for a couple of benchmarks to test

a) memory system
b) scalability (apart from pgbench)
c) network performance/throughput/network scalability
d) portion of CPU performance the system delivers for numerical
applications to the user apart from the system's own consumption
e) disk I/O performance and scalability
The majority of these benchmarks are already in Phoronix Test Suite.  
There is monitoring capability (temp, load, CPU states, etc).  The 
question is the mapping from system attribute to benchmark, as well as 
determine what the ambigious terms mean (scaling can mean on increasing 
workloads, as memory is increased, as cpus are increased).




it would also be nice to discuss some nice settings and performance
tunings for FreeBSD for several scenarios. I guess, starting developing
benchmarking test scenarios for several purposes would lead faster to
real numbers and non polemic than weird discussions ...

This is what Michael and I are wanting to see.  Adrian Chadd has 
offerered to help facilitate within the FreeBSD community.  As mentioned 
before, what I'd like to see is


1) Recommendations for more rounded benchmarks from the FreeBSD 
perspective

2) Tuning guide documented somewhere within the community
3) Comparative results based on the communities testing.

All concrete, and all achievable.

Regards,

Matthew
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: VM images for FreeBSD

2011-12-20 Thread Alexander Yerenkow
2011/12/21 Baptiste Daroussin b...@freebsd.org

 On Mon, Dec 19, 2011 at 10:31:51PM +0200, Alexander Yerenkow wrote:
  2011/12/19 Adrian Chadd adr...@freebsd.org
 
   Hi,
  
   Hm, so this lets us create a virtualbox image from what, a set of
   install tarballs? Or /usr/src build?
  
   I'm using cross-build and installation from sources dir (which is after
  that got svn-up'ed and all goes again).
  It shouldn't be complex to install to image from installation media
 and/or
  tarballs, but mine main idea is to have rolling image for making some
  automated tests.
  Currently I'm establishing building and providing images scheme, will do
  images with KMS+small graphical programs, with qt+unstable KDE, and
  probably with BHyVe. I think that's most useful setups currently. And
 maybe
  some image for benchmarking :)
 
 

 FYI I have been working on a ova file generator for the release, I manage
 to
 create ova images that do work on VirtualBox without problems, there are
 still
 some problems with vmware for now, the goal is to have a standard vm ready
 image
 (ova is standard) that would work on every system that do support it.


This is good :)
Do you have some scripts left?


 the image can be easily created from the release.


Well, I'm trying to build custom images, like rolling from svn [+ patches]
[+ packages] [+ some initial config].
A way for building images from release I think can be taken from PC-BSD,
they distribute upcoming 9 in VM formats too.




 regards,
 Bapt




-- 
Regards,
Alexander Yerenkow
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: VM images for FreeBSD

2011-12-20 Thread Outback Dingo
On Tue, Dec 20, 2011 at 6:52 PM, Alexander Yerenkow yeren...@gmail.com wrote:
 2011/12/21 Baptiste Daroussin b...@freebsd.org

 On Mon, Dec 19, 2011 at 10:31:51PM +0200, Alexander Yerenkow wrote:
  2011/12/19 Adrian Chadd adr...@freebsd.org
 
   Hi,
  
   Hm, so this lets us create a virtualbox image from what, a set of
   install tarballs? Or /usr/src build?
  
   I'm using cross-build and installation from sources dir (which is after
  that got svn-up'ed and all goes again).
  It shouldn't be complex to install to image from installation media
 and/or
  tarballs, but mine main idea is to have rolling image for making some
  automated tests.
  Currently I'm establishing building and providing images scheme, will do
  images with KMS+small graphical programs, with qt+unstable KDE, and
  probably with BHyVe. I think that's most useful setups currently. And
 maybe
  some image for benchmarking :)
 
 

 FYI I have been working on a ova file generator for the release, I manage
 to
 create ova images that do work on VirtualBox without problems, there are
 still
 some problems with vmware for now, the goal is to have a standard vm ready
 image
 (ova is standard) that would work on every system that do support it.



How about in XEN 4.1 a .vhd image would be awesome, especially boot from zfs


 This is good :)
 Do you have some scripts left?


 the image can be easily created from the release.


 Well, I'm trying to build custom images, like rolling from svn [+ patches]
 [+ packages] [+ some initial config].
 A way for building images from release I think can be taken from PC-BSD,
 they distribute upcoming 9 in VM formats too.




 regards,
 Bapt




 --
 Regards,
 Alexander Yerenkow
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread O. Hartmann
On 12/21/11 00:29, Jeremy Chadwick wrote:
 On Tue, Dec 20, 2011 at 11:54:23PM +0100, O. Hartmann wrote:
 On 12/20/11 22:45, Samuel J. Greear wrote:
 http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved

 PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux
 and Solaris. Steps to reproduce these benchmarks provided.

 Sam

 On Tue, Dec 20, 2011 at 1:20 PM, Igor Mozolevsky 
 i...@hybrid-lab.co.ukwrote:

 Interestingly, while people seem to be (arguably rightly) focused on
 criticising Phoronix's benchmarking, nobody has offered an alternative
 benchmark; and while (again, arguably rightly) it is important to
 benchmark real world performance, equally, nobody has offered any
 numbers in relation to, for example, HTTP or SMTP, or any other real
 world-application torture tests done on the aforementioned two
 platforms... IMO, this just goes to show that doing is hard and
 criticising is much easier (yes, I am aware of the irony involved in
 making this statement, but someone has to!)


 Cheers,
 Igor M :-)
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


 Thanks for those numbers.
 Impressive how Matthew Dillon's project jumps forward now. And it is
 still impressive to see that the picture is still in the right place
 when it comes to a comparison to Linux.
 Also, OpenIndiana shows an impressive performance.
 
 Preface to my long post below:
 
 The things being discussed here are benchmarks, as in how much work
 can you get out of Thing.  This is VERY DIFFERENT from testing
 interactivity in a scheduler, which is more of a test that says when
 Thing X is executed while heavier-Thing Y is also being executed, how
 much interaction is lost in Thing X.
 
 The reason people notice this when using Xorg is because it's visual,
 in an environment where responsiveness is absolutely mandatory above all
 else.  Nobody is going to put up with a system where during a buildworld
 they go to move a window or click a mouse button or type a key and find
 that the window doesn't move, the mouse click is lost, or the key typed
 has gone into the bit bucket -- or, that those things are SEVERELY
 delayed, to the point where interactivity is crap.

I whitnessed sticky, jumpy and non-responsive-for seconds FreeBSD
servers (serving homes, NFS/SAMBA and PostgreSQL database (small)).
Those seconds where enough to cut a ssh line. Not funny. Network
traffic droped significantly. X/Desktop makes the problem visible,
indeed. But not seeing it does not mean it isn't there.
This might be the reason why FreeBSD is so much behind when it comes to X?

 
 I just want to make that clear to folks.  This immense thread has been
 with regards to the latter -- bad interactivity/responsiveness on a
 system which was undergoing load that SHOULD be distributed more
 evenly across the system *while* keeping interactivity/responsiveness
 high.  Historically nice/renice has been used for this task, but that
 was when kernels were a little less complex and I/O subsystems were less
 complex.  Remember: we've now got schedulers for each type of thing,
 and who gets what priority?  You get my point I'm sure.
 
 So remember: this was to discuss that aspect, with regards to ULE vs.
 4BSD schedulers.
 
 Now, back to the benchmarks:
 
 This also interested me:
 
 * Linux system crashed
   http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg8.html
 
 * OpenIndiana system crashed same way as Linux system
   http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg00017.html
 
 I cannot help but wonder if the Linux and OpenIndiana installations were
 more stressful on the hardware -- getting more out of the system, maybe
 resulting in increased power/load, which in turn resulted in the systems
 locking up (shoddy PSU, unstable mainboard, MCH problems, etc.).

Is FreeBSD supposed to run on dumpyard equipment? In former times,
freeBSD was used on high value hardware, not the decomissioned crap with
shoddy PSUs or whatsoever.
If I need a server, I care about quality hardware as I do for my lab's
box and my own box at home. I expect a server garde hardware to act
like that and I expect the operating system to get the maximum out of
that hardware. Otherwise it is not worth one shot.

 
 My point is that Francois states these things in such a way to imply
 that DragonflyBSD was more stable, when in fact I happen to wonder the
 opposite point -- that is to say, Linux and OpenIndiana were trying to
 use the hardware more-so than DragonflyBSD, thus tickled what may be a
 hardware-level problem.
 
 But this is only one suite of testing. Scientific Linux is supposed to
 give the best performance for scientifi purposes, i.e. for longhaul
 calculations, much numerical stuff. It outperforms in a typical server
 application FreeBSd, were 

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Adrian Chadd
Is there a specific version of the test suite that should be used, to
compare against the published results?


Adrian

On 20 December 2011 17:18, Matthew Tippett matt...@phoronix.com wrote:
 For such a system, the greatest immediate value would be to attempt to
 reproduce the benchmarks in question.

 Install PTS from www.phoronix-test-suite.com or freshports.org.

 Run the benchmark against those used in the article

    phoronix-test-suite benchmark 1112113-AR-ORACLELIN37

 You will be asked to push the comparison up to openbenchmarking at the end.

 Matthew


 On 12/20/2011 01:39 PM, O. Hartmann wrote:

 On 12/20/11 21:20, Igor Mozolevsky wrote:

 Interestingly, while people seem to be (arguably rightly) focused on
 criticising Phoronix's benchmarking, nobody has offered an alternative
 benchmark; and while (again, arguably rightly) it is important to
 benchmark real world performance, equally, nobody has offered any
 numbers in relation to, for example, HTTP or SMTP, or any other real
 world-application torture tests done on the aforementioned two
 platforms... IMO, this just goes to show that doing is hard and
 criticising is much easier (yes, I am aware of the irony involved in
 making this statement, but someone has to!)


 Cheers,
 Igor M :-)

 Unfortunately, M. Larabel is the only one who's performing benchmarks on
 FreeBSD, comparing its performance to the Linux-opponents. Adn indeed,
 there is a lot of criticism, but no alternative.
 I said unfortunately - not offensive - since Larabel and Phoronix are
 sadly the only ones who do actually such bechmarking.

 It would be much more nicer and kind to support those people.

 Well, in January/February we get new hardware. One box is supposed to do
 number crunching via 12 cores and a TESLA GPU. My colleague is
 developing a high parallelized peice of software for satellite data
 transformation. The software package is CPU bound, partially GPU, but
 massively memory hungry (96 to 128 GB RAM is needed).
 What I can offer is, since I will also work on that machine and I've
 free hand to administer, in the spare time of doing my PhD, installing
 FreeBSD 9.0/10.0 besides SuSe Linux and looking forward having one ZFS
 data storage drive for homes, so both systems can perform on a most
 recent ZFS. I'm new to Linux, not a BSD guru, nor I'm a professional
 programmer/developer. My skills are sufficient for the daily scientific
 work. So, without pressure, I'm willing to perform some HPC benchmarks
 under advice if the day comes and those interested in bare numbers of
 FreeBSD vs. Linux performance with a real-world-scientific application.

 I would appreciate to see some of the developers and/or FreeBSD hackers
 to help Phoronix setting up a proper testenvironment instead of bashing
 M. Larabel and his fellows.

 Regards,
 Oliver


 ___
 freebsd-sta...@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-stable
 To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Matthew Tippett
For such a system, the greatest immediate value would be to attempt to 
reproduce the benchmarks in question.


Install PTS from www.phoronix-test-suite.com or freshports.org.

Run the benchmark against those used in the article

phoronix-test-suite benchmark 1112113-AR-ORACLELIN37

You will be asked to push the comparison up to openbenchmarking at the end.

Matthew

On 12/20/2011 01:39 PM, O. Hartmann wrote:

On 12/20/11 21:20, Igor Mozolevsky wrote:

Interestingly, while people seem to be (arguably rightly) focused on
criticising Phoronix's benchmarking, nobody has offered an alternative
benchmark; and while (again, arguably rightly) it is important to
benchmark real world performance, equally, nobody has offered any
numbers in relation to, for example, HTTP or SMTP, or any other real
world-application torture tests done on the aforementioned two
platforms... IMO, this just goes to show that doing is hard and
criticising is much easier (yes, I am aware of the irony involved in
making this statement, but someone has to!)


Cheers,
Igor M :-)

Unfortunately, M. Larabel is the only one who's performing benchmarks on
FreeBSD, comparing its performance to the Linux-opponents. Adn indeed,
there is a lot of criticism, but no alternative.
I said unfortunately - not offensive - since Larabel and Phoronix are
sadly the only ones who do actually such bechmarking.

It would be much more nicer and kind to support those people.

Well, in January/February we get new hardware. One box is supposed to do
number crunching via 12 cores and a TESLA GPU. My colleague is
developing a high parallelized peice of software for satellite data
transformation. The software package is CPU bound, partially GPU, but
massively memory hungry (96 to 128 GB RAM is needed).
What I can offer is, since I will also work on that machine and I've
free hand to administer, in the spare time of doing my PhD, installing
FreeBSD 9.0/10.0 besides SuSe Linux and looking forward having one ZFS
data storage drive for homes, so both systems can perform on a most
recent ZFS. I'm new to Linux, not a BSD guru, nor I'm a professional
programmer/developer. My skills are sufficient for the daily scientific
work. So, without pressure, I'm willing to perform some HPC benchmarks
under advice if the day comes and those interested in bare numbers of
FreeBSD vs. Linux performance with a real-world-scientific application.

I would appreciate to see some of the developers and/or FreeBSD hackers
to help Phoronix setting up a proper testenvironment instead of bashing
M. Larabel and his fellows.

Regards,
Oliver



___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread matthew

   The benchmarks themselves are versioned.  So in general most of the
   av= ailable versions of PTS itself should be fine.  PTS can be
   considered = an execution shell that doesn't affect the benchmark
   itself.
   Note th= at you'll download a pile of the benchmarks, build and
   install them.  = Then you run about 49 individual steps.
   Matthew

   -- Sent from my HP Pre3
 _

   On Dec 20, 2011 5:30 PM, Adrian Chadd adr...@freebsd.org= ; wrote:
   Is there a specific version of the test suite that = should be used,
   to
   compare against the published results?
   

   Adrian
   
   On 20 December 2011 17:18, Matthew Tippett = ;matt...@phoronix.com
   wrote:
For such a system, the greatest= immediate value would be to attempt
   to
reproduce the benchmarks= in question.
   
Install PTS from www.phoronix-test-suit= e.com or freshports.org.
   
Run the benchmark against th= ose used in the article
   
   phoronix-test-su= ite benchmark
   1112113-AR-ORACLELIN37
   
You will be aske= d to push the comparison up to openbenchmarking at
   the end.
   
 Matthew
   
   
On 12/20/2011 01:39 PM, O. = Hartmann wrote:
   
On 12/20/11 21:20, Igor Mozol= evsky wrote:
   
Interestingly, while peo= ple seem to be (arguably rightly)
   focused on
criticising= Phoronix's benchmarking, nobody has offered an
   alternative
   = gt; benchmark; and while (again, arguably rightly) it is
   important to
benchmark real world performance, equally, nobody has offered   any
numbers in relation to, for example, HTTP or SMTP, = or any other
   real
world-application torture tests done= on the aforementioned
   two
platforms... IMO, this just g= oes to show that doing is hard
   and
criticising is muc= h easier (yes, I am aware of the irony
   involved in
maki= ng this statement, but someone has to!)
   
   g= t;
Cheers,
Igor M :-)
   = 
Unfortunately, M. Larabel is the only one who's performingbenchmarks 
on
FreeBSD, comparing its performance to the Linu= x-opponents. Adn
   indeed,
there is a lot of criticism, but no= alternative.
I said unfortunately - not offensive - since L= arabel and Phoronix
   are
sadly the only ones who do actually = such bechmarking.
   
It would be much more nicer= and kind to support those people.
   
Well, in J= anuary/February we get new hardware. One box is
   supposed to do
   g= t; number crunching via 12 cores and a TESLA GPU. My colleague
   is
   = ; developing a high parallelized peice of software for satellite
   data= 
transformation. The software package is CPU bound, partiall= y GPU,
   but
massively memory hungry (96 to 128 GB RAM is need= ed).
What I can offer is, since I will also work on that mac= hine and
   I've
free hand to administer, in the spare time of = doing my PhD,
   installing
FreeBSD 9.0/10.0 besides SuSe Linux= and looking forward having one
   ZFS
data storage drive for h= omes, so both systems can perform on a
   most
recent ZFS. I'm = new to Linux, not a BSD guru, nor I'm a
   professional
program= mer/developer. My skills are sufficient for the daily
   scientific
   =  work. So, without pressure, I'm willing to perform some HPC
   benchmarks= 
under advice if the day comes and those interested in barenumbers of
FreeBSD vs. Linux performance with a real-world-s= cientific
   application.
   
I would appreciate to = see some of the developers and/or FreeBSD
   hackers
to help Ph= oronix setting up a proper testenvironment instead of
   bashing
   = ; M. Larabel and his fellows.
   
Regards,
   =  Oliver
   
   
__= _
freebsd-sta...@freebsd.org mailing lis= t
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to
   freebsd-stable-unsubscribe@freebsd.= org
   ___
   freebsd-pe= rforma...@freebsd.org mailing list
   http://lists.freebsd.org/mailman/l= istinfo/freebsd-performance
   To unsubscribe, send any mail to freebsd   
-performance-unsubscr...@freebsd.org

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

2011-12-20 Thread Michael Larabel
Any version is fine that's PTS 3.0 or newer in terms of being 
compatible, since the test profiles are versioned separately and 
automatically fetched to match the result file. However, I'd recommended 
the newest (PTS 3.6) as it contains the best FreeBSD support at present 
in terms of hardware/software information parsing (for the automated 
table), etc.


Michael

On 12/20/2011 07:29 PM, Adrian Chadd wrote:

Is there a specific version of the test suite that should be used, to
compare against the published results?


Adrian

On 20 December 2011 17:18, Matthew Tippettmatt...@phoronix.com  wrote:

For such a system, the greatest immediate value would be to attempt to
reproduce the benchmarks in question.

Install PTS from www.phoronix-test-suite.com or freshports.org.

Run the benchmark against those used in the article

phoronix-test-suite benchmark 1112113-AR-ORACLELIN37

You will be asked to push the comparison up to openbenchmarking at the end.

Matthew


On 12/20/2011 01:39 PM, O. Hartmann wrote:

On 12/20/11 21:20, Igor Mozolevsky wrote:

Interestingly, while people seem to be (arguably rightly) focused on
criticising Phoronix's benchmarking, nobody has offered an alternative
benchmark; and while (again, arguably rightly) it is important to
benchmark real world performance, equally, nobody has offered any
numbers in relation to, for example, HTTP or SMTP, or any other real
world-application torture tests done on the aforementioned two
platforms... IMO, this just goes to show that doing is hard and
criticising is much easier (yes, I am aware of the irony involved in
making this statement, but someone has to!)


Cheers,
Igor M :-)

Unfortunately, M. Larabel is the only one who's performing benchmarks on
FreeBSD, comparing its performance to the Linux-opponents. Adn indeed,
there is a lot of criticism, but no alternative.
I said unfortunately - not offensive - since Larabel and Phoronix are
sadly the only ones who do actually such bechmarking.

It would be much more nicer and kind to support those people.

Well, in January/February we get new hardware. One box is supposed to do
number crunching via 12 cores and a TESLA GPU. My colleague is
developing a high parallelized peice of software for satellite data
transformation. The software package is CPU bound, partially GPU, but
massively memory hungry (96 to 128 GB RAM is needed).
What I can offer is, since I will also work on that machine and I've
free hand to administer, in the spare time of doing my PhD, installing
FreeBSD 9.0/10.0 besides SuSe Linux and looking forward having one ZFS
data storage drive for homes, so both systems can perform on a most
recent ZFS. I'm new to Linux, not a BSD guru, nor I'm a professional
programmer/developer. My skills are sufficient for the daily scientific
work. So, without pressure, I'm willing to perform some HPC benchmarks
under advice if the day comes and those interested in bare numbers of
FreeBSD vs. Linux performance with a real-world-scientific application.

I would appreciate to see some of the developers and/or FreeBSD hackers
to help Phoronix setting up a proper testenvironment instead of bashing
M. Larabel and his fellows.

Regards,
Oliver


___
freebsd-sta...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

___
freebsd-sta...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org



___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: r228700 can't dhclient em0

2011-12-20 Thread Brooks Davis
On Tue, Dec 20, 2011 at 11:15:20PM +0400, Gleb Smirnoff wrote:
   Doug,
 
 On Tue, Dec 20, 2011 at 12:38:53AM -0800, Doug Barton wrote:
 D  I saw this too, when my kernel and userland were out of sync (e.g. just
 D  after installing a new kernel, and before installworld).  I suspect it
 D  is caused by the changes in r228571, which cause old ifconfig and
 D  dhclient to not recognize any interfaces.  I'm not 100% sure though...
 D 
 D I tried replacing both ifconfig and dhclient with the versions that were
 D built along with the new kernel, and that didn't work.
 
 This shouldn't happen. If you did 'make buildworld buildkernel', then
 your world in objdir would have binaries compiled with includes from
 source tree, not from /usr/include, thus compatible with new kernel.
 
 'make buildworld buildkernel' always produces compatible kernel and
 worlds.
 
 However, if you did 'cd /usr/src/sbin/ifconfig  make all install' then
 that didn't work, since used headers from /usr/include.
 
 D The traditional (and documented) upgrade process for many years has been
 D to boot the new kernel, make sure it's Ok, then update world. Obviously
 D something different is needed this time, so it needs an UPDATING entry
 D (assuming that all this is not just a bug).
 
 The documented one says 'Reboot into single user mode' and then install
 new world. This path was not broken, since single user mode doesn't
 imply network support.

While this is the documented path, it's not actually been required
except in edge cases for ages (the last I can remember is a.out-elf).
It's been long enough that I don't think we can really make people do
it except for a short period of time in HEAD.  I believe it's
unacceptable for a release to release upgrade.

 The undocumented brave way 'make installkernel installworld  reboot'
 works also, without any problems.

At least until someone screws up something else and you now can't use
kernel.old either.  This is somewhat ok for HEAD users, but I think we
should try harder to avoid this sort of situation.

-- Brooks


pgp309HM5Uyj4.pgp
Description: PGP signature


Re: r228700 can't dhclient em0

2011-12-20 Thread Brooks Davis
On Tue, Dec 20, 2011 at 11:23:54PM +0400, Gleb Smirnoff wrote:
   Brooks,
 
 On Tue, Dec 20, 2011 at 10:51:34AM -0600, Brooks Davis wrote:
 B We almost certainly need to back r228571 out.  This is not an acceptable
 B upgrade path that would be acceptable historically.  Specially, we have
 B effectively promised users that an X.Y world will work on an X+1.0
 B kernel for most of history.  There are obvious exceptions to this, but
 B we have never allowed ifconfig to be one of them (I broke it many years
 B ago with my first attempt to add if_epoch to if_data and had to back
 B that out).
 
 Pardon, where did we promise that? The applications in jail should work,
 but not kernel configuration tools. The network facilities like ipfw
 and pf has changed their ABI numerous times, making a new kernel
 with older world inaccessible via network after boot.

We've not promised it in writing, but by not breaking it for many years
we're created an effective promise IMO.

 Considering r228571: we need to specify vhid as additional address
 attribute in atomic manner, via one ioctl(). Address can't be first
 configured, and then made redundant, that would lead it to being
 static for a short period, sending gratutious ARP announcement, etc.
 
 An assumption that we are not allowed to change ABI for our own tools
 strongly discourages bringing in new features :(

I'm not saying we can't change the ABI.  I am saying that we should
make sure we can support a remote, console-less upgrade from what ever
9.x branches are practical to 10.0 in the average case.  We've set the
expectation that this will work and IMO it's a reasonable expectation.  
We've violated it in a few cases such as the emX-igbX fiasco, but by
and large most users have been able to do this.  I guess I'm ok with
dealing with this in HEAD, but I'm not OK with it for all upgrades from
9.x.

-- Brooks


pgpQrFZn3sdT9.pgp
Description: PGP signature


Re: Is the svn2cvs gateway down ?

2011-12-20 Thread Doug Barton
On 12/20/2011 02:01, Claude Buisson wrote:
 Hi,
 
 It seems (from my own csup's and cvswe.cgi) that the src commits are lost,
 starting with r228697 Sun Dec 18 22:04:55 2011)

Yeah, my warning 2 days ago that this was going to happen seems to have
gone un-heeded. :)  I'm sure you can take bz' word that it's being
looked at now though.


Doug

-- 

[^L]

Breadth of IT experience, and depth of knowledge in the DNS.
Yours for the right price.  :)  http://SupersetSolutions.com/

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: regression: Xorg get 100% cpu and freeze

2011-12-20 Thread Doug Barton
For those having this problem if you compile a custom kernel with the
4BSD scheduler instead of ULE, does the problem disappear?


Doug

-- 

[^L]

Breadth of IT experience, and depth of knowledge in the DNS.
Yours for the right price.  :)  http://SupersetSolutions.com/

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: clang (__builtin_ffs) vs /usr/include/string.h

2011-12-20 Thread Benjamin Kaduk

Hi Larry,

On Tue, 20 Dec 2011, Larry Rosenman wrote:


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Is anyone going to fix the following in the clang or FreeBSD system?


I haven't seen any mention of __builtin_ffs on any freebsd lists since 
your thread in october, system headers with clang?.  That rather makes 
me suspect that no one else is seeing this problem.  It's hard to fix 
something you don't know about.




In file included from /usr/include/string.h:45:
/usr/include/strings.h:47:6: error: conflicting types for '__builtin_ffs'
int  ffs(int) __pure2;
^
/usr/include/machine/cpufunc.h:140:24: note: expanded from:
#defineffs(x)  __builtin_ffs(x)
  ^
/usr/include/strings.h:47:6: note: '__builtin_ffs' is a builtin with
type 'int (unsigned int)'
7 warnings and 1 error generated.
*** Error code 1



As such, since we don't know what the problem is, some context for what 
is going on here would be useful -- what are you trying to compile?  Still 
lsof?




This looks like something that needs to change in clang/FreeBSD headers.


Not necessarily -- things from the machine/ hierarchy are pretty uncommon, 
and I wouldn't be surprised if they had dependencies and ordering 
requirements involved.  It could well be an application error, but we 
can't tell, since there is no context.


-Ben Kaduk
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Failure to compile world

2011-12-20 Thread Alex Kuster

Ok, just sent the PR → http://www.freebsd.org/cgi/query-pr.cgi?pr=163495
And hopefully I'll make another one describing some other issues I'm 
having (like world compilation taking libs/headers from system instead 
of the own src tree, failing to compile due to missing symbols in 
libraries or changes in headers)


Thanks for your time Garrett !

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: VM images for FreeBSD

2011-12-20 Thread Baptiste Daroussin
On Wed, Dec 21, 2011 at 01:52:40AM +0200, Alexander Yerenkow wrote:
 2011/12/21 Baptiste Daroussin b...@freebsd.org
 
  On Mon, Dec 19, 2011 at 10:31:51PM +0200, Alexander Yerenkow wrote:
   2011/12/19 Adrian Chadd adr...@freebsd.org
  
Hi,
   
Hm, so this lets us create a virtualbox image from what, a set of
install tarballs? Or /usr/src build?
   
I'm using cross-build and installation from sources dir (which is after
   that got svn-up'ed and all goes again).
   It shouldn't be complex to install to image from installation media
  and/or
   tarballs, but mine main idea is to have rolling image for making some
   automated tests.
   Currently I'm establishing building and providing images scheme, will do
   images with KMS+small graphical programs, with qt+unstable KDE, and
   probably with BHyVe. I think that's most useful setups currently. And
  maybe
   some image for benchmarking :)
  
  
 
  FYI I have been working on a ova file generator for the release, I manage
  to
  create ova images that do work on VirtualBox without problems, there are
  still
  some problems with vmware for now, the goal is to have a standard vm ready
  image
  (ova is standard) that would work on every system that do support it.
 
 
 This is good :)
 Do you have some scripts left?
 

You can try it here: http://git.etoilebsd.net/vmdkimage/ currently this is a
quick and dirty make-vmdk.sh script I imported vmdkimage.c from Haiku, converted
it from C++ to C and make some changes so that it is able to build scsi vmdk
images (vmware needs scsi disk).

You can use it that way: ./make-vmdk.sh 4 90-RC3 test9

You will get a tes9.ova.xz image the disk size will be a 4G one, the xz will
take 100M. you can import it in vbox it will work ootb, it should work on vmware
I hope, currently no feedback on vmware for the last changes.

Here is an example
http://files.etoilebsd.net/ova/test9.ova.xz

No tunning on the OS yet

regards,
Bapt


pgpYeLGcqQZdo.pgp
Description: PGP signature


[head tinderbox] failure on i386/i386

2011-12-20 Thread FreeBSD Tinderbox
TB --- 2011-12-21 06:40:00 - tinderbox 2.8 running on freebsd-current.sentex.ca
TB --- 2011-12-21 06:40:00 - starting HEAD tinderbox run for i386/i386
TB --- 2011-12-21 06:40:00 - cleaning the object tree
TB --- 2011-12-21 06:40:59 - cvsupping the source tree
TB --- 2011-12-21 06:40:59 - /usr/bin/csup -z -r 3 -g -L 1 -h cvsup.sentex.ca 
/tinderbox/HEAD/i386/i386/supfile
TB --- 2011-12-21 06:46:23 - building world
TB --- 2011-12-21 06:46:23 - CROSS_BUILD_TESTING=YES
TB --- 2011-12-21 06:46:23 - MAKEOBJDIRPREFIX=/obj
TB --- 2011-12-21 06:46:23 - PATH=/usr/bin:/usr/sbin:/bin:/sbin
TB --- 2011-12-21 06:46:23 - SRCCONF=/dev/null
TB --- 2011-12-21 06:46:23 - TARGET=i386
TB --- 2011-12-21 06:46:23 - TARGET_ARCH=i386
TB --- 2011-12-21 06:46:23 - TZ=UTC
TB --- 2011-12-21 06:46:23 - __MAKE_CONF=/dev/null
TB --- 2011-12-21 06:46:23 - cd /src
TB --- 2011-12-21 06:46:23 - /usr/bin/make -B buildworld
 World build started on Wed Dec 21 06:46:23 UTC 2011
 Rebuilding the temporary build tree
 stage 1.1: legacy release compatibility shims
 stage 1.2: bootstrap tools
 stage 2.1: cleaning up the object tree
 stage 2.2: rebuilding the object tree
 stage 2.3: build tools
 stage 3: cross tools
 stage 4.1: building includes
 stage 4.2: building libraries
[...]
c++ -O2 -pipe -I/src/lib/clang/libclanglex/../../../contrib/llvm/include 
-I/src/lib/clang/libclanglex/../../../contrib/llvm/tools/clang/include 
-I/src/lib/clang/libclanglex/../../../contrib/llvm/tools/clang/lib/Lex -I. 
-I/src/lib/clang/libclanglex/../../../contrib/llvm/../../lib/clang/include 
-DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS 
-DLLVM_HOSTTRIPLE=\i386-unknown-freebsd10.0\ -fstack-protector 
-fno-exceptions -fno-rtti -c 
/src/lib/clang/libclanglex/../../../contrib/llvm/tools/clang/lib/Lex/Lexer.cpp
c++ -O2 -pipe -I/src/lib/clang/libclanglex/../../../contrib/llvm/include 
-I/src/lib/clang/libclanglex/../../../contrib/llvm/tools/clang/include 
-I/src/lib/clang/libclanglex/../../../contrib/llvm/tools/clang/lib/Lex -I. 
-I/src/lib/clang/libclanglex/../../../contrib/llvm/../../lib/clang/include 
-DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS 
-DLLVM_HOSTTRIPLE=\i386-unknown-freebsd10.0\ -fstack-protector 
-fno-exceptions -fno-rtti -c 
/src/lib/clang/libclanglex/../../../contrib/llvm/tools/clang/lib/Lex/LiteralSupport.cpp
c++ -O2 -pipe -I/src/lib/clang/libclanglex/../../../contrib/llvm/include 
-I/src/lib/clang/libclanglex/../../../contrib/llvm/tools/clang/include 
-I/src/lib/clang/libclanglex/../../../contrib/llvm/tools/clang/lib/Lex -I. 
-I/src/lib/clang/libclanglex/../../../contrib/llvm/../../lib/clang/include 
-DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS 
-DLLVM_HOSTTRIPLE=\i386-unknown-freebsd10.0\ -fstack-protector 
-fno-exceptions -fno-rtti -c 
/src/lib/clang/libclanglex/../../../contrib/llvm/tools/clang/lib/Lex/MacroArgs.cpp
/src/lib/clang/libclanglex/../../../contrib/llvm/tools/clang/lib/Lex/MacroArgs.cpp:
 In static member function 'static clang::Token 
clang::MacroArgs::StringifyArgument(const clang::Token*, clang::Preprocessor, 
bool, clang::SourceLocation, clang::SourceLocation)':
/src/lib/clang/libclanglex/../../../contrib/llvm/tools/clang/lib/Lex/MacroArgs.cpp:193:
 internal compiler error: Segmentation fault: 11
Please submit a full bug report,
with preprocessed source if appropriate.
See URL:http://gcc.gnu.org/bugs.html for instructions.
*** Error code 1

Stop in /src/lib/clang/libclanglex.
*** Error code 1

Stop in /src/lib/clang.
*** Error code 1

Stop in /src/lib.
*** Error code 1

Stop in /src.
*** Error code 1

Stop in /src.
*** Error code 1

Stop in /src.
*** Error code 1

Stop in /src.
TB --- 2011-12-21 07:39:25 - WARNING: /usr/bin/make returned exit code  1 
TB --- 2011-12-21 07:39:25 - ERROR: failed to build world
TB --- 2011-12-21 07:39:25 - 2500.89 user 509.55 system 3564.81 real


http://tinderbox.freebsd.org/tinderbox-head-HEAD-i386-i386.full
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org