Re: New CAM locking preview

2013-08-15 Thread Steven Hartland


- Original Message - 
From: Hans Petter Selasky h...@bitfrost.no

To: Alexander Motin m...@freebsd.org
Cc: Scott Long sco...@freebsd.org; Jeff Roberson j...@freebsd.org; ken k...@freebsd.org; freebsd-hackers@FreeBSD.org; 
FreeBSD SCSI freebsd-s...@freebsd.org; Steven Hartland s...@freebsd.org; Justin T. Gibbs gi...@freebsd.org

Sent: Friday, August 16, 2013 1:38 AM
Subject: Re: New CAM locking preview



On 08/15/13 23:40, Alexander Motin wrote:

Hi.

Last weeks I've made substantial progress on my CAM locking work. In
fact, at this moment I think I've tied all loose ends good enough to
consider the new design viable and implementation worth further testing
and bug fixing. So I would like to ask for review of my work from
everybody who interested in CAM internals.

In short, my idea was to split single per-SIM lock, that creates huge
congestion under high IOPS, into several smaller ones. So design I've
finally chosen includes such locks:
  1) New per-device (per-LUN) locks to protect state of the devices and
respective periphs. In most cases peripheral drivers just use that lock
instead of SIM lock used before, so code modification is minimal and
straightforward.
  2) New per-target lock to protect list of LUNs fetched from the device.
  3) Old single per-SIM lock to protect SIM driver internals, but only
that. No parts of CAM itself use that lock. Keeping it for SIMs allows
to keep API and hopefully ABI compatibility. Reducing its scope allows
to reduce congestion.
  4) New per-SIM lock to protect SIM and device command queues. That
allows execute queued commands from any context unrelated to other
locks. Also this lock serializes accesses to sim_action() method for the
most of commands, this allows to mostly avoid busy spilling on SIM lock
collision.
  5) New per-bus locks to protect target, device and periphs reference
counters. It allows to create and destroy paths unrelated to other locks
in any possible context.



Sounds very good! I assume you have tested USB mass storage device unplug 
during various file system operations?


It does indeed sound like some very good work, thanks Alexander!

@Hans if USB mass storage device unplug is something important to you
then might I suggest its a good idea to grab the patches, run your own
tests and report any issues you might find as I'm sure this would be
most appreciated :)

   Regards
   Steve 




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [PATCH] Add 4K QUIRK for Intel X25-M, MARVELL SD88SA02 and OCZ Agility 2

2013-08-14 Thread Steven Hartland
- Original Message - 
From: Xin Li delp...@delphij.net




-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 8/11/13 12:03 PM, Richard Yao wrote:
Is there any possibility that these quirks could be added to the 
upcoming 9.2 release?


I think so but we need to act fast.


Pools composed of affected disks will suffer from performance
issues until they are reformatted with the correct block size. In
addition, a certain benchmarking site, whose benchmarks are often
fodder for trolls, uses the Intel X25-M in its benchmarks. Adding
the quirk to 9.2 would eliminate an unfair handicap on FreeBSD from
their next set of benchmarks.


Actually I'm not 100% sure here: the current FreeBSD ZFS code does not
handle stripesize at all, and thus if you create a pool with the
patch, they will still be ashift=9.  Justin (gibbs@) have a patchset
to address this by the way, but since it's not yet in HEAD I wouldn't
expect it be incorporated in 9.2...


Indeed, there is nothing which really makes proper use of QUIRKS that
I'm aware of and ZFS definitely doesn't.

There's a number of patches around which correct that, I've one here
too, but none as yet are a total solution to the ashift issue.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [PATCH] Add 4K QUIRK for Intel X25-M, MARVELL SD88SA02 and OCZ Agility 2

2013-08-13 Thread Steven Hartland

Could you provide the output from camcontrol identify for these
disks I want to just double check the formatting before 
commiting as I don't have these disks here in labs.


   Regards
   Steve

- Original Message - 
From: Richard Yao r...@gentoo.org

To: hack...@freebsd.org
Cc: Richard Yao r...@gentoo.org; Eitan Adler ead...@freebsd.org
Sent: Sunday, August 11, 2013 1:12 PM
Subject: [PATCH] Add 4K QUIRK for Intel X25-M, MARVELL SD88SA02 and OCZ Agility 
2



Signed-off-by: Richard Yao r...@gentoo.org
---
sys/cam/ata/ata_da.c   | 24 
sys/cam/scsi/scsi_da.c | 24 
2 files changed, 48 insertions(+)

diff --git a/sys/cam/ata/ata_da.c b/sys/cam/ata/ata_da.c
index f201231..b7f293d 100644
--- a/sys/cam/ata/ata_da.c
+++ b/sys/cam/ata/ata_da.c
@@ -349,6 +349,14 @@ static struct ada_quirk_entry ada_quirk_table[] =
 },
 {
 /*
+ * Intel X25-M Series SSDs
+ * 4k optimised  trim only works in 4k requests + 4k aligned
+ */
+ { T_DIRECT, SIP_MEDIA_FIXED, *, INTEL SSDSA2M*, * },
+ /*quirks*/ADA_Q_4K
+ },
+ {
+ /*
 * Kingston E100 Series SSDs
 * 4k optimised  trim only works in 4k requests + 4k aligned
 */
@@ -365,6 +373,22 @@ static struct ada_quirk_entry ada_quirk_table[] =
 },
 {
 /*
+ * Marvell SSD (entry taken from Open Solaris)
+ * 4k optimised  trim only works in 4k requests + 4k aligned
+ */
+ { T_DIRECT, SIP_MEDIA_FIXED, *, MARVELL SD88SA02*, * },
+ /*quirks*/ADA_Q_4K
+ },
+ {
+ /*
+ * OCZ Agility 2 SSDs
+ * 4k optimised  trim only works in 4k requests + 4k aligned
+ */
+ { T_DIRECT, SIP_MEDIA_FIXED, *, OCZ-AGILITY2*, * },
+ /*quirks*/ADA_Q_4K
+ },
+ {
+ /*
 * OCZ Agility 3 SSDs
 * 4k optimised  trim only works in 4k requests + 4k aligned
 */
diff --git a/sys/cam/scsi/scsi_da.c b/sys/cam/scsi/scsi_da.c
index 617afbd..df895be 100644
--- a/sys/cam/scsi/scsi_da.c
+++ b/sys/cam/scsi/scsi_da.c
@@ -1008,6 +1008,14 @@ static struct da_quirk_entry da_quirk_table[] =
 },
 {
 /*
+ * Intel X25-M Series SSDs
+ * 4k optimised  trim only works in 4k requests + 4k aligned
+ */
+ { T_DIRECT, SIP_MEDIA_FIXED, *, INTEL SSDSA2M*, * },
+ /*quirks*/ADA_Q_4K
+ },
+ {
+ /*
 * Kingston E100 Series SSDs
 * 4k optimised  trim only works in 4k requests + 4k aligned
 */
@@ -1024,6 +1032,22 @@ static struct da_quirk_entry da_quirk_table[] =
 },
 {
 /*
+ * Marvell SSD (entry taken from Open Solaris)
+ * 4k optimised  trim only works in 4k requests + 4k aligned
+ */
+ { T_DIRECT, SIP_MEDIA_FIXED, *, MARVELL SD88SA02*, * },
+ /*quirks*/ADA_Q_4K
+ },
+ {
+ /*
+ * OCZ Agility 2 SSDs
+ * 4k optimised  trim only works in 4k requests + 4k aligned
+ */
+ { T_DIRECT, SIP_MEDIA_FIXED, *, OCZ-AGILITY2*, * },
+ /*quirks*/ADA_Q_4K
+ },
+ {
+ /*
 * OCZ Agility 3 SSDs
 * 4k optimised  trim only works in 4k requests + 4k aligned
 */
--
1.8.1.5

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [PATCH] Add 4K QUIRK for Intel X25-M, MARVELL SD88SA02 and OCZ Agility 2

2013-08-11 Thread Steven Hartland

Thanks Richard I'll commit these when I get in the office next week.

   Regards
   Steve
- Original Message - 
From: Richard Yao r...@gentoo.org

To: hack...@freebsd.org
Cc: Richard Yao r...@gentoo.org; Eitan Adler ead...@freebsd.org
Sent: Sunday, August 11, 2013 1:12 PM
Subject: [PATCH] Add 4K QUIRK for Intel X25-M, MARVELL SD88SA02 and OCZ Agility 
2



Signed-off-by: Richard Yao r...@gentoo.org
---
sys/cam/ata/ata_da.c   | 24 
sys/cam/scsi/scsi_da.c | 24 
2 files changed, 48 insertions(+)

diff --git a/sys/cam/ata/ata_da.c b/sys/cam/ata/ata_da.c
index f201231..b7f293d 100644
--- a/sys/cam/ata/ata_da.c
+++ b/sys/cam/ata/ata_da.c
@@ -349,6 +349,14 @@ static struct ada_quirk_entry ada_quirk_table[] =
 },
 {
 /*
+ * Intel X25-M Series SSDs
+ * 4k optimised  trim only works in 4k requests + 4k aligned
+ */
+ { T_DIRECT, SIP_MEDIA_FIXED, *, INTEL SSDSA2M*, * },
+ /*quirks*/ADA_Q_4K
+ },
+ {
+ /*
 * Kingston E100 Series SSDs
 * 4k optimised  trim only works in 4k requests + 4k aligned
 */
@@ -365,6 +373,22 @@ static struct ada_quirk_entry ada_quirk_table[] =
 },
 {
 /*
+ * Marvell SSD (entry taken from Open Solaris)
+ * 4k optimised  trim only works in 4k requests + 4k aligned
+ */
+ { T_DIRECT, SIP_MEDIA_FIXED, *, MARVELL SD88SA02*, * },
+ /*quirks*/ADA_Q_4K
+ },
+ {
+ /*
+ * OCZ Agility 2 SSDs
+ * 4k optimised  trim only works in 4k requests + 4k aligned
+ */
+ { T_DIRECT, SIP_MEDIA_FIXED, *, OCZ-AGILITY2*, * },
+ /*quirks*/ADA_Q_4K
+ },
+ {
+ /*
 * OCZ Agility 3 SSDs
 * 4k optimised  trim only works in 4k requests + 4k aligned
 */
diff --git a/sys/cam/scsi/scsi_da.c b/sys/cam/scsi/scsi_da.c
index 617afbd..df895be 100644
--- a/sys/cam/scsi/scsi_da.c
+++ b/sys/cam/scsi/scsi_da.c
@@ -1008,6 +1008,14 @@ static struct da_quirk_entry da_quirk_table[] =
 },
 {
 /*
+ * Intel X25-M Series SSDs
+ * 4k optimised  trim only works in 4k requests + 4k aligned
+ */
+ { T_DIRECT, SIP_MEDIA_FIXED, *, INTEL SSDSA2M*, * },
+ /*quirks*/ADA_Q_4K
+ },
+ {
+ /*
 * Kingston E100 Series SSDs
 * 4k optimised  trim only works in 4k requests + 4k aligned
 */
@@ -1024,6 +1032,22 @@ static struct da_quirk_entry da_quirk_table[] =
 },
 {
 /*
+ * Marvell SSD (entry taken from Open Solaris)
+ * 4k optimised  trim only works in 4k requests + 4k aligned
+ */
+ { T_DIRECT, SIP_MEDIA_FIXED, *, MARVELL SD88SA02*, * },
+ /*quirks*/ADA_Q_4K
+ },
+ {
+ /*
+ * OCZ Agility 2 SSDs
+ * 4k optimised  trim only works in 4k requests + 4k aligned
+ */
+ { T_DIRECT, SIP_MEDIA_FIXED, *, OCZ-AGILITY2*, * },
+ /*quirks*/ADA_Q_4K
+ },
+ {
+ /*
 * OCZ Agility 3 SSDs
 * 4k optimised  trim only works in 4k requests + 4k aligned
 */
--
1.8.1.5

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Make ZFS use the physical sector size when computing initial ashift

2013-07-10 Thread Steven Hartland

Hi DES, unfortunately you need a quite bit more than this to work compatibly.

I've had a patch here that does just this for quite some time but there's been 
some
discussion on how we want additional control over this so its not been commited.

If others are interested I've attached this as it achieves what we needed here 
so
may also be of use for others too.

There's also a big discussion on illumos about this very subject ATM so I'm
monitoring that too.

Hopefully there will be a nice conclusion come from that how people want to
proceed and we'll be able to get a change in that works for everyone.

   Regards
   Steve
- Original Message - 
From: Dag-Erling Smørgrav d...@des.no

To: freebsd...@freebsd.org; freebsd-hackers@freebsd.org
Cc: ivo...@freebsd.org
Sent: Wednesday, July 10, 2013 10:02 AM
Subject: Make ZFS use the physical sector size when computing initial ashift


The attached patch causes ZFS to base the minimum transfer size for a
new vdev on the GEOM provider's stripesize (physical sector size) rather
than sectorsize (logical sector size), provided that stripesize is a
power of two larger than sectorsize and smaller than or equal to
VDEV_PAD_SIZE.  This should eliminate the need for ivoras@'s gnop trick
when creating ZFS pools on Advanced Format drives.

DES
--
Dag-Erling Smørgrav - d...@des.no








___
freebsd...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to freebsd-fs-unsubscr...@freebsd.org 




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

zzz-zfs-ashift-fix.patch
Description: Binary data
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org

Re: Make ZFS use the physical sector size when computing initial ashift

2013-07-10 Thread Steven Hartland

There's lots more to consider when considering a way foward not least of all
ashift isn't a zpool configuration option is per top level vdev, space
consideration of moving from 512b to 4k, see previous and current discussions
on zfs-de...@freebsd.org and z...@lists.illumos.org for details.

   Regards
   Steve

- Original Message - 
From: Borja Marcos bor...@sarenet.es


On Jul 10, 2013, at 11:25 AM, Steven Hartland wrote:


If others are interested I've attached this as it achieves what we needed here 
so
may also be of use for others too.

There's also a big discussion on illumos about this very subject ATM so I'm
monitoring that too.

Hopefully there will be a nice conclusion come from that how people want to
proceed and we'll be able to get a change in that works for everyone.


Hmm. I wonder if the simplest approach would be the better. I mean, adding a 
flag to zpool.

At home I have a playground FreeBSD machine with a ZFS zmirror, and, you 
guessed it, I was
careless when I purchased the components, I asked for two 1 TB drives and 
that I got, but different
models, one of them advanced format and the other one classic.

I don't think it's that bad to create a pool on a classic disk using 4 KB 
blocks, and it's quite likely that
replacement disks will be 4 KB in the near future. 


Also, if you use SSDs the situation is similar.



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Make ZFS use the physical sector size when computing initial ashift

2013-07-10 Thread Steven Hartland


- Original Message - 
From: Xin Li 


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 07/10/13 02:02, Dag-Erling Sm?rgrav wrote:

The attached patch causes ZFS to base the minimum transfer size for
a new vdev on the GEOM provider's stripesize (physical sector size)
rather than sectorsize (logical sector size), provided that
stripesize is a power of two larger than sectorsize and smaller
than or equal to VDEV_PAD_SIZE.  This should eliminate the need for
ivoras@'s gnop trick when creating ZFS pools on Advanced Format
drives.


I think there are multiple versions of this (I also have one[1]) but
the concern is that if one creates a pool with ashift=9, and now
ashift=12, the pool gets unimportable.  So there need a way to disable
this behavior.


I've tested my patch in all configurations I can think of including exported
ashift=9 pools being imported, all no issues.

For your example e.g.

# Create a 4K pool (min_create_ashift=4K, dev=512)
test:src sysctl vfs.zfs.min_create_ashift
vfs.zfs.min_create_ashift: 12
test:src mdconfig -a -t swap -s 128m -S 512 -u 0
test:src zpool create mdpool md0
test:src zdb mdpool | grep ashift
   ashift: 12
   ashift: 12

# Create a 512b pool (min_create_ashift=512, dev=512)
test:src zpool destroy mdpool
test:src sysctl vfs.zfs.min_create_ashift=9
vfs.zfs.min_create_ashift: 12 - 9
test:src zpool create mdpool md0 
test:src zdb mdpool | grep ashift

   ashift: 9
   ashift: 9

# Import a 512b pool (min_create_ashift=4K, dev=512)
test:src zpool export mdpool
test:src sysctl vfs.zfs.min_create_ashift=12
vfs.zfs.min_create_ashift: 9 - 12
test:src zpool import mdpool
test:src zdb mdpool | grep ashift
   ashift: 9
   ashift: 9

# Create a 4K pool (min_create_ashift=512, dev=4K)
test:src zpool destroy mdpool
test:src mdconfig -d -u 0
test:src mdconfig -a -t swap -s 128m -S 4096 -u 0   
test:src sysctl vfs.zfs.min_create_ashift=9

vfs.zfs.min_create_ashift: 12 - 9
test:src zpool create mdpool md0
test:src zdb mdpool | grep ashift
   ashift: 12
   ashift: 12

# Import a 4K pool (min_create_ashift=4K, dev=4K)
test:src zpool export mdpool
test:src sysctl vfs.zfs.min_create_ashift=12
vfs.zfs.min_create_ashift: 9 - 12
test:src zpool import mdpool
test:src zdb mdpool | grep ashift
   ashift: 12
   ashift: 12


Another thing (not really related to the automatic detection) is that
we need a way to manually override this setting from command line when
creating the pool, this is under active discussion at Illumos mailing
list right now.

[1]
https://github.com/trueos/trueos/commit/3d2e3a38faad8df4acf442b055c5e98ab873fb26


Yep has been on my list for a while, based on previous discussions on 
zfs-devel@. I've not had any time recently but I'm following the illumos

thread to see what conclusions they come to.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Make ZFS use the physical sector size when computing initial ashift

2013-07-10 Thread Steven Hartland
- Original Message - 
From: Justin T. Gibbs 

I'm sure lots of folks have some solution to this.  Here is an
old version of what we use at Spectra:

 http://people.freebsd.org/~gibbs/zfs_patches/zfs_auto_ashift.diff

The above patch is missing some cleanup that was motivated by my
discussions with George Wilson about this change in April.  I'll
dig that up later tonight.  Even if you don't read the full diff,
please read the included checkin comment since it explains the
motivation behind this particular solution.

This is on my list of things to upstream in the next week or so after
I add logic to the userspace tools to report whether or not the
TLVs in a pool are using an optimal allocation size.  This is only
possible if you actually make ZFS fully aware of logical, physical,
and the configured allocation size.  All of the other patches I've seen
just treat physical as logical.


Reading through your patch it seems that your logical_ashift equates to
the current ashift values which for geom devices is based off sectorsize
and your physical_ashift is based stripesize.

This is almost identical to the approach I used adding a desired ashift,
which equates to your physical_ashift, along side the standard ashift
i.e. required aka logical_ashift value :)

One issue I did spot in your patch is that you currently expose
zfs_max_auto_ashift as a sysctl but don't clamp its value which would
cause problems should a user configure values  13.

If your interested in the reason for this its explained in the comments in 
my version which does a very similar thing with validation.


   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Make ZFS use the physical sector size when computing initial ashift

2013-07-10 Thread Steven Hartland


- Original Message - 
From: Justin T. Gibbs

On Jul 10, 2013, at 1:06 PM, Steven Hartland wrote:
- Original Message - From: Justin T. Gibbs 

I'm sure lots of folks have some solution to this.  Here is an
old version of what we use at Spectra:
http://people.freebsd.org/~gibbs/zfs_patches/zfs_auto_ashift.diff
The above patch is missing some cleanup that was motivated by my
discussions with George Wilson about this change in April.  I'll
dig that up later tonight.  Even if you don't read the full diff,
please read the included checkin comment since it explains the
motivation behind this particular solution.

This is on my list of things to upstream in the next week or so after
I add logic to the userspace tools to report whether or not the
TLVs in a pool are using an optimal allocation size.  This is only
possible if you actually make ZFS fully aware of logical, physical,
and the configured allocation size.  All of the other patches I've seen
just treat physical as logical.


Reading through your patch it seems that your logical_ashift equates to
the current ashift values which for geom devices is based off sectorsize
and your physical_ashift is based stripesize.

This is almost identical to the approach I used adding a desired ashift,
which equates to your physical_ashift, along side the standard ashift
i.e. required aka logical_ashift value :)


Yes, the approaches are similar.  Our current version records the logical
access size in the vdev structure too, which might relate to the issue
below.

 One issue I did spot in your patch is that you currently expose
 zfs_max_auto_ashift as a sysctl but don't clamp its value which would
 cause problems should a user configure values  13.

I would expect the zio pipeline to simply insert an ashift aligned thunking
buffer for these operations, but I haven't tried going past an ashift of 13 in
my tests.  If it is an issue, it seems the restriction should be based on
logical access size, not optimal access size.


Yes with your methodology you'll only see the issue if zfs_max_auto_ashift
and physical_ashift are both  13, but this can be the case for example
on a RAID controller with large stripsize.

Looking back at my old patch it too suffers from the same issue along with
the current code base, but that would only happen if logical sector size
resulted in an ashift  13 which is going to be much less common ;-)

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Make ZFS use the physical sector size when computing initial ashift

2013-07-10 Thread Steven Hartland
- Original Message - 
From: Justin T. Gibbs

...

 One issue I did spot in your patch is that you currently expose
 zfs_max_auto_ashift as a sysctl but don't clamp its value which would
 cause problems should a user configure values  13.

I would expect the zio pipeline to simply insert an ashift aligned thunking
buffer for these operations, but I haven't tried going past an ashift of 13 in
my tests.  If it is an issue, it seems the restriction should be based on
logical access size, not optimal access size.


Yes with your methodology you'll only see the issue if zfs_max_auto_ashift
and physical_ashift are both  13, but this can be the case for example
on a RAID controller with large stripsize.


I'm not sure I follow.  logical_ashift is available in our latest code, as is 
the
physical_ashift.  But even without the logical_ashift, why doesn't the zio
pipeline properly thunk zio_phys_read() access based on the configured ashift?


When I looked at it, which was a long time ago now so please excuse me if
I'm a little rusty on the details, zio_phys_read() was working more luck than
judgement as the offsets passed in where calculated from a valid start + 
increment
based on the size of a structure within vdev_label_offset() with no ashift
logic applied that I cound find.

The result was pools created with large ashift's where unstable when I
tested.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: potential future proofing fix for aicasm build.

2013-05-02 Thread Steven Hartland

I just remembered I had an issue with aicasm when compiling our kernel that 
didn't
have ahc or ahd on an 8.3 box. The fix was to manually delete the kernel obj
directory before compiling after doing a full make buildworld.

For some reason there was some cruft left in there that running make buildkernel
wasn't cleaning out, could you be suffering from a similar issue?

- Original Message - 
From: Dimitry Andric dimi...@andric.com

To: Alfred Perlstein alf...@ixsystems.com
Cc: hack...@freebsd.org
Sent: Thursday, May 02, 2013 8:19 AM
Subject: Re: potential future proofing fix for aicasm build.



On May 1, 2013, at 18:44, Alfred Perlstein alf...@ixsystems.com wrote:

I took a shot at fixing this issue with building aicasm as part of 
buildkernel of an older 9.0 src on a machine running HEAD.

aicasm.o: In function `__getCurrentRuneLocale':  /usr/include/runetype.h:96: 
undefined reference to `_ThreadRuneLocale'


I don't understand this error message... It seems like a linker error, but it also seems to refer to an incorrect include file? 
Is this during linking or compiling?




The issue seems to be two-fold:

1) Paths are not fully set to pick up the bootstrap tools needed to build.


What do you mean, exactly?  In r230622 I explicitly set the PATH to ${BPATH}:${PATH}, which should be enough to pick up the 
bootstrap tools.  This is exactly the same path used to build the bootstrap-tools stage itself.  The kernel bootstrap tools 
(only aicasm, really) should be built by the host compiler, not the cross-tools compiler.




2) include files use the host's instead of the build trees.

The first problem is fixed by changing setting of PATH from ${BPATH}:${PATH} 
to ${TMPPATH}.

The second is fixed by using -nostdinc and setting strict include paths using 
-I directives to the compiler:

CFLAGS=-nostdinc -I${WORLDTMP}/usr/include -I. 
-I${KERNSRCDIR}/dev/aic7xxx/aicasm


I don't think this is correct, as aicasm should be compiled by the host compiler, and linked with the host libc.  So if you 
start including headers from the source directory, there will be a mismatch between what those headers declare, and what is 
available in the host libc.


___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org





This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: potential future proofing fix for aicasm build.

2013-05-02 Thread Steven Hartland
- Original Message - 
From: Dimitry Andric dimi...@andric.com




On 2013-05-02 09:47, Steven Hartland wrote:

I just remembered I had an issue with aicasm when compiling our kernel that 
didn't
have ahc or ahd on an 8.3 box. The fix was to manually delete the kernel obj
directory before compiling after doing a full make buildworld.

For some reason there was some cruft left in there that running make buildkernel
wasn't cleaning out, could you be suffering from a similar issue?


Well, a way to reproduce the problem would be nice.  I tried running
make buildkernel in a stable/9 tree on a 10.0 machine, and building
aicasm went just fine.  So it works for me...


I think what I did was the following:-
1. make buildkernel KERNCONF=MYCONF -j10 # failed due to aicasm
2. make buildworld -j10 # Realised I'd forgotten to build world
3. make buildkernel -j10 # still failed due to aicasm
4. rm -rf /usr/obj/usr/src/sys/MYCONF
5. make buildkernel -j10 # worked

So I think the mistake that triggered it was I didn't build world before
building kernel, however I've not tried reproducing.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: potential future proofing fix for aicasm build.

2013-05-02 Thread Steven Hartland
- Original Message - 
From: Dimitry Andric dimi...@andric.com




On 2013-05-02 11:54, Steven Hartland wrote:

From: Dimitry Andric dimi...@andric.com

...

Well, a way to reproduce the problem would be nice.  I tried running
make buildkernel in a stable/9 tree on a 10.0 machine, and building
aicasm went just fine.  So it works for me...


I think what I did was the following:-
1. make buildkernel KERNCONF=MYCONF -j10 # failed due to aicasm


And this is what I cannot reproduce.  Works just fine, either with or without 
-j.


Have just confirmed this reproduces the issue with a recent head source on 
8.3-RELEASE
machine.

1. rm -rf /usr/obj
2. make buildkernel KERNCONF=MYCONF -j10 # Fails with:-
cc1: warnings being treated as errors
aicasm_gram.c:1539: warning: no previous prototype for 'yyparse'
3. make buildworld
4. make buildkernel KERNCONF=MYCONF -j10 # Still fails with:-
cc1: warnings being treated as errors
aicasm_gram.c:1539: warning: no previous prototype for 'yyparse'

Fix is:
1. rm -rf /usr/obj/usr/src/sys/MYCONF
2. make buildkernel KERNCONF=MYCONF -j10 # Now works

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: potential future proofing fix for aicasm build.

2013-05-01 Thread Steven Hartland

I don't believe aicasm is actually needed if you don't have a driver
which requires e.g. ahd or ahc. It would be good to get that fixed too.

   Regards
   Steve

- Original Message - 
From: Alfred Perlstein bri...@mu.org

To: hack...@freebsd.org
Sent: Wednesday, May 01, 2013 5:46 PM
Subject: potential future proofing fix for aicasm build.




Hey folks,

I took a shot at fixing this issue with building aicasm as part of
buildkernel of an older 9.0 src on a machine running HEAD.

aicasm.o: In function `__getCurrentRuneLocale': 
/usr/include/runetype.h:96: undefined reference to `_ThreadRuneLocale'

The issue seems to be two-fold:

1) Paths are not fully set to pick up the bootstrap tools needed to build.
2) include files use the host's instead of the build trees.

The first problem is fixed by changing setting of PATH from
${BPATH}:${PATH} to ${TMPPATH}.

The second is fixed by using -nostdinc and setting strict include paths
using -I directives to the compiler:

CFLAGS=-nostdinc -I${WORLDTMP}/usr/include -I. 
-I${KERNSRCDIR}/dev/aic7xxx/aicasm



Can I get review on this patch?

https://gist.github.com/anonymous/5493734

Inline:

diff --git a/Makefile.inc1 b/Makefile.inc1
index e850cda..785e3180 100644
--- a/Makefile.inc1
+++ b/Makefile.inc1
@@ -830,17 +830,18 @@ buildkernel:
 @echo  stage 2.3: build tools
 @echo --
 cd ${KRNLOBJDIR}/${_kernel}; \
-   PATH=${BPATH}:${PATH} \
+   PATH=${TMPPATH} \
 MAKESRCPATH=${KERNSRCDIR}/dev/aic7xxx/aicasm \
-   ${MAKE} SSP_CFLAGS= -DNO_CPU_CFLAGS -DNO_CTF \
+   ${MAKE} SSP_CFLAGS= -DNO_CPU_CFLAGS -DNO_CTF 
CFLAGS=-nostdinc -I${WORLDTMP}/usr/include -I. -I${KERNSRCDIR}/dev/aic7xxx/aicasm \

 -f ${KERNSRCDIR}/dev/aic7xxx/aicasm/Makefile
  # XXX - Gratuitously builds aicasm in the ``makeoptions NO_MODULES'' case.
  .if !defined(MODULES_WITH_WORLD)  !defined(NO_MODULES)  
exists(${KERNSRCDIR}/modules)
  .for target in obj depend all
+   @echo  aicasm: ${target} 
 cd ${KERNSRCDIR}/modules/aic7xxx/aicasm; \
-   PATH=${BPATH}:${PATH} \
+   PATH=${TMPPATH} \
 MAKEOBJDIRPREFIX=${KRNLOBJDIR}/${_kernel}/modules \
-   ${MAKE} SSP_CFLAGS= -DNO_CPU_CFLAGS -DNO_CTF ${target}
+   ${MAKE} SSP_CFLAGS= -DNO_CPU_CFLAGS -DNO_CTF 
CFLAGS=-nostdinc -I${WORLDTMP}/usr/include -I. -I${KERNSRCDIR}/dev/aic7xxx/aicasm ${target}

  .endfor
  .endif
  .if !defined(NO_KERNELDEPEND)






___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org





This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Does any body agree ?

2013-03-09 Thread Steven Hartland


- Original Message - 
From: Mehmet Erol Sanliturk m.e.sanlit...@gmail.com

To: Adrian Chadd adr...@freebsd.org
Cc: freebsd-hackers@freebsd.org; ali mousa ali_mousa...@yahoo.com
Sent: Saturday, March 09, 2013 11:06 PM
Subject: Re: Does any body agree ?



On Sat, Mar 9, 2013 at 3:03 PM, Adrian Chadd adr...@freebsd.org wrote:


... you need to post a more useful/descrptive title and/or body in
your request. Most people won't look at the post. :-)



Adrian


On 9 March 2013 14:41, ali mousa ali_mousa...@yahoo.com wrote:
 Does any body agree ?
http://forums.freebsd.org/showthread.php?p=212423#post212423 ?


 Regards


 Ali
 _




Dear Adrian ,

You are really right , and correct .

Thank you very much .


You may wish to know I asked the ports security team if they where
working on this and they are. I've not seen it hit svn yet so hopefully
this will happen soon.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: IPMI console [Re: Chicken and egg, encrypted root FS on remote server]

2013-02-21 Thread Steven Hartland
- Original Message - 
From: Daniel O'Connor 

On 21/02/2013, at 9:06, Steven Hartland kill...@multiplay.co.uk wrote:

If I change the console redirect to com1, my screen stays blank.  Would
you perhaps know how to use com1 for redirect and connect to it using
ipmi-console (or ipmi-tool)?


We use the following on Supermicro servers works fine:-
http://blog.multiplay.co.uk/2012/12/freebsd-serial-over-lan/



Nice!

BTW do you know what flag 0x20 does for UART? 0x10 is documented but 0x20 is 
not.

I had a quick look at the code and AFAIK it doesn't do anything (on 9.1 anyway).

Actually at a guess I would say it's a hangover from sio(4) where 0x20 forced 
the
device in question to be the console.


According to the handbook, where I got the settings from, 0x20:
Forces this unit to be the console (unless there is another higher priority 
console),
regardless of the -h option discussed below. The flag 0x20 must be used 
together with
the 0x10 flag.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: IPMI console [Re: Chicken and egg, encrypted root FS on remote server]

2013-02-20 Thread Steven Hartland


- Original Message - 
From: Paul Schenkeveld free...@psconsult.nl

To: Daniel O'Connor docon...@gsoft.com.au
Cc: hack...@freebsd.org
Sent: Wednesday, February 20, 2013 8:31 PM
Subject: IPMI console [Re: Chicken and egg, encrypted root FS on remote server]



Hi Daniel,

On Wed, Feb 20, 2013 at 10:55:47PM +1030, Daniel O'Connor wrote:


On 20/02/2013, at 21:43, Paul Schenkeveld free...@psconsult.nl wrote:
 What about getting a remote console like HP's ILO or Dell's DRAC ?
 
 You get to login remotely, you can use some degree of access control... you can even remote boot.
 
 For new hardware I could indeed use this, the current hardware does not

 support remote console.
 
 I don't have experience with ILO nor DRAC but I do have experience with

 SuperMicro's KVM over LAN which does need a java client to run.  If I can
 enter the passphrase over ssh that would be better as I can use any device
 including a smartphone to dial in and enter the passphrase.


If you setup a serial console you don't need Java if you use ipmitool, eg
ipmitool -H remoteip -U ADMIN -I lanplus sol activate


Tried that with some Supermicro servers, the serial console allows me to
get into BIOS config and shows boot messages up to starting the kernel,
once the kernel starts output stops.

In the BIOS setup, console redirect defaults to com2 port which explains
why output stops after the loader passes control to the kernel.

BTW, ipmitool always gives me Info: cannot activate SOL payload with
encryption but ipmi-console (sysutils/freeipmi) works.

If I change the console redirect to com1, my screen stays blank.  Would
you perhaps know how to use com1 for redirect and connect to it using
ipmi-console (or ipmi-tool)?


We use the following on Supermicro servers works fine:-
http://blog.multiplay.co.uk/2012/12/freebsd-serial-over-lan/

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Looking for reviewers for patch that adds foreign disk support mfiutil

2013-02-19 Thread Steven Hartland
- Original Message - 
From: John Baldwin


Thanks for the feedback John appreciated, a couple of questions inline
below if you would be so kind.



On Sunday, February 17, 2013 1:06:40 pm Steven Hartland wrote:

Hi all I'm looking for someone to review the attached patch
to mfiutil which adds foreign disk support to mfiutil as
per:
http://www.freebsd.org/cgi/query-pr.cgi?pr=172091

Any and all feedback welcome :)


Some suggestions:

- Please stick with FreeBSD style, e.g. please use:

   if (foo == NULL)

 rather than:

   if (NULL == foo)

 I understand the reasons for the latter style (turn accidental assignments
 into compile errors) but I don't buy them because 1) modern compilers can
 already catch such things, but most importantly 2) it doesn't read
 correctly.  Above all else code should be readable, and one doesn't say
 if NULL the pointer is (unless one is Yoda), but if the pointer is NULL.


Acknowledged, I'm working through some old patches so need to bring them in
line, will check for others like this.


- Don't make dump_config() use a default prefix, just fix the existing call
 to dump_config() to pass in a prefix.


Done.


- Is dump_config() really the right choice for 'foreign config'?  It doesn't
 attempt to output things very pretty, and I think mfiutil's non-debug
 commands should aim to be human readable.


Will check this, just didn't want to re-invent the wheel ;-)


- This (human readable) is also why it doesn't include the opcode in the error
 message by default.  Sysadmins don't really care which opcode fails.  Maybe
 put that under '#ifdef DEBUG'?


Previously there was no information about what command failed, which made
the failure message kinda useless, so while debugging I added the opcode
to help me trace things.

In order to keep it more user friendly I'm thinking of using mfi_op_desc,
but this is currently only available under MFI_DEBUG from mfi_debug.c.

Would it be better to move this else where e.g. mfireg.h so human readable
command information is always available?


- mfireg.h should be kept in sync with the driver's version of that header, so
 don't reorder the enum's unless you are changing it to match what is in the
 device driver's mfireg.h.  In fact, mfiutil should probably be using the
 mfireg.h from sys/dev/mfi directly now that it is in the tree.  (mfiutil
 was originally developed outside of the tree as a standalone app)


There is only one mfireg.h and that is already in sys/dev/mfi :)


- Leaving out the 'MFI_DCMD_' prefix from the opcode description was
 intentional.  If you are ever fortunate enough to examine the manuals from
 LSI, they refer to the firmware commands as 'LD_CONFIG', etc.  (Maybe it's
 'MR_LD_CONFIG'?)  The MFI_DCMD_ prefix is specific to the FreeBSD driver.


Thanks for the info, changed.


- Please don't do assignments in declarations and leave a blank line between
 declarations and the bode of code.  Thus:

mfi_op_desc(...)
{
int i, num_ops;

num_ops = nitems(mfi_op_codes);
...

 (nitems() is nice to use when it is available as well)


Changed, this the case for constant initialisers too? e.g. is the
following incorrect or acceptable?
myfunc(...)
{
   int i = 0, j = 1;
...


- Reindent the call to mfi_ldprobe() if CFG_ADD or CFG_FOREIGN_IMPORT
 succeeds.


Nice spot, sorted.

   Regards
   Steve

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Looking for reviewers for patch that adds foreign disk support mfiutil

2013-02-17 Thread Steven Hartland

Hi all I'm looking for someone to review the attached patch
to mfiutil which adds foreign disk support to mfiutil as
per:
http://www.freebsd.org/cgi/query-pr.cgi?pr=172091

Any and all feedback welcome :)

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

z-mfi-foreign-timeout.patch
Description: Binary data
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org

Re: HDD write cache

2013-02-01 Thread Steven Hartland

Looking at the cam ata code I can't see how those values would do
anything after booting.

I suspect they should be loader only tunables with the current code
and cant be changed on the fly for a disk that's already been
registered.

Try setting the values in /boot/loader.conf if you haven't already?

You can check the actual status of the disk itself using:-
camcontrol identify ada0

   Regards
   Steve

- Original Message - 
From: Wojciech Puchar woj...@wojtek.tensor.gdynia.pl

To: freebsd-hackers@freebsd.org
Sent: Friday, February 01, 2013 2:26 PM
Subject: HDD write cache


after reading quite recent topics about disabling/enabling write cache, i 
tried to test in on desktop 3.5 drive


kern.cam.ada.write_cache: 1
kern.cam.ada.read_ahead: 1
kern.cam.ada.0.read_ahead: -1
kern.cam.ada.0.write_cache: -1

i tried writing 1 or 0 to kern.cam.ada.0.write_cache, and there were 
exactly no differences at all, and disk seems to do always write caching.


Does that drive lie and ignore commands or i do it wrong?

this is my disk.

ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: SAMSUNG HD501LJ CR100-10 ATA-8 SATA 2.x device
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 476940MB (976773168 512 byte sectors: 16H 63S/T 16383C)

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: HDD write cache

2013-02-01 Thread Steven Hartland

Investigating a bit more a device reset will also trigger the
change so after changing the value you can use camcontrol reset
to trigger the change to apply using e.g.
camcontrol reset 0:0:0

While this is stated in the man for ada its not obvious that
changing the sysctl without a reset won't work, so you might
want to raise a PR about it.

   Regards
   Steve

- Original Message - 
From: Steven Hartland 

Looking at the cam ata code I can't see how those values would do
anything after booting.

I suspect they should be loader only tunables with the current code
and cant be changed on the fly for a disk that's already been
registered.

Try setting the values in /boot/loader.conf if you haven't already?

You can check the actual status of the disk itself using:-
camcontrol identify ada0

   Regards
   Steve

- Original Message - 
From: Wojciech Puchar


after reading quite recent topics about disabling/enabling write cache, i 
tried to test in on desktop 3.5 drive


kern.cam.ada.write_cache: 1
kern.cam.ada.read_ahead: 1
kern.cam.ada.0.read_ahead: -1
kern.cam.ada.0.write_cache: -1

i tried writing 1 or 0 to kern.cam.ada.0.write_cache, and there were 
exactly no differences at all, and disk seems to do always write caching.


Does that drive lie and ignore commands or i do it wrong?

this is my disk.

ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: SAMSUNG HD501LJ CR100-10 ATA-8 SATA 2.x device
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 476940MB (976773168 512 byte sectors: 16H 63S/T 16383C)




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: disadvantages of running 8.3 kernel on freebsd 8.2 system

2013-01-17 Thread Steven Hartland
- Original Message - 
From: Devin Teske devin.te...@fisglobal.com

You're the perfect person to help us figure out why when we:

1. back-port mfi(4) from stable/8 into releng/8.3 (8.3-RELEASE-p5 at the time 
of back-port)

2. Succeed in getting 8.3 to boot on Thunderbolt card

3. mfiutil produces Inappropriate ioctl for device

Even after…

4. Recompiling mfiutil from stable/8 (albeit in a releng/8.3 build environment -- back ported headers applied for new macros 
even)


Any hints on where to go next to restore mfiutil access?


You also need to backport mfiutil too.

I have a patch set for 8.3 which include latest mfi driver, mfiutil and
a large set of mfi driver fixes, even head has some rather serious issues
although mainly around error handling on tbolt cards.

We're running this in production so believe its a good stable set.

If you would like the patch set just let me know.

   Regards
   Steve 




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: disadvantages of running 8.3 kernel on freebsd 8.2 system

2013-01-17 Thread Steven Hartland
- Original Message - 
From: dte...@freebsd.org

But I bet you're not sitting 88 units of Thunderbolt cards that don't work in 
8.3.

8.3 is also exhibiting major problems with the igb-based NICs on those same 88 
units.


Only effects igb init but might want to make sure you have r245334 back ported 
to
avoid memory leaks when mbuf clusters are exhausted.

8.3 version of the patch attached ;-) 


For reference not only does this prevent the nic initialising properly it can 
also hang
the boot process as when routing initialises route appears to trigger mbuf 
allocation
with wait and as mbufs are exhaused and not freed correctly this hangs forever.

This will happen on an untuned kernel if more than 2 igb nics are configured as
each igb requires 8k of mbuf clusters (1k per queue x 8 queues on a machine with
8 or more cores) and the default kern.ipc.nmbclusters is only 25600.

For clarity by configured I mean if the nic is initialised either by 
assigning an IP
or ifconfig igbX up the queues are not allocated if the nic is present but 
unused.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.
Fixed mbuf free when receive structures fail to allocate.

This prevents quad igb card on high core machines, without any nmbcluster or
igb queue tuning wedging the boot process if all nics are configured.
--- sys/dev/e1000/if_igb.c.orig 2013-01-10 21:44:03.017805977 +
+++ sys/dev/e1000/if_igb.c  2013-01-10 21:44:55.751355018 +
@@ -4335,8 +4335,8 @@
 * the rings that completed, the failing case will have
 * cleaned up for itself. 'i' is the endpoint.
 */
-   for (int j = 0; j  i; ++j) {
-   rxr = adapter-rx_rings[i];
+   for (int j = 0; j  i; ++j) {
+   rxr = adapter-rx_rings[j];
IGB_RX_LOCK(rxr);
igb_free_receive_ring(rxr);
IGB_RX_UNLOCK(rxr);
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org

Re: kgzip(1) is broken

2013-01-15 Thread Steven Hartland


- Original Message - 
From: dte...@freebsd.org

To: 'Ian Lepore' free...@damnhippie.dyndns.org
Cc: freebsd-hackers@freebsd.org; dte...@freebsd.org
Sent: Wednesday, January 16, 2013 12:56 AM
Subject: RE: kgzip(1) is broken






-Original Message-
From: Ian Lepore [mailto:free...@damnhippie.dyndns.org]
Sent: Tuesday, January 15, 2013 4:43 PM
To: Devin Teske
Cc: dte...@freebsd.org; freebsd-hackers@freebsd.org
Subject: RE: kgzip(1) is broken

On Tue, 2013-01-15 at 16:10 -0800, Devin Teske wrote:

  -Original Message-
  From: Devin Teske [mailto:devin.te...@fisglobal.com] On Behalf Of
  dte...@freebsd.org
  Sent: Tuesday, January 15, 2013 3:10 PM
  To: 'Ian Lepore'
  Cc: freebsd-hackers@freebsd.org; dte...@freebsd.org
  Subject: RE: kgzip(1) is broken
 
 
 
   -Original Message-
   From: Ian Lepore [mailto:free...@damnhippie.dyndns.org]
   Sent: Tuesday, January 15, 2013 3:05 PM
   To: dte...@freebsd.org
   Cc: freebsd-hackers@freebsd.org
   Subject: Re: kgzip(1) is broken
  
   On Tue, 2013-01-15 at 13:27 -0800, dte...@freebsd.org wrote:
Hello,
   
I have been sad of-late because kgzip(1) no longer produces a usable
 kernel.
   
All versions of 9.x suffer this.
   
And somewhere between 8.3-RELEASE-p1 and 8.3-RELEASE-p5 this
recently
   broke in
the 8.x series.
   
I haven't tried the 7 series lately, but if whatever is making the

rounds

  gets
MFC'd that far back, I expect the problem to percolate there too.
   
The symptom is that the machine reboots immediately and unexpectedly
the
   moment
the kernel is executed by the loader.
   
This is quite troubling and I am looking for someone to help find the
  culprit. I
don't know where to start looking.
  
   Here are some possible candidates from the things that were MFC'd to 8
   in that timeframe.  I haven't looked at what these do, they're just
   changes that affect files related to booting.
  
   r233211
   r233377
   r233469
   r234563
  
 
  Thanks Ian!
 
  I'll test each one individually to see if regressing any one (or all)
 addresses
  the problem.

 Progress...

 Looks like I found the culprit.

 Turns out it's a back-ported bxe(4) driver (back-ported from 9 -- where

kgzip

 seems to never work).

 I wonder why back-porting bxe(4) from stable/9 to releng/8.3 would cause
kgzip
 to produce non-working kernels.


Yeah, it'll be interesting to see how a device driver can lead to the
machine reboots immediately and unexpectedly the moment the kernel is
executed by the loader, which I took to mean before seeing the
copyright or anything.



Indeed... loader throws up the syms and upon execution *KABOOM* (screen goes
black and back to POST)

The copyright never appears.



 I'm emailing the maintainers (davidch + other Broadcom folk)



The current dossier is even more interesting... the back-ported driver (with
zero modifications mind you from stable/9 to stable/8) exhibits memory failures
(example below), and causes terminals to become wedged when attempting to (for
example) scp a file over an existing configured network (igb-based -- presumably
unrelated to bxe but in practice loading bxe causes igb to misbehave).

$ ifconfig bxe0 inet 192.168.1.5/24
bxe0: ../../../dev/bxe/if_bxe.c(10939): Memory allocation failure! Cannot fill
fp[00] RX chain.
bxe0: ../../../dev/bxe/if_bxe.c(3921): NIC initialization failed, aborting!
$ ifconfig bxe1 inet 192.168.1.6/24
bxe1: ../../../dev/bxe/if_bxe.c(10939): Memory allocation failure! Cannot fill
fp[00] RX chain.
bxe1: ../../../dev/bxe/if_bxe.c(3921): NIC initialization failed, aborting!

(as expected, also sent mail off to maintainers w/respect to above notes/errors)


Sounds like you may be out of mbufs which is easy, on a box with 4 igb's simply
booting without tuning with cause this so, if you have igb's and bxe's this
could be your cause.

Try adding the following to loader.conf and see if it helps:-
kern.ipc.nmbclusters=51200

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Threaded 6.4 code compiled under 9.0 uses a lot more memory?..

2012-10-30 Thread Steven Hartland


- Original Message - 
From: Karl Pielorz kpielorz_...@tdx.co.uk

To: freebsd-hackers@freebsd.org
Sent: Tuesday, October 30, 2012 11:12 AM
Subject: Threaded 6.4 code compiled under 9.0 uses a lot more memory?..




Hi All,

Can anyone think of any quick pointers as to why some code originally 
written under 6.4 amd64 - when re-compiled under 9.0-stable amd64 takes up 
a *lot* more memory when running?


The code involved is a sendmail Milter, and a TCP server type program (that 
runs up a large number of threads [~700] at startup).


Both were previously compiled with:

-O2 -pthread -lc_r

They're now compiled under 9.0-S with just:

-O2 -pthread


As an example, under 6.4 the size/res for one reported by top is 81Mb/48Mb 
- and for the other is 44Mb/9Mb [this is after it's been running for weeks].


Under 9.0-stable the initial memory used by the processes just after 
starting rises to 625Mb/128Mb and 519Mb/65Mb respectively.


Is that something I need to worry about?

They've not been running longing enough yet to see if anything is 'leaking' 
(i.e. if size/res continues to go up). Just thought I'd ask if there's a 
simple/possible explanation for this - and if it's something I need to 
worry about...


amd64 vs i386?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Looking for testers / feedback for ZFS recieve properties options

2012-09-30 Thread Steven Hartland

We encountered a problem receiving a full ZFS stream from
a disk we had backed up. The problem was the receive was
aborting due to quota being exceeded so I did some digging
around and found that Oracle ZFS now has -x and -o options
as documented here:-
http://docs.oracle.com/cd/E23824_01/html/821-1462/zfs-1m.html

Seems this has been raised as a feature request upstream:
https://www.illumos.org/issues/2745

Anyway being stuck with a backup we couldn't restore I had
a play a implementing these options and have a prototype up
and running, which I'd like feedback on.

This patch also adds a -l option which allows the streams to
be limited to those specified. Another option which I think
would be useful and seemed relatively painless to add.

The initial version of the patch which is based off
8.3-RELEASE can be found here:
http://blog.multiplay.co.uk/dropzone/freebsd/zfs-recv-properties.patch

Any feedback appreciated

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: AHCI Timeouts on SATA III with Intel 520 SSDs

2012-07-27 Thread Steven Hartland

Did you get anywhere with this?

Seeing a similar thing on some new Patsburg based machines with
KINGSTON SSD's on 8.3-RELEASE.

   Regards
   Steve

- Original Message - 
From: Caza, Aaron aaron.c...@ca.weatherford.com

To: freebsd-hackers@freebsd.org
Sent: Monday, February 13, 2012 9:58 PM
Subject: AHCI Timeouts on SATA III with Intel 520 SSDs


I've got a couple of Intel 520 SSDs that I'm running on an Intel Sandy-bridge based system(Core i5-2500K H67 chipset). 
Unfortunately, the drives experience AHCI Timeouts when connected to the SATA III ports.  If, however, I connect the drives to the 
SATA-II ports on the same system the drives do not timeout.  NCQ is enabled.  Below is the complete dmesg showing the issue.  For 
my testing, I'm just using a FreeBSD 9.0 Release (amd64) generic kernel using 'dd if=/dev/ada0 of=/dev/null bs=1m' to exhibit the 
behavior.  The drives, ofcourse, are brand new and again if I run them off the  SATA-II ports instead of the SATA-III ports the 
problem goes away but then so does the performance.


Suggestions?

gpart show ada0:
=   34  234441581  ada0  GPT  (111G)
34128 1  freebsd-boot  (64k)
   162  232783872 2  freebsd-ufs  (111G)
 2327840341657581- free -  (809M)

camcontrol identify ada0:
pass0: INTEL SSDSC2CW120A3 400i ATA-9 SATA 3.x device
pass0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)

protocol  ATA/ATAPI-9 SATA 3.x
device model  INTEL SSDSC2CW120A3
firmware revision 400i
serial number
WWN   5001517bb27d76f7
cylinders 16383
heads 16
sectors/track 63
sector size   logical 512, physical 512, offset 0
LBA supported 234441648 sectors
LBA48 supported   234441648 sectors
PIO supported PIO4
DMA supported WDMA2 UDMA6
media RPM non-rotating

Feature  Support  Enabled   Value   Vendor
read ahead yes  yes
write cacheyes  yes
flush cacheyes  yes
overlapno
Tagged Command Queuing (TCQ)   no   no
Native Command Queuing (NCQ)   yes32 tags
SMART  yes  yes
microcode download yes  yes
security   yes  no
power management   yes  yes
advanced power management  yes  yes   254/0xFE
automatic acoustic management  no   no
media status notification  no   no
power-up in Standbyyes  no
write-read-verify  no   no
unload yes  yes
free-fall  no   no
data set management (TRIM) yes

dmesg:
Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
 The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.0-RELEASE #0: Tue Jan  3 07:46:30 UTC 2012
   r...@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64
CPU: Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz (3292.59-MHz K8-class CPU)
 Origin = GenuineIntel  Id = 0x206a7  Family = 6  Model = 2a  Stepping = 7
 
Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE
 
Features2=0x179ae3bfSSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,TSCDLT,AESNI,XSAVE,AVX
 AMD Features=0x28100800SYSCALL,NX,RDTSCP,LM
 AMD Features2=0x1LAHF
 TSC: P-state invariant, performance statistics
real memory  = 17179869184 (16384 MB)
avail memory = 16459304960 (15696 MB)
Event timer LAPIC quality 600
ACPI APIC Table: Shuttl Shuttle 
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
cpu0 (BSP): APIC ID:  0
cpu1 (AP): APIC ID:  2
cpu2 (AP): APIC ID:  4
cpu3 (AP): APIC ID:  6
ioapic0 Version 2.0 irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: Shuttl Shuttle on motherboard
acpi0: Power Button (fixed)
Timecounter ACPI-fast frequency 3579545 Hz quality 900
acpi_timer0: 24-bit timer at 3.579545MHz port 0x408-0x40b on acpi0
cpu0: ACPI CPU on acpi0
cpu1: ACPI CPU on acpi0
cpu2: ACPI CPU on acpi0
cpu3: ACPI CPU on acpi0
pcib0: ACPI Host-PCI bridge port 0xcf8-0xcff on acpi0
pci0: ACPI PCI bus on pcib0
pcib1: ACPI PCI-PCI bridge irq 16 at device 1.0 on pci0
pci1: ACPI PCI bus on pcib1
vgapci0: VGA-compatible display port 0xf000-0xf03f mem 
0xfe00-0xfe3f,0xc000-0xcfff irq 16 at device 2.0 on pci0
pci0: simple comms at device 22.0 (no driver attached)
ehci0: EHCI (generic) USB 2.0 controller mem 0xfe603000-0xfe6033ff irq 16 at 
device 26.0 on pci0
usbus0: EHCI version 1.0
usbus0: EHCI (generic) USB 2.0 controller on ehci0
pcib2: ACPI PCI-PCI bridge irq 17 at device 28.0 on pci0
pci2: ACPI PCI bus on pcib2
pcib3: ACPI PCI-PCI bridge irq 16 at device 28.1 on pci0
pci3: ACPI PCI bus on pcib3
xhci0: XHCI (generic) USB 3.0 

Re: AHCI Timeouts on SATA III with Intel 520 SSDs

2012-07-27 Thread Steven Hartland

That's very useful and could well be what we have here.

I've got two new machines which run the SSD's fine @ SATA
3 on port 1 but an identical disk on port 2 is having issues.

If we switch it down to SATA 2 and all is good. So sounds
like the next move is to switch the disks round and see if
the problem stays with the disk or the port.

Here's hoping its the disk or the cable and not the
controller.

   Regards
   Steve
- Original Message - 
From: Caza, Aaron aaron.c...@ca.weatherford.com

To: Steven Hartland kill...@multiplay.co.uk; freebsd-hackers@freebsd.org
Sent: Friday, July 27, 2012 5:07 PM
Subject: RE: AHCI Timeouts on SATA III with Intel 520 SSDs


Yes.  In my case, the problem turned out to be a marginal SATA-III port on the motherboard which was determined after swapping 
SSDs, SATA cables, etcetera to finally pin down the problem.  When trouble-shooting this issue, I recall googling a particular 
missive by Alexander Motion in which he indicates these problems are potentially due to any number of hardware-related reasons 
hence the exhaustive search for the culprit which, as he suggested, did indeed turn out to be the hardware.It's actually 
rather interesting how borderline hardware can be - the port in question could handle an Intel 510 SSD running at full SATA-III 
speed but an Intel 520 pushed it over the brink.




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: geom - cam disk

2012-07-25 Thread Steven Hartland
- Original Message - 
From: Andriy Gapon a...@freebsd.org



on 26/07/2012 01:08 Alexander Motin said the following:

Different controllers have different command queueing limitations. If you are
testing with ahci(4) driver and modern disks, then their 32 command slots per
port can be enough for many workloads to enqueue all commands to the hardware
and leave queue empty as you've described. But if you take harder workload, or
controller/ device without command queueing support, extra requests will be
accumulated on that bioq and sorted there.


Alexander,

thank you for the reply.
Indeed, using 64 parallel dd processes with bs=512 I was able to 'kick in' the
disksort logic.  But I am not sure if the disksort algorithm makes much
difference in this case given the number of commands that a disk firmware can
internally re-order.  (Not mentioning that potentially disksort could starve
some I/O bound processes in favor of others -- but that's a totally different
topic).

But then, of course, for the less capable hardware the disksort could still be a
significant factor.


The sort is actually important for delete requests too as this can allow
the delete processing code to operate more effectively which can result in
significant performance increases if this then allows request combining.

For example Alexander is currently reviewing some changes I've written to the
delete processing which include an optimisation that increases SSD delete
performance from ~630MB/s to 1.3GB/s on 3rd gen sandforce controllers.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


dtrace crashes on ustack

2012-06-06 Thread Steven Hartland

When running dtrace looking at syscall::write:return for a java
app with a ustack call for the first few calls I get:-
 dtrace: ERROR: gelf_getehdr() failed: No error: 0

And then after a number dtrace crashes:-

#0  0x000800fcc59d in _malloc_prefork () from /lib/libc.so.7
[New Thread 80283c500 (LWP 100815/dtrace)]
[New Thread 8012041c0 (LWP 101294/initial thread)]
(gdb) bt
#0  0x000800fcc59d in _malloc_prefork () from /lib/libc.so.7
#1  0x000800fcdd2d in _malloc_thread_cleanup () from /lib/libc.so.7
#2  0x00080066190e in pthread_exit () from /lib/libthr.so.3
#3  0x000800658523 in pthread_getprio () from /lib/libthr.so.3
#4  0x in ?? ()
Cannot access memory at address 0x7fbff000

Kernel is compiled with dtrace, world has been rebuild with
WITH_CTF=1 as has openjdk7 port.

Any ideas?

   Regards
   Steve



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


dtrace filename lookups from fd

2012-06-01 Thread Steven Hartland

As a first foray into dtrace I wanted to create a little
script which shows the amount of disk read / write activity.

Now the DtraceToolkit includes rwsnoop but this uses Solaris
specific requests and on looking around it seems like
using rwsnoop vn_fullpath may be the way to go.

Has anyone done anything similar to this before or has any
tips on going about this?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: proper newfs options for SSD disk

2012-05-25 Thread Steven Hartland
- Original Message - 
From: Wojciech Puchar woj...@wojtek.tensor.gdynia.pl




On 2012-May-18 22:54:43 +0200, Dimitry Andric d...@freebsd.org wrote:

Be sure to use -t enable when creating the filesystem:


Only if your SSD supports TRIM.  Some consumer-grade SSDs don't and
get very confused if sent TRIM commands.


mine do.


The disk also has be be connected to a disk arch which supports
BIO_DELETE which ATM is only ata unless your running HEAD which
also has support in da

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Ways to promote FreeBSD?

2012-04-27 Thread Steven Hartland
- Original Message - 
From: Mehmet Erol Sanliturk



My opinion is that most important obstacle in front of FreeBSD is its
installation structure :


It is NOT possible to install and use a FreeBSD distribution directly as it
is .


I disagree, we find quite the opposite; FreeBSD's current install is perfect
its quick, doesn't install stuff we don't need and leaves a very nice base.

Linux on the other had takes ages, asks way to many questions, has issues
with some hardware with mouse and gui not work properly making the
install difficult to navigate, but most import its quite hard to get a nice 
simple
base as there are so many options, which is default with FreeBSD.

In essence it depends on what you want and how you use the OS. For
the way we use FreeBSD on our servers its perfect. So if your trying
to suggest its not suitable for all that's is incorrect as it depends on what
you want :)

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: 8 to 9: A longer wait early in the boot of a (damaged) CompaqPresario

2012-02-16 Thread Steven Hartland
- Original Message - 
From: Alex Goncharov alex-goncha...@comcast.net



About a week ago, I made a jump and upgraded the system's FreeBSD from
version 8 to 9.  Everything is great (I am typing this message on that
machine now) but the boot pause after the (looking new in 9) boot menu
is *much* longer now -- it will show the '\' character and wait for,
subjectively, half a minute before putting anything else on the
screen.


Two things spring to mind which could help:
1. The reduce the slice sampling size in sys/boot/zfs/zfs.c which was
increased recently
2. disable boot time mem tests using the attached patch

Patches for both from 8.2-RELEASE attached. For #2 you also need the
following in /boot/loader.conf
hw.memtest.tests=0

   Regards
   Steve




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.
--- sys/amd64/amd64/machdep.c   2011/06/08 08:12:15 222853
+++ sys/amd64/amd64/machdep.c   2011/07/30 13:33:05 224516
@@ -1309,7 +1309,7 @@
{
int i, physmap_idx, pa_indx, da_indx;
vm_paddr_t pa, physmap[PHYSMAP_SIZE];
-   u_long physmem_tunable;
+   u_long physmem_tunable, memtest, tmpul;
pt_entry_t *pte;
struct bios_smap *smapbase, *smap, *smapend;
u_int32_t smapsize;
@@ -1372,6 +1372,14 @@
Maxmem = atop(physmem_tunable);

/*
+* By default keep the memtest enabled.  Use a general name so that
+* one could eventually do more with the code than just disable it.
+*/
+   memtest = 1;
+   if (TUNABLE_ULONG_FETCH(hw.memtest.tests, tmpul))
+   memtest = tmpul;
+
+   /*
 * Don't allow MAXMEM or hw.physmem to extend the amount of memory
 * in the system.
 */
@@ -1433,6 +1441,8 @@
goto do_dump_avail;

page_bad = FALSE;
+   if (memtest == 0)
+   goto skip_memtest;

/*
 * map page into kernel: valid, read/write,non-cacheable
@@ -1470,6 +1480,7 @@
 */
*(int *)ptr = tmp;

+skip_memtest:
/*
 * Adjust array of valid/good pages.
 */

--- sys/boot/zfs/zfs.c.orig 2011-10-20 18:15:29.966685430 +
+++ sys/boot/zfs/zfs.c  2011-10-20 18:18:22.291033636 +
@@ -45,6 +45,12 @@

#include zfsimpl.c

+/*
+ * For GPT this should be 128 but leads to 50+ second delay in BTX loader so
+ * we use the original 4 pre r198420 by default for the boot process
+ */
+#define ZFS_MAX_SLICES 4
+
static int  zfs_open(const char *path, struct open_file *f);
static int  zfs_write(struct open_file *f, void *buf, size_t size, size_t 
*resid);
static int  zfs_close(struct open_file *f);
@@ -415,7 +421,7 @@
if (vdev_probe(vdev_read, (void*) (uintptr_t) fd, 0))
close(fd);

-   for (slice = 1; slice = 128; slice++) {
+   for (slice = 1; slice = ZFS_MAX_SLICES; slice++) {
sprintf(devname, disk%dp%d:, unit, slice);
fd = open(devname, O_RDONLY);
if (fd == -1) {
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org

Re: FreeBSD has serious problems with focus, longevity, and lifecycle

2012-01-17 Thread Steven Hartland
- Original Message - 
From: John Kozubik j...@kozubik.com


It's amazing how many people are in the exact same boats - waiting for 
8.3, getting locked out of new motherboards because em(4) can't be 
backported to even the production release...


This is not true, only last week did we take the version of e1000 from
HEAD into our 8.2-RELEASE tree as a patch. It wasnt totally trivial but
it also wasnt difficult either. But it would still be preferable for
many not to have to do this I assume?

This import brings the number of number release patches we manually apply
to our machines above 8.2-RELEASE to 18 which includes:-
updated areca driver, boot time fixes (disable memtest), devfs startup fix
ixgbe  e100 drivers, libz assembly crash fix, mps driver import, rc.subr
fix for scripts, increased max swap size, tcp reassembly fix, udp6 fix
for java, cam timeout fixes, zfs overflow fix, zfs slice boot delay,
camcontrol security options for ssds and jail uref panic fix.

I'm sure there are more that others would include but these changes are
important enough to our environment to prompt their inclusion in what
is effectively our own stream of 8.2-RELEASE.

Now I know the factastic work commiters do in bring us FreeBSD so
I can't bring myself to critisise in anyway the work they do. But its
defintiely interesting to see others are in the same boat as us looking
for something thats a bit more predicable than the current large change
releases.

I wonder if there is something that could be created to make maintaining
micro branches easier. I know mfsbsd has made our lives so much simpler
since we discovered it allowing us to take a standard source build on
one machine and roll an install cd customised to our requirements in
minutes.

What other undiscovered gems are our out there?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: FreeBSD has serious problems with focus, longevity, and lifecycle

2012-01-17 Thread Steven Hartland


- Original Message - 
From: Hans Petter Selasky hsela...@c2i.net




On Tuesday 17 January 2012 20:54:51 Steven Hartland wrote:

boot time fixes (disable memtest),


Hi,

Another noticeable part is that ufsread.c in boot2 uses very small block sizes 
to read the file system data. If that could be fixed boot times would drop too 
!


It might but not the same order of magnitude I suspect
as this was like 30-60+ delay depending on memory size ;-)

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: FreeBSD has serious problems with focus, longevity, and lifecycle

2012-01-16 Thread Steven Hartland


- Original Message - 
From: John Kozubik j...@kozubik.com

To: freebsd-hackers@freebsd.org
Sent: Monday, January 16, 2012 10:28 PM
Subject: FreeBSD has serious problems with focus, longevity, and lifecycle




Friends,

I was disappointed to see that 8.3-RELEASE is now slated to come out in 
March of 2012.  This will be ~13 months since 8.2-RELEASE and is typical 
of a trend towards longer gaps between minor releases.


...

I must say as a small company that runs ~200 machines on FreeBSD
I do see where John is coming from, as it is very time consuming to keep
things up to date and new is not always better e.g. we still have boxes
stuck on 6.x as issues introduced in the Linux compat after that caused
problems.

That said I'm in two minds as the features that have been brought in by
the more rapid dev cycle like ZFS have been great.

Where I do see an issue is where it feels like we've just got to a solid
8.2 release with p6 and some addition patches we see things like em driver
updates required to run newer hardware only in 9.

While we might like to push everything to 9 it brings with it a large
amount of untested changes like the HPN patches to core ssh which we
have seen problems with under openssh-portable when tested.

So this puts us in a dilemma, push to 9 and keep up to date or stick
with 8.2 with custom patches while we wait for 8.3 which we know is
good and assuming it has the patches need included in it?

The correct answer for us is currently unknown and is still being
debated, but in the mean time we are going to keep with 8.2 until
we've had the chance to fully evaluate what 9 has to offer.

   Regards
   Steve




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Invalid memory stats from vmstat and sysctl vm.vmtotal?

2011-12-02 Thread Steven Hartland


- Original Message - 
From: Jason Hellenthal jh...@dataix.net

Just to put some visuals to this...

.
`-- DIE
   |-- Core1  [Idle]
   |-- Core2  [35% ]
   |   `-- thread127
   |-- Core3  [40% ]
   |   `-- thread127
   `-- Core4  [100%]
   `-- thread127

In this case you would say the DIE should be at a total of 175% ?


I think your getting confused there; it sounds like your referring to
a single CPU capable of multiple tasks via either multiple cores or
HTT as an UP machine? If so that's your problem this isn't UP it SMP.

Have a look on your machine in /var/run/dmesg.boot if your see
it reporting more than one core then your SMP not UP hence the
confusion as each of these cores be they real, virtual or physical
represents a possible thread of 100% so even if you have a single
physical CPU with 4 cores that still represents a possible total
of 400%

e.g the following shows a machine capable of 1600% if all cores
are busy.

FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs
FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 SMT threads
cpu0 (BSP): APIC ID:  0
cpu1 (AP): APIC ID:  1
cpu2 (AP): APIC ID:  2
cpu3 (AP): APIC ID:  3
cpu4 (AP): APIC ID:  4
cpu5 (AP): APIC ID:  5
cpu6 (AP): APIC ID:  6
cpu7 (AP): APIC ID:  7
cpu8 (AP): APIC ID: 16
cpu9 (AP): APIC ID: 17
cpu10 (AP): APIC ID: 18
cpu11 (AP): APIC ID: 19
cpu12 (AP): APIC ID: 20
cpu13 (AP): APIC ID: 21
cpu14 (AP): APIC ID: 22
cpu15 (AP): APIC ID: 23

If you want proper UP which will total 100% you could remove
SMP from your kernel but I wouldnt advise that ;-)

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Invalid memory stats from vmstat and sysctl vm.vmtotal?

2011-12-02 Thread Steven Hartland
- Original Message - 
From: Andriy Gapon a...@freebsd.org



Totalling up RSS from ps axo rss gives a total in the region of that if
the vm stats are out by a factor of 4, in this case it should be: 8132557
which is 7.75GB a much more realistic value.

Am I totally missing something or is there problem here?


Likely more of the former than of latter.  Those virtual sizes are not
sufficiently explained, but you have been warned that those are not physical
sizes, so I am not sure why you try to compare the virtual figures with the
physical figures.


My miss-understanding was due to what virtual actually meant.


Here's an example.  Let' say you mmap-ed a 1GB file into a process memory space,
here you immediately increased your virtual size counts by 1GB, even if you 
hadn't
accessed any bytes in the file yet and so none of them were in physical memory.
The same applies to anonymous memory.

P.S. the above is reveled by a cursory look through the code (which is publicly
available btw) :-)


Yer I did have a dig around before posting and ended up the code for vm.vmtotal,
which is where vmstat gets its info from but that's just a summation of each 
object's
size from vm_object_list. Thats where I got lost without an insight into what
a vm_object.size actually represents.

Your info about mmap'ed files helped point me in the right direction as it 
identified space that shows as virtual but doesn't show in swap or real ram,

which is what I was missing.

Given this starting point the following links provided me with addtional
information:-
http://www.freebsd.org/doc/en/books/arch-handbook/vm.html
http://www.freebsd.org/doc/en/books/design-44bsd/overview-memory-management.html
http://www.freebsd.org/doc/en_US.ISO8859-1/articles/vm-design/
http://www.cse.chalmers.se/edu/course/EDA203/unix4.pdf

I was under the incorrect impression that Virtual Memory (VM) was so named as it
was a unified physical memory and swap (virtual memory), but its not that 
simple,
as other items such as file-backed objects also count to this total which would
never show in physical or swap allocation of other tools such as top and 
swapinfo.

So what I believe is now the big cause of virtual memory uplift vs the memory
totals shown by ps / top is that the vm totals include things like file backed
memory mapped process binaries, shared libs etc many multiple times.

This would explain why this specific machine shows the applification more than
others here as it runs thousands of very small lightweight processes.

Thanks for pointer Andy, I now understand a lot more about the BSD VMS :)

What do people think about expanding that entry in the man page of vmstat to
clarify just what active virtual pages really means?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Invalid memory stats from vmstat and sysctl vm.vmtotal?

2011-12-01 Thread Steven Hartland
- Original Message - 
From: Jason Hellenthal jh...@dataix.net



This goes along with the thoughts I had about 4 months ago tending to some
zfs statistics as well top showing greater than 100% actual CPU usage. This
is a big pet peave of mine. Its like saying you ate 134% of a bannanna when
in all reallity it is impossible. You can never have more than 100% usage of
anything and when seen is a clear notice that some math is considerably
incorrect leading to other such miscalculations to be performed. Things like
the above already have checks in place that ensure no boundries are being
crossed/overflowed or underrun but it surely makes processing results building
future products a bitch. One instance is the calculation of threads for example
firefox can be seen using upto or more 338% of the CPU. Thats impossible its
like saying anyones CPU grew by 400%.


I could understand a bit of overflow as stats are snapshots which may not
be instuntanious, but 31GB instead of under 8GB is hardly a rounding issue /
overflow.

With respect to top showing greater than 100% by how much are you talking?
Do your realise that each core = 100%? So if you have a quad core your system
total will be 400% not 100%?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Invalid memory stats from vmstat and sysctl vm.vmtotal?

2011-12-01 Thread Steven Hartland
- Original Message - 
From: Damien Fleuriot m...@my.gd



I could understand a bit of overflow as stats are snapshots which may not
be instuntanious, but 31GB instead of under 8GB is hardly a rounding
issue /
overflow.

With respect to top showing greater than 100% by how much are you talking?
Do your realise that each core = 100%? So if you have a quad core your
system
total will be 400% not 100%?



That's his point, you cannot use 400% of a system as a whole, his point
is that top should report 100% where each core accounts for 25%


Then I would have to disagree, keeping 100% to mean 100% of a single core
is much easier to manage than 100% of a machines total capacity.

If you went to 100% = the machine total capacity processes could be using
a lot of cpu without even registering 1% on today's machines where 24 cores
is common place.

It also makes detecting single process / thread bottlenecks easier as if
your seeing 100% you know its maxing a core, instead of having to calculate
it once you know how many cores the machine has.

If your looking for total machine usage then that's also their in the summary
at the top of the screen e.g.
CPU states: 13.6% user,  0.0% nice,  1.3% system,  0.0% interrupt, 85.1% idle

Anyway this is quite off topic, and I don't want to loose sight of the threads
goal which is to determine why we can see 31GB of usage on an 8GB machine
with very little shared memory usage an no swap usage.

   Regards
   Steve



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Invalid memory stats from vmstat and sysctl vm.vmtotal?

2011-12-01 Thread Steven Hartland


- Original Message - 
From: RW rwmailli...@googlemail.com

To: freebsd-hackers@freebsd.org
Sent: Thursday, December 01, 2011 1:58 PM
Subject: Re: Invalid memory stats from vmstat and sysctl vm.vmtotal?



On Wed, 30 Nov 2011 12:39:10 -
Steven Hartland wrote:


We're seeing some impossible memory usage stats reported on machines
here from vmstat and sysctl vm.vmtotal.

We have machines reporting to be using 31GB total when they only have
8GB physical and are not using any swap.

Here's an output from one of our machines:-
vmstat -c 2 -w 1 -n 0 
 procs  memory  page  faults cpu

 r b w avmfre   flt  re  pi  pofr  sr   in   sy   cs us
sy id 0 0 0  31768M  2112M   586   0   0   0   421   0  106  270
569  0  6 94 0 0 0  31768M  2112M 2   0   0   0 0   0  370
8139 3996  0  1 99

...
It looks like it may be out by a factor of 4, possibly due to the fact
the its a 4k page size not 1k as indicated by the vmstat man page:-



I don't think so, I have 16GB and tops shows:

Mem: 817M Active, 396M Inact, 1364M Wired, 82M Cache, 1282M Buf, 13G
Free

but  

vmstat -c 2 -w 1 -n 0 
procs  memory  page  faults cpu

r b w avmfre   flt  re  pi  pofr  sr   in   sy   cs us sy
id 0 0 0   9750M13G   450   5   3   1   560   0  234 50201 5206  2
2 96 1 0 0   9742M13G79   4   0   0   573   0  239 51886 4700
0  1 99



Hmm, yes on another machine same OS with 16GB memory we see:-

dmesg | grep memory
real memory  = 17179869184 (16384 MB)
avail memory = 16536604672 (15770 MB)

vmstat -c 2 -w 1 -n 0
procs  memory  page  faults cpu
r b w avmfre   flt  re  pi  pofr  sr   in   sy   cs us sy id
0 0 0   1948M   666M   395  14   0   0   398 1966 3237 5980 14085  2  4 93
1 0 0   1948M   636M 4   0   0   016   0 8215 18660 35322  2  7 92

sysctl vm.vmtotal
vm.vmtotal: 
System wide totals computed every five seconds: (values in kilobytes)

===
Processes:  (RUNQ: 1 Disk Wait: 0 Page Wait: 0 Sleep: 194)
Virtual Memory: (Total: 1075975332K Active: 1979056K)
Real Memory:(Total: 309048K Active: 160804K)
Shared Virtual Memory:  (Total: 75112K Active: 18860K)
Shared Real Memory: (Total: 15656K Active: 10572K)
Free Memory Pages:  731960K

but with top:-
last pid: 38187;  load averages:  0.15,  0.16,  0.16 up 2+23:33:55  15:13:02
195 processes: 1 running, 194 sleeping
CPU:  0.0% user,  0.0% nice,  4.1% system,  0.0% interrupt, 95.9% idle
Mem: 886M Active, 12G Inact, 2194M Wired, 599M Cache, 1630M Buf, 138M Free
Swap: 8192M Total, 1512K Used, 8190M Free

So I've no idea whats going on?

   Regards
   Steve



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Invalid memory stats from vmstat and sysctl vm.vmtotal?

2011-11-30 Thread Steven Hartland

We're seeing some impossible memory usage stats reported on machines
here from vmstat and sysctl vm.vmtotal.

We have machines reporting to be using 31GB total when they only have
8GB physical and are not using any swap.

Here's an output from one of our machines:-
vmstat -c 2 -w 1 -n 0 
procs  memory  page  faults cpu

r b w avmfre   flt  re  pi  pofr  sr   in   sy   cs us sy id
0 0 0  31768M  2112M   586   0   0   0   421   0  106  270  569  0  6 94
0 0 0  31768M  2112M 2   0   0   0 0   0  370 8139 3996  0  1 99

The raw output is:-
vmstat -c 2 -w 1 -n 0 -H
procs  memory  page  faults cpu
r b w avmfre   flt  re  pi  pofr  sr   in   sy   cs us sy id
0 0 0 32530228  2162524   586   0   0   0   421   0  106  270  569  0  6 94
0 0 0 32530228  2162524 2   0   0   0 0   0  286 8234 4347  0  1 99

Top shows:-
last pid:  6665;  load averages:  0.00,  0.00, 0.01 up 80+01:24:12  09:35:28
1893 processes:1 running, 1892 sleeping
CPU:  0.0% user,  0.0% nice,  0.3% system,  0.0% interrupt, 99.7% idle
Mem: 3754M Active, 84M Inact, 1976M Wired, 4K Cache, 2109M Free
Swap: 4096M Total, 4096M Free

sysctl vm.vmtotal
vm.vmtotal: 
System wide totals computed every five seconds: (values in kilobytes)

===
Processes:  (RUNQ: 1 Disk Wait: 0 Page Wait: 0 Sleep: 1893)
Virtual Memory: (Total: 1106403532K Active: 32540260K)
Real Memory:(Total: 4563648K Active: 3921644K)
Shared Virtual Memory:  (Total: 19976K Active: 16396K)
Shared Real Memory: (Total: 9040K Active: 8436K)
Free Memory Pages:  2161740K

As mentioned this machine has 8GB of ram and according to both top and
swapinfo is using no swap at all


From dmesg:-

real memory  = 8589934592 (8192 MB)
avail memory = 823536 (7873 MB)

swapinfo
Device  1K-blocks UsedAvail Capacity
/dev/gptid/09f211f7-39ce-11e0-8   41943040  4194304 0%

uname -a
FreeBSD test 8.2-RELEASE FreeBSD 8.2-RELEASE #2: Thu Mar 24 17:28:55 UTC 2011   
  root@test:/usr/obj/usr/src/sys/MULTIPLAY  amd64

sysctl hw.pagesize
hw.pagesize: 4096

It looks like it may be out by a factor of 4, possibly due to the fact
the its a 4k page size not 1k as indicated by the vmstat man page:-

memory  Information about the usage of virtual and real memory.  Virtual
pages (reported in units of 1024 bytes) are considered active if
they belong to processes which are running or have run in the
last 20 seconds.

avm active virtual pages
fre size of the free list


Totalling up RSS from ps axo rss gives a total in the region of that if
the vm stats are out by a factor of 4, in this case it should be: 8132557
which is 7.75GB a much more realistic value.

Am I totally missing something or is there problem here?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: iotop (dtrace?)

2011-10-25 Thread Steven Hartland

top and then use the IO mode, will give you an idea where the issue is.

   Regards
   Steve

- Original Message - 
From: Stefan Bethke s...@lassitu.de

To: freebsd-hackers@freebsd.org
Sent: Tuesday, October 25, 2011 9:34 PM
Subject: iotop (dtrace?)


I've got two systems with a constantly high rate of disk I/O that sometimes seems to be overwhelmed from it.  Before trying to 
decide if a hardware upgrade will help, I'd like to figure out which processes generate the load.


I've found a couple scripts named iotop which appear to produce what I would be interested in, but they appear to require Solaris 
or Linux.


Has someone ported over one of them, or would have a suggestion how to go about writing a custom dtrace script to gather this kind 
of information?


I can successfully run a couple of sample dtrace scripts on these 8-stable 
amd64 boxes.


Thanks,
Stefan

Solaris dtrace-based iotop: 
http://prefetch.net/articles/solaris.dtracetopten.html
Linux /proc-based Python script: http://guichaz.free.fr/iotop/

--
Stefan Bethke s...@lassitu.de   Fon +49 151 14070811



___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: debugging frequent kernel panics on 8.2-RELEASE

2011-08-18 Thread Steven Hartland
- Original Message - 
From: Andriy Gapon a...@freebsd.org

Thats interesting, are you using http as an example or is that something thats
been gleaned from the debugging of our output? I ask as there's only one process
running in each of our jails and thats a single java process.



It's from the debug data: p_comm = httpd


Hmm, there's only one httpd thats ever run on the machine and thats not in the 
jail
its on the raw machine.


I also would like to ask you to revert the last patch that I sent you (with 
tf_rip
comparisons) and try the patch from Kostik instead.


Sure.


Given what we suspect about the problem, can please also try to provoke the
problem by e.g. doing frequent jail restarts or something else that supposedly
should hit the bug.


I've tried doing this for quite some days on the test machine, but I've been
unable to provoke it, will continue to try.

   Regards
   Steve




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: debugging frequent kernel panics on 8.2-RELEASE

2011-08-18 Thread Steven Hartland
- Original Message - 
From: Andriy Gapon a...@freebsd.org



Probably I have mistakenly assumed that the 'prison' in prison_derefer() has
something to do with an actual jail, while it could have been just prison0 where
all non-jailed processes belong.


That makes sense as this particular panic was caused by a machine reboot,
which is slightly different from the more common jail panic we're seeing.

Doesn't help with our reproduction scenario though unfortunately. If we
don't have any joy reproducing on our single test machine I'll have this
kernel rolled out across a portion of the farm, which should mean we
see the panic results in a few days time.

I understand there's a risk involved in this but, its important for us
to determine the cause and get a confirmed fix, as well as being able
to prove that the panic fix works which will help everyone in the long
run.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: debugging frequent kernel panics on 8.2-RELEASE

2011-08-17 Thread Steven Hartland
- Original Message - 
From: Andriy Gapon a...@freebsd.org



Thanks to the debug that Steven provided and to the help that I received from
Kostik, I think that now I understand the basic mechanics of this panic, but,
unfortunately, not the details of its root cause.

It seems like everything starts with some kind of a race between terminating
processes in a jail and termination of the jail itself.  This is where the
details are very thin so far.  What we see is that a process (http) is in
exit(2) syscall, in exit1() function actually, and past the place where P_WEXIT
flag is set and even past the place where p_limit is freed and reset to NULL.
At that place the thread calls prison_proc_free(), which calls prison_deref().
Then, we see that in prison_deref() the thread gets a page fault because of what
seems like a NULL pointer dereference.  That's just the start of the problem and
its root cause.


Thats interesting, are you using http as an example or is that something thats
been gleaned from the debugging of our output? I ask as there's only one process
running in each of our jails and thats a single java process.

Now given your description there may be something I can add that may help
clarify what the cause could be.

In a nutshell the jail manager we're using will attempt to resurrect the jail
from a dieing state in a few specific scenarios.

Here's an exmaple:-
1. jail restart requested
2. jail is stopped, so the java processes is killed off, but active tcp sessions
may prevent the timely full shutdown of the jail.
3. if an existing jail is detected, i.e. a dieing jail from #2, instead of
starting a new jail we attach to the old one and exec the new java process.
4. if an existing jail isnt detected, i.e. where there where not hanging tcp
sessions and #2 cleanly shutdown the jail, a new jail is created, attached to
and the java exec'ed.

The system uses static jailid's so its possible to determine if an existing
jail for this service exists or not. This prevents duplicate services as
well as making services easy to identify by their jailid.

So what we could be seeing is a race between the jail shutdown and the attach
of the new process?

Now man 2 jail seems to indicate this is a valid use case for jail_set, as
it documents its support for JAIL_DYING as a valid option for flags, but I
suspect its something quite out of the ordinary to actually do, which may be
why this panic hasnt been seen before now.

As some background the reason we use static jailid's is to ensure only one
instance of the jailed service is running, and the reason we re-attach to
the dieing jail is so that jails can be restarted in a timely manor. Without
using the re-attach we would need to wait of all tcp sessions which have
been aborted to timeout.


So, of course, Steven is interested in finding and fixing the root cause.  I
hope we will get to that with some help from the prison guards :-)


Does the above potentially explain how we're getting to the situation
which generates the panic?

If so we can certainly look at using alternatives to the current design to
workaround this issue. Flagging the jail as permanent and using manual process
management and additional external locking to prevent duplicates, is what
instantly springs to mind.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: cam / ata timeout limited to 2147 due to overflow bug?

2011-08-05 Thread Steven Hartland

Fri, Aug 05, 2011 at 12:02:19AM +0100, Steven Hartland wrote:

So I suspect that this is what's happening resulting in an extremely
small timeout instead of a large one. Now I know that passed in value
to the timeout is seconds * 1000 so we should be seeing 2148000
for ccb-ccb_h.timeout now multiply that by 1000 (hz) and your over
the int wrap point 2147483647.

So instead of the wrap point being 2147483 seconds (24 days), I suspect
because of the way this is structured its actually 2147 seconds (26mins).

If this is the case the fix is likely to be something like:-
 callout_reset(slot-timeout, (int)(ccb-ccb_h.timeout * (hz / 2000)),


It will give you 0 timeout for all values of hz that are lower than
2000: hz is int, so you'll get integer division.  Since ccb_h.timeout
is u_int32_t, the proper way to handle this situation would be
{{{
(u_int64_t)ccb-ccb_h.timeout * (u_int32_t)hz)/2000
}}}
as long as the value of hz won't be greater than 2^32.


Ahh of course, was late ;-)


Can you try the patch at
 
http://codelabs.ru/fbsd/patches/ahci/AHCI-properly-convert-CAM-timeout-to-ticks.diff


What I don't understand is why the /2000


It gives (timeout_in_ticks)/2.  The code in ahci_timeout does the following:
{{{
   /* Check if slot was not being executed last time we checked. */
   if (slot-state  AHCI_SLOT_EXECUTING) {

snip..


So, my theory is that the first half of the timeout time is devoted
to the transition from AHCI_SLOT_RUNNING - AHCI_SLOT_EXECUTING and
the second one is the transition from AHCI_SLOT_RUNNING - TIMEOUT
to give the whole process the duration of a full timeout.  However,
judging by the code, if the slot won't start executing at the first
invocation of ahci_timeout that was spawned by the callout armed in
ahci_execute_transaction, we can have timeouts more than for the
specified amount of time.  And if the slot will never start its
execution, the callout will spin forever, unless I am missing something
important here.

May be Alexander can shed some light into this?


Interesting thanks for the explaination.

I've tried the patch and it a few cut and paste errors, which I've fixed,
and confirmed it works as expected, so thanks for that :)

There's also a load more drivers with the same issue so I've gone through
and fixed all the occurances I can find. Here's the updated patch:-
http://blog.multiplay.co.uk/dropzone/freebsd/ccb_timeout.patch

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


cam / ata timeout limited to 2147 due to overflow bug?

2011-08-04 Thread Steven Hartland

I'm working on adding security methods to camcontrol and have
come up against a strange issue. It seems that the timeout
value for cam, at least on ata (ahci), is limited to less than
2148 seconds.

This can be seen by running:-
camcontrol identify ada0 -t 2148 -v
(pass0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
(pass0:ahcich0:0:0:0): CAM status: Command timeout

Also seen in /var/log/messages at this time is:-
Aug  4 23:29:51 cfdev kernel: ahcich0: Timeout on slot 24
Aug  4 23:29:51 cfdev kernel: ahcich0: is  cs 0100 ss  rs 
0100 tfd d0 serr 

Dropping the timeout down to 2147 and the command runs fine.

I've done some digging and it seems like this is implemented via:-
sys/dev/ahci/ahci.c
ahci_execute_transaction(struct ahci_slot *slot)
{
...
   /* Start command execution timeout */
   callout_reset(slot-timeout, (int)ccb-ccb_h.timeout * hz / 2000,
   (timeout_t*)ahci_timeout, slot);

Now its documented that:-
Non-positive values of ticks are silently converted to the value 1

So I suspect that this is what's happening resulting in an extremely
small timeout instead of a large one. Now I know that passed in value
to the timeout is seconds * 1000 so we should be seeing 2148000
for ccb-ccb_h.timeout now multiply that by 1000 (hz) and your over
the int wrap point 2147483647.

So instead of the wrap point being 2147483 seconds (24 days), I suspect
because of the way this is structured its actually 2147 seconds (26mins).

If this is the case the fix is likely to be something like:-
callout_reset(slot-timeout, (int)(ccb-ccb_h.timeout * (hz / 2000)),

Does this sound reasonable? What I don't understand is why the /2000?

For reference the reason for wanting a large timeout is that a
secure erase of large media could take many hours so I'm using
the erase time reported by the drive for this, in my case here is
400 minutes.

Currently this instantly fails with a Command timeout which is
clearly not right.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: IPv4 socket bind using IPv6 socket on openjdk6 breaks udp send

2011-06-27 Thread Steven Hartland
- Original Message - 
From: Matthias Andree matthias.and...@gmx.de
No I'm not its replying to the sender. 


Yes you are, check your trace: The sendto address and port are the same
as the bound address.


This is a bug in truss it seems, Bjoern said he's gonna have a look at
it. Doesn't matter what you bind to it always reports this value in the
truss output.


In the java code we have:-
socket.send( new DatagramPacket( data, data.length,
src.getSocketAddress() ) );
Where src is the src packet. This works fine on IPv4 only machines and
when the jdk is told to use only IPv4 stack. So its not a problem with
the java code itself but could well be an issue with the


What data type is it?


All mute as the Bjoern found and fixed the issue, it was a bug in the
kernel fixed by:-
http://svnweb.freebsd.org/base?view=revisionrevision=220463


Oops sorry cut and paste error (wrong line) heres the correct line.
root java   894   20 udp4   85.236.109.212:25675  *:*


While a datagram socket, it does not match the socket()/bind() above.

An INET6-domain datagram socket would be listed as udp6 here.  Are you
sure you're tracing the right VM and are looking at the right thread?


Again, truss isn't showing the correct results, confused me too ;-)

Possibly another bug in sockstat / netstat as well, when the socket is a
IPv6 socket bound to an IPv4 address maybe it should indicate this e.g.
udp6-4 instead of either udp4 or udp6; alternatively maybe udp6 + a IPv4
address is enough to indicate this, but that could cause confusion.

Thanks for looking at this, appreciated :)

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: IPv4 socket bind using IPv6 socket on openjdk6 breaks udp send

2011-06-25 Thread Steven Hartland


- Original Message - 
From: Matthias Andree matthias.and...@gmx.de


I'm adding back in -java as based on you comments it may well be
something in the jdk passing invalid values down to the kernel
syscall.


The socket bind works fine and the packets sent to the server arrive
and are processed by the app but when it tries to reply using
send the result is:-
java.io.IOException: Invalid argument
   at java.net.PlainDatagramSocketImpl.send(Native Method)
   at java.net.DatagramSocket.send(DatagramSocket.java:629)

using truss we see the following:-
socket(PF_INET6,SOCK_DGRAM,0)= 20 (0x14)
setsockopt(0x14,0x29,0x1b,0x7edf0318,0x4,0x0) = 0 (0x0)
setsockopt(0x14,0x,0x20,0x7edf031c,0x4,0x0) = 0 (0x0)
bind(20,{ AF_INET6 [3800::10:0:0:0]:20736 },28)  = 0 (0x0)
..
sendto(20,\M^?\M^?\M^?\M^?I\aMultiplay :: ...,82,0x0,{ AF_INET6
[3800::10:0:0:0]:20736 },0x1c) ERR#22 'Invalid argument'


You're trying to send to your own address, but you're likely not using
the loopback interface for that.  Is that permitted by your firewall
configuration and routing?


No I'm not its replying to the sender. In the java code we have:-
socket.send( new DatagramPacket( data, data.length, src.getSocketAddress() ) );
Where src is the src packet. This works fine on IPv4 only machines and
when the jdk is told to use only IPv4 stack. So its not a problem with
the java code itself but could well be an issue with the




sockstat shows it binding correctly
root java   894   21 tcp4   85.236.109.212:25675  *:*


This is unrelated, as it has fd #21 not #20 as in the socket/bind/sendto
calls.  You've quoted the wrong line from sockstat output.


Oops sorry cut and paste error (wrong line) heres the correct line.
root java   894   20 udp4   85.236.109.212:25675  *:*

21 is the tcp port created in the same manor (ipv6 socket) which works
fine.




Note: net.inet6.ip6.v6only was set to the default 1 but changing
it to 0 has no effect on the issue.


You aren't using IPv4 mapped addresses, and you haven't stated whether
you're using wildcard listeners. Only in that case would it matter.


I'm not, its a bound port, as shown above now I have the correct line ;-)


inet6(4) reads:

 IPV6CTL_V6ONLY(ip6.v6only) Boolean: enable/disable the prohib-
   ited use of IPv4 mapped address on AF_INET6 sock-
   ets.  Defaults to on.


The jvm automatically sets this on all sockets for compatibility for
this exact reason. I'm not rulling out an issue with the IPv6 - v4
routing in the kernel though.


Are you sure that's what you seeing?  It's not a match for what you give
above, but anyways it's an implementation artifact because the tcp code
for v4 and v6 used to be shared and the udp code separate.


Thats not how the jdk works, its ment to be 100% transparent but isn't.
See the following for some interesting details:-
http://diario.beerensalat.info/2008/10/12/java_and_ipv6_on_bsd.html


It is best to set up one IPv4 and one IPv6 listening socket.


I don't believe there is any way to do this in java it either uses the
IPv4 stack only or the IPv6 stack only hence relies on the kernel
routing IPv4 packets through the IPv6 stack.

Thats the reason the jdk explicitly enables this for all the ports it
creates, which was added as a back port of the jdk7 fixes which can
be seen here:-
http://www.freebsd.org/cgi/cvsweb.cgi/ports/java/openjdk6/files/patch-set


Check the URL above, perhaps that helps your understanding a bit.  I
presume 3800::10:0:0:0 is your server?


Not that I'm aware of, here's the output from ifconfig if anyone can
tell me different, as I'm new to IPv6 and don't follow how its mapped
yet.

ifconfig
igb0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500
   
options=1bbRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4
   ether 00:25:90:2c:3c:b0
   inet 85.236.109.212 netmask 0xff00 broadcast 85.236.109.255
   inet6 fe80::225:90ff:fe2c:3cb0%igb0 prefixlen 64 scopeid 0x1 
   inet6 2001:4db0:20:2::1337 prefixlen 64 
   nd6 options=3PERFORMNUD,ACCEPT_RTADV

   media: Ethernet autoselect (1000baseSX full-duplex)
   status: active
igb1: flags=8802BROADCAST,SIMPLEX,MULTICAST metric 0 mtu 1500
   
options=1bbRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4
   ether 00:25:90:2c:3c:b1
   media: Ethernet autoselect (1000baseSX full-duplex)
   status: active
lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384
   options=3RXCSUM,TXCSUM
   inet 127.0.0.1 netmask 0xff00 
   inet6 ::1 prefixlen 128 
   inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 
   nd6 options=3PERFORMNUD,ACCEPT_RTADV


This is currently running on 8.2-RELEASE with openjdk6-b22_5

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the 

IPv4 socket bind using IPv6 socket on openjdk6 breaks udp send

2011-06-24 Thread Steven Hartland

We're trying to get our machines IPv6 enabled but in doing so this
seems to break java apps using openjdk6 for UDP sends.

The server seems quite happy to send and receive TCP packets on
IPv6 socket that are bound to IPv4 addresses, but the same is not
true for UDP.

The socket bind works fine and the packets sent to the server arrive
and are processed by the app but when it tries to reply using
send the result is:-
java.io.IOException: Invalid argument
   at java.net.PlainDatagramSocketImpl.send(Native Method)
   at java.net.DatagramSocket.send(DatagramSocket.java:629)

using truss we see the following:-
socket(PF_INET6,SOCK_DGRAM,0)= 20 (0x14)
setsockopt(0x14,0x29,0x1b,0x7edf0318,0x4,0x0) = 0 (0x0)
setsockopt(0x14,0x,0x20,0x7edf031c,0x4,0x0) = 0 (0x0)
bind(20,{ AF_INET6 [3800::10:0:0:0]:20736 },28)  = 0 (0x0)
..
recvfrom(20,0x7eaeb580,1460,0x0,0x7eaed580,0x7eaed5ac) ERR#60 
'Operation timed out'
..
sendto(20,\M^?\M^?\M^?\M^?I\aMultiplay :: ...,82,0x0,{ AF_INET6 
[3800::10:0:0:0]:20736 },0x1c) ERR#22 'Invalid argument'

sockstat shows it binding correctly
root java   894   21 tcp4   85.236.109.212:25675  *:*

The following PR seems relevant but also seems to indicate it was
fixed back in 2006
http://www.freebsd.org/cgi/query-pr.cgi?pr=92620

Setting -Djava.net.preferIPv4Stack=true does workaround the issue
but when we come to support IPv6 as well as IPv4 this won't work.

Note: net.inet6.ip6.v6only was set to the default 1 but changing
it to 0 has no effect on the issue.

An ideas why tcp in this setup works fine for udp fails only on
send?

Not sure which list is best for this so sorry about the cross
posting.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: quotas an essential feature? (was: svn commit: r218953 - stable/8/usr.sbin/sysinstall)

2011-02-25 Thread Steven Hartland

While I can understand some may want its not something we use on any of
our machines, and I suspect that's the case for many others.

Given adding it means the kernel will be doing extra work and hence a
drop in performance for a feature most will never use, I'm guessing here,
I would say just leave it out of generic unless there is a real pressing
requirement for it?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: machdep.hlt_cpus not safe with ULE?

2011-02-21 Thread Steven Hartland
- Original Message - 
From: Garrett Cooper gcoo...@freebsd.org

   As a followup to this and based on discussions with other folks,
the fact that it's using hlt to halt CPUs without rescheduling tasks /
masking interrupts, etc is not good. So none of the *hlt* sysctls are
really doing the right thing on x86.


Time to disable them until they are fixed properly then I would suggest?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


machdep.hlt_cpus not safe with ULE?

2011-02-19 Thread Steven Hartland

I'm trying to debug a possibly failing CPU, so I thought it would
be easy just disable the cores using machdep.hlt_cpus and see if
we see the panic's we've been seeing.

The problem is it seems ULE doesnt properly support machdep.hlt_cpus
and still schedules processes onto the halted cpus which obviously
causes problems.

Can anyone confirm this behaviour? Should machdep.hlt_cpus and I assume
the logical counterpart never be used with ULE?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: machdep.hlt_cpus not safe with ULE?

2011-02-19 Thread Steven Hartland


- Original Message - 
From: Gary Jennejohn gljennj...@googlemail.com


Looking at the kernel source it appears that only sched_4bsd.c makes use
of hlt_cpus_mask.


Given ULE is default do these need to be either removed totally or at least
conditionally based on the scheduler choice as currently they are quite
dangerous to a systems health?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: machdep.hlt_cpus not safe with ULE?

2011-02-19 Thread Steven Hartland

For reference I've found that an alternative is to set the following
in loader.conf:-
hint.lapic.2.disabled=1
hint.lapic.3.disabled=1

2 and 3 here are the apic numbers displayed by dmesg on boot for the
cpu's

Obviously this requires a reboot so no perfect for all uses but it does
work for what we're testing.



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: nginx + passenger = segv in _rtld_error on restart on FreeBSD 8.0?

2010-05-08 Thread Steven Hartland

As you may have guessed by my last reply no we didn't we had to revert to
apache + passenger, but seems you've found a fix anyway. Out of interest
what lead you to the close race condition PR as a potential fix?

- Original Message - 
From: Matt Reimer mattjrei...@gmail.com

Steve,

Did you figure this out? We're seeing something very similar with
nginx + passenger + FreeBSD 8.0.



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: nginx + passenger = segv in _rtld_error on restart onFreeBSD8.0?

2010-05-08 Thread Steven Hartland

Thanks Matt most appreciated!
- Original Message - 
From: Matt Reimer mattjrei...@gmail.com

Steve,

The patch for PR 144061 works for us.

http://lists.freebsd.org/pipermail/freebsd-hackers/2010-February/030741.html
http://www.freebsd.org/cgi/query-pr.cgi?pr=144061


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


To sendmail or to postfix that is the question?

2010-03-10 Thread Steven Hartland

Ok so I'm looking to replace our current windows mail
server using mdaemon with a FreeBSD solution, having
looked around there seems to be differing opinions
of which is the best option to go with between sendmail
and postfix.

The problem with looking for info on this is that a
lot of the high listed articles in Google etc are
from way back when, 2000 - 2007 during which time
quite a bit can change.

So what are peoples current day experience with the
two options?

A few key question come to mind:-
1. Has sendmail's config moved away from the black art
it once was?
2. Is postfix that much easier?
3. What would people use for:
3.1. POP / IMAP support?
3.2. Web Mail?
3.2. AV / Spam filtering?

Any advice, opinions on a full mail solution on FreeBSD
would be appreciated.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: To sendmail or to postfix that is the question?

2010-03-10 Thread Steven Hartland


- Original Message - 
From: Bill Moran wmo...@collaborativefusion.com



I recommend squirrelmail.  Unless you require some obscene feature-rich
craziness (in that case, have fun installing IMP).  I've used it with
Postfix/Dovecot for a few years now with no trouble.

Why is this discussion on hack...@?


Couldn't see a really appropriate list tbh, as pointed out though
perhaps isp@ would have been better.

Thanks for all the feedback and ideas so far :)

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: nginx hanging with state zoneli

2010-01-09 Thread Steven Hartland


- Original Message - 
From: Alexander Nesterov nester@gmail.com



On 08.01.2010 20:21, Steven Hartland wrote:

[..]
12582/218/12800/12800 4k (page size) jumbo clusters in use 
(current/cache/total/max)


Try to increase jumbo clusters (sysctl kern.ipc.nmbjumbop)


Thanks for the suggestion Alex I've doubled this now and will keep an eye.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


nginx hanging with state zoneli

2010-01-08 Thread Steven Hartland

The first time in a while be we just had nginx get stuck in the zoneli
state:
51630 www 1 -160 16616K  9396K zoneli 2  24:33  0.00% nginx
51627 www 1 -160 16616K  9376K zoneli 1  24:22  0.00% nginx
51629 www 1 -160 16616K  9372K zoneli 3  24:09  0.00% nginx
51628 www 1 -160 16616K  9104K zoneli 0  24:05  0.00% nginx

Usually I would suspect an mbufs issue but that doesnt seem to have
caused the problem this time, going by the info below, anyone ideas?

Running: 7.0-RELEASE-p11 AMD64

== netstat -m ==
14829/11796/26625 mbufs in use (current/cache/total)
2240/1024/3264/262144 mbuf clusters in use (current/cache/total/max)
1449/727 mbuf+clusters out of packet secondary zone in use (current/cache)
12582/218/12800/12800 4k (page size) jumbo clusters in use 
(current/cache/total/max)
0/0/0/6400 9k jumbo clusters in use (current/cache/total/max)
0/0/0/3200 16k jumbo clusters in use (current/cache/total/max)
58515K/5869K/64384K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0/0/0 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
3214 requests for I/O initiated by sendfile
0 calls to protocol drain routines


== /etc/sysctl.con ==
# Large amounts of files in directories fix
vfs.ufs.dirhash_maxmem=8379692

# Network performance tunning
net.inet.tcp.inflight.enable=0
net.inet.tcp.sendspace=65536
kern.ipc.maxsockbuf=16777216
net.inet.tcp.sendbuf_max=16777216
net.inet.tcp.recvbuf_max=16777216

# Prevent nginx issues:-
# Increase backlog max
kern.ipc.somaxconn=1024

# Prevent issue with pmap
vm.pmap.shpgperproc=500

# high server numbers tuning see:
# http://rerepi.wordpress.com/2008/04/19/tuning-freebsd-sysoev-rit/
kern.ipc.nmbclusters=262144
kern.ipc.maxsockets=204800
net.inet.tcp.msl=2

== /boot/loader.conf ==
accf_http_load=YES
net.inet.tcp.syncache.hashsize=1024
net.inet.tcp.syncache.bucketlimit=100
net.inet.tcp.tcbhashsize=4096
vm.kmem_size=768M
vm.kmem_size_max=768M
 



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Passenger hangs on live and SEGV on tests possible threading /kernel bug?

2009-12-21 Thread Steven Hartland
- Original Message - 
From: John Baldwin 

I've uploaded a two more traces for the oxt test failure / segv.
http://code.google.com/p/phusion-passenger/issues/detail?id=441#c1

From looking at the test case it testing the capture of failures and its 
ability
to create a stack trace output so that may give others some indication where
the issue may be?

I will look to do the same on for the hang issue but that's on a live site so
will need to schedule some downtime before I can get those rebuilt and then
wait for it to hang again, which could be quite some time :(


Hmmm, the only seg fault I see is happening down inside libgcc in the stack
unwinding code and that is 3rd party code from gcc.


Thanks for looking John, so you believe this may be an issue with the gcc code?

What would be the next step on this, raise it on a gcc mail list or something?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Passenger hangs on live and SEGV on tests possible threading / kernel bug?

2009-12-17 Thread Steven Hartland

We're having an issue with Passenger on FreeBSD where it will hang
and stop processing any more requests the details are attach to
the following bug report:
http://code.google.com/p/phusion-passenger/issues/detail?id=318#c14

In addition the test suite crashes in what seems to be a very
basic test, which I'm at a loss with.
http://code.google.com/p/phusion-passenger/issues/detail?id=441

I'm thinking this may be a bugs in the FreeBSD either kernel or
thread library as the crashes don't make any sense from the
application side.

Any advise on debugging or feedback on the stack traces would
be much appreciated.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Passenger hangs on live and SEGV on tests possible threading / kernel bug?

2009-12-17 Thread Steven Hartland
- Original Message - 
From: John Baldwin j...@freebsd.org
For the hang it seems you have a thread waiting in a blocking read(), a thread 
waiting in a blocking accept(), and lots of threads creating condition 
variables.  However, the pthread_cond_init() in libpthread (libthr on FreeBSD) 
doesn't call pthread_cleanup_push(), so your stack trace doesn't make sense to 
me.  However, that may be gdb getting confused.  The pthread_cleanup_push() 
frame may be cond_init().  However, it doesn't call umtx_op() (the 
_thr_umutex_init() call it makes just initializes the structure, it doesn't 
make a _umtx_op() system call).  You might try posting on threads@ to try to 
get more info on this, but your pthread_cond_init() stack traces don't really 
make sense.  Can you rebuild libc and libthr with debug symbols?


For example:

# cd /usr/src/lib/libc
# make clean 
# make DEBUG_FLAGS=-g

# make DEBUG_FLAGS=-g install

However, if you are hanging in read(), that usually means you have a socket 
that just doesn't have data.  That might be an application bug of some sort.


The segv trace doesn't include the first part of GDB messages which show which 
thread actually had a seg fault.  It looks like it was the thread that was 
throwing an exception.  However, nanosleep() doesn't throw exceptions, so that 
stack trace doesn't really make sense either.  Perhaps that stack is hosed by 
the exception handling code?


I've uploaded a two more traces for the oxt test failure / segv.
http://code.google.com/p/phusion-passenger/issues/detail?id=441#c1


From looking at the test case it testing the capture of failures and its ability

to create a stack trace output so that may give others some indication where
the issue may be?

I will look to do the same on for the hang issue but that's on a live site so
will need to schedule some downtime before I can get those rebuilt and then
wait for it to hang again, which could be quite some time :(

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: nginx + passenger = segv in _rtld_error on restart on FreeBSD8.0?

2009-12-09 Thread Steven Hartland


- Original Message - 
From: Kostik Belousov kostik...@gmail.com

To: Steven Hartland kill...@multiplay.co.uk
Cc: freebsd-hackers@freebsd.org; freebsd-sta...@freebsd.org
Sent: Wednesday, December 09, 2009 10:21 AM
Subject: Re: nginx + passenger = segv in _rtld_error on restart on FreeBSD8.0?

This is the trace once world had been recompiled with:-
CFLAGS=-pipe
WITH_CTF=1
DEBUG_FLAGS=-g


#0  0x000800c95eec in thr_kill () at thr_kill.S:3
#1  0x000800b22e9e in _thr_send_sig (thread=0x800f06600, sig=6) at 
/usr/src/lib/libthr/thread/thr_kern.c:92
#2  0x000800b1f878 in _raise (sig=6) at 
/usr/src/lib/libthr/thread/thr_sig.c:187
#3  0x000800d74003 in abort () at /usr/src/lib/libc/stdlib/abort.c:65
#4  0x0043b8a7 in Client::threadMain (this=0x800f9cf40) at 
ext/nginx/HelperServer.cpp:516
#5  0x00411302 in boost::_mfi::mf0void, Client::operator() (this=0x7fa45ea8, p=0x800f9cf40) at 
mem_fn_template.hpp:49
#6  0x00411651 in boost::_bi::list1boost::_bi::valueClient* ::operator()boost::_mfi::mf0void, Client, 
boost::_bi::list0 (this=0x7fa45eb8, f...@0x7fa45ea8, a...@0x7fa45d7f) at bind.hpp:232
#7  0x00411696 in boost::_bi::bind_tvoid, boost::_mfi::mf0void, Client, boost::_bi::list1boost::_bi::valueClient*  
::operator() (this=0x7fa45ea8) at bind_template.hpp:20
#8  0x004116bd in boost::detail::function::void_function_obj_invoker0boost::_bi::bind_tvoid, boost::_mfi::mf0void, 
Client, boost::_bi::list1boost::_bi::valueClient*  , void::invoke (

   function_obj_p...@0x7fa45ea8) at function_template.hpp:158
#9  0x0042e73a in boost::function0void, std::allocatorvoid ::operator() (this=0x7fa45ea0) at 
function_template.hpp:825

#10 0x00435760 in oxt::thread::thread_main (fu...@0x7fa45ea0, 
da...@0x7fa45e90) at thread.hpp:107
#11 0x0041310e in boost::_bi::list2boost::_bi::valueboost::functionvoid ()(), std::allocatorvoid  , 
boost::_bi::valueboost::shared_ptroxt::thread::thread_data  ::operator()void (*)(boost::functionvoid ()(), 
std::allocatorvoid , boost::shared_ptroxt::thread::thread_data), boost::_bi::list0 (this=0x800f3ee80, f...@0x800f3ee78, 
a...@0x7fa45f0f) at bind.hpp:289
#12 0x00413196 in boost::_bi::bind_tvoid, void (*)(boost::functionvoid ()(), std::allocatorvoid , 
boost::shared_ptroxt::thread::thread_data), boost::_bi::list2boost::_bi::valueboost::functionvoid ()(), std::allocatorvoid 
 , boost::_bi::valueboost::shared_ptroxt::thread::thread_data   ::operator() (this=0x800f3ee78) at bind_template.hpp:20
#13 0x004131b9 in boost::thread::thread_databoost::_bi::bind_tvoid, void (*)(boost::functionvoid ()(), 
std::allocatorvoid , boost::shared_ptroxt::thread::thread_data), boost::_bi::list2boost::_bi::valueboost::functionvoid 
()(), std::allocatorvoid  , boost::_bi::valueboost::shared_ptroxt::thread::thread_data::run (this=0x800f3ee00) at 
thread.hpp:130

#14 0x00443259 in thread_proxy (param=0x800f3ee00) at 
ext/boost/src/pthread/thread.cpp:127
#15 0x000800b1badd in thread_start (curthread=0x800f06600) at 
/usr/src/lib/libthr/thread/thr_create.c:288
#16 0x in ?? ()
Cannot access memory at address 0x7fa46000
Current language:  auto; currently asm

It seems that in the passenger client threads it calls closeStream which errors 
when
the socket close errors with ENOTCONN
   virtual void closeStream() {
   TRACE_POINT();
   if (fd != -1) {
   int ret = syscalls::close(fd);
   fd = -1;
   if (ret == -1) {
   if (errno == EIO) {
   throw SystemException(A write operation on the session 
stream failed,
   errno);
   } else {
   throw SystemException(Cannot close the session stream,
   errno);
   }
   }
   }
   }

This causes it to call abort on the the thread which then crashes the app with
the above stack trace, which seems really weird. Anyone got any ideas?

   Regards
   steve 




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


nginx + passenger = segv in _rtld_error on restart on FreeBSD 8.0?

2009-12-08 Thread Steven Hartland

I'm currently testing nginx + passenger on FreeBSD 8.0 and I'm seeing a strange
segv which seems to indicate a core library error in _rtld_error. Could this
be the case or is the stack just badly corrupted?

(gdb) bt
#0  0x0008005577dc in _rtld_error () from /libexec/ld-elf.so.1
#1  0x000800557c3f in _rtld_error () from /libexec/ld-elf.so.1
#2  0x000800557d5e in _rtld_error () from /libexec/ld-elf.so.1
#3  0x00080055851b in dladdr () from /libexec/ld-elf.so.1
#4  0x0008005585f3 in dladdr () from /libexec/ld-elf.so.1
#5  0x00080055576d in ?? () from /libexec/ld-elf.so.1
#6  0x0001 in ?? ()
#7  0x004117f8 in boost::detail::sp_counted_impl_pPassenger::Application::StandardSession::dispose (this=0x800768980) at 
sp_counted_impl.hpp:78

Previous frame inner to this frame (corrupt stack?)

   Regards
   Steve



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Dell M600 Blade: 6.4 works, 7.x and 8.x fail to boot

2009-11-14 Thread Steven Hartland

Yes I can confirm that.

   Regards
   Steve
- Original Message - 
From: Thierry Herbelot thierry.herbe...@free.fr


from the two URLs you provide, it seems that i386 is working (for all FreeBSD 
versions) and only amd64 is failing : can you confirm this ?



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


How to prevent 64bit library paths being searched for 32bit binaries on amd64?

2009-10-03 Thread Steven Hartland

I've just been trying to get a 32bit binary to work on amd64 7.0-RELEASE,
the binary had some qt4 dependencies so I set about copying these into
/usr/local/lib32 but it seems that for some reason /usr/local/lib is
searched first which obviously causes problems when here is a 64bit
library of the same name.

/etc/rc.d/ldconfig start 
shows:-

ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib
/usr/local/lib/compat/pkg /usr/local/lib/compat/pkg
/usr/local/lib/mysql /usr/local/lib/pth
32-bit compatibility ldconfig path: /usr/lib32 /usr/local/lib32
/usr/local/lib32/mysql /usr/local/lib32/qt4

LD_LIBARY_PATH isn't set

So the question is why is /usr/local/lib searched for a 32bit binary?

Is it possible the binary itself is setting it and if so how to I
override it?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: FreeBSD 7.0-RELEASE amd64 on Dell M600 Blade

2009-07-20 Thread Steven Hartland

Did anyone ever get anywhere with this, just retried with 8.0-BETA2
and still the same hangs as depicted in the attached screen shots.
Sorry got no serial console :(

Would like to get this working if possible so anything I can try
to gain some more info on this issue?

   Regards
   Steve

- Original Message - 
From: Adam Jacob Muller freebsd-hack...@adam.gs




Anyone ever get this to work? Perhaps this was fixed in a newer  
FreeBSD? Have some M600 that i'd like to get FreeBSD running on :)


hint.apic.0.disabled seemed to change things a bit, it would reach  



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org

i386_set_ldt on FreeBSD 7.x amd64

2009-07-19 Thread Steven Hartland

I'm trying to convert some audio streams and the only app
which seems to be capable of this mplayer with the w32codecs

Unfortunately the machine is 7.1 amd64. I've compiled the
binaries up on an old i386 box but when running said bins
on the amd64 box it errors with:

Opening audio decoder: [dmo] Win32/DMO decoders
install_fs: Invalid argument
Couldn't install fs segment, expect segfault

Looking at the code the call to i386_set_ldt is failing
is there any know workaround?

Looking on Google there is a thread:
i386_set_ldt and wine on AMD64 which mentions some patches by kib
but no mention if this ever worked.
http://docs.freebsd.org/cgi/getmsg.cgi?fetch=233316+0+archive/2008/freebsd-amd64/20081214.freebsd-amd64

The latest version I can find is:
http://people.freebsd.org/~kib/misc/amd64_ldt-pre.4.patch

Anyone info appreciated.

  Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: i386_set_ldt on FreeBSD 7.x amd64

2009-07-19 Thread Steven Hartland
- Original Message - 
From: Kostik Belousov kostik...@gmail.com

http://docs.freebsd.org/cgi/getmsg.cgi?fetch=233316+0+archive/2008/freebsd-amd64/20081214.freebsd-amd64

The latest version I can find is:
http://people.freebsd.org/~kib/misc/amd64_ldt-pre.4.patch


Anyone info appreciated.
The patches were committed to HEAD. mplayer/win32 codec's were tested,
it was one of the goal of the patch to have these codec's working on amd64.

No MFC to 7.x is planned.


Thanks for the confirmation Kostik. Do you believe there is any reason to
stop that patch working on 7.X if manually applied? Is the patch above the
final version that went into HEAD?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: i386_set_ldt on FreeBSD 7.x amd64

2009-07-19 Thread Steven Hartland
- Original Message - 
From: Kostik Belousov kostik...@gmail.com

Thanks for the confirmation Kostik. Do you believe there is any reason to
stop that patch working on 7.X if manually applied? Is the patch above the
final version that went into HEAD?


No and no, but I think that back-porting is not that trivial.
Upgrade is probably easier then backport.


When you say HEAD would that require an upgrade to 7-STABLE or 8-CURRENT? I
never did get what the official meaning of HEAD is, as it seems to be used by
different people to mean both of the above.

   Regards
   Steve



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Looking for someone to commit hptmv driver fixes

2009-04-04 Thread Steven Hartland

Hi guys I'm looking for someone to commit the hptmv v1.16
driver to the main source? Currently this driver in FreeBSD
core is at 1.12 which it totally unusable under 7.+, panics
on install, as well as being totally unstable on Seagate
drives on previous versions.

I can provide a full patch which is essentially v1.16
from highpoint + one minor fix for multi card support.

If anyone who can do this would let me know, that would
be most appreciated as getting a machine working with
this card is currently a major PITA involving building
a custom install CD which is a very lengthy process :(

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: How to increase the max pty's on Freebsd 7.0?

2009-04-02 Thread Steven Hartland
0n Wed, Apr 01, 2009 at 10:53:06PM +0200, Ed Schouten wrote: 

You can increase the maximum amount of PTYs by editing a lot of source
files on your system. There is some good news: in -CURRENT we switched
to Unix98-style PTYs (/dev/pts/%u). Right now the maximum amount of PTYs
is limited to 1000 (0 to 999).



Thanks for the confirmation I've managed to patch our local source tree
to increase this to 1024. Seems the last patch to raise to 512 has one
bug so fixed that while I was there.

If anyone wants the patch set shout.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: How to increase the max pty's on Freebsd 7.0?

2009-04-02 Thread Steven Hartland

Yep that's what I came up with after looking though the code thanks
for the link though, always good to get confirmation that I didn't
miss something.

   Regards
   Steve

- Original Message - 
From: John Baldwin j...@freebsd.org




On Wednesday 01 April 2009 1:55:12 pm Steven Hartland wrote:

How can I increase the maximum number or ptys available on FreeBSD 7.0?

It seems that currently the machine is maxing out at 512 but there is
still loads of capacity left on the machine.

Ideally would like to double at least the number of ttys available,
any help would be most appreciated.


http://people.freebsd.org/~jhb/patches/pty_1152.patch

This might require 7.1 instead of 7.0 as I simplified the pty allocation code 
in = 7.x so that it was easier to add new ones.



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


How to increase the max pty's on Freebsd 7.0?

2009-04-01 Thread Steven Hartland

How can I increase the maximum number or ptys available on FreeBSD 7.0?

It seems that currently the machine is maxing out at 512 but there is
still loads of capacity left on the machine.

Ideally would like to double at least the number of ttys available,
any help would be most appreciated.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: How to increase the max pty's on Freebsd 7.0?

2009-04-01 Thread Steven Hartland

I knew his sounded very familiar, seems I asked the same
question back in 2007 for 5.4 just it was capped at 256:
http://lists.freebsd.org/mailman/htdig/freebsd-hackers/2007-October/021852.html

I'm sure I also remember saying this limit had been removed in 7.x
but looking in the source for pqrsPQRS it seems there are several
places where this still hard coded:-

contrib/telnet/telnetd/sys_term.c:  for (cp = pqrsPQRS; *cp; cp++) {
lib/libc/stdlib/grantpt.c:#define   PTY_DEV1pqrsPQRSlmnoLMNO
lib/libutil/pty.c:  for (cp1 = pqrsPQRSlmnoLMNO; *cp1; cp1++) {
sys/kern/tty_pty.c:static const char names[] = pqrsPQRSlmnoLMNOtuvwTUVWxyzXYZ;
sys/kern/tty_pty.c: * pts == 
/dev/tty[pqrsPQRSlmnoLMNO][0123456789abcdefghijklmnopqrstuv]
sys/kern/tty_pty.c: * ptc == 
/dev/pty[pqrsPQRSlmnoLMNO][0123456789abcdefghijklmnopqrstuv]
usr.sbin/ac/ac.c:   strchr(pqrsPQRS, 
usr.ut_line[3]) != 0 ||

So the questions are:-
1. Is the fix to this still to just to update these settings?
2. Is there any significance to the upper case lower case thing?
3. Are there any other restrictions on the letters that can be used?

- Original Message - 
From: Steven Hartland



How can I increase the maximum number or ptys available on FreeBSD 7.0?

It seems that currently the machine is maxing out at 512 but there is
still loads of capacity left on the machine.

Ideally would like to double at least the number of ttys available,
any help would be most appreciated.




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


7.0 unusual performance issue - vmdaemon hang?

2008-12-10 Thread Steven Hartland

Just had one of hour webservers flag as down here and on
investigation the machine seems to be struggling due to
a hung vmdaemon process.

top is reporting vmdaemon as using a constant 55.57% CPU
yet CPU time is not increasing:-

last pid: 36492;  load averages:  0.04,  0.05,  .11   up 89+19:53:21  
14:36:08
223 processes: 9 running, 201 sleeping, 13 waiting
CPU states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
Mem: 644M Active, 2780M Inact, 480M Wired, 249M Cache, 214M Buf, 3759M Free
Swap: 4096M Total, 537M Used, 3559M Free, 13% Inuse

 PID USERNAME THR PRI NICE   SIZERES STATE  C   TIME   WCPU COMMAND
  11 root   1 171 ki31 0K16K CPU7   7 2116.4 100.00% idle: cpu7
  12 root   1 171 ki31 0K16K CPU6   6 2059.5 100.00% idle: cpu6
  13 root   1 171 ki31 0K16K CPU5   5 2029.3 100.00% idle: cpu5
  14 root   1 171 ki31 0K16K CPU4   4 1977.8 100.00% idle: cpu4
  15 root   1 171 ki31 0K16K CPU3   3 1912.0 100.00% idle: cpu3
  16 root   1 171 ki31 0K16K CPU2   2 1835.2 100.00% idle: cpu2
  17 root   1 171 ki31 0K16K CPU1   1 1763.1 100.00% idle: cpu1
  18 root   1 171 ki31 0K16K RUN0 1727.6 100.00% idle: cpu0
  37 root   1  20- 0K16K psleep 5   0:56 55.57% vmdaemon
60198 www1   4098M 13516K sbwait 2  35:21  1.46% httpd
60264 www1   40   133M  9248K sbwait 0  21:21  0.39% httpd
  30 root   1 -68- 0K16K -  7  18.3H  0.00% em1 taskq
  29 root   1 -68- 0K16K -  6 330:21  0.00% em0 taskq
  41 root   1  20- 0K16K syncer 1 212:42  0.00% syncer
  21 root   1 -44- 0K16K WAIT   0 201:02  0.00% swi1: net
  19 root   1 -32- 0K16K WAIT   0 120:15  0.00% swi4: clock
  22 root   1  44- 0K16K -  5  73:00  0.00% yarrow

I've tried to ktrace the process and it produced nothing, also tried
gdb and it failed to attach. Is there anything else I can try before
we reboot the machine to help determine what the problem is?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: unionfs kernel panic on 7.1-PRERELEASE

2008-12-06 Thread Steven Hartland

Did you have any ideas where the root of this problem may lie Kostik?

- Original Message - 
From: Steven Hartland [EMAIL PROTECTED]




Yes every time, I've got a half life 2 dedicated install mounted under unionfs:-
mount -t unionfs -o noatime -o below /usr/local/games/hl2ds 
/usr/local/games/servers/1

As soon as I start the server from under servers/1 the machine panics I'm 
thinking its
a combination of the Linux ABI and unionfs interaction which is causing the 
issue.

   Regards
   Steve

- Original Message - 
From: Kostik Belousov [EMAIL PROTECTED]

To: Steven Hartland [EMAIL PROTECTED]
Cc: freebsd-hackers@freebsd.org
Sent: Tuesday, December 02, 2008 8:39 PM
Subject: Re: unionfs kernel panic on 7.1-PRERELEASE

Is it reproducible ?

The start of *fdp structure looks very suspicious,
fd_ofiles = 0x140, fd_ofileflags = 0x154, fd_cdir = 0x168, fd_rdir = 0x17c,
fd_jdir = 0x18c
The values are consequently increasing by 0x14, except fd_jdir, and
pointer values are wrong for kernel.



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


unionfs kernel panic on 7.1-PRERELEASE

2008-12-02 Thread Steven Hartland

Not sure where to go with this one any help appreciated:-
FreeBSD dedicated11.multiplay.co.uk 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #4: Tue Dec  2 16:53:30 UTC 2008 
[EMAIL PROTECTED]:/usr/obj/usr/src/sys/MULTIPLAY  i386


kgdb kernel /var/crash/vmcore.1
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type show copying to see the conditions.
There is absolutely no warranty for GDB.  Type show warranty for details.
This GDB was configured as i386-marcel-freebsd...

Unread portion of the kernel message buffer:


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x150
fault code  = supervisor read, page not present
instruction pointer = 0x20:0xc0624115
stack pointer   = 0x28:0xe62c3b80
frame pointer   = 0x28:0xe62c3ba8
code segment= base 0x0, limit 0xf, type 0x1b
   = DPL 0, pres 1, def32 1, gran 1
processor eflags= interrupt enabled, resume, IOPL = 0
current process = 763 (srcds_i686)
trap number = 12
panic: page fault
cpuid = 0
Uptime: 2m5s
Physical memory: 1007 MB
Dumping 53 MB: 38 22 6


warning: kld_current_sos: Can't read filename: Input/output error

Reading symbols from /boot/kernel/acpi.ko...Reading symbols from 
/boot/kernel/acpi.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/acpi.ko
Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from 
/boot/kernel/linprocfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/linprocfs.ko
Reading symbols from /boot/kernel/linux.ko...Reading symbols from 
/boot/kernel/linux.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/linux.ko
Reading symbols from /boot/kernel/unionfs.ko...Reading symbols from 
/boot/kernel/unionfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/unionfs.ko
#0  doadump () at pcpu.h:196
196 pcpu.h: No such file or directory.
   in pcpu.h
(kgdb) list *0xc0624115
0xc0624115 is in getvnode (/usr/src/sys/kern/vfs_syscalls.c:3969).
3964fp = NULL;
3965if (fdp == NULL)
3966error = EBADF;
3967else {
3968FILEDESC_SLOCK(fdp);
3969if ((u_int)fd = fdp-fd_nfiles ||
3970(fp = fdp-fd_ofiles[fd]) == NULL)
3971error = EBADF;
3972else if (fp-f_vnode == NULL) {
3973fp = NULL;
(kgdb) bt
#0  doadump () at pcpu.h:196
#1  0xc05a0937 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
#2  0xc05a0c09 in panic (fmt=Variable fmt is not available.
) at /usr/src/sys/kern/kern_shutdown.c:574
#3  0xc072eb8c in trap_fatal (frame=0xe62c3b40, eva=336) at 
/usr/src/sys/i386/i386/trap.c:939
#4  0xc072ee10 in trap_pfault (frame=0xe62c3b40, usermode=0, eva=336) at 
/usr/src/sys/i386/i386/trap.c:852
#5  0xc072f7cc in trap (frame=0xe62c3b40) at /usr/src/sys/i386/i386/trap.c:530
#6  0xc071563b in calltrap () at /usr/src/sys/i386/i386/exception.s:159
#7  0xc0624115 in getvnode (fdp=0xc40b4d00, fd=4, fpp=0xe62c3c70) at 
/usr/src/sys/kern/vfs_syscalls.c:3969
#8  0xc3e2a13d in getdents_common (td=0xc408f460, args=0xe62c3cfc, is64bit=0) at 
/usr/src/sys/modules/linux/../../compat/linux/linux_file.c:446

#9  0xc072f165 in syscall (frame=0xe62c3d38) at 
/usr/src/sys/i386/i386/trap.c:1090
#10 0xc07156a0 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:255
#11 0x0033 in ?? ()
Previous frame inner to this frame (corrupt stack?)


(kgdb) frame 7
#7  0xc0624115 in getvnode (fdp=0xc40b4d00, fd=4, fpp=0xe62c3c70) at 
/usr/src/sys/kern/vfs_syscalls.c:3969
3969if ((u_int)fd = fdp-fd_nfiles ||
(kgdb) print *fdp
$1 = {fd_ofiles = 0x140, fd_ofileflags = 0x154 Address 0x154 out of bounds, fd_cdir = 0x168, fd_rdir = 0x17c, fd_jdir = 0x18c, 
fd_nfiles = 512, fd_map = 0xc3bed560, fd_lastfile = 4,
 fd_freefile = 5, fd_cmask = 18, fd_refcnt = 1, fd_holdcnt = 1, fd_sx = {lock_object = {lo_name = 0xc076e1c2 filedesc 
structure, lo_type = 0xc076e1c2 filedesc structure, lo_flags = 37421056,
 lo_witness_data = {lod_list = {stqe_next = 0x0}, lod_witness = 0x0}}, sx_lock = 17, sx_recurse = 0}, fd_kqlist = {slh_first 
= 0x0}, fd_holdleaderscount = 0, fd_holdleaderswakeup = 0}

(kgdb) print fd
$2 = 4
(kgdb) print fdp-fd_ofiles
$3 = (struct file **) 0x140
(kgdb) print fdp-fd_ofiles[fd]
Cannot access memory at address 0x150
(kgdb) print fdp-fd_ofiles[0]
Cannot access memory at address 0x140
(kgdb) print *fdp-fd_ofiles
Cannot access memory at address 0x140

0xc3e2a13d is in getdents_common 
(/usr/src/sys/modules/linux/../../compat/linux/linux_file.c:446).
441 nbytes = sizeof(linux_dirent);
442 justone = 1;
443 } else
444 justone = 0;
445
446 

Re: unionfs kernel panic on 7.1-PRERELEASE

2008-12-02 Thread Steven Hartland

Yes every time, I've got a half life 2 dedicated install mounted under unionfs:-
mount -t unionfs -o noatime -o below /usr/local/games/hl2ds 
/usr/local/games/servers/1

As soon as I start the server from under servers/1 the machine panics I'm 
thinking its
a combination of the Linux ABI and unionfs interaction which is causing the 
issue.

   Regards
   Steve

- Original Message - 
From: Kostik Belousov [EMAIL PROTECTED]

To: Steven Hartland [EMAIL PROTECTED]
Cc: freebsd-hackers@freebsd.org
Sent: Tuesday, December 02, 2008 8:39 PM
Subject: Re: unionfs kernel panic on 7.1-PRERELEASE

Is it reproducible ?

The start of *fdp structure looks very suspicious,
fd_ofiles = 0x140, fd_ofileflags = 0x154, fd_cdir = 0x168, fd_rdir = 0x17c,
fd_jdir = 0x18c
The values are consequently increasing by 0x14, except fd_jdir, and
pointer values are wrong for kernel.


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FreeBSD 7.0-RELEASE amd64 on Dell M600 Blade

2008-09-11 Thread Steven Hartland

Thanks Rudi, would really like to get is sorted as they would make
ideal app servers.

- Original Message - 
From: Rudi Kramer - MWEB [EMAIL PROTECTED]



Hi Steven,

We recently purchased a few M600's but haven't got around to loading
FBSD on them, we should start installing next week and I will let you
know if we run in to any problems.



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


FreeBSD 7.0-RELEASE amd64 on Dell M600 Blade

2008-09-08 Thread Steven Hartland
I've been trying to install FreeBSD 7.0-RELEASE amd64 on 
a Dell M600 Blade but it hangs just after initialising

the isa bus.

I've tried a number of things and the only thing that I
can get to work is the i386 build which boots into the
installer without issue.

Has anyone had any experience with the Dell M600 blade
on amd64 or had the amd64 build hang at this point
before.

I don't have access to the machines to try new things
with ATM as we needed them in production so was forced
to install ubuntu to get then live but I should get
them back for more testing next week some time so wanted
to see if anyone had any experience with this or a
similar issue?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


lighttpd failing to accept new connections ( connection reset )

2008-08-28 Thread Steven Hartland

We're using lighttpd here for a new project and we're having issues
where by it simply stops processing after a 1-2 days.

Having looked at it in some detail this morning it seems that
the kernel is resetting the connection without notifying the
lighttpd process there is a new connection attempt. I assume
that the listen queue is full but why kevent is not notifying
lighttpd that there are outstanding events is beyond me.


The following is a truss of the process which is currently in
this state:-
kevent(6,0x0,0,{},11096,{1.0})   = 0 (0x0)
gettimeofday({1219920575.149428},0x0)= 0 (0x0)
kevent(6,0x0,0,{},11096,{1.0})   = 0 (0x0)
gettimeofday({1219920576.150443},0x0)= 0 (0x0)

ktrace of the operation as well:-
28363 lighttpd RET   kevent 0
28363 lighttpd CALL  gettimeofday(0x7fffeb20,0)
28363 lighttpd RET   gettimeofday 0
28363 lighttpd CALL  kevent(0x6,0,0,0x800e66000,0x2b58,0x7fffeb20)
28363 lighttpd GIO   fd 6 wrote 0 bytes
  
28363 lighttpd GIO   fd 6 read 0 bytes
  
28363 lighttpd RET   kevent 0
28363 lighttpd CALL  gettimeofday(0x7fffeb20,0)
28363 lighttpd RET   gettimeofday 0
28363 lighttpd CALL  kevent(0x6,0,0,0x800e66000,0x2b58,0x7fffeb20)
28363 lighttpd GIO   fd 6 wrote 0 bytes
  
28363 lighttpd GIO   fd 6 read 0 bytes
  
28363 lighttpd RET   kevent 0
28363 lighttpd CALL  gettimeofday(0x7fffeb20,0)
28363 lighttpd RET   gettimeofday 0
28363 lighttpd CALL  kevent(0x6,0,0,0x800e66000,0x2b58,0x7fffeb20)
28363 lighttpd GIO   fd 6 wrote 0 bytes
  
28363 lighttpd GIO   fd 6 read 0 bytes
  
28363 lighttpd RET   kevent 0
28363 lighttpd CALL  gettimeofday(0x7fffeb20,0)
28363 lighttpd RET   gettimeofday 0
28363 lighttpd CALL  kevent(0x6,0,0,0x800e66000,0x2b58,0x7fffeb20)
28363 lighttpd GIO   fd 6 wrote 0 bytes
  
28363 lighttpd GIO   fd 6 read 0 bytes
  
28363 lighttpd RET   kevent 0
28363 lighttpd CALL  gettimeofday(0x7fffeb20,0)
28363 lighttpd RET   gettimeofday 0
28363 lighttpd CALL  kevent(0x6,0,0,0x800e66000,0x2b58,0x7fffeb20)
28363 lighttpd GIO   fd 6 wrote 0 bytes
  
28363 lighttpd GIO   fd 6 read 0 bytes
  
28363 lighttpd RET   kevent 0
28363 lighttpd CALL  gettimeofday(0x7fffeb20,0)
28363 lighttpd RET   gettimeofday 0
28363 lighttpd CALL  kevent(0x6,0,0,0x800e66000,0x2b58,0x7fffeb20)


tcpdump shows:-
12:10:29.475255 IP (tos 0x10, ttl  64, id 9536, offset 0, flags [DF], proto: TCP (6), length: 64) client.61224  server.80: S, 
cksum 0x6d22 (incorrect (- 0xedfa), 291994449:291994449(0) win 65535 mss 1460,nop,wscale 1,nop,nop,timestamp 3661727139 
0,sackOK,eol
12:10:29.481396 IP (tos 0x0, ttl  61, id 25503, offset 0, flags [DF], proto: TCP (6), length: 60) server.80  client.61224: S, 
cksum 0xbf22 (correct), 3444532576:3444532576(0) ack 291994450 win 65535 mss 1460,nop,wscale 9,sackOK,timestamp 3136311843 
3661727139
12:10:29.481419 IP (tos 0x10, ttl  64, id 9538, offset 0, flags [DF], proto: TCP (6), length: 52) client.61224  server.80: ., 
cksum 0x6d16 (incorrect (- 0x6bd2), 1:1(0) ack 1 win 33304 nop,nop,timestamp 3661727145 3136311843
12:10:29.487519 IP (tos 0x10, ttl  61, id 25504, offset 0, flags [DF], proto: TCP (6), length: 40) server.80  client.61224: R, 
cksum 0x20c7 (correct), 3444532577:3444532577(0) win 0


This may have been raised before back 2003 as bug kern/57380
but it was closed after no response from the reporter.

Another possible issues related to this is:-
http://trac.lighttpd.net/trac/ticket/1734


I've currently got one of the production machines offline
with this error ( hence the important flag ) in the hope
that someone can suggest a test which will shed more light
on the issue before I restart it.

   Regards
   Steve 





This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: lighttpd failing to accept new connections ( connection reset ) / possible kqueue bug

2008-08-28 Thread Steven Hartland


- Original Message - 
From: Jeremy Chadwick [EMAIL PROTECTED]


Can you change the polling method in lighttpd to use poll or select
instead of kqueue?  This would help in determining if the problem is
with the daemon itself or the kevent system.


Yep already scheduled that change for our London node tomorrow
morning. ATM we are seeing this issue every 1 - 2 days so it may
take a little while before I can answer that question.

I've had a look through the source and I can't see any reason why
kevent would suddenly stop notifying the app that new connections
are present. Event registration appears to only be done once on
app startup and similarly unregisters are only done on shutdown,
so my current thinking is there may be an problem with kqueue
itself.

I don't suppose your aware of any way to query the status of this
in the kernel or app given I have a node in this hung state?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: lighttpd failing to accept new connections ( connection reset )

2008-08-28 Thread Steven Hartland


- Original Message - 
From: Robert Watson [EMAIL PROTECTED]
The connections getting reset without application notification is a classic 
symptom of a full listen queue.  A couple of questions:


Yep thats what I thought.


(1) What FreeBSD version?


7.0-RELEASE-p2 (amd64)


(2) Are you using accept filters?


The modules there but not loaded, so no.


(3) If possibly, are you able to instrument lighthttpd so that you can trigger
it to query SO_LISTENQLIMIT, SO_LISTENQLEN, and SO_LISTENINCQLEN on the
listen socket once things have gone wrong?  The respectively (and perhaps
obviously) querye the current administrative limit on queue depth, the
number queue depth on completed connections, and the current queue depth
on incomplete connections.  The last of these will only be used with
accept filters on recent FreeBSD network stacks (since the syncache was
added).

Hopefully doing (3) will allow us to try to determine whether it's indeed the 
case that somehow the listen queue or event handling has gotten wedged in 
some way.


This should be possible, I'll have a look, assuming the kgdb stuff doesn't
turn up the required results.

   Regards
   Steve



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: ntpd jail problem

2008-06-08 Thread Steven Hartland

I assume as it would effect the entire machine and hence should be run
on the base machine instead, not the jail?

- Original Message - 
From: [EMAIL PROTECTED]




Anybody know why ntpd might not work in a jail?




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Using sendmsg for SCM_CREDS results in EINVAL on PF_INET socket

2008-05-18 Thread Steven Hartland

sendmsg is not documented as ever returning EINVAL but yet when
using the following code to send credentials to a remote host
results in EINVAL from sendmsg.

I suspect that SCM_CREDS is only valid for PF_LOCAL / PF_UNIX
sockets and not PF_INET sockets and hence the code in dbus
is actually invalid.

Can anyone confirm this is the case or not?

[code from dbus-sysdeps-unix.c]
write_credentials_byte (int server_fd,
   DBusError  *error)
{
 int bytes_written;
 char buf[1] = { '\0' };
#if defined(HAVE_CMSGCRED) 
 union {

 struct cmsghdr hdr;
 char cred[CMSG_SPACE (sizeof (struct cmsgcred))];
 } cmsg;
 struct iovec iov;
 struct msghdr msg;
 iov.iov_base = buf;
 iov.iov_len = 1;

 memset (msg, 0, sizeof (msg));
 msg.msg_iov = iov;
 msg.msg_iovlen = 1;

 msg.msg_control = (caddr_t) cmsg;
 msg.msg_controllen = CMSG_SPACE (sizeof (struct cmsgcred));
 memset (cmsg, 0, sizeof (cmsg));
 cmsg.hdr.cmsg_len = CMSG_LEN (sizeof (struct cmsgcred));
 cmsg.hdr.cmsg_level = SOL_SOCKET;
 cmsg.hdr.cmsg_type = SCM_CREDS;
#endif

 _DBUS_ASSERT_ERROR_IS_CLEAR (error);

again:

#if defined(HAVE_CMSGCRED) 
 bytes_written = sendmsg (server_fd, msg, 0);

#else
 bytes_written = write (server_fd, buf, 1);
#endif

 if (bytes_written  0  errno == EINTR)
   goto again;

 if (bytes_written  0)
   {
 dbus_set_error (error, _dbus_error_from_errno (errno),
 Failed to write credentials byte: %s,
_dbus_strerror (errno));
 return FALSE;
   }
 else if (bytes_written == 0)
   {
 dbus_set_error (error, DBUS_ERROR_IO_ERROR,
 wrote zero bytes writing credentials byte);
 return FALSE;
   }
 else
   {
 _dbus_assert (bytes_written == 1);
 _dbus_verbose (wrote credentials byte\n);
 return TRUE;
   }
}
[/code from dbus-sysdeps-unix.c]


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: MODULES_OVERRIDE magic needed

2008-03-31 Thread Steven Hartland

The following works fine here note its MODULE'S' not MODULE

makeoptions MODULES_OVERRIDE=linux linprocfs acpi nfsclient nfsserver nullfs 
accf_http

   Regards
   Steve

- Original Message - 
From: Danny Braniss [EMAIL PROTECTED]

To: freebsd-hackers@freebsd.org
Sent: Monday, March 31, 2008 3:11 PM
Subject: MODULES_OVERRIDE magic needed



hi,
I'm trying to compile only a few kernel modules, but
using MODULES_OVERRIDE I can only select one module.
MODULE_OVERRIDE=unionfs
or
MODULE_OVERRIDE=if_sis
is ok, but
MODULE_OVERRIDE=if_sis unionfs
failes with either
if_sis: not found
if it's defined in src.conf, or with
make: don't know how to make if_sis
from the command line.

danny


___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


pkg_add -r doesn't process sub dependencies?

2008-03-07 Thread Steven Hartland

Seems if you have package X which depends on package Y which
depends on package Z then pkg_add -r X will always fail unless
you first :-
1. pkg_add -r Y
2. pkg_add -r Z

The failure will be:-
pkg_add: could not find package Z !

This happens even though Z exists in the remote source package
directory.

Surely I've missed something very basic here and pkg_add isn't
that broken?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: pkg_add -r doesn't process sub dependencies?

2008-03-07 Thread Steven Hartland
- Original Message - 
From: walt [EMAIL PROTECTED]

If you are using RELENG_7, pkg_add is looking in the wrong place
on the remote server for packages.  I just submitted a patch on
the -STABLE list for this problem.


I'm actually using PACKAGESITE which is meant to bypass all path
calculations but it simply doesn't. I've now found PKG_ADD_BASE
which seems to fix this issue but is not documented anywhere.

I had to dig around in the code to find it, where its commented
as:-
/* Special tip that sysinstall left for us */

It seems really strange that the default behaviour of pkg_add -r
can't deal with sub dependencies properly without this undocumented
feature.

   Regards
   Steve




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: gettimeofday() in hping

2008-01-23 Thread Steven Hartland
- Original Message - 
From: Ivan Voras [EMAIL PROTECTED]

The other thing that bothers me is, that under freebsd is quite easy to get:
[send_ip] sendto: No buffer space available
It happens almost always on my laptop just few seconds after I start
hping with timecounter=TSC


I'm not sure, but from what I understood of Robert Watson's
explanation in the big ZFS thread on -current, maybe increasing
kmem_size (exactly as for ZFS...) could help you with these buffers.


Is this not just running out of mbufs? netstat -m will show if it is and the fix
is to just increase kern.ipc.nmbclusters.

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Jail with a back door interface?

2007-11-27 Thread Steven Hartland

Is it possible to create a jail with more than one IP?

What I'm looking to do is have a forward facing IP and
an backwards facing IP in the same jail so it can
talk to the internet and our back door systems.

Looking around this doesn't seem possible but is it?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: How to enable more than 256 pty's?

2007-10-02 Thread Steven Hartland

Any one got any pointers on this, the machine we running this app on is over
90% idle so I really don't want to have to install a second machine just to
workaround a limit on the number of pty's, surely there's a way to increase
this?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: How to enable more than 256 pty's?

2007-10-02 Thread Steven Hartland

Thanks for the tip there but I cant find any function called pty_create_slave
in the source.

N.B. Machine is running 5.4 but I also looked on 6.2 which we could upgrade
to but still couldn't find it, so I assume you may be talking about something
that's in current which we couldn't risk on this machine.

Is this something that's possible on 5.x / 6.2 or something that will need a
lot of work?

   Regards
   Steve

- Original Message - 
From: Dag-Erling Smørgrav [EMAIL PROTECTED]


You need to change the way ptys are named in pty_create_slave() and
pty_clone() in sys/kern/tty_pty.c.  Just changing names won't help as
the sequence is also hardcoded in pty_clone().

You also need to change grantpt(), openpty() and any other userland code
which has hardcoded knowledge of the naming scheme:

[EMAIL PROTECTED] ~% gfs pqrsPQRS
src/sys/kern/tty_pty.c: static char *names = pqrsPQRS;
src/sys/kern/tty_pty.c:  * pts == 
/dev/tty[pqrsPQRS][0123456789abcdefghijklmnopqrstuv]
src/sys/kern/tty_pty.c:  * ptc == 
/dev/pty[pqrsPQRS][0123456789abcdefghijklmnopqrstuv]
src/contrib/telnet/telnetd/sys_term.c:  for (cp = pqrsPQRS; *cp; cp++) {
src/usr.sbin/ac/ac.c:   strchr(pqrsPQRS, 
usr.ut_line[3]) != 0 ||
src/lib/libutil/pty.c:  for (cp1 = pqrsPQRS; *cp1; cp1++) {
src/lib/libc/stdlib/grantpt.c: #define  PT_DEV1 pqrsPQRS

Alternatively, set kern.pts.enable to 1, and find and fix the
hang-on-close bug in the pts code (if it hasn't been fixed already)

DES
--
Dag-Erling Smørgrav - [EMAIL PROTECTED]



This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


How to enable more than 256 pty's?

2007-10-01 Thread Steven Hartland

Is it really a matter of editing:
/usr/src/sys/kern/tty_pty.c

And changing:
static char *names = pqrsPQRS;

To increase the number of available pty's ( FreeBSD 5.4 ) or is there
a nicer way?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: How to enable more than 256 pty's?

2007-10-01 Thread Steven Hartland

Well looked like that might work but clearly not, should have looked closer.
Is there a way to do this?

   Regards
   Steve

- Original Message - 

Is it really a matter of editing:
/usr/src/sys/kern/tty_pty.c

And changing:
static char *names = pqrsPQRS;

To increase the number of available pty's ( FreeBSD 5.4 ) or is there
a nicer way?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the 
event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any 
information contained in it.

In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]





This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


truss with -f = application hang

2007-09-30 Thread Steven Hartland

When trying to trace an app with truss -f to trace descendants the application
will always hang. Without -f everything is fine but we dont get the decendants
calls.

This is when running on FreeBSD 6.2-RELEASE-p7, the application in this test
is a linux app and we are running linux_base-fc-4_9.

Anyone got any ideas about this?

   Regards
   Steve


This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to [EMAIL PROTECTED]

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


  1   2   3   >