Re: New CAM locking preview
- Original Message - From: Hans Petter Selasky h...@bitfrost.no To: Alexander Motin m...@freebsd.org Cc: Scott Long sco...@freebsd.org; Jeff Roberson j...@freebsd.org; ken k...@freebsd.org; freebsd-hackers@FreeBSD.org; FreeBSD SCSI freebsd-s...@freebsd.org; Steven Hartland s...@freebsd.org; Justin T. Gibbs gi...@freebsd.org Sent: Friday, August 16, 2013 1:38 AM Subject: Re: New CAM locking preview On 08/15/13 23:40, Alexander Motin wrote: Hi. Last weeks I've made substantial progress on my CAM locking work. In fact, at this moment I think I've tied all loose ends good enough to consider the new design viable and implementation worth further testing and bug fixing. So I would like to ask for review of my work from everybody who interested in CAM internals. In short, my idea was to split single per-SIM lock, that creates huge congestion under high IOPS, into several smaller ones. So design I've finally chosen includes such locks: 1) New per-device (per-LUN) locks to protect state of the devices and respective periphs. In most cases peripheral drivers just use that lock instead of SIM lock used before, so code modification is minimal and straightforward. 2) New per-target lock to protect list of LUNs fetched from the device. 3) Old single per-SIM lock to protect SIM driver internals, but only that. No parts of CAM itself use that lock. Keeping it for SIMs allows to keep API and hopefully ABI compatibility. Reducing its scope allows to reduce congestion. 4) New per-SIM lock to protect SIM and device command queues. That allows execute queued commands from any context unrelated to other locks. Also this lock serializes accesses to sim_action() method for the most of commands, this allows to mostly avoid busy spilling on SIM lock collision. 5) New per-bus locks to protect target, device and periphs reference counters. It allows to create and destroy paths unrelated to other locks in any possible context. Sounds very good! I assume you have tested USB mass storage device unplug during various file system operations? It does indeed sound like some very good work, thanks Alexander! @Hans if USB mass storage device unplug is something important to you then might I suggest its a good idea to grab the patches, run your own tests and report any issues you might find as I'm sure this would be most appreciated :) Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [PATCH] Add 4K QUIRK for Intel X25-M, MARVELL SD88SA02 and OCZ Agility 2
- Original Message - From: Xin Li delp...@delphij.net -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 8/11/13 12:03 PM, Richard Yao wrote: Is there any possibility that these quirks could be added to the upcoming 9.2 release? I think so but we need to act fast. Pools composed of affected disks will suffer from performance issues until they are reformatted with the correct block size. In addition, a certain benchmarking site, whose benchmarks are often fodder for trolls, uses the Intel X25-M in its benchmarks. Adding the quirk to 9.2 would eliminate an unfair handicap on FreeBSD from their next set of benchmarks. Actually I'm not 100% sure here: the current FreeBSD ZFS code does not handle stripesize at all, and thus if you create a pool with the patch, they will still be ashift=9. Justin (gibbs@) have a patchset to address this by the way, but since it's not yet in HEAD I wouldn't expect it be incorporated in 9.2... Indeed, there is nothing which really makes proper use of QUIRKS that I'm aware of and ZFS definitely doesn't. There's a number of patches around which correct that, I've one here too, but none as yet are a total solution to the ashift issue. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [PATCH] Add 4K QUIRK for Intel X25-M, MARVELL SD88SA02 and OCZ Agility 2
Could you provide the output from camcontrol identify for these disks I want to just double check the formatting before commiting as I don't have these disks here in labs. Regards Steve - Original Message - From: Richard Yao r...@gentoo.org To: hack...@freebsd.org Cc: Richard Yao r...@gentoo.org; Eitan Adler ead...@freebsd.org Sent: Sunday, August 11, 2013 1:12 PM Subject: [PATCH] Add 4K QUIRK for Intel X25-M, MARVELL SD88SA02 and OCZ Agility 2 Signed-off-by: Richard Yao r...@gentoo.org --- sys/cam/ata/ata_da.c | 24 sys/cam/scsi/scsi_da.c | 24 2 files changed, 48 insertions(+) diff --git a/sys/cam/ata/ata_da.c b/sys/cam/ata/ata_da.c index f201231..b7f293d 100644 --- a/sys/cam/ata/ata_da.c +++ b/sys/cam/ata/ata_da.c @@ -349,6 +349,14 @@ static struct ada_quirk_entry ada_quirk_table[] = }, { /* + * Intel X25-M Series SSDs + * 4k optimised trim only works in 4k requests + 4k aligned + */ + { T_DIRECT, SIP_MEDIA_FIXED, *, INTEL SSDSA2M*, * }, + /*quirks*/ADA_Q_4K + }, + { + /* * Kingston E100 Series SSDs * 4k optimised trim only works in 4k requests + 4k aligned */ @@ -365,6 +373,22 @@ static struct ada_quirk_entry ada_quirk_table[] = }, { /* + * Marvell SSD (entry taken from Open Solaris) + * 4k optimised trim only works in 4k requests + 4k aligned + */ + { T_DIRECT, SIP_MEDIA_FIXED, *, MARVELL SD88SA02*, * }, + /*quirks*/ADA_Q_4K + }, + { + /* + * OCZ Agility 2 SSDs + * 4k optimised trim only works in 4k requests + 4k aligned + */ + { T_DIRECT, SIP_MEDIA_FIXED, *, OCZ-AGILITY2*, * }, + /*quirks*/ADA_Q_4K + }, + { + /* * OCZ Agility 3 SSDs * 4k optimised trim only works in 4k requests + 4k aligned */ diff --git a/sys/cam/scsi/scsi_da.c b/sys/cam/scsi/scsi_da.c index 617afbd..df895be 100644 --- a/sys/cam/scsi/scsi_da.c +++ b/sys/cam/scsi/scsi_da.c @@ -1008,6 +1008,14 @@ static struct da_quirk_entry da_quirk_table[] = }, { /* + * Intel X25-M Series SSDs + * 4k optimised trim only works in 4k requests + 4k aligned + */ + { T_DIRECT, SIP_MEDIA_FIXED, *, INTEL SSDSA2M*, * }, + /*quirks*/ADA_Q_4K + }, + { + /* * Kingston E100 Series SSDs * 4k optimised trim only works in 4k requests + 4k aligned */ @@ -1024,6 +1032,22 @@ static struct da_quirk_entry da_quirk_table[] = }, { /* + * Marvell SSD (entry taken from Open Solaris) + * 4k optimised trim only works in 4k requests + 4k aligned + */ + { T_DIRECT, SIP_MEDIA_FIXED, *, MARVELL SD88SA02*, * }, + /*quirks*/ADA_Q_4K + }, + { + /* + * OCZ Agility 2 SSDs + * 4k optimised trim only works in 4k requests + 4k aligned + */ + { T_DIRECT, SIP_MEDIA_FIXED, *, OCZ-AGILITY2*, * }, + /*quirks*/ADA_Q_4K + }, + { + /* * OCZ Agility 3 SSDs * 4k optimised trim only works in 4k requests + 4k aligned */ -- 1.8.1.5 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [PATCH] Add 4K QUIRK for Intel X25-M, MARVELL SD88SA02 and OCZ Agility 2
Thanks Richard I'll commit these when I get in the office next week. Regards Steve - Original Message - From: Richard Yao r...@gentoo.org To: hack...@freebsd.org Cc: Richard Yao r...@gentoo.org; Eitan Adler ead...@freebsd.org Sent: Sunday, August 11, 2013 1:12 PM Subject: [PATCH] Add 4K QUIRK for Intel X25-M, MARVELL SD88SA02 and OCZ Agility 2 Signed-off-by: Richard Yao r...@gentoo.org --- sys/cam/ata/ata_da.c | 24 sys/cam/scsi/scsi_da.c | 24 2 files changed, 48 insertions(+) diff --git a/sys/cam/ata/ata_da.c b/sys/cam/ata/ata_da.c index f201231..b7f293d 100644 --- a/sys/cam/ata/ata_da.c +++ b/sys/cam/ata/ata_da.c @@ -349,6 +349,14 @@ static struct ada_quirk_entry ada_quirk_table[] = }, { /* + * Intel X25-M Series SSDs + * 4k optimised trim only works in 4k requests + 4k aligned + */ + { T_DIRECT, SIP_MEDIA_FIXED, *, INTEL SSDSA2M*, * }, + /*quirks*/ADA_Q_4K + }, + { + /* * Kingston E100 Series SSDs * 4k optimised trim only works in 4k requests + 4k aligned */ @@ -365,6 +373,22 @@ static struct ada_quirk_entry ada_quirk_table[] = }, { /* + * Marvell SSD (entry taken from Open Solaris) + * 4k optimised trim only works in 4k requests + 4k aligned + */ + { T_DIRECT, SIP_MEDIA_FIXED, *, MARVELL SD88SA02*, * }, + /*quirks*/ADA_Q_4K + }, + { + /* + * OCZ Agility 2 SSDs + * 4k optimised trim only works in 4k requests + 4k aligned + */ + { T_DIRECT, SIP_MEDIA_FIXED, *, OCZ-AGILITY2*, * }, + /*quirks*/ADA_Q_4K + }, + { + /* * OCZ Agility 3 SSDs * 4k optimised trim only works in 4k requests + 4k aligned */ diff --git a/sys/cam/scsi/scsi_da.c b/sys/cam/scsi/scsi_da.c index 617afbd..df895be 100644 --- a/sys/cam/scsi/scsi_da.c +++ b/sys/cam/scsi/scsi_da.c @@ -1008,6 +1008,14 @@ static struct da_quirk_entry da_quirk_table[] = }, { /* + * Intel X25-M Series SSDs + * 4k optimised trim only works in 4k requests + 4k aligned + */ + { T_DIRECT, SIP_MEDIA_FIXED, *, INTEL SSDSA2M*, * }, + /*quirks*/ADA_Q_4K + }, + { + /* * Kingston E100 Series SSDs * 4k optimised trim only works in 4k requests + 4k aligned */ @@ -1024,6 +1032,22 @@ static struct da_quirk_entry da_quirk_table[] = }, { /* + * Marvell SSD (entry taken from Open Solaris) + * 4k optimised trim only works in 4k requests + 4k aligned + */ + { T_DIRECT, SIP_MEDIA_FIXED, *, MARVELL SD88SA02*, * }, + /*quirks*/ADA_Q_4K + }, + { + /* + * OCZ Agility 2 SSDs + * 4k optimised trim only works in 4k requests + 4k aligned + */ + { T_DIRECT, SIP_MEDIA_FIXED, *, OCZ-AGILITY2*, * }, + /*quirks*/ADA_Q_4K + }, + { + /* * OCZ Agility 3 SSDs * 4k optimised trim only works in 4k requests + 4k aligned */ -- 1.8.1.5 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Make ZFS use the physical sector size when computing initial ashift
Hi DES, unfortunately you need a quite bit more than this to work compatibly. I've had a patch here that does just this for quite some time but there's been some discussion on how we want additional control over this so its not been commited. If others are interested I've attached this as it achieves what we needed here so may also be of use for others too. There's also a big discussion on illumos about this very subject ATM so I'm monitoring that too. Hopefully there will be a nice conclusion come from that how people want to proceed and we'll be able to get a change in that works for everyone. Regards Steve - Original Message - From: Dag-Erling Smørgrav d...@des.no To: freebsd...@freebsd.org; freebsd-hackers@freebsd.org Cc: ivo...@freebsd.org Sent: Wednesday, July 10, 2013 10:02 AM Subject: Make ZFS use the physical sector size when computing initial ashift The attached patch causes ZFS to base the minimum transfer size for a new vdev on the GEOM provider's stripesize (physical sector size) rather than sectorsize (logical sector size), provided that stripesize is a power of two larger than sectorsize and smaller than or equal to VDEV_PAD_SIZE. This should eliminate the need for ivoras@'s gnop trick when creating ZFS pools on Advanced Format drives. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to freebsd-fs-unsubscr...@freebsd.org This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. zzz-zfs-ashift-fix.patch Description: Binary data ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Make ZFS use the physical sector size when computing initial ashift
There's lots more to consider when considering a way foward not least of all ashift isn't a zpool configuration option is per top level vdev, space consideration of moving from 512b to 4k, see previous and current discussions on zfs-de...@freebsd.org and z...@lists.illumos.org for details. Regards Steve - Original Message - From: Borja Marcos bor...@sarenet.es On Jul 10, 2013, at 11:25 AM, Steven Hartland wrote: If others are interested I've attached this as it achieves what we needed here so may also be of use for others too. There's also a big discussion on illumos about this very subject ATM so I'm monitoring that too. Hopefully there will be a nice conclusion come from that how people want to proceed and we'll be able to get a change in that works for everyone. Hmm. I wonder if the simplest approach would be the better. I mean, adding a flag to zpool. At home I have a playground FreeBSD machine with a ZFS zmirror, and, you guessed it, I was careless when I purchased the components, I asked for two 1 TB drives and that I got, but different models, one of them advanced format and the other one classic. I don't think it's that bad to create a pool on a classic disk using 4 KB blocks, and it's quite likely that replacement disks will be 4 KB in the near future. Also, if you use SSDs the situation is similar. This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Make ZFS use the physical sector size when computing initial ashift
- Original Message - From: Xin Li -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 07/10/13 02:02, Dag-Erling Sm?rgrav wrote: The attached patch causes ZFS to base the minimum transfer size for a new vdev on the GEOM provider's stripesize (physical sector size) rather than sectorsize (logical sector size), provided that stripesize is a power of two larger than sectorsize and smaller than or equal to VDEV_PAD_SIZE. This should eliminate the need for ivoras@'s gnop trick when creating ZFS pools on Advanced Format drives. I think there are multiple versions of this (I also have one[1]) but the concern is that if one creates a pool with ashift=9, and now ashift=12, the pool gets unimportable. So there need a way to disable this behavior. I've tested my patch in all configurations I can think of including exported ashift=9 pools being imported, all no issues. For your example e.g. # Create a 4K pool (min_create_ashift=4K, dev=512) test:src sysctl vfs.zfs.min_create_ashift vfs.zfs.min_create_ashift: 12 test:src mdconfig -a -t swap -s 128m -S 512 -u 0 test:src zpool create mdpool md0 test:src zdb mdpool | grep ashift ashift: 12 ashift: 12 # Create a 512b pool (min_create_ashift=512, dev=512) test:src zpool destroy mdpool test:src sysctl vfs.zfs.min_create_ashift=9 vfs.zfs.min_create_ashift: 12 - 9 test:src zpool create mdpool md0 test:src zdb mdpool | grep ashift ashift: 9 ashift: 9 # Import a 512b pool (min_create_ashift=4K, dev=512) test:src zpool export mdpool test:src sysctl vfs.zfs.min_create_ashift=12 vfs.zfs.min_create_ashift: 9 - 12 test:src zpool import mdpool test:src zdb mdpool | grep ashift ashift: 9 ashift: 9 # Create a 4K pool (min_create_ashift=512, dev=4K) test:src zpool destroy mdpool test:src mdconfig -d -u 0 test:src mdconfig -a -t swap -s 128m -S 4096 -u 0 test:src sysctl vfs.zfs.min_create_ashift=9 vfs.zfs.min_create_ashift: 12 - 9 test:src zpool create mdpool md0 test:src zdb mdpool | grep ashift ashift: 12 ashift: 12 # Import a 4K pool (min_create_ashift=4K, dev=4K) test:src zpool export mdpool test:src sysctl vfs.zfs.min_create_ashift=12 vfs.zfs.min_create_ashift: 9 - 12 test:src zpool import mdpool test:src zdb mdpool | grep ashift ashift: 12 ashift: 12 Another thing (not really related to the automatic detection) is that we need a way to manually override this setting from command line when creating the pool, this is under active discussion at Illumos mailing list right now. [1] https://github.com/trueos/trueos/commit/3d2e3a38faad8df4acf442b055c5e98ab873fb26 Yep has been on my list for a while, based on previous discussions on zfs-devel@. I've not had any time recently but I'm following the illumos thread to see what conclusions they come to. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Make ZFS use the physical sector size when computing initial ashift
- Original Message - From: Justin T. Gibbs I'm sure lots of folks have some solution to this. Here is an old version of what we use at Spectra: http://people.freebsd.org/~gibbs/zfs_patches/zfs_auto_ashift.diff The above patch is missing some cleanup that was motivated by my discussions with George Wilson about this change in April. I'll dig that up later tonight. Even if you don't read the full diff, please read the included checkin comment since it explains the motivation behind this particular solution. This is on my list of things to upstream in the next week or so after I add logic to the userspace tools to report whether or not the TLVs in a pool are using an optimal allocation size. This is only possible if you actually make ZFS fully aware of logical, physical, and the configured allocation size. All of the other patches I've seen just treat physical as logical. Reading through your patch it seems that your logical_ashift equates to the current ashift values which for geom devices is based off sectorsize and your physical_ashift is based stripesize. This is almost identical to the approach I used adding a desired ashift, which equates to your physical_ashift, along side the standard ashift i.e. required aka logical_ashift value :) One issue I did spot in your patch is that you currently expose zfs_max_auto_ashift as a sysctl but don't clamp its value which would cause problems should a user configure values 13. If your interested in the reason for this its explained in the comments in my version which does a very similar thing with validation. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Make ZFS use the physical sector size when computing initial ashift
- Original Message - From: Justin T. Gibbs On Jul 10, 2013, at 1:06 PM, Steven Hartland wrote: - Original Message - From: Justin T. Gibbs I'm sure lots of folks have some solution to this. Here is an old version of what we use at Spectra: http://people.freebsd.org/~gibbs/zfs_patches/zfs_auto_ashift.diff The above patch is missing some cleanup that was motivated by my discussions with George Wilson about this change in April. I'll dig that up later tonight. Even if you don't read the full diff, please read the included checkin comment since it explains the motivation behind this particular solution. This is on my list of things to upstream in the next week or so after I add logic to the userspace tools to report whether or not the TLVs in a pool are using an optimal allocation size. This is only possible if you actually make ZFS fully aware of logical, physical, and the configured allocation size. All of the other patches I've seen just treat physical as logical. Reading through your patch it seems that your logical_ashift equates to the current ashift values which for geom devices is based off sectorsize and your physical_ashift is based stripesize. This is almost identical to the approach I used adding a desired ashift, which equates to your physical_ashift, along side the standard ashift i.e. required aka logical_ashift value :) Yes, the approaches are similar. Our current version records the logical access size in the vdev structure too, which might relate to the issue below. One issue I did spot in your patch is that you currently expose zfs_max_auto_ashift as a sysctl but don't clamp its value which would cause problems should a user configure values 13. I would expect the zio pipeline to simply insert an ashift aligned thunking buffer for these operations, but I haven't tried going past an ashift of 13 in my tests. If it is an issue, it seems the restriction should be based on logical access size, not optimal access size. Yes with your methodology you'll only see the issue if zfs_max_auto_ashift and physical_ashift are both 13, but this can be the case for example on a RAID controller with large stripsize. Looking back at my old patch it too suffers from the same issue along with the current code base, but that would only happen if logical sector size resulted in an ashift 13 which is going to be much less common ;-) Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Make ZFS use the physical sector size when computing initial ashift
- Original Message - From: Justin T. Gibbs ... One issue I did spot in your patch is that you currently expose zfs_max_auto_ashift as a sysctl but don't clamp its value which would cause problems should a user configure values 13. I would expect the zio pipeline to simply insert an ashift aligned thunking buffer for these operations, but I haven't tried going past an ashift of 13 in my tests. If it is an issue, it seems the restriction should be based on logical access size, not optimal access size. Yes with your methodology you'll only see the issue if zfs_max_auto_ashift and physical_ashift are both 13, but this can be the case for example on a RAID controller with large stripsize. I'm not sure I follow. logical_ashift is available in our latest code, as is the physical_ashift. But even without the logical_ashift, why doesn't the zio pipeline properly thunk zio_phys_read() access based on the configured ashift? When I looked at it, which was a long time ago now so please excuse me if I'm a little rusty on the details, zio_phys_read() was working more luck than judgement as the offsets passed in where calculated from a valid start + increment based on the size of a structure within vdev_label_offset() with no ashift logic applied that I cound find. The result was pools created with large ashift's where unstable when I tested. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: potential future proofing fix for aicasm build.
I just remembered I had an issue with aicasm when compiling our kernel that didn't have ahc or ahd on an 8.3 box. The fix was to manually delete the kernel obj directory before compiling after doing a full make buildworld. For some reason there was some cruft left in there that running make buildkernel wasn't cleaning out, could you be suffering from a similar issue? - Original Message - From: Dimitry Andric dimi...@andric.com To: Alfred Perlstein alf...@ixsystems.com Cc: hack...@freebsd.org Sent: Thursday, May 02, 2013 8:19 AM Subject: Re: potential future proofing fix for aicasm build. On May 1, 2013, at 18:44, Alfred Perlstein alf...@ixsystems.com wrote: I took a shot at fixing this issue with building aicasm as part of buildkernel of an older 9.0 src on a machine running HEAD. aicasm.o: In function `__getCurrentRuneLocale': /usr/include/runetype.h:96: undefined reference to `_ThreadRuneLocale' I don't understand this error message... It seems like a linker error, but it also seems to refer to an incorrect include file? Is this during linking or compiling? The issue seems to be two-fold: 1) Paths are not fully set to pick up the bootstrap tools needed to build. What do you mean, exactly? In r230622 I explicitly set the PATH to ${BPATH}:${PATH}, which should be enough to pick up the bootstrap tools. This is exactly the same path used to build the bootstrap-tools stage itself. The kernel bootstrap tools (only aicasm, really) should be built by the host compiler, not the cross-tools compiler. 2) include files use the host's instead of the build trees. The first problem is fixed by changing setting of PATH from ${BPATH}:${PATH} to ${TMPPATH}. The second is fixed by using -nostdinc and setting strict include paths using -I directives to the compiler: CFLAGS=-nostdinc -I${WORLDTMP}/usr/include -I. -I${KERNSRCDIR}/dev/aic7xxx/aicasm I don't think this is correct, as aicasm should be compiled by the host compiler, and linked with the host libc. So if you start including headers from the source directory, there will be a mismatch between what those headers declare, and what is available in the host libc. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: potential future proofing fix for aicasm build.
- Original Message - From: Dimitry Andric dimi...@andric.com On 2013-05-02 09:47, Steven Hartland wrote: I just remembered I had an issue with aicasm when compiling our kernel that didn't have ahc or ahd on an 8.3 box. The fix was to manually delete the kernel obj directory before compiling after doing a full make buildworld. For some reason there was some cruft left in there that running make buildkernel wasn't cleaning out, could you be suffering from a similar issue? Well, a way to reproduce the problem would be nice. I tried running make buildkernel in a stable/9 tree on a 10.0 machine, and building aicasm went just fine. So it works for me... I think what I did was the following:- 1. make buildkernel KERNCONF=MYCONF -j10 # failed due to aicasm 2. make buildworld -j10 # Realised I'd forgotten to build world 3. make buildkernel -j10 # still failed due to aicasm 4. rm -rf /usr/obj/usr/src/sys/MYCONF 5. make buildkernel -j10 # worked So I think the mistake that triggered it was I didn't build world before building kernel, however I've not tried reproducing. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: potential future proofing fix for aicasm build.
- Original Message - From: Dimitry Andric dimi...@andric.com On 2013-05-02 11:54, Steven Hartland wrote: From: Dimitry Andric dimi...@andric.com ... Well, a way to reproduce the problem would be nice. I tried running make buildkernel in a stable/9 tree on a 10.0 machine, and building aicasm went just fine. So it works for me... I think what I did was the following:- 1. make buildkernel KERNCONF=MYCONF -j10 # failed due to aicasm And this is what I cannot reproduce. Works just fine, either with or without -j. Have just confirmed this reproduces the issue with a recent head source on 8.3-RELEASE machine. 1. rm -rf /usr/obj 2. make buildkernel KERNCONF=MYCONF -j10 # Fails with:- cc1: warnings being treated as errors aicasm_gram.c:1539: warning: no previous prototype for 'yyparse' 3. make buildworld 4. make buildkernel KERNCONF=MYCONF -j10 # Still fails with:- cc1: warnings being treated as errors aicasm_gram.c:1539: warning: no previous prototype for 'yyparse' Fix is: 1. rm -rf /usr/obj/usr/src/sys/MYCONF 2. make buildkernel KERNCONF=MYCONF -j10 # Now works Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: potential future proofing fix for aicasm build.
I don't believe aicasm is actually needed if you don't have a driver which requires e.g. ahd or ahc. It would be good to get that fixed too. Regards Steve - Original Message - From: Alfred Perlstein bri...@mu.org To: hack...@freebsd.org Sent: Wednesday, May 01, 2013 5:46 PM Subject: potential future proofing fix for aicasm build. Hey folks, I took a shot at fixing this issue with building aicasm as part of buildkernel of an older 9.0 src on a machine running HEAD. aicasm.o: In function `__getCurrentRuneLocale': /usr/include/runetype.h:96: undefined reference to `_ThreadRuneLocale' The issue seems to be two-fold: 1) Paths are not fully set to pick up the bootstrap tools needed to build. 2) include files use the host's instead of the build trees. The first problem is fixed by changing setting of PATH from ${BPATH}:${PATH} to ${TMPPATH}. The second is fixed by using -nostdinc and setting strict include paths using -I directives to the compiler: CFLAGS=-nostdinc -I${WORLDTMP}/usr/include -I. -I${KERNSRCDIR}/dev/aic7xxx/aicasm Can I get review on this patch? https://gist.github.com/anonymous/5493734 Inline: diff --git a/Makefile.inc1 b/Makefile.inc1 index e850cda..785e3180 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -830,17 +830,18 @@ buildkernel: @echo stage 2.3: build tools @echo -- cd ${KRNLOBJDIR}/${_kernel}; \ - PATH=${BPATH}:${PATH} \ + PATH=${TMPPATH} \ MAKESRCPATH=${KERNSRCDIR}/dev/aic7xxx/aicasm \ - ${MAKE} SSP_CFLAGS= -DNO_CPU_CFLAGS -DNO_CTF \ + ${MAKE} SSP_CFLAGS= -DNO_CPU_CFLAGS -DNO_CTF CFLAGS=-nostdinc -I${WORLDTMP}/usr/include -I. -I${KERNSRCDIR}/dev/aic7xxx/aicasm \ -f ${KERNSRCDIR}/dev/aic7xxx/aicasm/Makefile # XXX - Gratuitously builds aicasm in the ``makeoptions NO_MODULES'' case. .if !defined(MODULES_WITH_WORLD) !defined(NO_MODULES) exists(${KERNSRCDIR}/modules) .for target in obj depend all + @echo aicasm: ${target} cd ${KERNSRCDIR}/modules/aic7xxx/aicasm; \ - PATH=${BPATH}:${PATH} \ + PATH=${TMPPATH} \ MAKEOBJDIRPREFIX=${KRNLOBJDIR}/${_kernel}/modules \ - ${MAKE} SSP_CFLAGS= -DNO_CPU_CFLAGS -DNO_CTF ${target} + ${MAKE} SSP_CFLAGS= -DNO_CPU_CFLAGS -DNO_CTF CFLAGS=-nostdinc -I${WORLDTMP}/usr/include -I. -I${KERNSRCDIR}/dev/aic7xxx/aicasm ${target} .endfor .endif .if !defined(NO_KERNELDEPEND) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Does any body agree ?
- Original Message - From: Mehmet Erol Sanliturk m.e.sanlit...@gmail.com To: Adrian Chadd adr...@freebsd.org Cc: freebsd-hackers@freebsd.org; ali mousa ali_mousa...@yahoo.com Sent: Saturday, March 09, 2013 11:06 PM Subject: Re: Does any body agree ? On Sat, Mar 9, 2013 at 3:03 PM, Adrian Chadd adr...@freebsd.org wrote: ... you need to post a more useful/descrptive title and/or body in your request. Most people won't look at the post. :-) Adrian On 9 March 2013 14:41, ali mousa ali_mousa...@yahoo.com wrote: Does any body agree ? http://forums.freebsd.org/showthread.php?p=212423#post212423 ? Regards Ali _ Dear Adrian , You are really right , and correct . Thank you very much . You may wish to know I asked the ports security team if they where working on this and they are. I've not seen it hit svn yet so hopefully this will happen soon. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: IPMI console [Re: Chicken and egg, encrypted root FS on remote server]
- Original Message - From: Daniel O'Connor On 21/02/2013, at 9:06, Steven Hartland kill...@multiplay.co.uk wrote: If I change the console redirect to com1, my screen stays blank. Would you perhaps know how to use com1 for redirect and connect to it using ipmi-console (or ipmi-tool)? We use the following on Supermicro servers works fine:- http://blog.multiplay.co.uk/2012/12/freebsd-serial-over-lan/ Nice! BTW do you know what flag 0x20 does for UART? 0x10 is documented but 0x20 is not. I had a quick look at the code and AFAIK it doesn't do anything (on 9.1 anyway). Actually at a guess I would say it's a hangover from sio(4) where 0x20 forced the device in question to be the console. According to the handbook, where I got the settings from, 0x20: Forces this unit to be the console (unless there is another higher priority console), regardless of the -h option discussed below. The flag 0x20 must be used together with the 0x10 flag. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: IPMI console [Re: Chicken and egg, encrypted root FS on remote server]
- Original Message - From: Paul Schenkeveld free...@psconsult.nl To: Daniel O'Connor docon...@gsoft.com.au Cc: hack...@freebsd.org Sent: Wednesday, February 20, 2013 8:31 PM Subject: IPMI console [Re: Chicken and egg, encrypted root FS on remote server] Hi Daniel, On Wed, Feb 20, 2013 at 10:55:47PM +1030, Daniel O'Connor wrote: On 20/02/2013, at 21:43, Paul Schenkeveld free...@psconsult.nl wrote: What about getting a remote console like HP's ILO or Dell's DRAC ? You get to login remotely, you can use some degree of access control... you can even remote boot. For new hardware I could indeed use this, the current hardware does not support remote console. I don't have experience with ILO nor DRAC but I do have experience with SuperMicro's KVM over LAN which does need a java client to run. If I can enter the passphrase over ssh that would be better as I can use any device including a smartphone to dial in and enter the passphrase. If you setup a serial console you don't need Java if you use ipmitool, eg ipmitool -H remoteip -U ADMIN -I lanplus sol activate Tried that with some Supermicro servers, the serial console allows me to get into BIOS config and shows boot messages up to starting the kernel, once the kernel starts output stops. In the BIOS setup, console redirect defaults to com2 port which explains why output stops after the loader passes control to the kernel. BTW, ipmitool always gives me Info: cannot activate SOL payload with encryption but ipmi-console (sysutils/freeipmi) works. If I change the console redirect to com1, my screen stays blank. Would you perhaps know how to use com1 for redirect and connect to it using ipmi-console (or ipmi-tool)? We use the following on Supermicro servers works fine:- http://blog.multiplay.co.uk/2012/12/freebsd-serial-over-lan/ Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Looking for reviewers for patch that adds foreign disk support mfiutil
- Original Message - From: John Baldwin Thanks for the feedback John appreciated, a couple of questions inline below if you would be so kind. On Sunday, February 17, 2013 1:06:40 pm Steven Hartland wrote: Hi all I'm looking for someone to review the attached patch to mfiutil which adds foreign disk support to mfiutil as per: http://www.freebsd.org/cgi/query-pr.cgi?pr=172091 Any and all feedback welcome :) Some suggestions: - Please stick with FreeBSD style, e.g. please use: if (foo == NULL) rather than: if (NULL == foo) I understand the reasons for the latter style (turn accidental assignments into compile errors) but I don't buy them because 1) modern compilers can already catch such things, but most importantly 2) it doesn't read correctly. Above all else code should be readable, and one doesn't say if NULL the pointer is (unless one is Yoda), but if the pointer is NULL. Acknowledged, I'm working through some old patches so need to bring them in line, will check for others like this. - Don't make dump_config() use a default prefix, just fix the existing call to dump_config() to pass in a prefix. Done. - Is dump_config() really the right choice for 'foreign config'? It doesn't attempt to output things very pretty, and I think mfiutil's non-debug commands should aim to be human readable. Will check this, just didn't want to re-invent the wheel ;-) - This (human readable) is also why it doesn't include the opcode in the error message by default. Sysadmins don't really care which opcode fails. Maybe put that under '#ifdef DEBUG'? Previously there was no information about what command failed, which made the failure message kinda useless, so while debugging I added the opcode to help me trace things. In order to keep it more user friendly I'm thinking of using mfi_op_desc, but this is currently only available under MFI_DEBUG from mfi_debug.c. Would it be better to move this else where e.g. mfireg.h so human readable command information is always available? - mfireg.h should be kept in sync with the driver's version of that header, so don't reorder the enum's unless you are changing it to match what is in the device driver's mfireg.h. In fact, mfiutil should probably be using the mfireg.h from sys/dev/mfi directly now that it is in the tree. (mfiutil was originally developed outside of the tree as a standalone app) There is only one mfireg.h and that is already in sys/dev/mfi :) - Leaving out the 'MFI_DCMD_' prefix from the opcode description was intentional. If you are ever fortunate enough to examine the manuals from LSI, they refer to the firmware commands as 'LD_CONFIG', etc. (Maybe it's 'MR_LD_CONFIG'?) The MFI_DCMD_ prefix is specific to the FreeBSD driver. Thanks for the info, changed. - Please don't do assignments in declarations and leave a blank line between declarations and the bode of code. Thus: mfi_op_desc(...) { int i, num_ops; num_ops = nitems(mfi_op_codes); ... (nitems() is nice to use when it is available as well) Changed, this the case for constant initialisers too? e.g. is the following incorrect or acceptable? myfunc(...) { int i = 0, j = 1; ... - Reindent the call to mfi_ldprobe() if CFG_ADD or CFG_FOREIGN_IMPORT succeeds. Nice spot, sorted. Regards Steve ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Looking for reviewers for patch that adds foreign disk support mfiutil
Hi all I'm looking for someone to review the attached patch to mfiutil which adds foreign disk support to mfiutil as per: http://www.freebsd.org/cgi/query-pr.cgi?pr=172091 Any and all feedback welcome :) Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. z-mfi-foreign-timeout.patch Description: Binary data ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: HDD write cache
Looking at the cam ata code I can't see how those values would do anything after booting. I suspect they should be loader only tunables with the current code and cant be changed on the fly for a disk that's already been registered. Try setting the values in /boot/loader.conf if you haven't already? You can check the actual status of the disk itself using:- camcontrol identify ada0 Regards Steve - Original Message - From: Wojciech Puchar woj...@wojtek.tensor.gdynia.pl To: freebsd-hackers@freebsd.org Sent: Friday, February 01, 2013 2:26 PM Subject: HDD write cache after reading quite recent topics about disabling/enabling write cache, i tried to test in on desktop 3.5 drive kern.cam.ada.write_cache: 1 kern.cam.ada.read_ahead: 1 kern.cam.ada.0.read_ahead: -1 kern.cam.ada.0.write_cache: -1 i tried writing 1 or 0 to kern.cam.ada.0.write_cache, and there were exactly no differences at all, and disk seems to do always write caching. Does that drive lie and ignore commands or i do it wrong? this is my disk. ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: SAMSUNG HD501LJ CR100-10 ATA-8 SATA 2.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 476940MB (976773168 512 byte sectors: 16H 63S/T 16383C) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: HDD write cache
Investigating a bit more a device reset will also trigger the change so after changing the value you can use camcontrol reset to trigger the change to apply using e.g. camcontrol reset 0:0:0 While this is stated in the man for ada its not obvious that changing the sysctl without a reset won't work, so you might want to raise a PR about it. Regards Steve - Original Message - From: Steven Hartland Looking at the cam ata code I can't see how those values would do anything after booting. I suspect they should be loader only tunables with the current code and cant be changed on the fly for a disk that's already been registered. Try setting the values in /boot/loader.conf if you haven't already? You can check the actual status of the disk itself using:- camcontrol identify ada0 Regards Steve - Original Message - From: Wojciech Puchar after reading quite recent topics about disabling/enabling write cache, i tried to test in on desktop 3.5 drive kern.cam.ada.write_cache: 1 kern.cam.ada.read_ahead: 1 kern.cam.ada.0.read_ahead: -1 kern.cam.ada.0.write_cache: -1 i tried writing 1 or 0 to kern.cam.ada.0.write_cache, and there were exactly no differences at all, and disk seems to do always write caching. Does that drive lie and ignore commands or i do it wrong? this is my disk. ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: SAMSUNG HD501LJ CR100-10 ATA-8 SATA 2.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 476940MB (976773168 512 byte sectors: 16H 63S/T 16383C) This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: disadvantages of running 8.3 kernel on freebsd 8.2 system
- Original Message - From: Devin Teske devin.te...@fisglobal.com You're the perfect person to help us figure out why when we: 1. back-port mfi(4) from stable/8 into releng/8.3 (8.3-RELEASE-p5 at the time of back-port) 2. Succeed in getting 8.3 to boot on Thunderbolt card 3. mfiutil produces Inappropriate ioctl for device Even after… 4. Recompiling mfiutil from stable/8 (albeit in a releng/8.3 build environment -- back ported headers applied for new macros even) Any hints on where to go next to restore mfiutil access? You also need to backport mfiutil too. I have a patch set for 8.3 which include latest mfi driver, mfiutil and a large set of mfi driver fixes, even head has some rather serious issues although mainly around error handling on tbolt cards. We're running this in production so believe its a good stable set. If you would like the patch set just let me know. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: disadvantages of running 8.3 kernel on freebsd 8.2 system
- Original Message - From: dte...@freebsd.org But I bet you're not sitting 88 units of Thunderbolt cards that don't work in 8.3. 8.3 is also exhibiting major problems with the igb-based NICs on those same 88 units. Only effects igb init but might want to make sure you have r245334 back ported to avoid memory leaks when mbuf clusters are exhausted. 8.3 version of the patch attached ;-) For reference not only does this prevent the nic initialising properly it can also hang the boot process as when routing initialises route appears to trigger mbuf allocation with wait and as mbufs are exhaused and not freed correctly this hangs forever. This will happen on an untuned kernel if more than 2 igb nics are configured as each igb requires 8k of mbuf clusters (1k per queue x 8 queues on a machine with 8 or more cores) and the default kern.ipc.nmbclusters is only 25600. For clarity by configured I mean if the nic is initialised either by assigning an IP or ifconfig igbX up the queues are not allocated if the nic is present but unused. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. Fixed mbuf free when receive structures fail to allocate. This prevents quad igb card on high core machines, without any nmbcluster or igb queue tuning wedging the boot process if all nics are configured. --- sys/dev/e1000/if_igb.c.orig 2013-01-10 21:44:03.017805977 + +++ sys/dev/e1000/if_igb.c 2013-01-10 21:44:55.751355018 + @@ -4335,8 +4335,8 @@ * the rings that completed, the failing case will have * cleaned up for itself. 'i' is the endpoint. */ - for (int j = 0; j i; ++j) { - rxr = adapter-rx_rings[i]; + for (int j = 0; j i; ++j) { + rxr = adapter-rx_rings[j]; IGB_RX_LOCK(rxr); igb_free_receive_ring(rxr); IGB_RX_UNLOCK(rxr); ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: kgzip(1) is broken
- Original Message - From: dte...@freebsd.org To: 'Ian Lepore' free...@damnhippie.dyndns.org Cc: freebsd-hackers@freebsd.org; dte...@freebsd.org Sent: Wednesday, January 16, 2013 12:56 AM Subject: RE: kgzip(1) is broken -Original Message- From: Ian Lepore [mailto:free...@damnhippie.dyndns.org] Sent: Tuesday, January 15, 2013 4:43 PM To: Devin Teske Cc: dte...@freebsd.org; freebsd-hackers@freebsd.org Subject: RE: kgzip(1) is broken On Tue, 2013-01-15 at 16:10 -0800, Devin Teske wrote: -Original Message- From: Devin Teske [mailto:devin.te...@fisglobal.com] On Behalf Of dte...@freebsd.org Sent: Tuesday, January 15, 2013 3:10 PM To: 'Ian Lepore' Cc: freebsd-hackers@freebsd.org; dte...@freebsd.org Subject: RE: kgzip(1) is broken -Original Message- From: Ian Lepore [mailto:free...@damnhippie.dyndns.org] Sent: Tuesday, January 15, 2013 3:05 PM To: dte...@freebsd.org Cc: freebsd-hackers@freebsd.org Subject: Re: kgzip(1) is broken On Tue, 2013-01-15 at 13:27 -0800, dte...@freebsd.org wrote: Hello, I have been sad of-late because kgzip(1) no longer produces a usable kernel. All versions of 9.x suffer this. And somewhere between 8.3-RELEASE-p1 and 8.3-RELEASE-p5 this recently broke in the 8.x series. I haven't tried the 7 series lately, but if whatever is making the rounds gets MFC'd that far back, I expect the problem to percolate there too. The symptom is that the machine reboots immediately and unexpectedly the moment the kernel is executed by the loader. This is quite troubling and I am looking for someone to help find the culprit. I don't know where to start looking. Here are some possible candidates from the things that were MFC'd to 8 in that timeframe. I haven't looked at what these do, they're just changes that affect files related to booting. r233211 r233377 r233469 r234563 Thanks Ian! I'll test each one individually to see if regressing any one (or all) addresses the problem. Progress... Looks like I found the culprit. Turns out it's a back-ported bxe(4) driver (back-ported from 9 -- where kgzip seems to never work). I wonder why back-porting bxe(4) from stable/9 to releng/8.3 would cause kgzip to produce non-working kernels. Yeah, it'll be interesting to see how a device driver can lead to the machine reboots immediately and unexpectedly the moment the kernel is executed by the loader, which I took to mean before seeing the copyright or anything. Indeed... loader throws up the syms and upon execution *KABOOM* (screen goes black and back to POST) The copyright never appears. I'm emailing the maintainers (davidch + other Broadcom folk) The current dossier is even more interesting... the back-ported driver (with zero modifications mind you from stable/9 to stable/8) exhibits memory failures (example below), and causes terminals to become wedged when attempting to (for example) scp a file over an existing configured network (igb-based -- presumably unrelated to bxe but in practice loading bxe causes igb to misbehave). $ ifconfig bxe0 inet 192.168.1.5/24 bxe0: ../../../dev/bxe/if_bxe.c(10939): Memory allocation failure! Cannot fill fp[00] RX chain. bxe0: ../../../dev/bxe/if_bxe.c(3921): NIC initialization failed, aborting! $ ifconfig bxe1 inet 192.168.1.6/24 bxe1: ../../../dev/bxe/if_bxe.c(10939): Memory allocation failure! Cannot fill fp[00] RX chain. bxe1: ../../../dev/bxe/if_bxe.c(3921): NIC initialization failed, aborting! (as expected, also sent mail off to maintainers w/respect to above notes/errors) Sounds like you may be out of mbufs which is easy, on a box with 4 igb's simply booting without tuning with cause this so, if you have igb's and bxe's this could be your cause. Try adding the following to loader.conf and see if it helps:- kern.ipc.nmbclusters=51200 Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Threaded 6.4 code compiled under 9.0 uses a lot more memory?..
- Original Message - From: Karl Pielorz kpielorz_...@tdx.co.uk To: freebsd-hackers@freebsd.org Sent: Tuesday, October 30, 2012 11:12 AM Subject: Threaded 6.4 code compiled under 9.0 uses a lot more memory?.. Hi All, Can anyone think of any quick pointers as to why some code originally written under 6.4 amd64 - when re-compiled under 9.0-stable amd64 takes up a *lot* more memory when running? The code involved is a sendmail Milter, and a TCP server type program (that runs up a large number of threads [~700] at startup). Both were previously compiled with: -O2 -pthread -lc_r They're now compiled under 9.0-S with just: -O2 -pthread As an example, under 6.4 the size/res for one reported by top is 81Mb/48Mb - and for the other is 44Mb/9Mb [this is after it's been running for weeks]. Under 9.0-stable the initial memory used by the processes just after starting rises to 625Mb/128Mb and 519Mb/65Mb respectively. Is that something I need to worry about? They've not been running longing enough yet to see if anything is 'leaking' (i.e. if size/res continues to go up). Just thought I'd ask if there's a simple/possible explanation for this - and if it's something I need to worry about... amd64 vs i386? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Looking for testers / feedback for ZFS recieve properties options
We encountered a problem receiving a full ZFS stream from a disk we had backed up. The problem was the receive was aborting due to quota being exceeded so I did some digging around and found that Oracle ZFS now has -x and -o options as documented here:- http://docs.oracle.com/cd/E23824_01/html/821-1462/zfs-1m.html Seems this has been raised as a feature request upstream: https://www.illumos.org/issues/2745 Anyway being stuck with a backup we couldn't restore I had a play a implementing these options and have a prototype up and running, which I'd like feedback on. This patch also adds a -l option which allows the streams to be limited to those specified. Another option which I think would be useful and seemed relatively painless to add. The initial version of the patch which is based off 8.3-RELEASE can be found here: http://blog.multiplay.co.uk/dropzone/freebsd/zfs-recv-properties.patch Any feedback appreciated Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: AHCI Timeouts on SATA III with Intel 520 SSDs
Did you get anywhere with this? Seeing a similar thing on some new Patsburg based machines with KINGSTON SSD's on 8.3-RELEASE. Regards Steve - Original Message - From: Caza, Aaron aaron.c...@ca.weatherford.com To: freebsd-hackers@freebsd.org Sent: Monday, February 13, 2012 9:58 PM Subject: AHCI Timeouts on SATA III with Intel 520 SSDs I've got a couple of Intel 520 SSDs that I'm running on an Intel Sandy-bridge based system(Core i5-2500K H67 chipset). Unfortunately, the drives experience AHCI Timeouts when connected to the SATA III ports. If, however, I connect the drives to the SATA-II ports on the same system the drives do not timeout. NCQ is enabled. Below is the complete dmesg showing the issue. For my testing, I'm just using a FreeBSD 9.0 Release (amd64) generic kernel using 'dd if=/dev/ada0 of=/dev/null bs=1m' to exhibit the behavior. The drives, ofcourse, are brand new and again if I run them off the SATA-II ports instead of the SATA-III ports the problem goes away but then so does the performance. Suggestions? gpart show ada0: = 34 234441581 ada0 GPT (111G) 34128 1 freebsd-boot (64k) 162 232783872 2 freebsd-ufs (111G) 2327840341657581- free - (809M) camcontrol identify ada0: pass0: INTEL SSDSC2CW120A3 400i ATA-9 SATA 3.x device pass0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) protocol ATA/ATAPI-9 SATA 3.x device model INTEL SSDSC2CW120A3 firmware revision 400i serial number WWN 5001517bb27d76f7 cylinders 16383 heads 16 sectors/track 63 sector size logical 512, physical 512, offset 0 LBA supported 234441648 sectors LBA48 supported 234441648 sectors PIO supported PIO4 DMA supported WDMA2 UDMA6 media RPM non-rotating Feature Support Enabled Value Vendor read ahead yes yes write cacheyes yes flush cacheyes yes overlapno Tagged Command Queuing (TCQ) no no Native Command Queuing (NCQ) yes32 tags SMART yes yes microcode download yes yes security yes no power management yes yes advanced power management yes yes 254/0xFE automatic acoustic management no no media status notification no no power-up in Standbyyes no write-read-verify no no unload yes yes free-fall no no data set management (TRIM) yes dmesg: Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-RELEASE #0: Tue Jan 3 07:46:30 UTC 2012 r...@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 CPU: Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz (3292.59-MHz K8-class CPU) Origin = GenuineIntel Id = 0x206a7 Family = 6 Model = 2a Stepping = 7 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0x179ae3bfSSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,TSCDLT,AESNI,XSAVE,AVX AMD Features=0x28100800SYSCALL,NX,RDTSCP,LM AMD Features2=0x1LAHF TSC: P-state invariant, performance statistics real memory = 17179869184 (16384 MB) avail memory = 16459304960 (15696 MB) Event timer LAPIC quality 600 ACPI APIC Table: Shuttl Shuttle FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 2 cpu2 (AP): APIC ID: 4 cpu3 (AP): APIC ID: 6 ioapic0 Version 2.0 irqs 0-23 on motherboard kbd1 at kbdmux0 acpi0: Shuttl Shuttle on motherboard acpi0: Power Button (fixed) Timecounter ACPI-fast frequency 3579545 Hz quality 900 acpi_timer0: 24-bit timer at 3.579545MHz port 0x408-0x40b on acpi0 cpu0: ACPI CPU on acpi0 cpu1: ACPI CPU on acpi0 cpu2: ACPI CPU on acpi0 cpu3: ACPI CPU on acpi0 pcib0: ACPI Host-PCI bridge port 0xcf8-0xcff on acpi0 pci0: ACPI PCI bus on pcib0 pcib1: ACPI PCI-PCI bridge irq 16 at device 1.0 on pci0 pci1: ACPI PCI bus on pcib1 vgapci0: VGA-compatible display port 0xf000-0xf03f mem 0xfe00-0xfe3f,0xc000-0xcfff irq 16 at device 2.0 on pci0 pci0: simple comms at device 22.0 (no driver attached) ehci0: EHCI (generic) USB 2.0 controller mem 0xfe603000-0xfe6033ff irq 16 at device 26.0 on pci0 usbus0: EHCI version 1.0 usbus0: EHCI (generic) USB 2.0 controller on ehci0 pcib2: ACPI PCI-PCI bridge irq 17 at device 28.0 on pci0 pci2: ACPI PCI bus on pcib2 pcib3: ACPI PCI-PCI bridge irq 16 at device 28.1 on pci0 pci3: ACPI PCI bus on pcib3 xhci0: XHCI (generic) USB 3.0
Re: AHCI Timeouts on SATA III with Intel 520 SSDs
That's very useful and could well be what we have here. I've got two new machines which run the SSD's fine @ SATA 3 on port 1 but an identical disk on port 2 is having issues. If we switch it down to SATA 2 and all is good. So sounds like the next move is to switch the disks round and see if the problem stays with the disk or the port. Here's hoping its the disk or the cable and not the controller. Regards Steve - Original Message - From: Caza, Aaron aaron.c...@ca.weatherford.com To: Steven Hartland kill...@multiplay.co.uk; freebsd-hackers@freebsd.org Sent: Friday, July 27, 2012 5:07 PM Subject: RE: AHCI Timeouts on SATA III with Intel 520 SSDs Yes. In my case, the problem turned out to be a marginal SATA-III port on the motherboard which was determined after swapping SSDs, SATA cables, etcetera to finally pin down the problem. When trouble-shooting this issue, I recall googling a particular missive by Alexander Motion in which he indicates these problems are potentially due to any number of hardware-related reasons hence the exhaustive search for the culprit which, as he suggested, did indeed turn out to be the hardware.It's actually rather interesting how borderline hardware can be - the port in question could handle an Intel 510 SSD running at full SATA-III speed but an Intel 520 pushed it over the brink. This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: geom - cam disk
- Original Message - From: Andriy Gapon a...@freebsd.org on 26/07/2012 01:08 Alexander Motin said the following: Different controllers have different command queueing limitations. If you are testing with ahci(4) driver and modern disks, then their 32 command slots per port can be enough for many workloads to enqueue all commands to the hardware and leave queue empty as you've described. But if you take harder workload, or controller/ device without command queueing support, extra requests will be accumulated on that bioq and sorted there. Alexander, thank you for the reply. Indeed, using 64 parallel dd processes with bs=512 I was able to 'kick in' the disksort logic. But I am not sure if the disksort algorithm makes much difference in this case given the number of commands that a disk firmware can internally re-order. (Not mentioning that potentially disksort could starve some I/O bound processes in favor of others -- but that's a totally different topic). But then, of course, for the less capable hardware the disksort could still be a significant factor. The sort is actually important for delete requests too as this can allow the delete processing code to operate more effectively which can result in significant performance increases if this then allows request combining. For example Alexander is currently reviewing some changes I've written to the delete processing which include an optimisation that increases SSD delete performance from ~630MB/s to 1.3GB/s on 3rd gen sandforce controllers. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
dtrace crashes on ustack
When running dtrace looking at syscall::write:return for a java app with a ustack call for the first few calls I get:- dtrace: ERROR: gelf_getehdr() failed: No error: 0 And then after a number dtrace crashes:- #0 0x000800fcc59d in _malloc_prefork () from /lib/libc.so.7 [New Thread 80283c500 (LWP 100815/dtrace)] [New Thread 8012041c0 (LWP 101294/initial thread)] (gdb) bt #0 0x000800fcc59d in _malloc_prefork () from /lib/libc.so.7 #1 0x000800fcdd2d in _malloc_thread_cleanup () from /lib/libc.so.7 #2 0x00080066190e in pthread_exit () from /lib/libthr.so.3 #3 0x000800658523 in pthread_getprio () from /lib/libthr.so.3 #4 0x in ?? () Cannot access memory at address 0x7fbff000 Kernel is compiled with dtrace, world has been rebuild with WITH_CTF=1 as has openjdk7 port. Any ideas? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
dtrace filename lookups from fd
As a first foray into dtrace I wanted to create a little script which shows the amount of disk read / write activity. Now the DtraceToolkit includes rwsnoop but this uses Solaris specific requests and on looking around it seems like using rwsnoop vn_fullpath may be the way to go. Has anyone done anything similar to this before or has any tips on going about this? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: proper newfs options for SSD disk
- Original Message - From: Wojciech Puchar woj...@wojtek.tensor.gdynia.pl On 2012-May-18 22:54:43 +0200, Dimitry Andric d...@freebsd.org wrote: Be sure to use -t enable when creating the filesystem: Only if your SSD supports TRIM. Some consumer-grade SSDs don't and get very confused if sent TRIM commands. mine do. The disk also has be be connected to a disk arch which supports BIO_DELETE which ATM is only ata unless your running HEAD which also has support in da Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Ways to promote FreeBSD?
- Original Message - From: Mehmet Erol Sanliturk My opinion is that most important obstacle in front of FreeBSD is its installation structure : It is NOT possible to install and use a FreeBSD distribution directly as it is . I disagree, we find quite the opposite; FreeBSD's current install is perfect its quick, doesn't install stuff we don't need and leaves a very nice base. Linux on the other had takes ages, asks way to many questions, has issues with some hardware with mouse and gui not work properly making the install difficult to navigate, but most import its quite hard to get a nice simple base as there are so many options, which is default with FreeBSD. In essence it depends on what you want and how you use the OS. For the way we use FreeBSD on our servers its perfect. So if your trying to suggest its not suitable for all that's is incorrect as it depends on what you want :) Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: 8 to 9: A longer wait early in the boot of a (damaged) CompaqPresario
- Original Message - From: Alex Goncharov alex-goncha...@comcast.net About a week ago, I made a jump and upgraded the system's FreeBSD from version 8 to 9. Everything is great (I am typing this message on that machine now) but the boot pause after the (looking new in 9) boot menu is *much* longer now -- it will show the '\' character and wait for, subjectively, half a minute before putting anything else on the screen. Two things spring to mind which could help: 1. The reduce the slice sampling size in sys/boot/zfs/zfs.c which was increased recently 2. disable boot time mem tests using the attached patch Patches for both from 8.2-RELEASE attached. For #2 you also need the following in /boot/loader.conf hw.memtest.tests=0 Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. --- sys/amd64/amd64/machdep.c 2011/06/08 08:12:15 222853 +++ sys/amd64/amd64/machdep.c 2011/07/30 13:33:05 224516 @@ -1309,7 +1309,7 @@ { int i, physmap_idx, pa_indx, da_indx; vm_paddr_t pa, physmap[PHYSMAP_SIZE]; - u_long physmem_tunable; + u_long physmem_tunable, memtest, tmpul; pt_entry_t *pte; struct bios_smap *smapbase, *smap, *smapend; u_int32_t smapsize; @@ -1372,6 +1372,14 @@ Maxmem = atop(physmem_tunable); /* +* By default keep the memtest enabled. Use a general name so that +* one could eventually do more with the code than just disable it. +*/ + memtest = 1; + if (TUNABLE_ULONG_FETCH(hw.memtest.tests, tmpul)) + memtest = tmpul; + + /* * Don't allow MAXMEM or hw.physmem to extend the amount of memory * in the system. */ @@ -1433,6 +1441,8 @@ goto do_dump_avail; page_bad = FALSE; + if (memtest == 0) + goto skip_memtest; /* * map page into kernel: valid, read/write,non-cacheable @@ -1470,6 +1480,7 @@ */ *(int *)ptr = tmp; +skip_memtest: /* * Adjust array of valid/good pages. */ --- sys/boot/zfs/zfs.c.orig 2011-10-20 18:15:29.966685430 + +++ sys/boot/zfs/zfs.c 2011-10-20 18:18:22.291033636 + @@ -45,6 +45,12 @@ #include zfsimpl.c +/* + * For GPT this should be 128 but leads to 50+ second delay in BTX loader so + * we use the original 4 pre r198420 by default for the boot process + */ +#define ZFS_MAX_SLICES 4 + static int zfs_open(const char *path, struct open_file *f); static int zfs_write(struct open_file *f, void *buf, size_t size, size_t *resid); static int zfs_close(struct open_file *f); @@ -415,7 +421,7 @@ if (vdev_probe(vdev_read, (void*) (uintptr_t) fd, 0)) close(fd); - for (slice = 1; slice = 128; slice++) { + for (slice = 1; slice = ZFS_MAX_SLICES; slice++) { sprintf(devname, disk%dp%d:, unit, slice); fd = open(devname, O_RDONLY); if (fd == -1) { ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD has serious problems with focus, longevity, and lifecycle
- Original Message - From: John Kozubik j...@kozubik.com It's amazing how many people are in the exact same boats - waiting for 8.3, getting locked out of new motherboards because em(4) can't be backported to even the production release... This is not true, only last week did we take the version of e1000 from HEAD into our 8.2-RELEASE tree as a patch. It wasnt totally trivial but it also wasnt difficult either. But it would still be preferable for many not to have to do this I assume? This import brings the number of number release patches we manually apply to our machines above 8.2-RELEASE to 18 which includes:- updated areca driver, boot time fixes (disable memtest), devfs startup fix ixgbe e100 drivers, libz assembly crash fix, mps driver import, rc.subr fix for scripts, increased max swap size, tcp reassembly fix, udp6 fix for java, cam timeout fixes, zfs overflow fix, zfs slice boot delay, camcontrol security options for ssds and jail uref panic fix. I'm sure there are more that others would include but these changes are important enough to our environment to prompt their inclusion in what is effectively our own stream of 8.2-RELEASE. Now I know the factastic work commiters do in bring us FreeBSD so I can't bring myself to critisise in anyway the work they do. But its defintiely interesting to see others are in the same boat as us looking for something thats a bit more predicable than the current large change releases. I wonder if there is something that could be created to make maintaining micro branches easier. I know mfsbsd has made our lives so much simpler since we discovered it allowing us to take a standard source build on one machine and roll an install cd customised to our requirements in minutes. What other undiscovered gems are our out there? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD has serious problems with focus, longevity, and lifecycle
- Original Message - From: Hans Petter Selasky hsela...@c2i.net On Tuesday 17 January 2012 20:54:51 Steven Hartland wrote: boot time fixes (disable memtest), Hi, Another noticeable part is that ufsread.c in boot2 uses very small block sizes to read the file system data. If that could be fixed boot times would drop too ! It might but not the same order of magnitude I suspect as this was like 30-60+ delay depending on memory size ;-) Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD has serious problems with focus, longevity, and lifecycle
- Original Message - From: John Kozubik j...@kozubik.com To: freebsd-hackers@freebsd.org Sent: Monday, January 16, 2012 10:28 PM Subject: FreeBSD has serious problems with focus, longevity, and lifecycle Friends, I was disappointed to see that 8.3-RELEASE is now slated to come out in March of 2012. This will be ~13 months since 8.2-RELEASE and is typical of a trend towards longer gaps between minor releases. ... I must say as a small company that runs ~200 machines on FreeBSD I do see where John is coming from, as it is very time consuming to keep things up to date and new is not always better e.g. we still have boxes stuck on 6.x as issues introduced in the Linux compat after that caused problems. That said I'm in two minds as the features that have been brought in by the more rapid dev cycle like ZFS have been great. Where I do see an issue is where it feels like we've just got to a solid 8.2 release with p6 and some addition patches we see things like em driver updates required to run newer hardware only in 9. While we might like to push everything to 9 it brings with it a large amount of untested changes like the HPN patches to core ssh which we have seen problems with under openssh-portable when tested. So this puts us in a dilemma, push to 9 and keep up to date or stick with 8.2 with custom patches while we wait for 8.3 which we know is good and assuming it has the patches need included in it? The correct answer for us is currently unknown and is still being debated, but in the mean time we are going to keep with 8.2 until we've had the chance to fully evaluate what 9 has to offer. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Invalid memory stats from vmstat and sysctl vm.vmtotal?
- Original Message - From: Jason Hellenthal jh...@dataix.net Just to put some visuals to this... . `-- DIE |-- Core1 [Idle] |-- Core2 [35% ] | `-- thread127 |-- Core3 [40% ] | `-- thread127 `-- Core4 [100%] `-- thread127 In this case you would say the DIE should be at a total of 175% ? I think your getting confused there; it sounds like your referring to a single CPU capable of multiple tasks via either multiple cores or HTT as an UP machine? If so that's your problem this isn't UP it SMP. Have a look on your machine in /var/run/dmesg.boot if your see it reporting more than one core then your SMP not UP hence the confusion as each of these cores be they real, virtual or physical represents a possible thread of 100% so even if you have a single physical CPU with 4 cores that still represents a possible total of 400% e.g the following shows a machine capable of 1600% if all cores are busy. FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 SMT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 cpu8 (AP): APIC ID: 16 cpu9 (AP): APIC ID: 17 cpu10 (AP): APIC ID: 18 cpu11 (AP): APIC ID: 19 cpu12 (AP): APIC ID: 20 cpu13 (AP): APIC ID: 21 cpu14 (AP): APIC ID: 22 cpu15 (AP): APIC ID: 23 If you want proper UP which will total 100% you could remove SMP from your kernel but I wouldnt advise that ;-) Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Invalid memory stats from vmstat and sysctl vm.vmtotal?
- Original Message - From: Andriy Gapon a...@freebsd.org Totalling up RSS from ps axo rss gives a total in the region of that if the vm stats are out by a factor of 4, in this case it should be: 8132557 which is 7.75GB a much more realistic value. Am I totally missing something or is there problem here? Likely more of the former than of latter. Those virtual sizes are not sufficiently explained, but you have been warned that those are not physical sizes, so I am not sure why you try to compare the virtual figures with the physical figures. My miss-understanding was due to what virtual actually meant. Here's an example. Let' say you mmap-ed a 1GB file into a process memory space, here you immediately increased your virtual size counts by 1GB, even if you hadn't accessed any bytes in the file yet and so none of them were in physical memory. The same applies to anonymous memory. P.S. the above is reveled by a cursory look through the code (which is publicly available btw) :-) Yer I did have a dig around before posting and ended up the code for vm.vmtotal, which is where vmstat gets its info from but that's just a summation of each object's size from vm_object_list. Thats where I got lost without an insight into what a vm_object.size actually represents. Your info about mmap'ed files helped point me in the right direction as it identified space that shows as virtual but doesn't show in swap or real ram, which is what I was missing. Given this starting point the following links provided me with addtional information:- http://www.freebsd.org/doc/en/books/arch-handbook/vm.html http://www.freebsd.org/doc/en/books/design-44bsd/overview-memory-management.html http://www.freebsd.org/doc/en_US.ISO8859-1/articles/vm-design/ http://www.cse.chalmers.se/edu/course/EDA203/unix4.pdf I was under the incorrect impression that Virtual Memory (VM) was so named as it was a unified physical memory and swap (virtual memory), but its not that simple, as other items such as file-backed objects also count to this total which would never show in physical or swap allocation of other tools such as top and swapinfo. So what I believe is now the big cause of virtual memory uplift vs the memory totals shown by ps / top is that the vm totals include things like file backed memory mapped process binaries, shared libs etc many multiple times. This would explain why this specific machine shows the applification more than others here as it runs thousands of very small lightweight processes. Thanks for pointer Andy, I now understand a lot more about the BSD VMS :) What do people think about expanding that entry in the man page of vmstat to clarify just what active virtual pages really means? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Invalid memory stats from vmstat and sysctl vm.vmtotal?
- Original Message - From: Jason Hellenthal jh...@dataix.net This goes along with the thoughts I had about 4 months ago tending to some zfs statistics as well top showing greater than 100% actual CPU usage. This is a big pet peave of mine. Its like saying you ate 134% of a bannanna when in all reallity it is impossible. You can never have more than 100% usage of anything and when seen is a clear notice that some math is considerably incorrect leading to other such miscalculations to be performed. Things like the above already have checks in place that ensure no boundries are being crossed/overflowed or underrun but it surely makes processing results building future products a bitch. One instance is the calculation of threads for example firefox can be seen using upto or more 338% of the CPU. Thats impossible its like saying anyones CPU grew by 400%. I could understand a bit of overflow as stats are snapshots which may not be instuntanious, but 31GB instead of under 8GB is hardly a rounding issue / overflow. With respect to top showing greater than 100% by how much are you talking? Do your realise that each core = 100%? So if you have a quad core your system total will be 400% not 100%? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Invalid memory stats from vmstat and sysctl vm.vmtotal?
- Original Message - From: Damien Fleuriot m...@my.gd I could understand a bit of overflow as stats are snapshots which may not be instuntanious, but 31GB instead of under 8GB is hardly a rounding issue / overflow. With respect to top showing greater than 100% by how much are you talking? Do your realise that each core = 100%? So if you have a quad core your system total will be 400% not 100%? That's his point, you cannot use 400% of a system as a whole, his point is that top should report 100% where each core accounts for 25% Then I would have to disagree, keeping 100% to mean 100% of a single core is much easier to manage than 100% of a machines total capacity. If you went to 100% = the machine total capacity processes could be using a lot of cpu without even registering 1% on today's machines where 24 cores is common place. It also makes detecting single process / thread bottlenecks easier as if your seeing 100% you know its maxing a core, instead of having to calculate it once you know how many cores the machine has. If your looking for total machine usage then that's also their in the summary at the top of the screen e.g. CPU states: 13.6% user, 0.0% nice, 1.3% system, 0.0% interrupt, 85.1% idle Anyway this is quite off topic, and I don't want to loose sight of the threads goal which is to determine why we can see 31GB of usage on an 8GB machine with very little shared memory usage an no swap usage. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Invalid memory stats from vmstat and sysctl vm.vmtotal?
- Original Message - From: RW rwmailli...@googlemail.com To: freebsd-hackers@freebsd.org Sent: Thursday, December 01, 2011 1:58 PM Subject: Re: Invalid memory stats from vmstat and sysctl vm.vmtotal? On Wed, 30 Nov 2011 12:39:10 - Steven Hartland wrote: We're seeing some impossible memory usage stats reported on machines here from vmstat and sysctl vm.vmtotal. We have machines reporting to be using 31GB total when they only have 8GB physical and are not using any swap. Here's an output from one of our machines:- vmstat -c 2 -w 1 -n 0 procs memory page faults cpu r b w avmfre flt re pi pofr sr in sy cs us sy id 0 0 0 31768M 2112M 586 0 0 0 421 0 106 270 569 0 6 94 0 0 0 31768M 2112M 2 0 0 0 0 0 370 8139 3996 0 1 99 ... It looks like it may be out by a factor of 4, possibly due to the fact the its a 4k page size not 1k as indicated by the vmstat man page:- I don't think so, I have 16GB and tops shows: Mem: 817M Active, 396M Inact, 1364M Wired, 82M Cache, 1282M Buf, 13G Free but vmstat -c 2 -w 1 -n 0 procs memory page faults cpu r b w avmfre flt re pi pofr sr in sy cs us sy id 0 0 0 9750M13G 450 5 3 1 560 0 234 50201 5206 2 2 96 1 0 0 9742M13G79 4 0 0 573 0 239 51886 4700 0 1 99 Hmm, yes on another machine same OS with 16GB memory we see:- dmesg | grep memory real memory = 17179869184 (16384 MB) avail memory = 16536604672 (15770 MB) vmstat -c 2 -w 1 -n 0 procs memory page faults cpu r b w avmfre flt re pi pofr sr in sy cs us sy id 0 0 0 1948M 666M 395 14 0 0 398 1966 3237 5980 14085 2 4 93 1 0 0 1948M 636M 4 0 0 016 0 8215 18660 35322 2 7 92 sysctl vm.vmtotal vm.vmtotal: System wide totals computed every five seconds: (values in kilobytes) === Processes: (RUNQ: 1 Disk Wait: 0 Page Wait: 0 Sleep: 194) Virtual Memory: (Total: 1075975332K Active: 1979056K) Real Memory:(Total: 309048K Active: 160804K) Shared Virtual Memory: (Total: 75112K Active: 18860K) Shared Real Memory: (Total: 15656K Active: 10572K) Free Memory Pages: 731960K but with top:- last pid: 38187; load averages: 0.15, 0.16, 0.16 up 2+23:33:55 15:13:02 195 processes: 1 running, 194 sleeping CPU: 0.0% user, 0.0% nice, 4.1% system, 0.0% interrupt, 95.9% idle Mem: 886M Active, 12G Inact, 2194M Wired, 599M Cache, 1630M Buf, 138M Free Swap: 8192M Total, 1512K Used, 8190M Free So I've no idea whats going on? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Invalid memory stats from vmstat and sysctl vm.vmtotal?
We're seeing some impossible memory usage stats reported on machines here from vmstat and sysctl vm.vmtotal. We have machines reporting to be using 31GB total when they only have 8GB physical and are not using any swap. Here's an output from one of our machines:- vmstat -c 2 -w 1 -n 0 procs memory page faults cpu r b w avmfre flt re pi pofr sr in sy cs us sy id 0 0 0 31768M 2112M 586 0 0 0 421 0 106 270 569 0 6 94 0 0 0 31768M 2112M 2 0 0 0 0 0 370 8139 3996 0 1 99 The raw output is:- vmstat -c 2 -w 1 -n 0 -H procs memory page faults cpu r b w avmfre flt re pi pofr sr in sy cs us sy id 0 0 0 32530228 2162524 586 0 0 0 421 0 106 270 569 0 6 94 0 0 0 32530228 2162524 2 0 0 0 0 0 286 8234 4347 0 1 99 Top shows:- last pid: 6665; load averages: 0.00, 0.00, 0.01 up 80+01:24:12 09:35:28 1893 processes:1 running, 1892 sleeping CPU: 0.0% user, 0.0% nice, 0.3% system, 0.0% interrupt, 99.7% idle Mem: 3754M Active, 84M Inact, 1976M Wired, 4K Cache, 2109M Free Swap: 4096M Total, 4096M Free sysctl vm.vmtotal vm.vmtotal: System wide totals computed every five seconds: (values in kilobytes) === Processes: (RUNQ: 1 Disk Wait: 0 Page Wait: 0 Sleep: 1893) Virtual Memory: (Total: 1106403532K Active: 32540260K) Real Memory:(Total: 4563648K Active: 3921644K) Shared Virtual Memory: (Total: 19976K Active: 16396K) Shared Real Memory: (Total: 9040K Active: 8436K) Free Memory Pages: 2161740K As mentioned this machine has 8GB of ram and according to both top and swapinfo is using no swap at all From dmesg:- real memory = 8589934592 (8192 MB) avail memory = 823536 (7873 MB) swapinfo Device 1K-blocks UsedAvail Capacity /dev/gptid/09f211f7-39ce-11e0-8 41943040 4194304 0% uname -a FreeBSD test 8.2-RELEASE FreeBSD 8.2-RELEASE #2: Thu Mar 24 17:28:55 UTC 2011 root@test:/usr/obj/usr/src/sys/MULTIPLAY amd64 sysctl hw.pagesize hw.pagesize: 4096 It looks like it may be out by a factor of 4, possibly due to the fact the its a 4k page size not 1k as indicated by the vmstat man page:- memory Information about the usage of virtual and real memory. Virtual pages (reported in units of 1024 bytes) are considered active if they belong to processes which are running or have run in the last 20 seconds. avm active virtual pages fre size of the free list Totalling up RSS from ps axo rss gives a total in the region of that if the vm stats are out by a factor of 4, in this case it should be: 8132557 which is 7.75GB a much more realistic value. Am I totally missing something or is there problem here? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: iotop (dtrace?)
top and then use the IO mode, will give you an idea where the issue is. Regards Steve - Original Message - From: Stefan Bethke s...@lassitu.de To: freebsd-hackers@freebsd.org Sent: Tuesday, October 25, 2011 9:34 PM Subject: iotop (dtrace?) I've got two systems with a constantly high rate of disk I/O that sometimes seems to be overwhelmed from it. Before trying to decide if a hardware upgrade will help, I'd like to figure out which processes generate the load. I've found a couple scripts named iotop which appear to produce what I would be interested in, but they appear to require Solaris or Linux. Has someone ported over one of them, or would have a suggestion how to go about writing a custom dtrace script to gather this kind of information? I can successfully run a couple of sample dtrace scripts on these 8-stable amd64 boxes. Thanks, Stefan Solaris dtrace-based iotop: http://prefetch.net/articles/solaris.dtracetopten.html Linux /proc-based Python script: http://guichaz.free.fr/iotop/ -- Stefan Bethke s...@lassitu.de Fon +49 151 14070811 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: debugging frequent kernel panics on 8.2-RELEASE
- Original Message - From: Andriy Gapon a...@freebsd.org Thats interesting, are you using http as an example or is that something thats been gleaned from the debugging of our output? I ask as there's only one process running in each of our jails and thats a single java process. It's from the debug data: p_comm = httpd Hmm, there's only one httpd thats ever run on the machine and thats not in the jail its on the raw machine. I also would like to ask you to revert the last patch that I sent you (with tf_rip comparisons) and try the patch from Kostik instead. Sure. Given what we suspect about the problem, can please also try to provoke the problem by e.g. doing frequent jail restarts or something else that supposedly should hit the bug. I've tried doing this for quite some days on the test machine, but I've been unable to provoke it, will continue to try. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: debugging frequent kernel panics on 8.2-RELEASE
- Original Message - From: Andriy Gapon a...@freebsd.org Probably I have mistakenly assumed that the 'prison' in prison_derefer() has something to do with an actual jail, while it could have been just prison0 where all non-jailed processes belong. That makes sense as this particular panic was caused by a machine reboot, which is slightly different from the more common jail panic we're seeing. Doesn't help with our reproduction scenario though unfortunately. If we don't have any joy reproducing on our single test machine I'll have this kernel rolled out across a portion of the farm, which should mean we see the panic results in a few days time. I understand there's a risk involved in this but, its important for us to determine the cause and get a confirmed fix, as well as being able to prove that the panic fix works which will help everyone in the long run. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: debugging frequent kernel panics on 8.2-RELEASE
- Original Message - From: Andriy Gapon a...@freebsd.org Thanks to the debug that Steven provided and to the help that I received from Kostik, I think that now I understand the basic mechanics of this panic, but, unfortunately, not the details of its root cause. It seems like everything starts with some kind of a race between terminating processes in a jail and termination of the jail itself. This is where the details are very thin so far. What we see is that a process (http) is in exit(2) syscall, in exit1() function actually, and past the place where P_WEXIT flag is set and even past the place where p_limit is freed and reset to NULL. At that place the thread calls prison_proc_free(), which calls prison_deref(). Then, we see that in prison_deref() the thread gets a page fault because of what seems like a NULL pointer dereference. That's just the start of the problem and its root cause. Thats interesting, are you using http as an example or is that something thats been gleaned from the debugging of our output? I ask as there's only one process running in each of our jails and thats a single java process. Now given your description there may be something I can add that may help clarify what the cause could be. In a nutshell the jail manager we're using will attempt to resurrect the jail from a dieing state in a few specific scenarios. Here's an exmaple:- 1. jail restart requested 2. jail is stopped, so the java processes is killed off, but active tcp sessions may prevent the timely full shutdown of the jail. 3. if an existing jail is detected, i.e. a dieing jail from #2, instead of starting a new jail we attach to the old one and exec the new java process. 4. if an existing jail isnt detected, i.e. where there where not hanging tcp sessions and #2 cleanly shutdown the jail, a new jail is created, attached to and the java exec'ed. The system uses static jailid's so its possible to determine if an existing jail for this service exists or not. This prevents duplicate services as well as making services easy to identify by their jailid. So what we could be seeing is a race between the jail shutdown and the attach of the new process? Now man 2 jail seems to indicate this is a valid use case for jail_set, as it documents its support for JAIL_DYING as a valid option for flags, but I suspect its something quite out of the ordinary to actually do, which may be why this panic hasnt been seen before now. As some background the reason we use static jailid's is to ensure only one instance of the jailed service is running, and the reason we re-attach to the dieing jail is so that jails can be restarted in a timely manor. Without using the re-attach we would need to wait of all tcp sessions which have been aborted to timeout. So, of course, Steven is interested in finding and fixing the root cause. I hope we will get to that with some help from the prison guards :-) Does the above potentially explain how we're getting to the situation which generates the panic? If so we can certainly look at using alternatives to the current design to workaround this issue. Flagging the jail as permanent and using manual process management and additional external locking to prevent duplicates, is what instantly springs to mind. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: cam / ata timeout limited to 2147 due to overflow bug?
Fri, Aug 05, 2011 at 12:02:19AM +0100, Steven Hartland wrote: So I suspect that this is what's happening resulting in an extremely small timeout instead of a large one. Now I know that passed in value to the timeout is seconds * 1000 so we should be seeing 2148000 for ccb-ccb_h.timeout now multiply that by 1000 (hz) and your over the int wrap point 2147483647. So instead of the wrap point being 2147483 seconds (24 days), I suspect because of the way this is structured its actually 2147 seconds (26mins). If this is the case the fix is likely to be something like:- callout_reset(slot-timeout, (int)(ccb-ccb_h.timeout * (hz / 2000)), It will give you 0 timeout for all values of hz that are lower than 2000: hz is int, so you'll get integer division. Since ccb_h.timeout is u_int32_t, the proper way to handle this situation would be {{{ (u_int64_t)ccb-ccb_h.timeout * (u_int32_t)hz)/2000 }}} as long as the value of hz won't be greater than 2^32. Ahh of course, was late ;-) Can you try the patch at http://codelabs.ru/fbsd/patches/ahci/AHCI-properly-convert-CAM-timeout-to-ticks.diff What I don't understand is why the /2000 It gives (timeout_in_ticks)/2. The code in ahci_timeout does the following: {{{ /* Check if slot was not being executed last time we checked. */ if (slot-state AHCI_SLOT_EXECUTING) { snip.. So, my theory is that the first half of the timeout time is devoted to the transition from AHCI_SLOT_RUNNING - AHCI_SLOT_EXECUTING and the second one is the transition from AHCI_SLOT_RUNNING - TIMEOUT to give the whole process the duration of a full timeout. However, judging by the code, if the slot won't start executing at the first invocation of ahci_timeout that was spawned by the callout armed in ahci_execute_transaction, we can have timeouts more than for the specified amount of time. And if the slot will never start its execution, the callout will spin forever, unless I am missing something important here. May be Alexander can shed some light into this? Interesting thanks for the explaination. I've tried the patch and it a few cut and paste errors, which I've fixed, and confirmed it works as expected, so thanks for that :) There's also a load more drivers with the same issue so I've gone through and fixed all the occurances I can find. Here's the updated patch:- http://blog.multiplay.co.uk/dropzone/freebsd/ccb_timeout.patch Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
cam / ata timeout limited to 2147 due to overflow bug?
I'm working on adding security methods to camcontrol and have come up against a strange issue. It seems that the timeout value for cam, at least on ata (ahci), is limited to less than 2148 seconds. This can be seen by running:- camcontrol identify ada0 -t 2148 -v (pass0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (pass0:ahcich0:0:0:0): CAM status: Command timeout Also seen in /var/log/messages at this time is:- Aug 4 23:29:51 cfdev kernel: ahcich0: Timeout on slot 24 Aug 4 23:29:51 cfdev kernel: ahcich0: is cs 0100 ss rs 0100 tfd d0 serr Dropping the timeout down to 2147 and the command runs fine. I've done some digging and it seems like this is implemented via:- sys/dev/ahci/ahci.c ahci_execute_transaction(struct ahci_slot *slot) { ... /* Start command execution timeout */ callout_reset(slot-timeout, (int)ccb-ccb_h.timeout * hz / 2000, (timeout_t*)ahci_timeout, slot); Now its documented that:- Non-positive values of ticks are silently converted to the value 1 So I suspect that this is what's happening resulting in an extremely small timeout instead of a large one. Now I know that passed in value to the timeout is seconds * 1000 so we should be seeing 2148000 for ccb-ccb_h.timeout now multiply that by 1000 (hz) and your over the int wrap point 2147483647. So instead of the wrap point being 2147483 seconds (24 days), I suspect because of the way this is structured its actually 2147 seconds (26mins). If this is the case the fix is likely to be something like:- callout_reset(slot-timeout, (int)(ccb-ccb_h.timeout * (hz / 2000)), Does this sound reasonable? What I don't understand is why the /2000? For reference the reason for wanting a large timeout is that a secure erase of large media could take many hours so I'm using the erase time reported by the drive for this, in my case here is 400 minutes. Currently this instantly fails with a Command timeout which is clearly not right. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: IPv4 socket bind using IPv6 socket on openjdk6 breaks udp send
- Original Message - From: Matthias Andree matthias.and...@gmx.de No I'm not its replying to the sender. Yes you are, check your trace: The sendto address and port are the same as the bound address. This is a bug in truss it seems, Bjoern said he's gonna have a look at it. Doesn't matter what you bind to it always reports this value in the truss output. In the java code we have:- socket.send( new DatagramPacket( data, data.length, src.getSocketAddress() ) ); Where src is the src packet. This works fine on IPv4 only machines and when the jdk is told to use only IPv4 stack. So its not a problem with the java code itself but could well be an issue with the What data type is it? All mute as the Bjoern found and fixed the issue, it was a bug in the kernel fixed by:- http://svnweb.freebsd.org/base?view=revisionrevision=220463 Oops sorry cut and paste error (wrong line) heres the correct line. root java 894 20 udp4 85.236.109.212:25675 *:* While a datagram socket, it does not match the socket()/bind() above. An INET6-domain datagram socket would be listed as udp6 here. Are you sure you're tracing the right VM and are looking at the right thread? Again, truss isn't showing the correct results, confused me too ;-) Possibly another bug in sockstat / netstat as well, when the socket is a IPv6 socket bound to an IPv4 address maybe it should indicate this e.g. udp6-4 instead of either udp4 or udp6; alternatively maybe udp6 + a IPv4 address is enough to indicate this, but that could cause confusion. Thanks for looking at this, appreciated :) Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: IPv4 socket bind using IPv6 socket on openjdk6 breaks udp send
- Original Message - From: Matthias Andree matthias.and...@gmx.de I'm adding back in -java as based on you comments it may well be something in the jdk passing invalid values down to the kernel syscall. The socket bind works fine and the packets sent to the server arrive and are processed by the app but when it tries to reply using send the result is:- java.io.IOException: Invalid argument at java.net.PlainDatagramSocketImpl.send(Native Method) at java.net.DatagramSocket.send(DatagramSocket.java:629) using truss we see the following:- socket(PF_INET6,SOCK_DGRAM,0)= 20 (0x14) setsockopt(0x14,0x29,0x1b,0x7edf0318,0x4,0x0) = 0 (0x0) setsockopt(0x14,0x,0x20,0x7edf031c,0x4,0x0) = 0 (0x0) bind(20,{ AF_INET6 [3800::10:0:0:0]:20736 },28) = 0 (0x0) .. sendto(20,\M^?\M^?\M^?\M^?I\aMultiplay :: ...,82,0x0,{ AF_INET6 [3800::10:0:0:0]:20736 },0x1c) ERR#22 'Invalid argument' You're trying to send to your own address, but you're likely not using the loopback interface for that. Is that permitted by your firewall configuration and routing? No I'm not its replying to the sender. In the java code we have:- socket.send( new DatagramPacket( data, data.length, src.getSocketAddress() ) ); Where src is the src packet. This works fine on IPv4 only machines and when the jdk is told to use only IPv4 stack. So its not a problem with the java code itself but could well be an issue with the sockstat shows it binding correctly root java 894 21 tcp4 85.236.109.212:25675 *:* This is unrelated, as it has fd #21 not #20 as in the socket/bind/sendto calls. You've quoted the wrong line from sockstat output. Oops sorry cut and paste error (wrong line) heres the correct line. root java 894 20 udp4 85.236.109.212:25675 *:* 21 is the tcp port created in the same manor (ipv6 socket) which works fine. Note: net.inet6.ip6.v6only was set to the default 1 but changing it to 0 has no effect on the issue. You aren't using IPv4 mapped addresses, and you haven't stated whether you're using wildcard listeners. Only in that case would it matter. I'm not, its a bound port, as shown above now I have the correct line ;-) inet6(4) reads: IPV6CTL_V6ONLY(ip6.v6only) Boolean: enable/disable the prohib- ited use of IPv4 mapped address on AF_INET6 sock- ets. Defaults to on. The jvm automatically sets this on all sockets for compatibility for this exact reason. I'm not rulling out an issue with the IPv6 - v4 routing in the kernel though. Are you sure that's what you seeing? It's not a match for what you give above, but anyways it's an implementation artifact because the tcp code for v4 and v6 used to be shared and the udp code separate. Thats not how the jdk works, its ment to be 100% transparent but isn't. See the following for some interesting details:- http://diario.beerensalat.info/2008/10/12/java_and_ipv6_on_bsd.html It is best to set up one IPv4 and one IPv6 listening socket. I don't believe there is any way to do this in java it either uses the IPv4 stack only or the IPv6 stack only hence relies on the kernel routing IPv4 packets through the IPv6 stack. Thats the reason the jdk explicitly enables this for all the ports it creates, which was added as a back port of the jdk7 fixes which can be seen here:- http://www.freebsd.org/cgi/cvsweb.cgi/ports/java/openjdk6/files/patch-set Check the URL above, perhaps that helps your understanding a bit. I presume 3800::10:0:0:0 is your server? Not that I'm aware of, here's the output from ifconfig if anyone can tell me different, as I'm new to IPv6 and don't follow how its mapped yet. ifconfig igb0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500 options=1bbRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4 ether 00:25:90:2c:3c:b0 inet 85.236.109.212 netmask 0xff00 broadcast 85.236.109.255 inet6 fe80::225:90ff:fe2c:3cb0%igb0 prefixlen 64 scopeid 0x1 inet6 2001:4db0:20:2::1337 prefixlen 64 nd6 options=3PERFORMNUD,ACCEPT_RTADV media: Ethernet autoselect (1000baseSX full-duplex) status: active igb1: flags=8802BROADCAST,SIMPLEX,MULTICAST metric 0 mtu 1500 options=1bbRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4 ether 00:25:90:2c:3c:b1 media: Ethernet autoselect (1000baseSX full-duplex) status: active lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384 options=3RXCSUM,TXCSUM inet 127.0.0.1 netmask 0xff00 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 nd6 options=3PERFORMNUD,ACCEPT_RTADV This is currently running on 8.2-RELEASE with openjdk6-b22_5 Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the
IPv4 socket bind using IPv6 socket on openjdk6 breaks udp send
We're trying to get our machines IPv6 enabled but in doing so this seems to break java apps using openjdk6 for UDP sends. The server seems quite happy to send and receive TCP packets on IPv6 socket that are bound to IPv4 addresses, but the same is not true for UDP. The socket bind works fine and the packets sent to the server arrive and are processed by the app but when it tries to reply using send the result is:- java.io.IOException: Invalid argument at java.net.PlainDatagramSocketImpl.send(Native Method) at java.net.DatagramSocket.send(DatagramSocket.java:629) using truss we see the following:- socket(PF_INET6,SOCK_DGRAM,0)= 20 (0x14) setsockopt(0x14,0x29,0x1b,0x7edf0318,0x4,0x0) = 0 (0x0) setsockopt(0x14,0x,0x20,0x7edf031c,0x4,0x0) = 0 (0x0) bind(20,{ AF_INET6 [3800::10:0:0:0]:20736 },28) = 0 (0x0) .. recvfrom(20,0x7eaeb580,1460,0x0,0x7eaed580,0x7eaed5ac) ERR#60 'Operation timed out' .. sendto(20,\M^?\M^?\M^?\M^?I\aMultiplay :: ...,82,0x0,{ AF_INET6 [3800::10:0:0:0]:20736 },0x1c) ERR#22 'Invalid argument' sockstat shows it binding correctly root java 894 21 tcp4 85.236.109.212:25675 *:* The following PR seems relevant but also seems to indicate it was fixed back in 2006 http://www.freebsd.org/cgi/query-pr.cgi?pr=92620 Setting -Djava.net.preferIPv4Stack=true does workaround the issue but when we come to support IPv6 as well as IPv4 this won't work. Note: net.inet6.ip6.v6only was set to the default 1 but changing it to 0 has no effect on the issue. An ideas why tcp in this setup works fine for udp fails only on send? Not sure which list is best for this so sorry about the cross posting. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: quotas an essential feature? (was: svn commit: r218953 - stable/8/usr.sbin/sysinstall)
While I can understand some may want its not something we use on any of our machines, and I suspect that's the case for many others. Given adding it means the kernel will be doing extra work and hence a drop in performance for a feature most will never use, I'm guessing here, I would say just leave it out of generic unless there is a real pressing requirement for it? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: machdep.hlt_cpus not safe with ULE?
- Original Message - From: Garrett Cooper gcoo...@freebsd.org As a followup to this and based on discussions with other folks, the fact that it's using hlt to halt CPUs without rescheduling tasks / masking interrupts, etc is not good. So none of the *hlt* sysctls are really doing the right thing on x86. Time to disable them until they are fixed properly then I would suggest? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
machdep.hlt_cpus not safe with ULE?
I'm trying to debug a possibly failing CPU, so I thought it would be easy just disable the cores using machdep.hlt_cpus and see if we see the panic's we've been seeing. The problem is it seems ULE doesnt properly support machdep.hlt_cpus and still schedules processes onto the halted cpus which obviously causes problems. Can anyone confirm this behaviour? Should machdep.hlt_cpus and I assume the logical counterpart never be used with ULE? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: machdep.hlt_cpus not safe with ULE?
- Original Message - From: Gary Jennejohn gljennj...@googlemail.com Looking at the kernel source it appears that only sched_4bsd.c makes use of hlt_cpus_mask. Given ULE is default do these need to be either removed totally or at least conditionally based on the scheduler choice as currently they are quite dangerous to a systems health? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: machdep.hlt_cpus not safe with ULE?
For reference I've found that an alternative is to set the following in loader.conf:- hint.lapic.2.disabled=1 hint.lapic.3.disabled=1 2 and 3 here are the apic numbers displayed by dmesg on boot for the cpu's Obviously this requires a reboot so no perfect for all uses but it does work for what we're testing. This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: nginx + passenger = segv in _rtld_error on restart on FreeBSD 8.0?
As you may have guessed by my last reply no we didn't we had to revert to apache + passenger, but seems you've found a fix anyway. Out of interest what lead you to the close race condition PR as a potential fix? - Original Message - From: Matt Reimer mattjrei...@gmail.com Steve, Did you figure this out? We're seeing something very similar with nginx + passenger + FreeBSD 8.0. This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: nginx + passenger = segv in _rtld_error on restart onFreeBSD8.0?
Thanks Matt most appreciated! - Original Message - From: Matt Reimer mattjrei...@gmail.com Steve, The patch for PR 144061 works for us. http://lists.freebsd.org/pipermail/freebsd-hackers/2010-February/030741.html http://www.freebsd.org/cgi/query-pr.cgi?pr=144061 This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
To sendmail or to postfix that is the question?
Ok so I'm looking to replace our current windows mail server using mdaemon with a FreeBSD solution, having looked around there seems to be differing opinions of which is the best option to go with between sendmail and postfix. The problem with looking for info on this is that a lot of the high listed articles in Google etc are from way back when, 2000 - 2007 during which time quite a bit can change. So what are peoples current day experience with the two options? A few key question come to mind:- 1. Has sendmail's config moved away from the black art it once was? 2. Is postfix that much easier? 3. What would people use for: 3.1. POP / IMAP support? 3.2. Web Mail? 3.2. AV / Spam filtering? Any advice, opinions on a full mail solution on FreeBSD would be appreciated. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: To sendmail or to postfix that is the question?
- Original Message - From: Bill Moran wmo...@collaborativefusion.com I recommend squirrelmail. Unless you require some obscene feature-rich craziness (in that case, have fun installing IMP). I've used it with Postfix/Dovecot for a few years now with no trouble. Why is this discussion on hack...@? Couldn't see a really appropriate list tbh, as pointed out though perhaps isp@ would have been better. Thanks for all the feedback and ideas so far :) Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: nginx hanging with state zoneli
- Original Message - From: Alexander Nesterov nester@gmail.com On 08.01.2010 20:21, Steven Hartland wrote: [..] 12582/218/12800/12800 4k (page size) jumbo clusters in use (current/cache/total/max) Try to increase jumbo clusters (sysctl kern.ipc.nmbjumbop) Thanks for the suggestion Alex I've doubled this now and will keep an eye. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
nginx hanging with state zoneli
The first time in a while be we just had nginx get stuck in the zoneli state: 51630 www 1 -160 16616K 9396K zoneli 2 24:33 0.00% nginx 51627 www 1 -160 16616K 9376K zoneli 1 24:22 0.00% nginx 51629 www 1 -160 16616K 9372K zoneli 3 24:09 0.00% nginx 51628 www 1 -160 16616K 9104K zoneli 0 24:05 0.00% nginx Usually I would suspect an mbufs issue but that doesnt seem to have caused the problem this time, going by the info below, anyone ideas? Running: 7.0-RELEASE-p11 AMD64 == netstat -m == 14829/11796/26625 mbufs in use (current/cache/total) 2240/1024/3264/262144 mbuf clusters in use (current/cache/total/max) 1449/727 mbuf+clusters out of packet secondary zone in use (current/cache) 12582/218/12800/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 58515K/5869K/64384K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 3214 requests for I/O initiated by sendfile 0 calls to protocol drain routines == /etc/sysctl.con == # Large amounts of files in directories fix vfs.ufs.dirhash_maxmem=8379692 # Network performance tunning net.inet.tcp.inflight.enable=0 net.inet.tcp.sendspace=65536 kern.ipc.maxsockbuf=16777216 net.inet.tcp.sendbuf_max=16777216 net.inet.tcp.recvbuf_max=16777216 # Prevent nginx issues:- # Increase backlog max kern.ipc.somaxconn=1024 # Prevent issue with pmap vm.pmap.shpgperproc=500 # high server numbers tuning see: # http://rerepi.wordpress.com/2008/04/19/tuning-freebsd-sysoev-rit/ kern.ipc.nmbclusters=262144 kern.ipc.maxsockets=204800 net.inet.tcp.msl=2 == /boot/loader.conf == accf_http_load=YES net.inet.tcp.syncache.hashsize=1024 net.inet.tcp.syncache.bucketlimit=100 net.inet.tcp.tcbhashsize=4096 vm.kmem_size=768M vm.kmem_size_max=768M This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Passenger hangs on live and SEGV on tests possible threading /kernel bug?
- Original Message - From: John Baldwin I've uploaded a two more traces for the oxt test failure / segv. http://code.google.com/p/phusion-passenger/issues/detail?id=441#c1 From looking at the test case it testing the capture of failures and its ability to create a stack trace output so that may give others some indication where the issue may be? I will look to do the same on for the hang issue but that's on a live site so will need to schedule some downtime before I can get those rebuilt and then wait for it to hang again, which could be quite some time :( Hmmm, the only seg fault I see is happening down inside libgcc in the stack unwinding code and that is 3rd party code from gcc. Thanks for looking John, so you believe this may be an issue with the gcc code? What would be the next step on this, raise it on a gcc mail list or something? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Passenger hangs on live and SEGV on tests possible threading / kernel bug?
We're having an issue with Passenger on FreeBSD where it will hang and stop processing any more requests the details are attach to the following bug report: http://code.google.com/p/phusion-passenger/issues/detail?id=318#c14 In addition the test suite crashes in what seems to be a very basic test, which I'm at a loss with. http://code.google.com/p/phusion-passenger/issues/detail?id=441 I'm thinking this may be a bugs in the FreeBSD either kernel or thread library as the crashes don't make any sense from the application side. Any advise on debugging or feedback on the stack traces would be much appreciated. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Passenger hangs on live and SEGV on tests possible threading / kernel bug?
- Original Message - From: John Baldwin j...@freebsd.org For the hang it seems you have a thread waiting in a blocking read(), a thread waiting in a blocking accept(), and lots of threads creating condition variables. However, the pthread_cond_init() in libpthread (libthr on FreeBSD) doesn't call pthread_cleanup_push(), so your stack trace doesn't make sense to me. However, that may be gdb getting confused. The pthread_cleanup_push() frame may be cond_init(). However, it doesn't call umtx_op() (the _thr_umutex_init() call it makes just initializes the structure, it doesn't make a _umtx_op() system call). You might try posting on threads@ to try to get more info on this, but your pthread_cond_init() stack traces don't really make sense. Can you rebuild libc and libthr with debug symbols? For example: # cd /usr/src/lib/libc # make clean # make DEBUG_FLAGS=-g # make DEBUG_FLAGS=-g install However, if you are hanging in read(), that usually means you have a socket that just doesn't have data. That might be an application bug of some sort. The segv trace doesn't include the first part of GDB messages which show which thread actually had a seg fault. It looks like it was the thread that was throwing an exception. However, nanosleep() doesn't throw exceptions, so that stack trace doesn't really make sense either. Perhaps that stack is hosed by the exception handling code? I've uploaded a two more traces for the oxt test failure / segv. http://code.google.com/p/phusion-passenger/issues/detail?id=441#c1 From looking at the test case it testing the capture of failures and its ability to create a stack trace output so that may give others some indication where the issue may be? I will look to do the same on for the hang issue but that's on a live site so will need to schedule some downtime before I can get those rebuilt and then wait for it to hang again, which could be quite some time :( Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: nginx + passenger = segv in _rtld_error on restart on FreeBSD8.0?
- Original Message - From: Kostik Belousov kostik...@gmail.com To: Steven Hartland kill...@multiplay.co.uk Cc: freebsd-hackers@freebsd.org; freebsd-sta...@freebsd.org Sent: Wednesday, December 09, 2009 10:21 AM Subject: Re: nginx + passenger = segv in _rtld_error on restart on FreeBSD8.0? This is the trace once world had been recompiled with:- CFLAGS=-pipe WITH_CTF=1 DEBUG_FLAGS=-g #0 0x000800c95eec in thr_kill () at thr_kill.S:3 #1 0x000800b22e9e in _thr_send_sig (thread=0x800f06600, sig=6) at /usr/src/lib/libthr/thread/thr_kern.c:92 #2 0x000800b1f878 in _raise (sig=6) at /usr/src/lib/libthr/thread/thr_sig.c:187 #3 0x000800d74003 in abort () at /usr/src/lib/libc/stdlib/abort.c:65 #4 0x0043b8a7 in Client::threadMain (this=0x800f9cf40) at ext/nginx/HelperServer.cpp:516 #5 0x00411302 in boost::_mfi::mf0void, Client::operator() (this=0x7fa45ea8, p=0x800f9cf40) at mem_fn_template.hpp:49 #6 0x00411651 in boost::_bi::list1boost::_bi::valueClient* ::operator()boost::_mfi::mf0void, Client, boost::_bi::list0 (this=0x7fa45eb8, f...@0x7fa45ea8, a...@0x7fa45d7f) at bind.hpp:232 #7 0x00411696 in boost::_bi::bind_tvoid, boost::_mfi::mf0void, Client, boost::_bi::list1boost::_bi::valueClient* ::operator() (this=0x7fa45ea8) at bind_template.hpp:20 #8 0x004116bd in boost::detail::function::void_function_obj_invoker0boost::_bi::bind_tvoid, boost::_mfi::mf0void, Client, boost::_bi::list1boost::_bi::valueClient* , void::invoke ( function_obj_p...@0x7fa45ea8) at function_template.hpp:158 #9 0x0042e73a in boost::function0void, std::allocatorvoid ::operator() (this=0x7fa45ea0) at function_template.hpp:825 #10 0x00435760 in oxt::thread::thread_main (fu...@0x7fa45ea0, da...@0x7fa45e90) at thread.hpp:107 #11 0x0041310e in boost::_bi::list2boost::_bi::valueboost::functionvoid ()(), std::allocatorvoid , boost::_bi::valueboost::shared_ptroxt::thread::thread_data ::operator()void (*)(boost::functionvoid ()(), std::allocatorvoid , boost::shared_ptroxt::thread::thread_data), boost::_bi::list0 (this=0x800f3ee80, f...@0x800f3ee78, a...@0x7fa45f0f) at bind.hpp:289 #12 0x00413196 in boost::_bi::bind_tvoid, void (*)(boost::functionvoid ()(), std::allocatorvoid , boost::shared_ptroxt::thread::thread_data), boost::_bi::list2boost::_bi::valueboost::functionvoid ()(), std::allocatorvoid , boost::_bi::valueboost::shared_ptroxt::thread::thread_data ::operator() (this=0x800f3ee78) at bind_template.hpp:20 #13 0x004131b9 in boost::thread::thread_databoost::_bi::bind_tvoid, void (*)(boost::functionvoid ()(), std::allocatorvoid , boost::shared_ptroxt::thread::thread_data), boost::_bi::list2boost::_bi::valueboost::functionvoid ()(), std::allocatorvoid , boost::_bi::valueboost::shared_ptroxt::thread::thread_data::run (this=0x800f3ee00) at thread.hpp:130 #14 0x00443259 in thread_proxy (param=0x800f3ee00) at ext/boost/src/pthread/thread.cpp:127 #15 0x000800b1badd in thread_start (curthread=0x800f06600) at /usr/src/lib/libthr/thread/thr_create.c:288 #16 0x in ?? () Cannot access memory at address 0x7fa46000 Current language: auto; currently asm It seems that in the passenger client threads it calls closeStream which errors when the socket close errors with ENOTCONN virtual void closeStream() { TRACE_POINT(); if (fd != -1) { int ret = syscalls::close(fd); fd = -1; if (ret == -1) { if (errno == EIO) { throw SystemException(A write operation on the session stream failed, errno); } else { throw SystemException(Cannot close the session stream, errno); } } } } This causes it to call abort on the the thread which then crashes the app with the above stack trace, which seems really weird. Anyone got any ideas? Regards steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
nginx + passenger = segv in _rtld_error on restart on FreeBSD 8.0?
I'm currently testing nginx + passenger on FreeBSD 8.0 and I'm seeing a strange segv which seems to indicate a core library error in _rtld_error. Could this be the case or is the stack just badly corrupted? (gdb) bt #0 0x0008005577dc in _rtld_error () from /libexec/ld-elf.so.1 #1 0x000800557c3f in _rtld_error () from /libexec/ld-elf.so.1 #2 0x000800557d5e in _rtld_error () from /libexec/ld-elf.so.1 #3 0x00080055851b in dladdr () from /libexec/ld-elf.so.1 #4 0x0008005585f3 in dladdr () from /libexec/ld-elf.so.1 #5 0x00080055576d in ?? () from /libexec/ld-elf.so.1 #6 0x0001 in ?? () #7 0x004117f8 in boost::detail::sp_counted_impl_pPassenger::Application::StandardSession::dispose (this=0x800768980) at sp_counted_impl.hpp:78 Previous frame inner to this frame (corrupt stack?) Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Dell M600 Blade: 6.4 works, 7.x and 8.x fail to boot
Yes I can confirm that. Regards Steve - Original Message - From: Thierry Herbelot thierry.herbe...@free.fr from the two URLs you provide, it seems that i386 is working (for all FreeBSD versions) and only amd64 is failing : can you confirm this ? This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
How to prevent 64bit library paths being searched for 32bit binaries on amd64?
I've just been trying to get a 32bit binary to work on amd64 7.0-RELEASE, the binary had some qt4 dependencies so I set about copying these into /usr/local/lib32 but it seems that for some reason /usr/local/lib is searched first which obviously causes problems when here is a 64bit library of the same name. /etc/rc.d/ldconfig start shows:- ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat/pkg /usr/local/lib/mysql /usr/local/lib/pth 32-bit compatibility ldconfig path: /usr/lib32 /usr/local/lib32 /usr/local/lib32/mysql /usr/local/lib32/qt4 LD_LIBARY_PATH isn't set So the question is why is /usr/local/lib searched for a 32bit binary? Is it possible the binary itself is setting it and if so how to I override it? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: FreeBSD 7.0-RELEASE amd64 on Dell M600 Blade
Did anyone ever get anywhere with this, just retried with 8.0-BETA2 and still the same hangs as depicted in the attached screen shots. Sorry got no serial console :( Would like to get this working if possible so anything I can try to gain some more info on this issue? Regards Steve - Original Message - From: Adam Jacob Muller freebsd-hack...@adam.gs Anyone ever get this to work? Perhaps this was fixed in a newer FreeBSD? Have some M600 that i'd like to get FreeBSD running on :) hint.apic.0.disabled seemed to change things a bit, it would reach This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk.___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
i386_set_ldt on FreeBSD 7.x amd64
I'm trying to convert some audio streams and the only app which seems to be capable of this mplayer with the w32codecs Unfortunately the machine is 7.1 amd64. I've compiled the binaries up on an old i386 box but when running said bins on the amd64 box it errors with: Opening audio decoder: [dmo] Win32/DMO decoders install_fs: Invalid argument Couldn't install fs segment, expect segfault Looking at the code the call to i386_set_ldt is failing is there any know workaround? Looking on Google there is a thread: i386_set_ldt and wine on AMD64 which mentions some patches by kib but no mention if this ever worked. http://docs.freebsd.org/cgi/getmsg.cgi?fetch=233316+0+archive/2008/freebsd-amd64/20081214.freebsd-amd64 The latest version I can find is: http://people.freebsd.org/~kib/misc/amd64_ldt-pre.4.patch Anyone info appreciated. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: i386_set_ldt on FreeBSD 7.x amd64
- Original Message - From: Kostik Belousov kostik...@gmail.com http://docs.freebsd.org/cgi/getmsg.cgi?fetch=233316+0+archive/2008/freebsd-amd64/20081214.freebsd-amd64 The latest version I can find is: http://people.freebsd.org/~kib/misc/amd64_ldt-pre.4.patch Anyone info appreciated. The patches were committed to HEAD. mplayer/win32 codec's were tested, it was one of the goal of the patch to have these codec's working on amd64. No MFC to 7.x is planned. Thanks for the confirmation Kostik. Do you believe there is any reason to stop that patch working on 7.X if manually applied? Is the patch above the final version that went into HEAD? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: i386_set_ldt on FreeBSD 7.x amd64
- Original Message - From: Kostik Belousov kostik...@gmail.com Thanks for the confirmation Kostik. Do you believe there is any reason to stop that patch working on 7.X if manually applied? Is the patch above the final version that went into HEAD? No and no, but I think that back-porting is not that trivial. Upgrade is probably easier then backport. When you say HEAD would that require an upgrade to 7-STABLE or 8-CURRENT? I never did get what the official meaning of HEAD is, as it seems to be used by different people to mean both of the above. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Looking for someone to commit hptmv driver fixes
Hi guys I'm looking for someone to commit the hptmv v1.16 driver to the main source? Currently this driver in FreeBSD core is at 1.12 which it totally unusable under 7.+, panics on install, as well as being totally unstable on Seagate drives on previous versions. I can provide a full patch which is essentially v1.16 from highpoint + one minor fix for multi card support. If anyone who can do this would let me know, that would be most appreciated as getting a machine working with this card is currently a major PITA involving building a custom install CD which is a very lengthy process :( Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: How to increase the max pty's on Freebsd 7.0?
0n Wed, Apr 01, 2009 at 10:53:06PM +0200, Ed Schouten wrote: You can increase the maximum amount of PTYs by editing a lot of source files on your system. There is some good news: in -CURRENT we switched to Unix98-style PTYs (/dev/pts/%u). Right now the maximum amount of PTYs is limited to 1000 (0 to 999). Thanks for the confirmation I've managed to patch our local source tree to increase this to 1024. Seems the last patch to raise to 512 has one bug so fixed that while I was there. If anyone wants the patch set shout. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: How to increase the max pty's on Freebsd 7.0?
Yep that's what I came up with after looking though the code thanks for the link though, always good to get confirmation that I didn't miss something. Regards Steve - Original Message - From: John Baldwin j...@freebsd.org On Wednesday 01 April 2009 1:55:12 pm Steven Hartland wrote: How can I increase the maximum number or ptys available on FreeBSD 7.0? It seems that currently the machine is maxing out at 512 but there is still loads of capacity left on the machine. Ideally would like to double at least the number of ttys available, any help would be most appreciated. http://people.freebsd.org/~jhb/patches/pty_1152.patch This might require 7.1 instead of 7.0 as I simplified the pty allocation code in = 7.x so that it was easier to add new ones. This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
How to increase the max pty's on Freebsd 7.0?
How can I increase the maximum number or ptys available on FreeBSD 7.0? It seems that currently the machine is maxing out at 512 but there is still loads of capacity left on the machine. Ideally would like to double at least the number of ttys available, any help would be most appreciated. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: How to increase the max pty's on Freebsd 7.0?
I knew his sounded very familiar, seems I asked the same question back in 2007 for 5.4 just it was capped at 256: http://lists.freebsd.org/mailman/htdig/freebsd-hackers/2007-October/021852.html I'm sure I also remember saying this limit had been removed in 7.x but looking in the source for pqrsPQRS it seems there are several places where this still hard coded:- contrib/telnet/telnetd/sys_term.c: for (cp = pqrsPQRS; *cp; cp++) { lib/libc/stdlib/grantpt.c:#define PTY_DEV1pqrsPQRSlmnoLMNO lib/libutil/pty.c: for (cp1 = pqrsPQRSlmnoLMNO; *cp1; cp1++) { sys/kern/tty_pty.c:static const char names[] = pqrsPQRSlmnoLMNOtuvwTUVWxyzXYZ; sys/kern/tty_pty.c: * pts == /dev/tty[pqrsPQRSlmnoLMNO][0123456789abcdefghijklmnopqrstuv] sys/kern/tty_pty.c: * ptc == /dev/pty[pqrsPQRSlmnoLMNO][0123456789abcdefghijklmnopqrstuv] usr.sbin/ac/ac.c: strchr(pqrsPQRS, usr.ut_line[3]) != 0 || So the questions are:- 1. Is the fix to this still to just to update these settings? 2. Is there any significance to the upper case lower case thing? 3. Are there any other restrictions on the letters that can be used? - Original Message - From: Steven Hartland How can I increase the maximum number or ptys available on FreeBSD 7.0? It seems that currently the machine is maxing out at 512 but there is still loads of capacity left on the machine. Ideally would like to double at least the number of ttys available, any help would be most appreciated. This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
7.0 unusual performance issue - vmdaemon hang?
Just had one of hour webservers flag as down here and on investigation the machine seems to be struggling due to a hung vmdaemon process. top is reporting vmdaemon as using a constant 55.57% CPU yet CPU time is not increasing:- last pid: 36492; load averages: 0.04, 0.05, .11 up 89+19:53:21 14:36:08 223 processes: 9 running, 201 sleeping, 13 waiting CPU states: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 644M Active, 2780M Inact, 480M Wired, 249M Cache, 214M Buf, 3759M Free Swap: 4096M Total, 537M Used, 3559M Free, 13% Inuse PID USERNAME THR PRI NICE SIZERES STATE C TIME WCPU COMMAND 11 root 1 171 ki31 0K16K CPU7 7 2116.4 100.00% idle: cpu7 12 root 1 171 ki31 0K16K CPU6 6 2059.5 100.00% idle: cpu6 13 root 1 171 ki31 0K16K CPU5 5 2029.3 100.00% idle: cpu5 14 root 1 171 ki31 0K16K CPU4 4 1977.8 100.00% idle: cpu4 15 root 1 171 ki31 0K16K CPU3 3 1912.0 100.00% idle: cpu3 16 root 1 171 ki31 0K16K CPU2 2 1835.2 100.00% idle: cpu2 17 root 1 171 ki31 0K16K CPU1 1 1763.1 100.00% idle: cpu1 18 root 1 171 ki31 0K16K RUN0 1727.6 100.00% idle: cpu0 37 root 1 20- 0K16K psleep 5 0:56 55.57% vmdaemon 60198 www1 4098M 13516K sbwait 2 35:21 1.46% httpd 60264 www1 40 133M 9248K sbwait 0 21:21 0.39% httpd 30 root 1 -68- 0K16K - 7 18.3H 0.00% em1 taskq 29 root 1 -68- 0K16K - 6 330:21 0.00% em0 taskq 41 root 1 20- 0K16K syncer 1 212:42 0.00% syncer 21 root 1 -44- 0K16K WAIT 0 201:02 0.00% swi1: net 19 root 1 -32- 0K16K WAIT 0 120:15 0.00% swi4: clock 22 root 1 44- 0K16K - 5 73:00 0.00% yarrow I've tried to ktrace the process and it produced nothing, also tried gdb and it failed to attach. Is there anything else I can try before we reboot the machine to help determine what the problem is? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: unionfs kernel panic on 7.1-PRERELEASE
Did you have any ideas where the root of this problem may lie Kostik? - Original Message - From: Steven Hartland [EMAIL PROTECTED] Yes every time, I've got a half life 2 dedicated install mounted under unionfs:- mount -t unionfs -o noatime -o below /usr/local/games/hl2ds /usr/local/games/servers/1 As soon as I start the server from under servers/1 the machine panics I'm thinking its a combination of the Linux ABI and unionfs interaction which is causing the issue. Regards Steve - Original Message - From: Kostik Belousov [EMAIL PROTECTED] To: Steven Hartland [EMAIL PROTECTED] Cc: freebsd-hackers@freebsd.org Sent: Tuesday, December 02, 2008 8:39 PM Subject: Re: unionfs kernel panic on 7.1-PRERELEASE Is it reproducible ? The start of *fdp structure looks very suspicious, fd_ofiles = 0x140, fd_ofileflags = 0x154, fd_cdir = 0x168, fd_rdir = 0x17c, fd_jdir = 0x18c The values are consequently increasing by 0x14, except fd_jdir, and pointer values are wrong for kernel. This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
unionfs kernel panic on 7.1-PRERELEASE
Not sure where to go with this one any help appreciated:- FreeBSD dedicated11.multiplay.co.uk 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #4: Tue Dec 2 16:53:30 UTC 2008 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/MULTIPLAY i386 kgdb kernel /var/crash/vmcore.1 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as i386-marcel-freebsd... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x150 fault code = supervisor read, page not present instruction pointer = 0x20:0xc0624115 stack pointer = 0x28:0xe62c3b80 frame pointer = 0x28:0xe62c3ba8 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags= interrupt enabled, resume, IOPL = 0 current process = 763 (srcds_i686) trap number = 12 panic: page fault cpuid = 0 Uptime: 2m5s Physical memory: 1007 MB Dumping 53 MB: 38 22 6 warning: kld_current_sos: Can't read filename: Input/output error Reading symbols from /boot/kernel/acpi.ko...Reading symbols from /boot/kernel/acpi.ko.symbols...done. done. Loaded symbols for /boot/kernel/acpi.ko Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from /boot/kernel/linprocfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/linprocfs.ko Reading symbols from /boot/kernel/linux.ko...Reading symbols from /boot/kernel/linux.ko.symbols...done. done. Loaded symbols for /boot/kernel/linux.ko Reading symbols from /boot/kernel/unionfs.ko...Reading symbols from /boot/kernel/unionfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/unionfs.ko #0 doadump () at pcpu.h:196 196 pcpu.h: No such file or directory. in pcpu.h (kgdb) list *0xc0624115 0xc0624115 is in getvnode (/usr/src/sys/kern/vfs_syscalls.c:3969). 3964fp = NULL; 3965if (fdp == NULL) 3966error = EBADF; 3967else { 3968FILEDESC_SLOCK(fdp); 3969if ((u_int)fd = fdp-fd_nfiles || 3970(fp = fdp-fd_ofiles[fd]) == NULL) 3971error = EBADF; 3972else if (fp-f_vnode == NULL) { 3973fp = NULL; (kgdb) bt #0 doadump () at pcpu.h:196 #1 0xc05a0937 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #2 0xc05a0c09 in panic (fmt=Variable fmt is not available. ) at /usr/src/sys/kern/kern_shutdown.c:574 #3 0xc072eb8c in trap_fatal (frame=0xe62c3b40, eva=336) at /usr/src/sys/i386/i386/trap.c:939 #4 0xc072ee10 in trap_pfault (frame=0xe62c3b40, usermode=0, eva=336) at /usr/src/sys/i386/i386/trap.c:852 #5 0xc072f7cc in trap (frame=0xe62c3b40) at /usr/src/sys/i386/i386/trap.c:530 #6 0xc071563b in calltrap () at /usr/src/sys/i386/i386/exception.s:159 #7 0xc0624115 in getvnode (fdp=0xc40b4d00, fd=4, fpp=0xe62c3c70) at /usr/src/sys/kern/vfs_syscalls.c:3969 #8 0xc3e2a13d in getdents_common (td=0xc408f460, args=0xe62c3cfc, is64bit=0) at /usr/src/sys/modules/linux/../../compat/linux/linux_file.c:446 #9 0xc072f165 in syscall (frame=0xe62c3d38) at /usr/src/sys/i386/i386/trap.c:1090 #10 0xc07156a0 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:255 #11 0x0033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) frame 7 #7 0xc0624115 in getvnode (fdp=0xc40b4d00, fd=4, fpp=0xe62c3c70) at /usr/src/sys/kern/vfs_syscalls.c:3969 3969if ((u_int)fd = fdp-fd_nfiles || (kgdb) print *fdp $1 = {fd_ofiles = 0x140, fd_ofileflags = 0x154 Address 0x154 out of bounds, fd_cdir = 0x168, fd_rdir = 0x17c, fd_jdir = 0x18c, fd_nfiles = 512, fd_map = 0xc3bed560, fd_lastfile = 4, fd_freefile = 5, fd_cmask = 18, fd_refcnt = 1, fd_holdcnt = 1, fd_sx = {lock_object = {lo_name = 0xc076e1c2 filedesc structure, lo_type = 0xc076e1c2 filedesc structure, lo_flags = 37421056, lo_witness_data = {lod_list = {stqe_next = 0x0}, lod_witness = 0x0}}, sx_lock = 17, sx_recurse = 0}, fd_kqlist = {slh_first = 0x0}, fd_holdleaderscount = 0, fd_holdleaderswakeup = 0} (kgdb) print fd $2 = 4 (kgdb) print fdp-fd_ofiles $3 = (struct file **) 0x140 (kgdb) print fdp-fd_ofiles[fd] Cannot access memory at address 0x150 (kgdb) print fdp-fd_ofiles[0] Cannot access memory at address 0x140 (kgdb) print *fdp-fd_ofiles Cannot access memory at address 0x140 0xc3e2a13d is in getdents_common (/usr/src/sys/modules/linux/../../compat/linux/linux_file.c:446). 441 nbytes = sizeof(linux_dirent); 442 justone = 1; 443 } else 444 justone = 0; 445 446
Re: unionfs kernel panic on 7.1-PRERELEASE
Yes every time, I've got a half life 2 dedicated install mounted under unionfs:- mount -t unionfs -o noatime -o below /usr/local/games/hl2ds /usr/local/games/servers/1 As soon as I start the server from under servers/1 the machine panics I'm thinking its a combination of the Linux ABI and unionfs interaction which is causing the issue. Regards Steve - Original Message - From: Kostik Belousov [EMAIL PROTECTED] To: Steven Hartland [EMAIL PROTECTED] Cc: freebsd-hackers@freebsd.org Sent: Tuesday, December 02, 2008 8:39 PM Subject: Re: unionfs kernel panic on 7.1-PRERELEASE Is it reproducible ? The start of *fdp structure looks very suspicious, fd_ofiles = 0x140, fd_ofileflags = 0x154, fd_cdir = 0x168, fd_rdir = 0x17c, fd_jdir = 0x18c The values are consequently increasing by 0x14, except fd_jdir, and pointer values are wrong for kernel. This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD 7.0-RELEASE amd64 on Dell M600 Blade
Thanks Rudi, would really like to get is sorted as they would make ideal app servers. - Original Message - From: Rudi Kramer - MWEB [EMAIL PROTECTED] Hi Steven, We recently purchased a few M600's but haven't got around to loading FBSD on them, we should start installing next week and I will let you know if we run in to any problems. This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
FreeBSD 7.0-RELEASE amd64 on Dell M600 Blade
I've been trying to install FreeBSD 7.0-RELEASE amd64 on a Dell M600 Blade but it hangs just after initialising the isa bus. I've tried a number of things and the only thing that I can get to work is the i386 build which boots into the installer without issue. Has anyone had any experience with the Dell M600 blade on amd64 or had the amd64 build hang at this point before. I don't have access to the machines to try new things with ATM as we needed them in production so was forced to install ubuntu to get then live but I should get them back for more testing next week some time so wanted to see if anyone had any experience with this or a similar issue? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
lighttpd failing to accept new connections ( connection reset )
We're using lighttpd here for a new project and we're having issues where by it simply stops processing after a 1-2 days. Having looked at it in some detail this morning it seems that the kernel is resetting the connection without notifying the lighttpd process there is a new connection attempt. I assume that the listen queue is full but why kevent is not notifying lighttpd that there are outstanding events is beyond me. The following is a truss of the process which is currently in this state:- kevent(6,0x0,0,{},11096,{1.0}) = 0 (0x0) gettimeofday({1219920575.149428},0x0)= 0 (0x0) kevent(6,0x0,0,{},11096,{1.0}) = 0 (0x0) gettimeofday({1219920576.150443},0x0)= 0 (0x0) ktrace of the operation as well:- 28363 lighttpd RET kevent 0 28363 lighttpd CALL gettimeofday(0x7fffeb20,0) 28363 lighttpd RET gettimeofday 0 28363 lighttpd CALL kevent(0x6,0,0,0x800e66000,0x2b58,0x7fffeb20) 28363 lighttpd GIO fd 6 wrote 0 bytes 28363 lighttpd GIO fd 6 read 0 bytes 28363 lighttpd RET kevent 0 28363 lighttpd CALL gettimeofday(0x7fffeb20,0) 28363 lighttpd RET gettimeofday 0 28363 lighttpd CALL kevent(0x6,0,0,0x800e66000,0x2b58,0x7fffeb20) 28363 lighttpd GIO fd 6 wrote 0 bytes 28363 lighttpd GIO fd 6 read 0 bytes 28363 lighttpd RET kevent 0 28363 lighttpd CALL gettimeofday(0x7fffeb20,0) 28363 lighttpd RET gettimeofday 0 28363 lighttpd CALL kevent(0x6,0,0,0x800e66000,0x2b58,0x7fffeb20) 28363 lighttpd GIO fd 6 wrote 0 bytes 28363 lighttpd GIO fd 6 read 0 bytes 28363 lighttpd RET kevent 0 28363 lighttpd CALL gettimeofday(0x7fffeb20,0) 28363 lighttpd RET gettimeofday 0 28363 lighttpd CALL kevent(0x6,0,0,0x800e66000,0x2b58,0x7fffeb20) 28363 lighttpd GIO fd 6 wrote 0 bytes 28363 lighttpd GIO fd 6 read 0 bytes 28363 lighttpd RET kevent 0 28363 lighttpd CALL gettimeofday(0x7fffeb20,0) 28363 lighttpd RET gettimeofday 0 28363 lighttpd CALL kevent(0x6,0,0,0x800e66000,0x2b58,0x7fffeb20) 28363 lighttpd GIO fd 6 wrote 0 bytes 28363 lighttpd GIO fd 6 read 0 bytes 28363 lighttpd RET kevent 0 28363 lighttpd CALL gettimeofday(0x7fffeb20,0) 28363 lighttpd RET gettimeofday 0 28363 lighttpd CALL kevent(0x6,0,0,0x800e66000,0x2b58,0x7fffeb20) tcpdump shows:- 12:10:29.475255 IP (tos 0x10, ttl 64, id 9536, offset 0, flags [DF], proto: TCP (6), length: 64) client.61224 server.80: S, cksum 0x6d22 (incorrect (- 0xedfa), 291994449:291994449(0) win 65535 mss 1460,nop,wscale 1,nop,nop,timestamp 3661727139 0,sackOK,eol 12:10:29.481396 IP (tos 0x0, ttl 61, id 25503, offset 0, flags [DF], proto: TCP (6), length: 60) server.80 client.61224: S, cksum 0xbf22 (correct), 3444532576:3444532576(0) ack 291994450 win 65535 mss 1460,nop,wscale 9,sackOK,timestamp 3136311843 3661727139 12:10:29.481419 IP (tos 0x10, ttl 64, id 9538, offset 0, flags [DF], proto: TCP (6), length: 52) client.61224 server.80: ., cksum 0x6d16 (incorrect (- 0x6bd2), 1:1(0) ack 1 win 33304 nop,nop,timestamp 3661727145 3136311843 12:10:29.487519 IP (tos 0x10, ttl 61, id 25504, offset 0, flags [DF], proto: TCP (6), length: 40) server.80 client.61224: R, cksum 0x20c7 (correct), 3444532577:3444532577(0) win 0 This may have been raised before back 2003 as bug kern/57380 but it was closed after no response from the reporter. Another possible issues related to this is:- http://trac.lighttpd.net/trac/ticket/1734 I've currently got one of the production machines offline with this error ( hence the important flag ) in the hope that someone can suggest a test which will shed more light on the issue before I restart it. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: lighttpd failing to accept new connections ( connection reset ) / possible kqueue bug
- Original Message - From: Jeremy Chadwick [EMAIL PROTECTED] Can you change the polling method in lighttpd to use poll or select instead of kqueue? This would help in determining if the problem is with the daemon itself or the kevent system. Yep already scheduled that change for our London node tomorrow morning. ATM we are seeing this issue every 1 - 2 days so it may take a little while before I can answer that question. I've had a look through the source and I can't see any reason why kevent would suddenly stop notifying the app that new connections are present. Event registration appears to only be done once on app startup and similarly unregisters are only done on shutdown, so my current thinking is there may be an problem with kqueue itself. I don't suppose your aware of any way to query the status of this in the kernel or app given I have a node in this hung state? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: lighttpd failing to accept new connections ( connection reset )
- Original Message - From: Robert Watson [EMAIL PROTECTED] The connections getting reset without application notification is a classic symptom of a full listen queue. A couple of questions: Yep thats what I thought. (1) What FreeBSD version? 7.0-RELEASE-p2 (amd64) (2) Are you using accept filters? The modules there but not loaded, so no. (3) If possibly, are you able to instrument lighthttpd so that you can trigger it to query SO_LISTENQLIMIT, SO_LISTENQLEN, and SO_LISTENINCQLEN on the listen socket once things have gone wrong? The respectively (and perhaps obviously) querye the current administrative limit on queue depth, the number queue depth on completed connections, and the current queue depth on incomplete connections. The last of these will only be used with accept filters on recent FreeBSD network stacks (since the syncache was added). Hopefully doing (3) will allow us to try to determine whether it's indeed the case that somehow the listen queue or event handling has gotten wedged in some way. This should be possible, I'll have a look, assuming the kgdb stuff doesn't turn up the required results. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ntpd jail problem
I assume as it would effect the entire machine and hence should be run on the base machine instead, not the jail? - Original Message - From: [EMAIL PROTECTED] Anybody know why ntpd might not work in a jail? This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Using sendmsg for SCM_CREDS results in EINVAL on PF_INET socket
sendmsg is not documented as ever returning EINVAL but yet when using the following code to send credentials to a remote host results in EINVAL from sendmsg. I suspect that SCM_CREDS is only valid for PF_LOCAL / PF_UNIX sockets and not PF_INET sockets and hence the code in dbus is actually invalid. Can anyone confirm this is the case or not? [code from dbus-sysdeps-unix.c] write_credentials_byte (int server_fd, DBusError *error) { int bytes_written; char buf[1] = { '\0' }; #if defined(HAVE_CMSGCRED) union { struct cmsghdr hdr; char cred[CMSG_SPACE (sizeof (struct cmsgcred))]; } cmsg; struct iovec iov; struct msghdr msg; iov.iov_base = buf; iov.iov_len = 1; memset (msg, 0, sizeof (msg)); msg.msg_iov = iov; msg.msg_iovlen = 1; msg.msg_control = (caddr_t) cmsg; msg.msg_controllen = CMSG_SPACE (sizeof (struct cmsgcred)); memset (cmsg, 0, sizeof (cmsg)); cmsg.hdr.cmsg_len = CMSG_LEN (sizeof (struct cmsgcred)); cmsg.hdr.cmsg_level = SOL_SOCKET; cmsg.hdr.cmsg_type = SCM_CREDS; #endif _DBUS_ASSERT_ERROR_IS_CLEAR (error); again: #if defined(HAVE_CMSGCRED) bytes_written = sendmsg (server_fd, msg, 0); #else bytes_written = write (server_fd, buf, 1); #endif if (bytes_written 0 errno == EINTR) goto again; if (bytes_written 0) { dbus_set_error (error, _dbus_error_from_errno (errno), Failed to write credentials byte: %s, _dbus_strerror (errno)); return FALSE; } else if (bytes_written == 0) { dbus_set_error (error, DBUS_ERROR_IO_ERROR, wrote zero bytes writing credentials byte); return FALSE; } else { _dbus_assert (bytes_written == 1); _dbus_verbose (wrote credentials byte\n); return TRUE; } } [/code from dbus-sysdeps-unix.c] This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: MODULES_OVERRIDE magic needed
The following works fine here note its MODULE'S' not MODULE makeoptions MODULES_OVERRIDE=linux linprocfs acpi nfsclient nfsserver nullfs accf_http Regards Steve - Original Message - From: Danny Braniss [EMAIL PROTECTED] To: freebsd-hackers@freebsd.org Sent: Monday, March 31, 2008 3:11 PM Subject: MODULES_OVERRIDE magic needed hi, I'm trying to compile only a few kernel modules, but using MODULES_OVERRIDE I can only select one module. MODULE_OVERRIDE=unionfs or MODULE_OVERRIDE=if_sis is ok, but MODULE_OVERRIDE=if_sis unionfs failes with either if_sis: not found if it's defined in src.conf, or with make: don't know how to make if_sis from the command line. danny ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
pkg_add -r doesn't process sub dependencies?
Seems if you have package X which depends on package Y which depends on package Z then pkg_add -r X will always fail unless you first :- 1. pkg_add -r Y 2. pkg_add -r Z The failure will be:- pkg_add: could not find package Z ! This happens even though Z exists in the remote source package directory. Surely I've missed something very basic here and pkg_add isn't that broken? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: pkg_add -r doesn't process sub dependencies?
- Original Message - From: walt [EMAIL PROTECTED] If you are using RELENG_7, pkg_add is looking in the wrong place on the remote server for packages. I just submitted a patch on the -STABLE list for this problem. I'm actually using PACKAGESITE which is meant to bypass all path calculations but it simply doesn't. I've now found PKG_ADD_BASE which seems to fix this issue but is not documented anywhere. I had to dig around in the code to find it, where its commented as:- /* Special tip that sysinstall left for us */ It seems really strange that the default behaviour of pkg_add -r can't deal with sub dependencies properly without this undocumented feature. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: gettimeofday() in hping
- Original Message - From: Ivan Voras [EMAIL PROTECTED] The other thing that bothers me is, that under freebsd is quite easy to get: [send_ip] sendto: No buffer space available It happens almost always on my laptop just few seconds after I start hping with timecounter=TSC I'm not sure, but from what I understood of Robert Watson's explanation in the big ZFS thread on -current, maybe increasing kmem_size (exactly as for ZFS...) could help you with these buffers. Is this not just running out of mbufs? netstat -m will show if it is and the fix is to just increase kern.ipc.nmbclusters. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Jail with a back door interface?
Is it possible to create a jail with more than one IP? What I'm looking to do is have a forward facing IP and an backwards facing IP in the same jail so it can talk to the internet and our back door systems. Looking around this doesn't seem possible but is it? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How to enable more than 256 pty's?
Any one got any pointers on this, the machine we running this app on is over 90% idle so I really don't want to have to install a second machine just to workaround a limit on the number of pty's, surely there's a way to increase this? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How to enable more than 256 pty's?
Thanks for the tip there but I cant find any function called pty_create_slave in the source. N.B. Machine is running 5.4 but I also looked on 6.2 which we could upgrade to but still couldn't find it, so I assume you may be talking about something that's in current which we couldn't risk on this machine. Is this something that's possible on 5.x / 6.2 or something that will need a lot of work? Regards Steve - Original Message - From: Dag-Erling Smørgrav [EMAIL PROTECTED] You need to change the way ptys are named in pty_create_slave() and pty_clone() in sys/kern/tty_pty.c. Just changing names won't help as the sequence is also hardcoded in pty_clone(). You also need to change grantpt(), openpty() and any other userland code which has hardcoded knowledge of the naming scheme: [EMAIL PROTECTED] ~% gfs pqrsPQRS src/sys/kern/tty_pty.c: static char *names = pqrsPQRS; src/sys/kern/tty_pty.c: * pts == /dev/tty[pqrsPQRS][0123456789abcdefghijklmnopqrstuv] src/sys/kern/tty_pty.c: * ptc == /dev/pty[pqrsPQRS][0123456789abcdefghijklmnopqrstuv] src/contrib/telnet/telnetd/sys_term.c: for (cp = pqrsPQRS; *cp; cp++) { src/usr.sbin/ac/ac.c: strchr(pqrsPQRS, usr.ut_line[3]) != 0 || src/lib/libutil/pty.c: for (cp1 = pqrsPQRS; *cp1; cp1++) { src/lib/libc/stdlib/grantpt.c: #define PT_DEV1 pqrsPQRS Alternatively, set kern.pts.enable to 1, and find and fix the hang-on-close bug in the pts code (if it hasn't been fixed already) DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
How to enable more than 256 pty's?
Is it really a matter of editing: /usr/src/sys/kern/tty_pty.c And changing: static char *names = pqrsPQRS; To increase the number of available pty's ( FreeBSD 5.4 ) or is there a nicer way? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: How to enable more than 256 pty's?
Well looked like that might work but clearly not, should have looked closer. Is there a way to do this? Regards Steve - Original Message - Is it really a matter of editing: /usr/src/sys/kern/tty_pty.c And changing: static char *names = pqrsPQRS; To increase the number of available pty's ( FreeBSD 5.4 ) or is there a nicer way? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
truss with -f = application hang
When trying to trace an app with truss -f to trace descendants the application will always hang. Without -f everything is fine but we dont get the decendants calls. This is when running on FreeBSD 6.2-RELEASE-p7, the application in this test is a linux app and we are running linux_base-fc-4_9. Anyone got any ideas about this? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]