9-STABLE: bce pulse(): Warning: bootcode thinks driver is absent
I just started to try and get VirtualBox up and running on this server .. same configuration as another server I already have a few of them running, but after a period of time with the install, I get the above error pop up on my remote console, and pinging to the server itself just dies … I'm up to date for src / 9-STABLE as of today … have never seen this one before, nor can I seem to find anything useful through google except for points to the code itself … When it dies, it isn't just the network that is dead, but the whole machine appears to be un-responsive … When I rebuilt the kernel, I did also rebuild VirtualBox/kmod, since I've experienced odd things with it in the past … Anyone have any ideas on what is causing this? Something with VirtualBox triggering … something? Thx ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 9-STABLE: bce pulse(): Warning: bootcode thinks driver is absent
Further to this … I just did a 'ctl-alt-del' through my remote console, which appears to have 'unstuck' whatever it was, in that I got a bunch of Login: prompts scroll up the screen (hit return a few times after it got stuck), after which it seems to do a full shutdown and reboot, as if nothing was wrong … I had a ping process running against that machine, which halted … but when I hit ctl-alt-del, a whack of responses came back with really high times: === 64 bytes from 200.46.151.146: icmp_seq=3104 ttl=64 time=215390.333 ms 64 bytes from 200.46.151.146: icmp_seq=3105 ttl=64 time=214389.342 ms 64 bytes from 200.46.151.146: icmp_seq=3286 ttl=64 time=33295.200 ms 64 bytes from 200.46.151.146: icmp_seq=3287 ttl=64 time=32294.228 ms 64 bytes from 200.46.151.146: icmp_seq=3288 ttl=64 time=31296.220 ms 64 bytes from 200.46.151.146: icmp_seq=3289 ttl=64 time=30296.262 ms whack deleted 64 bytes from 200.46.151.146: icmp_seq=3315 ttl=64 time=4299.235 ms 64 bytes from 200.46.151.146: icmp_seq=3316 ttl=64 time=3299.278 ms 64 bytes from 200.46.151.146: icmp_seq=3317 ttl=64 time=2298.324 ms 64 bytes from 200.46.151.146: icmp_seq=3318 ttl=64 time=1299.448 ms 64 bytes from 200.46.151.146: icmp_seq=3319 ttl=64 time=298.485 ms 64 bytes from 200.46.151.146: icmp_seq=3320 ttl=64 time=0.168 ms 64 bytes from 200.46.151.146: icmp_seq=3321 ttl=64 time=0.158 ms 64 bytes from 200.46.151.146: icmp_seq=3322 ttl=64 time=0.167 ms 64 bytes from 200.46.151.146: icmp_seq=3323 ttl=64 time=0.157 ms back to normal 64 bytes from 200.46.151.146: icmp_seq=3330 ttl=64 time=0.137 ms 64 bytes from 200.46.151.146: icmp_seq=3331 ttl=64 time=0.159 ms 64 bytes from 200.46.151.146: icmp_seq=3332 ttl=64 time=0.154 ms 64 bytes from 200.46.151.146: icmp_seq= ttl=64 time=0.204 ms no more ping response as server rebooting On 2013-02-24, at 12:22 AM, Marc Fournier free...@hub.org wrote: I just started to try and get VirtualBox up and running on this server .. same configuration as another server I already have a few of them running, but after a period of time with the install, I get the above error pop up on my remote console, and pinging to the server itself just dies … I'm up to date for src / 9-STABLE as of today … have never seen this one before, nor can I seem to find anything useful through google except for points to the code itself … When it dies, it isn't just the network that is dead, but the whole machine appears to be un-responsive … When I rebuilt the kernel, I did also rebuild VirtualBox/kmod, since I've experienced odd things with it in the past … Anyone have any ideas on what is causing this? Something with VirtualBox triggering … something? Thx ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Old ICH7 SATA-2 question
On 24.02.2013 05:09, Jeremy Chadwick wrote:
On Sat, Feb 23, 2013 at 04:13:07PM -0800, Jeremy Chadwick wrote:
On Sun, Feb 24, 2013 at 02:28:08AM +0400, Michael BlackHeart wrote:
2013/2/24 Jeremy Chadwick j...@koitsu.org:
{snipping irrelevant stuff and fixing formatting}
atapci1@pci0:0:31:2:class=0x01018f card=0x26011043 chip=0x27c08086
rev=0x01 hdr=0x00
vendor = 'Intel Corporation'
device = 'N10/ICH7 Family SATA IDE Controller'
{snip}
I had written up a significantly longer reply to this Email, but once I
finished and went back reviewing the information provided, my memory
recalled having this exact conversation some time ago. After some
extensive digging, I found it -- circa late 2008:
http://lists.freebsd.org/pipermail/freebsd-stable/2008-October/046306.html
The result of this conversation was that FreeBSD at the time -- this
would have been probably FreeBSD 8.0 or 8.1 -- contained a bug:
ata_chipset.c (part of the classic ata(4) driver) was misidentifying the
different revisions of ICH7 and therefore limiting controller capacities
incorrectly.
Possibly a regression has been introduced as a result of the ATA-via-CAM
migration (now the default), which uses a completely different set of
code.
CC'ing mav@ because I need some help/clarification on this one.
I received an off-list mail from another ICH7 user, particularly one who
is using an SSD. Their controller is identical (same vendor/device ID),
and all their devices also claim SATA as well as 150MBytes/sec.
However, diskinfo -t on the individuals' SSD shows quite clearly rates
of up to 200MBytes/second.
So the issue appears to be cosmetic. The question is why.
Let me clarify because some list members have already gotten confused
about some of the information output by the kernel and utilities. I'm
going to use my own system (different controller, etc.) to educate list
members. The fact I'm using AHCI has no bearing on this educational
section, let me make that clear!
* The following output during boot reflects the results of the ATA
IDENTIFY command, and indicates what the **disk itself** (or more
specifically, the controller on the disk itself) claims it is capable
of:
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: INTEL SSDSA2CW080G3 4PC10302 ATA-8 SATA 2.x device
This indicates the ada0 disk supports up to the SATA 2.x revision (i.e.
SATA300). This DOES NOT indicate what the motherboard (or HBA) SATA
controllers' PHY/port is operating at; it only indicates what the disk
supports up to.
* The subsequent output during boot reflects the actual motherboard (or
HBA) SATA controllers' PHY/port speed, including what has been
negotiated:
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
There's a 1:1 mapping between SATA revision and PHY speed, effectively,
unless otherwise throttled down or forced in some manner. I'm reminded
of the non-ATA_CAM function ata_satarev2str() where the revision map is
like this:
value 0= (default speed value is used?? unsure)
value 1= SATA150 (150MB/sec)
value 2= SATA300 (300MB/sec)
value 3= SATA600 (600MB/sec)
value 0xff = SATA(default speed value is used)
else = ??? (default speed value is used?? unsure)
Now for the rest...
I've taken a look at the code and it's very difficult for me to follow;
I'm not entirely sure, but it does look like pieces of
sys/dev/ata/chipsets are still used today. I need mav@ to verify that
fact however, because if ata_intel_probe() isn't used any more, then
that might explain what's going on here (possibly).
The code in sys/dev/ata/chipsets is still used for controllers not
supported by more advanced ahci, siis and mvs drivers. Depending on
specific chipset and BIOS revision and settings ICH7 may or may not
support AHCI mode.
The negotiated speed printing comes from sys/cam/ata/ata_xpt.c, in
ata_announce_periph(). The code logic here is simple, while the complex
bits (e.g. what sets CTS_SATA_VALID_REVISION) are elsewhere.
First, the speed:
2022 /* Report connection speed */
2023 speed = cpi.base_transfer_speed;
2024 if (cts.ccb_h.status == CAM_REQ_CMP cts.transport ==
XPORT_ATA) {
2025 struct ccb_trans_settings_pata *pata =
2026 cts.xport_specific.ata;
2027
2028 if (pata-valid CTS_ATA_VALID_MODE)
2029 speed = ata_mode2speed(pata-mode);
2030 }
2031 if (cts.ccb_h.status == CAM_REQ_CMP cts.transport ==
XPORT_SATA) {
2032 struct ccb_trans_settings_sata *sata =
2033 cts.xport_specific.sata;
2034
2035 if (sata-valid CTS_SATA_VALID_REVISION)
2036 speed = ata_revision2speed(sata-revision);
2037 }
2038 mb = speed / 1000;
2039 if (mb 0)
2040
Re: Panic at shutdown
Le mardi 12 février 2013 21:42:01 Ronald Klop a écrit : On Tue, 12 Feb 2013 19:44:49 +0100, David Demelier demelier.da...@gmail.com wrote: Le mardi 12 février 2013 10:01:10 Andriy Gapon a écrit : on 12/02/2013 09:57 David Demelier said the following: Yes I have added debugging option in my kernel. I have makeoptions DEBUG=-g in my config. Do I need more ? .symbols? I don't understand what you are saying, I have /boot/kernel/kernel.symbols. Please tell me what I'm doing wrong. I've just read and done the steps written there : http://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug- gdb.html So I've run # cd /usr/obj/usr/src/sys/Melon # kgdb kernel.debug /var/crash/vmcore.0 Why not something like kgdb /boot/kernel/kernel.symbols /var/crash/vmcore.0? That looks like what the manual page of kgdb seems to suggest. Regards, Ronald. and that's the only trace I get using bt full : 229 #define IS_BSP()(PCPU_GET(cpuid) == 0) (kgdb) bt full #0 doadump (textdump=value optimized out) at pcpu.h:229 No locals. #1 0x in ?? () No symbol table info available. I have that, in fact I was using bt full instead of short bt : #0 doadump (textdump=Variable textdump is not available. ) at pcpu.h:224 #1 0x0004 in ?? () #2 0x805146b6 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:448 #3 0x80514b79 in panic (fmt=0x1 Address 0x1 out of bounds) at /usr/src/sys/kern/kern_shutdown.c:636 #4 0x8071e919 in trap_fatal (frame=0xc, eva=Variable eva is not available. ) at /usr/src/sys/amd64/amd64/trap.c:857 #5 0x8071eca4 in trap_pfault (frame=0xff80d63db5d0, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:773 #6 0x8071f09b in trap (frame=0xff80d63db5d0) at /usr/src/sys/amd64/amd64/trap.c:456 #7 0x8070993f in calltrap () at /usr/src/sys/amd64/amd64/exception.S:228 #8 0x802ddbf5 in AcpiOsAcquireObject (Cache=0xfe00015de400) at /usr/src/sys/contrib/dev/acpica/utilities/utcache.c:316 #9 0x802e27c1 in AcpiUtAllocateObjectDescDbg (ModuleName=0x80795110 dsutils, LineNumber=703, ComponentId=Variable ComponentId is not available. ) at /usr/src/sys/contrib/dev/acpica/utilities/utobject.c:437 #10 0x802e2972 in AcpiUtCreateInternalObjectDbg (ModuleName=0x80795110 dsutils, LineNumber=703, ComponentId=64, Type=1) at /usr/src/sys/contrib/dev/acpica/utilities/utobject.c:112 #11 0x802b32ea in AcpiDsCreateOperand (WalkState=0xfe000380fc00, Arg=0xfe00017d39c0, ArgIndex=0) at /usr/src/sys/contrib/dev/acpica/dispatcher/dsutils.c:703 #12 0x802b3686 in AcpiDsCreateOperands (WalkState=0xfe000380fc00, FirstArg=0xfe00017d39c0) at /usr/src/sys/contrib/dev/acpica/dispatcher/dsutils.c:798 #13 0x802b4323 in AcpiDsExecEndOp (WalkState=0xfe000380fc00) at /usr/src/sys/contrib/dev/acpica/dispatcher/dswexec.c:449 #14 0x802d55e9 in AcpiPsParseLoop (WalkState=0xfe000380fc00) at /usr/src/sys/contrib/dev/acpica/parser/psloop.c:1249 #15 0x802d6a5b in AcpiPsParseAml (WalkState=0xfe000380fc00) at /usr/src/sys/contrib/dev/acpica/parser/psparse.c:525 #16 0x802d7ad7 in AcpiPsExecuteMethod (Info=0xfe00411b9500) at /usr/src/sys/contrib/dev/acpica/parser/psxface.c:368 #17 0x802ce3af in AcpiNsEvaluate (Info=0xfe00411b9500) at /usr/src/sys/contrib/dev/acpica/namespace/nseval.c:193 #18 0x802d3567 in AcpiEvaluateObject (Handle=0xfe00017d6d40, Pathname=0x807b2e9b _TMP, ExternalParams=0x0, ReturnBuffer=0xff80d63dba50) at /usr/src/sys/contrib/dev/acpica/namespace/nsxfeval.c:289 #19 0x8031ca84 in acpi_GetInteger (handle=0xfe00017d6d40, path=0x807b2e9b _TMP, number=0xff80d63dbab4) at /usr/src/sys/dev/acpica/acpi.c:2204 #20 0x80331756 in acpi_tz_get_temperature (sc=0xfe0001a2fa00) at /usr/src/sys/dev/acpica/acpi_thermal.c:462 #21 0x803329d6 in acpi_tz_monitor (Context=0xfe0001a2fa00) at /usr/src/sys/dev/acpica/acpi_thermal.c:497 #22 0x80332f92 in acpi_tz_thread (arg=Variable arg is not available. ) at /usr/src/sys/dev/acpica/acpi_thermal.c:864 #23 0x804e7a3d in fork_exit (callout=0x80332e50 acpi_tz_thread, arg=0x0, frame=0xff80d63dbc00) at /usr/src/sys/kern/kern_fork.c:992 #24 0x80709dfe in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:602 #25 0x in ?? () #26 0x in ?? () #27 0x0001 in ?? () #28 0x in ?? () #29 0x in ?? () #30 0x in ?? () #31 0x in ?? () #32 0x in ?? () #33 0x in ?? () #34 0x in ?? () #35 0x in ?? () #36 0x in ?? () #37 0x
Re: Old ICH7 SATA-2 question
Quoth Jeremy Chadwick j...@koitsu.org: If there are people out there using, for example, SSDs on an ICH7 in non-AHCI mode, it would be good to know and get pciconf -lvbc output (specifically the entry for their ATA/SATA controller). But as with all publicly released operating systems, most Requests For Feedback are ignored, things are then changed, and only afterwards do people crawl out of their hobbit holes and speak up. :-) Since you asked: isab0@pci0:0:31:0: class=0x060100 card=0x50011458 chip=0x27b88086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82801GB/GR (ICH7 Family) LPC Interface Bridge' class = bridge subclass = PCI-ISA cap 09[e0] = vendor (length 12) Intel cap 1 version 0 features: Quick Resume, SATA RAID-5, 6 PCI-e x1 slots, SATA RAID-0/1/10, SATA AHCI atapci0@pci0:0:31:1:class=0x01018a card=0xb0011458 chip=0x27df8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82801G (ICH7 Family) IDE Controller' class = mass storage subclass = ATA bar [20] = type I/O Port, range 32, base 0xf000, size 16, enabled atapci1@pci0:0:31:2:class=0x01018f card=0xb0021458 chip=0x27c08086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'N10/ICH7 Family SATA IDE Controller' class = mass storage subclass = ATA bar [10] = type I/O Port, range 32, base 0xe800, size 8, enabled bar [14] = type I/O Port, range 32, base 0xe900, size 4, enabled bar [18] = type I/O Port, range 32, base 0xea00, size 8, enabled bar [1c] = type I/O Port, range 32, base 0xeb00, size 4, enabled bar [20] = type I/O Port, range 32, base 0xec00, size 16, enabled cap 01[70] = powerspec 2 supports D0 D3 current D0 The motherboard manual claims it supports SATA 2 (3Gb/s). However, ada2 is an SSD, and camcontrol identify ada2 reports: pass2: KINGSTON SVP200S37A60G 502ABBF0 ATA-8 SATA 3.x device pass2: 150.000MB/s transfers (SATA, UDMA5, PIO 8192bytes) protocol ATA/ATAPI-8 SATA 3.x device model KINGSTON SVP200S37A60G [...] so FreeBSD doesn't appear to know that. Ben ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: svn - but smaller?
On Sat, 23 Feb 2013 22:31:10 -0800 Jeremy Chadwick j...@koitsu.org wrote: I downloaded it and looked at the source. svnup.c:1002: warning: zero-length printf format string svnup.c:1020: warning: zero-length printf format string svnup.c:1027: warning: zero-length printf format string svnup.c:1065: warning: zero-length printf format string 1002 sprintf(command, ); 1020 sprintf(command, ); 1027 sprintf(command, ); 1065 sprintf(command, ); I'm not sure what the intention is here. If it's to terminate the buffer, then all of these should be: command[0] = '\0'; Correct. Clang didn't complain when I went the sprintf route... :( Also, John, please consider using malloc(3) instead of heap-allocated buffers like file_buffer[6][] (196608 bytes) and command[] (32769 bytes). I'm referring to this: 47 #define COMMAND_BUFFER 32768 386 char new_path_target[BUFFER_UNIT], file_buffer[6][COMMAND_BUFFER], *path_source; 836 char *start, *value, *addr, *branch, *path_target, temp_file[BUFFER_UNIT], command[COMMAND_BUFFER + 1]; These were leftovers from a malloc debug session that I forgot to undo. Processing the ports tree needs way more that six buffers... face-palm/ I should also point out that watching this thing in top(1) shows RES/RSS increasing until the segfault (for me dies out around 4.5MBytes). I don't know if that's by design or not, but I don't see why this thing would need so much memory given what it's doing. You'd know better than I though. The code is definitely going to be memory intensive. The initial, naive implementation I wrote sent get-file and get-dir requests one at a time and the time penalty was atrocious -- it took four hours to process base/releng/8.3. To get around this, since the subversion server can handle multiple requests at a time, I have to cram as many get-file and get-dir requests together as I can to minimize the number of interactions with the server. You may want to consider running this under valgrind. It's remarkable what you can find with that during debugging. Will do. -- | Jeremy Chadwick j...@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | I've got a rough fix in place and the ports tree is chugging along now (I only tested my code against the base/releng and base/stable branches -- oops). I should be able to post revised code tonight. Thanks for taking a look at this. As a first-timer here, I definitely appreciate it! ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: svn - but smaller?
On Sat, 2013-02-23 at 22:31 -0800, Jeremy Chadwick wrote: Also, John, please consider using malloc(3) instead of heap-allocated buffers like file_buffer[6][] (196608 bytes) and command[] (32769 bytes). I'm referring to this: Why? I absolutely do not understand why people are always objecting to large stack-allocated arrays in userland code (sometimes with the definition of large being as small as 2k for some folks). -- Ian ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: svn - but smaller?
On Sun, Feb 24, 2013 at 10:19:57AM -0700, Ian Lepore wrote: On Sat, 2013-02-23 at 22:31 -0800, Jeremy Chadwick wrote: Also, John, please consider using malloc(3) instead of heap-allocated buffers like file_buffer[6][] (196608 bytes) and command[] (32769 bytes). I'm referring to this: Why? I absolutely do not understand why people are always objecting to large stack-allocated arrays in userland code (sometimes with the definition of large being as small as 2k for some folks). This is incredibly off-topic, but I'll bite. I should not have said heap-allocated, I was actually referring to statically-allocated. The issues as I see them: 1. Such buffers exist during the entire program's lifetime even if they aren't actively used/needed by the program. With malloc(3) and friends, you're allocating memory dynamically, and you can free(3) when done with it, rather than just having a gigantic portion of memory allocated sitting around potentially doing nothing. 2. If the length of the buffer exceeds the amount of stack space available at the time, the program will run but the behaviour is unknown (I know that on FreeBSD if it exceeds stacksize per resource limits, the program segfaults at runtime). With malloc and friends you can gracefully handle allocation failures. 3. Statically-allocated buffers can't grow; meaning what you've requested size-wise is all you get. Compare this to something that's dynamic -- think a linked list containing pointers to malloc'd memory, which can even be realloc(3)'d if needed. The definition of what's too large is up to the individual and the limits of the underlying application. For some people, sure, anything larger than 2048 might warrant use of malloc. I tend to use malloc for anything larger than 4096. That magic number comes from some piece of information I was told long ago about what size pages malloc internally uses, but looking at the IMPLEMENTATION NOTES section in malloc(3) it appears to be a lot more complex than that. If you want me to break down #1 for you with some real-world output and a very small C program, showing you the effects on RES/RSS and SIZE/VIRT of static vs. dynamic allocation, just ask. -- | Jeremy Chadwick j...@koitsu.org | | UNIX Systems Administratorhttp://jdc.koitsu.org/ | | Mountain View, CA, US| | Making life hard for others since 1977. PGP 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: svn - but smaller?
On 02/24/2013 03:24 PM, Jeremy Chadwick wrote: On Sun, Feb 24, 2013 at 10:19:57AM -0700, Ian Lepore wrote: On Sat, 2013-02-23 at 22:31 -0800, Jeremy Chadwick wrote: Also, John, please consider using malloc(3) instead of heap-allocated buffers like file_buffer[6][] (196608 bytes) and command[] (32769 bytes). I'm referring to this: Why? I absolutely do not understand why people are always objecting to large stack-allocated arrays in userland code (sometimes with the definition of large being as small as 2k for some folks). This is incredibly off-topic, but I'll bite. I should not have said heap-allocated, I was actually referring to statically-allocated. The issues as I see them: 2. If the length of the buffer exceeds the amount of stack space available at the time, the program will run but the behaviour is unknown (I know that on FreeBSD if it exceeds stacksize per resource limits, the program segfaults at runtime). With malloc and friends you can gracefully handle allocation failures. This actually happened to me. The program mkctm used to allocate space using alloca (which is the same as declaring it like char file_buffer[big_no] if big_no could be variable). But this is space on the stack as opposed to the heap, and if you type the command limit you will see that the stack size is typically much smaller than the heap size. And on 32 bit machines, the default value of stack size is rather small. Anyway, one day mkctm stopped working for me, and precisely for this reason. It took me a day to trace the fault, and I ended up rewriting the code to use malloc instead. (mkctm is a little known program, whose code is included in the FreeBSD base, to create updates for the CTM delivery method.) Probably on 64 bit machines it is not such a big issue. And on Linux, I think it makes no difference at all, because on Linux all data is stored on the heap. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: svn - but smaller?
On Sun, 2013-02-24 at 13:24 -0800, Jeremy Chadwick wrote: On Sun, Feb 24, 2013 at 10:19:57AM -0700, Ian Lepore wrote: On Sat, 2013-02-23 at 22:31 -0800, Jeremy Chadwick wrote: Also, John, please consider using malloc(3) instead of heap-allocated buffers like file_buffer[6][] (196608 bytes) and command[] (32769 bytes). I'm referring to this: Why? I absolutely do not understand why people are always objecting to large stack-allocated arrays in userland code (sometimes with the definition of large being as small as 2k for some folks). This is incredibly off-topic, but I'll bite. I should not have said heap-allocated, I was actually referring to statically-allocated. The issues as I see them: 1. Such buffers exist during the entire program's lifetime even if they aren't actively used/needed by the program. With malloc(3) and friends, you're allocating memory dynamically, and you can free(3) when done with it, rather than just having a gigantic portion of memory allocated sitting around potentially doing nothing. 2. If the length of the buffer exceeds the amount of stack space available at the time, the program will run but the behaviour is unknown (I know that on FreeBSD if it exceeds stacksize per resource limits, the program segfaults at runtime). With malloc and friends you can gracefully handle allocation failures. 3. Statically-allocated buffers can't grow; meaning what you've requested size-wise is all you get. Compare this to something that's dynamic -- think a linked list containing pointers to malloc'd memory, which can even be realloc(3)'d if needed. The definition of what's too large is up to the individual and the limits of the underlying application. For some people, sure, anything larger than 2048 might warrant use of malloc. I tend to use malloc for anything larger than 4096. That magic number comes from some piece of information I was told long ago about what size pages malloc internally uses, but looking at the IMPLEMENTATION NOTES section in malloc(3) it appears to be a lot more complex than that. If you want me to break down #1 for you with some real-world output and a very small C program, showing you the effects on RES/RSS and SIZE/VIRT of static vs. dynamic allocation, just ask. Actually, after seeing that the userland limit for an unpriveleged user on freebsd is a mere 64k, I'd say the only valid reason to not allocate big things on the stack is because freebsd has completely broken defaults. I see no reason why there should even be a distinction between stack size and memory use limits in general. Pages are pages, it really doesn't matter what part of your virtual address space they live in. Almost everything I've ever done with freebsd runs as root on an embedded system, so I'm not used to thinking in terms of limits at all. -- Ian [P.S. Having mail troubles today, sorry if this gets duplicated.] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: svn - but smaller?
On 02/24/2013 05:43 PM, Ian Lepore wrote: On Sun, 2013-02-24 at 13:24 -0800, Jeremy Chadwick wrote: On Sun, Feb 24, 2013 at 10:19:57AM -0700, Ian Lepore wrote: On Sat, 2013-02-23 at 22:31 -0800, Jeremy Chadwick wrote: Also, John, please consider using malloc(3) instead of heap-allocated buffers like file_buffer[6][] (196608 bytes) and command[] (32769 bytes). I'm referring to this: Why? I absolutely do not understand why people are always objecting to large stack-allocated arrays in userland code (sometimes with the definition of large being as small as 2k for some folks). This is incredibly off-topic, but I'll bite. I should not have said heap-allocated, I was actually referring to statically-allocated. The issues as I see them: 1. Such buffers exist during the entire program's lifetime even if they aren't actively used/needed by the program. With malloc(3) and friends, you're allocating memory dynamically, and you can free(3) when done with it, rather than just having a gigantic portion of memory allocated sitting around potentially doing nothing. 2. If the length of the buffer exceeds the amount of stack space available at the time, the program will run but the behaviour is unknown (I know that on FreeBSD if it exceeds stacksize per resource limits, the program segfaults at runtime). With malloc and friends you can gracefully handle allocation failures. 3. Statically-allocated buffers can't grow; meaning what you've requested size-wise is all you get. Compare this to something that's dynamic -- think a linked list containing pointers to malloc'd memory, which can even be realloc(3)'d if needed. The definition of what's too large is up to the individual and the limits of the underlying application. For some people, sure, anything larger than 2048 might warrant use of malloc. I tend to use malloc for anything larger than 4096. That magic number comes from some piece of information I was told long ago about what size pages malloc internally uses, but looking at the IMPLEMENTATION NOTES section in malloc(3) it appears to be a lot more complex than that. If you want me to break down #1 for you with some real-world output and a very small C program, showing you the effects on RES/RSS and SIZE/VIRT of static vs. dynamic allocation, just ask. Actually, after seeing that the userland limit for an unpriveleged user on freebsd is a mere 64k, I'd say the only valid reason to not allocate big things on the stack is because freebsd has completely broken defaults. I see no reason why there should even be a distinction between stack size and memory use limits in general. Pages are pages, it really doesn't matter what part of your virtual address space they live in. I think that the stacksize and datasize cannot together add up to more than 4G, or maybe it is only 2G, at least on a 32 bit machine. This is because (I think) they compete for virtual memory address space. I guess this wasn't a problem in the old days, when 4G of RAM was unthinkable. Also, as I said in another thread, I think Linux doesn't make this distinction. So at least someone else out there agrees with you. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: svn - but smaller?
On Sun, Feb 24, 2013 at 04:43:33PM -0700, Ian Lepore wrote:
On Sun, 2013-02-24 at 13:24 -0800, Jeremy Chadwick wrote:
On Sun, Feb 24, 2013 at 10:19:57AM -0700, Ian Lepore wrote:
On Sat, 2013-02-23 at 22:31 -0800, Jeremy Chadwick wrote:
Also, John, please consider using malloc(3) instead of heap-allocated
buffers like file_buffer[6][] (196608 bytes) and command[] (32769
bytes). I'm referring to this:
Why? I absolutely do not understand why people are always objecting to
large stack-allocated arrays in userland code (sometimes with the
definition of large being as small as 2k for some folks).
This is incredibly off-topic, but I'll bite.
I should not have said heap-allocated, I was actually referring to
statically-allocated.
The issues as I see them:
1. Such buffers exist during the entire program's lifetime even if they
aren't actively used/needed by the program. With malloc(3) and friends,
you're allocating memory dynamically, and you can free(3) when done with
it, rather than just having a gigantic portion of memory allocated
sitting around potentially doing nothing.
2. If the length of the buffer exceeds the amount of stack space
available at the time, the program will run but the behaviour is unknown
(I know that on FreeBSD if it exceeds stacksize per resource limits,
the program segfaults at runtime). With malloc and friends you can
gracefully handle allocation failures.
3. Statically-allocated buffers can't grow; meaning what you've
requested size-wise is all you get. Compare this to something that's
dynamic -- think a linked list containing pointers to malloc'd memory,
which can even be realloc(3)'d if needed.
The definition of what's too large is up to the individual and the
limits of the underlying application. For some people, sure, anything
larger than 2048 might warrant use of malloc. I tend to use malloc for
anything larger than 4096. That magic number comes from some piece of
information I was told long ago about what size pages malloc internally
uses, but looking at the IMPLEMENTATION NOTES section in malloc(3) it
appears to be a lot more complex than that.
If you want me to break down #1 for you with some real-world output and
a very small C program, showing you the effects on RES/RSS and SIZE/VIRT
of static vs. dynamic allocation, just ask.
Actually, after seeing that the userland limit for an unpriveleged user
on freebsd is a mere 64k, I'd say the only valid reason to not allocate
big things on the stack is because freebsd has completely broken
defaults.
The limits (i.e. what's shown via limits(1)) differs per architecture
(ex. i386 vs. amd64) and may adjust based on amount of physical memory
available (not sure on the latter part). The 64k value you're talking
about, I think, is memorylocked -- I'm referring to stacksize.
I see no reason why there should even be a distinction
between stack size and memory use limits in general. Pages are pages,
it really doesn't matter what part of your virtual address space they
live in.
You're thinking purely of SIZE/VIRT.
I guess I'd best break the C program out. Apologise in advance for the
crappy code (system(3)!), but I wanted something that made the task
easy.
$ limits -a
Resource limits (current):
cputime infinity secs
filesize infinity kB
datasize 2621440 kB
stacksize 262144 kB
coredumpsize infinity kB
memoryuseinfinity kB
memorylocked 64 kB
maxprocesses 5547
openfiles 11095
sbsize infinity bytes
vmemoryuse infinity kB
pseudo-terminals infinity
swapuse infinity kB
$ cat x.c
#include stdio.h
#include stdlib.h
#include sys/types.h
#include unistd.h
#include string.h
#define SIZE_MFATTY 512*1024*1024 /* 512MB */
#define SIZE_SFATTY 128*1024*1024 /* 128MB; must be smaller than limits
stacksize! */
int main(int argc, char *argv[]) {
char procstat[BUFSIZ];
char topgrep[BUFSIZ];
pid_t mypid;
char *mfatty;
char sfatty[SIZE_SFATTY];
sfatty[0] = '\0'; /* squelch gcc unused var warning */
mypid = getpid();
snprintf(procstat, sizeof(procstat),
procstat -v %u, mypid);
snprintf(topgrep, sizeof(topgrep),
top -b 9 | egrep '^(Mem:|[ ]+PID|[ ]*%u)', mypid);
printf(at startup\n);
printf(\n);
system(topgrep);
printf(-\n);
system(procstat);
sleep(5);
mfatty = malloc(SIZE_MFATTY);
printf(\n);
printf(after malloc mfatty\n);
printf(=\n);
system(topgrep);
printf(-\n);
system(procstat);
sleep(5);
memset(mfatty, 0x0, SIZE_MFATTY);
memset(sfatty,
Re: svn - but smaller?
Quoth Jeremy Chadwick j...@koitsu.org: On Sun, Feb 24, 2013 at 10:19:57AM -0700, Ian Lepore wrote: On Sat, 2013-02-23 at 22:31 -0800, Jeremy Chadwick wrote: Also, John, please consider using malloc(3) instead of heap-allocated buffers like file_buffer[6][] (196608 bytes) and command[] (32769 bytes). I'm referring to this: Why? I absolutely do not understand why people are always objecting to large stack-allocated arrays in userland code (sometimes with the definition of large being as small as 2k for some folks). This is incredibly off-topic, but I'll bite. I should not have said heap-allocated, I was actually referring to statically-allocated. The issues as I see them: 1. Such buffers exist during the entire program's lifetime even if they aren't actively used/needed by the program. With malloc(3) and friends, you're allocating memory dynamically, and you can free(3) when done with it, rather than just having a gigantic portion of memory allocated sitting around potentially doing nothing. If the 'allocated' memory isn't touched, it will never be paged in. Even once it is paged in, if it then goes back to being unused it can be paged out to swap (when necessary) and then stay there. (It would be nice if there were some way to tell the system 'this memory is dead, don't write it out to swap', but I don't think there is.) Of course, you may get up to two pages of an object paged in just because some other object on the same page was touched, but 'two pages' is not a lot of memory to waste (especially given that you're saving the malloc overhead). I don't know if the compiler is clever enough to page-align large static allocations; that would reduce the potential waste to one page. 2. If the length of the buffer exceeds the amount of stack space available at the time, the program will run but the behaviour is unknown (I know that on FreeBSD if it exceeds stacksize per resource limits, the program segfaults at runtime). With malloc and friends you can gracefully handle allocation failures. (SIGSEGV on stack overflow is specified by POSIX, including the fact that the signal must be fatal unless the signal handler uses an alternate stack.) Statically-allocated objects are not allocated on the stack, they are allocated in the bss or data sections of the executable. When it is possible and reasonable to do so, it's actually better to allocate all memory statically, as then once the process has started successfully you can be certain it won't fail because of a memory shortage. Large auto objects (including alloca) *are* dangerous. 3. Statically-allocated buffers can't grow; meaning what you've requested size-wise is all you get. Compare this to something that's dynamic -- think a linked list containing pointers to malloc'd memory, which can even be realloc(3)'d if needed. Under many circumstances a statically-allocated fixed-size buffer, *if properly used*, is safer than a potentially-unbounded malloced one. Of course, it's possible to put limits on the size of a malloced buffer as well, but (given that parts of the buffer you don't use won't ever be paged in) there isn't much advantage in doing so. Fixed allocations are only useful in small, single-use programs where you can accurately predict the memory requirements in advance. I wouldn't be surprised if svnsup fell into that bracket. Ben ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: RELENG_8: amdtemp module and newer CPUs not working. MFC?
On 20 Feb, Jeremy Chadwick wrote: On Wed, Feb 20, 2013 at 10:29:05PM -0800, Don Lewis wrote: On 17 Feb, Torfinn Ingolfsen wrote: Hello, I'm running FreeBSD 8.3-stable on a machine with an AMD A8-5600K cpu. tingo@kg-quiet$ uname -a FreeBSD kg-quiet.kg4.no 8.3-STABLE FreeBSD 8.3-STABLE #2: Fri Jan 4 19:18:15 CET 2013 r...@kg-quiet.kg4.no:/usr/obj/usr/src/sys/GENERIC amd64 tingo@kg-quiet$ dmesg | grep CPU | head -1 CPU: AMD A8-5600K APU with Radeon(tm) HD Graphics(3618.02-MHz K8-class CPU) Unfortunately, the amdtemp.ko module doesn't work: tingo@kg-quiet$ kldstat | grep temp 101 0x8123e000 f0f amdtemp.ko tingo@kg-quiet$ sysctl dev.amdtemp sysctl: unknown oid 'dev.amdtemp' Based on a thread[1] on the forums, amdtemp.c from -CURRENT work. But it doesn't compile under FreeBSD 8.3-stable: Updating amdtemp is on my TODO list. It has some issues even on -CURRENT. This is kind of far down my priority list because on most of my AMD machines, I can also get the temperature without amdtemp: % sysctl hw.acpi.thermal.tz0.temperature hw.acpi.thermal.tz0.temperature: 30.0C There's an implication in your statement here, so I want to clarify for readers (as the author of sysutils/bsdhwmon): acpi_thermal(4) does not necessarily tie in to an on-die DTS within the CPU. Your motherboards and CPUs (both matter! (e.g. for Intel CPUs, see PECI (not a typo)) may offer this tie-in, but such is not the case for many people. I tend to find ACPI thermal zones used in laptops and very rarely anywhere else. acpi_thermal(4) may return temperatures from zones that are mapped to readings from Super I/O chips or dedicated H/W monitoring ICs (such as ones provided by Nuvuton/Winbond, LM, ITE, ADT, etc.). It all depends on how the BIOSes ACPI tables are written/what maps to what. Such ICs DO NOT have anything to do with the on-die DTS which both amdtemp(4) and coretemp(4) use -- instead, these chips use external thermistors which may be placed anywhere on the motherboard (such as under the CPU socket, or wherever the manufacturer chooses (and more often than not, does not document)). My point: under the CPU thermistor != within the CPU DTS. They measure two different things, and are not guaranteed to be even remotely similar. I can show proof of this (a very large delta between Core i5 core DTSes and an on-board IT87xxx) if requested. You are correct. It had been several months since I looked at this and was misremembering the details. With amdtemp loaded on one of my systems where it works: hw.acpi.thermal.tz0.temperature: 34.0C dev.cpu.0.temperature: 37.2C dev.cpu.1.temperature: 42.2C dev.amdtemp.0.%desc: AMD CPU On-Die Thermal Sensors dev.amdtemp.0.%driver: amdtemp dev.amdtemp.0.%parent: hostb3 dev.amdtemp.0.sensor_offset: 0 dev.amdtemp.0.core0.sensor0: 37.5C dev.amdtemp.0.core0.sensor1: 32.7C dev.amdtemp.0.core1.sensor0: 42.2C dev.amdtemp.0.core1.sensor1: 28.2C When I looked at this previously (on another system with only one DTS), I noticed that dev.amdtemp.0.core0.sensor0 was giving the same answer as dev.cpu.0.temperature. I was unaware that amdtemp was responsible for both sysctl nodes and thought that some other kernel driver was responsible for dev.cpu.0.temperature, which is why I stopped work on my amdtemp updates. I see that the amdtemp(4) man page explains the situation. Thanks for the heads up about sysutils/bsdhwmon. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: svn - but smaller?
Ben Morrow wrote this message on Mon, Feb 25, 2013 at 00:28 +: 1. Such buffers exist during the entire program's lifetime even if they aren't actively used/needed by the program. With malloc(3) and friends, you're allocating memory dynamically, and you can free(3) when done with it, rather than just having a gigantic portion of memory allocated sitting around potentially doing nothing. If the 'allocated' memory isn't touched, it will never be paged in. Even once it is paged in, if it then goes back to being unused it can be paged out to swap (when necessary) and then stay there. (It would be nice if there were some way to tell the system 'this memory is dead, don't write it out to swap', but I don't think there is.) madvise(2) w/ MADV_FREE, though there was some discussion on -current about what these different flags will/should do not too long ago... -- John-Mark Gurney Voice: +1 415 225 5579 All that I will do, has been done, All that I have, has not. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
