ladmins that cannot type a command

2013-05-03 Thread Dieter BSD
>> good for today and future ladmins that cannot type a command.
>>
>> Any USEFUL proposals that add some real functionality?
>
> Since this will enable more people to run FreeBSD that otherwise
> wouldn't give it a second glance, I would say it is VERY useful.

Really?  How useful is FreeBSD going to be to someone who cannot type?
Who is the target here?  Idiots?  People without fingers?

I see lots of chatter about adding new stuff, much of which is ...
rather questionable.  Meanwhile many of the features FreeBSD
already has are broken.  No shortage of PRs that need fixing.
Many of these PRs have been sitting around, unloved, for *years*.
Some of them even contain patches (that *work*)!

If you're building a race car, make sure the engine runs before
spending your time polishing the wheels.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: HDD write cache

2013-02-01 Thread Dieter BSD
Wojciech writes:
> after reading quite recent topics about disabling/enabling write cache, i
> tried to test in on desktop 3.5" drive
>
> kern.cam.ada.write_cache: 1
> kern.cam.ada.read_ahead: 1
> kern.cam.ada.0.read_ahead: -1
> kern.cam.ada.0.write_cache: -1
>
> i tried writing 1 or 0 to kern.cam.ada.0.write_cache, and there were
> exactly no differences at all, and disk seems to do always write caching.
>
> Does that drive lie and ignore commands or i do it wrong?
>
> this is my disk.
>
> ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
> ada0:  ATA-8 SATA 2.x device
> ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
> ada0: Command Queueing enabled
> ada0: 476940MB (976773168 512 byte sectors: 16H 63S/T 16383C)

1) See the other followups about making sure this change actually takes
effect.

2) How, exactly, are you testing this?  With Command Queueing enabled,
(and assuming that it is working properly) you will not see any difference
in speed with a casual test.

Does ata(4) support your controller?  Last time I looked, ata did not
support NCQ. :-( So turning the write cache on/off gives a huge difference
in speed.

When ahci and siis came out, NCQ was so fast that it looked like the
write cache was on.  I had to write a special pathological test to
be able to tell the difference between having the write cache on and
using NCQ.  I needed to verify that the write cache was really off so
that my data was safe.

In anything resembling normal use, NCQ with the write cache off is
just as fast as having the write cache on.  You really get the best
of both worlds, speed and safety.  Now all we need is NCQ support
for all the controllers that have it.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: receiving aerial digital TV

2013-02-01 Thread Dieter BSD
[ This topic is better discussed in -multimedia@, suggest that followups
drop -hackers@ ]

> i am out of current knowledge about common TV for about 10 years.
>
> Currently in Poland there is aerial TV broadcasted in DVB-T standard.
> There are TVs with builtin decoder/demodulator or separate
> decoders/demodulator with HDMI output.
>
> But how about just receiving demodulated data and letting mplayer play it,
> and - most importantly - recording it directly. I know there are USB
> receivers on market.
>
> Are such devices supported under FreeBSD? if so -  what driver, what
> chipset to look when buying such a thing? any other recommendation.
>
> With analog TV broadcast there were PCI cards with brooktree chipset that
> worked with FreeBSD.

There are PCI and PCIe tuner cards, USB tuners, and Ethernet tuners.
There used to be firewire tuners but these have disappeared the last
time I looked.  Each type has advantages and disadvantages.  Cards
require expansion slots, which you may or may not have available.
Ethernet tuners allow locating the tuner near the antenna, giving
less signal loss in the coax.  And they don't need a slot, and are OS
independent.  Unfortunately, there seems to be only 1 brand of
Ethernet tuner (Silicondust HDhomerun).  The ATSC/8VSB version has
better debugging info than other tuners, but there are many reports
that other tuners give better reception. I don't know if this
applies to the DVB-T version.  Word is that some of the very small
USB tuners suffer from poor reception.  A high-gain "outdoor" antenna
will give much better reception (due to less multipath) than a low
gain antenna.  Garbage in garbage out. If you manage to get a signal
that is too strong, an attenuator is very inexpensive.

Yes, you should be able to record to a file on disk and watch it later.
Cron(8) and at(1) are useful for this. Some people like mythtv.

Many of the cards use a cx88 family chip. cx88 (in ports) supports
many of these cards.
http://corona.homeunix.net/cx88wiki

Ethernet tuners
http://www.silicondust.com/

Useful forums for tuners, antennas, how to fix reception problems, ...
http://www.avsforum.com/

Useful info about antennas and reception
http://www.hdtvprimer.com
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: IBM blade server abysmal disk write performances

2013-01-19 Thread Dieter BSD
Stefan writes:
> I seem to remember, that drives of that time required the write cache
> to be enabled to get any speed-up from tagged commands. This was no
> risk with SCSI drives, since the cache did not make the drives lye
> about command completion (i.e. the status for the write was only
> returned when the cached data had been written to disk, independently
> of the write cache enable).

Interesting.  Is there a way to tell, other than coming up with
some way to actually test it, whether a particular drive waits until
the data has been written to non-volatile memory (the platters in
conventional disks) before sending the command completion message?

I'm having thoughts of putting sensing resistors in the disk's
power cable, attaching an oscilloscope, and displaying the
timing of data on the data cable along with power usage from
seeking.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: IBM blade server abysmal disk write performances

2013-01-18 Thread Dieter BSD
Matthew writes:
> There is also no information in the original email as to which direction
> the I/O was being sent.

In one of the followups, Karim reported:
  # dd if=/dev/zero of=foo count=10 bs=1024000
  10+0 records in
  10+0 records out
  1024 bytes transferred in 19.615134 secs (522046 bytes/sec)

522 KB/s is pathetic.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: IBM blade server abysmal disk write performances

2013-01-18 Thread Dieter BSD
Scott writes:
> If I had my way, the WC would be off, everyone would be using SAS,
> and anyone who enabled SATA WC or complained about I/O slowness
> would be forced into Siberian salt mines for the remainder of their lives.

Actually, If you are running SAS, having SATA WC on or off wouldn't
matter, it would be SCSI's WC you'd care about.  :-)

>> The bigger problem is that
>> FreeBSD does not support queuing on all controllers that support it.
>> Not something that admins can fix, and inexcusable for an OS that
>> claims to care about performance.
>
> You keep saying this, but I'm unclear on what you mean.  Can you
> explain?

For most applications you need the write cache to be off.
Having the write cache off is fine as long as you have queuing.
But with the write cache off, if you don't have queuing, performance
sucks. Like getting only 6% of the performance you should be getting.
Some of the early SATA controllers didn't have NCQ.  Knowing that
queuing was very important, I made sure to choose a mainboard with
NCQ, giving up other useful features to get it.  But FreeBSD does
not support NCQ on the nforce4-ultra's SATA controllers.  Even the
sad joke of an OS Linux has had NCQ on nforce4 since Oct 2006.
But Linux is such crap it is unusable.  Linux is slowly improving,
but I don't expect to live long enough to see it become usable.
Seriously. I've tried it several times but I have completely
given up on it.  Anyway, even after all these years the supposedly
performance oriented FreeBSD still does not support NCQ on nforce4,
which isn't some obscure chip. they sold a lot them.  I've added
3 additional SATA controllers on expansion cards, and FreeBSD
supports NCQ on them, so the slow controllers limited by PCIe-x1
have much better write performance than the much faster controllers
in the chipset with all the bandwidth they need.  I can't add
more controllers, there aren't any free slots.  The nforce
will remain in service for years, aside from the monetary cost,
silicon has a huge amount of environmental cost: embedded energy,
water, pollution, etc.  And there are a lot of them.

Wojciech writes:
>> That is incorrect.  A UPS reduces the risk, but does not eliminate it.
>
> nothing eliminate all risks.

Turning the write cache off eliminates the risk of having the write cache
on.  Yes you can still lose data for other reasons.  Backups are still a
good idea.

>> But for most applications, you must have the write cache off,
>> and you need queuing (e.g. TCQ or NCQ) for performance.  If
>> you have queuing, there is no need to turn the write cache
>> on.
>
> did you tested the above claim? i have SATA drives everywhere, all in ahci
> mode, all with NCQ active.

Yes, turn the write cache off and NCQ will give you the performance.
As long as you have queuing you can have the best of both worlds.

Which is why Karim's problem is so odd.  Driver says there is queuing,
but performance (1 write per rev) looks exactly like there is no queuing.
Maybe there is something else that causes only 1 write per rev but
I don't know what that might be.

Peter writes:
> Apart from continuous whinging and whining on mailing lists, what have
> you done to add support for queuing?

Submitted PR, it was closed without being fixed.  Looked at code,
but Greek to me, even though I have successfully modified a BSD based device
driver in the past giving major performance improvement.  If I were
a C-level exec of a Fortune 500 company I'd just hire some device driver
wizard.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: IBM blade server abysmal disk write performances

2013-01-18 Thread Dieter BSD
Wojciech writes:
> If computer have UPS then write caching is fine. even if FreeBSD crash,
> disk would write data

That is incorrect.  A UPS reduces the risk, but does not eliminate it.
It is impossible to completely eliminate the risk of having the
write cache on.  If you care about your data you must turn the disk's
write cache off.

If you are using the drive in an application where the data does
not matter, or can easily be regenerated (e.g. disk duplication,
if it fails, just start over), then turning the write cache on
for that one drive can be ok. There is a patch that allows turning
the write cache on and off on a per drive basis. The patch is for
ata(4), but should be possible with other drivers.  camcontrol(8)
may work for SCSI and SAS drives. I have yet to see a USB-to-*ATA
bridge that allows turning the write cache off, so USB disks are
useless for most applications.

But for most applications, you must have the write cache off,
and you need queuing (e.g. TCQ or NCQ) for performance.  If
you have queuing, there is no need to turn the write cache
on.

It is inexcusable that FreeBSD defaults to leaving the write cache on
for SATA & PATA drives.  At least the admin can easily fix this by
adding hw.ata.wc=0 to /boot/loader.conf.  The bigger problem is that
FreeBSD does not support queuing on all controllers that support it.
Not something that admins can fix, and inexcusable for an OS that
claims to care about performance.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: IBM blade server abysmal disk write performances

2013-01-17 Thread Dieter BSD
> I am thinking that something fancy in that SAS drive is
> not being handled correctly by the FreeBSD driver.

I think so too, and I think the something fancy is "tagged command queuing".
The driver prints "da0: Command Queueing enabled" and yet your SAS drive
is only getting 1 write per rev, and queuing should get you more than that.
Your SATA drive is getting the expected performance, which means that NCQ
must be working.

> Please let me know if there is anything you would like me to run on the
> BSD 9.1 system to help diagnose this issue?

Looking at the mpt driver, a verbose boot may give more info.
Looks like you can set a "debug" device hint, but I don't
see any documentation on what to set it to.

I think it is time to ask the driver wizards why TCQ isn't working,
so I'm cc-ing the authors listed on the mpt man page.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: IBM blade server abysmal disk write performances

2013-01-15 Thread Dieter BSD
Karim writes:
> It is quite obvious that something is awfully slow on SAS drives,
> whatever it is and regardless of OS comparison. We swapped the SAS
> drives for SATA and we're seeing much higher speeds. Basically on par
> with what we were expecting (roughly 300 to 400 times faster then what
> we see with SAS...).

Major clue there!  According to wikipedia: "Most SAS drives provide
tagged command queuing, while most newer SATA drives provide native
command queuing" [1]

Note that the driver says "Command Queueing enabled" without
specifying which.  If the driver is trying to use SATA's NCQ but
the drive only speaks SCSI's TCQ, that could explain it. Or if
the TCQ isn't working for some other reason.

See if there are any error messages in dmesg or /var/log.
If not, perhaps the driver has extra debugging you could turn on.

Get TCQ working and make sure your partitions are aligned on
4 KiB boundaries (in case the drive actually has 4 KiB sectors),
and you should get the expected performance.

[1] http://en.wikipedia.org/wiki/Serial_attached_SCSI
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: IBM blade server abysmal disk write performances

2013-01-15 Thread Dieter BSD
> 25.9 MB/s

Even Linux is pretty slow.

> Transfer rates:
> outside:   102400 kbytes in   0.685483 sec = 149384 kbytes/sec
> middle:102400 kbytes in   0.747424 sec = 137004 kbytes/sec
> inside:102400 kbytes in   1.051036 sec = 97428 kbytes/sec

That's more like it.  I assume these numbers are reading.  You should get
numbers nearly this high when writing.

Can you try writing to the bare drive without a filesystem?

time dd if=/dev/da0 of=/dev/null bs=124k count=25
time (dd if=/dev/zero if=/dev/da0 bs=124k count=25; sync)

Between writing more data than the size of memory and the sync,
this should hopefully reduce any buffering effects down into the noise
and make the numbers more comparable between FreeBSD and Linux.
(and more honest) Also eliminates any effect from the filesystem, which
will be different between FreeBSD and Linux.

Writing should be almost as fast as reading.

Is the disk healthy?  Smartctl might give a clue.

If the disk is healthy and you still get numbers that indicate one
write per rev without a filesystem, then the question is why does
the driver claim queueing but not deliver it?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: IBM blade server abysmal disk write performances

2013-01-15 Thread Dieter BSD
I wrote:
> The kernel must be doing write-behind even to a raw disk, otherwise
> waiting for write(2) to return before issuing the next write would
> slow it down as Matthew suggests.

And a minute after hitting send, I remembered that FreeBSD does not
provide the traditional "raw" disk devices, e.g. /dev/rda0 with an 'r'.
(Now if I could just remember *why* it doesn't.)
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: IBM blade server abysmal disk write performances

2013-01-15 Thread Dieter BSD
Karim writes:
> dd to the
> raw drive and no compression/encryption or some other features, just a
> naive boot off a live 9.1 CD then dd (see below). The following results
> have been gathered on the FreeBSD 9.1 system:
>
> # dd if=/dev/zero of=toto count=100
> 100+0 records in
> 100+0 records out
> 51200 bytes transferred in 1.057507 secs (48416 bytes/sec)

By "raw drive" I meant something like
dd if=/dev/zero of=/dev/da0 bs=1m count=1000
"of=toto" implies that you are using a filesystem. (FFS? ZFS? other?)

Matthew writes:
>> But dmesg claims "Command Queueing enabled", so you should be
>> getting more than one op per rev, and writes should be fast.
>
> Queueing would only help if your load threw multiple ops at the drive
> before waiting for any of them to complete.  I'd expect a dd to a raw
> device to throw a single, wait for it to return complete, then throw
> the next, leading to no more than 1 op per rev.

I see a huge speedup from NCQ on both raw disks and with FFS/su.
Without NCQ I only get 6% of the expected performance, even with
a large blocksize.

The kernel must be doing write-behind even to a raw disk, otherwise
waiting for write(2) to return before issuing the next write would
slow it down as Matthew suggests.

Writing an entire 3TB disk (raw disk, no fs) gives:
21378.98 real 2.00 user   440.98 sys
or 140 MB/s (133 MiB/s) on slow controller in PCIe x1 slot.
The same test on the same make & model disk on a much faster controller
in the chipset takes over 10x as long because FreeBSD does not support
NCQ on that controller. :-(

Karim's data sure looks like 1 op per rev. Either it isn't really doing
NCQ or the filesystem is doing something to keep NCQ from being effective.
For example, mounting the fs with the sync option would probably have
that effect.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Vice versa of 'pkg_info -W'

2013-01-03 Thread Dieter BSD
Domagoj writes:
> I've attempted to grep '/usr/ports/*/*/pkg-plist' for 'bin/lynx'
> and shot myself in a foot! :P
> Even if it did returned "sane" amount of matches, speed was atrocious.

time find /usr/ports/ -name pkg-plist | xargs grep bin/lynx$
/usr/ports/finance/ledgersmb/pkg-plist:@dirrm ledger-smb/bin/lynx
/usr/ports/finance/sql-ledger/pkg-plist:@dirrm sql-ledger/bin/lynx
/usr/ports/japanese/lynx/pkg-plist:bin/lynx
/usr/ports/japanese/lynx-current/pkg-plist:bin/lynx
/usr/ports/www/lynx/pkg-plist:bin/lynx
/usr/ports/www/lynx-current/pkg-plist:bin/lynx

real0m2.065s
user0m0.379s
sys 0m1.338s

Not bad for a system that has been repeatedly insulted on this
list recently.

(I'm assuming that 6 matches is a "sane" amount.)
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: FreeBSD for serious performance?

2012-12-26 Thread Dieter BSD
> If the driver is doing something daft like DELAY(x) in a fast
> interrupt handler which would lead to that behaviour, it should be
> fixed.
>
> If it's doing a DELAY(x) in a critical section, it shuld be fixed.

They are doing *something* that completely locks out everything else.
It is always a device driver.

> Now, it's quite likely you hit some kind of ata(4) bug which kept it
> in a tight loop

Hard to imagine locking everything out for 19 minutes without being
in a loop.

> So it was likely just spun
> in some high priority loop that nothing lower-priority could really do
> anything about.

Would several different drivers have this same bug?

> The next time it happens, please break into the debugger and grab some
> debugging output. Show alllocks, ps, should be a good couple of things
> to start with.

I've only caught it hanging forever once. It only takes a few
milliseconds to cause incoming data to be lost, so I usually
don't know about it until looking at the log file later. Not
that I could jump into the debugger and gather data in a few
milliseconds even if I knew when it was happening.

BTW, how do I break into the debugger and gather data when all of
the devices are locked out, including the console?

I assume that once it recovers, there is no point in gathering data.

> Alternately - please find a currently actively maintained SATA chipset.

The ata controller is soldered to the mainboard, a gazillion pins
I'm sure, and no doubt requires very specialized equipment to replace,
and I don't know of any pin-compatable replacements. Besides the
hardware itself has never caused any problems. The problem is caused
by the software, it is the software that needs to be fixed.

Ata isn't maintained? Why the bleep not? Disk drivers are essential.

I was under the impression that siis(4) and ahci(4) were actively
maintained? I'm running four sata controllers using three different
drivers and all three drivers lock out other drivers for too long
when something unusual happens.

And other, non-disk drivers have the same problem of locking out
other drivers, even during normal operation. And this happens on
yet other drivers on other people's hardware, not just mine.

> help migrate the nvidia chipset support out of ata(4)

I've looked at several of FreeBSD's device drivers (including,
as you might expect, ata, siis, and ahci) and I can't make
heads or tails out of any of them. Back before FreeBSD existed, I
did manage to make a significant improvement to a driver in a
BSD-derived system, so I'm not a complete idiot.

Several different drivers cause the same problem. Are they all
making the same mistake? Or is there a problem in something
they all use? Whether a design problem or an implementation bug.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: FreeBSD for serious performance?

2012-12-25 Thread Dieter BSD
> Which device drivers?  We can't fix problems we don't know about.

ata(4) completely hung the system for 19 minutes (at which point
I manually intervened, see the PR), probably an infinite loop.

http://www.freebsd.org/cgi/query-pr.cgi?pr=170675

Siis(4) and ahci(4) have also caused data loss, presumably by
blocking interrupts for too long.

Improving these drivers would be wonderful. But better yet,
can we please find a way to fix the underlying problem?

When a device driver handles an interrupt, it needs to block
further interrupts while it modifies its data structures. Otherwise
another interrupt coming in might cause it to mangle the data.
Right? But! Why does it need to block interrupts for everything?
Why does a disk driver need to block interrupts from Ethernet?
Why does Ethernet need to block Firewire? Why does Firewire
need to block USB? And so on. Can't the disk driver block just
its own interrupts and leave the other devices alone?

That way, when some device driver writer puts in DELAY(TOO_LONG),
at least the other devices will still work.

Alternately, why couldn't the data structures be protected with
a mutex? Then the drivers shouldn't have to block even themselves.

Alternately, why can't drivers have a polling option?
Yes, the extra overhead of polling sucks, but losing incoming
data sucks a lot more. I am not suggesting that polling should
be the default, just an option for those who need it.

Alternately, 

Current machines can have multiple disks, multiple Ethernets,
multiple pretty-much-any-device, multiple CPUs, etc. etc.
We have SMP kernel to juggle those multiple CPUs. But we still
have this absurd bottleneck where the device drivers bring
everything to a screaching halt every time an interrupt happens.
And if the driver has a bug, or thinks there is a problem and
decides to keep DELAY()ing over and over, the entire machine
just locks up and stays locked up, often forever.

It isn't just me. I have seen quite a few threads where other
people are having the same problem.

This needs to be fixed.

(Fixing this is at *least* a Usenix paper.)
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: NetBSD's boot select MBR

2012-12-15 Thread Dieter BSD
Domagoj writes:

> MBR supports max of 4 slices/partitions.

4 primary partitions, there are the "extended/logical partitions",
which allow more.

> The '/usr/mdec/mbr_bootsel', which you've mentioned, is equivalent
> to FreeBSD's '/boot/boot0', which is tuned via 'boot0cfg' util.
> It will also show menu, to choose from, a slice to boot or another
> drive  OR => 'F6' to choose PXE boot.
> If nothing has been selected, it will simply boot an active slice.
> So I see FreeBSD's, as superior (additionally supports PXE) => 8.0 onward ...

If I understand this correctly FreeBSD's boot0 allows you to set
what to boot from the next time and go away while the system reboots.
And this method allow booting from not only any slice but any disk,
or PXE?

But, it does this by writing to the MBR, yes? Some sysadmins want
the MBR to be read-only for various reasons.

OSes other than FreeBSD aren't going to have boot0cfg. If you're
running NetBSD and want to boot Plan 9, do you have to reboot to
FreeBSD, run boot0cfg, then reboot to Plan 9?

As far as I know, NetBSD's MBR works with any combination of OSes.
If the next OS to be booted is on the 1st disk and you don't mind
writing to the MBR, you can do the set-and-walk-away thing by
setting the active partition. Or you can use something like grub
as the next stage and edit grub's default using any text editor,
including sed in a script. If you want to boot a slice that is not
on the first disk you have to either sit there and type at the menu,
or use something like grub. As far as I know, NetBSD's MBR has no
support for PXE, but then PXE is intended for diskless machines.

> I thought NetBSD's had a MBR code, which could be told via a tool,
> from fully booted and running FreeBSD:
> "Active slice is 2, but please, boot slice 4 at a next boot".
> After that, I would '# reboot' and without any menu, slice 4
> would be booted, for that one time, instaed of an active slice 2.

I don't know of a way to do that with just NetBSD's MBR. You could
either change the active flag which would remain that way until you
changed it again, or sit there and watch for the menu. Given the
limited space in the MBR it would be difficult or impossible to
add that without giving up something. That feature could be
added to a later stage bootstrap.

Next time I build a boot disk I'm going to take another shot
at multibooting with GPT. I really want to get away from the MBR
kludgefest.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

NetBSD's boot select MBR

2012-12-13 Thread Dieter BSD
[ from the FreeBSD for serious performance? thread ]

> > So I use NetBSD's MBR for disks I want
> > to boot from.
>
> Can I have a CMD sequence?
>
> First would be ...
> # fetch ...

Read NetBSD's fdisk(8) and mbr(8).

The MBR is only 512 bytes, and must contain the code and data.
This is very limiting, so there are multiple versions depending
on which features you need. IIRC, /usr/mdec/mbr_bootsel is the one I use.
You can specify a label (up to 7 characters) for each bootable partition
("slice" in FreeBSD-speak). When the system executes the MBR code,
it presents a menu with these labels. You can type '1' for the first
partition/slice, '2' for the second, and so on. You can also type a
function key to run the MBR of a different disk. The disk numbering
can be rather insane, so it must be using the bios disk numbering. If
you don't type anything it times out and boots the active partition/slice.

Sadly, the FreeBSD bootstrap insists on booting the partition/slice
marked active in the MBR. (The MBR code should look at the active
bit, the later bootstraps should not.) So if you want to have
multiple FreeBSD versions, it works best to put each on its own disk.

The last time I used fdisk, large disks triggered a bug where some
variable overflowed, giving negative numbers. Hopefully this is fixed
by now. FreeBSD's fdisk also had some bug, so I had great fun
handcrafting a MBR.

It's been a long time since I installed NetBSD from scratch, (I usually
just unpack the tar files, run installboot, and edit some config files.)
I assume you can just boot a CD and get a shell, and experiment with
fdisk on a scratch disk without actually installing NetBSD.
To avoid a disaster, do something like
dd if=$DISK of=/boot/MBR_backup bs=512 count=1 ; sync
beforehand, in case you accidently mess up your boot drive.

man pages:
http://netbsd.gw.com/cgi-bin/man-cgi?++NetBSD-current

isos:
http://www.netbsd.org/mirrors/
for example:
ftp://iso.netbsd.org/pub/NetBSD/iso/5.2/amd64cd-5.2.iso
or
ftp://iso.netbsd.org/pub/NetBSD/iso/5.2/i386cd-5.2.iso
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: FreeBSD for serious performance?

2012-12-11 Thread Dieter BSD
>>Ronald writes:
>>> the last Alpha to be produced was shipped way back in 2004... eight years
>>> ago... with a top speed of 1.3 GHz I now have a cheap little media player
>>> thingy sitting on my desk, and _each_ of its two cores runs faster than tha\
t.
>>> In short, Alphas hardly constitute high-end hardware in this day and age.
>>
>> So clock rate is the only thing that matters in your world?
>
> Yea, pretty much.
>
> As regards to reliability, except for the occasional low-level quirk (which
> is usually taken care of for me by the kernel guys) I've never had a processo\
r
> fail on me.  Once, about five or six years ago I accidentally burnt up an
> Athlon XP (by not having the heatsink properly seated) but that was entirely
> my fault.

I care about data integrity, so things like ECC are on my must-have list.
I suspect that your "cheap little media player thingy" doesn't have ECC.
If you don't care about getting the correct answer you can have infinite
speed.

A high clock rate doesn't help when some device driver does

block_all_interrupts();
while(1)
 DELAY(MIGHT_AS_WELL_BE_FOREVER);

At least four device drivers have caused me to lose data this way.
Not what I call high performance.

>> In my world, high-end means high quality. It doesn't necessarily
>> mean fast, or recent.

Data integrity, and yes, reliability, that sort of thing.

Performance-wise, in most cases I don't expect to get 99.%
of the theoretical best case, I'm usually happy with 90-95%.
But without NCQ I'm only getting ~6% of what I should be getting.
Pretty pathetic for an OS that claims to be all about performance.
All the more so when the crappy OSes do support NCQ on that chip.
It's not some rare, obscure chip. Lots of boxes have it.

>> I never found a way to boot from different partitions, much less
>> different disks with GPT.
>
> Having just been recently convinced to switch over to GPT (from MBR) I do
> most sincerly hope that you are either joking or mistaken about this.

I am neither joking nor mistaken. I looked but could not find a way.
I am not claiming that a way does not exist, merely that I couldn't
find it. Perhaps there wasn't a way when I looked (it was awhile ago)
but does exist now? I have never been a fan of MBRs, they are for
pee-cees with the expected ugly kludges and limitations, real machines
don't use them. GPT isn't perfect, but it seems much nicer than MBRs.

Warren writes:
> Grub (or grub2) can do it.

Back when I was triple-booting FreeBSD, NetBSD and Linux I used
grub (rev number forgotten). It was supposed to be able to boot
BSD from a partition but I never got that to work. I had to have it
boot the MBR of a different disk which then booted BSD. I wrote
3 little shell scripts that edited grub's menu to change the default.
So I could be running FreeBSD and type "boot_netbsd", and go have lunch
while it rebooted. Other than not booting BSD from a partition it
wasn't that bad for something that smells of penguin.

> If you're booting multiple versions of FreeBSD, see gpart(8)
> for some partition attributes that may help.

You mean the bootme bootonce stuff? That looks promising, assuming
I can manage to decode the man page, and figure out what it actually
does. Mostly multiple versions of FreeBSD. (for example 7.0 had a couple
of bugs preventing it from booting, so having 6 still available was
essential.) I no long need Linux (YEA!!!), and after a certain fubar
incident have declared a Linux Free Zone. It would be nice if I could
also boot Net/Open/Dragonfly. I don't see a way to boot multiple disks,
but GPT allows enough partitions that I probably won't care.

Not sure if the bootme bootonce stuff wasn't there yet when I looked,
or if I just missed it. Thanks for the pointer.

> Or even consider ZFS boot environments.

I plan to stick with FFS w/softdeps.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: FreeBSD for serious performance?

2012-12-09 Thread Dieter BSD
[ lack of SATA NCQ support for nforce4-ultra ]

Adrian writes:
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commitdiff;h=e2e031eb09760c36099ac127eeb175e06d257aef

which is:

The mcp61 has bug with ncq.
- { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA), SWNCQ },
- { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA2), SWNCQ },
- { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA3), SWNCQ },
+ { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA), GENERIC },
+ { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA2), GENERIC },
+ { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA3), GENERIC },

http://en.wikipedia.org/wiki/GeForce_6_Series
says: "MCP61 introduced a bug in the SATA NCQ implementation.
As a result, Nvidia employees have contributed code to disable NCQ operations 
under Linux."
I have not found a description of the bug.

But FreeBSD says I have:

 port 
0x9f0-0x9f7,0xbf0-0xbf3,0x970-0x977,0xb70-0xb73,0xcc00-0xcc0f
 mem 0xfebfb000-0xfebfbfff irq 21 at device 7.0 on pci0
 port 
0x9e0-0x9e7,0xbe0-0xbe3,0x960-0x967,0xb60-0xb63,0xb800-0xb80f
 mem 0xfebfa000-0xfebfafff irq 22 at device 8.0 on pci0

The CK804 (a few lines higher in the source file) did not get changed to 
GENERIC.
As far as I can figure out, the bug (whatever it is) is limited to the MCP61.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: FreeBSD for serious performance?

2012-12-09 Thread Dieter BSD
Ronald writes:
> the last Alpha to be produced was shipped way back in 2004... eight years
> ago... with a top speed of 1.3 GHz.  I now have a cheap little media player
> thingy sitting on my desk, and _each_ of its two cores runs faster than that.
> In short, Alphas hardly constitute high-end hardware in this day and age.

So clock rate is the only thing that matters in your world?
In my world, high-end means high quality. It doesn't necessarily
mean fast, or recent.

Warren writes:
> A. Use ahci(4):
>    "The ada device driver takes full advantage of NCQ, when supported."

Achi and siis support NCQ, but neither attach to the nforce4-ultra,
which does support NCQ. I knew that NCQ would be required for acceptable
performance and gave up other useful features to get it. Silly me,
assuming that the performance orientated version of BSD would
support such an essential performance feature on a very popular chipset.
Linux has had NCQ on nforce4 since Oct 2006.

Without NCQ, writes are slower than USB2, even with purely sequential
writes (to the raw drive, no filesystem, so no seeking other then to
the next track):

device    r/s    w/s     kr/s     kw/s wait  svc_t  %b  controller
   ad6 1726.5    0.0 217535.0      0.0    1    0.6  96  ata3-master

   ad6    0.0  109.5      0.0  13794.8    1    9.1 100  ata3-master

   da0  249.9    0.0  15741.4      0.0    1    6.0 100  umass-sim0

ad6 is ST3000DM001-9YN166 on nforce4-ultra chipset
da0 is ST3000DM001-9YN166 USB3 disk on a USB2 port (from nforce)

Sorry no write performance data for da0, I yanked the USB-to-SATA bridge
off after doing some testing. The bridges they're currently shipping
are slightly less crappy than they used to be, but they are still crap.

> B. Use GPT, which does not have the CHS baggage.  It is easier and more
>    versatile.  My systems with GPT disks don't complain about track
>    alignment.  Or maybe that's ahci(4)'s doing.

I never found a way to boot from different partitions, much less
different disks with GPT. So I use NetBSD's MBR for disks I want
to boot from. FreeBSD's and NetBSD's fdisk are both broken.
Building MBRs by hand is such fun. Non-boot disks with multiple
partitions use GPT. Big data disks get newfs-ed directly, no
partitioning.

The useless CHS baggage hangs around for decades, but useful
hardware loses all support 5 nanoseconds after the last machine
is sold. Other useful hardware waits years hoping to get support.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

FreeBSD for serious performance? (was: Re: 9.x -- New Install -- serious partition misalignment)

2012-12-08 Thread Dieter BSD
Ronald writes:
> This probably wouldn't be such a big deal if we were just talking about
> Linux.  But FreeBSD has always prided itself on being a serious OS for
> serious people with serious work to do... like major server farms and
> such.  In the context of high-end applications on high-end hardware where
> people are often trying to squeeze out that last drop of performance,

Linux is certainly a steaming pile of crap. BSD is orders of magnitude
better, but hey, that doesn't take much.

But don't brag about high-end hardware.  But FreeBSD has dropped support
for even semi-high-end hardware (DEC Alpha). So I'm stuck running it on
AMD64. Nothing against AMD, they did what they could to try and make a silk
purse (amd64) out of a sow's ear (x86). But even getting what passes for
a high quality board in amd64/x86 land with good reviews doesn't compare.
The firmware is absolute crap, and it's not like it is something you can
ignore. BTW, real high end hardware is redundant, better than mil-spec,
and provides better than 5-9s uptime. Been there, done that.

Several chips/features aren't supported properly. PRs sit for years on end.

> Performance has been degraded by a whopping 75% !

Having a 4KiB misalignment is nothing compared with not having NCQ
support. (Which even linux has, btw.) 25% performance would be a massive
upgrade. Or even worse, having the disk driver go into an infinite loop
with interrupts blocked, so *nothing* happens and all your incoming data
is lost until you manually intervene.

Speaking of alignment, I still get "partition 1 does not end on a
track boundary" messages. FreeBSD has no clue where the track boundaries
are and neither do I. Disks have used varying numbers of sectors/track
for longer than FreeBSD has existed.

This is your idea of serious?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: old style kernel configuration

2012-11-23 Thread Dieter BSD
Julian writes:
> it is however a good way to get mismatching kernel and userland
> but that's not what we are discussing.

The method recommended on
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-building.html
"# make buildkernel KERNCONF=MYKERNEL"
is also a good way to get mismatching kernel and userland. Or any other
way of building just the kernel rather than everything.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Memory reserves or lack thereof

2012-11-11 Thread Dieter BSD
Alan writes:
> In conclusion, I think it's time that we change M_NOWAIT so that it doesn't
> dig any deeper into the cache/free page queues than M_WAITOK does and
> reintroduce a M_USE_RESERVE-like flag that says dig deep into the
> cache/free page queues.  The trouble is that we then need to identify all
> of those places that are implicitly depending on the current behavior of
> M_NOWAIT also digging deep into the cache/free page queues so that we can
> add an explicit M_USE_RESERVE.

find /usr/src/sys | xargs grep M_NOWAIT | wc -l
2101

Sounds like a lot of work that would need to happen atomically.
Would this work:

M_NO_WAIT       do not sleep, do not dig deep unless M_USE_RESERVE also set
M_USE_RESERVE   dig deep
M_NOWAIT        M_NO_WAIT | M_USE_RESERVE (deprecated)

New code avoids using M_NOWAIT. Existing code continues working the same way.
As time permits, old code is converted to new flags. Eventually M_NOWAIT
goes away.

Pro: the amount of code that needs to change atomically is much smaller.

Con: (1) Have to remember (or look up) difference between M_NOWAIT
and M_NO_WAIT. Maybe calling the new flag M_NO_SLEEP would help?
(2) Would M_NOWAIT really ever go away? The spl() calls haven't,
even after some cage rattling.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: FreeBSD 8.0 suddenly freezing

2012-09-20 Thread Dieter BSD
> In the last 72 hours, we've had two different systems freeze; they don't
> apparently recognize any interrupts, they won't respond to ping, and
> they require a powercycle to reboot. We can't easily generate an NMI on
> these boxes.

> Just on the off chance, does this sound like a familiar -- and
> preferably soluble -- problem?

Only theory I can think of is that some device driver shuts off
all interrupts and then goes into an infinite loop.

I have had trouble with at least four different device drivers
locking everything out long enough to lose incoming data.
Usually it recovers without a reboot. Not an infinte loop, just
way way too long to leave interrupts shut off.

See also the "How to diagnose system freezes?" thread from July-August.
Yuri found that a particular rev of the nvidia driver went into
an infinite loop.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: How to diagnose system freezes?

2012-08-27 Thread Dieter BSD
Yuri writes:

> Anything else I can try?
>
> One thing of importance here is that there is an older graphics card
> 9400 GT on this system and current nvidia-driver-295.71 has an issue
> with 9400 GT: it makes graphics to malfunction (unpainted windows, long
> delays switching to terminal mode) or freezes Xorg (but not OS). So I
> run the older nvidia-driver-285.05.09 which appears to work.
> That's why I think that nvidia driver is probably to blame for these
> periodic OS freezes. Also the latest driver version must be, obviously,
> working for most people because (I think) they mostly have newer than
> mine nvidia cards.

Have you found a way to trigger the bug on demand?

Since you suspect the nvidia-driver-285.05.09, try some other
driver, and do whatever triggers the bug and see if you get the freeze.

> So maybe I should also just get the newer nvidia card
> and shut up, not sure.

If you can demonstrate that the various nvidia drivers are broken
in various ways, submit a problem report to whoever wrote the drivers
(Nvidia presumably). If Nvidia supports their products, then they
will fix their drivers. If they don't support their products,
why would you want to reward them by buying another nvidia card?

But the question I have is: why are device drivers allowed to
freeze the entire machine?

I have at *least* 4 drivers that freeze the machine long enough for
data to be lost. My theory is that:

1) An interrupt comes in.
2) ALL INTERRUPTS ARE BLOCKED !
3) The device driver sits around too long.
4) Eventually the interrupts are turned back on.

If the device driver gets stuck in an infinite loop, the machine
hangs forever.

Assuming my theory is correct (anyone disagree?), then WHY are ALL
interrupts blocked? Why can't we just block interrupts for that
particular device?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: How to diagnose system freezes?

2012-08-03 Thread Dieter BSD
Adrian writes:
> the "cycle over a short period" may not be the driver writing the same
> crap to the card. I've seen similar failure modes in windows where
> during playback, if the system hangs for whatever reason, the card
> plays the last sample over and over in a loop.

Ok, that makes sense.  So it could be any device driver.

Seems like diagnosing a device driver that is stuck in an
infinite loop (or equivalent) with all interrupts shut off
would be rather difficult.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: How to diagnose system freezes?

2012-08-02 Thread Dieter BSD
Yuri writes:
> One of my 9.1-BETA1 systems periodically freezes. If sound was playing,
> it would usually cycle with a very short period. And system stops being
> sensitive to keyboard/mouse. Also ping of this system doesn't get a
> response.

So the sound continues, on and off, while everything else freezes?
I would suspect that the sound device driver is locking out other
devices and hogging the system.

WHY does FreeBSD allow one device driver to lock out out everything
else? And what can we do about it? I've seen a device driver lock
out everything else for minutes-on-end, which of course resulted
in data being lost. Losing data is one of the worst things an OS
can do. This needs to be fixed.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Awful FreeBSD 9 block IO performance in KVM

2012-07-21 Thread Dieter BSD
>>> da0: 3.300MB/s transfers
>>> da0: Command Queueing enabled
>
> root@freebsd:/root # dd if=/dev/zero of=/dev/da1 bs=16384 count=262144
>
> 4294967296 bytes transferred in 615.840721 secs (6974153 bytes/sec)

1) Does a larger block size (bs=1m) help?

2) That's roughly the speed I'd expect without queueing. Is it really
making effective use of queueing, or is something limiting queueing to
one transfer at a time?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Pull in upstream before 9.1 code freeze?

2012-07-17 Thread Dieter BSD
>> Why not create a command wtf(1)?
>>
> there are really lot of good features that can be made in FreeBSD.
> actually good, instead of that crap

While this is certainly not the most important improvement that could
be made (Fix the PRs!), the proposed wtf command could be useful.
And, importantly, putting this functionality into a seperate utility
is the correct way to do it. We don't want to add more complexity to
the shell for several good reasons (bloat, possiblilty of bugs,
slowness after every typo, not the Unix way[1], etc.).

[1] "Make each program do one thing well. To do a new job, build afresh
rather than complicate old programs by adding new features."

http://www.faqs.org/docs/artu/ch01s06.html
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Training wheels for commandline

2012-07-05 Thread Dieter BSD
> As long as it can be toggled off system-wide, persistently (sysctl?), I
> can't see the harm in bringing that in.

It violates the Unix Philosophy.

"Make each program do one thing well. To do a new job, build afresh
rather than complicate old programs by adding new features."

http://www.faqs.org/docs/artu/ch01s06.html
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


geom metadata (Re: [CFC/CFT] large changes in the loader(8) code)

2012-06-28 Thread Dieter BSD
These schemes to just put the metadata in some special location
and have all the tools know about it create a lot of problems.
There is always some tool that doesn't know. There is always
some human that doesn't know. Telling the difference between
real metadata and some other data that happens to look similar.
Convoluted logic that is prone to bugs. I have seen complaints
from people that have lost data when some tool wrote metadata
on top of it. Losing data is absolutely unacceptable.

There is a time to be clever and a time to just keep it simple.

Define a "FreeBSD geom metadata" GPT partition type.
Create a 6 sector (3 KiB) "FreeBSD geom metadata" GPT partition just after
the GPT header.

PMBR
pri GPT header
pri GPT table
FreeBSD geom metadata
data partition(s)
sec GPT table
sec GPT header

Advantages:
1) All OSes will know that this space is taken.
2) Humans looking at the GPT partition table will know that this space is
taken, and what it is being used for.
3) The 1st data partition becomes 4 KiB aligned, which is important for
many recent disks (yes the metadata partition is not 4K aligned, but is
presumably accessed only rarely, so it is not a performance problem)

Disadvantages
1) uses up a partition type
2) uses up a partition
With GPT neither of these disadvantages is significant.

Alternately one could make the geom metadata partition smaller and
add a spaceholder partition to get 4K alignment. Yes you can just
leave a hole, but putting a partition there labled "4K_alignment"
makes it obvious why it is there.

So, what have I missed?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Freeze when running freebsd-update

2012-06-27 Thread Dieter BSD
 Robert writes:
> 3) the box is responsive to hitting enter at the console (it produces
> another login: prompt)

 Getty is in memory and can run.

> 5) if I try to login to the console, it lets me enter a username then
> locks up totally, it does not present me with a password: prompt.

 Login(1) is not in memory, and the kernel cannot read it from disk
 for some reason.

 I can get this symptom by writing a large file to a disk on a
 controller that FreeBSD doesn't support NCQ on. I assume there
 is a logjam in the buffer cache. Something trivial like reading
 login in from disk that would normally happen in well under a
 second can take many minutes.

 Perhaps geli is causing a similar logjam? Does it hang forever or
 is it just obscenely slow? If it truely hangs forever it is
 probably something else. Is there disk activity after it hangs?
 Can you try it without geli? systat -vmstat might provide a clue.
>>>
>>> Well, it is geli. I'm unable to reproduce the freeze on the same
>>> exact system with everything else the same except for no geli. I'm
>>> going to move this thread over to geom, and continue it there. Thanks
>>> for your help!
>>
>> It occurs to me that it will need twice as much memory for disk i/o.
>> 1 buffer for encrypted and 1 for unencrypted. I know nothing about geli,
>> so I don't know if it uses the buffer cache for both, or what.
>> Could it be that the kernel isn't keeping enough memory free and
>> manages to paint itself into a corner and not have space to store
>> the unencrypted version of disk reads, and can't page/swap anything
>> out to make space because it doesn't have space to store the encrypted
>> version to write?
>
> I think that's probably about what is happening. I'm still waiting
> for an answer on the geom mailing list, but I will do some testing
> with increasing memory sizes and see where the problem stops
> occurring.

Some of the vfs.*buf sysctls might be useful?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Freeze when running freebsd-update

2012-06-26 Thread Dieter BSD
>> Robert writes:
>>> 3) the box is responsive to hitting enter at the console (it produces
>>> another login: prompt)
>>
>> Getty is in memory and can run.
>>
>>> 5) if I try to login to the console, it lets me enter a username then
>>> locks up totally, it does not present me with a password: prompt.
>>
>> Login(1) is not in memory, and the kernel cannot read it from disk
>> for some reason.
>>
>> I can get this symptom by writing a large file to a disk on a
>> controller that FreeBSD doesn't support NCQ on. I assume there
>> is a logjam in the buffer cache. Something trivial like reading
>> login in from disk that would normally happen in well under a
>> second can take many minutes.
>>
>> Perhaps geli is causing a similar logjam? Does it hang forever or
>> is it just obscenely slow? If it truely hangs forever it is
>> probably something else. Is there disk activity after it hangs?
>> Can you try it without geli? systat -vmstat might provide a clue.
>
> Well, it is geli. I'm unable to reproduce the freeze on the same
> exact system with everything else the same except for no geli. I'm
> going to move this thread over to geom, and continue it there. Thanks
> for your help!

It occurs to me that it will need twice as much memory for disk i/o.
1 buffer for encrypted and 1 for unencrypted. I know nothing about geli,
so I don't know if it uses the buffer cache for both, or what.
Could it be that the kernel isn't keeping enough memory free and
manages to paint itself into a corner and not have space to store
the unencrypted version of disk reads, and can't page/swap anything
out to make space because it doesn't have space to store the encrypted
version to write?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Freeze when running freebsd-update

2012-06-26 Thread Dieter BSD
Robert writes:
> 3) the box is responsive to hitting enter at the console (it produces
> another login: prompt)

Getty is in memory and can run.

> 5) if I try to login to the console, it lets me enter a username then
> locks up totally, it does not present me with a password: prompt.

Login(1) is not in memory, and the kernel cannot read it from disk
for some reason.

I can get this symptom by writing a large file to a disk on a
controller that FreeBSD doesn't support NCQ on. I assume there
is a logjam in the buffer cache. Something trivial like reading
login in from disk that would normally happen in well under a
second can take many minutes.

Perhaps geli is causing a similar logjam? Does it hang forever or
is it just obscenely slow? If it truely hangs forever it is
probably something else. Is there disk activity after it hangs?
Can you try it without geli? systat -vmstat might provide a clue.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: MAGIC with HP KVM - someone will help?

2012-06-23 Thread Dieter BSD
Wojciech writes:
> 1) i know for a friend that on some motherboards (it was embedded VIA
> based CPU) geli doesn't work at typing password.
>
> 2) in my case (as well as his case) problems happen everytime kernel
> function cngets is used, it maybe after being unable to mount root in
> a kernel prompt.
>
> The problem IS FreeBSD specific AND some hardware specific.
>
> this seems like an interaction between FreeBSD and some hardware
> (timings?). Really have no idea.


>> I refuse to move from the keyboard I have currently, and occasionally
>> keys get 'stuck' but it's so obvious when it happens it's no big
>> hardship. I have a Belkin adaptor one by the way, and it works
> but the PROBABLY FReeBSD specific is that some configuration, not
> only USB to PS/2 adapters, have problems only when kernel cngets routine.
>
> my friend switched to linux, in spite of lower performance, on his
> dedicated fileserver machine with VIA processor, only because keyboard
> didn't work when geli asked for password at boot, or - for whatever reason
> - in kernel command prompt activated.
>
> it works every other case properly.

This reminds me of a problem I have when using a RS-232 console.
When the loader is sitting there waiting for a few seconds to see
if you will type something, it doesn't always notice keystrokes.
I would have to hold the key down (letting it autorepeat) and then
it would usually notice. This problem does not happen with the
PS/2 keyboard. (no KVM, no USB adapters) If it is a timing problem,
perhaps it is related?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: how to turn my computer into a TV

2012-06-22 Thread Dieter BSD
user.vdr writes:
> Tuners do NOT provide raw audio/video to the system in any case.

 http://corona.homeunix.net/cx88wiki/Overview/RawVideo
>>>
>>> While that's technically possible in _some_ cases, and assuming it's
>>> fully implemented and functional, I'm unaware of any software that
>>> actually provides raw data to the user. I suppose I should have worded
>>> my point differently.
>>
>> The cx88wiki URL above describes the cx88 software (in ports).
>> For tuners without a hardware encoder, raw video/audio is the only
>> thing you can get from the tuner when receiving NTSC.
>
> Nope.

Prove me wrong.  Post the command line to have cx88 (in ports) output
encoded (mpeg or similar) video with a pcHDTV HD3000 tuner card
receiving a NTSC input.

Since you claim that the cx88 output is already encoded, piping the
output into mplayer or similar to do the encoding doesn't count.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: how to turn my computer into a TV

2012-06-22 Thread Dieter BSD
user.vdr writes:
>> As long as there remain some NTSC broadcasts, there might be some
>> that you wish to watch. That's why I wrote:
>
> Yes, technically there are still some that exist, for now. However,
> their death certificate is signed and they're so few that it's not
> worth mentioning.

If you don't think NTSC is worth mentioning, why do you keep posting
the same incorrect statements over and over again?

>>> You absolutely do NOT have to reencode a stream
>>
>> I did not say anything about RE-encoding anything. Only about
>> encoding/compressing the high bandwidth datastream the tuner
>> generates from NTSC. And to be clear, this only applies to
>> NTSC, not to ATSC.
>
> NTSC streams are not broadcast raw. What do you call encoding data
> that's already encoded if you don't think it's reencoding? Also, doing
> so causes degredation so unless there's a need for the user to do so,
> he's better off not wasting his time.

NTSC is not a stream of bits. NTSC is analog. The tuner converts
the NTSC analog waveform into a raw stream of bits. This raw
stream of bits is too large to conviently store on disk, so it
needs to be compressed/encoded into mpeg or similar. Some
tuners include a hardware encoder, but many do not.

>>> Tuners do NOT provide raw audio/video to the system in any case.
>>
>> http://corona.homeunix.net/cx88wiki/Overview/RawVideo
>
> While that's technically possible in _some_ cases, and assuming it's
> fully implemented and functional, I'm unaware of any software that
> actually provides raw data to the user. I suppose I should have worded
> my point differently.

The cx88wiki URL above describes the cx88 software (in ports).
For tuners without a hardware encoder, raw video/audio is the only
thing you can get from the tuner when receiving NTSC.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


(no subject)

2012-06-21 Thread Dieter BSD
user.vdr writes:
>>> Recording doesn't require any compression unless you are transcoding
>>> in real-time. There's no difference between recording ATSC, NTSC, PAL,
>>> etc, and it's actually irrelevant what the stream is.
>>
>> This is incorrect. ATSC is compressed before broadcast, so
>> you receive the data already compresed.  NTSC and PAL are
>> broadcast in analog. The tuner performs A-to-D which gives
>> an uncompressed data stream.  Have fun trying to store that.
>> As a practical matter, you have to compress the data in real time.
>> Some, not all, tuners include hardware compression.
>
> With very very very few exceptions, all analog NTSC broadcasts have
> been switched to digital, by the FCC mandated deadline of June 12,
> 2009.

As long as there remain some NTSC broadcasts, there might be some
that you wish to watch.  That's why I wrote:

 You'll need to know if you have any NTSC (analog) stations you
 care about or if everything is ATSC (digital).

Aryeh may or may not have any NTSC stations of interest. Given
"we are using a broadcast signal only [current US {NYC} standards]"
I rather doubt that Aryeh has any PAL stations to worry about.

> You absolutely do NOT have to reencode a stream

I did not say anything about RE-encoding anything.  Only about
encoding/compressing the high bandwidth datastream the tuner
generates from NTSC.  And to be clear, this only applies to
NTSC, not to ATSC.

> Tuners do NOT provide raw audio/video to the system in any case.

http://corona.homeunix.net/cx88wiki/Overview/RawVideo
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: EFI development tools

2012-06-18 Thread Dieter BSD
> This is a known issue, and had been around for a long time.
> You can't reliably build 32 bit binaries (what the -m32 flag specifies)
> on a 64 bit system. The header files (and possibly other things) are wrong.

People build 32 bit binaries on 64 bit systems all the time.
It is called cross-compiling. I rather doubt that just
setting the -m32 flag is sufficient to set up a proper
cross-compiling environment.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: how to turn my computer into a TV

2012-06-18 Thread Dieter BSD
user.vdr writes:
> Recording doesn't require any compression unless you are transcoding
> in real-time. There's no difference between recording ATSC, NTSC, PAL,
> etc, and it's actually irrelevant what the stream is.

This is incorrect.  ATSC is compressed before broadcast, so
you receive the data already compresed.  NTSC and PAL are
broadcast in analog.  The tuner performs A-to-D which gives
an uncompressed data stream.  Have fun trying to store that.
As a practical matter, you have to compress the data in real time.
Some, not all, tuners include hardware compression.

> Lastly, it's possible to save a single channel or the entire stream
> which usually contains several channels. Even when saving the full
> stream, it likely uses far less bandwidth than your media offers so
> there's no problem there.

This appariently refers to ATSC.  Yes, modern disks have plenty
of bandwidth to store the entire ATSC stream.  The main reason
to filter PIDs is to save disk *space*.  Also, some software
can't select which program to decode.

Wojciech writes:
> most people vastly underestimate power of modern CPUs.

Many people overestimate the "moderness" of most people's CPUs.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: how to turn my computer into a TV

2012-06-17 Thread Dieter BSD
[ Added multimedia@ as that is a more appropriate list than hackers ]

> I just moved into a very cramped apartment
> we are using a broadcast signal only [current US {NYC} standards]

You'll need to know if you have any NTSC (analog) stations you
care about or if everything is ATSC (digital).  Hopefully your
building has a good antenna system.

Next, select your tuner(s).  You have a choice of tuners that connect
via Ethernet (HDHomeRun, pros: small external box, doesn't need a slot,
works with any OS, better diagnostic info than others. cons: I've seen
a *lot* of postings with people saying that various other tuners get better
reception, digital only no NTSC analog), Firewire (if they are still
available?), USB, PCIe cards (needs a slot), PCI cards (needs a slot).
Some cards also do FM radio.

> 2. What ports to install

Depends on what tuner(s) you select.

Cards based on cx88 need multimedia/cx88 and multimedia/libtuner
http://corona.homeunix.net/cx88wiki
has a list of supported cards.

Some cards are supported by bktr(4).

Make sure the tuner you select is supported by FreeBSD.

Recording ATSC takes very little CPU.  Recording NTSC takes either
a lot of CPU or hardware compression.  Decoding either takes a lot of CPU
(or hardware decoding which AFAIK FreeBSD doesn't have). You can use
at(1) for automated recordings.  A full ATSC channel is 19.3 Mbps.
Some tuners allow filtering by PID, which saves disk space.

For playback you can use mplayer or any of several similar programs.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Solving the great resource problem

2012-06-14 Thread Dieter BSD
Spending resources to create more releases is pointless when
the PRs aren't getting fixed.  "Oh, Look!  Release 9.2.2.2.2.2
is out!  The system still crashes every 5 seconds, but a typo
on the true(1) man page is fixed."

We need a more global discussion about all the things that
resources are spent on, and which are the most useful.

Replacing perfectly good components simply because they
are GPL. The purpose of BSD is supposed to be creating a
great OS, not providing software hoarders with a supply
of free code to abuse.

Sending people to conferences. Nice, but clearly a luxury.

Meanwhile the hardware support is a disaster. PRs sit for
years and years and years.  The documentation has plenty of
room for improvement.

It seems there are never enough resources to fix problems,
but somehow there are always resources to do yet another fork.
FreeBSD, NetBSD, OpenBSD, DragonflyBSD, and now Bitrig.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: FreeBSD Boot Times

2012-06-14 Thread Dieter BSD
Brandon writes:
> Booting into Ubuntu minimal or my own custom Linux distro,
> literally takes 0.5-2 seconds to boot up to shell

0.5-2 seconds from power-on to a shell prompt?  How do you
get through the firmware that fast, much less firmware plus
an OS?

Which reminds me, back when I was triple-booting Free, Net, and
penguinix, I noticed that when rebooting, penguinix didn't go
through all the firmware stuff like the BSDs do.  That is one
way to save a lot of time, at least for reboots.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: GSoC Project: Automated Kernel Crash Reporting System - Discussion

2012-05-25 Thread Dieter BSD
1) tar up files
2) encrypt tarball
3) copy encrypted tarball with rcp, ftp, uucp, ...
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: csh builtin command problems

2012-05-10 Thread Dieter BSD
Robert writes:
> I want this:
>
> # echo test\ttest > test
> # cat test
> test    test

I have given up on using echo for anything the least bit fancy,
in favor of printf(1) which gives much better control.

printf "test\ttest\n"
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Ways to promote FreeBSD?

2012-05-04 Thread Dieter BSD
*WHY* is Linux so much more popular than the BSDs?

GPL vs BSDL ? (Create a GPLed BSD and see if it takes off.)

the obese cartoon penguin?

Do most people actually prefer the lower quality product?
Popularity is inversly proportional to quality in many
areas, not just OSes.

marketing?

Is there something that Linux does better than the BSDs,
that many people care about?

--

Is there a list somewhere of companies/organizations that
contribute to FreeBSD? A list of those that freeload
(Use FreeBSD as part of their product but do not contribute
back.)? Do we need to recrute a diplomatic fund raiser
person to shake the bushes?

--

Are the resources we do have being used as effectively as possible?
Looks to me like funds are being spent on things that would likely
happen anyway (because they are "fun", would result in a paper, or
are commercially useful), while things that need doing but aren't
always fun (such as fixing PRs) don't get funding and never get done.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Is there any modern alternative to pstack?

2012-04-16 Thread Dieter BSD
Konstantin Belousov wrote:
> My opinion is that such tool should be imported into the base.

Why?

Don't optional tools belong in ports?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


mlock/mlockall (was: Re: problems with mmap() and disk caching)

2012-04-10 Thread Dieter BSD
Andrey writes:
> Wired memory: kernel memory and yes, application may get wired memory
> through mlock()/mlockall(), but I haven't seen any real application
> which calls mlock().

Apps with real time considerations may need to lock memory to prevent
having to wait for page/swap.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Graphical Terminal Environment

2012-04-09 Thread Dieter BSD
Brandon writes:
> I'm still avidly trying to work on this idea, but right now the issue
> seems to be with AMD and NVIDIA not documenting their protocols. Intel
> does a good job, but I don't have any Intel chips with graphics laying
> around.

I thought that AMD had documented most of it by now, with the
major exception of the UVD?

> I'm working on a minimal GPU right now on an FPGA.

> Currently it looks like I could run 4 monitors at 1080p for about $50

Have FPGA prices come down that much?  The OGP-D1 was quite a bit
more than that, last time I looked.  Or would that be the price for
a production version with an ASIC?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

2012-03-30 Thread Dieter BSD
> Subsequent inspection suggested that it was happening during the
> periodic daily, though we never managed to get it to happen by manually
> forcing periodic daily, so that's only a theory.

Perhaps due to a bunch of VMs all running periodic daily at the same time?

> We had a perfectly functional, nearly zero-traffic VM, since Jabber
> traffic averages no more than a few messages per hour.  It was working
> for quite some time.
>
> We moved it from a local datastore to an iSCSI datastore that ended up
> getting periodically crushed by the load (in particular during the
> periodic daily load imposed by a bunch of VM's all running at once).
> At this point, this one VM started hanging on I/O.  We expected that
> this would clear up upon return to a host with a local datastore.  It
> did not.
>
> This ended up as a broken VM, one that would hang up overnite, maybe
> not every night, but several times a week at least.

...

> For the problem to "follow" the VM in this manner, and afflict *only*
> the one VM, strongly suggests that it is something that is contained
> within the VM files that constitute this VM.  That is consistent with
> the observation that the problem arose at a point where the VM is
> known to have had all those files moved from one location to a dodgy
> location.
>
> That's why I believe the evidence points to corruption of some sort.

Compare a backup of the VM before it broke to a backup of the same VM
after it broke.  Hopefully the haystack of insignificant differences
isn't too large, or the significant difference needle might be a
lot of "fun" to find.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: mlock(2) man page errata

2012-03-30 Thread Dieter BSD
>> mlock(2) says:
>>
>> > A single process can mlock() the minimum of a system-wide
>> > ``wired pages'' limit and the per-process RLIMIT_MEMLOCK
>> > resource limit.
>>
>> Shouldn't this say maximum rather than minimum?
>
> I don't think so.  The minimum of the two would be the limit that you
> will hit first, and presumably is the point at which you cannot mlock
> any more pages.

Ok, but "can mlock() the minimum of" is easy to misread as
"can mlock() at least".  Perhaps it would be more clear to say
something like

 The amount of memory that a process can mlock() is limited by the
 per-process RLIMIT_MEMLOCK resource limit, and by a system-wide
 ``wired pages'' limit.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

2012-03-29 Thread Dieter BSD
> FreeBSD ?? - 7.4 never crash
> FreeBSD 8.0 - 8.2 crashes

Obvious short term workaround is to run production on 7.4 (assuming you can)
until you figure out what is wrong with 8.x.

What filesystem(s) are you running? UFS? ZFS? other?

> started randomly disconnecting people every morning

Due to timeouts? Something might be keeping interrupts disabled
too long.

> there were other good reasons to reload the
> VM, so I nuked the VM, which, of course, fixed it.

> I can look at recovering the faulty VM from backup

Sounds like corruption.  Can you compare the bad VM against a good
one?  If you find corruption, the question then becomes what is causing
the corruption?  Sounds like the same thing is getting corrupted
every time, rather than something at random.

Sounds like the corruption is causing a deadlock in something
common, like the buffer cache, or filesystem, or...

Is it possible to have root be a ramdisk?  This might give you
access to the utilities, depending on where the problem is.

I have vague memories that the sticky bit used to lock a program in
memory, but sticky(8) indicates that this is no longer the case.
Is there a way to lock a program in memory? (So that it will be available
when the system can't do disk i/o.)  If not, you could keep some
windows open with things like top and systat -vmstat running.

Some of the utilities have options to look at a disk file rather than
the live system, if you can get a core dump (swap to NFS?).
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


mlock(2) man page errata

2012-03-29 Thread Dieter BSD
mlock(2) says:

> A single process can mlock() the minimum of a system-wide
> ``wired pages'' limit and the per-process RLIMIT_MEMLOCK
> resource limit.

Shouldn't this say maximum rather than minimum?

> [EAGAIN] Locking the indicated range would exceed either the
> system or per-process limit for locked memory.
>
> [ENOMEM] Some portion of the indicated address range is not
> allocated. There was an error faulting/mapping a page.

In some cases, the code returns ENOMEM if the limit is exceeded.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Reigning in disk cache buffer hogs

2012-03-18 Thread Dieter BSD
amd64
4 GiB
nVidia nForce CK804 (aka nforce4-ultra), SiI3132, JMB363
sata disks
FreeBSD 8.2
FFS/SU

Problem: I/O to one disk can fill up the disk buffer cache,
starving other processes of disk I/O.  If the other processes
are logging real time data (from a closed source "black box"),
then data is lost.  Each process has its own private disk
to minimise latency due to seeking and such, but that isn't
sufficient to prevent the problem.

It is always the onboard nforce4-ultra controller that causes the
problem. The expansion card controllers never cause the problem.
The onboard nforce4-ultra controller is supported by ata(4).
The expansion card controllers are supported by ahci(4) and siis(4).
The onboard nforce4-ultra controller is much faster than the
expansion card controllers, except that ata doesn't support NCQ,
so the nforce has a serious bottleneck writing.

Is there something about NCQ other than the obvious performance
improvement that would explain why the NCQ controllers never have the
problem?

What sort of miracle would it take to get NCQ support for the
nforce4-ultra controller?  Linix has had NCQ support since 2006.

Is there some knob to control reserving a few buffers per disk
to prevent one disk from hogging all the buffer cache?

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Graphical Terminal Environment

2012-03-06 Thread Dieter BSD
Brandon writes:
> (If you haven't noticed, I'm going to keep finding excuses to
> write this as I really am kindof excited to learn/work on it)

ideas:

Display PostScript

rio (from Plan 9)

If you're mainly looking for a low-level graphics project,
maybe reverse engineer something on the GPU (e.g. UVD)
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: OS support for fault tolerance

2012-02-24 Thread Dieter BSD
>> The problem then is how to feed both machines the same inputs, and
>> compare the outputs. Do we need a third machine to supervise?
>> Can we have each machine keep an eye on the other, avoiding the
>> need for a third machine?
>
> A pair would work as long as the only failures are "obvious" (e.g.
> crashes).  If they simply disagree as to the result, how would we
> determine which one was right?

Depends on what sort of work the machine is doing.  If the job is
something that can be done again, you could simply try again, if
you still get different answers try a third machine or wade in and
start manually inspecting things until you find the problem.
If the job is time critical or you can't get the same inputs again,
then the machine needs to get it right the first time.  How many
9s of reliability do you need and how many resources can you throw
at it?  2x hardware can be good for better than 5 9s. (high quality
hardware and software, and technicians standing by with cold spares)
I've heard that mil gear uses 3x hardware.

Building a 5 9s system is... non-trivial.  So I'm wondering what sort
of reliability we can get with 2x off the shelf commodity hardware
and a bit of software?  Similar to mirroring/RAID but with whole
computers rather than just disks.  Classic Unix technique of doing
10-20% of the work and getting 80-90% of the result.

>> Which then leads to the issue of how to avoid problems when *it*
>> breaks.
>
> For some reason, this reminds me of a Dr. Seuss story:
> http://www.goodreads.com/review/show/49519038

*grin*  Gotta love Dr. Seuss.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: OS support for fault tolerance

2012-02-20 Thread Dieter BSD
Rayson writes:
> The question is, are we planning to handle >95% of the errors for >99%
> of the hardware we run on, or are we really planning to spend years
> trying to design something that would require special hardware
> support?

I assume this started as: "Oh look, most CPUs have multiple cores
these days, maybe we could play with fault tolerance".  Which
could be useful if CPU cores failed a lot, but in reality what
fails is disks, disks, controllers, disks, random other things,
and disks.  Assuming you have avoided the garbage-quality stuff,
and have the system on a UPS.  If you have enough ports you can
add more disks and mirror or some other version of RAID.

The next step is to duplicate everything.  Not by looking for
a mainboard with redundant everything, but by simply adding
another computer.  And rather than getting two of the same machine,
you're better off if they are different, so that they don't have
the same bugs.

The problem then is how to feed both machines the same inputs,
and compare the outputs.  Do we need a third machine to supervise?
Which then leads to the issue of how to avoid problems when *it* breaks.
Can we have each machine keep an eye on the other, avoiding the
need for a third machine?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Giant lock gone?

2012-01-19 Thread Dieter BSD
>>> The original goal for 5.0 was to completely remove the Giant lock (and
>>> do other cool SMP-related stuff). Eventually it was realized that this
>>> was too big a goal to fully accomplish in 5.0 (albeit too late in the
>>> process) and the goal was changed to do the basic framework for the new
>>> SMP model; and lay the groundwork for "some things run under Giant for
>>> now, and we'll remove it from them ASAP." That actually turned out to
>>> last through 6, making 7 the realization of what 5.0 was supposed to be.
>>
>> So you are saying that the Giant lock was completely removed in 7.0?
>>
>> 8.2 says:
>>
>> atkbd0: [GIANT-LOCKED]
>> psm0: [GIANT-LOCKED]
>
> Yeah, I think nitpicking minor details about Giant still being used on
> non-critical drivers is definitely the key to solving the problems that
> face FreeBSD today.
>
> I could have more thoroughly clarified the reality of the when/where/how
> of Giant but it really wasn't central to my point.

Sigh. I wasn't intending to nitpick, I was confused and curious by the
apparient contradiction between "completely remove" and
what I saw.

8.2 again:
find /usr/src/sys | xargs grep -i giant | wc -l
2018
Skimming though the output of grep, a few are just false positives,
but most appear to involve the Giant lock. Perhaps these are all
unimportant, I don't know.

Apologies to anyone I upset.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Getting PRs fixed

2012-01-19 Thread Dieter BSD
Igor writes:
> You mean something like: http://people.freebsd.org/~edwin/gnats/ ?

Daniel writes:
> http://www.oook.cz/bsd/prstats/

Yes, something like these.

Stephen writes:
> You should get extra points for difficult PR's. One way to measure this
> would be to give more points for fixing older PR's than newer PR's.

Older might be harder, or it might mean boring, or seen as less important.
It might be worth giving more points for old PRs regardless, to help
get the old ones fixed. The main goal here is to get PRs fixed.
It just feels wrong to have PRs sitting around for years on end,
and if it is a significant problem you can be sure that the submitter
is unhappy about not having a fix.

A longer PR might be more difficult. A complex problem takes longer to
describe, and longer to fix, than just "there is a typo in the foo(1)
man page".

One problem is PRs that are closed without being fixed. Some of these
are legitimate (dups, submitter error, already fixed in newer release, ...)
but some shouldn't have been closed.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Getting PRs fixed (was: Re: ...focus, longevity, and lifecycle)

2012-01-18 Thread Dieter BSD
Andriy writes:
> And dealing with PRs is not always exciting.

Neither is brushing your teeth or cleaning the kitchen, but most of us
manage to do them at least occasionally. Part of being a grown up.

Instead of looking for a stick to hold over developers to get them
to fix PRs, let's look for carrots to make fixing PRs more appealing.

Idea 1: Fix 'n' PRs, get a tee-shirt, fridge magnet, plush daemon, ...

Idea 2: Give it status. Set up a web page with PR fixing stats

name/handle..total PRs fixed...fixed in last 12 months...average fixed/year
Sheldon..150...9072
Leonard..131..11067
Howard...104...2052
Raj...80...8080
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Giant lock gone? (was: Re: ...focus, longevity, and lifecycle)

2012-01-18 Thread Dieter BSD
> The original goal for 5.0 was to completely remove the Giant lock (and
> do other cool SMP-related stuff). Eventually it was realized that this
> was too big a goal to fully accomplish in 5.0 (albeit too late in the
> process) and the goal was changed to do the basic framework for the new
> SMP model; and lay the groundwork for "some things run under Giant for
> now, and we'll remove it from them ASAP." That actually turned out to
> last through 6, making 7 the realization of what 5.0 was supposed to be.

So you are saying that the Giant lock was completely removed in 7.0?

8.2 says:

atkbd0: [GIANT-LOCKED]
psm0: [GIANT-LOCKED]
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: FreeBSD has serious problems with focus, longevity, and lifecycle

2012-01-18 Thread Dieter BSD
John writes:
> - EOL 7
> - mark 8 as legacy
> - mark 9 as the _only_ production release
> - release 10.0 in January 2017

Until a few days ago 8 was the latest, shinest release.
So you want to suddenly demote it all the way down to legacy?
I thought the goal was to have releases that can be used for a long time?

On the one hand, many users want/need releases to have a long lifetime.
On the other hand, we want to be able to start a new release when
some new major feature is mature enough. If these features come out
often enough, we end up with a lot of releases to support, and
supporting a lot of releases uses up a lot of time and effort.
Assuming that a lot more resources aren't going to magically appear,
perhaps the solution is to ramp down the level of support for
older releases.

7 - legacy (only gets fixes for security, panic/hang, data loss)
8 - supported (security, panic/hang, data loss bugs, plus others as requested)
9 - very supported (most improvements that aren't massively disruptive)
stable - not quite bleeding edge, not a "release" yet, not recommended
for production, brave users can try out new features
current - bleeding edge

The legacy level shouldn't have many fixes, so it shouldn't take
too much time and effort. It should be possible to support multiple
"legacy" releases. If fixes only get backported to "supported"
releases when users request it, the amount of time and effort may
be low enough? I don't have a good idea of how many requests would
be made. If the requests are infrequent enough, it might be possible
to support more than one "supported" release. (Better names for
"supported" and "very supported" would be welcome.)

A time based schedule can make sense if there are a lot of small
changes. It doesn't work as well with fewer large changes. If the
clock says it is time for a new major release but none of the major
new features are ready, it doesn't make sense to do a major release.
A time based schedule might make more sense for minor and point
releases?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: FreeBSD has serious problems with focus, longevity, and lifecycle

2012-01-17 Thread Dieter BSD
Atom writes:
> i bought myself a LENOVO T510 when it first came out, around early 2010.
> it's got an i5 CPU and Arrandale GPU. it's two years old and on freeBSD i
> STILL can't run xorg properly with it.

I have a machine from 2005-08 that FreeBSD still doesn't support properly
in 2012. After much research I figured out that SATA NCQ was an essential
feature, and choose a mainboard with nforce4 to get it. FreeBSD still
doesn't support NCQ on nforce after all these years. Linux has had
NCQ on nforce4 since Oct 2006. FreeBSD also doesn't properly support
the onboard VIA firewire chip, which is still found on new mainboards
today. I don't necessarily expect support for every exotic chip out
there the first day they hit the street, but these are both popular
chips, and it is 6.5 years later. I'm not sure how an OS is supposed
to have "The power to serve" when it can't even talk to disks properly.
And all the device drivers that think it is ok to lollygag around
for absurd lengths of time with interrupts turned off, thus causing
data to be lost.

Damien writes:
> Check this PR I opened some months ago:
> http://www.freebsd.org/cgi/query-pr.cgi?pr=161123&cat=kern
>
> It was planned for 9.0-RELEASE, there is no mention of 8.x
> That's just the kind of problem John raises here.

Hey, at least you have a fix, and it is even in a release (I'm
assuming it made it into 9.0). The bigger problem is all the
bugs that don't have fixes at all.

Igor writes:
> patches go ignored (no, I don't have a reference)

Here is a PR that contains a patch, yet is still open after
several years. Also fixes an even older PR from Dec 2006.
Dinky little patch, works great. Should be easy enough
to code inspect, test, and check in.
http://www.FreeBSD.org/cgi/query-pr.cgi?pr=127717&cat=

Mark writes:
> Why is everyone so afraid of running -STABLE?

Experience.

John writes:
> Is three per year an insane schedule ? Remember, I am simultaneously
> advocating that FreeBSD stop publishing two production releases
> simultaneously, which is almost oxymoronic

Why is publishing two production releases almost oxymoronic?
Seems like a very good policy to me. Say you are running 8.1.
It is good to have the choice of going to a low risk 8.2 with bugfixes
or going to 9.0 with some major new feature at the expense of more work
and higher risk. If you want the option of sticking with a major
release series (say 8.x) for a long time, then there needs to be at
least two production releases supported. As fast as major releases
are coming out, probably 3 or 4. Why are major releases coming out
so often? Gotta compete in the large number war. NetBSD was at
1.x for years and years, then suddenly someone decided to change
the numbering scheme and they're off to the races. Firefox has
caught the same insanity.

I see complaints from medium-to-large sites, yet FreeBSD's budget is
so small. Surely it must be possible for these medium-to-large sites
to pay into a fund to improve things. FreeBSD clearly needs more
developers to fix problems, to code review, test, and check in patches,
and to improve the documentation. Judging from this email thread there
is a demand for more/better release engineering and backporting as well.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Sound on the system briefly interrupts when kde4 switches windows with nvidia card

2012-01-15 Thread Dieter BSD
> I have kde4 on 8.2 with translucency windows effect enabled and nvidia
> graphics driver.
> Every time I switch an active window or maximize some window, sound
> played by mplayer interrupts for ~0.5 sec. Very annoying effect. Problem
> disappears when windows effects in kde4 are turned off.
> Is this likely a bug in nvidia driver that it locks up the system when a
> lot of OpenGL operations are in progress? Or what might be a problem?

Some drivers do things like DELAY(big number) while interrupts are turned
off. Very bad. Look through the source, maybe you can find it.

We REALLY need a way to service one device without shutting off
interrupts from other devices.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: cron(8) mis-feature with @reboot long after system startup

2011-11-25 Thread Dieter BSD
The system doesn't go multiuser until the rc jobs complete,
even if you attempt to background them with '&'.  This can be
a problem with long running jobs.  I started using cron @reboot
for this reason.

I haven't run into the problem since I've never needed to run
/etc/rc.d/cron restart.

> Add an option to cron to check lastlog and if within 5 or 10 minutes
> of the last reboot, then call run_reboot_jobs().

Depending on timestamps might be okay as a temporary quick-and-dirty
workaround, but there is likely to be a case where it will also do the
wrong thing.  What if you need to restart cron within the 5-10 minutes?

Maybe something like: rc script sets a flag, cron_reboot script checks
and resets flag.  The flag could be a file ("> /tmp/rebooting_system").
Better yet, a run-the-reboot-script command line option could be added
to cron.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: easy way to determine if a stream or fd is seekable

2011-11-20 Thread Dieter BSD
> something like the following inside lseek() would take care of tape drives:
>
>        if (S_ISCHR(sb.st_mode) || S_ISBLK(sb.st_mode)) {
>                if (ioctl(io->fd, FIODTYPE, &type) == -1)
>                        err(1, "%s", io->name);
>
>                if (type & D_TAPE)
>                        return(EBADF)
>        }

I'd suggest ENODEV ("Operation not supported by device") rather than
EBADF ("Bad file descriptor").

To do this correctly, we'd need some standard way to ask the
device driver if the device can perform the seek or not.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: easy way to determine if a stream or fd is seekable

2011-11-17 Thread Dieter BSD
> lseek() on a tape drive does not return an error, nor does it
> actually do anything.

IIRC some tape drives can seek, while others cannot.
Vague memories that it is supposed to be possible to put a
filesystem on a DECtape and mount the filesystem.

It might be that FreeBSD doesn't currently support seeking
on a tape, but we shouldn't paint ourselves into a corner
by assuming that it is fundimentally impossible.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: checksum offload

2011-09-19 Thread Dieter BSD
>>> The data sheet for intel 82576 advertises IP TX/RX checksum offload
>>> but the driver does not set CSUM_IP in ifp->if_hwassist. Does this mean
>>> that driver (and chip) do not support IP TX checksum offload or the
>>> support for TX is not yet included in the driver?
>>
>> The first question is "is checksum offload a good idea?"
>> There is less protection against errors.
>
> This assumes firmware flaws, right? Albeit, it also does confuse some
> software like tcpdump.

"Unfortunately lots of high performance people use checksum offload
which removes much of the end to end protection"

      Alan Cox

http://kerneltrap.org/Linux/Data_Errors_During_Drive_Communication

Bugs in the hardware/firmware/software are not required to have a
problem, but if present would make things even worse.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: checksum offload

2011-09-18 Thread Dieter BSD
> The data sheet for intel 82576 advertises IP TX/RX checksum offload
> but the driver does not set CSUM_IP in ifp->if_hwassist. Does this mean that
> driver (and chip) do not support IP TX checksum offload or the support for
> TX is not yet included in the driver?

The first question is "is checksum offload a good idea?"
There is less protection against errors.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: excessive use of gettimeofday(2) and other syscalls

2011-09-09 Thread Dieter BSD
>> Firefox 5 and 6 has more gettimeofday call than 2 per second on my
>> amd64-8.2-stable box.

> i don't see why chromium needs
> to call gettimeofday(2) or any library function that triggers it more
> than 3000 times a second.

What the  are web browsers doing that they "need" the clock
so often?

I suspect the answer is the same as why firefox uses significant amounts
of CPU when it should be idle, why it uses memory without bound
(I actually had to add ulimit to my shell's rc file :-( ), and
so on.

Using "links -g",
"ktrace -di -tc -p6951; sleep 1; ktrace -C; kdump|wc -l"
gives a typical count of 300-400, highest count seen: 1454.

What we need, is a sanely written web browser that has the
features we need. Unfortunately the last time I checked,
links and dillo both lacked features needed for online
shopping/banking.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


optimum fs param sizes (was: ZFS installs on HD with 4k physical...)

2011-08-22 Thread Dieter BSD
>> I guess formatting the filesystem for 4k sectors on a 512b drive would still
>> work but it would be suboptimal.  What would the performance penalty be in
>> reality?
>
> It would be suboptimal but only for the slight waste of space that would
> have otherwise been reclaimed if the block or fragment size remained 512
> or 2K. This waste of space is insignificant for the vast majority of
> users and there are no performance penalties, so it seems that switching
> to 4K sectors by default for all file systems would actually be a good idea.

If you have large files, then large block/frag sizes waste less space than
small ones.  Remember that keeping track of all those blocks and frags
has overhead, and that overhead can waste more space than you save at the
end of files.

Is anyone looking at extending the range of tuning variables for FFS?
Allowing even larger block/frag sizes would be useful, as well as
larger cylinder groups and fewer inodes.  Fsck runs a *lot* faster with
fewer inodes.  The expected average file size and expected average number
of files per directory limits could use work as well.

newfs -e 1 -b 16384 -f 2048 -g 67108864 -h 16 -i 67108864 -U
 -o space -L 58data /dev/ada9p1
density reduced from 67108864 to 3676160
/dev/ada9p1: 2861588.5MB (5860533100 sectors) block size 16384,
 fragment size 2048
        using 12754 cylinder groups of 224.38MB, 14360 blks, 64 inodes.
        with soft updates

A cylinder group size of 224.38MB is just insanely small with today's
disk and file sizes.

IIRC the block size limit is 64K, but there was some bug with using
block/frag larger than 16K/2K.  Even 64K is too small.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: CONF class of files

2011-06-19 Thread Dieter BSD
> CONFDIR is for base, not ports

Perhaps ${BASE_CONF_DIR}, ${PORTS_CONF_DIR}, ...
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Testing a change to printf(9)

2011-06-07 Thread Dieter BSD
>> I've been working on fixing problems with printf(9), log(9) and
>> related functions. Today I tried converting printf(9) to write
>> to the log rather than directly to the console, unless the log is
>> not open, in which case the message is also sent to the console.
>> Printf(...) is now the same as log(LOG_INFO, ...).
> oh please no!
>
> from my perspective, I want my printf output to go to the console.
> immediately, regardless of where it goes after that.
> I don't care if there is or is not a log.
> I do NOT want to EVER have the problem I've had on linux where
> the last vital bit of output never made it out before the system stopped.
>
> once it's been shown on the console I don't care what happens to it..

Printfs to the console cause data loss. Easily repeatable.
Absolutely unacceptable.

You are welcome to fix the actual underlying problem. I would
love to see the underlying problem fixed! I've asked a few times
before, but I'll ask again. Why does a driver for one piece of
hardware have to block interrupts for all hardware? I thought
changing from spl to mutex was supposed to fix this?

My changes do not prevent a sysadmin from having printf output
go to the console, it gives them a choice. Currently there is
no choice.

>> I commented out the line in /etc/syslog.conf that sends
>> some log messages to the console. In multiuser mode,
>> normal printfs go to log, but not the console, as expected.
>>
>> Bootup messages, shutdown messages, and panic messages all
>> show up on the console as expected.
>>
>> Are there any other special cases to test?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Testing a change to printf(9)

2011-06-07 Thread Dieter BSD
I've been working on fixing problems with printf(9), log(9) and
related functions.  Today I tried converting printf(9) to write
to the log rather than directly to the console, unless the log is
not open, in which case the message is also sent to the console.
Printf(...) is now the same as log(LOG_INFO, ...).

I commented out the line in /etc/syslog.conf that sends
some log messages to the console.  In multiuser mode,
normal printfs go to log, but not the console, as expected.

Bootup messages, shutdown messages, and panic messages all
show up on the console as expected.

Are there any other special cases to test?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Active slice, only for a next boot

2011-05-31 Thread Dieter BSD
Peter writes:

>> A better approach is to be able to boot whatever slice you
>> want without having to change the active slice.
>>
>> NetBSD can do this. The MBR puts up a menu of the bootable
>> slices on the disk being booted. You can allow the timer
>> to time out and boot the default. Or you can enter the number
>> of the slice you want to boot. Or you can type a function key
>> F1 F2 ... to boot a different disk, and it will load the MBR
>> from that disk and run it. There is an alternative for keyboards
>> without function keys.

> So can FreeBSD - though only for MBR

If so, the documentation could use improvement.
Examples would be great.

> - this functionality doesn't seem
> to have made it into the GPT bootcode.

Is anyone working on this?  MBR is only good for 2 TB
and the 3 TB disks are becoming cost competitive.  I've
switched over to GPT for everything except boot disks,
as it is less brain damaged than MBR.

>> And it works great. Except that one of the 27 stages of boot
>> code that FreeBSD uses INSISTS on booting the active slice,
>> so you can tell the MBR to boot slice 3 and slice 3's boot
>> code sees that slice 4 is active and boots slice 4.
>
> Multibooting worked correctly when I last used it (a few years ago).
> Have you raised this as a PR?

No, partly because I haven't had much luck with PRs.  Mainly because
I'd rather spend my time trying to migrate to GPT than improving MBR.
So many bugs/misfeatures so little time,

>> RS-232 console + hardware modem + POTS = remote console
>
> And even that doesn't fully work unless you have a serial-aware BIOS.

Real computers have RS-232 consoles.

If you neglected to specify RS-232 console in the requirements,
there is this thing.  I haven't tried it.
http://www.realweasel.com/

Somebody probably sells a KVM-over-IP box.

You could see if openbios supports your mainboard.

John writes:

>> And it works great.  Except that one of the 27 stages of boot
>> code that FreeBSD uses INSISTS on booting the active slice,
>> so you can tell the MBR to boot slice 3 and slice 3's boot
>> code sees that slice 4 is active and boots slice 4.
>
> There are only 3 stages,

It feels like more.  :-)

> and boot1.S is what looks at the active slice.  
> Unfortunately it doesn't have a better way to do this as the only input it 
> gets from boot0 or any other MBR boot loader is the BIOS drive number in %dl.
> I'm not sure how else you would detect that a non-active slice was booted 
> from 
> when that is your only input.

The NetBSD boot code manages to do it.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: compiler warnings (was: Re: [rfc] a few kern.mk and bsd.sys.mk related changes)

2011-05-31 Thread Dieter BSD
>> please keep in mind that -Wfoo does reflect the ideas of the GNU people
>> regarding *proper* code. the warnings themselves are sometimes wrong,
>> because they complain about perfectly correct code.

I attempted to get the gcc people to improve this:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=9072

Most of the warnings I see are either due to someone thinking
all the world is ILP32 and doing things like storing a 64 bit
pointer or long in a 32 bit int, or due to the compiler needing
more info to insure that they are not trying to stuff 64 bits into
32, such as missing prototypes.  Either way it needs to be fixed.

In many cases the developers that claim to write such great code,
and claim that the compiler warnings don't matter are the ones
whose code has the most bugs (seg faults, floating point exceptions, ...).

> Pretty much the entire kernel is compiled
> with quite a large number of warning classes enabled, and -Werror set, for
> example.

Whoever did this, THANK YOU!!!

> fixing warnings in other people's code is useful only if
> you can get them to accept the fixes back

Fixing bugs is always useful.  Certainly it is a *lot* more
efficient if you can get them fixed at the source rather than
having to maintain patches.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: compiler warnings (was: Re: [rfc] a few kern.mk and bsd.sys.mk related changes)

2011-05-30 Thread Dieter BSD
Chris writes:
>> Ports need attention. The warnings I get there are frightening.
>
> I find it comforting that they're just that: warnings.
>
> How do they frighten you?

High quality code does not have any warnings.

The most frightening thing is the attitute that "They're just warnings,
so I'll ignore them."  Most compiler warnings should be fatal errors.
And a lot of the warnings that require a -Wwhatever should be on by
default.

Code that doesn't compile cleanly often has other problems, like assuming
that all CPUs are ILP32 little endian, like not checking return codes, etc.

But hey, the next time the weather service issues a tornado warning,
feel free to go outside and fly a kite.  After all it's just a warning.

a13x writes:
> -Wcoercion seems to have only been a SoC project in 2006 [1]. i checked gcc
> trunk and it's not in the gcc(1) manual.
>
> [1] http://gcc.gnu.org/wiki/Wcoercion

And yet someone marked the bug fixed.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=9072
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: compiler warnings (was: Re: [rfc] a few kern.mk and bsd.sys.mk related changes)

2011-05-30 Thread Dieter BSD
> maybe we find some nice -Wwarning options which are reasonable
> to have

-Wmissing-declarations
-Wimplicit

FreeBSD's gcc doesn't seem to have  -Wcoercion  ???
Bugzilla indicates that it was added years ago (2006?).

It would be really really nice if -static worked on (nearly) everything.

> and - most importantly - don't break tinderbox

That's a matter of fixing the bugs before adding the warning flags
to tinderbox.

Ports need attention.  The warnings I get there are frightening.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Active slice, only for a next boot

2011-05-30 Thread Dieter BSD
>> I have i.e; 3 slices, of which first is active.
>> Now I wana set slice 2 active, but only for a one/next boot.
>> Once slice 2 is booted and system is shutdown or rebooted,
>> once again, first slice is active and booted, without user's intervention.

I think that setting the active slice is the wrong approach.
Because then whatever OS you boot must then somehow know to
automagically change the active slice back to the default.

A better approach is to be able to boot whatever slice you
want without having to change the active slice.

NetBSD can do this.  The MBR puts up a menu of the bootable
slices on the disk being booted.  You can allow the timer
to time out and boot the default.  Or you can enter the number
of the slice you want to boot.  Or you can type a function key
F1 F2 ... to boot a different disk, and it will load the MBR
from that disk and run it.  There is an alternative for keyboards
without function keys.

And it works great.  Except that one of the 27 stages of boot
code that FreeBSD uses INSISTS on booting the active slice,
so you can tell the MBR to boot slice 3 and slice 3's boot
code sees that slice 4 is active and boots slice 4.

My ugly workaround is to have my backup FreeBSD slice on
a different disk.  The better solution would be to fix
FreeBSD's boot code so that it boots the slice it is in
rather than insisting on booting the active slice.  And
import the NetBSD boot selector MBR.

> Anyone has any idea, for a case of a remote server,
> which is accessible over ssh, only when it is "up"?

Remote access only when the system is up will bite you,
sooner or later.  The classic formula is:

RS-232 console + hardware modem + POTS = remote console
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: [RELEASE] New Boot-Loader Menu

2011-05-08 Thread Dieter BSD
> There's really only room for one or two more menu items.

Perhaps some items could be moved to a 2nd level menu?

1) boot multiuser mode ( default )
2) boot single user mode
3) menu to set boot options
4) help

 Would be nice: a fix for having to lean on a key autorepeating
 for a couple seconds.
>>>
>>> Could you explain? I don't follow.
>>
>> On my Tyan Tomcat k8e 2865, just entering the number rarely if
>> ever works. I have to either repeatedly bang away at the key or
>> hold it down, letting the RS-232 terminal do the autorepeat thing,
>> while hoping that it notices before the timer runs out.
>
> Is that with the current code that's in CVS? I'd love for you to
> try my code on that hardware. One of the things that I worked on
> in the very beginning was the responsiveness.

That's with 6.x, 7,x and 8.x, currently running 8.2.  I haven't
looked to see if anything has changed in CVS since 8.2.  I'll
try your code, but I can't promise when.  I'm swamped at the
moment and windows when the machine is free keep getting smaller
and less frequent.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: [RELEASE] New Boot-Loader Menu

2011-04-30 Thread Dieter BSD
> Already on the to-do list is to support ``loader_logo=...'' in
> /boot/loader.conf

Including an option for no logo? (For consoles that are slow and/or
small, and for people that just don't like the logos.)

>> Putting brackets around letters (and numbers) sounds good.
>> If there is room, perhaps add a message explaining that
>> the user should enter one of the choices in brackets.
>
> I think I'm going to have to play with this and see what we come up with. I
> don't want to make it too "busy" if you know what I mean. That's with
> respect to the brackets. As for adding a messages... things are a bit tight
> and again, I'm afraid of making it too cluttered. I'll post some screenshots
> of some mock-ups tomorrow, incorporating the various requests.
>
>> A help option would be useful, giving a reminder of what
>> things like ACPI and APIC stand for, what "safe" mode does, etc.
>
> This is not altogether infeasible. Since this menu (unlike the current one)
> has the ability to be wiped from screen and then recalled completely in the
> original state is was left in ... implementing an F1 feature that reads text
> from a file would be very do-able.

If there is a help option that the user can figure out how to execute,
the explaination about brackets (if you go that route), entering numbers,
letters, and such could be included in the help screen(s) instead of the
main menu page.

>> Would be nice: a fix for having to lean on a key autorepeating
>> for a couple seconds.
>
> Could you explain? I don't follow.

On my Tyan Tomcat k8e 2865, just entering the number rarely if
ever works. I have to either repeatedly bang away at the key or
hold it down, letting the RS-232 terminal do the autorepeat thing,
while hoping that it notices before the timer runs out.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: [RELEASE] New Boot-Loader Menu

2011-04-30 Thread Dieter BSD
[ attempt #2 - grumble - sorry about the blank message, hope it
works this time - grumble- ]

> I hope that works for serial console.  VT100 may be a reasonable
> default in that case, but it would be good to make sure that menu
> works even on a dumb terminal. Perhaps we should put 'key' letter
> in brackets then?

This needs to work, correctly, everywhere.  This needs to be easy to
understand by a stressed out user whose machine is having problems.

Therefore:

Thou shalt not assume graphics.
Thou shalt not assume color.
Thou shalt not assume VT100 or any specific terminal.
Thou shalt not assume ability to display bold.
Thou shalt not assume ability to underline text.
Thou shalt not assume availability of multiple fonts.
Thou shalt not assume more than 24x80 chars.
Thou shalt not assume scrollback.
Thou shalt not assume fancy cursor movements.
Thou shalt not assume presence of function keys.
Thou shalt not assume presence of arrow keys.
Thou shalt not assume a fast interface.
Thou shalt not assume the three-finger-salute works.

Putting brackets around letters (and numbers) sounds good.
If there is room, perhaps add a message explaining that
the user should enter one of the choices in brackets.

A help option would be useful, giving a reminder of what
things like ACPI and APIC stand for, what "safe" mode does, etc.

Would be nice: "uname -v" of the kernel it will boot.

Would be nice: a user friendly way to boot from a different
disk/partition/kernel.  Without the user having to know the
mapping between what the firmware calls disks and what FreeBSD
calls disks.  And without writing anything to disk.

Would be nice: a fix for having to lean on a key autorepeating
for a couple seconds.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: [RELEASE] New Boot-Loader Menu

2011-04-30 Thread Dieter BSD

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: CFT: final patches for NGROUPS>>16

2009-06-18 Thread ttw+bsd
posted patches to this effect some months ago.  they needed some
clean-up and validation but the also mapped correctly into an
RPC_NGROUPS_MAX and IPC_NGROUPS_MAX for consistent (and possibly
dynamic) mapping to those problem areas.  they build and run but are
untested beyond that.  i do not expect to have time to re-visit the
problem in reasonable order.  they also included some other userland
changes that may be of interest.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: does Copyright on source files expire ?

2009-03-26 Thread ttw+bsd
On 25.03-05:31, David Schultz wrote:
[ ... ]
> A person's Copyright doesn't go away just because they die,
> disappear, or fail to respond. If you can't contact them, their
> heirs, or whomever they transferred the Copyright to, you're stuck.

yeah but it's a little like finding something.  if there not about
and not reachable there isn't much they can do to stop you using it.
if they popup and make demands later then you get to choose between
re-writes and haggling (twenty shekels is standard).

point is you "can" use it, the actual copyright owner needs to sue
you; not like saying "jehovah" which may result in action by the agents
of the state.

n.b: using the above opinion may get you crucified.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Doing away with NGROUPS_MAX in src/sys/sys/syslimits.h?

2009-03-23 Thread ttw+bsd
On 22.03-22:33, Boris Kochergin wrote:
> Ahoy. I got bitten by this today--a system I administer for someone had 
> users in more than 16 groups, so I had to bump the value, recompile the 
> kernel, and reboot. It seems desirable to (at the very least) make this 
> a read-only tunable that can be set using /boot/loader.conf, so as to 
> avoid source modification and kernel recompilation. I had a look around, 
> and noticed that NGROUPS_MAX is used to construct static arrays in a 
> couple of locations ("ibcs2_gid_t iset[NGROUPS_MAX];"). It seems that 
> malloc(9)/MALLOC(9) can be used to allocate memory for the array 
> instead, and panic() (or something) can be called if the allocation 
> fails, no? Is that about the gist of it? If I'm not overlooking 
> something major, I'd like to take a stab at it.

i've sumbitted a patch for this to hackers@' list but actually
bumping the groups limit is more work.  i'm pretty far on with it
but am unsure wwhen it'll be completed.  if anyone wishes a copy of
the patches or current working patch then i'd be happy to post it.

note that bumping NGROUPS_MAX will do little in itself.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: Doing away with NGROUPS_MAX in src/sys/sys/syslimits.h?

2009-03-23 Thread ttw+bsd
On 23.03-00:39, David Schultz wrote:
[ ... ]
> There's already a kern.ngroups sysctl, but there are many places
> where `ngroups' needs to be used in preference to NGROUPS in the
> kernel. In userland, sysconf(_SC_NGROUPS_MAX) needs to be used in
> preference to NGROUPS_MAX.

you will also note that, as you look at this more, NGROUPS_MAX controls
very little regarding the relevant buffers and, generally, without
reviewing it again to be specific i'd suggest that you may expose a
number of buffer overruns but will most certainally not get the
'correct' behaviour from the change.  i.e. removing NGROUPS_MAX may
remove an error message from setgroups but will not increase the buffer
allocations or alter relevant code to check NGROUPS_MAX correctly.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: removal of NGROUPS_MAX dependancy from base

2009-02-22 Thread ttw+bsd
On 21.02-22:49, Julian Elischer wrote:
[ ... ]
> >this patch should remove the dependancy on the definition of
> >NGROUPS_MAX as a static constant and implement it as a writable
> >sysconf variable of the same.  it should also make the necessary
> >changes to the codebase to support those.
[ ... ]
> What do you do about NFS?
> I seem to remember that NFWS had a maximum number of groups supported..

NFS will be supported by mapping 16 groups into the auth_unix structure
dynamically.  my intention is to try and make this transparent by
allocating moving the 'most used' groups into that mapping as user
processes check them, however, this is very conceptual at the moment
and needs more thought as well as validation from others with more
experience.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


removal of NGROUPS_MAX dependancy from base

2009-02-13 Thread ttw+bsd
attached is the first in a series of patches that is intended to
remove the current limitation on group membership.

this patch should remove the dependancy on the definition of
NGROUPS_MAX as a static constant and implement it as a writable
sysconf variable of the same.  it should also make the necessary
changes to the codebase to support those.

i need some guidance as to what i should re-define NGROUPS_MAX to be
(so that code that depends on it can continue to operate, i'm thinking
just make it 16 but perhaps it would be worth extending the default
while we're at it to something like 64??).  i also need feedback on
any braindamage in the current changes.

the next step will be to extend the kernel groups and map them back
to the user structs / calls.   finally i'll extend the user groups
and implement those calls.

nb: not tested the code (it builds) ... was intending to test it on
my XEN box but only just realised that Xen on amd64 isn't working.
:-(

happy for any questions that may help guide the process.
Index: contrib/openpam/lib/openpam_borrow_cred.c
===
RCS file: 
/home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/contrib/openpam/lib/openpam_borrow_cred.c,v
retrieving revision 1.1.1.9
diff -b -u -r1.1.1.9 openpam_borrow_cred.c
--- contrib/openpam/lib/openpam_borrow_cred.c   21 Dec 2007 11:49:29 -  
1.1.1.9
+++ contrib/openpam/lib/openpam_borrow_cred.c   4 Feb 2009 16:38:46 -
@@ -60,6 +60,7 @@
struct pam_saved_cred *scred;
const void *scredp;
int r;
+   int ngroups ;
 
ENTERI(pwd->pw_uid);
r = pam_get_data(pamh, PAM_SAVED_CRED, &scredp);
@@ -73,26 +74,55 @@
(int)geteuid());
RETURNC(PAM_PERM_DENIED);
}
-   scred = calloc(1, sizeof *scred);
-   if (scred == NULL)
-   RETURNC(PAM_BUF_ERR);
-   scred->euid = geteuid();
-   scred->egid = getegid();
-   r = getgroups(NGROUPS_MAX, scred->groups);
-   if (r < 0) {
-   FREE(scred);
-   RETURNC(PAM_SYSTEM_ERR);
-   }
-   scred->ngroups = r;
+/* get the maximum number of system groups */
+#if _POSIX_VERSION > 199212
+   ngroups = sysconf( _SC_NGROUPS_MAX ) ;
+#elif defined(NGROUPS_MAX)
+   ngroups = NGROUPS_MAX ;
+#else
+   ngroups = _NGROUPS_COMPAT ;
+#endif
+/* initally allocate enough memory for max_groups */
+   scred = malloc( sizeof(struct pam_saved_cred) +
+   ngroups*sizeof(gid_t) ) ;
+   if( scred == NULL )
+   RETURNC( PAM_BUF_ERR ) ;
+/* set the save values */
+   scred->euid = geteuid() ;
+   scred->egid = getegid() ;
+/* save groups into our (probably) oversized memory allocation */
+   r = getgroups( ngroups, scred->groups ) ;
+   if( r < 0 ) {
+   FREE( scred ) ; /* call PAM's free macro */
+   RETURNC( PAM_SYSTEM_ERR ) ;
+   } ;
+   scred->ngroups = r ;
+   ngroups = r < ngroups ? r : ngroups ; /* choose the smallest */
+   /* ... number of groups to allocate */
+   ngroups = ngroups < _NGROUPS_COMPAT ? ngroups : _NGROUPS_COMPAT ;
+   /* but keep it within expected minimum value */
+   /*  XXX: we don't really want this but 
until we get
+*  educated on the implications this is 
probably safe
+*  and certainaly compatible */
+/* realloc, releasing unneeded memory */
+   scred = realloc( (void*)scred,
+   sizeof(struct pam_saved_cred)+ngroups*sizeof(gid_t) ) ;
+   /* nb: we ignore failure and try to store the 
larger
+* ... structure as initially requested. 
catching the
+* ... error in 'pam_set_data' if neccessary. */
+/* save the credentials to PAM user data area */
r = pam_set_data(pamh, PAM_SAVED_CRED, scred, &openpam_free_data);
if (r != PAM_SUCCESS) {
FREE(scred);
RETURNC(r);
}
+/* set the new credentials */
if (geteuid() == pwd->pw_uid)
RETURNC(PAM_SUCCESS);
if (initgroups(pwd->pw_name, pwd->pw_gid) < 0 ||
- setegid(pwd->pw_gid) < 0 || seteuid(pwd->pw_uid) < 0) {
+   setegid(pwd->pw_gid) < 0 || seteuid(pwd->pw_uid) < 0)
+   {
+   /* if any of the set calls failed, then restore and fail */
openpam_restore_cred(pamh);
RETURNC(PAM_SYSTEM_ERR);
}
Index: contrib/openpam/lib/openpam_impl.h
===
RCS file: 
/home/__orole/dev/cabinet/zeeNi/ai/freebsd/src/contrib/openpam/lib/openpam_impl.h,v
retrieving revision 1.1.1.17
diff -b -u -r1.1.1.17 openpam_impl.h
--- contrib/openpam/lib/openpam_impl.h  21 Dec 2007 11:49:29 -00

Apache & PV_ENTRY

2006-01-19 Thread BSD

Hello list,

I am configuring a very heavily used apache webserver, that required 
some special needs.


This particular configuration needs to have at least 1024 httpds always 
running. Reaching this number is not a problem, but whenever I stop 
apache via apachectl stop, I notice all the httpds take a long time (1 
min) to vanish - and meanwhile, top shows most of them locked.


The states I see in top are:
*vm ob
RUN

while the header shows:
1038 processes:332 running, 12 sleeping, 694 lock

The numbers vary a lot. Eventually, they get killed, but it takes too 
much time.


After some searching, I found out the cause for this was running out of 
PVs. I've added the following relevant options to the kernel:

options KVA_PAGES=768
options PMAP_SHPGPERPROC=5120

sysctl vm.zone | grep PV shows: (note: httpd stopped here)
(limit) (used)  (free)   
(requests)

PV ENTRY: 24, 84410445,   5522, 1632543,  5381679


So, if my assumptions are correct,  and running out of PVs was the cause 
(I did get that message about increasing PMAP_SHPGPERPROC before I 
placed that in the kernel), there is a problem now. If you look at the 
vm.zone output, there are 84,410,445 PVs that could exist, but the 
system shows only 1,632,543 free.


This leads me to believe there is another thing to tune. I already have 
a high PVs limit, but according to this output, it's not going further 
than 1,6 millions.


My question is, what can I do to make the system use the full 84 
millions, as I have not found any way to do this.






___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


6.0-STABLE & powerd: eating my keyboard input

2005-11-23 Thread BSD

Hi all,

I just upgraded from 6.0-RELEASE to -STABLE today (FreeBSD 6.0-STABLE 
#0: Thu Nov 24 00:40:48 WET 2005). After the upgrade I couldn't login 
anymore, so I thought I probably erased master.passwd by mistake with 
mergemaster, and rebooted into single user mode. Here I reset both root 
and my user account password, and processed to multiuser, only to fail 
the login again. Both on KDE and console. I rebooted again into single 
user, and changed both password to 3 characters. Now I could login, but 
every word I typed would have some letters wiped out (that was why the 
login was failing, it was "eating" my password).


A quick 'top' inspection showed powerd using ~30% CPU, which is 
abnormal. I killed it, and voila! No more letter-eating. I started it 
again, and back to the same.


powerd has always worked fine, it started behaving like this when I 
upgraded just an hour ago. This is a centrino laptop from acer. 
Comments, suggestions ?



___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


Kernel options

2004-07-28 Thread bsd hack
Hi,
I am working with the Kernel config file to optimize it and also to improve the 
overall security of the system!
 
I have the following quetions:
(1) There are a few options that are not available in the default kernel... like the 
IPFIREWALL options(and the like)... I basically need to know all possible options I 
can add to the kernel config file! 
(2) I guess these options can be used to set the kernel variables accessible through 
the sysctl command. So can I create my own options so that I can set a few kernel 
variables as and when I build the custom kernel?
(3) and also my aim includes optimizing the kernel... so by enabling only the options 
I need to I should get a get optimization... is there anything else that can be done?
(4) My aim is to improve local and network security. I guess enabling IPFIREWALL helps 
with the network security part are there any special options for local security?
 
Thank you.
 
HKR


-
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


Kernel GENERIC config file

2004-07-27 Thread bsd hack
 Hi,
I need the kernel GENERIC config file for freebsd 4.7.  I am able to find only the 
config file for freeBSD 5.2 online... can n'ybody either mail me the freeBSD 4.7 
GENERIC file or gimme a link to it?



Thank you.

HKR

 



-
Do you Yahoo!?
Y! Messenger - Communicate in real time. Download now.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


FreeBSD 5.2 and ACPI

2004-07-20 Thread bsd hack
Hi,
   I am building a custom kernel with FreeBSD 5.2... I need to disable ACPI (otherwise 
the system fails to recognize the disk!!)
 ... so is it fine if I just do a make and then a make install of the new kernel and 
then add hint.acpi.0.disabled to /boot/loader.conf  ? or is anything to be done before 
doin a make and a make install?
 
any help will be appreciated. 
 
Thank you.
HKR


-
Do you Yahoo!?
Vote for the stars of Yahoo!'s next ad campaign!
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


FreeBSD Custom Kernel

2004-07-16 Thread bsd hack
Hi All,
  I have a few basic questions regarding building a custom Kernel:
 
(1) Once I configure, make and make install the custom kernel... it will get written 
to /boot/kernel. Now I have already made a backup of the working kernel.  what I need 
to know is when I get the boot loader menu(when it counts from 1 to 10) is there any 
way in which I can Include another option for my Custom Kernel. I want the default to 
remain as the working kernel... and I want an option for my Custom kernel in the 
menu... 
 
(2) Secondly I dont understand the boot prompt and the loader prompt completely. at 
the boot prompt I guess I can specify the kernel I want to load. So lets say I have my 
custom kernel in the directory /boot/mydir/custom-kernel. so how do I specify at the 
boot prompt or the loader prompt to load the custom kernel
 
(3) And finally I had downloaded the source code of freeBSD and I configured, did a 
make and a make install without making any changes ...(Just to get it started!) ... 
when I rebooted the system it failed to recognize the disk and so I reinstalled 
FreeBSD again...( hadnt enabled any debug options so could not recover any 
information!!)... Is there anything that "HAS" to be done before installing the new 
kernel?
 
I'm working on FreeBSD 5.2...
 
any help will be appreciated.
 
Thank you
HKR



-
Do you Yahoo!?
Yahoo! Mail is new and improved - Check it out!
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


Re: Discussion on the future of floppies in 5.x and 6.x

2004-01-08 Thread BSD
On Thu, Jan 08, 2004 at 04:10:38PM -0600, Matthew D. Fuller wrote:
> On Thu, Jan 08, 2004 at 07:36:10AM -0800 I heard the voice of
> Avleen Vig, and lo! it spake thus:
> > 
> > If I understand you right..
> > A floppy boot, which loads the absolutely basic stuff (network drivers,
> > and some easy way to config the network) and then goes and grabs the
> > installer would otherwise be on the current floppies and "boots" it?
> 
> Many (most?) Linux dists do this for floppy installs.  I've come around
> to thinking it a better and better idea lately.  It makes it easy to have
> much more bloa... er, "featureful" installers, particularly more
> graphical ones, since you're no longer limited by the size of a floppy.
> And even cheap DSL is faster than a floppy drive for loading it, to boot
> (no pun ;).  And you can even provide for loading it off a local CD, if
> you have a CD drive you can't boot from.
> 
> The downside is that writing such a beast is a lot more work...

Take what I'm babbling about with a huge grain of salt, since I probably
have no idea what I'm talking about...

How hard would it be to take something like etherboot and building a
single install floppy from that? And have a FreeBSD kernel/installer
hosted on a public nfs/tftp server?

Ie., the user pops in the etherboot floppy, it asks for network setup
info, we tell it to use ftp.freebsd.org, and it goes out and either nfs
mounts or tftp the kernel and installer and drivers. Then proceed from
there to installing FreeBSD onto the system.

I suppose that's what you just said.. But was just thinking that it'd be
easier modifying an existing system (ie., etherboot) than writing one
from scratch..


Note: I just mentioned etherboot because that was the first thing that
came to mind. I'm sure there are other systems better suited or licensed
for what's needed.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


My jail can not ssh..

2003-09-15 Thread maillist bsd
Hi ,
 
I am just testing jail on my FreeBSD4.8-stable box, i found i can not ssh to the jail 
environment, but i can telnet to jail environment, the sshd is running both inside and 
outside jail.  What's the problem.
 
With following link is i ask in www.freebsdforums.org .
 
http://www.freebsdforums.org/forums/showthread.php?s=&threadid=13656
 
Regards,
kinux

最新鈴聲推介:十面埋伏,多謝失戀,心淡...
http://ringtone.yahoo.com.hk
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


about /usr/ports make with options..

2003-08-22 Thread maillist bsd

hi all,

as i install a lot package work together, such apache+php+mysql..
i always download the source code compile by myself, it just need to make with some 
options such as ..
make --prefix=/usr/local/apache --enable-module=so

But i found someone said install by /usr/ports also can do that, it also add options 
at make in /usr/ports, i want to know how can i check the options the packages have, 
how can i know it..

it seems like..
cd /usr/ports/www/apahe13
make With 

Can anyone explain it more..??
Any document for reference..??

Thanks.


³Ì·s¹aÁn±À¤¶:¤Q­±®I¥ñ¡A¦hÁÂ¥¢ÅÊ¡A¤ß²H...
http://ringtone.yahoo.com.hk
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


IP Network Multipathing failover on FreeBSD..??

2003-08-14 Thread maillist bsd
Hi all,
 
Is it there have IP Network Multipathing failover on FreeBSD..??  how to do so??
 
Thanks

³Ì·s¹aÁn±À¤¶:¤Q­±®I¥ñ¡A¦hÁÂ¥¢ÅÊ¡A¤ß²H...
http://ringtone.yahoo.com.hk
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


Re: System Crash - please help me figure it out...

2003-06-09 Thread bsd
[EMAIL PROTECTED] writes: 

Dear Hackers,  

I have experienced a system crash for the first time in 10 months (last 
year in august I had a vinum issue which has been resolved with your 
help).  

Fortunately I still have a kernel with debugging enabled running, and the 
gdbmods script, so I can submit some useful data with this email.  

On further reflection, here is more info which might be usefult to know: 

1) I recently updated to 4.8 RELEASE.  I just looked at the kernel config 
file and noted that I removed the "options VINUMDEBUG" a while ago.  This 
might explain frames 13 through 15 in the crash dump being ?? .  Help me out 
here please... 

2) When upgrading I noted that the source for vinum had been changed in a 
similar way to the patches I received from Greg Lehey and applied in August 
2002, so I did NOT apply the patches again.  Perhaps I should have? 

3) I also recently began using 'gre' devices for some VPN tunnels.  This is 
new in 4.8.  I find it odd (in hindsight) that these are working OK thought 
the "pseudo-device gre" line is absent from my kernel config file ??? 

FYI: kldstat is at http://docs.perimeter.co.za/crash/2003-06/kldstat 

Regards,
Patrick.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


  1   2   >