Re: patch(1) depends on RCS - should it?

2013-10-07 Thread Poul-Henning Kamp
In message caf6rxgni6kw6qtlmwwqdc2suqp+wa5-ptqwgsbtpa1-x_vz...@mail.gmail.com
, Eitan Adler writes:

patch(1) explicitly tries to use RCS (and SCCS) in certain cases.  Are
we okay with a base system utility that behaves differently depending
on whether a port is installed? Should the relevant code be removed
from patch(1)?

See head/usr.bin/patch/inp.c lines 166 to 240 for details.

Yes, that code should be removed.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Importing tradcpp (traditional (KR-style) C macro preprocessor) into base?

2013-06-12 Thread Poul-Henning Kamp
In message 20130611221124.gc84...@ithaqua.etoilebsd.net, Baptiste Daroussin w
rites:

I have been working in importing tradcpp (developped by David A. Holland from
NetBSD) into the ports tree, it is a traditional (KR-style) C macro
preprocessor BSD licensed. I first worked on it so that imake can work properly
without gcc.

As a user of certain antique X11 apps, I applaud this effort.

I discovered that some part of the base system still needs a traditional
preprocessor, like (calendar), what I propose it to import tradcpp into the 
base
system (not the version in port right now but what will become version 0.2).

However, I think these programs should be fixed, rather than put tradcpp
in the src tree.


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Adding a FOREACH_CONTINUE() variant to queue(3)

2013-05-01 Thread Poul-Henning Kamp
In message 518092bf.9070...@freebsd.org, Lawrence Stewart writes:
[reposting from freebsd-arch@ - was probably the wrong list]

#define TAILQ_FOREACH_CONTINUE(var, head, field)   \

Obligatory bikeshedding:

I find the suffix _CONTINUE non-obvious, as there may not have
been any previos FOREACH involved.

TAILQ_FOREACH_FROM(...) ?


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: devstat overhead VS precision

2013-04-16 Thread Poul-Henning Kamp
In message 516c71bc.4000...@freebsd.org, Alexander Motin writes:
On 15.04.2013 23:43, Poul-Henning Kamp wrote:
 In message 516c515a.9090...@freebsd.org, Alexander Motin writes:


 For tuning anything on a non-ridiculous SSD device or modern
 harddisks, it will be useless because of the bias you introduce is
 *not* one which averages out over many operations.

Could you please explain why?

 The fundamental problem is that on a busy system, getbinuptime()
 does not get called at random times, it will be heavily affected
 by the I/O traffic, because of the interrupts, the bus-traffic
 itself, the cache-effects of I/O transfers and the context-switches
 by the processes causing the I/O.

I'm sorry, but I am not sure I understand above paragraphs.

That was the exact explanation you asked for, and I'm not sure I can
find a better way to explain it, but I'll try:

Your assumption that the error will cancel out, implicitly assumes
that the timestamp returned from getbinuptime() is updated at
times which are totally independent from the I/O traffic you are
trying to measure the latency of.

That is not the case.  The interrupt which updates getbinuptime()'s
cached timestamp is affected a lot by the I/O traffic, for the various
reasons I mention above.

Sure, getbinuptime() won't allow to answer how many requests completed 
within 0.5ms, but present API doesn't allow to calculate that any way, 
providing only total/average times. And why _5-10_ timecounter interrupts?

A: Yes it actually does, a userland application running on a dedicated
CPU core can poll the shared memory devstat structure at a very high
rate and get very useful information about short latencies.

Most people don't do that, becuase they don't care about the difference
between 0.5 and 0.45 milliseconds.

B: To get the systematic bias down to 10-20% of the measured interval.

  Latency distribution:

  5msec: 92.12 %
  10msec: 0.17 %
  20msec: 1.34 %
  50msec: 6.37 %
  50msec: 0.00 %

I agree that such functionality could be interesting. The only worry is 
which buckets should be there. For modern HDDs above buckets could be 
fine. For high-end SSD it may go about microseconds then milliseconds. I 
have doubt that 5 buckets will be universal enough, unless separated by 
factor of 5-10.

Remember what people use this for:  Answering the question Does my
disk subsystem suck, and if so, how much

Buckets like the ones proposed will tell you that.

 The %busy crap should be killed, all it does is confuse people.

I agree that it heavily lies, especially for cached writes, but at least 
it allows to make some very basic estimates. 

For rotating disks:  It always lies.

For SSD: It almost always lies.

Kill it.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: devstat overhead VS precision

2013-04-15 Thread Poul-Henning Kamp
In message 516c515a.9090...@freebsd.org, Alexander Motin writes:

 I propose to switch that
 statistics from using binuptime() to getbinuptime() to solve the problem
 globally.

 No objections here, but I wonder if you were able to compare the results
 somehow before and after the change so we have some hard numbers to show
 that we don't lose much by applying the change.

I haven't tested it statistically, but I haven't noticed any visual 
difference in gstat output with its 0.1ms displayed resolution.

I have tested it statistically, back when I wrote GEOM:  It leads
to very significant statistical bias.

Just about the only thing in devstat that has any predictive power
with respect to filesystem performance, is the latency, which measures
how long time it takes to satisfy each I/O request.

If you run gstat(8), this is the ms/* numbers:  milliseconds per
this or that.

The rest of what's in devstat, with the exception of the queue-length
(L(q)) has almost no predictive power, and is IMO, practically
pointless.  In particular the %busy is totally misleading and I
deeply regret that I didn't fight to kill it back then.

If you switch to getbinuptime(), the latency measurements will only
be precise if the I/O operations take much longer than the timecounter
update period, which is not guaranteed to be 1000 Hz btw.

For measuring how much USB-sticks suck, that will work fine.

For tuning anything on a non-ridiculous SSD device or modern
harddisks, it will be useless because of the bias you introduce is
*not* one which averages out over many operations.

The fundamental problem is that on a busy system, getbinuptime()
does not get called at random times, it will be heavily affected
by the I/O traffic, because of the interrupts, the bus-traffic
itself, the cache-effects of I/O transfers and the context-switches
by the processes causing the I/O.

So yes, you can switch to getbinuptime(), but the only statistical
responsible way to do so, would be to supress latency measurements
on all I/O operations which complete in less than 5-10 timecounter
interrupts.

Apart from some practical issues implementing it, the numbers
that came out would be pretty useless.

The right idea is probably to bucketize the latencies, so that
rather than having to keep track of devstat in real time to find
out, you could get a histogram at any time showing past
performance something like:

Latency distribution:

5msec: 92.12 %
10msec: 0.17 %
20msec: 1.34 %
50msec: 6.37 %
50msec: 0.00 %

Doing that with getbinuptime() would be statistically defensible
provided the top bucket is 5msec and it would very clearly tell
people if they have I/O trouble or not, which IMO is what people
want to know.

The cost 20 64bit counters in struct devstat (N|R|W|E)*5*8 = 160
bytes, but since devstat is already 288 bytes, that isn't a major
catastropy.

The ability to measure latency precisly should be retained, but it
could be made a sysctl enabled debugging facility.

The %busy crap should be killed, all it does is confuse people.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [patch] md(4) and preloaded memory disks

2012-11-06 Thread Poul-Henning Kamp

In message 20121106182752.gb2...@a91-153-116-96.elisa-laajakaista.fi, Jaakko 
Heinonen writes:

I plan to commit the patch below to disallow attaching preloaded memory
disks via ioctl. I didn't find anything that would really use this
undocumented feature.

If I remember right, this was for the case where you had preloaded
multiple images, and the kernel would only auto-discover the first
one.

All things (such as the disapperance of floppy disks),  considered
this feature should be removed.


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: please review: patch to retain device name for dumpdev.

2012-11-01 Thread Poul-Henning Kamp

In message 50921b44.20...@ixsystems.com, Alfred Perlstein writes:

Poul-Henning, what do you think?  Is there a nicer way?  Perhaps a way 
to include the /dev/$device

I think there are private implemenations where dumpdev is a network thing,
so too much magic string editing is probably not a good idea.

Given that /dev is really just a view into GEOMs namespace, one could
argue for GEOM:ada0p3 that that may be going overboard in sematic
correctness.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program

2012-09-24 Thread Poul-Henning Kamp
In message cagh67wqty4krgwspa5jaht-4hqd4veykley-r3c6k9f5xaf...@mail.gmail.com
, Garrett Cooper writes:

No difference proven at 95.0% confidence

This is the important bit of information...

Thanks for the tip :)!

You're welcome :-)

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Providing a default graphical environment on FreeBSD

2012-09-18 Thread Poul-Henning Kamp
In message alpine.bsf.2.00.1209181004510.44...@wojtek.tensor.gdynia.pl, Wojci
ech Puchar writes:

That's all true. But do anyone understand why there is still so much 
pressure for every open source OS and specifically *BSD on default 
desktop environment or similar ideas?

Tell you what:

When you have at least 75% of the user population of FreeBSD agreeing
on which window manager we should offer as the default, we can talk
about this.

Until such a consensus exists, this discussion is just a waste of time.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Providing a default graphical environment on FreeBSD

2012-09-18 Thread Poul-Henning Kamp
In message alpine.bsf.2.00.1209181039140.44...@wojtek.tensor.gdynia.pl, Wojci
ech Puchar writes:
 desktop environment or similar ideas?

 Tell you what:

 When you have at least 75% of the user population of FreeBSD agreeing
 on which window manager we should offer as the default, we can talk
 about this.

so if 76% would decide that FreeBSD should have KDE included in system - 
it means that it should?

Just to clarify: when I write offer by default I do not mean cram
down peoples throat.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Providing a default graphical environment on FreeBSD

2012-09-17 Thread Poul-Henning Kamp
In message blu0-smtp510b16745b704c714268e2d5...@phx.gbl, Lorenzo Cogotti writ
es:
Hi,
I was wondering about the possibility of FreeBSD to provide an official
supported graphical environment.

We already do:  It's called X11 :-)

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Providing a default graphical environment on FreeBSD

2012-09-17 Thread Poul-Henning Kamp
In message CAGsORuAnDs_E=l747+tp95nxjxdonnsqfvfco+xd2hjsj-u...@mail.gmail.com
, Zhihao Yuan writes:
On Mon, Sep 17, 2012 at 10:42 AM, Poul-Henning Kamp p...@phk.freebsd.dk 
wrote:
 In message blu0-smtp510b16745b704c714268e2d5...@phx.gbl, Lorenzo Cogotti 
 writ
 es:
Hi,
I was wondering about the possibility of FreeBSD to provide an official
supported graphical environment.

 We already do:  It's called X11 :-)

How about Wikipedia graphical environment before u say this?

How about you try to install ports/x11-vm/twm, turn your CPU
speed down to 20 MHz and get a good feel for how a graphical
environment felt 25 years ago, before you make a fool of yourself ?

:-)

There is no way that FreeBSD is going to annoint a canonical
window manager (look that up too!), we've been down that road
before and the landscape is ugly and filled with bikesheds.

My suggest was 100% serious:  Assume X11 _is_ the graphical
environment, pick a toolkit which is written to work with
any window manager, which all good toolkits are, and move on.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Providing a default graphical environment on FreeBSD

2012-09-17 Thread Poul-Henning Kamp
In message cagsorub4yd8rknlrwmctx16idohwjkd1rnyarb98nwn+pwv...@mail.gmail.com
, Zhihao Yuan writes:
On Mon, Sep 17, 2012 at 11:20 AM, Poul-Henning Kamp p...@freebsd.org wrote:

 My suggest was 100% serious:  Assume X11 _is_ the graphical
 environment, pick a toolkit which is written to work with
 any window manager, which all good toolkits are, and move on.

You can assume, but you can't deny that X11 is not GUI at all, and
twm is not a modern GUI either.

You are confusing window manager and graphical user interface,
one is layered on the other, your homework is to figure out which.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Providing a default graphical environment on FreeBSD

2012-09-17 Thread Poul-Henning Kamp
In message CAGsORuBqiodwt_EmVqB+fO=tgOVeZOERopSE2y=mla8jp6z...@mail.gmail.com
, Zhihao Yuan writes:

Well, let's make it more straightforward. I hope people can agree with
this: a default, officially supported modern desktop environment is
essential to FreeBSD.

No, it is not.

It would certainly be nice to have as an option, but I would hate
to have to deal with it, when I squeeze FreeBSD into embedded systems
which have neither graphics outputs nor keyboard or mouse inputs.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [maybe spam] Re: FreeBSD 1.x Binaries Work Except under Chroot

2012-08-16 Thread Poul-Henning Kamp
In message 20120816081336.gb27...@e-new.0x20.net, Lars Engels writes:

 386BSD was even better, and I have a machine that boots it in less
 than 15 seconds from power-on...

Me too, it's running Linux. ;-)

You should upgrade the OS then... :-)

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [maybe spam] Re: FreeBSD 1.x Binaries Work Except under Chroot

2012-08-15 Thread Poul-Henning Kamp
In message CAJ-Vmo=yzcz_jdhtvq3kyhgxdnutvzqzpp+sdsz+zwe7cym...@mail.gmail.com
, Adrian Chadd writes:
Holy. Crap. 17 seconds?

Can we please go back to having it take this long? please?

386BSD was even better, and I have a machine that boots it in less
than 15 seconds from power-on...

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: FreeBSD 1.x Binaries Work Except under Chroot

2012-08-11 Thread Poul-Henning Kamp
In message 201208102249.q7amn8gf066...@fire.js.berklix.net, Julian H. Stacey
 writes:

I dont see 1.1.5:

It is not in our VCS because of the USL-BSD lawsuit.

You can find the bits here: http://phk.freebsd.dk/FreeBSD/

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: CVE-2012-0217 Intel's sysret Kernel Privilege Escalation and FreeBSD 6.2/6.3

2012-07-13 Thread Poul-Henning Kamp
In message 201207130831.59211@freebsd.org, John Baldwin writes:

Every FreeBSD/amd64 kernel in existent is vulnerable.  In truth, my personal 
opinion is that Intel screwed up their implementation of that instruction 
whereas AMD got it right, and we are merely working around Intel's CPU bug. :(

Given that the instruction set of AMD64 is defined by AMD originally,
while Intel was trying very hard to ram Itanic down everybodys
throat, that diagnosis is a given:  Intel copied AMD, and difference
in functionality is a screwup on Intels part, even if they documented
their screwup in their manual.

TL;DR: Which part of compatible doesn't Intel get ?

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [CFC/CFT] large changes in the loader(8) code

2012-06-27 Thread Poul-Henning Kamp

I would like to point out that all other operating system which has
had this precise problem, have solved it by adding a bootfs partition
to hold the kernel+modules required to truly understand the disk-layout ?

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: FreeBSD Boot Times

2012-06-13 Thread Poul-Henning Kamp
In message alpine.bsf.2.00.1206130909310.73...@wojtek.tensor.gdynia.pl, Wojci
ech Puchar writes:

One of the major slowdowns is that we do all the device drivers
serially  synchronously.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: reverse USB driver - is it possible?

2012-06-03 Thread Poul-Henning Kamp
In message alpine.bsf.2.00.1206031022360.55...@wojtek.tensor.gdynia.pl, Wojci
ech Puchar writes:

is it the same possible with USB?
i mean if i can make my laptop to simulate say USB CDROM.

No, the hardware is not up to it.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [Hash function Ipv4]

2012-06-02 Thread Poul-Henning Kamp
In message dub116-w41324220eb6a5f7fd8b06a97...@phx.gbl, enrico d'urso writes:


I'm looking for an Hash function for Ipv4 addresses.

What are good ones?

They are generally very hard to hash well, for all sorts of reasons
related to how we use them.

One way that used to work reasonably well for me:

uint32_t ipv4, h;

h = ipv4 % HASH;
ipv4 /= HASH;
h ^= ipv4 % HASH;
h ^= ipv4 / HASH;

Where HASH was a prime number near to 2^11

However, I cannot rule out that the good results I saw was a result
if RIPE's allocation policy at the time.


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: PID/UID namespaces

2012-05-21 Thread Poul-Henning Kamp
In message cadlo838vov_xi+a_wjd3h7e_d4qi+odjynhpoim5bbzawnx...@mail.gmail.com
, Chris Rees writes:

It would certainly prevent many common problems when setting up jails;
UID collision is much more common than you'd think, given that the
default UIDs remain the same.

Uhm... jails have separate UID/GID spaces.

Filesystems mounted or visible in multiple jails act as shared UID/GID
(sub-)spaces for those jails, but there is now way to avoid that, it's
a direct consequence of the sharing of the filesystems.


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


A dual-ISP hack with jail/vnet and ipfw

2012-02-04 Thread Poul-Henning Kamp
 127.0.0.1
echo
echo instance vr2
echo port 2
echo alias_address $VR2_IP
echo redirect_port tcp ${EPAIR_IN}:22 ${VR2_IP}:22
echo 
echo instance vr3
echo port 3
echo alias_address $VR3_IP
echo redirect_port tcp ${EPAIR_IN}:22 ${VR3_IP}:22
)  $jdir/etc/natd_ext.conf

jexec ext natd -f /etc/natd_ext.conf

# Remove the roadblock
$F delete 1

# Remove the evidence
# XXX: Even safer: put jail in md(4) disk, rm, remount r/o
rm -rf $jdir/*


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: mmap implementation for cdev

2011-12-03 Thread Poul-Henning Kamp
In message a51f9b30-a71a-49a5-b684-16ec3c127...@gmail.com, Filippo Sironi wri
tes:

The problem is that when I come =
back to user space all the threads write in memory page mapped by the =
first thread that called the mmap leaving their memory pages untouched.

That's how cdev-mmap works, it is global rather than per-thread.


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: mmap implementation for cdev

2011-12-03 Thread Poul-Henning Kamp
In message a842c229-1846-408a-999a-9347770ef...@gmail.com, Filippo Sironi wri
tes:

Ok, that's what I was suspecting.
Do you have any ideas on how to get the behavior I described in the =
previous mail?

Why don't you just use mmap(2) ?  I couldn't see anything you
couldn't do with it.

There's also support in pthread for thread specific storage, which
should be your first choice.


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: mmap implementation for cdev

2011-12-03 Thread Poul-Henning Kamp
In message 0011fd6a-e29d-4f67-913c-897ba1b2f...@gmail.com, Filippo Sironi wri
tes:
I need to access the memory both from user space and kernel space, I =
cannot do that (simply) with an mmap or thread-specific storage if I =
recall correctly.

Have you looked how shm_open(2) and friends are implemented ?  That
might allow you to.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: 9-beta1 installer - partition editor

2011-09-13 Thread Poul-Henning Kamp
In message 4e6f26d1.GZdzm/zhxjjqfow1%per...@pluto.rain.com, per...@pluto.rain
.com writes:
Freddie Cash fjwc...@gmail.com wrote:

 Unix partitioning has always been this way:
   - create partition on disk for OS
   - create sub-partitions for filesystems

No, it has not.

In fact, it is only on PC like hardware that you can reliably share
a disk between different mutually competitive operating systems.

Most unix-machines don't have a concept of what you call partitions,
and neither did BSD unix until 386BSD introduced it.

Until then: One OS, one disk(-pack|-drive).

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: A style proposal for referring to upper-level directories in Makefiles

2011-07-29 Thread Poul-Henning Kamp
In message 4e31aed9.4000...@aldan.algebra.com, Mikhail T. writes:
The most common method to refer to the upper directory in Makefile is as 
${.CURDIR}/..

I'd like to propose we begin using ${.CURDIR:H} instead. 

This will make it even harder for people who try to compile our
bits on alien systems without bmake.

I am not sure if that is a concern we should care about.


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: int64_t and printf

2011-06-05 Thread Poul-Henning Kamp
In message 4debc741.1020...@links.org, Ben Laurie writes:

So, for example int64_t has no printf modifier I am aware of. Likewise
its many friends.

but I have no idea where to put such a thing in FreeBSD. Opinions? 

I have totally given up on this mess.

At best you get incredibly messy source code, at worst you waste a
lot of time figuring out why who to define stuff to make some platform
you have only heard rumours about behave.

I have therefore resorted to printf'ing any typedefed integer type using
%jd and an explicit cast to (intmax_t):

printf(%-30s - %jd - %s\n, s, (intmax_t)t, buf);

If some system introduces int512_t that may not be optimal, but
since printf is a pretty slow operation anyway, I doubt it will
hurt even half as much as the alternative.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: int64_t and printf

2011-06-05 Thread Poul-Henning Kamp
In message 4debe469.5060...@links.org, Ben Laurie writes:
On 05/06/2011 19:21, Poul-Henning Kamp wrote:
 In message 4debc741.1020...@links.org, Ben Laurie writes:

 I have therefore resorted to printf'ing any typedefed integer type using
 %jd and an explicit cast to (intmax_t):
 
  printf(%-30s - %jd - %s\n, s, (intmax_t)t, buf);

My objection to this approach is the lack of type-safety - t could be
anything and this would continue to work.

Using PRId64 at least ensures that t is of an appropriate type.

Uhm, no, it doesn't.

At best it allows the compiler to make well chosen assumptions
about what the printf(3) function does.

Printf-format warnings are usually lost as soon as you go through
stdarg/v*printf, because people don't know they should add
__printflike() and other nonportable gunk to their prototypes.

And they are totally lost if you use extended printf formatting
of any kind, because there is no way to tell the compiler that
%Y takes a void *

This is basically why the work I did in printf.h is practically
useless.

But worse: PRId64 only works if you know your variable actually is 64bit.

If you are trying to write code which works with typedefs on both
32 and 64 bit platforms you cannot know this.

It's all nice and dandy that they made a magic z letter for size_t,
but what about uid_t, gid_t, pid_t, off_t, ino_t, mode_t, nlink_t,
fflags_t and so on ?

You will therefore be forced to cast your argument to (int64_t)
before you can use PRId64 safely on it.

Now you have messed up the format string without loosing the cast
and now your code will DTWT once somebody typedefs pid_t to int71_t.

Providing a better printf seems like an even smarter idea, e.g.

   printf(%-30s - %I64d - %s\n, s, t, buf);

Same problem as above.

There is no way to do this sanely, without involving the compiler.

At the very least, the compiler would need to mangle the format
string, so that you write:

printf(%-30?d\n, sometype)

and the compiler replaces the '?' with whatever is suitable
for the width of the argument.

Alternatively, and more useful, would be a type-safe or at least
type-aware stdarg, so that prinf(3) could ask about the width
and type of the next argument.

Both would be wonderful additions to ISO-C but you can produce a
college fresh-man from scratch starting now, before that happens.

(See also Bjarnes approx. 1985 discussion of why C++ overloads 
instead of providing printf(3)).

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: int64_t and printf

2011-06-05 Thread Poul-Henning Kamp
In message 4debf0e7.3040...@links.org, Ben Laurie writes:

I note that you didn't react to my other wherein you cast from known
type A to known type B. I supposed it would be smart to also assert that
the cast was non-narrowing.

Well, if casting to intmax_t is narrowing I think I have bigger
problems on my hands :-)

I've spent a fair amount of time agonizing over this in Varnish and
I came to the conclusion that the my time spent trying to establish
if something narrower than intmax_t was safe would never amortize
the performance difference of printing an intmax_t vs. intN_t, so
now I just cast anything that that's typedef'ed to intmax_t and
move on.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Galep5 chip programmer works under FreeBSD

2011-06-01 Thread Poul-Henning Kamp

I recently needed a new chip programmer for various hardware work
and since I know this is a recurring issue for people, I thought I
would share this information.

After researching the market, I decided to get the Galep5 from 
the German company Conitec.net   Price EUR 417 + sales tax

It's a nifty little device with an embedded ARM9 processor which
talks ethernet over USB which neatly solves the Damn, now we
also have to write a USB device driver issue for Conitec.  Good
thinking there.

Getting the Linux GUI application was pretty trivial:

http://phk.freebsd.dk/Galep5.html

There are various hooks into this product which allows it to be
controlled by programs, I have not used those (yet?)

Recommended,

Poul-Henning

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [patch] extending alloc_unr(9) to allocate specific unit numbers

2010-06-21 Thread Poul-Henning Kamp
In message 201006211109.11653@freebsd.org, John Baldwin writes:
On Saturday 19 June 2010 11:48:22 am Jaakko Heinonen wrote:

 As an example here is md(4) converted to use
 alloc_unr() / alloc_unr_specific():
 
  http://people.freebsd.org/~jh/patches/md-alloc_unr.diff

This sounds useful to me.  Perhaps ask p...@?

My only worry is that if people start to use this indiscriminantly to
store random collections of numbers, then it is far from the optimal
data structure for it.

Other than that: go for it.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Improving the kernel/i386 timecounter performance (GSoC proposal)

2009-03-30 Thread Poul-Henning Kamp
In message 20090329180745.gb38...@server.vk2pj.dyndns.org, Peter Jeremy write
s:

I'm assuming folks are still in love with the TSC because it still the
cheapest as oppose ACPI-fast or HPET to even contemplate this?

That is its major advantage.  It might be feasible to export all the
data necessary to implement the complete CLOCK_*_FAST family.

The general attraction is that it can be read from userland by unpriviledged
programs.

On systems where the ACPI or HPET hardware can be memory-mapped, I should
be equally possible to map those read-only into userland processes.

Now _THAT_ would be interesting.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Improving the kernel/i386 timecounter performance (GSoC proposal)

2009-03-27 Thread Poul-Henning Kamp
In message 17560ccf0903270555oe7d1652p7414a221aa2d6...@mail.gmail.com, 
Prashant Vaibhav writes:

[...] these must provide a monotonic timescale when queried interleaved
 ? Be aware that the TSC may not be, and may not stay synchronized across
 multiple cores.

The TSC is documented to be monotonically increasing [...]

Notice the absence of the word regular ? 

That it is monotonically increasing just means that it does not
count backwards (except on the buggy cpu-revs where it does).  It
does not mean that it counts upwards at a stable or constant rate.

Further more, the TSC is not constant frequency and in particular not
 known frequency at all times.

The TSC is guaranteed to be constant frequency on relatively modern
processors from Intel and AMD [...]

Which is why you will neeed a {CPU+MOBO+BIOS} table of known good
combinations: the majority of systems out there does not guarantee
and some of those that do lie.  Or have bugs.  Or both.

There are a lot of nasty cases to check,

They're not very involved in my opinion.

Then you likely have not done enough :-)

and a nasty interpolation required,

Could you please elaborate or hint me on some terms I can google about the
interpolations that are required? Are you referring to the interpolation
needed during measuring the tsc frequency to account for the (weird)
duration of PIT? This happens during bootup only.

I'm talking about the systems where SMM bios operations cause the different
CPU's TSC to develop skew over time.

which, in my tests some years back, totally negated any speedup from using
 the TSC in the first place.

This could be an issue: I have not made extensive benchmarks. The benefit of
using TSC could still be: the availability of a higher resolution timer
which can be accessed from userspace.

We have the same resolution today, if you dare to enable TSC in the kernel.

In fact, we have even better resolution, because the struct bintime format
is much more precise than both timespec and timeval.  So far I doubt anybody
but me have tried to measure that this makes a difference :-)

At the very minimum, you will have to add a quirk table where known good
 {CPU+MOBO+BIOS} combinations can be entered, as we find them.

Perhaps.
Or alternatively, a quirk table for known *bad* combinations.

No, FreeBSD is shipped working by default, not possibly working by
default and particularly not in an area, where the signs of trouble are
so subtle that most people don't recognize them at all and just blame
it on random buggy crap.

Rubbish.  Timecounters are not even closely associated with the tick or

My understanding could be flawed here, but the reasoning was: for a tickles
kernel, we need some sort of monotonically increasing, known-rate counter as
a replacement for periodic timer interrupts.

We already have that in FreeBSD for CPU time accounting.

The crucial fact about a tickless kernel, is that it does not take an
interrupt N times a second just to see if there is anything to do in the
callout queue, but instead uses the hardware timer to aim an interrupt
at the next time it needs to wake up.

the bios may autonomously change the cpu speed

True. This could be an issue.

Your optimism is cute but misguided.

On most laptops the bios WILL change the CPU speed without notice
in reaction to temperature and battery power.

Let me repeat:

 [1] In my mind, reworking the callout system in the kernel would
 be a much better more neded and much more worthwhile project.

Poul-Henning

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Improving the kernel/i386 timecounter performance (GSoC proposal)

2009-03-27 Thread Poul-Henning Kamp
In message 49cd0405.1060...@samsco.org, Scott Long writes:

I've been talking about this for years.  All I need is help with the VM 
magic to create the page on fork.  I also want two pages, one global
for gettimeofday (and any other global data we can think of) and one
per-process for static data like getpid/getgid.

Agreed, that is a good place to start.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Improving the kernel/i386 timecounter performance (GSoC proposal)

2009-03-27 Thread Poul-Henning Kamp
In message alpine.bsf.2.00.0903272254460.12...@fledge.watson.org, Robert Wats
on writes:

I guess interesting questions are whether (a) it would be desirable to have 
per-page, per-cpu, or per-thread mappings.  If there are non-synchronized 
TSCs, then there might be some interesting advantages to a per-CPU page.

Rule #3:
The only thing worse than generalizing from one example is
generalizing from no examples at all.

We can add those mappings when we know why we would want them.


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Improving the kernel/i386 timecounter performance (GSoC proposal)

2009-03-27 Thread Poul-Henning Kamp
In message alpine.bsf.2.00.0903272303040.12...@fledge.watson.org, Robert Wats
on writes:

In which case user application threads will need to 
know their CPU [...]

Didn't jemalloc solve that problem once already ?

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Improving the kernel/i386 timecounter performance (GSoC proposal)

2009-03-26 Thread Poul-Henning Kamp
In message 17560ccf0903260551v1f5cba9eu87727c0bae7b...@mail.gmail.com, Prasha
nt Vaibhav writes:

The gettimeofday() function's implementation will then be
changed to read the timestamp counter (TSC) from the processor, and use the
reading in conjunction with the timing info exported by the kernel to
calculate and return the time info in proper format.

I take it as read, that you know that there are other relvant
functions than gettimeofday() and that these must provide a
monotonic timescale when queried interleaved ?

Be aware that the TSC may not be, and may not stay synchronized
across multiple cores.

Further more, the TSC is not constant frequency and in particular
not known frequency at all times.

There are a lot of nasty cases to check, and a nasty interpolation
required, which, in my tests some years back, totally negated any
speedup from using the TSC in the first place.

At the very minimum, you will have to add a quirk table where
known good {CPU+MOBO+BIOS} combinations can be entered, as we
find them.

This will also pave way for optionally making the
FreeBSD kernel tickless,

Rubbish.  Timecounters are not even closely associated with the
tick or ticklessness of the kernel. [1]

   - The TSC frequency might change on certain processors with non-constant
   TSC rate (because of SpeedStep, dynamic freq scaling etc.). The only way to
   combat this is that the kernel be notified every time the processor
   frequency changes. Every cpu frequency driver will need to be updated to
   notify the kernel before and after a cpu freq change. 

That is not good enough, the bios may autonomously change the cpu speed
and the skew from not knowing exactly _when_ and _how_ the cpu clock
changed, is a significant number of microseconds, plenty of time
to make strange things happen.

You will want to study carefully Dave Mills work to tame the alpha
chips wandering SAW clocks.

Poul-Henning

[1] In my mind, reworking the callout system in the kernel would
be a much better more neded and much more worthwhile project.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: write-only variables in src/sys/ - possible bugs

2009-02-02 Thread Poul-Henning Kamp
In message 49874ca8.5090...@gmx.de, Christoph Mallon writes:

I compiled a list of all local variables in src/sys/ (r188000), which 
are only written to, but never read.

Bravo!

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: atacontrol broken in 7.1-PR

2008-09-29 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Andrey V. Elsukov writes:

This is known problem and it fixed in CURRENT.
You need to apply this patch:
http://www.freebsd.org/cgi/cvsweb.cgi/src/sbin/atacontrol/atacontrol.c.diff?r1=1.47;r2=1.48

I cc'ed person, who commited this fix.
Hi, Poul-Henning, I think it should be MFCed before release.

I agree, but I'm ENOTIME.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: fs/udf: vm pages overlap while reading large dir

2008-02-12 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Andriy Gapon writes:
on 06/02/2008 18:29 Andriy Gapon said the following:
 Small summary of the above long description.
 For directory reading fs/udf performs bread() on a (underlying) device
 vnode. It passes block number as if block size was 512 bytes (i.e.
 byte_offset_within_dev/512).

We have three sizes of relevance here, the sectorsize of the provider,
the blocksize of the filesystem and the page size of the system.

In general it is adventurous to have any of them be anything but
powers of two, and it is at best ill-adviced and more likely asking
for trouble to do requests that are not multiple of and aligned to
the sectorsize of the provider.

So up front, I would say that it is an UDF bug to do 512 byte reads off
a 2k sector provider.

Making the buffer cache handle this is possible, but it is not the
direction we have planned for the buffercache (ie: that it should
become a wrapper for using the VM system, rather than being a
separately allocated cache).

So if the objective is to make UDF work in the short term, your
change might work, if the objective is to move FreeBSD's kernel
architecture forward, the right thing to do is to fix UDF to not
access subsectors.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: fs/udf: vm pages overlap while reading large dir

2008-02-12 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Andriy Gapon writes:

2.3. this code passes to bread blkno that is calculated as 4*sector,
where sector is a number of a physical 2048-byte sector. [**]
[**] - I think that this is a requirement of buffcache system, because
internally it performs many calculations that seem to assume that block
size is always 512.

Yes, the buf-cache works in 512 bytes units throughout.

3.1. for a fresh buf getlbk would assign the following:
bsize = bo-bo_bsize;
offset = blkno * bsize;

That's clearly wrong.

We need to assert that the blkno is aligned to the start of a sector
and use the 512 byte units, so I guess it would be:

offset = dbtob(blkno);
KASSERT(!(offset  (bsize - 1)), (suitable diagnostic));


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Progress for 7.0 - the what's cooking page

2007-09-04 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Ivan Voras writes:

As some of you may know, I'm maintaining a web page which aims to=20
enumerate and describe major new features for FreeBSD 7, located at=20
http://ivoras.sharanet.org/freebsd/freebsd7.html .

Feel free to add:

ACPI suspend/resume does not work on SMP systems, including
multi-core laptops.

:-/

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Tester needed, picture heavy webserver preferred.

2006-06-20 Thread Poul-Henning Kamp

(For picture heavy substitute the word you all have in your spamfilters.)

I'm writing a serverside web-cache and I am looking for a web-operator
in the heavy duty picture category to get some input and possibly a
beta tester.

I would prefer a site which runs FreeBSD, and I it to be somebody who
is into serious bandwidth AND clue.

Should any of you know of a good candidate, please tell them to email
me.

Thanks in advance!

Poul-Henning

PS: Full discretion!  The software will be delivered in plain brown IP
packets with no source address :-)

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Any D-Link employees here ?

2006-03-10 Thread Poul-Henning Kamp

Apologies for this interruption:

If we have any D-Link employees in the audience, please contact me by
private email, thanks!

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: [CALL FOR TESTERS] New system call: abort2()

2005-12-15 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Warner Losh writes:

 On the similar note, the ability to move declarations closer to the
 point of use in code is IMO nice feature, too. The style(9) doesn't
 mention this either.

C doesn't allow it, or didn't until recently.  That style tends to
lead to really gross things too.  Functions should be short enough
that it doesn't matter.

Also, it tends to make it harder to judge the amount of stackspace
a function uses, something which is not entirely uninteresting in
kernel programming.

And yes, changing style(9) is just not worth the time it takes.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: GEOM for multipath? How?

2005-11-10 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Sergey 
Babkin writes:
From: Danny Howard [EMAIL PROTECTED]

Hey ... yes, I recall there being issues with the QLogic drivers ... I
wonder if anyone has given the mpt drivers a shot?  I was able to speak
with an engineer at Engenio (now owned by LSI) and she said there were
some issues with the QLogic dual-port cards that were interesting to
her, but the LSI dual-port cards behaved differently ...

QLogic worked fine in multi-path configuration with UnixWare.
I think LSI and Adaptec did too. The only trick is to make sure 
that the IRQs of the cards are not shared between the cards or with
any other device.

I suspect it is not the card as much as the driver, but I am not sure.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: dev_lock() question

2005-09-29 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], John Baldwin writes:


Actually, you would think that it could be initialized either via an early 
SYSINIT() or in the init_mutexes() function in kern_mutex.c and thus not need 
the early check and avoid penalizing dev_lock().

phk, how early his dev_lock needed?

Far too early due to console madness (in syscons I belive).


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: dev_lock() question

2005-09-29 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], John Baldwin writes:
On Thursday 29 September 2005 02:14 pm, Poul-Henning Kamp wrote:
 In message [EMAIL PROTECTED], John Baldwin writes:
 Actually, you would think that it could be initialized either via an early
 SYSINIT() or in the init_mutexes() function in kern_mutex.c and thus not
  need the early check and avoid penalizing dev_lock().
 
 phk, how early his dev_lock needed?

 Far too early due to console madness (in syscons I belive).

So would mutex_init() work?

Havn't tried.  It basically has to work right before the copyright
is printed.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: devfs file name NLS encoding?

2005-09-13 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], [EMAIL PROTECTED]
org writes:
Hi, as you know, we can create arbitaly file name on devfs.
But for now, all file names on a devfs are encoded in ASCII.

If we want to put Japanese file names in devfs, how should  it 
be encoded? UTF-8 or something convinient for the source encoding?

All filenames in DEVFS are either created from the device driver
'C' source (as a string literal via sprintf mostly) or from userland
as symlink.

So whatever works.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Kernel Source Divergence, Security (was: booting gbde-encrypted filesystem)

2005-07-31 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Allan Fields writes:

Yes, this is all very nice, but when is someone actually going to
commit it? ;)

I'm (as always) short of time, and GBDE is not the top priority
for me for the time being.

So I am more than happy to see people band together and improve
gbde.

The main work necessary is to polish the userland program and that
is relatively trivial programming, so anyone should be able to pick
that up: just go for it.

Giving gbde a taste function so that the root filesystem can be
protected by GBDE, this is also OK by me in principle, but I'd like
to review the patch before it gets committed because there are a
large number of dragons.

In P4:phk_gbde there is the beginning of hw-crypto support through
opencrypto(9), if somebody wants to work on that, get in touch with
me.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Kernel Source Divergence, Security (was: booting gbde-encrypted filesystem)

2005-07-31 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Pawel Jakub Dawidek writes:

I'm starting to wonder if we couldn't create one storage-crypto-base
and rewrite gbde, geli on top of it.

Could be, it all depends how much you actually gain from generalizing
common code.

Best way to find out is to try :-)

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: booting gbde-encrypted filesystem

2005-07-29 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Alexander Leidinger writes:
Pawel Jakub Dawidek [EMAIL PROTECTED] wrote:

 This is not not possible with current GBDE.
 I've patches which allows this here:

  http://people.freebsd.org/~pjd/patches/gbde.patch

I fail to see how this allows an encryted root-FS, it doesn't add gbde
support to boot0(ext) or to the loader. It needs access to an unencrypted
kernel. I don't think this is what Ronnel had in mind (overlooking the fact
that his suggestion to save the passphrase in the loader is insecure).

There is a difference between loading the kernel from an encrypted volume
(very hard!) and mounting the root filesystem from an encrypted volume
(possible with pawels patch.

Now of course, if your kernel has been trojaned, you're in trouble, but
then again, most people just worry about their data if the machine gets
stolen.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Ancient FreeBSD releases online

2005-07-03 Thread Poul-Henning Kamp

ftp://phk.freebsd.dk

./386BSD/cd1.iso
./BSD4.4-LITE/cover.pnm
./BSD4.4-LITE/cd1.iso
./BSD4.4-LITE/cd2.iso
./BSD4.4-LITE/cd3.iso
./FreeBSD-1.0-RELEASE/cover.pnm
./FreeBSD-1.0-RELEASE/cd1.iso
./FreeBSD-1.1-RELEASE/cd1.iso
./FreeBSD-1.1.5.1/cd1.iso
./FreeBSD-2.0-RELEASE/cd1.iso
./FreeBSD-2.0-RELEASE/cover_black.pnm
./FreeBSD-2.0-RELEASE/cover_green.pnm
./FreeBSD-2.0.5-RELEASE/cover.pnm
./FreeBSD-2.0.5-RELEASE/cd1.iso
./FreeBSD-2.0.5-RELEASE/cd2.iso
./FreeBSD-2.1-RELEASE/cd1.iso

Still in upload queue:

./FreeBSD-2.1-RELEASE/cd2.iso
./FreeBSD-2.1.5-RELEASE/cd1.iso
./FreeBSD-2.1.5-RELEASE/cd2.iso
./FreeBSD-2.1.6-RELEASE/cd1.iso
./FreeBSD-2.1.6-RELEASE/cd2.iso
./FreeBSD-2.1.7-RELEASE/cd1.iso
./FreeBSD-2.1.7-RELEASE/cd2.iso
./FreeBSD-2.2-960501-SNAP/cd1.iso
./FreeBSD-2.2-960801-SNAP/cd1.iso
./FreeBSD-2.2-961014-SNAP/cd1.iso
./FreeBSD-2.2.1-RELEASE/cd1.iso
./FreeBSD-2.2.1-RELEASE/cd2.iso
./FreeBSD-2.2.2-RELEASE/cd1.iso
./FreeBSD-2.2.2-RELEASE/cd2.iso
./FreeBSD-2.2.5-RELEASE/cd1.iso
./FreeBSD-2.2.5-RELEASE/cd2.iso
./FreeBSD-2.2.5-RELEASE/cd3.iso
./FreeBSD-2.2.5-RELEASE/cd4.iso

The server is a Soekris NET4801 and it's primary task is my
email, so if this gets abused it'll disappear again.

Mirrors welcome.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


wishlist for sysutils/xbatt: two batteries

2005-06-01 Thread Poul-Henning Kamp

My thinkpad has two batteries and apm reports this:

APM version: 1.2
APM Management: Disabled
AC Line status: on-line
Battery Status: charging
Remaining battery life: 49%
Remaining battery time: unknown
Number of batteries: 2
Battery 0:
Battery Status: charging
Remaining battery life: 98%
Remaining battery time:  0:00:00
Battery 1:
Battery Status: critical
Remaining battery life: 0%
Remaining battery time:  0:00:00
Resume timer: unknown
Resume on ring indicator: disabled


Isn't there some kind soul who can make sysutils/xbatt (or some other
X11 tool) show the status for the two batteries individually ?

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: the current status of nullfs, unionfs

2005-03-10 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Erez Zadok wr
ites:

   Anyone can download our unionfs software and the testsuite within from
   here:

   http://www.filesystems.org/project-unionfs.html

   You may consider it the first ever response to phk's request. :-)


yEHA!

Thankyouthankyouthankyou!

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-07 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], [EMAIL PROTECTED] writes:

 I also believe that it would be beneficial to implement regular rewriting
 of randomly picked lock sector(s) at random times during a user specified
 interval (up to x rewrites within n seconds) in order to further obscure
 the write pattern and provide additional protection for lock sectors.
 ALeine

I agree. I would also add random reads (or specially designed, combined
random reads and writes) to make traffic analysis and differential attacks
a real PITA for the hacker (although this idea may not be very effective
against a highly motivated and determined attacker, such as some government,
for instance).

If you want to do something like this, you want to do sectorrenaming
and journaling since that means you can only see that something
was written but not what it was that was written.

The performance cost can be considerable and the complexity formidable.
There are incredibly many cornercases to handle.

BTW, PHK, why did you choose the scheme of encrypting offsets of lock
sectors with part of key material and storing them somewhere, instead
of just using part of the key material itself to determine the offsets?

Because if I used part of the key material you would have to change
the location of the lock sectors when you changed the key material.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-07 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], [EMAIL PROTECTED] writes:


 If you want to do something like this, you want to do sectorrenaming
 and journaling since that means you can only see that something
 was written but not what it was that was written.

So you think that just adding specially crafted, random reads/writes
will have no significant positive impact on security of hot disks?

No I don't think so, because I tried it and were able to see straight
through it myself.  The trouble is that normal disk activity is not
random.

Randomness sticks out like a sore thumb in any sort of analysis, this
is why plausible denial of existence of encrypted data is so hard
that it is almost impossible when faced with a good adversary.

 The performance cost can be considerable and the complexity formidable.
 There are incredibly many cornercases to handle.

But you do not deny that providing strong protection for hot disks
is very important? After all, user protection is only available when
the disk is hot.

It is very important, but it is also a task which is very formidable
in comparison with protecting cold disks.  (Hint: attend BSDcan2005).

Speaking of user protection, how did you implement the procedure of
erasing keys? Did you account for the properties of magnetic media
and RAM that make data recovery possible? See, for example:
http://www.cs.auckland.ac.nz/~pgut001/pubs/secure_del.html

No I did not, because there is no reliable way to counter it from
software.

There may have been once upon a time when that paper was written
10 years ago (although I seriously doubt the actual efficiency of
the proposed schedules), but these days with Giant Magnetoresitive head
technology and Partial Response Maximum Likelyhood decoding you can
write and write and write and you will have no idea if it works or
not.  Bad sector forwarding is another issue in this area.

There are commercial companies who have specialized in recovering
deleted data from trackfringes.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-06 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Charles M. Hannum wri
tes:

While you might claim that the dedication to study the user's behavior and 
mount such an attack is fanciful, I claim that it is not.  Under observation, 
GBDE's additional techniques do not stand up to the claim of being spook 
strength.

GBDE only makes that claim for a cold disk for pretty much the very
reasons you mention.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-06 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Steven M. Bell
ovin writes:

etc.  I think we need to be careful about phrases like one can.  I 
decided to stop supposing and gather some real data, so I wrote some 
analysis tools to measure the entropy of disk drives.  I need to 
rewrite some of my tools and do a lot more analysis, but I think the 
results thus far are quite interesting.  See
http://www.cs.columbia.edu/~smb/rawdisk-entropy.ps

I did something similar when I studied if more intricate sector
placement hiding algorithms would be worthwhile.

In addition to various UNIX disks I also analyzed disk images from
windows server, windows laptop, s390/MVS volumes, archive disk
images from an official government library and optical disk images
from an archive of scanned documents.

Overall, almost all the disks except the archive and mainframe disks
had a zero peak with sectors never written to.

The rest of the curve can have any shape you can imagine but will
often have some number of distinct peaks for certain content types.

The highest entropy I found was a disk-images from a public FTP
server and a not-quite-warez server both of which extensively
used file compression and the scanned image archive.

The UNIX software developer disks had particularly low entropy
because of vast amounts of source code in ASCII.

My guess is that an attacker who would have access to these
distribution curves for his target data would be very likely to
also know more detailed/specific things about it, and that
informatio would likely be much more helpful to his attack.

Under the assumption that the disk is flushed with PRNG bits
initially, and that the output of the cipher has high entropy too,
I decided that trying to obscure these patterns, as well as the
physical layout patterns of the storage managment (filesystem/
database) beyond the basic rotation, would just slow things down
and not add any incremental security worth it.

Poul-Henning

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-04 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Perry E. Metzger writes:

The best I can say, however, is that the US
government has approved the use of AES with 256 bit keys for very
highly secure communications, and they have a very demanding user
community.

(There is a big difference in what crypto you need for communications,
storage, and archiving.)

My philosophy in this, (as implemented in GBDE), is to trust the
algorithms to do their job, but to not offer an attacker any more
leverage against them than I absolutely have to.

It has been said by various people that there are [currently]
no cause to think that using the same key for many millions of
sectors is a problem.

I belive that is a fair and balanced summary, and I trust the
people to know what they talk about and I belive them.

Nontheless, I do not consider it good engineering to expose the
algorithm to unnecessary stress, even if we currently belive we
have a 220 bits margin of safety, if by trivial means, I can avoid
that stress on the algorithm and maintain 256 bits of margin.

Exactly where the line is between overkill and conservative engineering
is always subject to individual preference and taste.

I don't seriously think that either of CGD or GBDE will be broken
through any other path than guessing or otherwise obtaining the
passphrase.

But in the case something unforeseen happens, I know that GBDE will
yield its secrets only one sector at a time and CGD will spill all
the eggs at once because it has only one basket.  (Hans Christian
Andersen wrote quite amuzingly about this 150 years ago in the story
The woman with the eggs.)


As for making user selectable ciphers and keysizes I decided against
it for two reasons.

The first reason is that it adds complexity.

The second is that it offers complexity.


Adding complexity in the implementation is wrong for all the reasons
we all agree on.  To justify that complexity, a benefit must be
proven.

In all my interviews and talks with people, I found nobody who
wanted that level of flexibility.  Everybody, almost in unison sang
from the AES is the annointed king and 128 bit is the his key
size. hymn.

That surprised me initially.

It transpired that people had a very simple and prosaic reasoning:
Anything else will give us footnotes during audits.  Different
ciphers will make the auditor write the standard AES should have
been used even if the cipher used is agreed by everybody to be
stronger.  A longer keylength will result in a note about unneccessary
overkill and waste of resources.


The other complexity is for the user.  The more questions the user
has to determine the correct answer to, the less likely he is to
get it all right if he is not a subject matter specialist.

My favourite question from a UNIX installer was something like:

Do you want this system configured according the the
regulations set forth in [45 character of legal reference]
? [yes/no] _

And as you can guess, it didn't even offer a default to hint
what one should choose.   (As it later transpired, the option
would disable the support for a locally connected printer.)

Offering options in a situation where users will generally not dare
using anything but the default is not only quite pointless, it is
down right detrimal to user experience, and, probably, security.

It used to be that I, as a UNIX wizard of some renown, knew what
happened when a user logged into a FreeBSD box.  But today, between
ssh, opie, PAM, access.conf and what else gets in the way, I have
to say that unless all settings are the default, it will take me
considerable time to determine that no holes have been opened.

I fully agree that we have gained fantastic flexibility through all
these features, but I am not always convinced that they are a net
improvement of security when measured over the set of all FreeBSD
machines in the world.


So I made the choice to structure the source code and the crypto
path so that other algorithms could be slotted in by any minimally
competent programmer, and stuck to the choice of algorithm which
seems to be the king of the hill these days.  If the wind or the
king changes, the source code will adapt in less than a day.


Finally, on keying schemes:  I didn't put a keying scheme on GBDE
because I belive in the tools not policies dogma.  The gbde(8)
userland program should be (but isn't quite) flexible enough to
support any keying scheme people decide to use.

And I also belive in the many hands make light work, so I sort
of hoped that people who knew more about public key crypto than me
would produce scripts or frontends which implement sound keying
schemes for GBDE based on these technologies.

Poul-Henning

PS: Will you be at BSDcan ?  It would be a good chance to corner
a whiteboard and some beer.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately

Re: FUD about CGD and GBDE

2005-03-04 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Jari Ruusu writes:

Early versions of loop-AES were FUBARed, true. But why do you insist on
ranting about fuckups that were fixed long time ago?

I don't :-)

The topic at hand was why I made certain choices for GBDE the way
I did, what loop-AES did subsequent to that time is not really
relevant.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Bernd Walter writes:

No matter what disk you take - writes never have been atomic.
The major difference I see is that you get a read error back in
the disk failure case, while such a crypto failure produces more or
less random data without any error.
Mounting unclean filesystems rw for bg_fsck can be considered
dangerous with such unexpected data corruption.
And how would you know that a restore from backup is required for
a damaged file?

100% true.

The trouble is that it would cost a lot in performance and a doubling
in metadata to protect yourself against this.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], ALeine writes:

Not necessarily, if one were to implement the ideas I proposed
I believe the performance could be kept at the same level as now.

I gave up on journalling myself because IMO it complicates
things a lot and the problem it solves is very very small.

The impact in disk seeks is non-trivial to predict, but it is
very hard to argue that it will not lead to an increase in
disk seeks.  (This is really a variant of the age old argument
between jounaling filesystems and traditional filesystems)

I can only recommend that you try :-)

We need more ideas and more people trying out ideas.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Thor Lancelot Simon writes:

No, it would not.  What it _would_ take would be an abandonment of the
adamant position that your home-grown cryptosystem is superior to
simply encrypting the disk with 256-bit AES.

Where I come from home-grown is not derogative.  All cryptosystems
are by necessity home-grown for somebody somewhere.

If you are _convinced_ that there will be no attacks which can
exploit the ample data CGD offers for two-way leverage on the crypto
algorithm during the relevant lifetime of your data, then stick
with CGD and be happy.

If like me that makes you quite uneasy, look for something which
mitigates that issue, like for instance GBDE.

If neither suits you, design your own.

Generally, complexity is not considered a desirable property in
cryptosystems.  GBDE violates this rule in spades.  There are _reasons_
why complexity is not good: to begin with, a very complex cryptographic
construct will require detailed analysis (which it does not appear
GBDE has had by anyone but its author until Roland started looking at
it) in order that we may know that it is even as secure as the underlying
algorithmic building blocks it uses.

Both Lucky Green and David Wagner has nodded vertical on GBDE.

[crypto sermon]

I fully agree with you about the philosophical points, but not on
the implications.

I can not convince myself that encrypting a 40 GB disk sector by
sector using the same key, even if it is 256 bits, is a safe design.

You seem to belive otherwise.

And that's where it ends.

Have a good life.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Thor Lancelot Simon writes:

I could not disagree more.  When it comes to nonstandard homebrewed
cryptosystems foisted off on unsuspecting users with a bundle of
claims of algorithm strength that they're not competent to evaluate
for themselves, we do not need more ideas, nor more people trying
out ideas; we need less.

Standard, widely analyzed cryptographic algorithms are good.

s/ are good/, when applied with caution and wisdom, are good/

:-)

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Roland Dowdeswell writes:

I chose CGD with AES256 for two reasons.  First I wanted to compare
systems with comparable performance.

More computing sins are committed in the name of efficiency
(without necessarily achieving it) than for any other single
reason - including blind stupidity.

-- W.A.Wulf

GBDE has two major designgoals: Security and Usability/Deployability.

Performance is somewhere down the list after those two.

I'm looking forward to seeing something from you which amounts to more
than name-calling and hand-waving.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], ALeine writes:
[EMAIL PROTECTED] wrote: 

 I gave up on journalling myself because IMO it complicates
 things a lot and the problem it solves is very very small.

If only hardware manufacturers were to equip hard drives with
a mechanism to ensure atomic writes.

I would prefer If only some bloddy laywer would stop suing
any person who tries to market a NVRAM PCI card :-(

Journaling would definitely cause more seeks, but the question
is how well that can be compensated for by the reduction of
writes gained from using the other speedup mechanisms I proposed.
I might try implementing this in GBDE first to see what the
benchmarks say.

I can not encourage you enough to try it.

Don't let peole like Thor scare you away, progress happens when people
try to follow their ideas, even if told that they are fools by people
who (think they) know better.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Richard Coleman writes:

For instance, the NIST specification for AES and CCM mode (NIST Special 
Publication 800-38C) specifically states that you must limit the number 
of invocations of the block cipher (specifically AES) to 2^61.  Now, I 
realize that is an upper bound.  But even after removing several orders 
of magnitude, that leaves a huge amount of material you can encrypt with 
a single key.

Just throwing out a data point.

This would be much more interesting if you qouted the number they will
say ten and twenty years from now.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Thor Lancelot Simon writes:

It also uses MD5 in a way that I would characterize as not exactly
ordinary 

The only role MD5 has is as a bit-blender.  Any strength it may
add is just a bonus.

Indeed, the large number of algorithms
used in the keying and encryption process for any block in GBDE
does not necessarily increase its security: in fact, certain
kinds of flaws in any one of those algorithms could in fact make
the decryption of any particular block _more_ likely -- and Roland
has pointed out how the design of GBDE allows such failures to
cascade through the entire set of encrypted data.

I'm very much looking forward to Rolands analysis as compared to
his hand-waving.

The very complexity of your system makes it very, very
difficult to evaluate just how secure it is, and you seem to think
that that is a benefit: comparing the incommensurables I don't
believe and I don't know, but I suspect, you land on the side of
I suspect.

I invite you to analyse GBDE, and once you have determined which
kinds of vulnerabilities in the compontent ciphers it would require
before the house of cards come tumbling down.

Then, before you fly off the handle like Roland did, take a moment
to consider what else those flaws would doom.

Then report your findings in a professional way.

The argument I hear right now is I have not bothered to actually
analyse GBDE at all but I heard there were a neck-tie party going
on so I thought'd I'd lend them a hand since it is nobody I know.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Perry E. Metzger writes:

There is a profession called cryptographer out there. They are the
folks who try out these new ideas, and they fill lots of conference
proceedings with their new ideas, including things like crypto modes
designed specifically for disk encryption.


There is a world out here that's called the IT industry.

When they badly needed a new password scrambler nobody from that
cryptographic discipline could be bothered with a problem already
long since solved in their academic journals and the task fell to
a more or less random programmer in the end.

Today that algorithm, with all its flaws, seems to protect a very
good sized chunk of the passwords on the internet:  All cisco
routers, all FreeBSD systems, the majority of Linux systems etc etc
etc.

At the time where I wrote GBDE, the best that was offered was CGD (and
similar) and users (not cryptographers!) didn't trust it and history
have so far repeated.

I am not claiming that things couldn't be done smarter than GBDE,
but I do notice an distinct lack of attempts to do so from the
cryptographic establishment.

I can add another property of the elite society of cryptographers:
if you are not a card carrying member of their society, the majority
of their members can not even be bothered to reply to an email from
an outsider.  This does hamper communiation a bit.

Maybe the problem is that cryptographers, like true computer
scientists, write in nothing less portable than pencil number two ?

If some card-carrying member of the cryptographic establishment were
to offer the Open Source operating systems an implementation which
were approved by all (or some qualified quorum of) the high priests
of their brother hood, then I am sure that it would be received with
praise and thanks of no end.

Poul-Henning

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Thor Lancelot Simon writes:
On Thu, Mar 03, 2005 at 08:25:18PM +0100, Poul-Henning Kamp wrote:

To quote David Hume, Never an ought from an is.

I'm Danish by birth so english is only my second language, so I
apologize for mangling it.

That users (who
are they?  how many of them?  What criterion or criteria  of trust
do they apply?) _did_ not trust X says precisely nothing about whether
users _should_ not trust X.

soapbox

If there is one word I have come to detest, then should is at the
top of the list.  Voters _should_ vote based on intelligently informed
opinions.  Researchers _should_ report their findings uncolored by
personal bias.  Kids _should_ listen to their parents.  Somebody
_should_ fix this bug.

I increasingly associate will not happen when I read should.

Let me twist it around:  How would the users know if they should or
should not trust something ?  They form their opinion based on the
information they have under the constraints they have.  And then,
more often than not, the remaining 30% is gut feeling.

When it comes to crypto gut feeling has about 70% of quorum.

The crypto establishment has a big problem communicating to the
rest of the world what their findings are in a way that makes this
information usable for people.  (IMO).

/soapbox

You seem to deny that there is a particular domain of expertise that is
cryptography, or perhaps more rightly two domains, one being largely
a subset of the other: how to design good cryptographic algorithms and
how to use good cryptographic algorithms safely.

No I certainly don't.

I have personally the deepest respect and admiration for the craft.

I spent a lot of time before going into GBDE reading theory.
Interestingly again, the best book from a practitioners point of
view is written by an outsider in the crypto-clerigy.

I also spent a lot of time studying what was already available.

But in difference from everybody else (it seems) I also asked users
and administrators what they needed and wanted from a cryptographic
disk facility.

Interestingly I found that the users focus were very different from
the points which the crypto community emphasized.

And then I designed and wrote GBDE from that angle.

Despite what some people in this dicussion seems to belive, I did
not write GBDE using 1 iteration random-seed genetic programming.
A lot of thought and consideration went into it. 

I may not be a world renowned cryptographer, nor even claiming to
be one at all, but I am not totally without ability either.

I am fully aware of the arguments against complexity and I tried
very hard to simplify GBDE to the simplest possible algorithm while
maintaining the design goals fulfillment.  That is why there is no
journaling, no MAC, only a very simple level of positional hiding
and no heavy duty support for plausible denial.

And then I tried very hard to engage somebody with the right
union-card to do a review for me, and despite the fact that funding
were available under the DARPA contract nobody would bite.

Lucky Green, on his own initiative contacted me because he heard
the rumour that I was working on something, and he convinced David
Wager to take a peek as well.  I am more grateful to them both than
my words can express.

They gave me a lot of sound advice and I tried my best to implement
according to it, but any blame for mistakes is entirely mine.

Now, if you could stop defending the cryptographers-local-64 union
and accept that non-union people might try to make the world a
better place by applying some of the craft in actual code, instead
of banning the code because an infidel wrote it, then you could
really help by giving said code a professional review.

It would be much appreciated if you did.

If you sit down and study GBDE, you will find that I have used all
the cryptographic algorithms in a conservative way and likely as
not, you will end up saying overkill.  The users will call the
same safety margin.

The truth is somewhere between, because the real world is shades
between dark white and light black.

You call Roland's criticisms of GBDE handwaving.

I have yet to see anything solid from him where he didn't overlook
something in his haste to prove his own product superior.

Poul-Henning

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Todd Vierling writes:
On Thu, 3 Mar 2005, Poul-Henning Kamp wrote:

 At the time where I wrote GBDE, the best that was offered was CGD (and
 similar) and users (not cryptographers!) didn't trust it

Could you back up this claim, insofar that users did not trust cgd?  I
haven't seen any distrust of cgd -- in fact, I've seen quite a bit of
welcome acceptace of cgd by both users *and* cryptographers.

Some of the people I talked to were very unhappy about the same key
being used for all sectors on the disk.  Even a small weakness in
the cipher becomes a big hole because of the amount of data this
offers for analysis.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Perry E. Metzger writes:

 There is a world out here that's called the IT industry.

Yes, there is. They routinely deploy bad security because they don't
get people who know what they are doing involved. See WEP, for
example, or a thousand other things.

Yes, it would really be desirable for the cryptographers to come
down from their mount olympus more often.  Too bad they never
answer invitations :-(

I have no idea what you're talking about,

Thanks for confirming what I wrote just a second ago above.

If you're talking about MD5 which is used in many modern Unixes, it
was done by Ron Rivest, and even though he's really good, it has
recently been (quite badly) broken.

Again.

 At the time where I wrote GBDE, the best that was offered was CGD (and
 similar) and users (not cryptographers!) didn't trust it and history
 have so far repeated.

I have no idea what you are talking about here.

And again.

 I can add another property of the elite society of cryptographers:
 if you are not a card carrying member of their society, the majority
 of their members can not even be bothered to reply to an email from
 an outsider.  This does hamper communiation a bit.

Actually, you can show up at any crypto conference you like,

I have a better idea: Why don't we get the cryptographers to
show up at computer science conferences ?  That would get the
gospel out to a far wider crowd without spoiling the highly
specialized conferences for the cryptographers.

 Maybe the problem is that cryptographers, like true computer
 scientists, write in nothing less portable than pencil number two ?

It is rare to see a new algorithm show up from someone like Ron Rivest
without some C code also appearing. That's pretty common in the crypto
world. When the Chinese team that cracked a bunch of hash algorithms
last summer presented their work, they had worked examples of their
stuff.

You seem to misunderstand something:  Computer users don't call MD5
directly.  They use software which makes the calls for them.  Sometimes
this software has a goal which is different from calling crypto
algorithms, in fact some of them even have the impropiety of
regarding the crypt algorithms as mere tools.

I think you don't quite get it the point.

There are many points not being got here.

1) No one claims that you need to be a cryptographer to write
   something like GBDE. What is being claimed is that you should not
   have invented your own cryptographic modes, and that you might have
   wanted to ask some professionals about your approach.

You have not actually studied GBDE yet, right ?  You don't actually
know if I invented my own cryptographic modes or not, do you ?

2) CGD *has* been looked at by a bunch of people, and was written to
   carefully use standard algorithms in a standard way. If you don't
   like using NetBSD code because NetBSD people have cooties, fine --
   I don't care, write your own. However, you should at least pay the
   same attention to conservative use of cryptographic algorithms and
   having people review your work is a good idea, too.

Even if I were alone in the world with the sentiment, I would never
call CGDs use of the same key for all sectors conservative.

3) You've made some very bizarre claims about the security of your
   system. Some of these claims have already been shown on their face
   to be incorrect, such as your claimed work factor for breaking your
   new improved crypto modes.

Sorry, they have only been disproved in a significantly larger universe
than the one my users inhabit.  That doesn't count to me.

   Instead, he admitted his mistakes and wrote a version 2.

Any qualified, factually correct critique of GBDE will be taken very
serious by me.  I am very much looking forward to it.  What Roland
has provided is not it.

   Are your users better served by you digging in your heels and
   saying GDBE is perfect as it is, 

Now, there is one thing I have never said and would never say.


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Perry E. Metzger writes:

Poul-Henning Kamp [EMAIL PROTECTED] writes:
 Don't let peole like Thor scare you away, progress happens when people
 try to follow their ideas, even if told that they are fools by people
 who (think they) know better.

They laughed at Fulton.

They also laughed at Bozo the Clown.

You're in good company then :-)

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Perry E. Metzger writes:

Poul-Henning Kamp [EMAIL PROTECTED] writes:
 In message [EMAIL PROTECTED], Todd Vierling writes:
On Thu, 3 Mar 2005, Poul-Henning Kamp wrote:

 At the time where I wrote GBDE, the best that was offered was CGD (and
 similar) and users (not cryptographers!) didn't trust it

Could you back up this claim, insofar that users did not trust cgd?  I
haven't seen any distrust of cgd -- in fact, I've seen quite a bit of
welcome acceptace of cgd by both users *and* cryptographers.

 Some of the people I talked to were very unhappy about the same key
 being used for all sectors on the disk.

Now, was that in the first day after cgd was committed or the second?
As I recall, you committed GBDE 48 hours after CGD was committed in
NetBSD. I'd be curious to hear about how much you changed your design
in that period in response to feedback on cgd. (Please correct me if
I'm wrong about the time gap.)

I am being a bit unfair here because I am lumping CGD in with the
equally defficient code in Linux (Loop-AES etc).  It was mostly the
linux code I talked to people about, but CGD makes the same exact
mistake.

 Some of the people I talked to were very unhappy about the same key
 being used for all sectors on the disk. Even a small weakness in
 the cipher becomes a big hole because of the amount of data this
 offers for analysis.

I think we've already established that this fear, though
understandable, is not a reasonable one under the circumstances. See
several postings already made. You are better off just using AES with
a longer key than the GBDE mechanism.

I'm sorry, I reached the exact opposite conclusion.

The work that was referred to earlier of defactorizing AES into a
very large number of equations would be exactly the kind of attack
I would worry about if I have 80 million sectors with the same key.

If that attack comes through, but relies on some partiular bit
combination being present on the plaintext or ciphertext of the
algoritm, I see no reason why I would want to improve the attackers
odds by a factor of 80 million.

And if CGD is _so_ officially approved as you say, then I can not
for the life of me understand how it can use the same key to generate
the IV and perform the encryption.  At the very least two different
keys should have been used at the expense of making the masterkey
512 bits instead of 256.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], ALeine writes:
[EMAIL PROTECTED] wrote: 

 I can not encourage you enough to try it.
 
 Don't let peole like Thor scare you away, progress happens when
 people try to follow their ideas, even if told that they are fools by
 people who (think they) know better.

Thor? Who is Thor? :- Seriously, this discussion is only making me
more eager to implement my ideas, I know exactly what I want to
implement, it just might take me more time than you seasoned kernel
programmers who live the code. :-)

Start out doing it in userland, it's much easier to work with.  In
FreeBSD we have something called geom-gate which allows you to
implement disks in userland.  I'm sure something similar exists or
can be trivially created in your favourite OS.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Todd Vierling writes:
On Thu, 3 Mar 2005, Poul-Henning Kamp wrote:

 And if CGD is _so_ officially approved as you say, then I can not
 for the life of me understand how it can use the same key to generate
 the IV and perform the encryption.  At the very least two different
 keys should have been used at the expense of making the masterkey
 512 bits instead of 256.

Technically, two different keys are used.  The IV is generated from the
block number (although it's pluggable for other IV generation methods,
should one be desired; take a look!).

As I read it, he encrypts the block number using the key to get the IV
which he then uses with the key to encrypt the data.

Since the attacker know the block number the IV generation doesn't
add strength.

In fact expose any weakness in the algorithm even more because it
offers two-way leverage on the algorithm.

It also adds a very efficient hit-detector for a brute force attack.

It would have been much better to use a different key to generate the IV.

And did he salt the block number at all ?  I don't think so...


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Thor Lancelot Simon writes:
On Thu, Mar 03, 2005 at 10:15:55PM +0100, Poul-Henning Kamp wrote:
 
 And if CGD is _so_ officially approved as you say, then I can not
 for the life of me understand how it can use the same key to generate
 the IV and perform the encryption.  At the very least two different
 keys should have been used at the expense of making the masterkey
 512 bits instead of 256.

Why should two different keys have been used?  It is possible that I
misunderstand the underlying theory, but so far as I do understand it
the only real requirement for IVs is that the Hamming distance between
any two used with the same encryption key be large.

I have studied the AES papers and in particular the attacks and
critisisms of it very carefully, and they have proven a whole lot
of things to be impossible, but they have not proven that there
are not more that needs to be proven impossible.

When DES was designed, nobody knew that differential attacks existed.

Shortly after AES was gold-plated the earlier mentioned attack
method where it is decomposed into a massive number of equations
was presented.

Since we cannot possibly know how AES can be attacked in the future,
we have to think about how we provide the least help to _any_
potential attack.

So my approach is to regard the algorithms as black boxes, and
study the information flow and degrees of freedom involved.

Try to zoom out a bit, and think of this as a black box with wires
going into it.  I belive CGD looks like this:


   Block no- AES -
   ^|
   ||
   ||
   KEY(256b) --+|
   ||
   ||
   v|
   DATA --- AES --
   |
   |
   v
to disk

By constraining the two AES instances to use the same key, we give
the attacker a lot less degrees of freedom to figure out.

Even worse, we may be helping him because worst case, the two uses
of the key could cancel something out in some way when two instances
of AES are hooked up this exact way.

Since the master keysize could have been 512 bits at no expense to
anybody, it only takes a minute modification to improve the diagram
above:


   KEY1(256b) -
   |
   |
   v
   Block no- AES -
|
|
|
   KEY2(256b) --|
   ||
   ||
   v|
   DATA --- AES --
   |
   |
   v
to disk

Now, if there are any weakness in hooking two AES instances together,
we at least are not giving the attacker any help exploiting it, this is
what I call lack of two-way leverage.

But there is (I belive) still a trivial mistake:  The IV generation
has totally predictable input.  Lets add another handful of bits
to the masterkey and fix that also:


   KEY1(256b) -
   |
   |
   v
   Block no- AES -
   + SALT (256b)|
|
|
|
   KEY2(256b) --|
   ||
   ||
   v|
   DATA --- AES --
   |
   |
   v
to disk


There.  I'm not a card-carrying cryptographer, but I belive I just
improved CGD in a fairly obvious way.

I would be very surprised if anybody can show me to have done any
harm with these two modifications because all I did was to improve
how the components were hooked together and add more bits to the
input side of the whoe thing, and that can never be a bad thing,
provided those bits are generated by a competent (P)RNG of course.

I am still uncomfortable with the KEY2 being constant over all
sectors and that is why GBDE doesn't look like this.  (We can not
just derive KEY2 from the block number like we do the IV because
then we

Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Roland Dowdeswell writes:

Now, the key--key sector protects 32 disk
sectors which contain 32 * 512 * 8 = 131072 bits.

The key-key protect only a single sector-key, however, the actual
sectorsize may be bigger than 512 and I recommend that people use
the FFS fragment size (typically 2k) if possible.

I have not taken the time to figure out how many of the 2^2048
possible sector contents are actually valid superblocks but it would
actually be an interesting number to know in this dicussion, just
like the fraction of valid inode blocks and directory blocks would
be interesting to know.

My guess is that the super block probability is about 1E-9, inode
block around 1E-8 and directory around 1E-9, but these are pure
guesses and likely off by a many orders of magnitude.

The bsd disklabel is not relevant in normal usage, people tend to
encrypt each filesystem individually as far as I have heard.

But Roland,

These continued false starts because you didn't take the time to
really understand what goes on in GBDE are a waste of both your and
my time and doesn't add to your reputation in any way.

I am old enough to actually appreciate a competently done review
because that is one of the best ways for me to improve my skills,
so even if your review ends up tearing GBDE apart at the seams,
you can trust me to appreciate that you did it.

And I am convinced you can give me a competent review if you give
it a fair shot, but right now you seem hell-bent on acing me rather
than doing your homework properly.

Please sit down and take the time it must take and then come back
with a professionally done review.  If there are things you are
unclear about, send me email and I'll do my best to explain them.

Thanks in advance,

Poul-Henning

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Perry E. Metzger writes:

I remember a certain talk at BSDCon where someone criticized the
design of the kernel RNG during the talk on it. He mentioned that the
person giving the presentation had stated a few inaccurate things,
such as claiming that there was a proof of security for Schneier's
Yarrow algorithm and a few other howlers. As I recall, he was
thoroughly criticized for mentioning these things. As I also recall,
when said person brought the topic up with a certain person named PHK,
he said I don't want to hear about it.

No, that is not what I said.  I said something about Yarrow being
so many times better than what we had before that it didn't really
affect the picture at this time and that if you had a beef with
Marks presentation you should take it up with Mark because I had
not myself read the Yarrow paper at the time.

 You don't actually know if I invented my own cryptographic modes
 or not, do you ?

You did.

I did ?  Cool, I should patent them!  :-)

Seriously, I don't think I did, but this may be a matter of semantics
which I am not aware of on account of not being a full time
cryptographer.

If by cryptographic modes you mean use algorithms in ways we have
not seen before, then yes, I may have.  That is not the same as
I havn't thought a lot about the implications.  For instance, the
role of MD5 is not to add strength but merely to give a better
statistical distribution of bits.

But either way, I'm very interested in getting a competent review,
but saying only you did something new is not that.

 Sorry, they have only been disproved in a significantly larger universe
 than the one my users inhabit.  That doesn't count to me.

Being stubborn on this isn't going to help your users. The math is
pretty obvious here. Sure a brute force isn't practical -- but neither
is a brute force of AES-256.

No, not right now.

But do we know that a brute force attack is unfeasible on AES-256
tens years from now ?  And are we sure that the reuse of key material
which happens in CGD will not materially aid any attacks that may
be developed in the next decade ?

If we do, please forward the paper.

The fact that you just need to break one single sector in CGD before
you get the entire disk contents gives a disadvantage to CGD of
2^26 before we even consider the nature of the attack.  That is not
conservative when it could have been trivially avoided.

someone points out an obvious flaw in your logic and shows the work
factor is lower than that for AES-256, the gentlemanly thing to do is
say you are correct, I was mistaken.

And you can trust me to do so.

But the flaw must be possible to exploit in the current universe.

The goal for GBDE is to give credible denial of access for up to
25 years, and if nothing else that limits the storage capactity
available for an attacker to within an order of magnitude of number
of particles within a 12.5 lightyear radius of Earth.

If you do not consider such limitations, you are just producing
silly math along the lines of the old joke: First, we assume a
hexagonal sheep.

 Any qualified, factually correct critique of GBDE will be taken very
 serious by me.  I am very much looking forward to it.  What Roland
 has provided is not it.

Roland's criticism is reasonable.

As you saw from the email preceeding this one, Roland still hasn't
found out how GBDE actually works.

Rather than getting angry, why don't
you consider it?

What I got angry about was the fact that Roland was spreading fud
about GBDE in all sorts of mailinglists without having enough honour
to give me a Cc: so I could have a chance to participate in the
discussion.

As I said in my reply just a second ago, I would very much appreciate
if Roland would take the time to give me a competent review of GBDE,
but he cannot do that as long as he is blinded by the desire to ace
me instantly instead of thinking his arguments properly through and
test his hypothesis.  Unfortunately I can no longer offer him financial
compensation for the effort, the DARPA contract is long since closed.

Poul-Henning

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Perry E. Metzger writes:

 MD5 was believed to be heavily understood in literature. It was
 well established. Look at what happened to it.

Yup. And Roland made the algorithm you use for encrypting your disk
*pluggable*. That way, if AES is broken, you can replace it with the
next big thing and move on with your life.

Now, if AES is indeed broken, GBDE is in serious trouble, but CGD is
not. Specific users of CGD have to change their drives, but the
framework continues to work as advertised.

Gee Perry, now you're spreading FUD.

You know perfectly well that it would take less than one hour to
substitute another algorithm in the GBDE source code.

Poul-Henning

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Perry E. Metzger writes:

I also very strongly suggest that the biggest real threat you face
isn't someone cracking AES but key management issues. CGD is in some
sense largely a framework for letting you do all sorts of neat things
with key management in a disk encryption context. You may want to add
similar features -- the most practical attack against your system as
it stands is a dictionary attack.

This is where it would have been nice that you didn't jump into
the middle of a discussion without reading the basic material.

See my paper please.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Steven M. Bellovin writes:

You can subscribe to that list by sending to [EMAIL PROTECTED]

Feel free to post the link to my paper there.

For reasons of mental bandwidth I must decline to subscribe to more mailing
lists than I'm currently infected with, but that doesn't mean I am not
interested in a discussion on the subject: just put me on the Cc:.

Poul-Henning

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Perry E. Metzger writes:

My strong suggestion for you is that you adopt a similar approach --
build a good framework that, given good algorithms, will provide
security, and make it easy for users to change over if an algorithm
falls.

If you actually look at GBDE, you will see that any and all of the
algorithms can be changed.  They are used only in their most basic
capability.  This was part of the design from the start: not to
rely on any single-source algorithm.

Well, so is stock AES 256. I don't see why I should assume your
construction is any better. What do you know that the NIST/NSA review
of AES did not know?

That neither the authors of Rinjdael, its reviewers, nor NIST are 
willing to offer a 25 year warranty on it.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-03 Thread Poul-Henning Kamp

In message [EMAIL PROTECTED], Roland Dowdeswell writes:

The claim is made that there is at least O(2^256) work to crack a
disk and O(2^384) to crack the disk if the lock sectors are destroyed.

Roland, in particular when you get into big numbers you have to
pay attention to small details.

The O(2^384) number takes the current estimate for the number of
atoms in the universe into account.

In the absense of a way to store more than one bit on every single
hydrogen atom in the universe, you cannot possibly correlate a
matrix which is 2^128 on boths sides without repeatedly recalculating
the results for one side as you iterate through the results of the
other.

The fact that you are not even guaranteed to be able to eliminate
a single candidate solution until you brute force at least 17 sectors
than two does not improve the situation any.

Poul-Henning

PS: I get a lot of bounces from various lists I'm not on.  I put
my faith in somebody forwarding my replies faithfully onto those
lists ?

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


FUD about CGD and GBDE

2005-03-02 Thread Poul-Henning Kamp
 to appreciate this flexibility.

All the talk about what happens during a powerfail/crash is rather
uninformed:  On any contemporary filesystem you will loose data,
encrypted or not, if the system crashes before you have a zero
return value from fsync(2).   If fsync(2) completed successfully,
your data is safe on the disk, both with GBDE and CGD.  Adding
journaling or before/after images to the disk encryption is totally
the wrong way to address this problem since all your filesystems
and disk device drivers suffer from the same issue.

(Adding journaling to your disk encryption has merits for other
reasons as it effectively obscures the write access pattern, but
the cost in performance and fragility of doing sector renames is
prohibitive IMO).


Finally, Yes, I will agree that GBDE takes a performance hit to
implement the PRNG single-use sector keys, but in practice that hit
is barely noticeable and the benefits it brings in cryptographic
strength can not be obtained in any cheaper way.


There are a lot of things GBDE doesn't try to protect against:
   illicit modification of disks (If people can modify your disk
  without you knowing about it, how do you know they did not also
  modify your other hardware ?)
   protection of running computers (This is an entirely different
  ball game).
   manipulation of entropy/randomness (If the attacker has this level
  of control over your computer, don't you think you have other
  problems which are more pressing ?)


The goal of GBDE was to write a trustworthy protection for cold
disks which also can protect the user and which can be used in
practice in real world organizations.

All reviews I have heard so far say that this goal was attained.

Several people have done detailed reviews for their respective
organizations, and none have yet to find anything wrong with GBDE.

Unfortunately, most people in this business seems to be very keen
on having their names not mentioned, so so far I think only Lucky
Green and David Wagner have stood by their opinion with their name.

I personally cherish this judgement from a government employed
analyst:

It is refreshingly clear that it is just over the dotted
line into over-kill, yet simple enough that any theory about
loopholes can be disposed with instantly.  Nobody will have
any doubt that the pass-phrase is the only viable access
path.


I personally find that the keyhandling is the only bit of CGD with
any real merits, what they do to the actual data on the disk appears
to be a mere afterthought to their work on key management.  Since
I will not deny that the keyhandling is probably at little too much
left in the hands of the user in GBDE, it is obvious that it would
benefit boths parties to make the CGD key handling an option for
GBDE.

So how about it guys:  Instead of spreading FUD, lets work together
and make the world an even better place ?

Poul-Henning


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-02 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], ALeine writes:

1. Introducing a sysctl variable to control how often (after how many writes)
the random key is regenerated:

kern.geom.bde.random_key_regeneration_cycle

The way it works now would correspond to the value being 1 with
regeneration happening on every write. Setting this higher would
improve the performance and people could decide for themselves
what value fits their needs.

I don't really think you would gain any performance, but I am open
to benchmarks proving me wrong.

2. Backporting to 4.x - this has more or less come to the point that
I would rather write the whole thing from scratch, similar to vncrypt
(ports/security/vncrypt), only implementing the main principles found
in GBDE.

I wouldn't bother.  4.x is nearing the EOL and you wouldn't get
much useful lifetime out of it.  It wouldn't be too much work to
do it however.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: FUD about CGD and GBDE

2005-03-02 Thread Poul-Henning Kamp
 of those before you continue.

(MD5 may or may not add any real strengt at this point, it's there
only as a bit blender.  If MD5 adds strength, it is multiplicative
not additive, because it applies to all of the 2^128 possibilities.)

At this point you will need to find storage space for the 2^128 *
16 bytes if you want to avoid recomputations.  I belive this means
that you'll need to store 2^32 bits on every hydrogen atom in the
universe (and then some.)

Once you break the second data sector open, you can start to weed
some of the 2^128 possibilities out, but it will be a long time
before you get that down to a manageable number because only
impossible output codes from MD5 will prune it for you.

Of the 2^128 possibilities from the first sector and the 2^128
possibilities for the second data sector you broke, some, but not
all, will have shared values for some of the 16 bytes from the
masterkey.  Being identical is no guarantee that they have the same
index in the masterkey, but you can nontheless at this point start
to attack the toplevel MD5 based on that theory.  Getting
more like 10-20 sectors would improve your chances a lot but
then you need more storage space.

Attacking from the top side means breaking the 512 bytes which
protect one of the master keys.  Given that there are four of them,
I will conceeede to 508 bits of work.

If we cut that in half just on general principles of being very
very conservative, we get down to where the 256 bit version of CGD
is if we ignore the CGD's little issue with key reuse.

So, if I were attacking GBDE, what I would do would be to look for
the superblock which is ~trivial to recognise and then walk down
the directory chain until I found the files/directories that I
want.

I'll give you the headstart that you know where the superblock is.

So you brute force the superblock. (2^128).

From that you can find the relative sector position of the inode
table (we totally ignore the uncertainty of the for keysectors
interleaving here).

Then you bruteforce the first inode block (2^128)

Then you bruteforce the first sector of the root directory (2^128)
and you are lucky and find the file right there.

You're even more lucky that the file has an inode in the same inode
sector you already handled.

And now you can start to bruteforce the data sector of the
file (of which there is only one) (2^128).

Now, you can't possibly be more lucky than this so that was
2^132 in the ultimate best case, downhill etc etc.

But let us be a bit realistic:  In case you get a false hit on the
superblock, when do you find out ?   The superblock is highly
structured, but it is not free of entropy, so this is a very
real risk.

The answer is that you will not be sure to find out until you have
broken the inode sector as well.

Well, what if that gives you false hit too ?

You get the picture.

With CGD if I suspect a hit I can just run fsck or mount it.  Fsck
may be slow when you are in a hurry, but it is a lot faster than
bruteforcing a 128 bit AES.

By the way: you keep comparing your AES256 version of CGD to
my AES128 version of GBDE, but at the same time you want me
to conceede that your 256 bit key is almost 1024 bits when
seen in the right light.

Lets us be fair and use a level ground:  Let us compare two 128 bit
version or two 256 bit versions.

Now, which algorithm is stronger ?

If a breakthrough makes AES128 very easy to crack, then you can
just crack the sectors individually, ignoring the rest of the
system.

Yes, but is would be Nsect easier with CGD as the entire thing
unravels at the first sector I break.  With GBDE I have to
attack them one by one.

And since it is almost a given that any attack on AES will be
statistical in nature, GBDE is much more resistant because
there is no two-way leverage on any of the algoritms.

So, I will still claim that you get points for your key handling
and flunk on the disk encryption side.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


GPIB: (very) basic ibfoo() library support

2005-02-06 Thread Poul-Henning Kamp

I needed to get some data out from some of my measurement equipment
for some $work and the hacked up userland /dev/io code I have used
for some time just didn't cut it.

So now we have the skeleton of ibfoo() support in FreeBSD.  We only
support PCIIA compatible adapters, and only a very small subset of
the ibfoo() functions are implemented, but the framework has been
written so that both other hardware and more functions should be a
matter of very little work.

Completing this is on my wish-list, but only on my todo list to the
extent that I need it for something else.  Help to get this more
complete is most welcome!

The wish list:

   o Complete more of the ibfoo() functions.
   o Support for different GPIB cards.
   o IEEE488.2 API implentation.
   o Manual pages.  (Either just reference the NI docs or rewrite it
 all to avoid copyright issues).

Reading material:

http://www.ni.com/pdf/manuals/321038g.pdf 
http://www.measurementcomputing.com/gpibhelp_Lib4881.html
http://linux-gpib.sourceforge.net/doc_html/index.html

The following program should be able to pull out the identity of
any SCPI instrument which happens to sit on PAD21 on your GPIB bus,

/* compile with:  cc -o ibtest ibtest.c -lgpib */

#include stdio.h
#include err.h
#include vis.h

#include gpib/gpib.h

int
main(int argc __unused, char **argv __unused)
{
int dmm;
unsigned char buf[100];
char vbuf[sizeof buf * 4];

/* DVM */
dmm = ibdev(0, 21, 0, T10s, 1, 0);
if (dmm  0)
errx(1, ibdev = %d\n, dmm);
ibwrt(dmm, *IDN?\r\n, 7);
ibrd(dmm, buf, sizeof buf - 1);
strvisx(vbuf, buf, ibcnt, VIS_WHITE | VIS_CSTYLE);
printf(%s\n, vbuf);
return (0);
}

Poul-Henning

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: GENERIC build broken

2005-01-24 Thread Poul-Henning Kamp

I have stared with fascination on this email for a full 30 minutes.

What could possibly be going on in the mind which came up with the
idea to take a five year old email, change Matt Dillons name and
repost it to our mailing list ?

In some weird way I feel honoured to have made such an impact on
this persons life that they would go to such bother.

And equally fascinating is the thought there is a town somewhere
in the world, where this derailed mind sneaks out at night to
spray-paint obscene things about me on walls I'll never see.

The mind boggles...


To a person sitting in darkness is a famous piece by Mark Twain
and I can never attempt to match it, but nontheless:


To a troll sitting in darkness:

I don't doubt you have reason to hate me, and I doubt you
will open your mind and listen to what I say to you, but
I will say it anyway, that is the least I can do in return
for your effort:

On a day where the only real newspaper in Europe broke the
news that we have a decade or less to do something about
global warming, while the rest of the press still dithers
about which catastrophe Bush is going to unleash first in
his second term.

One a day where 14 people were killed by snowstorms, in the
USA and thousands died from AIDS and you took the effort
to curse my name in the most circumspect manner.

There are so many other ways you could have used your time.

You could have written code or documentation, you could
have listened to music, played it yourself, read a good
book maybe ?

You could have offered an elderly neighbor help with
something, you could have given a helping hand in a local
shelter or played with the kids in the park, just for the
sheer heck of it.

You could have gone all the way and signed up to some cause
worthy of a whole life of dedication: religion, emergency
response, medicine, education or the toughest of all,
parenthood.

You did not.

You mananged this day to locate a five year old email from
me to Matt Dillon, and for some reason which we cannot even
imagine the depth off, it upset you a LOT.

And your anger gave you energy.

And the energy inpired you to do something!

You would right this wrong!

And you came up with the most diabolic scheme to transpose
all of the anger and frustration away from your own miserable
life and onto me, by changing Matt Dillons name and repost
it to our mailing lists.

I can almost tell how good that felt for you.

You are forgiven,

Poul-Henning




In message [EMAIL PROTECTED], Saul Bloom writes:
tries to use them directly.  PXE sounds cool, but coolness doesn't
count unless all the motherboard manufacturers start using it.

Saul,

Please try to do as Mike says, it would save a lot of time and windmills
if you would check the facts rather than keep arguing your unfounded
dogma.

--
Poul-Henning Kamp FreeBSD coreteam member
[EMAIL PROTECTED]   Real hackers run -current on their laptop.

   Go fly a kite, Poul.  I'm not interesting in listening to your spouting
   off any more.

 -Saul
 Saul Bloom 
 [EMAIL PROTECTED]


-
This e-mail was sent using a CentralPets WebMail account
Get yours at: http://mail.centralpets.com



___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: My project wish-list for the next 12 months

2004-12-02 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Scott Long writes:
All,

I know that I said last month that we were going to stop promising
specific features for the next major release.  However, I'd like to
throw out a list of things that would be really nice to have in the
future, whether its 6.0 or 7.0 or whatever.
[...]
1.  Keyboard multiplexer.  [...]
2.  New installer.  [...]
3.  Native PCI Express support.  [...]
4.  Journaled filesystem.  [...]
5.  Clustered FS support.  [...]
6.  Overhaul CAM, add iSCSI.  [...]

I agree on all of the above but I think we also need to have things
on the list that doesn't take super hackers, and architectural
reviews.

So let us add the following points which I think are just as, if
not more important for FreeBSDs future:

7. More people writing FreeBSD related articles for online and
   traditional media.

   If you have never written a piece about FreeBSD, how about sending
   something to your local IT trade rag ?   Heck, even your local
   paper will probably run it if you send them a piece.


8. More people stomping for FreeBSD in universities and schools.

   Have you actually offered the science/IT teachers at the local
   hi-school to pop around and give a lesson on this open source
   phenomena to their pupils ?

   Or call your local college and ask if they need a teaching
   assistant for their evening courses in IT ?


9. A band of happy 1st line reponders dealing with PRs etc.

   We're getting better at this, but one way to really gain users
   is to help them when they need it most: right when they begin.


10. A more dynamic and interesting www.freebsd.org frontpage.

Come on, at least we should be able to beat the Congressional
Record when it comes to being interesting.


11. More people attending BSDcon conferences.

Come to BSDcan2005! come to the next EuroBSDcon or AsiaBSDcon.

Or make your own mini conference!

Many Linux User groups would welcome you if you offered to give
a talk about FreeBSD on one of their evenings.


12. Research/Coding grants (3/6/12 months) from the FreeBSD Foundation
and other deep(er) pockets to help some of the heavy lifting happen.

We're not only in it for the money, but money surely helps.


And I'd like to stress that none of the above requires you to get
permission from the core team, just go out and do it!

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: sysctl hacks

2004-08-21 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Alfred Perlstein writes:
I'm doing some work that requires that I have a sysctl structure
be passed around, but inside that structure are several pointers I
may need to dereference.

Basically:

struct mysysctldata {
    (data here)
   void *moredata;
   size_t morelen;
};

What is the proper way of sysctl'ing IN the data from moredata?

I need to make a copy of the sysctl req, but... I'm not sure what
to initialize the 'lock' member to.

Just use the SYSCTL_IN() and ..._OUT() functions.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


[TEST] rewritten floppy driver

2004-08-20 Thread Poul-Henning Kamp

Please help test this rewrite of the floppy driver so it can qualify
to get into 5.3.

Can the PC98 coders please try to create a fdc_cbus.c so we can use
the same driver on PC98 ?

If anybody wants to play with 2.88M floppies, I belive I found at
least one bug which would have prevented it from working with the
old driver.

If you get into trouble with this driver, please set:
sysctl debug.fdc.debugflags=255
and recreate the problem, then send me the stuff it prints.

Thanks!

Poul-Henning

phk 2004-08-20 15:14:25 UTC

  FreeBSD src repository

  Modified files:
sys/sys  fdcio.h 
usr.sbin/fdcontrol   fdcontrol.c 
usr.sbin/fdformatfdformat.c 
usr.sbin/fdread  fdutil.c 
sys/dev/fdc  fdc.c fdc_acpi.c fdc_isa.c fdc_pccard.c 
 fdcvar.h 
  Log:
  Rewrite of the floppy driver to make it MPsafe  GEOM friendly:

[...]

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: [patch] Raw sockets in jails

2004-04-20 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Christian S.J. Peron w
rites:

Although RAW sockets can be used when specifying the source
address of packets (defeating one of the aspects of the jail)
some people may find it usefull to use utilities like ping(8)
or traceroute(8) from inside jails.

Enclosed is a patch I have written which gives you the option
of allowing prison-root to create raw sockets inside the prison,
so that programs various network debugging programs like ping
and traceroute etc can be used.

This patch will create the security.jail.allow_raw_sockets sysctl
MIB. I would appriciate any feed-back from testers

   See PR #:
   http://www.freebsd.org/cgi/query-pr.cgi?pr=65800

Could you take a peek and see how hard it would be to enforce source-IP
compliance with the jail restriction ?

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: [patch] Raw sockets in jails

2004-04-20 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Christian S.J. Peron w
rites:
Poul/group

The following patch makes raw sockets comply with prison IP addresses.
Some tools such as traceroute(8) may require that the prison IP address
be specified on the command line. I.E.

   traceroute -s prison ip dest address

Otherwise it might fail.

How does traceroute and ping normally determine which source address 
to use ?   Can't we use that mechanism to default them to the right thing ?

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Fundraising for FreeBSD development.

2004-04-08 Thread Poul-Henning Kamp

http://people.freebsd.org/~phk/funding.html

Poul-Henning

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Fundraising for FreeBSD development.

2004-04-08 Thread Poul-Henning Kamp
In message [EMAIL PROTECTED], Dan Langille writes:
On Thu, 8 Apr 2004, Poul-Henning Kamp wrote:


 http://people.freebsd.org/~phk/funding.html

typo :(An before any of you get an

Should be And, not An.

Fixed, thanks!

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


  1   2   3   4   5   6   7   >