Re: max phy mem known working with FreeBSD 4.x

2002-11-06 Thread Terry Lambert
Nate Lawson wrote:
> On Tue, 5 Nov 2002, Terry Lambert wrote:
> > As far as PCI goes (or anything they publish, for that matter), the
> > MindShare books are very, very good.  But for the particular question
> > of how much physical address space is eaten, you really have to go to
> > the chipset spec. sheets to get the right answer these days.  8-(.
> 
> The PCI book is good.  The others less so.  Bottom of the barrel is "plug
> and play arch" -- terrible.  Amazon's reviews are pretty close to reality
> IMO.

I actually really like the "Protected Mode Software Architecture", the
"EISA", and the "ISA" books.  The "Cardbus" and "Firewire" titles are
also useful.

Sorry to contradict you, but the "Plug and Play" is sufficient to let
someone write "PnP OS" code, by manually scanning devices in the OS,
instead of trusting the BIOS, which is good enough for me (I had an
old box that had a built in bus mouse on IRQ 12 that wasn't in the
BIOS; a PnP OS got this right; non-PnP OS's always failed to do the
right thing, and reassigned IRQ 12 to a conflicting device (the second
disk controller, on that particular box).

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message



Re: max phy mem known working with FreeBSD 4.x

2002-11-06 Thread Nate Lawson
On Tue, 5 Nov 2002, Terry Lambert wrote:
> As far as PCI goes (or anything they publish, for that matter), the
> MindShare books are very, very good.  But for the particular question
> of how much physical address space is eaten, you really have to go to
> the chipset spec. sheets to get the right answer these days.  8-(.

The PCI book is good.  The others less so.  Bottom of the barrel is "plug
and play arch" -- terrible.  Amazon's reviews are pretty close to reality
IMO.

-Nate


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message



Re: max phy mem known working with FreeBSD 4.x

2002-11-05 Thread Terry Lambert
"M. Warner Losh" wrote:
> In message: <[EMAIL PROTECTED]>
> Terry Lambert <[EMAIL PROTECTED]> writes:
> : The CDROM and a full set of the specifications will run you US$1500,
> : plus US$15 shipping in the US, or US$40 shipping, international.
> :
> : The price goes from $1500 to $75, if you can order through a company
> : which is already a member of the PCI SIG.
> 
> The cdrom was $50 when I ordered it with only the PCI 2.2 spec on
> paper.  Now that 3.0 is out, looks like they have jacked up the price
> again.  Looks like it costs a lot more now.  The mindshare PCI book
> has a good explaination.

I paid $150 in the 2.x era, but I got the full boxed set, with
the PCI-PCI bridge specifications, etc..

I think the thing that's bloating it now is that they have the
PCI-X and some other stuff that no one will really use much.  I've
already spent a little over 2 feet of shelf space, just in Microsoft
MSDN CDROM's and Windows SDK's and DDK's (~300 CDROMs).  I expect
the printed versions of the PCI specifications would probably come
in around a foot now, if they are hard bound, when before they were
around 4 inches wide.

I've noticed that most people who sell standards sell them cheap to
start, and then jump the price up after they get adopted as standards,
thinking they have a captive market.

One I can recommend is the ECMA standards CDROM; the ECMA people seem
to actually be interested in people complying with their standards,
so there was no cost for the CDROM, as long as you have a business
address to get it sent to...

As far as PCI goes (or anything they publish, for that matter), the
MindShare books are very, very good.  But for the particular question
of how much physical address space is eaten, you really have to go to
the chipset spec. sheets to get the right answer these days.  8-(.

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message



Re: max phy mem known working with FreeBSD 4.x

2002-11-05 Thread M. Warner Losh
In message: <[EMAIL PROTECTED]>
Terry Lambert <[EMAIL PROTECTED]> writes:
: The CDROM and a full set of the specifications will run you US$1500,
: plus US$15 shipping in the US, or US$40 shipping, international.
: 
: The price goes from $1500 to $75, if you can order through a company
: which is already a member of the PCI SIG.

The cdrom was $50 when I ordered it with only the PCI 2.2 spec on
paper.  Now that 3.0 is out, looks like they have jacked up the price
again.  Looks like it costs a lot more now.  The mindshare PCI book
has a good explaination.

Warner

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message



Re: max phy mem known working with FreeBSD 4.x

2002-11-05 Thread Terry Lambert
Dinesh Nair wrote:
> On Tue, 5 Nov 2002, Richard Sharpe wrote:
> > Well, we have 4GB in a 4.6.2 system, and I think that we ran 4.3 on those
> > systems for a while.
> >
> > However, you lose anywhere between 128M and 512M because of the PCI
> > address space.
> 
> what PCI address space ? could someone explain about this loss ?
> curiousity beckons. :)

Short answer


Memory on devices on the PCI bus is mappes into the physical memory
address space, and so "covers up" RAM in the window in which it is
mapped.  Good chipsets will only map in memory that actually exists
for devices which are actually installed.  Bad chipsets will reserve
the entire window, and therefore take a much larger chunk out.

Long answer
---

See http://www.pcisig.com/specifications/ordering_information

The CDROM and a full set of the specifications will run you US$1500,
plus US$15 shipping in the US, or US$40 shipping, international.

The price goes from $1500 to $75, if you can order through a company
which is already a member of the PCI SIG.

Your company can join the PCI SIG for a US$3000 per year fee, if it
is not already a member.

See http://www.pcisig.com/membership/join_pci_sig/new_member_forms/
for membership information.

FWIW, I got my copy when I was an employeed of a member company; it's
the way to go, if you can do it.

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message



Re: max phy mem known working with FreeBSD 4.x

2002-11-05 Thread Dinesh Nair

On Tue, 5 Nov 2002, Richard Sharpe wrote:

> Well, we have 4GB in a 4.6.2 system, and I think that we ran 4.3 on those
> systems for a while.
>
> However, you lose anywhere between 128M and 512M because of the PCI
> address space.

what PCI address space ? could someone explain about this loss ?
curiousity beckons. :)

Regards,   /\_/\   "All dogs go to heaven."
[EMAIL PROTECTED](0 0)http://www.alphaque.com/
+==oOO--(_)--OOo==+
| for a in past present future; do|
|   for b in clients employers associates relatives neighbours pets; do   |
|   echo "The opinions here in no way reflect the opinions of my $a $b."  |
| done; done  |
+=+


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message



Re: max phy mem known working with FreeBSD 4.x

2002-11-05 Thread Terry Lambert
PS:

Richard Sharpe wrote:
> Well, the P4 does have an address extension that allows addressing of up
> to 64GB, EPA or something like that.

It has segments, too.  We don't use those, either.  8-).

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message



Re: max phy mem known working with FreeBSD 4.x

2002-11-05 Thread Terry Lambert
Richard Sharpe wrote:
> Well, the P4 does have an address extension that allows addressing of up
> to 64GB, EPA or something like that.

This is useless to discuss in the context of 4.3, or even 4.x,
unless Paul Saab is doing his work in 4.7, and it makes it
into 4.8.  Given that we are pretty much forced to do all new
work 0n -current in order to get committed (or everyone would
be working on 4.x instead), it'll end up a forward port with
no back-port (expect it after the 5.x release, if Yahoo gives
it back to FreeBSD).


> However, it requires some work. Would be nice in large Samba servers,
> though, to be able to cache enormous amounts of file data :-)

It would be useless for this purpose.

Specifically, you would need to guarantee a locality of reference
for clients that you could not guarantee, for this to be useful at
all.  Mostly, you would be swapping the PSE window around all the
time, shooting down TLB's and cache lines, and you would burn up
any benefits of having the data in main RAM instead of in RAM on
the disk controller or the disk itself, very quickly.

In addition, you would not be able to use "sendfile" or similar
interfaces, and you would not be able to use scatter/gather DMA,
unless you knew for certain that the card doing the DMA had 64 bit
access to physical RAM.  Otherwise, you would need to copy all data
to be DMA'ed to/from any physical RAM above 4G to bounce buffers
below 4G.  The extra copying might as well be from the controller
RAM or the RAM on the disk itself, than from/to RAM above 4G.

The 2G limit on Alpha is basically the same issue, where the data
needs to be bounced through lower memory in order to be accessible
to the hardware DMA engines on the other side of the PCI bus (hence
the need to convert the drivers to "bus space" to be able to use
more than 2G in an Alpha box).

Basically, the extra RAM makes assumptions about code that make us
all realize that hardware designers don't really ask software
designers why they ask for things like the ability to access more
than 4G of RAM.  8-).

If you want to discuss this further, please read the archives of
the last 3 times the issue was discusees, on the -current and
-arch lists.

No matter how you look at it, though, unless you pay some schmuck
to do the work, you're never going to see this in 4.3 anyway, so
you might as well move the discussion over to the -current list.

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message



Re: max phy mem known working with FreeBSD 4.x

2002-11-05 Thread Terry Lambert
Julian Elischer wrote:
> The hardware changes to do > 32 bit physical addresses
> include a redefinition of how page tables and page directories are layed
> out and to be able to use it we'd have to define
> and turn on code for that differnt mode. iIt has not yet been written.
> It wouldn't be hard to make a system that ONLY ran in that mode
> but one that can run on new systems withthat mode and also run on old
> 486 machines is much harder.

It's easier to do it with PSE-36 than PAE, actually.  The double
indirect is also easy to setup, and then not use, if it comes to
that, and 4K vs. 8K allocated in assembly is pretty much nothing.

The big issue is the 2M vs. 4M "jumbo pages": you have to expect
that, and FreeBSD, if PSE is present, will try to set up a 4M page
for the kernel.

What's really needed is a "remap4kto4m()" function that gets used
for that; there's a trick you have to pull with CR3, CR0, and CR4
to make it not break, though, because there's a CPU bug, and it
leaves you with a chicken-and-egg problem, otherwise.  This would
work for 2M pages, as well (just use two of them).

If you understand the code, you should be able to write it in
about two weeks (including the CPU bug workaround), once you
have your head around things.

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message



Re: max phy mem known working with FreeBSD 4.x

2002-11-05 Thread Richard Sharpe
On Tue, 5 Nov 2002, Terry Lambert wrote:

> Matt wrote:
> > > > Anyone knows the max physical mem that can be used  with FreeBSD4.3?
> [ ... ]
> > any chance going more than 4G?
> 
> Sure, if you want to install it to warm things up.
> 
> No, if you want to access it; access is limited to 4G, because
> that's 32 bits of address space, and your machine is a 32 bit
> machine.

Well, the P4 does have an address extension that allows addressing of up 
to 64GB, EPA or something like that.

However, it requires some work. Would be nice in large Samba servers, 
though, to be able to cache enormous amounts of file data :-)

Regards
-
Richard Sharpe, [EMAIL PROTECTED], [EMAIL PROTECTED], 
[EMAIL PROTECTED], http://www.richardsharpe.com


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message



Re: max phy mem known working with FreeBSD 4.x

2002-11-05 Thread Terry Lambert
Matt wrote:
> > > Anyone knows the max physical mem that can be used  with FreeBSD4.3?
[ ... ]
> any chance going more than 4G?

Sure, if you want to install it to warm things up.

No, if you want to access it; access is limited to 4G, because
that's 32 bits of address space, and your machine is a 32 bit
machine.

Unless you run IA-64 or SPARC-64, or run Alpha, and fix several
of the drivers to use "bus space" properly.

Of course, for these platforms, you wouldn't be running 4.3.

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message



Re: max phy mem known working with FreeBSD 4.x

2002-11-04 Thread Julian Elischer


On Mon, 4 Nov 2002, Matt wrote:

> any chance going more than 4G?

Not yet. (on x86)

The hardware changes to do > 32 bit physical addresses 
include a redefinition of how page tables and page directories are layed
out and to be able to use it we'd have to define
and turn on code for that differnt mode. iIt has not yet been written.
It wouldn't be hard to make a system that ONLY ran in that mode
but one that can run on new systems withthat mode and also run on old
486 machines is much harder.


Julian





To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message



Re: max phy mem known working with FreeBSD 4.x

2002-11-04 Thread Terry Lambert
Matt wrote:
> Anyone knows the max physical mem that can be used  with FreeBSD4.3?

2G on Alpha.

4G on Intel, if you tune your kernel and modify your KVA size to 3G.

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message



Re: max phy mem known working with FreeBSD 4.x

2002-11-04 Thread Matt
any chance going more than 4G?

Best Regards


Matt
http://www.xgforce.com/product.html

The Next Generation Server Clustering and
Clustered Enterprise SQL database, and
Firewall/VPN Solutions.

- Original Message -
From: "Richard Sharpe" <[EMAIL PROTECTED]>
To: "Matt" <[EMAIL PROTECTED]>
Cc: <[EMAIL PROTECTED]>
Sent: Monday, November 04, 2002 10:50 PM
Subject: Re: max phy mem known working with FreeBSD 4.x


> On Mon, 4 Nov 2002, Matt wrote:
>
> > Anyone knows the max physical mem that can be used  with FreeBSD4.3?
>
> Well, we have 4GB in a 4.6.2 system, and I think that we ran 4.3 on those
> systems for a while.
>
> However, you lose anywhere between 128M and 512M because of the PCI
> address space.
>
> Regards
> -
> Richard Sharpe, [EMAIL PROTECTED], [EMAIL PROTECTED],
> [EMAIL PROTECTED], http://www.richardsharpe.com
>


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message



Re: max phy mem known working with FreeBSD 4.x

2002-11-04 Thread Richard Sharpe
On Mon, 4 Nov 2002, Matt wrote:

> Anyone knows the max physical mem that can be used  with FreeBSD4.3?

Well, we have 4GB in a 4.6.2 system, and I think that we ran 4.3 on those 
systems for a while.

However, you lose anywhere between 128M and 512M because of the PCI 
address space.

Regards
-
Richard Sharpe, [EMAIL PROTECTED], [EMAIL PROTECTED], 
[EMAIL PROTECTED], http://www.richardsharpe.com


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message



max phy mem known working with FreeBSD 4.x

2002-11-04 Thread Matt
Anyone knows the max physical mem that can be used  with FreeBSD4.3?

Thanks in advance


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message