Re: ioremap_nocache problem?
Hi, On Thu, Jan 25, 2001 at 11:53:01AM -0600, Timur Tabi wrote: > > > As in an MMIO aperture? If its MMIO on the bus you should be able to > > just call ioremap with the bus address. By nature of it being outside > > of real ram, it should automatically be uncached (unless you've set an > > MTRR over that region saying otherwise). > > It's not outside of real RAM. The device is inside real RAM (it sits on the > DIMM itself), but I need to poke through the entire 4GB range to see how it > responds. kmap() is designed for that, not ioremap(). Is it absolutely essential that your mapping is uncached? If so, extending kmap() to support kmap_nocache() would seem to make a lot more sense than using ioremap(): kmap is there for temporarily poking around in high memory, whereas ioremap is really intended to be used for persistent maps. --Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Hi, On Thu, Jan 25, 2001 at 10:49:50AM -0600, Timur Tabi wrote: > > > set_bit(PG_reserved, >flags); > > ioremap(); > > ... > > iounmap(); > > clear_bit(PG_reserved, >flags); > > The problem with this is that between the ioremap and iounmap, the page is > reserved. What happens if that page belongs to some disk buffer or user > process, and some other process tries to free it. Won't that cause a problem? It depends on how it is being used, but yes, it is likely to --- page reference counts won't be updated properly on reserved pages, for example. Why on earth do you want to do ioremap of something like a page cache page, though? That is _not_ what ioremap is designed for! --Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Hi, On Thu, Jan 25, 2001 at 09:56:32AM -0600, Timur Tabi wrote: > > ioremap*() is only supposed to be used on IO regions or reserved > > pages. If you haven't marked the pages as reserved, then iounmap will > > do the wrong thing, so it's up to you to reserve the pages. > > Au contraire! > > I mark the page as reserved when I ioremap() it. However, if I leave it marked > reserved, then iounmap() will not unmap it. It certainly should do, and the 2.4 source certainly looks as if it does. At least on i386, iounmap calls vfree, which ends up in free_area_pte(), which will unconditionally clear the pte (hence unmapping the page). Cheers, Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Hi, On Thu, Jan 25, 2001 at 09:56:32AM -0600, Timur Tabi wrote: ioremap*() is only supposed to be used on IO regions or reserved pages. If you haven't marked the pages as reserved, then iounmap will do the wrong thing, so it's up to you to reserve the pages. Au contraire! I mark the page as reserved when I ioremap() it. However, if I leave it marked reserved, then iounmap() will not unmap it. It certainly should do, and the 2.4 source certainly looks as if it does. At least on i386, iounmap calls vfree, which ends up in free_area_pte(), which will unconditionally clear the pte (hence unmapping the page). Cheers, Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Hi, On Thu, Jan 25, 2001 at 10:49:50AM -0600, Timur Tabi wrote: set_bit(PG_reserved, page-flags); ioremap(); ... iounmap(); clear_bit(PG_reserved, page-flags); The problem with this is that between the ioremap and iounmap, the page is reserved. What happens if that page belongs to some disk buffer or user process, and some other process tries to free it. Won't that cause a problem? It depends on how it is being used, but yes, it is likely to --- page reference counts won't be updated properly on reserved pages, for example. Why on earth do you want to do ioremap of something like a page cache page, though? That is _not_ what ioremap is designed for! --Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Hi, On Thu, Jan 25, 2001 at 11:53:01AM -0600, Timur Tabi wrote: As in an MMIO aperture? If its MMIO on the bus you should be able to just call ioremap with the bus address. By nature of it being outside of real ram, it should automatically be uncached (unless you've set an MTRR over that region saying otherwise). It's not outside of real RAM. The device is inside real RAM (it sits on the DIMM itself), but I need to poke through the entire 4GB range to see how it responds. kmap() is designed for that, not ioremap(). Is it absolutely essential that your mapping is uncached? If so, extending kmap() to support kmap_nocache() would seem to make a lot more sense than using ioremap(): kmap is there for temporarily poking around in high memory, whereas ioremap is really intended to be used for persistent maps. --Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Timur Tabi wrote: > ** Reply to message from Jeff Hartmann <[EMAIL PROTECTED]> on Thu, 25 Jan > 2001 11:13:35 -0700 > > > >> You need to have your driver in the early bootup process then. When >> memory is being detected (but before the free lists are created.), you >> can set your page as being reserved. > > > But doesn't this mean that my driver has to be built as part of the kernel? > The end-user won't have the source code, so he won't be able to compile it, only > link it. As it stands now, our driver is a binary that can be shipped > separately. Sorry, this is the only way to do it properly. Binary kernel drivers are intensely evil. ;) Open the driver and you have no problems. You also do know that binary kernel drivers mean you'll be chasing every kernel release, having to provide several different flavors of your binary depending on the users kernel configuration. It also means that when kernel interfaces change, people won't be nice and change your code over to the new interfaces for you. For instance if a function depreciates, your code might be automatically moved to use the replacement function if your in the standard kernel. If your a binary module, you have to do all that maintaining yourself. (There are several other reasons to have open kernel modules. I won't go into the entire argument, since that could take all day.) You might be able to get away with making detection of this page open, and keep the rest of the driver closed. However that is something for Linus to decided, not I. I believe he doesn't like putting in hooks in the kernel for binary modules. Since all you really want to do is reserve the page during early bootup, perhaps he might let you get away with it. Not my call though. -Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
** Reply to message from Jeff Hartmann <[EMAIL PROTECTED]> on Thu, 25 Jan 2001 11:13:35 -0700 > You need to have your driver in the early bootup process then. When > memory is being detected (but before the free lists are created.), you > can set your page as being reserved. But doesn't this mean that my driver has to be built as part of the kernel? The end-user won't have the source code, so he won't be able to compile it, only link it. As it stands now, our driver is a binary that can be shipped separately. -- Timur Tabi - [EMAIL PROTECTED] Interactive Silicon - http://www.interactivesi.com When replying to a mailing-list message, please direct the reply to the mailing list only. Don't send another copy to me. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Timur Tabi wrote: > ** Reply to message from Jeff Hartmann <[EMAIL PROTECTED]> on Thu, 25 Jan > 2001 10:47:13 -0700 > > > >> As in an MMIO aperture? If its MMIO on the bus you should be able to >> just call ioremap with the bus address. By nature of it being outside >> of real ram, it should automatically be uncached (unless you've set an >> MTRR over that region saying otherwise). > > > It's not outside of real RAM. The device is inside real RAM (it sits on the > DIMM itself), but I need to poke through the entire 4GB range to see how it > responds. > > >> Look at the functions agp_generic_free_gatt_table and >> agp_generic_create_gatt_table in agpgart_be.c (drivers/char/agp). They >> do the ioremap_nocache on real ram for the GATT/GART table. > > > Unfortunately, the memory they remap is allocated: > > table = (char *) __get_free_pages(GFP_KERNEL, page_order); > > ... > > CACHE_FLUSH(); > agp_bridge.gatt_table = ioremap_nocache(virt_to_phys(table), (PAGE_SIZE * (1 << > page_order))); > CACHE_FLUSH(); > > I've searched high and low for examples of code that does what I do, and I > can't find any. You need to have your driver in the early bootup process then. When memory is being detected (but before the free lists are created.), you can set your page as being reserved. Then the kernel will leave it alone when it creates its free lists. This does mean that this driver can not be a module and that it must run at least part of itself in the early bootup process. I don't remember exactly where you need to do this, you might try looking at arch/i386/mm/init.c (Just an educated guess.) -Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
** Reply to message from Jeff Hartmann <[EMAIL PROTECTED]> on Thu, 25 Jan 2001 10:47:13 -0700 > As in an MMIO aperture? If its MMIO on the bus you should be able to > just call ioremap with the bus address. By nature of it being outside > of real ram, it should automatically be uncached (unless you've set an > MTRR over that region saying otherwise). It's not outside of real RAM. The device is inside real RAM (it sits on the DIMM itself), but I need to poke through the entire 4GB range to see how it responds. > Look at the functions agp_generic_free_gatt_table and > agp_generic_create_gatt_table in agpgart_be.c (drivers/char/agp). They > do the ioremap_nocache on real ram for the GATT/GART table. Unfortunately, the memory they remap is allocated: table = (char *) __get_free_pages(GFP_KERNEL, page_order); ... CACHE_FLUSH(); agp_bridge.gatt_table = ioremap_nocache(virt_to_phys(table), (PAGE_SIZE * (1 << page_order))); CACHE_FLUSH(); I've searched high and low for examples of code that does what I do, and I can't find any. -- Timur Tabi - [EMAIL PROTECTED] Interactive Silicon - http://www.interactivesi.com When replying to a mailing-list message, please direct the reply to the mailing list only. Don't send another copy to me. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Timur Tabi wrote:
> ** Reply to message from Jeff Hartmann <[EMAIL PROTECTED]> on Thu, 25 Jan
> 2001 10:04:47 -0700
>
>
>
>>> The problem with this is that between the ioremap and iounmap, the page is
>>> reserved. What happens if that page belongs to some disk buffer or user
>>> process, and some other process tries to free it. Won't that cause a problem?
>>
>> The page can't belong to some other process/kernel component. You own
>> the page if you allocated it.
>
>
> Ok, my mistake. I wasn't paying attention to the "get_free_pages" call. My
> problem is that I'm ioremap'ing someone else's page, but my hardware sits on the
> memory bus disguised as real memory, and so I need to poke around the 4GB space
> trying to find it.
As in an MMIO aperture? If its MMIO on the bus you should be able to
just call ioremap with the bus address. By nature of it being outside
of real ram, it should automatically be uncached (unless you've set an
MTRR over that region saying otherwise).
>
>
>
>> (I was the one who added support to
>> the kernel to ioremap real ram, trust me.)
>
>
> I really appreciate that feature, because it helps me a lot. Any
> recommendations on how I can do what I do without causing any problems? Right
> now, my driver never calls iounmap on memory that's in real RAM, even when it
> exits. Fortunately, the driver isn't supposed to exit, so all it does is waste
> a few KB of virtual memory.
Look at the functions agp_generic_free_gatt_table and
agp_generic_create_gatt_table in agpgart_be.c (drivers/char/agp). They
do the ioremap_nocache on real ram for the GATT/GART table. Heres some
quick pseudo code as well.
I_want_a_no_cached_page() {
alloc a page
reserve the page
flush every cpu's cache
ioremap_nocache the page
}
I_want_to_free_a_no_cached_page() {
iounmap the page
unreserve the page
free the page
}
-Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
** Reply to message from Jeff Hartmann <[EMAIL PROTECTED]> on Thu, 25 Jan 2001 10:04:47 -0700 > > The problem with this is that between the ioremap and iounmap, the page is > > reserved. What happens if that page belongs to some disk buffer or user > > process, and some other process tries to free it. Won't that cause a problem? > > The page can't belong to some other process/kernel component. You own > the page if you allocated it. Ok, my mistake. I wasn't paying attention to the "get_free_pages" call. My problem is that I'm ioremap'ing someone else's page, but my hardware sits on the memory bus disguised as real memory, and so I need to poke around the 4GB space trying to find it. > (I was the one who added support to > the kernel to ioremap real ram, trust me.) I really appreciate that feature, because it helps me a lot. Any recommendations on how I can do what I do without causing any problems? Right now, my driver never calls iounmap on memory that's in real RAM, even when it exits. Fortunately, the driver isn't supposed to exit, so all it does is waste a few KB of virtual memory. -- Timur Tabi - [EMAIL PROTECTED] Interactive Silicon - http://www.interactivesi.com When replying to a mailing-list message, please direct the reply to the mailing list only. Don't send another copy to me. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Timur Tabi wrote: > ** Reply to message from Roman Zippel <[EMAIL PROTECTED]> on Thu, 25 Jan 2001 > 17:44:51 +0100 > > > >> set_bit(PG_reserved, >flags); >> ioremap(); >> ... >> iounmap(); >> clear_bit(PG_reserved, >flags); > > > The problem with this is that between the ioremap and iounmap, the page is > reserved. What happens if that page belongs to some disk buffer or user > process, and some other process tries to free it. Won't that cause a problem? The page can't belong to some other process/kernel component. You own the page if you allocated it. The kernel will only muck with memory you allocated if its GFP_HIGHMEM, or under certain circumstances if you map it into a user process (There are several rules here and I won't go into them, look at the DRM mmap setup for a start if your interested.) This is the correct ordering of the calls (I was the one who added support to the kernel to ioremap real ram, trust me.) -Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
** Reply to message from Roman Zippel <[EMAIL PROTECTED]> on Thu, 25 Jan 2001 17:44:51 +0100 > set_bit(PG_reserved, >flags); > ioremap(); > ... > iounmap(); > clear_bit(PG_reserved, >flags); The problem with this is that between the ioremap and iounmap, the page is reserved. What happens if that page belongs to some disk buffer or user process, and some other process tries to free it. Won't that cause a problem? -- Timur Tabi - [EMAIL PROTECTED] Interactive Silicon - http://www.interactivesi.com When replying to a mailing-list message, please direct the reply to the mailing list only. Don't send another copy to me. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Hi,
Timur Tabi wrote:
> I mark the page as reserved when I ioremap() it. However, if I leave it marked
> reserved, then iounmap() will not unmap it. If I mark it "unreserved" (i.e.
> reset the reserved bit), then iounmap will unmap it, but it will decrement the
> page counter to -1 and the whole system will crash soon thereafter.
>
> I've been asking about this problem for months, but no one has bothered to help
> me out.
The order is important:
get_free_page();
set_bit(PG_reserved, >flags);
ioremap();
...
iounmap();
clear_bit(PG_reserved, >flags);
free_page();
Alternatively something like this should also be possible:
get_free_page();
ioremap();
...
iounmap();
nopage() {
...
atomic_inc(>count);
return page;
}
But I never tried this version, so I can't guarantee anything. :)
bye, Roman
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
** Reply to message from "Stephen C. Tweedie" <[EMAIL PROTECTED]> on Thu, 25 Jan 2001 15:16:55 + > ioremap*() is only supposed to be used on IO regions or reserved > pages. If you haven't marked the pages as reserved, then iounmap will > do the wrong thing, so it's up to you to reserve the pages. Au contraire! I mark the page as reserved when I ioremap() it. However, if I leave it marked reserved, then iounmap() will not unmap it. If I mark it "unreserved" (i.e. reset the reserved bit), then iounmap will unmap it, but it will decrement the page counter to -1 and the whole system will crash soon thereafter. I've been asking about this problem for months, but no one has bothered to help me out. -- Timur Tabi - [EMAIL PROTECTED] Interactive Silicon - http://www.interactivesi.com When replying to a mailing-list message, please direct the reply to the mailing list only. Don't send another copy to me. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Hi, On Tue, Jan 23, 2001 at 10:53:51AM -0600, Timur Tabi wrote: > > My problem is that it's very easy to map memory with ioremap_nocache, but if > you use iounmap() the un-map it, the entire system will crash. No one has been > able to explain that one to me, either. ioremap*() is only supposed to be used on IO regions or reserved pages. If you haven't marked the pages as reserved, then iounmap will do the wrong thing, so it's up to you to reserve the pages. Cheers, Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Hi, On Tue, Jan 23, 2001 at 10:53:51AM -0600, Timur Tabi wrote: My problem is that it's very easy to map memory with ioremap_nocache, but if you use iounmap() the un-map it, the entire system will crash. No one has been able to explain that one to me, either. ioremap*() is only supposed to be used on IO regions or reserved pages. If you haven't marked the pages as reserved, then iounmap will do the wrong thing, so it's up to you to reserve the pages. Cheers, Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
** Reply to message from "Stephen C. Tweedie" [EMAIL PROTECTED] on Thu, 25 Jan 2001 15:16:55 + ioremap*() is only supposed to be used on IO regions or reserved pages. If you haven't marked the pages as reserved, then iounmap will do the wrong thing, so it's up to you to reserve the pages. Au contraire! I mark the page as reserved when I ioremap() it. However, if I leave it marked reserved, then iounmap() will not unmap it. If I mark it "unreserved" (i.e. reset the reserved bit), then iounmap will unmap it, but it will decrement the page counter to -1 and the whole system will crash soon thereafter. I've been asking about this problem for months, but no one has bothered to help me out. -- Timur Tabi - [EMAIL PROTECTED] Interactive Silicon - http://www.interactivesi.com When replying to a mailing-list message, please direct the reply to the mailing list only. Don't send another copy to me. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Hi,
Timur Tabi wrote:
I mark the page as reserved when I ioremap() it. However, if I leave it marked
reserved, then iounmap() will not unmap it. If I mark it "unreserved" (i.e.
reset the reserved bit), then iounmap will unmap it, but it will decrement the
page counter to -1 and the whole system will crash soon thereafter.
I've been asking about this problem for months, but no one has bothered to help
me out.
The order is important:
get_free_page();
set_bit(PG_reserved, page-flags);
ioremap();
...
iounmap();
clear_bit(PG_reserved, page-flags);
free_page();
Alternatively something like this should also be possible:
get_free_page();
ioremap();
...
iounmap();
nopage() {
...
atomic_inc(page-count);
return page;
}
But I never tried this version, so I can't guarantee anything. :)
bye, Roman
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Timur Tabi wrote: ** Reply to message from Roman Zippel [EMAIL PROTECTED] on Thu, 25 Jan 2001 17:44:51 +0100 set_bit(PG_reserved, page-flags); ioremap(); ... iounmap(); clear_bit(PG_reserved, page-flags); The problem with this is that between the ioremap and iounmap, the page is reserved. What happens if that page belongs to some disk buffer or user process, and some other process tries to free it. Won't that cause a problem? The page can't belong to some other process/kernel component. You own the page if you allocated it. The kernel will only muck with memory you allocated if its GFP_HIGHMEM, or under certain circumstances if you map it into a user process (There are several rules here and I won't go into them, look at the DRM mmap setup for a start if your interested.) This is the correct ordering of the calls (I was the one who added support to the kernel to ioremap real ram, trust me.) -Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
** Reply to message from Jeff Hartmann [EMAIL PROTECTED] on Thu, 25 Jan 2001 10:04:47 -0700 The problem with this is that between the ioremap and iounmap, the page is reserved. What happens if that page belongs to some disk buffer or user process, and some other process tries to free it. Won't that cause a problem? The page can't belong to some other process/kernel component. You own the page if you allocated it. Ok, my mistake. I wasn't paying attention to the "get_free_pages" call. My problem is that I'm ioremap'ing someone else's page, but my hardware sits on the memory bus disguised as real memory, and so I need to poke around the 4GB space trying to find it. (I was the one who added support to the kernel to ioremap real ram, trust me.) I really appreciate that feature, because it helps me a lot. Any recommendations on how I can do what I do without causing any problems? Right now, my driver never calls iounmap on memory that's in real RAM, even when it exits. Fortunately, the driver isn't supposed to exit, so all it does is waste a few KB of virtual memory. -- Timur Tabi - [EMAIL PROTECTED] Interactive Silicon - http://www.interactivesi.com When replying to a mailing-list message, please direct the reply to the mailing list only. Don't send another copy to me. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Timur Tabi wrote:
** Reply to message from Jeff Hartmann [EMAIL PROTECTED] on Thu, 25 Jan
2001 10:04:47 -0700
The problem with this is that between the ioremap and iounmap, the page is
reserved. What happens if that page belongs to some disk buffer or user
process, and some other process tries to free it. Won't that cause a problem?
The page can't belong to some other process/kernel component. You own
the page if you allocated it.
Ok, my mistake. I wasn't paying attention to the "get_free_pages" call. My
problem is that I'm ioremap'ing someone else's page, but my hardware sits on the
memory bus disguised as real memory, and so I need to poke around the 4GB space
trying to find it.
As in an MMIO aperture? If its MMIO on the bus you should be able to
just call ioremap with the bus address. By nature of it being outside
of real ram, it should automatically be uncached (unless you've set an
MTRR over that region saying otherwise).
(I was the one who added support to
the kernel to ioremap real ram, trust me.)
I really appreciate that feature, because it helps me a lot. Any
recommendations on how I can do what I do without causing any problems? Right
now, my driver never calls iounmap on memory that's in real RAM, even when it
exits. Fortunately, the driver isn't supposed to exit, so all it does is waste
a few KB of virtual memory.
Look at the functions agp_generic_free_gatt_table and
agp_generic_create_gatt_table in agpgart_be.c (drivers/char/agp). They
do the ioremap_nocache on real ram for the GATT/GART table. Heres some
quick pseudo code as well.
I_want_a_no_cached_page() {
alloc a page
reserve the page
flush every cpu's cache
ioremap_nocache the page
}
I_want_to_free_a_no_cached_page() {
iounmap the page
unreserve the page
free the page
}
-Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Timur Tabi wrote: ** Reply to message from Jeff Hartmann [EMAIL PROTECTED] on Thu, 25 Jan 2001 10:47:13 -0700 As in an MMIO aperture? If its MMIO on the bus you should be able to just call ioremap with the bus address. By nature of it being outside of real ram, it should automatically be uncached (unless you've set an MTRR over that region saying otherwise). It's not outside of real RAM. The device is inside real RAM (it sits on the DIMM itself), but I need to poke through the entire 4GB range to see how it responds. Look at the functions agp_generic_free_gatt_table and agp_generic_create_gatt_table in agpgart_be.c (drivers/char/agp). They do the ioremap_nocache on real ram for the GATT/GART table. Unfortunately, the memory they remap is allocated: table = (char *) __get_free_pages(GFP_KERNEL, page_order); ... CACHE_FLUSH(); agp_bridge.gatt_table = ioremap_nocache(virt_to_phys(table), (PAGE_SIZE * (1 page_order))); CACHE_FLUSH(); I've searched high and low for examples of code that does what I do, and I can't find any. You need to have your driver in the early bootup process then. When memory is being detected (but before the free lists are created.), you can set your page as being reserved. Then the kernel will leave it alone when it creates its free lists. This does mean that this driver can not be a module and that it must run at least part of itself in the early bootup process. I don't remember exactly where you need to do this, you might try looking at arch/i386/mm/init.c (Just an educated guess.) -Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
** Reply to message from Jeff Hartmann [EMAIL PROTECTED] on Thu, 25 Jan 2001 11:13:35 -0700 You need to have your driver in the early bootup process then. When memory is being detected (but before the free lists are created.), you can set your page as being reserved. But doesn't this mean that my driver has to be built as part of the kernel? The end-user won't have the source code, so he won't be able to compile it, only link it. As it stands now, our driver is a binary that can be shipped separately. -- Timur Tabi - [EMAIL PROTECTED] Interactive Silicon - http://www.interactivesi.com When replying to a mailing-list message, please direct the reply to the mailing list only. Don't send another copy to me. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Timur Tabi wrote: ** Reply to message from Jeff Hartmann [EMAIL PROTECTED] on Thu, 25 Jan 2001 11:13:35 -0700 You need to have your driver in the early bootup process then. When memory is being detected (but before the free lists are created.), you can set your page as being reserved. But doesn't this mean that my driver has to be built as part of the kernel? The end-user won't have the source code, so he won't be able to compile it, only link it. As it stands now, our driver is a binary that can be shipped separately. Sorry, this is the only way to do it properly. Binary kernel drivers are intensely evil. ;) Open the driver and you have no problems. You also do know that binary kernel drivers mean you'll be chasing every kernel release, having to provide several different flavors of your binary depending on the users kernel configuration. It also means that when kernel interfaces change, people won't be nice and change your code over to the new interfaces for you. For instance if a function depreciates, your code might be automatically moved to use the replacement function if your in the standard kernel. If your a binary module, you have to do all that maintaining yourself. (There are several other reasons to have open kernel modules. I won't go into the entire argument, since that could take all day.) You might be able to get away with making detection of this page open, and keep the rest of the driver closed. However that is something for Linus to decided, not I. I believe he doesn't like putting in hooks in the kernel for binary modules. Since all you really want to do is reserve the page during early bootup, perhaps he might let you get away with it. Not my call though. -Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Timur Tabi <[EMAIL PROTECTED]> writes: > ** Reply to message from David Wragg <[EMAIL PROTECTED]> on 24 Jan 2001 > 00:50:20 + > > (x86 processors with PAT and IA64 can set write-combining through > >page flags. x86 processors with MTRRs but not PAT would need a more > >elaborate implementation for write-combining.) > > What is PAT? I desperately need to figure out how to turn on write > combining on a per-page level. I thought I had to use MTRRs, but now > you're saying I can use this "PAT" thing instead. Please explain! PAT is basically the MTRR memory types on a per-page basis. It adds a new flag bit to the x86 page table entry, then that bit together with the PCD and PWT bits is used to do a look-up in an 8-entry table that gives the effective memory type (the table is set through an MSR). All the details are in the Intel x86 manual, volume 3 http://developer.intel.com/design/pentium4/manuals/> (at the end of chapter 9). Quite a lot of the x86 CPUs out there support PAT: The PII except the first couple of models, the Celeron except the first model, the PIII, all PII and PIII Xeons, the P4, all AMD K7 models. I'm guessing, but I suspect that the majority of x86 CPUs supporting write combining in any form that have been made also support PAT. I wish Intel had put PAT in the PPro, rather than messing everyone around with MTRRs (MTRRs are good for BIOS writers, but a pain for everyone else). David Wragg - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
** Reply to message from David Wragg <[EMAIL PROTECTED]> on 24 Jan 2001 00:50:20 + > (x86 processors with PAT and IA64 can set write-combining through page > flags. x86 processors with MTRRs but not PAT would need a more > elaborate implementation for write-combining.) What is PAT? I desperately need to figure out how to turn on write combining on a per-page level. I thought I had to use MTRRs, but now you're saying I can use this "PAT" thing instead. Please explain! -- Timur Tabi - [EMAIL PROTECTED] Interactive Silicon - http://www.interactivesi.com When replying to a mailing-list message, please direct the reply to the mailing list only. Don't send another copy to me. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
** Reply to message from David Wragg [EMAIL PROTECTED] on 24 Jan 2001 00:50:20 + (x86 processors with PAT and IA64 can set write-combining through page flags. x86 processors with MTRRs but not PAT would need a more elaborate implementation for write-combining.) What is PAT? I desperately need to figure out how to turn on write combining on a per-page level. I thought I had to use MTRRs, but now you're saying I can use this "PAT" thing instead. Please explain! -- Timur Tabi - [EMAIL PROTECTED] Interactive Silicon - http://www.interactivesi.com When replying to a mailing-list message, please direct the reply to the mailing list only. Don't send another copy to me. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Timur Tabi [EMAIL PROTECTED] writes: ** Reply to message from David Wragg [EMAIL PROTECTED] on 24 Jan 2001 00:50:20 + (x86 processors with PAT and IA64 can set write-combining through page flags. x86 processors with MTRRs but not PAT would need a more elaborate implementation for write-combining.) What is PAT? I desperately need to figure out how to turn on write combining on a per-page level. I thought I had to use MTRRs, but now you're saying I can use this "PAT" thing instead. Please explain! PAT is basically the MTRR memory types on a per-page basis. It adds a new flag bit to the x86 page table entry, then that bit together with the PCD and PWT bits is used to do a look-up in an 8-entry table that gives the effective memory type (the table is set through an MSR). All the details are in the Intel x86 manual, volume 3 URL:http://developer.intel.com/design/pentium4/manuals/ (at the end of chapter 9). Quite a lot of the x86 CPUs out there support PAT: The PII except the first couple of models, the Celeron except the first model, the PIII, all PII and PIII Xeons, the P4, all AMD K7 models. I'm guessing, but I suspect that the majority of x86 CPUs supporting write combining in any form that have been made also support PAT. I wish Intel had put PAT in the PPro, rather than messing everyone around with MTRRs (MTRRs are good for BIOS writers, but a pain for everyone else). David Wragg - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Timur Tabi <[EMAIL PROTECTED]> writes: > ** Reply to message from Roman Zippel <[EMAIL PROTECTED]> on > Tue, 23 Jan 2001 19:12:36 +0100 (MET) > > ioremap creates a new mapping that shouldn't interfere with MTRR, > >whereas you can map a MTRR mapped area into userspace. But I'm not > >sure if it's correct that no flag is set for boot_cpu_data.x86 <= > >3... > > I was under the impression that the "don't cache" bit that > ioremap_nocache sets overrides any MTRR. Nope. There's a table explaining how page flags and MTRRs interact in the Intel x86 manual, volume 3 (it's in section 9.5.1 "Precedence of Cache Controls" in the fairly recent edition I have here). For example, with PCD set, PWT clear, and the MTRRs saying WC, the effective memory type is WC. In addition, there's a note saying this may change in future models. So you have to set PCD | PWT if you want to get uncached in all cases. David Wragg - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
From: David Wragg <[EMAIL PROTECTED]> Gcc: nnfolder:mail.sent --text follows this line-- Roman Zippel <[EMAIL PROTECTED]> writes: > On Tue, 23 Jan 2001, Mark Mokryn wrote: > > ioremap_nocache does the following: > > return __ioremap(offset, size, _PAGE_PCD); You have a point. It would be nice if ioremap took a argument indicating the desired memory type -- normal, nocache, write-through, write-combining, etc. Then it could look in an architecture-specific table to get the appropriate page flags for that type. (x86 processors with PAT and IA64 can set write-combining through page flags. x86 processors with MTRRs but not PAT would need a more elaborate implementation for write-combining.) > > > > However, in drivers/char/mem.c (2.4.0), we see the following: > > > > /* On PPro and successors, PCD alone doesn't always mean > > uncached because of interactions with the MTRRs. PCD | PWT > > means definitely uncached. */ > > if (boot_cpu_data.x86 > 3) > > prot |= _PAGE_PCD | _PAGE_PWT; > > > > Does this mean ioremap_nocache() may not do the job? > > ioremap creates a new mapping that shouldn't interfere with MTRR, whereas > you can map a MTRR mapped area into userspace. But I'm not sure if it's > correct that no flag is set for boot_cpu_data.x86 <= 3... The boot_cpu_data.x86 > 3 test is there because the 386 doesn't have PWT. David Wragg - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
** Reply to message from Roman Zippel <[EMAIL PROTECTED]> on Tue, 23 Jan 2001 19:12:36 +0100 (MET) > ioremap creates a new mapping that shouldn't interfere with MTRR, whereas > you can map a MTRR mapped area into userspace. But I'm not sure if it's > correct that no flag is set for boot_cpu_data.x86 <= 3... I was under the impression that the "don't cache" bit that ioremap_nocache sets overrides any MTRR. -- Timur Tabi - [EMAIL PROTECTED] Interactive Silicon - http://www.interactivesi.com When replying to a mailing-list message, please direct the reply to the mailing list only. Don't send another copy to me. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Hi, On Tue, 23 Jan 2001, Mark Mokryn wrote: > ioremap_nocache does the following: > return __ioremap(offset, size, _PAGE_PCD); > > However, in drivers/char/mem.c (2.4.0), we see the following: > > /* On PPro and successors, PCD alone doesn't always mean > uncached because of interactions with the MTRRs. PCD | PWT > means definitely uncached. */ > if (boot_cpu_data.x86 > 3) > prot |= _PAGE_PCD | _PAGE_PWT; > > Does this mean ioremap_nocache() may not do the job? ioremap creates a new mapping that shouldn't interfere with MTRR, whereas you can map a MTRR mapped area into userspace. But I'm not sure if it's correct that no flag is set for boot_cpu_data.x86 <= 3... bye, Roman - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
** Reply to message from Mark Mokryn <[EMAIL PROTECTED]> on Tue, 23 Jan 2001 12:30:00 +0200 > Does this mean ioremap_nocache() may not do the job? Good luck trying to get an answer. I've been asking questions on ioremap for months, but no one's ever been able to tell me anything. According to the comments, mem.c provides /dev/zero support, whatever that is. It doesn't appear to be connected to ioremap in any way, so I understand your question. I can tell you that I have written a driver that depends on ioremap_nocache, and it does work, so it appears that ioremap_nocache is doing something. My problem is that it's very easy to map memory with ioremap_nocache, but if you use iounmap() the un-map it, the entire system will crash. No one has been able to explain that one to me, either. -- Timur Tabi - [EMAIL PROTECTED] Interactive Silicon - http://www.interactivesi.com When replying to a mailing-list message, please direct the reply to the mailing list only. Don't send another copy to me. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
** Reply to message from Mark Mokryn [EMAIL PROTECTED] on Tue, 23 Jan 2001 12:30:00 +0200 Does this mean ioremap_nocache() may not do the job? Good luck trying to get an answer. I've been asking questions on ioremap for months, but no one's ever been able to tell me anything. According to the comments, mem.c provides /dev/zero support, whatever that is. It doesn't appear to be connected to ioremap in any way, so I understand your question. I can tell you that I have written a driver that depends on ioremap_nocache, and it does work, so it appears that ioremap_nocache is doing something. My problem is that it's very easy to map memory with ioremap_nocache, but if you use iounmap() the un-map it, the entire system will crash. No one has been able to explain that one to me, either. -- Timur Tabi - [EMAIL PROTECTED] Interactive Silicon - http://www.interactivesi.com When replying to a mailing-list message, please direct the reply to the mailing list only. Don't send another copy to me. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Hi, On Tue, 23 Jan 2001, Mark Mokryn wrote: ioremap_nocache does the following: return __ioremap(offset, size, _PAGE_PCD); However, in drivers/char/mem.c (2.4.0), we see the following: /* On PPro and successors, PCD alone doesn't always mean uncached because of interactions with the MTRRs. PCD | PWT means definitely uncached. */ if (boot_cpu_data.x86 3) prot |= _PAGE_PCD | _PAGE_PWT; Does this mean ioremap_nocache() may not do the job? ioremap creates a new mapping that shouldn't interfere with MTRR, whereas you can map a MTRR mapped area into userspace. But I'm not sure if it's correct that no flag is set for boot_cpu_data.x86 = 3... bye, Roman - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
** Reply to message from Roman Zippel [EMAIL PROTECTED] on Tue, 23 Jan 2001 19:12:36 +0100 (MET) ioremap creates a new mapping that shouldn't interfere with MTRR, whereas you can map a MTRR mapped area into userspace. But I'm not sure if it's correct that no flag is set for boot_cpu_data.x86 = 3... I was under the impression that the "don't cache" bit that ioremap_nocache sets overrides any MTRR. -- Timur Tabi - [EMAIL PROTECTED] Interactive Silicon - http://www.interactivesi.com When replying to a mailing-list message, please direct the reply to the mailing list only. Don't send another copy to me. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
From: David Wragg [EMAIL PROTECTED] Gcc: nnfolder:mail.sent --text follows this line-- Roman Zippel [EMAIL PROTECTED] writes: On Tue, 23 Jan 2001, Mark Mokryn wrote: ioremap_nocache does the following: return __ioremap(offset, size, _PAGE_PCD); You have a point. It would be nice if ioremap took a argument indicating the desired memory type -- normal, nocache, write-through, write-combining, etc. Then it could look in an architecture-specific table to get the appropriate page flags for that type. (x86 processors with PAT and IA64 can set write-combining through page flags. x86 processors with MTRRs but not PAT would need a more elaborate implementation for write-combining.) However, in drivers/char/mem.c (2.4.0), we see the following: /* On PPro and successors, PCD alone doesn't always mean uncached because of interactions with the MTRRs. PCD | PWT means definitely uncached. */ if (boot_cpu_data.x86 3) prot |= _PAGE_PCD | _PAGE_PWT; Does this mean ioremap_nocache() may not do the job? ioremap creates a new mapping that shouldn't interfere with MTRR, whereas you can map a MTRR mapped area into userspace. But I'm not sure if it's correct that no flag is set for boot_cpu_data.x86 = 3... The boot_cpu_data.x86 3 test is there because the 386 doesn't have PWT. David Wragg - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: ioremap_nocache problem?
Timur Tabi [EMAIL PROTECTED] writes: ** Reply to message from Roman Zippel [EMAIL PROTECTED] on Tue, 23 Jan 2001 19:12:36 +0100 (MET) ioremap creates a new mapping that shouldn't interfere with MTRR, whereas you can map a MTRR mapped area into userspace. But I'm not sure if it's correct that no flag is set for boot_cpu_data.x86 = 3... I was under the impression that the "don't cache" bit that ioremap_nocache sets overrides any MTRR. Nope. There's a table explaining how page flags and MTRRs interact in the Intel x86 manual, volume 3 (it's in section 9.5.1 "Precedence of Cache Controls" in the fairly recent edition I have here). For example, with PCD set, PWT clear, and the MTRRs saying WC, the effective memory type is WC. In addition, there's a note saying this may change in future models. So you have to set PCD | PWT if you want to get uncached in all cases. David Wragg - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
