Re: MTRR failure revisited (nVidia) 8-STABLE/RELEASE
On Sat, 2009-12-12 at 20:08 -0800, Chris H wrote: On Sat, December 12, 2009 6:36 am, Robert Noland wrote: On Sat, 2009-12-12 at 03:47 -0800, Chris H wrote: Greetings, I brought this same error to the list back in May 2009. Under: failed to set mtrr: invalid argument. Well, I'm back using the same card: GeForce4 MX 440-SE - VideoRam 65536 - BusID PCI:1:3:0. The driver is different, I'm using: nvidia-driver-96.43.13 out of ports on a custom 8-STABLE kernel. Xorg starts up, and produces a desktop. But it's dog slow UPDATE: Disabling HAL /greatly/ increased performance eg; hal_enable=YES -- hal_enable=NO in /etc/rc.conf More specifically, response times are now closer to what one would anticipate/ expect now that HAL has been dis-abled in rc.conf. , and the nvidia driver emits the following error: NVIDIA: failed to set MTRR 0xf000, 0M (write-combining) several times. I understand John Baldwin provided some invaluable help some time ago: http://lists.freebsd.org/pipermail/freebsd-hackers/2006-June/016995.html and I was wondering if anyone has gained any further insight with these cards, and how to better interface them in BSD. Last I spoke on the topic, I was informed that the memory was basically untouchable - or perhaps in other words; can't be manipulated. Has this changed? Surely someone else has had to deal with this besides me. It seems crazy to spend a boat load of $$ on these high performers, and not be able to use them on a high performing OS - no? :) Sure, the one I'm working with now is legacy. But I have 3 near new, top of their line cards, and thus far it appears that if I ever hope to use them, I'll be forced to... hack, choke.. spin up a WIN CD. :( Thank you for all your time, consideration, and insight. Greetings Robert, and thank you for taking the time to respond. The mtrr issue has to do with the system / bios, not the graphics card. While I've not used the blob driver, the issue in Nouveau and other drm drivers is that on many systems if you run memcontrol list, you will see a line something like this: 0x0/0x1 BIOS write-back set-by-firmware active I see the following (condensed for brevity): 0x0/0x1 BIOS write-back fixed-base fixed-length set-by-firmware active 0x1/0x1-0x7/0x1 BIOS write-back fixed-base fixed-length set-by-firmware active 0x8/0x4000 BIOS-0x9c000/0x4000 write-back fixed-base fixed-length set-by-firmware active 0xa/0x4000-0xbc000/0x4000 BIOS uncacheable fixed-base fixed-length set-by-firmware active 0xc/0x1xc7000/0x1000 BIOS write-protect fixed-base fixed-length set-by-firmware active 0xc8000/0x1000-0xff000/0x1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active 0x0/0x4000 BIOS write-back set-by-firmware active The above entry is the one that causes setting write-combine MTRR to fail. 0xe000/0x2000 BIOS uncacheable set-by-firmware active While I could pull the BIOS out of it's socket after POST. I don't suppose I could read it's contents to file, and then allow manipulation of the regions currently off limits? This is more easily achieved in our MTRR code I expect, certainly than BIOS hacking all of the effected machines. Frankly, all of my more modern machines have this issue. This says that all of memory defaults to write-back. We aren't allowed to overlap write-combined on top of write-back, so the setting of mtrr fails. Isn't it /best/ to choose write-back, so as to mark the memory dirty? I /could/ choose write-ahead, or write-through. I've looked at ways to try to fix this in the past, but generally found it more practical to use PAT than try to override/fix bios behavior. Marius Nünnerich also mentioned this in a response to this thread. Would you be willing to share any additional information, based on your experiences using PAT? PAT and MTRR both allow for the setting of cache attributes for a range of memory. For MTRR, there are a certain number of variable range registers in a given CPU which allow the setting of global cache attributes for a given memory range. There are a number of rules about what memory types may overlap though, so having that entry that covers all of memory means that setting MTRR to anything other than uncached will always fail. PAT is less restrictive in this regard and a table exists in the intel documentation showing the effective cache method for various combinations of MTRR/PAT. The difficulty with PAT is that it is an attribute of the individual page mapping and so all mappings of the same physical region of memory need to have the same PAT type set. jhb@, alc@ and I (my contribution has mostly consisted of whining and testing) have recently (over the last year at least) been adding the ability to handle allowing userspace mappings of memory regions to be mapped with consistent PAT attributes. This was also
Re: MTRR failure revisited (nVidia) 8-STABLE/RELEASE
On Sun, December 13, 2009 6:04 am, Robert Noland wrote: On Sat, 2009-12-12 at 20:08 -0800, Chris H wrote: On Sat, December 12, 2009 6:36 am, Robert Noland wrote: On Sat, 2009-12-12 at 03:47 -0800, Chris H wrote: Greetings, I brought this same error to the list back in May 2009. Under: failed to set mtrr: invalid argument. Well, I'm back using the same card: GeForce4 MX 440-SE - VideoRam 65536 - BusID PCI:1:3:0. The driver is different, I'm using: nvidia-driver-96.43.13 out of ports on a custom 8-STABLE kernel. Xorg starts up, and produces a desktop. But it's dog slow UPDATE: Disabling HAL /greatly/ increased performance eg; hal_enable=YES -- hal_enable=NO in /etc/rc.conf More specifically, response times are now closer to what one would anticipate/ expect now that HAL has been dis-abled in rc.conf. , and the nvidia driver emits the following error: NVIDIA: failed to set MTRR 0xf000, 0M (write-combining) several times. I understand John Baldwin provided some invaluable help some time ago: http://lists.freebsd.org/pipermail/freebsd-hackers/2006-June/016995.html and I was wondering if anyone has gained any further insight with these cards, and how to better interface them in BSD. Last I spoke on the topic, I was informed that the memory was basically untouchable - or perhaps in other words; can't be manipulated. Has this changed? Surely someone else has had to deal with this besides me. It seems crazy to spend a boat load of $$ on these high performers, and not be able to use them on a high performing OS - no? :) Sure, the one I'm working with now is legacy. But I have 3 near new, top of their line cards, and thus far it appears that if I ever hope to use them, I'll be forced to... hack, choke.. spin up a WIN CD. :( Thank you for all your time, consideration, and insight. Greetings Robert, and thank you for taking the time to respond. The mtrr issue has to do with the system / bios, not the graphics card. While I've not used the blob driver, the issue in Nouveau and other drm drivers is that on many systems if you run memcontrol list, you will see a line something like this: 0x0/0x1 BIOS write-back set-by-firmware active I see the following (condensed for brevity): 0x0/0x1 BIOS write-back fixed-base fixed-length set-by-firmware active 0x1/0x1-0x7/0x1 BIOS write-back fixed-base fixed-length set-by-firmware active 0x8/0x4000 BIOS-0x9c000/0x4000 write-back fixed-base fixed-length set-by-firmware active 0xa/0x4000-0xbc000/0x4000 BIOS uncacheable fixed-base fixed-length set-by-firmware active 0xc/0x1xc7000/0x1000 BIOS write-protect fixed-base fixed-length set-by-firmware active 0xc8000/0x1000-0xff000/0x1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active 0x0/0x4000 BIOS write-back set-by-firmware active Hello Robert, and thank you for your thoughtful response. The above entry is the one that causes setting write-combine MTRR to fail. 0xe000/0x2000 BIOS uncacheable set-by-firmware active While I could pull the BIOS out of it's socket after POST. I don't suppose I could read it's contents to file, and then allow manipulation of the regions currently off limits? This is more easily achieved in our MTRR code I expect, certainly than BIOS hacking all of the effected machines. Frankly, all of my more modern machines have this issue. Ahhh, OK. This says that all of memory defaults to write-back. We aren't allowed to overlap write-combined on top of write-back, so the setting of mtrr fails. Isn't it /best/ to choose write-back, so as to mark the memory dirty? I /could/ choose write-ahead, or write-through. I've looked at ways to try to fix this in the past, but generally found it more practical to use PAT than try to override/fix bios behavior. Marius Nünnerich also mentioned this in a response to this thread. Would you be willing to share any additional information, based on your experiences using PAT? PAT and MTRR both allow for the setting of cache attributes for a range of memory. For MTRR, there are a certain number of variable range registers in a given CPU which allow the setting of global cache attributes for a given memory range. There are a number of rules about what memory types may overlap though, so having that entry that covers all of memory means that setting MTRR to anything other than uncached will always fail. PAT is less restrictive in this regard and a table exists in the intel documentation showing the effective cache method for various combinations of MTRR/PAT. The difficulty with PAT is that it is an attribute of the individual page mapping and so all mappings of the same physical region of memory need to have the same PAT type set. jhb@, alc@ and I (my contribution has mostly consisted of whining and testing) have recently (over the last year at least) been adding the ability to handle allowing userspace
Re: MTRR failure revisited (nVidia) 8-STABLE/RELEASE
On Sun, 2009-12-13 at 06:53 -0800, Chris H wrote: On Sun, December 13, 2009 6:04 am, Robert Noland wrote: On Sat, 2009-12-12 at 20:08 -0800, Chris H wrote: On Sat, December 12, 2009 6:36 am, Robert Noland wrote: On Sat, 2009-12-12 at 03:47 -0800, Chris H wrote: Greetings, I brought this same error to the list back in May 2009. Under: failed to set mtrr: invalid argument. Well, I'm back using the same card: GeForce4 MX 440-SE - VideoRam 65536 - BusID PCI:1:3:0. The driver is different, I'm using: nvidia-driver-96.43.13 out of ports on a custom 8-STABLE kernel. Xorg starts up, and produces a desktop. But it's dog slow UPDATE: Disabling HAL /greatly/ increased performance eg; hal_enable=YES -- hal_enable=NO in /etc/rc.conf More specifically, response times are now closer to what one would anticipate/ expect now that HAL has been dis-abled in rc.conf. , and the nvidia driver emits the following error: NVIDIA: failed to set MTRR 0xf000, 0M (write-combining) several times. I understand John Baldwin provided some invaluable help some time ago: http://lists.freebsd.org/pipermail/freebsd-hackers/2006-June/016995.html and I was wondering if anyone has gained any further insight with these cards, and how to better interface them in BSD. Last I spoke on the topic, I was informed that the memory was basically untouchable - or perhaps in other words; can't be manipulated. Has this changed? Surely someone else has had to deal with this besides me. It seems crazy to spend a boat load of $$ on these high performers, and not be able to use them on a high performing OS - no? :) Sure, the one I'm working with now is legacy. But I have 3 near new, top of their line cards, and thus far it appears that if I ever hope to use them, I'll be forced to... hack, choke.. spin up a WIN CD. :( Thank you for all your time, consideration, and insight. Greetings Robert, and thank you for taking the time to respond. The mtrr issue has to do with the system / bios, not the graphics card. While I've not used the blob driver, the issue in Nouveau and other drm drivers is that on many systems if you run memcontrol list, you will see a line something like this: 0x0/0x1 BIOS write-back set-by-firmware active I see the following (condensed for brevity): 0x0/0x1 BIOS write-back fixed-base fixed-length set-by-firmware active 0x1/0x1-0x7/0x1 BIOS write-back fixed-base fixed-length set-by-firmware active 0x8/0x4000 BIOS-0x9c000/0x4000 write-back fixed-base fixed-length set-by-firmware active 0xa/0x4000-0xbc000/0x4000 BIOS uncacheable fixed-base fixed-length set-by-firmware active 0xc/0x1xc7000/0x1000 BIOS write-protect fixed-base fixed-length set-by-firmware active 0xc8000/0x1000-0xff000/0x1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active 0x0/0x4000 BIOS write-back set-by-firmware active Hello Robert, and thank you for your thoughtful response. The above entry is the one that causes setting write-combine MTRR to fail. 0xe000/0x2000 BIOS uncacheable set-by-firmware active While I could pull the BIOS out of it's socket after POST. I don't suppose I could read it's contents to file, and then allow manipulation of the regions currently off limits? This is more easily achieved in our MTRR code I expect, certainly than BIOS hacking all of the effected machines. Frankly, all of my more modern machines have this issue. Ahhh, OK. This says that all of memory defaults to write-back. We aren't allowed to overlap write-combined on top of write-back, so the setting of mtrr fails. Isn't it /best/ to choose write-back, so as to mark the memory dirty? I /could/ choose write-ahead, or write-through. I've looked at ways to try to fix this in the past, but generally found it more practical to use PAT than try to override/fix bios behavior. Marius Nünnerich also mentioned this in a response to this thread. Would you be willing to share any additional information, based on your experiences using PAT? PAT and MTRR both allow for the setting of cache attributes for a range of memory. For MTRR, there are a certain number of variable range registers in a given CPU which allow the setting of global cache attributes for a given memory range. There are a number of rules about what memory types may overlap though, so having that entry that covers all of memory means that setting MTRR to anything other than uncached will always fail. PAT is less restrictive in this regard and a table exists in the intel documentation showing the effective cache method for various combinations of MTRR/PAT. The difficulty with PAT is that it is an attribute of the individual page mapping and so all mappings of the same physical region of memory need to have the same PAT type set.
Re: MTRR failure revisited (nVidia) 8-STABLE/RELEASE
On Sat, Dec 12, 2009 at 12:47, Chris H chr...@1command.com wrote: Greetings, I brought this same error to the list back in May 2009. Under: failed to set mtrr: invalid argument. Well, I'm back using the same card: GeForce4 MX 440-SE - VideoRam 65536 - BusID PCI:1:3:0. The driver is different, I'm using: nvidia-driver-96.43.13 out of ports on a custom 8-STABLE kernel. Xorg starts up, and produces a desktop. But it's dog slow, and the nvidia driver emits the following error: NVIDIA: failed to set MTRR 0xf000, 0M (write-combining) several times. I understand John Baldwin provided some invaluable help some time ago: http://lists.freebsd.org/pipermail/freebsd-hackers/2006-June/016995.html and I was wondering if anyone has gained any further insight with these cards, and how to better interface them in BSD. Last I spoke on the topic, I was informed that the memory was basically untouchable - or perhaps in other words; can't be manipulated. Has this changed? Surely someone else has had to deal with this besides me. It seems crazy to spend a boat load of $$ on these high performers, and not be able to use them on a high performing OS - no? :) Sure, the one I'm working with now is legacy. But I have 3 near new, top of their line cards, and thus far it appears that if I ever hope to use them, I'll be forced to... hack, choke.. spin up a WIN CD. :( Thank you for all your time, consideration, and insight. FreeBSD as of 8.0-RELEASE supports PAT which obsoletes MTRR. It was a requirement for the nvidia amd64 blob. So in the future it should all work fine if you have at least a card which is supported by the newest nvidia driver. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: MTRR failure revisited (nVidia) 8-STABLE/RELEASE
On Sat, 2009-12-12 at 03:47 -0800, Chris H wrote: Greetings, I brought this same error to the list back in May 2009. Under: failed to set mtrr: invalid argument. Well, I'm back using the same card: GeForce4 MX 440-SE - VideoRam 65536 - BusID PCI:1:3:0. The driver is different, I'm using: nvidia-driver-96.43.13 out of ports on a custom 8-STABLE kernel. Xorg starts up, and produces a desktop. But it's dog slow, and the nvidia driver emits the following error: NVIDIA: failed to set MTRR 0xf000, 0M (write-combining) several times. I understand John Baldwin provided some invaluable help some time ago: http://lists.freebsd.org/pipermail/freebsd-hackers/2006-June/016995.html and I was wondering if anyone has gained any further insight with these cards, and how to better interface them in BSD. Last I spoke on the topic, I was informed that the memory was basically untouchable - or perhaps in other words; can't be manipulated. Has this changed? Surely someone else has had to deal with this besides me. It seems crazy to spend a boat load of $$ on these high performers, and not be able to use them on a high performing OS - no? :) Sure, the one I'm working with now is legacy. But I have 3 near new, top of their line cards, and thus far it appears that if I ever hope to use them, I'll be forced to... hack, choke.. spin up a WIN CD. :( Thank you for all your time, consideration, and insight. The mtrr issue has to do with the system / bios, not the graphics card. While I've not used the blob driver, the issue in Nouveau and other drm drivers is that on many systems if you run memcontrol list, you will see a line something like this: 0x0/0x1 BIOS write-back set-by-firmware active This says that all of memory defaults to write-back. We aren't allowed to overlap write-combined on top of write-back, so the setting of mtrr fails. I've looked at ways to try to fix this in the past, but generally found it more practical to use PAT than try to override/fix bios behavior. I've been told that linux does apply some BIOS fixups to address this issue, which I might look into again, but I make no promises. There are also a very limited number of variable mtrr registers (7 on most hardware, iirc) for managing caching. robert. --Chris H ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org -- Robert Noland rnol...@freebsd.org FreeBSD ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: MTRR failure revisited (nVidia) 8-STABLE/RELEASE
On Sat, December 12, 2009 6:36 am, Robert Noland wrote: On Sat, 2009-12-12 at 03:47 -0800, Chris H wrote: Greetings, I brought this same error to the list back in May 2009. Under: failed to set mtrr: invalid argument. Well, I'm back using the same card: GeForce4 MX 440-SE - VideoRam 65536 - BusID PCI:1:3:0. The driver is different, I'm using: nvidia-driver-96.43.13 out of ports on a custom 8-STABLE kernel. Xorg starts up, and produces a desktop. But it's dog slow UPDATE: Disabling HAL /greatly/ increased performance eg; hal_enable=YES -- hal_enable=NO in /etc/rc.conf More specifically, response times are now closer to what one would anticipate/ expect now that HAL has been dis-abled in rc.conf. , and the nvidia driver emits the following error: NVIDIA: failed to set MTRR 0xf000, 0M (write-combining) several times. I understand John Baldwin provided some invaluable help some time ago: http://lists.freebsd.org/pipermail/freebsd-hackers/2006-June/016995.html and I was wondering if anyone has gained any further insight with these cards, and how to better interface them in BSD. Last I spoke on the topic, I was informed that the memory was basically untouchable - or perhaps in other words; can't be manipulated. Has this changed? Surely someone else has had to deal with this besides me. It seems crazy to spend a boat load of $$ on these high performers, and not be able to use them on a high performing OS - no? :) Sure, the one I'm working with now is legacy. But I have 3 near new, top of their line cards, and thus far it appears that if I ever hope to use them, I'll be forced to... hack, choke.. spin up a WIN CD. :( Thank you for all your time, consideration, and insight. Greetings Robert, and thank you for taking the time to respond. The mtrr issue has to do with the system / bios, not the graphics card. While I've not used the blob driver, the issue in Nouveau and other drm drivers is that on many systems if you run memcontrol list, you will see a line something like this: 0x0/0x1 BIOS write-back set-by-firmware active I see the following (condensed for brevity): 0x0/0x1 BIOS write-back fixed-base fixed-length set-by-firmware active 0x1/0x1-0x7/0x1 BIOS write-back fixed-base fixed-length set-by-firmware active 0x8/0x4000 BIOS-0x9c000/0x4000 write-back fixed-base fixed-length set-by-firmware active 0xa/0x4000-0xbc000/0x4000 BIOS uncacheable fixed-base fixed-length set-by-firmware active 0xc/0x1xc7000/0x1000 BIOS write-protect fixed-base fixed-length set-by-firmware active 0xc8000/0x1000-0xff000/0x1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active 0x0/0x4000 BIOS write-back set-by-firmware active 0xe000/0x2000 BIOS uncacheable set-by-firmware active While I could pull the BIOS out of it's socket after POST. I don't suppose I could read it's contents to file, and then allow manipulation of the regions currently off limits? This says that all of memory defaults to write-back. We aren't allowed to overlap write-combined on top of write-back, so the setting of mtrr fails. Isn't it /best/ to choose write-back, so as to mark the memory dirty? I /could/ choose write-ahead, or write-through. I've looked at ways to try to fix this in the past, but generally found it more practical to use PAT than try to override/fix bios behavior. Marius Nünnerich also mentioned this in a response to this thread. Would you be willing to share any additional information, based on your experiences using PAT? I've been told that linux does apply some BIOS fixups to address this issue, which I might look into again, but I make no promises. Is there anything I could do that would help you in this regard? There are also a very limited number of variable mtrr registers (7 on most hardware, iirc) for managing caching. Thank you again for taking the time to respond. --Chris H robert. --Chris H ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org -- Robert Noland rnol...@freebsd.org FreeBSD ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
