On Thu, Aug 11, 2016 at 6:37 PM, Warren Young <w...@etr-usa.com> wrote:
> On Aug 11, 2016, at 3:19 PM, Scott Robison <sc...@casaderobison.com> > wrote: > > > > I think you guys are just talking past each other. > > Well, at least one of us isn’t communicating clearly, that’s certain. I > just don’t yet know if it’s me, him, or both of us. :) > I'm just glad I never have problems expressing myself klearly. > > Windows versions that > > support PAE have the Address Windowing Extensions (AWE) which allows a > > single process to access more than 4 GiB total in a single process > > From what I can see, AWE and PAE are orthogonal, but AWE without PAE > doesn’t let you get beyond 4 GiB in a single process on IA-32. > Correct. AWE depends on PAE to enable windowed access into a memory space > 4 GiB. I've never used it so I don't know how much utility it might have outside PAE. https://en.wikipedia.org/wiki/Address_Windowing_Extensions probably gives a better summary that I could hope to. > But — and this may be where Keith was trying to go — a 32-bit app running > on a 64-bit OS doesn’t need PAE because the host system does support more > than 4 GiB of virtual memory. I have yet to see anything that says that > AWE on 64-bit Windows couldn’t give a 32-bit app access to some of the VM > beyond 4 GiB, even on consumer versions of Windows. > What I assumed was being said was that "sqlite.exe could access more than 2 GiB even on a 32 bit OS thanks to PAE" which is a true statement. I very well may not have read something carefully enough and missed something though. > If so, that’s what I was trying to get at with my request that he give a > reference to the specific technology he’s talking about, instead of > describing it in prose. > > > That being said, I don't think it is a reasonable or practical thing to > > expect of cross platform source code like SQLite to use such a platform > > specific API. > > Couldn’t it be abstracted behind a layer that used mmap() + tmpfs to pull > off a similar trick on other 64-bit OSes? (I’m not talking about > 4 GiB > on 32-bit OSes here.) > Perhaps, but I was only addressing the > 4 GiB on a 32-bit OS scenario. It was a great solution for apps that needed to process a lot of data in memory at one time back before AMD64 was a thing. http://stackoverflow.com/questions/1709243/how-to-use-more-than-3-gb-in-a-process-on-32-bit-pae-enabled-linux-app suggests that a 32-bit Linux system could use mmap to do the equivalent bit of functionality. > It’d be a lot of work just to avoid rebuilding for 64-bit, but maybe it > would be an interesting project for someone. Like a master’s university > project, maybe. > At first I thought to myself that a custom memory allocator for SQLite could do this, but the real problem would be once a pointer is given to SQLite, it is expected that pointer will be valid until disposed of in some way. It would take a lot of effort to make SQLite access pointers indirectly so that the virtual address window could be set to the proper physical memory range for every memory access. It would be returning to a segment:offset scheme that I certainly do not miss. Certainly a valuable tool for heavy processes that need to run on 32-bit PAE hardware with > 4 GiB of addressable ram. Anyone want to start work on SQLHeavy? ;) > Still, it looks like we’re on the cusp of all the major OSes moving to > 64-bit-only, so a lot of work made here might be obsolete soon. OS X and > RHEL have already made the leap. Ubuntu and Microsoft both threatened to > do this recently, but both backed down after user outcry. > Right. PAE & AWE (and related interfaces on other OSes) were a great solution from 1994 until the end of the 32-bit server era. They might still have some application, but with every passing day their utility diminishes, I expect. -- Scott Robison _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
