I brought up this idea with Lars Bak (HotSpot architect at Sun back then) at a conference some years back when Sun introduced the HotSpot VM. The argument back then was that a program mays not execute in the same pattern every time and the optimization made previously may no longer apply.
True, but if you saved the entire state of the JVM memory on disk (an JVM 'hibernation'?) then you could just start from where you left, instruction pointer included.
Not sure how hard it is to write, but doesn't sound like a bad idea to me.
(wasn't this was EMACS was doing with the 'core dump trick'?)
-- Stefano.