The motivation for the observer-moment concept is that it is intended to capture the bare minimum that we know to be true about the world. We don't know that our pasts are real. They could be imagined, synthesized, or faked. We may have been created one second ago and be destroyed one second in the future.
To start with the idea that we are observers, with a given history and past timeline, is to assume more than is in evidence. For a valid philosophical inquiry, we need to distinguish what we know from what we assume. All we know is the present moment. We assume a history to explain it, but we must at least consider the possibility that the history is wrong. The program I outlined at the start of this thread provides an in-principle way of calculating how much contribution "fake" versions of an observer-moment (such as brains in vats, or living in The Matrix) make versus "real" versions (where conventional reality is as it seems). Of course it is not tractable in practice; you'd have to simulate every possible universe and see which ones instantiated a particular OM. But the point is that we must consider the possibility that our pasts are not real. And in truth, scientific experiments have shown that many of our memories are partially incorrect or even entirely fabricated. All that we know is the present moment. It is the raw content of our experience as observers and it is what we must explain. Hal Finney