I found it. It's in the MPC8272 family errata. When setting pipeline depth to one everything works. Can you Alex confirm this works also for you? This errata should probably be included in the Linux kernel, since it really depends on it. Right now I set this from the boot loader, but perhaps head.S in the kernel would be the just as good.
Part I System Interface Unit (SIU) Errata SIU18: ARTRY assertion when using pipeline depth of 0. Devices: MPC8272, MPC8271, MPC8248, MPC8247 Description: Internal (60x) slave maintains a pipeline depth of zero by asserting AACK only after TA. When ARTRY is asserted, the 60x bus access is terminated and TA is not asserted. The internal (60x) slave does not assert AACK, since TA was not asserted. Workaround: Use a pipeline depth of one (BCR[PLDP]=0) for applications that require memory coherency. Fix Plan: No fix plan at this time.
