No, I think that can't happen, but it would be interesting to bisect what the guest is doing exactly when this happens. The guest should not move the "next command" pointer before if has written the command entirely, this should be enough to guard against executing a partial command. Unless there's another timing issue somewhere obviously.
Well, the guest driver is essentially the one in X.Org git tree here: http://cgit.freedesktop.org/xorg/driver/xf86-video-vmware/tree/src Looking at e.g. vmwareSendSVGACmdUpdate and vmwareWriteWordToFIFO in vmware.c, the command seems to be inserted into the FIFO one value at a time. Now, is the whole sequence somehow atomic wrt the host FIFO access or not?
Hmm, I don't know about the guest.. would be good to check too, but it should be only fixable in the guest if it is so.
Is the FIFO protocol documented somewhere? Like what kind of atomicity is expected?
I'm not sure if it's likely that the FIFO is really getting full? Most guest software will not write too many commands without knowing that the previous content has appeared on the screen, right?
Yes, I agree. That's more of a theoretical issue.
The FIFO can run at any moment but everything else stops until all the commands currently in the FIFO have been executed. vmware_fifo_run is called by the UI update which in turn is called from the main select() loop. Guest code is also executed in that loop.
"At any time" as in between guest calls to vmwareWriteWordToFIFO? Or not? It seems to me that the GUI is updated from a timer, but can it go off at any time?