It is working now, at least no black screen. Mouse can move outof vm window seamless, and copy/paste function is working between Host and osol Guest in vm. Unity function is not working, but I did not find the module after installing, maybe this is not available now. I will test other functions later.
It was in wks 6.5.1, pls get the sreps below: 1. Install vmtools. 2. In /usr/X11/lib/modules/drivers, rename file 'rmware_drv.so' to 'vmware_drv.so.new', and rename 'vmware_drv.so.BeforeVMwareToolsInstall' to 'vmware_drv.so'. - File 'vmware_drv.so.BeforeVMwareToolsInstall' is the old video driver within osol, and the 'vmware_drv.so' before changing is the one in vmtools. I just changed the built-in one back. As I checked, osol has video driver for vmware already, so it is working only because of video driver back. -- This message posted from opensolaris.org