For some reason mouse pointers dont seem to roam very well between three locations.
I too have had this problem, the only way I could find round it was to use remote desktop to remote into the virtual machine itself directly, either that or go to and setup a free account and use that to login to the virtual machine directly.