I'm looking at the instructions at http://wiki.lyx.org/Windows/PortableInstallation
I have put USBTeX on my memory stick, and copied C:\Program Files\Lyx16 to that stick. But the installation instructions say to copy some tl-portable files from the TexLive DVD, which I have not used. If anyone has got this combination working (USBTeX with Lyx16), could you please share what you did? I will update the wiki after I get it to work myself. Thanks!
