Hi Tom, What a great answer!! Thank you so much for the several proposals. For me, I already did the training on Linux, so I just need to do the decoding on Windows. I will take a look on your solution "Slate Toolkit", it seems interesting. Anyway, I have multiple choices, I must choose the best/right one.
Thank you again for your help. Warm regards. Amine. On Thu, May 10, 2018, 08:13 Tom Hoar <[email protected]> wrote: > Hi everyone. Sorry I joined the conversation a bit late. > > Here are all the options for Moses on Windows that I've tried: > > *Virtual machine* - VMs are more efficient than they used to be. Any good > hypervisor will work. You need to build your own tools to create a sockets > layer to communicate from the host machine to the VM. > > *Moses with Cygwin* - I never got it to work. There are too many > variables and the underlying open source components change drastically. It > could be one person's full-time job to track keep Moses updated for those > build tools. > > *Moses with Visual Studio* - It's a good option if you're looking for > only the moses.exe runtime binary. This does not address the serious > challenges of updating all the other C++, Perl and Python components to run > on Windows. Nor does it address collecting and installing all the GNU > utilities. > > *Windows Subsystem for Linux (WSL)* - WSL is native Linux running on the > Windows. Microsoft has done a great job of mapping the Linux kernel calls > to the Windows kernel. There's no virtual machine. It's very much like > running Windows applications on Linux using WINE... but without the > complexities of re-mapping Microsoft's GUI APIs. > > From Moses' perspective, WSL is nothing special. It sees the environment > as any native Linux. You can compile Linux binaries on Linux and copy them > to your Windows 10 machine and run them with WSL. You will need to > verify/install all of the other dependencies because WSL distributions are > sparse at best. For example, the Ubuntu 16.04 distro for WSL does not > install Python 2.7. The `python` command will fail. It installs Python 3.5 > by default with `python3` as the executable name. This will cause Moses to > fail. You'll need to do a lot of troubleshooting. > > WSL also falls short when it comes to inter-process communications between > WSL's native Linux apps and native Windows apps. I found one Microsoft > instruction that WSL and Windows apps share the network sockets stack, and > that is the only inter-process communication between the two layers. So, > STDIN/STDOUT/STDERR are not available between native Windows apps and Linux > apps on WSL. This means from a practical standpoint, you're back to working > with WSL as though it's a VM. That is, you can build a sockets layer to > communicate between the two. Maybe you can use the moses binary in Moses > Server mode and use XML-RPC, but I never tried it. > > There's another awkward inter-process strategy. If you're in a Windows > cmd.exe terminal or PowerShell terminal, you can execute a command line > that will launch a Linux app in the WSL space. (and vice-versa. From a > native Linux Bash terminal, you can launch a native Windows .exe, but > that's irrelevant to Moses) So, you could have your Windows app save a > text file to the file system, then execute the Moses Linux binary with the > -i argument pointing to the input file. The challenge here is translating > the Windows file system "C:\Users\...." to the WSL file system > "/mnt/c/Users/...". You'll also have to figure out a way to pass the STDOUT > redirection to the moses binary to save it's output. I don't know if this > is possible. Then, your Windows application that spawned the Linux Moses > binary will need to detect when the binary is done, and then retrieve the > output file. WARNING: do not directly access the Linux file system tree > from Windows > [%LOCALAPPDATA%\Packages\CanonicalGroupLimited.UbuntuonWindows_xxxxxxxxxxxxx\LocalState\rootfs]. > You are virtually guaranteed to corrupt the files. Only have Windows apps > save to it's native C:\ drives and then access them from Linux using the > mount director tree. > > Bottom line... you should read what Microsoft's engineers have written > about why they created WSL. It is not intended to be a production tool. It > is designed as a development environment that facilitates developing > cross-platform applications/tools. Microsoft is still putting great > development effort into WSL, it will likely evolve to become a much better > production environment. WSL also has some funky quirks relating to path > lengths and permissions, but they were edge case for me and you'll probably > never encounter them. > > *Slate Toolkit* - I have a free Windows and Linux package distributions > that you can download from my website ( > https://slate.rocks/slate-toolkit-edition/). I won't repeat the README > here. In brief, the toolkit packages includes all of the the MGIZA++ tools > and the Moses tools for each platform as described in the README. The > Windows package includes the required native Windows GNU tools. These are > the GIT commits used to compile the binaries from Moses github trunk. > > MGIZA_COMMIT = d643960de98565d208114780ba8025799208afa7 > MOSES_COMMIT_LINUX = 3a0631a05b7f53a7f387ca8ddca432f5ddb22029 (Jan 2018?) > MOSES_COMMIT_WINDOWS = a64468a9919867174beba71962ada795a4ce12d3 (pre-merge > of Moses server) > > The Linux binaries were compiled using Moses' BJAM. The Windows binaries > were compiled using my proprietary build environment (Visual Studio or > Cygwin). Both Linux and Windows were compiled with these options enabled: > > - 64-bit architecture only > - static compiled -- no .dll/.so dependencies > - KenLM > - XML-RPC > - legacy binarized translation and reordering tables > > The packages include Perl and Python scripts from MGIZA++ and Moses that I > updated for cross-platform use. This means these packages are full-stack > phrase-based SMT that create SMT models and decode with them. Models > trained/tuned on Windows can be copied to Linux for decoding and vice > versa. > > The files in the packages are laid out so that you unzip the package, open > a command line and execute the demo shell scripts. There is a sample data > set so you can confirm everything works. There are demo shell scripts (Bash > for Linux, .CMD batch files for Windows) that execute complete toolchains > to build a baseline SMT model. They are not suitable for production, but > they should be a good learning environment to compare Linux vs Windows > usage. Standard pipes, sockets, file system... they're all available for > inter-process communications between the Moses binaries and other Windows > apps. > > Licensing: All of the components, including the binaries, are distributed > under their respective open source licenses. For the most part, that's > LGPL-3 for Moses. The package has a complete list of open source licenses > per component. This means you can freely re-distribute the binaries and > scripts within the terms of the respective open source licenses. I.e., I > don't make any money on them. > > *Slate Desktop* - Lastly, I'll mention my our flagship application. For > the SMT component, Slate Desktop uses Slate Toolkit. It includes many many > more tools for corpus preparation and integration with popular CAT tools. > We distribute Slate Desktop under a proprietary EULA. License fees are > based per-host. There are no subscription fees or royalties. Slate > Desktop's underlying architecture includes a primitive API. You could, for > example, license our API and build your application around it. This means > you could have your application running and tested in a few days. If this > interests you, please contact me off-list. > > I hope this helps everyone and puts the Windows support in perspective. > > Regards, > Tom > > On 5/9/2018 11:58 PM, [email protected] wrote: > > Date: Wed, 9 May 2018 18:52:48 +0200 > From: Mohamed Amine MENACER <[email protected]> <[email protected]> > Subject: Re: [Moses-support] Install moses on Windows > To: Nisheeth Joshi <[email protected]> <[email protected]>, > [email protected] > Cc: [email protected] > > Hi Nisheeth, > > Great!! I will try it out right away. > I also think that it is a good idea to put this solution on the official > website. > > Warm regards. > Amine. > > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
