Hi Hieu,
Just to provide more info, I had compiled moses using the following command:
"./bjam -j8 -q --with-cmph=/cs/natlang-user/jasneet/softwares/cmph-2.0/
--with-boost=/cs/natlang-user/jasneet/softwares/boost/ --max-kenlm-order=8 -a
--with-mm --with-probing-pt”.
Following are some more
Added load= option in 7a1baee, deprecating lazy=. Valid load= options
are the lowercase version of the enum shown below.
There's copies of the loading code in Backward.cpp and Reordering.h that
blames back to Lane. I've put ? : hacks in and hope he'll pay the cost
for code copying.
Kenneth
On
Hi,
The default is mmap with MAP_POPULATE (see man mmap). As to whether
GPFS implements MAP_POPULATE correctly, I defer to the former IBM
employee.
KenLM implements the following options via config.load_method:
typedef enum {
// mmap with no prepopulate
LAZY,
// On linux,
Hi Lane,
For the compact phrase table and reordering table you can use
--minphr-memory and --minlexr-memory respectively. That will disable
memory mapping entirely and just read both into RAM.
Best,
Marcin
On 20.02.2016 00:29, Lane Schwartz wrote:
> Hey,
>
> This is mostly addressed to Kenneth,
Hey,
This is mostly addressed to Kenneth, since as far as I know he's the author
of the data structures involved.
I have access to a cluster at the University of Illinois. The cluster here
uses GPFS as its file system.
I've observed that when running moses, especially with lots of threads,
that
