[
https://issues.apache.org/jira/browse/JOSHUA-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138814#comment-16138814
]
Jeffrey Smith commented on JOSHUA-277:
--------------------------------------
Getting closer:
I deleted runs/1 and re-ran:
$JOSHUA/bin/pipeline.pl \
--rundir 1 \
--readme "Baseline Hiero run" \
--source es \
--target en \
--type hiero \
--corpus $FISHER/corpus/asr/fisher_train \
--tune $FISHER/corpus/asr/fisher_dev \
--test $FISHER/corpus/asr/fisher_dev2 \
--maxlen 11 \
--maxlen-tune 11 \
--maxlen-test 11 \
--tuner-iterations 1 \
--lm-order 3
The example got farther but ended with the following error.
...
* Packing grammar at "grammar.gz" to
"/data/joshua-tutorial/runs/1/tune/model/grammar.gz.packed"
* Running the grammar-packer.pl script with the command:
/data/joshua/scripts/support/grammar-packer.pl -a -T /tmp -g grammar.gz -o
/data/joshua-tutorial/runs/1/tune/model/grammar.gz.packed
Exception in thread "main" java.util.NoSuchElementException
at org.apache.joshua.util.io.LineReader.next(LineReader.java:276)
at
org.apache.joshua.tools.GrammarPacker.getGrammarReader(GrammarPacker.java:239)
at org.apache.joshua.tools.GrammarPacker.pack(GrammarPacker.java:184)
at
org.apache.joshua.tools.GrammarPackerCli.run(GrammarPackerCli.java:120)
at
org.apache.joshua.tools.GrammarPackerCli.main(GrammarPackerCli.java:137)
* FATAL: Couldn't pack the grammar.
> UnsatisfiedLinkError: no ken in java.library.path
> -------------------------------------------------
>
> Key: JOSHUA-277
> URL: https://issues.apache.org/jira/browse/JOSHUA-277
> Project: Joshua
> Issue Type: Bug
> Reporter: Thamme Gowda
>
> I followed this guide http://joshua.incubator.apache.org/6.0/quick-start.html
> to test the latest build.
> Assuming there few things are broken due to newer maven build system, I tried
> to fix pipeline.pl to get the quick start guide working.
> Which files from kenlm build should I add to JNI path? (I am unable to locate
> the library file in the kenlm build output)
> Here is the full log:
> {code}
> $JOSHUA/bin/pipeline.pl --source bn --target en --type hiero
> --no-prepare --aligner berkeley --corpus input/bn-en/tok/training.bn-en
> --tune input/bn-en/tok/dev.bn-en --test input/bn-en/tok/devtest.bn-en
> [train-copy-and-filter] cached, skipping...
> [train-vocab-bn] cached, skipping...
> [train-vocab-en] cached, skipping...
> [tune-copy-and-filter] cached, skipping...
> [tune-vocab-bn] cached, skipping...
> [tune-vocab-en.0] cached, skipping...
> [tune-vocab-en.1] cached, skipping...
> [tune-vocab-en.2] cached, skipping...
> [tune-vocab-en.3] cached, skipping...
> [test-copy-and-filter] cached, skipping...
> [test-vocab-bn] cached, skipping...
> [test-vocab-en.0] cached, skipping...
> [test-vocab-en.1] cached, skipping...
> [test-vocab-en.2] cached, skipping...
> [test-vocab-en.3] cached, skipping...
> [source-numlines] cached, skipping...
> [source-numlines] retrieved cached result => 20788
> [berkeley-aligner-chunk-0] cached, skipping...
> [aligner-combine] cached, skipping...
> [pack-grammar] cached, skipping...
> [lm-sort-uniq] cached, skipping...
> [kenlm] cached, skipping...
> [compile-kenlm] cached, skipping...
> [glue-tune] cached, skipping...
> Error: Could not find or load main class
> joshua.util.encoding.EncoderConfiguration
> [tune-bundle] cached, skipping...
> [mert-1] rebuilding...
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
> [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> [CHANGED]
> dep=tune/model/grammar.packed/slice_00000.source [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config.final
> [NOT FOUND]
>
> cmd=/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.en
> --tunedir
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune --tuner
> mert --decoder
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/decoder_command
> --decoder-config
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> --decoder-output-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/output.nbest
> --decoder-log-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.log
> --iterations 10 --metric 'BLEU 4 closest'
> JOB FAILED (return code 1)
> Exception in thread "main" java.lang.RuntimeException: Unable to instantiate
> feature function 'StateMinimizingLanguageModel -lm_order 5 -lm_file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/model/lm.kenlm'!
> at
> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:761)
> at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514)
> at org.apache.joshua.decoder.Decoder.<init>(Decoder.java:122)
> at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:757)
> ... 3 more
> Caused by: java.lang.ExceptionInInitializerError
> at
> org.apache.joshua.decoder.ff.lm.StateMinimizingLanguageModel.initializeLM(StateMinimizingLanguageModel.java:75)
> at
> org.apache.joshua.decoder.ff.lm.LanguageModelFF.<init>(LanguageModelFF.java:156)
> at
> org.apache.joshua.decoder.ff.lm.StateMinimizingLanguageModel.<init>(StateMinimizingLanguageModel.java:50)
> ... 8 more
> Caused by: java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: no ken
> in java.library.path
> at org.apache.joshua.decoder.ff.lm.KenLM.<clinit>(KenLM.java:52)
> ... 11 more
> Caused by: java.lang.UnsatisfiedLinkError: no ken in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at org.apache.joshua.decoder.ff.lm.KenLM.<clinit>(KenLM.java:43)
> ... 11 more
> Traceback (most recent call last):
> File
> "/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py",
> line 553, in <module>
> main(sys.argv)
> File
> "/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py",
> line 536, in main
> run_zmert(opts.tunedir, opts.source, opts.target, opts.decoder,
> opts.decoder_config, opts.decoder_output_file, opts)
> File
> "/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py",
> line 417, in run_zmert
> opts.metric, opts.iterations or 10)
> File
> "/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py",
> line 399, in setup_configs
> for feature,weight in get_features(config):
> File
> "/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py",
> line 351, in get_features
> output = check_output("%s/bin/joshua-decoder -c %s -show-weights -v 0" %
> (JOSHUA, config_file), shell=True)
> File
> "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py",
> line 573, in check_output
> raise CalledProcessError(retcode, cmd, output=output)
> subprocess.CalledProcessError: Command
> '/Users/thammegr/work/projects/apache/incubator-joshua/bin/joshua-decoder -c
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> -show-weights -v 0' returned non-zero exit status 1
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
