Marcin Junczys-Dowmunt <junczys@...> writes: > > Hi Rico, > cmph is not probabilistic, but the check for phrases outside the set of > training phrases (false positives) is. For encodings others than None, > the compression also serves as a test, so the fp rate is much smaller > then. You can also modifiy the fp probability by increasing the number > of fingerprint bits, for instance to 20 (-fingerprint 20), default is > 16. The probability of a FP slipping through per query is 1/2^b where b > is the number of fingerprint bits. Using b=16 and -encoding PREnc > (default) I have never seen a false positive, so just stick to the > defaults :)
Hi Marcin, sorry, I should have seen that false positives are not a bug per se. The problem was just that they broke my code, and apparently your code also sometimes segfaults when encountering them. here are the steps to reproduce the bug (looking up "various authorities" causes the segfault; it's not in the table) =================== command executed: echo "various authorities" | gdb "~/mosesdecoder/bin/queryPhraseTableMin -t table -n 3 -a" =================== resources used: table.minphr (compiled on 64bit Linux), hope you can use it https://docs.google.com/open?id=0B5nFI-wykGgnUEttRTNlRk94Z3c =================== backtrace: #0 AddWord (newWord=..., this=0x22e1550) at moses/Phrase.h:131 #1 Moses::Phrase::Append (this=0x22e1550, endPhrase=...) at moses/Phrase.cpp:138 #2 0x0000000000594471 in Moses::PhraseDecoder::DecodeCollection (this=0x228d490, tpv=..., encodedBitStream=..., sourcePhrase=..., topLevel=true) at moses/CompactPT/PhraseDecoder.cpp:412 #3 0x0000000000593657 in Moses::PhraseDecoder::CreateTargetPhraseCollection (this=0x228d490, sourcePhrase=..., topLevel=true) at moses/CompactPT/PhraseDecoder.cpp:245 #4 0x00000000004ac0b8 in Moses::PhraseDictionaryCompact::GetTargetPhraseCollectionRaw (this=0x7fff9694c560, sourcePhrase=...) at moses/CompactPT/PhraseDictionaryCompact.cpp:158 #5 0x0000000000409749 in main (argc=<optimized out>, argv=<optimized out>) at misc/queryPhraseTableMin.cpp:75 _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
