Hi,
ChartHypothesis.{h,cpp} maintain a hypothesis counter and each
ChartHypothesis is assigned an identifier "for logging reasons." Such
logging includes e.g. moses-chart-cmd/src/IOWrapper.cpp: 271.
if (StaticData::Instance().GetOutputHypoScore()) {
out << hypo->GetTotalScore() << " "
<< ChartHypothesis::GetHypoCount() << " ";
}
In the presence of multi-threading, this is broken because the ids were
not assigned in a thread-safe fashion. Philip Williams made id handout
thread-safe, but there's a high cost: a lock every time a rule is
applied. Further, code such as the one above will print out a
non-deterministic number depending on how many hypotheses were generated
by other threads in between.
Since ids are not handed out in a deterministic fashion and can be
sparse as a result of threading, I would like to remove the id and
replace it with the pointer to the ChartHypothesis object. These were
designed to be in bijective correspondence. It will also save memory.
Further, I want to remove GetHypoCount and omit it from output, as these
numbers are anyway misleading. stderr output will change as a result.
Further, log output will use pointers with 0xhex instead of integers.
The same issue impacts phrase-based hypothesis ids.
Shout now if this is a problem,
Kenneth
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
