Hi, this may have some impact on other processes, but we can deal with that there. So, I'd say go ahead.
-phi On Sat, Sep 24, 2011 at 1:43 PM, Kenneth Heafield <[email protected]> wrote: > Hi, > > ChartHypothesis.{h,cpp} maintain a hypothesis counter and each > ChartHypothesis is assigned an identifier "for logging reasons." Such > logging includes e.g. moses-chart-cmd/src/IOWrapper.cpp: 271. > > if (StaticData::Instance().GetOutputHypoScore()) { > out << hypo->GetTotalScore() << " " > << ChartHypothesis::GetHypoCount() << " "; > } > > In the presence of multi-threading, this is broken because the ids were > not assigned in a thread-safe fashion. Philip Williams made id handout > thread-safe, but there's a high cost: a lock every time a rule is > applied. Further, code such as the one above will print out a > non-deterministic number depending on how many hypotheses were generated > by other threads in between. > > Since ids are not handed out in a deterministic fashion and can be > sparse as a result of threading, I would like to remove the id and > replace it with the pointer to the ChartHypothesis object. These were > designed to be in bijective correspondence. It will also save memory. > Further, I want to remove GetHypoCount and omit it from output, as these > numbers are anyway misleading. stderr output will change as a result. > Further, log output will use pointers with 0xhex instead of integers. > > The same issue impacts phrase-based hypothesis ids. > > Shout now if this is a problem, > > Kenneth > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
