Hi,Yang, Thanks your suggestion. By outputting related anything information with CubePruning Search step by step, fortunately I finally understood the effect of BitmapContainer and the implementation of CubePruning. what a perfect the CubePruning and BitmapContainer.
cheers, chen 2012/8/13 <[email protected]> > Send Moses-support mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.mit.edu/mailman/listinfo/moses-support > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Moses-support digest..." > > > Today's Topics: > > 1. Re: Who can you give me some explanation about the Class > BitmapContainer in details? (Yang Gao) > 2. Re: help (Barry Haddow) > 3. Re: Factored decoding performance (Hieu Hoang) > 4. First Call for Participation: SMS Based FAQ Retrieval at FIRE > 2012 (Danish Contractor) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sun, 12 Aug 2012 10:02:22 -0700 > From: Yang Gao <[email protected]> > Subject: Re: [Moses-support] Who can you give me some explanation > about the Class BitmapContainer in details? > To: MT NLP <[email protected]> > Cc: [email protected], [email protected] > Message-ID: > < > ca+o3wvu7j0yrshn1fwne9fw0m-+7adpse7-ncmpybke60zf...@mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Hi, maybe looking at the methods that call this class can give you an idea? > If you are confused, it helps to run a toy example and print out something. > > cheers, > > yang > > On Sun, Aug 12, 2012 at 4:15 AM, MT NLP <[email protected]> wrote: > > > Hi,all > > > > Presently,I read the implemention of CubePruning search algorithm in the > > Moses,but I confuse on the effect of Class BitmapContainer. > > Who could you give me some explanation about the Class BitmapContainer in > > details? > > > > chen > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > > > -- > Yang Gao (Vivian) > ---------------------------------------- > Graduate Student, USC/ISI > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mailman.mit.edu/mailman/private/moses-support/attachments/20120812/f0597053/attachment-0001.htm > > ------------------------------ > > Message: 2 > Date: Mon, 13 Aug 2012 09:23:53 +0100 > From: Barry Haddow <[email protected]> > Subject: Re: [Moses-support] help > To: Mohadeseh Sheikhoveisi <[email protected]> > Cc: [email protected] > Message-ID: <[email protected]> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Hi Mohadeseh > > You're using a very old version of Moses, so I'd suggest you update to > the latest github version. > > As to the Giza error, does this answer your question? > http://comments.gmane.org/gmane.comp.nlp.moses.user/7231 > > cheers - Barry > > cheers - Barry > > On 12/08/12 09:32, Mohadeseh Sheikhoveisi wrote: > > Hi, > > I want to get word-to-word alignment (IBM-Model-1) using giza++ while I > run > > "./train-model.perl -scripts-root-dir /opt/tools/scripts-20120707-0326 > > -root-dir /home/msho/Parallel-Corpus/Now/ -corpus > > /home/msho/Parallel-Corpus/clean/Clean-Corpus -f en -e fa -alignment > > grow-diag-final-and -reordering msd-bidirectional-fe -lm > > 0:3:/home/msho/Parallel-Corpus/LM/lm/Data.fa.lm -parallel -giza-option > > m1=5,m2=0,m3=0,m4=0,mh=0,t1=5,nodumps=0 " I get this error: > > > > ERROR: Giza did not produce the output file > > /home/msho/Parallel-Corpus/New/giza.fa-en/fa-en.A3.final. Is your corpus > > clean (reasonably-sized sentences)? at ./train-model.perl line 958. > > > > how can i get word-to-word alignment? > > please help me. > > > > Best Regards, > > > > > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > > > > ------------------------------ > > Message: 3 > Date: Mon, 13 Aug 2012 10:28:52 +0100 > From: Hieu Hoang <[email protected]> > Subject: Re: [Moses-support] Factored decoding performance > To: [email protected] > Message-ID: <[email protected]> > Content-Type: text/plain; charset="iso-8859-1" > > your factor model does looks fairly complicated so I'm not surprised > that it would be slow. It prob uses a significant amount of memory too. > > There's 2 generation steps, where 1 would be sufficient. > > On 09/08/2012 10:35, Michal Krajn(ansk? wrote: > > Hi, > > > > I am experimenting with factored training and I've got a question > > about decoding performance. > > I experience hangups - probably not real hangups, just too long > > computation - while decoding some sentences. > > > > My models are trained on text annotated in the following way: > > > > SURF|LEMM|POS|OTHER > > > > Training is done with the following parameters: > > > > --lm 0:3:news-commentary-v7.cs-en.blm.en:8 \ > > --lm 2:3:pos.blm.en:8 \ > > --translation-factors 1-1+3-2+0-0,2 \ > > --generation-factors 1-2+1,2-0 \ > > --decoding-steps t0,g0,t1,g1:t2 > > > > I suspect some things that can cause the issue: > > > > > > - the POS tags are in the form of a number: 1-10, when ambiguous, the > > possibilities are separated by comma, so e.g. POS= > > 1 > > 3,5 > > 7,0,2,6,1 > > > > POS tags are more abiguous at the target language side (en), where > > this is about 1/2 of cases. > > I see this can cause the sparsity problem but believe this is not a > > fundamental issue. > > > > > > - sometimes I do not get the values for some factors, so I introduce > > an universal placeholder '_' which I put in place of unknown factors > > > > I imagine this could cause some "sink" problem, in the sense of too > > common token (this is not really uncommon in my data). > > > > I post a sample sentence: > > the|the|3|Node government|government|1|Gsub in|in,inly|7,2,6,1|Node > > Washington|Washington|1|Gsub has|have|5|Node published|publish|5|Pred > > a|a|3|Node prohibition|prohibition|1|Obje to|to|6|Advi > > that|that|8|Node effect|effect|5,1|Node _-_|_|_|Node > > thereby|thereby|6|Node definitively|definitively|6|Node > > scrapping|scrap|5|Node earlier|earlier,early|2,6|Node plans|plan|1|Gsub > > > > > > Could you please point me to the possible problems of this setup? > > > > Thanks in advance and regards, > > > > Michal Krajn(ansk? > > > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mailman.mit.edu/mailman/private/moses-support/attachments/20120813/c162db0e/attachment-0001.htm > > ------------------------------ > > Message: 4 > Date: Mon, 13 Aug 2012 15:05:53 +0530 > From: Danish Contractor <[email protected]> > Subject: [Moses-support] First Call for Participation: SMS Based FAQ > Retrieval at FIRE 2012 > To: [email protected] > Message-ID: > < > capia5v6slp0blkhdeb8_gjd0p+lyrwg-diqgfjznhsomznx...@mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > [Apologies if you receive multiple copies of this call] > > > First Call for Participation > > > SMS-Based FAQ Retrieval > (at Forum for Information Retrieval Evaluation 2012) > > 17 - 19 December 2012 > Indian Statistical Institute, Kolkata > > http://www.isical.ac.in/~fire/faq-retrieval/2012/faq-retrieval.html > > > > The number of internet users in the world is estimated to be around 2 > Billion. > India, having nearly 17% of the world population accounts for merely 81 > million users > (6.9 % of its population). On the other hand, the number of telecom service > users, > specifically mobile phone users in India is nearly 10 times larger than > the number of internet users. The mobile phone is a cheap and > easy device for communication and is increasingly being used as > a source of information. It is keeping this fact in mind, that FIRE 2012 is > continuing the "SMS based FAQ Retrieval" task introduced last year at FIRE. > (Results from last year's participation can be seen at : > http://www.isical.ac.in/~fire/faq-retrieval/2011/results.html) > > The goal of this task is to find a question Q* from the corpora of FAQs > (Frequently asked questions) that best answers/matches the SMS query S. > > SMS queries which are written in "SMS language" tend to be noisy as > users try and compress text by omitting letters, using slang, etc., due to > a cap on the length of messages (160 characters constitutes one SMS), lack > of screen space (which makes reading large amounts of text difficult), etc. > > The messages also frequently contain unintended typographical errors due > to the small size of keypads on mobile phones as also the poor language > skills of the users. The presence of such noise makes this task different > and more challenging than traditional QA retrieval tasks. > > The task comprises mono-, cross- and multi-lingual FAQ Retrieval > sub-tasks. > > All participating teams also need to submit a working note that outlines > the approach followed by them. The Working note is due on November > 28, 2012. The post-proceedings will be published in Lecture Notes in > Computer Science (http://www.springer.com/computer/lncs), Springer. > Authors will be invited to submit a revised and expanded version of > their working note to be published in the post-proceedings. > > > > IMPORTANT DATES: > (http://www.isical.ac.in/~fire/faq-retrieval/2012/dates.html) > Training data release: Aug 4,2012 > Test data release: Sept 20,2012 > Run Submission:Oct 15,2012 > Qrel Release:Nov 15,2012 > Working Note Due: Nov 28,2012 > > > > > TASK COORDINATORS: > > Danish Contractor, [email protected] > Ankush Mittal, [email protected] > Deepak S Padmanabhan, [email protected] > LV Subramaniam, [email protected] > > ****************************** > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mailman.mit.edu/mailman/private/moses-support/attachments/20120813/43fcdd21/attachment.htm > > ------------------------------ > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > > End of Moses-support Digest, Vol 70, Issue 54 > ********************************************* >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
