Hi, I just submitted a update to the wrapper script that takes care of the limit (by breaking up the input text into smaller chunks) without the need to change the parser code.
-phi On Mon, Oct 18, 2010 at 1:15 PM, Hwidong Na <[email protected]> wrote: > Hi, > > You don't need to know the total number of the lines in advance. I > attached my modified version of "main.c", downsizing the number of > sentences in memory for each iteration to 250. I tested it using > "examples/sec??.tagged", and the both input file is parsed without any > segmentation fault. > > Best regards, > -- > Hwidong Na <[email protected]> > KLE lab, POSTECH, KOREA > > > 2010-10-15 (금), 16:02 +0200, marco turchi: >> Hi >> thanks, I'm trying to modify the main.c in a way that it reads the >> file twice, the first time to get the number of lines and the second >> to run the parser. It is not the best solution, but if ti works it can >> solve the problem. >> >> I do not yet take in account the segmentation fault. >> >> thanks >> Marco >> >> On Fri, Oct 15, 2010 at 3:49 PM, Raphael Payen <[email protected]> >> wrote: >> I also had the same problem, and I am also interested by the >> modifications to make to avoid the segmentation fault. >> >> Since when I tried it was for a simple test and I didnt bother >> correcting, I made this script, which you might use also. It >> splits the >> input into chunks of 2500 lines, It is used like this: >> <file split-file-wrapper.py 2500 parse-en-collins >outfile >> (But it makes the processing much slower, modifying the source >> would be >> better). >> >> -- >> Raphael Payen >> >> >> >> On Fri, 2010-10-15 at 14:40 +0200, marco turchi wrote: >> > Hi >> > I have the same problem with the Collins' parser. Do u know >> exactly >> > what I need to change in the source code of the parser? or u >> have a >> > modified version? >> > >> > Thanks a lot >> > Marco >> > >> > On Thu, Jun 3, 2010 at 5:03 PM, Hwidong Na >> <[email protected]> >> > wrote: >> > Hi, >> > >> > This is not because of the wrapper script, but the >> Collins' >> > parser. You >> > can modify the source to iterate the read_sentences >> function >> > in the file >> > "main.c". In addition, you need to modify defined >> values in >> > "grammar.h" >> > to avoid segmentation faults of long sentences. >> > >> > -- >> > Hwidong Na <[email protected]> >> > KLE lab, POSTECH, KOREA >> > >> > >> > 2010-05-27 (목), 19:20 +0800, dongxinghua0213: >> > >> > > hello, >> > > when parsing sentences using >> parse-en-collins.perl,I find >> > only 2500 >> > > parsed sentences are available ,but the number of >> sentences >> > are more >> > > than one hundred thousand , what can I do to >> parse all >> > sentences ? >> > > >> > > thank you ! >> > > >> > > >> > > >> > > >> > >> >> ______________________________________________________________________ >> > > 网易为中小企业免费提供企业邮箱（自主域名） >> > >> > > _______________________________________________ >> > > Moses-support mailing list >> > > [email protected] >> > > >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > >> > >> > >> > >> > >> > _______________________________________________ >> > Moses-support mailing list >> > [email protected] >> > >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > >> > _______________________________________________ >> > Moses-support mailing list >> > [email protected] >> > http://mailman.mit.edu/mailman/listinfo/moses-support >> >> >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
