Re: [ngram] Using huge-count.pl with lots of files

[Ted Pedersen tpede...@d.umn.edu [ngram]]

I guess my first thought would be to see if there is a simple way to
compute the input you are providing to huge count into fewer files. If you
have a lot of files that start with the letter 'a', for example, you could
concatentate them all together via a (Linux) command like

cat a* > myafiles.txt

and then use myafiles.txt as an input to huge_count.

This is just one idea, but it's a start perhaps. If this isn't helpful
please let us know and we can try again!

On Sun, Apr 15, 2018 at 1:19 PM, catherine.dejage...@gmail.com [ngram] <
ngram@yahoogroups.com> wrote:

>
>
> I am trying to get the bigram counts aggregated across a lot of files.
> However, when I ran huge-count.pl using the list of files as an input, I
> got the error "Argument list too long". What would you recommend for
> combining many files, when there are too many files to just run
> huge-count.pl as is?
>
>
> Thank you,
>
> Catherine
>
>
> 
>

[ngram] Using huge-count.pl with lots of files

[catherine.dejage...@gmail.com [ngram]]

I am trying to get the bigram counts aggregated across a lot of files. However, 
when I ran huge-count.pl using the list of files as an input, I got the error 
"Argument list too long". What would you recommend for combining many files, 
when there are too many files to just run huge-count.pl as is?
 

 Thank you,
 Catherine