I have been having great success with the @parallel decorator, but
sometimes I cannot work out why it is not doing everything that I
expect. Perhaps someone can help?
The setup is simple. I have a function process_one(L, N1, N2) which
takes a string L as input (plus a couple of other parameters) and
returns a string as output. This gets the parallel decorator. It is
managed by a controlling function process_all() which opens an input
file inf and an output file, uses inf.readlines() to get an iterator
over all lines of the input, calls process_one() on each and writes
the result to the screen and the output file. I do not care that the
output files's lines are in random order since I can easily sort them
afterwards.
The code for this function looks very like this (I only changed the name)
def process_all(infile, N1, N2, outfile=None):
inf = open(infile)
if outfile:
outf = open(outfile,'w')
output_lines = rank1_line([(L,N1,N2) for L in inf.readlines()])
for input_line, output_line in output_lines:
print output_line
if outfile:
outf.write(output_line)
outf.write("\n")
outf.flush()
inf.close()
if outfile:
outf.close()
When I run this on a file with 10000 lines it works fine (say with
@parallel(20)) but when I run it on a file with two or a few lines it
tends to process the first lines but not all (and never returns), or
on a file with 10 lines then most will run but not the others and the
function never returns.
In case it is relevant, the inner function process_one() uses system()
to call gp with some input, whose output is piped to a file which is
then read. I don't use Sage's own pari/gp since the function I need
is only in pari-2.6.0.
John
--
You received this message because you are subscribed to the Google Groups
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/groups/opt_out.
