On Jul 9, 12:04 pm, dp_pearce <[EMAIL PROTECTED]> wrote: > I have some code that takes data from an Access database and processes > it into text files for another application. At the moment, I am using > a number of loops that are pretty slow. I am not a hugely experienced > python user so I would like to know if I am doing anything > particularly wrong or that can be hugely improved through the use of > another method. > > Currently, all of the values that are to be written to file are pulled > from the database and into a list called "domainVa". These values > represent 3D data and need to be written to text files using line > breaks to seperate 'layers'. I am currently looping through the list > and appending a string, which I then write to file. This list can > regularly contain upwards of half a million values... > > count = 0 > dmntString = "" > for z in range(0, Z): > for y in range(0, Y): > for x in range(0, X): > fraction = domainVa[count] > dmntString += " " > dmntString += fraction > count = count + 1 > dmntString += "\n" > dmntString += "\n" > dmntString += "\n***\n > > dmntFile = open(dmntFilename, 'wt') > dmntFile.write(dmntString) > dmntFile.close() > > I have found that it is currently taking ~3 seconds to build the > string but ~1 second to write the string to file, which seems wrong (I > would normally guess the CPU/Memory would out perform disc writing > speeds). > > Can anyone see a way of speeding this loop up? Perhaps by changing the > data format? Is it wrong to append a string and write once, or should > hold a file open and write at each instance? > > Thank you in advance for your time, > > Dan
Maybe try something like this ... count = 0 dmntList = [] for z in xrange(Z): for y in xrange(Y): dmntList.extend([" "+domainVa[count+x] for x in xrange(X)]) dmntList.append("\n") count += X dmntList.append("\n") dmntList.append("\n***\n") dmntString = ''.join(dmntList) -- http://mail.python.org/mailman/listinfo/python-list