Re: Not Responding When Dealing with Large Data

[MRAB]

On 2014-06-19 09:17, cutey Love wrote:

update_idletasks didn't work.

The code is this

     file_path = filedialog.askopenfilename(filetypes=[('text files', '.txt')], 
multiple=True, defaultextension=".txt")

     for path in file_path:

         fo = open(path, "r")

         for line in fo:
             if myCase(line.lower()):
                 myList.append(line.lower())
         fo.close()


def myCase(c):

         if c in myList:
             return False

         if len(c) < 8 or len(c) > 80:
             return False

return True



This processes a fair bit of data

It's quicker to look for something in a set than in a list, so if you
can use a set instead of a list, do so.

Also, checking the length of a string is quick, quicker than searching
a list.

Therefore, before processing the file, do:

    mySet = set(myList)

and then you can say:

    def myCase(c):
        if len(c) < 8 or len(c) > 80:
            return False

        if c in mySet:
            return False

        return True

which can be shortened to:

    def myCase(c):
        return 8 <= len(c) <= 80 and c in mySet

--
https://mail.python.org/mailman/listinfo/python-list