Hi,
I don't know why you are forking shells using os.system. You should
either use system commands to do the job or plain python, but mixing
both should never be an option.
Here is a plain portable python script (using the with statement. Your
python should not be to old)
from __future__ import with_statement
import os
for i in xrange(1, 1001):
os.mkdir('Folder%s' %i)
for j in xrange(1, 51):
with open('Folder%s%sTextFile%s.txt' %(i, os.sep, j), "w") as f:
f.write('TextFile%s.txt' %j)
Here are the results on a ext3 file system
time python test.py
real 0m5.299s
user 0m0.939s
sys 0m3.539s
Regards,
Laurent
venutaurus...@gmail.com a écrit :
On Mar 18, 6:35 pm, Peter Otten <__pete...@web.de> wrote:
venutaurus...@gmail.com wrote:
Hello all,
I've an application where I need to create 50K files spread
uniformly across 50 folders in python. The content can be the name of
file itself repeated 10 times.I wrote a code using normal for loops
but it is taking hours together for that. Can some one please share
the code for it using Multithreading. As am new to Python, I know
little about threading concepts.
This is my requiremnt:
C:\TestFolder....
That folder contains 5 Folders.. Folder1, Folder2, Folder3.....
Folder5
Each folder in turn contains 10 folders:
and Each of those folder contains 1000 text files.
Please let me know if you are not clear.
Thank you,
Venu Madhav.
I've just tried it, creating the 50,000 text files took 17 seconds on my not
very fast machine. Python is certainly not the bottleneck here.
Peter
Really...!!!! I just can't beleive it. I am running my scirpt on an
INTEL Core2duo CPU machine with 2GB of RAM. I've started it two hours
back, but still it is running. This is how my code looks like
def createFiles(path):
m.write(strftime("%Y-%m-%d %H:%M:%S") +" Creating files in the
folder "+path+"\n")
global c
global d
os.chdir (path)
k = 1
for k in range (1,1001):
p = "%.2d"%(k)
FName = "TextFile"+c+"_"+d+"_"+p+".txt"
l =1
for l in range(1 , 11):
os.system ("\"echo "+FName+" >> "+FName+"\"")
l = l +1
k = k+1
MainPath = "C:\\Many_50000_1KB"
try:
os.mkdir (MainPath)
m.write(strftime("%Y-%m-%d %H:%M:%S") +" Created the base directory
\n")
except:
m.write(strftime("%Y-%m-%d %H:%M:%S") +" base directory already
exists\n")
os.chdir (MainPath)
for i in range (1 , 6):
j = 1
c = "%.2d"%(i)
FolderName ="Folder"+c
try:
os.mkdir (FolderName)
m.write(strftime("%Y-%m-%d %H:%M:%S") +" Created the folder
"+FolderName+"\n")
except:
m.write(strftime("%Y-%m-%d %H:%M:%S") +" Folder "+FolderName+"
already exists \n")
os.chdir (FolderName)
path = os.getcwd ()
#createFiles(path)
for j in range ( 1 , 11):
d = "%.2d"%(j)
FolderName = "Folder"+c+"_"+d
try:
os.mkdir (FolderName)
m.write(strftime("%Y-%m-%d %H:%M:%S") +" Created the
folder "+FolderName+"\n")
except:
m.write(strftime("%Y-%m-%d %H:%M:%S") +" the folder
"+FolderName+" exists \n")
os.chdir (FolderName)
path = os.getcwd ()
createFiles(path)
os.chdir ("..")
j = j + 1
os.chdir ("..")
i = i + 1
Can you please let me know where do I have to modify it, to make it
faster.
Thank you,
Venu Madhav
--
http://mail.python.org/mailman/listinfo/python-list
