depends on the underlying filesystem. Just try it out before hitting your platform's limits. Someone's got to help me with math. Suppose that for ~10 millions of files in a single table, uuid are "evenly distributed". Folders are "2 letter prefixes" of an uuid, that is composed of hexadecimal characters. This means 16^2 folders in total, 256. 10 millions images evenly distributed in such 256 folders make ~39062 images per single folder. FAT32 holds 65634 files per folder, ok. NTFS 4 millions, ok. XFS, etx3, etx4, jfs, etc are ok (depends on available inodes, but for normal disk-sizes noaday it's a no-issue)
Depending on the filesystem and on the number of actual files, you may encounter some delays with high number of files in a single folder because every filesystem implements it's own indexing system (not of the file contents, just "pointers to where the file is"). BTW: lately there have been much improvements on supporting in web2py whatever you need to use as a "filesystem", even if it's virtual. pyfilesystem is supported and you can also write your own function to store/retrieve the file (e.g. if you want to store it in a similar way, but with a "4 character prefix" (10 millions files would end up being in folders of 152 files each), or for using GridFS, MogileFS, etc). There is no docs ready yet on it, but see dal source (from line 8360), it's quite straightforward. On Friday, October 5, 2012 9:02:26 PM UTC+2, apps in tables wrote: > > Thank you for the immediate reply... > > I did it manually. > is uploadseparate=True the best configuration for having millions of > images with reasonable performance? > > On Friday, October 5, 2012 9:56:37 PM UTC+3, Niphlod wrote: >> >> it's stated very clearly on the book that it's not possible to "migrate" >> between uploadseparate or not. Web2py should have to move within one >> request millions of files. >> """ >> ATTENTION: You cannot change the value of uploadseparate from True to >> False without breaking the system. web2py either uses the separate >> subfolders or it does not. Changing the behavior after files have been >> uploaded will prevent web2py from being able to retrieve those files. If >> this happens it is possible to move files and fix the problem but this is >> not described here. >> """ >> you can do that by hand if you need to, just move every file in a >> subfolder that is composed as the first 2 letters of the file itself. >> >> On Friday, October 5, 2012 8:44:06 PM UTC+2, apps in tables wrote: >>> >>> Hi >>> >>> Web2py is great... >>> >>> I always get a smooth migration after changing the structure of the >>> database. >>> >>> unfortunately, when i changed 'upload' data to uploadseparate = True , I >>> lost them. >>> >>> is this needs to be included in the auto migration? >>> >>> regards, >>> >>> Ashraf >>> >> --
