Hi, i've implemented your new code. I still have some trouble with it being very slow. Listing 12 directories takes about 100 seconds. This is from a bit complicated view, which might be some of the reason. Still shouldn't take that long? Any ideas how to make it even faster?
Kenneth -----Original Message----- From: Kenneth Vanvik Hansen [mailto:[email protected]] Sent: 27. april 2010 23:30 To: [email protected] Subject: Re: LIST from database Niice :-) I'll take a look at it tomorrow. I wrote a db view that creates a folder structure form existing blobs. And there are several thousand blobs, which is how i found the list command being slow. Thank you. Kenneth On Apr 27, 2010, at 11:13 PM, Brett M. Bergquist wrote: > I just did a quick modification of my code to not do a select on each file. Basically when the "list" is done, I gather all of the information and cache it in the DbFile object. This greatly increased performance so that at directory with a couple of hundred files takes less than a second to return a directory listing. > > I have created a Jira issue and attached a Netbeans project which is a running sample against a Derby database. This has this changed code in it. So you might want to download the code and make the same changes in your version. > > This is Jira Issue: FTPSERVER-372 </jira/browse/FTPSERVER-372> > > https://issues.apache.org/jira/browse/FTPSERVER-372 > > > Brett M. Bergquist wrote: >> Sorry, I just saw this among the hundreds of emails that I get ;) >> >> I don't have a quick answer. It is doing 1 select to get the list of files and then for each other check on each file (isDirectory, isReadable, etc.) it is doing another select. It is a very naive implementation in that it is caching nothing and always refreshing its knowledge from the database. And your right, with many files in a directory it is slow and resource intensive. >> >> Kenneth Vanvik Hansen wrote: >>> No, i don't have a need to do a select for every file. It just seems >>> it does a lot of queries when it receives a LIST command. Only >>> change I have done to the original code is I have changed from Derby >>> to ODBC. It works great for folders with up to 10 or so files in it, >>> after that it starts to get really slow. >>> >>> -----Original Message----- >>> From: David Latorre [mailto:[email protected]] Sent: 27. april 2010 >>> 14:10 >>> To: ftpserver-users >>> Subject: Re: LIST from database >>> >>> If I inderstood you correctly, it is your code that is doing a >>> SELECT for every file, isn't it? >>> >>> Since I haven't seen your code, I cannot give you advice on this, >>> why do you need to do a select for every file? >>> >>> Maybe Brett's solution can help you... did you get the time to work >>> on this, Brett? >>> >>> >>> >>> >>> 2010/4/27 Kenneth Vanvik Hansen <[email protected]>: >>> >>>> Hi, I've been working on this a bit now but I'm having some trouble >>>> when there are many files/folders in a folder. Seems like LIST does >>>> a select for every file in a folder. Any good ideas on how to solve >>>> this? In my db(which is a bit slow) it can take several minutes to >>>> list >>>> >>> 40+ files/folders. >>> >>>> Kenneth >>>> >>>> -----Original Message----- >>>> From: Brett M. Bergquist [mailto:[email protected]] >>>> Sent: 23. mars 2010 17:40 >>>> To: [email protected] >>>> Subject: Re: LIST from database >>>> >>>> Okay. That will work. Thanks. >>>> >>>> Niklas Gustavsson wrote: >>>> >>>>> On Tue, Mar 23, 2010 at 4:08 PM, Brett M. Bergquist <[email protected]> wrote: >>>>> >>>>> >>>>>> I actually have a working Netbeans project that works against a >>>>>> Derby database. I am going to do a little more cleanup and see >>>>>> if I cannot factor out the database specifics like the db user >>>>>> manager does and then I would like to provide the whole thing, >>>>>> probably to go into the examples. So should I send this to you >>>>>> Niklas when I am >>>>>> >>> ready? >>> >>>>> If you would like to contribute the code to the project, the best way is to create a JIRA issue and attach the code there (making sure you check the donate to ASF checkbox). >>>>> >>>>> /niklas >>>>> >>>>> >>>>> >>>> >>>> >>> >>> >>> >>> >> > > Kenneth Vanvik Hansen [email protected]
