I think i actually found something now. Need to rewrite some stuff to make it work with the views again. Thanks for your help anyway :-)
Kenneth -----Original Message----- From: Brett Bergquist [mailto:[email protected]] Sent: 3. mai 2010 12:50 To: [email protected] Subject: Re: LIST from database Actually right now, no. Maybe throw some debug statements around or profile it to see where it is spending the time. Maybe that will give clue as to where it needs to be optimized. Brett On May 3, 2010, at 6:37 AM, Kenneth Vanvik Hansen wrote: > Hi, i've implemented your new code. I still have some trouble with it > being very slow. Listing 12 directories takes about 100 seconds. This > is from a bit complicated view, which might be some of the reason. > Still shouldn't take that long? Any ideas how to make it even faster? > > Kenneth > > -----Original Message----- > From: Kenneth Vanvik Hansen [mailto:[email protected]] > Sent: 27. april 2010 23:30 > To: [email protected] > Subject: Re: LIST from database > > Niice :-) I'll take a look at it tomorrow. I wrote a db view that > creates a folder structure form existing blobs. And there are several > thousand blobs, which is how i found the list command being slow. Thank you. > > Kenneth > > On Apr 27, 2010, at 11:13 PM, Brett M. Bergquist wrote: > >> I just did a quick modification of my code to not do a select on each > file. Basically when the "list" is done, I gather all of the > information and cache it in the DbFile object. This greatly increased > performance so that at directory with a couple of hundred files takes > less than a second to return a directory listing. >> >> I have created a Jira issue and attached a Netbeans project which is >> a > running sample against a Derby database. This has this changed code in it. > So you might want to download the code and make the same changes in > your version. >> >> This is Jira Issue: FTPSERVER-372 </jira/browse/FTPSERVER-372> >> >> https://issues.apache.org/jira/browse/FTPSERVER-372 >> >> >> Brett M. Bergquist wrote: >>> Sorry, I just saw this among the hundreds of emails that I get ;) >>> >>> I don't have a quick answer. It is doing 1 select to get the list >>> of > files and then for each other check on each file (isDirectory, > isReadable, > etc.) it is doing another select. It is a very naive implementation > in that it is caching nothing and always refreshing its knowledge from the database. > And your right, with many files in a directory it is slow and resource > intensive. >>> >>> Kenneth Vanvik Hansen wrote: >>>> No, i don't have a need to do a select for every file. It just >>>> seems it does a lot of queries when it receives a LIST command. >>>> Only change I have done to the original code is I have changed from >>>> Derby to ODBC. It works great for folders with up to 10 or so files >>>> in it, after that it starts to get really slow. >>>> >>>> -----Original Message----- >>>> From: David Latorre [mailto:[email protected]] Sent: 27. april 2010 >>>> 14:10 >>>> To: ftpserver-users >>>> Subject: Re: LIST from database >>>> >>>> If I inderstood you correctly, it is your code that is doing a >>>> SELECT for every file, isn't it? >>>> >>>> Since I haven't seen your code, I cannot give you advice on this, >>>> why do you need to do a select for every file? >>>> >>>> Maybe Brett's solution can help you... did you get the time to work >>>> on this, Brett? >>>> >>>> >>>> >>>> >>>> 2010/4/27 Kenneth Vanvik Hansen <[email protected]>: >>>> >>>>> Hi, I've been working on this a bit now but I'm having some >>>>> trouble when there are many files/folders in a folder. Seems like >>>>> LIST does a select for every file in a folder. Any good ideas on >>>>> how to solve this? In my db(which is a bit slow) it can take >>>>> several minutes to list >>>>> >>>> 40+ files/folders. >>>> >>>>> Kenneth >>>>> >>>>> -----Original Message----- >>>>> From: Brett M. Bergquist [mailto:[email protected]] >>>>> Sent: 23. mars 2010 17:40 >>>>> To: [email protected] >>>>> Subject: Re: LIST from database >>>>> >>>>> Okay. That will work. Thanks. >>>>> >>>>> Niklas Gustavsson wrote: >>>>> >>>>>> On Tue, Mar 23, 2010 at 4:08 PM, Brett M. Bergquist > <[email protected]> wrote: >>>>>> >>>>>> >>>>>>> I actually have a working Netbeans project that works against a >>>>>>> Derby database. I am going to do a little more cleanup and see >>>>>>> if I cannot factor out the database specifics like the db user >>>>>>> manager does and then I would like to provide the whole thing, >>>>>>> probably to go into the examples. So should I send this to you >>>>>>> Niklas when I am >>>>>>> >>>> ready? >>>> >>>>>> If you would like to contribute the code to the project, the best >>>>>> way > is to create a JIRA issue and attach the code there (making sure you > check the donate to ASF checkbox). >>>>>> >>>>>> /niklas >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>>> >>>> >>> >> >> > > Kenneth Vanvik Hansen > [email protected] > > > >
