Re: Have we got any *runtime* performance tests?
Not yet, so all my evidence is anecdotal. What I was trying to figure out was whether or not there are any preexisting runtime performance tests I could just run and compare. I know at some point in time we had some. I have been running with the patches now for over a month with no troubles or unusual behavior. I had to do a smidgen further patching (NIO streams will fire a ClosedByInterruptException if you try to use them with the current thread's Interrupted flag set), and if you expand a folder with thousands of unrecognized files, you can get a "too many open files" FileSystemException, easily dealt with by just closing 1/4 of the cache's open streams and retrying. It does occur to me that this might not play nicely at all with Windows' automatic file-locking, and might have to be disabled for windows. I was not able to get using the existence of an open stream (even touching it to ensure accuracy) as a substitute for File.exists() to behave similarly enough for tests to pass, though this may be an artifact of the tests, not the approach. -Tim On Thu, Sep 12, 2019 at 9:45 AM Jean-Marc Borer wrote: > Hi Tim, > > If file access is improved that may also improve indexing and by > consequence symbol finding or refactoring. Did you try such operations to > check the difference in speed? > > Cheers, > > JMB > > On Sun, Aug 18, 2019 at 8:47 AM Tim Boudreau wrote: > > > I've been playing with some tweaks to masterfs - specifically, I had a > > library i wrote for a completely different purpose, which keeps a cache > of > > open NIO FileChannels. Partly out of curiosity, and partly because I've > > wondered for a while what using NIO more for File IO in NetBeans might > > accomplish, I dropped it into masterfs. > > > > Basically, opening a file opens a file channel (the cache key includes > the > > read/write/truncate options), which is cached, and closed after 60 > seconds > > if it has not been touched. Input and output streams just take a lock on > > the stream, set its state back to what it was when they last touched it, > > and then release the lock. > > > > Subjective impression, after running with these patches for about a week > is > > that my machine is quieter and the IDE snappier - which makes some sense, > > since simply editing a file can trigger a bunch of reads of related file, > > so this is dramatically reducing interaction with the OS (in the back of > my > > mind, I'm wondering if some of the Windows file-stat performance problems > > could be bypassed by checking if an open channel exists for some file - > > thought you might need to at least try to position the channel to trigger > > the IDE noticing if it had been deleted and the channel was invalid). > > > > What would be useful to actually measure if my perception matches reality > > would be some performance tests (not startup performance!!) that open a > > large number of projects and files, perform edits, switch between things, > > and take timings for those operations. I have the vague sense that such > > might exist from one or another round of performance tuning years ago, > but > > I have no idea where. > > > > Any ideas? > > > > If anyone wants to play with this stuff, it's on the nio-mfs branch of my > > fork here: > > https://github.com/timboudreau/incubator-netbeans/tree/nio-mfs > > > > (for the time being I just copied the code from three libraries of mine > > that live elsewhere; ideally they would be used as libraries instead, > but > > it was a quick bit of patching to see if it was even useful to try this) > > > > -Tim > > > > -- > > http://timboudreau.com > > > -- http://timboudreau.com
Re: Have we got any *runtime* performance tests?
Hi Tim, If file access is improved that may also improve indexing and by consequence symbol finding or refactoring. Did you try such operations to check the difference in speed? Cheers, JMB On Sun, Aug 18, 2019 at 8:47 AM Tim Boudreau wrote: > I've been playing with some tweaks to masterfs - specifically, I had a > library i wrote for a completely different purpose, which keeps a cache of > open NIO FileChannels. Partly out of curiosity, and partly because I've > wondered for a while what using NIO more for File IO in NetBeans might > accomplish, I dropped it into masterfs. > > Basically, opening a file opens a file channel (the cache key includes the > read/write/truncate options), which is cached, and closed after 60 seconds > if it has not been touched. Input and output streams just take a lock on > the stream, set its state back to what it was when they last touched it, > and then release the lock. > > Subjective impression, after running with these patches for about a week is > that my machine is quieter and the IDE snappier - which makes some sense, > since simply editing a file can trigger a bunch of reads of related file, > so this is dramatically reducing interaction with the OS (in the back of my > mind, I'm wondering if some of the Windows file-stat performance problems > could be bypassed by checking if an open channel exists for some file - > thought you might need to at least try to position the channel to trigger > the IDE noticing if it had been deleted and the channel was invalid). > > What would be useful to actually measure if my perception matches reality > would be some performance tests (not startup performance!!) that open a > large number of projects and files, perform edits, switch between things, > and take timings for those operations. I have the vague sense that such > might exist from one or another round of performance tuning years ago, but > I have no idea where. > > Any ideas? > > If anyone wants to play with this stuff, it's on the nio-mfs branch of my > fork here: > https://github.com/timboudreau/incubator-netbeans/tree/nio-mfs > > (for the time being I just copied the code from three libraries of mine > that live elsewhere; ideally they would be used as libraries instead, but > it was a quick bit of patching to see if it was even useful to try this) > > -Tim > > -- > http://timboudreau.com >
Have we got any *runtime* performance tests?
I've been playing with some tweaks to masterfs - specifically, I had a library i wrote for a completely different purpose, which keeps a cache of open NIO FileChannels. Partly out of curiosity, and partly because I've wondered for a while what using NIO more for File IO in NetBeans might accomplish, I dropped it into masterfs. Basically, opening a file opens a file channel (the cache key includes the read/write/truncate options), which is cached, and closed after 60 seconds if it has not been touched. Input and output streams just take a lock on the stream, set its state back to what it was when they last touched it, and then release the lock. Subjective impression, after running with these patches for about a week is that my machine is quieter and the IDE snappier - which makes some sense, since simply editing a file can trigger a bunch of reads of related file, so this is dramatically reducing interaction with the OS (in the back of my mind, I'm wondering if some of the Windows file-stat performance problems could be bypassed by checking if an open channel exists for some file - thought you might need to at least try to position the channel to trigger the IDE noticing if it had been deleted and the channel was invalid). What would be useful to actually measure if my perception matches reality would be some performance tests (not startup performance!!) that open a large number of projects and files, perform edits, switch between things, and take timings for those operations. I have the vague sense that such might exist from one or another round of performance tuning years ago, but I have no idea where. Any ideas? If anyone wants to play with this stuff, it's on the nio-mfs branch of my fork here: https://github.com/timboudreau/incubator-netbeans/tree/nio-mfs (for the time being I just copied the code from three libraries of mine that live elsewhere; ideally they would be used as libraries instead, but it was a quick bit of patching to see if it was even useful to try this) -Tim -- http://timboudreau.com
