have created a patch that addresses the poor cachefile save performance.
This patch addresses this bug, as well as bug 29742.
The revised ModifiedSelector now implements the BuildListener interface.
Upon configuration, it registers itself with the Project as a
BuildListener. The ModifiedSelector is notified of finished BuildEvents,
like taskFinished, targetFinished and buildFinished. Saving of the
cachefile is now delayed until the next finished BuildEvent occurs, most
likely, a taskFinished call.
There is also a new attribute called "delayupdate". It defaults to true
to gain the new performance increase of delaying the save of the
cachefile. Setting the "delayupdate" attribute to false allows continual
cachefile updates ( like the current ModifiedSelector ).
I also ran a sample population of files to note the reduction in time to
cache the file changes:
sample: 6596 image files averaging 31K, total size of 204 MB
the cache is empty, so all files are seen as revised,
this is the most extreme case to show the largest performance gain
MD5 digest mode with continual update: 9 minutes, 8 seconds
MD5 digest mode with delayed update: 36 seconds
CRC checksum mode with continual update: 7 minutes, 36 seconds
CRC checksum mode with delayed update: 43 seconds
On Wed, 23 Jun 2004 [EMAIL PROTECTED] wrote:
> I´ll have a deeper look into that
> (but now I have to ensure that my new computer is able to run - the old was
> damaged :(
>
>
> Jan
>
>
> > -----Ursprüngliche Nachricht-----
> > Von: Robert Rice [mailto:[EMAIL PROTECTED]
> > Gesendet am: Dienstag, 22. Juni 2004 20:23
> > An: [EMAIL PROTECTED]
> > Betreff: Issues with <modified /> selector.
> >
> > Recently, I submitted two bugs related to the modified selector.
> >
> > Bug 29742 addresses the issue that the Modified selector
> > doesn't allow
> > algorithm or comparator selection. Only the default choices
> > of "digest"
> > and "equal" respectively are allowed.
> >
> > I have created a bug fix for the modified selector that
> > addresses this
> > issue.
> > In the bug fix, the algorithm and comparator initialization
> > logic has been
> > changed to allow choosing away from default.
> >
> > At the same time, I have created another algorithm type,
> > "checksum", that
> > makes
> > use of the java.util.zip.Checksum interface. It is chosen by
> > setting the
> > alogrithm attribute to "checksum" ( <modified
> > algorithm="checksum" /> ).
> >
> > This checksum allows the choice of CRC32 or Alder32, which utilize
> > java.util.zip.CRC32 and java.util.zip.Adler32 respectively.
> > This choice
> > is made
> > by setting the algorithm.algorithm parameter to "CRC" or
> > "ADLER", with the
> > default being "CRC" ( <modified algorithm="checksum"><param
> > name="algorithm.algorithm" value="ADLER" /></modified> )
> >
> > How do I submit such changes for evaluation and possible inclusion?
> >
> > Bug 29743 addresses a bigger issue in that the modified
> > selector has poor
> > cachefile save performance. The architecture of the selector
> > framework
> > dictates that the selector is notified of files, one at a
> > time, through
> > the isSelected method. It is at this method call, that the cache
> > properties file is saved. This properties file is saved every time a
> > file modification is found, possibly as many times as there
> > are files in
> > the fileset. This leads to poor performance when process
> > large filesets
> > with multiple changes, in my case around 15000.
> >
> > I would like to delay the saving of the file until we are
> > finished with
> > the fileset or finished with the selector. It looks to me, like this
> > requires the addition of one of more methods to the selector
> > framework,
> > likely to BaseSelector.
> >
> > In the case of the modified selector, we may be able to get away with
> > knowing we are finished with the selector through some method
> > call like
> > "tearDown". At this point, the modified selector could save its
> > properites file.
> >
> > Another option is to add filesetListener type of behavior to
> > the BaseSelector.
> > This would notify a selector when a fileset selection has started and
> > ended. With this option, the modified selector could save its
> > properties
> > file at the end of fileset selection.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
