Thanks for all the proposals. Kristian had a good idea which I try to work on now.
Instead of having an own Scanner for this, our good ole DirectoryScanner might return a ScanResult. This ScanResult can offer a diff method or something similar. Will do some experiments. LieGrue, strub ----- Original Message ----- > From: Hervé BOUTEMY <[email protected]> > To: Maven Developers List <[email protected]> > Cc: > Sent: Saturday, September 15, 2012 11:17 AM > Subject: Re: svn commit: r1384747 - in > /maven/shared/trunk/maven-shared-utils/src: > main/java/org/apache/maven/shared/utils/io/DirectorySnapshotScanner.java > test/java/org/apache/maven/shared/utils/io/DirectorySnapshotScannerTest.java > >& quot;Listing"? > > Le vendredi 14 septembre 2012 20:04:23 Anders Hammar a écrit : >> Yes, "Snapshot" is not good. "PointInTime"? >> >> /Anders >> >> On Fri, Sep 14, 2012 at 3:03 PM, Mark Struberg <[email protected]> > wrote: >> > Hi! >> > >> > If anyone has a better name for the 'DirectorySnapshotScanner' > I would be >> > happy to rename it. >> > >> > >> > What it makes: it takes a snapshot capture of a directory structure > and >> > compares it with another snapshot capture of that directory to > calculate >> > a diff (files added/removed) >> > >> > Not sure though if the name 'snapshot' isn't too heavily > associated with >> > dependency-snapshots, wdyt? >> > >> > >> > We will need this for the incremental build support. >> > >> > LieGrue, >> > strub >> > >> > >> > >> > ----- Original Message ----- >> > >> >> From: "[email protected]" <[email protected]> >> >> To: [email protected] >> >> Cc: >> >> Sent: Friday, September 14, 2012 2:57 PM >> >> Subject: svn commit: r1384747 - in >> >> /maven/shared/trunk/maven-shared-utils/src: >> >> > main/java/org/apache/maven/shared/utils/io/DirectorySnapshotScanner.java >> >> > test/java/org/apache/maven/shared/utils/io/DirectorySnapshotScannerTest. >> >> java >> >> >> >> Author: struberg >> >> Date: Fri Sep 14 12:57:24 2012 >> >> New Revision: 1384747 >> >> >> >> URL: http://svn.apache.org/viewvc?rev=1384747&view=rev >> >> Log: >> >> MSHARED-243 add DirectorySnapshotScanner >> >> >> >> This allows to take snapshot captures of a directory and >> >> calculate the 'diff' between those (files added/removed). >> >> >> >> Added: >> >> >> >> > maven/shared/trunk/maven-shared-utils/src/main/java/org/apache/maven/shar >> >> ed/utils/io/DirectorySnapshotScanner.java (with props) >> >> >> >> > maven/shared/trunk/maven-shared-utils/src/test/java/org/apache/maven/shar >> >> ed/utils/io/DirectorySnapshotScannerTest.java (with props) >> >> >> >> Added: >> >> > maven/shared/trunk/maven-shared-utils/src/main/java/org/apache/maven/shar >> >> ed/utils/io/DirectorySnapshotScanner.java URL: >> >> > http://svn.apache.org/viewvc/maven/shared/trunk/maven-shared-utils/src/ma >> >> > in/java/org/apache/maven/shared/utils/io/DirectorySnapshotScanner.java?re >> >> v=1384747&view=auto >> >> > ======================================================================== >> >> ====== --- >> >> > maven/shared/trunk/maven-shared-utils/src/main/java/org/apache/maven/shar >> >> ed/utils/io/DirectorySnapshotScanner.java (added) >> >> +++ >> >> > maven/shared/trunk/maven-shared-utils/src/main/java/org/apache/maven/shar >> >> ed/utils/io/DirectorySnapshotScanner.java Fri Sep 14 12:57:24 2012 >> >> @@ -0,0 +1,139 @@ >> >> +package org.apache.maven.shared.utils.io; >> >> + >> >> +/* >> >> + * Licensed to the Apache Software Foundation (ASF) under one >> >> + * or more contributor license agreements. See the NOTICE file >> >> + * distributed with this work for additional information >> >> + * regarding copyright ownership. The ASF licenses this file >> >> + * to you under the Apache License, Version 2.0 (the >> >> + * "License"); you may not use this file except in > compliance >> >> + * with the License. You may obtain a copy of the License at >> >> + * >> >> + * http://www.apache.org/licenses/LICENSE-2.0 >> >> + * >> >> + * Unless required by applicable law or agreed to in writing, >> >> + * software distributed under the License is distributed on an >> >> + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF > ANY >> >> + * KIND, either express or implied. See the License for the >> >> + * specific language governing permissions and limitations >> >> + * under the License. >> >> + */ >> >> + >> >> +import org.apache.maven.shared.utils.CollectionUtils; >> >> + >> >> +import java.io.File; >> >> +import java.util.ArrayList; >> >> +import java.util.List; >> >> +import java.util.Set; >> >> + >> >> +/** >> >> + * Scan for files in a directory at a given time and reports > removed and >> >> added files >> >> + * between captures. >> >> + */ >> >> +public class DirectorySnapshotScanner >> >> +{ >> >> + private final static String[] NO_FILES = new String[0]; >> >> + >> >> + private File rootDirectory; >> >> + private String[] scannedOldFiles; >> >> + private String[] scannedNewFiles = NO_FILES; >> >> + private String[] filesAdded; >> >> + private String[] filesRemoved; >> >> + >> >> + public DirectorySnapshotScanner( File rootDirectory ) >> >> + { >> >> + this.rootDirectory = rootDirectory; >> >> + } >> >> + >> >> + public void capture() >> >> + { >> >> + DirectoryScanner ds = new DirectoryScanner(); >> >> + capture( ds ); >> >> + } >> >> + >> >> + public void capture( DirectoryScanner ds ) >> >> + { >> >> + ds.setBasedir( rootDirectory ); >> >> + ds.scan(); >> >> + >> >> + scannedNewFiles = ds.getIncludedFiles(); >> >> + >> >> + if ( scannedOldFiles != null ) >> >> + { >> >> + calculateDiff( scannedOldFiles, scannedNewFiles ); >> >> + >> >> + } >> >> + else >> >> + { >> >> + filesAdded = NO_FILES; >> >> + filesRemoved = NO_FILES; >> >> + } >> >> + >> >> + // this attempts new files is the baseline for the next > scan >> >> + scannedOldFiles = scannedNewFiles; >> >> + } >> >> + >> >> + /** >> >> + * @return all files which got scanned during the last > capture. >> >> + */ >> >> + public String[] getScannedFiles() >> >> + { >> >> + return scannedNewFiles; >> >> + } >> >> + >> >> + /** >> >> + * @return all files which got detected as being added > between 2 >> >> capture calls >> >> + */ >> >> + public String[] getFilesAdded() >> >> + { >> >> + return filesAdded; >> >> + } >> >> + >> >> + /** >> >> + * @return all files which got detected as being removed > between 2 >> >> capture calls >> >> + */ >> >> + public String[] getFilesRemoved() >> >> + { >> >> + return filesRemoved; >> >> + } >> >> + >> >> + /** >> >> + * Determine the file differences between the oldFiles and > newFiles. >> >> + * This method will not look for a changed in content but > sole in >> >> the >> >> + * list of files given. >> >> + * >> >> + * The result of the diff can be queried by the methods >> >> + * {@link #getFilesAdded()} and {@link #getFilesRemoved()} >> >> + * >> >> + * @param oldFiles >> >> + * @param newFiles >> >> + */ >> >> + public void calculateDiff( String[] oldFiles, String[] > newFiles ) >> >> + { >> >> + Set<String> oldFileSet = > CollectionUtils.arrayAsHashSet( >> >> oldFiles >> >> ); >> >> + Set<String> newFileSet = > CollectionUtils.arrayAsHashSet( >> >> newFiles >> >> ); >> >> + >> >> + List<String> added = new ArrayList<String>(); >> >> + List<String> removed = new > ArrayList<String>(); >> >> + >> >> + for ( String oldFile : oldFileSet ) >> >> + { >> >> + if ( !newFileSet.contains( oldFile ) ) >> >> + { >> >> + removed.add( oldFile ); >> >> + } >> >> + } >> >> + >> >> + for ( String newFile : newFileSet ) >> >> + { >> >> + if ( !oldFileSet.contains( newFile ) ) >> >> + { >> >> + added.add( newFile ); >> >> + } >> >> + } >> >> + >> >> + filesAdded = added.toArray( new String[ added.size() ] ); >> >> + filesRemoved = removed.toArray( new String[ > removed.size() ] ); >> >> + } >> >> + >> >> +} >> >> >> >> Propchange: >> >> > maven/shared/trunk/maven-shared-utils/src/main/java/org/apache/maven/shar >> >> ed/utils/io/DirectorySnapshotScanner.java >> >> > ------------------------------------------------------------------------ >> >> ------>> >> >> svn:eol-style = native >> >> >> >> Added: >> >> > maven/shared/trunk/maven-shared-utils/src/test/java/org/apache/maven/shar >> >> ed/utils/io/DirectorySnapshotScannerTest.java URL: >> >> > http://svn.apache.org/viewvc/maven/shared/trunk/maven-shared-utils/src/te >> >> > st/java/org/apache/maven/shared/utils/io/DirectorySnapshotScannerTest.jav >> >> a?rev=1384747&view=auto >> >> > ======================================================================== >> >> ====== --- >> >> > maven/shared/trunk/maven-shared-utils/src/test/java/org/apache/maven/shar >> >> ed/utils/io/DirectorySnapshotScannerTest.java (added) >> >> +++ >> >> > maven/shared/trunk/maven-shared-utils/src/test/java/org/apache/maven/shar >> >> ed/utils/io/DirectorySnapshotScannerTest.java Fri Sep 14 12:57:24 > 2012 >> >> @@ -0,0 +1,161 @@ >> >> +package org.apache.maven.shared.utils.io; >> >> + >> >> +/* >> >> + * Licensed to the Apache Software Foundation (ASF) under one >> >> + * or more contributor license agreements. See the NOTICE file >> >> + * distributed with this work for additional information >> >> + * regarding copyright ownership. The ASF licenses this file >> >> + * to you under the Apache License, Version 2.0 (the >> >> + * "License"); you may not use this file except in > compliance >> >> + * with the License. You may obtain a copy of the License at >> >> + * >> >> + * http://www.apache.org/licenses/LICENSE-2.0 >> >> + * >> >> + * Unless required by applicable law or agreed to in writing, >> >> + * software distributed under the License is distributed on an >> >> + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF > ANY >> >> + * KIND, either express or implied. See the License for the >> >> + * specific language governing permissions and limitations >> >> + * under the License. >> >> + */ >> >> + >> >> + >> >> +import org.apache.maven.shared.utils.testhelpers.FileTestHelper; >> >> +import org.junit.Test; >> >> +import org.junit.Rule; >> >> +import org.junit.Ignore; >> >> +import org.junit.Assert; >> >> +import org.junit.rules.TemporaryFolder; >> >> + >> >> +import java.io.File; >> >> +import java.io.IOException; >> >> + >> >> +public class DirectorySnapshotScannerTest >> >> +{ >> >> + >> >> + @Rule >> >> + public TemporaryFolder tempFolder = new TemporaryFolder(); >> >> + >> >> + private void createTestData() throws IOException >> >> + { >> >> + File rootDir = tempFolder.getRoot(); >> >> + File folder1 = new File( rootDir, "folder1" ); >> >> + folder1.mkdirs(); >> >> + >> >> + FileTestHelper.generateTestFile( new File( rootDir, >> >> "file1.txt" ), 11 ); >> >> + FileTestHelper.generateTestFile( new File( rootDir, >> >> "file2.txt" ), 12 ); >> >> + FileTestHelper.generateTestFile( new File( rootDir, >> >> "file3.dat" ), 13 ); >> >> + >> >> + FileTestHelper.generateTestFile( new File( folder1, >> >> "file4.txt" ), 14 ); >> >> + FileTestHelper.generateTestFile( new File( folder1, >> >> "file5.dat" ), 15 ); >> >> + >> >> + File folder2 = new File( folder1, > "ignorefolder" ); >> >> + folder2.mkdirs(); >> >> + FileTestHelper.generateTestFile( new File( folder2, >> >> "file7.txt" ), 17 ); >> >> + >> >> + } >> >> + >> >> + private void removeAndAddSomeFiles() throws IOException >> >> + { >> >> + File rootDir = tempFolder.getRoot(); >> >> + File file2 = new File( rootDir, "file2.txt" ); >> >> + file2.delete(); >> >> + >> >> + FileTestHelper.generateTestFile( new File( rootDir, >> >> "folder1/file9.txt" ), 15 ); >> >> + >> >> + File folder2 = new File( rootDir, > "folder1/ignorefolder" ); >> >> + FileUtils.deleteDirectory( folder2 ); >> >> + } >> >> + >> >> + @Test >> >> + public void testInitialScan() throws Exception >> >> + { >> >> + createTestData(); >> >> + >> >> + DirectorySnapshotScanner dss = new > DirectorySnapshotScanner( >> >> tempFolder.getRoot() ); >> >> + Assert.assertNotNull( dss ); >> >> + >> >> + // we take the initial snapshot which should result in an > empty >> >> diff + dss.capture(); >> >> + >> >> + //X TODO define result of the initial scan. Full > directory tree >> >> or just an empty array? >> >> + String[] addedFiles = dss.getFilesAdded(); >> >> + String[] removedFiles = dss.getFilesRemoved(); >> >> + Assert.assertNotNull( addedFiles ); >> >> + Assert.assertNotNull( removedFiles ); >> >> + Assert.assertEquals( 0, addedFiles.length ); >> >> + Assert.assertEquals(0, removedFiles.length ); >> >> + >> >> + // now we change 3 files. add one and remove >> >> + removeAndAddSomeFiles(); >> >> + >> >> + dss.capture(); >> >> + >> >> + addedFiles = dss.getFilesAdded(); >> >> + removedFiles = dss.getFilesRemoved(); >> >> + Assert.assertNotNull( addedFiles ); >> >> + Assert.assertNotNull( removedFiles ); >> >> + Assert.assertEquals( 1, addedFiles.length ); >> >> + Assert.assertEquals(2, removedFiles.length ); >> >> + >> >> + String[] allFiles = dss.getScannedFiles(); >> >> + Assert.assertNotNull( allFiles ); >> >> + Assert.assertEquals(5, allFiles.length ); >> >> + >> >> + } >> >> + >> >> + >> >> + >> >> + @Ignore("Enable this test to run performance > checks") >> >> + @Test >> >> + public void performanceTest() throws Exception { >> >> + >> >> + File rootFolder = tempFolder.getRoot(); >> >> + >> >> + // do some warmup >> >> + for ( int i = 1; i < 200; i++ ) >> >> + { >> >> + createTestData(); >> >> + removeAndAddSomeFiles(); >> >> + FileUtils.deleteDirectory( rootFolder ); >> >> + } >> >> + >> >> + int cycles = 2000; >> >> + >> >> + // and now we take the time _without_ >> >> + long startTime = System.nanoTime(); >> >> + for ( int i = 1; i < cycles; i++ ) >> >> + { >> >> + createTestData(); >> >> + removeAndAddSomeFiles(); >> >> + FileUtils.deleteDirectory( rootFolder ); >> >> + rootFolder.mkdir(); >> >> + } >> >> + long endTime = System.nanoTime(); >> >> + >> >> + long durationEmptyRun = endTime - startTime; >> >> + System.out.println( "durationEmptyRun > [ns]: " + >> >> durationEmptyRun); >> >> + >> >> + startTime = System.nanoTime(); >> >> + for ( int i = 1; i < cycles; i++ ) >> >> + { >> >> + createTestData(); >> >> + DirectorySnapshotScanner dss = new > DirectorySnapshotScanner( >> >> rootFolder ); >> >> + dss.capture(); >> >> + removeAndAddSomeFiles(); >> >> + dss.capture(); >> >> + >> >> + FileUtils.deleteDirectory( rootFolder ); >> >> + rootFolder.mkdir(); >> >> + } >> >> + endTime = System.nanoTime(); >> >> + >> >> + long durationWithSnapshotScanner = endTime - startTime; >> >> + System.out.println( "durationWithSnapshotScanner > [ns]: " + >> >> durationWithSnapshotScanner); >> >> + >> >> + long dirScannerOverhead = durationWithSnapshotScanner - >> >> durationEmptyRun; >> >> + >> >> + System.out.println( "Overhead for n cycles [ns]: > " + >> >> dirScannerOverhead); >> >> + } >> >> + >> >> +} >> >> >> >> Propchange: >> >> > maven/shared/trunk/maven-shared-utils/src/test/java/org/apache/maven/shar >> >> ed/utils/io/DirectorySnapshotScannerTest.java >> >> > ------------------------------------------------------------------------ >> >> ------>> >> >> svn:eol-style = native >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: [email protected] >> > For additional commands, e-mail: [email protected] >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
