See http://hudson.zones.apache.org/hudson/job/HBase-Patch/92/changes
Changes: [stack] HBASE-586 HRegion runs HStore memcache snapshotting -- fix it so only HStore knows about workings of memcache HStore knows about workings of memcache This patch moves the running of mecache snapshots out of the control of HRegion and hides the memcache details in HStore. This patch also does a bunch of work on the merge tool fixing a bug in the metautils along the way. The merge tool test was failing. We weren't setting into the HLog the maximum sequence id after we'd opened a region -- as HRS does -- so were losing edits. On the way, refactored the merge tool test to get rid of duplicated code. Finally, cleans up logging in HStore to aid debugging; e.g. we always refer to the sequence id as the 'sequence id' in log messages rather than as seqId or 'sequence record' so can sort log as see state of sequence id transitions. Version 2 changes the order in which things are run in memcache. 532 made it so flushing did snapshot and then cleared the snapshot. Now, we snapshot before we flush a store, then inside in the store flush, we call getSnapshot and then clearSnapshot. M src/test/org/apache/hadoop/hbase/regionserver/TestHMemcache.java How snapshotting changed. Change test in accordance. M src/test/org/apache/hadoop/hbase/regionserver/TestHLog.java Use accessor to get file number. M src/test/org/apache/hadoop/hbase/util/TestMergeTool.java Refactored to remove duplicated code so could tell what was going on. (mergeAndVerify, verifyMerge): Addd. M src/java/org/apache/hadoop/hbase/regionserver/Memcache.java (snapshot): Changed so it no longer returns snapshot. M src/java/org/apache/hadoop/hbase/regionserver/HStore.java Changed log messages removing the useless and adding info to others. (this.maxSeqId): We used to add 1 to this in here in HStore. Let HRegion do it. Its the one that does the machinations w/ sequenceids anyways. Make flushes return the amount flushed. Use this updating the regions memcacheSize accounting. (snapshot): Added method for the region to call. M src/java/org/apache/hadoop/hbase/regionserver/HLog.java Made data members private. (getFileNum): Added accessor for tests. M src/java/org/apache/hadoop/hbase/regionserver/HRegion.java Added info to logs.. (snapshotMemcaches): removed. (internalFlushcache): No longer takes startime. Internally now does some of what used happen in snapshotMemcaches including sending of message to stores to snapshot. (getEntrySize): Added method for calculating size of an update. Used by HRegion and flushing so both come up w/ same answer. M src/java/org/apache/hadoop/hbase/util/Merge.java Add logging of whats happening during merges and fail earlier than we used if stuff is not right. Renamed local variables from region1 to r1, etc., so didn't clash with data members of same name. M src/java/org/apache/hadoop/hbase/util/MetaUtils.java Added a TODO ------------------------------------------ started Updating https://svn.apache.org/repos/asf/hadoop/hbase/trunk U CHANGES.txt U src/test/org/apache/hadoop/hbase/regionserver/TestHLog.java U src/test/org/apache/hadoop/hbase/regionserver/TestHMemcache.java U src/test/org/apache/hadoop/hbase/util/TestMergeTool.java U src/java/org/apache/hadoop/hbase/regionserver/Memcache.java U src/java/org/apache/hadoop/hbase/regionserver/HStoreFile.java U src/java/org/apache/hadoop/hbase/regionserver/HStore.java U src/java/org/apache/hadoop/hbase/regionserver/HLog.java U src/java/org/apache/hadoop/hbase/regionserver/HRegion.java U src/java/org/apache/hadoop/hbase/util/Merge.java U src/java/org/apache/hadoop/hbase/util/MetaUtils.java U src/java/org/apache/hadoop/hbase/util/FSUtils.java At revision 650309 [trunk] $ ant -Dtest.junit.output.format=xml -Dtest.output=yes clean jar test package tar Buildfile: build.xml clean: [delete] Deleting directory /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build init: [mkdir] Created dir: /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build [mkdir] Created dir: /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build/classes [mkdir] Created dir: /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build/test [mkdir] Created dir: /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build/examples [mkdir] Created dir: /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build/webapps [copy] Copying 8 files to /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build/webapps [mkdir] Created dir: /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build/lib [copy] Copying 29 files to /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build/lib [mkdir] Created dir: /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build/conf [copy] Copying 5 files to /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build/conf [mkdir] Created dir: /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build/bin [copy] Copying 7 files to /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build/bin jspc: javacc: compile: [javac] Compiling 195 source files to /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build/classes [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. jar: [copy] Copying 1 file to /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build/classes [jar] Building jar: /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build/hbase-0.2.0-dev.jar init: [copy] Copying 2 files to /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build/webapps jspc: BUILD FAILED /zonestorage/hudson/home/hudson/hudson/jobs/HBase-Patch/workspace/trunk/build.xml:178: java.lang.ExceptionInInitializerError Total time: 11 seconds ERROR: No artifacts found that match the file pattern "trunk/build/*.tar.gz". Configuration error? ERROR: 'trunk/build/*.tar.gz' doesn't match anything: 'trunk' exists but not 'trunk/build/*.tar.gz' Recording test results
