[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[ https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136626#comment-13136626 ] Hudson commented on HBASE-4634: --- Integrated in HBase-0.92 #83 (See [https://builds.apache.org/job/HBase-0.92/83/]) HBASE-4634 'test.build.data' property overused leading to write data at the wrong place stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestFSTableDescriptorForceCreation.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestHBaseTestingUtility.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestInfoServers.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestMultiVersions.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestHTablePool.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/replication/TestReplicationAdmin.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/filter/TestColumnPrefixFilter.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/filter/TestDependentColumnFilter.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/filter/TestMultipleColumnPrefixFilter.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestFixedFileTrailer.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapred/TestTableMapReduce.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMasterRestartAfterDisablingTable.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMasterTransitions.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestOpenedRegionHandler.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestColumnSeeking.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java *
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[ https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136701#comment-13136701 ] Hudson commented on HBASE-4634: --- Integrated in HBase-TRUNK #2372 (See [https://builds.apache.org/job/HBase-TRUNK/2372/]) HBASE-4634 'test.build.data' property overused leading to write data at the wrong place stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestFSTableDescriptorForceCreation.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestHBaseTestingUtility.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestInfoServers.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestMultiVersions.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestHTablePool.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/replication/TestReplicationAdmin.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/filter/TestColumnPrefixFilter.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/filter/TestDependentColumnFilter.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/filter/TestMultipleColumnPrefixFilter.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestFixedFileTrailer.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapred/TestTableMapReduce.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterRestartAfterDisablingTable.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterTransitions.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestOpenedRegionHandler.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestColumnSeeking.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestGetClosestAtOrBefore.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java *
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[ https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134171#comment-13134171 ] stack commented on HBASE-4634: -- @nkeywal Mind refreshing your TRUNK? Patch did not apply cleanly and I got this when trying to build it: {code} [INFO] [ERROR] BUILD FAILURE [INFO] [INFO] Compilation failure /Users/stack/checkout/clean-trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java:[265,8] cannot find symbol symbol: variable ShowTime ShowTime.start(); /Users/stack/checkout/clean-trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java:[270,8] cannot find symbol symbol: variable ShowTime ShowTime.pub(public void run() ); /Users/stack/checkout/clean-trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java:[401,4] cannot find symbol symbol : variable ShowTime location: class org.apache.hadoop.hbase.master.TestDistributedLogSplitting /Users/stack/checkout/clean-trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java:[403,4] cannot find symbol symbol : variable ShowTime location: class org.apache.hadoop.hbase.master.TestDistributedLogSplitting /Users/stack/checkout/clean-trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java:[228,39] cannot find symbol symbol : method getTestDir() location: class org.apache.hadoop.hbase.HBaseTestingUtility {code} Thanks boss. test.build.data property overused leading to write data at the wrong place Key: HBASE-4634 URL: https://issues.apache.org/jira/browse/HBASE-4634 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.0 Environment: all Reporter: nkeywal Assignee: nkeywal Attachments: 2011020_4634_all.patch, 20111020_4639_TestStoreFile.patch, 20111021_4634_all.v2.patch, 20111022_4634_all.v3.patch, 20111023_4634_all.v4.patch test.build.data is overloaded in HBase.At the beginning, it's the Default parent directory for test output., but then it's rewritten to be the directory itself in functions like HBaseTestingUtility#startMiniDFSCluster It seems that this value is already used by MiniDFS (i.e. outside of HBase): Name is as it is because mini dfs has hard-codings to put test data here. As it is today, there is at least a bug in HBaseTestingUtility: {noformat} public void initTestDir() { if (System.getProperty(TEST_DIRECTORY_KEY) == null) { clusterTestBuildDir = setupClusterTestBuildDir(); System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath()); } } {noformat} if you set a value for test.build.data, the test dir will be the parent directory and not a temp subdir, leading to issues as multiple tests will end-ups in the same (bad) directory. This function is barely used today, hence it's not visible, but I would like to use it in some new code. A possible fix is to remove the check for null and continue with the overloading, but I don't think it would be a big issue to create a new key(like test.build.data.rootdirectory) specific to the root directory and to use test.build.data only to communicate with MiniDFS. Feedback welcome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[ https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134181#comment-13134181 ] nkeywal commented on HBASE-4634: that's an error I missed in my patch. Ok to refresh. On Mon, Oct 24, 2011 at 5:51 PM, stack (Commented) (JIRA) test.build.data property overused leading to write data at the wrong place Key: HBASE-4634 URL: https://issues.apache.org/jira/browse/HBASE-4634 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.0 Environment: all Reporter: nkeywal Assignee: nkeywal Attachments: 2011020_4634_all.patch, 20111020_4639_TestStoreFile.patch, 20111021_4634_all.v2.patch, 20111022_4634_all.v3.patch, 20111023_4634_all.v4.patch test.build.data is overloaded in HBase.At the beginning, it's the Default parent directory for test output., but then it's rewritten to be the directory itself in functions like HBaseTestingUtility#startMiniDFSCluster It seems that this value is already used by MiniDFS (i.e. outside of HBase): Name is as it is because mini dfs has hard-codings to put test data here. As it is today, there is at least a bug in HBaseTestingUtility: {noformat} public void initTestDir() { if (System.getProperty(TEST_DIRECTORY_KEY) == null) { clusterTestBuildDir = setupClusterTestBuildDir(); System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath()); } } {noformat} if you set a value for test.build.data, the test dir will be the parent directory and not a temp subdir, leading to issues as multiple tests will end-ups in the same (bad) directory. This function is barely used today, hence it's not visible, but I would like to use it in some new code. A possible fix is to remove the check for null and continue with the overloading, but I don't think it would be a big issue to create a new key(like test.build.data.rootdirectory) specific to the root directory and to use test.build.data only to communicate with MiniDFS. Feedback welcome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[ https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13133415#comment-13133415 ] nkeywal commented on HBASE-4634: v4 coming after: - some impacts of shutdown throwing Exception instead of IOException were not taken into account - startZK before starting the cluster was puting the file in the current directory (old bug I think). Full test in progress. test.build.data property overused leading to write data at the wrong place Key: HBASE-4634 URL: https://issues.apache.org/jira/browse/HBASE-4634 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.0 Environment: all Reporter: nkeywal Assignee: nkeywal Attachments: 2011020_4634_all.patch, 20111020_4639_TestStoreFile.patch, 20111021_4634_all.v2.patch, 20111022_4634_all.v3.patch test.build.data is overloaded in HBase.At the beginning, it's the Default parent directory for test output., but then it's rewritten to be the directory itself in functions like HBaseTestingUtility#startMiniDFSCluster It seems that this value is already used by MiniDFS (i.e. outside of HBase): Name is as it is because mini dfs has hard-codings to put test data here. As it is today, there is at least a bug in HBaseTestingUtility: {noformat} public void initTestDir() { if (System.getProperty(TEST_DIRECTORY_KEY) == null) { clusterTestBuildDir = setupClusterTestBuildDir(); System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath()); } } {noformat} if you set a value for test.build.data, the test dir will be the parent directory and not a temp subdir, leading to issues as multiple tests will end-ups in the same (bad) directory. This function is barely used today, hence it's not visible, but I would like to use it in some new code. A possible fix is to remove the check for null and continue with the overloading, but I don't think it would be a big issue to create a new key(like test.build.data.rootdirectory) specific to the root directory and to use test.build.data only to communicate with MiniDFS. Feedback welcome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[ https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13133045#comment-13133045 ] nkeywal commented on HBASE-4634: I am going to attach a second version of the patch. I checked indent, it seems ok to me. It took more time than expected because I haven't managed to escape some modifications to the get files functions. Not much impacts for the user. Basically, there is now a clear distinction between: - base directory: configurable directory where all tests put their data - data directory: sub directory used by the test - cluster directory: subdirectory for the cluster stuff (dfs or hbase) This will help to have multiple HBaseTestingUtility in the same JVM (// or not). I have also removed a duplicate getter (something as getHbaseCluster while getHBase cluster exists). It includes the fixes for HBASE-4639 and most if not all the bad usage mentioned in HBASE-4326 (but I did not fix the root issue of HBASE-4326). It seems to work on my env, currently n-checking again. I have a systematic errors in the test, but it's in #org.apache.hadoop.hbase.io.hfile.TestHFileBlock#testBlockHeapSize , I believe it was there at the beginning. I have random errors in: TestDistributedLogSplitting, regionserver.TestServerCustomProtocol, client.TestAdmin master.TestHMasterRPCException, coprocessor.TestRegionServerCoprocessorExceptionWithAbort, wal.TestLogRolling, CatalogTrackerOnCluster and a few else, but it works when I replay the test set, so I tend to think they belong to the flaky test sets. At the end of the day, the patch is not that medium... Sorry for that. Will add a new comment when the tests will be over and I will have rechecked the flaky one, but you can already have a look. test.build.data property overused leading to write data at the wrong place Key: HBASE-4634 URL: https://issues.apache.org/jira/browse/HBASE-4634 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.0 Environment: all Reporter: nkeywal Assignee: nkeywal Attachments: 2011020_4634_all.patch, 20111020_4639_TestStoreFile.patch test.build.data is overloaded in HBase.At the beginning, it's the Default parent directory for test output., but then it's rewritten to be the directory itself in functions like HBaseTestingUtility#startMiniDFSCluster It seems that this value is already used by MiniDFS (i.e. outside of HBase): Name is as it is because mini dfs has hard-codings to put test data here. As it is today, there is at least a bug in HBaseTestingUtility: {noformat} public void initTestDir() { if (System.getProperty(TEST_DIRECTORY_KEY) == null) { clusterTestBuildDir = setupClusterTestBuildDir(); System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath()); } } {noformat} if you set a value for test.build.data, the test dir will be the parent directory and not a temp subdir, leading to issues as multiple tests will end-ups in the same (bad) directory. This function is barely used today, hence it's not visible, but I would like to use it in some new code. A possible fix is to remove the check for null and continue with the overloading, but I don't think it would be a big issue to create a new key(like test.build.data.rootdirectory) specific to the root directory and to use test.build.data only to communicate with MiniDFS. Feedback welcome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[ https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13133048#comment-13133048 ] Ted Yu commented on HBASE-4634: --- TestHFileBlock#testBlockHeapSize is fixed by addendum to HBASE-4219 Please refresh your workspace. test.build.data property overused leading to write data at the wrong place Key: HBASE-4634 URL: https://issues.apache.org/jira/browse/HBASE-4634 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.0 Environment: all Reporter: nkeywal Assignee: nkeywal Attachments: 2011020_4634_all.patch, 20111020_4639_TestStoreFile.patch, 20111021_4634_all.v2.patch test.build.data is overloaded in HBase.At the beginning, it's the Default parent directory for test output., but then it's rewritten to be the directory itself in functions like HBaseTestingUtility#startMiniDFSCluster It seems that this value is already used by MiniDFS (i.e. outside of HBase): Name is as it is because mini dfs has hard-codings to put test data here. As it is today, there is at least a bug in HBaseTestingUtility: {noformat} public void initTestDir() { if (System.getProperty(TEST_DIRECTORY_KEY) == null) { clusterTestBuildDir = setupClusterTestBuildDir(); System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath()); } } {noformat} if you set a value for test.build.data, the test dir will be the parent directory and not a temp subdir, leading to issues as multiple tests will end-ups in the same (bad) directory. This function is barely used today, hence it's not visible, but I would like to use it in some new code. A possible fix is to remove the check for null and continue with the overloading, but I don't think it would be a big issue to create a new key(like test.build.data.rootdirectory) specific to the root directory and to use test.build.data only to communicate with MiniDFS. Feedback welcome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[ https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131722#comment-13131722 ] Ted Yu commented on HBASE-4634: --- From MiniDFSCluster.java: {code} public static String getBaseDirectory() { return System.getProperty(PROP_TEST_BUILD_DATA, build/test/data) + /dfs/; } {code} I think we can introduce a new key for the test data used by hbase. test.build.data property overused leading to write data at the wrong place Key: HBASE-4634 URL: https://issues.apache.org/jira/browse/HBASE-4634 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.0 Environment: all Reporter: nkeywal Assignee: nkeywal test.build.data is overloaded in HBase.At the beginning, it's the Default parent directory for test output., but then it's rewritten to be the directory itself in functions like HBaseTestingUtility#startMiniDFSCluster It seems that this value is already used by MiniDFS (i.e. outside of HBase): Name is as it is because mini dfs has hard-codings to put test data here. As it is today, there is at least a bug in HBaseTestingUtility: {noformat} public void initTestDir() { if (System.getProperty(TEST_DIRECTORY_KEY) == null) { clusterTestBuildDir = setupClusterTestBuildDir(); System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath()); } } {noformat} if you set a value for test.build.data, the test dir will be the parent directory and not a temp subdir, leading to issues as multiple tests will end-ups in the same (bad) directory. This function is barely used today, hence it's not visible, but I would like to use it in some new code. A possible fix is to remove the check for null and continue with the overloading, but I don't think it would be a big issue to create a new key(like test.build.data.rootdirectory) specific to the root directory and to use test.build.data only to communicate with MiniDFS. Feedback welcome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[ https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131912#comment-13131912 ] nkeywal commented on HBASE-4634: Ok, I've implemented something, currently under testing. test.build.data property overused leading to write data at the wrong place Key: HBASE-4634 URL: https://issues.apache.org/jira/browse/HBASE-4634 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.0 Environment: all Reporter: nkeywal Assignee: nkeywal test.build.data is overloaded in HBase.At the beginning, it's the Default parent directory for test output., but then it's rewritten to be the directory itself in functions like HBaseTestingUtility#startMiniDFSCluster It seems that this value is already used by MiniDFS (i.e. outside of HBase): Name is as it is because mini dfs has hard-codings to put test data here. As it is today, there is at least a bug in HBaseTestingUtility: {noformat} public void initTestDir() { if (System.getProperty(TEST_DIRECTORY_KEY) == null) { clusterTestBuildDir = setupClusterTestBuildDir(); System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath()); } } {noformat} if you set a value for test.build.data, the test dir will be the parent directory and not a temp subdir, leading to issues as multiple tests will end-ups in the same (bad) directory. This function is barely used today, hence it's not visible, but I would like to use it in some new code. A possible fix is to remove the check for null and continue with the overloading, but I don't think it would be a big issue to create a new key(like test.build.data.rootdirectory) specific to the root directory and to use test.build.data only to communicate with MiniDFS. Feedback welcome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[ https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131987#comment-13131987 ] stack commented on HBASE-4634: -- Please use the same indent as the surrounding file (two spaces for a tab). Should we deprecate ' public static final String TEST_DIRECTORY_KEY = test.build.data;' so we can later change it to private or protected access? 'root' is a pretty overloaded term in these parts. There is unix root and then there is hbase root region and hbase.rootdir. You are using ROOT_TEST_DIRECTORY in place of TEST_DIRECTORY. Maybe BASE_TEST_DIRECTORY instead? FYI, the surrounding code has spaces between operators. e.g. '+if (clusterTestBuildDir==null){' should be '+if (clusterTestBuildDir == null) {' Otherwise, patch is going in right direction IMO. Good stuff. I'm not sure what to do with the patch 20111020_4639_TestStoreFile.patch. Is it to be applied here or over in hbase-4639? test.build.data property overused leading to write data at the wrong place Key: HBASE-4634 URL: https://issues.apache.org/jira/browse/HBASE-4634 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.0 Environment: all Reporter: nkeywal Assignee: nkeywal Attachments: 2011020_4634_all.patch, 20111020_4639_TestStoreFile.patch test.build.data is overloaded in HBase.At the beginning, it's the Default parent directory for test output., but then it's rewritten to be the directory itself in functions like HBaseTestingUtility#startMiniDFSCluster It seems that this value is already used by MiniDFS (i.e. outside of HBase): Name is as it is because mini dfs has hard-codings to put test data here. As it is today, there is at least a bug in HBaseTestingUtility: {noformat} public void initTestDir() { if (System.getProperty(TEST_DIRECTORY_KEY) == null) { clusterTestBuildDir = setupClusterTestBuildDir(); System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath()); } } {noformat} if you set a value for test.build.data, the test dir will be the parent directory and not a temp subdir, leading to issues as multiple tests will end-ups in the same (bad) directory. This function is barely used today, hence it's not visible, but I would like to use it in some new code. A possible fix is to remove the check for null and continue with the overloading, but I don't think it would be a big issue to create a new key(like test.build.data.rootdirectory) specific to the root directory and to use test.build.data only to communicate with MiniDFS. Feedback welcome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[ https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13132003#comment-13132003 ] nkeywal commented on HBASE-4634: For TestStoreFile, It would need to be applied here, even it's doing more than just removing the static dependency on the TestingTool : it includes 4639. It will take into account your comments and publish an updated version. - with BASE_TEST_DIRECTORY - private for TEST_DIRECTORY_KEY should to it, I will try. + indent stuff On Thu, Oct 20, 2011 at 10:25 PM, stack (Commented) (JIRA) test.build.data property overused leading to write data at the wrong place Key: HBASE-4634 URL: https://issues.apache.org/jira/browse/HBASE-4634 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.0 Environment: all Reporter: nkeywal Assignee: nkeywal Attachments: 2011020_4634_all.patch, 20111020_4639_TestStoreFile.patch test.build.data is overloaded in HBase.At the beginning, it's the Default parent directory for test output., but then it's rewritten to be the directory itself in functions like HBaseTestingUtility#startMiniDFSCluster It seems that this value is already used by MiniDFS (i.e. outside of HBase): Name is as it is because mini dfs has hard-codings to put test data here. As it is today, there is at least a bug in HBaseTestingUtility: {noformat} public void initTestDir() { if (System.getProperty(TEST_DIRECTORY_KEY) == null) { clusterTestBuildDir = setupClusterTestBuildDir(); System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath()); } } {noformat} if you set a value for test.build.data, the test dir will be the parent directory and not a temp subdir, leading to issues as multiple tests will end-ups in the same (bad) directory. This function is barely used today, hence it's not visible, but I would like to use it in some new code. A possible fix is to remove the check for null and continue with the overloading, but I don't think it would be a big issue to create a new key(like test.build.data.rootdirectory) specific to the root directory and to use test.build.data only to communicate with MiniDFS. Feedback welcome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira