[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[ https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136626#comment-13136626 ] Hudson commented on HBASE-4634: --- Integrated in HBase-0.92 #83 (See [https://builds.apache.org/job/HBase-0.92/83/]) HBASE-4634 'test.build.data' property overused leading to write data at the wrong place stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestFSTableDescriptorForceCreation.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestHBaseTestingUtility.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestInfoServers.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestMultiVersions.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestHTablePool.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/replication/TestReplicationAdmin.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/filter/TestColumnPrefixFilter.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/filter/TestDependentColumnFilter.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/filter/TestMultipleColumnPrefixFilter.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestFixedFileTrailer.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapred/TestTableMapReduce.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMasterRestartAfterDisablingTable.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMasterTransitions.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestOpenedRegionHandler.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestColumnSeeking.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java *
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[ https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136701#comment-13136701 ] Hudson commented on HBASE-4634: --- Integrated in HBase-TRUNK #2372 (See [https://builds.apache.org/job/HBase-TRUNK/2372/]) HBASE-4634 'test.build.data' property overused leading to write data at the wrong place stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestFSTableDescriptorForceCreation.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestHBaseTestingUtility.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestInfoServers.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestMultiVersions.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestHTablePool.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/replication/TestReplicationAdmin.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/filter/TestColumnPrefixFilter.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/filter/TestDependentColumnFilter.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/filter/TestMultipleColumnPrefixFilter.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestFixedFileTrailer.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapred/TestTableMapReduce.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterRestartAfterDisablingTable.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterTransitions.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestOpenedRegionHandler.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestColumnSeeking.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestGetClosestAtOrBefore.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java *
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[
https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134171#comment-13134171
]
stack commented on HBASE-4634:
--
@nkeywal Mind refreshing your TRUNK? Patch did not apply cleanly and I got
this when trying to build it:
{code}
[INFO]
[ERROR] BUILD FAILURE
[INFO]
[INFO] Compilation failure
/Users/stack/checkout/clean-trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java:[265,8]
cannot find symbol
symbol: variable ShowTime
ShowTime.start();
/Users/stack/checkout/clean-trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java:[270,8]
cannot find symbol
symbol: variable ShowTime
ShowTime.pub(public void run() );
/Users/stack/checkout/clean-trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java:[401,4]
cannot find symbol
symbol : variable ShowTime
location: class org.apache.hadoop.hbase.master.TestDistributedLogSplitting
/Users/stack/checkout/clean-trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java:[403,4]
cannot find symbol
symbol : variable ShowTime
location: class org.apache.hadoop.hbase.master.TestDistributedLogSplitting
/Users/stack/checkout/clean-trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java:[228,39]
cannot find symbol
symbol : method getTestDir()
location: class org.apache.hadoop.hbase.HBaseTestingUtility
{code}
Thanks boss.
test.build.data property overused leading to write data at the wrong place
Key: HBASE-4634
URL: https://issues.apache.org/jira/browse/HBASE-4634
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.92.0
Environment: all
Reporter: nkeywal
Assignee: nkeywal
Attachments: 2011020_4634_all.patch,
20111020_4639_TestStoreFile.patch, 20111021_4634_all.v2.patch,
20111022_4634_all.v3.patch, 20111023_4634_all.v4.patch
test.build.data is overloaded in HBase.At the beginning, it's the Default
parent directory for test output., but then it's rewritten to be the
directory itself in functions like HBaseTestingUtility#startMiniDFSCluster
It seems that this value is already used by MiniDFS (i.e. outside of HBase):
Name is as it is because mini dfs has hard-codings to put test data here.
As it is today, there is at least a bug in HBaseTestingUtility:
{noformat}
public void initTestDir() {
if (System.getProperty(TEST_DIRECTORY_KEY) == null) {
clusterTestBuildDir = setupClusterTestBuildDir();
System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath());
}
}
{noformat}
if you set a value for test.build.data, the test dir will be the parent
directory and not a temp subdir, leading to issues as multiple tests will
end-ups in the same (bad) directory. This function is barely used today,
hence it's not visible, but I would like to use it in some new code.
A possible fix is to remove the check for null and continue with the
overloading, but I don't think it would be a big issue to create a new
key(like test.build.data.rootdirectory) specific to the root directory and
to use test.build.data only to communicate with MiniDFS. Feedback welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[
https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134181#comment-13134181
]
nkeywal commented on HBASE-4634:
that's an error I missed in my patch. Ok to refresh.
On Mon, Oct 24, 2011 at 5:51 PM, stack (Commented) (JIRA)
test.build.data property overused leading to write data at the wrong place
Key: HBASE-4634
URL: https://issues.apache.org/jira/browse/HBASE-4634
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.92.0
Environment: all
Reporter: nkeywal
Assignee: nkeywal
Attachments: 2011020_4634_all.patch,
20111020_4639_TestStoreFile.patch, 20111021_4634_all.v2.patch,
20111022_4634_all.v3.patch, 20111023_4634_all.v4.patch
test.build.data is overloaded in HBase.At the beginning, it's the Default
parent directory for test output., but then it's rewritten to be the
directory itself in functions like HBaseTestingUtility#startMiniDFSCluster
It seems that this value is already used by MiniDFS (i.e. outside of HBase):
Name is as it is because mini dfs has hard-codings to put test data here.
As it is today, there is at least a bug in HBaseTestingUtility:
{noformat}
public void initTestDir() {
if (System.getProperty(TEST_DIRECTORY_KEY) == null) {
clusterTestBuildDir = setupClusterTestBuildDir();
System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath());
}
}
{noformat}
if you set a value for test.build.data, the test dir will be the parent
directory and not a temp subdir, leading to issues as multiple tests will
end-ups in the same (bad) directory. This function is barely used today,
hence it's not visible, but I would like to use it in some new code.
A possible fix is to remove the check for null and continue with the
overloading, but I don't think it would be a big issue to create a new
key(like test.build.data.rootdirectory) specific to the root directory and
to use test.build.data only to communicate with MiniDFS. Feedback welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[
https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13133415#comment-13133415
]
nkeywal commented on HBASE-4634:
v4 coming after:
- some impacts of shutdown throwing Exception instead of IOException were not
taken into account
- startZK before starting the cluster was puting the file in the current
directory (old bug I think).
Full test in progress.
test.build.data property overused leading to write data at the wrong place
Key: HBASE-4634
URL: https://issues.apache.org/jira/browse/HBASE-4634
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.92.0
Environment: all
Reporter: nkeywal
Assignee: nkeywal
Attachments: 2011020_4634_all.patch,
20111020_4639_TestStoreFile.patch, 20111021_4634_all.v2.patch,
20111022_4634_all.v3.patch
test.build.data is overloaded in HBase.At the beginning, it's the Default
parent directory for test output., but then it's rewritten to be the
directory itself in functions like HBaseTestingUtility#startMiniDFSCluster
It seems that this value is already used by MiniDFS (i.e. outside of HBase):
Name is as it is because mini dfs has hard-codings to put test data here.
As it is today, there is at least a bug in HBaseTestingUtility:
{noformat}
public void initTestDir() {
if (System.getProperty(TEST_DIRECTORY_KEY) == null) {
clusterTestBuildDir = setupClusterTestBuildDir();
System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath());
}
}
{noformat}
if you set a value for test.build.data, the test dir will be the parent
directory and not a temp subdir, leading to issues as multiple tests will
end-ups in the same (bad) directory. This function is barely used today,
hence it's not visible, but I would like to use it in some new code.
A possible fix is to remove the check for null and continue with the
overloading, but I don't think it would be a big issue to create a new
key(like test.build.data.rootdirectory) specific to the root directory and
to use test.build.data only to communicate with MiniDFS. Feedback welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[
https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13133045#comment-13133045
]
nkeywal commented on HBASE-4634:
I am going to attach a second version of the patch. I checked indent, it seems
ok to me.
It took more time than expected because I haven't managed to escape some
modifications to the get files functions. Not much impacts for the user.
Basically, there is now a clear distinction between:
- base directory: configurable directory where all tests put their data
- data directory: sub directory used by the test
- cluster directory: subdirectory for the cluster stuff (dfs or hbase)
This will help to have multiple HBaseTestingUtility in the same JVM (// or not).
I have also removed a duplicate getter (something as getHbaseCluster while
getHBase cluster exists).
It includes the fixes for HBASE-4639 and most if not all the bad usage
mentioned in HBASE-4326 (but I did not fix the root issue of HBASE-4326).
It seems to work on my env, currently n-checking again.
I have a systematic errors in the test, but it's in
#org.apache.hadoop.hbase.io.hfile.TestHFileBlock#testBlockHeapSize , I believe
it was there at the beginning.
I have random errors in: TestDistributedLogSplitting,
regionserver.TestServerCustomProtocol, client.TestAdmin
master.TestHMasterRPCException,
coprocessor.TestRegionServerCoprocessorExceptionWithAbort, wal.TestLogRolling,
CatalogTrackerOnCluster and a few else, but it works when I replay the test
set, so I tend to think they belong to the flaky test sets.
At the end of the day, the patch is not that medium... Sorry for that.
Will add a new comment when the tests will be over and I will have rechecked
the flaky one, but you can already have a look.
test.build.data property overused leading to write data at the wrong place
Key: HBASE-4634
URL: https://issues.apache.org/jira/browse/HBASE-4634
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.92.0
Environment: all
Reporter: nkeywal
Assignee: nkeywal
Attachments: 2011020_4634_all.patch, 20111020_4639_TestStoreFile.patch
test.build.data is overloaded in HBase.At the beginning, it's the Default
parent directory for test output., but then it's rewritten to be the
directory itself in functions like HBaseTestingUtility#startMiniDFSCluster
It seems that this value is already used by MiniDFS (i.e. outside of HBase):
Name is as it is because mini dfs has hard-codings to put test data here.
As it is today, there is at least a bug in HBaseTestingUtility:
{noformat}
public void initTestDir() {
if (System.getProperty(TEST_DIRECTORY_KEY) == null) {
clusterTestBuildDir = setupClusterTestBuildDir();
System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath());
}
}
{noformat}
if you set a value for test.build.data, the test dir will be the parent
directory and not a temp subdir, leading to issues as multiple tests will
end-ups in the same (bad) directory. This function is barely used today,
hence it's not visible, but I would like to use it in some new code.
A possible fix is to remove the check for null and continue with the
overloading, but I don't think it would be a big issue to create a new
key(like test.build.data.rootdirectory) specific to the root directory and
to use test.build.data only to communicate with MiniDFS. Feedback welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[
https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13133048#comment-13133048
]
Ted Yu commented on HBASE-4634:
---
TestHFileBlock#testBlockHeapSize is fixed by addendum to HBASE-4219
Please refresh your workspace.
test.build.data property overused leading to write data at the wrong place
Key: HBASE-4634
URL: https://issues.apache.org/jira/browse/HBASE-4634
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.92.0
Environment: all
Reporter: nkeywal
Assignee: nkeywal
Attachments: 2011020_4634_all.patch,
20111020_4639_TestStoreFile.patch, 20111021_4634_all.v2.patch
test.build.data is overloaded in HBase.At the beginning, it's the Default
parent directory for test output., but then it's rewritten to be the
directory itself in functions like HBaseTestingUtility#startMiniDFSCluster
It seems that this value is already used by MiniDFS (i.e. outside of HBase):
Name is as it is because mini dfs has hard-codings to put test data here.
As it is today, there is at least a bug in HBaseTestingUtility:
{noformat}
public void initTestDir() {
if (System.getProperty(TEST_DIRECTORY_KEY) == null) {
clusterTestBuildDir = setupClusterTestBuildDir();
System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath());
}
}
{noformat}
if you set a value for test.build.data, the test dir will be the parent
directory and not a temp subdir, leading to issues as multiple tests will
end-ups in the same (bad) directory. This function is barely used today,
hence it's not visible, but I would like to use it in some new code.
A possible fix is to remove the check for null and continue with the
overloading, but I don't think it would be a big issue to create a new
key(like test.build.data.rootdirectory) specific to the root directory and
to use test.build.data only to communicate with MiniDFS. Feedback welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[
https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131722#comment-13131722
]
Ted Yu commented on HBASE-4634:
---
From MiniDFSCluster.java:
{code}
public static String getBaseDirectory() {
return System.getProperty(PROP_TEST_BUILD_DATA, build/test/data) +
/dfs/;
}
{code}
I think we can introduce a new key for the test data used by hbase.
test.build.data property overused leading to write data at the wrong place
Key: HBASE-4634
URL: https://issues.apache.org/jira/browse/HBASE-4634
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.92.0
Environment: all
Reporter: nkeywal
Assignee: nkeywal
test.build.data is overloaded in HBase.At the beginning, it's the Default
parent directory for test output., but then it's rewritten to be the
directory itself in functions like HBaseTestingUtility#startMiniDFSCluster
It seems that this value is already used by MiniDFS (i.e. outside of HBase):
Name is as it is because mini dfs has hard-codings to put test data here.
As it is today, there is at least a bug in HBaseTestingUtility:
{noformat}
public void initTestDir() {
if (System.getProperty(TEST_DIRECTORY_KEY) == null) {
clusterTestBuildDir = setupClusterTestBuildDir();
System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath());
}
}
{noformat}
if you set a value for test.build.data, the test dir will be the parent
directory and not a temp subdir, leading to issues as multiple tests will
end-ups in the same (bad) directory. This function is barely used today,
hence it's not visible, but I would like to use it in some new code.
A possible fix is to remove the check for null and continue with the
overloading, but I don't think it would be a big issue to create a new
key(like test.build.data.rootdirectory) specific to the root directory and
to use test.build.data only to communicate with MiniDFS. Feedback welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[
https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131912#comment-13131912
]
nkeywal commented on HBASE-4634:
Ok, I've implemented something, currently under testing.
test.build.data property overused leading to write data at the wrong place
Key: HBASE-4634
URL: https://issues.apache.org/jira/browse/HBASE-4634
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.92.0
Environment: all
Reporter: nkeywal
Assignee: nkeywal
test.build.data is overloaded in HBase.At the beginning, it's the Default
parent directory for test output., but then it's rewritten to be the
directory itself in functions like HBaseTestingUtility#startMiniDFSCluster
It seems that this value is already used by MiniDFS (i.e. outside of HBase):
Name is as it is because mini dfs has hard-codings to put test data here.
As it is today, there is at least a bug in HBaseTestingUtility:
{noformat}
public void initTestDir() {
if (System.getProperty(TEST_DIRECTORY_KEY) == null) {
clusterTestBuildDir = setupClusterTestBuildDir();
System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath());
}
}
{noformat}
if you set a value for test.build.data, the test dir will be the parent
directory and not a temp subdir, leading to issues as multiple tests will
end-ups in the same (bad) directory. This function is barely used today,
hence it's not visible, but I would like to use it in some new code.
A possible fix is to remove the check for null and continue with the
overloading, but I don't think it would be a big issue to create a new
key(like test.build.data.rootdirectory) specific to the root directory and
to use test.build.data only to communicate with MiniDFS. Feedback welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[
https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131987#comment-13131987
]
stack commented on HBASE-4634:
--
Please use the same indent as the surrounding file (two spaces for a tab).
Should we deprecate ' public static final String TEST_DIRECTORY_KEY =
test.build.data;' so we can later change it to private or protected access?
'root' is a pretty overloaded term in these parts. There is unix root and then
there is hbase root region and hbase.rootdir. You are using
ROOT_TEST_DIRECTORY in place of TEST_DIRECTORY. Maybe BASE_TEST_DIRECTORY
instead?
FYI, the surrounding code has spaces between operators. e.g. '+if
(clusterTestBuildDir==null){' should be '+if (clusterTestBuildDir == null)
{'
Otherwise, patch is going in right direction IMO. Good stuff.
I'm not sure what to do with the patch 20111020_4639_TestStoreFile.patch. Is
it to be applied here or over in hbase-4639?
test.build.data property overused leading to write data at the wrong place
Key: HBASE-4634
URL: https://issues.apache.org/jira/browse/HBASE-4634
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.92.0
Environment: all
Reporter: nkeywal
Assignee: nkeywal
Attachments: 2011020_4634_all.patch, 20111020_4639_TestStoreFile.patch
test.build.data is overloaded in HBase.At the beginning, it's the Default
parent directory for test output., but then it's rewritten to be the
directory itself in functions like HBaseTestingUtility#startMiniDFSCluster
It seems that this value is already used by MiniDFS (i.e. outside of HBase):
Name is as it is because mini dfs has hard-codings to put test data here.
As it is today, there is at least a bug in HBaseTestingUtility:
{noformat}
public void initTestDir() {
if (System.getProperty(TEST_DIRECTORY_KEY) == null) {
clusterTestBuildDir = setupClusterTestBuildDir();
System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath());
}
}
{noformat}
if you set a value for test.build.data, the test dir will be the parent
directory and not a temp subdir, leading to issues as multiple tests will
end-ups in the same (bad) directory. This function is barely used today,
hence it's not visible, but I would like to use it in some new code.
A possible fix is to remove the check for null and continue with the
overloading, but I don't think it would be a big issue to create a new
key(like test.build.data.rootdirectory) specific to the root directory and
to use test.build.data only to communicate with MiniDFS. Feedback welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4634) test.build.data property overused leading to write data at the wrong place
[
https://issues.apache.org/jira/browse/HBASE-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13132003#comment-13132003
]
nkeywal commented on HBASE-4634:
For TestStoreFile, It would need to be applied here, even it's doing more
than just removing the static dependency on the TestingTool : it includes
4639.
It will take into account your comments and publish an updated version.
- with BASE_TEST_DIRECTORY
- private for TEST_DIRECTORY_KEY should to it, I will try.
+ indent stuff
On Thu, Oct 20, 2011 at 10:25 PM, stack (Commented) (JIRA)
test.build.data property overused leading to write data at the wrong place
Key: HBASE-4634
URL: https://issues.apache.org/jira/browse/HBASE-4634
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.92.0
Environment: all
Reporter: nkeywal
Assignee: nkeywal
Attachments: 2011020_4634_all.patch, 20111020_4639_TestStoreFile.patch
test.build.data is overloaded in HBase.At the beginning, it's the Default
parent directory for test output., but then it's rewritten to be the
directory itself in functions like HBaseTestingUtility#startMiniDFSCluster
It seems that this value is already used by MiniDFS (i.e. outside of HBase):
Name is as it is because mini dfs has hard-codings to put test data here.
As it is today, there is at least a bug in HBaseTestingUtility:
{noformat}
public void initTestDir() {
if (System.getProperty(TEST_DIRECTORY_KEY) == null) {
clusterTestBuildDir = setupClusterTestBuildDir();
System.setProperty(TEST_DIRECTORY_KEY, clusterTestBuildDir.getPath());
}
}
{noformat}
if you set a value for test.build.data, the test dir will be the parent
directory and not a temp subdir, leading to issues as multiple tests will
end-ups in the same (bad) directory. This function is barely used today,
hence it's not visible, but I would like to use it in some new code.
A possible fix is to remove the check for null and continue with the
overloading, but I don't think it would be a big issue to create a new
key(like test.build.data.rootdirectory) specific to the root directory and
to use test.build.data only to communicate with MiniDFS. Feedback welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
