[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-9108: -- Resolution: Incomplete Assignee: (was: gautam) Status: Resolved (was: Patch Available) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10 >Reporter: gautam >Priority: Critical > Attachments: 9108.patch._trunk.5, 9108.patch._trunk.6, > HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3, > HBASE-9108.patch._trunk.4, HBASE-9108.patch._trunk.7, > HBASE-9108.patch._trunk.8 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Status: Patch Available (was: In Progress) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.94.10, 0.94.9, 0.95.1, 0.95.0 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: 9108.patch._trunk.5, 9108.patch._trunk.6, > HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3, > HBASE-9108.patch._trunk.4, HBASE-9108.patch._trunk.7, > HBASE-9108.patch._trunk.8 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Attachment: HBASE-9108.patch._trunk.8 I was doing git apply , which was not returning anything. And the compilation was also fine. When I did patch -p1 < that really applied and I could fix the compilation issue thereafter. Learnt patching... > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: 9108.patch._trunk.5, 9108.patch._trunk.6, > HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3, > HBASE-9108.patch._trunk.4, HBASE-9108.patch._trunk.7, > HBASE-9108.patch._trunk.8 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Status: In Progress (was: Patch Available) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.94.10, 0.94.9, 0.95.1, 0.95.0 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: 9108.patch._trunk.5, 9108.patch._trunk.6, > HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3, > HBASE-9108.patch._trunk.4, HBASE-9108.patch._trunk.7 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Status: In Progress (was: Patch Available) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.94.10, 0.94.9, 0.95.1, 0.95.0 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: 9108.patch._trunk.5, 9108.patch._trunk.6, > HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3, > HBASE-9108.patch._trunk.4, HBASE-9108.patch._trunk.7 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Status: Patch Available (was: In Progress) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.94.10, 0.94.9, 0.95.1, 0.95.0 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: 9108.patch._trunk.5, 9108.patch._trunk.6, > HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3, > HBASE-9108.patch._trunk.4, HBASE-9108.patch._trunk.7 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Attachment: HBASE-9108.patch._trunk.7 Looks like things have changed a bit. This is with the latest changes. > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: 9108.patch._trunk.5, 9108.patch._trunk.6, > HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3, > HBASE-9108.patch._trunk.4, HBASE-9108.patch._trunk.7 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Status: Patch Available (was: In Progress) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.94.10, 0.94.9, 0.95.1, 0.95.0 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: 9108.patch._trunk.5, 9108.patch._trunk.6, > HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3, > HBASE-9108.patch._trunk.4 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Status: In Progress (was: Patch Available) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.94.10, 0.94.9, 0.95.1, 0.95.0 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: 9108.patch._trunk.5, 9108.patch._trunk.6, > HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3, > HBASE-9108.patch._trunk.4 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Attachment: 9108.patch._trunk.6 Here is with the isEmpty() added. > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: 9108.patch._trunk.5, 9108.patch._trunk.6, > HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3, > HBASE-9108.patch._trunk.4 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-9108: -- Attachment: 9108.patch._trunk.5 > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: 9108.patch._trunk.5, HBASE-9108.patch._trunk.2, > HBASE-9108.patch._trunk.3, HBASE-9108.patch._trunk.4 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Status: Patch Available (was: In Progress) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.94.10, 0.94.9, 0.95.1, 0.95.0 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3, > HBASE-9108.patch._trunk.4 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Attachment: HBASE-9108.patch._trunk.4 How about this? > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3, > HBASE-9108.patch._trunk.4 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Status: In Progress (was: Patch Available) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.94.10, 0.94.9, 0.95.1, 0.95.0 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3, > HBASE-9108.patch._trunk.4 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Status: Patch Available (was: In Progress) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.94.10, 0.94.9, 0.95.1, 0.95.0 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Status: In Progress (was: Patch Available) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.94.10, 0.94.9, 0.95.1, 0.95.0 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Attachment: HBASE-9108.patch._trunk.3 Here i go. Its the same as trunk.2 > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: HBASE-9108.patch._trunk.2, HBASE-9108.patch._trunk.3 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Attachment: (was: HBASE-9108.patch._0.94.2) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: HBASE-9108.patch._trunk.2 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Status: In Progress (was: Patch Available) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.94.10, 0.94.9, 0.95.1, 0.95.0 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: HBASE-9108.patch._trunk.2 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Attachment: (was: HBASE-9108.patch._trunk) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: HBASE-9108.patch._trunk.2 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Status: Patch Available (was: In Progress) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.94.10, 0.94.9, 0.95.1, 0.95.0 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: HBASE-9108.patch._trunk.2 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Attachment: (was: HBASE-9108.patch._0.94) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: HBASE-9108.patch._trunk.2 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Attachment: HBASE-9108.patch._trunk.2 HBASE-9108.patch._0.94.2 Except changing to package private, rest of the changes are done. I dont think there is a need to mark it package private. > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: HBASE-9108.patch._0.94, HBASE-9108.patch._0.94.2, > HBASE-9108.patch._trunk, HBASE-9108.patch._trunk.2 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Status: Patch Available (was: Open) > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.94.10, 0.94.9, 0.95.1, 0.95.0 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: HBASE-9108.patch._0.94, HBASE-9108.patch._trunk > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Attachment: HBASE-9108.patch._trunk Here is the patch file for trunk/0.95. > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: HBASE-9108.patch._0.94, HBASE-9108.patch._trunk > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9108) LoadTestTool need to have a way to ignore keys which were failed during write.
[ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gautam updated HBASE-9108: -- Attachment: HBASE-9108.patch._0.94 Here is the change for 0.94 branch. > LoadTestTool need to have a way to ignore keys which were failed during > write. > --- > > Key: HBASE-9108 > URL: https://issues.apache.org/jira/browse/HBASE-9108 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10 >Reporter: gautam >Assignee: gautam >Priority: Critical > Attachments: HBASE-9108.patch._0.94 > > Original Estimate: 48h > Remaining Estimate: 48h > > While running the chaosmonkey integration tests, it is found that write > sometimes fails when the cluster components are restarted/stopped/killed etc.. > The data key which was being put, using the LoadTestTool, is added to the > failed key set, and at the end of the test, this failed key set is checked > for any entries to assert failures. > While doing fail-over testing, it is expected that some of the keys may go > un-written. The point here is to validate that whatever gets into hbase for > an unstable cluster really goes in, and hence read should be 100% for > whatever keys went in successfully. > Currently LoadTestTool has strict checks to validate every key being written > or not. In case any keys is not written, it fails. > I wanted to loosen this constraint by allowing users to pass in a set of > exceptions they expect when doing put/write operations over hbase. If one of > these expected exception set is thrown while writing key to hbase, the failed > key would be ignored, and hence wont even be considered again for subsequent > write as well as read. > This can be passed to the load test tool as csv list parameter > -allowed_write_exceptions, or it can be passed through hbase-site.xml by > writing a value for "test.ignore.exceptions.during.write" > Here is the usage: > -allowed_write_exceptions > "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException" > Hence, by doing this the existing integration tests can also make use of this > change by passing it as property in hbase-site.xml, as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
