[jira] [Commented] (HBASE-5821) Incorrect handling of null value in Coprocessor aggregation function min()
[
https://issues.apache.org/jira/browse/HBASE-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257280#comment-13257280
]
Himanshu Vashishtha commented on HBASE-5821:
Yes, I meant LongColumnInterpreter. There is a minor nit.
As per the existing code, max is correct (it will not override the value with
an incoming null result); but min is buggy. Maryann said that too.
I think we should have a uniform check at both the places (Client and Protocol)
ie. in client we are checking :
{code}
max = ci.compare(max, result) 0 ? result : max;
{code}
and in Protocol we are checking:
{code}
max = (max == null || ci.compare(temp, max) 0) ? temp : max;
{code}
The ordering of arguments in the compare method is not same.
That was also why I said that its good to have these sort of check at single
place (and Column interpreter implementations are good candidate for that as
they can define their own way to handle them. They just have to follow the
Interface contract.).
Anyway, if we go with the existing one, I think we should use the same order
just to avoid any confusion.
I know its a minor nit, but it would be good to have a more consistent code?
Thanks for looking into this.
Incorrect handling of null value in Coprocessor aggregation function min()
--
Key: HBASE-5821
URL: https://issues.apache.org/jira/browse/HBASE-5821
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.1
Reporter: Maryann Xue
Assignee: Maryann Xue
Fix For: 0.92.2, 0.96.0, 0.94.1
Attachments: HBASE-5821.patch
Both in AggregateImplementation and AggregationClient, the evaluation of the
current minimum value is like:
min = (min == null || ci.compare(result, min) 0) ? result : min;
The LongColumnInterpreter takes null value is treated as the least value,
while the above expression takes min as the greater value when it is null.
Thus, the real minimum value gets discarded if a null value comes later.
max() could also be wrong if a different ColumnInterpreter other than
LongColumnInterpreter treats null value differently (as the greatest).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5821) Incorrect handling of null value in Coprocessor aggregation function min()
[ https://issues.apache.org/jira/browse/HBASE-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257542#comment-13257542 ] Himanshu Vashishtha commented on HBASE-5821: I see your point about different use cases. With the current patch we are invoking the comparator method only when both arguments are non-null. Anyway, its working correctly now, I am good :) Thanks for correcting it Maryann. Incorrect handling of null value in Coprocessor aggregation function min() -- Key: HBASE-5821 URL: https://issues.apache.org/jira/browse/HBASE-5821 Project: HBase Issue Type: Bug Components: coprocessors Affects Versions: 0.92.1 Reporter: Maryann Xue Assignee: Maryann Xue Fix For: 0.92.2, 0.96.0, 0.94.1 Attachments: HBASE-5821.patch Both in AggregateImplementation and AggregationClient, the evaluation of the current minimum value is like: min = (min == null || ci.compare(result, min) 0) ? result : min; The LongColumnInterpreter takes null value is treated as the least value, while the above expression takes min as the greater value when it is null. Thus, the real minimum value gets discarded if a null value comes later. max() could also be wrong if a different ColumnInterpreter other than LongColumnInterpreter treats null value differently (as the greatest). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5821) Incorrect handling of null value in Coprocessor aggregation function min()
[ https://issues.apache.org/jira/browse/HBASE-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13256794#comment-13256794 ] Himanshu Vashishtha commented on HBASE-5821: Yes, its a bug. Thanks for pointing this out. I think the change should be done at the LongColumnInterpreter level? Incorrect handling of null value in Coprocessor aggregation function min() -- Key: HBASE-5821 URL: https://issues.apache.org/jira/browse/HBASE-5821 Project: HBase Issue Type: Bug Components: coprocessors Affects Versions: 0.92.1 Reporter: Maryann Xue Assignee: Maryann Xue Attachments: HBASE-5821.patch Both in AggregateImplementation and AggregationClient, the evaluation of the current minimum value is like: min = (min == null || ci.compare(result, min) 0) ? result : min; The LongColumnInterpreter takes null value is treated as the least value, while the above expression takes min as the greater value when it is null. Thus, the real minimum value gets discarded if a null value comes later. max() could also be wrong if a different ColumnInterpreter other than LongColumnInterpreter treats null value differently (as the greatest). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5821) Incorrect handling of null value in Coprocessor aggregation function min()
[ https://issues.apache.org/jira/browse/HBASE-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257202#comment-13257202 ] Himanshu Vashishtha commented on HBASE-5821: Yeah, LCM is just an implementation and I thought to better keep the semantics of CI right and tight such that we just call compare and it gives what we want (as per CI.compareTo() javadoc). Otherwise, I am good. Incorrect handling of null value in Coprocessor aggregation function min() -- Key: HBASE-5821 URL: https://issues.apache.org/jira/browse/HBASE-5821 Project: HBase Issue Type: Bug Components: coprocessors Affects Versions: 0.92.1 Reporter: Maryann Xue Assignee: Maryann Xue Attachments: HBASE-5821.patch Both in AggregateImplementation and AggregationClient, the evaluation of the current minimum value is like: min = (min == null || ci.compare(result, min) 0) ? result : min; The LongColumnInterpreter takes null value is treated as the least value, while the above expression takes min as the greater value when it is null. Thus, the real minimum value gets discarded if a null value comes later. max() could also be wrong if a different ColumnInterpreter other than LongColumnInterpreter treats null value differently (as the greatest). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5741) ImportTsv does not check for table existence
[ https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13253637#comment-13253637 ] Himanshu Vashishtha commented on HBASE-5741: I am waiting for Clint to give me some numbers of such use cases to make the cut; (Apparently, he is on vacation these days and will be back this Monday) Thanks. ImportTsv does not check for table existence - Key: HBASE-5741 URL: https://issues.apache.org/jira/browse/HBASE-5741 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Clint Heath Assignee: Himanshu Vashishtha Fix For: 0.96.0, 0.94.1 Attachments: 5741-94.txt, 5741-v3.txt, HBase-5741-v2.patch, HBase-5741.patch The usage statement for the importtsv command to hbase claims this: Note: if you do not use this option, then the target table must already exist in HBase (in reference to the importtsv.bulk.output command-line option) The truth is, the table must exist no matter what, importtsv cannot and will not create it for you. This is the case because the createSubmittableJob method of ImportTsv does not even attempt to check if the table exists already, much less create it: (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java) 305 HTable table = new HTable(conf, tableName); The HTable method signature in use there assumes the table exists and runs a meta scan on it: (From org.apache.hadoop.hbase.client.HTable.java) 142 * Creates an object to access a HBase table. ... 151 public HTable(Configuration conf, final String tableName) What we should do inside of createSubmittableJob is something similar to what the completebulkloads command would do: (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java) 690 boolean tableExists = this.doesTableExist(tableName); 691 if (!tableExists) this.createTable(tableName,dirPath); Currently the docs are misleading, the table in fact must exist prior to running importtsv. We should check if it exists rather than assume it's already there and throw the below exception: 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: Encountered problems when prefetch META table: org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for table: myTable2, row=myTable2,,99 at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5741) ImportTsv does not check for table existence
[ https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252087#comment-13252087 ] Himanshu Vashishtha commented on HBASE-5741: As Clint mentioned, there are some workloads that use ImportTsv to create HFiles, which later on are used in the bulk import method : my motivation for the patch. What you suggest Lars, failing the importtsv, or adding a similar thing in the Import tool too? I think later makes it consistent. ImportTsv does not check for table existence - Key: HBASE-5741 URL: https://issues.apache.org/jira/browse/HBASE-5741 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Clint Heath Assignee: Himanshu Vashishtha Fix For: 0.94.0, 0.96.0 Attachments: 5741-94.txt, 5741-v3.txt, HBase-5741-v2.patch, HBase-5741.patch The usage statement for the importtsv command to hbase claims this: Note: if you do not use this option, then the target table must already exist in HBase (in reference to the importtsv.bulk.output command-line option) The truth is, the table must exist no matter what, importtsv cannot and will not create it for you. This is the case because the createSubmittableJob method of ImportTsv does not even attempt to check if the table exists already, much less create it: (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java) 305 HTable table = new HTable(conf, tableName); The HTable method signature in use there assumes the table exists and runs a meta scan on it: (From org.apache.hadoop.hbase.client.HTable.java) 142 * Creates an object to access a HBase table. ... 151 public HTable(Configuration conf, final String tableName) What we should do inside of createSubmittableJob is something similar to what the completebulkloads command would do: (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java) 690 boolean tableExists = this.doesTableExist(tableName); 691 if (!tableExists) this.createTable(tableName,dirPath); Currently the docs are misleading, the table in fact must exist prior to running importtsv. We should check if it exists rather than assume it's already there and throw the below exception: 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: Encountered problems when prefetch META table: org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for table: myTable2, row=myTable2,,99 at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5741) ImportTsv does not check for table existence
[ https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250813#comment-13250813 ] Himanshu Vashishtha commented on HBASE-5741: You are right. I am on it now. Thanks. ImportTsv does not check for table existence - Key: HBASE-5741 URL: https://issues.apache.org/jira/browse/HBASE-5741 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Clint Heath Assignee: Himanshu Vashishtha The usage statement for the importtsv command to hbase claims this: Note: if you do not use this option, then the target table must already exist in HBase (in reference to the importtsv.bulk.output command-line option) The truth is, the table must exist no matter what, importtsv cannot and will not create it for you. This is the case because the createSubmittableJob method of ImportTsv does not even attempt to check if the table exists already, much less create it: (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java) 305 HTable table = new HTable(conf, tableName); The HTable method signature in use there assumes the table exists and runs a meta scan on it: (From org.apache.hadoop.hbase.client.HTable.java) 142 * Creates an object to access a HBase table. ... 151 public HTable(Configuration conf, final String tableName) What we should do inside of createSubmittableJob is something similar to what the completebulkloads command would do: (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java) 690 boolean tableExists = this.doesTableExist(tableName); 691 if (!tableExists) this.createTable(tableName,dirPath); Currently the docs are misleading, the table in fact must exist prior to running importtsv. We should check if it exists rather than assume it's already there and throw the below exception: 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: Encountered problems when prefetch META table: org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for table: myTable2, row=myTable2,,99 at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2223) Handle 10min+ network partitions between clusters
[ https://issues.apache.org/jira/browse/HBASE-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250160#comment-13250160 ] Himanshu Vashishtha commented on HBASE-2223: I was skimming through the patch; and it looks like its not there in the current trunk? Will be great if anyone can provide its current status, or follow up jiras? Handle 10min+ network partitions between clusters - Key: HBASE-2223 URL: https://issues.apache.org/jira/browse/HBASE-2223 Project: HBase Issue Type: Sub-task Reporter: Jean-Daniel Cryans Assignee: Jean-Daniel Cryans Fix For: 0.90.0 Attachments: HBASE-2223.patch We need a nice way of handling long network partitions without impacting a master cluster (which pushes the data). Currently it will just retry over and over again. I think we could: - Stop replication to a slave cluster if it didn't respond for more than 10 minutes - Keep track of the duration of the partition - When the slave cluster comes back, initiate a MR job like HBASE-2221 Maybe we want less than 10 minutes, maybe we want this to be all automatic or just the first 2 parts. Discuss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5741) ImportTsv does not check for table existence
[ https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13249442#comment-13249442 ] Himanshu Vashishtha commented on HBASE-5741: Yes, the javadoc says it half correct when it says that the table must exist in HBase if you are not using the option importtsv.bulk.output. The behavior, in absence of the above property is to do inserts into the provided table; so it throws an exception if the table doesn't exist. When we use this option, importtsv tries to configure job by making an attempt to read all the regions of this table and then use the start keys of these regions as partition-markers for the TotalOrderPartitioner class. This usage eliminates the first start-row (which is always a EMPTY_BYTE_ARRAY), so even if we create a new table, it will be useless as for using it for configuring a job. For the case with LoadIncrementalHFile, the destination of the directory containing the hFiles is assumed to be in a specific format: tableName/cf1/listOfHFiles /cf2/listOfHFiles --where we are storing the hfiles for a specific column family in a separate sub-directory. If we do create a table based on the parameters given to the importtsv command, it will not be useful for the case of bulkload usecase as in the importtsv job, we dump hfiles based on rows; so all coulmn famlies for a specific row lands in one hfile. It would be great to know if we actually use this workflow: create HFiles from importtsv job, and then use bulkload to insert those HFiles in HBase table. I think we should change the javadoc. Please let me know if you have any questions; hbase-mapreduce use cases are exciting. ImportTsv does not check for table existence - Key: HBASE-5741 URL: https://issues.apache.org/jira/browse/HBASE-5741 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Clint Heath Assignee: Himanshu Vashishtha The usage statement for the importtsv command to hbase claims this: Note: if you do not use this option, then the target table must already exist in HBase (in reference to the importtsv.bulk.output command-line option) The truth is, the table must exist no matter what, importtsv cannot and will not create it for you. This is the case because the createSubmittableJob method of ImportTsv does not even attempt to check if the table exists already, much less create it: (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java) 305 HTable table = new HTable(conf, tableName); The HTable method signature in use there assumes the table exists and runs a meta scan on it: (From org.apache.hadoop.hbase.client.HTable.java) 142 * Creates an object to access a HBase table. ... 151 public HTable(Configuration conf, final String tableName) What we should do inside of createSubmittableJob is something similar to what the completebulkloads command would do: (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java) 690 boolean tableExists = this.doesTableExist(tableName); 691 if (!tableExists) this.createTable(tableName,dirPath); Currently the docs are misleading, the table in fact must exist prior to running importtsv. We should check if it exists rather than assume it's already there and throw the below exception: 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: Encountered problems when prefetch META table: org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for table: myTable2, row=myTable2,,99 at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5741) ImportTsv does not check for table existence
[ https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13249443#comment-13249443 ] Himanshu Vashishtha commented on HBASE-5741: btw, by making an attempt to read all the regions of this table and then use the start keys of these regions, I mean get the start rows for all the existing regions of the table and then use them for job configuration. ImportTsv does not check for table existence - Key: HBASE-5741 URL: https://issues.apache.org/jira/browse/HBASE-5741 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.4 Reporter: Clint Heath Assignee: Himanshu Vashishtha The usage statement for the importtsv command to hbase claims this: Note: if you do not use this option, then the target table must already exist in HBase (in reference to the importtsv.bulk.output command-line option) The truth is, the table must exist no matter what, importtsv cannot and will not create it for you. This is the case because the createSubmittableJob method of ImportTsv does not even attempt to check if the table exists already, much less create it: (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java) 305 HTable table = new HTable(conf, tableName); The HTable method signature in use there assumes the table exists and runs a meta scan on it: (From org.apache.hadoop.hbase.client.HTable.java) 142 * Creates an object to access a HBase table. ... 151 public HTable(Configuration conf, final String tableName) What we should do inside of createSubmittableJob is something similar to what the completebulkloads command would do: (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java) 690 boolean tableExists = this.doesTableExist(tableName); 691 if (!tableExists) this.createTable(tableName,dirPath); Currently the docs are misleading, the table in fact must exist prior to running importtsv. We should check if it exists rather than assume it's already there and throw the below exception: 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: Encountered problems when prefetch META table: org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for table: myTable2, row=myTable2,,99 at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150) ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4348) Add metrics for regions in transition
[ https://issues.apache.org/jira/browse/HBASE-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247941#comment-13247941 ] Himanshu Vashishtha commented on HBASE-4348: I tested it with latest trunk commit hash (f3168b76f48541ce89e88016b333c0d66fb12f40) and it applies good. Is hadoop qa is applying it some other branch? 0.92? Add metrics for regions in transition - Key: HBASE-4348 URL: https://issues.apache.org/jira/browse/HBASE-4348 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Himanshu Vashishtha Priority: Minor Labels: noob Fix For: 0.96.0 Attachments: 4348-metrics-v3.patch, 4348-v1.patch, 4348-v2.patch, RITs.png, RegionInTransitions2.png, hbase-4348.patch, metrics-v2.patch The following metrics would be useful for monitoring the master: - the number of regions in transition - the number of regions in transition that have been in transition for more than a minute - how many seconds has the oldest region-in-transition been in transition -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4348) Add metrics for regions in transition
[ https://issues.apache.org/jira/browse/HBASE-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248079#comment-13248079 ] Himanshu Vashishtha commented on HBASE-4348: The failed test (org.apache.hadoop.hbase.coprocessor.TestRowProcessorEndpoint) passes locally for me, though. Add metrics for regions in transition - Key: HBASE-4348 URL: https://issues.apache.org/jira/browse/HBASE-4348 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Himanshu Vashishtha Priority: Minor Labels: noob Fix For: 0.96.0 Attachments: 4348-metrics-v3.patch, 4348-v1.patch, 4348-v2.patch, RITs.png, RegionInTransitions2.png, hbase-4348-v4.patch, hbase-4348.patch, metrics-v2.patch The following metrics would be useful for monitoring the master: - the number of regions in transition - the number of regions in transition that have been in transition for more than a minute - how many seconds has the oldest region-in-transition been in transition -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242927#comment-13242927 ] Himanshu Vashishtha commented on HBASE-5655: Is it ok to mark it resolved now? Cap space usage of default log4j rolling policy --- Key: HBASE-5655 URL: https://issues.apache.org/jira/browse/HBASE-5655 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.96.0 Attachments: 5655-v1.patch, HBase-5655-v2.patch, HBase-5655-v3.patch The current default log4j policy is to use Daily Rolling File Appender (DRFA). At times, its good to have a cap on the maximum size of the logs in order to limit its disk usage. Here is a proposal to set a new file appemder (RFA) as the default appender. It can be configured via env so that existing tools can use the current behavior of using DRFA instead. This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13241460#comment-13241460 ] Himanshu Vashishtha commented on HBASE-5655: I have tested this by running HBase with this settings. It uses the RFA policy. Using an env property with HBASE_ROOT_LOGGER=DRFA takes the old DRFA policy. Cap space usage of default log4j rolling policy --- Key: HBASE-5655 URL: https://issues.apache.org/jira/browse/HBASE-5655 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Attachments: 5655-v1.patch The current default log4j policy is to use Daily Rolling File Appender (DRFA). At times, its good to have a cap on the maximum size of the logs in order to limit its disk usage. Here is a proposal to set a new file appemder (RFA) as the default appender. It can be configured via env so that existing tools can use the current behavior of using DRFA instead. This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13241883#comment-13241883 ] Himanshu Vashishtha commented on HBASE-5655: It will be good to know whether these failures are caused by this patch or are they just flaky. Looking at the logs for TestTAbleMapReduce's failures, it seems metrics2 were not loaded, and it gave out NFE: attempt_20120329220930011_0001_r_02_0: 2012-03-29 22:09:50,378 WARN [main] impl.MetricsSystemImpl(137): Metrics system not started: Cannot locate configuration: tried hadoop-metrics2-reducetask.properties, hadoop-metrics2.properties Shall I look into it, or are they just flaky. I can't correlate these failures with log4j rolling policy change as of now. Cap space usage of default log4j rolling policy --- Key: HBASE-5655 URL: https://issues.apache.org/jira/browse/HBASE-5655 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Attachments: 5655-v1.patch, HBase-5655-v2.patch The current default log4j policy is to use Daily Rolling File Appender (DRFA). At times, its good to have a cap on the maximum size of the logs in order to limit its disk usage. Here is a proposal to set a new file appemder (RFA) as the default appender. It can be configured via env so that existing tools can use the current behavior of using DRFA instead. This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[
https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13239578#comment-13239578
]
Himanshu Vashishtha commented on HBASE-5655:
It involves adding a new appender description in log4j.properties file, and
making the hbase-daemon.sh to set the root logger after reading from env rather
than hard coding.
More specifically, in hbase-daemon.sh:
using export HBASE_ROOT_LOGGER=${HBASE_ROOT_LOGGER:-INFO,RFA}, in place of
export HBASE_ROOT_LOGGER=INFO,DRFA
I have done the changes and tested on my local. Please share your thoughts.
Cap space usage of default log4j rolling policy
---
Key: HBASE-5655
URL: https://issues.apache.org/jira/browse/HBASE-5655
Project: HBase
Issue Type: Improvement
Affects Versions: 0.92.1
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
Fix For: 0.96.0
The current default log4j policy is to use Daily Rolling File Appender
(DRFA). At times, its good to have a cap on the maximum size of the logs in
order to limit its disk usage. Here is a proposal to set a new file appemder
(RFA) as the default appender. It can be configured via env so that existing
tools can use the current behavior of using DRFA instead.
This is in parallel with jira Hadoop-8149.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13239627#comment-13239627 ] Himanshu Vashishtha commented on HBASE-5655: You can define its maximum size and the number of history files. More specifically, log4j.appender.RFA.MaxFileSize=user_defined_prop. log4j.appender.RFA.MaxBackupIndex=number_of_history_files // This is similar to DRFA.MaxBackupIndex though. Cap space usage of default log4j rolling policy --- Key: HBASE-5655 URL: https://issues.apache.org/jira/browse/HBASE-5655 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha The current default log4j policy is to use Daily Rolling File Appender (DRFA). At times, its good to have a cap on the maximum size of the logs in order to limit its disk usage. Here is a proposal to set a new file appemder (RFA) as the default appender. It can be configured via env so that existing tools can use the current behavior of using DRFA instead. This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy
[ https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240012#comment-13240012 ] Himanshu Vashishtha commented on HBASE-5655: Yes, it will not roll over the log on a date change. Disk space is not the only issue for DRFA; its java doc says there are also possible concurrency issues (or data loss). DailyRollingFileAppender has been observed to exhibit synchronization issues and data loss. The log4j extras companion includes alternatives which should be considered for new deployments and which are discussed in the documentation for org.apache.log4j.rolling.RollingFileAppender. Therefore, I think it will be a positive move if we switch to RFA as default; and let users set an env param in case they do want to use DRFA one. Its just an idea, though :) Cap space usage of default log4j rolling policy --- Key: HBASE-5655 URL: https://issues.apache.org/jira/browse/HBASE-5655 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha The current default log4j policy is to use Daily Rolling File Appender (DRFA). At times, its good to have a cap on the maximum size of the logs in order to limit its disk usage. Here is a proposal to set a new file appemder (RFA) as the default appender. It can be configured via env so that existing tools can use the current behavior of using DRFA instead. This is in parallel with jira Hadoop-8149. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5222) Stopping replication via the stop_replication command in hbase shell on a slave cluster isn't acknowledged in the replication sink
[ https://issues.apache.org/jira/browse/HBASE-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13236639#comment-13236639 ] Himanshu Vashishtha commented on HBASE-5222: @JD: When you want to use replication, you ought to run these commands (plus setting replication hbase.replication to true in the hase-config.xml) on the master cluster. The slave cluster configs is not changed (in case of simple Master-slave replication). So, in case when hbase.replication is false (or default) on the slave, its replication specific code will be null, and make these commands non-effective; no? Stopping replication via the stop_replication command in hbase shell on a slave cluster isn't acknowledged in the replication sink Key: HBASE-5222 URL: https://issues.apache.org/jira/browse/HBASE-5222 Project: HBase Issue Type: Bug Components: replication, shell Affects Versions: 0.90.4 Reporter: Josh Wymer After running stop_replication in the hbase shell on our slave cluster we saw replication continue for weeks. Turns out that the replication sink is missing a check to get the replication state and therefore continued to write. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5222) Stopping replication via the stop_replication command in hbase shell on a slave cluster isn't acknowledged in the replication sink
[ https://issues.apache.org/jira/browse/HBASE-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13236742#comment-13236742 ] Himanshu Vashishtha commented on HBASE-5222: @Josh: In case you want to do some maintenance on the slave cluster, while you do want to resume the replication once it is restored, you don't need to pull the stop trigger. The master cluster RS see that they can't connect to the slave cluster's RS anymore and will keep on waiting till they are up (sleeping/awaking loop). But in case you are also stopping the slave cluster's ZK, then you might have to remove/add it again. As the Master cluster just stops caring about it then. There is also a jira in the upstream in which you can enable/disable a particular peer (HBase-3143). So, afaik, running commands on the slave cluster are futile as its the master cluster which does all the work. PS: This is based on few days using plus 1 day code digging (yesterday:) experience. So, let's see what JD says. Stopping replication via the stop_replication command in hbase shell on a slave cluster isn't acknowledged in the replication sink Key: HBASE-5222 URL: https://issues.apache.org/jira/browse/HBASE-5222 Project: HBase Issue Type: Bug Components: replication, shell Affects Versions: 0.90.4 Reporter: Josh Wymer After running stop_replication in the hbase shell on our slave cluster we saw replication continue for weeks. Turns out that the replication sink is missing a check to get the replication state and therefore continued to write. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5222) Stopping replication via the stop_replication command in hbase shell on a slave cluster isn't acknowledged in the replication sink
[ https://issues.apache.org/jira/browse/HBASE-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13235331#comment-13235331 ] Himanshu Vashishtha commented on HBASE-5222: Shouldn't this command be run on the master cluster instead? Stopping replication via the stop_replication command in hbase shell on a slave cluster isn't acknowledged in the replication sink Key: HBASE-5222 URL: https://issues.apache.org/jira/browse/HBASE-5222 Project: HBase Issue Type: Bug Components: replication, shell Affects Versions: 0.90.4 Reporter: Josh Wymer After running stop_replication in the hbase shell on our slave cluster we saw replication continue for weeks. Turns out that the replication sink is missing a check to get the replication state and therefore continued to write. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4348) Add metrics for regions in transition
[ https://issues.apache.org/jira/browse/HBASE-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13232468#comment-13232468 ] Himanshu Vashishtha commented on HBASE-4348: Thanks for the review Jon. I incorporated your suggestions, and have tested this new patch on a 5 node cluster and randomly killing regions servers to bring some regions in transition, and noting the change in metrics. I wonder how the metrics for RIT can be tested by some simple unit tests. Attached is the new UI for region in transition section on the master UI. I will upload a review request on the board soon. Add metrics for regions in transition - Key: HBASE-4348 URL: https://issues.apache.org/jira/browse/HBASE-4348 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Himanshu Vashishtha Priority: Minor Labels: noob Attachments: 4348-metrics-v3.patch, 4348-v1.patch, 4348-v2.patch, RITs.png, metrics-v2.patch The following metrics would be useful for monitoring the master: - the number of regions in transition - the number of regions in transition that have been in transition for more than a minute - how many seconds has the oldest region-in-transition been in transition -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5543) Add a keepalive option for IPC connections
[
https://issues.apache.org/jira/browse/HBASE-5543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13227333#comment-13227333
]
Himanshu Vashishtha commented on HBASE-5543:
What is the scope of the uuid token in the Coprocessor context. The current
approach is to subdivide the calls in terms of regions; then submit a Callable
object for each of these Regions; obtain a Future object on each of these calls
and block until all of them have returned some result.
So, a uuid from the client side server proxy object, or a list of uuids from
all the involved regions, or something more elegant which I am missing. Please
suggest. Thanks.
Add a keepalive option for IPC connections
--
Key: HBASE-5543
URL: https://issues.apache.org/jira/browse/HBASE-5543
Project: HBase
Issue Type: Improvement
Components: client, coprocessors, ipc
Reporter: Andrew Purtell
On the user list someone wrote in with a connection failure due to a long
running coprocessor:
{quote}
On Wed, Mar 7, 2012 at 10:59 PM, raghavendhra rahul wrote:
2012-03-08 12:03:09,475 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server
Responder, call execCoprocessor([B@50cb21, getProjection(), rpc version=1,
client version=0, methodsFingerPrint=0), rpc version=1, client version=29,
methodsFingerPrint=54742778 from 10.184.17.26:46472: output error
2012-03-08 12:03:09,476 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server
handler 7 on 60020 caught: java.nio.channels.ClosedChannelException
{quote}
I suggested in response we might consider give our RPC a keepalive option for
calls that may run for a long time (like execCoprocessor).
LarsH +1ed the idea:
{quote}
+1 on keepalive. It's a shame (especially for long running server code) to
do all the work, just to find out at the end that the client has given up.
Or maybe there should be a way to cancel an operation if the clients decides
it does not want to wait any longer (PostgreSQL does that for example). Here
that would mean the server would need to check periodically and coprocessors
would need to be written to support that - so maybe that's no-starter.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4348) Add metrics for regions in transition
[ https://issues.apache.org/jira/browse/HBASE-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13223969#comment-13223969 ] Himanshu Vashishtha commented on HBASE-4348: I found one issue with the existing patch; will upload a new one soon! Add metrics for regions in transition - Key: HBASE-4348 URL: https://issues.apache.org/jira/browse/HBASE-4348 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Himanshu Vashishtha Priority: Minor Labels: noob Attachments: 4348-v1.patch, 4348-v2.patch, RITs.png, metrics-v2.patch The following metrics would be useful for monitoring the master: - the number of regions in transition - the number of regions in transition that have been in transition for more than a minute - how many seconds has the oldest region-in-transition been in transition -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4348) Add metrics for regions in transition
[ https://issues.apache.org/jira/browse/HBASE-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13222173#comment-13222173 ] Himanshu Vashishtha commented on HBASE-4348: I have some some analysis and some questions regarding the metrics here. Please correct me if I am wrong. RegionServer updates its metrics at fix intervals defined by hbase.regionserver.msginterval and the metrics are pushed to the record after specified interval. Master don't use this methodology; rather whenever the master statistics are changed, the caller is supposed to update the metrics value on its own. I am wondering why we don't follow the same approach for master (because the stats we are recording are not changing that often?). The reason I am asking is because the regionsInTransition map is mutated at bunch of places in the AssignmentManager class; so either I should invoke the method to compute and push the stats at all such callstacks, or may be we can have a similar approach as used by RegionServer. Add metrics for regions in transition - Key: HBASE-4348 URL: https://issues.apache.org/jira/browse/HBASE-4348 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Himanshu Vashishtha Priority: Minor Labels: noob Attachments: 4348-v1.patch, 4348-v2.patch, RITs.png The following metrics would be useful for monitoring the master: - the number of regions in transition - the number of regions in transition that have been in transition for more than a minute - how many seconds has the oldest region-in-transition been in transition -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4348) Add metrics for regions in transition
[ https://issues.apache.org/jira/browse/HBASE-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13222174#comment-13222174 ] Himanshu Vashishtha commented on HBASE-4348: I have done some analysis and some questions regarding the metrics here. Please correct me if I am wrong. RegionServer updates its metrics at fix intervals defined by hbase.regionserver.msginterval and the metrics are pushed to the record after specified interval. Master don't use this methodology; rather whenever the master statistics are changed, the caller is supposed to update the metrics value on its own. I am wondering why we don't follow the same approach for master (because the stats we are recording are not changing that often?). The reason I am asking is because the regionsInTransition map is mutated at bunch of places in the AssignmentManager class; so either I should invoke the method to compute and push the stats at all such callstacks, or may be we can have a similar approach as used by RegionServer. Add metrics for regions in transition - Key: HBASE-4348 URL: https://issues.apache.org/jira/browse/HBASE-4348 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Himanshu Vashishtha Priority: Minor Labels: noob Attachments: 4348-v1.patch, 4348-v2.patch, RITs.png The following metrics would be useful for monitoring the master: - the number of regions in transition - the number of regions in transition that have been in transition for more than a minute - how many seconds has the oldest region-in-transition been in transition -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4348) Add metrics for regions in transition
[ https://issues.apache.org/jira/browse/HBASE-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13220335#comment-13220335 ] Himanshu Vashishtha commented on HBASE-4348: It involves a UI related change, so I did test it on a 2 node cluster. Shall I attach a screenshot for the new metrics? Add metrics for regions in transition - Key: HBASE-4348 URL: https://issues.apache.org/jira/browse/HBASE-4348 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Himanshu Vashishtha Priority: Minor Labels: noob The following metrics would be useful for monitoring the master: - the number of regions in transition - the number of regions in transition that have been in transition for more than a minute - how many seconds has the oldest region-in-transition been in transition -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4348) Add metrics for regions in transition
[ https://issues.apache.org/jira/browse/HBASE-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13220393#comment-13220393 ] Himanshu Vashishtha commented on HBASE-4348: I like the idea of having only one table for RITs and adding a new column (at the end) showing in-transition time and marking the longest one in red. But the jira description also says to show number of regions that are in transition for more than a minute too? Should that be in a separate line below the table? (Sorry, my ui senses are quite raw I know) Add metrics for regions in transition - Key: HBASE-4348 URL: https://issues.apache.org/jira/browse/HBASE-4348 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Himanshu Vashishtha Priority: Minor Labels: noob Attachments: 4348-v1.patch, 4348-v2.patch, RITs.png The following metrics would be useful for monitoring the master: - the number of regions in transition - the number of regions in transition that have been in transition for more than a minute - how many seconds has the oldest region-in-transition been in transition -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4348) Add metrics for regions in transition
[ https://issues.apache.org/jira/browse/HBASE-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13220697#comment-13220697 ] Himanshu Vashishtha commented on HBASE-4348: yeah, I am working on the metrics. I am new to this part, so its taking a bit longer. Add metrics for regions in transition - Key: HBASE-4348 URL: https://issues.apache.org/jira/browse/HBASE-4348 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Himanshu Vashishtha Priority: Minor Labels: noob Attachments: 4348-v1.patch, 4348-v2.patch, RITs.png The following metrics would be useful for monitoring the master: - the number of regions in transition - the number of regions in transition that have been in transition for more than a minute - how many seconds has the oldest region-in-transition been in transition -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5486) Warn message in HTable: Stringify the byte[]
[
https://issues.apache.org/jira/browse/HBASE-5486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13218361#comment-13218361
]
Himanshu Vashishtha commented on HBASE-5486:
This is converting a static byte array into a String, I don't think testing
will add anything. Please let me know if you have other thoughts.
Warn message in HTable: Stringify the byte[]
Key: HBASE-5486
URL: https://issues.apache.org/jira/browse/HBASE-5486
Project: HBase
Issue Type: Bug
Components: client
Affects Versions: 0.92.0
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
Priority: Trivial
Labels: noob
Attachments: 5486.patch
The warn message in the method getStartEndKeys() in HTable can be improved by
stringifying the byte array for Regions.Qualifier
Currently, a sample message is like :
12/01/17 16:36:34 WARN client.HTable: Null [B@552c8fa8 cell in
keyvalues={test5,\xC9\xA2\x00\x00\x00\x00\x00\x00/00_0,1326642537734.dbc62b2765529a9ad2ddcf8eb58cb2dc./info:server/1326750341579/Put/vlen=28,
test5,\xC9\xA2\x00\x00\x00\x00\x00\x00/00_0,1326642537734.dbc62b2765529a9ad2ddcf8eb58cb2dc./info:serverstartcode/1326750341579/Put/vlen=8}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4348) Add metrics for regions in transition
[ https://issues.apache.org/jira/browse/HBASE-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215769#comment-13215769 ] Himanshu Vashishtha commented on HBASE-4348: I have created a patch, which involves a new method in org.apache.hadoop.hbase.master.AssignmentManager and supporting code in src/main/jamon/org/apache/hbase/tmpl/master/AssignmentManagerStatusTmpl.jamon. I am running it on my local system, and wonder about how to test this, i.e., to get some regions in RIT. Any suggestions please? Add metrics for regions in transition - Key: HBASE-4348 URL: https://issues.apache.org/jira/browse/HBASE-4348 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Himanshu Vashishtha Priority: Minor Labels: noob The following metrics would be useful for monitoring the master: - the number of regions in transition - the number of regions in transition that have been in transition for more than a minute - how many seconds has the oldest region-in-transition been in transition -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
