[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[ https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13767690#comment-13767690 ] stack commented on HBASE-8810: -- Purging the unused would be coolio and fixing up the mismatches would help too. Bring in code constants in line with default xml's -- Key: HBASE-8810 URL: https://issues.apache.org/jira/browse/HBASE-8810 Project: HBase Issue Type: Bug Components: Usability Reporter: Elliott Clark Assignee: Elliott Clark Attachments: 8810.txt, 8810v2.txt, hbase-default_to_java_constants.xsl, HBaseDefaultXMLConstants.java After the defaults were changed in the xml some constants were left the same. DEFAULT_HBASE_CLIENT_PAUSE for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[ https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760395#comment-13760395 ] Nicolas Liochon commented on HBASE-8810: here is the result of a pass on the current defaults: hbase.master.logcleaner.ttl ok hbase.master.catalog.timeout NOK not used anymore hbase.master.dns.interface ok hbase.master.dns.nameserver ok hbase.regionserver.port ok hbase.regionserver.info.port.auto NOK not used hbase.regionserver.msginterval ok hbase.regionserver.optionallogflushinterval ok hbase.regionserver.regionSplitLimit ok hbase.regionserver.logroll.period ok hbase.regionserver.logroll.errors.tolerated NOK 2 or 0 hbase.regionserver.hlog.reader.impl ok hbase.regionserver.global.memstore.upperLimit ok hbase.regionserver.global.memstore.lowerLimit NOK .38 or .35 hbase.regionserver.optionalcacheflushinterval ok hbase.regionserver.catalog.timeout NOK not used hbase.regionserver.dns.interface ok zookeeper.session.timeout NOK 3 minutes, 90s, 180 * 1000 zookeeper.znode.rootserver ok hbase.zookeeper.peerport ok hbase.zookeeper.leaderport ok hbase.zookeeper.useMulti NOK true false hbase.config.read.zookeeper.config ok hbase.client.write.buffer ok hbase.client.pause NOK 100 1000 hbase.client.retries.number NOK 31 35 hbase.client.scanner.caching ok hbase.client.keyvalue.maxsize NOK -1 and other hbase.client.scanner.timeout.period ok hbase.bulkload.retries.number ok hbase.balancer.period ok hbase.regions.slop NOK 0.01 .02 hbase.server.thread.wakefrequency NOK 1 6 hbase.server.versionfile.writeattempts ok hbase.hregion.memstore.flush.size ok hbase.hregion.preclose.flush.size ok hbase.hregion.memstore.block.multiplier ok hbase.hregion.memstore.mslab.enabled ok hbase.hregion.max.filesize ok hbase.hregion.majorcompaction ok hbase.hregion.majorcompaction.jitter ok hbase.hstore.compactionThreshold ok hbase.hstore.blockingStoreFiles NOK 10 7 hbase.hstore.blockingWaitTime ok hbase.hstore.compaction.max ok hbase.hstore.compaction.kv.max ok hbase.storescanner.parallel.seek.enable ok hbase.storescanner.parallel.seek.threads ok hfile.block.cache.size NOK 0.4 0.25 hfile.block.index.cacheonwrite ok hfile.format.version ok hfile.block.bloom.cacheonwrite ok hbase.rs.cacheblocksonwrite ok hbase.ipc.client.tcpnodelay ok hbase.hstore.bytes.per.checksum ok Bring in code constants in line with default xml's -- Key: HBASE-8810 URL: https://issues.apache.org/jira/browse/HBASE-8810 Project: HBase Issue Type: Bug Components: Usability Reporter: Elliott Clark Assignee: Elliott Clark Attachments: 8810.txt, 8810v2.txt, hbase-default_to_java_constants.xsl, HBaseDefaultXMLConstants.java After the defaults were changed in the xml some constants were left the same. DEFAULT_HBASE_CLIENT_PAUSE for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[ https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13758123#comment-13758123 ] Nicolas Liochon commented on HBASE-8810: I forgot about this one. Sorry. I'm gonna propose a stupid patch for the main differences. By stupid I mean nothing fancy, just grep change. Bring in code constants in line with default xml's -- Key: HBASE-8810 URL: https://issues.apache.org/jira/browse/HBASE-8810 Project: HBase Issue Type: Bug Components: Usability Reporter: Elliott Clark Assignee: Elliott Clark Attachments: 8810.txt, 8810v2.txt, hbase-default_to_java_constants.xsl, HBaseDefaultXMLConstants.java After the defaults were changed in the xml some constants were left the same. DEFAULT_HBASE_CLIENT_PAUSE for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[ https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13702618#comment-13702618 ] stack commented on HBASE-8810: -- [~nkeywal] Sounds like aligning the constants w/ the hbase-default.xml would be a big undertaking. How to proceed with this issue? Let Elliott and Anoops have their issue back and commit their patch for now that does some fixup and then continue the make-it-all-javadoc in another issue? Bring in code constants in line with default xml's -- Key: HBASE-8810 URL: https://issues.apache.org/jira/browse/HBASE-8810 Project: HBase Issue Type: Bug Components: Usability Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.95.2 Attachments: 8810.txt, 8810v2.txt, hbase-default_to_java_constants.xsl, HBaseDefaultXMLConstants.java After the defaults were changed in the xml some constants were left the same. DEFAULT_HBASE_CLIENT_PAUSE for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[
https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13700544#comment-13700544
]
Nicolas Liochon commented on HBASE-8810:
I've written this
{code}
package org.apache.hadoop.hbase;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.hadoop.conf.Configuration;
import org.junit.Test;
import org.junit.experimental.categories.Category;
import java.lang.reflect.Field;
import java.util.Map;
@Category(SmallTests.class)
public class TestDefaultSettings {
public static final Log LOG =
LogFactory.getLog(TestDefaultSettings.class);
private String codeNameFromXMLName(String xmlName){
String codeName = DEFAULT_ + xmlName.toUpperCase().replace('.',
'_').trim();
return codeName;
}
@Test
public void testDefaultSettings() throws IllegalAccessException {
Configuration codeConf = new Configuration();
Configuration xmlConf = HBaseConfiguration.create();
Class? hcc = HConstants.class;
for (Map.EntryString, String e : xmlConf){
String xmlName = e.getKey();
String codeName = codeNameFromXMLName(xmlName);
try {
Field f = hcc.getField(codeName);
String codeVal = ( + f.get(null)).trim();
if (e.getValue().equals(codeVal)){
System.out.println(OK xmlName: + xmlName + , codeConf: code= +
codeVal +xml= + e.getValue());
}else {
System.err.println(NOK xmlName: + xmlName + , codeConf: code= +
codeVal +xml= + e.getValue());
}
} catch (NoSuchFieldException e1) {
System.out.println( NoSuchFieldException: xmlName: +
xmlName + , + codeConf: code= + codeName +xml= + e.getKey());
}
}
}
}
{code}
But it's not a huge success because the naming differs in most cases.
Bring in code constants in line with default xml's
--
Key: HBASE-8810
URL: https://issues.apache.org/jira/browse/HBASE-8810
Project: HBase
Issue Type: Bug
Components: Usability
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 0.95.2
Attachments: 8810.txt, 8810v2.txt,
hbase-default_to_java_constants.xsl, HBaseDefaultXMLConstants.java
After the defaults were changed in the xml some constants were left the same.
DEFAULT_HBASE_CLIENT_PAUSE for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[
https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13699832#comment-13699832
]
Nicolas Liochon commented on HBASE-8810:
bq. So, styling hbase-default.xml to produce a constants file to include would
be a bunch of work, too much work. (bunch of ugly xsl).
I was thinking about something like this:
- hand written XML file, as today, except it is NOT loaded. We always use the
code defaults
- so we have one duplication (on this file and in the code), and we check it
with a test.
void testDefaultOk(){
Configuration file = new Configuration();
file.load(hbase-default-sample.xml);
Class? code = HConstant.class;
for (String key: file.keys){
String fileVal = file.get(key);
String codeVal = + code.getField(DEFAULT_ + key).val;
Assert.assertEquals(codeVal, fileVal)
}
}
The issue with the generation .xml -- java is that we will need to add the
typing in the xml (w/o this, we won't find an heuristic to choose between long
integer imho).
Bring in code constants in line with default xml's
--
Key: HBASE-8810
URL: https://issues.apache.org/jira/browse/HBASE-8810
Project: HBase
Issue Type: Bug
Components: Usability
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 0.95.2
Attachments: 8810.txt, 8810v2.txt,
hbase-default_to_java_constants.xsl, HBaseDefaultXMLConstants.java
After the defaults were changed in the xml some constants were left the same.
DEFAULT_HBASE_CLIENT_PAUSE for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[
https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13700199#comment-13700199
]
stack commented on HBASE-8810:
--
[~nkeywal] I'd have to do the interpolations in code:
{code}
namehbase.tmp.dir/name
value${java.io.tmpdir}/hbase-${user.name}/value
{code}
... which is then used later in stuff like:
{code}
namehbase.rootdir/name
valuefile://${hbase.tmp.dir}/hbase/value
{code}
Won't we have typing issue if we go your approach too?
file.get returns a String.
code.getField returns an Object (could be a String, could be long, etc.)
Since types are not same, overloading will not be able to discern the right
assertEquals
I'm fine hand-coding this all up but would like to avoid having to do update in
two places.
Could we mark public configs w/ annotation and generate the xml for inclusion
in doc? (There would still be the interpolation issue -- though I suppose this
ain't the end of the world; we could just do system properties).
Bring in code constants in line with default xml's
--
Key: HBASE-8810
URL: https://issues.apache.org/jira/browse/HBASE-8810
Project: HBase
Issue Type: Bug
Components: Usability
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 0.95.2
Attachments: 8810.txt, 8810v2.txt,
hbase-default_to_java_constants.xsl, HBaseDefaultXMLConstants.java
After the defaults were changed in the xml some constants were left the same.
DEFAULT_HBASE_CLIENT_PAUSE for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[
https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13700233#comment-13700233
]
Nicolas Liochon commented on HBASE-8810:
bq. Won't we have typing issue if we go your approach too?
I think it will work because I'm converting the code value to a String, so I
compare two strings.
bq. valuefile://${hbase.tmp.dir}/hbase/value
I don't know for this one.
bq. public configs w/ annotation and generate the xml for inclusion in doc
We can do that as well. Or we can ensure that the javadoc is very good and use
it as a reference (this would solve everything actually).
Bring in code constants in line with default xml's
--
Key: HBASE-8810
URL: https://issues.apache.org/jira/browse/HBASE-8810
Project: HBase
Issue Type: Bug
Components: Usability
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 0.95.2
Attachments: 8810.txt, 8810v2.txt,
hbase-default_to_java_constants.xsl, HBaseDefaultXMLConstants.java
After the defaults were changed in the xml some constants were left the same.
DEFAULT_HBASE_CLIENT_PAUSE for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[ https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13700292#comment-13700292 ] stack commented on HBASE-8810: -- bq. Or we can ensure that the javadoc is very good and use it as a reference (this would solve everything actually). We'd break out the configs we think operators might want to tweak to a special java file and then we'd point at it in the refguide? bq. I think it will work because I'm converting the code value to a String, so I compare two strings. I see. Yes. bq. I don't know for this one. I think we could work around this using system property so its easy-enough to override Bring in code constants in line with default xml's -- Key: HBASE-8810 URL: https://issues.apache.org/jira/browse/HBASE-8810 Project: HBase Issue Type: Bug Components: Usability Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.95.2 Attachments: 8810.txt, 8810v2.txt, hbase-default_to_java_constants.xsl, HBaseDefaultXMLConstants.java After the defaults were changed in the xml some constants were left the same. DEFAULT_HBASE_CLIENT_PAUSE for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[ https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13698781#comment-13698781 ] Nicolas Liochon commented on HBASE-8810: Following the discussion in HBASE-6295: bq. I do agree that we should combine all of the test hbase-site.xml's into one so that there's less confusion on which is being used. I think we could have: - a file called hbase-settings-sample.xml that would not be included when we read the conf (while we read hbase-default and hbase-site today). It would be for documentation only. - a unit test to load this file and compare with the code default, to ensure our doc is in line with the code. Bring in code constants in line with default xml's -- Key: HBASE-8810 URL: https://issues.apache.org/jira/browse/HBASE-8810 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Elliott Clark After the defaults were changed in the xml some constants were left the same. DEFAULT_HBASE_CLIENT_PAUSE for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[
https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13699395#comment-13699395
]
stack commented on HBASE-8810:
--
bq. a file called hbase-settings-sample.xml that would not be included when we
read the conf (while we read hbase-default and hbase-site today). It would be
for documentation only.
We could generate a hbase-default.{html,txt} from hbase-default.xml into conf
dir? Or just put a README.txt into the conf or an xml comment into
hbase-site.template.xml which pointed at the refguide or where to find
hbase-default.xml in src or in jar? This would be less likely to rot in a user
deploy?
bq. a unit test to load this file and compare with the code default, to ensure
our doc is in line with the code.
This is a good idea except except how would it work? How would the unit test
find the code default setting?
Bring in code constants in line with default xml's
--
Key: HBASE-8810
URL: https://issues.apache.org/jira/browse/HBASE-8810
Project: HBase
Issue Type: Bug
Reporter: Elliott Clark
Assignee: Elliott Clark
After the defaults were changed in the xml some constants were left the same.
DEFAULT_HBASE_CLIENT_PAUSE for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[ https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13699419#comment-13699419 ] Nicolas Liochon commented on HBASE-8810: bq. Or just put a README.txt into the conf or an xml comment into hbase-site.template.xml which pointed at the refguide or where to find hbase-default.xml in src or in jar? This would be less likely to rot in a user deploy? I think so :-) bq. This is a good idea except except how would it work? How would the unit test find the code default setting? I was thinking about loading hbase-default-sample and then checking against the value in HConstant by reflexivity. Bring in code constants in line with default xml's -- Key: HBASE-8810 URL: https://issues.apache.org/jira/browse/HBASE-8810 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Elliott Clark After the defaults were changed in the xml some constants were left the same. DEFAULT_HBASE_CLIENT_PAUSE for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[
https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13699453#comment-13699453
]
stack commented on HBASE-8810:
--
Here is result of a little experiment where I generate a java constants file
from hbase-default.xml.
Here is how I ran did the generation:
{code}
$ java -classpath /opt/local/share/java/saxon9he.jar net.sf.saxon.Transform
-xsl:hbase-default_to_java_constants.xsl
~/checkouts/hbase/hbase-common/src/main/resources/hbase-default.xml
HBaseDefaultXMLConstants.java
{code}
We could have the build do this for us if we wanted.
Attached are stylesheet and example generated class. Needs a bit of work
including marking the hbase-default.xml w/ type information; e.g. if a String,
I'd supply quotes.
Bring in code constants in line with default xml's
--
Key: HBASE-8810
URL: https://issues.apache.org/jira/browse/HBASE-8810
Project: HBase
Issue Type: Bug
Reporter: Elliott Clark
Assignee: Elliott Clark
After the defaults were changed in the xml some constants were left the same.
DEFAULT_HBASE_CLIENT_PAUSE for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[ https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13699743#comment-13699743 ] stack commented on HBASE-8810: -- The hbase-default.xml is hard to get rid of. While it is a pain to update it and code, hbase-default.xml is used to publish those configs. we think operators should be concerned with, it is used to generate documentation (the descriptions are in the xml file and no where else), and we sometimes make use of the interpolation feature which substitutes variables w/ environment defines: e.g. user or tmp.dir. So, styling hbase-default.xml to produce a constants file to include would be a bunch of work, too much work. 1. I'd have to add type attributes to hbase-default.xml and then in stylesheet dependent on whether string, boolean, int or long, format the value appropriately (bunch of ugly xsl). 2. Change build to generate then include this new file. 3. Go through code and change all hbase-default.xml properties to instead get their values out of this constants file. 4. Put in place docs and some constraint so adding a value to hbase-default.xml, you somehow would get it all right. Let me see if can generate unit test from hbase-default.xml. Will have same typing issue though Bring in code constants in line with default xml's -- Key: HBASE-8810 URL: https://issues.apache.org/jira/browse/HBASE-8810 Project: HBase Issue Type: Bug Components: Usability Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.95.2 Attachments: hbase-default_to_java_constants.xsl, HBaseDefaultXMLConstants.java After the defaults were changed in the xml some constants were left the same. DEFAULT_HBASE_CLIENT_PAUSE for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[ https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694948#comment-13694948 ] Elliott Clark commented on HBASE-8810: -- Thanks I'll add those to my patch. Bring in code constants in line with default xml's -- Key: HBASE-8810 URL: https://issues.apache.org/jira/browse/HBASE-8810 Project: HBase Issue Type: Bug Reporter: Elliott Clark After the defaults were changed in the xml some constants were left the same. DEFAULT_HBASE_CLIENT_PAUSE for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[
https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694480#comment-13694480
]
Anoop Sam John commented on HBASE-8810:
---
Some more
{noformat}
HConstants
public static final float HFILE_BLOCK_CACHE_SIZE_DEFAULT = 0.25f;
MemStoreFlusher
private static final float DEFAULT_LOWER = 0.35f;
{noformat}
Pls see my comment here
https://issues.apache.org/jira/browse/HBASE-8450?focusedCommentId=13671133page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13671133
Bring in code constants in line with default xml's
--
Key: HBASE-8810
URL: https://issues.apache.org/jira/browse/HBASE-8810
Project: HBase
Issue Type: Bug
Reporter: Elliott Clark
After the defaults were changed in the xml some constants were left the same.
DEFAULT_HBASE_CLIENT_PAUSE for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
