[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371572#comment-16371572 ] Mike Drob commented on HBASE-19920: --- subtask is fair. thanks, stack > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch, > HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch, > HBASE-19920.v9.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367988#comment-16367988 ] Hudson commented on HBASE-19920: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4596 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4596/]) HBASE-19920 Lazy init for ProtobufUtil classloader (mdrob: rev 8cf7a9d51ee92444e5e2c730045d2289ea97e82a) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RemoteWithExtrasException.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/token/TokenUtil.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/security/token/TestTokenUtil.java > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch, > HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch, > HBASE-19920.v9.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366540#comment-16366540 ] Francis Liu commented on HBASE-19920: - Test looks good. +1 post QA issues. Thanks [~mdrob]. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch, > HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch, > HBASE-19920.v9.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366210#comment-16366210 ] Hadoop QA commented on HBASE-19920: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 15s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 24s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m 46s{color} | {color:green} branch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 44s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 31s{color} | {color:red} hbase-client: The patch generated 3 new + 297 unchanged - 0 fixed = 300 total (was 297) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 59s{color} | {color:red} hbase-server: The patch generated 1 new + 5 unchanged - 0 fixed = 6 total (was 5) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m 19s{color} | {color:green} patch has no errors when building our shaded downstream artifacts. {color} | | {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m 27s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color} | | {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m 40s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color} | | {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m 58s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 3s{color} | {color:green} hbase-client in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green}102m 18s{color} | {color:green} hbase-server in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 28s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}137m 9s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 | | JIRA Issue | HBASE-19920 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12910777/HBASE-19920.v9.patch | | Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366044#comment-16366044 ] Sean Busbey commented on HBASE-19920: - +1 pending QA feedback. {code} --- a/hbase-server/src/main/java/org/apache/hadoop/hbase/security/token/TokenUtil.java +++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/security/token/TokenUtil.java ... +import org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting; + {code} nit: no longer used. fine to fix this on commit. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch, > HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch, > HBASE-19920.v9.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366019#comment-16366019 ] Mike Drob commented on HBASE-19920: --- v9 to address Sean's comments > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch, > HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch, > HBASE-19920.v9.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365980#comment-16365980 ] Mike Drob commented on HBASE-19920: --- bq. my intuition is telling me that the two ProtobufUtils should be sharing a common place for it though Probably. There's a TODO at the top of one to generate it from the other, I'd rather leave it until then. bq. Is there a reason we can't just hide the holder behind something like DynamicClassloader.getInstance()? Not all DCLs are created the same. There's one in RemoteWithExtrasException too that I will update to use this same lazy pattern though. {quote} So long as you're changing this up to do lazy init, should we also clean up how we init? HBC.create can throw (like if the xml config version is off from library). do we want that to propagate or should we do a try/catch to set CLASS_LOADER to null? AFAICT the use of CLASS_LOADER would then default to the bootstrap classloader, which is probably correct. Also HBC.create expressly sets the classloader for the Configuration to be whatever classloader was used to make the HBaseConfiguration instance. Should it be created with parent? {quote} Probably better to do in a follow on? bq. This feels wrong. I think this variable needs to be volatile or AtomicBoolean? It doesn't have the same guarantees about visibility since it's not within the holder, right? It works currently because it's only accessed by a single thread, but can make it volatile for future proofing, sure. bq. Given that, why not also make the fault injection private and/or final and then use reflection in the test to allow changing it in the test? private, sure, we can call setAccessible via the reflection. final, i'd rather not have to deal with manually removing flags though and I don't see much benefit. bq. Should specifically check that it's the injected ServiceException. I guess by text of the message? can make this explicit setting exception instead of boolean > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch, > HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365830#comment-16365830 ] Sean Busbey commented on HBASE-19920: - I like the move to the singleton holder idiom. I don't have a specific reason, but my intuition is telling me that the two ProtobufUtils should be sharing a common place for it though. Is there a reason we can't just hide the holder behind something like *DynamicClassloader.getInstance()*? {code} /** - * Dynamic class loader to load filter/comparators - */ - private final static ClassLoader CLASS_LOADER; + * Dynamic class loader to load filter/comparators + */ + private final static class ClassLoaderHolder { + private final static ClassLoader CLASS_LOADER; - static { - ClassLoader parent = ProtobufUtil.class.getClassLoader(); - Configuration conf = HBaseConfiguration.create(); - CLASS_LOADER = new DynamicClassLoader(conf, parent); + static { + ClassLoader parent = ProtobufUtil.class.getClassLoader(); + Configuration conf = HBaseConfiguration.create(); + CLASS_LOADER = new DynamicClassLoader(conf, parent); + } } {code} So long as you're changing this up to do lazy init, should we also clean up how we init? HBC.create can throw (like if the xml config version is off from library). do we want that to propagate or should we do a try/catch to set CLASS_LOADER to null? AFAICT the use of CLASS_LOADER would then default to the bootstrap classloader, which is probably correct. Also HBC.create expressly sets the classloader for the Configuration to be whatever classloader was used to make the HBaseConfiguration instance. Should it be created with parent? {code} + private static boolean classLoaderLoaded = false; + {code} This feels wrong. I think this variable needs to be volatile or AtomicBoolean? It doesn't have the same guarantees about visibility since it's not within the holder, right? {code} +ClassLoader cl = new URLClassLoader(new URL[] { urlPU, urlTU }, getClass().getClassLoader()); + +Class tokenUtil = cl.loadClass(TokenUtil.class.getCanonicalName()); +tokenUtil.getDeclaredField("shouldInjectFault").setBoolean(null, true); {code} We have to go through all this reflection so that the TokenUtil class specifically won't be in the classloader of the test in case some other test needed to do something that caused the dynamic classloader to get loaded, right? Given that, why not also make the fault injection private and/or final and then use reflection in the test to allow changing it in the test? {code} +} catch (InvocationTargetException e) { + Throwable t = e; + boolean serviceExceptionFound = false; + while ((t = t.getCause()) != null) { +if (t instanceof com.google.protobuf.ServiceException) { + serviceExceptionFound = true; + break; +} + } {code} Should specifically check that it's the injected ServiceException. I guess by text of the message? > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch, > HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365766#comment-16365766 ] Sean Busbey commented on HBASE-19920: - oh it grabbed the first patch instead of v8. let me figure out if I need to upgrade or manually specify the version or what. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch, > HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365720#comment-16365720 ] Sean Busbey commented on HBASE-19920: - hurm. lemme see what smart-apply-patch did. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch, > HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365692#comment-16365692 ] Mike Drob commented on HBASE-19920: --- You were looking at the wrong patch, boss. {{ProtobufExceptionUtil}} has been dropped. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch, > HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365624#comment-16365624 ] Sean Busbey commented on HBASE-19920: - I don't want to go down a rabbit hole with a late review, so if [~toffer] wants to jump back in here, feel free to take the rest of this with a grain of salt. nit: Having two classes with the same name and different packages is a code smell for me. In logging contexts plenty of things try to "cleverly" shorten names and drop package names, especially in the middle of deep package nesting. ProtobufUtil and ProtobufExceptionUtil both seem likely to get this treatment, which will be confusing for debugging later. The notes in ProtobufUtil certainly help, but is there a big disadvantage to naming one e.g. NonShadedProtobufUtil? Given the guidance about keeping the two ProtobufUtil implementations in sync, notes about why they currently differ would help. For example, the *toScan* methods in the non-shaded version skip the stanzas around handling *NeedCursorResult* in both directions. nit: the DeserializationException construction in *expectPBMagicPrefix* in shaded.ProtobufUtil is different from the non-shaded version and does an extra concatenation of string literals. Looking at the two *ProtobufExceptionUtil.java* implementations in here, they should have similar notices to those in ProtobufUtil and I have the same concerns about overlapping class names. Maybe in this case the non-shaded version could leverage the shaded one since it already references both versions of ServiceException? > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch, > HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361587#comment-16361587 ] Mike Drob commented on HBASE-19920: --- Looking for a review please > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch, > HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359248#comment-16359248 ] Mike Drob commented on HBASE-19920: --- Not sure why it couldn't comment... https://builds.apache.org/job/PreCommit-HBASE-Build/11457 ran against v8 {noformat} 02:35:47.560 02:35:47.560 02:35:47.560 Adding comment to JIRA 02:35:47.560 02:35:47.560 02:35:47.560 02:35:47.560 02:37:11.642 02:37:11.642 02:37:11.642 +1 overall 02:37:11.642 02:37:11.764 | Vote | Subsystem | Runtime | Comment 02:37:11.764 02:37:11.822 | 0 | reexec | 0m 12s | Docker mode activated. 02:37:11.853 | | || Prechecks 02:37:11.883 | 0 | findbugs | 0m 0s | Findbugs executables are not available. 02:37:11.910 | +1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. 02:37:11.939 | +1 |@author | 0m 0s | The patch does not contain any @author 02:37:11.939 | | || tags. 02:37:11.967 | +1 | test4tests | 0m 0s | The patch appears to include 1 new or 02:37:11.967 | | || modified test files. 02:37:11.994 | | || master Compile Tests 02:37:12.026 | 0 | mvndep | 0m 23s | Maven dependency ordering for branch 02:37:12.061 | +1 | mvninstall | 4m 25s | master passed 02:37:12.091 | +1 |compile | 1m 3s | master passed 02:37:12.116 | +1 | checkstyle | 1m 39s | master passed 02:37:12.145 | +1 | shadedjars | 6m 21s | branch has no errors when building our 02:37:12.145 | | || shaded downstream artifacts. 02:37:12.173 | +1 |javadoc | 0m 48s | master passed 02:37:12.197 | | || Patch Compile Tests 02:37:12.221 | 0 | mvndep | 0m 14s | Maven dependency ordering for patch 02:37:12.252 | +1 | mvninstall | 4m 21s | the patch passed 02:37:12.281 | +1 |compile | 1m 2s | the patch passed 02:37:12.307 | +1 | javac | 1m 2s | the patch passed 02:37:12.339 | +1 | checkstyle | 1m 37s | the patch passed 02:37:12.372 | +1 | whitespace | 0m 0s | The patch has no whitespace issues. 02:37:12.401 | +1 | shadedjars | 4m 42s | patch has no errors when building our 02:37:12.401 | | || shaded downstream artifacts. 02:37:12.437 | +1 |hadoopcheck | 18m 58s | Patch does not cause any errors with 02:37:12.437 | | || Hadoop 2.6.5 2.7.4 or 3.0.0. 02:37:12.480 | +1 |javadoc | 0m 50s | the patch passed 02:37:12.509 | | || Other Tests 02:37:12.538 | +1 | unit | 3m 4s | hbase-client in the patch passed. 02:37:12.563 | +1 | unit | 111m 12s | hbase-server in the patch passed. 02:37:12.593 | +1 | asflicense | 0m 40s | The patch does not generate ASF License 02:37:12.593 | | || warnings. 02:37:12.625 | | | 155m 36s | 02:37:12.626 02:37:12.626 02:37:12.626 || Subsystem || Report/Notes || 02:37:12.626 02:37:12.632 | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 | 02:37:12.637 | JIRA Issue | HBASE-19920 | 02:37:12.642 | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12909862/HBASE-19920.v8.patch | 02:37:12.647 | Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | 02:37:12.652 | uname | Linux 3b98d43f13bd 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 GNU/Linux | 02:37:12.656 | Build tool | maven | 02:37:12.660 | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | 02:37:12.663 | git revision | master / f1a81618fd | 02:37:12.688 | maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) | 02:37:12.688 | Default Java | 1.8.0_151 | 02:37:12.688 | Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11457/testReport/ | 02:37:12.688 | Max. process+thread count | 4404 (vs. ulimit of 1) | 02:37:12.688 | modules | C: hbase-client hbase-server U:
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357768#comment-16357768 ] Hadoop QA commented on HBASE-19920: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 16s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 45s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 8s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 37s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m 19s{color} | {color:green} branch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 5s{color} | {color:red} hbase-server: The patch generated 1 new + 5 unchanged - 0 fixed = 6 total (was 5) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m 43s{color} | {color:green} patch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 23m 37s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 1s{color} | {color:green} hbase-client in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 97m 59s{color} | {color:green} hbase-server in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 36s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}147m 7s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 | | JIRA Issue | HBASE-19920 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12909843/HBASE-19920.v7.patch | | Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 06a6f053dfcd 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19 15:49:21 UTC 2017 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / f1a81618fd | | maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) | | Default Java | 1.8.0_151 | | checkstyle |
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357650#comment-16357650 ] Mike Drob commented on HBASE-19920: --- Ugh. v7, try again with checkstyle. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch, > HBASE-19920.v6.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357509#comment-16357509 ] Hadoop QA commented on HBASE-19920: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 30s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 20s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 28s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 8s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 7m 46s{color} | {color:green} branch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 5s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 17s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 13s{color} | {color:red} hbase-server: The patch generated 2 new + 5 unchanged - 0 fixed = 7 total (was 5) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m 11s{color} | {color:green} patch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 21m 58s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 10s{color} | {color:green} hbase-client in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 99m 0s{color} | {color:green} hbase-server in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 42s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}151m 4s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 | | JIRA Issue | HBASE-19920 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12909825/HBASE-19920.v6.patch | | Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d515c4d769a6 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh | | git revision | master / 277ce3d8a3 | | maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) | | Default Java | 1.8.0_151 | | checkstyle |
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357270#comment-16357270 ] Mike Drob commented on HBASE-19920: --- v6: checkstyle > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch, > HBASE-19920.v6.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356339#comment-16356339 ] Hadoop QA commented on HBASE-19920: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 33s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 2s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 34s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 11s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 8m 9s{color} | {color:green} branch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 22s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 26s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 10s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 33s{color} | {color:red} hbase-client: The patch generated 3 new + 295 unchanged - 0 fixed = 298 total (was 295) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 5s{color} | {color:red} hbase-server: The patch generated 3 new + 5 unchanged - 0 fixed = 8 total (was 5) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m 48s{color} | {color:green} patch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 18m 27s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 54s{color} | {color:green} hbase-client in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 96m 56s{color} | {color:green} hbase-server in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 34s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}146m 19s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 | | JIRA Issue | HBASE-19920 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12909677/HBASE-19920.v5.patch | | Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux be72b4e3c7af 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19 15:49:21 UTC 2017 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / f5dbdf0dab | | maven |
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356194#comment-16356194 ] Mike Drob commented on HBASE-19920: --- I'm not very happy with this test because we're still only testing the side effects of "did dynamic class loader load" instead of the actual effects we care about preventing (did file system access occur) but given all the static methods at play here, I think this is the best we're going to do. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355931#comment-16355931 ] Francis Liu commented on HBASE-19920: - Skimmed the patch, the approach looks better. Tho don't we still need a test to verify no regression occurs down the road? > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354653#comment-16354653 ] Mike Drob commented on HBASE-19920: --- hadoopcheck issues have been failing on other issues and don't reproduce locally for me. can address checkstyle on commit. what do folks think of v4? > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354565#comment-16354565 ] Hadoop QA commented on HBASE-19920: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 32s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 31s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m 4s{color} | {color:green} branch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 18s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 27s{color} | {color:red} hbase-client: The patch generated 2 new + 295 unchanged - 0 fixed = 297 total (was 295) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m 27s{color} | {color:green} patch has no errors when building our shaded downstream artifacts. {color} | | {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m 38s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color} | | {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m 46s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color} | | {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 11m 1s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 48s{color} | {color:green} hbase-client in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 7s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 30m 6s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 | | JIRA Issue | HBASE-19920 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12909508/HBASE-19920.v4.patch | | Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 69624364d390 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / 001f9cc5ea | | maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) | | Default Java | 1.8.0_151 | | checkstyle |
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354530#comment-16354530 ] Mike Drob commented on HBASE-19920: --- While trying to write good tests, I realized there's a much more straightforward way to accomplish this that doesn't involve creating new classes. v4 moves the classloader into a singleton holder idiom I didn't write tests for this because at that point we'd be verifying that the JVM works as expected, which I think places it outside of the scope of what we're trying to do here. If somebody feels strongly about this, I can try to write one anyway though. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch, HBASE-19920.v4.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353185#comment-16353185 ] Hadoop QA commented on HBASE-19920: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m 21s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 18s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 48s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 28s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 6s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m 56s{color} | {color:green} branch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 11s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 17s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 46s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 24s{color} | {color:green} The patch hbase-common passed checkstyle {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 34s{color} | {color:green} hbase-client: The patch generated 0 new + 374 unchanged - 2 fixed = 374 total (was 376) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 5s{color} | {color:red} hbase-server: The patch generated 1 new + 5 unchanged - 0 fixed = 6 total (was 5) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m 56s{color} | {color:green} patch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 20m 29s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 22s{color} | {color:green} hbase-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 8s{color} | {color:green} hbase-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 6s{color} | {color:red} hbase-server in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 33s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 75m 45s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hbase.security.token.TestTokenUtil | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 | | JIRA Issue | HBASE-19920 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12909306/HBASE-19920.v3.patch | | Optional Tests | asflicense
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353157#comment-16353157 ] Ted Yu commented on HBASE-19920: Looks good overall. {code} + * @return IOException wrapped in ServiceException. {code} The above is backwards, right (ServiceException wrapped in IOException) ? {code} + TokenUtil.obtainToken(null); +} catch (Exception e) { {code} Since exception is expected, you can add assert(false) following TokenUtil.obtainToken() in the try block. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353065#comment-16353065 ] Mike Drob commented on HBASE-19920: --- v3: * import order * test classrule > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch, > HBASE-19920.v3.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353049#comment-16353049 ] Hadoop QA commented on HBASE-19920: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m 31s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 15s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 14s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 47s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m 52s{color} | {color:green} branch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 12s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 12s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 21s{color} | {color:red} hbase-common: The patch generated 2 new + 26 unchanged - 0 fixed = 28 total (was 26) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 30s{color} | {color:red} hbase-client: The patch generated 2 new + 374 unchanged - 2 fixed = 376 total (was 376) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 54s{color} | {color:red} hbase-server: The patch generated 5 new + 4 unchanged - 1 fixed = 9 total (was 5) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m 9s{color} | {color:green} patch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 16m 29s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 16s{color} | {color:red} hbase-client generated 3 new + 2 unchanged - 0 fixed = 5 total (was 2) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 19s{color} | {color:green} hbase-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 55s{color} | {color:green} hbase-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 46s{color} | {color:red} hbase-server in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 25s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 68m 34s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hbase.security.token.TestTokenUtil | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 | | JIRA Issue | HBASE-19920 | | JIRA Patch URL |
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352961#comment-16352961 ] Mike Drob commented on HBASE-19920: --- v2: * checkstyle * javadoc * add a unit test * added fault injection to TokenUtil for the test > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch, HBASE-19920.v2.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352813#comment-16352813 ] Francis Liu commented on HBASE-19920: - {quote}I think I'm in strong agreement with this now. Let's address the immediate concern to help Rohini and Attila, then possibly have a dev@ list discussion about what to do with the dynamic class loading and what appropriate use cases are. {quote} +1 > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352811#comment-16352811 ] Francis Liu commented on HBASE-19920: - {quote} That said, clients using ProtobufUtil::toException, ::toComparator, and ::toFilter still might have reason to need the dynamic jars. Not sure how prevalent this is. {quote} Took a quick look last week, except for toException All of them are used server-side. And even with toException if it can't instantiate the custom exception it just wraps it in an existing class. So I think even that is acceptable. Because if the user really needed to act on a custom exception it would need to have that exception in the classpath. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352783#comment-16352783 ] Mike Drob commented on HBASE-19920: --- bq. Having said that the approach you've currently taken sounds fine as it addresses the immediate concern. Tho a bit tricky as future code changes may make use of ProtobufUtil (sounds like we need to add an IT test to avoid regression). I think I'm in strong agreement with this now. Let's address the immediate concern to help Rohini and Attila, then possibly have a dev@ list discussion about what to do with the dynamic class loading and what appropriate use cases are. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352775#comment-16352775 ] Mike Drob commented on HBASE-19920: --- Ah, I think I misunderstood your question earlier. It's a good point about the statically referenced classes and needing to set the thread CL. I wasn't thinking in that direction. The CPEP example doesn't fit here. That said, clients using ProtobufUtil::toException, ::toComparator, and ::toFilter still might have reason to need the dynamic jars. Not sure how prevalent this is. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352758#comment-16352758 ] Francis Liu commented on HBASE-19920: - {quote}Sure, they compile against CP in maven parlance as a "provided" dependency and then when they deploy, it's already there on the cluster. {quote} Understood. My concern is how the statically referenced custom classes get loaded since the default contextclassloader in the thread won't be able to load it. Do we expect the user to directly use the DynamicClassLoader to load the classes which reference the custom cp, set the thread classloader, etc? All of which sounds undesirable. What am I missing? > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352705#comment-16352705 ] Mike Drob commented on HBASE-19920: --- Sure, they compile against CP in maven parlance as a "provided" dependency and then when they deploy, it's already there on the cluster. This kind of model works great when several clients are using the same CP libs, and you can deploy them to DFS instead of having to bundle them with each application. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352700#comment-16352700 ] Francis Liu commented on HBASE-19920: - {quote}This was my initial assessment, but when I chatted with Stack he pointed out that clients can use endpoint coprocessors. {quote} If clients use endpoint cps then don't they need to statically reference the custom service class and possibly other custom classes to make use of the endpoint? > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352684#comment-16352684 ] Mike Drob commented on HBASE-19920: --- {quote}In any case, it sounds reasonable to assume that if a client (not running in regionserver, master, etc) needs to use a custom filter/coprocessor/etc it would have direct access to the classes (ie in classpath) to use the apis. So it would seem reasonable to assume that we only need to enable DynamicClassLoader on clients running in an HBase daemon? {quote} This was my initial assessment, but when I chatted with Stack he pointed out that clients can use endpoint coprocessors. Look at this [table API|https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Table.html#coprocessorService-java.lang.Class-byte:A-byte:A-org.apache.hadoop.hbase.client.coprocessor.Batch.Call-] for example. I think it is reasonable if users know that the coproc exists in dynamic classpath to let client code look for it there. > Please check unit test failure. Tests pass locally, thanks. I'm thinking about how to add a test for this to prevent regression. I think I have an idea, will put up a new patch soon. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351992#comment-16351992 ] Francis Liu commented on HBASE-19920: - [~mdrob] Rohini and I were talking about this last friday. As far as I can tell DynamicClassLoader is mainly for supporting custom filters tho there are some other cases (eg custom comparator for checkAndPut). In any case, it sounds reasonable to assume that if a client (not running in regionserver, master, etc) needs to use a custom filter/coprocessor/etc it would have direct access to the classes (ie in classpath) to use the apis. So it would seem reasonable to assume that we only need to enable DynamicClassLoader on clients running in an HBase daemon? Having said that the approach you've currently taken sounds fine as it addresses the immediate concern. Tho a bit tricky as future code changes may make use of ProtobufUtil (sounds like we need to add an IT test to avoid regression). > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351430#comment-16351430 ] Ted Yu commented on HBASE-19920: {code} 34 * @return Exception wrapped in ServiceException or 35 * a new IOException that wraps the unexpected ServiceException. 36 */ 37public static IOException getRemoteException(com.google.protobuf.ServiceException se) { {code} For @return, IOException is returned. {code} 61 * @param e 62 */ 63public static IOException handleRemoteException(Exception e) { {code} Add comment for @param Are you going to add a test case in next patch ? Please check unit test failure. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351423#comment-16351423 ] Attila Sasvari commented on HBASE-19920: [~mdrob] we were using 1.2.0 (cdh5-1.2.0_5.13). > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351414#comment-16351414 ] Mike Drob commented on HBASE-19920: --- [~rohini] - what version of HBase were you using, so that we know to set the fixversion appropriately > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Mike Drob >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351221#comment-16351221 ] Hadoop QA commented on HBASE-19920: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m 55s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 1s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 30s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 5s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 8s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 39s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m 25s{color} | {color:green} branch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 32s{color} | {color:red} hbase-client: The patch generated 6 new + 374 unchanged - 2 fixed = 380 total (was 376) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m 37s{color} | {color:green} patch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 19m 58s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 24s{color} | {color:red} hbase-client generated 3 new + 2 unchanged - 0 fixed = 5 total (was 2) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 56s{color} | {color:green} hbase-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red}146m 58s{color} | {color:red} hbase-server in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 33s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}197m 9s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hbase.procedure.TestFailedProcCleanup | | | hadoop.hbase.TestJMXListener | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 | | JIRA Issue | HBASE-19920 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12909057/HBASE-19920.patch | | Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 9a888c7e6bbc 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4 19:09:19 UTC 2017 x86_64 GNU/Linux | | Build tool | maven | | Personality |
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351039#comment-16351039 ] Mike Drob commented on HBASE-19920: --- Attaching a patch that makes an attempt at this. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Priority: Major > Fix For: 2.0 > > Attachments: HBASE-19920.patch > > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351012#comment-16351012 ] Rohini Palaniswamy commented on HBASE-19920: There was no failure because of this. We just noticed that this was happening while debugging a different issue. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Priority: Major > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351005#comment-16351005 ] Mike Drob commented on HBASE-19920: --- [~rohini] - thanks for clarifying, yep that makes perfect sense and is something that we can address, I think. Do you have any code you can share from around the failure that we can use to create a test case? > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Priority: Major > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class > to access hbase.dynamic.jars.dir. > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350991#comment-16350991 ] Rohini Palaniswamy commented on HBASE-19920: bq. Do you want to submit a patch ? No >From my perspective, a call to get delegation token should not be 1) Creating a local directory 2) Instantiating a filesystem class be it local or remote. It is worse when it is remote because of the overhead involved with instantiating a DFSClient (opening sockets, etc). I do not have a problem if DynamicClassLoader actually does those things when the client intends to use coprocessors. Would just prefer it to be taken out of the code path of getting delegation tokens. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Priority: Major > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.rootdir}/lib > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L115-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350943#comment-16350943 ] Mike Drob commented on HBASE-19920: --- It looks like there are a couple of subtle things going on here. # Clients could use dynamic jars for coprocessors, for example if they are making a request that involves endpoint coprocessors. So I think I disagree with the implied solution. # Dynamic jars shouldn't be getting loaded from the local file system, the intended use it to load them from a shared file system like HDFS. This might break in use cases where HBase is running on LocalFS instead of HDFS, I suspect this to be seen in a test environment? # Maybe we shouldn't be creating this directory, but limit to checking if it exists and is readable. Not having the directory there shouldn't be a fatal error, probably sufficient to log a warning and move on. > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Priority: Major > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.rootdir}/lib > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L115-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350662#comment-16350662 ] Ted Yu commented on HBASE-19920: Rohini: Do you want to submit a patch ? > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Priority: Major > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.rootdir}/lib > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L115-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350623#comment-16350623 ] Mike Drob commented on HBASE-19920: --- Ah, ok a few lines up there is the mkdir > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Priority: Major > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.rootdir}/lib > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L115-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[ https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350569#comment-16350569 ] Mike Drob commented on HBASE-19920: --- I don't see any directory creation code there, can you be more specific? > TokenUtil.obtainToken unnecessarily creates a local directory > - > > Key: HBASE-19920 > URL: https://issues.apache.org/jira/browse/HBASE-19920 > Project: HBase > Issue Type: Bug >Reporter: Rohini Palaniswamy >Priority: Major > > On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil > which in its static block initializes DynamicClassLoader and that creates the > directory ${hbase.rootdir}/lib > https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L115-L127 > Since this is region server specific code, not expecting this to happen when > one accesses hbase as a client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)