[jira] [Created] (HBASE-20629) Oozie cannot start with Hadoop 3.1.0
Artem Ervits created HBASE-20629:
Summary: Oozie cannot start with Hadoop 3.1.0
Key: HBASE-20629
URL: https://issues.apache.org/jira/browse/HBASE-20629
Project: HBase
Issue Type: Bug
Environment: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z)
Maven home: /opt/maven/apache-maven-3.5.3
Java version: 1.8.0_171, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-8.b10.el7_5.x86_64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-862.2.3.el7.x86_64", arch: "amd64", family:
"unix"
Reporter: Artem Ervits
{code:java}
bin/oozied.sh start{code}
{code:java}
Setting up oozie DB
Validate DB Connection
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/util/ReflectionUtils
at org.apache.oozie.service.Services.setServiceInternal(Services.java:377)
at org.apache.oozie.service.Services.(Services.java:111)
at org.apache.oozie.tools.OozieDBCLI.getJdbcConf(OozieDBCLI.java:169)
at org.apache.oozie.tools.OozieDBCLI.createConnection(OozieDBCLI.java:918)
at org.apache.oozie.tools.OozieDBCLI.validateConnection(OozieDBCLI.java:926)
at org.apache.oozie.tools.OozieDBCLI.createDB(OozieDBCLI.java:188)
at org.apache.oozie.tools.OozieDBCLI.run(OozieDBCLI.java:131)
at org.apache.oozie.tools.OozieDBCLI.main(OozieDBCLI.java:79)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.util.ReflectionUtils
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 8 more{code}
{code:java}
/opt/oozie/oozie-5.1.0-SNAPSHOT/bin/oozie-setup.sh sharelib create -fs
hdfs://hadoop.example.com:8020 -locallib ~/share/{code}
{code:java}
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/fs/Path
at org.apache.oozie.tools.OozieSharelibCLI.run(OozieSharelibCLI.java:153)
at org.apache.oozie.tools.OozieSharelibCLI.main(OozieSharelibCLI.java:67)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.Path
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 2 more{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-20629) Oozie cannot start with Hadoop 3.1.0
[
https://issues.apache.org/jira/browse/HBASE-20629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits resolved HBASE-20629.
--
Resolution: Invalid
> Oozie cannot start with Hadoop 3.1.0
>
>
> Key: HBASE-20629
> URL: https://issues.apache.org/jira/browse/HBASE-20629
> Project: HBase
> Issue Type: Bug
> Environment: Apache Maven 3.5.3
> (3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z)
> Maven home: /opt/maven/apache-maven-3.5.3
> Java version: 1.8.0_171, vendor: Oracle Corporation
> Java home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-8.b10.el7_5.x86_64/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "3.10.0-862.2.3.el7.x86_64", arch: "amd64",
> family: "unix"
>Reporter: Artem Ervits
>Priority: Major
>
> {code:java}
> bin/oozied.sh start{code}
>
> {code:java}
> Setting up oozie DB
> Validate DB Connection
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/util/ReflectionUtils
> at org.apache.oozie.service.Services.setServiceInternal(Services.java:377)
> at org.apache.oozie.service.Services.(Services.java:111)
> at org.apache.oozie.tools.OozieDBCLI.getJdbcConf(OozieDBCLI.java:169)
> at org.apache.oozie.tools.OozieDBCLI.createConnection(OozieDBCLI.java:918)
> at org.apache.oozie.tools.OozieDBCLI.validateConnection(OozieDBCLI.java:926)
> at org.apache.oozie.tools.OozieDBCLI.createDB(OozieDBCLI.java:188)
> at org.apache.oozie.tools.OozieDBCLI.run(OozieDBCLI.java:131)
> at org.apache.oozie.tools.OozieDBCLI.main(OozieDBCLI.java:79)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.util.ReflectionUtils
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 8 more{code}
> {code:java}
> /opt/oozie/oozie-5.1.0-SNAPSHOT/bin/oozie-setup.sh sharelib create -fs
> hdfs://hadoop.example.com:8020 -locallib ~/share/{code}
> {code:java}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/fs/Path
> at org.apache.oozie.tools.OozieSharelibCLI.run(OozieSharelibCLI.java:153)
> at org.apache.oozie.tools.OozieSharelibCLI.main(OozieSharelibCLI.java:67)
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.Path
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 2 more{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-21238) MapReduceHFileSplitterJob#run shouldn't call System.exit
[
https://issues.apache.org/jira/browse/HBASE-21238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits reassigned HBASE-21238:
Assignee: Artem Ervits
> MapReduceHFileSplitterJob#run shouldn't call System.exit
>
>
> Key: HBASE-21238
> URL: https://issues.apache.org/jira/browse/HBASE-21238
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: mapreduce
>
> {code}
> if (args.length < 2) {
> usage("Wrong number of arguments: " + args.length);
> System.exit(-1);
> {code}
> Correct way of handling error condition is through return value of run method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21238) MapReduceHFileSplitterJob#run shouldn't call System.exit
[
https://issues.apache.org/jira/browse/HBASE-21238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21238:
-
Attachment: HBASE-21238.v01.patch
> MapReduceHFileSplitterJob#run shouldn't call System.exit
>
>
> Key: HBASE-21238
> URL: https://issues.apache.org/jira/browse/HBASE-21238
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: mapreduce
> Attachments: HBASE-21238.v01.patch
>
>
> {code}
> if (args.length < 2) {
> usage("Wrong number of arguments: " + args.length);
> System.exit(-1);
> {code}
> Correct way of handling error condition is through return value of run method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21238) MapReduceHFileSplitterJob#run shouldn't call System.exit
[
https://issues.apache.org/jira/browse/HBASE-21238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21238:
-
Status: Patch Available (was: Open)
> MapReduceHFileSplitterJob#run shouldn't call System.exit
>
>
> Key: HBASE-21238
> URL: https://issues.apache.org/jira/browse/HBASE-21238
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: mapreduce
> Attachments: HBASE-21238.v01.patch
>
>
> {code}
> if (args.length < 2) {
> usage("Wrong number of arguments: " + args.length);
> System.exit(-1);
> {code}
> Correct way of handling error condition is through return value of run method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-21198) Exclude dependency on net.minidev:json-smart
[
https://issues.apache.org/jira/browse/HBASE-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits reassigned HBASE-21198:
Assignee: Artem Ervits
> Exclude dependency on net.minidev:json-smart
>
>
> Key: HBASE-21198
> URL: https://issues.apache.org/jira/browse/HBASE-21198
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/14414/artifact/patchprocess/patch-javac-3.0.0.txt
> :
> {code}
> [ERROR] Failed to execute goal on project hbase-common: Could not resolve
> dependencies for project org.apache.hbase:hbase-common:jar:3.0.0-SNAPSHOT:
> Failed to collect dependencies at org.apache.hadoop:hadoop-common:jar:3.0.0
> -> org.apache.hadoop:hadoop-auth:jar:3.0.0 ->
> com.nimbusds:nimbus-jose-jwt:jar:4.41.1 ->
> net.minidev:json-smart:jar:2.3-SNAPSHOT: Failed to read artifact descriptor
> for net.minidev:json-smart:jar:2.3-SNAPSHOT: Could not transfer artifact
> net.minidev:json-smart:pom:2.3-SNAPSHOT from/to dynamodb-local-oregon
> (https://s3-us-west-2.amazonaws.com/dynamodb-local/release): Access denied
> to:
> https://s3-us-west-2.amazonaws.com/dynamodb-local/release/net/minidev/json-smart/2.3-SNAPSHOT/json-smart-2.3-SNAPSHOT.pom
> , ReasonPhrase:Forbidden. -> [Help 1]
> {code}
> We should exclude dependency on net.minidev:json-smart
> hbase-common/bin/pom.xml has done so.
> The other pom.xml should do the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21198) Exclude dependency on net.minidev:json-smart
[
https://issues.apache.org/jira/browse/HBASE-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21198:
-
Attachment: HBASE-21198.v01.patch
> Exclude dependency on net.minidev:json-smart
>
>
> Key: HBASE-21198
> URL: https://issues.apache.org/jira/browse/HBASE-21198
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21198.v01.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/14414/artifact/patchprocess/patch-javac-3.0.0.txt
> :
> {code}
> [ERROR] Failed to execute goal on project hbase-common: Could not resolve
> dependencies for project org.apache.hbase:hbase-common:jar:3.0.0-SNAPSHOT:
> Failed to collect dependencies at org.apache.hadoop:hadoop-common:jar:3.0.0
> -> org.apache.hadoop:hadoop-auth:jar:3.0.0 ->
> com.nimbusds:nimbus-jose-jwt:jar:4.41.1 ->
> net.minidev:json-smart:jar:2.3-SNAPSHOT: Failed to read artifact descriptor
> for net.minidev:json-smart:jar:2.3-SNAPSHOT: Could not transfer artifact
> net.minidev:json-smart:pom:2.3-SNAPSHOT from/to dynamodb-local-oregon
> (https://s3-us-west-2.amazonaws.com/dynamodb-local/release): Access denied
> to:
> https://s3-us-west-2.amazonaws.com/dynamodb-local/release/net/minidev/json-smart/2.3-SNAPSHOT/json-smart-2.3-SNAPSHOT.pom
> , ReasonPhrase:Forbidden. -> [Help 1]
> {code}
> We should exclude dependency on net.minidev:json-smart
> hbase-common/bin/pom.xml has done so.
> The other pom.xml should do the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21198) Exclude dependency on net.minidev:json-smart
[
https://issues.apache.org/jira/browse/HBASE-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650675#comment-16650675
]
Artem Ervits commented on HBASE-21198:
--
[~yuzhih...@gmail.com] [~stack] I've seen this before and what I learned was
that it was fixed in Hadoop 3.1,
https://issues.apache.org/jira/browse/HADOOP-15265. Hadoop 3 profile is set to
3.0.3 so this message will appear until explicitly excluded with a patch like I
just attached or bump Hadoop to 3.1. Please review.
> Exclude dependency on net.minidev:json-smart
>
>
> Key: HBASE-21198
> URL: https://issues.apache.org/jira/browse/HBASE-21198
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21198.v01.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/14414/artifact/patchprocess/patch-javac-3.0.0.txt
> :
> {code}
> [ERROR] Failed to execute goal on project hbase-common: Could not resolve
> dependencies for project org.apache.hbase:hbase-common:jar:3.0.0-SNAPSHOT:
> Failed to collect dependencies at org.apache.hadoop:hadoop-common:jar:3.0.0
> -> org.apache.hadoop:hadoop-auth:jar:3.0.0 ->
> com.nimbusds:nimbus-jose-jwt:jar:4.41.1 ->
> net.minidev:json-smart:jar:2.3-SNAPSHOT: Failed to read artifact descriptor
> for net.minidev:json-smart:jar:2.3-SNAPSHOT: Could not transfer artifact
> net.minidev:json-smart:pom:2.3-SNAPSHOT from/to dynamodb-local-oregon
> (https://s3-us-west-2.amazonaws.com/dynamodb-local/release): Access denied
> to:
> https://s3-us-west-2.amazonaws.com/dynamodb-local/release/net/minidev/json-smart/2.3-SNAPSHOT/json-smart-2.3-SNAPSHOT.pom
> , ReasonPhrase:Forbidden. -> [Help 1]
> {code}
> We should exclude dependency on net.minidev:json-smart
> hbase-common/bin/pom.xml has done so.
> The other pom.xml should do the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21198) Exclude dependency on net.minidev:json-smart
[
https://issues.apache.org/jira/browse/HBASE-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21198:
-
Status: Patch Available (was: Open)
> Exclude dependency on net.minidev:json-smart
>
>
> Key: HBASE-21198
> URL: https://issues.apache.org/jira/browse/HBASE-21198
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21198.v01.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/14414/artifact/patchprocess/patch-javac-3.0.0.txt
> :
> {code}
> [ERROR] Failed to execute goal on project hbase-common: Could not resolve
> dependencies for project org.apache.hbase:hbase-common:jar:3.0.0-SNAPSHOT:
> Failed to collect dependencies at org.apache.hadoop:hadoop-common:jar:3.0.0
> -> org.apache.hadoop:hadoop-auth:jar:3.0.0 ->
> com.nimbusds:nimbus-jose-jwt:jar:4.41.1 ->
> net.minidev:json-smart:jar:2.3-SNAPSHOT: Failed to read artifact descriptor
> for net.minidev:json-smart:jar:2.3-SNAPSHOT: Could not transfer artifact
> net.minidev:json-smart:pom:2.3-SNAPSHOT from/to dynamodb-local-oregon
> (https://s3-us-west-2.amazonaws.com/dynamodb-local/release): Access denied
> to:
> https://s3-us-west-2.amazonaws.com/dynamodb-local/release/net/minidev/json-smart/2.3-SNAPSHOT/json-smart-2.3-SNAPSHOT.pom
> , ReasonPhrase:Forbidden. -> [Help 1]
> {code}
> We should exclude dependency on net.minidev:json-smart
> hbase-common/bin/pom.xml has done so.
> The other pom.xml should do the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21198) Exclude dependency on net.minidev:json-smart
[
https://issues.apache.org/jira/browse/HBASE-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652243#comment-16652243
]
Artem Ervits commented on HBASE-21198:
--
FWIW, [~stack], I ran individual failed tests and they all pass locally
{code:java}
mvn clean
-Dtest=org.apache.hadoop.hbase.replication.TestReplicationChangingPeerRegionservers
-Dhadoop.profile=3.0 test
mvn clean
-Dtest=org.apache.hadoop.hbase.client.TestMobRestoreSnapshotFromClient
-Dhadoop.profile=3.0 test
mvn clean -Dtest=org.apache.hadoop.hbase.client.TestFromClientSide#testDeletes
-Dhadoop.profile=3.0 test
mvn clean
-Dtest=org.apache.hadoop.hbase.client.TestFromClientSide#testUnmanagedHConnection
-Dhadoop.profile=3.0 test{code}
> Exclude dependency on net.minidev:json-smart
>
>
> Key: HBASE-21198
> URL: https://issues.apache.org/jira/browse/HBASE-21198
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21198.v01.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/14414/artifact/patchprocess/patch-javac-3.0.0.txt
> :
> {code}
> [ERROR] Failed to execute goal on project hbase-common: Could not resolve
> dependencies for project org.apache.hbase:hbase-common:jar:3.0.0-SNAPSHOT:
> Failed to collect dependencies at org.apache.hadoop:hadoop-common:jar:3.0.0
> -> org.apache.hadoop:hadoop-auth:jar:3.0.0 ->
> com.nimbusds:nimbus-jose-jwt:jar:4.41.1 ->
> net.minidev:json-smart:jar:2.3-SNAPSHOT: Failed to read artifact descriptor
> for net.minidev:json-smart:jar:2.3-SNAPSHOT: Could not transfer artifact
> net.minidev:json-smart:pom:2.3-SNAPSHOT from/to dynamodb-local-oregon
> (https://s3-us-west-2.amazonaws.com/dynamodb-local/release): Access denied
> to:
> https://s3-us-west-2.amazonaws.com/dynamodb-local/release/net/minidev/json-smart/2.3-SNAPSHOT/json-smart-2.3-SNAPSHOT.pom
> , ReasonPhrase:Forbidden. -> [Help 1]
> {code}
> We should exclude dependency on net.minidev:json-smart
> hbase-common/bin/pom.xml has done so.
> The other pom.xml should do the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-21279) Split TestAdminShell into several tests
[
https://issues.apache.org/jira/browse/HBASE-21279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits reassigned HBASE-21279:
Assignee: Artem Ervits
> Split TestAdminShell into several tests
> ---
>
> Key: HBASE-21279
> URL: https://issues.apache.org/jira/browse/HBASE-21279
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: testAdminShell-output.tar.gz
>
>
> In the flaky test board, TestAdminShell often timed out
> (https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html).
> I ran the test on Linux with SSD and reproduced the timeout (see attached
> test output).
> {code}
> 2018-10-08 02:36:09,146 DEBUG [main] hbase.HBaseTestingUtility(351): Setting
> hbase.rootdir to
> /mnt/disk2/a/2-hbase/hbase-shell/target/test-data/a103d8e4-695c-a5a9-6690-1ef2580050f9
> ...
> 2018-10-08 02:49:09,093 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=7]
> master.MasterRpcServices(1171): Checking to see if procedure is done pid=871
> Took 0.7262 seconds2018-10-08 02:49:09,324 DEBUG [PEWorker-1]
> util.FSTableDescriptors(684): Wrote into
> hdfs://localhost:43859/user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-
> d935316c9241/.tmp/data/default/hbase_shell_tests_table/.tabledesc/.tableinfo.01
> 2018-10-08 02:49:09,328 INFO
> [RegionOpenAndInitThread-hbase_shell_tests_table-1]
> regionserver.HRegion(7004): creating HRegion hbase_shell_tests_table HTD ==
> 'hbase_shell_tests_table', {NAME => 'x', VERSIONS => '5',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE =>
> 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS =>
> '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'}, {NAME => 'y', VERSIONS => '1',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'} RootDir = hdfs://localhost:43859/
> user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-d935316c9241/.tmp Table name ==
> hbase_shell_tests_table
> ^[[38;5;226mE^[[0m
> ===
> Error: ^[[48;5;16;38;5;226;1mtest_Get_simple_status(Hbase::StatusTest)^[[0m:
> Java::JavaIo::InterruptedIOException: Interrupt while waiting on Operation:
> CREATE, Table Name: default:hbase_shell_tests_table, procId: 871
> 2018-10-08 02:49:09,361 INFO [Block report processor]
> blockmanagement.BlockManager(2645): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:41338 is added to
> blk_1073742193_1369{UCState=COMMITTED, truncateBlock=null,
> primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-ecc89143-e0a5-4a1c-b552-120be2561334:NORMAL:127.0.0.1:
>41338|RBW]]} size 58
> > TEST TIMED OUT. PRINTING THREAD DUMP. <
> {code}
> We can see that the procedure #871 wasn't stuck - the timeout cut in and
> stopped the test.
> We should separate the current test into two (or more) test files (with
> corresponding .rb) so that the execution time consistently would not exceed
> limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21279) Split TestAdminShell into several tests
[
https://issues.apache.org/jira/browse/HBASE-21279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21279:
-
Status: Patch Available (was: Open)
> Split TestAdminShell into several tests
> ---
>
> Key: HBASE-21279
> URL: https://issues.apache.org/jira/browse/HBASE-21279
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21279.v01.patch, testAdminShell-output.tar.gz
>
>
> In the flaky test board, TestAdminShell often timed out
> (https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html).
> I ran the test on Linux with SSD and reproduced the timeout (see attached
> test output).
> {code}
> 2018-10-08 02:36:09,146 DEBUG [main] hbase.HBaseTestingUtility(351): Setting
> hbase.rootdir to
> /mnt/disk2/a/2-hbase/hbase-shell/target/test-data/a103d8e4-695c-a5a9-6690-1ef2580050f9
> ...
> 2018-10-08 02:49:09,093 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=7]
> master.MasterRpcServices(1171): Checking to see if procedure is done pid=871
> Took 0.7262 seconds2018-10-08 02:49:09,324 DEBUG [PEWorker-1]
> util.FSTableDescriptors(684): Wrote into
> hdfs://localhost:43859/user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-
> d935316c9241/.tmp/data/default/hbase_shell_tests_table/.tabledesc/.tableinfo.01
> 2018-10-08 02:49:09,328 INFO
> [RegionOpenAndInitThread-hbase_shell_tests_table-1]
> regionserver.HRegion(7004): creating HRegion hbase_shell_tests_table HTD ==
> 'hbase_shell_tests_table', {NAME => 'x', VERSIONS => '5',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE =>
> 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS =>
> '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'}, {NAME => 'y', VERSIONS => '1',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'} RootDir = hdfs://localhost:43859/
> user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-d935316c9241/.tmp Table name ==
> hbase_shell_tests_table
> ^[[38;5;226mE^[[0m
> ===
> Error: ^[[48;5;16;38;5;226;1mtest_Get_simple_status(Hbase::StatusTest)^[[0m:
> Java::JavaIo::InterruptedIOException: Interrupt while waiting on Operation:
> CREATE, Table Name: default:hbase_shell_tests_table, procId: 871
> 2018-10-08 02:49:09,361 INFO [Block report processor]
> blockmanagement.BlockManager(2645): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:41338 is added to
> blk_1073742193_1369{UCState=COMMITTED, truncateBlock=null,
> primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-ecc89143-e0a5-4a1c-b552-120be2561334:NORMAL:127.0.0.1:
>41338|RBW]]} size 58
> > TEST TIMED OUT. PRINTING THREAD DUMP. <
> {code}
> We can see that the procedure #871 wasn't stuck - the timeout cut in and
> stopped the test.
> We should separate the current test into two (or more) test files (with
> corresponding .rb) so that the execution time consistently would not exceed
> limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21279) Split TestAdminShell into several tests
[
https://issues.apache.org/jira/browse/HBASE-21279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21279:
-
Attachment: HBASE-21279.v01.patch
> Split TestAdminShell into several tests
> ---
>
> Key: HBASE-21279
> URL: https://issues.apache.org/jira/browse/HBASE-21279
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21279.v01.patch, testAdminShell-output.tar.gz
>
>
> In the flaky test board, TestAdminShell often timed out
> (https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html).
> I ran the test on Linux with SSD and reproduced the timeout (see attached
> test output).
> {code}
> 2018-10-08 02:36:09,146 DEBUG [main] hbase.HBaseTestingUtility(351): Setting
> hbase.rootdir to
> /mnt/disk2/a/2-hbase/hbase-shell/target/test-data/a103d8e4-695c-a5a9-6690-1ef2580050f9
> ...
> 2018-10-08 02:49:09,093 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=7]
> master.MasterRpcServices(1171): Checking to see if procedure is done pid=871
> Took 0.7262 seconds2018-10-08 02:49:09,324 DEBUG [PEWorker-1]
> util.FSTableDescriptors(684): Wrote into
> hdfs://localhost:43859/user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-
> d935316c9241/.tmp/data/default/hbase_shell_tests_table/.tabledesc/.tableinfo.01
> 2018-10-08 02:49:09,328 INFO
> [RegionOpenAndInitThread-hbase_shell_tests_table-1]
> regionserver.HRegion(7004): creating HRegion hbase_shell_tests_table HTD ==
> 'hbase_shell_tests_table', {NAME => 'x', VERSIONS => '5',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE =>
> 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS =>
> '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'}, {NAME => 'y', VERSIONS => '1',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'} RootDir = hdfs://localhost:43859/
> user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-d935316c9241/.tmp Table name ==
> hbase_shell_tests_table
> ^[[38;5;226mE^[[0m
> ===
> Error: ^[[48;5;16;38;5;226;1mtest_Get_simple_status(Hbase::StatusTest)^[[0m:
> Java::JavaIo::InterruptedIOException: Interrupt while waiting on Operation:
> CREATE, Table Name: default:hbase_shell_tests_table, procId: 871
> 2018-10-08 02:49:09,361 INFO [Block report processor]
> blockmanagement.BlockManager(2645): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:41338 is added to
> blk_1073742193_1369{UCState=COMMITTED, truncateBlock=null,
> primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-ecc89143-e0a5-4a1c-b552-120be2561334:NORMAL:127.0.0.1:
>41338|RBW]]} size 58
> > TEST TIMED OUT. PRINTING THREAD DUMP. <
> {code}
> We can see that the procedure #871 wasn't stuck - the timeout cut in and
> stopped the test.
> We should separate the current test into two (or more) test files (with
> corresponding .rb) so that the execution time consistently would not exceed
> limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21279) Split TestAdminShell into several tests
[
https://issues.apache.org/jira/browse/HBASE-21279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21279:
-
Attachment: HBASE-21279.v02.patch
> Split TestAdminShell into several tests
> ---
>
> Key: HBASE-21279
> URL: https://issues.apache.org/jira/browse/HBASE-21279
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21279.v01.patch, HBASE-21279.v02.patch,
> testAdminShell-output.tar.gz
>
>
> In the flaky test board, TestAdminShell often timed out
> (https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html).
> I ran the test on Linux with SSD and reproduced the timeout (see attached
> test output).
> {code}
> 2018-10-08 02:36:09,146 DEBUG [main] hbase.HBaseTestingUtility(351): Setting
> hbase.rootdir to
> /mnt/disk2/a/2-hbase/hbase-shell/target/test-data/a103d8e4-695c-a5a9-6690-1ef2580050f9
> ...
> 2018-10-08 02:49:09,093 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=7]
> master.MasterRpcServices(1171): Checking to see if procedure is done pid=871
> Took 0.7262 seconds2018-10-08 02:49:09,324 DEBUG [PEWorker-1]
> util.FSTableDescriptors(684): Wrote into
> hdfs://localhost:43859/user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-
> d935316c9241/.tmp/data/default/hbase_shell_tests_table/.tabledesc/.tableinfo.01
> 2018-10-08 02:49:09,328 INFO
> [RegionOpenAndInitThread-hbase_shell_tests_table-1]
> regionserver.HRegion(7004): creating HRegion hbase_shell_tests_table HTD ==
> 'hbase_shell_tests_table', {NAME => 'x', VERSIONS => '5',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE =>
> 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS =>
> '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'}, {NAME => 'y', VERSIONS => '1',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'} RootDir = hdfs://localhost:43859/
> user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-d935316c9241/.tmp Table name ==
> hbase_shell_tests_table
> ^[[38;5;226mE^[[0m
> ===
> Error: ^[[48;5;16;38;5;226;1mtest_Get_simple_status(Hbase::StatusTest)^[[0m:
> Java::JavaIo::InterruptedIOException: Interrupt while waiting on Operation:
> CREATE, Table Name: default:hbase_shell_tests_table, procId: 871
> 2018-10-08 02:49:09,361 INFO [Block report processor]
> blockmanagement.BlockManager(2645): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:41338 is added to
> blk_1073742193_1369{UCState=COMMITTED, truncateBlock=null,
> primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-ecc89143-e0a5-4a1c-b552-120be2561334:NORMAL:127.0.0.1:
>41338|RBW]]} size 58
> > TEST TIMED OUT. PRINTING THREAD DUMP. <
> {code}
> We can see that the procedure #871 wasn't stuck - the timeout cut in and
> stopped the test.
> We should separate the current test into two (or more) test files (with
> corresponding .rb) so that the execution time consistently would not exceed
> limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21279) Split TestAdminShell into several tests
[
https://issues.apache.org/jira/browse/HBASE-21279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653774#comment-16653774
]
Artem Ervits commented on HBASE-21279:
--
[~yuzhih...@gmail.com] split the admin_test.rb based delete_test_table calls as
create_test_table is not evenly distributed across the script. With .02 patch
the following can be observed:
{code:java}
# master
time mvn test -Dtest=TestAdminShell -rf :hbase-shell
[INFO] Total time: 09:15 min
[INFO] Finished at: 2018-10-17T11:51:31-04:00
[INFO]
real9m16.152s
user5m3.471s
sys 0m37.542s
# with patch
time mvn test -Dtest=TestAdminShell -rf :hbase-shell
[INFO] Total time: 05:10 min
[INFO] Finished at: 2018-10-17T11:58:54-04:00
[INFO]
real5m11.896s
user4m42.509s
sys 0m28.630s
time mvn test -Dtest=TestAdminShell2 -rf :hbase-shell
[INFO] Total time: 03:29 min
[INFO] Finished at: 2018-10-17T12:03:42-04:00
[INFO]
real3m30.390s
user3m58.885s
sys 0m21.751s{code}
> Split TestAdminShell into several tests
> ---
>
> Key: HBASE-21279
> URL: https://issues.apache.org/jira/browse/HBASE-21279
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21279.v01.patch, HBASE-21279.v02.patch,
> testAdminShell-output.tar.gz
>
>
> In the flaky test board, TestAdminShell often timed out
> (https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html).
> I ran the test on Linux with SSD and reproduced the timeout (see attached
> test output).
> {code}
> 2018-10-08 02:36:09,146 DEBUG [main] hbase.HBaseTestingUtility(351): Setting
> hbase.rootdir to
> /mnt/disk2/a/2-hbase/hbase-shell/target/test-data/a103d8e4-695c-a5a9-6690-1ef2580050f9
> ...
> 2018-10-08 02:49:09,093 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=7]
> master.MasterRpcServices(1171): Checking to see if procedure is done pid=871
> Took 0.7262 seconds2018-10-08 02:49:09,324 DEBUG [PEWorker-1]
> util.FSTableDescriptors(684): Wrote into
> hdfs://localhost:43859/user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-
> d935316c9241/.tmp/data/default/hbase_shell_tests_table/.tabledesc/.tableinfo.01
> 2018-10-08 02:49:09,328 INFO
> [RegionOpenAndInitThread-hbase_shell_tests_table-1]
> regionserver.HRegion(7004): creating HRegion hbase_shell_tests_table HTD ==
> 'hbase_shell_tests_table', {NAME => 'x', VERSIONS => '5',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE =>
> 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS =>
> '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'}, {NAME => 'y', VERSIONS => '1',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'} RootDir = hdfs://localhost:43859/
> user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-d935316c9241/.tmp Table name ==
> hbase_shell_tests_table
> ^[[38;5;226mE^[[0m
> ===
> Error: ^[[48;5;16;38;5;226;1mtest_Get_simple_status(Hbase::StatusTest)^[[0m:
> Java::JavaIo::InterruptedIOException: Interrupt while waiting on Operation:
> CREATE, Table Name: default:hbase_shell_tests_table, procId: 871
> 2018-10-08 02:49:09,361 INFO [Block report processor]
> blockmanagement.BlockManager(2645): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:41338 is added to
> blk_1073742193_1369{UCState=COMMITTED, truncateBlock=null,
> primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-ecc89143-e0a5-4a1c-b552-120be2561334:NORMAL:127.0.0.1:
>41338|RBW]]} size 58
> > TEST TIMED OUT. PRINTING THREAD DUMP. <
> {code}
> We can see that the procedure #871 wasn't stuck - the timeout cut in and
> stopped the test.
> We should separate the current test into two (or more) test files (with
> corresponding
[jira] [Updated] (HBASE-21279) Split TestAdminShell into several tests
[
https://issues.apache.org/jira/browse/HBASE-21279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21279:
-
Attachment: HBASE-21279.v03.patch
> Split TestAdminShell into several tests
> ---
>
> Key: HBASE-21279
> URL: https://issues.apache.org/jira/browse/HBASE-21279
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21279.v01.patch, HBASE-21279.v02.patch,
> HBASE-21279.v03.patch, testAdminShell-output.tar.gz
>
>
> In the flaky test board, TestAdminShell often timed out
> (https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html).
> I ran the test on Linux with SSD and reproduced the timeout (see attached
> test output).
> {code}
> 2018-10-08 02:36:09,146 DEBUG [main] hbase.HBaseTestingUtility(351): Setting
> hbase.rootdir to
> /mnt/disk2/a/2-hbase/hbase-shell/target/test-data/a103d8e4-695c-a5a9-6690-1ef2580050f9
> ...
> 2018-10-08 02:49:09,093 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=7]
> master.MasterRpcServices(1171): Checking to see if procedure is done pid=871
> Took 0.7262 seconds2018-10-08 02:49:09,324 DEBUG [PEWorker-1]
> util.FSTableDescriptors(684): Wrote into
> hdfs://localhost:43859/user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-
> d935316c9241/.tmp/data/default/hbase_shell_tests_table/.tabledesc/.tableinfo.01
> 2018-10-08 02:49:09,328 INFO
> [RegionOpenAndInitThread-hbase_shell_tests_table-1]
> regionserver.HRegion(7004): creating HRegion hbase_shell_tests_table HTD ==
> 'hbase_shell_tests_table', {NAME => 'x', VERSIONS => '5',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE =>
> 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS =>
> '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'}, {NAME => 'y', VERSIONS => '1',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'} RootDir = hdfs://localhost:43859/
> user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-d935316c9241/.tmp Table name ==
> hbase_shell_tests_table
> ^[[38;5;226mE^[[0m
> ===
> Error: ^[[48;5;16;38;5;226;1mtest_Get_simple_status(Hbase::StatusTest)^[[0m:
> Java::JavaIo::InterruptedIOException: Interrupt while waiting on Operation:
> CREATE, Table Name: default:hbase_shell_tests_table, procId: 871
> 2018-10-08 02:49:09,361 INFO [Block report processor]
> blockmanagement.BlockManager(2645): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:41338 is added to
> blk_1073742193_1369{UCState=COMMITTED, truncateBlock=null,
> primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-ecc89143-e0a5-4a1c-b552-120be2561334:NORMAL:127.0.0.1:
>41338|RBW]]} size 58
> > TEST TIMED OUT. PRINTING THREAD DUMP. <
> {code}
> We can see that the procedure #871 wasn't stuck - the timeout cut in and
> stopped the test.
> We should separate the current test into two (or more) test files (with
> corresponding .rb) so that the execution time consistently would not exceed
> limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21279) Split TestAdminShell into several tests
[
https://issues.apache.org/jira/browse/HBASE-21279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653801#comment-16653801
]
Artem Ervits commented on HBASE-21279:
--
[~yuzhih...@gmail.com] certainly, please see patch .03
> Split TestAdminShell into several tests
> ---
>
> Key: HBASE-21279
> URL: https://issues.apache.org/jira/browse/HBASE-21279
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21279.v01.patch, HBASE-21279.v02.patch,
> HBASE-21279.v03.patch, testAdminShell-output.tar.gz
>
>
> In the flaky test board, TestAdminShell often timed out
> (https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html).
> I ran the test on Linux with SSD and reproduced the timeout (see attached
> test output).
> {code}
> 2018-10-08 02:36:09,146 DEBUG [main] hbase.HBaseTestingUtility(351): Setting
> hbase.rootdir to
> /mnt/disk2/a/2-hbase/hbase-shell/target/test-data/a103d8e4-695c-a5a9-6690-1ef2580050f9
> ...
> 2018-10-08 02:49:09,093 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=7]
> master.MasterRpcServices(1171): Checking to see if procedure is done pid=871
> Took 0.7262 seconds2018-10-08 02:49:09,324 DEBUG [PEWorker-1]
> util.FSTableDescriptors(684): Wrote into
> hdfs://localhost:43859/user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-
> d935316c9241/.tmp/data/default/hbase_shell_tests_table/.tabledesc/.tableinfo.01
> 2018-10-08 02:49:09,328 INFO
> [RegionOpenAndInitThread-hbase_shell_tests_table-1]
> regionserver.HRegion(7004): creating HRegion hbase_shell_tests_table HTD ==
> 'hbase_shell_tests_table', {NAME => 'x', VERSIONS => '5',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE =>
> 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS =>
> '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'}, {NAME => 'y', VERSIONS => '1',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'} RootDir = hdfs://localhost:43859/
> user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-d935316c9241/.tmp Table name ==
> hbase_shell_tests_table
> ^[[38;5;226mE^[[0m
> ===
> Error: ^[[48;5;16;38;5;226;1mtest_Get_simple_status(Hbase::StatusTest)^[[0m:
> Java::JavaIo::InterruptedIOException: Interrupt while waiting on Operation:
> CREATE, Table Name: default:hbase_shell_tests_table, procId: 871
> 2018-10-08 02:49:09,361 INFO [Block report processor]
> blockmanagement.BlockManager(2645): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:41338 is added to
> blk_1073742193_1369{UCState=COMMITTED, truncateBlock=null,
> primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-ecc89143-e0a5-4a1c-b552-120be2561334:NORMAL:127.0.0.1:
>41338|RBW]]} size 58
> > TEST TIMED OUT. PRINTING THREAD DUMP. <
> {code}
> We can see that the procedure #871 wasn't stuck - the timeout cut in and
> stopped the test.
> We should separate the current test into two (or more) test files (with
> corresponding .rb) so that the execution time consistently would not exceed
> limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21198) Exclude dependency on net.minidev:json-smart
[
https://issues.apache.org/jira/browse/HBASE-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653980#comment-16653980
]
Artem Ervits commented on HBASE-21198:
--
[~stack] branches running on hadoop 3 so 2.x and 3.x.
> Exclude dependency on net.minidev:json-smart
>
>
> Key: HBASE-21198
> URL: https://issues.apache.org/jira/browse/HBASE-21198
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21198.v01.patch, HBASE-21198.v01.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/14414/artifact/patchprocess/patch-javac-3.0.0.txt
> :
> {code}
> [ERROR] Failed to execute goal on project hbase-common: Could not resolve
> dependencies for project org.apache.hbase:hbase-common:jar:3.0.0-SNAPSHOT:
> Failed to collect dependencies at org.apache.hadoop:hadoop-common:jar:3.0.0
> -> org.apache.hadoop:hadoop-auth:jar:3.0.0 ->
> com.nimbusds:nimbus-jose-jwt:jar:4.41.1 ->
> net.minidev:json-smart:jar:2.3-SNAPSHOT: Failed to read artifact descriptor
> for net.minidev:json-smart:jar:2.3-SNAPSHOT: Could not transfer artifact
> net.minidev:json-smart:pom:2.3-SNAPSHOT from/to dynamodb-local-oregon
> (https://s3-us-west-2.amazonaws.com/dynamodb-local/release): Access denied
> to:
> https://s3-us-west-2.amazonaws.com/dynamodb-local/release/net/minidev/json-smart/2.3-SNAPSHOT/json-smart-2.3-SNAPSHOT.pom
> , ReasonPhrase:Forbidden. -> [Help 1]
> {code}
> We should exclude dependency on net.minidev:json-smart
> hbase-common/bin/pom.xml has done so.
> The other pom.xml should do the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21279) Split TestAdminShell into several tests
[
https://issues.apache.org/jira/browse/HBASE-21279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21279:
-
Attachment: HBASE-21279.v04.patch
> Split TestAdminShell into several tests
> ---
>
> Key: HBASE-21279
> URL: https://issues.apache.org/jira/browse/HBASE-21279
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21279.v01.patch, HBASE-21279.v02.patch,
> HBASE-21279.v03.patch, HBASE-21279.v04.patch, testAdminShell-output.tar.gz
>
>
> In the flaky test board, TestAdminShell often timed out
> (https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html).
> I ran the test on Linux with SSD and reproduced the timeout (see attached
> test output).
> {code}
> 2018-10-08 02:36:09,146 DEBUG [main] hbase.HBaseTestingUtility(351): Setting
> hbase.rootdir to
> /mnt/disk2/a/2-hbase/hbase-shell/target/test-data/a103d8e4-695c-a5a9-6690-1ef2580050f9
> ...
> 2018-10-08 02:49:09,093 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=7]
> master.MasterRpcServices(1171): Checking to see if procedure is done pid=871
> Took 0.7262 seconds2018-10-08 02:49:09,324 DEBUG [PEWorker-1]
> util.FSTableDescriptors(684): Wrote into
> hdfs://localhost:43859/user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-
> d935316c9241/.tmp/data/default/hbase_shell_tests_table/.tabledesc/.tableinfo.01
> 2018-10-08 02:49:09,328 INFO
> [RegionOpenAndInitThread-hbase_shell_tests_table-1]
> regionserver.HRegion(7004): creating HRegion hbase_shell_tests_table HTD ==
> 'hbase_shell_tests_table', {NAME => 'x', VERSIONS => '5',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE =>
> 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS =>
> '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'}, {NAME => 'y', VERSIONS => '1',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'} RootDir = hdfs://localhost:43859/
> user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-d935316c9241/.tmp Table name ==
> hbase_shell_tests_table
> ^[[38;5;226mE^[[0m
> ===
> Error: ^[[48;5;16;38;5;226;1mtest_Get_simple_status(Hbase::StatusTest)^[[0m:
> Java::JavaIo::InterruptedIOException: Interrupt while waiting on Operation:
> CREATE, Table Name: default:hbase_shell_tests_table, procId: 871
> 2018-10-08 02:49:09,361 INFO [Block report processor]
> blockmanagement.BlockManager(2645): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:41338 is added to
> blk_1073742193_1369{UCState=COMMITTED, truncateBlock=null,
> primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-ecc89143-e0a5-4a1c-b552-120be2561334:NORMAL:127.0.0.1:
>41338|RBW]]} size 58
> > TEST TIMED OUT. PRINTING THREAD DUMP. <
> {code}
> We can see that the procedure #871 wasn't stuck - the timeout cut in and
> stopped the test.
> We should separate the current test into two (or more) test files (with
> corresponding .rb) so that the execution time consistently would not exceed
> limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21279) Split TestAdminShell into several tests
[
https://issues.apache.org/jira/browse/HBASE-21279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654087#comment-16654087
]
Artem Ervits commented on HBASE-21279:
--
[~yuzhih...@gmail.com] addressed all your comments, duration is more or less
split even now
{code:java}
[INFO] Running org.apache.hadoop.hbase.client.TestAdminShell
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 216.568
s - in org.apache.hadoop.hbase.client.TestAdminShell
[INFO] Running org.apache.hadoop.hbase.client.TestAdminShell2
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 162.588
s - in org.apache.hadoop.hbase.client.TestAdminShell2{code}
The table in admin2_test.rb is named after the script as well
{code:java}
create 'hbase_shell_admin2_test_table'{code}
please review
> Split TestAdminShell into several tests
> ---
>
> Key: HBASE-21279
> URL: https://issues.apache.org/jira/browse/HBASE-21279
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21279.v01.patch, HBASE-21279.v02.patch,
> HBASE-21279.v03.patch, HBASE-21279.v04.patch, testAdminShell-output.tar.gz
>
>
> In the flaky test board, TestAdminShell often timed out
> (https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html).
> I ran the test on Linux with SSD and reproduced the timeout (see attached
> test output).
> {code}
> 2018-10-08 02:36:09,146 DEBUG [main] hbase.HBaseTestingUtility(351): Setting
> hbase.rootdir to
> /mnt/disk2/a/2-hbase/hbase-shell/target/test-data/a103d8e4-695c-a5a9-6690-1ef2580050f9
> ...
> 2018-10-08 02:49:09,093 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=7]
> master.MasterRpcServices(1171): Checking to see if procedure is done pid=871
> Took 0.7262 seconds2018-10-08 02:49:09,324 DEBUG [PEWorker-1]
> util.FSTableDescriptors(684): Wrote into
> hdfs://localhost:43859/user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-
> d935316c9241/.tmp/data/default/hbase_shell_tests_table/.tabledesc/.tableinfo.01
> 2018-10-08 02:49:09,328 INFO
> [RegionOpenAndInitThread-hbase_shell_tests_table-1]
> regionserver.HRegion(7004): creating HRegion hbase_shell_tests_table HTD ==
> 'hbase_shell_tests_table', {NAME => 'x', VERSIONS => '5',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE =>
> 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS =>
> '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'}, {NAME => 'y', VERSIONS => '1',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'} RootDir = hdfs://localhost:43859/
> user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-d935316c9241/.tmp Table name ==
> hbase_shell_tests_table
> ^[[38;5;226mE^[[0m
> ===
> Error: ^[[48;5;16;38;5;226;1mtest_Get_simple_status(Hbase::StatusTest)^[[0m:
> Java::JavaIo::InterruptedIOException: Interrupt while waiting on Operation:
> CREATE, Table Name: default:hbase_shell_tests_table, procId: 871
> 2018-10-08 02:49:09,361 INFO [Block report processor]
> blockmanagement.BlockManager(2645): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:41338 is added to
> blk_1073742193_1369{UCState=COMMITTED, truncateBlock=null,
> primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-ecc89143-e0a5-4a1c-b552-120be2561334:NORMAL:127.0.0.1:
>41338|RBW]]} size 58
> > TEST TIMED OUT. PRINTING THREAD DUMP. <
> {code}
> We can see that the procedure #871 wasn't stuck - the timeout cut in and
> stopped the test.
> We should separate the current test into two (or more) test files (with
> corresponding .rb) so that the execution time consistently would not exceed
> limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21279) Split TestAdminShell into several tests
[
https://issues.apache.org/jira/browse/HBASE-21279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21279:
-
Attachment: HBASE-21279.v05.patch
> Split TestAdminShell into several tests
> ---
>
> Key: HBASE-21279
> URL: https://issues.apache.org/jira/browse/HBASE-21279
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21279.v01.patch, HBASE-21279.v02.patch,
> HBASE-21279.v03.patch, HBASE-21279.v04.patch, HBASE-21279.v05.patch,
> testAdminShell-output.tar.gz
>
>
> In the flaky test board, TestAdminShell often timed out
> (https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html).
> I ran the test on Linux with SSD and reproduced the timeout (see attached
> test output).
> {code}
> 2018-10-08 02:36:09,146 DEBUG [main] hbase.HBaseTestingUtility(351): Setting
> hbase.rootdir to
> /mnt/disk2/a/2-hbase/hbase-shell/target/test-data/a103d8e4-695c-a5a9-6690-1ef2580050f9
> ...
> 2018-10-08 02:49:09,093 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=7]
> master.MasterRpcServices(1171): Checking to see if procedure is done pid=871
> Took 0.7262 seconds2018-10-08 02:49:09,324 DEBUG [PEWorker-1]
> util.FSTableDescriptors(684): Wrote into
> hdfs://localhost:43859/user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-
> d935316c9241/.tmp/data/default/hbase_shell_tests_table/.tabledesc/.tableinfo.01
> 2018-10-08 02:49:09,328 INFO
> [RegionOpenAndInitThread-hbase_shell_tests_table-1]
> regionserver.HRegion(7004): creating HRegion hbase_shell_tests_table HTD ==
> 'hbase_shell_tests_table', {NAME => 'x', VERSIONS => '5',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE =>
> 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS =>
> '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'}, {NAME => 'y', VERSIONS => '1',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'} RootDir = hdfs://localhost:43859/
> user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-d935316c9241/.tmp Table name ==
> hbase_shell_tests_table
> ^[[38;5;226mE^[[0m
> ===
> Error: ^[[48;5;16;38;5;226;1mtest_Get_simple_status(Hbase::StatusTest)^[[0m:
> Java::JavaIo::InterruptedIOException: Interrupt while waiting on Operation:
> CREATE, Table Name: default:hbase_shell_tests_table, procId: 871
> 2018-10-08 02:49:09,361 INFO [Block report processor]
> blockmanagement.BlockManager(2645): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:41338 is added to
> blk_1073742193_1369{UCState=COMMITTED, truncateBlock=null,
> primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-ecc89143-e0a5-4a1c-b552-120be2561334:NORMAL:127.0.0.1:
>41338|RBW]]} size 58
> > TEST TIMED OUT. PRINTING THREAD DUMP. <
> {code}
> We can see that the procedure #871 wasn't stuck - the timeout cut in and
> stopped the test.
> We should separate the current test into two (or more) test files (with
> corresponding .rb) so that the execution time consistently would not exceed
> limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21279) Split TestAdminShell into several tests
[
https://issues.apache.org/jira/browse/HBASE-21279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654191#comment-16654191
]
Artem Ervits commented on HBASE-21279:
--
v. 05 patch removes a tab character
> Split TestAdminShell into several tests
> ---
>
> Key: HBASE-21279
> URL: https://issues.apache.org/jira/browse/HBASE-21279
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21279.v01.patch, HBASE-21279.v02.patch,
> HBASE-21279.v03.patch, HBASE-21279.v04.patch, HBASE-21279.v05.patch,
> testAdminShell-output.tar.gz
>
>
> In the flaky test board, TestAdminShell often timed out
> (https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html).
> I ran the test on Linux with SSD and reproduced the timeout (see attached
> test output).
> {code}
> 2018-10-08 02:36:09,146 DEBUG [main] hbase.HBaseTestingUtility(351): Setting
> hbase.rootdir to
> /mnt/disk2/a/2-hbase/hbase-shell/target/test-data/a103d8e4-695c-a5a9-6690-1ef2580050f9
> ...
> 2018-10-08 02:49:09,093 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=7]
> master.MasterRpcServices(1171): Checking to see if procedure is done pid=871
> Took 0.7262 seconds2018-10-08 02:49:09,324 DEBUG [PEWorker-1]
> util.FSTableDescriptors(684): Wrote into
> hdfs://localhost:43859/user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-
> d935316c9241/.tmp/data/default/hbase_shell_tests_table/.tabledesc/.tableinfo.01
> 2018-10-08 02:49:09,328 INFO
> [RegionOpenAndInitThread-hbase_shell_tests_table-1]
> regionserver.HRegion(7004): creating HRegion hbase_shell_tests_table HTD ==
> 'hbase_shell_tests_table', {NAME => 'x', VERSIONS => '5',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE =>
> 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS =>
> '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'}, {NAME => 'y', VERSIONS => '1',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'} RootDir = hdfs://localhost:43859/
> user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-d935316c9241/.tmp Table name ==
> hbase_shell_tests_table
> ^[[38;5;226mE^[[0m
> ===
> Error: ^[[48;5;16;38;5;226;1mtest_Get_simple_status(Hbase::StatusTest)^[[0m:
> Java::JavaIo::InterruptedIOException: Interrupt while waiting on Operation:
> CREATE, Table Name: default:hbase_shell_tests_table, procId: 871
> 2018-10-08 02:49:09,361 INFO [Block report processor]
> blockmanagement.BlockManager(2645): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:41338 is added to
> blk_1073742193_1369{UCState=COMMITTED, truncateBlock=null,
> primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-ecc89143-e0a5-4a1c-b552-120be2561334:NORMAL:127.0.0.1:
>41338|RBW]]} size 58
> > TEST TIMED OUT. PRINTING THREAD DUMP. <
> {code}
> We can see that the procedure #871 wasn't stuck - the timeout cut in and
> stopped the test.
> We should separate the current test into two (or more) test files (with
> corresponding .rb) so that the execution time consistently would not exceed
> limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21279) Split TestAdminShell into several tests
[
https://issues.apache.org/jira/browse/HBASE-21279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654191#comment-16654191
]
Artem Ervits edited comment on HBASE-21279 at 10/17/18 9:14 PM:
v. 05 patch removes a tab character that was inherited from earlier script
was (Author: dbist13):
v. 05 patch removes a tab character
> Split TestAdminShell into several tests
> ---
>
> Key: HBASE-21279
> URL: https://issues.apache.org/jira/browse/HBASE-21279
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21279.v01.patch, HBASE-21279.v02.patch,
> HBASE-21279.v03.patch, HBASE-21279.v04.patch, HBASE-21279.v05.patch,
> testAdminShell-output.tar.gz
>
>
> In the flaky test board, TestAdminShell often timed out
> (https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html).
> I ran the test on Linux with SSD and reproduced the timeout (see attached
> test output).
> {code}
> 2018-10-08 02:36:09,146 DEBUG [main] hbase.HBaseTestingUtility(351): Setting
> hbase.rootdir to
> /mnt/disk2/a/2-hbase/hbase-shell/target/test-data/a103d8e4-695c-a5a9-6690-1ef2580050f9
> ...
> 2018-10-08 02:49:09,093 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=7]
> master.MasterRpcServices(1171): Checking to see if procedure is done pid=871
> Took 0.7262 seconds2018-10-08 02:49:09,324 DEBUG [PEWorker-1]
> util.FSTableDescriptors(684): Wrote into
> hdfs://localhost:43859/user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-
> d935316c9241/.tmp/data/default/hbase_shell_tests_table/.tabledesc/.tableinfo.01
> 2018-10-08 02:49:09,328 INFO
> [RegionOpenAndInitThread-hbase_shell_tests_table-1]
> regionserver.HRegion(7004): creating HRegion hbase_shell_tests_table HTD ==
> 'hbase_shell_tests_table', {NAME => 'x', VERSIONS => '5',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE =>
> 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS =>
> '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'}, {NAME => 'y', VERSIONS => '1',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'} RootDir = hdfs://localhost:43859/
> user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-d935316c9241/.tmp Table name ==
> hbase_shell_tests_table
> ^[[38;5;226mE^[[0m
> ===
> Error: ^[[48;5;16;38;5;226;1mtest_Get_simple_status(Hbase::StatusTest)^[[0m:
> Java::JavaIo::InterruptedIOException: Interrupt while waiting on Operation:
> CREATE, Table Name: default:hbase_shell_tests_table, procId: 871
> 2018-10-08 02:49:09,361 INFO [Block report processor]
> blockmanagement.BlockManager(2645): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:41338 is added to
> blk_1073742193_1369{UCState=COMMITTED, truncateBlock=null,
> primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-ecc89143-e0a5-4a1c-b552-120be2561334:NORMAL:127.0.0.1:
>41338|RBW]]} size 58
> > TEST TIMED OUT. PRINTING THREAD DUMP. <
> {code}
> We can see that the procedure #871 wasn't stuck - the timeout cut in and
> stopped the test.
> We should separate the current test into two (or more) test files (with
> corresponding .rb) so that the execution time consistently would not exceed
> limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21279) Split TestAdminShell into several tests
[
https://issues.apache.org/jira/browse/HBASE-21279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654191#comment-16654191
]
Artem Ervits edited comment on HBASE-21279 at 10/17/18 9:16 PM:
v. 05 patch removes a tab character that was inherited from earlier script
https://github.com/apache/hbase/blob/master/hbase-shell/src/test/ruby/hbase/admin_test.rb#L690
was (Author: dbist13):
v. 05 patch removes a tab character that was inherited from earlier script
> Split TestAdminShell into several tests
> ---
>
> Key: HBASE-21279
> URL: https://issues.apache.org/jira/browse/HBASE-21279
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-21279.v01.patch, HBASE-21279.v02.patch,
> HBASE-21279.v03.patch, HBASE-21279.v04.patch, HBASE-21279.v05.patch,
> testAdminShell-output.tar.gz
>
>
> In the flaky test board, TestAdminShell often timed out
> (https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html).
> I ran the test on Linux with SSD and reproduced the timeout (see attached
> test output).
> {code}
> 2018-10-08 02:36:09,146 DEBUG [main] hbase.HBaseTestingUtility(351): Setting
> hbase.rootdir to
> /mnt/disk2/a/2-hbase/hbase-shell/target/test-data/a103d8e4-695c-a5a9-6690-1ef2580050f9
> ...
> 2018-10-08 02:49:09,093 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=7]
> master.MasterRpcServices(1171): Checking to see if procedure is done pid=871
> Took 0.7262 seconds2018-10-08 02:49:09,324 DEBUG [PEWorker-1]
> util.FSTableDescriptors(684): Wrote into
> hdfs://localhost:43859/user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-
> d935316c9241/.tmp/data/default/hbase_shell_tests_table/.tabledesc/.tableinfo.01
> 2018-10-08 02:49:09,328 INFO
> [RegionOpenAndInitThread-hbase_shell_tests_table-1]
> regionserver.HRegion(7004): creating HRegion hbase_shell_tests_table HTD ==
> 'hbase_shell_tests_table', {NAME => 'x', VERSIONS => '5',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE =>
> 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS =>
> '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'}, {NAME => 'y', VERSIONS => '1',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'} RootDir = hdfs://localhost:43859/
> user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-d935316c9241/.tmp Table name ==
> hbase_shell_tests_table
> ^[[38;5;226mE^[[0m
> ===
> Error: ^[[48;5;16;38;5;226;1mtest_Get_simple_status(Hbase::StatusTest)^[[0m:
> Java::JavaIo::InterruptedIOException: Interrupt while waiting on Operation:
> CREATE, Table Name: default:hbase_shell_tests_table, procId: 871
> 2018-10-08 02:49:09,361 INFO [Block report processor]
> blockmanagement.BlockManager(2645): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:41338 is added to
> blk_1073742193_1369{UCState=COMMITTED, truncateBlock=null,
> primaryNodeIndex=-1,
> replicas=[ReplicaUC[[DISK]DS-ecc89143-e0a5-4a1c-b552-120be2561334:NORMAL:127.0.0.1:
>41338|RBW]]} size 58
> > TEST TIMED OUT. PRINTING THREAD DUMP. <
> {code}
> We can see that the procedure #871 wasn't stuck - the timeout cut in and
> stopped the test.
> We should separate the current test into two (or more) test files (with
> corresponding .rb) so that the execution time consistently would not exceed
> limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21339) Address ruby lint warnings in shell module
Artem Ervits created HBASE-21339:
Summary: Address ruby lint warnings in shell module
Key: HBASE-21339
URL: https://issues.apache.org/jira/browse/HBASE-21339
Project: HBase
Issue Type: Task
Components: shell
Reporter: Artem Ervits
Assignee: Artem Ervits
While working on HBASE-21279 the built reported warnings in ruby-lint
[https://builds.apache.org/job/PreCommit-HBASE-Build/14735/artifact/patchprocess/diff-patch-ruby-lint.txt]
{code:java}
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:37:7:
undefined method drop_test_table
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:38:7:
undefined method create_test_table
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:42:7:
undefined method shutdown
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:45:5:
undefined method define_test
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:46:16:
undefined method capture_stdout
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:46:33:
undefined method replication_status
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:48:7:
undefined method assert
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:49:7:
undefined method assert
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:52:5:
undefined method define_test
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:53:16:
undefined method capture_stdout
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:53:33:
undefined method replication_status
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:55:7:
undefined method assert
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:58:5:
undefined method define_test
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:59:16:
undefined method capture_stdout
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:59:33:
undefined method replication_status
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:61:7:
undefined method assert
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:64:5:
undefined method define_test
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:65:16:
undefined method capture_stdout
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:65:33:
undefined method admin
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:67:7:
undefined method assert
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:70:5:
undefined method define_test
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:71:16:
undefined method capture_stdout
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:71:33:
undefined method admin
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:74:7:
undefined method assert
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:81:13:
undefined constant TestHelpers
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:84:7:
undefined method setup_hbase
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:87:7:
undefined method drop_test_table
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:88:7:
undefined method create_test_table
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:94:7:
undefined method shutdown
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:98:5:
undefined method define_test
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:99:7:
undefined method assert_raise
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:100:9:
undefined method command
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:104:5:
undefined method define_test
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:105:7:
undefined method assert_raise
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:106:9:
undefined method command
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:110:5:
undefined method define_test
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:111:7:
undefined method assert_raise
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:112:9:
undefined method command
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:116:5:
undefined method define_test
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:117:7:
undefined method drop_test_snapshot
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:118:7:
undefined method command
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:119:14:
undefined method command
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:E:120:7:
undefined method assert_equal
/testptch/hbase/hbase-shell/src/test/ruby/h
[jira] [Created] (HBASE-21340) Address rubocop warnings in shell module
Artem Ervits created HBASE-21340:
Summary: Address rubocop warnings in shell module
Key: HBASE-21340
URL: https://issues.apache.org/jira/browse/HBASE-21340
Project: HBase
Issue Type: Task
Reporter: Artem Ervits
Assignee: Artem Ervits
While working on HBASE-21279 build reported warnings with rubocop
[https://builds.apache.org/job/PreCommit-HBASE-Build/14735/artifact/patchprocess/diff-patch-rubocop.txt]
{code:java}
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:86:20: C:
Style/StringLiterals: Prefer single-quoted strings when you don't need string
interpolation or special symbols.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:97:81: C:
Metrics/LineLength: Line is too long. [84/80]
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:98:17: C:
Style/StringLiterals: Prefer single-quoted strings when you don't need string
interpolation or special symbols.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:104:17: C:
Style/StringLiterals: Prefer single-quoted strings when you don't need string
interpolation or special symbols.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:110:17: C:
Style/StringLiterals: Prefer single-quoted strings when you don't need string
interpolation or special symbols.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:116:17: C:
Style/StringLiterals: Prefer single-quoted strings when you don't need string
interpolation or special symbols.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:117:25: C:
Style/MethodCallWithoutArgsParentheses: Do not use parentheses for method calls
with no arguments.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:123:17: C:
Style/StringLiterals: Prefer single-quoted strings when you don't need string
interpolation or special symbols.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:124:25: C:
Style/MethodCallWithoutArgsParentheses: Do not use parentheses for method calls
with no arguments.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:125:61: C:
Layout/SpaceInsideHashLiteralBraces: Space inside { missing.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:125:61: C:
Style/BracesAroundHashParameters: Redundant curly braces around a hash
parameter.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:125:80: C:
Layout/SpaceInsideHashLiteralBraces: Space inside } missing.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:125:81: C:
Metrics/LineLength: Line is too long. [81/80]
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:130:17: C:
Style/StringLiterals: Prefer single-quoted strings when you don't need string
interpolation or special symbols.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:131:25: C:
Style/MethodCallWithoutArgsParentheses: Do not use parentheses for method calls
with no arguments.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:137:17: C:
Style/StringLiterals: Prefer single-quoted strings when you don't need string
interpolation or special symbols.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:138:39: C:
Style/StringLiterals: Prefer single-quoted strings when you don't need string
interpolation or special symbols.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:142:17: C:
Style/StringLiterals: Prefer single-quoted strings when you don't need string
interpolation or special symbols.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:178:17: C:
Style/StringLiterals: Prefer single-quoted strings when you don't need string
interpolation or special symbols.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:184:17: C:
Style/StringLiterals: Prefer single-quoted strings when you don't need string
interpolation or special symbols.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:190:17: C:
Style/StringLiterals: Prefer single-quoted strings when you don't need string
interpolation or special symbols.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:191:25: C:
Style/MethodCallWithoutArgsParentheses: Do not use parentheses for method calls
with no arguments.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:192:21: C:
Style/StringLiterals: Prefer single-quoted strings when you don't need string
interpolation or special symbols.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:193:20: C:
Security/Eval: The use of `eval` is a serious security risk.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:193:25: C:
Style/StringLiterals: Prefer single-quoted strings when you don't need string
interpolation or special symbols.
/testptch/hbase/hbase-shell/src/test/ruby/hbase/admin2_test.rb:193:31: C:
Style/StringLiterals: Prefer single-quoted str
[jira] [Created] (HBASE-21343) hbase-rest shouldn't import unshaded protobuf
Artem Ervits created HBASE-21343: Summary: hbase-rest shouldn't import unshaded protobuf Key: HBASE-21343 URL: https://issues.apache.org/jira/browse/HBASE-21343 Project: HBase Issue Type: Bug Reporter: Artem Ervits Assignee: Artem Ervits -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21343) hbase-rest shouldn't import unshaded protobuf
[ https://issues.apache.org/jira/browse/HBASE-21343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21343: - Attachment: HBASE-21343.v01.patch > hbase-rest shouldn't import unshaded protobuf > - > > Key: HBASE-21343 > URL: https://issues.apache.org/jira/browse/HBASE-21343 > Project: HBase > Issue Type: Bug >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-21343.v01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21343) hbase-rest shouldn't import unshaded protobuf
[ https://issues.apache.org/jira/browse/HBASE-21343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21343: - Status: Patch Available (was: Open) > hbase-rest shouldn't import unshaded protobuf > - > > Key: HBASE-21343 > URL: https://issues.apache.org/jira/browse/HBASE-21343 > Project: HBase > Issue Type: Bug >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-21343.v01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21343) hbase-rest shouldn't import unshaded protobuf
[ https://issues.apache.org/jira/browse/HBASE-21343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21343: - Status: Open (was: Patch Available) > hbase-rest shouldn't import unshaded protobuf > - > > Key: HBASE-21343 > URL: https://issues.apache.org/jira/browse/HBASE-21343 > Project: HBase > Issue Type: Bug >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-21343.v01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21343) hbase-rest shouldn't import unshaded protobuf
[ https://issues.apache.org/jira/browse/HBASE-21343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655761#comment-16655761 ] Artem Ervits commented on HBASE-21343: -- you're right [~stack] it broke my local build. I'll close the Jira as invalid. > hbase-rest shouldn't import unshaded protobuf > - > > Key: HBASE-21343 > URL: https://issues.apache.org/jira/browse/HBASE-21343 > Project: HBase > Issue Type: Bug >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-21343.v01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-21343) hbase-rest shouldn't import unshaded protobuf
[ https://issues.apache.org/jira/browse/HBASE-21343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits resolved HBASE-21343. -- Resolution: Invalid > hbase-rest shouldn't import unshaded protobuf > - > > Key: HBASE-21343 > URL: https://issues.apache.org/jira/browse/HBASE-21343 > Project: HBase > Issue Type: Bug >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-21343.v01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21194) Add TestCopyTable which exercises MOB feature
[ https://issues.apache.org/jira/browse/HBASE-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits reassigned HBASE-21194: Assignee: Artem Ervits > Add TestCopyTable which exercises MOB feature > - > > Key: HBASE-21194 > URL: https://issues.apache.org/jira/browse/HBASE-21194 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Minor > Labels: mob > > Currently TestCopyTable doesn't cover table(s) with MOB feature enabled. > We should add variant that enables MOB on the table being copied and verify > that MOB content is copied correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21194) Add TestCopyTable which exercises MOB feature
[ https://issues.apache.org/jira/browse/HBASE-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21194: - Status: Patch Available (was: Open) > Add TestCopyTable which exercises MOB feature > - > > Key: HBASE-21194 > URL: https://issues.apache.org/jira/browse/HBASE-21194 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Minor > Labels: mob > Attachments: HBASE-21194.v01.patch > > > Currently TestCopyTable doesn't cover table(s) with MOB feature enabled. > We should add variant that enables MOB on the table being copied and verify > that MOB content is copied correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21194) Add TestCopyTable which exercises MOB feature
[ https://issues.apache.org/jira/browse/HBASE-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21194: - Attachment: HBASE-21194.v01.patch > Add TestCopyTable which exercises MOB feature > - > > Key: HBASE-21194 > URL: https://issues.apache.org/jira/browse/HBASE-21194 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Minor > Labels: mob > Attachments: HBASE-21194.v01.patch > > > Currently TestCopyTable doesn't cover table(s) with MOB feature enabled. > We should add variant that enables MOB on the table being copied and verify > that MOB content is copied correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21194) Add TestCopyTable which exercises MOB feature
[ https://issues.apache.org/jira/browse/HBASE-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21194: - Attachment: HBASE-21194.v02.patch > Add TestCopyTable which exercises MOB feature > - > > Key: HBASE-21194 > URL: https://issues.apache.org/jira/browse/HBASE-21194 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Minor > Labels: mob > Attachments: HBASE-21194.v01.patch, HBASE-21194.v02.patch > > > Currently TestCopyTable doesn't cover table(s) with MOB feature enabled. > We should add variant that enables MOB on the table being copied and verify > that MOB content is copied correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21194) Add TestCopyTable which exercises MOB feature
[ https://issues.apache.org/jira/browse/HBASE-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656880#comment-16656880 ] Artem Ervits commented on HBASE-21194: -- [~yuzhih...@gmail.com] should I lower it even more? Set it to 0? > Add TestCopyTable which exercises MOB feature > - > > Key: HBASE-21194 > URL: https://issues.apache.org/jira/browse/HBASE-21194 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Minor > Labels: mob > Attachments: HBASE-21194.v01.patch, HBASE-21194.v02.patch > > > Currently TestCopyTable doesn't cover table(s) with MOB feature enabled. > We should add variant that enables MOB on the table being copied and verify > that MOB content is copied correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21194) Add TestCopyTable which exercises MOB feature
[ https://issues.apache.org/jira/browse/HBASE-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21194: - Attachment: HBASE-21194.v03.patch > Add TestCopyTable which exercises MOB feature > - > > Key: HBASE-21194 > URL: https://issues.apache.org/jira/browse/HBASE-21194 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Minor > Labels: mob > Attachments: HBASE-21194.v01.patch, HBASE-21194.v02.patch, > HBASE-21194.v03.patch > > > Currently TestCopyTable doesn't cover table(s) with MOB feature enabled. > We should add variant that enables MOB on the table being copied and verify > that MOB content is copied correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21194) Add TestCopyTable which exercises MOB feature
[ https://issues.apache.org/jira/browse/HBASE-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656948#comment-16656948 ] Artem Ervits commented on HBASE-21194: -- [~yuzhih...@gmail.com] should I open a separate Jira and move the rest of the count() methods into the MobTestUtil class? Please review v.03 > Add TestCopyTable which exercises MOB feature > - > > Key: HBASE-21194 > URL: https://issues.apache.org/jira/browse/HBASE-21194 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Minor > Labels: mob > Attachments: HBASE-21194.v01.patch, HBASE-21194.v02.patch, > HBASE-21194.v03.patch > > > Currently TestCopyTable doesn't cover table(s) with MOB feature enabled. > We should add variant that enables MOB on the table being copied and verify > that MOB content is copied correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21194) Add TestCopyTable which exercises MOB feature
[ https://issues.apache.org/jira/browse/HBASE-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16657105#comment-16657105 ] Artem Ervits commented on HBASE-21194: -- thanks [~yuzhih...@gmail.com] please see v.06 > Add TestCopyTable which exercises MOB feature > - > > Key: HBASE-21194 > URL: https://issues.apache.org/jira/browse/HBASE-21194 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Minor > Labels: mob > Attachments: HBASE-21194.v01.patch, HBASE-21194.v02.patch, > HBASE-21194.v03.patch, HBASE-21194.v06.patch > > > Currently TestCopyTable doesn't cover table(s) with MOB feature enabled. > We should add variant that enables MOB on the table being copied and verify > that MOB content is copied correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21194) Add TestCopyTable which exercises MOB feature
[ https://issues.apache.org/jira/browse/HBASE-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21194: - Attachment: HBASE-21194.v06.patch > Add TestCopyTable which exercises MOB feature > - > > Key: HBASE-21194 > URL: https://issues.apache.org/jira/browse/HBASE-21194 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Minor > Labels: mob > Attachments: HBASE-21194.v01.patch, HBASE-21194.v02.patch, > HBASE-21194.v03.patch, HBASE-21194.v06.patch > > > Currently TestCopyTable doesn't cover table(s) with MOB feature enabled. > We should add variant that enables MOB on the table being copied and verify > that MOB content is copied correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21194) Add TestCopyTable which exercises MOB feature
[ https://issues.apache.org/jira/browse/HBASE-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21194: - Attachment: HBASE-21194.v07.patch > Add TestCopyTable which exercises MOB feature > - > > Key: HBASE-21194 > URL: https://issues.apache.org/jira/browse/HBASE-21194 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Minor > Labels: mob > Attachments: HBASE-21194.v01.patch, HBASE-21194.v02.patch, > HBASE-21194.v03.patch, HBASE-21194.v06.patch, HBASE-21194.v07.patch > > > Currently TestCopyTable doesn't cover table(s) with MOB feature enabled. > We should add variant that enables MOB on the table being copied and verify > that MOB content is copied correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21194) Add TestCopyTable which exercises MOB feature
[
https://issues.apache.org/jira/browse/HBASE-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16657204#comment-16657204
]
Artem Ervits commented on HBASE-21194:
--
I cannot tell in my environment the sort of checkstyle warnings you're getting,
I was only able to see them after the build completes. I did address the
warnings in
[https://builds.apache.org/job/PreCommit-HBASE-Build/14758/artifact/patchprocess/diff-checkstyle-hbase-mapreduce.txt]
I tried to run
{code:java}
mvn clean checkstyle:checkstyle -pl :hbase-mapreduce{code}
but the checkstyle-result.xml only shows me the warning, not the location of
the offending line.
> Add TestCopyTable which exercises MOB feature
> -
>
> Key: HBASE-21194
> URL: https://issues.apache.org/jira/browse/HBASE-21194
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: mob
> Attachments: HBASE-21194.v01.patch, HBASE-21194.v02.patch,
> HBASE-21194.v03.patch, HBASE-21194.v06.patch, HBASE-21194.v07.patch
>
>
> Currently TestCopyTable doesn't cover table(s) with MOB feature enabled.
> We should add variant that enables MOB on the table being copied and verify
> that MOB content is copied correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21194) Add TestCopyTable which exercises MOB feature
[
https://issues.apache.org/jira/browse/HBASE-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16657204#comment-16657204
]
Artem Ervits edited comment on HBASE-21194 at 10/19/18 5:55 PM:
I cannot tell in my environment the sort of checkstyle warnings you're getting,
I was only able to see them after the build completes. I did address the
warnings in
[https://builds.apache.org/job/PreCommit-HBASE-Build/14758/artifact/patchprocess/diff-checkstyle-hbase-mapreduce.txt]
I tried to run
{code:java}
mvn clean checkstyle:checkstyle -pl :hbase-mapreduce{code}
-but the checkstyle-result.xml only shows me the warning, not the location of
the offending line.-
I'm able to find the warnings in subdirectory "site", file checkstyle.html
was (Author: dbist13):
I cannot tell in my environment the sort of checkstyle warnings you're getting,
I was only able to see them after the build completes. I did address the
warnings in
[https://builds.apache.org/job/PreCommit-HBASE-Build/14758/artifact/patchprocess/diff-checkstyle-hbase-mapreduce.txt]
I tried to run
{code:java}
mvn clean checkstyle:checkstyle -pl :hbase-mapreduce{code}
but the checkstyle-result.xml only shows me the warning, not the location of
the offending line.
> Add TestCopyTable which exercises MOB feature
> -
>
> Key: HBASE-21194
> URL: https://issues.apache.org/jira/browse/HBASE-21194
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: mob
> Attachments: HBASE-21194.v01.patch, HBASE-21194.v02.patch,
> HBASE-21194.v03.patch, HBASE-21194.v06.patch, HBASE-21194.v07.patch
>
>
> Currently TestCopyTable doesn't cover table(s) with MOB feature enabled.
> We should add variant that enables MOB on the table being copied and verify
> that MOB content is copied correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21194) Add TestCopyTable which exercises MOB feature
[ https://issues.apache.org/jira/browse/HBASE-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16657249#comment-16657249 ] Artem Ervits commented on HBASE-21194: -- patch v.08 addresses any checkstyle warnings I may have contributed to. > Add TestCopyTable which exercises MOB feature > - > > Key: HBASE-21194 > URL: https://issues.apache.org/jira/browse/HBASE-21194 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Minor > Labels: mob > Attachments: HBASE-21194.v01.patch, HBASE-21194.v02.patch, > HBASE-21194.v03.patch, HBASE-21194.v06.patch, HBASE-21194.v07.patch, > HBASE-21194.v08.patch > > > Currently TestCopyTable doesn't cover table(s) with MOB feature enabled. > We should add variant that enables MOB on the table being copied and verify > that MOB content is copied correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21194) Add TestCopyTable which exercises MOB feature
[ https://issues.apache.org/jira/browse/HBASE-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21194: - Attachment: HBASE-21194.v08.patch > Add TestCopyTable which exercises MOB feature > - > > Key: HBASE-21194 > URL: https://issues.apache.org/jira/browse/HBASE-21194 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Minor > Labels: mob > Attachments: HBASE-21194.v01.patch, HBASE-21194.v02.patch, > HBASE-21194.v03.patch, HBASE-21194.v06.patch, HBASE-21194.v07.patch, > HBASE-21194.v08.patch > > > Currently TestCopyTable doesn't cover table(s) with MOB feature enabled. > We should add variant that enables MOB on the table being copied and verify > that MOB content is copied correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits reassigned HBASE-21175:
Assignee: Artem Ervits
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Attachment: HBASE-21175.v01.patch
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Status: Patch Available (was: Open)
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Status: Open (was: Patch Available)
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Attachment: HBASE-21175.v03.patch
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v03.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Status: Patch Available (was: Open)
[~yuzhih...@gmail.com] please review
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v03.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Status: Open (was: Patch Available)
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v03.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Comment: was deleted
(was: [~yuzhih...@gmail.com] please review)
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v04.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Attachment: (was: HBASE-21175.v03.patch)
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v04.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Status: Patch Available (was: Open)
[~yuzhih...@gmail.com] please review
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v04.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Attachment: HBASE-21175.v04.patch
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v04.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Attachment: HBASE-21175.v05.patch
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v04.patch,
> HBASE-21175.v05.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16665347#comment-16665347
]
Artem Ervits commented on HBASE-21175:
--
[~yuzhih...@gmail.com] please review v.05 addressing any newly-introduced and
existing checkstyle warnings.
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v04.patch,
> HBASE-21175.v05.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Attachment: HBASE-21175.v06.patch
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v04.patch,
> HBASE-21175.v05.patch, HBASE-21175.v06.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Status: Open (was: Patch Available)
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v04.patch,
> HBASE-21175.v05.patch, HBASE-21175.v06.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16665431#comment-16665431
]
Artem Ervits commented on HBASE-21175:
--
v. 06 patch addresses last comments [~yuzhih...@gmail.com].
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v04.patch,
> HBASE-21175.v05.patch, HBASE-21175.v06.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Attachment: (was: HBASE-21175.v06.patch)
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v04.patch,
> HBASE-21175.v05.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16665431#comment-16665431
]
Artem Ervits edited comment on HBASE-21175 at 10/26/18 5:24 PM:
nevermind v.06 causes some compilation problems
was (Author: dbist13):
v. 06 patch addresses last comments [~yuzhih...@gmail.com].
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v04.patch,
> HBASE-21175.v05.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Comment: was deleted
(was: nevermind v.06 causes some compilation problems)
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v04.patch,
> HBASE-21175.v05.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Status: Patch Available (was: Open)
[~yuzhih...@gmail.com] v.07 addresses all previous concerns, compiles and test
passes locally.
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v04.patch,
> HBASE-21175.v05.patch, HBASE-21175.v07.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21175) Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving
[
https://issues.apache.org/jira/browse/HBASE-21175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21175:
-
Attachment: HBASE-21175.v07.patch
> Partially initialized SnapshotHFileCleaner leads to NPE during
> TestHFileArchiving
> -
>
> Key: HBASE-21175
> URL: https://issues.apache.org/jira/browse/HBASE-21175
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: snapshot
> Attachments: HBASE-21175.v01.patch, HBASE-21175.v04.patch,
> HBASE-21175.v05.patch, HBASE-21175.v07.patch
>
>
> TestHFileArchiving#testCleaningRace creates HFileCleaner instance within the
> test.
> When SnapshotHFileCleaner.init() is called, there is no master parameter
> passed in {{params}}.
> When the chore runs the cleaner during the test, NPE comes out of this line
> in getDeletableFiles():
> {code}
> return cache.getUnreferencedFiles(files, master.getSnapshotManager());
> {code}
> since master is null.
> We should either check for the null master or, pass master instance properly
> when constructing the cleaner instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits reassigned HBASE-21141: Assignee: Artem Ervits > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backup&restore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21458) Error: Could not find or load main class org.apache.hadoop.hbase.util.GetJavaProperty
[ https://issues.apache.org/jira/browse/HBASE-21458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16680354#comment-16680354 ] Artem Ervits commented on HBASE-21458: -- yes [~stack] that's the message I reported in the 2.1.1 vote. > Error: Could not find or load main class > org.apache.hadoop.hbase.util.GetJavaProperty > - > > Key: HBASE-21458 > URL: https://issues.apache.org/jira/browse/HBASE-21458 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Major > Attachments: HBASE-21458.branch-2.1.001.patch > > > I get this when I run bin/hbase classpath whether a built checkout or an > undone tarball. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21141: - Status: Patch Available (was: Open) > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backup&restore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > Attachments: HBASE-21141.v01.patch > > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688611#comment-16688611 ] Artem Ervits commented on HBASE-21141: -- [~yuzhih...@gmail.com] JFYI, after patch, the test runs a bit faster, wonder if changing to calling static method and using try-with-resources had helped test duration. > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backup&restore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > Attachments: HBASE-21141.v01.patch > > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21141: - Attachment: HBASE-21141.v01.patch > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backup&restore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > Attachments: HBASE-21141.v01.patch > > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21141: - Attachment: HBASE-21141.v02.patch > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backup&restore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch > > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16689693#comment-16689693 ] Artem Ervits commented on HBASE-21141: -- [~yuzhih...@gmail.com] v.02 patch addresses your comments. > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backup&restore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch > > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21141: - Attachment: HBASE-21141.v03.patch > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backup&restore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch, > HBASE-21141.v03.patch > > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[
https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16689810#comment-16689810
]
Artem Ervits commented on HBASE-21141:
--
[~yuzhih...@gmail.com] added your comments, there's a checkstyle warning but I
am not sure how it's getting generated
{code:java}
Error sizes MethodLengthMethod length is 157 lines (max allowed is
150).76{code}
> Enable MOB in backup / restore test involving incremental backup
>
>
> Key: HBASE-21141
> URL: https://issues.apache.org/jira/browse/HBASE-21141
> Project: HBase
> Issue Type: Test
> Components: backup&restore
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Major
> Labels: mob
> Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch,
> HBASE-21141.v03.patch
>
>
> Currently we only have one test (TestRemoteBackup) where MOB feature is
> enabled. The test only performs full backup.
> This issue is to enable MOB in backup / restore test(s) involving incremental
> backup.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16689859#comment-16689859 ] Artem Ervits commented on HBASE-21141: -- that's a new one for me, I thought it was referring to method name length, not the actual method. The message is gone with v.04. [~yuzhih...@gmail.com] > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backup&restore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch, > HBASE-21141.v03.patch, HBASE-21141.v04.patch > > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21141) Enable MOB in backup / restore test involving incremental backup
[ https://issues.apache.org/jira/browse/HBASE-21141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21141: - Attachment: HBASE-21141.v04.patch > Enable MOB in backup / restore test involving incremental backup > > > Key: HBASE-21141 > URL: https://issues.apache.org/jira/browse/HBASE-21141 > Project: HBase > Issue Type: Test > Components: backup&restore >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: mob > Attachments: HBASE-21141.v01.patch, HBASE-21141.v02.patch, > HBASE-21141.v03.patch, HBASE-21141.v04.patch > > > Currently we only have one test (TestRemoteBackup) where MOB feature is > enabled. The test only performs full backup. > This issue is to enable MOB in backup / restore test(s) involving incremental > backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21304) HBaseBulkGetExample throws NullPointerException
[
https://issues.apache.org/jira/browse/HBASE-21304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits reassigned HBASE-21304:
Assignee: Artem Ervits
> HBaseBulkGetExample throws NullPointerException
> ---
>
> Key: HBASE-21304
> URL: https://issues.apache.org/jira/browse/HBASE-21304
> Project: HBase
> Issue Type: Bug
> Components: spark
>Affects Versions: 3.0.0
>Reporter: Carlos A. Morillo
>Assignee: Artem Ervits
>Priority: Minor
> Labels: beginner
>
> Running the HBaseContext Examples available at
> [https://github.com/apache/hbase/tree/master/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/hbasecontext]
>
> If you run them in sequence, let's say you run first HBaseBulkPutExample
> [https://github.com/apache/hbase/blob/master/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/rdd/HBaseBulkPutExample.scala]
> and immediately after you run HBaseBulkGetExample
> [https://github.com/apache/hbase/blob/master/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/rdd/HBaseBulkGetExample.scala]
> You get a NullPointerException.
> In the API
> [https://github.com/apache/hbase/blob/master/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/hbasecontext/HBaseBulkGetExample.scala]
> we have:
> h4. defhbaseBulkGet(hc:
> [HBaseContext|http://hbase.apache.org/hbase-spark/scaladocs/org/apache/hadoop/hbase/spark/HBaseContext.html],
> tableName: TableName, batchSize: Int, f: (T) ⇒ Get):
> RDD[(ImmutableBytesWritable, Result)]
> Implicit method that gives easy access to HBaseContext's bulk get. This will
> return a new RDD. Think about it as a RDD map function. In that every RDD
> value will get a new value out of HBase. That new value will populate the
> newly generated RDD.
> hc
> The hbaseContext object to identify which HBase cluster connection to use
> tableName
> The tableName that the put will be sent to
> batchSize
> How many gets to execute in a single batch
> f
> The function that will turn the RDD values in HBase Get objects
> returns
> A resulting RDD with type R objects
> So it seems the function f passed to should be modified as an Scala partial
> function to handle the case when the Result is null.
> One possible fix would be {color:#00}to call in an{color}{color:#00}
> if
> {{[Result.isEmpty()|https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Result.html#isEmpty--]
> to make sure it isn't empty.}}{color}
> {color:#00}{{The API for Result.listCells expressly says it can return
> null if there are no results.}}{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-20676) Give .hbase-snapshot proper ownership upon directory creation
[ https://issues.apache.org/jira/browse/HBASE-20676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits reassigned HBASE-20676: Assignee: Artem Ervits > Give .hbase-snapshot proper ownership upon directory creation > - > > Key: HBASE-20676 > URL: https://issues.apache.org/jira/browse/HBASE-20676 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Artem Ervits >Priority: Major > Labels: security, snapshot > > This is continuation of the discussion over HBASE-20668. > Tthe .hbase-snapshot directory is not created at cluster startup. Normally it > is created when snapshot operation is initiated. > However, if before any snapshot operation is performed, some non-super user > from another cluster conducts ExportSnapshot to this cluster, the > .hbase-snapshot directory would be created as that user. > (This is just one scenario that can lead to wrong ownership) > This JIRA is to seek proper way(s) to ensure that .hbase-snapshot directory > would always carry proper onwership and permission upon creation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21501) Add “force-stop” to skip all close region logic in HBaseMiniCluster
Artem Ervits created HBASE-21501: Summary: Add “force-stop” to skip all close region logic in HBaseMiniCluster Key: HBASE-21501 URL: https://issues.apache.org/jira/browse/HBASE-21501 Project: HBase Issue Type: Improvement Components: test Reporter: Artem Ervits Assignee: Artem Ervits spoke to [~elserj] as a result of Phoenix dev meetup, this may be a much needed feature. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21506) Fix for warning: Multiple versions of scala detected in phoenix-spark
Artem Ervits created HBASE-21506:
Summary: Fix for warning: Multiple versions of scala detected in
phoenix-spark
Key: HBASE-21506
URL: https://issues.apache.org/jira/browse/HBASE-21506
Project: HBase
Issue Type: Improvement
Reporter: Artem Ervits
Assignee: Artem Ervits
{code:java}
[WARNING] Expected all dependencies to require Scala version: 2.11.8
[WARNING] org.apache.phoenix:phoenix-spark:5.1.0-HBase-2.0-SNAPSHOT requires
scala version: 2.11.8
[WARNING] com.twitter:chill_2.11:0.8.4 requires scala version: 2.11.8
[WARNING] org.apache.spark:spark-core_2.11:2.3.0 requires scala version: 2.11.8
[WARNING] org.json4s:json4s-jackson_2.11:3.2.11 requires scala version: 2.11.0
[WARNING] Multiple versions of scala libraries detected!{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-21506) Fix for warning: Multiple versions of scala detected in phoenix-spark
[
https://issues.apache.org/jira/browse/HBASE-21506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits resolved HBASE-21506.
--
Resolution: Invalid
> Fix for warning: Multiple versions of scala detected in phoenix-spark
> -
>
> Key: HBASE-21506
> URL: https://issues.apache.org/jira/browse/HBASE-21506
> Project: HBase
> Issue Type: Improvement
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
>
> {code:java}
> [WARNING] Expected all dependencies to require Scala version: 2.11.8
> [WARNING] org.apache.phoenix:phoenix-spark:5.1.0-HBase-2.0-SNAPSHOT requires
> scala version: 2.11.8
> [WARNING] com.twitter:chill_2.11:0.8.4 requires scala version: 2.11.8
> [WARNING] org.apache.spark:spark-core_2.11:2.3.0 requires scala version:
> 2.11.8
> [WARNING] org.json4s:json4s-jackson_2.11:3.2.11 requires scala version: 2.11.0
> [WARNING] Multiple versions of scala libraries detected!{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-21501) Add “force-stop” to skip all close region logic in HBaseMiniCluster
[ https://issues.apache.org/jira/browse/HBASE-21501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits resolved HBASE-21501. -- Resolution: Duplicate will track in the original Jira https://issues.apache.org/jira/browse/HBASE-18735 > Add “force-stop” to skip all close region logic in HBaseMiniCluster > --- > > Key: HBASE-21501 > URL: https://issues.apache.org/jira/browse/HBASE-21501 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Major > > spoke to [~elserj] as a result of Phoenix dev meetup, this may be a much > needed feature. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits reassigned HBASE-18735: Assignee: Artem Ervits > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-18735: - Status: Patch Available (was: Open) I provided two tests to demonstrate performance of both calls forceShutdownMiniHBaseCluster() and shutdownMiniHBaseCluster(). I tested with 100 tables and 1000 rows each, on average, new method is 40-50secs faster. I'm open to suggestions [~jamestaylor]. I also need to fix some checkstyle warnings but seems hbase-server module has tons of them so this patch is only to demonstrate functionality, I'll follow up with checkstyle fixes after all reviews are in. > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-18735.v01.patch > > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-18735: - Attachment: HBASE-18735.v01.patch > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-18735.v01.patch > > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21481) [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
[
https://issues.apache.org/jira/browse/HBASE-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701209#comment-16701209
]
Artem Ervits commented on HBASE-21481:
--
[~reidchan] your patch fails in
[https://builds.apache.org/job/PreCommit-HBASE-Build/15119/artifact/patchprocess/patch-unit-hbase-server.txt]
does it pass locally?
> [acl] Superuser's permissions should not be granted or revoked by any non-su
> global admin
> -
>
> Key: HBASE-21481
> URL: https://issues.apache.org/jira/browse/HBASE-21481
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Labels: ACL, security-issue
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21481.master.001.patch,
> HBASE-21481.master.002.patch, HBASE-21481.master.003.patch,
> HBASE-21481.master.004.patch, HBASE-21481.master.005.patch
>
>
> Superusers are {{hbase.superuser}} listed in configuration and plus the one
> who start master process, these two may be overlap.
> A superuser must be a global admin, but a global admin may not be a
> superuser, possibly granted afterwards.
> For now, an non-su global admin with a Global.ADMIN permission can grant or
> revoke any superuser's permission, accidentally or deliberately.
> The purpose of this issue is to ban this action.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701213#comment-16701213 ] Artem Ervits commented on HBASE-18735: -- [~stack] what is the suggested method name? You may have pasted the same name twice forceShutdownMiniHBaseCluster > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-18735.v01.patch > > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-18735: - Attachment: HBASE-18735.v02.patch > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch > > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701993#comment-16701993 ] Artem Ervits commented on HBASE-18735: -- .02 patch addresses [~stack] comment as well as fixes some checkstyle issues. There are over 7000 checkstyle issues in hbase-server and hundreds of issues in HBaseTestingUtility alone, if need be, I can address those in a separate jira. My question remains, is this a useful patch [~jamestaylor] [~lhofhansl] and do I need to keep the tests as they're not testing but merely there for timing purposes. [~elserj] please advise. > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch > > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-18735: - Attachment: HBASE-18735.v03.patch > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch, > HBASE-18735.v03.patch > > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703229#comment-16703229 ] Artem Ervits commented on HBASE-18735: -- [~elserj] v.03 patch reverts all checkstyle changes, removes test for shutdown, reduces sample size to 1 table. > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch, > HBASE-18735.v03.patch > > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703508#comment-16703508 ] Artem Ervits commented on HBASE-18735: -- [~jamestaylor] it's hard to say as I imagine the proportion of improvement is tied to number of tables and rows ingested. I originally tested with 1, 10 and 100 tables. Improvement was 1 second faster for 1 table, 5 seconds or so for 10 tables and 50 seconds for 100 tables. On 100 tables, improvement is 12%. If you want to identify a few tests in Phoenix to test this on, I will be happy to take this upon myself, let's do this in a Phoenix Jira, feel free to assign to me? > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch, > HBASE-18735.v03.patch > > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[
https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703634#comment-16703634
]
Artem Ervits edited comment on HBASE-18735 at 11/29/18 6:38 PM:
The {org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner} test passes
locally, albeit runs for 575 seconds on my Mac and unrelated to my patch.
was (Author: dbist13):
The [org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner] test passes
locally, albeit runs for 575 seconds on my Mac and unrelated to my patch.
> Provide a fast mechanism for shutting down mini cluster
> ---
>
> Key: HBASE-18735
> URL: https://issues.apache.org/jira/browse/HBASE-18735
> Project: HBase
> Issue Type: Wish
>Reporter: Samarth Jain
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch,
> HBASE-18735.v03.patch
>
>
> The current mechanism of shutting down a mini cluster through
> HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini
> cluster almost has a lot of tables. A lot of this time is spent in closing
> all the user regions. It would be nice to have a mechanism where this
> shutdown can happen quickly without having to worry about closing these user
> regions. At the same time, this mechanism would need to make sure that all
> the critical system resources like file handles and network ports are still
> released so that subsequently initialized mini clusters on the same JVM or
> system won't run into resource issues. This would make testing using HBase
> mini clusters much faster and immensely help out test frameworks of dependent
> projects like Phoenix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703634#comment-16703634 ] Artem Ervits commented on HBASE-18735: -- The [org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner] test passes locally, albeit runs for 575 seconds on my Mac and unrelated to my patch. > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch, > HBASE-18735.v03.patch > > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[
https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703634#comment-16703634
]
Artem Ervits edited comment on HBASE-18735 at 11/29/18 6:39 PM:
The
{code:java}
org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner{code}
test passes locally, albeit runs for 575 seconds on my Mac and unrelated to my
patch.
was (Author: dbist13):
The {org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner} test passes
locally, albeit runs for 575 seconds on my Mac and unrelated to my patch.
> Provide a fast mechanism for shutting down mini cluster
> ---
>
> Key: HBASE-18735
> URL: https://issues.apache.org/jira/browse/HBASE-18735
> Project: HBase
> Issue Type: Wish
>Reporter: Samarth Jain
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch,
> HBASE-18735.v03.patch
>
>
> The current mechanism of shutting down a mini cluster through
> HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini
> cluster almost has a lot of tables. A lot of this time is spent in closing
> all the user regions. It would be nice to have a mechanism where this
> shutdown can happen quickly without having to worry about closing these user
> regions. At the same time, this mechanism would need to make sure that all
> the critical system resources like file handles and network ports are still
> released so that subsequently initialized mini clusters on the same JVM or
> system won't run into resource issues. This would make testing using HBase
> mini clusters much faster and immensely help out test frameworks of dependent
> projects like Phoenix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21528) hbase-connectors need scala library convergence
Artem Ervits created HBASE-21528: Summary: hbase-connectors need scala library convergence Key: HBASE-21528 URL: https://issues.apache.org/jira/browse/HBASE-21528 Project: HBase Issue Type: Task Components: hbase-connectors Reporter: Artem Ervits Assignee: Artem Ervits this issue reappeared after migration of hbase-spark to hbase-connectors repo. Originally addressed in https://issues.apache.org/jira/browse/HBASE-20175 introducing scala-maven-plugin and adding a flag to honor compatible major scala version makes warning disappear. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21528) hbase-connectors need scala dependency convergence
[ https://issues.apache.org/jira/browse/HBASE-21528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21528: - Summary: hbase-connectors need scala dependency convergence (was: hbase-connectors need scala library convergence) > hbase-connectors need scala dependency convergence > -- > > Key: HBASE-21528 > URL: https://issues.apache.org/jira/browse/HBASE-21528 > Project: HBase > Issue Type: Task > Components: hbase-connectors >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Minor > > this issue reappeared after migration of hbase-spark to hbase-connectors > repo. Originally addressed in > https://issues.apache.org/jira/browse/HBASE-20175 > introducing scala-maven-plugin and adding a flag to honor compatible major > scala version makes warning disappear. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21528) hbase-connectors need scala dependency convergence
[ https://issues.apache.org/jira/browse/HBASE-21528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21528: - Status: Patch Available (was: Open) > hbase-connectors need scala dependency convergence > -- > > Key: HBASE-21528 > URL: https://issues.apache.org/jira/browse/HBASE-21528 > Project: HBase > Issue Type: Task > Components: hbase-connectors >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Minor > Attachments: HBASE-21528.v01.patch > > > this issue reappeared after migration of hbase-spark to hbase-connectors > repo. Originally addressed in > https://issues.apache.org/jira/browse/HBASE-20175 > introducing scala-maven-plugin and adding a flag to honor compatible major > scala version makes warning disappear. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
