[jira] [Commented] (HBASE-20642) IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException
[ https://issues.apache.org/jira/browse/HBASE-20642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517820#comment-16517820 ] stack commented on HBASE-20642: --- Skimmed. +1 > IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException > - > > Key: HBASE-20642 > URL: https://issues.apache.org/jira/browse/HBASE-20642 > Project: HBase > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Attachments: HBASE-20642.001.patch, HBASE-20642.002.patch, > HBASE-20642.patch > > > [~romil.choksi] reported that IntegrationTestDDLMasterFailover is failing > while adding column family during the time master is restarting. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20615) emphasize use of shaded client jars when they're present in an install
[ https://issues.apache.org/jira/browse/HBASE-20615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20615: Release Note: HBase's built in scripts now rely on the downstream facing shaded artifacts where possible. In particular interest to downstream users, the `hbase classpath` and `hbase mapredcp` commands now return the relevant shaded client artifact and only those third paty jars needed to make use of them (e.g. slf4j-api, commons-logging, htrace, etc). Downstream users should note that by default the `hbase classpath` command will treat having `hadoop` on the shell's PATH as an implicit request to include the output of the `hadoop classpath` command in the returned classpath. This long-existing behavior can be opted out of by setting the environment variable `HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP` to the value "true". For example: `HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP="true" bin/hbase classpath`. was: HBase's built in scripts now rely on the downstream facing shaded artifacts where possible. In particular interest to downstream users, the `hadoop classpath` and `hadoop mapredcp` commands now return the relevant shaded client artifact and only those third paty jars needed to make use of them (e.g. slf4j-api, commons-logging, htrace, etc). Downstream users should note that by default the `hbase classpath` command will treat having `hadoop` on the shell's PATH as an implicit request to include the output of the `hadoop classpath` command in the returned classpath. This long-existing behavior can be opted out of by setting the environment variable `HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP` to the value "true". For example: `HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP="true" bin/hbase classpath`. > emphasize use of shaded client jars when they're present in an install > -- > > Key: HBASE-20615 > URL: https://issues.apache.org/jira/browse/HBASE-20615 > Project: HBase > Issue Type: Sub-task > Components: build, Client, Usability >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20615.0.patch, HBASE-20615.1.patch, > HBASE-20615.2.patch > > > Working through setting up an IT for our shaded artifacts in HBASE-20334 > makes our lack of packaging seem like an oversight. While I could work around > by pulling the shaded clients out of whatever build process built the > convenience binary that we're trying to test, it seems v awkward. > After reflecting on it more, it makes more sense to me for there to be a > common place in the install that folks running jobs against the cluster can > rely on. If they need to run without a full hbase install, that should still > work fine via e.g. grabbing from the maven repo. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20642) IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException
[
https://issues.apache.org/jira/browse/HBASE-20642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517810#comment-16517810
]
Ankit Singhal commented on HBASE-20642:
---
Thanks [~stack] and [~elserj] for all the help in getting this fixed.
{quote}Doing nonce creation outside the call method is what makes it so we
don't make a new nonce on each invocation?
{quote}
Yes , so that call method will be called with the same nonce for retries.
{quote}So, the patch here makes it so call could complete even though Master
was restarted while call was going on (as long as new Master came up before
timeout)?
{quote}
Yes , the test restarts procedure executor after every step to simulate the
master failure.
{quote}Bit of doc for this...
public interface StepHook{
?
{quote}
Done in the latest patch. Fixed javac, whitespace and checkstyle warning as
well.
{quote}I learned/re-learned stuff reviewing this work.
{quote}
Same here, great learnings for me as well.
> IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException
> -
>
> Key: HBASE-20642
> URL: https://issues.apache.org/jira/browse/HBASE-20642
> Project: HBase
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Attachments: HBASE-20642.001.patch, HBASE-20642.002.patch,
> HBASE-20642.patch
>
>
> [~romil.choksi] reported that IntegrationTestDDLMasterFailover is failing
> while adding column family during the time master is restarting.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20756) reference guide examples still contain references to EOM versions
Sean Busbey created HBASE-20756:
---
Summary: reference guide examples still contain references to EOM
versions
Key: HBASE-20756
URL: https://issues.apache.org/jira/browse/HBASE-20756
Project: HBase
Issue Type: Bug
Components: community, documentation
Reporter: Sean Busbey
the reference guide still has examples that refer to EOM versions. e.g. this
shell output that has 0.98 in it:
{code}
$ echo "describe 'test1'" | ./hbase shell -n
Version 0.98.3-hadoop2, rd5e65a9144e315bb0a964e7730871af32f5018d5, Sat May 31
19:56:09 PDT 2014
describe 'test1'
DESCRIPTION ENABLED
'test1', {NAME => 'cf', DATA_BLOCK_ENCODING => 'NON true
E', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0',
VERSIONS => '1', COMPRESSION => 'NONE', MIN_VERSIO
NS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS =>
'false', BLOCKSIZE => '65536', IN_MEMORY => 'false'
, BLOCKCACHE => 'true'}
1 row(s) in 3.2410 seconds
{code}
these should be redone with a current release. Ideally a version in the minor
release line the docs are for, but even just updating to the stable pointer
would be a big improvement.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20642) IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException
[ https://issues.apache.org/jira/browse/HBASE-20642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal updated HBASE-20642: -- Attachment: HBASE-20642.002.patch > IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException > - > > Key: HBASE-20642 > URL: https://issues.apache.org/jira/browse/HBASE-20642 > Project: HBase > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Attachments: HBASE-20642.001.patch, HBASE-20642.002.patch, > HBASE-20642.patch > > > [~romil.choksi] reported that IntegrationTestDDLMasterFailover is failing > while adding column family during the time master is restarting. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20403) Prefetch sometimes doesn't work with encrypted file system
[
https://issues.apache.org/jira/browse/HBASE-20403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517805#comment-16517805
]
Todd Lipcon commented on HBASE-20403:
-
Hello from the peanut gallery!
Looking at the implementation of prefetch, it seems like the prefetch task
scheduled on a separate thread calls readBlock() on the HFileReaderImpl even
though there might be concurrent calls from the main (scanner) thread. It calls
readBlock() with pread == false, which means that it ends up screwing with the
file position, buffers, and underlying codec from the main thread. Seems like
that could easily cause invalid data reads, weird buffer offsets, and crypto
library crashes (due to concurrent usage of the same cipher).
Am I mis-remembering the thread safety guarantees of HFileReader? I had thought
it was not meant to be thread-safe, but the prefetching is basically
multi-threaded access to a single instance.
> Prefetch sometimes doesn't work with encrypted file system
> --
>
> Key: HBASE-20403
> URL: https://issues.apache.org/jira/browse/HBASE-20403
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-beta-2
>Reporter: Umesh Agashe
>Assignee: Umesh Agashe
>Priority: Major
> Fix For: 3.0.0
>
>
> Log from long running test has following stack trace a few times:
> {code}
> 2018-04-09 18:33:21,523 WARN
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl: Prefetch
> path=hdfs://ns1/hbase/data/default/IntegrationTestBigLinkedList_20180409172704/35f1a7ef13b9d327665228abdbcdffae/meta/9089d98b2a6b4847b3fcf6aceb124988,
> offset=36884200, end=231005989
> java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:275)
> at
> org.apache.hadoop.hdfs.ByteBufferStrategy.readFromBlock(ReaderStrategy.java:183)
> at org.apache.hadoop.hdfs.DFSInputStream.readBuffer(DFSInputStream.java:705)
> at
> org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:766)
> at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:831)
> at
> org.apache.hadoop.crypto.CryptoInputStream.read(CryptoInputStream.java:197)
> at java.io.DataInputStream.read(DataInputStream.java:149)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock.readWithExtra(HFileBlock.java:762)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readAtOffset(HFileBlock.java:1559)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1771)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1594)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl.readBlock(HFileReaderImpl.java:1488)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$1.run(HFileReaderImpl.java:278)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> Size on disk calculations seem to get messed up due to encryption. Possible
> fixes can be:
> * if file is encrypted with FileStatus#isEncrypted() and do not prefetch.
> * document that hbase.rs.prefetchblocksonopen cannot be true if file is
> encrypted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20755) quickstart note about Web UI port changes in ref guide is rendered incorrectly
[ https://issues.apache.org/jira/browse/HBASE-20755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20755: Attachment: Untitled.png > quickstart note about Web UI port changes in ref guide is rendered incorrectly > -- > > Key: HBASE-20755 > URL: https://issues.apache.org/jira/browse/HBASE-20755 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Sean Busbey >Priority: Minor > Attachments: Untitled.png > > > The note in the quickstart guide about how the web ui ports changed only > renders the title as a note. the text is just a normal paragraph afterwards. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20755) quickstart note about Web UI port changes in ref guide is rendered incorrectly
Sean Busbey created HBASE-20755: --- Summary: quickstart note about Web UI port changes in ref guide is rendered incorrectly Key: HBASE-20755 URL: https://issues.apache.org/jira/browse/HBASE-20755 Project: HBase Issue Type: Bug Components: documentation Reporter: Sean Busbey The note in the quickstart guide about how the web ui ports changed only renders the title as a note. the text is just a normal paragraph afterwards. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20753) reference guide should direct security related issues to priv...@hbase.apache.org
[
https://issues.apache.org/jira/browse/HBASE-20753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20753:
Component/s: security
> reference guide should direct security related issues to
> priv...@hbase.apache.org
> -
>
> Key: HBASE-20753
> URL: https://issues.apache.org/jira/browse/HBASE-20753
> Project: HBase
> Issue Type: Bug
> Components: documentation, security
>Reporter: Sean Busbey
>Priority: Critical
> Labels: beginner
>
> the reference guide currently directs folks to send security issues to
> priv...@apache.org:
> {quote}
> To protect existing HBase installations from new vulnerabilities, please do
> not use JIRA to report security-related bugs. Instead, send your report to
> the mailing list priv...@apache.org, which allows anyone to send messages,
> but restricts who can read them. Someone on that list will contact you to
> follow up on your report.
> {quote}
> This address does not exist. It should tell folks to send the email to
> priv...@hbase.apache.org.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20754) quickstart guide should instruct folks to set JAVA_HOME to a JDK installation.
Sean Busbey created HBASE-20754:
---
Summary: quickstart guide should instruct folks to set JAVA_HOME
to a JDK installation.
Key: HBASE-20754
URL: https://issues.apache.org/jira/browse/HBASE-20754
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Sean Busbey
The quickstart guide currently instructs folks to set JAVA_HOME, but to the
wrong place
{code}
The JAVA_HOME variable should be set to a directory which contains the
executable file bin/java. Most modern Linux operating systems provide a
mechanism, such as /usr/bin/alternatives on RHEL or CentOS, for transparently
switching between versions of executables such as Java. In this case, you can
set JAVA_HOME to the directory containing the symbolic link to bin/java, which
is usually /usr.
JAVA_HOME=/usr
{code}
instead, it should tell folks to point it to a jdk installation and help them
on how to find that.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20732) Shutdown scan pool when master is stopped.
[
https://issues.apache.org/jira/browse/HBASE-20732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517792#comment-16517792
]
Reid Chan commented on HBASE-20732:
---
Refactored {{TestCleanerChore}} and ran few times on my machine, all passed.
Let's see what QA say.
> Shutdown scan pool when master is stopped.
> --
>
> Key: HBASE-20732
> URL: https://issues.apache.org/jira/browse/HBASE-20732
> Project: HBase
> Issue Type: Bug
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-20732.master.001.patch,
> HBASE-20732.master.002.patch
>
>
> If master is stopped, {{DirScanPool}} is kept open. This is found by
> [~chia7712] when reviewing HBASE-20352.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20732) Shutdown scan pool when master is stopped.
[
https://issues.apache.org/jira/browse/HBASE-20732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-20732:
--
Attachment: HBASE-20732.master.002.patch
> Shutdown scan pool when master is stopped.
> --
>
> Key: HBASE-20732
> URL: https://issues.apache.org/jira/browse/HBASE-20732
> Project: HBase
> Issue Type: Bug
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-20732.master.001.patch,
> HBASE-20732.master.002.patch
>
>
> If master is stopped, {{DirScanPool}} is kept open. This is found by
> [~chia7712] when reviewing HBASE-20352.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20753) reference guide should direct security related issues to priv...@hbase.apache.org
Sean Busbey created HBASE-20753:
---
Summary: reference guide should direct security related issues to
priv...@hbase.apache.org
Key: HBASE-20753
URL: https://issues.apache.org/jira/browse/HBASE-20753
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Sean Busbey
the reference guide currently directs folks to send security issues to
priv...@apache.org:
{quote}
To protect existing HBase installations from new vulnerabilities, please do not
use JIRA to report security-related bugs. Instead, send your report to the
mailing list priv...@apache.org, which allows anyone to send messages, but
restricts who can read them. Someone on that list will contact you to follow up
on your report.
{quote}
This address does not exist. It should tell folks to send the email to
priv...@hbase.apache.org.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20642) IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException
[
https://issues.apache.org/jira/browse/HBASE-20642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517787#comment-16517787
]
Josh Elser commented on HBASE-20642:
{quote}Doing nonce creation outside the call method is what makes it so we
don't make a new nonce on each invocation?
{quote}
{quote}So, the patch here makes it so call could complete even though Master
was restarted while call was going on (as long as new Master came up before
timeout)?
{quote}
That's what I'm seeing with this.
I think after the javadoc for StepHook (and the corresponding
\{{MasterProcedureTestingUtility}} usage), this is good to go, too.
> IntegrationTestDDLMasterFailover throws 'InvalidFamilyOperationException
> -
>
> Key: HBASE-20642
> URL: https://issues.apache.org/jira/browse/HBASE-20642
> Project: HBase
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Attachments: HBASE-20642.001.patch, HBASE-20642.patch
>
>
> [~romil.choksi] reported that IntegrationTestDDLMasterFailover is failing
> while adding column family during the time master is restarting.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20587) Replace Jackson with shaded thirdparty gson in client
[ https://issues.apache.org/jira/browse/HBASE-20587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-20587: --- Fix Version/s: (was: 2.1.0) 2.2.0 > Replace Jackson with shaded thirdparty gson in client > - > > Key: HBASE-20587 > URL: https://issues.apache.org/jira/browse/HBASE-20587 > Project: HBase > Issue Type: Bug > Components: dependencies >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-20587.001.patch > > > HBASE-20582 got me looking at how we use Jackson. It appears that we moved > some JSON code from hbase-server into hbase-common via HBASE-19053. But, > there seems to be no good reason why this code should live there and not in > hbase-http instead. Keeping Jackson off the user's classpath is a nice goal. > FYI [~appy], [~mdrob] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20635) Support to convert the shaded user permission proto to client user permission object
[
https://issues.apache.org/jira/browse/HBASE-20635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517780#comment-16517780
]
Josh Elser commented on HBASE-20635:
{quote}The patch is already in branch-2? No?
{quote}
Yeah, [~Apache9], it's in branch-2 and master. Didn't push a revert yet, but
can if we want to. Not sure if [~stack] was thinking that we should revert, or
just expressing concern over what Rajesh was trying to do downstream.
> Support to convert the shaded user permission proto to client user permission
> object
>
>
> Key: HBASE-20635
> URL: https://issues.apache.org/jira/browse/HBASE-20635
> Project: HBase
> Issue Type: Bug
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20635.patch, HBASE-20635_v2.patch,
> PHOENIX-4528_5.x-HBase-2.0_v2.patch
>
>
> Currently we have API to build the protobuf UserPermission to client user
> permission in AccessControlUtil but we cannot do the same when we use shaded
> protobufs.
> {noformat}
> /**
>* Converts a user permission proto to a client user permission object.
>*
>* @param proto the protobuf UserPermission
>* @return the converted UserPermission
>*/
> public static UserPermission
> toUserPermission(AccessControlProtos.UserPermission proto) {
> return new UserPermission(proto.getUser().toByteArray(),
> toTablePermission(proto.getPermission()));
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20587) Replace Jackson with shaded thirdparty gson in client
[ https://issues.apache.org/jira/browse/HBASE-20587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517783#comment-16517783 ] Josh Elser commented on HBASE-20587: Nah, haven't gotten around to this yet, Duo. Bumped to 2.2.0. Thanks for checking. > Replace Jackson with shaded thirdparty gson in client > - > > Key: HBASE-20587 > URL: https://issues.apache.org/jira/browse/HBASE-20587 > Project: HBase > Issue Type: Bug > Components: dependencies >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-20587.001.patch > > > HBASE-20582 got me looking at how we use Jackson. It appears that we moved > some JSON code from hbase-server into hbase-common via HBASE-19053. But, > there seems to be no good reason why this code should live there and not in > hbase-http instead. Keeping Jackson off the user's classpath is a nice goal. > FYI [~appy], [~mdrob] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-15320) HBase connector for Kafka Connect
[ https://issues.apache.org/jira/browse/HBASE-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1651#comment-1651 ] stack commented on HBASE-15320: --- Hello [~MikeWingert] Thanks for keeping at this. It looks great. Have you seen HBASE-18846? We changed the way the hbase-indexer gets its edits so you don't have to do your own dumbed down regionserver; rather, with a bit of config., you can just use the current RS and do your plugin against a public API, one that is less likely to rot or break on every change to our internals? Thanks. > HBase connector for Kafka Connect > - > > Key: HBASE-15320 > URL: https://issues.apache.org/jira/browse/HBASE-15320 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Andrew Purtell >Assignee: Mike Wingert >Priority: Major > Labels: beginner > Fix For: 3.0.0 > > Attachments: HBASE-15320.master.1.patch, HBASE-15320.master.10.patch, > HBASE-15320.master.2.patch, HBASE-15320.master.3.patch, > HBASE-15320.master.4.patch, HBASE-15320.master.5.patch, > HBASE-15320.master.6.patch, HBASE-15320.master.7.patch, > HBASE-15320.master.8.patch, HBASE-15320.master.8.patch, > HBASE-15320.master.9.patch, HBASE-15320.pdf, HBASE-15320.pdf > > > Implement an HBase connector with source and sink tasks for the Connect > framework (http://docs.confluent.io/2.0.0/connect/index.html) available in > Kafka 0.9 and later. > See also: > http://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines > An HBase source > (http://docs.confluent.io/2.0.0/connect/devguide.html#task-example-source-task) > could be implemented as a replication endpoint or WALObserver, publishing > cluster wide change streams from the WAL to one or more topics, with > configurable mapping and partitioning of table changes to topics. > An HBase sink task > (http://docs.confluent.io/2.0.0/connect/devguide.html#sink-tasks) would > persist, with optional transformation (JSON? Avro?, map fields to native > schema?), Kafka SinkRecords into HBase tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20742) Always create WAL directory for region server
[ https://issues.apache.org/jira/browse/HBASE-20742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517773#comment-16517773 ] stack commented on HBASE-20742: --- +1 > Always create WAL directory for region server > - > > Key: HBASE-20742 > URL: https://issues.apache.org/jira/browse/HBASE-20742 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20742.patch > > > After HBASE-20708, when master restart, we will scan the wal directory to > find out the live servers. In most cases this is OK, as when we create a > HRegion instance at RS side, we will create a WAL for it, and the directory > which contains the server name will be there, even if user always use > SKIP_WAL. > But there could still be problem as the directory is created in the > implementation of WAL, not in the initialization of region server, so if user > uses DisabledWALProvider then we will be in trouble. > So let's fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-15320) HBase connector for Kafka Connect
[
https://issues.apache.org/jira/browse/HBASE-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517749#comment-16517749
]
Ted Yu commented on HBASE-15320:
Unit test encountered:
{code}
[ERROR] Caused by:
org.apache.maven.surefire.booter.SurefireBooterForkException: There was an
error in the forked process
[ERROR] java.lang.ClassNotFoundException:
org.apache.hadoop.hbase.ResourceCheckerJUnitListener
{code}
Please add license header to conf/kafka-route-rules.xml
> HBase connector for Kafka Connect
> -
>
> Key: HBASE-15320
> URL: https://issues.apache.org/jira/browse/HBASE-15320
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Reporter: Andrew Purtell
>Assignee: Mike Wingert
>Priority: Major
> Labels: beginner
> Fix For: 3.0.0
>
> Attachments: HBASE-15320.master.1.patch, HBASE-15320.master.10.patch,
> HBASE-15320.master.2.patch, HBASE-15320.master.3.patch,
> HBASE-15320.master.4.patch, HBASE-15320.master.5.patch,
> HBASE-15320.master.6.patch, HBASE-15320.master.7.patch,
> HBASE-15320.master.8.patch, HBASE-15320.master.8.patch,
> HBASE-15320.master.9.patch, HBASE-15320.pdf, HBASE-15320.pdf
>
>
> Implement an HBase connector with source and sink tasks for the Connect
> framework (http://docs.confluent.io/2.0.0/connect/index.html) available in
> Kafka 0.9 and later.
> See also:
> http://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines
> An HBase source
> (http://docs.confluent.io/2.0.0/connect/devguide.html#task-example-source-task)
> could be implemented as a replication endpoint or WALObserver, publishing
> cluster wide change streams from the WAL to one or more topics, with
> configurable mapping and partitioning of table changes to topics.
> An HBase sink task
> (http://docs.confluent.io/2.0.0/connect/devguide.html#sink-tasks) would
> persist, with optional transformation (JSON? Avro?, map fields to native
> schema?), Kafka SinkRecords into HBase tables.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20369) Document incompatibilities between HBase 1.x and HBase 2.0
[ https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517742#comment-16517742 ] Thiriguna Bharat Rao commented on HBASE-20369: -- Many thanks [~elserj] for the feedback and continuous support. I highly appreciate it. Best Regards, Triguna > Document incompatibilities between HBase 1.x and HBase 2.0 > -- > > Key: HBASE-20369 > URL: https://issues.apache.org/jira/browse/HBASE-20369 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Thiriguna Bharat Rao >Assignee: Thiriguna Bharat Rao >Priority: Critical > Labels: patch > Fix For: 3.0.0 > > Attachments: HBASE-20369.patch, HBASE-20369_v1.patch, > HBASE-20369_v2.patch, HBASE-20369_v3.patch, HBASE-20369_v4.patch, > HBASE-20369_v5.patch, HBASE-20369_v6.patch, HBASE-20369_v7.patch, > HBASE-20369_v8.patch, HBASE-20369_v9.patch, HBase-20369_v10.patch, book.adoc > > > Hi, > I compiled a draft document for the HBase incompatibilities from the raw > source content that was available in HBase Beta 1 site. Can someone please > review and provide a feedback or share your comments on this document? > Appreciate your support and time. > > Best Regards, > Triguna -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20720) Make 2.0.1RC0
[ https://issues.apache.org/jira/browse/HBASE-20720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517736#comment-16517736 ] stack commented on HBASE-20720: --- Forgot to include this step to include report in RC vote: $ ./dev-support/checkcompatibility.py --annotation org.apache.hadoop.hbase.classification.InterfaceAudience.Public rel/2.0.0 rel/2.0.1 > Make 2.0.1RC0 > - > > Key: HBASE-20720 > URL: https://issues.apache.org/jira/browse/HBASE-20720 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.1 > > > Let me push out a release off branch-2.0, a 2.0.1. It has a bunch of fixes > and some perf improvements. A nightly run just passed clean: > https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-2.0/421/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20745) Log when master proc wal rolls
[ https://issues.apache.org/jira/browse/HBASE-20745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20745: -- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to branch-2.0+. Just a bit of extra output showing when Master Proc WALs roll. > Log when master proc wal rolls > -- > > Key: HBASE-20745 > URL: https://issues.apache.org/jira/browse/HBASE-20745 > Project: HBase > Issue Type: Sub-task > Components: debugging >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.2 > > Attachments: HBASE-20745.master.001.patch > > > Emit when we roll master proc WAL so can see when they happen. Want to > correlate instances of corruption w/ events on Master. Currently hard to do > on a server where log-level is INFO (default for many deploys). > Also, we log STUCK regions every 5 seconds. If a bundle of regions get stuck, > we can log so frequently, we roll away where the problem happened so lose the > chance to debug. Let me fix that too > Need both debugging instances of parent issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19735) Create a minimal "client" tarball installation
[
https://issues.apache.org/jira/browse/HBASE-19735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517688#comment-16517688
]
Hudson commented on HBASE-19735:
Results for branch master
[build #370 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/370/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Create a minimal "client" tarball installation
> --
>
> Key: HBASE-19735
> URL: https://issues.apache.org/jira/browse/HBASE-19735
> Project: HBase
> Issue Type: New Feature
> Components: build, Client
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-19735.000.patch, HBASE-19735.001.branch-2.patch,
> HBASE-19735.002.branch-2.patch, HBASE-19735.003.patch, HBASE-19735.004.patch
>
>
> We're moving ourselves towards more controlled dependencies. A logical next
> step is to try to do the same for our "binary" artifacts that we create
> during releases.
> There is code (our's and our dependency's) which the HMaster and RegionServer
> require which, obviously, clients do not need.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20332) shaded mapreduce module shouldn't include hadoop
[
https://issues.apache.org/jira/browse/HBASE-20332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517686#comment-16517686
]
Hudson commented on HBASE-20332:
Results for branch master
[build #370 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/370/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> shaded mapreduce module shouldn't include hadoop
>
>
> Key: HBASE-20332
> URL: https://issues.apache.org/jira/browse/HBASE-20332
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, shading
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20332.0.patch, HBASE-20332.1.WIP.patch,
> HBASE-20332.2.WIP.patch, HBASE-20332.3.patch, HBASE-20332.4.patch,
> HBASE-20332.5.patch, HBASE-20332.6.patch, HBASE-20332.7.patch
>
>
> AFAICT, we should just entirely skip including hadoop in our shaded mapreduce
> module
> 1) Folks expect to run yarn / mr apps via {{hadoop jar}} / {{yarn jar}}
> 2) those commands include all the needed Hadoop jars in your classpath by
> default (both client side and in the containers)
> 3) If you try to use "user classpath first" for your job as a workaround
> (e.g. for some library your application needs that hadoop provides) then our
> inclusion of *some but not all* hadoop classes then causes everything to fall
> over because of mixing rewritten and non-rewritten hadoop classes
> 4) if you don't use "user classpath first" then all of our
> non-relocated-but-still-shaded hadoop classes are ignored anyways so we're
> just wasting space
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20727) Persist FlushedSequenceId to speed up WAL split after cluster restart
[
https://issues.apache.org/jira/browse/HBASE-20727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517691#comment-16517691
]
Hudson commented on HBASE-20727:
Results for branch master
[build #370 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/370/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Persist FlushedSequenceId to speed up WAL split after cluster restart
> -
>
> Key: HBASE-20727
> URL: https://issues.apache.org/jira/browse/HBASE-20727
> Project: HBase
> Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20727.002.patch, HBASE-20727.003.patch,
> HBASE-20727.004.patch, HBASE-20727.005.patch, HBASE-20727.patch
>
>
> We use flushedSequenceIdByRegion and storeFlushedSequenceIdsByRegion in
> ServerManager to record the latest flushed seqids of regions and stores. So
> during log split, we can use seqids stored in those maps to filter out the
> edits which do not need to be replayed. But, those maps are not persisted.
> After cluster restart or master restart, info of flushed seqids are all lost.
> Here I offer a way to persist those info to HDFS, even if master restart, we
> can still use those info to filter WAL edits and then to speed up replay.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20334) add a test that expressly uses both our shaded client and the one from hadoop 3
[
https://issues.apache.org/jira/browse/HBASE-20334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517690#comment-16517690
]
Hudson commented on HBASE-20334:
Results for branch master
[build #370 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/370/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> add a test that expressly uses both our shaded client and the one from hadoop
> 3
> ---
>
> Key: HBASE-20334
> URL: https://issues.apache.org/jira/browse/HBASE-20334
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop3, shading
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20334.0.patch, HBASE-20334.1.patch
>
>
> Since we're making a shaded client that bleed out of our namespace and into
> Hadoop's, we should ensure that we can show our clients coexisting. Even if
> it's just an IT that successfully talks to both us and HDFS via our
> respective shaded clients, that'd be a big help in keeping us proactive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20333) break up shaded client into one with no Hadoop and one that's standalone
[
https://issues.apache.org/jira/browse/HBASE-20333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517687#comment-16517687
]
Hudson commented on HBASE-20333:
Results for branch master
[build #370 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/370/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> break up shaded client into one with no Hadoop and one that's standalone
>
>
> Key: HBASE-20333
> URL: https://issues.apache.org/jira/browse/HBASE-20333
> Project: HBase
> Issue Type: Sub-task
> Components: shading
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20333.1.patch, HBASE-20333.WIP.0.patch
>
>
> there are contexts where we want to stay out of our downstream users way wrt
> dependencies, but they need more Hadoop classes than we provide. i.e. any
> downstream client that wants to use both HBase and HDFS in their application,
> or any non-MR YARN application.
> Now that Hadoop also has shaded client artifacts for Hadoop 3, we're also
> providing less incremental benefit by including our own rewritten Hadoop
> classes to avoid downstream needing to pull in all of Hadoop's transitive
> dependencies.
> right now those users need to ensure that any jars from the Hadoop project
> are loaded in the classpath prior to our shaded client jar. This is brittle
> and prone to weird debugging trouble.
> instead, we should have two artifacts: one that just lists Hadoop as a
> prerequisite and one that still includes the rewritten-but-not-relocated
> Hadoop classes.
> We can then use docs to emphasize when each of these is appropriate to use.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20708) Remove the usage of RecoverMetaProcedure in master startup
[
https://issues.apache.org/jira/browse/HBASE-20708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517692#comment-16517692
]
Hudson commented on HBASE-20708:
Results for branch master
[build #370 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/370/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove the usage of RecoverMetaProcedure in master startup
> --
>
> Key: HBASE-20708
> URL: https://issues.apache.org/jira/browse/HBASE-20708
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20708-v1.patch, HBASE-20708-v2.patch,
> HBASE-20708-v3.patch, HBASE-20708-v4.patch, HBASE-20708-v5.patch,
> HBASE-20708-v6.patch, HBASE-20708-v7.patch, HBASE-20708-v8.patch,
> HBASE-20708-v9.patch, HBASE-20708-v9.patch, HBASE-20708.patch
>
>
> In HBASE-20700, we make RecoverMetaProcedure use a special lock which is only
> used by RMP to avoid dead lock with MoveRegionProcedure. But we will always
> schedule a RMP when master starting up, so we still need to make sure that
> there is no race between this RMP and other RMPs and SCPs scheduled before
> the master restarts.
> Please see [#[accompanying design document
> |https://docs.google.com/document/d/1_872oHzrhJq4ck7f6zmp1J--zMhsIFvXSZyX1Mxg5MA/edit#heading=h.xy1z4alsq7uy]
> ]where we call out the problem being addressed by this issue in more detail
> and in which we describe our new approach to Master startup.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20615) emphasize use of shaded client jars when they're present in an install
[
https://issues.apache.org/jira/browse/HBASE-20615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517689#comment-16517689
]
Hudson commented on HBASE-20615:
Results for branch master
[build #370 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/370/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> emphasize use of shaded client jars when they're present in an install
> --
>
> Key: HBASE-20615
> URL: https://issues.apache.org/jira/browse/HBASE-20615
> Project: HBase
> Issue Type: Sub-task
> Components: build, Client, Usability
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20615.0.patch, HBASE-20615.1.patch,
> HBASE-20615.2.patch
>
>
> Working through setting up an IT for our shaded artifacts in HBASE-20334
> makes our lack of packaging seem like an oversight. While I could work around
> by pulling the shaded clients out of whatever build process built the
> convenience binary that we're trying to test, it seems v awkward.
> After reflecting on it more, it makes more sense to me for there to be a
> common place in the install that folks running jobs against the cluster can
> rely on. If they need to run without a full hbase install, that should still
> work fine via e.g. grabbing from the maven repo.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20369) Document incompatibilities between HBase 1.x and HBase 2.0
[
https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517693#comment-16517693
]
Hudson commented on HBASE-20369:
Results for branch master
[build #370 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/370/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/370//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Document incompatibilities between HBase 1.x and HBase 2.0
> --
>
> Key: HBASE-20369
> URL: https://issues.apache.org/jira/browse/HBASE-20369
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Thiriguna Bharat Rao
>Assignee: Thiriguna Bharat Rao
>Priority: Critical
> Labels: patch
> Fix For: 3.0.0
>
> Attachments: HBASE-20369.patch, HBASE-20369_v1.patch,
> HBASE-20369_v2.patch, HBASE-20369_v3.patch, HBASE-20369_v4.patch,
> HBASE-20369_v5.patch, HBASE-20369_v6.patch, HBASE-20369_v7.patch,
> HBASE-20369_v8.patch, HBASE-20369_v9.patch, HBase-20369_v10.patch, book.adoc
>
>
> Hi,
> I compiled a draft document for the HBase incompatibilities from the raw
> source content that was available in HBase Beta 1 site. Can someone please
> review and provide a feedback or share your comments on this document?
> Appreciate your support and time.
>
> Best Regards,
> Triguna
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20747) Cut branch-2.1
[ https://issues.apache.org/jira/browse/HBASE-20747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517666#comment-16517666 ] Duo Zhang commented on HBASE-20747: --- Skimmed the issues which fix versions contains 2.1.0 and moved a bunch of them out of 2.1.0. Plan to cut branch-2.1 at the end of this week. > Cut branch-2.1 > -- > > Key: HBASE-20747 > URL: https://issues.apache.org/jira/browse/HBASE-20747 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20691) Storage policy should allow deferring to HDFS
[
https://issues.apache.org/jira/browse/HBASE-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517662#comment-16517662
]
Sean Busbey commented on HBASE-20691:
-
CommonFSUtils should be updated to check against
{{DEFER_TO_HDFS_STORAGE_POLICY}} rather than against a passed in default of
{{DEFAULT_WAL_STORAGE_POLICY}}. They're the same thing now, but they won't
necessarily be.
Since the {{setStoragePolicy}} code is in CommonFSUtils, the test should be in
TestCommonFSUtils.
{code}
354 LOG.debug("Before set storage policy to NONE");
355 FSUtils.setStoragePolicy(testFs, conf, new Path("non-exist"),
HConstants.WAL_STORAGE_POLICY,
356 HConstants.DEFAULT_WAL_STORAGE_POLICY);
357 LOG.debug("After set storage policy to NONE");
358 conf.set(HConstants.WAL_STORAGE_POLICY, "HOT");
359 // warning log is expected when passing some valid policy
360 FSUtils.setStoragePolicy(testFs, conf, new Path("non-exist"),
HConstants.WAL_STORAGE_POLICY,
361 HConstants.DEFAULT_WAL_STORAGE_POLICY);
{code}
Shouldn't this second invocation have thrown an IOException?
> Storage policy should allow deferring to HDFS
> -
>
> Key: HBASE-20691
> URL: https://issues.apache.org/jira/browse/HBASE-20691
> Project: HBase
> Issue Type: Bug
> Components: Filesystem Integration, wal
>Affects Versions: 1.5.0, 2.0.0
>Reporter: Sean Busbey
>Assignee: Yu Li
>Priority: Minor
> Fix For: 2.1.0, 1.5.0
>
> Attachments: HBASE-20691.patch, HBASE-20691.v2.patch,
> HBASE-20691.v3.patch
>
>
> In HBase 1.1 - 1.4 we can defer storage policy decisions to HDFS by using
> "NONE" as the storage policy in hbase configs.
> As described on this [dev@hbase thread "WAL storage policies and interactions
> with Hadoop admin
> tools."|https://lists.apache.org/thread.html/d220726fab4bb4c9e117ecc8f44246402dd97bfc986a57eb2237@%3Cdev.hbase.apache.org%3E]
> we no longer have that option in 2.0.0 and 1.5.0 (as the branch is now).
> Additionally, we can't set the policy to HOT in the event that HDFS has
> changed the policy for a parent directory of our WALs.
> We should put back that ability. Presuming this is done by re-adopting the
> "NONE" placeholder variable, we need to ensure that value doesn't get passed
> to HDFS APIs. Since it isn't a valid storage policy attempting to use it will
> cause a bunch of logging churn (which will be a regression of the problem
> HBASE-18118 sought to fix).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-15320) HBase connector for Kafka Connect
[
https://issues.apache.org/jira/browse/HBASE-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517655#comment-16517655
]
Hadoop QA commented on HBASE-15320:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
4s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
27s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-assembly . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 7m 51s{color}
| {color:red} root generated 104 new + 1299 unchanged - 0 fixed = 1403 total
(was 1299) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
1s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 9 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
8s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 0m
13s{color} | {color:red} patch has 7 errors when building our shaded downstream
artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 6s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-kafka-model hbase-assembly . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}206m 12s{color}
| {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 2m
1s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}267m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce
[jira] [Work started] (HBASE-20448) update ref guide to expressly use shaded clients for examples
[ https://issues.apache.org/jira/browse/HBASE-20448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-20448 started by Sean Busbey. --- > update ref guide to expressly use shaded clients for examples > - > > Key: HBASE-20448 > URL: https://issues.apache.org/jira/browse/HBASE-20448 > Project: HBase > Issue Type: Sub-task > Components: Client, documentation, mapreduce >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 2.1.0 > > > the whole mapreduce section, especially, should be using the shaded version. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20448) update ref guide to expressly use shaded clients for examples
[ https://issues.apache.org/jira/browse/HBASE-20448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reassigned HBASE-20448: --- Assignee: Sean Busbey > update ref guide to expressly use shaded clients for examples > - > > Key: HBASE-20448 > URL: https://issues.apache.org/jira/browse/HBASE-20448 > Project: HBase > Issue Type: Sub-task > Components: Client, documentation, mapreduce >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 2.1.0 > > > the whole mapreduce section, especially, should be using the shaded version. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20448) update ref guide to expressly use shaded clients for examples
[ https://issues.apache.org/jira/browse/HBASE-20448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20448: Fix Version/s: 2.1.0 > update ref guide to expressly use shaded clients for examples > - > > Key: HBASE-20448 > URL: https://issues.apache.org/jira/browse/HBASE-20448 > Project: HBase > Issue Type: Sub-task > Components: Client, documentation, mapreduce >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 2.1.0 > > > the whole mapreduce section, especially, should be using the shaded version. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HBASE-20331) clean up shaded packaging for 2.1
[ https://issues.apache.org/jira/browse/HBASE-20331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-20331 started by Sean Busbey. --- > clean up shaded packaging for 2.1 > - > > Key: HBASE-20331 > URL: https://issues.apache.org/jira/browse/HBASE-20331 > Project: HBase > Issue Type: Umbrella > Components: Client, mapreduce, shading >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 2.1.0 > > > polishing pass on shaded modules for 2.0 based on trying to use them in more > contexts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20369) Document incompatibilities between HBase 1.x and HBase 2.0
[ https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-20369: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Thanks for the documentation improvement, with all of the back and forth, Triguna!! > Document incompatibilities between HBase 1.x and HBase 2.0 > -- > > Key: HBASE-20369 > URL: https://issues.apache.org/jira/browse/HBASE-20369 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Thiriguna Bharat Rao >Assignee: Thiriguna Bharat Rao >Priority: Critical > Labels: patch > Fix For: 3.0.0 > > Attachments: HBASE-20369.patch, HBASE-20369_v1.patch, > HBASE-20369_v2.patch, HBASE-20369_v3.patch, HBASE-20369_v4.patch, > HBASE-20369_v5.patch, HBASE-20369_v6.patch, HBASE-20369_v7.patch, > HBASE-20369_v8.patch, HBASE-20369_v9.patch, HBase-20369_v10.patch, book.adoc > > > Hi, > I compiled a draft document for the HBase incompatibilities from the raw > source content that was available in HBase Beta 1 site. Can someone please > review and provide a feedback or share your comments on this document? > Appreciate your support and time. > > Best Regards, > Triguna -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20369) Document incompatibilities between HBase 1.x and HBase 2.0
[
https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517494#comment-16517494
]
Josh Elser commented on HBASE-20369:
{quote}Incorporated the feedback and uploaded the changes in v10 patch. Many
thanks.
{quote}
It looks like there was a missing line (copy-paste error, I guess). I can fix
this on commit, with the tab characters as well that QA has flagged.
> Document incompatibilities between HBase 1.x and HBase 2.0
> --
>
> Key: HBASE-20369
> URL: https://issues.apache.org/jira/browse/HBASE-20369
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Thiriguna Bharat Rao
>Assignee: Thiriguna Bharat Rao
>Priority: Critical
> Labels: patch
> Attachments: HBASE-20369.patch, HBASE-20369_v1.patch,
> HBASE-20369_v2.patch, HBASE-20369_v3.patch, HBASE-20369_v4.patch,
> HBASE-20369_v5.patch, HBASE-20369_v6.patch, HBASE-20369_v7.patch,
> HBASE-20369_v8.patch, HBASE-20369_v9.patch, HBase-20369_v10.patch, book.adoc
>
>
> Hi,
> I compiled a draft document for the HBase incompatibilities from the raw
> source content that was available in HBase Beta 1 site. Can someone please
> review and provide a feedback or share your comments on this document?
> Appreciate your support and time.
>
> Best Regards,
> Triguna
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-15320) HBase connector for Kafka Connect
[ https://issues.apache.org/jira/browse/HBASE-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517429#comment-16517429 ] Mike Wingert commented on HBASE-15320: -- Update the patch so it compiles again > HBase connector for Kafka Connect > - > > Key: HBASE-15320 > URL: https://issues.apache.org/jira/browse/HBASE-15320 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Andrew Purtell >Assignee: Mike Wingert >Priority: Major > Labels: beginner > Fix For: 3.0.0 > > Attachments: HBASE-15320.master.1.patch, HBASE-15320.master.10.patch, > HBASE-15320.master.2.patch, HBASE-15320.master.3.patch, > HBASE-15320.master.4.patch, HBASE-15320.master.5.patch, > HBASE-15320.master.6.patch, HBASE-15320.master.7.patch, > HBASE-15320.master.8.patch, HBASE-15320.master.8.patch, > HBASE-15320.master.9.patch, HBASE-15320.pdf, HBASE-15320.pdf > > > Implement an HBase connector with source and sink tasks for the Connect > framework (http://docs.confluent.io/2.0.0/connect/index.html) available in > Kafka 0.9 and later. > See also: > http://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines > An HBase source > (http://docs.confluent.io/2.0.0/connect/devguide.html#task-example-source-task) > could be implemented as a replication endpoint or WALObserver, publishing > cluster wide change streams from the WAL to one or more topics, with > configurable mapping and partitioning of table changes to topics. > An HBase sink task > (http://docs.confluent.io/2.0.0/connect/devguide.html#sink-tasks) would > persist, with optional transformation (JSON? Avro?, map fields to native > schema?), Kafka SinkRecords into HBase tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15320) HBase connector for Kafka Connect
[ https://issues.apache.org/jira/browse/HBASE-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Wingert updated HBASE-15320: - Attachment: HBASE-15320.master.10.patch > HBase connector for Kafka Connect > - > > Key: HBASE-15320 > URL: https://issues.apache.org/jira/browse/HBASE-15320 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Andrew Purtell >Assignee: Mike Wingert >Priority: Major > Labels: beginner > Fix For: 3.0.0 > > Attachments: HBASE-15320.master.1.patch, HBASE-15320.master.10.patch, > HBASE-15320.master.2.patch, HBASE-15320.master.3.patch, > HBASE-15320.master.4.patch, HBASE-15320.master.5.patch, > HBASE-15320.master.6.patch, HBASE-15320.master.7.patch, > HBASE-15320.master.8.patch, HBASE-15320.master.8.patch, > HBASE-15320.master.9.patch, HBASE-15320.pdf, HBASE-15320.pdf > > > Implement an HBase connector with source and sink tasks for the Connect > framework (http://docs.confluent.io/2.0.0/connect/index.html) available in > Kafka 0.9 and later. > See also: > http://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines > An HBase source > (http://docs.confluent.io/2.0.0/connect/devguide.html#task-example-source-task) > could be implemented as a replication endpoint or WALObserver, publishing > cluster wide change streams from the WAL to one or more topics, with > configurable mapping and partitioning of table changes to topics. > An HBase sink task > (http://docs.confluent.io/2.0.0/connect/devguide.html#sink-tasks) would > persist, with optional transformation (JSON? Avro?, map fields to native > schema?), Kafka SinkRecords into HBase tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19064) Synchronous replication for HBase
[
https://issues.apache.org/jira/browse/HBASE-19064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517414#comment-16517414
]
Hudson commented on HBASE-19064:
Results for branch HBASE-19064
[build #166 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/166/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/166//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/166//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/166//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Synchronous replication for HBase
> -
>
> Key: HBASE-19064
> URL: https://issues.apache.org/jira/browse/HBASE-19064
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
> The guys from Alibaba made a presentation on HBaseCon Asia about the
> synchronous replication for HBase. We(Xiaomi) think this is a very useful
> feature for HBase so we want to bring it into the community version.
> This is a big feature so we plan to do it in a feature branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20740) StochasticLoadBalancer should consider CoprocessorService request factor when computing cost
[ https://issues.apache.org/jira/browse/HBASE-20740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenxu updated HBASE-20740: --- Description: When compute region load cost, ReadRequest, WriteRequest, MemStoreSize and StoreFileSize are considered, But CoprocessorService requests are ignored. In our KYLIN cluster, there only have CoprocessorService requests, and the cluster sometimes unbalanced. (was: When compute region load cost, ReadRequest, WriteRequest, MemStoreSize and StoreFileSize are considered, But CoprocessorService requests are not include. In our KYLIN cluster, there only have CoprocessorService requests, and the cluster sometimes unbalanced.) > StochasticLoadBalancer should consider CoprocessorService request factor when > computing cost > > > Key: HBASE-20740 > URL: https://issues.apache.org/jira/browse/HBASE-20740 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: chenxu >Assignee: chenxu >Priority: Major > Attachments: HBASE-20740-master-v1.patch > > > When compute region load cost, ReadRequest, WriteRequest, MemStoreSize and > StoreFileSize are considered, But CoprocessorService requests are ignored. In > our KYLIN cluster, there only have CoprocessorService requests, and the > cluster sometimes unbalanced. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20740) StochasticLoadBalancer should consider CoprocessorService request factor when computing cost
[ https://issues.apache.org/jira/browse/HBASE-20740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenxu updated HBASE-20740: --- Description: When compute region load cost, ReadRequest, WriteRequest, MemStoreSize and StoreFileSize are considered, But CoprocessorService requests are not include. In our KYLIN cluster, there only have CoprocessorService requests, and the cluster sometimes unbalanced. (was: When compute region load cost, ReadRequest, WriteRequest, MemStoreSize and StoreFileSize are considered, But CoprocessorRequests are not include. In our kylin cluster, there only have coprocessor requests, and the cluster sometimes unbalanced.) > StochasticLoadBalancer should consider CoprocessorService request factor when > computing cost > > > Key: HBASE-20740 > URL: https://issues.apache.org/jira/browse/HBASE-20740 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: chenxu >Assignee: chenxu >Priority: Major > Attachments: HBASE-20740-master-v1.patch > > > When compute region load cost, ReadRequest, WriteRequest, MemStoreSize and > StoreFileSize are considered, But CoprocessorService requests are not > include. In our KYLIN cluster, there only have CoprocessorService requests, > and the cluster sometimes unbalanced. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20740) StochasticLoadBalancer should consider CoprocessorService request factor when computing cost
[ https://issues.apache.org/jira/browse/HBASE-20740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenxu updated HBASE-20740: --- Summary: StochasticLoadBalancer should consider CoprocessorService request factor when computing cost (was: StochasticLoadBalancer should consider CoprocessorService request count when computing cost) > StochasticLoadBalancer should consider CoprocessorService request factor when > computing cost > > > Key: HBASE-20740 > URL: https://issues.apache.org/jira/browse/HBASE-20740 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: chenxu >Assignee: chenxu >Priority: Major > Attachments: HBASE-20740-master-v1.patch > > > When compute region load cost, ReadRequest, WriteRequest, MemStoreSize and > StoreFileSize are considered, But CoprocessorRequests are not include. In our > kylin cluster, there only have coprocessor requests, and the cluster > sometimes unbalanced. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20740) StochasticLoadBalancer should consider CoprocessorService request count when computing cost
[ https://issues.apache.org/jira/browse/HBASE-20740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenxu updated HBASE-20740: --- Summary: StochasticLoadBalancer should consider CoprocessorService request count when computing cost (was: StochasticLoadBalancer should consider Coprocessor request count when computing cost) > StochasticLoadBalancer should consider CoprocessorService request count when > computing cost > --- > > Key: HBASE-20740 > URL: https://issues.apache.org/jira/browse/HBASE-20740 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: chenxu >Assignee: chenxu >Priority: Major > Attachments: HBASE-20740-master-v1.patch > > > When compute region load cost, ReadRequest, WriteRequest, MemStoreSize and > StoreFileSize are considered, But CoprocessorRequests are not include. In our > kylin cluster, there only have coprocessor requests, and the cluster > sometimes unbalanced. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20708) Remove the usage of RecoverMetaProcedure in master startup
[
https://issues.apache.org/jira/browse/HBASE-20708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517252#comment-16517252
]
Hudson commented on HBASE-20708:
Results for branch branch-2
[build #880 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/880/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/880//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/880//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/880//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove the usage of RecoverMetaProcedure in master startup
> --
>
> Key: HBASE-20708
> URL: https://issues.apache.org/jira/browse/HBASE-20708
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20708-v1.patch, HBASE-20708-v2.patch,
> HBASE-20708-v3.patch, HBASE-20708-v4.patch, HBASE-20708-v5.patch,
> HBASE-20708-v6.patch, HBASE-20708-v7.patch, HBASE-20708-v8.patch,
> HBASE-20708-v9.patch, HBASE-20708-v9.patch, HBASE-20708.patch
>
>
> In HBASE-20700, we make RecoverMetaProcedure use a special lock which is only
> used by RMP to avoid dead lock with MoveRegionProcedure. But we will always
> schedule a RMP when master starting up, so we still need to make sure that
> there is no race between this RMP and other RMPs and SCPs scheduled before
> the master restarts.
> Please see [#[accompanying design document
> |https://docs.google.com/document/d/1_872oHzrhJq4ck7f6zmp1J--zMhsIFvXSZyX1Mxg5MA/edit#heading=h.xy1z4alsq7uy]
> ]where we call out the problem being addressed by this issue in more detail
> and in which we describe our new approach to Master startup.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20740) StochasticLoadBalancer should consider Coprocessor request count when computing cost
[
https://issues.apache.org/jira/browse/HBASE-20740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517250#comment-16517250
]
chenxu commented on HBASE-20740:
{quote}Please describe your use case in more detail: how does coprocessor
generate requests (if there is no request from clients).
{quote}
what i means is that only *RSRpcServices#execService* are called, and
cpRequestCount means coprocessor service request count.
When performing heartbeat, _RegionLoad_s are report to the _HMaster_, but it
only contains readRequestsCount and writeRequestsCount and no cpRequestsCount,
so the balancer ignored cpRequestsCount factor.
Thank you for review this [~yuzhih...@gmail.com]
[https://reviews.apache.org/r/67647/]
> StochasticLoadBalancer should consider Coprocessor request count when
> computing cost
>
>
> Key: HBASE-20740
> URL: https://issues.apache.org/jira/browse/HBASE-20740
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
>Reporter: chenxu
>Assignee: chenxu
>Priority: Major
> Attachments: HBASE-20740-master-v1.patch
>
>
> When compute region load cost, ReadRequest, WriteRequest, MemStoreSize and
> StoreFileSize are considered, But CoprocessorRequests are not include. In our
> kylin cluster, there only have coprocessor requests, and the cluster
> sometimes unbalanced.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-15654) Optimize client's MetaCache handling
[ https://issues.apache.org/jira/browse/HBASE-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517217#comment-16517217 ] stack commented on HBASE-15654: --- Meta perf is coming up more and more. In hbase2 we go to meta more for stuff like table state. Master does not currently cache the last thing read from meta though it is the only writer so meta is hit harder in hbase2. need to start working through these issues. > Optimize client's MetaCache handling > > > Key: HBASE-15654 > URL: https://issues.apache.org/jira/browse/HBASE-15654 > Project: HBase > Issue Type: Umbrella > Components: Client >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Critical > Fix For: 3.0.0, 1.5.0, 2.2.0 > > > This is an umbrella jira to track all individual issues, bugfixes and small > optimizations around MetaCache (region locations cache) in the client. > Motivation is that under the load one could see a spikes in the number of > requests going to meta - reaching tens of thousands requests per second. > That covers issues when we clear entries from location cache unnecessary, as > well as when we do more lookups than necessary when entries are legitimately > evicted. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15654) Optimize client's MetaCache handling
[ https://issues.apache.org/jira/browse/HBASE-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15654: -- Priority: Critical (was: Major) > Optimize client's MetaCache handling > > > Key: HBASE-15654 > URL: https://issues.apache.org/jira/browse/HBASE-15654 > Project: HBase > Issue Type: Umbrella > Components: Client >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Critical > Fix For: 3.0.0, 1.5.0, 2.2.0 > > > This is an umbrella jira to track all individual issues, bugfixes and small > optimizations around MetaCache (region locations cache) in the client. > Motivation is that under the load one could see a spikes in the number of > requests going to meta - reaching tens of thousands requests per second. > That covers issues when we clear entries from location cache unnecessary, as > well as when we do more lookups than necessary when entries are legitimately > evicted. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15814) Miss important information in Document of HBase Security
[
https://issues.apache.org/jira/browse/HBASE-15814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-15814:
--
Priority: Critical (was: Major)
> Miss important information in Document of HBase Security
>
>
> Key: HBASE-15814
> URL: https://issues.apache.org/jira/browse/HBASE-15814
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Heng Chen
>Priority: Critical
>
> I have deployed secure cluster recently, and found we miss important
> information in http://hbase.apache.org/book.html#security
> Some configurations like
> {code}
>
> hbase.regionserver.kerberos.principal
> hbase/_h...@your-realm.com
>
>
> hbase.regionserver.keytab.file
> /etc/hbase/conf/hbase.keytab
>
>
> hbase.master.kerberos.principal
> hbase/_h...@your-realm.com
>
>
> hbase.master.keytab.file
> /etc/hbase/conf/hbase.keytab
>
> {code}
> And i found more detailed document in
> http://www.cloudera.com/documentation/enterprise/5-5-x/topics/cdh_sg_hbase_authentication.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20708) Remove the usage of RecoverMetaProcedure in master startup
[ https://issues.apache.org/jira/browse/HBASE-20708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20708: -- Release Note: Introduce an InitMetaProcedure to initialize meta table for a new HBase deploy. Marked RecoverMetaProcedure deprecated and remove the usage of it in the current code base. We still need to keep it in place for compatibility. The code in RecoverMetaProcedure has been moved to ServerCrashProcedure, and SCP will always be enabled and we will rely on it to bring meta region online. For more on the issue addressed by this commit, see the design doc for overview and plan: https://docs.google.com/document/d/1_872oHzrhJq4ck7f6zmp1J--zMhsIFvXSZyX1Mxg5MA/edit#heading=h.xy1z4alsq7uy was: Introduce an InitMetaProcedure to initialize meta table for a new HBase deploy. Marked RecoverMetaProcedure deprecated and remove the usage of it in the current code base. We still need to keep it in place for compatibility. The code in RecoverMetaProcedure has been moved to ServerCrashProcedure, and SCP will always be enabled and we will rely on it to bring meta region online. > Remove the usage of RecoverMetaProcedure in master startup > -- > > Key: HBASE-20708 > URL: https://issues.apache.org/jira/browse/HBASE-20708 > Project: HBase > Issue Type: Bug > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20708-v1.patch, HBASE-20708-v2.patch, > HBASE-20708-v3.patch, HBASE-20708-v4.patch, HBASE-20708-v5.patch, > HBASE-20708-v6.patch, HBASE-20708-v7.patch, HBASE-20708-v8.patch, > HBASE-20708-v9.patch, HBASE-20708-v9.patch, HBASE-20708.patch > > > In HBASE-20700, we make RecoverMetaProcedure use a special lock which is only > used by RMP to avoid dead lock with MoveRegionProcedure. But we will always > schedule a RMP when master starting up, so we still need to make sure that > there is no race between this RMP and other RMPs and SCPs scheduled before > the master restarts. > Please see [#[accompanying design document > |https://docs.google.com/document/d/1_872oHzrhJq4ck7f6zmp1J--zMhsIFvXSZyX1Mxg5MA/edit#heading=h.xy1z4alsq7uy] > ]where we call out the problem being addressed by this issue in more detail > and in which we describe our new approach to Master startup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20708) Remove the usage of RecoverMetaProcedure in master startup
[ https://issues.apache.org/jira/browse/HBASE-20708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20708: -- Release Note: Introduce an InitMetaProcedure to initialize meta table for a new HBase deploy. Marked RecoverMetaProcedure deprecated and remove the usage of it in the current code base. We still need to keep it in place for compatibility. The code in RecoverMetaProcedure has been moved to ServerCrashProcedure, and SCP will always be enabled and we will rely on it to bring meta region online. was:Introduce an InitMetaProcedure to initialize meta table for a new HBase deploy. Marked RecoverMetaProcedure deprecated and remove the usage of it in the current code base. We still need to keep it in place for compatibility. The code in RecoverMetaProcedure has been moved to ServerCrashProcedure, and SCP will always be enabled and we will rely on it to bring meta region online. > Remove the usage of RecoverMetaProcedure in master startup > -- > > Key: HBASE-20708 > URL: https://issues.apache.org/jira/browse/HBASE-20708 > Project: HBase > Issue Type: Bug > Components: proc-v2, Region Assignment >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20708-v1.patch, HBASE-20708-v2.patch, > HBASE-20708-v3.patch, HBASE-20708-v4.patch, HBASE-20708-v5.patch, > HBASE-20708-v6.patch, HBASE-20708-v7.patch, HBASE-20708-v8.patch, > HBASE-20708-v9.patch, HBASE-20708-v9.patch, HBASE-20708.patch > > > In HBASE-20700, we make RecoverMetaProcedure use a special lock which is only > used by RMP to avoid dead lock with MoveRegionProcedure. But we will always > schedule a RMP when master starting up, so we still need to make sure that > there is no race between this RMP and other RMPs and SCPs scheduled before > the master restarts. > Please see [#[accompanying design document > |https://docs.google.com/document/d/1_872oHzrhJq4ck7f6zmp1J--zMhsIFvXSZyX1Mxg5MA/edit#heading=h.xy1z4alsq7uy] > ]where we call out the problem being addressed by this issue in more detail > and in which we describe our new approach to Master startup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20542) Better heap utilization for IMC with MSLABs
[ https://issues.apache.org/jira/browse/HBASE-20542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517160#comment-16517160 ] Eshcar Hillel commented on HBASE-20542: --- bq. Is it possible to see the patch on the review board? Thanks! I was having some trouble before, but now it is available. > Better heap utilization for IMC with MSLABs > --- > > Key: HBASE-20542 > URL: https://issues.apache.org/jira/browse/HBASE-20542 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Attachments: HBASE-20542.branch-2.001.patch, run.sh, workloada, > workloadc, workloadx, workloady > > > Following HBASE-20188 we realized in-memory compaction combined with MSLABs > may suffer from heap under-utilization due to internal fragmentation. This > jira presents a solution to circumvent this problem. The main idea is to have > each update operation check if it will cause overflow in the active segment > *before* it is writing the new value (instead of checking the size after the > write is completed), and if it is then the active segment is atomically > swapped with a new empty segment, and is pushed (full-yet-not-overflowed) to > the compaction pipeline. Later on the IMC deamon will run its compaction > operation (flatten index/merge indices/data compaction) in the background. > Some subtle concurrency issues should be handled with care. We next elaborate > on them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20300) Minor refactor for RpcExecutor
[
https://issues.apache.org/jira/browse/HBASE-20300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517127#comment-16517127
]
Hadoop QA commented on HBASE-20300:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-20300 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-20300 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12916749/HBASE-20300.v0.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13314/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Minor refactor for RpcExecutor
> --
>
> Key: HBASE-20300
> URL: https://issues.apache.org/jira/browse/HBASE-20300
> Project: HBase
> Issue Type: Task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-20300.v0.patch, HBASE-20300.v0.patch.patch
>
>
> Plan to do the following changes.
> # make Handler be static class
> # move the threadlocal variables of MonitoredRPCHandler from RpcServer to
> FifoRpcScheduler since only FifoRpcScheduler use it
> # create MonitoredRPCHandler in Handler constuction instead of saving the
> MonitoredRPCHandler in threadlocal variables. In FPBQ mode, the web UI can
> display all Handlers info even if the rpc Handlers are not used yet.
> # Threshhold -> Threshold
> # make RpcExecutor extend ConfigurationObserver
> # don't create task filter repeatly
> # add a ut to check whether each Handler has created own MonitoredTask even
> if no ops
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20064) Disable MOB threads that are running whether you MOB or not
[
https://issues.apache.org/jira/browse/HBASE-20064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517126#comment-16517126
]
Hadoop QA commented on HBASE-20064:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-20064 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-20064 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12912215/HBASE-20064.master.002.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13311/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Disable MOB threads that are running whether you MOB or not
> ---
>
> Key: HBASE-20064
> URL: https://issues.apache.org/jira/browse/HBASE-20064
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Reid Chan
>Priority: Major
> Fix For: 2.2.0
>
> Attachments: HBASE-20064.master.001.patch,
> HBASE-20064.master.002.patch
>
>
> Master starts up some cleaner and compacting threads even though no MOB.
> Disable them and have users explicitly enable it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19452) Turn ON off heap Bucket Cache by default
[
https://issues.apache.org/jira/browse/HBASE-19452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517125#comment-16517125
]
Hadoop QA commented on HBASE-19452:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HBASE-19452 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-19452 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12901671/HBASE-19452_V3.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13313/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Turn ON off heap Bucket Cache by default
>
>
> Key: HBASE-19452
> URL: https://issues.apache.org/jira/browse/HBASE-19452
> Project: HBase
> Issue Type: Sub-task
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Major
> Fix For: 2.2.0
>
> Attachments: HBASE-19452.patch, HBASE-19452_V2.patch,
> HBASE-19452_V3.patch
>
>
> BC's hbase.bucketcache.ioengine by default is empty now. Means now BC.
> Make this default to be 'offheap'. And the default off heap size for the BC
> also to be provided. This can be 8 GB? Better to make it also a % of the
> Xmx. Lets continue to be 40% of Xmx as LRU cache default size.
> When user has to disable BC, configure size as 0. An empty value of this
> config will be treated as default size.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20741) Split of a region with replicas creates all daughter regions and its replica in same server
[ https://issues.apache.org/jira/browse/HBASE-20741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20741: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Split of a region with replicas creates all daughter regions and its replica > in same server > --- > > Key: HBASE-20741 > URL: https://issues.apache.org/jira/browse/HBASE-20741 > Project: HBase > Issue Type: Bug > Components: read replicas >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 2.2.0 > > > Generally it is better that the parent region when split creates the daughter > region in the same target server. > But for replicas also we do the same and all the replica regions are created > in the same target server. We should ideally be doing a round robin and only > the primary daughter region should be opened in the intended target server > (where the parent was previously opened). > [~huaxiang] FYI. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20691) Storage policy should allow deferring to HDFS
[ https://issues.apache.org/jira/browse/HBASE-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517116#comment-16517116 ] Duo Zhang commented on HBASE-20691: --- Any updates here? Seems the patch is ready. > Storage policy should allow deferring to HDFS > - > > Key: HBASE-20691 > URL: https://issues.apache.org/jira/browse/HBASE-20691 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, wal >Affects Versions: 1.5.0, 2.0.0 >Reporter: Sean Busbey >Assignee: Yu Li >Priority: Minor > Fix For: 2.1.0, 1.5.0 > > Attachments: HBASE-20691.patch, HBASE-20691.v2.patch, > HBASE-20691.v3.patch > > > In HBase 1.1 - 1.4 we can defer storage policy decisions to HDFS by using > "NONE" as the storage policy in hbase configs. > As described on this [dev@hbase thread "WAL storage policies and interactions > with Hadoop admin > tools."|https://lists.apache.org/thread.html/d220726fab4bb4c9e117ecc8f44246402dd97bfc986a57eb2237@%3Cdev.hbase.apache.org%3E] > we no longer have that option in 2.0.0 and 1.5.0 (as the branch is now). > Additionally, we can't set the policy to HOT in the event that HDFS has > changed the policy for a parent directory of our WALs. > We should put back that ability. Presuming this is done by re-adopting the > "NONE" placeholder variable, we need to ensure that value doesn't get passed > to HDFS APIs. Since it isn't a valid storage policy attempting to use it will > cause a bunch of logging churn (which will be a regression of the problem > HBASE-18118 sought to fix). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20694) Consolidate warning on SecureBulkLoad directory permissions
[ https://issues.apache.org/jira/browse/HBASE-20694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20694: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Consolidate warning on SecureBulkLoad directory permissions > --- > > Key: HBASE-20694 > URL: https://issues.apache.org/jira/browse/HBASE-20694 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Minor > Fix For: 3.0.0, 1.5.0, 1.2.7, 1.3.3, 1.4.6, 2.2.0 > > > Follow-on from HBASE-20605: > HBase 1.x has a check which ignores a directory permission check if you're > using a specific filesystem which we think doesnt' do security properly. > HBase 2.x dropped this check. > Since the security of bulk-loaded data is dependent upon this directory > permission (and thus the capabilities of the FileSystem), it would be better > to have a consistent warning across branches. > [~busbey] suggested that we make a WARN message which points admins to our > Book (and write such a section if we don't have something sufficient already) > and our supported filesystems, coupled with an option to disable that warning > message. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20719) HTable.batch() doesn't handle TableNotFound correctly.
[
https://issues.apache.org/jira/browse/HBASE-20719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-20719:
--
Fix Version/s: (was: 2.1.0)
2.2.0
> HTable.batch() doesn't handle TableNotFound correctly.
> --
>
> Key: HBASE-20719
> URL: https://issues.apache.org/jira/browse/HBASE-20719
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.1.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Minor
> Fix For: 2.2.0
>
>
> batch() as well as delete() are processing using AsyncRequest. To report
> about problems we are using RetriesExhaustedWithDetailsException and there is
> no special handling for TableNotFound exception. So, the final result for
> running batch or delete operations against not existing table looks really
> weird and missleading:
> {noformat}
> hbase(main):003:0> delete 't1', 'r1', 'c1'
> 2018-06-12 15:02:50,742 ERROR [main] client.AsyncRequestFutureImpl: Cannot
> get replica 0 location for
> {"totalColumns":1,"row":"r1","families":{"c1":[{"qualifier":"","vlen":0,"tag":[],"timestamp":9223372036854775807}]},"ts":9223372036854775807}
> ERROR: Failed 1 action: t1: 1 time, servers with issues: null
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20643) Getting HDFSBlockDist in Master by querying RegionServers
[ https://issues.apache.org/jira/browse/HBASE-20643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20643: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Getting HDFSBlockDist in Master by querying RegionServers > - > > Key: HBASE-20643 > URL: https://issues.apache.org/jira/browse/HBASE-20643 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 1.5.0, 1.4.6, 2.2.0 > > > Region locality information is needed by the balancer to generate region > plans. Computing HDFSBlockDistribution is expensive on larger clusters and > adds load to the NameNode. This also needs to be recomputed on a master > restart. The proposal is to get the HDFSBlockDistribution from the > RegionServers instead of computing it in Master. RS already has this > information and we could just reuse it by querying it. RS already passes > dataLocality info via RegionLoad today. > Proposed Implementation: This is a high-level overview. > # A RegionServer API has to be added which will return HDFSBlockDistribution > for all the regions it hosts. RS already has this info. Since ClusterStatus > has already become bulky and we don’t need updated locality so fast, it’s > better to have another API rather than add this to RegionLoad and pass it > along with RSReport. > # Master will have a Chore to query all RegionServers and will cache the > HDFSBlockDistribution for those regions. This is easy and quick. Admins can > tune the frequency based on size of the cluster. On a ~90 nodes cluster with > 500k regions and a prototype implementation and no load, it took about 5 > seconds to get all HDFSBlockDistribution from RS. > # The cache will be an extension of RegionLocationFinder (subclass), if > needed to keep the implementation simple. Probably will get clear with > implementation. > # Balancer will use the new cache to get all HDFSBlockDistribution. If there > is a new region and Chore didn’t get the block distribution from RS during > its previous run, then it will be computed by RegionLocationFinder the same > way it has been done now. If the Chore runs more frequently like every hour, > then this recomputation will be drastically reduced. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20635) Support to convert the shaded user permission proto to client user permission object
[
https://issues.apache.org/jira/browse/HBASE-20635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517114#comment-16517114
]
Duo Zhang commented on HBASE-20635:
---
The patch is already in branch-2? No?
> Support to convert the shaded user permission proto to client user permission
> object
>
>
> Key: HBASE-20635
> URL: https://issues.apache.org/jira/browse/HBASE-20635
> Project: HBase
> Issue Type: Bug
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20635.patch, HBASE-20635_v2.patch,
> PHOENIX-4528_5.x-HBase-2.0_v2.patch
>
>
> Currently we have API to build the protobuf UserPermission to client user
> permission in AccessControlUtil but we cannot do the same when we use shaded
> protobufs.
> {noformat}
> /**
>* Converts a user permission proto to a client user permission object.
>*
>* @param proto the protobuf UserPermission
>* @return the converted UserPermission
>*/
> public static UserPermission
> toUserPermission(AccessControlProtos.UserPermission proto) {
> return new UserPermission(proto.getUser().toByteArray(),
> toTablePermission(proto.getPermission()));
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20623) Introduce the helper method "getCellBuilder()" to Mutation
[
https://issues.apache.org/jira/browse/HBASE-20623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-20623:
--
Fix Version/s: (was: 2.1.0)
2.2.0
> Introduce the helper method "getCellBuilder()" to Mutation
> --
>
> Key: HBASE-20623
> URL: https://issues.apache.org/jira/browse/HBASE-20623
> Project: HBase
> Issue Type: Task
> Components: API
>Reporter: Chia-Ping Tsai
>Assignee: maoling
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
>
> see
> [https://lists.apache.org/thread.html/d05bfaa0134502a47f6e1aca56cb0b096d4dd32ddefbbdf28db4952a@%3Cdev.hbase.apache.org%3E]
> for more details.
> {code:java}
> How about a "getCellBuilder" or "getCellBuilderFactory" method for
> Mutation implementations that gives you a CellBuilder instance that
> already has relevant parts set? Like for a Put instance it should be
> able to already have the Type and Row set.{code}
> mentioned a day or so ago by [~busbey]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20587) Replace Jackson with shaded thirdparty gson in client
[ https://issues.apache.org/jira/browse/HBASE-20587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517105#comment-16517105 ] Duo Zhang commented on HBASE-20587: --- Any updates here? > Replace Jackson with shaded thirdparty gson in client > - > > Key: HBASE-20587 > URL: https://issues.apache.org/jira/browse/HBASE-20587 > Project: HBase > Issue Type: Bug > Components: dependencies >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20587.001.patch > > > HBASE-20582 got me looking at how we use Jackson. It appears that we moved > some JSON code from hbase-server into hbase-common via HBASE-19053. But, > there seems to be no good reason why this code should live there and not in > hbase-http instead. Keeping Jackson off the user's classpath is a nice goal. > FYI [~appy], [~mdrob] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20538) TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
[ https://issues.apache.org/jira/browse/HBASE-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20538: -- Fix Version/s: (was: 2.1.0) 2.1.1 > TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: > Rejected by the jceks.key.serialFilter or jdk.serialFilter property > > > Key: HBASE-20538 > URL: https://issues.apache.org/jira/browse/HBASE-20538 > Project: HBase > Issue Type: Bug > Components: java, security >Reporter: stack >Priority: Critical > Fix For: 3.0.0, 2.0.1, 2.1.1 > > Attachments: > 0001-HBASE-20538-TestSaslFanOutOneBlockAsyncDFSOutput-DISABLE.patch > > > Infra must have updated our JDK over the weekend. The test > TestSaslFanOutOneBlockAsyncDFSOutput fails solidly since. > [~gabor.bota] ran into it already over in HDFS-13494 and provided nice > pointers as to cause. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20553) Add dependency CVE checking to nightly tests
[
https://issues.apache.org/jira/browse/HBASE-20553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-20553:
--
Fix Version/s: (was: 2.1.0)
2.2.0
> Add dependency CVE checking to nightly tests
>
>
> Key: HBASE-20553
> URL: https://issues.apache.org/jira/browse/HBASE-20553
> Project: HBase
> Issue Type: Umbrella
> Components: dependencies
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sakthi
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
>
> We should proactively work to flag dependencies with known CVEs so that we
> can then update them early in our development instead of near a release.
> YETUS-441 is working to add a plugin for this, we should grab a copy early to
> make sure it works for us.
> Rough outline:
> 1. [install yetus locally|http://yetus.apache.org/downloads/]
> 2. [install the dependency-check
> cli|https://www.owasp.org/index.php/OWASP_Dependency_Check] (homebrew
> instructions on right hand margin)
> 3. Get a local copy of the OWASP datafile ({{dependency-check --updateonly
> --data /some/local/path/to/dir}})
> 4. Run {{hbase_nightly_yetus.sh}} using matching environment variables from
> the “yetus general check” (currently [line #126 in our nightly
> Jenkinsfile|https://github.com/apache/hbase/blob/master/dev-support/Jenkinsfile#L126])
> 5. Grab the plugin definition and suppression file from from YETUS-441
> 6. put the plugin definition either in a directory of dev-support or into the
> hbase-personality.sh directly
> 7. Re-run {{hbase_nightly_yetus.sh}} to verify that the plugin results show
> up. (Probably this will involve adding new pointers for “where is the
> suppression file”, “where is the OWASP datafile” and pointing them somewhere
> locally.)
> Once all of that is in place we’ll get the changes needed into a branch that
> we can test out. Over in YETUS-441 I’ll need to add a jenkins job that’ll
> handle periodically updating a copy of the datafile for the OWASP dependency
> checker. Presuming I have that in place by the time we have a nightly branch
> to check this out, then we’ll also need to update our nightly Jenkinsfile to
> fetch the data file from that job.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20540) [umbrella] Hadoop 3 compatibility
[ https://issues.apache.org/jira/browse/HBASE-20540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20540: -- Fix Version/s: (was: 2.1.0) 2.1.1 > [umbrella] Hadoop 3 compatibility > - > > Key: HBASE-20540 > URL: https://issues.apache.org/jira/browse/HBASE-20540 > Project: HBase > Issue Type: Umbrella >Reporter: Duo Zhang >Priority: Major > Fix For: 2.0.1, 2.1.1 > > > There are known issues about the hadoop 3 compatibility for hbase 2. But > hadoop 3 is still not production ready. So we will link the issues here and > once there is a production ready hadoop 3 release, we will fix these issues > soon and upgrade our dependencies on hadoop, and also update the support > matrix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20516) Offheap read-path needs more detail
[ https://issues.apache.org/jira/browse/HBASE-20516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20516: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Offheap read-path needs more detail > --- > > Key: HBASE-20516 > URL: https://issues.apache.org/jira/browse/HBASE-20516 > Project: HBase > Issue Type: Task > Components: Offheaping >Reporter: stack >Priority: Major > Fix For: 2.2.0 > > Attachments: 20516.initial.patch.txt > > > Needs notes on what an operator should look for to see that all is on and > what to monitor in a running cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20455) [IHC] Workloadx
[
https://issues.apache.org/jira/browse/HBASE-20455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-20455:
--
Fix Version/s: (was: 2.1.0)
2.2.0
> [IHC] Workloadx
> ---
>
> Key: HBASE-20455
> URL: https://issues.apache.org/jira/browse/HBASE-20455
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
>
> I tried workloadx from parent issue, the ycsb workload that is meant to make
> IHC shine. I'm doing something wrong. I tried just plugging it in and
> doing this:
> {code}
> ycsb run hbase12 -P /home/stack/ycsb/workloads/workloadx -p table=ycsb
> -threads 48 -cp /home/stack/conf_hbase -p columnfamily=family -p
> clientbuffering=true -p writebuffersize=2097152 -s -p maxexecutiontime=1200
> -jvm-args=-Xmx8192
> m -Djava.security.egd=file:/dev/./urandom -p recordcount=2500 -p
> operationcount=2500 -p
> exportfile=logs/ycsb-workloadx-measurements-ve0524-20180419T02:58:14Z.json -p
> exporter=com.yahoo.ycsb.measurements.exporter.JSONArrayMeasurementsExporter
> {code}
> It completes in a minute and a half. Claims stuff like this for throughput:
> 236825 ops/second. In this case I have in-memory-compaction set to NONE. I
> was going to compare before and after.
> [~eshcar]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20413) IntegrationTestRegionReplicaReplication fails
[ https://issues.apache.org/jira/browse/HBASE-20413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20413: -- Fix Version/s: (was: 2.1.0) 2.2.0 > IntegrationTestRegionReplicaReplication fails > - > > Key: HBASE-20413 > URL: https://issues.apache.org/jira/browse/HBASE-20413 > Project: HBase > Issue Type: Bug > Components: test >Reporter: stack >Priority: Critical > Fix For: 3.0.0, 2.2.0 > > > Following [~psomogyi]'s inspiration, I've been running unit tests on a GCE > instance. IntegrationTestRegionReplicaReplication fails always for > branch-2.0. It fails for branch-1 too but for a different looking reason. On > branch-2.0, its OOME'ing. Downing the size of ingested data, it seems to hang > unable to update because it is unable to flush. > Marking this a blocker for now. Fix or disable it for branch-2.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20460) Doc offheap write-path
[ https://issues.apache.org/jira/browse/HBASE-20460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20460: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Doc offheap write-path > -- > > Key: HBASE-20460 > URL: https://issues.apache.org/jira/browse/HBASE-20460 > Project: HBase > Issue Type: Bug > Components: documentation, Offheaping >Reporter: stack >Priority: Critical > Fix For: 2.2.0 > > > We have an empty section in refguide that needs filling in on how to enable > offheap write-path, how to know you've set it up right or not, how to tune > it, and how it relates to direct memory allocation and offheap read-path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20511) Set the scan's read type on the subsequent scan requests
[ https://issues.apache.org/jira/browse/HBASE-20511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20511: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Set the scan's read type on the subsequent scan requests > > > Key: HBASE-20511 > URL: https://issues.apache.org/jira/browse/HBASE-20511 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 2.2.0 > > > See discussion in > https://issues.apache.org/jira/browse/HBASE-20457?focusedCommentId=16445329=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16445329 > Its better we set the scan's read type to the subsequent scan requests in > cases where the scan actually switched over from preads to STREAM type. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20383) [AMv2] AssignmentManager: Failed transition XYZ is not OPEN
[
https://issues.apache.org/jira/browse/HBASE-20383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-20383:
--
Fix Version/s: (was: 2.1.0)
2.2.0
> [AMv2] AssignmentManager: Failed transition XYZ is not OPEN
> ---
>
> Key: HBASE-20383
> URL: https://issues.apache.org/jira/browse/HBASE-20383
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Reporter: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-20383.master.001.patch
>
>
> Seeing a bunch of this testing 2.0.0:
> {code}
> 2018-04-10 13:57:09,430 WARN
> [RpcServer.default.FPBQ.Fifo.handler=46,queue=1,port=16000]
> assignment.AssignmentManager: Failed transition
>
>
> org.apache.hadoop.hbase.client.DoNotRetryRegionException:
> 19a2cd6f88abae0036415ee1ea041c2e is not OPEN
> at
> org.apache.hadoop.hbase.master.procedure.AbstractStateMachineTableProcedure.checkOnline(AbstractStateMachineTableProcedure.java:193)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:112)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:769)
>
>
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.updateRegionSplitTransition(AssignmentManager.java:911)
>
>
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportRegionStateTransition(AssignmentManager.java:819)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.reportRegionStateTransition(MasterRpcServices.java:1538)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:11093)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
>
>
>at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {code}
> Looks like report back from Master OK'ing a split to go ahead but the split
> is already running. Figure how to shut these down. They are noisy at least.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20300) Minor refactor for RpcExecutor
[ https://issues.apache.org/jira/browse/HBASE-20300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20300: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Minor refactor for RpcExecutor > -- > > Key: HBASE-20300 > URL: https://issues.apache.org/jira/browse/HBASE-20300 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-20300.v0.patch, HBASE-20300.v0.patch.patch > > > Plan to do the following changes. > # make Handler be static class > # move the threadlocal variables of MonitoredRPCHandler from RpcServer to > FifoRpcScheduler since only FifoRpcScheduler use it > # create MonitoredRPCHandler in Handler constuction instead of saving the > MonitoredRPCHandler in threadlocal variables. In FPBQ mode, the web UI can > display all Handlers info even if the rpc Handlers are not used yet. > # Threshhold -> Threshold > # make RpcExecutor extend ConfigurationObserver > # don't create task filter repeatly > # add a ut to check whether each Handler has created own MonitoredTask even > if no ops > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20252) Admin.move will not fail if we move region to a nonexistent region server
[ https://issues.apache.org/jira/browse/HBASE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20252: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Admin.move will not fail if we move region to a nonexistent region server > -- > > Key: HBASE-20252 > URL: https://issues.apache.org/jira/browse/HBASE-20252 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-20252-UT.patch > > > The region will just be reopened on the source regionserver... > This is a bit confusing I think... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20270) Turn off command help that follows all errors in shell
[
https://issues.apache.org/jira/browse/HBASE-20270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-20270:
--
Fix Version/s: (was: 2.1.0)
2.2.0
> Turn off command help that follows all errors in shell
> --
>
> Key: HBASE-20270
> URL: https://issues.apache.org/jira/browse/HBASE-20270
> Project: HBase
> Issue Type: Task
> Components: shell
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sakthi
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-20270.master.001.patch,
> hbase-20270.master.002.patch, hbase-20270.master.003.patch,
> hbase-20270.master.004.patch, hbase-20270.master.005.patch,
> hbase-20270.master.006.patch
>
>
> Right now if a shell command gives an error, any error, it then echos the
> command help. It makes it harder to see the actual error text and is annoying.
> example:
> {code}
>
>
>
> hbase(main):007:0> create 'test:a_table', 'family', { NUMREGIONS => 20,
> SPLITALGO => 'HexStringSplit'}
> ERROR: Unknown namespace test!
> Creates a table. Pass a table name, and a set of column family
> specifications (at least one), and, optionally, table configuration.
> Column specification can be a simple string (name), or a dictionary
> (dictionaries are described below in main help output), necessarily
> including NAME attribute.
> Examples:
> Create a table with namespace=ns1 and table qualifier=t1
> hbase> create 'ns1:t1', {NAME => 'f1', VERSIONS => 5}
> Create a table with namespace=default and table qualifier=t1
> hbase> create 't1', {NAME => 'f1'}, {NAME => 'f2'}, {NAME => 'f3'}
> hbase> # The above in shorthand would be the following:
> hbase> create 't1', 'f1', 'f2', 'f3'
> hbase> create 't1', {NAME => 'f1', VERSIONS => 1, TTL => 2592000,
> BLOCKCACHE => true}
> hbase> create 't1', {NAME => 'f1', CONFIGURATION =>
> {'hbase.hstore.blockingStoreFiles' => '10'}}
> hbase> create 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD =>
> 100, MOB_COMPACT_PARTITION_POLICY => 'weekly'}
> Table configuration options can be put at the end.
> Examples:
> hbase> create 'ns1:t1', 'f1', SPLITS => ['10', '20', '30', '40']
> hbase> create 't1', 'f1', SPLITS => ['10', '20', '30', '40']
> hbase> create 't1', 'f1', SPLITS_FILE => 'splits.txt', OWNER => 'johndoe'
> hbase> create 't1', {NAME => 'f1', VERSIONS => 5}, METADATA => { 'mykey' =>
> 'myvalue' }
> hbase> # Optionally pre-split the table into NUMREGIONS, using
> hbase> # SPLITALGO ("HexStringSplit", "UniformSplit" or classname)
> hbase> create 't1', 'f1', {NUMREGIONS => 15, SPLITALGO => 'HexStringSplit'}
> hbase> create 't1', 'f1', {NUMREGIONS => 15, SPLITALGO => 'HexStringSplit',
> REGION_REPLICATION => 2, CONFIGURATION =>
> {'hbase.hregion.scan.loadColumnFamiliesOnDemand' => 'true'}}
> hbase> create 't1', {NAME => 'f1', DFS_REPLICATION => 1}
> You can also keep around a reference to the created table:
> hbase> t1 = create 't1', 'f1'
> Which gives you a reference to the table named 't1', on which you can then
> call methods.
> Took 0.0221 seconds
>
>
> hbase(main):008:0> create_namespace 'test'
> Took 0.2554 seconds
>
>
> hbase(main):009:0> create 'test:a_table', 'family', { NUMREGIONS => 20,
> SPLITALGO => 'HexStringSplit'}
> Created table test:a_table
> Took 1.2264 seconds
> {code}
> I was trying to make a table in the test namespace before making the
> namespace. Much faster to recognize and move on when the error text isn't
> followed by 80x the text.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20249) Investigate behavior of hbase.client.operation.timeout
[ https://issues.apache.org/jira/browse/HBASE-20249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20249: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Investigate behavior of hbase.client.operation.timeout > -- > > Key: HBASE-20249 > URL: https://issues.apache.org/jira/browse/HBASE-20249 > Project: HBase > Issue Type: Bug >Reporter: Appy >Priority: Critical > Fix For: 3.0.0, 2.2.0 > > > hbase.client.operation.timeout should be hard limit on client operations, and > independent of number of retires. > Previous discussions here: > https://issues.apache.org/jira/browse/HBASE-17449?focusedCommentId=16390085=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16390085 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20151) Bug with SingleColumnValueFilter and FamilyFilter
[
https://issues.apache.org/jira/browse/HBASE-20151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-20151:
--
Fix Version/s: (was: 2.1.0)
2.2.0
> Bug with SingleColumnValueFilter and FamilyFilter
> -
>
> Key: HBASE-20151
> URL: https://issues.apache.org/jira/browse/HBASE-20151
> Project: HBase
> Issue Type: Bug
> Environment: MacOS 10.13.3
> HBase 1.3.1
>Reporter: Steven Sadowski
>Assignee: Reid Chan
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-20151.master.001.patch,
> HBASE-20151.master.002.patch, HBASE-20151.master.003.patch,
> HBASE-20151.master.004.patch, HBASE-20151.master.004.patch,
> HBASE-20151.master.005.patch
>
>
> When running the following queries, the result is sometimes return correctly
> and other times incorrectly based on the qualifier queried.
> Setup:
> {code:java}
> create 'test', 'a', 'b'
> test = get_table 'test'
> test.put '1', 'a:1', nil
> test.put '1', 'a:10', nil
> test.put '1', 'b:2', nil
> {code}
>
> This query works fine when the SCVF's qualifier has length 1 (i.e. '1') :
> {code:java}
> test.scan({ FILTER => "(
> SingleColumnValueFilter('a','1',=,'binary:',true,true) AND
> FamilyFilter(=,'binary:b') )"})
> ROW COLUMN+CELL
> 1column=b:2,
> timestamp=1520455888059, value=
> 1 row(s) in 0.0060 seconds
> {code}
>
> The query should return the same result when passed a qualifier of length 2
> (i.e. '10') :
> {code:java}
> test.scan({ FILTER => "(
> SingleColumnValueFilter('a','10',=,'binary:',true,true) AND
> FamilyFilter(=,'binary:b') )"})
> ROW COLUMN+CELL
> 0 row(s) in 0.0110 seconds
> {code}
> However, in this case, it does not return any row (expected result would be
> to return the same result as the first query).
>
> Removing the family filter while the qualifier is '10' yields expected
> results:
> {code:java}
> test.scan({ FILTER => "(
> SingleColumnValueFilter('a','10',=,'binary:',true,true) )"})
> ROW COLUMN+CELL
> 1column=a:1,
> timestamp=1520455887954, value=
> 1column=a:10,
> timestamp=1520455888024, value=
> 1column=b:2,
> timestamp=1520455888059, value=
> 1 row(s) in 0.0140 seconds
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20064) Disable MOB threads that are running whether you MOB or not
[ https://issues.apache.org/jira/browse/HBASE-20064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20064: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Disable MOB threads that are running whether you MOB or not > --- > > Key: HBASE-20064 > URL: https://issues.apache.org/jira/browse/HBASE-20064 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Reid Chan >Priority: Major > Fix For: 2.2.0 > > Attachments: HBASE-20064.master.001.patch, > HBASE-20064.master.002.patch > > > Master starts up some cleaner and compacting threads even though no MOB. > Disable them and have users explicitly enable it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20099) Need to support per procedure killAndToggleBeforeStoreUpdate
[ https://issues.apache.org/jira/browse/HBASE-20099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20099: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Need to support per procedure killAndToggleBeforeStoreUpdate > > > Key: HBASE-20099 > URL: https://issues.apache.org/jira/browse/HBASE-20099 > Project: HBase > Issue Type: Improvement > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 2.2.0 > > > Currently there is procedure framework wide killAndToggleBeforeStoreUpdate. > This affects all procedures being executed by procedure framework including > sub-procedures. Procedure specific toggle can be added as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19952) Find tests which are declared with wrong category
[ https://issues.apache.org/jira/browse/HBASE-19952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19952: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Find tests which are declared with wrong category > - > > Key: HBASE-19952 > URL: https://issues.apache.org/jira/browse/HBASE-19952 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.2.0 > > Attachments: category-report > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20040) Master UI should include "Cluster Key" needed to use the cluster as a replication sink
[
https://issues.apache.org/jira/browse/HBASE-20040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-20040:
--
Fix Version/s: (was: 2.1.0)
2.2.0
> Master UI should include "Cluster Key" needed to use the cluster as a
> replication sink
> --
>
> Key: HBASE-20040
> URL: https://issues.apache.org/jira/browse/HBASE-20040
> Project: HBase
> Issue Type: Improvement
> Components: Replication, Usability
>Reporter: Sean Busbey
>Assignee: Sakthi
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-20040.branch-1.001.patch,
> hbase-20040.master.001.patch, hbase-20040.master.002.patch
>
>
> The ref guide defines a "Cluster Key" needed to add an hbase cluster as a
> replication peer
> {quote}
> CLUSTER_KEY: composed using the following template, with appropriate
> place-holders:
> {code}hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent{code}
> {quote}
> the Master UI has all of the pieces displayed currently, but it should
> include a single field that operators can copy/paste.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20060) Add details of off heap memstore into book.
[ https://issues.apache.org/jira/browse/HBASE-20060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20060: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Add details of off heap memstore into book. > --- > > Key: HBASE-20060 > URL: https://issues.apache.org/jira/browse/HBASE-20060 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.2.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19562) Purge mirror writing of region and table info into fs at .tableinfo and .regioninfo
[ https://issues.apache.org/jira/browse/HBASE-19562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19562: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Purge mirror writing of region and table info into fs at .tableinfo and > .regioninfo > --- > > Key: HBASE-19562 > URL: https://issues.apache.org/jira/browse/HBASE-19562 > Project: HBase > Issue Type: Bug > Components: fs >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.2.0 > > Attachments: > 0002-HBASE-19562-Purge-mirror-writing-of-region-and-table.patch > > > We don't use these files in hbase2 yet we keep writing them when we create a > table or region. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19452) Turn ON off heap Bucket Cache by default
[ https://issues.apache.org/jira/browse/HBASE-19452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19452: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Turn ON off heap Bucket Cache by default > > > Key: HBASE-19452 > URL: https://issues.apache.org/jira/browse/HBASE-19452 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Major > Fix For: 2.2.0 > > Attachments: HBASE-19452.patch, HBASE-19452_V2.patch, > HBASE-19452_V3.patch > > > BC's hbase.bucketcache.ioengine by default is empty now. Means now BC. > Make this default to be 'offheap'. And the default off heap size for the BC > also to be provided. This can be 8 GB? Better to make it also a % of the > Xmx. Lets continue to be 40% of Xmx as LRU cache default size. > When user has to disable BC, configure size as 0. An empty value of this > config will be treated as default size. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19104) Add filtering during restore in HBase backups.
[ https://issues.apache.org/jira/browse/HBASE-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19104: -- Fix Version/s: (was: 2.2.0) 3.0.0 > Add filtering during restore in HBase backups. > -- > > Key: HBASE-19104 > URL: https://issues.apache.org/jira/browse/HBASE-19104 > Project: HBase > Issue Type: New Feature > Components: backuprestore >Reporter: Amit Kabra >Priority: Major > Fix For: 3.0.0 > > > When we deal with large amount of data, it would be great , if we can do data > restore from backups based on tenant , based on time range , etc , so that if > finishes faster and we restore only what's required. > Currently restore take backup id as input and restore all the data will that > backup id time stamp. We may not need to restore all data in a given backup > id. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19380) B: auto-repair mode of execution for create/merge/delete after failure.
[ https://issues.apache.org/jira/browse/HBASE-19380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19380: -- Fix Version/s: (was: 2.1.0) 3.0.0 > B: auto-repair mode of execution for create/merge/delete after failure. > - > > Key: HBASE-19380 > URL: https://issues.apache.org/jira/browse/HBASE-19380 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Labels: backup > Fix For: 3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19222) update jruby to 9.1.14.0+
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19222: -- Fix Version/s: (was: 2.1.0) 2.2.0 > update jruby to 9.1.14.0+ > - > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.2.0 > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19107) Backup should resume from last failed state
[ https://issues.apache.org/jira/browse/HBASE-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19107: -- Fix Version/s: (was: 2.1.0) 3.0.0 > Backup should resume from last failed state > --- > > Key: HBASE-19107 > URL: https://issues.apache.org/jira/browse/HBASE-19107 > Project: HBase > Issue Type: New Feature > Components: backuprestore >Reporter: Vishal Khandelwal >Priority: Major > Labels: backup > Fix For: 3.0.0 > > > Consider that incremental backup is happenng for 10 tables for a set of 100 > WALs. if backup @ 10th fails, then on net iteration it would be triggered > again from 100 WALs + new WALs all tables > This is a issue because resource and time used to do he backup of 9 table & > 100 WALs are wasted > Rather it should have functionality to resume from previous state complete > that backup and then do new set of WALs. This will make it more performant -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19106) Backup self validation for its correctness.
[ https://issues.apache.org/jira/browse/HBASE-19106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19106: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Backup self validation for its correctness. > --- > > Key: HBASE-19106 > URL: https://issues.apache.org/jira/browse/HBASE-19106 > Project: HBase > Issue Type: Improvement > Components: backuprestore >Reporter: Amit Kabra >Priority: Major > Fix For: 3.0.0 > > > Backups are critical and if they don't work when we need them at the time of > restore than they are not useful. We should do sanity test for each backup > job we run that it is restorable and hence can be trusted. > A self validation feature can be added for the same to the backups where > whenever a backup is run , once it finishes it will trigger a validation job > that will do a sample restoration of the backed up data and will make sure > that it compares well with actual data. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19106) Backup self validation for its correctness.
[ https://issues.apache.org/jira/browse/HBASE-19106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19106: -- Fix Version/s: (was: 2.2.0) 3.0.0 > Backup self validation for its correctness. > --- > > Key: HBASE-19106 > URL: https://issues.apache.org/jira/browse/HBASE-19106 > Project: HBase > Issue Type: Improvement > Components: backuprestore >Reporter: Amit Kabra >Priority: Major > Fix For: 3.0.0 > > > Backups are critical and if they don't work when we need them at the time of > restore than they are not useful. We should do sanity test for each backup > job we run that it is restorable and hence can be trusted. > A self validation feature can be added for the same to the backups where > whenever a backup is run , once it finishes it will trigger a validation job > that will do a sample restoration of the backed up data and will make sure > that it compares well with actual data. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18785) Blocker doc issues for 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-18785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-18785: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Blocker doc issues for 2.0.0 > > > Key: HBASE-18785 > URL: https://issues.apache.org/jira/browse/HBASE-18785 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 3.0.0, 2.2.0 > > > Hang blocker documentation issues off this one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19104) Add filtering during restore in HBase backups.
[ https://issues.apache.org/jira/browse/HBASE-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19104: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Add filtering during restore in HBase backups. > -- > > Key: HBASE-19104 > URL: https://issues.apache.org/jira/browse/HBASE-19104 > Project: HBase > Issue Type: New Feature > Components: backuprestore >Reporter: Amit Kabra >Priority: Major > Fix For: 2.2.0 > > > When we deal with large amount of data, it would be great , if we can do data > restore from backups based on tenant , based on time range , etc , so that if > finishes faster and we restore only what's required. > Currently restore take backup id as input and restore all the data will that > backup id time stamp. We may not need to restore all data in a given backup > id. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18738) Ride over HDFS 'safe mode' post-launch
[ https://issues.apache.org/jira/browse/HBASE-18738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-18738: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Ride over HDFS 'safe mode' post-launch > -- > > Key: HBASE-18738 > URL: https://issues.apache.org/jira/browse/HBASE-18738 > Project: HBase > Issue Type: Improvement > Components: Operability >Reporter: stack >Priority: Major > Fix For: 2.2.0 > > > A coworker has an integration test that includes setting HDFS into 'safe > mode' post-launch of the cluster. He finds that HBase just falls over. We > should do better. We should be able to recognize the switch and going into > stasis recovering once 'safe mode' has expired. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18622) Mitigate API compatibility concerns between branch-1 and branch-2
[ https://issues.apache.org/jira/browse/HBASE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517085#comment-16517085 ] Duo Zhang commented on HBASE-18622: --- Do we still have unfinished work for this issue? It is marked as blocker... > Mitigate API compatibility concerns between branch-1 and branch-2 > - > > Key: HBASE-18622 > URL: https://issues.apache.org/jira/browse/HBASE-18622 > Project: HBase > Issue Type: Bug > Components: API >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 3.0.0, 2.1.0 > > Attachments: report.1.2_2.0.html.gz > > > This project is to do what [~apurtell] did in the issue "HBASE-18431 Mitigate > compatibility concerns between branch-1.3 and branch-1.4" only do it between > branch-1 and branch-2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18561) Blog post w/intro to SyncTable
[ https://issues.apache.org/jira/browse/HBASE-18561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-18561: -- Fix Version/s: (was: 2.1.0) 2.2.0 > Blog post w/intro to SyncTable > -- > > Key: HBASE-18561 > URL: https://issues.apache.org/jira/browse/HBASE-18561 > Project: HBase > Issue Type: Task > Components: community, documentation, Replication >Affects Versions: 1.2.0 >Reporter: Sean Busbey >Priority: Critical > Fix For: 2.2.0 > > > In addition to HBASE-15557, we should have a blog post that goes over > SyncTable / HashTable . They're a great addition that needs more surfacing to > operators. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18495) [AMv2] Make methods isServerOnline() and isServerDead() in ServerManager mutually exclusive
[ https://issues.apache.org/jira/browse/HBASE-18495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-18495: -- Fix Version/s: (was: 2.1.0) 2.2.0 > [AMv2] Make methods isServerOnline() and isServerDead() in ServerManager > mutually exclusive > --- > > Key: HBASE-18495 > URL: https://issues.apache.org/jira/browse/HBASE-18495 > Project: HBase > Issue Type: Bug > Components: amv2 >Affects Versions: 2.0.0 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > During debugging HBASE-18366, it was found that > ServerManager.isServerOnline() and ServerManager.isServerDead() can return > true at the same time. This causes problems with the code depending on which > method is used. Making them mutually exclusive seems line right thing to do. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18415) The local timeout may cause Admin to submit duplicate request
[ https://issues.apache.org/jira/browse/HBASE-18415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-18415: -- Fix Version/s: (was: 2.1.0) 2.2.0 > The local timeout may cause Admin to submit duplicate request > - > > Key: HBASE-18415 > URL: https://issues.apache.org/jira/browse/HBASE-18415 > Project: HBase > Issue Type: Bug >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.3.3, 1.4.6, 2.2.0 > > Attachments: HBASE-18415.branch-1.ut.patch, > HBASE-18415.branch-1.v0.patch, HBASE-18415.branch-1.v1.patch, > HBASE-18415.branch-1.v2.patch, HBASE-18415.branch-1.v3.patch, > HBASE-18415.branch-1.v3.patch, HBASE-18415.branch-1.v3.patch, > HBASE-18415.branch-1.v4.patch, HBASE-18415.branch-1.v4.patch, > HBASE-18415.branch-1.v4.patch > > > After a timeout occurs on first request, client will retry the request with > distinct group/nonce. The second request may bring the TableXXXException back > if the first request have changed the table state. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
