[jira] [Comment Edited] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only
[
https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041234#comment-15041234
]
Duo Zhang edited comment on HBASE-14790 at 12/4/15 8:06 AM:
Considering these features:
Hflush is much faster than hsync, especially in pipeline mode. So we have to
use hflush for hbase writing.
The data in DN that is hflushed but not hsynced may only in memory not disk,
but it can be read by client.
So if we hflush data to DNs, and it is read by ReplicationSource and
transferred to slave cluster, then three DNs and RS in master cluster crash.
And after replaying WALs, slave will have data that master loses...
The only way to prevent any data losses is hsync every time but it is too slow,
and I think most users can bear data lose to speed up writing operation but can
not bear slave has more data than master.
Therefore, I think we can do these:
hflush every time, not hsync;
hsync periodically, for example, default per 1000ms? It can be configured by
users, and users can also configure that we hsync every time, so there will not
have any data loses unless all DNs disk fail...
RS tells "acked length" to ReplicationSource which is the data we hsynced, not
hflushed.
ReplicationSource only transfer data which is not larger than acked length. So
the slave cluster will never have inconsistency.
WAL reading can handle duplicate entries.
On WAL logging, if we get error on hflush, we open a new file and rewrite this
entry, and recover/hsync/close old file asynchronously.
was (Author: yangzhe1991):
Considering these features:
Hflush is much faster than hsync, especially in pipeline mode. So we have to
use hflush for hbase writing.
The data in DN that is hflushed but not hsynced may only in memory not disk,
but it can be read by client.
So if we hflush data to DNs, and it is read by ReplicationSource and
transferred to slave cluster, then three DNs and RS in master cluster crash.
And after replaying WALs, slave will have data that master loses...
The only way to prevent any data losses is hsync every time but it is too slow,
and I think most users can bear data lose to speed up writing operation but can
not bear slave has more data than master.
Therefore, I think we can do these:
hflush every time, not fsync;
hsync periodically, for example, default per 1000ms? It can be configured by
users, and users can also configure that we hsync every time, so there will not
have any data loses unless all DNs disk fail...
RS tells "acked length" to ReplicationSource which is the data we hsynced, not
hflushed.
ReplicationSource only transfer data which is not larger than acked length. So
the slave cluster will never have inconsistency.
WAL reading can handle duplicate entries.
On WAL logging, if we get error on hflush, we open a new file and rewrite this
entry, and recover/hsync/close old file asynchronously.
> Implement a new DFSOutputStream for logging WAL only
>
>
> Key: HBASE-14790
> URL: https://issues.apache.org/jira/browse/HBASE-14790
> Project: HBase
> Issue Type: Improvement
>Reporter: Duo Zhang
>
> The original {{DFSOutputStream}} is very powerful and aims to serve all
> purposes. But in fact, we do not need most of the features if we only want to
> log WAL. For example, we do not need pipeline recovery since we could just
> close the old logger and open a new one. And also, we do not need to write
> multiple blocks since we could also open a new logger if the old file is too
> large.
> And the most important thing is that, it is hard to handle all the corner
> cases to avoid data loss or data inconsistency(such as HBASE-14004) when
> using original DFSOutputStream due to its complicated logic. And the
> complicated logic also force us to use some magical tricks to increase
> performance. For example, we need to use multiple threads to call {{hflush}}
> when logging, and now we use 5 threads. But why 5 not 10 or 100?
> So here, I propose we should implement our own {{DFSOutputStream}} when
> logging WAL. For correctness, and also for performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only
[
https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041234#comment-15041234
]
Duo Zhang edited comment on HBASE-14790 at 12/4/15 8:06 AM:
Considering these features:
Hflush is much faster than hsync, especially in pipeline mode. So we have to
use hflush for hbase writing.
The data in DN that is hflushed but not hsynced may only in memory not disk,
but it can be read by client.
So if we hflush data to DNs, and it is read by ReplicationSource and
transferred to slave cluster, then three DNs and RS in master cluster crash.
And after replaying WALs, slave will have data that master loses...
The only way to prevent any data losses is hsync every time but it is too slow,
and I think most users can bear data lose to speed up writing operation but can
not bear slave has more data than master.
Therefore, I think we can do these:
hflush every time, not fsync;
hsync periodically, for example, default per 1000ms? It can be configured by
users, and users can also configure that we hsync every time, so there will not
have any data loses unless all DNs disk fail...
RS tells "acked length" to ReplicationSource which is the data we hsynced, not
hflushed.
ReplicationSource only transfer data which is not larger than acked length. So
the slave cluster will never have inconsistency.
WAL reading can handle duplicate entries.
On WAL logging, if we get error on hflush, we open a new file and rewrite this
entry, and recover/hsync/close old file asynchronously.
was (Author: yangzhe1991):
Considering these features:
Hflush is much faster than hsync, especially in pipeline mode. So we have to
use hflush for hbase writing.
The data in DN that is hflushed but not hsynced may only in memory not disk,
but it can be read by client.
So if we hflush data to DNs, and it is read by ReplicationSource and
transferred to slave cluster, then three DNs and RS in master cluster crash.
And after replaying WALs, slave will have data that master loses...
The only way to prevent any data losses is hsync every time but it is too slow,
and I think most users can bear data lose to speed up writing operation but can
not bear slave has more data than master.
Therefore, I think we can do these:
hflush every time, not fsync;
hsync periodically, for example, default per 1000ms? It can be configured by
users, and users can also configure that we every each time, so there will not
have any data loses unless all DNs disk fail...
RS tells "acked length" to ReplicationSource which is the data we hsynced, not
hflushed.
ReplicationSource only transfer data which is not larger than acked length. So
the slave cluster will never have inconsistency.
WAL reading can handle duplicate entries.
On WAL logging, if we get error on hflush, we open a new file and rewrite this
entry, and recover/hsync/close old file asynchronously.
> Implement a new DFSOutputStream for logging WAL only
>
>
> Key: HBASE-14790
> URL: https://issues.apache.org/jira/browse/HBASE-14790
> Project: HBase
> Issue Type: Improvement
>Reporter: Duo Zhang
>
> The original {{DFSOutputStream}} is very powerful and aims to serve all
> purposes. But in fact, we do not need most of the features if we only want to
> log WAL. For example, we do not need pipeline recovery since we could just
> close the old logger and open a new one. And also, we do not need to write
> multiple blocks since we could also open a new logger if the old file is too
> large.
> And the most important thing is that, it is hard to handle all the corner
> cases to avoid data loss or data inconsistency(such as HBASE-14004) when
> using original DFSOutputStream due to its complicated logic. And the
> complicated logic also force us to use some magical tricks to increase
> performance. For example, we need to use multiple threads to call {{hflush}}
> when logging, and now we use 5 threads. But why 5 not 10 or 100?
> So here, I propose we should implement our own {{DFSOutputStream}} when
> logging WAL. For correctness, and also for performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14905) VerifyReplication does not honour versions option
[
https://issues.apache.org/jira/browse/HBASE-14905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041263#comment-15041263
]
Hudson commented on HBASE-14905:
FAILURE: Integrated in HBase-1.2 #421 (See
[https://builds.apache.org/job/HBase-1.2/421/])
HBASE-14905 VerifyReplication does not honour versions option (Vishal (tedyu:
rev eb777ef289827ae385c0cae71ea64cd6618e14af)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationBase.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSmallTests.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
> VerifyReplication does not honour versions option
> -
>
> Key: HBASE-14905
> URL: https://issues.apache.org/jira/browse/HBASE-14905
> Project: HBase
> Issue Type: Bug
> Components: tooling
>Affects Versions: 2.0.0, 0.98.16
>Reporter: Vishal Khandelwal
>Assignee: Vishal Khandelwal
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14905-v2.txt, HBASE-14905.patch, HBASE-14905_v3.patch,
> HBASE-14905_v4.patch, test.patch
>
>
> source:
> hbase(main):001:0> scan 't1', {RAW => true, VERSIONS => 100}
> ROW COLUMN+CELL
>
>
> r1 column=f1:, timestamp=1449030102091,
> value=value1112
>
> r1 column=f1:, timestamp=1449029774173,
> value=value1001
>
> r1 column=f1:, timestamp=1449029709974,
> value=value1002
>
> target:
> hbase(main):023:0> scan 't1', {RAW => true, VERSIONS => 100}
> ROW COLUMN+CELL
>
>
> r1 column=f1:, timestamp=1449030102091,
> value=value1112
>
> r1 column=f1:, timestamp=1449030090758,
> value=value1112
>
> r1 column=f1:, timestamp=1449029984282,
> value=value
>
> r1 column=f1:, timestamp=1449029774173,
> value=value1001
>
> r1 column=f1:, timestamp=1449029709974,
> value=value1002
> /bin/hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication
> --versions=100 1 t1
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication$Verifier$Counters
> GOODROWS=1
> Does not show any mismatch. Ideally it should show. This is because in
> VerifyReplication Class maxVersion is not correctly set.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14876) Provide maven archetypes
[ https://issues.apache.org/jira/browse/HBASE-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Vimont updated HBASE-14876: -- Attachment: archetype_shaded_prototype01.zip A new prototype of an archetype, this one to fulfill the second requirement: to produce client application with dependency upon *hbase-shaded-client*. This one may also be installed in a sandbox environment to work with, by following the same instructions that were listed above for the first prototype. > Provide maven archetypes > > > Key: HBASE-14876 > URL: https://issues.apache.org/jira/browse/HBASE-14876 > Project: HBase > Issue Type: New Feature > Components: build, Usability >Reporter: Nick Dimiduk >Assignee: Daniel Vimont > Labels: beginner > Attachments: archetype_prototype.zip, archetype_prototype02.zip, > archetype_shaded_prototype01.zip > > > To help onboard new users, we should provide maven archetypes for hbase > client applications. Off the top of my head, we should have templates for > - hbase client application with all dependencies > - hbase client application using client-shaded jar > - mapreduce application with hbase as input and output (ie, copy table) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only
[
https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041307#comment-15041307
]
Heng Chen commented on HBASE-14790:
---
{quote}
hsync periodically, for example, default per 1000ms? It can be configured by
users, and users can also configure that we hsync every time, so there will not
have any data loses unless all DNs disk fail...
{quote}
Sounds reasonable. we update 'acked length' after hsync, I think 'acked
length' logical is completed. Any thoughts?
> Implement a new DFSOutputStream for logging WAL only
>
>
> Key: HBASE-14790
> URL: https://issues.apache.org/jira/browse/HBASE-14790
> Project: HBase
> Issue Type: Improvement
>Reporter: Duo Zhang
>
> The original {{DFSOutputStream}} is very powerful and aims to serve all
> purposes. But in fact, we do not need most of the features if we only want to
> log WAL. For example, we do not need pipeline recovery since we could just
> close the old logger and open a new one. And also, we do not need to write
> multiple blocks since we could also open a new logger if the old file is too
> large.
> And the most important thing is that, it is hard to handle all the corner
> cases to avoid data loss or data inconsistency(such as HBASE-14004) when
> using original DFSOutputStream due to its complicated logic. And the
> complicated logic also force us to use some magical tricks to increase
> performance. For example, we need to use multiple threads to call {{hflush}}
> when logging, and now we use 5 threads. But why 5 not 10 or 100?
> So here, I propose we should implement our own {{DFSOutputStream}} when
> logging WAL. For correctness, and also for performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14904) Mark Base[En|De]coder LimitedPrivate and fix binary compat issue
[
https://issues.apache.org/jira/browse/HBASE-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041249#comment-15041249
]
Hudson commented on HBASE-14904:
SUCCESS: Integrated in HBase-1.1-JDK8 #1700 (See
[https://builds.apache.org/job/HBase-1.1-JDK8/1700/])
HBASE-14904 Mark Base[En|De]coder LimitedPrivate and fix binary compat (enis:
rev f3d3bd9d3b8eca176166391ef078391816b34bed)
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseEncoder.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/CompressionContext.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALPrettyPrinter.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseDecoder.java
> Mark Base[En|De]coder LimitedPrivate and fix binary compat issue
>
>
> Key: HBASE-14904
> URL: https://issues.apache.org/jira/browse/HBASE-14904
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4
>
> Attachments: hbase-14904_v1.patch, hbase-14904_v2.patch
>
>
> PHOENIX-2477 revealed that the changes from HBASE-14501 breaks binary
> compatibility in Phoenix compiled with earlier versions of HBase and run
> agains later versions.
> This is one of the areas that the boundary is not clear, but it won't hurt us
> to fix it.
> The exception trace is:
> {code}
> Exception in thread "main" java.lang.NoSuchFieldError: in
> at
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec$PhoenixBaseDecoder.(IndexedWALEditCodec.java:106)
> at
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec$IndexKeyValueDecoder.(IndexedWALEditCodec.java:121)
> at
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec.getDecoder(IndexedWALEditCodec.java:63)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.initAfterCompression(ProtobufLogReader.java:292)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:82)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.init(ProtobufLogReader.java:148)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:316)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:281)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:269)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:418)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.processFile(WALPrettyPrinter.java:247)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.run(WALPrettyPrinter.java:422)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.main(WALPrettyPrinter.java:357)
> {code}
> Although {{BaseDecoder.in}} is still there, it got changed to be a class
> rather than an interface. BaseDecoder is marked Private, thus the binary
> compat check is not run at all. Not sure whether it would have caught this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14873) Problems around BoundedByteBufferPool providing direct buffers
[
https://issues.apache.org/jira/browse/HBASE-14873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041304#comment-15041304
]
Hiroshi Ikeda commented on HBASE-14873:
---
In practice the most problematic thing seems unlimited Unsafe.copyMemory if we
worry about stop-the-world. Unsafe is mysteriously dangerous and I doubt the
performance improvement is practically effective at the cost of uncertain risk.
> Problems around BoundedByteBufferPool providing direct buffers
> --
>
> Key: HBASE-14873
> URL: https://issues.apache.org/jira/browse/HBASE-14873
> Project: HBase
> Issue Type: Bug
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
> Attachments: HBASE-14873-V2.patch, HBASE-14873.patch,
> HBASE-14873.patch, HBASE-14873.patch
>
>
> HBASE-13819 made BoundedByteBufferPool provide direct buffers.
> See RpcServer.java:
> {code}
> ...
> class Call implements RpcCallContext {
> protected synchronized void setResponse(...) {
> ...
> this.cellBlock = ipcUtil.buildCellBlock(..., reservoir);
> ...
> bc = new BufferChain(..., this.cellBlock);
> if (connection.useWrap) {
> bc = wrapWithSasl(bc);
> }
> ...
> private BufferChain wrapWithSasl(BufferChain bc) throws IOException {
> ...
> byte[] responseBytes = bc.getBytes();
> ...
> {code}
> {{cellBlock}} is expected to be a direct buffer retrieved from {{reservoir}}
> (but not always), and {{bc}} may be composed of both direct and non-direct
> buffers.
> And then, see BufferChain.java:
> {code}
> byte [] getBytes() {
> ...
> for (ByteBuffer bb: this.buffers) {
> System.arraycopy(bb.array(), ...);
> {code}
> A direct buffer doesn't give its array, and will throw
> UnsupportedOperationException.
> Another problem; {{cellBlock}} is allowed to be a non-direct buffer, and
> after use it will be put to {{reservoir}}, mixing direct and non-direct
> buffers in the pool.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14876) Provide maven archetypes
[ https://issues.apache.org/jira/browse/HBASE-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041244#comment-15041244 ] Daniel Vimont commented on HBASE-14876: --- This comment refers to the zip file I just uploaded: archetype_shaded_prototype01.zip > Provide maven archetypes > > > Key: HBASE-14876 > URL: https://issues.apache.org/jira/browse/HBASE-14876 > Project: HBase > Issue Type: New Feature > Components: build, Usability >Reporter: Nick Dimiduk >Assignee: Daniel Vimont > Labels: beginner > Attachments: archetype_prototype.zip, archetype_prototype02.zip, > archetype_shaded_prototype01.zip > > > To help onboard new users, we should provide maven archetypes for hbase > client applications. Off the top of my head, we should have templates for > - hbase client application with all dependencies > - hbase client application using client-shaded jar > - mapreduce application with hbase as input and output (ie, copy table) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14927) Backport HBASE-13014 and HBASE-14749 to branch-1 and 0.98
Abhishek Singh Chouhan created HBASE-14927: -- Summary: Backport HBASE-13014 and HBASE-14749 to branch-1 and 0.98 Key: HBASE-14927 URL: https://issues.apache.org/jira/browse/HBASE-14927 Project: HBase Issue Type: Improvement Reporter: Abhishek Singh Chouhan Assignee: Abhishek Singh Chouhan Fix For: 1.3.0, 0.98.17 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only
[
https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041234#comment-15041234
]
Duo Zhang edited comment on HBASE-14790 at 12/4/15 8:02 AM:
Considering these features:
Hflush is much faster than hsync, especially in pipeline mode. So we have to
use hflush for hbase writing.
The data in DN that is hflushed but not hsynced may only in memory not disk,
but it can be read by client.
So if we hflush data to DNs, and it is read by ReplicationSource and
transferred to slave cluster, then three DNs and RS in master cluster crash.
And after replaying WALs, slave will have data that master loses...
The only way to prevent any data losses is hsync every time but it is too slow,
and I think most users can bear data lose to speed up writing operation but can
not bear slave has more data than master.
Therefore, I think we can do these:
hflush every time, not fsync;
hsync periodically, for example, default per 1000ms? It can be configured by
users, and users can also configure that we every each time, so there will not
have any data loses unless all DNs disk fail...
RS tells "acked length" to ReplicationSource which is the data we hsynced, not
hflushed.
ReplicationSource only transfer data which is not larger than acked length. So
the slave cluster will never have inconsistency.
WAL reading can handle duplicate entries.
On WAL logging, if we get error on hflush, we open a new file and rewrite this
entry, and recover/hsync/close old file asynchronously.
was (Author: yangzhe1991):
Considering these features:
Hflush is much faster than hsync, especially in pipeline mode. So we have to
use hflush for hbase writing.
The data in DN that is hflushed but not hsynced may only in memory not disk,
but it can be read by client.
So if we hflush data to DNs, and it is read by ReplicationSource and
transferred to slave cluster, then three DNs and RS in master cluster crash.
And after replaying WALs, slave will have data that master loses...
The only way to prevent any data losses is hsync every time but it is too slow,
and I think most users can bear data lose to speed up writing operation but can
not bear slave has more data than master.
Therefore, I think we can do these:
hflush every time, not fsync;
hfsync periodically, for example, default per 1000ms? It can be configured by
users, and users can also configure that we hfsync each time, so there will not
have any data loses unless all DNs disk fail...
RS tells "acked length" to ReplicationSource which is the data we hsynced, not
hflushed.
ReplicationSource only transfer data which is not larger than acked length. So
the slave cluster will never have inconsistency.
WAL reading can handle duplicate entries.
On WAL logging, if we get error on hflush, we open a new file and rewrite this
entry, and recover/hsync/close old file asynchronously.
> Implement a new DFSOutputStream for logging WAL only
>
>
> Key: HBASE-14790
> URL: https://issues.apache.org/jira/browse/HBASE-14790
> Project: HBase
> Issue Type: Improvement
>Reporter: Duo Zhang
>
> The original {{DFSOutputStream}} is very powerful and aims to serve all
> purposes. But in fact, we do not need most of the features if we only want to
> log WAL. For example, we do not need pipeline recovery since we could just
> close the old logger and open a new one. And also, we do not need to write
> multiple blocks since we could also open a new logger if the old file is too
> large.
> And the most important thing is that, it is hard to handle all the corner
> cases to avoid data loss or data inconsistency(such as HBASE-14004) when
> using original DFSOutputStream due to its complicated logic. And the
> complicated logic also force us to use some magical tricks to increase
> performance. For example, we need to use multiple threads to call {{hflush}}
> when logging, and now we use 5 threads. But why 5 not 10 or 100?
> So here, I propose we should implement our own {{DFSOutputStream}} when
> logging WAL. For correctness, and also for performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14904) Mark Base[En|De]coder LimitedPrivate and fix binary compat issue
[
https://issues.apache.org/jira/browse/HBASE-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041264#comment-15041264
]
Hudson commented on HBASE-14904:
FAILURE: Integrated in HBase-1.2 #421 (See
[https://builds.apache.org/job/HBase-1.2/421/])
HBASE-14904 Mark Base[En|De]coder LimitedPrivate and fix binary compat (enis:
rev a75a93f98ca003a172cc966464308b013b1769e4)
* hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALPrettyPrinter.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseDecoder.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseEncoder.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/CompressionContext.java
> Mark Base[En|De]coder LimitedPrivate and fix binary compat issue
>
>
> Key: HBASE-14904
> URL: https://issues.apache.org/jira/browse/HBASE-14904
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4
>
> Attachments: hbase-14904_v1.patch, hbase-14904_v2.patch
>
>
> PHOENIX-2477 revealed that the changes from HBASE-14501 breaks binary
> compatibility in Phoenix compiled with earlier versions of HBase and run
> agains later versions.
> This is one of the areas that the boundary is not clear, but it won't hurt us
> to fix it.
> The exception trace is:
> {code}
> Exception in thread "main" java.lang.NoSuchFieldError: in
> at
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec$PhoenixBaseDecoder.(IndexedWALEditCodec.java:106)
> at
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec$IndexKeyValueDecoder.(IndexedWALEditCodec.java:121)
> at
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec.getDecoder(IndexedWALEditCodec.java:63)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.initAfterCompression(ProtobufLogReader.java:292)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:82)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.init(ProtobufLogReader.java:148)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:316)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:281)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:269)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:418)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.processFile(WALPrettyPrinter.java:247)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.run(WALPrettyPrinter.java:422)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.main(WALPrettyPrinter.java:357)
> {code}
> Although {{BaseDecoder.in}} is still there, it got changed to be a class
> rather than an interface. BaseDecoder is marked Private, thus the binary
> compat check is not run at all. Not sure whether it would have caught this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14926) Hung ThriftServer; no timeout on read from client; if client crashes, worker thread gets stuck reading
[
https://issues.apache.org/jira/browse/HBASE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041376#comment-15041376
]
Hadoop QA commented on HBASE-14926:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12775738/14926.patch
against master branch at commit 8b3d1f144408e4a7a014c5ac46418c9e91b9b0db.
ATTACHMENT ID: 12775738
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ 4. Here is a lazy example that just pulls in all hbase dependency
jars and that goes against default location on localhost.
+ {java -cp
./hbase-examples/target/hbase-examples-2.0.0-SNAPSHOT.jar:`./bin/hbase
classpath` org.apache.hadoop.hbase.thrift.DemoClient localhost 9090}
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16764//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16764//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16764//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16764//console
This message is automatically generated.
> Hung ThriftServer; no timeout on read from client; if client crashes, worker
> thread gets stuck reading
> --
>
> Key: HBASE-14926
> URL: https://issues.apache.org/jira/browse/HBASE-14926
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 1.0.3, 0.98.16
>Reporter: stack
>Assignee: stack
> Attachments: 14926.patch
>
>
> Thrift server is hung. All worker threads are doing this:
> {code}
> "thrift-worker-0" daemon prio=10 tid=0x7f0bb95c2800 nid=0xf6a7 runnable
> [0x7f0b956e]
>java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> - locked <0x00066d859490> (a java.io.BufferedInputStream)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> at
> org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:601)
> at
> org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:470)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
> at
> org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
>
[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only
[
https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041321#comment-15041321
]
Duo Zhang commented on HBASE-14790:
---
{quote}
This is clean up of a broken WAL? This is being able to ask each DN what it
thinks the length is? While this is going on, we would be holding on to the
hbase handlers not letting response go back to the client? Would we have to do
some weird accounting where three clients A, B, and C and each written an edit,
and then the length we get back from exisiting DNs after a crash say does not
include the edit written by client C... we'll have to figure out how to fail
client C's write (though we'd moved on from append and were trying to
sync/hflush the append)?
{quote}
I think we can implement in this way. When a WAL is broken(one datanode fail
means broken)
1. Open a new WAL synchronously.
2. Write all the un-acked WAL entries to the new WAL file(which means we should
keep all the un-acked WAL entries).
3. Schedule a background task to close the old WAL file.
We should hold the sync WAL request if we consider that some of WAL entries
after last sync has already been written out but not acked until we
successfully write them to new WAL file and get ack back.
And the task in phase 3
1. Doing standard pipeline recovery. Maybe a little difference is that, we can
truncate the block length to our acked length when negotiating with datanodes.
2. endBlock on each datanode.
3. complete file on namenode.
It is does not matter if the rs is crashed during the recovery because we can
make sure that the file length after lease recovery should be longer than acked
length(unless the 3 datanodes are all crashed, we can not handle this using
hflush).
Thanks. [~stack]
> Implement a new DFSOutputStream for logging WAL only
>
>
> Key: HBASE-14790
> URL: https://issues.apache.org/jira/browse/HBASE-14790
> Project: HBase
> Issue Type: Improvement
>Reporter: Duo Zhang
>
> The original {{DFSOutputStream}} is very powerful and aims to serve all
> purposes. But in fact, we do not need most of the features if we only want to
> log WAL. For example, we do not need pipeline recovery since we could just
> close the old logger and open a new one. And also, we do not need to write
> multiple blocks since we could also open a new logger if the old file is too
> large.
> And the most important thing is that, it is hard to handle all the corner
> cases to avoid data loss or data inconsistency(such as HBASE-14004) when
> using original DFSOutputStream due to its complicated logic. And the
> complicated logic also force us to use some magical tricks to increase
> performance. For example, we need to use multiple threads to call {{hflush}}
> when logging, and now we use 5 threads. But why 5 not 10 or 100?
> So here, I propose we should implement our own {{DFSOutputStream}} when
> logging WAL. For correctness, and also for performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14929) There is a space missing from Table "foo" is not currently available.
Ted Malaska created HBASE-14929:
---
Summary: There is a space missing from Table "foo" is not
currently available.
Key: HBASE-14929
URL: https://issues.apache.org/jira/browse/HBASE-14929
Project: HBase
Issue Type: Bug
Reporter: Ted Malaska
Priority: Trivial
Go to the following line in LoadIncrementalHFiles.java
throw new TableNotFoundException("Table " + table.getName() + "is not currently
available.");
and add a space before is and after '
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner
[ https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041432#comment-15041432 ] Anoop Sam John commented on HBASE-13082: Can we have it in 1.3 also? > Coarsen StoreScanner locks to RegionScanner > --- > > Key: HBASE-13082 > URL: https://issues.apache.org/jira/browse/HBASE-13082 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: 13082-test.txt, 13082-v2.txt, 13082-v3.txt, > 13082-v4.txt, 13082.txt, 13082.txt, HBASE-13082.pdf, HBASE-13082_1.pdf, > HBASE-13082_12.patch, HBASE-13082_13.patch, HBASE-13082_14.patch, > HBASE-13082_15.patch, HBASE-13082_16.patch, HBASE-13082_17.patch, > HBASE-13082_18.patch, HBASE-13082_19.patch, HBASE-13082_1_WIP.patch, > HBASE-13082_2.pdf, HBASE-13082_2_WIP.patch, HBASE-13082_3.patch, > HBASE-13082_4.patch, HBASE-13082_9.patch, HBASE-13082_9.patch, > HBASE-13082_withoutpatch.jpg, HBASE-13082_withpatch.jpg, > LockVsSynchronized.java, gc.png, gc.png, gc.png, hits.png, next.png, next.png > > > Continuing where HBASE-10015 left of. > We can avoid locking (and memory fencing) inside StoreScanner by deferring to > the lock already held by the RegionScanner. > In tests this shows quite a scan improvement and reduced CPU (the fences make > the cores wait for memory fetches). > There are some drawbacks too: > * All calls to RegionScanner need to be remain synchronized > * Implementors of coprocessors need to be diligent in following the locking > contract. For example Phoenix does not lock RegionScanner.nextRaw() and > required in the documentation (not picking on Phoenix, this one is my fault > as I told them it's OK) > * possible starving of flushes and compaction with heavy read load. > RegionScanner operations would keep getting the locks and the > flushes/compactions would not be able finalize the set of files. > I'll have a patch soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14895) Seek only to the newly flushed file on scanner reset on flush
[ https://issues.apache.org/jira/browse/HBASE-14895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-14895: --- Attachment: HBASE-14895.patch Patch for QA. > Seek only to the newly flushed file on scanner reset on flush > - > > Key: HBASE-14895 > URL: https://issues.apache.org/jira/browse/HBASE-14895 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-14895.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14895) Seek only to the newly flushed file on scanner reset on flush
[ https://issues.apache.org/jira/browse/HBASE-14895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-14895: --- Status: Patch Available (was: Open) > Seek only to the newly flushed file on scanner reset on flush > - > > Key: HBASE-14895 > URL: https://issues.apache.org/jira/browse/HBASE-14895 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-14895.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14928) Start row should be set for query through HBase REST gateway involving globbing option
Ted Yu created HBASE-14928: -- Summary: Start row should be set for query through HBase REST gateway involving globbing option Key: HBASE-14928 URL: https://issues.apache.org/jira/browse/HBASE-14928 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu As Ben Sutton reported in the thread, Slow response on HBase REST api using globbing option, query through the Rest API with a globbing option i.e. http://:/table/key\* executes extremely slowly. Jerry He pointed out that PrefixFilter is used for query involving globbing option. This issue is to fix this bug by setting start row for such queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14928) Start row should be set for query through HBase REST gateway involving globbing option
[ https://issues.apache.org/jira/browse/HBASE-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14928: --- Attachment: 14928-v1.txt > Start row should be set for query through HBase REST gateway involving > globbing option > -- > > Key: HBASE-14928 > URL: https://issues.apache.org/jira/browse/HBASE-14928 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 14928-v1.txt > > > As Ben Sutton reported in the thread, Slow response on HBase REST api using > globbing option, query through the Rest API with a globbing option i.e. > http://:/table/key\* executes extremely slowly. > Jerry He pointed out that PrefixFilter is used for query involving globbing > option. > This issue is to fix this bug by setting start row for such queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14928) Start row should be set for query through HBase REST gateway involving globbing option
[ https://issues.apache.org/jira/browse/HBASE-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14928: --- Status: Patch Available (was: Open) > Start row should be set for query through HBase REST gateway involving > globbing option > -- > > Key: HBASE-14928 > URL: https://issues.apache.org/jira/browse/HBASE-14928 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 14928-v1.txt > > > As Ben Sutton reported in the thread, Slow response on HBase REST api using > globbing option, query through the Rest API with a globbing option i.e. > http://:/table/key\* executes extremely slowly. > Jerry He pointed out that PrefixFilter is used for query involving globbing > option. > This issue is to fix this bug by setting start row for such queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14928) Start row should be set for query through HBase REST gateway involving globbing option
[ https://issues.apache.org/jira/browse/HBASE-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14928: --- Fix Version/s: 1.1.3 1.3.0 1.2.0 2.0.0 > Start row should be set for query through HBase REST gateway involving > globbing option > -- > > Key: HBASE-14928 > URL: https://issues.apache.org/jira/browse/HBASE-14928 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3 > > Attachments: 14928-v1.txt > > > As Ben Sutton reported in the thread, Slow response on HBase REST api using > globbing option, query through the Rest API with a globbing option i.e. > http://:/table/key\* executes extremely slowly. > Jerry He pointed out that PrefixFilter is used for query involving globbing > option. > This issue is to fix this bug by setting start row for such queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13857) Slow WAL Append count in ServerMetricsTmpl.jamon is hardcoded to zero
[
https://issues.apache.org/jira/browse/HBASE-13857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041651#comment-15041651
]
Vrishal Kulkarni commented on HBASE-13857:
--
There's one failing test
org.apache.hadoop.hbase.ipc.TestAsyncIPC.testRTEDuringAsyncConnectionSetup[0]
The failure is unrelated to this change!
> Slow WAL Append count in ServerMetricsTmpl.jamon is hardcoded to zero
> -
>
> Key: HBASE-13857
> URL: https://issues.apache.org/jira/browse/HBASE-13857
> Project: HBase
> Issue Type: Bug
> Components: regionserver, UI
>Affects Versions: 0.98.0
>Reporter: Lars George
>Assignee: Vrishal Kulkarni
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-13857.patch
>
>
> The template has this:
> {noformat}
>
> ...
> Slow WAL Append Count
>
>
>
> <% 0 %>
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14895) Seek only to the newly flushed file on scanner reset on flush
[
https://issues.apache.org/jira/browse/HBASE-14895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041680#comment-15041680
]
Hadoop QA commented on HBASE-14895:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12775763/HBASE-14895.patch
against master branch at commit 8b3d1f144408e4a7a014c5ac46418c9e91b9b0db.
ATTACHMENT ID: 12775763
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 16 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16766//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16766//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16766//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16766//console
This message is automatically generated.
> Seek only to the newly flushed file on scanner reset on flush
> -
>
> Key: HBASE-14895
> URL: https://issues.apache.org/jira/browse/HBASE-14895
> Project: HBase
> Issue Type: Sub-task
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-14895.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13857) Slow WAL Append count in ServerMetricsTmpl.jamon is hardcoded to zero
[
https://issues.apache.org/jira/browse/HBASE-13857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041650#comment-15041650
]
Vrishal Kulkarni commented on HBASE-13857:
--
There's one failing test
org.apache.hadoop.hbase.ipc.TestAsyncIPC.testRTEDuringAsyncConnectionSetup[0]
The failure is unrelated to this change!
> Slow WAL Append count in ServerMetricsTmpl.jamon is hardcoded to zero
> -
>
> Key: HBASE-13857
> URL: https://issues.apache.org/jira/browse/HBASE-13857
> Project: HBase
> Issue Type: Bug
> Components: regionserver, UI
>Affects Versions: 0.98.0
>Reporter: Lars George
>Assignee: Vrishal Kulkarni
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-13857.patch
>
>
> The template has this:
> {noformat}
>
> ...
> Slow WAL Append Count
>
>
>
> <% 0 %>
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14928) Start row should be set for query through HBase REST gateway involving globbing option
[ https://issues.apache.org/jira/browse/HBASE-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14928: --- Fix Version/s: 0.98.17 > Start row should be set for query through HBase REST gateway involving > globbing option > -- > > Key: HBASE-14928 > URL: https://issues.apache.org/jira/browse/HBASE-14928 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17 > > Attachments: 14928-v1.txt > > > As Ben Sutton reported in the thread, Slow response on HBase REST api using > globbing option, query through the Rest API with a globbing option i.e. > http://:/table/key\* executes extremely slowly. > Jerry He pointed out that PrefixFilter is used for query involving globbing > option. > This issue is to fix this bug by setting start row for such queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14904) Mark Base[En|De]coder LimitedPrivate and fix binary compat issue
[
https://issues.apache.org/jira/browse/HBASE-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041519#comment-15041519
]
Hudson commented on HBASE-14904:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1140 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1140/])
HBASE-14904 Mark Base[En|De]coder LimitedPrivate and fix binary compat (enis:
rev b04a09da352a396bcbb2fbd4bbcbd7aceaa8fdfb)
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseDecoder.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogPrettyPrinter.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseEncoder.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/CompressionContext.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java
> Mark Base[En|De]coder LimitedPrivate and fix binary compat issue
>
>
> Key: HBASE-14904
> URL: https://issues.apache.org/jira/browse/HBASE-14904
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4
>
> Attachments: hbase-14904_v1.patch, hbase-14904_v2.patch
>
>
> PHOENIX-2477 revealed that the changes from HBASE-14501 breaks binary
> compatibility in Phoenix compiled with earlier versions of HBase and run
> agains later versions.
> This is one of the areas that the boundary is not clear, but it won't hurt us
> to fix it.
> The exception trace is:
> {code}
> Exception in thread "main" java.lang.NoSuchFieldError: in
> at
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec$PhoenixBaseDecoder.(IndexedWALEditCodec.java:106)
> at
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec$IndexKeyValueDecoder.(IndexedWALEditCodec.java:121)
> at
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec.getDecoder(IndexedWALEditCodec.java:63)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.initAfterCompression(ProtobufLogReader.java:292)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:82)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.init(ProtobufLogReader.java:148)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:316)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:281)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:269)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:418)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.processFile(WALPrettyPrinter.java:247)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.run(WALPrettyPrinter.java:422)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.main(WALPrettyPrinter.java:357)
> {code}
> Although {{BaseDecoder.in}} is still there, it got changed to be a class
> rather than an interface. BaseDecoder is marked Private, thus the binary
> compat check is not run at all. Not sure whether it would have caught this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14895) Seek only to the newly flushed file on scanner reset on flush
[ https://issues.apache.org/jira/browse/HBASE-14895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041527#comment-15041527 ] Anoop Sam John commented on HBASE-14895: After HBASE-13082, we will notify ChangedReadersObserver only after a flush operation. After bulk load and compaction we dont need inform the readers. After a flush, which results in exactly one file, the notify mechanism can pass that file's detail also? So on that we can do the reset. We will remove the scanner on the snapshot also (?).. So this can be achieved with lesser code change? Rather than maintain new Lists and all? > Seek only to the newly flushed file on scanner reset on flush > - > > Key: HBASE-14895 > URL: https://issues.apache.org/jira/browse/HBASE-14895 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-14895.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14928) Start row should be set for query through HBase REST gateway involving globbing option
[
https://issues.apache.org/jira/browse/HBASE-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041540#comment-15041540
]
Hadoop QA commented on HBASE-14928:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12775755/14928-v1.txt
against master branch at commit 8b3d1f144408e4a7a014c5ac46418c9e91b9b0db.
ATTACHMENT ID: 12775755
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16765//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16765//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16765//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16765//console
This message is automatically generated.
> Start row should be set for query through HBase REST gateway involving
> globbing option
> --
>
> Key: HBASE-14928
> URL: https://issues.apache.org/jira/browse/HBASE-14928
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 14928-v1.txt
>
>
> As Ben Sutton reported in the thread, Slow response on HBase REST api using
> globbing option, query through the Rest API with a globbing option i.e.
> http://:/table/key\* executes extremely slowly.
> Jerry He pointed out that PrefixFilter is used for query involving globbing
> option.
> This issue is to fix this bug by setting start row for such queries.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14904) Mark Base[En|De]coder LimitedPrivate and fix binary compat issue
[
https://issues.apache.org/jira/browse/HBASE-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041541#comment-15041541
]
Hudson commented on HBASE-14904:
FAILURE: Integrated in HBase-0.98-matrix #267 (See
[https://builds.apache.org/job/HBase-0.98-matrix/267/])
HBASE-14904 Mark Base[En|De]coder LimitedPrivate and fix binary compat (enis:
rev b04a09da352a396bcbb2fbd4bbcbd7aceaa8fdfb)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/CompressionContext.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseDecoder.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseEncoder.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogPrettyPrinter.java
> Mark Base[En|De]coder LimitedPrivate and fix binary compat issue
>
>
> Key: HBASE-14904
> URL: https://issues.apache.org/jira/browse/HBASE-14904
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4
>
> Attachments: hbase-14904_v1.patch, hbase-14904_v2.patch
>
>
> PHOENIX-2477 revealed that the changes from HBASE-14501 breaks binary
> compatibility in Phoenix compiled with earlier versions of HBase and run
> agains later versions.
> This is one of the areas that the boundary is not clear, but it won't hurt us
> to fix it.
> The exception trace is:
> {code}
> Exception in thread "main" java.lang.NoSuchFieldError: in
> at
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec$PhoenixBaseDecoder.(IndexedWALEditCodec.java:106)
> at
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec$IndexKeyValueDecoder.(IndexedWALEditCodec.java:121)
> at
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec.getDecoder(IndexedWALEditCodec.java:63)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.initAfterCompression(ProtobufLogReader.java:292)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:82)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.init(ProtobufLogReader.java:148)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:316)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:281)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:269)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:418)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.processFile(WALPrettyPrinter.java:247)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.run(WALPrettyPrinter.java:422)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.main(WALPrettyPrinter.java:357)
> {code}
> Although {{BaseDecoder.in}} is still there, it got changed to be a class
> rather than an interface. BaseDecoder is marked Private, thus the binary
> compat check is not run at all. Not sure whether it would have caught this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14223) Meta WALs are not cleared if meta region was closed and RS aborts
[
https://issues.apache.org/jira/browse/HBASE-14223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041781#comment-15041781
]
Hudson commented on HBASE-14223:
SUCCESS: Integrated in HBase-1.0 #1119 (See
[https://builds.apache.org/job/HBase-1.0/1119/])
Revert "HBASE-14223 Meta WALs are not cleared if meta region was closed (enis:
rev c4aab43a1ac1617890bb4535c9611fa0e31a59c2)
* hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALFactory.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestMetaWALsAreClosed.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
*
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/factories/SlowDeterministicMonkeyFactory.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
*
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/factories/NoKillMonkeyFactory.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/CloseMetaHandler.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/MockRegionServerServices.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/CloseRegionHandler.java
*
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/MoveMetaAction.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java
*
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/factories/StressAssignmentManagerMonkeyFactory.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
> Meta WALs are not cleared if meta region was closed and RS aborts
> -
>
> Key: HBASE-14223
> URL: https://issues.apache.org/jira/browse/HBASE-14223
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 1.0.4
>
> Attachments: HBASE-14223logs, hbase-14223_v0.patch,
> hbase-14223_v1-branch-1.patch, hbase-14223_v2-branch-1.patch,
> hbase-14223_v3-branch-1.patch, hbase-14223_v3-branch-1.patch,
> hbase-14223_v3-master.patch
>
>
> When an RS opens meta, and later closes it, the WAL(FSHlog) is not closed.
> The last WAL file just sits there in the RS WAL directory. If RS stops
> gracefully, the WAL file for meta is deleted. Otherwise if RS aborts, WAL for
> meta is not cleaned. It is also not split (which is correct) since master
> determines that the RS no longer hosts meta at the time of RS abort.
> From a cluster after running ITBLL with CM, I see a lot of {{-splitting}}
> directories left uncleaned:
> {code}
> [root@os-enis-dal-test-jun-4-7 cluster-os]# sudo -u hdfs hadoop fs -ls
> /apps/hbase/data/WALs
> Found 31 items
> drwxr-xr-x - hbase hadoop 0 2015-06-05 01:14
> /apps/hbase/data/WALs/hregion-58203265
> drwxr-xr-x - hbase hadoop 0 2015-06-05 07:54
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433489308745-splitting
> drwxr-xr-x - hbase hadoop 0 2015-06-05 09:28
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433494382959-splitting
> drwxr-xr-x - hbase hadoop 0 2015-06-05 10:01
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433498252205-splitting
> ...
> {code}
> The directories contain WALs from meta:
> {code}
> [root@os-enis-dal-test-jun-4-7 cluster-os]# sudo -u hdfs hadoop fs -ls
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting
> Found 2 items
> -rw-r--r-- 3 hbase hadoop 201608 2015-06-05 03:15
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433470511501.meta
> -rw-r--r-- 3 hbase hadoop 44420 2015-06-05 04:36
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433474111645.meta
> {code}
> The RS hosted the meta region for some time:
> {code}
> 2015-06-05 03:14:28,692 INFO [PostOpenDeployTasks:1588230740]
> zookeeper.MetaTableLocator: Setting hbase:meta region location in ZooKeeper
> as os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285
> ...
> 2015-06-05 03:15:17,302 INFO
> [RS_CLOSE_META-os-enis-dal-test-jun-4-5:16020-0] regionserver.HRegion: Closed
> hbase:meta,,1.1588230740
> {code}
> In between, a WAL is created:
> {code}
> 2015-06-05 03:15:11,707 INFO
> [RS_OPEN_META-os-enis-dal-test-jun-4-5:16020-0-MetaLogRoller]
[jira] [Commented] (HBASE-14904) Mark Base[En|De]coder LimitedPrivate and fix binary compat issue
[
https://issues.apache.org/jira/browse/HBASE-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041782#comment-15041782
]
Hudson commented on HBASE-14904:
SUCCESS: Integrated in HBase-1.0 #1119 (See
[https://builds.apache.org/job/HBase-1.0/1119/])
HBASE-14904 Mark Base[En|De]coder LimitedPrivate and fix binary compat (enis:
rev 46536baa97d3c15a3c2233ea26e7b9034dfa520c)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseDecoder.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/codec/BaseEncoder.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/CompressionContext.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALPrettyPrinter.java
> Mark Base[En|De]coder LimitedPrivate and fix binary compat issue
>
>
> Key: HBASE-14904
> URL: https://issues.apache.org/jira/browse/HBASE-14904
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4
>
> Attachments: hbase-14904_v1.patch, hbase-14904_v2.patch
>
>
> PHOENIX-2477 revealed that the changes from HBASE-14501 breaks binary
> compatibility in Phoenix compiled with earlier versions of HBase and run
> agains later versions.
> This is one of the areas that the boundary is not clear, but it won't hurt us
> to fix it.
> The exception trace is:
> {code}
> Exception in thread "main" java.lang.NoSuchFieldError: in
> at
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec$PhoenixBaseDecoder.(IndexedWALEditCodec.java:106)
> at
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec$IndexKeyValueDecoder.(IndexedWALEditCodec.java:121)
> at
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec.getDecoder(IndexedWALEditCodec.java:63)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.initAfterCompression(ProtobufLogReader.java:292)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:82)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.init(ProtobufLogReader.java:148)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:316)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:281)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:269)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:418)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.processFile(WALPrettyPrinter.java:247)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.run(WALPrettyPrinter.java:422)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.main(WALPrettyPrinter.java:357)
> {code}
> Although {{BaseDecoder.in}} is still there, it got changed to be a class
> rather than an interface. BaseDecoder is marked Private, thus the binary
> compat check is not run at all. Not sure whether it would have caught this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14719) Add metric for number of MasterProcWALs
[ https://issues.apache.org/jira/browse/HBASE-14719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041854#comment-15041854 ] Vrishal Kulkarni commented on HBASE-14719: -- Have you had a chance? > Add metric for number of MasterProcWALs > --- > > Key: HBASE-14719 > URL: https://issues.apache.org/jira/browse/HBASE-14719 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Vrishal Kulkarni > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-14719.1.patch, HBASE-14719.2.patch, > HBASE-14719.3.patch, HBASE-14719.patch > > > Lets add monitoring to this so that we can see when it starts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14928) Start row should be set for query through HBase REST gateway involving globbing option
[ https://issues.apache.org/jira/browse/HBASE-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041859#comment-15041859 ] Nick Dimiduk commented on HBASE-14928: -- Perhaps the use of PrefixFilter should be detected and a startkey be added automatically if one is not present. > Start row should be set for query through HBase REST gateway involving > globbing option > -- > > Key: HBASE-14928 > URL: https://issues.apache.org/jira/browse/HBASE-14928 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17 > > Attachments: 14928-v1.txt > > > As Ben Sutton reported in the thread, Slow response on HBase REST api using > globbing option, query through the Rest API with a globbing option i.e. > http://:/table/key\* executes extremely slowly. > Jerry He pointed out that PrefixFilter is used for query involving globbing > option. > This issue is to fix this bug by setting start row for such queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14900) Make global config option for ReplicationEndpoint
[ https://issues.apache.org/jira/browse/HBASE-14900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041764#comment-15041764 ] Cody Marcel commented on HBASE-14900: - I should have a patch today. I got side tracked. I was also trying to refactor the existing tests a little to leverage those. Unsurprisingly that's taking longer than the few lines for the code change ;) > Make global config option for ReplicationEndpoint > - > > Key: HBASE-14900 > URL: https://issues.apache.org/jira/browse/HBASE-14900 > Project: HBase > Issue Type: Sub-task > Components: Replication >Affects Versions: 2.0.0 >Reporter: Cody Marcel >Assignee: Cody Marcel >Priority: Minor > Fix For: 2.0.0 > > > Currently ReplicationEndpoint implementations can only be configured through > the HBase shell. We should be able to to use a property in the hbase-site.xml > to globally set an alternate default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-14929) There is a space missing from Table "foo" is not currently available.
[
https://issues.apache.org/jira/browse/HBASE-14929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carlos A. Morillo reassigned HBASE-14929:
-
Assignee: Carlos A. Morillo
> There is a space missing from Table "foo" is not currently available.
> -
>
> Key: HBASE-14929
> URL: https://issues.apache.org/jira/browse/HBASE-14929
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Malaska
>Assignee: Carlos A. Morillo
>Priority: Trivial
>
> Go to the following line in LoadIncrementalHFiles.java
> throw new TableNotFoundException("Table " + table.getName() + "is not
> currently available.");
> and add a space before is and after '
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14926) Hung ThriftServer; no timeout on read from client; if client crashes, worker thread gets stuck reading
[
https://issues.apache.org/jira/browse/HBASE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041735#comment-15041735
]
Andrew Purtell commented on HBASE-14926:
bq. I'm a bit stuck on how to manufacture this circumstance in a test
Yeah, hard to mock
It's an obvious improvement even without a test, I'd be +1 to commit as is
> Hung ThriftServer; no timeout on read from client; if client crashes, worker
> thread gets stuck reading
> --
>
> Key: HBASE-14926
> URL: https://issues.apache.org/jira/browse/HBASE-14926
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 1.0.3, 0.98.16
>Reporter: stack
>Assignee: stack
> Attachments: 14926.patch
>
>
> Thrift server is hung. All worker threads are doing this:
> {code}
> "thrift-worker-0" daemon prio=10 tid=0x7f0bb95c2800 nid=0xf6a7 runnable
> [0x7f0b956e]
>java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> - locked <0x00066d859490> (a java.io.BufferedInputStream)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> at
> org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:601)
> at
> org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:470)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
> at
> org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
> at
> org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> They never recover.
> I don't have client side logs.
> We've been here before: HBASE-4967 "connected client thrift sockets should
> have a server side read timeout" but this patch only got applied to fb branch
> (and thrift has changed since then).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work
[ https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041711#comment-15041711 ] Andrew Purtell commented on HBASE-14822: lgtm, and has the benefit of not needing to be concerned about the behavior of older server side code. New flag will simply be ignored. > Renewing leases of scanners doesn't work > > > Key: HBASE-14822 > URL: https://issues.apache.org/jira/browse/HBASE-14822 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.14 >Reporter: Samarth Jain >Assignee: Lars Hofhansl > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4 > > Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt, > 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14869) Better request latency histograms
[ https://issues.apache.org/jira/browse/HBASE-14869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041714#comment-15041714 ] Andrew Purtell commented on HBASE-14869: Ok, no further concerns here > Better request latency histograms > - > > Key: HBASE-14869 > URL: https://issues.apache.org/jira/browse/HBASE-14869 > Project: HBase > Issue Type: Brainstorming >Reporter: Lars Hofhansl >Assignee: Vikas Vishwakarma > Fix For: 2.0.0, 1.3.0, 0.98.17 > > Attachments: 14869-test-0.98.txt, 14869-v1-0.98.txt, > 14869-v1-2.0.txt, 14869-v2-0.98.txt, 14869-v2-2.0.txt, 14869-v3-0.98.txt, > 14869-v4-0.98.txt, 14869-v5-0.98.txt, AppendSizeTime.png, Get.png > > > I just discussed this with a colleague. > The get, put, etc, histograms that each region server keeps are somewhat > useless (depending on what you want to achieve of course), as they are > aggregated and calculated by each region server. > It would be better to record the number of requests in certainly latency > bands in addition to what we do now. > For example the number of gets that took 0-5ms, 6-10ms, 10-20ms, 20-50ms, > 50-100ms, 100-1000ms, > 1000ms, etc. (just as an example, should be > configurable). > That way we can do further calculations after the fact, and answer questions > like: How often did we miss our SLA? Percentage of requests that missed an > SLA, etc. > Comments? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only
[
https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041761#comment-15041761
]
Yu Li commented on HBASE-14790:
---
bq. hsync periodically, for example, default per 1000ms
AFAIK, DN will do this if {{dfs.datanode.sync.behind.writes}} is set to true,
and the window is 8MB, just correct me if I'm wrong.
Sorry for the interruption but is the discussion here back to the issue
HBASE-14004 reports, say preventing ReplicationSource to replicate any
non-persisted and later-lost data, due to the fact that we depends on hflush
not hsync currently? And it seems we could not resolve the issue by
implementing new DFSOutputStream?
Would it be a better idea to continue the discussion on master-slave
replication consistency issue in HBASE-14004, while focusing on the new
DFSOutputStream here, to make "an optimized DataOutputStream for small writes
like WAL entry"?
> Implement a new DFSOutputStream for logging WAL only
>
>
> Key: HBASE-14790
> URL: https://issues.apache.org/jira/browse/HBASE-14790
> Project: HBase
> Issue Type: Improvement
>Reporter: Duo Zhang
>
> The original {{DFSOutputStream}} is very powerful and aims to serve all
> purposes. But in fact, we do not need most of the features if we only want to
> log WAL. For example, we do not need pipeline recovery since we could just
> close the old logger and open a new one. And also, we do not need to write
> multiple blocks since we could also open a new logger if the old file is too
> large.
> And the most important thing is that, it is hard to handle all the corner
> cases to avoid data loss or data inconsistency(such as HBASE-14004) when
> using original DFSOutputStream due to its complicated logic. And the
> complicated logic also force us to use some magical tricks to increase
> performance. For example, we need to use multiple threads to call {{hflush}}
> when logging, and now we use 5 threads. But why 5 not 10 or 100?
> So here, I propose we should implement our own {{DFSOutputStream}} when
> logging WAL. For correctness, and also for performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14917) Log in console if individual tests in test-patch.sh fail or pass.
[ https://issues.apache.org/jira/browse/HBASE-14917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-14917: - Attachment: HBASE-14917-v2.patch v2: same patch, new name to trigger the build > Log in console if individual tests in test-patch.sh fail or pass. > - > > Key: HBASE-14917 > URL: https://issues.apache.org/jira/browse/HBASE-14917 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-14917-v2.patch, HBASE-14917.patch > > > Got 2 runs like > https://issues.apache.org/jira/browse/HBASE-14865?focusedCommentId=15037056=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15037056 > where can't figure out what went wrong in patch testing. > Logging results from individual tests to console as discussed > [here|https://mail-archives.apache.org/mod_mbox/hbase-dev/201512.mbox/%3CCAAjhxrrL4-qty562%3DcMyBJ2xyhGqHi3MFAgf9ygrzQf1%2BZmHtw%40mail.gmail.com%3E] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042285#comment-15042285 ] Enis Soztutar commented on HBASE-6721: -- Thanks [~toffer] for the updated patch. All of the review items have been addressed, and hence I'm +1 for committing this. [~apurtell], [~eclark] or others have any more concerns, feedback? Given that this is an optional coprocessor based thing, I think this can directly go to master. > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, immediateAssignments Sequence Diagram.svg, > randomAssignment Sequence Diagram.svg, retainAssignment Sequence Diagram.svg, > roundRobinAssignment Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[
https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042319#comment-15042319
]
Ted Yu commented on HBASE-6721:
---
Running TestShell with latest patch, I got:
{code}
1) Error:
test_Test_Basic_Group_Commands(Hbase::GroupShellTest):
NativeException: org.apache.hadoop.hbase.exceptions.UnknownProtocolException:
org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered
master coprocessor service found for name hbase.pb.GroupAdminService
at
org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:682)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:57964)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2184)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
...
2) Failure:
test_Test_bogus_arguments(Hbase::GroupShellTest)
[./src/test/ruby/shell/group_shell_test.rb:85:in `test_Test_bogus_arguments'
org/jruby/RubyProc.java:270:in `call'
org/jruby/RubyKernel.java:2105:in `send'
org/jruby/RubyArray.java:1620:in `each'
org/jruby/RubyArray.java:1620:in `each']:
exception expected but was
Class:
Message: <"org.apache.hadoop.hbase.exceptions.UnknownProtocolException:
org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered
master coprocessor service found for name hbase.pb.GroupAdminService\n\tat
org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:682)\n\tat
org.apache.hadoop.hbase.
protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:57964)\n\tat
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2184)\n\tat org.
apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)\n\tat
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)\n\tat
org.apache.hadoop.hbase.ipc. RpcExecutor$1.run(RpcExecutor.java:108)\n\tat
java.lang.Thread.run(Thread.java:745)\n">
{code}
Francis:
Do the tests pass for you ?
> RegionServer Group based Assignment
> ---
>
> Key: HBASE-6721
> URL: https://issues.apache.org/jira/browse/HBASE-6721
> Project: HBase
> Issue Type: New Feature
>Reporter: Francis Liu
>Assignee: Francis Liu
> Labels: hbase-6721
> Attachments: 6721-master-webUI.patch, HBASE-6721
> GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf,
> HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf,
> HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch,
> HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch,
> HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch,
> HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch,
> HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch,
> HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch,
> HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch,
> HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch,
> HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch,
> HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg,
> hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch,
> hbase-6721-v18.patch, immediateAssignments Sequence Diagram.svg,
> randomAssignment Sequence Diagram.svg, retainAssignment Sequence Diagram.svg,
> roundRobinAssignment Sequence Diagram.svg
>
>
> In multi-tenant deployments of HBase, it is likely that a RegionServer will
> be serving out regions from a number of different tables owned by various
> client applications. Being able to group a subset of running RegionServers
> and assign specific tables to it, provides a client application a level of
> isolation and resource allocation.
> The proposal essentially is to have an AssignmentManager which is aware of
> RegionServer groups and assigns tables to region servers based on groupings.
> Load balancing will occur on a per group basis as well.
> This is essentially a simplification of the approach taken in HBASE-4120. See
> attached document.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14930) check_compatibility.sh needs smarter exit codes
[ https://issues.apache.org/jira/browse/HBASE-14930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-14930: Attachment: HBASE-14930_master_v1.patch Patch that changes the exit codes we use during the setup before running the tool from "1" to "2." This lets us continue to use the script's exit code overall as the exit code of the Java API Compliance Checker tool. > check_compatibility.sh needs smarter exit codes > --- > > Key: HBASE-14930 > URL: https://issues.apache.org/jira/browse/HBASE-14930 > Project: HBase > Issue Type: Bug >Reporter: Dima Spivak >Assignee: Dima Spivak > Attachments: HBASE-14930_master_v1.patch > > > The check_compatibility.sh tool in dev_support uses the Java API Compliance > Checker to do static analysis of source/binary incompatibilties between two > HBase branches. One problem, though, is that the script has a few instances > where it may return an exit code of 1 (e.g. if Maven steps fail), but this is > the same exit code that the Java ACC tool itself uses to denote that the tool > succeeded, but found incompatibilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14930) check_compatibility.sh needs smarter exit codes
[ https://issues.apache.org/jira/browse/HBASE-14930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-14930: Status: Patch Available (was: Open) > check_compatibility.sh needs smarter exit codes > --- > > Key: HBASE-14930 > URL: https://issues.apache.org/jira/browse/HBASE-14930 > Project: HBase > Issue Type: Bug >Reporter: Dima Spivak >Assignee: Dima Spivak > Attachments: HBASE-14930_master_v1.patch > > > The check_compatibility.sh tool in dev_support uses the Java API Compliance > Checker to do static analysis of source/binary incompatibilties between two > HBase branches. One problem, though, is that the script has a few instances > where it may return an exit code of 1 (e.g. if Maven steps fail), but this is > the same exit code that the Java ACC tool itself uses to denote that the tool > succeeded, but found incompatibilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14580) Make the HBaseMiniCluster compliant with Kerberos
[
https://issues.apache.org/jira/browse/HBASE-14580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-14580:
Fix Version/s: (was: 1.2.1)
1.2.0
> Make the HBaseMiniCluster compliant with Kerberos
> -
>
> Key: HBASE-14580
> URL: https://issues.apache.org/jira/browse/HBASE-14580
> Project: HBase
> Issue Type: Improvement
> Components: security, test
>Affects Versions: 2.0.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.16
>
> Attachments: hbase-14580.v2.patch, hbase-14580.v2.patch,
> patch-14580.v1.patch
>
>
> Whne using MiniKDC and the minicluster in a unit test, there is a conflict
> causeed by HBaseTestingUtility:
> {code}
> public static User getDifferentUser(final Configuration c,
> final String differentiatingSuffix)
> throws IOException {
>// snip
> String username = User.getCurrent().getName() +
> differentiatingSuffix; < problem here
> User user = User.createUserForTesting(c, username,
> new String[]{"supergroup"});
> return user;
> }
> {code}
> This creates users like securedUser/localh...@example.com.hfs.0, and this
> does not work.
> My fix is to return the current user when Kerberos is set. I don't think that
> there is another option (any other opinion?). However this user is not in a
> group so we have logs like 'WARN [IPC Server handler 9 on 61366]
> security.UserGroupInformation (UserGroupInformation.java:getGroupNames(1521))
> - No groups available for user securedUser' I'm not sure of its impact.
> [~apurtell], what do you think?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14014) Explore row-by-row grouping options
[ https://issues.apache.org/jira/browse/HBASE-14014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042070#comment-15042070 ] Sean Busbey commented on HBASE-14014: - Parent was HBASE-12988 > Explore row-by-row grouping options > --- > > Key: HBASE-14014 > URL: https://issues.apache.org/jira/browse/HBASE-14014 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Lars Hofhansl > > See discussion in parent. > We need to considering the following attributes of WALKey: > * The cluster ids > * Table Name > * write time (here we could use the latest of any batch) > * seqNum > As long as we preserve these we can rearrange the cells between WALEdits. > Since seqNum is unique this will be a challenge. Currently it is not used, > but we shouldn't design anything that prevents us guaranteeing better > ordering guarantees using seqNum. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-14580) Make the HBaseMiniCluster compliant with Kerberos
[
https://issues.apache.org/jira/browse/HBASE-14580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reopened HBASE-14580:
-
> Make the HBaseMiniCluster compliant with Kerberos
> -
>
> Key: HBASE-14580
> URL: https://issues.apache.org/jira/browse/HBASE-14580
> Project: HBase
> Issue Type: Improvement
> Components: security, test
>Affects Versions: 2.0.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.16
>
> Attachments: hbase-14580.v2.patch, hbase-14580.v2.patch,
> patch-14580.v1.patch
>
>
> Whne using MiniKDC and the minicluster in a unit test, there is a conflict
> causeed by HBaseTestingUtility:
> {code}
> public static User getDifferentUser(final Configuration c,
> final String differentiatingSuffix)
> throws IOException {
>// snip
> String username = User.getCurrent().getName() +
> differentiatingSuffix; < problem here
> User user = User.createUserForTesting(c, username,
> new String[]{"supergroup"});
> return user;
> }
> {code}
> This creates users like securedUser/localh...@example.com.hfs.0, and this
> does not work.
> My fix is to return the current user when Kerberos is set. I don't think that
> there is another option (any other opinion?). However this user is not in a
> group so we have logs like 'WARN [IPC Server handler 9 on 61366]
> security.UserGroupInformation (UserGroupInformation.java:getGroupNames(1521))
> - No groups available for user securedUser' I'm not sure of its impact.
> [~apurtell], what do you think?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14089) Remove unnecessary draw of system entropy from RecoverableZooKeeper
[ https://issues.apache.org/jira/browse/HBASE-14089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14089: Fix Version/s: (was: 1.2.1) 1.2.0 > Remove unnecessary draw of system entropy from RecoverableZooKeeper > --- > > Key: HBASE-14089 > URL: https://issues.apache.org/jira/browse/HBASE-14089 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.2, 1.3.0 > > Attachments: HBASE-14089.patch > > > I had a look at instances where we use SecureRandom, which could block if > insufficient entropy, in the 0.98 and master branch code. (Random in contrast > is a PRNG seeded by System#nanoTime, it doesn't draw from system entropy.) > Most uses are in encryption related code, our native encryption and SSL, but > we do also use SecureRandom for salting znode metadata in > RecoverableZooKeeper#appendMetadata, which is called whenever we do setData. > Conceivably we could block unexpectedly when constructing data to write out > to a znode if entropy gets too low until more is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-14580) Make the HBaseMiniCluster compliant with Kerberos
[
https://issues.apache.org/jira/browse/HBASE-14580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey resolved HBASE-14580.
-
Resolution: Fixed
> Make the HBaseMiniCluster compliant with Kerberos
> -
>
> Key: HBASE-14580
> URL: https://issues.apache.org/jira/browse/HBASE-14580
> Project: HBase
> Issue Type: Improvement
> Components: security, test
>Affects Versions: 2.0.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.16
>
> Attachments: hbase-14580.v2.patch, hbase-14580.v2.patch,
> patch-14580.v1.patch
>
>
> Whne using MiniKDC and the minicluster in a unit test, there is a conflict
> causeed by HBaseTestingUtility:
> {code}
> public static User getDifferentUser(final Configuration c,
> final String differentiatingSuffix)
> throws IOException {
>// snip
> String username = User.getCurrent().getName() +
> differentiatingSuffix; < problem here
> User user = User.createUserForTesting(c, username,
> new String[]{"supergroup"});
> return user;
> }
> {code}
> This creates users like securedUser/localh...@example.com.hfs.0, and this
> does not work.
> My fix is to return the current user when Kerberos is set. I don't think that
> there is another option (any other opinion?). However this user is not in a
> group so we have logs like 'WARN [IPC Server handler 9 on 61366]
> security.UserGroupInformation (UserGroupInformation.java:getGroupNames(1521))
> - No groups available for user securedUser' I'm not sure of its impact.
> [~apurtell], what do you think?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14021) Quota table has a wrong description on the UI
[ https://issues.apache.org/jira/browse/HBASE-14021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14021: Fix Version/s: (was: 1.2.1) 1.2.0 > Quota table has a wrong description on the UI > - > > Key: HBASE-14021 > URL: https://issues.apache.org/jira/browse/HBASE-14021 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 1.1.0 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.1.2, 1.3.0 > > Attachments: HBASE-14021.patch, HBASE-14021.patch, > HBASE-14021branch-1.1.patch, error.png, fix.png > > > !error.png! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14758) Add UT case for unchecked error/exception thrown in AsyncProcess#sendMultiAction
[ https://issues.apache.org/jira/browse/HBASE-14758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14758: Fix Version/s: (was: 1.2.1) 1.2.0 > Add UT case for unchecked error/exception thrown in > AsyncProcess#sendMultiAction > > > Key: HBASE-14758 > URL: https://issues.apache.org/jira/browse/HBASE-14758 > Project: HBase > Issue Type: Test >Affects Versions: 1.1.2, 0.98.15 >Reporter: Yu Li >Assignee: Yu Li >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 0.98.17, 1.0.4 > > Attachments: HBASE-14758.branch-0.98.patch, HBASE-14758.patch > > > This is an addendum for HBASE-14359, open a new JIRA to trigger HadoopQA run -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14539) Slight improvement of StoreScanner.optimize
[ https://issues.apache.org/jira/browse/HBASE-14539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14539: Fix Version/s: (was: 1.2.1) 1.2.0 > Slight improvement of StoreScanner.optimize > --- > > Key: HBASE-14539 > URL: https://issues.apache.org/jira/browse/HBASE-14539 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.15, 1.0.3, 1.1.3 > > Attachments: 14539-0.98.txt > > > While looking at the code I noticed that StoreScanner.optimize does not some > unnecessary work. This is a very tight loop and even just looking up a > reference can throw off the CPUs cache lines. This does safe a few percent of > performance (not a lot, though). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14734) TestGenerateDelegationToken fails with BindAddress clash
[ https://issues.apache.org/jira/browse/HBASE-14734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042111#comment-15042111 ] stack commented on HBASE-14734: --- Again https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.3/lastCompletedBuild/jdk=latest1.7,label=Hadoop/testReport/org.apache.hadoop.hbase.security.token/TestGenerateDelegationToken/org_apache_hadoop_hbase_security_token_TestGenerateDelegationToken/ > TestGenerateDelegationToken fails with BindAddress clash > > > Key: HBASE-14734 > URL: https://issues.apache.org/jira/browse/HBASE-14734 > Project: HBase > Issue Type: Sub-task > Components: flakey, test >Reporter: stack > > From > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/330/jdk=latest1.7,label=Hadoop/testReport/junit/org.apache.hadoop.hbase.security.token/TestGenerateDelegationToken/org_apache_hadoop_hbase_security_token_TestGenerateDelegationToken/ > Error Message > Address already in use > Stacktrace > java.net.BindException: Address already in use > at sun.nio.ch.Net.bind0(Native Method) > at sun.nio.ch.Net.bind(Net.java:444) > at sun.nio.ch.Net.bind(Net.java:436) > at > sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214) > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) > at > org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:198) > at > org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:51) > at > org.apache.mina.core.polling.AbstractPollingIoAcceptor.registerHandles(AbstractPollingIoAcceptor.java:547) > at > org.apache.mina.core.polling.AbstractPollingIoAcceptor.access$400(AbstractPollingIoAcceptor.java:68) > at > org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:422) > at > org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Can this utility be made to not fail if address taken? Try another? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14891) Add log for uncaught exception in RegionServerMetricsWrapperRunnable
[
https://issues.apache.org/jira/browse/HBASE-14891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-14891:
Fix Version/s: (was: 1.2.1)
1.2.0
> Add log for uncaught exception in RegionServerMetricsWrapperRunnable
>
>
> Key: HBASE-14891
> URL: https://issues.apache.org/jira/browse/HBASE-14891
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14891.branch-1.2.patch,
> HBASE-14891.branch-1.patch, HBASE-14891.patch, HBASE-14891.v2.patch,
> HBASE-14891.v2.patch
>
>
> We are using ScheduledExecutorService#scheduleWithFixedDelay to periodically
> reporting metrics. From javadoc of this method
> {quote}
> If any execution of the task encounters an exception, subsequent executions
> are suppressed.
> {quote}
> We could see if any uncaught exception like NPE thrown in the task thread,
> there won't be any error thrown but the task would be silently terminated.
> In our real case we were making some modification to the metrics and causing
> NPE carelessly, then we found metrics disappeared but cannot get a clue from
> RS log.
> Suggest to catch Throwable in
> MetricsRegionServerWrapperImpl$RegionServerMetricsWrapperRunnable, add some
> log there and suppress the exception to give it another try at next scheduled
> run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14903) Table Or Region?
[ https://issues.apache.org/jira/browse/HBASE-14903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042122#comment-15042122 ] Andrew Purtell commented on HBASE-14903: Prepare a patch for the site documentation and attach it here. > Table Or Region? > > > Key: HBASE-14903 > URL: https://issues.apache.org/jira/browse/HBASE-14903 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 2.0.0 >Reporter: 胡托 >Priority: Blocker > > I've been reading on Latest Reference Guide and try to translated into > Chinese! > I think this sentence "When a table is in the process of splitting," > should be "When a Region is in the process of splitting," on chapter 【62.2. > hbase:meta】。 > By the way,is this document the > latest?【http://hbase.apache.org/book.html#arch.overview】I will translate it！ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14354) Minor improvements for usage of the mlock agent
[ https://issues.apache.org/jira/browse/HBASE-14354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14354: Fix Version/s: (was: 1.2.1) 1.2.0 > Minor improvements for usage of the mlock agent > --- > > Key: HBASE-14354 > URL: https://issues.apache.org/jira/browse/HBASE-14354 > Project: HBase > Issue Type: Bug > Components: hbase, regionserver >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez >Priority: Trivial > Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.15, 1.1.3 > > Attachments: > 0001-HBASE-14354-Minor-improvements-for-usage-of-the-mloc.patch, > HBASE-14354.v1.patch > > > 1. MLOCK_AGENT points to the wrong path in hbase-config.sh > When the mlock agent is build, the binary is installed under > $HBASE_HOME/lib/native and not under $HBASE_HOME/native > 2. By default we pass $HBASE_REGIONSERVER_UID to the agent options which > causes the mlock agent to attempt to do a setuid in order to mlock the memory > of the RS process. We should only pass that user if specified in the > environment, not by default. (the agent currently handles that gracefully) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14769) Remove unused functions and duplicate javadocs from HBaseAdmin
[ https://issues.apache.org/jira/browse/HBASE-14769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-14769: - Attachment: HBASE-14769-master-v9.patch v6 as v9 to trigger build. > Remove unused functions and duplicate javadocs from HBaseAdmin > --- > > Key: HBASE-14769 > URL: https://issues.apache.org/jira/browse/HBASE-14769 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Fix For: 2.0.0 > > Attachments: HBASE-14769-master-v2.patch, > HBASE-14769-master-v3.patch, HBASE-14769-master-v4.patch, > HBASE-14769-master-v5.patch, HBASE-14769-master-v6.patch, > HBASE-14769-master-v7.patch, HBASE-14769-master-v8.patch, > HBASE-14769-master-v9.patch, HBASE-14769-master.patch > > > HBaseAdmin is marked private, so removing the functions not being used > anywhere. > Also, the javadocs of overridden functions are same as corresponding ones in > Admin.java. Since javadocs are automatically inherited from the interface > class, we can remove these redundant 100s of lines. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-4698) Let the HFile Pretty Printer print all the key values for a specific row.
[ https://issues.apache.org/jira/browse/HBASE-4698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov reassigned HBASE-4698: -- Assignee: Mikhail Antonov (was: Liyin Tang) > Let the HFile Pretty Printer print all the key values for a specific row. > - > > Key: HBASE-4698 > URL: https://issues.apache.org/jira/browse/HBASE-4698 > Project: HBase > Issue Type: New Feature >Reporter: Liyin Tang >Assignee: Mikhail Antonov > Fix For: 0.94.0 > > Attachments: ASF.LICENSE.NOT.GRANTED--D111.1.patch, > ASF.LICENSE.NOT.GRANTED--D111.1.patch, ASF.LICENSE.NOT.GRANTED--D111.1.patch, > ASF.LICENSE.NOT.GRANTED--D111.2.patch, ASF.LICENSE.NOT.GRANTED--D111.3.patch, > ASF.LICENSE.NOT.GRANTED--D111.4.patch, HBASE-4689-trunk.patch > > > When using HFile Pretty Printer to debug HBase issues, > it would very nice to allow the Pretty Printer to seek to a specific row, and > only print all the key values for this row. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only
[
https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042228#comment-15042228
]
Zhe Zhang commented on HBASE-14790:
---
Thanks for the catch. If needed I guess we can make it public. Or at least have
a public API to return the acked length.
> Implement a new DFSOutputStream for logging WAL only
>
>
> Key: HBASE-14790
> URL: https://issues.apache.org/jira/browse/HBASE-14790
> Project: HBase
> Issue Type: Improvement
>Reporter: Duo Zhang
>
> The original {{DFSOutputStream}} is very powerful and aims to serve all
> purposes. But in fact, we do not need most of the features if we only want to
> log WAL. For example, we do not need pipeline recovery since we could just
> close the old logger and open a new one. And also, we do not need to write
> multiple blocks since we could also open a new logger if the old file is too
> large.
> And the most important thing is that, it is hard to handle all the corner
> cases to avoid data loss or data inconsistency(such as HBASE-14004) when
> using original DFSOutputStream due to its complicated logic. And the
> complicated logic also force us to use some magical tricks to increase
> performance. For example, we need to use multiple threads to call {{hflush}}
> when logging, and now we use 5 threads. But why 5 not 10 or 100?
> So here, I propose we should implement our own {{DFSOutputStream}} when
> logging WAL. For correctness, and also for performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14928) Start row should be set for query through HBase REST gateway involving globbing option
[ https://issues.apache.org/jira/browse/HBASE-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14928: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the review, Jerry. > Start row should be set for query through HBase REST gateway involving > globbing option > -- > > Key: HBASE-14928 > URL: https://issues.apache.org/jira/browse/HBASE-14928 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17 > > Attachments: 14928-v1.txt > > > As Ben Sutton reported in the thread, Slow response on HBase REST api using > globbing option, query through the Rest API with a globbing option i.e. > http://:/table/key\* executes extremely slowly. > Jerry He pointed out that PrefixFilter is used for query involving globbing > option. > This issue is to fix this bug by setting start row for such queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14901) There is duplicated code to create/manage encryption keys
[ https://issues.apache.org/jira/browse/HBASE-14901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nate Edel updated HBASE-14901: -- Status: Patch Available (was: Open) > There is duplicated code to create/manage encryption keys > - > > Key: HBASE-14901 > URL: https://issues.apache.org/jira/browse/HBASE-14901 > Project: HBase > Issue Type: Bug > Components: encryption >Affects Versions: 2.0.0 >Reporter: Nate Edel >Assignee: Nate Edel >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-14901.1.patch, HBASE-14901.2.patch, > HBASE-14901.3.patch, HBASE-14901.5.patch, HBASE-14901.6.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > There is duplicated code from MobUtils.createEncryptionContext in HStore, and > there is a subset of that code in HFileReaderImpl. > Refactored key selection > Moved both to EncryptionUtil.java > Can't figure out how to write a unit test for this, but there's no new code > just refactoring. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14901) There is duplicated code to create/manage encryption keys
[ https://issues.apache.org/jira/browse/HBASE-14901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nate Edel updated HBASE-14901: -- Attachment: HBASE-14901.6.patch Merge in HStore from change yesterday... > There is duplicated code to create/manage encryption keys > - > > Key: HBASE-14901 > URL: https://issues.apache.org/jira/browse/HBASE-14901 > Project: HBase > Issue Type: Bug > Components: encryption >Affects Versions: 2.0.0 >Reporter: Nate Edel >Assignee: Nate Edel >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-14901.1.patch, HBASE-14901.2.patch, > HBASE-14901.3.patch, HBASE-14901.5.patch, HBASE-14901.6.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > There is duplicated code from MobUtils.createEncryptionContext in HStore, and > there is a subset of that code in HFileReaderImpl. > Refactored key selection > Moved both to EncryptionUtil.java > Can't figure out how to write a unit test for this, but there's no new code > just refactoring. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-4698) Let the HFile Pretty Printer print all the key values for a specific row.
[ https://issues.apache.org/jira/browse/HBASE-4698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov reassigned HBASE-4698: -- Assignee: Liyin Tang (was: Mikhail Antonov) oops, reassigned by mistake > Let the HFile Pretty Printer print all the key values for a specific row. > - > > Key: HBASE-4698 > URL: https://issues.apache.org/jira/browse/HBASE-4698 > Project: HBase > Issue Type: New Feature >Reporter: Liyin Tang >Assignee: Liyin Tang > Fix For: 0.94.0 > > Attachments: ASF.LICENSE.NOT.GRANTED--D111.1.patch, > ASF.LICENSE.NOT.GRANTED--D111.1.patch, ASF.LICENSE.NOT.GRANTED--D111.1.patch, > ASF.LICENSE.NOT.GRANTED--D111.2.patch, ASF.LICENSE.NOT.GRANTED--D111.3.patch, > ASF.LICENSE.NOT.GRANTED--D111.4.patch, HBASE-4689-trunk.patch > > > When using HFile Pretty Printer to debug HBase issues, > it would very nice to allow the Pretty Printer to seek to a specific row, and > only print all the key values for this row. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14930) check_compatibility.sh needs smarter exit codes
[
https://issues.apache.org/jira/browse/HBASE-14930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042405#comment-15042405
]
Hadoop QA commented on HBASE-14930:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12775837/HBASE-14930_master_v1.patch
against master branch at commit d18c1af3bb12d6bf1800f22a5289bcb71020bb65.
ATTACHMENT ID: 12775837
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation, build,
or dev-support patch that doesn't require tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16768//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16768//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16768//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16768//console
This message is automatically generated.
> check_compatibility.sh needs smarter exit codes
> ---
>
> Key: HBASE-14930
> URL: https://issues.apache.org/jira/browse/HBASE-14930
> Project: HBase
> Issue Type: Bug
>Reporter: Dima Spivak
>Assignee: Dima Spivak
> Attachments: HBASE-14930_master_v1.patch
>
>
> The check_compatibility.sh tool in dev_support uses the Java API Compliance
> Checker to do static analysis of source/binary incompatibilties between two
> HBase branches. One problem, though, is that the script has a few instances
> where it may return an exit code of 1 (e.g. if Maven steps fail), but this is
> the same exit code that the Java ACC tool itself uses to denote that the tool
> succeeded, but found incompatibilities.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14926) Hung ThriftServer; no timeout on read from client; if client crashes, worker thread gets stuck reading
[
https://issues.apache.org/jira/browse/HBASE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14926:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 1.3.0
1.2.0
2.0.0
Release Note: Adds a timeout to server read from clients. Adds new configs
hbase.thrift.server.socket.read.timeout for setting read timeout on server
socket in milliseconds. Default is 6;
Status: Resolved (was: Patch Available)
Pushed to 1.2+ Thrift starts to vary if I go back more. Let me know if want it
on 0.98 and I'll hack it up.
> Hung ThriftServer; no timeout on read from client; if client crashes, worker
> thread gets stuck reading
> --
>
> Key: HBASE-14926
> URL: https://issues.apache.org/jira/browse/HBASE-14926
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 1.0.3, 0.98.16
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14926.patch, 14926v2.txt
>
>
> Thrift server is hung. All worker threads are doing this:
> {code}
> "thrift-worker-0" daemon prio=10 tid=0x7f0bb95c2800 nid=0xf6a7 runnable
> [0x7f0b956e]
>java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> - locked <0x00066d859490> (a java.io.BufferedInputStream)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> at
> org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:601)
> at
> org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:470)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
> at
> org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
> at
> org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> They never recover.
> I don't have client side logs.
> We've been here before: HBASE-4967 "connected client thrift sockets should
> have a server side read timeout" but this patch only got applied to fb branch
> (and thrift has changed since then).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14795) Enhance the spark-hbase scan operations
[ https://issues.apache.org/jira/browse/HBASE-14795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042336#comment-15042336 ] Ted Yu commented on HBASE-14795: In the groups field for reviewers, please fill in hbase so that people would receive notifications of reviews. > Enhance the spark-hbase scan operations > --- > > Key: HBASE-14795 > URL: https://issues.apache.org/jira/browse/HBASE-14795 > Project: HBase > Issue Type: Improvement >Reporter: Ted Malaska >Assignee: Zhan Zhang >Priority: Minor > Attachments: > 0001-HBASE-14795-Enhance-the-spark-hbase-scan-operations.patch > > > This is a sub-jira of HBASE-14789. This jira is to focus on the replacement > of TableInputFormat for a more custom scan implementation that will make the > following use case more effective. > Use case: > In the case you have multiple scan ranges on a single table with in a single > query. TableInputFormat will scan the the outer range of the scan start and > end range where this implementation can be more pointed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14926) Hung ThriftServer; no timeout on read from client; if client crashes, worker thread gets stuck reading
[
https://issues.apache.org/jira/browse/HBASE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042387#comment-15042387
]
Andrew Purtell commented on HBASE-14926:
Yes please to 0.98 if not too much trouble, thanks!
> Hung ThriftServer; no timeout on read from client; if client crashes, worker
> thread gets stuck reading
> --
>
> Key: HBASE-14926
> URL: https://issues.apache.org/jira/browse/HBASE-14926
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 1.0.3, 0.98.16
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14926.patch, 14926v2.txt
>
>
> Thrift server is hung. All worker threads are doing this:
> {code}
> "thrift-worker-0" daemon prio=10 tid=0x7f0bb95c2800 nid=0xf6a7 runnable
> [0x7f0b956e]
>java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> - locked <0x00066d859490> (a java.io.BufferedInputStream)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> at
> org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:601)
> at
> org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:470)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
> at
> org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
> at
> org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> They never recover.
> I don't have client side logs.
> We've been here before: HBASE-4967 "connected client thrift sockets should
> have a server side read timeout" but this patch only got applied to fb branch
> (and thrift has changed since then).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14916) Add checkstyle_report.py to other branches
[ https://issues.apache.org/jira/browse/HBASE-14916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-14916: - Attachment: HBASE-14916-branch-1-v2.patch uploading exact same patch as 'v2' to trigger build. > Add checkstyle_report.py to other branches > -- > > Key: HBASE-14916 > URL: https://issues.apache.org/jira/browse/HBASE-14916 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-14916-branch-1-v2.patch, HBASE-14916-branch-1.patch > > > Given test-patch.sh is always run from master, and that it now uses > checkstyle_report.py, we should pull back the script to other branches too. > Otherwise we see error like: > /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/jenkins.build/dev-support/test-patch.sh: > line 662: > /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/dev-support/checkstyle_report.py: > No such file or directory > [reference|https://builds.apache.org/job/PreCommit-HBASE-Build/16734//consoleFull] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14926) Hung ThriftServer; no timeout on read from client; if client crashes, worker thread gets stuck reading
[
https://issues.apache.org/jira/browse/HBASE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14926:
--
Attachment: 14926v2.txt
Ok. Played more trying to capture thriftserver in described state. I saw the
issue in thread dump but proved elusive (wasn't sure if it my timeout that
fixed it or not). The change is pretty basic so should be safe enough...
setting a timeout on server socket... which we want anyways. Going to commit
on back of @apurtell +1. v2 adds logging of the timeout to thrift2 version too
and adds a sentence or two more to examples on how to get going with thrift
server.
> Hung ThriftServer; no timeout on read from client; if client crashes, worker
> thread gets stuck reading
> --
>
> Key: HBASE-14926
> URL: https://issues.apache.org/jira/browse/HBASE-14926
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 1.0.3, 0.98.16
>Reporter: stack
>Assignee: stack
> Attachments: 14926.patch, 14926v2.txt
>
>
> Thrift server is hung. All worker threads are doing this:
> {code}
> "thrift-worker-0" daemon prio=10 tid=0x7f0bb95c2800 nid=0xf6a7 runnable
> [0x7f0b956e]
>java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> - locked <0x00066d859490> (a java.io.BufferedInputStream)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> at
> org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:601)
> at
> org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:470)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
> at
> org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
> at
> org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> They never recover.
> I don't have client side logs.
> We've been here before: HBASE-4967 "connected client thrift sockets should
> have a server side read timeout" but this patch only got applied to fb branch
> (and thrift has changed since then).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14925) Develop HBase shell command/tool to list table's region info through command line
[ https://issues.apache.org/jira/browse/HBASE-14925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042297#comment-15042297 ] Enis Soztutar commented on HBASE-14925: --- I like the idea for a way to iterate over rows. But an CVS like output out of the box for all regions or all regions of a table, which is parsable by other scripts would be very useful. > Develop HBase shell command/tool to list table's region info through command > line > - > > Key: HBASE-14925 > URL: https://issues.apache.org/jira/browse/HBASE-14925 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Romil Choksi >Assignee: huaxiang sun > > I am going through the hbase shell commands to see if there is anything I can > use to get all the regions info just for a particular table. I don’t see any > such command that provides me that information. > It would be better to have a command that provides region info, start key, > end key etc taking a table name as the input parameter. This is available > through HBase UI on clicking on a particular table's link > A tool/shell command to get a list of regions for a table or all tables in a > tabular structured output (that is machine readable) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13153) Bulk Loaded HFile Replication
[ https://issues.apache.org/jira/browse/HBASE-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042321#comment-15042321 ] Ted Yu commented on HBASE-13153: [~anoop.hbase] [~jerryhe]: Any more comment(s) ? > Bulk Loaded HFile Replication > - > > Key: HBASE-13153 > URL: https://issues.apache.org/jira/browse/HBASE-13153 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: sunhaitao >Assignee: Ashish Singhi > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-13153-branch-1-v18.patch, HBASE-13153-v1.patch, > HBASE-13153-v10.patch, HBASE-13153-v11.patch, HBASE-13153-v12.patch, > HBASE-13153-v13.patch, HBASE-13153-v14.patch, HBASE-13153-v15.patch, > HBASE-13153-v16.patch, HBASE-13153-v17.patch, HBASE-13153-v18.patch, > HBASE-13153-v2.patch, HBASE-13153-v3.patch, HBASE-13153-v4.patch, > HBASE-13153-v5.patch, HBASE-13153-v6.patch, HBASE-13153-v7.patch, > HBASE-13153-v8.patch, HBASE-13153-v9.patch, HBASE-13153.patch, HBase Bulk > Load Replication-v1-1.pdf, HBase Bulk Load Replication-v2.pdf, HBase Bulk > Load Replication-v3.pdf, HBase Bulk Load Replication.pdf, HDFS_HA_Solution.PNG > > > Currently we plan to use HBase Replication feature to deal with disaster > tolerance scenario.But we encounter an issue that we will use bulkload very > frequently,because bulkload bypass write path, and will not generate WAL, so > the data will not be replicated to backup cluster. It's inappropriate to > bukload twice both on active cluster and backup cluster. So i advise do some > modification to bulkload feature to enable bukload to both active cluster and > backup cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14866) VerifyReplication should use peer configuration in peer connection
[
https://issues.apache.org/jira/browse/HBASE-14866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Helmling updated HBASE-14866:
--
Attachment: hbase-14866-v5.patch
Attaching a new patch that moves the ZK related configuration methods to
ZKConfig, which also moves from hbase-client to hbase-common, in order to make
it available to HBaseConfiguration.createClusterConf().
TestZKConfig moves from hbase-server (don't know why it was there) to
hbase-common, to keep it together with ZKConfig.
I left the {{applyClusterKeyToConf()}} method in HBaseConfiguration, so that it
can be private. This should avoid any future independent use of it.
> VerifyReplication should use peer configuration in peer connection
> --
>
> Key: HBASE-14866
> URL: https://issues.apache.org/jira/browse/HBASE-14866
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Reporter: Gary Helmling
>Assignee: Gary Helmling
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14866.patch, HBASE-14866_v1.patch,
> hbase-14866-v4.patch, hbase-14866-v5.patch, hbase-14866_v2.patch,
> hbase-14866_v3.patch
>
>
> VerifyReplication uses the replication peer's configuration to construct the
> ZooKeeper quorum address for the peer connection. However, other
> configuration properties in the peer's configuration are dropped. It should
> merge all configuration properties from the {{ReplicationPeerConfig}} when
> creating the peer connection and obtaining a credentials for the peer cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14901) There is duplicated code to create/manage encryption keys
[ https://issues.apache.org/jira/browse/HBASE-14901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nate Edel updated HBASE-14901: -- Status: Open (was: Patch Available) > There is duplicated code to create/manage encryption keys > - > > Key: HBASE-14901 > URL: https://issues.apache.org/jira/browse/HBASE-14901 > Project: HBase > Issue Type: Bug > Components: encryption >Affects Versions: 2.0.0 >Reporter: Nate Edel >Assignee: Nate Edel >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-14901.1.patch, HBASE-14901.2.patch, > HBASE-14901.3.patch, HBASE-14901.5.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > There is duplicated code from MobUtils.createEncryptionContext in HStore, and > there is a subset of that code in HFileReaderImpl. > Refactored key selection > Moved both to EncryptionUtil.java > Can't figure out how to write a unit test for this, but there's no new code > just refactoring. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14928) Start row should be set for query through HBase REST gateway involving globbing option
[ https://issues.apache.org/jira/browse/HBASE-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042423#comment-15042423 ] Hudson commented on HBASE-14928: FAILURE: Integrated in HBase-1.3-IT #355 (See [https://builds.apache.org/job/HBase-1.3-IT/355/]) HBASE-14928 Start row should be set for query through HBase REST gateway (tedyu: rev 7bd3b94db1011a0f5742e22dd48af3e298c5dfe4) * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java > Start row should be set for query through HBase REST gateway involving > globbing option > -- > > Key: HBASE-14928 > URL: https://issues.apache.org/jira/browse/HBASE-14928 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17 > > Attachments: 14928-v1.txt > > > As Ben Sutton reported in the thread, Slow response on HBase REST api using > globbing option, query through the Rest API with a globbing option i.e. > http://:/table/key\* executes extremely slowly. > Jerry He pointed out that PrefixFilter is used for query involving globbing > option. > This issue is to fix this bug by setting start row for such queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14926) Hung ThriftServer; no timeout on read from client; if client crashes, worker thread gets stuck reading
[
https://issues.apache.org/jira/browse/HBASE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042422#comment-15042422
]
Hudson commented on HBASE-14926:
FAILURE: Integrated in HBase-1.3-IT #355 (See
[https://builds.apache.org/job/HBase-1.3-IT/355/])
HBASE-14926 Hung ThriftServer; no timeout on read from client; if client
(stack: rev 322d59839e60311590a4bc19adb83296925c704f)
* hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java
* hbase-examples/README.txt
* hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java
*
hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
> Hung ThriftServer; no timeout on read from client; if client crashes, worker
> thread gets stuck reading
> --
>
> Key: HBASE-14926
> URL: https://issues.apache.org/jira/browse/HBASE-14926
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 1.0.3, 0.98.16
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14926.patch, 14926v2.txt
>
>
> Thrift server is hung. All worker threads are doing this:
> {code}
> "thrift-worker-0" daemon prio=10 tid=0x7f0bb95c2800 nid=0xf6a7 runnable
> [0x7f0b956e]
>java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> - locked <0x00066d859490> (a java.io.BufferedInputStream)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> at
> org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:601)
> at
> org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:470)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
> at
> org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
> at
> org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> They never recover.
> I don't have client side logs.
> We've been here before: HBASE-4967 "connected client thrift sockets should
> have a server side read timeout" but this patch only got applied to fb branch
> (and thrift has changed since then).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14916) Add checkstyle_report.py to other branches
[
https://issues.apache.org/jira/browse/HBASE-14916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042428#comment-15042428
]
Hadoop QA commented on HBASE-14916:
---
{color:red}-1 overall{color}.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16769//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16769//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16769//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16769//console
This message is automatically generated.
> Add checkstyle_report.py to other branches
> --
>
> Key: HBASE-14916
> URL: https://issues.apache.org/jira/browse/HBASE-14916
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-14916-branch-1-v2.patch, HBASE-14916-branch-1.patch
>
>
> Given test-patch.sh is always run from master, and that it now uses
> checkstyle_report.py, we should pull back the script to other branches too.
> Otherwise we see error like:
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/jenkins.build/dev-support/test-patch.sh:
> line 662:
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/dev-support/checkstyle_report.py:
> No such file or directory
> [reference|https://builds.apache.org/job/PreCommit-HBASE-Build/16734//consoleFull]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14928) Start row should be set for query through HBase REST gateway involving globbing option
[ https://issues.apache.org/jira/browse/HBASE-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042430#comment-15042430 ] Hudson commented on HBASE-14928: FAILURE: Integrated in HBase-1.1-JDK8 #1701 (See [https://builds.apache.org/job/HBase-1.1-JDK8/1701/]) HBASE-14928 Start row should be set for query through HBase REST gateway (tedyu: rev fe7a0a2d51eabd7f210bcc9f4944de75ea304795) * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java > Start row should be set for query through HBase REST gateway involving > globbing option > -- > > Key: HBASE-14928 > URL: https://issues.apache.org/jira/browse/HBASE-14928 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17 > > Attachments: 14928-v1.txt > > > As Ben Sutton reported in the thread, Slow response on HBase REST api using > globbing option, query through the Rest API with a globbing option i.e. > http://:/table/key\* executes extremely slowly. > Jerry He pointed out that PrefixFilter is used for query involving globbing > option. > This issue is to fix this bug by setting start row for such queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14922) Delayed flush doesn't work causing flush storms.
[ https://issues.apache.org/jira/browse/HBASE-14922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-14922: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > Delayed flush doesn't work causing flush storms. > > > Key: HBASE-14922 > URL: https://issues.apache.org/jira/browse/HBASE-14922 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.0, 1.1.2 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14922-v1.patch, HBASE-14922-v2.patch, > HBASE-14922.patch > > > Starting all regionservers at the same time will mean that most > PeriodicMemstoreFlusher's will be running at the same time. So all of these > threads will queue flushes at about the same time. > This was supposed to be mitigated by Delayed. However that isn't nearly > enough. This results in the immediate filling up and then draining of the > flush queues every hour. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14928) Start row should be set for query through HBase REST gateway involving globbing option
[ https://issues.apache.org/jira/browse/HBASE-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042521#comment-15042521 ] Hudson commented on HBASE-14928: SUCCESS: Integrated in HBase-1.2-IT #326 (See [https://builds.apache.org/job/HBase-1.2-IT/326/]) HBASE-14928 Start row should be set for query through HBase REST (tedyu: rev f57bb78db83ac9680191a9d345ef5eaf941392e2) * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java > Start row should be set for query through HBase REST gateway involving > globbing option > -- > > Key: HBASE-14928 > URL: https://issues.apache.org/jira/browse/HBASE-14928 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17 > > Attachments: 14928-v1.txt > > > As Ben Sutton reported in the thread, Slow response on HBase REST api using > globbing option, query through the Rest API with a globbing option i.e. > http://:/table/key\* executes extremely slowly. > Jerry He pointed out that PrefixFilter is used for query involving globbing > option. > This issue is to fix this bug by setting start row for such queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14926) Hung ThriftServer; no timeout on read from client; if client crashes, worker thread gets stuck reading
[
https://issues.apache.org/jira/browse/HBASE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042520#comment-15042520
]
Hudson commented on HBASE-14926:
SUCCESS: Integrated in HBase-1.2-IT #326 (See
[https://builds.apache.org/job/HBase-1.2-IT/326/])
HBASE-14926 Hung ThriftServer; no timeout on read from client; if (stack: rev
8954091dfba7950199bcab772e318028e6b66f0f)
* hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java
* hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java
* hbase-examples/README.txt
*
hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
> Hung ThriftServer; no timeout on read from client; if client crashes, worker
> thread gets stuck reading
> --
>
> Key: HBASE-14926
> URL: https://issues.apache.org/jira/browse/HBASE-14926
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 1.0.3, 0.98.16
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17
>
> Attachments: 14926.patch, 14926v2.txt
>
>
> Thrift server is hung. All worker threads are doing this:
> {code}
> "thrift-worker-0" daemon prio=10 tid=0x7f0bb95c2800 nid=0xf6a7 runnable
> [0x7f0b956e]
>java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> - locked <0x00066d859490> (a java.io.BufferedInputStream)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> at
> org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:601)
> at
> org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:470)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
> at
> org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
> at
> org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> They never recover.
> I don't have client side logs.
> We've been here before: HBASE-4967 "connected client thrift sockets should
> have a server side read timeout" but this patch only got applied to fb branch
> (and thrift has changed since then).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14930) check_compatibility.sh needs smarter exit codes
[ https://issues.apache.org/jira/browse/HBASE-14930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042459#comment-15042459 ] Andrew Purtell commented on HBASE-14930: lgtm, if no concerns will commit soon everywhere > check_compatibility.sh needs smarter exit codes > --- > > Key: HBASE-14930 > URL: https://issues.apache.org/jira/browse/HBASE-14930 > Project: HBase > Issue Type: Bug >Reporter: Dima Spivak >Assignee: Dima Spivak > Attachments: HBASE-14930_master_v1.patch > > > The check_compatibility.sh tool in dev_support uses the Java API Compliance > Checker to do static analysis of source/binary incompatibilties between two > HBase branches. One problem, though, is that the script has a few instances > where it may return an exit code of 1 (e.g. if Maven steps fail), but this is > the same exit code that the Java ACC tool itself uses to denote that the tool > succeeded, but found incompatibilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14928) Start row should be set for query through HBase REST gateway involving globbing option
[ https://issues.apache.org/jira/browse/HBASE-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042464#comment-15042464 ] Hudson commented on HBASE-14928: FAILURE: Integrated in HBase-1.1-JDK7 #1613 (See [https://builds.apache.org/job/HBase-1.1-JDK7/1613/]) HBASE-14928 Start row should be set for query through HBase REST (tedyu: rev fe7a0a2d51eabd7f210bcc9f4944de75ea304795) * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java > Start row should be set for query through HBase REST gateway involving > globbing option > -- > > Key: HBASE-14928 > URL: https://issues.apache.org/jira/browse/HBASE-14928 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17 > > Attachments: 14928-v1.txt > > > As Ben Sutton reported in the thread, Slow response on HBase REST api using > globbing option, query through the Rest API with a globbing option i.e. > http://:/table/key\* executes extremely slowly. > Jerry He pointed out that PrefixFilter is used for query involving globbing > option. > This issue is to fix this bug by setting start row for such queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14926) Hung ThriftServer; no timeout on read from client; if client crashes, worker thread gets stuck reading
[
https://issues.apache.org/jira/browse/HBASE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14926:
--
Fix Version/s: 0.98.17
Pushed to 0.98.
0.98 has different thrift version so different classes. Gave it a run to make
sure it basically worked when done and it seems too
> Hung ThriftServer; no timeout on read from client; if client crashes, worker
> thread gets stuck reading
> --
>
> Key: HBASE-14926
> URL: https://issues.apache.org/jira/browse/HBASE-14926
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 1.0.3, 0.98.16
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17
>
> Attachments: 14926.patch, 14926v2.txt
>
>
> Thrift server is hung. All worker threads are doing this:
> {code}
> "thrift-worker-0" daemon prio=10 tid=0x7f0bb95c2800 nid=0xf6a7 runnable
> [0x7f0b956e]
>java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> - locked <0x00066d859490> (a java.io.BufferedInputStream)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> at
> org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:601)
> at
> org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:470)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
> at
> org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
> at
> org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> They never recover.
> I don't have client side logs.
> We've been here before: HBASE-4967 "connected client thrift sockets should
> have a server side read timeout" but this patch only got applied to fb branch
> (and thrift has changed since then).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14917) Log in console if individual tests in test-patch.sh fail or pass.
[
https://issues.apache.org/jira/browse/HBASE-14917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042475#comment-15042475
]
Hadoop QA commented on HBASE-14917:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12775852/HBASE-14917-v2.patch
against master branch at commit d18c1af3bb12d6bf1800f22a5289bcb71020bb65.
ATTACHMENT ID: 12775852
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16770//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16770//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16770//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16770//console
This message is automatically generated.
> Log in console if individual tests in test-patch.sh fail or pass.
> -
>
> Key: HBASE-14917
> URL: https://issues.apache.org/jira/browse/HBASE-14917
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
>Priority: Minor
> Attachments: HBASE-14917-v2.patch, HBASE-14917.patch
>
>
> Got 2 runs like
> https://issues.apache.org/jira/browse/HBASE-14865?focusedCommentId=15037056=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15037056
> where can't figure out what went wrong in patch testing.
> Logging results from individual tests to console as discussed
> [here|https://mail-archives.apache.org/mod_mbox/hbase-dev/201512.mbox/%3CCAAjhxrrL4-qty562%3DcMyBJ2xyhGqHi3MFAgf9ygrzQf1%2BZmHtw%40mail.gmail.com%3E]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-4907) Port 89-fb changes to trunk
[ https://issues.apache.org/jira/browse/HBASE-4907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042481#comment-15042481 ] Hudson commented on HBASE-4907: --- FAILURE: Integrated in HBase-Trunk_matrix #531 (See [https://builds.apache.org/job/HBase-Trunk_matrix/531/]) HBASE-4907 NPE of MobUtils.hasMobColumns in Build failed in Jenkins: (stack: rev 03e4712f0ca08d57586b3fc4d93cf02c999515d8) * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCreateTableProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java > Port 89-fb changes to trunk > --- > > Key: HBASE-4907 > URL: https://issues.apache.org/jira/browse/HBASE-4907 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Reporter: Nicolas Spiegelberg >Assignee: Nicolas Spiegelberg >Priority: Blocker > > A super task to track the progress of porting 89-fb functionality & fixes to > trunk. Note that these tasks are focused on RegionServer functionality. > 89-specific master functionality doesn't have a time frame for porting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14928) Start row should be set for query through HBase REST gateway involving globbing option
[ https://issues.apache.org/jira/browse/HBASE-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042536#comment-15042536 ] Hudson commented on HBASE-14928: FAILURE: Integrated in HBase-1.3 #416 (See [https://builds.apache.org/job/HBase-1.3/416/]) HBASE-14928 Start row should be set for query through HBase REST gateway (tedyu: rev 7bd3b94db1011a0f5742e22dd48af3e298c5dfe4) * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java > Start row should be set for query through HBase REST gateway involving > globbing option > -- > > Key: HBASE-14928 > URL: https://issues.apache.org/jira/browse/HBASE-14928 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17 > > Attachments: 14928-v1.txt > > > As Ben Sutton reported in the thread, Slow response on HBase REST api using > globbing option, query through the Rest API with a globbing option i.e. > http://:/table/key\* executes extremely slowly. > Jerry He pointed out that PrefixFilter is used for query involving globbing > option. > This issue is to fix this bug by setting start row for such queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13153) Bulk Loaded HFile Replication
[ https://issues.apache.org/jira/browse/HBASE-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042453#comment-15042453 ] Jerry He commented on HBASE-13153: -- I have a use case that this feature would be quite useful. We have a SQL on Hadoop/HBase. When inserting into HBase, we try to be smart and optimize using bulk load some times. For example, when doing 'INSERT INTO my-hbase-table SELECT col1 from table1', we will try to see if the cardinalities are big (say > 2). If yes, we will generate hfile to bulk load, not running table puts. The problem is that replication will not kick in for this new data. For across cluster bulk load, people would probably use an external tool (e.g distCp) to move the MR generated hfiles to the target cluster. But in this case, it would be difficult to save and transport the hfiles for bulk load to the peer cluster since they are generated on-the-fly inside the SQL engine. So this is a good feature to have. Regarding the network latency and impact on HBase instances, I think we should add notes/best practice/warning in the release notes. Mention that potentially large files need to copied over the network by HBase handlers, and potential impact on the source and peer clusters. And recommendations like the rpc timeout values need to be increased. > Bulk Loaded HFile Replication > - > > Key: HBASE-13153 > URL: https://issues.apache.org/jira/browse/HBASE-13153 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: sunhaitao >Assignee: Ashish Singhi > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-13153-branch-1-v18.patch, HBASE-13153-v1.patch, > HBASE-13153-v10.patch, HBASE-13153-v11.patch, HBASE-13153-v12.patch, > HBASE-13153-v13.patch, HBASE-13153-v14.patch, HBASE-13153-v15.patch, > HBASE-13153-v16.patch, HBASE-13153-v17.patch, HBASE-13153-v18.patch, > HBASE-13153-v2.patch, HBASE-13153-v3.patch, HBASE-13153-v4.patch, > HBASE-13153-v5.patch, HBASE-13153-v6.patch, HBASE-13153-v7.patch, > HBASE-13153-v8.patch, HBASE-13153-v9.patch, HBASE-13153.patch, HBase Bulk > Load Replication-v1-1.pdf, HBase Bulk Load Replication-v2.pdf, HBase Bulk > Load Replication-v3.pdf, HBase Bulk Load Replication.pdf, HDFS_HA_Solution.PNG > > > Currently we plan to use HBase Replication feature to deal with disaster > tolerance scenario.But we encounter an issue that we will use bulkload very > frequently,because bulkload bypass write path, and will not generate WAL, so > the data will not be replicated to backup cluster. It's inappropriate to > bukload twice both on active cluster and backup cluster. So i advise do some > modification to bulkload feature to enable bukload to both active cluster and > backup cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[
https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042474#comment-15042474
]
Ted Yu commented on HBASE-6721:
---
Test failure in TestSimpleRegionNormalizer is reproducible:
{code}
testNoNormalizationForMetaTable(org.apache.hadoop.hbase.master.normalizer.TestSimpleRegionNormalizer)
Time elapsed: 0.337 sec <<< ERROR!
java.lang.IllegalArgumentException: null
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:76)
at com.google.common.net.HostAndPort.fromParts(HostAndPort.java:130)
at org.apache.hadoop.hbase.ServerName.(ServerName.java:105)
at org.apache.hadoop.hbase.ServerName.valueOf(ServerName.java:158)
at
org.apache.hadoop.hbase.master.normalizer.TestSimpleRegionNormalizer.setupMocksForNormalizer(TestSimpleRegionNormalizer.java:223)
at
org.apache.hadoop.hbase.master.normalizer.TestSimpleRegionNormalizer.testNoNormalizationForMetaTable(TestSimpleRegionNormalizer.java:71)
testNoNormalizationIfTooFewRegions(org.apache.hadoop.hbase.master.normalizer.TestSimpleRegionNormalizer)
Time elapsed: 0.045 sec <<< ERROR!
java.lang.IllegalArgumentException: null
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:76)
at com.google.common.net.HostAndPort.fromParts(HostAndPort.java:130)
at org.apache.hadoop.hbase.ServerName.(ServerName.java:105)
at org.apache.hadoop.hbase.ServerName.valueOf(ServerName.java:158)
at
org.apache.hadoop.hbase.master.normalizer.TestSimpleRegionNormalizer.setupMocksForNormalizer(TestSimpleRegionNormalizer.java:223)
at
org.apache.hadoop.hbase.master.normalizer.TestSimpleRegionNormalizer.testNoNormalizationIfTooFewRegions(TestSimpleRegionNormalizer.java:90)
{code}
> RegionServer Group based Assignment
> ---
>
> Key: HBASE-6721
> URL: https://issues.apache.org/jira/browse/HBASE-6721
> Project: HBase
> Issue Type: New Feature
>Reporter: Francis Liu
>Assignee: Francis Liu
> Labels: hbase-6721
> Attachments: 6721-master-webUI.patch, HBASE-6721
> GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf,
> HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf,
> HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch,
> HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch,
> HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch,
> HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch,
> HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch,
> HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch,
> HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch,
> HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch,
> HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch,
> HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg,
> hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch,
> hbase-6721-v18.patch, immediateAssignments Sequence Diagram.svg,
> randomAssignment Sequence Diagram.svg, retainAssignment Sequence Diagram.svg,
> roundRobinAssignment Sequence Diagram.svg
>
>
> In multi-tenant deployments of HBase, it is likely that a RegionServer will
> be serving out regions from a number of different tables owned by various
> client applications. Being able to group a subset of running RegionServers
> and assign specific tables to it, provides a client application a level of
> isolation and resource allocation.
> The proposal essentially is to have an AssignmentManager which is aware of
> RegionServer groups and assigns tables to region servers based on groupings.
> Load balancing will occur on a per group basis as well.
> This is essentially a simplification of the approach taken in HBASE-4120. See
> attached document.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14901) There is duplicated code to create/manage encryption keys
[
https://issues.apache.org/jira/browse/HBASE-14901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042506#comment-15042506
]
Hadoop QA commented on HBASE-14901:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12775846/HBASE-14901.6.patch
against master branch at commit d18c1af3bb12d6bf1800f22a5289bcb71020bb65.
ATTACHMENT ID: 12775846
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16771//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16771//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16771//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16771//console
This message is automatically generated.
> There is duplicated code to create/manage encryption keys
> -
>
> Key: HBASE-14901
> URL: https://issues.apache.org/jira/browse/HBASE-14901
> Project: HBase
> Issue Type: Bug
> Components: encryption
>Affects Versions: 2.0.0
>Reporter: Nate Edel
>Assignee: Nate Edel
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-14901.1.patch, HBASE-14901.2.patch,
> HBASE-14901.3.patch, HBASE-14901.5.patch, HBASE-14901.6.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> There is duplicated code from MobUtils.createEncryptionContext in HStore, and
> there is a subset of that code in HFileReaderImpl.
> Refactored key selection
> Moved both to EncryptionUtil.java
> Can't figure out how to write a unit test for this, but there's no new code
> just refactoring.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14926) Hung ThriftServer; no timeout on read from client; if client crashes, worker thread gets stuck reading
[
https://issues.apache.org/jira/browse/HBASE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042535#comment-15042535
]
Hudson commented on HBASE-14926:
FAILURE: Integrated in HBase-1.3 #416 (See
[https://builds.apache.org/job/HBase-1.3/416/])
HBASE-14926 Hung ThriftServer; no timeout on read from client; if client
(stack: rev 322d59839e60311590a4bc19adb83296925c704f)
* hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java
* hbase-examples/README.txt
*
hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
* hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java
> Hung ThriftServer; no timeout on read from client; if client crashes, worker
> thread gets stuck reading
> --
>
> Key: HBASE-14926
> URL: https://issues.apache.org/jira/browse/HBASE-14926
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 1.0.3, 0.98.16
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17
>
> Attachments: 14926.patch, 14926v2.txt
>
>
> Thrift server is hung. All worker threads are doing this:
> {code}
> "thrift-worker-0" daemon prio=10 tid=0x7f0bb95c2800 nid=0xf6a7 runnable
> [0x7f0b956e]
>java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> - locked <0x00066d859490> (a java.io.BufferedInputStream)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> at
> org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:601)
> at
> org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:470)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
> at
> org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
> at
> org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> They never recover.
> I don't have client side logs.
> We've been here before: HBASE-4967 "connected client thrift sockets should
> have a server side read timeout" but this patch only got applied to fb branch
> (and thrift has changed since then).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14926) Hung ThriftServer; no timeout on read from client; if client crashes, worker thread gets stuck reading
[
https://issues.apache.org/jira/browse/HBASE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042547#comment-15042547
]
Hudson commented on HBASE-14926:
SUCCESS: Integrated in HBase-1.2 #423 (See
[https://builds.apache.org/job/HBase-1.2/423/])
HBASE-14926 Hung ThriftServer; no timeout on read from client; if client
(stack: rev 8954091dfba7950199bcab772e318028e6b66f0f)
* hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java
*
hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
* hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java
* hbase-examples/README.txt
> Hung ThriftServer; no timeout on read from client; if client crashes, worker
> thread gets stuck reading
> --
>
> Key: HBASE-14926
> URL: https://issues.apache.org/jira/browse/HBASE-14926
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 1.0.3, 0.98.16
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17
>
> Attachments: 14926.patch, 14926v2.txt
>
>
> Thrift server is hung. All worker threads are doing this:
> {code}
> "thrift-worker-0" daemon prio=10 tid=0x7f0bb95c2800 nid=0xf6a7 runnable
> [0x7f0b956e]
>java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> - locked <0x00066d859490> (a java.io.BufferedInputStream)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> at
> org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:601)
> at
> org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:470)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
> at
> org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
> at
> org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> They never recover.
> I don't have client side logs.
> We've been here before: HBASE-4967 "connected client thrift sockets should
> have a server side read timeout" but this patch only got applied to fb branch
> (and thrift has changed since then).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14922) Delayed flush doesn't work causing flush storms.
[ https://issues.apache.org/jira/browse/HBASE-14922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042566#comment-15042566 ] Hudson commented on HBASE-14922: FAILURE: Integrated in HBase-1.3 #417 (See [https://builds.apache.org/job/HBase-1.3/417/]) HBASE-14922 Delayed flush doesn't work causing flush storms. (eclark: rev 85167849ab934c577c8103dfc23ff555801b0c62) * hbase-common/src/test/java/org/apache/hadoop/hbase/TestChoreService.java * hbase-common/src/main/java/org/apache/hadoop/hbase/ChoreService.java * hbase-common/src/main/java/org/apache/hadoop/hbase/JitterScheduledThreadPoolExecutorImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Delayed flush doesn't work causing flush storms. > > > Key: HBASE-14922 > URL: https://issues.apache.org/jira/browse/HBASE-14922 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.2.0, 1.1.2 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14922-v1.patch, HBASE-14922-v2.patch, > HBASE-14922.patch > > > Starting all regionservers at the same time will mean that most > PeriodicMemstoreFlusher's will be running at the same time. So all of these > threads will queue flushes at about the same time. > This was supposed to be mitigated by Delayed. However that isn't nearly > enough. This results in the immediate filling up and then draining of the > flush queues every hour. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14769) Remove unused functions and duplicate javadocs from HBaseAdmin
[
https://issues.apache.org/jira/browse/HBASE-14769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042571#comment-15042571
]
Hadoop QA commented on HBASE-14769:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12775870/HBASE-14769-master-v9.patch
against master branch at commit 26dd0d17f81627d3688f28bba1a293513ff5d702.
ATTACHMENT ID: 12775870
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 33 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16773//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16773//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16773//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16773//console
This message is automatically generated.
> Remove unused functions and duplicate javadocs from HBaseAdmin
> ---
>
> Key: HBASE-14769
> URL: https://issues.apache.org/jira/browse/HBASE-14769
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Fix For: 2.0.0
>
> Attachments: HBASE-14769-master-v2.patch,
> HBASE-14769-master-v3.patch, HBASE-14769-master-v4.patch,
> HBASE-14769-master-v5.patch, HBASE-14769-master-v6.patch,
> HBASE-14769-master-v7.patch, HBASE-14769-master-v8.patch,
> HBASE-14769-master-v9.patch, HBASE-14769-master.patch
>
>
> HBaseAdmin is marked private, so removing the functions not being used
> anywhere.
> Also, the javadocs of overridden functions are same as corresponding ones in
> Admin.java. Since javadocs are automatically inherited from the interface
> class, we can remove these redundant 100s of lines.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-7171) Initial web UI for region/memstore/storefiles details
[ https://issues.apache.org/jira/browse/HBASE-7171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-7171: --- Summary: Initial web UI for region/memstore/storefiles details (was: Click on a region in UI and get a listing of hfiles in HDFS and summary of memstore content; click on an HFile and see its content) > Initial web UI for region/memstore/storefiles details > - > > Key: HBASE-7171 > URL: https://issues.apache.org/jira/browse/HBASE-7171 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: stack >Assignee: Mikhail Antonov > Labels: beginner > Attachments: region_details.png, region_list.png, > storefile_details.png > > > Click on a region in UI and get a listing of hfiles in HDFS and summary of > memstore content; click on an HFile and see its content -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-7171) Click on a region in UI and get a listing of hfiles in HDFS and summary of memstore content; click on an HFile and see its content
[ https://issues.apache.org/jira/browse/HBASE-7171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-7171: --- Description: Click on a region in UI and get a listing of hfiles in HDFS and summary of memstore content; click on an HFile and see its content > Click on a region in UI and get a listing of hfiles in HDFS and summary of > memstore content; click on an HFile and see its content > -- > > Key: HBASE-7171 > URL: https://issues.apache.org/jira/browse/HBASE-7171 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: stack >Assignee: Mikhail Antonov > Labels: beginner > Attachments: region_details.png, region_list.png, > storefile_details.png > > > Click on a region in UI and get a listing of hfiles in HDFS and summary of > memstore content; click on an HFile and see its content -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-7171) Initial web UI for region/memstore/storefiles details
[
https://issues.apache.org/jira/browse/HBASE-7171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-7171:
---
Attachment: HBASE-7171.patch
Here's draft patch. Added two new pages, region details and storefile details.
Some notes (cc [~stack], [~enis]):
- regarding using "logical" api to traverse store files (think
region.getStores().get('cfname').getStoreFiles() ) vs "physical" (traversing fs
assuming certain physical layout and linking to NN-style printout) I think
first one would be easier to maintain and cleaner going forward? What do we
need to display on region page. Memstore params (besides current size, what
else? timestamps, snapshot info? File information - size, lmd, what else is of
interest given the limited horizontal space.
- for now I just used quick and dirty approach when clicking at store file
name link simply invokes another java process (via 'hbase hfile...') and
redirects the output to the page. Is this information what we want to see at
this store file details page (assuming that browsing individual cells is not
something I'm doing now, that needs more thinking)? If this information is
kind of what we want here, then to avoid having to call another process I can
change HFilePrettyPrinter, pull KeyValueStatsCollector out of it, and a bit
more refactoring would be needed there.
> Initial web UI for region/memstore/storefiles details
> -
>
> Key: HBASE-7171
> URL: https://issues.apache.org/jira/browse/HBASE-7171
> Project: HBase
> Issue Type: Improvement
> Components: UI
>Reporter: stack
>Assignee: Mikhail Antonov
> Labels: beginner
> Attachments: HBASE-7171.patch, region_details.png, region_list.png,
> storefile_details.png
>
>
> Click on a region in UI and get a listing of hfiles in HDFS and summary of
> memstore content; click on an HFile and see its content
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14916) Add checkstyle_report.py to other branches
[ https://issues.apache.org/jira/browse/HBASE-14916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-14916: - Attachment: HBASE-14916-branch-1-v3.patch v3: Changing new file mode of checkstyle_report.py from 644 to 755 > Add checkstyle_report.py to other branches > -- > > Key: HBASE-14916 > URL: https://issues.apache.org/jira/browse/HBASE-14916 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-14916-branch-1-v2.patch, > HBASE-14916-branch-1-v3.patch, HBASE-14916-branch-1.patch > > > Given test-patch.sh is always run from master, and that it now uses > checkstyle_report.py, we should pull back the script to other branches too. > Otherwise we see error like: > /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/jenkins.build/dev-support/test-patch.sh: > line 662: > /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/dev-support/checkstyle_report.py: > No such file or directory > [reference|https://builds.apache.org/job/PreCommit-HBASE-Build/16734//consoleFull] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14923) VerifyReplication should not mask the exception during result comparison
[
https://issues.apache.org/jira/browse/HBASE-14923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-14923:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: (was: 0.98.16)
1.0.4
0.98.17
1.1.3
1.3.0
1.2.0
Status: Resolved (was: Patch Available)
Thanks for the patch [~vishk]
> VerifyReplication should not mask the exception during result comparison
> -
>
> Key: HBASE-14923
> URL: https://issues.apache.org/jira/browse/HBASE-14923
> Project: HBase
> Issue Type: Bug
> Components: tooling
>Affects Versions: 2.0.0, 0.98.16
>Reporter: Vishal Khandelwal
>Assignee: Vishal Khandelwal
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4
>
> Attachments: HBASE-14923_v1.patch
>
>
> hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
> Line:154
> } catch (Exception e) {
> logFailRowAndIncreaseCounter(context,
> Counters.CONTENT_DIFFERENT_ROWS, value);
> }
> Just LOG.error needs to be added for more information for the failure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14930) check_compatibility.sh needs smarter exit codes
[ https://issues.apache.org/jira/browse/HBASE-14930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-14930: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.0.4 0.98.17 1.1.3 1.3.0 1.2.0 2.0.0 Status: Resolved (was: Patch Available) > check_compatibility.sh needs smarter exit codes > --- > > Key: HBASE-14930 > URL: https://issues.apache.org/jira/browse/HBASE-14930 > Project: HBase > Issue Type: Bug >Reporter: Dima Spivak >Assignee: Dima Spivak > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4 > > Attachments: HBASE-14930_master_v1.patch > > > The check_compatibility.sh tool in dev_support uses the Java API Compliance > Checker to do static analysis of source/binary incompatibilties between two > HBase branches. One problem, though, is that the script has a few instances > where it may return an exit code of 1 (e.g. if Maven steps fail), but this is > the same exit code that the Java ACC tool itself uses to denote that the tool > succeeded, but found incompatibilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14926) Hung ThriftServer; no timeout on read from client; if client crashes, worker thread gets stuck reading
[
https://issues.apache.org/jira/browse/HBASE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042644#comment-15042644
]
Hudson commented on HBASE-14926:
FAILURE: Integrated in HBase-Trunk_matrix #532 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/532/])
HBASE-14926 Hung ThriftServer; no timeout on read from client; if client
(stack: rev 26dd0d17f81627d3688f28bba1a293513ff5d702)
* hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java
* hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java
* hbase-examples/README.txt
*
hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
> Hung ThriftServer; no timeout on read from client; if client crashes, worker
> thread gets stuck reading
> --
>
> Key: HBASE-14926
> URL: https://issues.apache.org/jira/browse/HBASE-14926
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 1.0.3, 0.98.16
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17
>
> Attachments: 14926.patch, 14926v2.txt
>
>
> Thrift server is hung. All worker threads are doing this:
> {code}
> "thrift-worker-0" daemon prio=10 tid=0x7f0bb95c2800 nid=0xf6a7 runnable
> [0x7f0b956e]
>java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> - locked <0x00066d859490> (a java.io.BufferedInputStream)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
> at
> org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:601)
> at
> org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:470)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
> at
> org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
> at
> org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> They never recover.
> I don't have client side logs.
> We've been here before: HBASE-4967 "connected client thrift sockets should
> have a server side read timeout" but this patch only got applied to fb branch
> (and thrift has changed since then).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
