[jira] [Commented] (HBASE-13071) Hbase Streaming Scan Feature
[ https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381587#comment-14381587 ] Edward Bortnikov commented on HBASE-13071: -- +1 on this feature Hbase Streaming Scan Feature Key: HBASE-13071 URL: https://issues.apache.org/jira/browse/HBASE-13071 Project: HBase Issue Type: New Feature Reporter: Eshcar Hillel Attachments: 99.eshcar.png, HBASE-13071_98_1.patch, HBASE-13071_trunk_1.patch, HBASE-13071_trunk_10.patch, HBASE-13071_trunk_2.patch, HBASE-13071_trunk_3.patch, HBASE-13071_trunk_4.patch, HBASE-13071_trunk_5.patch, HBASE-13071_trunk_6.patch, HBASE-13071_trunk_7.patch, HBASE-13071_trunk_8.patch, HBASE-13071_trunk_9.patch, HBaseStreamingScanDesign.pdf, HbaseStreamingScanEvaluation.pdf, HbaseStreamingScanEvaluationwithMultipleClients.pdf, gc.eshcar.png, hits.eshcar.png, network.png A scan operation iterates over all rows of a table or a subrange of the table. The synchronous nature in which the data is served at the client side hinders the speed the application traverses the data: it increases the overall processing time, and may cause a great variance in the times the application waits for the next piece of data. The scanner next() method at the client side invokes an RPC to the regionserver and then stores the results in a cache. The application can specify how many rows will be transmitted per RPC; by default this is set to 100 rows. The cache can be considered as a producer-consumer queue, where the hbase client pushes the data to the queue and the application consumes it. Currently this queue is synchronous, i.e., blocking. More specifically, when the application consumed all the data from the cache --- so the cache is empty --- the hbase client retrieves additional data from the server and re-fills the cache with new data. During this time the application is blocked. Under the assumption that the application processing time can be balanced by the time it takes to retrieve the data, an asynchronous approach can reduce the time the application is waiting for data. We attach a design document. We also have a patch that is based on a private branch, and some evaluation results of this code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13252) Get rid of managed connections and connection caching
[ https://issues.apache.org/jira/browse/HBASE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13252: Attachment: HBASE-13252-v1.patch updated patch, will see how it goes. Previously failing (as reported) tests pass for me now, if started individually from ide. Get rid of managed connections and connection caching - Key: HBASE-13252 URL: https://issues.apache.org/jira/browse/HBASE-13252 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Attachments: HBASE-13252-v0.patch, HBASE-13252-v1.patch (Need to): - Remove CONNECTION_INSTANCES from ConnectionManager - Remove HConnectionKey - Remove 'managed' property from ClusterConnection, HCI and places where it's used now - AFAIS this property isn't visible to client (ClusterConnection is private interface), so technically this would not even be backward-incompatible change, and no release note needed? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13252) Get rid of managed connections and connection caching
[ https://issues.apache.org/jira/browse/HBASE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13252: Status: Patch Available (was: Open) Get rid of managed connections and connection caching - Key: HBASE-13252 URL: https://issues.apache.org/jira/browse/HBASE-13252 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Attachments: HBASE-13252-v0.patch, HBASE-13252-v1.patch (Need to): - Remove CONNECTION_INSTANCES from ConnectionManager - Remove HConnectionKey - Remove 'managed' property from ClusterConnection, HCI and places where it's used now - AFAIS this property isn't visible to client (ClusterConnection is private interface), so technically this would not even be backward-incompatible change, and no release note needed? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13252) Get rid of managed connections and connection caching
[ https://issues.apache.org/jira/browse/HBASE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13252: Status: Open (was: Patch Available) Get rid of managed connections and connection caching - Key: HBASE-13252 URL: https://issues.apache.org/jira/browse/HBASE-13252 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Attachments: HBASE-13252-v0.patch, HBASE-13252-v1.patch (Need to): - Remove CONNECTION_INSTANCES from ConnectionManager - Remove HConnectionKey - Remove 'managed' property from ClusterConnection, HCI and places where it's used now - AFAIS this property isn't visible to client (ClusterConnection is private interface), so technically this would not even be backward-incompatible change, and no release note needed? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13337) Table regions are not assigning back, after restarting all regionservers at once.
Y. SREENIVASULU REDDY created HBASE-13337:
-
Summary: Table regions are not assigning back, after restarting
all regionservers at once.
Key: HBASE-13337
URL: https://issues.apache.org/jira/browse/HBASE-13337
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 2.0.0
Reporter: Y. SREENIVASULU REDDY
Priority: Blocker
Fix For: 2.0.0
Regions of the table are continouly in state=FAILED_CLOSE.
{noformat}
Region State
RIT time (ms)
8f62e819b356736053e06240f7f7c6fd
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. state=FAILED_CLOSE,
ts=Thu Mar 26 15:05:36 IST 2015 (113s ago), server=VM1,16040,1427362531818
113929
caf59209ae65ea80fca6bdc6996a7d68
t1,,1427362431330.caf59209ae65ea80fca6bdc6996a7d68. state=FAILED_CLOSE,
ts=Thu Mar 26 15:05:36 IST 2015 (113s ago), server=VM2,16040,1427362533691
113929
db52a74988f71e5cf257bbabf31f26f3
t1,,1427362431330.db52a74988f71e5cf257bbabf31f26f3. state=FAILED_CLOSE,
ts=Thu Mar 26 15:05:36 IST 2015 (113s ago), server=VM3,16040,1427362533691
113920
43f3a65b9f9ff283f598c5450feab1f8
t1,,1427362431330.43f3a65b9f9ff283f598c5450feab1f8. state=FAILED_CLOSE,
ts=Thu Mar 26 15:05:36 IST 2015 (113s ago), server=VM1,16040,1427362531818
113920
{noformat}
*Steps to reproduce:*
1. Start HBase cluster with more than one regionserver.
2. Create a table with precreated regions. (lets say 15 regions)
3. Make sure the regions are well balanced.
4. Restart all the Regionservers process at once across the cluster, except
HMaster process
5. After restarting the Regionservers, successfully will connect to the HMaster.
*Bug:*
But no regions are assigning back to the Regionservers.
*Master log shows as follows:*
{noformat}
2015-03-26 15:05:36,201 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=OFFLINE, ts=1427362536106, server=VM2,16040,1427362242602} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,202 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_OPENsn=VM1,16040,1427362531818
2015-03-26 15:05:36,244 DEBUG [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Force region state offline
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=PENDING_OPEN, ts=1427362536201, server=VM1,16040,1427362531818} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_CLOSE, ts=1427362536244,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_CLOSE
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=1 of 10
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=2 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=3 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=4 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=5 of 10
2015-03-26 15:05:36,250 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
[jira] [Commented] (HBASE-13337) Table regions are not assigning back, after restarting all regionservers at once.
[
https://issues.apache.org/jira/browse/HBASE-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381665#comment-14381665
]
ramkrishna.s.vasudevan commented on HBASE-13337:
I think the same thing happens when there is one RS. When we stop and start
the RS we are not able to get any regions assigned.
Table regions are not assigning back, after restarting all regionservers at
once.
-
Key: HBASE-13337
URL: https://issues.apache.org/jira/browse/HBASE-13337
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 2.0.0
Reporter: Y. SREENIVASULU REDDY
Priority: Blocker
Fix For: 2.0.0
Regions of the table are continouly in state=FAILED_CLOSE.
{noformat}
RegionState
RIT time (ms)
8f62e819b356736053e06240f7f7c6fd
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113929
caf59209ae65ea80fca6bdc6996a7d68
t1,,1427362431330.caf59209ae65ea80fca6bdc6996a7d68.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM2,16040,1427362533691 113929
db52a74988f71e5cf257bbabf31f26f3
t1,,1427362431330.db52a74988f71e5cf257bbabf31f26f3.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM3,16040,1427362533691 113920
43f3a65b9f9ff283f598c5450feab1f8
t1,,1427362431330.43f3a65b9f9ff283f598c5450feab1f8.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113920
{noformat}
*Steps to reproduce:*
1. Start HBase cluster with more than one regionserver.
2. Create a table with precreated regions. (lets say 15 regions)
3. Make sure the regions are well balanced.
4. Restart all the Regionservers process at once across the cluster, except
HMaster process
5. After restarting the Regionservers, successfully will connect to the
HMaster.
*Bug:*
But no regions are assigning back to the Regionservers.
*Master log shows as follows:*
{noformat}
2015-03-26 15:05:36,201 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=OFFLINE, ts=1427362536106, server=VM2,16040,1427362242602} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,202 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_OPENsn=VM1,16040,1427362531818
2015-03-26 15:05:36,244 DEBUG [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Force region state offline
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=PENDING_OPEN, ts=1427362536201, server=VM1,16040,1427362531818} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_CLOSE, ts=1427362536244,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_CLOSE
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=1 of 10
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=2 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=3 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=4 of 10
2015-03-26 15:05:36,249 INFO
[jira] [Commented] (HBASE-13326) Disabled table can't be enabled after HBase is restarted
[ https://issues.apache.org/jira/browse/HBASE-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381706#comment-14381706 ] Hudson commented on HBASE-13326: FAILURE: Integrated in HBase-TRUNK #6306 (See [https://builds.apache.org/job/HBase-TRUNK/6306/]) HBASE-13326 Disabled table can't be enabled after HBase is restarted (addendum) (octo47: rev bfeb067e3a50513cfc20bb8ee57c49f1d5c1f19f) * dev-support/test-patch.sh Revert HBASE-13326 Disabled table can't be enabled after HBase is restarted (addendum) (octo47: rev a85c1d9d30db473c048f778b211a2730261020df) * dev-support/test-patch.sh Disabled table can't be enabled after HBase is restarted Key: HBASE-13326 URL: https://issues.apache.org/jira/browse/HBASE-13326 Project: HBase Issue Type: Bug Affects Versions: 1.0.0, 1.1.0, 0.98.12 Reporter: Dima Spivak Assignee: Andrey Stepachev Priority: Blocker Fix For: 1.0.1, 1.1.0, 0.98.12 Attachments: HBASE-13326-0.98.patch, HBASE-13326-branch-1.0.patch, HBASE-13326-branch-1.0.patch, HBASE-13326-branch-1.0.v2.patch, HBASE-13326-branch-1.patch, HBASE-13326-branch-1.v2.patch The folks at Intel discovered a pretty nasty bug in 1.0 and 1.1 (but not master). Steps to reproduce: 1. Create a table, any table. 2. Disable the table. 3. Restart HBase. 4. Try enabling the table. The table won't become enabled and the master web UI will indicate a never-ending region in transition. Also worth noting is that [~mbertozzi] dug in and noted that this isn't happening in the master branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13334) FindBugs should create precise report for new bugs introduced
[ https://issues.apache.org/jira/browse/HBASE-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381705#comment-14381705 ] Hudson commented on HBASE-13334: FAILURE: Integrated in HBase-TRUNK #6306 (See [https://builds.apache.org/job/HBase-TRUNK/6306/]) HBASE-13334 FindBugs should create precise report for new bugs introduced (addendum) (octo47: rev 69f6a2e7b44d33e01866928672700cde1366f429) * dev-support/test-patch.sh FindBugs should create precise report for new bugs introduced - Key: HBASE-13334 URL: https://issues.apache.org/jira/browse/HBASE-13334 Project: HBase Issue Type: Improvement Components: build Reporter: Andrey Stepachev Assignee: Andrey Stepachev Priority: Minor Attachments: HBASE-13334.patch, HBASE-13334.v2.patch Currently findbugs build process reports only number of bugs introduced. And there is no report on what acutally was introduced. Lets improve that: we can use computeBugHistory to generate precise report on new bugs (and optionally missed bugs). Report should be available in html format. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13337) Table regions are not assigning back, after restarting all regionservers at once.
[
https://issues.apache.org/jira/browse/HBASE-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381714#comment-14381714
]
Ashish Singhi commented on HBASE-13337:
---
I am assigning this to me.
If any one else wants to take over, please feel free.
Table regions are not assigning back, after restarting all regionservers at
once.
-
Key: HBASE-13337
URL: https://issues.apache.org/jira/browse/HBASE-13337
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 2.0.0
Reporter: Y. SREENIVASULU REDDY
Priority: Blocker
Fix For: 2.0.0
Regions of the table are continouly in state=FAILED_CLOSE.
{noformat}
RegionState
RIT time (ms)
8f62e819b356736053e06240f7f7c6fd
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113929
caf59209ae65ea80fca6bdc6996a7d68
t1,,1427362431330.caf59209ae65ea80fca6bdc6996a7d68.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM2,16040,1427362533691 113929
db52a74988f71e5cf257bbabf31f26f3
t1,,1427362431330.db52a74988f71e5cf257bbabf31f26f3.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM3,16040,1427362533691 113920
43f3a65b9f9ff283f598c5450feab1f8
t1,,1427362431330.43f3a65b9f9ff283f598c5450feab1f8.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113920
{noformat}
*Steps to reproduce:*
1. Start HBase cluster with more than one regionserver.
2. Create a table with precreated regions. (lets say 15 regions)
3. Make sure the regions are well balanced.
4. Restart all the Regionservers process at once across the cluster, except
HMaster process
5. After restarting the Regionservers, successfully will connect to the
HMaster.
*Bug:*
But no regions are assigning back to the Regionservers.
*Master log shows as follows:*
{noformat}
2015-03-26 15:05:36,201 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=OFFLINE, ts=1427362536106, server=VM2,16040,1427362242602} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,202 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_OPENsn=VM1,16040,1427362531818
2015-03-26 15:05:36,244 DEBUG [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Force region state offline
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=PENDING_OPEN, ts=1427362536201, server=VM1,16040,1427362531818} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_CLOSE, ts=1427362536244,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_CLOSE
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=1 of 10
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=2 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=3 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=4 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager:
[jira] [Assigned] (HBASE-13337) Table regions are not assigning back, after restarting all regionservers at once.
[
https://issues.apache.org/jira/browse/HBASE-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Singhi reassigned HBASE-13337:
-
Assignee: Ashish Singhi
Table regions are not assigning back, after restarting all regionservers at
once.
-
Key: HBASE-13337
URL: https://issues.apache.org/jira/browse/HBASE-13337
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 2.0.0
Reporter: Y. SREENIVASULU REDDY
Assignee: Ashish Singhi
Priority: Blocker
Fix For: 2.0.0
Regions of the table are continouly in state=FAILED_CLOSE.
{noformat}
RegionState
RIT time (ms)
8f62e819b356736053e06240f7f7c6fd
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113929
caf59209ae65ea80fca6bdc6996a7d68
t1,,1427362431330.caf59209ae65ea80fca6bdc6996a7d68.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM2,16040,1427362533691 113929
db52a74988f71e5cf257bbabf31f26f3
t1,,1427362431330.db52a74988f71e5cf257bbabf31f26f3.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM3,16040,1427362533691 113920
43f3a65b9f9ff283f598c5450feab1f8
t1,,1427362431330.43f3a65b9f9ff283f598c5450feab1f8.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113920
{noformat}
*Steps to reproduce:*
1. Start HBase cluster with more than one regionserver.
2. Create a table with precreated regions. (lets say 15 regions)
3. Make sure the regions are well balanced.
4. Restart all the Regionservers process at once across the cluster, except
HMaster process
5. After restarting the Regionservers, successfully will connect to the
HMaster.
*Bug:*
But no regions are assigning back to the Regionservers.
*Master log shows as follows:*
{noformat}
2015-03-26 15:05:36,201 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=OFFLINE, ts=1427362536106, server=VM2,16040,1427362242602} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,202 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_OPENsn=VM1,16040,1427362531818
2015-03-26 15:05:36,244 DEBUG [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Force region state offline
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=PENDING_OPEN, ts=1427362536201, server=VM1,16040,1427362531818} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_CLOSE, ts=1427362536244,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_CLOSE
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=1 of 10
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=2 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=3 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=4 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
[jira] [Created] (HBASE-13338) Hbase to use PowerPC supported Jruby version 1.7.20
Ayappan created HBASE-13338: --- Summary: Hbase to use PowerPC supported Jruby version 1.7.20 Key: HBASE-13338 URL: https://issues.apache.org/jira/browse/HBASE-13338 Project: HBase Issue Type: Task Components: dependencies Affects Versions: 0.98.12, 1.0.0 Environment: PowerPC64, PowerPC64LE Reporter: Ayappan Older versions of jffi (till 1.2.7) don't have native PPC64 PPC64LE libraries. The latest released 1.2.8 version has PowerPC libraries and the jruby development version(1.7.20--SNAPSHOT) has been updated to make use of this version. Hbase still uses much older jruby 1.6.8 version which don't have the native libraries and this affects the Hbase shell in PowerPCs. So Hbase needs to be updated to make use of the upcoming Jruby release 1.7.20 to support PowerPC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13338) Hbase to use PowerPC supported Jruby version 1.7.20
[ https://issues.apache.org/jira/browse/HBASE-13338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayappan updated HBASE-13338: Component/s: shell Hbase to use PowerPC supported Jruby version 1.7.20 --- Key: HBASE-13338 URL: https://issues.apache.org/jira/browse/HBASE-13338 Project: HBase Issue Type: Task Components: dependencies, shell Affects Versions: 1.0.0, 0.98.12 Environment: PowerPC64, PowerPC64LE Reporter: Ayappan Older versions of jffi (till 1.2.7) don't have native PPC64 PPC64LE libraries. The latest released 1.2.8 version has PowerPC libraries and the jruby development version(1.7.20--SNAPSHOT) has been updated to make use of this version. Hbase still uses much older jruby 1.6.8 version which don't have the native libraries and this affects the Hbase shell in PowerPCs. So Hbase needs to be updated to make use of the upcoming Jruby release 1.7.20 to support PowerPC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13327) Use Admin in ConnectionCache
[ https://issues.apache.org/jira/browse/HBASE-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Solomon Duskis updated HBASE-13327: --- Attachment: HBASE-13327-addendum.patch I replaced the nested try with a cast of getAdmin() to HBaseAdmin and a call to the original compact/majorCompact method call. I added a TODO to deprecate the thrift method and create table and region counterparts. Use Admin in ConnectionCache Key: HBASE-13327 URL: https://issues.apache.org/jira/browse/HBASE-13327 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0, 1.0.1, 1.1.0 Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 1.0.1, 1.1.0 Attachments: HBASE-13327-addendum.patch, HBASE-13327.patch Replace HBaseAdmin usage with Admin in ConnectionCache and users of ConnectionCache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12413) Mismatch in the equals and hashcode methods of KeyValue
[ https://issues.apache.org/jira/browse/HBASE-12413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381806#comment-14381806 ] Gabriel Reid commented on HBASE-12413: -- [~tedyu] if this change is incompatible (i.e. if it breaks anything), I would assume that it is just making it more clear that that thing was broken anyhow. The case I'm imagining is that KeyValues are being put into a HashSet or used as keys in a HashMap -- in this case, the current (unpatched) behavior would be non-determinsitic, but may appear to work in many cases (i.e. the fact that the client code is broken might not be immediately visible). I think that committing this can only be a good thing -- I think that the worst case is that it would make it more clear that already-broken code is actually broken. Mismatch in the equals and hashcode methods of KeyValue --- Key: HBASE-12413 URL: https://issues.apache.org/jira/browse/HBASE-12413 Project: HBase Issue Type: Bug Reporter: Jingcheng Du Assignee: Jingcheng Du Priority: Minor Attachments: HBASE-12413-V2.diff, HBASE-12413.diff In the equals method of KeyValue only row key is compared, and in the hashcode method all bacing bytes are calculated. This breaks the Java rule. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13252) Get rid of managed connections and connection caching
[
https://issues.apache.org/jira/browse/HBASE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381813#comment-14381813
]
Hadoop QA commented on HBASE-13252:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707480/HBASE-13252-v1.patch
against master branch at commit 69f6a2e7b44d33e01866928672700cde1366f429.
ATTACHMENT ID: 12707480
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 22 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestAcidGuarantees
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
{color:red}-1 core zombie tests{color}. There are 6 zombie test(s):
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testRegionHole(TestHBaseFsck.java:1136)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testOverlapAndOrphan(TestHBaseFsck.java:1082)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1832)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testLingeringReferenceFile(TestHBaseFsck.java:2312)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testNoTableState(TestHBaseFsck.java:1528)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testFixAssignmentsAndNoHdfsChecking(TestHBaseFsck.java:1968)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testHbckAfterRegionMerge(TestHBaseFsck.java:2661)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testSidelineOverlapRegion(TestHBaseFsck.java:1038)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testFixByTable(TestHBaseFsck.java:1635)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testMissingLastRegion(TestHBaseFsck.java:1917)
at
org.apache.hadoop.hbase.client.TestFromClientSide.testWeirdCacheBehaviour(TestFromClientSide.java:313)
at
org.apache.hadoop.hbase.client.TestFromClientSide.testNullWithReverseScan(TestFromClientSide.java:5768)
at
org.apache.hadoop.hbase.client.TestHCM.testConnectionIdle(TestHCM.java:410)
at
org.apache.hadoop.hbase.TestPartialResultsFromClientSide.testOrderingOfCellsInPartialResults(TestPartialResultsFromClientSide.java:256)
at
org.apache.hadoop.hbase.TestPartialResultsFromClientSide.testOrderingOfCellsInPartialResults(TestPartialResultsFromClientSide.java:236)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13438//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13438//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13438//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13438//console
This message is automatically generated.
Get rid of managed connections and connection caching
-
Key: HBASE-13252
URL: https://issues.apache.org/jira/browse/HBASE-13252
Project: HBase
Issue Type: Sub-task
Components: API
Affects Versions: 2.0.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
Attachments: HBASE-13252-v0.patch, HBASE-13252-v1.patch
(Need to):
- Remove CONNECTION_INSTANCES from ConnectionManager
- Remove HConnectionKey
- Remove 'managed' property from ClusterConnection, HCI and places where
it's used now
- AFAIS this property isn't visible to client (ClusterConnection is private
interface), so technically this would not even be backward-incompatible
change, and no release note needed?
--
This message was sent by
[jira] [Commented] (HBASE-13338) Hbase to use PowerPC supported Jruby version 1.7.20
[ https://issues.apache.org/jira/browse/HBASE-13338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381794#comment-14381794 ] Jean-Marc Spaggiari commented on HBASE-13338: - Hi Ayappan, Are you going to provide a patch for that? Testing might be a bit hard since JRuby is mostily used for the shell and there is no unit test for that. Also, any improvement on the JRuby side will not really have any impact on the HBase side since HBase mostly use the Java API even from the JRuby calls. What is the expectation of this upgrade? Thanks. Hbase to use PowerPC supported Jruby version 1.7.20 --- Key: HBASE-13338 URL: https://issues.apache.org/jira/browse/HBASE-13338 Project: HBase Issue Type: Task Components: dependencies, shell Affects Versions: 1.0.0, 0.98.12 Environment: PowerPC64, PowerPC64LE Reporter: Ayappan Older versions of jffi (till 1.2.7) don't have native PPC64 PPC64LE libraries. The latest released 1.2.8 version has PowerPC libraries and the jruby development version(1.7.20--SNAPSHOT) has been updated to make use of this version. Hbase still uses much older jruby 1.6.8 version which don't have the native libraries and this affects the Hbase shell in PowerPCs. So Hbase needs to be updated to make use of the upcoming Jruby release 1.7.20 to support PowerPC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13329) Memstore flush fails if data has always the same value, breaking the region
[ https://issues.apache.org/jira/browse/HBASE-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381804#comment-14381804 ] Ruben Aguiar commented on HBASE-13329: -- After further testing and tweaking, I'm starting to doubt this is related to the data being written itself. At the moment, I have 4 different servers sending random data, each one of these sending to a random metric between its range of 10 possible ones. This means writes are never repeated constantly, either the data or the metrics itself. Even regarding tags, I'm constantly changing between two of them. However, after deleting the whole database and starting the whole cluster from a clean sheet, after a few minutes a similar problem arises. This time, it fails a bit further on the code with a negative array size exception: 2015-03-26 11:59:42,597 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://10.2.0.74:8020/hbase/data/default/tsdb/59f6a2c16cd9c 51677efa96ddf038fe8/t/d999a9dee66c4948862440ba1459a826, entries=806262, sequenceid=16116, filesize=6.8 M 2015-03-26 11:59:42,599 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished memstore flush of ~129.72 MB/136019416, currentsize=1.7 5 MB/1837224 for region tsdb,,1427368142097.59f6a2c16cd9c51677efa96ddf038fe8. in 717ms, sequenceid=16116, compaction requested=false 2015-03-26 12:02:15,957 INFO [MemStoreFlusher.0] regionserver.HRegion: Started memstore flush for tsdb,,1427368142097.59f6a2c16cd9c5167 7efa96ddf038fe8., current region memstore size 128.06 MB 2015-03-26 12:02:16,077 FATAL [MemStoreFlusher.0] regionserver.HRegionServer: ABORTING region server 10.2.0.73,16020,1427368055017: Repl ay of WAL required. Forcing server shutdown org.apache.hadoop.hbase.DroppedSnapshotException: region: tsdb,,1427368142097.59f6a2c16cd9c51677efa96ddf038fe8. at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1999) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1770) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1702) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:445) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:407) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$800(MemStoreFlusher.java:69) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:225) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NegativeArraySizeException at org.apache.hadoop.hbase.CellComparator.getMinimumMidpointArray(CellComparator.java:490) at org.apache.hadoop.hbase.CellComparator.getMidpoint(CellComparator.java:448) at org.apache.hadoop.hbase.io.hfile.HFileWriterV2.finishBlock(HFileWriterV2.java:165) at org.apache.hadoop.hbase.io.hfile.HFileWriterV2.checkBlockBoundary(HFileWriterV2.java:146) at org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:263) at org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87) at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:932) at org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:121) at org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:71) at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:879) at org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2128) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1953) ... 7 more After some investigation, I ran a hdfs fsck and it reported healthy. However, with the flag -openforwrite (include files opened for writes) I get corrupted blocks that refer to the WALS on the other slave machines. This might be due to the region server crashing unexpectedly. The fsck output: /hbase/WALs/10.2.0.171,16020,1427368055792/10.2.0.171%2C16020%2C1427368055792.default.1427368062105 83 bytes, 1 block(s), OPENFORWRITE: /hbase/WALs/10.2.0.171,16020,1427368055792/10.2.0.171%2C16020%2C1427368055792.default.1427368062105: MISSING 1 blocks of total size 83 B../hbase/WALs/10.2.0.224,16020,1427368062011-splitting/10.2.0.224%2C16020%2C1427368062011.default.1427368068287 83 bytes, 1 block(s), OPENFORWRITE: /hbase/WALs/10.2.0.224,16020,1427368062011-splitting/10.2.0.224%2C16020%2C1427368062011.default.1427368068287: MISSING 1 blocks of total size 83 B.Status: CORRUPT Total size:105297956 B Total dirs:54 Total files: 39 Total symlinks:0 Total blocks (validated): 33 (avg. block size 3190847 B) CORRUPT FILES:
[jira] [Commented] (HBASE-13338) Hbase to use PowerPC supported Jruby version 1.7.20
[ https://issues.apache.org/jira/browse/HBASE-13338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381810#comment-14381810 ] Jonathan Hsieh commented on HBASE-13338: I agree with [~jmspaggi] here about testing being the concern here. Do you know if the old version you are using is compatible with the new? [~Ayappan], if you make the patch (hopefully just a straight foroward change to pom), can you make sure to run [1] and report on if manually verifies that things seem to work? [1] https://github.com/apache/hbase/blob/master/hbase-shell/src/test/java/org/apache/hadoop/hbase/client/TestShell.java Hbase to use PowerPC supported Jruby version 1.7.20 --- Key: HBASE-13338 URL: https://issues.apache.org/jira/browse/HBASE-13338 Project: HBase Issue Type: Task Components: dependencies, shell Affects Versions: 1.0.0, 0.98.12 Environment: PowerPC64, PowerPC64LE Reporter: Ayappan Older versions of jffi (till 1.2.7) don't have native PPC64 PPC64LE libraries. The latest released 1.2.8 version has PowerPC libraries and the jruby development version(1.7.20--SNAPSHOT) has been updated to make use of this version. Hbase still uses much older jruby 1.6.8 version which don't have the native libraries and this affects the Hbase shell in PowerPCs. So Hbase needs to be updated to make use of the upcoming Jruby release 1.7.20 to support PowerPC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13262) ResultScanner doesn't return all rows in Scan
[
https://issues.apache.org/jira/browse/HBASE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-13262:
Fix Version/s: 1.0.1
ResultScanner doesn't return all rows in Scan
-
Key: HBASE-13262
URL: https://issues.apache.org/jira/browse/HBASE-13262
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 2.0.0, 1.1.0
Environment: Single node, pseduo-distributed 1.1.0-SNAPSHOT
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Blocker
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13
Attachments: 13262-0.98-testpatch.txt, HBASE-13262-branch-1-v2.patch,
HBASE-13262-branch-1-v3.patch, HBASE-13262-branch-1.patch,
HBASE-13262-v1.patch, HBASE-13262-v2.patch, HBASE-13262-v3.patch,
HBASE-13262-v4.patch, HBASE-13262-v5.patch, HBASE-13262-v6.patch,
HBASE-13262-v7.patch, HBASE-13262-v7.patch, HBASE-13262.patch,
regionserver-logging.diff, testrun_0.98.txt, testrun_branch1.0.txt
Tried to write a simple Java client again 1.1.0-SNAPSHOT.
* Write 1M rows, each row with 1 family, and 10 qualifiers (values [0-9]),
for a total of 10M cells written
* Read back the data from the table, ensure I saw 10M cells
Running it against {{04ac1891}} (and earlier) yesterday, I would get ~20% of
the actual rows. Running against 1.0.0, returns all 10M records as expected.
[Code I was
running|https://github.com/joshelser/hbase-hwhat/blob/master/src/main/java/hbase/HBaseTest.java]
for the curious.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13262) ResultScanner doesn't return all rows in Scan
[
https://issues.apache.org/jira/browse/HBASE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383244#comment-14383244
]
Ted Yu commented on HBASE-13262:
+1 on patch v7.
ResultScanner doesn't return all rows in Scan
-
Key: HBASE-13262
URL: https://issues.apache.org/jira/browse/HBASE-13262
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 2.0.0, 1.1.0
Environment: Single node, pseduo-distributed 1.1.0-SNAPSHOT
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 0.98.13
Attachments: 13262-0.98-testpatch.txt, HBASE-13262-branch-1-v2.patch,
HBASE-13262-branch-1-v3.patch, HBASE-13262-branch-1.patch,
HBASE-13262-v1.patch, HBASE-13262-v2.patch, HBASE-13262-v3.patch,
HBASE-13262-v4.patch, HBASE-13262-v5.patch, HBASE-13262-v6.patch,
HBASE-13262-v7.patch, HBASE-13262-v7.patch, HBASE-13262.patch,
regionserver-logging.diff, testrun_0.98.txt, testrun_branch1.0.txt
Tried to write a simple Java client again 1.1.0-SNAPSHOT.
* Write 1M rows, each row with 1 family, and 10 qualifiers (values [0-9]),
for a total of 10M cells written
* Read back the data from the table, ensure I saw 10M cells
Running it against {{04ac1891}} (and earlier) yesterday, I would get ~20% of
the actual rows. Running against 1.0.0, returns all 10M records as expected.
[Code I was
running|https://github.com/joshelser/hbase-hwhat/blob/master/src/main/java/hbase/HBaseTest.java]
for the curious.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12413) Mismatch in the equals and hashcode methods of KeyValue
[
https://issues.apache.org/jira/browse/HBASE-12413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383247#comment-14383247
]
Ted Yu commented on HBASE-12413:
lgtm
{code}
1068 * In line with {@link #equals(Object)}, only compares the key
portion,
1069 * not the value.
{code}
Looks like the verb 'compares' is not accurate: we're calculating hash code,
right ?
Mismatch in the equals and hashcode methods of KeyValue
---
Key: HBASE-12413
URL: https://issues.apache.org/jira/browse/HBASE-12413
Project: HBase
Issue Type: Bug
Reporter: Jingcheng Du
Assignee: Jingcheng Du
Priority: Minor
Attachments: HBASE-12413-V2.diff, HBASE-12413.diff,
HBASE-12413.v3.patch
In the equals method of KeyValue only row key is compared, and in the
hashcode method all bacing bytes are calculated. This breaks the Java rule.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13295) TestInfoServers hang
[
https://issues.apache.org/jira/browse/HBASE-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383303#comment-14383303
]
zhangduo commented on HBASE-13295:
--
Pushed to master and branch-1.
TestInfoServers hang
Key: HBASE-13295
URL: https://issues.apache.org/jira/browse/HBASE-13295
Project: HBase
Issue Type: Bug
Components: test
Reporter: zhangduo
Assignee: zhangduo
Attachments: HBASE-13295.patch
https://builds.apache.org/job/HBase-TRUNK-jacoco/16/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.TestInfoServers-output.txt
No progress after this line
{noformat}
2015-03-19 22:46:06,809 INFO [main] hbase.TestInfoServers(127): Testing
http://localhost:44749/table.jsp?name=testMasterServerReadOnlyaction=splitkey=
has Table action request accepted
{noformat}
I think the problem maybe we do not wait for master finish becoming active,
and there is no timeout when doing http request.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13337) Table regions are not assigning back, after restarting all regionservers at once.
[
https://issues.apache.org/jira/browse/HBASE-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383335#comment-14383335
]
Y. SREENIVASULU REDDY commented on HBASE-13337:
---
[~lhofhansl]
All regionservers are restarted graceful,
I tested one more scenario like out of 3 regionservers make down one
regionserver graceful, in this case it is assigning properly.
All regionservers making down and startup again only causing problem.
Table regions are not assigning back, after restarting all regionservers at
once.
-
Key: HBASE-13337
URL: https://issues.apache.org/jira/browse/HBASE-13337
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 2.0.0
Reporter: Y. SREENIVASULU REDDY
Assignee: Ashish Singhi
Priority: Blocker
Fix For: 2.0.0
Regions of the table are continouly in state=FAILED_CLOSE.
{noformat}
RegionState
RIT time (ms)
8f62e819b356736053e06240f7f7c6fd
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113929
caf59209ae65ea80fca6bdc6996a7d68
t1,,1427362431330.caf59209ae65ea80fca6bdc6996a7d68.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM2,16040,1427362533691 113929
db52a74988f71e5cf257bbabf31f26f3
t1,,1427362431330.db52a74988f71e5cf257bbabf31f26f3.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM3,16040,1427362533691 113920
43f3a65b9f9ff283f598c5450feab1f8
t1,,1427362431330.43f3a65b9f9ff283f598c5450feab1f8.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113920
{noformat}
*Steps to reproduce:*
1. Start HBase cluster with more than one regionserver.
2. Create a table with precreated regions. (lets say 15 regions)
3. Make sure the regions are well balanced.
4. Restart all the Regionservers process at once across the cluster, except
HMaster process
5. After restarting the Regionservers, successfully will connect to the
HMaster.
*Bug:*
But no regions are assigning back to the Regionservers.
*Master log shows as follows:*
{noformat}
2015-03-26 15:05:36,201 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=OFFLINE, ts=1427362536106, server=VM2,16040,1427362242602} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,202 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_OPENsn=VM1,16040,1427362531818
2015-03-26 15:05:36,244 DEBUG [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Force region state offline
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=PENDING_OPEN, ts=1427362536201, server=VM1,16040,1427362531818} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_CLOSE, ts=1427362536244,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_CLOSE
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=1 of 10
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=2 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=3 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
[jira] [Commented] (HBASE-13252) Get rid of managed connections and connection caching
[ https://issues.apache.org/jira/browse/HBASE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383351#comment-14383351 ] Mikhail Antonov commented on HBASE-13252: - It's getting better.. Tests run: 18, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 198.478 sec FAILURE! - in org.apache.hadoop.hbase.master.TestDistributedLogSplitting testLogReplayTwoSequentialRSDown(org.apache.hadoop.hbase.master.TestDistributedLogSplitting) Time elapsed: 37.21 sec FAILURE! java.lang.AssertionError: expected:1000 but was:891 at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) That's the only test which failed, and this failure doesn't look like something caused by connection rewiring..gonna retry. Get rid of managed connections and connection caching - Key: HBASE-13252 URL: https://issues.apache.org/jira/browse/HBASE-13252 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Attachments: HBASE-13252-v0.patch, HBASE-13252-v1.patch, HBASE-13252-v1.patch, HBASE-13252-v2.patch (Need to): - Remove CONNECTION_INSTANCES from ConnectionManager - Remove HConnectionKey - Remove 'managed' property from ClusterConnection, HCI and places where it's used now - AFAIS this property isn't visible to client (ClusterConnection is private interface), so technically this would not even be backward-incompatible change, and no release note needed? - In fact, ConnectionManager can be removed, too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13252) Get rid of managed connections and connection caching
[ https://issues.apache.org/jira/browse/HBASE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13252: Status: Patch Available (was: Open) Get rid of managed connections and connection caching - Key: HBASE-13252 URL: https://issues.apache.org/jira/browse/HBASE-13252 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Attachments: HBASE-13252-v0.patch, HBASE-13252-v1.patch, HBASE-13252-v1.patch, HBASE-13252-v2.patch (Need to): - Remove CONNECTION_INSTANCES from ConnectionManager - Remove HConnectionKey - Remove 'managed' property from ClusterConnection, HCI and places where it's used now - AFAIS this property isn't visible to client (ClusterConnection is private interface), so technically this would not even be backward-incompatible change, and no release note needed? - In fact, ConnectionManager can be removed, too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13252) Get rid of managed connections and connection caching
[ https://issues.apache.org/jira/browse/HBASE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13252: Status: Open (was: Patch Available) Get rid of managed connections and connection caching - Key: HBASE-13252 URL: https://issues.apache.org/jira/browse/HBASE-13252 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Attachments: HBASE-13252-v0.patch, HBASE-13252-v1.patch, HBASE-13252-v1.patch, HBASE-13252-v2.patch (Need to): - Remove CONNECTION_INSTANCES from ConnectionManager - Remove HConnectionKey - Remove 'managed' property from ClusterConnection, HCI and places where it's used now - AFAIS this property isn't visible to client (ClusterConnection is private interface), so technically this would not even be backward-incompatible change, and no release note needed? - In fact, ConnectionManager can be removed, too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13291) Lift the scan ceiling
[ https://issues.apache.org/jira/browse/HBASE-13291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383367#comment-14383367 ] stack commented on HBASE-13291: --- Oh, my test has 50 clients going over a network. Maybe I'm going faster than you. Lift the scan ceiling - Key: HBASE-13291 URL: https://issues.apache.org/jira/browse/HBASE-13291 Project: HBase Issue Type: Improvement Components: Scanners Affects Versions: 1.0.0 Reporter: stack Assignee: stack Attachments: 13291.inlining.txt, Screen Shot 2015-03-26 at 12.12.13 PM.png, Screen Shot 2015-03-26 at 3.39.33 PM.png, hack_to_bypass_bb.txt, nonBBposAndInineMvccVint.txt, q (1).png, traces.7.svg, traces.filterall.svg, traces.nofilter.svg, traces.small2.svg, traces.smaller.svg Scanning medium sized rows with multiple concurrent scanners exhibits interesting 'ceiling' properties. A server runs at about 6.7k ops a second using 450% of possible 1600% of CPUs when 4 clients each with 10 threads doing scan 1000 rows. If I add '--filterAll' argument (do not return results), then we run at 1450% of possible 1600% possible but we do 8k ops a second. Let me attach flame graphs for two cases. Unfortunately, there is some frustrating dark art going on. Let me try figure it... Filing issue in meantime to keep score in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13291) Lift the scan ceiling
[ https://issues.apache.org/jira/browse/HBASE-13291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383381#comment-14383381 ] ramkrishna.s.vasudevan commented on HBASE-13291: Using PE tool with ScanRange will do seek because it is using Explicit Column Tracker. Right? FilterallFilter usage will also do the same I think. Lift the scan ceiling - Key: HBASE-13291 URL: https://issues.apache.org/jira/browse/HBASE-13291 Project: HBase Issue Type: Improvement Components: Scanners Affects Versions: 1.0.0 Reporter: stack Assignee: stack Attachments: 13291.inlining.txt, Screen Shot 2015-03-26 at 12.12.13 PM.png, Screen Shot 2015-03-26 at 3.39.33 PM.png, hack_to_bypass_bb.txt, nonBBposAndInineMvccVint.txt, q (1).png, traces.7.svg, traces.filterall.svg, traces.nofilter.svg, traces.small2.svg, traces.smaller.svg Scanning medium sized rows with multiple concurrent scanners exhibits interesting 'ceiling' properties. A server runs at about 6.7k ops a second using 450% of possible 1600% of CPUs when 4 clients each with 10 threads doing scan 1000 rows. If I add '--filterAll' argument (do not return results), then we run at 1450% of possible 1600% possible but we do 8k ops a second. Let me attach flame graphs for two cases. Unfortunately, there is some frustrating dark art going on. Let me try figure it... Filing issue in meantime to keep score in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13262) ResultScanner doesn't return all rows in Scan
[
https://issues.apache.org/jira/browse/HBASE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383242#comment-14383242
]
Hadoop QA commented on HBASE-13262:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707651/HBASE-13262-v7.patch
against master branch at commit 18256fc9544866fc7182dfc5dd9d73b8cef02d66.
ATTACHMENT ID: 12707651
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13455//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13455//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13455//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13455//console
This message is automatically generated.
ResultScanner doesn't return all rows in Scan
-
Key: HBASE-13262
URL: https://issues.apache.org/jira/browse/HBASE-13262
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 2.0.0, 1.1.0
Environment: Single node, pseduo-distributed 1.1.0-SNAPSHOT
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 0.98.13
Attachments: 13262-0.98-testpatch.txt, HBASE-13262-branch-1-v2.patch,
HBASE-13262-branch-1-v3.patch, HBASE-13262-branch-1.patch,
HBASE-13262-v1.patch, HBASE-13262-v2.patch, HBASE-13262-v3.patch,
HBASE-13262-v4.patch, HBASE-13262-v5.patch, HBASE-13262-v6.patch,
HBASE-13262-v7.patch, HBASE-13262-v7.patch, HBASE-13262.patch,
regionserver-logging.diff, testrun_0.98.txt, testrun_branch1.0.txt
Tried to write a simple Java client again 1.1.0-SNAPSHOT.
* Write 1M rows, each row with 1 family, and 10 qualifiers (values [0-9]),
for a total of 10M cells written
* Read back the data from the table, ensure I saw 10M cells
Running it against {{04ac1891}} (and earlier) yesterday, I would get ~20% of
the actual rows. Running against 1.0.0, returns all 10M records as expected.
[Code I was
running|https://github.com/joshelser/hbase-hwhat/blob/master/src/main/java/hbase/HBaseTest.java]
for the curious.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13327) Use Admin in ConnectionCache
[ https://issues.apache.org/jira/browse/HBASE-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383276#comment-14383276 ] Hudson commented on HBASE-13327: SUCCESS: Integrated in HBase-TRUNK #6308 (See [https://builds.apache.org/job/HBase-TRUNK/6308/]) HBASE-13327 Addendum replaced the nested try with a cast of getAdmin() to HBaseAdmin and a call to the original compact/majorCompact method (Solomon) (tedyu: rev 306c44156a0f8248f9271d03000244182cecb3ae) * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java Use Admin in ConnectionCache Key: HBASE-13327 URL: https://issues.apache.org/jira/browse/HBASE-13327 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0, 1.0.1, 1.1.0 Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 1.0.1, 1.1.0 Attachments: HBASE-13327-addendum.patch, HBASE-13327.patch Replace HBaseAdmin usage with Admin in ConnectionCache and users of ConnectionCache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13252) Get rid of managed connections and connection caching
[
https://issues.apache.org/jira/browse/HBASE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383345#comment-14383345
]
Hadoop QA commented on HBASE-13252:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707677/HBASE-13252-v2.patch
against master branch at commit 306c44156a0f8248f9271d03000244182cecb3ae.
ATTACHMENT ID: 12707677
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 38 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13456//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13456//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13456//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13456//console
This message is automatically generated.
Get rid of managed connections and connection caching
-
Key: HBASE-13252
URL: https://issues.apache.org/jira/browse/HBASE-13252
Project: HBase
Issue Type: Sub-task
Components: API
Affects Versions: 2.0.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
Attachments: HBASE-13252-v0.patch, HBASE-13252-v1.patch,
HBASE-13252-v1.patch, HBASE-13252-v2.patch
(Need to):
- Remove CONNECTION_INSTANCES from ConnectionManager
- Remove HConnectionKey
- Remove 'managed' property from ClusterConnection, HCI and places where
it's used now
- AFAIS this property isn't visible to client (ClusterConnection is private
interface), so technically this would not even be backward-incompatible
change, and no release note needed?
- In fact, ConnectionManager can be removed, too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13337) Table regions are not assigning back, after restarting all regionservers at once.
[
https://issues.apache.org/jira/browse/HBASE-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383352#comment-14383352
]
Lars Hofhansl commented on HBASE-13337:
---
Right. But when all region servers are gone, HMaster cannot assign any regions.
As soon as the first few region servers are seen by the master again, it'll
assign all the regions to them.
In some case that might be what you want, in others it's not. Unless we tell
the master it can't know.
We can add some guessing of course. Maybe when more than 1/2 of the region
server are suddenly gone, wait a bit before reassigning the regions. But I will
bet that any guessing we do will have somebody complain.
Or we can invent a new API to tell the master to hold off assigning regions.
In the end the the solution is simple: If you plan to bring down many region
servers, stop the HMaster first.
Table regions are not assigning back, after restarting all regionservers at
once.
-
Key: HBASE-13337
URL: https://issues.apache.org/jira/browse/HBASE-13337
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 2.0.0
Reporter: Y. SREENIVASULU REDDY
Assignee: Ashish Singhi
Priority: Blocker
Fix For: 2.0.0
Regions of the table are continouly in state=FAILED_CLOSE.
{noformat}
RegionState
RIT time (ms)
8f62e819b356736053e06240f7f7c6fd
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113929
caf59209ae65ea80fca6bdc6996a7d68
t1,,1427362431330.caf59209ae65ea80fca6bdc6996a7d68.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM2,16040,1427362533691 113929
db52a74988f71e5cf257bbabf31f26f3
t1,,1427362431330.db52a74988f71e5cf257bbabf31f26f3.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM3,16040,1427362533691 113920
43f3a65b9f9ff283f598c5450feab1f8
t1,,1427362431330.43f3a65b9f9ff283f598c5450feab1f8.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113920
{noformat}
*Steps to reproduce:*
1. Start HBase cluster with more than one regionserver.
2. Create a table with precreated regions. (lets say 15 regions)
3. Make sure the regions are well balanced.
4. Restart all the Regionservers process at once across the cluster, except
HMaster process
5. After restarting the Regionservers, successfully will connect to the
HMaster.
*Bug:*
But no regions are assigning back to the Regionservers.
*Master log shows as follows:*
{noformat}
2015-03-26 15:05:36,201 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=OFFLINE, ts=1427362536106, server=VM2,16040,1427362242602} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,202 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_OPENsn=VM1,16040,1427362531818
2015-03-26 15:05:36,244 DEBUG [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Force region state offline
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=PENDING_OPEN, ts=1427362536201, server=VM1,16040,1427362531818} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_CLOSE, ts=1427362536244,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_CLOSE
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=1 of 10
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd.,
[jira] [Updated] (HBASE-13252) Get rid of managed connections and connection caching
[ https://issues.apache.org/jira/browse/HBASE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13252: Status: Open (was: Patch Available) Get rid of managed connections and connection caching - Key: HBASE-13252 URL: https://issues.apache.org/jira/browse/HBASE-13252 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Attachments: HBASE-13252-v0.patch, HBASE-13252-v1.patch, HBASE-13252-v1.patch, HBASE-13252-v2.patch, HBASE-13252-v2.patch (Need to): - Remove CONNECTION_INSTANCES from ConnectionManager - Remove HConnectionKey - Remove 'managed' property from ClusterConnection, HCI and places where it's used now - AFAIS this property isn't visible to client (ClusterConnection is private interface), so technically this would not even be backward-incompatible change, and no release note needed? - In fact, ConnectionManager can be removed, too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12972) Region, a supportable public/evolving subset of HRegion
[
https://issues.apache.org/jira/browse/HBASE-12972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383210#comment-14383210
]
Sean Busbey commented on HBASE-12972:
-
good by me. apologies that I haven't been able to give more feedback.
Region, a supportable public/evolving subset of HRegion
---
Key: HBASE-12972
URL: https://issues.apache.org/jira/browse/HBASE-12972
Project: HBase
Issue Type: New Feature
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 2.0.0, 1.1.0
Attachments: HBASE-12972-0.98.patch, HBASE-12972.patch,
HBASE-12972.patch, HBASE-12972.patch, HBASE-12972.patch
On HBASE-12566, [~lhofhansl] proposed:
{quote}
Maybe we can have a {{Region}} interface that is to {{HRegion}} is what
{{Store}} is to {{HStore}}. Store marked with {{@InterfaceAudience.Private}}
but used in some coprocessor hooks.
{quote}
By example, now coprocessors have to reach into HRegion in order to
participate in row and region locking protocols, this is one area where the
functionality is legitimate for coprocessors but not for users, so an
in-between interface make sense.
In addition we should promote {{Store}}'s interface audience to
LimitedPrivate(COPROC).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13327) Use Admin in ConnectionCache
[ https://issues.apache.org/jira/browse/HBASE-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383261#comment-14383261 ] Hudson commented on HBASE-13327: FAILURE: Integrated in HBase-1.0 #833 (See [https://builds.apache.org/job/HBase-1.0/833/]) HBASE-13327 Addendum replaced the nested try with a cast of getAdmin() to HBaseAdmin and a call to the original compact/majorCompact method (Solomon) (tedyu: rev ea03b10b0de29abc6ee72e8254c226f4aa529028) * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java Use Admin in ConnectionCache Key: HBASE-13327 URL: https://issues.apache.org/jira/browse/HBASE-13327 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0, 1.0.1, 1.1.0 Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 1.0.1, 1.1.0 Attachments: HBASE-13327-addendum.patch, HBASE-13327.patch Replace HBaseAdmin usage with Admin in ConnectionCache and users of ConnectionCache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13337) Table regions are not assigning back, after restarting all regionservers at once.
[
https://issues.apache.org/jira/browse/HBASE-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Singhi updated HBASE-13337:
--
Assignee: (was: Ashish Singhi)
Table regions are not assigning back, after restarting all regionservers at
once.
-
Key: HBASE-13337
URL: https://issues.apache.org/jira/browse/HBASE-13337
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 2.0.0
Reporter: Y. SREENIVASULU REDDY
Priority: Blocker
Fix For: 2.0.0
Regions of the table are continouly in state=FAILED_CLOSE.
{noformat}
RegionState
RIT time (ms)
8f62e819b356736053e06240f7f7c6fd
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113929
caf59209ae65ea80fca6bdc6996a7d68
t1,,1427362431330.caf59209ae65ea80fca6bdc6996a7d68.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM2,16040,1427362533691 113929
db52a74988f71e5cf257bbabf31f26f3
t1,,1427362431330.db52a74988f71e5cf257bbabf31f26f3.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM3,16040,1427362533691 113920
43f3a65b9f9ff283f598c5450feab1f8
t1,,1427362431330.43f3a65b9f9ff283f598c5450feab1f8.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113920
{noformat}
*Steps to reproduce:*
1. Start HBase cluster with more than one regionserver.
2. Create a table with precreated regions. (lets say 15 regions)
3. Make sure the regions are well balanced.
4. Restart all the Regionservers process at once across the cluster, except
HMaster process
5. After restarting the Regionservers, successfully will connect to the
HMaster.
*Bug:*
But no regions are assigning back to the Regionservers.
*Master log shows as follows:*
{noformat}
2015-03-26 15:05:36,201 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=OFFLINE, ts=1427362536106, server=VM2,16040,1427362242602} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,202 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_OPENsn=VM1,16040,1427362531818
2015-03-26 15:05:36,244 DEBUG [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Force region state offline
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=PENDING_OPEN, ts=1427362536201, server=VM1,16040,1427362531818} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_CLOSE, ts=1427362536244,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_CLOSE
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=1 of 10
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=2 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=3 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=4 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
[jira] [Commented] (HBASE-13222) Provide means of non-destructive balancer inspection
[ https://issues.apache.org/jira/browse/HBASE-13222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383265#comment-14383265 ] Josh Elser commented on HBASE-13222: Checkstyle and lineLength issues appear to be limited to generated code (jamon and protobuf, respectively). Provide means of non-destructive balancer inspection Key: HBASE-13222 URL: https://issues.apache.org/jira/browse/HBASE-13222 Project: HBase Issue Type: Improvement Components: Balancer Reporter: Nick Dimiduk Assignee: Josh Elser Priority: Minor Fix For: 2.0.0, 1.1.0, 0.98.13 Attachments: HBASE-13222-v1.patch, HBASE-13222-v2.patch, HBASE-13222-v3.patch, HBASE-13222.00-0.98.patch, HBASE-13222.patch, master.jpg At least on 0.98 (haven't checked master, 1.0), it seems we don't have a means for accessing balancer status w.o changing it in the process. Here's a little script to tickle ZK and see what the flag is. Would be nice if this was available from the shell, on the master status page, or something similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13252) Get rid of managed connections and connection caching
[ https://issues.apache.org/jira/browse/HBASE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13252: Status: Open (was: Patch Available) Get rid of managed connections and connection caching - Key: HBASE-13252 URL: https://issues.apache.org/jira/browse/HBASE-13252 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Attachments: HBASE-13252-v0.patch, HBASE-13252-v1.patch, HBASE-13252-v1.patch, HBASE-13252-v2.patch (Need to): - Remove CONNECTION_INSTANCES from ConnectionManager - Remove HConnectionKey - Remove 'managed' property from ClusterConnection, HCI and places where it's used now - AFAIS this property isn't visible to client (ClusterConnection is private interface), so technically this would not even be backward-incompatible change, and no release note needed? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13252) Get rid of managed connections and connection caching
[ https://issues.apache.org/jira/browse/HBASE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13252: Attachment: HBASE-13252-v2.patch Fixed some connection leaks, removed HConnectionKey and ConnectionManager. Get rid of managed connections and connection caching - Key: HBASE-13252 URL: https://issues.apache.org/jira/browse/HBASE-13252 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Attachments: HBASE-13252-v0.patch, HBASE-13252-v1.patch, HBASE-13252-v1.patch, HBASE-13252-v2.patch (Need to): - Remove CONNECTION_INSTANCES from ConnectionManager - Remove HConnectionKey - Remove 'managed' property from ClusterConnection, HCI and places where it's used now - AFAIS this property isn't visible to client (ClusterConnection is private interface), so technically this would not even be backward-incompatible change, and no release note needed? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13252) Get rid of managed connections and connection caching
[ https://issues.apache.org/jira/browse/HBASE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13252: Status: Patch Available (was: Open) Get rid of managed connections and connection caching - Key: HBASE-13252 URL: https://issues.apache.org/jira/browse/HBASE-13252 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Attachments: HBASE-13252-v0.patch, HBASE-13252-v1.patch, HBASE-13252-v1.patch, HBASE-13252-v2.patch (Need to): - Remove CONNECTION_INSTANCES from ConnectionManager - Remove HConnectionKey - Remove 'managed' property from ClusterConnection, HCI and places where it's used now - AFAIS this property isn't visible to client (ClusterConnection is private interface), so technically this would not even be backward-incompatible change, and no release note needed? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13332) Fix the usage of doAs/runAs in Visibility Controller tests.
[ https://issues.apache.org/jira/browse/HBASE-13332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-13332: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to branch-1.0. Thanks for the patch [~srikanth235] Fix the usage of doAs/runAs in Visibility Controller tests. --- Key: HBASE-13332 URL: https://issues.apache.org/jira/browse/HBASE-13332 Project: HBase Issue Type: Sub-task Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Fix For: 2.0.0, 1.0.1, 1.1.0 Attachments: HBASE-13332.patch, HBASE-13332.patch, HBASE-13332_v1.patch The doAs bug is also affecting VC related tests too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13252) Get rid of managed connections and connection caching
[ https://issues.apache.org/jira/browse/HBASE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13252: Status: Patch Available (was: Open) Get rid of managed connections and connection caching - Key: HBASE-13252 URL: https://issues.apache.org/jira/browse/HBASE-13252 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Attachments: HBASE-13252-v0.patch, HBASE-13252-v1.patch, HBASE-13252-v1.patch, HBASE-13252-v2.patch, HBASE-13252-v2.patch (Need to): - Remove CONNECTION_INSTANCES from ConnectionManager - Remove HConnectionKey - Remove 'managed' property from ClusterConnection, HCI and places where it's used now - AFAIS this property isn't visible to client (ClusterConnection is private interface), so technically this would not even be backward-incompatible change, and no release note needed? - In fact, ConnectionManager can be removed, too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13252) Get rid of managed connections and connection caching
[ https://issues.apache.org/jira/browse/HBASE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13252: Attachment: HBASE-13252-v2.patch same patch (to wake up build bot) Get rid of managed connections and connection caching - Key: HBASE-13252 URL: https://issues.apache.org/jira/browse/HBASE-13252 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Attachments: HBASE-13252-v0.patch, HBASE-13252-v1.patch, HBASE-13252-v1.patch, HBASE-13252-v2.patch, HBASE-13252-v2.patch (Need to): - Remove CONNECTION_INSTANCES from ConnectionManager - Remove HConnectionKey - Remove 'managed' property from ClusterConnection, HCI and places where it's used now - AFAIS this property isn't visible to client (ClusterConnection is private interface), so technically this would not even be backward-incompatible change, and no release note needed? - In fact, ConnectionManager can be removed, too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13295) TestInfoServers hang
[
https://issues.apache.org/jira/browse/HBASE-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383371#comment-14383371
]
Hudson commented on HBASE-13295:
FAILURE: Integrated in HBase-1.1 #328 (See
[https://builds.apache.org/job/HBase-1.1/328/])
HBASE-13295 TestInfoServers hung (zhangduo: rev
e8059b0dc570e4a7ec8915510e5b4405cfebef59)
* hbase-server/src/test/java/org/apache/hadoop/hbase/TestInfoServers.java
TestInfoServers hang
Key: HBASE-13295
URL: https://issues.apache.org/jira/browse/HBASE-13295
Project: HBase
Issue Type: Bug
Components: test
Reporter: zhangduo
Assignee: zhangduo
Attachments: HBASE-13295.patch
https://builds.apache.org/job/HBase-TRUNK-jacoco/16/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.TestInfoServers-output.txt
No progress after this line
{noformat}
2015-03-19 22:46:06,809 INFO [main] hbase.TestInfoServers(127): Testing
http://localhost:44749/table.jsp?name=testMasterServerReadOnlyaction=splitkey=
has Table action request accepted
{noformat}
I think the problem maybe we do not wait for master finish becoming active,
and there is no timeout when doing http request.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13332) Fix the usage of doAs/runAs in Visibility Controller tests.
[
https://issues.apache.org/jira/browse/HBASE-13332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383215#comment-14383215
]
Hadoop QA commented on HBASE-13332:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707643/HBASE-13332_v1.patch
against master branch at commit 18256fc9544866fc7182dfc5dd9d73b8cef02d66.
ATTACHMENT ID: 12707643
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 33 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+try (Table table = createTableAndWriteDataWithLabels(tableName, ( +
SECRET + | + CONFIDENTIAL
+try (Table table = createTableAndWriteDataWithLabels(tableName, ( +
SECRET + | + CONFIDENTIAL
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13454//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13454//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13454//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13454//console
This message is automatically generated.
Fix the usage of doAs/runAs in Visibility Controller tests.
---
Key: HBASE-13332
URL: https://issues.apache.org/jira/browse/HBASE-13332
Project: HBase
Issue Type: Sub-task
Reporter: Srikanth Srungarapu
Assignee: Srikanth Srungarapu
Fix For: 2.0.0, 1.0.1, 1.1.0
Attachments: HBASE-13332.patch, HBASE-13332.patch,
HBASE-13332_v1.patch
The doAs bug is also affecting VC related tests too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13262) ResultScanner doesn't return all rows in Scan
[
https://issues.apache.org/jira/browse/HBASE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383248#comment-14383248
]
Sean Busbey commented on HBASE-13262:
-
perfectly reasonable. I was only checking so I would know what to look for when
reviewing.
I updated the targets to include branch-1.0.
ResultScanner doesn't return all rows in Scan
-
Key: HBASE-13262
URL: https://issues.apache.org/jira/browse/HBASE-13262
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 2.0.0, 1.1.0
Environment: Single node, pseduo-distributed 1.1.0-SNAPSHOT
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Blocker
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13
Attachments: 13262-0.98-testpatch.txt, HBASE-13262-branch-1-v2.patch,
HBASE-13262-branch-1-v3.patch, HBASE-13262-branch-1.patch,
HBASE-13262-v1.patch, HBASE-13262-v2.patch, HBASE-13262-v3.patch,
HBASE-13262-v4.patch, HBASE-13262-v5.patch, HBASE-13262-v6.patch,
HBASE-13262-v7.patch, HBASE-13262-v7.patch, HBASE-13262.patch,
regionserver-logging.diff, testrun_0.98.txt, testrun_branch1.0.txt
Tried to write a simple Java client again 1.1.0-SNAPSHOT.
* Write 1M rows, each row with 1 family, and 10 qualifiers (values [0-9]),
for a total of 10M cells written
* Read back the data from the table, ensure I saw 10M cells
Running it against {{04ac1891}} (and earlier) yesterday, I would get ~20% of
the actual rows. Running against 1.0.0, returns all 10M records as expected.
[Code I was
running|https://github.com/joshelser/hbase-hwhat/blob/master/src/main/java/hbase/HBaseTest.java]
for the curious.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13295) TestInfoServers hang
[
https://issues.apache.org/jira/browse/HBASE-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383267#comment-14383267
]
zhangduo commented on HBASE-13295:
--
Let me pick this up. At least fix for master and branch-1.
TestInfoServers hang
Key: HBASE-13295
URL: https://issues.apache.org/jira/browse/HBASE-13295
Project: HBase
Issue Type: Bug
Components: test
Reporter: zhangduo
Assignee: zhangduo
Attachments: HBASE-13295.patch
https://builds.apache.org/job/HBase-TRUNK-jacoco/16/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.TestInfoServers-output.txt
No progress after this line
{noformat}
2015-03-19 22:46:06,809 INFO [main] hbase.TestInfoServers(127): Testing
http://localhost:44749/table.jsp?name=testMasterServerReadOnlyaction=splitkey=
has Table action request accepted
{noformat}
I think the problem maybe we do not wait for master finish becoming active,
and there is no timeout when doing http request.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13252) Get rid of managed connections and connection caching
[ https://issues.apache.org/jira/browse/HBASE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13252: Description: (Need to): - Remove CONNECTION_INSTANCES from ConnectionManager - Remove HConnectionKey - Remove 'managed' property from ClusterConnection, HCI and places where it's used now - AFAIS this property isn't visible to client (ClusterConnection is private interface), so technically this would not even be backward-incompatible change, and no release note needed? - In fact, ConnectionManager can be removed, too. was: (Need to): - Remove CONNECTION_INSTANCES from ConnectionManager - Remove HConnectionKey - Remove 'managed' property from ClusterConnection, HCI and places where it's used now - AFAIS this property isn't visible to client (ClusterConnection is private interface), so technically this would not even be backward-incompatible change, and no release note needed? Get rid of managed connections and connection caching - Key: HBASE-13252 URL: https://issues.apache.org/jira/browse/HBASE-13252 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Attachments: HBASE-13252-v0.patch, HBASE-13252-v1.patch, HBASE-13252-v1.patch, HBASE-13252-v2.patch (Need to): - Remove CONNECTION_INSTANCES from ConnectionManager - Remove HConnectionKey - Remove 'managed' property from ClusterConnection, HCI and places where it's used now - AFAIS this property isn't visible to client (ClusterConnection is private interface), so technically this would not even be backward-incompatible change, and no release note needed? - In fact, ConnectionManager can be removed, too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13291) Lift the scan ceiling
[
https://issues.apache.org/jira/browse/HBASE-13291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-13291:
--
Attachment: nonBBposAndInineMvccVint.txt
This is the hack to bypass bb plus inlining of the vint of mvcc. No longer
shows in perf top. Here is what it looks like now:
{code}
Samples: 4M of event 'cycles', Event count (approx.): 422533226118
21.09% perf-13042.map [.]
Lorg/apache/hadoop/hbase/io/hfile/HFileReaderV2$ScannerV2;._next in
Lorg/apache/hadoop/hbase/io/hfile/HFileReaderV2$ScannerV2;.next
12.76% perf-13042.map [.]
Lorg/apache/hadoop/hbase/regionserver/ScanQueryMatcher;.match
11.61% perf-13042.map [.] A04=^DF4^?
6.00% perf-13042.map [.]
Lorg/apache/hadoop/hbase/regionserver/StoreScanner;.optimize in
Lorg/apache/hadoop/hbase/regionserver/StoreScanner;.next
5.84% perf-13042.map [.]
Lorg/apache/hadoop/hbase/regionserver/KeyValueHeap;.next
4.75% perf-13042.map [.]
Ljava/util/concurrent/locks/AbstractQueuedSynchronizer;.release in
Lorg/apache/hadoop/hbase/regionserver/StoreScanner;.peek
3.87% perf-13042.map [.]
Ljava/util/PriorityQueue;.siftUpUsingComparator in
Lorg/apache/hadoop/hbase/regionserver/KeyValueHeap;.next
3.57% perf-13042.map [.]
Lorg/apache/hadoop/hbase/regionserver/KeyValueHeap;.peek in
Lorg/apache/hadoop/hbase/regionserver/KeyValueHeap;.next
3.09% perf-13042.map [.]
Lorg/apache/hadoop/hbase/util/Bytes$LexicographicalComparerHolder$UnsafeComparer;.compareTo
in Lorg/apache/hadoop/hbase/regionserver/ScanQueryMatcher;.match
2.00% perf-13042.map [.]
Lorg/apache/hadoop/hbase/regionserver/ScanDeleteTracker;.reset in
Lorg/apache/hadoop/hbase/regionserver/StoreScanner;.next
1.96% libjvm.so [.]
BlockOffsetArrayNonContigSpace::block_start_unsafe(void const*) const
1.38% perf-13042.map [.]
Lorg/apache/hadoop/hbase/regionserver/StoreFileScanner;.next in
Lorg/apache/hadoop/hbase/regionserver/KeyValueHeap;.next
1.03% libjvm.so [.]
ClearNoncleanCardWrapper::do_MemRegion(MemRegion)
0.95% perf-13042.map [.] Ljava/util/concurrent/ConcurrentHashMap;.get
in Lorg/apache/hadoop/hbase/io/hfile/LruBlockCache;.getBlock
0.93% perf-13042.map [.]
Ljava/util/concurrent/locks/LockSupport;.unpark in
Lorg/apache/hadoop/hbase/regionserver/StoreScanner;.next
0.91% perf-13042.map [.] Ljava/nio/HeapByteBuffer;.slice in
Lorg/apache/hadoop/hbase/io/hfile/HFileReaderV2$ScannerV2;.updateCurrBlock
0.77% libjvm.so [.] typeArrayKlass::allocate_common(int, bool,
Thread*)
0.72% libjvm.so [.] CompactibleFreeListSpace::block_size(HeapWord
const*) const
0.60% perf-13042.map [.]
Lorg/apache/hadoop/hbase/util/Bytes$LexicographicalComparerHolder$UnsafeComparer;.compareTo
in
Lorg/apache/hadoop/hbase/regionserver/HRegion$RegionScannerImpl;.nextInternal
0.55% perf-13042.map [.] D0)=^DF4^?
0.49% libjvm.so [.] SpinPause
0.45% perf-13042.map [.]
Lorg/apache/hadoop/hbase/util/Bytes$LexicographicalComparerHolder$UnsafeComparer;.compareTo
in
Lorg/apache/hadoop/hbase/regionserver/HRegion$RegionScannerImpl;.populateResult
{code}
At about 8.5k/s. Up but not much.
Lift the scan ceiling
-
Key: HBASE-13291
URL: https://issues.apache.org/jira/browse/HBASE-13291
Project: HBase
Issue Type: Improvement
Components: Scanners
Affects Versions: 1.0.0
Reporter: stack
Assignee: stack
Attachments: 13291.inlining.txt, Screen Shot 2015-03-26 at 12.12.13
PM.png, Screen Shot 2015-03-26 at 3.39.33 PM.png, hack_to_bypass_bb.txt,
nonBBposAndInineMvccVint.txt, q (1).png, traces.7.svg, traces.filterall.svg,
traces.nofilter.svg, traces.small2.svg, traces.smaller.svg
Scanning medium sized rows with multiple concurrent scanners exhibits
interesting 'ceiling' properties. A server runs at about 6.7k ops a second
using 450% of possible 1600% of CPUs when 4 clients each with 10 threads
doing scan 1000 rows. If I add '--filterAll' argument (do not return
results), then we run at 1450% of possible 1600% possible but we do 8k ops a
second.
Let me attach flame graphs for two cases. Unfortunately, there is some
frustrating dark art going on. Let me try figure it... Filing issue in
meantime to keep score in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13336) Consistent rules for security meta table protections
[ https://issues.apache.org/jira/browse/HBASE-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383359#comment-14383359 ] Srikanth Srungarapu commented on HBASE-13336: - Yeah, would be happy to work on this :). Consistent rules for security meta table protections Key: HBASE-13336 URL: https://issues.apache.org/jira/browse/HBASE-13336 Project: HBase Issue Type: Improvement Reporter: Andrew Purtell The AccessController and VisibilityController do different things regarding protecting their meta tables. The AC allows schema changes and disable/enable if the user has permission. The VC unconditionally disallows all admin actions. Generally, bad things will happen if these meta tables are damaged, disabled, or dropped. The likely outcome is random frequent (or constant) server side op failures with nasty stack traces. On the other hand some things like column family and table attribute changes can have valid use cases. We should have consistent and sensible rules for protecting security meta tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13291) Lift the scan ceiling
[ https://issues.apache.org/jira/browse/HBASE-13291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383366#comment-14383366 ] stack commented on HBASE-13291: --- bq. Yeah. Was thinking we use the index I build there to avoid the repeated getKeyValueLen calls. k. bq. I'm surprised that that StoreScanner.next() is only in there with 6% and StoreScanner.peek() not all. peek is way down... 1%. SS#next is yeah 6%/8%. bq. Is there anything different about P/E? Rows here are ten column and on average 180k so if I do my math right, we are going at about same speed (I'm slower by some). Maybe I should move to smaller cells because then changes will be more noticeable? Issue now is reading the mvcc off the end of the Cell. Its where I am spending time: 18.70% perf-16373.map [.] Lorg/apache/hadoop/hbase/io/hfile/HFileReaderV2$ScannerV2;.readMvccVersion in Lorg/apache/hadoop/hbase/io/hfile/HFileReaderV3$ScannerV3;.readKeyValueLen I've refactored it some. Will look more. And then SQM#match... and StoreScanner#optimize Lift the scan ceiling - Key: HBASE-13291 URL: https://issues.apache.org/jira/browse/HBASE-13291 Project: HBase Issue Type: Improvement Components: Scanners Affects Versions: 1.0.0 Reporter: stack Assignee: stack Attachments: 13291.inlining.txt, Screen Shot 2015-03-26 at 12.12.13 PM.png, Screen Shot 2015-03-26 at 3.39.33 PM.png, hack_to_bypass_bb.txt, nonBBposAndInineMvccVint.txt, q (1).png, traces.7.svg, traces.filterall.svg, traces.nofilter.svg, traces.small2.svg, traces.smaller.svg Scanning medium sized rows with multiple concurrent scanners exhibits interesting 'ceiling' properties. A server runs at about 6.7k ops a second using 450% of possible 1600% of CPUs when 4 clients each with 10 threads doing scan 1000 rows. If I add '--filterAll' argument (do not return results), then we run at 1450% of possible 1600% possible but we do 8k ops a second. Let me attach flame graphs for two cases. Unfortunately, there is some frustrating dark art going on. Let me try figure it... Filing issue in meantime to keep score in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13291) Lift the scan ceiling
[ https://issues.apache.org/jira/browse/HBASE-13291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383331#comment-14383331 ] Lars Hofhansl commented on HBASE-13291: --- bq. That work seems to address other issues than what I am seeing here. Yeah. Was thinking we use the index I build there to avoid the repeated getKeyValueLen calls. I'm surprised that that StoreScanner.next() is only in there with 6% and StoreScanner.peek() not all. By any chance is this still with the lock coarsening patch? (or you're testing with kinda larger column values too, so maybe that's why). Lift the scan ceiling - Key: HBASE-13291 URL: https://issues.apache.org/jira/browse/HBASE-13291 Project: HBase Issue Type: Improvement Components: Scanners Affects Versions: 1.0.0 Reporter: stack Assignee: stack Attachments: 13291.inlining.txt, Screen Shot 2015-03-26 at 12.12.13 PM.png, Screen Shot 2015-03-26 at 3.39.33 PM.png, hack_to_bypass_bb.txt, nonBBposAndInineMvccVint.txt, q (1).png, traces.7.svg, traces.filterall.svg, traces.nofilter.svg, traces.small2.svg, traces.smaller.svg Scanning medium sized rows with multiple concurrent scanners exhibits interesting 'ceiling' properties. A server runs at about 6.7k ops a second using 450% of possible 1600% of CPUs when 4 clients each with 10 threads doing scan 1000 rows. If I add '--filterAll' argument (do not return results), then we run at 1450% of possible 1600% possible but we do 8k ops a second. Let me attach flame graphs for two cases. Unfortunately, there is some frustrating dark art going on. Let me try figure it... Filing issue in meantime to keep score in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13337) Table regions are not assigning back, after restarting all regionservers at once.
[
https://issues.apache.org/jira/browse/HBASE-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383361#comment-14383361
]
Mikhail Antonov commented on HBASE-13337:
-
Just to make sure I understood correctly - so when all RS's are restarted
simultaneously, HMaster decides not to assign to prevent re-assignment storm? I
thought in such situation master will start assignment as soon as restarted RSs
start checking back in, and in next minutes (or may be tens of minutes) the
regions would be reassigned?
Table regions are not assigning back, after restarting all regionservers at
once.
-
Key: HBASE-13337
URL: https://issues.apache.org/jira/browse/HBASE-13337
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 2.0.0
Reporter: Y. SREENIVASULU REDDY
Assignee: Ashish Singhi
Priority: Blocker
Fix For: 2.0.0
Regions of the table are continouly in state=FAILED_CLOSE.
{noformat}
RegionState
RIT time (ms)
8f62e819b356736053e06240f7f7c6fd
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113929
caf59209ae65ea80fca6bdc6996a7d68
t1,,1427362431330.caf59209ae65ea80fca6bdc6996a7d68.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM2,16040,1427362533691 113929
db52a74988f71e5cf257bbabf31f26f3
t1,,1427362431330.db52a74988f71e5cf257bbabf31f26f3.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM3,16040,1427362533691 113920
43f3a65b9f9ff283f598c5450feab1f8
t1,,1427362431330.43f3a65b9f9ff283f598c5450feab1f8.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113920
{noformat}
*Steps to reproduce:*
1. Start HBase cluster with more than one regionserver.
2. Create a table with precreated regions. (lets say 15 regions)
3. Make sure the regions are well balanced.
4. Restart all the Regionservers process at once across the cluster, except
HMaster process
5. After restarting the Regionservers, successfully will connect to the
HMaster.
*Bug:*
But no regions are assigning back to the Regionservers.
*Master log shows as follows:*
{noformat}
2015-03-26 15:05:36,201 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=OFFLINE, ts=1427362536106, server=VM2,16040,1427362242602} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,202 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_OPENsn=VM1,16040,1427362531818
2015-03-26 15:05:36,244 DEBUG [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Force region state offline
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=PENDING_OPEN, ts=1427362536201, server=VM1,16040,1427362531818} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_CLOSE, ts=1427362536244,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_CLOSE
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=1 of 10
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=2 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=3 of 10
2015-03-26 15:05:36,249 INFO
[jira] [Commented] (HBASE-13291) Lift the scan ceiling
[ https://issues.apache.org/jira/browse/HBASE-13291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383376#comment-14383376 ] Lars Hofhansl commented on HBASE-13291: --- Hmm... StoreScanner.optimize is what I added in HBASE-13109. It only does anything when we are trying to seek in the scanner... FilterAllFilter never seeks. Something's fishy :) Lift the scan ceiling - Key: HBASE-13291 URL: https://issues.apache.org/jira/browse/HBASE-13291 Project: HBase Issue Type: Improvement Components: Scanners Affects Versions: 1.0.0 Reporter: stack Assignee: stack Attachments: 13291.inlining.txt, Screen Shot 2015-03-26 at 12.12.13 PM.png, Screen Shot 2015-03-26 at 3.39.33 PM.png, hack_to_bypass_bb.txt, nonBBposAndInineMvccVint.txt, q (1).png, traces.7.svg, traces.filterall.svg, traces.nofilter.svg, traces.small2.svg, traces.smaller.svg Scanning medium sized rows with multiple concurrent scanners exhibits interesting 'ceiling' properties. A server runs at about 6.7k ops a second using 450% of possible 1600% of CPUs when 4 clients each with 10 threads doing scan 1000 rows. If I add '--filterAll' argument (do not return results), then we run at 1450% of possible 1600% possible but we do 8k ops a second. Let me attach flame graphs for two cases. Unfortunately, there is some frustrating dark art going on. Let me try figure it... Filing issue in meantime to keep score in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13337) Table regions are not assigning back, after restarting all regionservers at once.
[
https://issues.apache.org/jira/browse/HBASE-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383379#comment-14383379
]
Y. SREENIVASULU REDDY commented on HBASE-13337:
---
correct [~ram_krish]
I verified the same scenario in HBase-0.98.11 version, there it is assigning
properly back to the newly restarted regionservers, and able to perform the
operations on those regions.
Table regions are not assigning back, after restarting all regionservers at
once.
-
Key: HBASE-13337
URL: https://issues.apache.org/jira/browse/HBASE-13337
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 2.0.0
Reporter: Y. SREENIVASULU REDDY
Priority: Blocker
Fix For: 2.0.0
Regions of the table are continouly in state=FAILED_CLOSE.
{noformat}
RegionState
RIT time (ms)
8f62e819b356736053e06240f7f7c6fd
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113929
caf59209ae65ea80fca6bdc6996a7d68
t1,,1427362431330.caf59209ae65ea80fca6bdc6996a7d68.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM2,16040,1427362533691 113929
db52a74988f71e5cf257bbabf31f26f3
t1,,1427362431330.db52a74988f71e5cf257bbabf31f26f3.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM3,16040,1427362533691 113920
43f3a65b9f9ff283f598c5450feab1f8
t1,,1427362431330.43f3a65b9f9ff283f598c5450feab1f8.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113920
{noformat}
*Steps to reproduce:*
1. Start HBase cluster with more than one regionserver.
2. Create a table with precreated regions. (lets say 15 regions)
3. Make sure the regions are well balanced.
4. Restart all the Regionservers process at once across the cluster, except
HMaster process
5. After restarting the Regionservers, successfully will connect to the
HMaster.
*Bug:*
But no regions are assigning back to the Regionservers.
*Master log shows as follows:*
{noformat}
2015-03-26 15:05:36,201 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=OFFLINE, ts=1427362536106, server=VM2,16040,1427362242602} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,202 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_OPENsn=VM1,16040,1427362531818
2015-03-26 15:05:36,244 DEBUG [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Force region state offline
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=PENDING_OPEN, ts=1427362536201, server=VM1,16040,1427362531818} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_CLOSE, ts=1427362536244,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_CLOSE
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=1 of 10
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=2 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=3 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
[jira] [Commented] (HBASE-13262) ResultScanner doesn't return all rows in Scan
[
https://issues.apache.org/jira/browse/HBASE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383241#comment-14383241
]
Josh Elser commented on HBASE-13262:
I was actually thinking that branch-1.0 would be omitted, but, on second look,
I'm not seeing anything that would actually violate the patch update rules. I
agree with everything Jonathan said WRT necessary patch alterations on 0.98 and
branch-1.0.
I'm happy to apply the patch for those branches as well, but I'm kind of
waiting on someone to tell/ask this of me rather than just do it on my own..
ResultScanner doesn't return all rows in Scan
-
Key: HBASE-13262
URL: https://issues.apache.org/jira/browse/HBASE-13262
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 2.0.0, 1.1.0
Environment: Single node, pseduo-distributed 1.1.0-SNAPSHOT
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 0.98.13
Attachments: 13262-0.98-testpatch.txt, HBASE-13262-branch-1-v2.patch,
HBASE-13262-branch-1-v3.patch, HBASE-13262-branch-1.patch,
HBASE-13262-v1.patch, HBASE-13262-v2.patch, HBASE-13262-v3.patch,
HBASE-13262-v4.patch, HBASE-13262-v5.patch, HBASE-13262-v6.patch,
HBASE-13262-v7.patch, HBASE-13262-v7.patch, HBASE-13262.patch,
regionserver-logging.diff, testrun_0.98.txt, testrun_branch1.0.txt
Tried to write a simple Java client again 1.1.0-SNAPSHOT.
* Write 1M rows, each row with 1 family, and 10 qualifiers (values [0-9]),
for a total of 10M cells written
* Read back the data from the table, ensure I saw 10M cells
Running it against {{04ac1891}} (and earlier) yesterday, I would get ~20% of
the actual rows. Running against 1.0.0, returns all 10M records as expected.
[Code I was
running|https://github.com/joshelser/hbase-hwhat/blob/master/src/main/java/hbase/HBaseTest.java]
for the curious.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12413) Mismatch in the equals and hashcode methods of KeyValue
[ https://issues.apache.org/jira/browse/HBASE-12413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383253#comment-14383253 ] Jingcheng Du commented on HBASE-12413: -- Thanks [~gabriel.reid]. In the patch V3, it tries to calculate the hashCode with mvcc whereas in equal it doesn't. Should we use hashCodeIgnoreMvcc instead of hashCode of CellComparator when calculating KeyValue hashcode? Mismatch in the equals and hashcode methods of KeyValue --- Key: HBASE-12413 URL: https://issues.apache.org/jira/browse/HBASE-12413 Project: HBase Issue Type: Bug Reporter: Jingcheng Du Assignee: Jingcheng Du Priority: Minor Attachments: HBASE-12413-V2.diff, HBASE-12413.diff, HBASE-12413.v3.patch In the equals method of KeyValue only row key is compared, and in the hashcode method all bacing bytes are calculated. This breaks the Java rule. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12413) Mismatch in the equals and hashcode methods of KeyValue
[ https://issues.apache.org/jira/browse/HBASE-12413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383254#comment-14383254 ] Jingcheng Du commented on HBASE-12413: -- Sorry [~yuzhih...@gmail.com], I did miss your reply, sorry for that! Mismatch in the equals and hashcode methods of KeyValue --- Key: HBASE-12413 URL: https://issues.apache.org/jira/browse/HBASE-12413 Project: HBase Issue Type: Bug Reporter: Jingcheng Du Assignee: Jingcheng Du Priority: Minor Attachments: HBASE-12413-V2.diff, HBASE-12413.diff, HBASE-12413.v3.patch In the equals method of KeyValue only row key is compared, and in the hashcode method all bacing bytes are calculated. This breaks the Java rule. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13327) Use Admin in ConnectionCache
[ https://issues.apache.org/jira/browse/HBASE-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383269#comment-14383269 ] Hudson commented on HBASE-13327: SUCCESS: Integrated in HBase-1.1 #327 (See [https://builds.apache.org/job/HBase-1.1/327/]) HBASE-13327 Addendum replaced the nested try with a cast of getAdmin() to HBaseAdmin and a call to the original compact/majorCompact method (Solomon) (tedyu: rev 9d6aa4a8df39cb44dec2d36c11f0a4d51c73febb) * hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java Use Admin in ConnectionCache Key: HBASE-13327 URL: https://issues.apache.org/jira/browse/HBASE-13327 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0, 1.0.1, 1.1.0 Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 1.0.1, 1.1.0 Attachments: HBASE-13327-addendum.patch, HBASE-13327.patch Replace HBaseAdmin usage with Admin in ConnectionCache and users of ConnectionCache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces
[ https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383338#comment-14383338 ] Rajeshbabu Chintaguntla commented on HBASE-12975: - [~apurtell] I am thinking more about split for APPROACH #3 at PHOENIX-1734. For index column family store files we need to create both top and bottom reference files irrespective of split key in the store file key range. To support this we need to add splitStoreFile to SplitTransaction interface and provide APIs to 1)create both top and bottom references irrespective of split key in the store file key range 2) create top and/or bottom references based on split key in the store file key range or not. What do you say? And also as stack mentioned here https://issues.apache.org/jira/browse/HBASE-12975?focusedCommentId=14367729page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14367729 Instead of using SplitTransactionImpl directly It's better to use split transaction implementation created from factory based on configuration. Supportable SplitTransaction and RegionMergeTransaction interfaces -- Key: HBASE-12975 URL: https://issues.apache.org/jira/browse/HBASE-12975 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Andrew Purtell Fix For: 2.0.0, 1.1.0 Attachments: HBASE-12975.patch, HBASE-12975.patch, HBASE-12975.patch Making SplitTransaction, RegionMergeTransaction limited private is required to support local indexing feature in Phoenix to ensure regions colocation. We can ensure region split, regions merge in the coprocessors in few method calls without touching internals like creating zk's, file layout changes or assignments. 1) stepsBeforePONR, stepsAfterPONR we can ensure split. 2) meta entries can pass through coprocessors to atomically update with the normal split/merge. 3) rollback on failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13337) Table regions are not assigning back, after restarting all regionservers at once.
[
https://issues.apache.org/jira/browse/HBASE-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383357#comment-14383357
]
ramkrishna.s.vasudevan commented on HBASE-13337:
I think the intent here is that previously when such thing happen when all RS
were stopped, and again brought up the HMaster was in a position to assign
regions and not making the cluster hang. Same was the case with a single RS
set up also. The HMaster was able to assign once the RS was restarted. Now it
is not so.
Am i right about the intention of this issue [~sreenivasulu]?
Table regions are not assigning back, after restarting all regionservers at
once.
-
Key: HBASE-13337
URL: https://issues.apache.org/jira/browse/HBASE-13337
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 2.0.0
Reporter: Y. SREENIVASULU REDDY
Assignee: Ashish Singhi
Priority: Blocker
Fix For: 2.0.0
Regions of the table are continouly in state=FAILED_CLOSE.
{noformat}
RegionState
RIT time (ms)
8f62e819b356736053e06240f7f7c6fd
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113929
caf59209ae65ea80fca6bdc6996a7d68
t1,,1427362431330.caf59209ae65ea80fca6bdc6996a7d68.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM2,16040,1427362533691 113929
db52a74988f71e5cf257bbabf31f26f3
t1,,1427362431330.db52a74988f71e5cf257bbabf31f26f3.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM3,16040,1427362533691 113920
43f3a65b9f9ff283f598c5450feab1f8
t1,,1427362431330.43f3a65b9f9ff283f598c5450feab1f8.
state=FAILED_CLOSE, ts=Thu Mar 26 15:05:36 IST 2015 (113s ago),
server=VM1,16040,1427362531818 113920
{noformat}
*Steps to reproduce:*
1. Start HBase cluster with more than one regionserver.
2. Create a table with precreated regions. (lets say 15 regions)
3. Make sure the regions are well balanced.
4. Restart all the Regionservers process at once across the cluster, except
HMaster process
5. After restarting the Regionservers, successfully will connect to the
HMaster.
*Bug:*
But no regions are assigning back to the Regionservers.
*Master log shows as follows:*
{noformat}
2015-03-26 15:05:36,201 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=OFFLINE, ts=1427362536106, server=VM2,16040,1427362242602} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,202 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_OPENsn=VM1,16040,1427362531818
2015-03-26 15:05:36,244 DEBUG [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Force region state offline
{8f62e819b356736053e06240f7f7c6fd state=PENDING_OPEN, ts=1427362536201,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStates: Transition {8f62e819b356736053e06240f7f7c6fd
state=PENDING_OPEN, ts=1427362536201, server=VM1,16040,1427362531818} to
{8f62e819b356736053e06240f7f7c6fd state=PENDING_CLOSE, ts=1427362536244,
server=VM1,16040,1427362531818}
2015-03-26 15:05:36,244 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.RegionStateStore: Updating row
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd. with
state=PENDING_CLOSE
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=1 of 10
2015-03-26 15:05:36,248 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=2 of 10
2015-03-26 15:05:36,249 INFO [VM2,16020,1427362216887-GeneralBulkAssigner-0]
master.AssignmentManager: Server VM1,16040,1427362531818 returned
java.nio.channels.ClosedChannelException for
t1,,1427362431330.8f62e819b356736053e06240f7f7c6fd., try=3 of 10
2015-03-26
[jira] [Commented] (HBASE-13291) Lift the scan ceiling
[ https://issues.apache.org/jira/browse/HBASE-13291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383356#comment-14383356 ] Lars Hofhansl commented on HBASE-13291: --- Just ran a test with my setup. All local on one box. I wrote 1m rows with 1 col of 1000 bytes each, i.e. ~1GB of data (verified the data size on HDFS). I can scan through that (in 0.98) 0.4s with filtering all, so that's over 2m rows/s. Is there anything different about P/E? Lift the scan ceiling - Key: HBASE-13291 URL: https://issues.apache.org/jira/browse/HBASE-13291 Project: HBase Issue Type: Improvement Components: Scanners Affects Versions: 1.0.0 Reporter: stack Assignee: stack Attachments: 13291.inlining.txt, Screen Shot 2015-03-26 at 12.12.13 PM.png, Screen Shot 2015-03-26 at 3.39.33 PM.png, hack_to_bypass_bb.txt, nonBBposAndInineMvccVint.txt, q (1).png, traces.7.svg, traces.filterall.svg, traces.nofilter.svg, traces.small2.svg, traces.smaller.svg Scanning medium sized rows with multiple concurrent scanners exhibits interesting 'ceiling' properties. A server runs at about 6.7k ops a second using 450% of possible 1600% of CPUs when 4 clients each with 10 threads doing scan 1000 rows. If I add '--filterAll' argument (do not return results), then we run at 1450% of possible 1600% possible but we do 8k ops a second. Let me attach flame graphs for two cases. Unfortunately, there is some frustrating dark art going on. Let me try figure it... Filing issue in meantime to keep score in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13336) Consistent rules for security meta table protections
[ https://issues.apache.org/jira/browse/HBASE-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382187#comment-14382187 ] Andrew Purtell commented on HBASE-13336: No plans yet, do you want it [~srikanth235]? Consistent rules for security meta table protections Key: HBASE-13336 URL: https://issues.apache.org/jira/browse/HBASE-13336 Project: HBase Issue Type: Improvement Reporter: Andrew Purtell The AccessController and VisibilityController do different things regarding protecting their meta tables. The AC allows schema changes and disable/enable if the user has permission. The VC unconditionally disallows all admin actions. Generally, bad things will happen if these meta tables are damaged, disabled, or dropped. The likely outcome is random frequent (or constant) server side op failures with nasty stack traces. On the other hand some things like column family and table attribute changes can have valid use cases. We should have consistent and sensible rules for protecting security meta tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13341) Add option to disable filtering on interface annotations for the API compatibility report
Andrew Purtell created HBASE-13341: -- Summary: Add option to disable filtering on interface annotations for the API compatibility report Key: HBASE-13341 URL: https://issues.apache.org/jira/browse/HBASE-13341 Project: HBase Issue Type: Improvement Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 2.0.0 The API compatibility checker script added in HBASE-12808 passes a file containing annotations to the JavaACC tool. When JavaACC is invoked with that option it will filter out all interfaces that do not have that annotation. We should add a command line option to the compatibility checker which turns off this filtering in case we want to look at the impact of changes to all interfaces, even private ones. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13329) Memstore flush fails if data has always the same value, breaking the region
[ https://issues.apache.org/jira/browse/HBASE-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382131#comment-14382131 ] Ruben Aguiar commented on HBASE-13329: -- So, after having the error every time I restarted the cluster (because it tried to replay the edits), I've manually build hbase with the change I proposed above (changing diffIdx from short to int). After restarting the region server, the region was sucessfully flushed. So the fix does solve the problem. I'll keep benchmarking my cluster and see if this error continues to happen. Memstore flush fails if data has always the same value, breaking the region --- Key: HBASE-13329 URL: https://issues.apache.org/jira/browse/HBASE-13329 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 1.0.1 Environment: linux-debian-jessie ec2 - t2.micro instances Reporter: Ruben Aguiar Priority: Critical Fix For: 2.0.0, 1.1.0 While trying to benchmark my opentsdb cluster, I've created a script that sends to hbase always the same value (in this case 1). After a few minutes, the whole region server crashes and the region itself becomes impossible to open again (cannot assign or unassign). After some investigation, what I saw on the logs is that when a Memstore flush is called on a large region (128mb) the process errors, killing the regionserver. On restart, replaying the edits generates the same error, making the region unavailable. Tried to manually unassign, assign or close_region. That didn't work because the code that reads/replays it crashes. From my investigation this seems to be an overflow issue. The logs show that the function getMinimumMidpointArray tried to access index -32743 of an array, extremely close to the minimum short value in Java. Upon investigation of the source code, it seems an index short is used, being incremented as long as the two vectors are the same, probably making it overflow on large vectors with equal data. Changing it to int should solve the problem. Here follows the hadoop logs of when the regionserver went down. Any help is appreciated. Any other information you need please do tell me: 2015-03-24 18:00:56,187 INFO [regionserver//10.2.0.73:16020.logRoller] wal.FSHLog: Rolled WAL /hbase/WALs/10.2.0.73,16020,1427216382590/10.2.0.73%2C16020%2C1427216382590.default.1427220018516 with entries=143, filesize=134.70 MB; new WAL /hbase/WALs/10.2.0.73,16020,1427216382590/10.2.0.73%2C16020%2C1427216382590.default.1427220056140 2015-03-24 18:00:56,188 INFO [regionserver//10.2.0.73:16020.logRoller] wal.FSHLog: Archiving hdfs://10.2.0.74:8020/hbase/WALs/10.2.0.73,16020,1427216382590/10.2.0.73%2C16020%2C1427216382590.default.1427219987709 to hdfs://10.2.0.74:8020/hbase/oldWALs/10.2.0.73%2C16020%2C1427216382590.default.1427219987709 2015-03-24 18:04:35,722 INFO [MemStoreFlusher.0] regionserver.HRegion: Started memstore flush for tsdb,,1427133969325.52bc1994da0fea97563a4a656a58bec2., current region memstore size 128.04 MB 2015-03-24 18:04:36,154 FATAL [MemStoreFlusher.0] regionserver.HRegionServer: ABORTING region server 10.2.0.73,16020,1427216382590: Replay of WAL required. Forcing server shutdown org.apache.hadoop.hbase.DroppedSnapshotException: region: tsdb,,1427133969325.52bc1994da0fea97563a4a656a58bec2. at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1999) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1770) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1702) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:445) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:407) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$800(MemStoreFlusher.java:69) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:225) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArrayIndexOutOfBoundsException: -32743 at org.apache.hadoop.hbase.CellComparator.getMinimumMidpointArray(CellComparator.java:478) at org.apache.hadoop.hbase.CellComparator.getMidpoint(CellComparator.java:448) at org.apache.hadoop.hbase.io.hfile.HFileWriterV2.finishBlock(HFileWriterV2.java:165) at org.apache.hadoop.hbase.io.hfile.HFileWriterV2.checkBlockBoundary(HFileWriterV2.java:146) at org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:263) at org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87) at
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382174#comment-14382174 ] Nick Dimiduk commented on HBASE-13339: -- In line with [~busbey]'s concerns, I'd like to see this an a chance to exercise [~dimaspivak]'s new check_compatibility.sh script. Dima, can you give us some guidance on how to use the script to check transitive dependencies of this upgrade? Ideally we can get this as simple as a +1/-1/-0 vote from the script and incorporate it into jenkins. I'd like to review all deps on branch-1 as well, maybe hadoop version to 2.5.2 -- but only if I have confidence in not breaking folks. Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13338) Hbase to use PowerPC supported Jruby version 1.7.20
[ https://issues.apache.org/jira/browse/HBASE-13338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382194#comment-14382194 ] Jonathan Hsieh commented on HBASE-13338: +1 to what Andrew says. Hbase to use PowerPC supported Jruby version 1.7.20 --- Key: HBASE-13338 URL: https://issues.apache.org/jira/browse/HBASE-13338 Project: HBase Issue Type: Task Components: dependencies, shell Affects Versions: 1.0.0, 0.98.12 Environment: PowerPC64, PowerPC64LE Reporter: Ayappan Older versions of jffi (till 1.2.7) don't have native PPC64 PPC64LE libraries. The latest released 1.2.8 version has PowerPC libraries and the jruby development version(1.7.20--SNAPSHOT) has been updated to make use of this version. Hbase still uses much older jruby 1.6.8 version which don't have the native libraries and this affects the Hbase shell in PowerPCs. So Hbase needs to be updated to make use of the upcoming Jruby release 1.7.20 to support PowerPC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13262) ResultScanner doesn't return all rows in Scan
[
https://issues.apache.org/jira/browse/HBASE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382251#comment-14382251
]
Lars Hofhansl commented on HBASE-13262:
---
Looks good to me.
{{+ if (null != values values.length 0
callable.getServerHasMoreResultsContext()) {}}
I'd probably name this just {{callable.hasMoreResultsContext}}. So that will
work right with an old client, right? It'll just not read the context and fall
back to the old logic. That's what the code does, just making sure it's
something we tested.
ResultScanner doesn't return all rows in Scan
-
Key: HBASE-13262
URL: https://issues.apache.org/jira/browse/HBASE-13262
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 2.0.0, 1.1.0
Environment: Single node, pseduo-distributed 1.1.0-SNAPSHOT
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 0.98.13
Attachments: 13262-0.98-testpatch.txt, HBASE-13262-branch-1-v2.patch,
HBASE-13262-branch-1-v3.patch, HBASE-13262-branch-1.patch,
HBASE-13262-v1.patch, HBASE-13262-v2.patch, HBASE-13262-v3.patch,
HBASE-13262-v4.patch, HBASE-13262-v5.patch, HBASE-13262-v6.patch,
HBASE-13262.patch, regionserver-logging.diff, testrun_0.98.txt,
testrun_branch1.0.txt
Tried to write a simple Java client again 1.1.0-SNAPSHOT.
* Write 1M rows, each row with 1 family, and 10 qualifiers (values [0-9]),
for a total of 10M cells written
* Read back the data from the table, ensure I saw 10M cells
Running it against {{04ac1891}} (and earlier) yesterday, I would get ~20% of
the actual rows. Running against 1.0.0, returns all 10M records as expected.
[Code I was
running|https://github.com/joshelser/hbase-hwhat/blob/master/src/main/java/hbase/HBaseTest.java]
for the curious.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382305#comment-14382305 ] Elliott Clark commented on HBASE-13339: --- You are correct on that. I was just going by the resolved state. Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[
https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382332#comment-14382332
]
Sean Busbey commented on HBASE-13339:
-
{quote}
I still think the point stands. If we are shipping a feature that requires a
version of hadoop then we're pulling that version in; be it by doing it
ourselves, or by asking users to do it. So if we have two different versions of
hadoop that we're bound to ( one that we shipped in 1.0.0 and one that features
depend on) then I would rather ship the better newer version by default.
{quote}
It's the difference between letting downstream users decide if the feature is
worth the pain of upgrading or making part of the decision for them. Granted,
folks are supposed to replace the Hadoop related client jars from our lib
directory to be those from their Hadoop distribution. However, giving clear
instructions on replacing Hadoop's transitive dependencies is difficult (and I
don't think we even try right now).
Additionally, it's dangerous to use a backwards-compatible (but not forward
compatible) newer version when we claim support for the older version. It's
doable given test infra, but right now we have a gap on that. Pre-commit QA
keeps us honest on compile and the part of the runtime covered by unit tests,
but we have a clear gap on coverage.
Update default Hadoop version to 2.6.0
--
Key: HBASE-13339
URL: https://issues.apache.org/jira/browse/HBASE-13339
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.1.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Attachments: HBASE-13339.patch
Current default Hadoop version is getting a little long in the tooth. We
should update to the latest version. The latest version is backwards
compatible with 2.5.1's dfs and mr so this should be painless.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13340) Include LimitedPrivate interfaces in the API compatibility report
Andrew Purtell created HBASE-13340: -- Summary: Include LimitedPrivate interfaces in the API compatibility report Key: HBASE-13340 URL: https://issues.apache.org/jira/browse/HBASE-13340 Project: HBase Issue Type: Improvement Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 2.0.0 The API compatibility checker script added in HBASE-12808 passes a file containing annotations to the JavaACC tool. When JavaACC is invoked with that option it will filter out all interfaces that do not have that annotation. Currently only Public interfaces are checked. We should add LimitedPrivate to the annotation list, otherwise we will miss changes that impact coprocessors and other users of those interfaces. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382243#comment-14382243 ] Nick Dimiduk commented on HBASE-13339: -- bq. I don't think our HBase API compat tool should check the APIs of transitive dependencies. Considering those part of _our_ API would be madness. Yes, but see also the row around Jackson on HBASE-13149 -- our transitive dependencies are effectively our API according to the semver compat matrix. Having the tools to detect these changes earlier will make all of our lives easier. bq. The JavaACC tool under the covers can be made to check compat between any two versions of any set of jars, but you'll have to do it by hand. Yes, this is exactly what I'm hoping we can exploit. Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13340) Include LimitedPrivate interfaces in the API compatibility report
[ https://issues.apache.org/jira/browse/HBASE-13340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382246#comment-14382246 ] Nick Dimiduk commented on HBASE-13340: -- Yes please! Include LimitedPrivate interfaces in the API compatibility report - Key: HBASE-13340 URL: https://issues.apache.org/jira/browse/HBASE-13340 Project: HBase Issue Type: Improvement Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 2.0.0 The API compatibility checker script added in HBASE-12808 passes a file containing annotations to the JavaACC tool. When JavaACC is invoked with that option it will filter out all interfaces that do not have that annotation. Currently only Public interfaces are checked. We should add LimitedPrivate to the annotation list, otherwise we will miss changes that impact coprocessors and other users of those interfaces. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[
https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382290#comment-14382290
]
Sean Busbey commented on HBASE-13339:
-
{quote}
Lars Hofhansl Normally I'd agree with you however we're shipping a feature in
1.1 that requires 2.6.
{quote}
We're not supposed to require downstream folks to upgrade Hadoop as a part of
upgrading. Currently, I believe we're supposed to be making sure any HBase 1.y
release works with Hadoop 2.4 and 2.5 (with 2.6 presumably getting added in for
1.1).
What feature breaks if someone is not on Hadoop 2.6?
Update default Hadoop version to 2.6.0
--
Key: HBASE-13339
URL: https://issues.apache.org/jira/browse/HBASE-13339
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.1.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Attachments: HBASE-13339.patch
Current default Hadoop version is getting a little long in the tooth. We
should update to the latest version. The latest version is backwards
compatible with 2.5.1's dfs and mr so this should be painless.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382161#comment-14382161 ] Elliott Clark commented on HBASE-13339: --- Hadoop 2.6.0 is required for several of the features currently in master and branch-1. So by not using it by default we're really just making the users lives harder (along with not testing the features). They have to re-compile add the dependencies themselves to get the features that we've promised them. Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13170) Allow block cache to be external
[ https://issues.apache.org/jira/browse/HBASE-13170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-13170: -- Attachment: HBASE-13170-v3.patch Allow block cache to be external Key: HBASE-13170 URL: https://issues.apache.org/jira/browse/HBASE-13170 Project: HBase Issue Type: New Feature Components: io Affects Versions: 2.0.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 1.1.0 Attachments: HBASE-13170-v1.patch, HBASE-13170-v2.patch, HBASE-13170-v3.patch, HBASE-13170.patch Allow an external service to provide the block cache. This has the nice property of allowing failover/upgrades to happen without causing a fully cold cache. Additionally this allows read replicas to share some of the same memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13342) Fix incorrect interface annotations
Elliott Clark created HBASE-13342: - Summary: Fix incorrect interface annotations Key: HBASE-13342 URL: https://issues.apache.org/jira/browse/HBASE-13342 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Still some old annotations. Have slipped in. Lets remove them and add in a patch check for them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12848) Utilize Flash storage for WAL
[ https://issues.apache.org/jira/browse/HBASE-12848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-12848: Release Note: For users on a version of Hadoop that supports tiered storage policies (i.e. Apache Hadoop 2.6.0+), HBase now allows users to opt-in to having the write ahead log placed on the SSD tier. Users on earlier versions of Hadoop will be unable to take advantage of this feature. Use of tiered storage is controlled by a new RegionServer config, hbase.wal.storage.policy. It defaults to the value 'NONE', which will rely on HDFS defaults for a policy decision. User can specify ONE_SSD or ALL_SSD as the value: ONE_SSD: place only one replica of WAL files in SSD and the remaining in default storage ALL_SSD: all replica for WAL files are placed on SSD See [the HDFS docs on storage policy|http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/ArchivalStorage.html] was: New config, hbase.wal.storage.policy, is added with default value 'NONE' User can specify ONE_SSD or ALL_SSD as the value: ONE_SSD: place only one replica of WAL files in SSD and the remaining in default storage ALL_SSD: all replica for WAL files are placed on SSD See [the HDFS docs on storage policy|http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/ArchivalStorage.html] Utilize Flash storage for WAL - Key: HBASE-12848 URL: https://issues.apache.org/jira/browse/HBASE-12848 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Assignee: Ted Yu Fix For: 2.0.0, 1.1.0 Attachments: 12848-v1.patch, 12848-v2.patch, 12848-v3.patch, 12848-v4.patch, 12848-v4.patch One way to improve data ingestion rate is to make use of Flash storage. HDFS is doing the heavy lifting - see HDFS-7228. We assume an environment where: 1. Some servers have a mix of flash, e.g. 2 flash drives and 4 traditional drives. 2. Some servers have all traditional storage. 3. RegionServers are deployed on both profiles within one HBase cluster. This JIRA allows WAL to be managed on flash in a mixed-profile environment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-13339: -- Attachment: HBASE-13339.patch Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13338) Hbase to use PowerPC supported Jruby version 1.7.20
[ https://issues.apache.org/jira/browse/HBASE-13338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382233#comment-14382233 ] Sean Busbey commented on HBASE-13338: - Upgrading this won't be just a simple pom change. JRuby 1.7 changes the default Ruby version from 1.8 to 1.9, which is *very* incompatible. You should be able to tell JRuby to still run in Ruby 1.8 mode ([ref|https://github.com/jruby/jruby/wiki/FAQs#does-jruby-support-ruby-19x-features]). Please make sure to do this. I really want to update or Ruby version because Ruby 1.8 is super old, has be EOL by several Ruby communities, and finally starting to die off. I've been waiting for the release of JRuby 9000 to update us off of Ruby 1.8 (release 9.0.0.0 pre-1 just finally landed in january). It updates things to be Ruby 2.2 (and only 2.2 IIRC). Hbase to use PowerPC supported Jruby version 1.7.20 --- Key: HBASE-13338 URL: https://issues.apache.org/jira/browse/HBASE-13338 Project: HBase Issue Type: Task Components: dependencies, shell Affects Versions: 1.0.0, 0.98.12 Environment: PowerPC64, PowerPC64LE Reporter: Ayappan Older versions of jffi (till 1.2.7) don't have native PPC64 PPC64LE libraries. The latest released 1.2.8 version has PowerPC libraries and the jruby development version(1.7.20--SNAPSHOT) has been updated to make use of this version. Hbase still uses much older jruby 1.6.8 version which don't have the native libraries and this affects the Hbase shell in PowerPCs. So Hbase needs to be updated to make use of the upcoming Jruby release 1.7.20 to support PowerPC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382280#comment-14382280 ] Sean Busbey commented on HBASE-13339: - To be clear, I'm not suggesting any policy change. I just want to be cognizant of what impact our update will have, since this is targeting 1.1 and I want that to be a low-risk upgrade. I'd also like a plan for how we're going to solve the general problem (presumably in the 2.0+ time frame). I don't know how strongly I feel about wanting that plan, because I don't know what we miss out on by not upgrading. Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382279#comment-14382279 ] Elliott Clark commented on HBASE-13339: --- [~lhofhansl] Normally I'd agree with you however we're shipping a feature in 1.1 that requires 2.6. Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382300#comment-14382300 ] Sean Busbey commented on HBASE-13339: - I'll go update the docs for tiered storage for WAL to make sure that's called out. And verify that the feature fails gracefully on pre 2.6. I thought tiered storage for flushing didn't happen? Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13001) NullPointer in master logs for table.jsp
[
https://issues.apache.org/jira/browse/HBASE-13001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-13001:
---
Fix Version/s: 1.0.1
NullPointer in master logs for table.jsp
Key: HBASE-13001
URL: https://issues.apache.org/jira/browse/HBASE-13001
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.10
Reporter: Vikas Vishwakarma
Assignee: Vikas Vishwakarma
Priority: Trivial
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11
Attachments: HBASE-13001-0.98.patch, HBASE-13001-branch-1.patch,
HBASE-13001.patch, Table_not_ready.png
Seeing a NullPointer issue in master logs for table.jsp probably similar to
HBASE-6607
{noformat}
2015-02-09 14:04:00,622 ERROR org.mortbay.log: /table.jsp
java.lang.NullPointerException
at
org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:71)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at
org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1087)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382322#comment-14382322 ] Elliott Clark commented on HBASE-13339: --- I still think the point stands. If we are shipping a feature that requires a version of hadoop then we're pulling that version in; be it by doing it ourselves, or by asking users to do it. So if we have two different versions of hadoop that we're bound to ( one that we shipped in 1.0.0 and one that features depend on) then I would rather ship the better newer version by default. Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382323#comment-14382323 ] Sean Busbey commented on HBASE-13339: - I have a solution for HBASE-13221 that is at 80% that doesn't require any Hadoop update. I need to get some pref numbers to make sure it's fine. If it turns out my intuition about the JIT is wrong, then the solution is going to require Hadoop 2.7.0 if HDFS encryption is on. But docs + a pointer for mitigation will just mean pre-2.7.0 Hadoops work only without HDFS encryption (and those users can use HBase's encryption instead). Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382172#comment-14382172 ] Sean Busbey commented on HBASE-13339: - shoot. are we tracking which features require Hadoop 2.6+? HBASE-12848 comes to mind, but that's a minor bit I don't see a straightforward way to find the rest. Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382214#comment-14382214 ] Andrew Purtell commented on HBASE-13339: I was also going to suggest we do Hadoop 2.5 instead, I see folks above have already done so. bq. guidance on how to use the script to check transitive dependencies of this upgrade It doesn't check transitive dependencies. It only checks HBase jars, and unless you comment out part of the script (I'm thinking of adding a new command line option) it will only check Public annotated interfaces. I don't think our HBase API compat tool should check the APIs of transitive dependencies. Considering those part of _our_ API would be madness. The JavaACC tool under the covers can be made to check compat between any two versions of any set of jars, but you'll have to do it by hand. Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13262) ResultScanner doesn't return all rows in Scan
[
https://issues.apache.org/jira/browse/HBASE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382232#comment-14382232
]
Jonathan Lawlor commented on HBASE-13262:
-
+1, lgtm
Anyone else have some time to review this nice patch?
ResultScanner doesn't return all rows in Scan
-
Key: HBASE-13262
URL: https://issues.apache.org/jira/browse/HBASE-13262
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 2.0.0, 1.1.0
Environment: Single node, pseduo-distributed 1.1.0-SNAPSHOT
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 0.98.13
Attachments: 13262-0.98-testpatch.txt, HBASE-13262-branch-1-v2.patch,
HBASE-13262-branch-1-v3.patch, HBASE-13262-branch-1.patch,
HBASE-13262-v1.patch, HBASE-13262-v2.patch, HBASE-13262-v3.patch,
HBASE-13262-v4.patch, HBASE-13262-v5.patch, HBASE-13262-v6.patch,
HBASE-13262.patch, regionserver-logging.diff, testrun_0.98.txt,
testrun_branch1.0.txt
Tried to write a simple Java client again 1.1.0-SNAPSHOT.
* Write 1M rows, each row with 1 family, and 10 qualifiers (values [0-9]),
for a total of 10M cells written
* Read back the data from the table, ensure I saw 10M cells
Running it against {{04ac1891}} (and earlier) yesterday, I would get ~20% of
the actual rows. Running against 1.0.0, returns all 10M records as expected.
[Code I was
running|https://github.com/joshelser/hbase-hwhat/blob/master/src/main/java/hbase/HBaseTest.java]
for the curious.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382256#comment-14382256 ] Andrew Purtell commented on HBASE-13339: bq. Yes, but see also the row around Jackson on HBASE-13149 -- our transitive dependencies are effectively our API according to the semver compat matrix. That was a point of discussion but I don't believe that was any kind of policy outcome. Anyway, I am -1 on making any statement that transitive dependencies are effectively our API. They are not our API. Our API is our API. Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13320) 'hbase.bucketcache.size' configuration value is not correct in hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Y. SREENIVASULU REDDY updated HBASE-13320:
--
Attachment: HBASE-v2-13320.patch
In previous patch
xml parser exception is throwing because of
{noformat}
cache (if 1.0) OR, it is the capacity
{noformat}
so changing it to
{noformat}
cache (if lessthan 1.0) OR, it is the capacity
{noformat}
'hbase.bucketcache.size' configuration value is not correct in
hbase-default.xml
-
Key: HBASE-13320
URL: https://issues.apache.org/jira/browse/HBASE-13320
Project: HBase
Issue Type: Bug
Components: hbase
Affects Versions: 2.0.0
Reporter: Y. SREENIVASULU REDDY
Assignee: Y. SREENIVASULU REDDY
Fix For: 2.0.0
Attachments: HBASE-13320.patch, HBASE-v2-13320.patch
In hbase-default.xml file
* 'hbase.bucketcache.size' is not correct
We either specify it as a float or in MB's and the default value that is
mentioned is never used
{code}
property
namehbase.bucketcache.size/name
value65536/value
sourcehbase-default.xml/source
/property
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382272#comment-14382272 ] Lars Hofhansl commented on HBASE-13339: --- Definitely for 2.0. For 1.1 this needs a bit more discussion I think. Per our sem var we should not force a version of Hadoop forward in a minor HBase upgrade. Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382297#comment-14382297 ] Elliott Clark commented on HBASE-13339: --- The tiered storage for wal and flushing doesn't work if 2.6.0 isn't there. Whatever we find for HBASE-13221 will also likely require 2.6.0 Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-13339: -- Status: Patch Available (was: Open) Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13338) Hbase to use PowerPC supported Jruby version 1.7.20
[ https://issues.apache.org/jira/browse/HBASE-13338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382192#comment-14382192 ] stack commented on HBASE-13338: --- In past we have been reluctant to upgrade our jruby. See HBASE-7028. JRuby changed their licensing graciously so we could bundle it in an apache project. Would be worth checking it still compatible too before bringing it in. Hbase to use PowerPC supported Jruby version 1.7.20 --- Key: HBASE-13338 URL: https://issues.apache.org/jira/browse/HBASE-13338 Project: HBase Issue Type: Task Components: dependencies, shell Affects Versions: 1.0.0, 0.98.12 Environment: PowerPC64, PowerPC64LE Reporter: Ayappan Older versions of jffi (till 1.2.7) don't have native PPC64 PPC64LE libraries. The latest released 1.2.8 version has PowerPC libraries and the jruby development version(1.7.20--SNAPSHOT) has been updated to make use of this version. Hbase still uses much older jruby 1.6.8 version which don't have the native libraries and this affects the Hbase shell in PowerPCs. So Hbase needs to be updated to make use of the upcoming Jruby release 1.7.20 to support PowerPC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13339) Update default Hadoop version to 2.6.0
[ https://issues.apache.org/jira/browse/HBASE-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382268#comment-14382268 ] Elliott Clark commented on HBASE-13339: --- I really don't see how forcing users to do this is any better than doing it ourselves. 2.6.0 is better in just about every way. Hence I can't see recommending 2.5 to anyone ever. Yes I'd love to live in a world where Hadoop and it's dependencies are hidden from our users. However I don't think that stagnating our version of hadoop and making some features ( tiered storage, and AES over the wire ) un-usable is a solution. It's just playing ostrich and ignoring that things are moving forward. I just went through and from eyeballing all that changes is that some dependencies are added. I don't see the large harm on adding some transitive dependencies ( that ship has already sailed ). Update default Hadoop version to 2.6.0 -- Key: HBASE-13339 URL: https://issues.apache.org/jira/browse/HBASE-13339 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13339.patch Current default Hadoop version is getting a little long in the tooth. We should update to the latest version. The latest version is backwards compatible with 2.5.1's dfs and mr so this should be painless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13001) NullPointer in master logs for table.jsp
[
https://issues.apache.org/jira/browse/HBASE-13001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382315#comment-14382315
]
Andrew Purtell commented on HBASE-13001:
I'll commit to branch-1.0 now. Thanks [~ndimiduk]
NullPointer in master logs for table.jsp
Key: HBASE-13001
URL: https://issues.apache.org/jira/browse/HBASE-13001
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.10
Reporter: Vikas Vishwakarma
Assignee: Vikas Vishwakarma
Priority: Trivial
Fix For: 2.0.0, 1.1.0, 0.98.11
Attachments: HBASE-13001-0.98.patch, HBASE-13001-branch-1.patch,
HBASE-13001.patch, Table_not_ready.png
Seeing a NullPointer issue in master logs for table.jsp probably similar to
HBASE-6607
{noformat}
2015-02-09 14:04:00,622 ERROR org.mortbay.log: /table.jsp
java.lang.NullPointerException
at
org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:71)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at
org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1087)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-8725) Add total time RPC call metrics
[ https://issues.apache.org/jira/browse/HBASE-8725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382544#comment-14382544 ] Hudson commented on HBASE-8725: --- SUCCESS: Integrated in HBase-1.1 #326 (See [https://builds.apache.org/job/HBase-1.1/326/]) HBASE-8725 Add total time RPC call metrics (eclark: rev 797eee33ab2367805b99984aceea15258a95f766) * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSourceImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestRpcMetrics.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSource.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServer.java Add total time RPC call metrics --- Key: HBASE-8725 URL: https://issues.apache.org/jira/browse/HBASE-8725 Project: HBase Issue Type: Bug Components: metrics Affects Versions: 0.98.9 Reporter: Elliott Clark Assignee: Nate Edel Fix For: 2.0.0, 1.1.0 Attachments: HBASE-8275.patch, HBASE-8725-r1.patch Right now we have: * queueCallTime ** exposes the ammount of time that a call is in the queue * processCallTime ** exposes how long a call was processing We don't have a stat for total time. It's pretty easy to think that the two metrics might not track together so getting a total time could be useful. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13303) Fix size calculation of results on the region server
[
https://issues.apache.org/jira/browse/HBASE-13303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382554#comment-14382554
]
Jonathan Lawlor commented on HBASE-13303:
-
[~apurtell] Apologies for the late catch, but while working on the addendum to
HBASE-11544 I just came across a calculation that was missed with this patch in
branch-1+. Specifically in {{StoreScanner#next(List, int, long)}} the
calculation used for totalHeapSize still uses to
{{CellUtil#estimatedHeapSizeOf}}. Same goes for
{{RegionScannerImpl#calculateResultSize}}.
These places should use the new estimate without tags as well because size
calculations are performed once down in StoreScanner and the calculation
propagates up to RSRpcServices through the NextState (thus the change in
RSRpcServices alone won't be sufficient since {{skipResultSizeCalculation}}
would be true). Would this be fixed as an addendum here or should I file
elsewhere? Or, given that the point of the addendum to HBASE-11544 is to get
rid of NextState, would it be more appropriate to fix there?
Fix size calculation of results on the region server
Key: HBASE-13303
URL: https://issues.apache.org/jira/browse/HBASE-13303
Project: HBase
Issue Type: Sub-task
Components: Client
Reporter: Lars Hofhansl
Assignee: Andrew Purtell
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Attachments: HBASE-13303-0.98.patch, HBASE-13303-0.98.patch,
HBASE-13303-0.98.patch, HBASE-13303.patch, HBASE-13303.patch,
HBASE-13303.patch, TestResultSizeEstimation.java
One of the problems in the parent is due to different size calculation
between client and server when HFilev3 is used.
Since tags are _never_ shipped to the client in a scan, we can have special
size function (or a flag on the current one) that does not include the tags
and the tags meta information (the length is what causes the issue), so that
client and server will always calculate the same size.
I'll make a patch within the hour, unless somebody beats me to it.
[~apurtell], FYI.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13342) Fix incorrect interface annotations
[ https://issues.apache.org/jira/browse/HBASE-13342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382563#comment-14382563 ] Enis Soztutar commented on HBASE-13342: --- +1. Fix incorrect interface annotations --- Key: HBASE-13342 URL: https://issues.apache.org/jira/browse/HBASE-13342 Project: HBase Issue Type: Improvement Affects Versions: 2.0.0, 1.1.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-13342.patch Still some old annotations. Have slipped in. Lets remove them and add in a patch check for them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
