[jira] Resolved: (HBASE-2400) new connector for Avro RPC access to HBase cluster
[ https://issues.apache.org/jira/browse/HBASE-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ryan rawson resolved HBASE-2400. Hadoop Flags: [Reviewed] Fix Version/s: 0.21.0 Resolution: Fixed I just committed v3 of the patch from reviewboard. new connector for Avro RPC access to HBase cluster -- Key: HBASE-2400 URL: https://issues.apache.org/jira/browse/HBASE-2400 Project: HBase Issue Type: Task Components: avro Reporter: Andrew Purtell Priority: Minor Fix For: 0.21.0 Attachments: HBASE-2400-v0.patch Build a new connector contrib architecturally equivalent to the Thrift connector, but using Avro serialization and associated transport and RPC server work. Support AAA (audit, authentication, authorization). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HBASE-2718) Update .gitignore for trunk after removal of contribs
Update .gitignore for trunk after removal of contribs - Key: HBASE-2718 URL: https://issues.apache.org/jira/browse/HBASE-2718 Project: HBase Issue Type: Improvement Affects Versions: 0.21.0 Reporter: Lars Francke Priority: Trivial This just updates the .gitignore file: Removes all references to core/ and contrib/ Also removes /build which should be a relic from the Ant/Ivy times Adds /.idea/ for the IntelliJ users -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-2718) Update .gitignore for trunk after removal of contribs
[
https://issues.apache.org/jira/browse/HBASE-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-2718.
--
Hadoop Flags: [Reviewed]
Fix Version/s: 0.21.0
Resolution: Fixed
Applied. Thanks for that patch Lars F. I had to apply it manually because
complained
{code}
pynchon-8:trunk stack$ patch -p0 HBASE-2718.patch
patch unexpectedly ends in middle of line
patch: Only garbage was found in the patch input.
{code}
I also left 'build' in there. The minidfscluster and minimapreduce cluster
have hardcoded build/test as location to keep test data so you'll see build
created as tests run (I need to file an issue)
Update .gitignore for trunk after removal of contribs
-
Key: HBASE-2718
URL: https://issues.apache.org/jira/browse/HBASE-2718
Project: HBase
Issue Type: Improvement
Affects Versions: 0.21.0
Reporter: Lars Francke
Priority: Trivial
Fix For: 0.21.0
Attachments: HBASE-2718.patch
This just updates the .gitignore file:
Removes all references to core/ and contrib/
Also removes /build which should be a relic from the Ant/Ivy times
Adds /.idea/ for the IntelliJ users
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2718) Update .gitignore for trunk after removal of contribs
[ https://issues.apache.org/jira/browse/HBASE-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12878405#action_12878405 ] Lars Francke commented on HBASE-2718: - Thanks and sorry. Something must have gone wrong on my side :( Update .gitignore for trunk after removal of contribs - Key: HBASE-2718 URL: https://issues.apache.org/jira/browse/HBASE-2718 Project: HBase Issue Type: Improvement Affects Versions: 0.21.0 Reporter: Lars Francke Priority: Trivial Fix For: 0.21.0 Attachments: HBASE-2718.patch This just updates the .gitignore file: Removes all references to core/ and contrib/ Also removes /build which should be a relic from the Ant/Ivy times Adds /.idea/ for the IntelliJ users -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-2468) Improvements to prewarm META cache on clients
[ https://issues.apache.org/jira/browse/HBASE-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-2468. -- Hadoop Flags: [Reviewed] Resolution: Fixed Committed. Thank you for the patch Mingjie Lai. Improvements to prewarm META cache on clients - Key: HBASE-2468 URL: https://issues.apache.org/jira/browse/HBASE-2468 Project: HBase Issue Type: Improvement Components: client Reporter: Todd Lipcon Assignee: Mingjie Lai Fix For: 0.21.0 Attachments: HBASE-2468-trunk.patch A couple different use cases cause storms of reads to META during startup. For example, a large MR job will cause each map task to hit meta since it starts with an empty cache. A couple possible improvements have been proposed: - MR jobs could ship a copy of META for the table in the DistributedCache - Clients could prewarm cache by doing a large scan of all the meta for the table instead of random reads for each miss - Each miss could fetch ahead some number of rows in META -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2468) Improvements to prewarm META cache on clients
[ https://issues.apache.org/jira/browse/HBASE-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12878406#action_12878406 ] HBase Review Board commented on HBASE-2468: --- Message from: st...@duboce.net --- This is an automatically generated e-mail. To reply, visit: http://review.hbase.org/r/98/#review210 --- Ship it! +1 I did a quick skim. All looks good. This is a weird one in that to prewarm we are going to do a getClosestOrBefore (which we'll be doing anyway) and then we'll open scanner, and scan next 10 rows... then close scanner; i.e. extra rpcs inline w/ first lookup against tale. This latter will actually slow down first lookup some but we can make it faster by making open scanner return results so we don't have to then go call next (already an issue to do this) and also, making it so we scan forward always by changing keys in .META. such that region names are keyed by endkey rather than startkey... also another issue to do this. - stack Improvements to prewarm META cache on clients - Key: HBASE-2468 URL: https://issues.apache.org/jira/browse/HBASE-2468 Project: HBase Issue Type: Improvement Components: client Reporter: Todd Lipcon Assignee: Mingjie Lai Fix For: 0.21.0 Attachments: HBASE-2468-trunk.patch A couple different use cases cause storms of reads to META during startup. For example, a large MR job will cause each map task to hit meta since it starts with an empty cache. A couple possible improvements have been proposed: - MR jobs could ship a copy of META for the table in the DistributedCache - Clients could prewarm cache by doing a large scan of all the meta for the table instead of random reads for each miss - Each miss could fetch ahead some number of rows in META -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2718) Update .gitignore for trunk after removal of contribs
[ https://issues.apache.org/jira/browse/HBASE-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12878408#action_12878408 ] stack commented on HBASE-2718: -- No worries. Update .gitignore for trunk after removal of contribs - Key: HBASE-2718 URL: https://issues.apache.org/jira/browse/HBASE-2718 Project: HBase Issue Type: Improvement Affects Versions: 0.21.0 Reporter: Lars Francke Priority: Trivial Fix For: 0.21.0 Attachments: HBASE-2718.patch This just updates the .gitignore file: Removes all references to core/ and contrib/ Also removes /build which should be a relic from the Ant/Ivy times Adds /.idea/ for the IntelliJ users -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-50) Snapshot of table
[ https://issues.apache.org/jira/browse/HBASE-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12878411#action_12878411 ] stack commented on HBASE-50: .bq What if the table with the same name is still online when we want to restore a snapshot Fail with a warning. A nice-to-have would be your suggestion of restoring snapshot into a table named something other than the original table's name (Fixing this issue is low-priority IMO). There a few things in the above that make me want to go over the design again. I'll report back after I've done that. Specifically: .bq Rename 'reference' family in .META. to 'snapshot' ... didn't we discuss that .META. might not be the place to keep snapshot data since regions are deleted when the system is done w/ them (but a snapshot may outlive a particular region). Snapshot of table - Key: HBASE-50 URL: https://issues.apache.org/jira/browse/HBASE-50 Project: HBase Issue Type: New Feature Reporter: Billy Pearson Assignee: Li Chongxin Priority: Minor Attachments: HBase Snapshot Design Report V2.pdf, HBase Snapshot Design Report V3.pdf, snapshot-src.zip Havening an option to take a snapshot of a table would be vary useful in production. What I would like to see this option do is do a merge of all the data into one or more files stored in the same folder on the dfs. This way we could save data in case of a software bug in hadoop or user code. The other advantage would be to be able to export a table to multi locations. Say I had a read_only table that must be online. I could take a snapshot of it when needed and export it to a separate data center and have it loaded there and then i would have it online at multi data centers for load balancing and failover. I understand that hadoop takes the need out of havening backup to protect from failed servers, but this does not protect use from software bugs that might delete or alter data in ways we did not plan. We should have a way we can roll back a dataset. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HBASE-2719) Fix test failures for Avro server on Hudson
Fix test failures for Avro server on Hudson --- Key: HBASE-2719 URL: https://issues.apache.org/jira/browse/HBASE-2719 Project: HBase Issue Type: Bug Components: avro Reporter: Jeff Hammerbacher Hudson is failing the tests from HBASE-2400: http://hudson.hbase.org/job/hbase-trunk/119/#showFailuresLink. These tests pass locally, so it's not clear why they are failing with Hudson. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2719) Fix test failures for Avro server on Hudson
[ https://issues.apache.org/jira/browse/HBASE-2719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12878418#action_12878418 ] Jeff Hammerbacher commented on HBASE-2719: -- Interestingly, http://hudson.zones.apache.org/hudson/job/HBase-TRUNK/1321/testReport/org.apache.hadoop.hbase.avro/TestAvroServer/ says these tests are passing. I'll try to investigate a bit more. Fix test failures for Avro server on Hudson --- Key: HBASE-2719 URL: https://issues.apache.org/jira/browse/HBASE-2719 Project: HBase Issue Type: Bug Components: avro Reporter: Jeff Hammerbacher Hudson is failing the tests from HBASE-2400: http://hudson.hbase.org/job/hbase-trunk/119/#showFailuresLink. These tests pass locally, so it's not clear why they are failing with Hudson. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-2353) HBASE-2283 removed bulk sync optimization for multi-row puts
[ https://issues.apache.org/jira/browse/HBASE-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HBASE-2353. Hadoop Flags: [Reviewed] Resolution: Fixed Committed to trunk. Thanks for review, Stack and Ryan. HBASE-2283 removed bulk sync optimization for multi-row puts Key: HBASE-2353 URL: https://issues.apache.org/jira/browse/HBASE-2353 Project: HBase Issue Type: Bug Reporter: ryan rawson Assignee: Todd Lipcon Priority: Blocker Fix For: 0.21.0 Attachments: HBASE-2353_def_log_flush.patch previously to HBASE-2283 we used to call flush/sync once per put(Put[]) call (ie: batch of commits). Now we do for every row. This makes bulk uploads slower if you are using WAL. Is there an acceptable solution to achieve both safety and performance by bulk-sync'ing puts? Or would this not work in face of atomic guarantees? discuss! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2353) HBASE-2283 removed bulk sync optimization for multi-row puts
[ https://issues.apache.org/jira/browse/HBASE-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HBASE-2353: --- Attachment: hbase-2353.txt Attaching patch I committed (r2 from review board) HBASE-2283 removed bulk sync optimization for multi-row puts Key: HBASE-2353 URL: https://issues.apache.org/jira/browse/HBASE-2353 Project: HBase Issue Type: Bug Reporter: ryan rawson Assignee: Todd Lipcon Priority: Blocker Fix For: 0.21.0 Attachments: hbase-2353.txt, HBASE-2353_def_log_flush.patch previously to HBASE-2283 we used to call flush/sync once per put(Put[]) call (ie: batch of commits). Now we do for every row. This makes bulk uploads slower if you are using WAL. Is there an acceptable solution to achieve both safety and performance by bulk-sync'ing puts? Or would this not work in face of atomic guarantees? discuss! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1025) Reconstruction log playback has no bounds on memory used
[ https://issues.apache.org/jira/browse/HBASE-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-1025: - Attachment: 1025-v3.txt Lost edits was actually in interesting issue having to do w/ sequenceids (I love unit tests!) Here is v3. Let me give it one more review.. then will post to review.hbase.org. Reconstruction log playback has no bounds on memory used Key: HBASE-1025 URL: https://issues.apache.org/jira/browse/HBASE-1025 Project: HBase Issue Type: Bug Reporter: stack Assignee: Kannan Muthukkaruppan Fix For: 0.21.0 Attachments: 1025-v2.txt, 1025-v3.txt, 1025.txt Makes a TreeMap and just keeps adding edits without regard for size of edits applied; could cause OOME (I've not seen a definitive case though have seen an OOME around time of a reconstructionlog replay -- perhaps this the straw that broke the fleas antlers?) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HBASE-2720) TestFromClientSide fails for client region cache prewarm on Hudson
TestFromClientSide fails for client region cache prewarm on Hudson -- Key: HBASE-2720 URL: https://issues.apache.org/jira/browse/HBASE-2720 Project: HBase Issue Type: Bug Components: client, test Affects Versions: 0.21.0 Environment: hudson Reporter: Mingjie Lai Assignee: Mingjie Lai Fix For: 0.21.0 TestFromClientSide failed by HBASE-2468 patch: http://hudson.zones.apache.org/hudson/job/HBase-TRUNK/1322/testReport/junit/org.apache.hadoop.hbase.client/TestFromClientSide/testRegionCachePreWarm/ It seems the number of actual cached regions was less than expected (as configured) on hudson. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
