[jira] Created: (HBASE-2919) initTableReducerJob: Unused method parameter.
initTableReducerJob: Unused method parameter. - Key: HBASE-2919 URL: https://issues.apache.org/jira/browse/HBASE-2919 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.89.20100621 Reporter: Libor Dener In method org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableReducerJob(String, Class? extends TableReducer, Job, Class) the partitioner parameter was not passed to called org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableReducerJob(String, Class? extends TableReducer, Job, Class) method. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2919) initTableReducerJob: Unused method parameter.
[ https://issues.apache.org/jira/browse/HBASE-2919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Libor Dener updated HBASE-2919: --- Attachment: hbase-2919.patch Fixes the bug by passing the parameter. initTableReducerJob: Unused method parameter. - Key: HBASE-2919 URL: https://issues.apache.org/jira/browse/HBASE-2919 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.89.20100621 Reporter: Libor Dener Attachments: hbase-2919.patch Original Estimate: 0.02h Remaining Estimate: 0.02h In method org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableReducerJob(String, Class? extends TableReducer, Job, Class) the partitioner parameter was not passed to called org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableReducerJob(String, Class? extends TableReducer, Job, Class) method. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2919) initTableReducerJob: Unused method parameter.
[ https://issues.apache.org/jira/browse/HBASE-2919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Libor Dener updated HBASE-2919: --- Status: Patch Available (was: Open) Release Note: HBASE-2919 initTableReducerJob: method parameter not passed to work horse function. initTableReducerJob: Unused method parameter. - Key: HBASE-2919 URL: https://issues.apache.org/jira/browse/HBASE-2919 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.89.20100621 Reporter: Libor Dener Attachments: hbase-2919.patch Original Estimate: 0.02h Remaining Estimate: 0.02h In method org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableReducerJob(String, Class? extends TableReducer, Job, Class) the partitioner parameter was not passed to called org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableReducerJob(String, Class? extends TableReducer, Job, Class) method. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-2908) Wrong order of null-check
[ https://issues.apache.org/jira/browse/HBASE-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reassigned HBASE-2908: Assignee: Libor Dener Wrong order of null-check - Key: HBASE-2908 URL: https://issues.apache.org/jira/browse/HBASE-2908 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.89.20100621 Reporter: Libor Dener Assignee: Libor Dener Priority: Trivial Fix For: 0.90.0 Attachments: hbase-2908-fix.patch In method org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(JobContext) this.table is used before null-throw check. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HBASE-2920) HTable.checkAndPut/Delete doesn't handle null values
HTable.checkAndPut/Delete doesn't handle null values
Key: HBASE-2920
URL: https://issues.apache.org/jira/browse/HBASE-2920
Project: HBase
Issue Type: Bug
Affects Versions: 0.89.20100621
Reporter: Jean-Daniel Cryans
Priority: Critical
Fix For: 0.90.0
From John Beatty on the ML:
{quote}
Thanks Ryan, but I seem to be missing something then. It NPEs for me.
When running against 0.89.20100726 and providing a null expected value
I get the below stack trace (and works like a champ when I provide a
byte[0]. I also don't see the transformation you're referring to in
HTable.
(for reference,
http://svn.apache.org/viewvc/hbase/branches/0.89.20100726/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java?view=markup)
java.io.IOException: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:845)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:835)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.checkAndMutate(HRegionServer.java:1754)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.checkAndPut(HRegionServer.java:1773)
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:576)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:919)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.HRegion.checkAndMutate(HRegion.java:1616)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.checkAndMutate(HRegionServer.java:1751)
... 6 more
{quote}
Looking in the code, I'm not sure either where the null conversion is done,
even worse is that we don't even have unit tests! It should be put
intoTestFromClientSide.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-7) [hbase] Provide a HBase checker and repair tool similar to fsck
[
https://issues.apache.org/jira/browse/HBASE-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12899087#action_12899087
]
Luke Forehand commented on HBASE-7:
---
I just ran this utility on our corrupt META table and got this exception:
Exception in thread main java.io.IOException: Two entries in META are same
REGION = {NAME = 'feedData,20100717
0463ff1d352930dc354f35358d3d11997c2fa050,1281549071340.d905798b9e7dca1c24a09a908bd1081c.',
STARTKEY = '20100717 0463ff1d352930dc354f35358d3d11997c2fa050', ENDKEY =
'20100717 25beb50790d72e3e2bf1fbebc0656dbcd82cdbd3', ENCODED =
d905798b9e7dca1c24a09a908bd1081c, TABLE = {{NAME = 'feedData', FAMILIES =
[{NAME = 'core', BLOOMFILTER = 'NONE', REPLICATION_SCOPE = '0', COMPRESSION
= 'LZO', VERSIONS = '3', TTL = '2147483647', BLOCKSIZE = '131072',
IN_MEMORY = 'false', BLOCKCACHE = 'true'}, {NAME = 'tf', BLOOMFILTER =
'NONE', REPLICATION_SCOPE = '0', COMPRESSION = 'LZO', VERSIONS = '3', TTL =
'2147483647', BLOCKSIZE = '131072', IN_MEMORY = 'false', BLOCKCACHE =
'true'}, {NAME = 'tfidf', BLOOMFILTER = 'NONE', REPLICATION_SCOPE = '0',
COMPRESSION = 'LZO', VERSIONS = '3', TTL = '2147483647', BLOCKSIZE =
'131072', IN_MEMORY = 'false', BLOCKCACHE = 'true'}]}}
at
org.apache.hadoop.hbase.client.HBaseFsck$1.processRow(HBaseFsck.java:420)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:156)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:68)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:53)
at com.ni.ods.hadoop.utils.HBaseFsck.getMetaEntries(HBaseFsck.java:435)
at com.ni.ods.hadoop.utils.HBaseFsck.doWork(HBaseFsck.java:109)
at com.ni.ods.hadoop.utils.HBaseFsck.main(HBaseFsck.java:522)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
[hbase] Provide a HBase checker and repair tool similar to fsck
---
Key: HBASE-7
URL: https://issues.apache.org/jira/browse/HBASE-7
Project: HBase
Issue Type: New Feature
Components: util
Reporter: Jim Kellerman
Assignee: dhruba borthakur
Fix For: 0.90.0
Attachments: add_region.rb, check_meta.rb, check_meta.rb,
HBASE-7.patch, hbase.fsck1.txt, HBaseConfiguration.java, HBaseFsck.java,
patch.txt
We need a tool to verify (and repair) HBase much like fsck
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2921) HBase shell prompt is not configured when used as a subprocess
[ https://issues.apache.org/jira/browse/HBASE-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12899135#action_12899135 ] Aditya Acharya commented on HBASE-2921: --- Yeah, good observation. It looks like lib/ruby/1.x/irb/init.rb has this line: @CONF[:PROMPT_MODE] = (STDIN.tty? ? :DEFAULT : :NULL) which is the source of this issue. I'll investigate using a pty with subprocess for my use case. HBase shell prompt is not configured when used as a subprocess -- Key: HBASE-2921 URL: https://issues.apache.org/jira/browse/HBASE-2921 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.89.20100621 Reporter: Aditya Acharya Original Estimate: 1h Remaining Estimate: 1h When you start the HBase shell from bash, you see the following prompt: hbase(main):001:0 And typing in conf as the command yields the following prompt-related information: conf.prompt_c=%N(%m):%03n:%i* conf.prompt_i=%N(%m):%03n:%i conf.prompt_mode=:DEFAULT conf.prompt_n=%N(%m):%03n:%i conf.prompt_s=%N(%m):%03n:%i%l On the other hand, opening the HBase shell as python subprocess yields an empty string as the prompt string. Furthermore, sending it the conf command through a pipe yields the following output: conf.prompt_c=nil conf.prompt_i=nil conf.prompt_mode=:NULL conf.prompt_n=nil conf.prompt_s=nil I think this is a bug in the HBase shell. I'm not sure where it occurs, but I have found that it can be easily patched up by hard-coding the prompt information into bin/hirb.rb. This seems like the most appropriate fix, as bin/hirb.rb already modifies the conf for the interpreter. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HBASE-2922) HLog cleanup is done under the updateLock, major slowdown
HLog cleanup is done under the updateLock, major slowdown
-
Key: HBASE-2922
URL: https://issues.apache.org/jira/browse/HBASE-2922
Project: HBase
Issue Type: Bug
Affects Versions: 0.89.20100621, 0.20.6
Reporter: Jean-Daniel Cryans
Something I've seen quite often in our production environment:
{quote}
2010-08-16 16:17:27,104 INFO org.apache.hadoop.hbase.regionserver.HLog:
removing old hlog file
/hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000385321 whose highest
sequence/edit id is 64837079950
2010-08-16 16:17:27,286 INFO org.apache.hadoop.hbase.regionserver.HLog:
removing old hlog file
/hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000392770 whose highest
sequence/edit id is 64837088260
2010-08-16 16:17:27,452 INFO org.apache.hadoop.hbase.regionserver.HLog:
removing old hlog file
/hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000399300 whose highest
sequence/edit id is 64837096566
2010-08-16 16:17:27,635 INFO org.apache.hadoop.hbase.regionserver.HLog:
removing old hlog file
/hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000406997 whose highest
sequence/edit id is 64837104865
2010-08-16 16:17:27,827 INFO org.apache.hadoop.hbase.regionserver.HLog:
removing old hlog file
/hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000413803 whose highest
sequence/edit id is 64837113153
2010-08-16 16:17:27,993 INFO org.apache.hadoop.hbase.regionserver.HLog:
removing old hlog file
/hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000421709 whose highest
sequence/edit id is 64837121467
2010-08-16 16:17:28,160 INFO org.apache.hadoop.hbase.regionserver.HLog:
removing old hlog file
/hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000427333 whose highest
sequence/edit id is 64837129775
2010-08-16 16:17:28,432 INFO org.apache.hadoop.hbase.regionserver.HLog:
removing old hlog file
/hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000434365 whose highest
sequence/edit id is 64837138074
2010-08-16 16:17:28,518 INFO org.apache.hadoop.hbase.regionserver.HLog:
removing old hlog file
/hbase/.logs/rs22,60020,1280909840873/hlog.dat.1282000440347 whose highest
sequence/edit id is 64837146376
2010-08-16 16:17:28,612 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
Server handler 39 on 60020 took 1801ms appending an edit to hlog; editcount=0
2010-08-16 16:17:28,615 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
Server handler 37 on 60020 took 1804ms appending an edit to hlog; editcount=1
2010-08-16 16:17:28,615 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
Server handler 25 on 60020 took 1805ms appending an edit to hlog; editcount=2
...
2010-08-16 16:17:28,619 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
Server handler 41 on 60020 took 1875ms appending an edit to hlog; editcount=50
2010-08-16 16:17:28,619 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
Server handler 24 on 60020 took 1876ms appending an edit to hlog; editcount=51
2010-08-16 16:17:28,619 WARN org.apache.hadoop.hbase.regionserver.HLog: IPC
Server handler 48 on 60020 took 1881ms appending an edit to hlog; editcount=54
{quote}
And looking at HLog.rollWriter, we roll then cleanup those unused hlog files
under updateLock, which blocks all the appenders (as shown). We should only do
the first part under that lock
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-50) Snapshot of table
[ https://issues.apache.org/jira/browse/HBASE-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12899195#action_12899195 ] HBase Review Board commented on HBASE-50: - Message from: Ted Yu ted...@yahoo.com --- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/467/#review924 --- src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWrapper.java http://review.cloudera.org/r/467/#comment3015 Should call currentThread().interrupt() src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWrapper.java http://review.cloudera.org/r/467/#comment3016 Should call currentThread().interrupt() - Ted Snapshot of table - Key: HBASE-50 URL: https://issues.apache.org/jira/browse/HBASE-50 Project: HBase Issue Type: New Feature Reporter: Billy Pearson Assignee: Li Chongxin Priority: Minor Attachments: HBase Snapshot Design Report V2.pdf, HBase Snapshot Design Report V3.pdf, HBase Snapshot Implementation Plan.pdf, Snapshot Class Diagram.png Havening an option to take a snapshot of a table would be vary useful in production. What I would like to see this option do is do a merge of all the data into one or more files stored in the same folder on the dfs. This way we could save data in case of a software bug in hadoop or user code. The other advantage would be to be able to export a table to multi locations. Say I had a read_only table that must be online. I could take a snapshot of it when needed and export it to a separate data center and have it loaded there and then i would have it online at multi data centers for load balancing and failover. I understand that hadoop takes the need out of havening backup to protect from failed servers, but this does not protect use from software bugs that might delete or alter data in ways we did not plan. We should have a way we can roll back a dataset. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-50) Snapshot of table
[ https://issues.apache.org/jira/browse/HBASE-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12899208#action_12899208 ] HBase Review Board commented on HBASE-50: - Message from: Ted Yu ted...@yahoo.com --- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/467/#review927 --- src/main/java/org/apache/hadoop/hbase/master/HMaster.java http://review.cloudera.org/r/467/#comment3024 Write a script that calls this method. - Ted Snapshot of table - Key: HBASE-50 URL: https://issues.apache.org/jira/browse/HBASE-50 Project: HBase Issue Type: New Feature Reporter: Billy Pearson Assignee: Li Chongxin Priority: Minor Attachments: HBase Snapshot Design Report V2.pdf, HBase Snapshot Design Report V3.pdf, HBase Snapshot Implementation Plan.pdf, Snapshot Class Diagram.png Havening an option to take a snapshot of a table would be vary useful in production. What I would like to see this option do is do a merge of all the data into one or more files stored in the same folder on the dfs. This way we could save data in case of a software bug in hadoop or user code. The other advantage would be to be able to export a table to multi locations. Say I had a read_only table that must be online. I could take a snapshot of it when needed and export it to a separate data center and have it loaded there and then i would have it online at multi data centers for load balancing and failover. I understand that hadoop takes the need out of havening backup to protect from failed servers, but this does not protect use from software bugs that might delete or alter data in ways we did not plan. We should have a way we can roll back a dataset. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
