[jira] [Comment Edited] (HBASE-18540) Ad-hoc job for checking if a test is flaky
[
https://issues.apache.org/jira/browse/HBASE-18540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118915#comment-16118915
]
Jonathan Hsieh edited comment on HBASE-18540 at 8/8/17 7:45 PM:
I use this script on my lappy. Making it job sounds like it could be useful.
edit: removing private repo link and adding code instead
{code}
#!/bin/bash
CMD="$@"
N=${N:-5}
echo "N = $N"
for i in `seq $N`
do
$CMD
RET=$?
if [ $RET != 0 ]; then
echo "Failed on iteration $i"
exit $RET
fi
done
echo "Succeeded $i iterations"
{code}
was (Author: jmhsieh):
I use this script on my lappy. Making it job sounds like it could be useful.
edt: removing private repo link and adding code instead
{code}
#!/bin/bash
CMD="$@"
N=${N:-5}
echo "N = $N"
for i in `seq $N`
do
$CMD
RET=$?
if [ $RET != 0 ]; then
echo "Failed on iteration $i"
exit $RET
fi
done
echo "Succeeded $i iterations"
{code}
> Ad-hoc job for checking if a test is flaky
> --
>
> Key: HBASE-18540
> URL: https://issues.apache.org/jira/browse/HBASE-18540
> Project: HBase
> Issue Type: New Feature
> Components: test
>Reporter: Sean Busbey
>
> It would be great if we had an ad-hoc job that could run a given test a bunch
> of times and tell me if it is flaky.
> I'm trying to evaluate the state of our new nightly jobs, and I'm not sure if
> hte failures I'm seeing are a) things I need to get fixed soon, b) things
> that are flaky but previously unrecognized, c) things that are flaky but
> missed because our current flaky list is only based on the behavior of the
> master branch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18540) Ad-hoc job for checking if a test is flaky
[
https://issues.apache.org/jira/browse/HBASE-18540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118915#comment-16118915
]
Jonathan Hsieh edited comment on HBASE-18540 at 8/8/17 7:45 PM:
I use this script on my lappy. Making it job sounds like it could be useful.
edt: removing private repo link and adding code instead
{code}
#!/bin/bash
CMD="$@"
N=${N:-5}
echo "N = $N"
for i in `seq $N`
do
$CMD
RET=$?
if [ $RET != 0 ]; then
echo "Failed on iteration $i"
exit $RET
fi
done
echo "Succeeded $i iterations"
{code}
was (Author: jmhsieh):
I use this script on my lappy. Making it job sounds like it could be useful.
http://github.mtv.cloudera.com/jon/bin/blob/master/loopfail.sh
> Ad-hoc job for checking if a test is flaky
> --
>
> Key: HBASE-18540
> URL: https://issues.apache.org/jira/browse/HBASE-18540
> Project: HBase
> Issue Type: New Feature
> Components: test
>Reporter: Sean Busbey
>
> It would be great if we had an ad-hoc job that could run a given test a bunch
> of times and tell me if it is flaky.
> I'm trying to evaluate the state of our new nightly jobs, and I'm not sure if
> hte failures I'm seeing are a) things I need to get fixed soon, b) things
> that are flaky but previously unrecognized, c) things that are flaky but
> missed because our current flaky list is only based on the behavior of the
> master branch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18540) Ad-hoc job for checking if a test is flaky
[ https://issues.apache.org/jira/browse/HBASE-18540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118915#comment-16118915 ] Jonathan Hsieh commented on HBASE-18540: I use this script on my lappy. Making it job sounds like it could be useful. http://github.mtv.cloudera.com/jon/bin/blob/master/loopfail.sh > Ad-hoc job for checking if a test is flaky > -- > > Key: HBASE-18540 > URL: https://issues.apache.org/jira/browse/HBASE-18540 > Project: HBase > Issue Type: New Feature > Components: test >Reporter: Sean Busbey > > It would be great if we had an ad-hoc job that could run a given test a bunch > of times and tell me if it is flaky. > I'm trying to evaluate the state of our new nightly jobs, and I'm not sure if > hte failures I'm seeing are a) things I need to get fixed soon, b) things > that are flaky but previously unrecognized, c) things that are flaky but > missed because our current flaky list is only based on the behavior of the > master branch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18228) HBCK improvements
[ https://issues.apache.org/jira/browse/HBASE-18228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052788#comment-16052788 ] Jonathan Hsieh commented on HBASE-18228: The problem in 2.x is that some of the repairs (especially around assignment) are likely no longer valid because they use operations that subsumed by the new assignment manager, and new operations for repairs of proc wal may be needed. > HBCK improvements > - > > Key: HBASE-18228 > URL: https://issues.apache.org/jira/browse/HBASE-18228 > Project: HBase > Issue Type: Improvement > Components: hbck >Reporter: Lars Hofhansl >Priority: Critical > Fix For: 1.4.0 > > > We just had a prod issue and running HBCK the way we did actually causes more > problems. > In part HBCK did stuff we did not expect, in part we had little visibility > into what HBCK was doing, and in part the logging was confusing. > I'm proposing 2 improvements: > 1. A dry-run mode. Run, and just list what would have been done. > 2. An interactive mode. Run, and for each action request Y/N user input. So > that a user can opt-out of stuff. > [~jmhsieh], FYI -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17996) HBase master fails to start sometimes on RHEL7
[
https://issues.apache.org/jira/browse/HBASE-17996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052503#comment-16052503
]
Jonathan Hsieh commented on HBASE-17996:
Thanks [~elserj]!
> HBase master fails to start sometimes on RHEL7
> --
>
> Key: HBASE-17996
> URL: https://issues.apache.org/jira/browse/HBASE-17996
> Project: HBase
> Issue Type: Bug
> Components: master, test
>Reporter: David Knupp
> Attachments:
> hbase-jenkins-master-impala-boost-static-burst-slave-el7-02f4.vpc.cloudera.com.out,
>
> hbase-jenkins-master-impala-boost-static-burst-slave-el7-11ef.vpc.cloudera.com.out
>
>
> Impala includes HBase in its local test environment, and we have found that
> intermittently, the HBase master node fails to start when we are testing on
> RHEL7.
> In these failures, what we typically see in the logs is this:
> {noformat}
> 17/04/29 21:33:47 INFO zookeeper.ClientCnxn: Session establishment complete
> on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x15bbd21b797000a,
> negotiated timeout = 9
> 17/04/29 21:33:47 INFO client.ZooKeeperRegistry: ClusterId read in ZooKeeper
> is null
> 17/04/29 21:33:48 INFO master.ActiveMasterManager: Deleting ZNode for
> /hbase/backup-masters/localhost,16000,1493526758211 from backup master
> directory
> {noformat}
> On a successful startup, the log looks like this:
> {noformat}
> 17/04/16 21:32:29 INFO zookeeper.ClientCnxn: Session establishment complete
> on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x15b7a2ed6860005,
> negotiated timeout = 9
> 17/04/16 21:32:29 INFO client.ZooKeeperRegistry: ClusterId read in ZooKeeper
> is null
> 17/04/16 21:32:30 INFO util.FSUtils: Created version file at
> hdfs://localhost:20500/hbase with version=8
> 17/04/16 21:32:31 INFO master.MasterFileSystem: BOOTSTRAP: creating
> hbase:meta region
> {noformat}
> So the event that we don't see in the failed start up attempts is
> {{master.MasterFileSystem: BOOTSTRAP: creating hbase:meta region}}.
> The full logs will be attached.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18228) HBCK improvements
[ https://issues.apache.org/jira/browse/HBASE-18228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052400#comment-16052400 ] Jonathan Hsieh commented on HBASE-18228: These sound like fine ideas. I think we'd want to limit the scope of this to Hbase 1.x since Hbase 2.x will likely rewrite hbck for its particular problems. [~larsh], do you plan on writing these improvements? > HBCK improvements > - > > Key: HBASE-18228 > URL: https://issues.apache.org/jira/browse/HBASE-18228 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl > > We just had a prod issue and running HBCK the way we did actually causes more > problems. > In part HBCK did stuff we did not expect, in part we had little visibility > into what HBCK was doing, and in part the logging was confusing. > I'm proposing 2 improvements: > 1. A dry-run mode. Run, and just list what would have been done. > 2. An interactive mode. Run, and for each action request Y/N user input. So > that a user can opt-out of stuff. > [~jmhsieh], FYI -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-17996) HBase master fails to start sometimes on RHEL7
[
https://issues.apache.org/jira/browse/HBASE-17996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh resolved HBASE-17996.
Resolution: Fixed
> HBase master fails to start sometimes on RHEL7
> --
>
> Key: HBASE-17996
> URL: https://issues.apache.org/jira/browse/HBASE-17996
> Project: HBase
> Issue Type: Bug
> Components: master, test
>Reporter: David Knupp
> Attachments:
> hbase-jenkins-master-impala-boost-static-burst-slave-el7-02f4.vpc.cloudera.com.out,
>
> hbase-jenkins-master-impala-boost-static-burst-slave-el7-11ef.vpc.cloudera.com.out
>
>
> Impala includes HBase in its local test environment, and we have found that
> intermittently, the HBase master node fails to start when we are testing on
> RHEL7.
> In these failures, what we typically see in the logs is this:
> {noformat}
> 17/04/29 21:33:47 INFO zookeeper.ClientCnxn: Session establishment complete
> on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x15bbd21b797000a,
> negotiated timeout = 9
> 17/04/29 21:33:47 INFO client.ZooKeeperRegistry: ClusterId read in ZooKeeper
> is null
> 17/04/29 21:33:48 INFO master.ActiveMasterManager: Deleting ZNode for
> /hbase/backup-masters/localhost,16000,1493526758211 from backup master
> directory
> {noformat}
> On a successful startup, the log looks like this:
> {noformat}
> 17/04/16 21:32:29 INFO zookeeper.ClientCnxn: Session establishment complete
> on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x15b7a2ed6860005,
> negotiated timeout = 9
> 17/04/16 21:32:29 INFO client.ZooKeeperRegistry: ClusterId read in ZooKeeper
> is null
> 17/04/16 21:32:30 INFO util.FSUtils: Created version file at
> hdfs://localhost:20500/hbase with version=8
> 17/04/16 21:32:31 INFO master.MasterFileSystem: BOOTSTRAP: creating
> hbase:meta region
> {noformat}
> So the event that we don't see in the failed start up attempts is
> {{master.MasterFileSystem: BOOTSTRAP: creating hbase:meta region}}.
> The full logs will be attached.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17996) HBase master fails to start sometimes on RHEL7
[
https://issues.apache.org/jira/browse/HBASE-17996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052166#comment-16052166
]
Jonathan Hsieh commented on HBASE-17996:
corresponding issue IMPALA-5223 was resolved. Instead of using the inprocess
minicluster, they scripted a deploy of a standalong cluster or psuedo-dist
cluster and fixed their issue by waiting for nodes to show before starting
tests.
> HBase master fails to start sometimes on RHEL7
> --
>
> Key: HBASE-17996
> URL: https://issues.apache.org/jira/browse/HBASE-17996
> Project: HBase
> Issue Type: Bug
> Components: master, test
>Reporter: David Knupp
> Attachments:
> hbase-jenkins-master-impala-boost-static-burst-slave-el7-02f4.vpc.cloudera.com.out,
>
> hbase-jenkins-master-impala-boost-static-burst-slave-el7-11ef.vpc.cloudera.com.out
>
>
> Impala includes HBase in its local test environment, and we have found that
> intermittently, the HBase master node fails to start when we are testing on
> RHEL7.
> In these failures, what we typically see in the logs is this:
> {noformat}
> 17/04/29 21:33:47 INFO zookeeper.ClientCnxn: Session establishment complete
> on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x15bbd21b797000a,
> negotiated timeout = 9
> 17/04/29 21:33:47 INFO client.ZooKeeperRegistry: ClusterId read in ZooKeeper
> is null
> 17/04/29 21:33:48 INFO master.ActiveMasterManager: Deleting ZNode for
> /hbase/backup-masters/localhost,16000,1493526758211 from backup master
> directory
> {noformat}
> On a successful startup, the log looks like this:
> {noformat}
> 17/04/16 21:32:29 INFO zookeeper.ClientCnxn: Session establishment complete
> on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x15b7a2ed6860005,
> negotiated timeout = 9
> 17/04/16 21:32:29 INFO client.ZooKeeperRegistry: ClusterId read in ZooKeeper
> is null
> 17/04/16 21:32:30 INFO util.FSUtils: Created version file at
> hdfs://localhost:20500/hbase with version=8
> 17/04/16 21:32:31 INFO master.MasterFileSystem: BOOTSTRAP: creating
> hbase:meta region
> {noformat}
> So the event that we don't see in the failed start up attempts is
> {{master.MasterFileSystem: BOOTSTRAP: creating hbase:meta region}}.
> The full logs will be attached.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18146) Long running integration test similar to TestAcidGuarantees
[ https://issues.apache.org/jira/browse/HBASE-18146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033977#comment-16033977 ] Jonathan Hsieh commented on HBASE-18146: Ugh, I realize that was hard to understand/parse but I think you got the jist. -- An alternative would be to change the flush size config. That said, a monkeyfactory that flushes frequently and causes compactions frequently (and similarly, a split/merge monkey factory) make sense. Would it make sense to just file those as separate tickets? We coudl also either close this out as info provided or convert it to a docs improvement jira?) > Long running integration test similar to TestAcidGuarantees > --- > > Key: HBASE-18146 > URL: https://issues.apache.org/jira/browse/HBASE-18146 > Project: HBase > Issue Type: Test > Components: integration tests >Reporter: Andrew Purtell > > TestAcidGuarantees and IntegrationTestAcidGuarantees both really only work > with minicluster based testing and do not run for a long duration. Consider a > new integration test that makes similar atomicity checks while running for, > potentially, a very long time, determined by test parameters supplied on the > command line (perhaps as property definitions). The new integration test > should expect to run against a distributed cluster, support specification of > desired monkey policy, and not require any special non-default site > configuration settings. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18146) Long running integration test similar to TestAcidGuarantees
[ https://issues.apache.org/jira/browse/HBASE-18146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033884#comment-16033884 ] Jonathan Hsieh commented on HBASE-18146: There is always room for improvement. :) For your initial specific asks, I think this is sufficient. That suggested monkey sounds useful. An alternative appraoch that would require config as opposed to code woudl be to flush size on the cluster under test which would to exercise flush and compact those mechanisms more frequently (would affect a few of my other favorites like ITBLL and ITIngest) The fact that this was filed means we should probably add a section to the docs to make it more obvious that this and other tests are runnable from the command line. (some IT tests use the class's static main function to run, other use the ITDriver to execute some canned runs). > Long running integration test similar to TestAcidGuarantees > --- > > Key: HBASE-18146 > URL: https://issues.apache.org/jira/browse/HBASE-18146 > Project: HBase > Issue Type: Test > Components: integration tests >Reporter: Andrew Purtell > > TestAcidGuarantees and IntegrationTestAcidGuarantees both really only work > with minicluster based testing and do not run for a long duration. Consider a > new integration test that makes similar atomicity checks while running for, > potentially, a very long time, determined by test parameters supplied on the > command line (perhaps as property definitions). The new integration test > should expect to run against a distributed cluster, support specification of > desired monkey policy, and not require any special non-default site > configuration settings. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18146) Long running integration test similar to TestAcidGuarantees
[ https://issues.apache.org/jira/browse/HBASE-18146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033556#comment-16033556 ] Jonathan Hsieh commented on HBASE-18146: IntegrationTestAcidGuarantees works against real clusters for a configurable amount of time. See: https://github.com/apache/hbase/blob/master/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestAcidGuarantees.java#L113 This can be done via the command line like so: hbase org.apache.hadoop.hbase.IntegrationTestAcidGuarantees -DnumWriters=5 -DnumGetters=1 -DnumScanners=1 -Dmillis=60 -m calm In some internal setups, we've been running it against multiple branches for 10 minute runs with each monkey enabled against various configuration settings (e.g. all writes through multiwal, through mob write path, under kerberos security etc). > Long running integration test similar to TestAcidGuarantees > --- > > Key: HBASE-18146 > URL: https://issues.apache.org/jira/browse/HBASE-18146 > Project: HBase > Issue Type: Test > Components: integration tests >Reporter: Andrew Purtell > > TestAcidGuarantees and IntegrationTestAcidGuarantees both really only work > with minicluster based testing and do not run for a long duration. Consider a > new integration test that makes similar atomicity checks while running for, > potentially, a very long time, determined by test parameters supplied on the > command line (perhaps as property definitions). The new integration test > should expect to run against a distributed cluster, support specification of > desired monkey policy, and not require any special non-default site > configuration settings. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15930) Make IntegrationTestReplication's waitForReplication() smarter
[
https://issues.apache.org/jira/browse/HBASE-15930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023197#comment-16023197
]
Jonathan Hsieh commented on HBASE-15930:
Did you test the pre-setup case?
line 190 seems backwards. -- shouldn't it be 'if
(!tableCFs.getcolumnFamilyMap(0.isEmpty()) ' ?
{code}
187 Admin admin = source.getConnection().getAdmin();
188 for (TableCFs tableCFs : admin.listReplicatedTableCFs()) {
189 if (tableCFs.getTable().equals(expected)) {
190 if (tableCFs.getColumnFamilyMap().isEmpty()) {
191 // Replicating at least one CF is good enough
192 return;
193 }
194 }
195 }
196 throw new RuntimeException("Aborting test: replication not
configured and we were instructed to not set it up.");
197 } finally {
{code}
> Make IntegrationTestReplication's waitForReplication() smarter
> --
>
> Key: HBASE-15930
> URL: https://issues.apache.org/jira/browse/HBASE-15930
> Project: HBase
> Issue Type: Improvement
> Components: integration tests
>Reporter: Dima Spivak
>Assignee: Mike Drob
> Fix For: 2.0.0
>
> Attachments: HBASE-15930.patch
>
>
> {{IntegrationTestReplication}} is a great test, but can improved by changing
> how we handle waiting between generation of the linked list on the source
> cluster and verifying the linked list on the destination cluster. [Even the
> code suggests this should be
> done|https://github.com/apache/hbase/blob/master/hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestReplication.java#L251-252],
> so I'd like to take it on. [~mbertozzi] and [~busbey] have both suggested a
> simple solution wherein we write a row into each region on the source cluster
> after the linked list generation and then assume replication has gone through
> once these rows are detected on the destination cluster.
> Since you lads at Facebook are some of the heaviest users, [~eclark], would
> you prefer I maintain the API and add a new command line option (say {{\-c |
> \-\-check-replication}}) that would run before any {{--generateVerifyGap}}
> sleep is carried out as it is now?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (HBASE-15930) Make IntegrationTestReplication's waitForReplication() smarter
[
https://issues.apache.org/jira/browse/HBASE-15930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023197#comment-16023197
]
Jonathan Hsieh edited comment on HBASE-15930 at 5/24/17 4:41 PM:
-
Did you test the pre-setup case?
line 190 seems backwards. -- shouldn't it be 'if
(!tableCFs.getcolumnFamilyMap().isEmpty()) ' ?
{code}
187 Admin admin = source.getConnection().getAdmin();
188 for (TableCFs tableCFs : admin.listReplicatedTableCFs()) {
189 if (tableCFs.getTable().equals(expected)) {
190 if (tableCFs.getColumnFamilyMap().isEmpty()) {
191 // Replicating at least one CF is good enough
192 return;
193 }
194 }
195 }
196 throw new RuntimeException("Aborting test: replication not
configured and we were instructed to not set it up.");
197 } finally {
{code}
was (Author: jmhsieh):
Did you test the pre-setup case?
line 190 seems backwards. -- shouldn't it be 'if
(!tableCFs.getcolumnFamilyMap(0.isEmpty()) ' ?
{code}
187 Admin admin = source.getConnection().getAdmin();
188 for (TableCFs tableCFs : admin.listReplicatedTableCFs()) {
189 if (tableCFs.getTable().equals(expected)) {
190 if (tableCFs.getColumnFamilyMap().isEmpty()) {
191 // Replicating at least one CF is good enough
192 return;
193 }
194 }
195 }
196 throw new RuntimeException("Aborting test: replication not
configured and we were instructed to not set it up.");
197 } finally {
{code}
> Make IntegrationTestReplication's waitForReplication() smarter
> --
>
> Key: HBASE-15930
> URL: https://issues.apache.org/jira/browse/HBASE-15930
> Project: HBase
> Issue Type: Improvement
> Components: integration tests
>Reporter: Dima Spivak
>Assignee: Mike Drob
> Fix For: 2.0.0
>
> Attachments: HBASE-15930.patch
>
>
> {{IntegrationTestReplication}} is a great test, but can improved by changing
> how we handle waiting between generation of the linked list on the source
> cluster and verifying the linked list on the destination cluster. [Even the
> code suggests this should be
> done|https://github.com/apache/hbase/blob/master/hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestReplication.java#L251-252],
> so I'd like to take it on. [~mbertozzi] and [~busbey] have both suggested a
> simple solution wherein we write a row into each region on the source cluster
> after the linked list generation and then assume replication has gone through
> once these rows are detected on the destination cluster.
> Since you lads at Facebook are some of the heaviest users, [~eclark], would
> you prefer I maintain the API and add a new command line option (say {{\-c |
> \-\-check-replication}}) that would run before any {{--generateVerifyGap}}
> sleep is carried out as it is now?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18043) Institute a hard limit for individual cell size that cannot be overridden by clients
[ https://issues.apache.org/jira/browse/HBASE-18043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009559#comment-16009559 ] Jonathan Hsieh commented on HBASE-18043: +1 from me. > Institute a hard limit for individual cell size that cannot be overridden by > clients > > > Key: HBASE-18043 > URL: https://issues.apache.org/jira/browse/HBASE-18043 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, regionserver >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-18043-branch-1.patch, HBASE-18043-branch-1.patch, > HBASE-18043-branch-1.patch, HBASE-18043.patch, HBASE-18043.patch, > HBASE-18043.patch, HBASE-18043.patch > > > For sake of service protection we should not give absolute trust to clients > regarding resource limits that can impact stability, like cell size limits. > We should add a server side configuration that sets a hard limit for > individual cell size that cannot be overridden by the client. We can keep the > client side check, because it's expensive to reject a RPC that has already > come in. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18043) Institute a hard limit for individual cell size that cannot be overridden by clients
[
https://issues.apache.org/jira/browse/HBASE-18043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009422#comment-16009422
]
Jonathan Hsieh commented on HBASE-18043:
Any reason why the cell's row key/column fam/column is not in the error
message? Also the include the max size configuration name in the message?
This would help with debugging and providing the operator hints if clients
claim data is "missing".
{code}
884 int size = CellUtil.estimatedSerializedSizeOf(cells.current());
885 if (size > r.maxCellSize) {
886 String msg = "Cell with size " + size + " exceeds limit of "
+ r.maxCellSize + " bytes";
887 if (LOG.isDebugEnabled()) {
888 LOG.debug(msg);
889 }
890 throw new DoNotRetryIOException(msg);
891 }
{code}
Also please add a release note about the new config.
> Institute a hard limit for individual cell size that cannot be overridden by
> clients
>
>
> Key: HBASE-18043
> URL: https://issues.apache.org/jira/browse/HBASE-18043
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC, regionserver
>Affects Versions: 2.0.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-18043-branch-1.patch, HBASE-18043-branch-1.patch,
> HBASE-18043.patch, HBASE-18043.patch, HBASE-18043.patch
>
>
> For sake of service protection we should not give absolute trust to clients
> regarding resource limits that can impact stability, like cell size limits.
> We should add a server side configuration that sets a hard limit for
> individual cell size that cannot be overridden by the client. We can keep the
> client side check, because it's expensive to reject a RPC that has already
> come in.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-18035) Meta replica does not give any primaryOperationTimeout to primary meta region
[ https://issues.apache.org/jira/browse/HBASE-18035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-18035: --- Summary: Meta replica does not give any primaryOperationTimeout to primary meta region (was: Meta replica does not give any primaryOperationTimeout to primary mete region) > Meta replica does not give any primaryOperationTimeout to primary meta region > - > > Key: HBASE-18035 > URL: https://issues.apache.org/jira/browse/HBASE-18035 > Project: HBase > Issue Type: Bug >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Critical > > I was working on my unittest and it failed with TableNotFoundException. I > debugged a bit and found out that for meta scan, it does not give any > primaryOperationTimeout to primary meta region. This will be an issue as the > meta replica will contain stale data and it is possible that the meta replica > will return back first than primary. > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java#L823 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16432) Revisit the asynchronous ipc implementation
[ https://issues.apache.org/jira/browse/HBASE-16432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16432: --- Fix Version/s: (was: 1.4.0) > Revisit the asynchronous ipc implementation > --- > > Key: HBASE-16432 > URL: https://issues.apache.org/jira/browse/HBASE-16432 > Project: HBase > Issue Type: Umbrella > Components: rpc >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > > Seems the current approach of implementing AsyncTable is in trouble as we > expose some netty classes in our public interfaces. > I agree that we should not do this. The AsyncTable should be implemented > using the asynchronous protobuf stub, and we could use CompletableFuture or > Deferred instead of netty's Future. I think the problem of netty's future is > that it is tighten with netty's executor. This makes it impossible to use > without netty. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16432) Revisit the asynchronous ipc implementation
[ https://issues.apache.org/jira/browse/HBASE-16432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16432: --- Fix Version/s: 1.4.0 > Revisit the asynchronous ipc implementation > --- > > Key: HBASE-16432 > URL: https://issues.apache.org/jira/browse/HBASE-16432 > Project: HBase > Issue Type: Umbrella > Components: rpc >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.4.0 > > > Seems the current approach of implementing AsyncTable is in trouble as we > expose some netty classes in our public interfaces. > I agree that we should not do this. The AsyncTable should be implemented > using the asynchronous protobuf stub, and we could use CompletableFuture or > Deferred instead of netty's Future. I think the problem of netty's future is > that it is tighten with netty's executor. This makes it impossible to use > without netty. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17925) mvn assembly:single fails against hadoop3-alpha2
[
https://issues.apache.org/jira/browse/HBASE-17925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973754#comment-15973754
]
Jonathan Hsieh commented on HBASE-17925:
Thanks for the review Ted. Committed.
> mvn assembly:single fails against hadoop3-alpha2
>
>
> Key: HBASE-17925
> URL: https://issues.apache.org/jira/browse/HBASE-17925
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0, 1.4.0, 1.1.9, 1.2.6, 1.3.2
>
> Attachments: hbase-17925.patch
>
>
> generally to build tarballs against hadoop3 alpha2 we'd use
> {code}
> mvn -Dhadoop.profile=3.0 -Dhadoop.three-version=3.0.0-alpha2 assembly:single
> -DskipTests
> {code}
> This currently fails.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17925) mvn assembly:single fails against hadoop3-alpha2
[
https://issues.apache.org/jira/browse/HBASE-17925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-17925:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 1.3.2
1.2.6
1.1.9
1.4.0
2.0.0
Status: Resolved (was: Patch Available)
> mvn assembly:single fails against hadoop3-alpha2
>
>
> Key: HBASE-17925
> URL: https://issues.apache.org/jira/browse/HBASE-17925
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0, 1.4.0, 1.1.9, 1.2.6, 1.3.2
>
> Attachments: hbase-17925.patch
>
>
> generally to build tarballs against hadoop3 alpha2 we'd use
> {code}
> mvn -Dhadoop.profile=3.0 -Dhadoop.three-version=3.0.0-alpha2 assembly:single
> -DskipTests
> {code}
> This currently fails.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17920) TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2
[ https://issues.apache.org/jira/browse/HBASE-17920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15971156#comment-15971156 ] Jonathan Hsieh commented on HBASE-17920: Looks like the precommit test failure may related to HBASE-16438. [~ramkrishna.s.vasude...@gmail.com] and [~chia7712], do you guys agree? > TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2 > > > Key: HBASE-17920 > URL: https://issues.apache.org/jira/browse/HBASE-17920 > Project: HBase > Issue Type: Sub-task > Components: hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-17920.patch > > > TestFSHDFSUtils always fails against hadoop-3.0.0-alpha1 and > hadoop-3.0.0-alpha2. This is because HDFS-9427 in those versions change the > default nn port from 8020 to 9820. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17920) TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2
[ https://issues.apache.org/jira/browse/HBASE-17920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15971086#comment-15971086 ] Jonathan Hsieh commented on HBASE-17920: Failed TestMemStoreLAB.testLABChunkQueue test is unrelated. Flaky dashboard says it has failed 8/8 times in previous attempts. https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html > TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2 > > > Key: HBASE-17920 > URL: https://issues.apache.org/jira/browse/HBASE-17920 > Project: HBase > Issue Type: Sub-task > Components: hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-17920.patch > > > TestFSHDFSUtils always fails against hadoop-3.0.0-alpha1 and > hadoop-3.0.0-alpha2. This is because HDFS-9427 in those versions change the > default nn port from 8020 to 9820. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17920) TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2
[ https://issues.apache.org/jira/browse/HBASE-17920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-17920: --- Attachment: hbase-17920.patch reattaching. Previous mvn install failures was due to an errant commit that was later reverted. > TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2 > > > Key: HBASE-17920 > URL: https://issues.apache.org/jira/browse/HBASE-17920 > Project: HBase > Issue Type: Sub-task > Components: hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-17920.patch > > > TestFSHDFSUtils always fails against hadoop-3.0.0-alpha1 and > hadoop-3.0.0-alpha2. This is because HDFS-9427 in those versions change the > default nn port from 8020 to 9820. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17920) TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2
[ https://issues.apache.org/jira/browse/HBASE-17920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-17920: --- Attachment: (was: hbase-17920.patch) > TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2 > > > Key: HBASE-17920 > URL: https://issues.apache.org/jira/browse/HBASE-17920 > Project: HBase > Issue Type: Sub-task > Components: hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > > TestFSHDFSUtils always fails against hadoop-3.0.0-alpha1 and > hadoop-3.0.0-alpha2. This is because HDFS-9427 in those versions change the > default nn port from 8020 to 9820. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17922) TestRegionServerHostname always fails against hadoop 3.0.0-alpha2
[
https://issues.apache.org/jira/browse/HBASE-17922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969583#comment-15969583
]
Jonathan Hsieh commented on HBASE-17922:
looks like TestRegionServerHostname#testInvalidRegionServerHostnameAbortsServer
has a problem and then when TestRegionServerHostname#testRegionServerHostname
after it it is in a bad state.
Here's the suspicious lines from the earlier test case
{code}
2017-04-14 14:48:42,285 ERROR [Time-limited test] hbase.MiniHBaseCluster(230):
Error starting cluster
java.lang.RuntimeException: Failed construction of RegionServer: class
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:96)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:183)
at
org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:197)
at
org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:194)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1857)
at
org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:311)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:193)
at
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:222)
at
org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1123)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1077)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:948)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:942)
at
org.apache.hadoop.hbase.regionserver.TestRegionServerHostname.testInvalidRegionServerHostnameAbortsServer(TestRegionServerHostname.java:54)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Failed resolve of
hostAddr.invalid:0
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:1070)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:699)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:563)
at
org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.(MiniHBaseCluster.java:114)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:91)
... 27 more
{code}
> TestRegionServerHostname always fails against hadoop 3.0.0-alpha2
> -
>
> Key: HBASE-17922
> URL: https://issues.apache.org/jira/browse/HBASE-17922
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>
> {code}
> Running org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
> Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 126.363 sec
> <<< FAILURE! - in
> org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
>
[jira] [Commented] (HBASE-17922) TestRegionServerHostname always fails against hadoop 3.0.0-alpha2
[
https://issues.apache.org/jira/browse/HBASE-17922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969580#comment-15969580
]
Jonathan Hsieh commented on HBASE-17922:
Huh, when the test is run along it passes.
{code}
mvn -Dhadoop.profile=3.0 -Dhadoop.three-version=3.0.0-alpha2 clean test
-Dtest=TestRegionServerHostname#testRegionServerHostname
{code}
When the whole class is run it fails.
{code}
mvn -Dhadoop.profile=3.0 -Dhadoop.three-version=3.0.0-alpha2 clean test
-Dtest=TestRegionServerHostname
{code}
> TestRegionServerHostname always fails against hadoop 3.0.0-alpha2
> -
>
> Key: HBASE-17922
> URL: https://issues.apache.org/jira/browse/HBASE-17922
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>
> {code}
> Running org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
> Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 126.363 sec
> <<< FAILURE! - in
> org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
> testRegionServerHostname(org.apache.hadoop.hbase.regionserver.TestRegionServerHostname)
> Time elapsed: 120.029 sec <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 12
> milliseconds
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:221)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:405)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:225)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1123)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1077)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:948)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:942)
> at
> org.apache.hadoop.hbase.regionserver.TestRegionServerHostname.testRegionServerHostname(TestRegionServerHostname.java:88)
> Results :
> Tests in error:
> TestRegionServerHostname.testRegionServerHostname:88 » TestTimedOut test
> timed...
> Tests run: 2, Failures: 0, Errors: 1, Skipped: 0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17920) TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2
[ https://issues.apache.org/jira/browse/HBASE-17920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969575#comment-15969575 ] Jonathan Hsieh commented on HBASE-17920: Thrift failure unrelated (was tested betwee HBASE-17906 and its revert) > TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2 > > > Key: HBASE-17920 > URL: https://issues.apache.org/jira/browse/HBASE-17920 > Project: HBase > Issue Type: Sub-task > Components: hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-17920.patch > > > TestFSHDFSUtils always fails against hadoop-3.0.0-alpha1 and > hadoop-3.0.0-alpha2. This is because HDFS-9427 in those versions change the > default nn port from 8020 to 9820. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17920) TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2
[ https://issues.apache.org/jira/browse/HBASE-17920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-17920: --- Status: Open (was: Patch Available) > TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2 > > > Key: HBASE-17920 > URL: https://issues.apache.org/jira/browse/HBASE-17920 > Project: HBase > Issue Type: Sub-task > Components: hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-17920.patch > > > TestFSHDFSUtils always fails against hadoop-3.0.0-alpha1 and > hadoop-3.0.0-alpha2. This is because HDFS-9427 in those versions change the > default nn port from 8020 to 9820. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17920) TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2
[ https://issues.apache.org/jira/browse/HBASE-17920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-17920: --- Status: Patch Available (was: Open) > TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2 > > > Key: HBASE-17920 > URL: https://issues.apache.org/jira/browse/HBASE-17920 > Project: HBase > Issue Type: Sub-task > Components: hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-17920.patch > > > TestFSHDFSUtils always fails against hadoop-3.0.0-alpha1 and > hadoop-3.0.0-alpha2. This is because HDFS-9427 in those versions change the > default nn port from 8020 to 9820. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17925) mvn assembly:single fails against hadoop3-alpha2
[
https://issues.apache.org/jira/browse/HBASE-17925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-17925:
---
Affects Version/s: 2.0.0
> mvn assembly:single fails against hadoop3-alpha2
>
>
> Key: HBASE-17925
> URL: https://issues.apache.org/jira/browse/HBASE-17925
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-17925.patch
>
>
> generally to build tarballs against hadoop3 alpha2 we'd use
> {code}
> mvn -Dhadoop.profile=3.0 -Dhadoop.three-version=3.0.0-alpha2 assembly:single
> -DskipTests
> {code}
> This currently fails.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17925) mvn assembly:single fails against hadoop3-alpha2
[
https://issues.apache.org/jira/browse/HBASE-17925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-17925:
---
Status: Patch Available (was: Open)
> mvn assembly:single fails against hadoop3-alpha2
>
>
> Key: HBASE-17925
> URL: https://issues.apache.org/jira/browse/HBASE-17925
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop3
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-17925.patch
>
>
> generally to build tarballs against hadoop3 alpha2 we'd use
> {code}
> mvn -Dhadoop.profile=3.0 -Dhadoop.three-version=3.0.0-alpha2 assembly:single
> -DskipTests
> {code}
> This currently fails.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17925) mvn assembly:single fails against hadoop3-alpha2
[
https://issues.apache.org/jira/browse/HBASE-17925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-17925:
---
Attachment: hbase-17925.patch
hadoop 3 profile points to a non-exsitent
src/main/assembly/hadoop-three-compat.xml file. Since we don't have anything
requires use to have a different version yet, changing to point it at the
hadoop-two-compat.xml version.
ran this and it passed.
{code}
mvn -Dhadoop.profile=3.0 -Dhadoop.three-version=3.0.0-alpha2 clean install
assembly:single -DskipTests
{code}
{code}
jon@swoop:~/proj/hbase$ tar tvfz
hbase-assembly/target/hbase-2.0.0-SNAPSHOT-bin.tar.gz | grep hadoop
-rw-r--r-- jon/jon 51125 2017-04-14 14:01
hbase-2.0.0-SNAPSHOT/lib/hbase-hadoop-compat-2.0.0-SNAPSHOT.jar
-rw-r--r-- jon/jon 130771 2017-04-14 14:01
hbase-2.0.0-SNAPSHOT/lib/hbase-hadoop2-compat-2.0.0-SNAPSHOT.jar
-rw-r--r-- jon/jon 3790945 2017-04-11 14:24
hbase-2.0.0-SNAPSHOT/lib/hadoop-common-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 58901 2017-04-11 14:24
hbase-2.0.0-SNAPSHOT/lib/hadoop-annotations-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 132876 2017-04-11 14:24
hbase-2.0.0-SNAPSHOT/lib/hadoop-auth-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 1544875 2016-11-04 14:35
hbase-2.0.0-SNAPSHOT/lib/hadoop-mapreduce-client-core-2.7.1.jar
-rw-r--r-- jon/jon 1654097 2016-11-04 14:35
hbase-2.0.0-SNAPSHOT/lib/hadoop-yarn-common-2.7.1.jar
-rw-r--r-- jon/jon 2669006 2017-04-11 14:25
hbase-2.0.0-SNAPSHOT/lib/hadoop-yarn-api-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 166560 2017-04-11 14:25
hbase-2.0.0-SNAPSHOT/lib/hadoop-distcp-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 42442 2017-04-11 14:25
hbase-2.0.0-SNAPSHOT/lib/hadoop-minicluster-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 148142 2017-04-11 14:25
hbase-2.0.0-SNAPSHOT/lib/hadoop-yarn-server-tests-3.0.0-alpha2-tests.jar
-rw-r--r-- jon/jon 738774 2017-04-11 14:25
hbase-2.0.0-SNAPSHOT/lib/hadoop-yarn-server-common-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 1011522 2017-04-11 14:25
hbase-2.0.0-SNAPSHOT/lib/hadoop-yarn-server-nodemanager-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 1807118 2017-04-11 14:25
hbase-2.0.0-SNAPSHOT/lib/hadoop-yarn-server-resourcemanager-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 296603 2017-04-11 14:25
hbase-2.0.0-SNAPSHOT/lib/hadoop-yarn-server-applicationhistoryservice-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 180753 2017-04-11 14:25
hbase-2.0.0-SNAPSHOT/lib/hadoop-yarn-server-timelineservice-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 78444 2017-04-11 14:25
hbase-2.0.0-SNAPSHOT/lib/hadoop-yarn-server-web-proxy-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 5217677 2017-04-11 14:25
hbase-2.0.0-SNAPSHOT/lib/hadoop-hdfs-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 601570 2017-04-11 14:25
hbase-2.0.0-SNAPSHOT/lib/hadoop-mapreduce-client-app-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 95722 2017-04-11 14:25
hbase-2.0.0-SNAPSHOT/lib/hadoop-mapreduce-client-shuffle-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 80578 2017-04-11 14:25
hbase-2.0.0-SNAPSHOT/lib/hadoop-mapreduce-client-jobclient-3.0.0-alpha2.jar
-rw-r--r-- jon/jon 214772 2017-04-11 14:25
hbase-2.0.0-SNAPSHOT/lib/hadoop-mapreduce-client-hs-3.0.0-alpha2.jar
-rw-r--r-- jon/jon2545 2016-11-04 14:35
hbase-2.0.0-SNAPSHOT/lib/hadoop-client-2.7.1.jar
-rw-r--r-- jon/jon1811 2017-03-16 14:53 hbase-
{code}
> mvn assembly:single fails against hadoop3-alpha2
>
>
> Key: HBASE-17925
> URL: https://issues.apache.org/jira/browse/HBASE-17925
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop3
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-17925.patch
>
>
> generally to build tarballs against hadoop3 alpha2 we'd use
> {code}
> mvn -Dhadoop.profile=3.0 -Dhadoop.three-version=3.0.0-alpha2 assembly:single
> -DskipTests
> {code}
> This currently fails.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17925) mvn assembly:single fails against hadoop3-alpha2
[
https://issues.apache.org/jira/browse/HBASE-17925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-17925:
---
Summary: mvn assembly:single fails against hadoop3-alpha2 (was: mvn
assembly:single fails against hadoop3-alpah2)
> mvn assembly:single fails against hadoop3-alpha2
>
>
> Key: HBASE-17925
> URL: https://issues.apache.org/jira/browse/HBASE-17925
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop3
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>
> generally to build tarballs against hadoop3 alpha2 we'd use
> {code}
> mvn -Dhadoop.profile=3.0 -Dhadoop.three-version=3.0.0-alpha2 assembly:single
> -DskipTests
> {code}
> This currently fails.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (HBASE-17925) mvn assembly:single fails against hadoop3-alpah2
[
https://issues.apache.org/jira/browse/HBASE-17925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh reassigned HBASE-17925:
--
Assignee: Jonathan Hsieh
> mvn assembly:single fails against hadoop3-alpah2
>
>
> Key: HBASE-17925
> URL: https://issues.apache.org/jira/browse/HBASE-17925
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop3
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>
> generally to build tarballs against hadoop3 alpha2 we'd use
> {code}
> mvn -Dhadoop.profile=3.0 -Dhadoop.three-version=3.0.0-alpha2 assembly:single
> -DskipTests
> {code}
> This currently fails.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HBASE-17925) mvn assembly:single fails against hadoop3-alpah2
Jonathan Hsieh created HBASE-17925:
--
Summary: mvn assembly:single fails against hadoop3-alpah2
Key: HBASE-17925
URL: https://issues.apache.org/jira/browse/HBASE-17925
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Hsieh
generally to build tarballs against hadoop3 alpha2 we'd use
{code}
mvn -Dhadoop.profile=3.0 -Dhadoop.three-version=3.0.0-alpha2 assembly:single
-DskipTests
{code}
This currently fails.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17922) TestRegionServerHostname always fails against hadoop 3.0.0-alpha2
[
https://issues.apache.org/jira/browse/HBASE-17922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-17922:
---
Affects Version/s: 2.0.0
> TestRegionServerHostname always fails against hadoop 3.0.0-alpha2
> -
>
> Key: HBASE-17922
> URL: https://issues.apache.org/jira/browse/HBASE-17922
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>
> {code}
> Running org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
> Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 126.363 sec
> <<< FAILURE! - in
> org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
> testRegionServerHostname(org.apache.hadoop.hbase.regionserver.TestRegionServerHostname)
> Time elapsed: 120.029 sec <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 12
> milliseconds
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:221)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:405)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:225)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1123)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1077)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:948)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:942)
> at
> org.apache.hadoop.hbase.regionserver.TestRegionServerHostname.testRegionServerHostname(TestRegionServerHostname.java:88)
> Results :
> Tests in error:
> TestRegionServerHostname.testRegionServerHostname:88 » TestTimedOut test
> timed...
> Tests run: 2, Failures: 0, Errors: 1, Skipped: 0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17922) TestRegionServerHostname always fails against hadoop 3.0.0-alpha2
[
https://issues.apache.org/jira/browse/HBASE-17922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-17922:
---
Description:
{code}
Running org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 126.363 sec <<<
FAILURE! - in org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
testRegionServerHostname(org.apache.hadoop.hbase.regionserver.TestRegionServerHostname)
Time elapsed: 120.029 sec <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 12
milliseconds
at java.lang.Thread.sleep(Native Method)
at
org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:221)
at
org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:405)
at
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:225)
at
org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1123)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1077)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:948)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:942)
at
org.apache.hadoop.hbase.regionserver.TestRegionServerHostname.testRegionServerHostname(TestRegionServerHostname.java:88)
Results :
Tests in error:
TestRegionServerHostname.testRegionServerHostname:88 » TestTimedOut test
timed...
Tests run: 2, Failures: 0, Errors: 1, Skipped: 0
{code}
> TestRegionServerHostname always fails against hadoop 3.0.0-alpha2
> -
>
> Key: HBASE-17922
> URL: https://issues.apache.org/jira/browse/HBASE-17922
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop3
>Reporter: Jonathan Hsieh
>
> {code}
> Running org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
> Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 126.363 sec
> <<< FAILURE! - in
> org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
> testRegionServerHostname(org.apache.hadoop.hbase.regionserver.TestRegionServerHostname)
> Time elapsed: 120.029 sec <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 12
> milliseconds
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:221)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:405)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:225)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1123)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1077)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:948)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:942)
> at
> org.apache.hadoop.hbase.regionserver.TestRegionServerHostname.testRegionServerHostname(TestRegionServerHostname.java:88)
> Results :
> Tests in error:
> TestRegionServerHostname.testRegionServerHostname:88 » TestTimedOut test
> timed...
> Tests run: 2, Failures: 0, Errors: 1, Skipped: 0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HBASE-17922) TestRegionServerHostname always fails against hadoop 3.0.0-alpha2
Jonathan Hsieh created HBASE-17922: -- Summary: TestRegionServerHostname always fails against hadoop 3.0.0-alpha2 Key: HBASE-17922 URL: https://issues.apache.org/jira/browse/HBASE-17922 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17921) TestBlockReorder always fails against hadoop3.0.0-alpha2
Jonathan Hsieh created HBASE-17921:
--
Summary: TestBlockReorder always fails against hadoop3.0.0-alpha2
Key: HBASE-17921
URL: https://issues.apache.org/jira/browse/HBASE-17921
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Hsieh
{code}
java.lang.ExceptionInInitializerError: null
at
org.apache.hadoop.hbase.fs.TestBlockReorder.(TestBlockReorder.java:83)
testBlockLocation(org.apache.hadoop.hbase.fs.TestBlockReorder) Time elapsed:
0.023 sec <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class
org.apache.hadoop.hbase.fs.TestBlockReorder
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:217)
at
org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:266)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
testHBaseCluster(org.apache.hadoop.hbase.fs.TestBlockReorder) Time elapsed:
0.011 sec <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class
org.apache.hadoop.hbase.fs.TestBlockReorder
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:217)
at
org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:266)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
[jira] [Updated] (HBASE-17919) HBase 2.x over hadoop 3.x umbrella
[ https://issues.apache.org/jira/browse/HBASE-17919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-17919: --- Description: We should try to get hbase 2.x branch working against the recently release hadoop 3.0.0 alphas. These days 3.0.0-alpha2 is the latest. HBASE-16733 and HBASE-17593 got the compile level checks in but we should progress to getting unit tests to pass and a build against hadoop3 up. This umbrella issue will capture issues around this project. was: We should try to get hbase 2.x branch working against the recently release hadoop 3.0.0 alphas. These days 3.0.0-alpha2 is the latest. HBASE-16733 and HBASE-17953 got the compile level checks in but we should progress to getting unit tests to pass and a build against hadoop3 up. This umbrella issue will capture issues around this project. > HBase 2.x over hadoop 3.x umbrella > --- > > Key: HBASE-17919 > URL: https://issues.apache.org/jira/browse/HBASE-17919 > Project: HBase > Issue Type: Umbrella > Components: hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh > > We should try to get hbase 2.x branch working against the recently release > hadoop 3.0.0 alphas. These days 3.0.0-alpha2 is the latest. > HBASE-16733 and HBASE-17593 got the compile level checks in but we should > progress to getting unit tests to pass and a build against hadoop3 up. > This umbrella issue will capture issues around this project. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17919) HBase 2.x over hadoop 3.x umbrella
[ https://issues.apache.org/jira/browse/HBASE-17919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-17919: --- Description: We should try to get hbase 2.x branch working against the recently release hadoop 3.0.0 alphas. These days 3.0.0-alpha2 is the latest. HBASE-16733 and HBASE-17953 got the compile level checks in but we should progress to getting unit tests to pass and a build against hadoop3 up. This umbrella issue will capture issues around this project. was: We should try to get hbase 2.x branch working against the recently release hadoop 3.0.0 alphas. These data 3.0.0-alpha2 is the latest. HBASE-16733 and HBASE-17953 got the compile level checks in but we should progress to getting unit tests to pass and a build against hadoop3 up. This umbrella issue will capture issues around this project. > HBase 2.x over hadoop 3.x umbrella > --- > > Key: HBASE-17919 > URL: https://issues.apache.org/jira/browse/HBASE-17919 > Project: HBase > Issue Type: Umbrella > Components: hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh > > We should try to get hbase 2.x branch working against the recently release > hadoop 3.0.0 alphas. These days 3.0.0-alpha2 is the latest. > HBASE-16733 and HBASE-17953 got the compile level checks in but we should > progress to getting unit tests to pass and a build against hadoop3 up. > This umbrella issue will capture issues around this project. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17919) HBase 2.x over hadoop 3.x umbrella
[ https://issues.apache.org/jira/browse/HBASE-17919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-17919: --- Summary: HBase 2.x over hadoop 3.x umbrella (was: HBase 2.x over hadoop 3.x umrella) > HBase 2.x over hadoop 3.x umbrella > --- > > Key: HBASE-17919 > URL: https://issues.apache.org/jira/browse/HBASE-17919 > Project: HBase > Issue Type: Umbrella > Components: hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh > > We should try to get hbase 2.x branch working against the recently release > hadoop 3.0.0 alphas. These data 3.0.0-alpha2 is the latest. > HBASE-16733 and HBASE-17953 got the compile level checks in but we should > progress to getting unit tests to pass and a build against hadoop3 up. > This umbrella issue will capture issues around this project. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17920) TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2
[
https://issues.apache.org/jira/browse/HBASE-17920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-17920:
---
Attachment: hbase-17920.patch
Tested patch with these two command lines and it passed.
{code}
# hadoop 2
mvn -U clean test -Dtest=TestFSHDFSUtils
# hadoop 3.0.0-alpha2
mvn -Dhadoop.profile=3.0 -Dhadoop.three-version=3.0.0-alpha2 -U clean test
-Dtest=TestFSHDFSUtils
{code}
> TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2
>
>
> Key: HBASE-17920
> URL: https://issues.apache.org/jira/browse/HBASE-17920
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: hbase-17920.patch
>
>
> TestFSHDFSUtils always fails against hadoop-3.0.0-alpha1 and
> hadoop-3.0.0-alpha2. This is because HDFS-9427 in those versions change the
> default nn port from 8020 to 9820.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17920) TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2
[ https://issues.apache.org/jira/browse/HBASE-17920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-17920: --- Fix Version/s: 2.0.0 Status: Patch Available (was: Open) > TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2 > > > Key: HBASE-17920 > URL: https://issues.apache.org/jira/browse/HBASE-17920 > Project: HBase > Issue Type: Sub-task > Components: hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-17920.patch > > > TestFSHDFSUtils always fails against hadoop-3.0.0-alpha1 and > hadoop-3.0.0-alpha2. This is because HDFS-9427 in those versions change the > default nn port from 8020 to 9820. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (HBASE-17920) TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2
[ https://issues.apache.org/jira/browse/HBASE-17920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh reassigned HBASE-17920: -- Assignee: Jonathan Hsieh > TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2 > > > Key: HBASE-17920 > URL: https://issues.apache.org/jira/browse/HBASE-17920 > Project: HBase > Issue Type: Sub-task > Components: hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > TestFSHDFSUtils always fails against hadoop-3.0.0-alpha1 and > hadoop-3.0.0-alpha2. This is because HDFS-9427 in those versions change the > default nn port from 8020 to 9820. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17920) TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2
Jonathan Hsieh created HBASE-17920: -- Summary: TestFSHDFSUtils always fails against hadoop 3.0.0-alpha2 Key: HBASE-17920 URL: https://issues.apache.org/jira/browse/HBASE-17920 Project: HBase Issue Type: Sub-task Components: hadoop3 Affects Versions: 2.0.0 Reporter: Jonathan Hsieh TestFSHDFSUtils always fails against hadoop-3.0.0-alpha1 and hadoop-3.0.0-alpha2. This is because HDFS-9427 in those versions change the default nn port from 8020 to 9820. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17919) HBase 2.x over hadoop 3.x umrella
Jonathan Hsieh created HBASE-17919: -- Summary: HBase 2.x over hadoop 3.x umrella Key: HBASE-17919 URL: https://issues.apache.org/jira/browse/HBASE-17919 Project: HBase Issue Type: Umbrella Components: hadoop3 Affects Versions: 2.0.0 Reporter: Jonathan Hsieh We should try to get hadoop 2.x branch working against the recently release hadoop 3.0.0 alphas. These data 3.0.0-alpha2 is the latest. HBASE-16733 and HBASE-17953 got the compile level checks in but we should progress to getting unit tests to pass and a build against hadoop3 up. This umbrella issue will capture issues around this project. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16775) Flakey test with TestExportSnapshot#testExportRetry and TestMobExportSnapshot#testExportRetry
[
https://issues.apache.org/jira/browse/HBASE-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965877#comment-15965877
]
Jonathan Hsieh commented on HBASE-16775:
Hm.. this is pretty subtle. Looking at the test case and if I understand
correctly, we have different export to directories and different table names in
each case and on the surface the test looks to be doing reasonable things to
try isolate concurrent runs of versions of the test. (and failing because mr
tmp dirs are trampling each other).
This makes most important change in the patch the HTU#getDataTestDir call which
calls HTU#setupDataTestDir which setup MR localizing properties[1].
If that is the case I'm +1 for the patch if you add more comments about this
specific trampling scenario in each place where you currently have this comment
"// This will setup separate directory for use in MR cluster."
As a follow up, is there any reason why this should't always be run whenever a
miniMR cluster is setup?
[1]
https://github.com/apache/hbase/blob/master/hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseCommonTestingUtility.java#L102
> Flakey test with TestExportSnapshot#testExportRetry and
> TestMobExportSnapshot#testExportRetry
> --
>
> Key: HBASE-16775
> URL: https://issues.apache.org/jira/browse/HBASE-16775
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: Appy
> Attachments: disable.patch, HBASE-16775.master.001.patch,
> HBASE-16775.master.002.patch, HBASE-16775.master.003.patch,
> HBASE-16775.master.004.patch, HBASE-16775.master.005.patch,
> HBASE-16775.master.006.patch, HBASE-16775.master.007.patch
>
>
> The root cause is that conf.setInt("mapreduce.map.maxattempts", 10) is not
> taken by the mapper job, so the retry is actually 0. Debugging to see why
> this is the case.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-16775) Flakey test with TestExportSnapshot#testExportRetry and TestMobExportSnapshot#testExportRetry
[
https://issues.apache.org/jira/browse/HBASE-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965837#comment-15965837
]
Jonathan Hsieh commented on HBASE-16775:
Hey Appy, to clarify, when you say they are running parallely, do you mean that
TestExportSnapshot#testConsecutiveExport and TestMobExportSnapshot#
testConsecutiveExport were stepping on each other when run in parallel or do
you mean the different test cases within each Test*ExportSnapshot class (e.g.
TestExportSnapshot#testConsecutiveExport and
TestExportSnapshot#testExportFileSystemStateWithSkipTemp ) were stepping on
each other?
I think from the failure report here it is the former case.
https://builds.apache.org/job/PreCommit-HBASE-Build/6337/testReport/
> Flakey test with TestExportSnapshot#testExportRetry and
> TestMobExportSnapshot#testExportRetry
> --
>
> Key: HBASE-16775
> URL: https://issues.apache.org/jira/browse/HBASE-16775
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: Appy
> Attachments: disable.patch, HBASE-16775.master.001.patch,
> HBASE-16775.master.002.patch, HBASE-16775.master.003.patch,
> HBASE-16775.master.004.patch, HBASE-16775.master.005.patch,
> HBASE-16775.master.006.patch, HBASE-16775.master.007.patch
>
>
> The root cause is that conf.setInt("mapreduce.map.maxattempts", 10) is not
> taken by the mapper job, so the retry is actually 0. Debugging to see why
> this is the case.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17224) There are lots of spelling errors in the HBase logging and exception messages
[ https://issues.apache.org/jira/browse/HBASE-17224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-17224: --- Assignee: Grant Sohn > There are lots of spelling errors in the HBase logging and exception messages > - > > Key: HBASE-17224 > URL: https://issues.apache.org/jira/browse/HBASE-17224 > Project: HBase > Issue Type: Bug > Components: Client, io, mapreduce, master, regionserver, security, > wal >Reporter: Grant Sohn >Assignee: Grant Sohn >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-17224.1.patch, hbase-17224.branch-1.patch > > > Found a bunch of spelling errors in log messages and exception messages such > as "Stoping" instead of "Stopping", "alligned" instead of "aligned". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17224) There are lots of spelling errors in the HBase logging and exception messages
[ https://issues.apache.org/jira/browse/HBASE-17224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15712527#comment-15712527 ] Jonathan Hsieh commented on HBASE-17224: Alright. I've committed to master/branch-1 and branch-1.3 but ran out of time to go further back. Thanks [~gsohn]! > There are lots of spelling errors in the HBase logging and exception messages > - > > Key: HBASE-17224 > URL: https://issues.apache.org/jira/browse/HBASE-17224 > Project: HBase > Issue Type: Bug > Components: Client, io, mapreduce, master, regionserver, security, > wal >Reporter: Grant Sohn >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-17224.1.patch, hbase-17224.branch-1.patch > > > Found a bunch of spelling errors in log messages and exception messages such > as "Stoping" instead of "Stopping", "alligned" instead of "aligned". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17224) There are lots of spelling errors in the HBase logging and exception messages
[ https://issues.apache.org/jira/browse/HBASE-17224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-17224: --- Fix Version/s: 1.3.0 > There are lots of spelling errors in the HBase logging and exception messages > - > > Key: HBASE-17224 > URL: https://issues.apache.org/jira/browse/HBASE-17224 > Project: HBase > Issue Type: Bug > Components: Client, io, mapreduce, master, regionserver, security, > wal >Reporter: Grant Sohn >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-17224.1.patch, hbase-17224.branch-1.patch > > > Found a bunch of spelling errors in log messages and exception messages such > as "Stoping" instead of "Stopping", "alligned" instead of "aligned". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17224) There are lots of spelling errors in the HBase logging and exception messages
[ https://issues.apache.org/jira/browse/HBASE-17224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-17224: --- Attachment: hbase-17224.branch-1.patch Attached version committed to branch-1 > There are lots of spelling errors in the HBase logging and exception messages > - > > Key: HBASE-17224 > URL: https://issues.apache.org/jira/browse/HBASE-17224 > Project: HBase > Issue Type: Bug > Components: Client, io, mapreduce, master, regionserver, security, > wal >Reporter: Grant Sohn >Priority: Trivial > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17224.1.patch, hbase-17224.branch-1.patch > > > Found a bunch of spelling errors in log messages and exception messages such > as "Stoping" instead of "Stopping", "alligned" instead of "aligned". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17224) There are lots of spelling errors in the HBase logging and exception messages
[ https://issues.apache.org/jira/browse/HBASE-17224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-17224: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.4.0 2.0.0 Status: Resolved (was: Patch Available) > There are lots of spelling errors in the HBase logging and exception messages > - > > Key: HBASE-17224 > URL: https://issues.apache.org/jira/browse/HBASE-17224 > Project: HBase > Issue Type: Bug > Components: Client, io, mapreduce, master, regionserver, security, > wal >Reporter: Grant Sohn >Priority: Trivial > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17224.1.patch, hbase-17224.branch-1.patch > > > Found a bunch of spelling errors in log messages and exception messages such > as "Stoping" instead of "Stopping", "alligned" instead of "aligned". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17224) There are lots of spelling errors in the HBase logging and exception messages
[ https://issues.apache.org/jira/browse/HBASE-17224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15712449#comment-15712449 ] Jonathan Hsieh commented on HBASE-17224: Looks good to me. Let me do a few quick checks and then I'll commit. > There are lots of spelling errors in the HBase logging and exception messages > - > > Key: HBASE-17224 > URL: https://issues.apache.org/jira/browse/HBASE-17224 > Project: HBase > Issue Type: Bug > Components: Client, io, mapreduce, master, regionserver, security, > wal >Reporter: Grant Sohn >Priority: Trivial > Attachments: HBASE-17224.1.patch > > > Found a bunch of spelling errors in log messages and exception messages such > as "Stoping" instead of "Stopping", "alligned" instead of "aligned". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17166) ITBLL fails on master unable to find hbase-protocol-shaded content
[
https://issues.apache.org/jira/browse/HBASE-17166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15688728#comment-15688728
]
Jonathan Hsieh commented on HBASE-17166:
I did a quick confirmation that ClientProtos was contained in
hbase-protocol-shaded-*.jar.
lgtm. +1.
> ITBLL fails on master unable to find hbase-protocol-shaded content
> --
>
> Key: HBASE-17166
> URL: https://issues.apache.org/jira/browse/HBASE-17166
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-17166.master.001.patch
>
>
> From an internal rig found by [~jmhsieh] running generator step:
> {code}
> 16/11/22 16:51:17 INFO mapreduce.Job: Task Id :
> attempt_1479833370377_0002_m_00_0, Status : FAILED
> Error: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$BlockingInterface
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:225)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:122)
> at
> org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator$GeneratorMapper.setup(IntegrationTestBigLinkedList.java:425)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1790)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17005) Improve log message in MobFileCache
[ https://issues.apache.org/jira/browse/HBASE-17005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630897#comment-15630897 ] Jonathan Hsieh commented on HBASE-17005: The other caches report using '=' instead of ':'. See [1] Mind making it consistent with them? [1] https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java#L510 > Improve log message in MobFileCache > --- > > Key: HBASE-17005 > URL: https://issues.apache.org/jira/browse/HBASE-17005 > Project: HBase > Issue Type: Improvement > Components: mob >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Trivial > Attachments: HBASE-17005.master.001.patch, > HBASE-17005.master.002.patch > > > Currently, the MobFileCache prints "MobFileCache is initialized, and the > cache size is 2". When MobFileCache is enabled, it can print out config > params and it is disabled, it prints explicitly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14551) Procedure v2 - Reimplement split
[ https://issues.apache.org/jira/browse/HBASE-14551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15625256#comment-15625256 ] Jonathan Hsieh commented on HBASE-14551: Hey [~syuanjiang], can you update the release notes with info about the change in semantics that phil brings up? > Procedure v2 - Reimplement split > > > Key: HBASE-14551 > URL: https://issues.apache.org/jira/browse/HBASE-14551 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-14551.v1-master.patch, > HBASE-14551.v2-master.patch, HBASE-14551.v3-master.patch > > > use the proc-v2 state machine for split. also update the logic to have a > single meta-writer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16975) TestSerialReplication occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-16975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15625253#comment-15625253
]
Jonathan Hsieh commented on HBASE-16975:
Instead of sleeping can you get what should be a returned procid and wait for
it to complete?
> TestSerialReplication occasionally fails
>
>
> Key: HBASE-16975
> URL: https://issues.apache.org/jira/browse/HBASE-16975
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Phil Yang
>Priority: Minor
> Attachments: HBASE-16975-v1.patch
>
>
> As seen in
> https://builds.apache.org/job/PreCommit-HBASE-Build/4246/artifact/patchprocess/patch-unit-hbase-server.txt
> :
> {code}
> testRegionMerge(org.apache.hadoop.hbase.replication.TestSerialReplication)
> Time elapsed: 14.763 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.replication.TestSerialReplication.assertIntegerList(TestSerialReplication.java:396)
> at
> org.apache.hadoop.hbase.replication.TestSerialReplication.testRegionMerge(TestSerialReplication.java:326)
> testRegionSplit(org.apache.hadoop.hbase.replication.TestSerialReplication)
> Time elapsed: 203.88 sec <<< ERROR!
> java.lang.Exception: Not all logs have been pushed
> at
> org.apache.hadoop.hbase.replication.TestSerialReplication.testRegionSplit(TestSerialReplication.java:264)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16733) add hadoop 3.0.0-alpha1 to precommit checks
[ https://issues.apache.org/jira/browse/HBASE-16733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15585960#comment-15585960 ] Jonathan Hsieh commented on HBASE-16733: Hey [~Apache9], I'd be find with dropping them but we should bring this up on the dev list so that folks are aware and can chime in on the decision. Can you start the thread? We should also do a corresponding update to this section of the do https://hbase.apache.org/book.html#hadoop to cover 1.4, and 2.0. > add hadoop 3.0.0-alpha1 to precommit checks > --- > > Key: HBASE-16733 > URL: https://issues.apache.org/jira/browse/HBASE-16733 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-16733.v0.patch > > > Been working on getting hadoop3 related build up and running and woudl ike to > add a precommit check so that new commits don't break the mvn compile/install. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16733) add hadoop 3.0.0-alpha1 to precommit checks
[ https://issues.apache.org/jira/browse/HBASE-16733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16733: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the review [~busbey]. I'll watch the next few builds to see how this works out. > add hadoop 3.0.0-alpha1 to precommit checks > --- > > Key: HBASE-16733 > URL: https://issues.apache.org/jira/browse/HBASE-16733 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-16733.v0.patch > > > Been working on getting hadoop3 related build up and running and woudl ike to > add a precommit check so that new commits don't break the mvn compile/install. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16774) [shell] Add coverage to TestShell when ZooKeeper is not reachable
[ https://issues.apache.org/jira/browse/HBASE-16774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15583074#comment-15583074 ] Jonathan Hsieh commented on HBASE-16774: Hey [~esteban], is the TestRSStatusServlet failure here legit or is it a flaky? > [shell] Add coverage to TestShell when ZooKeeper is not reachable > - > > Key: HBASE-16774 > URL: https://issues.apache.org/jira/browse/HBASE-16774 > Project: HBase > Issue Type: Improvement > Components: shell, test >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Attachments: > 0001-HBASE-16774-shell-Add-coverage-to-TestShell-when-Zoo.patch, > 0001-HBASE-16774-shell-Add-coverage-to-TestShell-when-Zoo.patch, > HBASE-16774.001.patch, HBASE-16774.002.patch > > > While testing a couple of things in master I noticed that after some of the > changes done in HBASE-16117 the hbase shell would die when there is no > ZooKeeper server up or if we get another ZK exception. This is to add > coverage to test the shell when ZK is not up or if we get another exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15583067#comment-15583067 ] Jonathan Hsieh commented on HBASE-16712: Will fix whitespaces on commit. Tests shouldn't be affected by this patch. TestAdmin2, TestSnapshotCloneIndependence are on the flaky page. TestHCM was flakey for me locally, but I don't see how this would affect it. > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch, hbase-16712.v2.patch, hbase-16712.v3.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[
https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576887#comment-15576887
]
Jonathan Hsieh commented on HBASE-16712:
In the next version I'll update the hadoop 3 version to 3.0.0-alpha1.
[~busbey] and I did some experiments. The xercesImpl being brought in seems to
be a bug in mvn (I'm using 3.3.9) not propagaing the excludes properly when
using -Dprofile.hadoop=3.0 . If I force the hadoop.profile to use 3.0 by
chaning the pom like below things work.
{code}
diff --git a/pom.xml b/pom.xml
index 3a24622..68600b9 100644
--- a/pom.xml
+++ b/pom.xml
@@ -2046,7 +2046,8 @@
- !hadoop.profile
+ hadoop.profile
+ 2.0
@@ -2237,8 +2238,8 @@
hadoop-3.0
- hadoop.profile
- 3.0
+ !hadoop.profile
+
@@ -2316,7 +2317,7 @@
hadoop-hdfs
${hadoop-three.version}
test-jar
- test
+
javax.servlet.jsp
{code}
the xerces exclude is not needed in the hadoop3.0 profile.
I plan on documenting this in the example.
> fix hadoop-3.0 profile mvn install
> --
>
> Key: HBASE-16712
> URL: https://issues.apache.org/jira/browse/HBASE-16712
> Project: HBase
> Issue Type: Bug
> Components: build, hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: h2-deps, h3-deps, hbase-16712.v0.patch,
> hbase-16712.v1.patch, hbase-16712.v2.patch
>
>
> After the compile is fixed, mvn install fails due to transitive dependencies
> coming from hadoop3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16811) Remove mob sweep job
[ https://issues.apache.org/jira/browse/HBASE-16811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566572#comment-15566572 ] Jonathan Hsieh commented on HBASE-16811: +1 > Remove mob sweep job > > > Key: HBASE-16811 > URL: https://issues.apache.org/jira/browse/HBASE-16811 > Project: HBase > Issue Type: Task >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-16811.master.001.patch > > > Discussed here: > http://mail-archives.apache.org/mod_mbox/hbase-dev/201610.mbox/%3CCAAjhxro%3Dt62K44dV2wUtq1hqYLogZ45M3oeNOFZPcnwcSY4_DQ%40mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16775) Flakey test with TestExportSnapshot#testExportRetry and TestMobExportSnapshot#testExportRetry
[
https://issues.apache.org/jira/browse/HBASE-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15565765#comment-15565765
]
Jonathan Hsieh commented on HBASE-16775:
+1 on disabling patch for now. Mind creating a "disable testExportRetry"
subtask jira and having the commit message for it use that number so we can
keep this open for the true fix?
> Flakey test with TestExportSnapshot#testExportRetry and
> TestMobExportSnapshot#testExportRetry
> --
>
> Key: HBASE-16775
> URL: https://issues.apache.org/jira/browse/HBASE-16775
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
> Attachments: disable.patch
>
>
> The root cause is that conf.setInt("mapreduce.map.maxattempts", 10) is not
> taken by the mapper job, so the retry is actually 0. Debugging to see why
> this is the case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16775) Flakey test with TestExportSnapshot#testExportRetry and TestMobExportSnapshot#testExportRetry
[
https://issues.apache.org/jira/browse/HBASE-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15565765#comment-15565765
]
Jonathan Hsieh edited comment on HBASE-16775 at 10/11/16 3:30 PM:
--
+1 on disabling patch for now. [~appy], mind creating a "disable
testExportRetry" subtask jira and having the commit message for it use that
number so we can keep this open for the true fix?
was (Author: jmhsieh):
+1 on disabling patch for now. Mind creating a "disable testExportRetry"
subtask jira and having the commit message for it use that number so we can
keep this open for the true fix?
> Flakey test with TestExportSnapshot#testExportRetry and
> TestMobExportSnapshot#testExportRetry
> --
>
> Key: HBASE-16775
> URL: https://issues.apache.org/jira/browse/HBASE-16775
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
> Attachments: disable.patch
>
>
> The root cause is that conf.setInt("mapreduce.map.maxattempts", 10) is not
> taken by the mapper job, so the retry is actually 0. Debugging to see why
> this is the case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16767) Mob compaction needs to clean up files in /hbase/mobdir/.tmp and /hbase/mobdir/.tmp/.bulkload when running into IO exceptions
[ https://issues.apache.org/jira/browse/HBASE-16767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15563477#comment-15563477 ] Jonathan Hsieh commented on HBASE-16767: [~tedyu], thanks for catching this. [~busbey] thanks for fixing it. My bad. > Mob compaction needs to clean up files in /hbase/mobdir/.tmp and > /hbase/mobdir/.tmp/.bulkload when running into IO exceptions > -- > > Key: HBASE-16767 > URL: https://issues.apache.org/jira/browse/HBASE-16767 > Project: HBase > Issue Type: Bug > Components: mob >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Fix For: 2.0.0 > > Attachments: HBASE-16767.master.001.patch, > HBASE-16767.master.002.patch, HBASE-16767.master.003.patch, > HBASE-16767.master.004.patch, HBASE-16767.master.004.patch, > HBASE-16767.master.005.patch > > > For the following lines, when it runs into IOException, it does not clean up > the files under /hbase/mobdir/.tmp and /hbase/mobdir/.tmp/.bulkload. > It could be due to creating of new mob file or new reference file under > .bulkload directory fails. Or when mob files from ./tmp to the mob region > directory fails. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java#L433 > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java#L433 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16767) Mob compaction needs to clean up files in /hbase/mobdir/.tmp and /hbase/mobdir/.tmp/.bulkload when running into IO exceptions
[ https://issues.apache.org/jira/browse/HBASE-16767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16767: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Status: Resolved (was: Patch Available) > Mob compaction needs to clean up files in /hbase/mobdir/.tmp and > /hbase/mobdir/.tmp/.bulkload when running into IO exceptions > -- > > Key: HBASE-16767 > URL: https://issues.apache.org/jira/browse/HBASE-16767 > Project: HBase > Issue Type: Bug > Components: mob >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Fix For: 2.0.0 > > Attachments: HBASE-16767.master.001.patch, > HBASE-16767.master.002.patch, HBASE-16767.master.003.patch, > HBASE-16767.master.004.patch, HBASE-16767.master.004.patch, > HBASE-16767.master.005.patch > > > For the following lines, when it runs into IOException, it does not clean up > the files under /hbase/mobdir/.tmp and /hbase/mobdir/.tmp/.bulkload. > It could be due to creating of new mob file or new reference file under > .bulkload directory fails. Or when mob files from ./tmp to the mob region > directory fails. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java#L433 > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java#L433 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16767) Mob compaction needs to clean up files in /hbase/mobdir/.tmp and /hbase/mobdir/.tmp/.bulkload when running into IO exceptions
[ https://issues.apache.org/jira/browse/HBASE-16767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556804#comment-15556804 ] Jonathan Hsieh commented on HBASE-16767: Thanks Huaxiang. I'm committing now. > Mob compaction needs to clean up files in /hbase/mobdir/.tmp and > /hbase/mobdir/.tmp/.bulkload when running into IO exceptions > -- > > Key: HBASE-16767 > URL: https://issues.apache.org/jira/browse/HBASE-16767 > Project: HBase > Issue Type: Bug > Components: mob >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Fix For: 2.0.0 > > Attachments: HBASE-16767.master.001.patch, > HBASE-16767.master.002.patch, HBASE-16767.master.003.patch, > HBASE-16767.master.004.patch, HBASE-16767.master.004.patch, > HBASE-16767.master.005.patch > > > For the following lines, when it runs into IOException, it does not clean up > the files under /hbase/mobdir/.tmp and /hbase/mobdir/.tmp/.bulkload. > It could be due to creating of new mob file or new reference file under > .bulkload directory fails. Or when mob files from ./tmp to the mob region > directory fails. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java#L433 > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java#L433 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16117) Fix Connection leak in mapred.TableOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-16117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16117: --- Resolution: Fixed Fix Version/s: (was: 1.2.5) (was: 1.1.7) (was: 1.3.1) Release Note: (This change will be irrelevant after HBASE-16774 lands). There is a subtle change with error handling when a connection is not able to connect to ZK. Attempts to create a connection when ZK is not up will now fail immediately instead of silently creating and then failing on a subsequent HBaseAdmin call. was:There is a subtle change with error handling when a connection is not able to connect to ZK. Attempts to create a connection when ZK is not up will now fail immediately instead of silently creating and then failing on a subsequent HBaseAdmin call. Status: Resolved (was: Patch Available) As noted earlier, I'm abandoning the attempt to port to 1.x currently. > Fix Connection leak in mapred.TableOutputFormat > > > Key: HBASE-16117 > URL: https://issues.apache.org/jira/browse/HBASE-16117 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.0.0, 1.3.0, 1.2.2, 1.1.6 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: HBASE-16117.branch-1.001.patch, > hbase-16117-part2.v5.patch, hbase-16117.branch-1.002.patch, > hbase-16117.branch-1.patch, hbase-16117.part2.v6.patch, hbase-16117.patch, > hbase-16117.v2.branch-1.patch, hbase-16117.v2.patch, > hbase-16117.v3.branch-1.patch, hbase-16117.v3.patch, hbase-16117.v4.patch > > > Spark seems to instantiate multiple instances of output formats within a > single process. When mapred.TableOutputFormat (not > mapreduce.TableOutputFormat) is used, this may cause connection leaks that > slowly exhaust the cluster's zk connections. > This patch fixes that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16767) Mob compaction needs to clean up files in /hbase/mobdir/.tmp and /hbase/mobdir/.tmp/.bulkload when running into IO exceptions
[ https://issues.apache.org/jira/browse/HBASE-16767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556532#comment-15556532 ] Jonathan Hsieh commented on HBASE-16767: Latest version looks good huaxiang. Thanks! I'll wait for qabot to give a satisfactory return before committing. > Mob compaction needs to clean up files in /hbase/mobdir/.tmp and > /hbase/mobdir/.tmp/.bulkload when running into IO exceptions > -- > > Key: HBASE-16767 > URL: https://issues.apache.org/jira/browse/HBASE-16767 > Project: HBase > Issue Type: Bug > Components: mob >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-16767.master.001.patch, > HBASE-16767.master.002.patch, HBASE-16767.master.003.patch, > HBASE-16767.master.004.patch, HBASE-16767.master.004.patch, > HBASE-16767.master.005.patch > > > For the following lines, when it runs into IOException, it does not clean up > the files under /hbase/mobdir/.tmp and /hbase/mobdir/.tmp/.bulkload. > It could be due to creating of new mob file or new reference file under > .bulkload directory fails. Or when mob files from ./tmp to the mob region > directory fails. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java#L433 > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java#L433 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16774) [shell] Add coverage to TestShell when ZooKeeper is not reachable
[
https://issues.apache.org/jira/browse/HBASE-16774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1823#comment-1823
]
Jonathan Hsieh commented on HBASE-16774:
A few nits in the ruby parts:
{code}
54@Test
55public void testRunNoClusterShellTests() throws IOException {
56 // Start ruby tests without cluster
57 // System.setProperty("shell.test",
"test_connection_no_cluster.rb"); // REMOVE THIS LINE?
58 jruby.runScriptlet(PathType.ABSOLUTE,
"src/test/ruby/no_cluster_tests_runner.rb");
59}
{code}
Add comment near top of no_cluster_tests_runner.rb saying intent of file (e.g.
for testing shell with hbase is not up, pulling in all *no_cluster tests from
dir xxx)
otherwise ruby parts look good to me.
> [shell] Add coverage to TestShell when ZooKeeper is not reachable
> -
>
> Key: HBASE-16774
> URL: https://issues.apache.org/jira/browse/HBASE-16774
> Project: HBase
> Issue Type: Improvement
> Components: shell, test
>Reporter: Esteban Gutierrez
>Assignee: Esteban Gutierrez
> Attachments:
> 0001-HBASE-16774-shell-Add-coverage-to-TestShell-when-Zoo.patch,
> 0001-HBASE-16774-shell-Add-coverage-to-TestShell-when-Zoo.patch,
> HBASE-16774.001.patch
>
>
> While testing a couple of things in master I noticed that after some of the
> changes done in HBASE-16117 the hbase shell would die when there is no
> ZooKeeper server up or if we get another ZK exception. This is to add
> coverage to test the shell when ZK is not up or if we get another exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16179) Fix compilation errors when building hbase-spark against Spark 2.0
[ https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553309#comment-15553309 ] Jonathan Hsieh commented on HBASE-16179: [~tedyu], for the hadoop 3 changes, even though there are two profiles, there is only one set of code and one set of hbase-server binaries that could be used against either hadoop 2.x or hadoop 3.x.The profiles simplify unit testing does not require separate binaries to run against hadoop 2 or hadoop3. The discussion here is about having two different modules and binaries -- one for spark 1.x and one for spark 2.x. If you can you make it one binary the two cases are similar. If you have to make different binaries for each version, it is very different from a release manager's point of view. > Fix compilation errors when building hbase-spark against Spark 2.0 > -- > > Key: HBASE-16179 > URL: https://issues.apache.org/jira/browse/HBASE-16179 > Project: HBase > Issue Type: Bug > Components: spark >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 2.0.0 > > Attachments: 16179.v0.txt, 16179.v1.txt, 16179.v1.txt, 16179.v10.txt, > 16179.v11.txt, 16179.v4.txt, 16179.v5.txt, 16179.v7.txt, 16179.v8.txt, > 16179.v9.txt > > > I tried building hbase-spark module against Spark-2.0 snapshot and got the > following compilation errors: > http://pastebin.com/bg3w247a > Some Spark classes such as DataTypeParser and Logging are no longer > accessible to downstream projects. > hbase-spark module should not depend on such classes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16775) Flakey test with TestExportSnapshot#testExportRetry and TestMobExportSnapshot#testExportRetry
[
https://issues.apache.org/jira/browse/HBASE-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553284#comment-15553284
]
Jonathan Hsieh edited comment on HBASE-16775 at 10/6/16 9:49 PM:
-
Based off of what you are saying it would be better to just always a
deterministic failures when in testing mode so that we always exercise the
failure recovery paths of the manifests.
This probably could be done differently than it is in the current code.
Realistically it might be better to have a fault / failure mid file copy
instead of per map so that we exercise the "sameFile" optimization in the case
of a failure or a retry. [1]
[1]
https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java#L262
was (Author: jmhsieh):
Based off of what you are saying it would be better to just always a
deterministic failures when in testing mode so that we always exercise the
failure recovery paths of the manifests.
This probably could be done differently than it is in the current code -- a
subclass of ExportSnapshot that failes every nth attempt. Realistically it
might be interested to have a fault / failure mid file copy instead of per map
so that we exercise the "sameFile" coptimization in the case of a failure or a
retry. [1]
[1]
https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java#L262
> Flakey test with TestExportSnapshot#testExportRetry and
> TestMobExportSnapshot#testExportRetry
> --
>
> Key: HBASE-16775
> URL: https://issues.apache.org/jira/browse/HBASE-16775
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>
> The root cause is that conf.setInt("mapreduce.map.maxattempts", 10) is not
> taken by the mapper job, so the retry is actually 0. Debugging to see why
> this is the case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16775) Flakey test with TestExportSnapshot#testExportRetry and TestMobExportSnapshot#testExportRetry
[
https://issues.apache.org/jira/browse/HBASE-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553284#comment-15553284
]
Jonathan Hsieh commented on HBASE-16775:
Based off of what you are saying it would be better to just always a
deterministic failures when in testing mode so that we always exercise the
failure recovery paths of the manifests.
This probably could be done differently than it is in the current code -- a
subclass of ExportSnapshot that failes every nth attempt. Realistically it
might be interested to have a fault / failure mid file copy instead of per map
so that we exercise the "sameFile" coptimization in the case of a failure or a
retry. [1]
[1]
https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java#L262
> Flakey test with TestExportSnapshot#testExportRetry and
> TestMobExportSnapshot#testExportRetry
> --
>
> Key: HBASE-16775
> URL: https://issues.apache.org/jira/browse/HBASE-16775
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>
> The root cause is that conf.setInt("mapreduce.map.maxattempts", 10) is not
> taken by the mapper job, so the retry is actually 0. Debugging to see why
> this is the case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552559#comment-15552559 ] Jonathan Hsieh edited comment on HBASE-16712 at 10/6/16 5:17 PM: - v2 changes how we handle the go license, and adds an explicit fail in NOTICE.vm. Doesn't handle xerces yet since I'm still investigating what to do there. was (Author: jmhsieh): v2 changes how we handle the go license, and adds an explicit fail in NOTICE.vm > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch, hbase-16712.v2.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16712: --- Attachment: hbase-16712.v2.patch v2 changes how we handle the go license, and adds an explicit fail in NOTICE.vm > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch, hbase-16712.v2.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16733) add hadoop 3.0.0-alpha1 to precommit checks
[ https://issues.apache.org/jira/browse/HBASE-16733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15551092#comment-15551092 ] Jonathan Hsieh commented on HBASE-16733: at the moment 'mvn clean test -Dhadoop.profile=3.0 -DskipTests' fails due to licensing enforcer issues. This means this is blocked by HBASE-16712 > add hadoop 3.0.0-alpha1 to precommit checks > --- > > Key: HBASE-16733 > URL: https://issues.apache.org/jira/browse/HBASE-16733 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-16733.v0.patch > > > Been working on getting hadoop3 related build up and running and woudl ike to > add a precommit check so that new commits don't break the mvn compile/install. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16776) Remove duplicated versions of countRow() in tests
[ https://issues.apache.org/jira/browse/HBASE-16776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550089#comment-15550089 ] Jonathan Hsieh commented on HBASE-16776: +1 lgtm. > Remove duplicated versions of countRow() in tests > - > > Key: HBASE-16776 > URL: https://issues.apache.org/jira/browse/HBASE-16776 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0 > > Attachments: HBASE-16776-v0.patch > > > A bunch of tests have their copy of countRow() use the ones in TestingUtility -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16774) [shell] Add coverage to TestShell when ZooKeeper is not reachable
[ https://issues.apache.org/jira/browse/HBASE-16774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550081#comment-15550081 ] Jonathan Hsieh edited comment on HBASE-16774 at 10/5/16 10:08 PM: -- The v6 version of the patch in HBASE-16117 fixed this problem with manual testing. Esteban is writting a test case for this. We will combine and submit a patch. was (Author: jmhsieh): The v6 version of the patch in HBASE-16117 fixed this problem with manual testing. Esteban is written a test case for this. We will combine and submit a patch. > [shell] Add coverage to TestShell when ZooKeeper is not reachable > - > > Key: HBASE-16774 > URL: https://issues.apache.org/jira/browse/HBASE-16774 > Project: HBase > Issue Type: Improvement > Components: shell, test >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > > While testing a couple of things in master I noticed that after some of the > changes done in HBASE-16117 the hbase shell would die when there is no > ZooKeeper server up or if we get another ZK exception. This is to add > coverage to test the shell when ZK is not up or if we get another exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16774) [shell] Add coverage to TestShell when ZooKeeper is not reachable
[ https://issues.apache.org/jira/browse/HBASE-16774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550081#comment-15550081 ] Jonathan Hsieh commented on HBASE-16774: The v6 version of the patch in HBASE-16117 fixed this problem with manual testing. Esteban is written a test case for this. We will combine and submit a patch. > [shell] Add coverage to TestShell when ZooKeeper is not reachable > - > > Key: HBASE-16774 > URL: https://issues.apache.org/jira/browse/HBASE-16774 > Project: HBase > Issue Type: Improvement > Components: shell, test >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > > While testing a couple of things in master I noticed that after some of the > changes done in HBASE-16117 the hbase shell would die when there is no > ZooKeeper server up or if we get another ZK exception. This is to add > coverage to test the shell when ZK is not up or if we get another exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16117) Fix Connection leak in mapred.TableOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-16117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549682#comment-15549682 ] Jonathan Hsieh edited comment on HBASE-16117 at 10/5/16 7:18 PM: - [~esteban] found a behavior change in the hbase shell due to the change in error handling. v6 fixes it. (and takes into account other recently landed patches). I'm posting here to get a precommit job working through it. When esteban creates the new issue with the issue that came up, I'll move this to the new issue and close this out (and close out the attempt to backport to 1.x in lieu of a new patch since the original was commited back in june/july) was (Author: jmhsieh): [~esteban] found a behavior change in the hbase shell due to the change in error handling. I'll v6 fixes it. (and takes into account other recently landed patches). I'm posting here to get a precommit job working through it. When esteban creates the new issue with the issue that came up, I'll move this to the new issue and close this out (and close out the attempt to backport to 1.x in lieu of a new patch since the original was commited back in june/july) > Fix Connection leak in mapred.TableOutputFormat > > > Key: HBASE-16117 > URL: https://issues.apache.org/jira/browse/HBASE-16117 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.0.0, 1.3.0, 1.2.2, 1.1.6 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0, 1.3.1, 1.1.7, 1.2.5 > > Attachments: HBASE-16117.branch-1.001.patch, > hbase-16117-part2.v5.patch, hbase-16117.branch-1.002.patch, > hbase-16117.branch-1.patch, hbase-16117.part2.v6.patch, hbase-16117.patch, > hbase-16117.v2.branch-1.patch, hbase-16117.v2.patch, > hbase-16117.v3.branch-1.patch, hbase-16117.v3.patch, hbase-16117.v4.patch > > > Spark seems to instantiate multiple instances of output formats within a > single process. When mapred.TableOutputFormat (not > mapreduce.TableOutputFormat) is used, this may cause connection leaks that > slowly exhaust the cluster's zk connections. > This patch fixes that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16117) Fix Connection leak in mapred.TableOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-16117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549682#comment-15549682 ] Jonathan Hsieh edited comment on HBASE-16117 at 10/5/16 7:19 PM: - [~esteban] found a behavior change in the hbase shell due to the change in error handling. v6 fixes it. (and takes into account other recently landed patches). I'm posting here to get a precommit job working through it. When esteban creates the new issue with the issue that came up, I'll move this to the new issue and close this out (and close out the attempt to backport to 1.x in lieu of a new jira since the original was commited back in june/july) was (Author: jmhsieh): [~esteban] found a behavior change in the hbase shell due to the change in error handling. v6 fixes it. (and takes into account other recently landed patches). I'm posting here to get a precommit job working through it. When esteban creates the new issue with the issue that came up, I'll move this to the new issue and close this out (and close out the attempt to backport to 1.x in lieu of a new patch since the original was commited back in june/july) > Fix Connection leak in mapred.TableOutputFormat > > > Key: HBASE-16117 > URL: https://issues.apache.org/jira/browse/HBASE-16117 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.0.0, 1.3.0, 1.2.2, 1.1.6 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0, 1.3.1, 1.1.7, 1.2.5 > > Attachments: HBASE-16117.branch-1.001.patch, > hbase-16117-part2.v5.patch, hbase-16117.branch-1.002.patch, > hbase-16117.branch-1.patch, hbase-16117.part2.v6.patch, hbase-16117.patch, > hbase-16117.v2.branch-1.patch, hbase-16117.v2.patch, > hbase-16117.v3.branch-1.patch, hbase-16117.v3.patch, hbase-16117.v4.patch > > > Spark seems to instantiate multiple instances of output formats within a > single process. When mapred.TableOutputFormat (not > mapreduce.TableOutputFormat) is used, this may cause connection leaks that > slowly exhaust the cluster's zk connections. > This patch fixes that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16117) Fix Connection leak in mapred.TableOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-16117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16117: --- Attachment: hbase-16117.part2.v6.patch [~esteban] found a behavior change in the hbase shell due to the change in error handling. I'll v6 fixes it. (and takes into account other recently landed patches). I'm posting here to get a precommit job working through it. When esteban creates the new issue with the issue that came up, I'll move this to the new issue and close this out (and close out the attempt to backport to 1.x in lieu of a new patch since the original was commited back in june/july) > Fix Connection leak in mapred.TableOutputFormat > > > Key: HBASE-16117 > URL: https://issues.apache.org/jira/browse/HBASE-16117 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.0.0, 1.3.0, 1.2.2, 1.1.6 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0, 1.3.1, 1.1.7, 1.2.5 > > Attachments: HBASE-16117.branch-1.001.patch, > hbase-16117-part2.v5.patch, hbase-16117.branch-1.002.patch, > hbase-16117.branch-1.patch, hbase-16117.part2.v6.patch, hbase-16117.patch, > hbase-16117.v2.branch-1.patch, hbase-16117.v2.patch, > hbase-16117.v3.branch-1.patch, hbase-16117.v3.patch, hbase-16117.v4.patch > > > Spark seems to instantiate multiple instances of output formats within a > single process. When mapred.TableOutputFormat (not > mapreduce.TableOutputFormat) is used, this may cause connection leaks that > slowly exhaust the cluster's zk connections. > This patch fixes that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16763) Remove unintentional dependency on net.sf.ehcache.search.Results
[ https://issues.apache.org/jira/browse/HBASE-16763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16763: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Status: Resolved (was: Patch Available) > Remove unintentional dependency on net.sf.ehcache.search.Results > > > Key: HBASE-16763 > URL: https://issues.apache.org/jira/browse/HBASE-16763 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-16763.v0.patch > > > HBASE-15638 introduced what looks like an inadvertent dependency on > net.sf.ehcache.search.Results. This removes it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16763) Remove unintentional dependency on net.sf.ehcache.search.Results
[ https://issues.apache.org/jira/browse/HBASE-16763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547067#comment-15547067 ] Jonathan Hsieh commented on HBASE-16763: Thanks for the look see stack. committing. Failures are known flakys [1] and completely unrelated. [1] https://builds.apache.org/job/HBASE-Find-Flaky-Tests/3134/artifact/dashboard.html > Remove unintentional dependency on net.sf.ehcache.search.Results > > > Key: HBASE-16763 > URL: https://issues.apache.org/jira/browse/HBASE-16763 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: hbase-16763.v0.patch > > > HBASE-15638 introduced what looks like an inadvertent dependency on > net.sf.ehcache.search.Results. This removes it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16763) Remove unintentional dependency on net.sf.ehcache.search.Results
[ https://issues.apache.org/jira/browse/HBASE-16763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547067#comment-15547067 ] Jonathan Hsieh edited comment on HBASE-16763 at 10/4/16 11:42 PM: -- Thanks for the look see stack. committing. Failures are known flakys [1] or completely unrelated. [1] https://builds.apache.org/job/HBASE-Find-Flaky-Tests/3134/artifact/dashboard.html was (Author: jmhsieh): Thanks for the look see stack. committing. Failures are known flakys [1] and completely unrelated. [1] https://builds.apache.org/job/HBASE-Find-Flaky-Tests/3134/artifact/dashboard.html > Remove unintentional dependency on net.sf.ehcache.search.Results > > > Key: HBASE-16763 > URL: https://issues.apache.org/jira/browse/HBASE-16763 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: hbase-16763.v0.patch > > > HBASE-15638 introduced what looks like an inadvertent dependency on > net.sf.ehcache.search.Results. This removes it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-16763) Remove unintentional dependency on net.sf.ehcache.search.Results
[ https://issues.apache.org/jira/browse/HBASE-16763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh reassigned HBASE-16763: -- Assignee: Jonathan Hsieh > Remove unintentional dependency on net.sf.ehcache.search.Results > > > Key: HBASE-16763 > URL: https://issues.apache.org/jira/browse/HBASE-16763 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: hbase-16763.v0.patch > > > HBASE-15638 introduced what looks like an inadvertent dependency on > net.sf.ehcache.search.Results. This removes it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16763) Remove unintentional dependency on net.sf.ehcache.search.Results
[ https://issues.apache.org/jira/browse/HBASE-16763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16763: --- Status: Patch Available (was: Open) > Remove unintentional dependency on net.sf.ehcache.search.Results > > > Key: HBASE-16763 > URL: https://issues.apache.org/jira/browse/HBASE-16763 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh > Attachments: hbase-16763.v0.patch > > > HBASE-15638 introduced what looks like an inadvertent dependency on > net.sf.ehcache.search.Results. This removes it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16763) Remove unintentional dependency on net.sf.ehcache.search.Results
[ https://issues.apache.org/jira/browse/HBASE-16763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16763: --- Attachment: hbase-16763.v0.patch > Remove unintentional dependency on net.sf.ehcache.search.Results > > > Key: HBASE-16763 > URL: https://issues.apache.org/jira/browse/HBASE-16763 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh > Attachments: hbase-16763.v0.patch > > > HBASE-15638 introduced what looks like an inadvertent dependency on > net.sf.ehcache.search.Results. This removes it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16763) Remove unintentional dependency on net.sf.ehcache.search.Results
Jonathan Hsieh created HBASE-16763: -- Summary: Remove unintentional dependency on net.sf.ehcache.search.Results Key: HBASE-16763 URL: https://issues.apache.org/jira/browse/HBASE-16763 Project: HBase Issue Type: Bug Reporter: Jonathan Hsieh HBASE-15638 introduced what looks like an inadvertent dependency on net.sf.ehcache.search.Results. This removes it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-16733) add hadoop 3.0.0-alpha1 to precommit checks
[ https://issues.apache.org/jira/browse/HBASE-16733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh reassigned HBASE-16733: -- Assignee: Jonathan Hsieh > add hadoop 3.0.0-alpha1 to precommit checks > --- > > Key: HBASE-16733 > URL: https://issues.apache.org/jira/browse/HBASE-16733 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-16733.v0.patch > > > Been working on getting hadoop3 related build up and running and woudl ike to > add a precommit check so that new commits don't break the mvn compile/install. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16733) add hadoop 3.0.0-alpha1 to precommit checks
[ https://issues.apache.org/jira/browse/HBASE-16733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16733: --- Attachment: hbase-16733.v0.patch v0 - testing via submitting a patch. > add hadoop 3.0.0-alpha1 to precommit checks > --- > > Key: HBASE-16733 > URL: https://issues.apache.org/jira/browse/HBASE-16733 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-16733.v0.patch > > > Been working on getting hadoop3 related build up and running and woudl ike to > add a precommit check so that new commits don't break the mvn compile/install. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16733) add hadoop 3.0.0-alpha1 to precommit checks
Jonathan Hsieh created HBASE-16733: -- Summary: add hadoop 3.0.0-alpha1 to precommit checks Key: HBASE-16733 URL: https://issues.apache.org/jira/browse/HBASE-16733 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0 Reporter: Jonathan Hsieh Fix For: 2.0.0 Been working on getting hadoop3 related build up and running and woudl ike to add a precommit check so that new commits don't break the mvn compile/install. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534064#comment-15534064 ] Jonathan Hsieh commented on HBASE-16712: No reason we shouldn't be able to that I'm aware of. I'll create a new jira to point there and send an email to dev@hbase to give a heads up. > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533939#comment-15533939 ] Jonathan Hsieh commented on HBASE-16712: [~busbey] check out the last commit here -- I've included the h?-deps and a log file with a -X of a 'mvn install -Dhadoop.profile=3.0. https://github.com/jmhsieh/hbase/commits/hbase-16712 > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533241#comment-15533241 ] Jonathan Hsieh commented on HBASE-16712: updated h3-deps to a dependency:tree that acutally has h3 in it. :) The strange thing is that there is no xerces in the tree! If you comment out the xerces bit in the supplemental, you end up with the error I posted earlier. Where else would this dependency get pulled in from? > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16712: --- Attachment: (was: h3-deps) > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16712: --- Attachment: h3-deps > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
