[jira] [Created] (HBASE-21118) Add Yetus Smart Apply Patch
Jack Bearden created HBASE-21118:
Summary: Add Yetus Smart Apply Patch
Key: HBASE-21118
URL: https://issues.apache.org/jira/browse/HBASE-21118
Project: HBase
Issue Type: New Feature
Reporter: Jack Bearden
Assignee: Jack Bearden
Hadoop trunk has a smart-apply-patch routine contributed by [~aw] and the Yetus
team. It is convenient for quickly pulling down the latest patch tagged with
"Patch Available" and applying it locally.
I think it would be a great addition to HBase and would like to port it over.
Usage:
{code:java}
./dev-support/yetus-smarter-apply-patch.sh --plugins=jira HBASE-123{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21112) [Auth] IPC client fallback to simple auth (forward-port to master)
Jack Bearden created HBASE-21112: Summary: [Auth] IPC client fallback to simple auth (forward-port to master) Key: HBASE-21112 URL: https://issues.apache.org/jira/browse/HBASE-21112 Project: HBase Issue Type: Bug Reporter: Jack Bearden Assignee: Jack Bearden -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21111) [Auth] IPC client fallback to simple auth (forward-port to branch-2)
Jack Bearden created HBASE-2: Summary: [Auth] IPC client fallback to simple auth (forward-port to branch-2) Key: HBASE-2 URL: https://issues.apache.org/jira/browse/HBASE-2 Project: HBase Issue Type: Bug Affects Versions: 2.1.0 Reporter: Jack Bearden Assignee: Jack Bearden -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [DISCUSS] Expanded "work items" for HBase-in-the-Cloud doc
Thanks Josh that was helpful. I'll start doing some of my own research around these topics and look into that Ratis ticket. Much appreciated! On Mon, Aug 6, 2018 at 8:04 AM, Josh Elser wrote: > Yup, replication is a big one to "unravel". Repeating myself from a branch > in the thread, but I'd expect some initial suggestions on what a new API > could be this week. Certainly the first draft won't be the final -- would > be great to get your input after your AsyncWAL work, Duo. > > Using AWS SimpleQueryService, or much anything else, would be great. I > want to make sure that, while we try to "scratch this one itch", we pave > the way for whatever else folks want to experiment with. > > > On 8/4/18 5:10 AM, 张铎(Duo Zhang) wrote: > >> Yes, maybe we could write WAL to SQS and HFile to S3, then we can deploy >> HBase on AWS without any local storage volumes... >> >> But we also need a good abstraction for Replication, as the current design >> is file based... >> >> 2018-07-27 1:28 GMT+08:00 Zach York : >> >> I would REALLY hope that the WAL interface/API changes would go into >>> master >>> even if the feature work for Ratis is going in a feature branch. Not only >>> would this enable other backends to be developed in parallel with the >>> Ratis >>> solution if there are other good fits for a non-HDFS WAL, but also it >>> would >>> save the burden of having to rebase these core changes onto the latest >>> master to maintain compatibility. I'm assuming the Ratis portion of the >>> code would be mostly new files so these would be less of a concern. >>> >>> On Thu, Jul 26, 2018 at 9:24 AM, Josh Elser wrote: >>> >>> On 7/26/18 1:00 AM, Stack wrote: All this said, I'd like to start moving toward the point where we start > >> breaking out this work into a feature-branch off of master and start >> building code. My hope is that this is amenable to everyone, with the >> acknowledge that the Ratis work is considered "experimental" and not >> an >> attempt to make all of HBase use Ratis-backed WALs. >> >> >> Go for it. >> > > The branch would have WAL API changes only or would it include Ratis > WAL > dev? (If the latter, would that be better done over on Ratis project?). > > I think we would start with WAL API changes, get those "blessed", and >>> then >>> continue Ratis WAL dev after that. >>> >>
[jira] [Created] (HBASE-21010) HBase Docker Development Environment
Jack Bearden created HBASE-21010: Summary: HBase Docker Development Environment Key: HBASE-21010 URL: https://issues.apache.org/jira/browse/HBASE-21010 Project: HBase Issue Type: Improvement Reporter: Jack Bearden Assignee: Jack Bearden Hi all, I have been using the following environment (see patch) for conveniently building and testing my HBase patches before they hit precommit. This improvement is a port from Hadoop trunk that was modified to work in our codebase instead. This Linux environment should more closely resembles Jenkins. Usage is simple, just run the script and it will build and run a docker container with your maven cache and hbase directory already set up. From there, you can execute your maven goals as usual. As a kicker, this can also be used to run HBase in docker with low resources to perhaps sniff out and debug flakey tests. Again, not my work, but I was surprised it wasn't in master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [DISCUSS] Expanded "work items" for HBase-in-the-Cloud doc
Great work! I'm excited about this feature and want to help with development. What do you guys suggest are the best topics to ramp up in preparation for these upcoming sprints? On Fri, Aug 3, 2018 at 10:18 AM, Josh Elser wrote: > Yup, we're on the same page :) > > > On 7/26/18 1:28 PM, Zach York wrote: > >> I would REALLY hope that the WAL interface/API changes would go into >> master >> even if the feature work for Ratis is going in a feature branch. Not only >> would this enable other backends to be developed in parallel with the >> Ratis >> solution if there are other good fits for a non-HDFS WAL, but also it >> would >> save the burden of having to rebase these core changes onto the latest >> master to maintain compatibility. I'm assuming the Ratis portion of the >> code would be mostly new files so these would be less of a concern. >> >> On Thu, Jul 26, 2018 at 9:24 AM, Josh Elser wrote: >> >> On 7/26/18 1:00 AM, Stack wrote: >>> >>> All this said, I'd like to start moving toward the point where we start > breaking out this work into a feature-branch off of master and start > building code. My hope is that this is amenable to everyone, with the > acknowledge that the Ratis work is considered "experimental" and not an > attempt to make all of HBase use Ratis-backed WALs. > > > Go for it. > The branch would have WAL API changes only or would it include Ratis WAL dev? (If the latter, would that be better done over on Ratis project?). >>> I think we would start with WAL API changes, get those "blessed", and >>> then >>> continue Ratis WAL dev after that. >>> >>> >>
Re: Question regarding hbase-shell JRuby testing workflow
Great feedback. Thanks all
On Tue, Jul 31, 2018 at 10:59 AM, Sean Busbey wrote:
> check out the configs for the maven-dependency-plugin. My guess would
> be that your call to dependency:build-classpath is getting the
> artifacts needed for runtime scope and not test scope.
>
> On Mon, Jul 30, 2018 at 10:16 PM, Jack Bearden
> wrote:
> > Hey all! I was hacking hbase-shell and JRuby over the weekend and wanted
> to
> > get some feedback on workflow. My objective was to execute a single Ruby
> > unit test in isolation from the TestShell.java class via the jruby
> binary.
> > I was able to accomplish this by doing the following steps:
> >
> > 1. Pulled down branch-2
> > 2. Installed and cleaned via maven at the base directory (mvn
> > -Dmaven.javadoc.skip -DskipTests install)
> > 3. Changed to the hbase-shell directory and exported the classpath (mvn
> > dependency:build-classpath -Dmdep.outputFile=/path/to/cpath.txt)
> > 4. Exported the path to that file to shell env (export
> > TEST_PATH="/path/to/cpath.txt")
> > 5. Hacked tests_runner.rb to just load("path/to/test") for the test I
> > wanted to run
> > 6. From the hbase-shell project directory ran the following:
> >
> > jruby \
> > -J-cp `cat $TEST_PATH` \
> > -d -w \
> > -I src/test/ruby \
> > -I src/main/ruby \
> > src/test/ruby/tests_runner.rb
> >
> > The problem is, is this only worked on *most* of the hbase-shell Ruby
> > tests. The only way to get, for example, list_procedures_test.rb to work
> > completely, was to run it from the TestShell.java file. When ran from the
> > jruby binary, I get a "class not found" when
> > org.apache.hadoop.hbase.client.procedure.ShellTestProcedure.new was
> being
> > referenced. I can't figure out how to load this class adhoc and not
> through
> > what appears to be Maven magic.
> >
> > Any suggestions or better ideas on how to do this?
>
Question regarding hbase-shell JRuby testing workflow
Hey all! I was hacking hbase-shell and JRuby over the weekend and wanted to
get some feedback on workflow. My objective was to execute a single Ruby
unit test in isolation from the TestShell.java class via the jruby binary.
I was able to accomplish this by doing the following steps:
1. Pulled down branch-2
2. Installed and cleaned via maven at the base directory (mvn
-Dmaven.javadoc.skip -DskipTests install)
3. Changed to the hbase-shell directory and exported the classpath (mvn
dependency:build-classpath -Dmdep.outputFile=/path/to/cpath.txt)
4. Exported the path to that file to shell env (export
TEST_PATH="/path/to/cpath.txt")
5. Hacked tests_runner.rb to just load("path/to/test") for the test I
wanted to run
6. From the hbase-shell project directory ran the following:
jruby \
-J-cp `cat $TEST_PATH` \
-d -w \
-I src/test/ruby \
-I src/main/ruby \
src/test/ruby/tests_runner.rb
The problem is, is this only worked on *most* of the hbase-shell Ruby
tests. The only way to get, for example, list_procedures_test.rb to work
completely, was to run it from the TestShell.java file. When ran from the
jruby binary, I get a "class not found" when
org.apache.hadoop.hbase.client.procedure.ShellTestProcedure.new was being
referenced. I can't figure out how to load this class adhoc and not through
what appears to be Maven magic.
Any suggestions or better ideas on how to do this?
[jira] [Created] (HBASE-20918) Re-enable TestRpcHandlerException
Jack Bearden created HBASE-20918: Summary: Re-enable TestRpcHandlerException Key: HBASE-20918 URL: https://issues.apache.org/jira/browse/HBASE-20918 Project: HBase Issue Type: Bug Reporter: Jack Bearden Assignee: Jack Bearden Work done to re-enable this test as part as the HBASE-20266 review. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Requesting Jira contributor status
Also, I just read you guys have a Slack channel. An invite to that would be awesome if possible too, thank you! On Wed, Jul 18, 2018 at 7:23 PM, Jack Bearden wrote: > Hello, > > My name is Jack Bearden and I work on the Altiscale (Hadoop) team at SAP. > I've been contributing to the Hadoop project the last few months but > noticed some HBase issues I could help with also. I cannot upload patches > and was wondering if someone has a minute to fix this for me? > > Jira: jackbearden > > Thanks! >
Requesting Jira contributor status
Hello, My name is Jack Bearden and I work on the Altiscale (Hadoop) team at SAP. I've been contributing to the Hadoop project the last few months but noticed some HBase issues I could help with also. I cannot upload patches and was wondering if someone has a minute to fix this for me? Jira: jackbearden Thanks!
