Jon, Thanks for offering to take a look!
Your question prompts me to come up with a bit more explicit rubric for assessment! We want to be sure that the proposed hbase-archetypes infrastructure "works" on at least three levels: 1. it has to be workably consistent with the philosophy and structures of the overall hbase project; 2. it has to be workable for HBase contributors and committers for the full SDLC of current and future archetypes developed using this infrastructure; and 3. the projects that the archetypes ultimately generate for end-users need to be as close to 100% rock-solid, robust, and straightforward as possible. (Using these archetypes could become many people's first hands-on experience with HBase, and it had better result in pleasure and wonder, not disappointment and confusion.) To assess items (1) and (2) above seems to require (a) reading the hbase-archetype README text (which is now rendered here in markdown format <https://github.com/dvimont/test_hbasearchetypes_readme> -- a little easier on the eyes), and (b) potentially inspecting the components of the hbase-archetypes infrastructure described in the README. Specific questions for items (1) and (2): - With regard to the new infrastructure described in the README, does the infrastructure seem robust and workable enough to commit to for present and future development/maintenance of archetypes? - Is the README text readily comprehensible? - Is the README text comprehensive enough in its instructions to contributors, or is something missing that needs to be there? To assess item (3) above requires installing the patch (to a clone of the master branch) in a sandbox, running a "clean install", and following the instructions given in the README under the heading "How to do additional testing/inspection of an archetype in this collection". (Those unfamiliar with Maven archetypes can follow footnotes 1 and 4 in the README for basic information.) Following the instructions there should result in creation of a new Maven project, preconfigured with dependency on hbase-client and test-scope dependency on hbase-testing-util. Specific questions for item (3): - Were any problems encountered in the project generation process? - Does the generated project provide all basic components needed in a simple hbase-client dependent project? - Is the sample code that is provided (in both the main and test subdirectories) clear in its structures and comments, so as to give a good starting foundation to a programmer unfamiliar with the HBase client API? That is all that comes to mind at the moment! Thanks, Dan On Thu, Jan 28, 2016 at 9:53 PM, Jonathan Hsieh <[email protected]> wrote: > I think this is a great dev and new user friendly idea. I saw some jiras > relate to this being created. > > Aside from folks trying it and reviewing, is there anything else you need? > My time is limited but will try to give it a spin in the next week. > > Jon > > On Wednesday, January 27, 2016, Daniel Vimont <[email protected]> > wrote: > > > One point to add to this discussion: the intention is to backport this to > > make it available in an upcoming 1.x minor release, so that new > archetypes > > are publicly available via the Maven central repository as soon as > > possible. > > > > As Sean brings up over in the HBASE-14877 JIRA entry > > <https://issues.apache.org/jira/browse/HBASE-14877>, we need to > determine > > "what level of 'done' we're looking for before we do that". > > > > While I imagine that this will not be anywhere near as involved as the > > backporting of the new hbase-spark stuff to branch-1 (see HBASE-14160 > > <https://issues.apache.org/jira/browse/HBASE-14160>), this patch does > > introduce *new infrastructure* for developing and maintaining new > artifacts > > (in this case, Maven archetypes), so it certainly does merit some review > > before we open up a new JIRA and start adding this functionality to > > branch-1. > > > > On Wed, Jan 27, 2016 at 4:07 PM, Daniel Vimont <[email protected] > > <javascript:;>> wrote: > > > > > Work has been underway for the last month on task HBASE-14877 > > > <https://issues.apache.org/jira/browse/HBASE-14877>, the first subtask > > of > > > HBASE-14876 (in Nick's original wording from that JIRA entry: "To help > > > onboard new users, we should *provide maven archetypes for hbase client > > > applications*"). > > > > > > The three originally proposed *Maven archetypes* in HBASE-14876 were > > > split into three subtasks, for: > > > > > > 1. an hbase-client dependent application, > > > 2. an hbase-shaded-client dependent application, and > > > 3. a map-reduce job example with hbase-server dependency. > > > > > > *(Note that a fourth subtask has since been added -- for an hbase-spark > > > oriented archetype.)* > > > > > > *The first subtask (HBASE-14877) provides:* > > > > > > *[a] a new infrastructure* for creation and maintenance of > HBase-related > > > Maven archetypes, and > > > [*b] the first archetype* from the list above (to be used by end-users > to > > > autogenerate a fully-configured Maven project with hbase-client > > dependency > > > and fully-functional sample Java code). > > > > > > > > > *It is item [a] above -- a new infrastructure for creation/maintenance > of > > > archetypes -- that we would particularly like for > contributors/committers > > > to take a look at, and to offer up comments, questions, suggestions, > > etc.* > > > The README text in the new hbase-archetypes subdirectory is intended to > > be > > > a brief yet comprehensive introduction to how this new infrastructure > > > works, and how a contributor would add new Maven archetypes in the > > future. *(It > > > is intended that it be very simple to contribute new archetypes, > without > > > the need for any special expertise with regard to Maven archetypes in > > > general.)* > > > > > > For ease of access, I'll just paste in the README text below. *(Note > that > > > this text will soon be reformatted into markdown format, but this is > the > > > intended content.)* > > > > > > Besides reviewing this text, you can earn extra gratitude-points if > you: > > > > > > - > > > *set up a sandbox environment, * > > > - > > > *install the latest patch (HBASE-14877-v4.patch > > > < > > > https://issues.apache.org/jira/secure/attachment/12784323/HBASE-14877-v4.patch > > >) > > > against the HBase master branch, * > > > - > > > *follow the instructions at the end of the README (below) to generate a > > > new project from the archetype (emulating the experience of an > > end-user), > > > and * > > > - *review the results of that process, as well!* > > > > > > THANKS VERY MUCH, > > > Dan Vimont > > > > > > > > > > > > > > > ------------------------------------------------------------------------------- > > > *README.txt* > > > > > > > > > ------------------------------------------------------------------------------- > > > *The hbase-archetypes subproject of hbase provides an infrastructure > for* > > > *creation and maintenance of Maven archetypes[1] pertinent to HBase. > > Upon* > > > *deployment to the archetype catalog[2] of the central Maven > > > repository[3],* > > > *these archetypes may be used by end-user developers to autogenerate > > > completely* > > > *configured Maven projects (including fully-functioning sample code) > > > through* > > > *invocation of the archetype:generate goal of the > > > maven-archetype-plugin[4].* > > > > > > *Notes for contributors and committers to the HBase project* > > > *==========================================================* > > > *The hbase-archetypes project contains a separate subproject for each > > > archetype.* > > > *The top level components of such a subproject comprise a complete, > > > standalone* > > > *exemplar Maven project containing (a) a src directory with sample, > > fully-* > > > *functioning code in the ./main and ./test subdirectories, (b) a > pom.xml > > > file* > > > *defining all required dependencies, and (c) any additional resources > > > required* > > > *by the exemplar project. (For example, the components of the > > > hbase-client-* > > > *project consist of (a) sample code ./src/main/.../HelloHBase.java and* > > > *./src/test/.../TestHelloHBase.java, (b) a pom.xml file establishing > > > dependency* > > > *upon hbase-client and test-scope dependency upon hbase-testing-util, > and > > > (c) a* > > > *log4j.properties resource file.)* > > > > > > *** Overview of the creation of archetypes during the hbase install > > > process: *** > > > *During the mvn install process, all standalone exemplar projects in > the* > > > *hbase-archetypes subdirectory are first packaged/tested/installed, and > > > then the* > > > *following steps are executed in > hbase-archetypes/hbase-archetype-builder > > > (via* > > > *the pom.xml, bash scripts, and xsl templates in that subdirectory):* > > > * (1) For each exemplar project, resources are copied (via > > > maven-resources-* > > > *plugin) and transformed (via xml-maven-plugin xslt functionality) to > > the* > > > *exemplar project's ./target/build-archetype subdirectory[5].* > > > * (2) The script createArchetypes.sh is executed to invoke the > > > maven-archetype-* > > > *plugin's create-from-project goal within each exemplar project's* > > > *./target/build-archetype subdirectory. For each exemplar project, this > > > creates* > > > *a corresponding Maven archetype in the ./target/build-* > > > *archetype/target/generate-sources/archetype subdirectory. (Note that > > this > > > step* > > > *always issues two platform-encoding warnings per archetype, due to > > > hard-wired* > > > *behavior of the maven-archetype-plugin[6].)* > > > * (3) The pom.xml file of each newly-created archetype is copied (via > > > maven-* > > > *resources-plugin) and transformed (via xml-maven-plugin xslt > > > functionality)[7].* > > > * (4) The script installArchetypes.sh is executed to install each > > > archetype* > > > *into the local Maven repository, ready for deployment to the central > > > Maven* > > > *repository. (Note that installation of an archetype automatically > > > includes* > > > *invocation of integration-testing prior to install, which performs a > > test* > > > *generation of a project from the archetype.)* > > > > > > *** To add a new archetype to the hbase-archetypes collection: *** > > > * (1) Create a new subdirectory in hbase-archetypes, populated with a* > > > *completely configured Maven project, which will serve as the exemplar > > > project* > > > *of the new archetype. (It may be most straightforward to simply copy > the > > > src* > > > *and pom.xml components from one of the existing exemplar projects, > > > replace the* > > > *src/main and src/test code, and modify the pom.xml file's > > <dependencies>,* > > > *<artifactId>, <name>, and <description> elements.)* > > > * (2) Modify the hbase-archetype-builder/pom.xml file: (a) add the new > > > exemplar* > > > *project to the <modules> element, and (b) add appropriate <execution> > > > elements* > > > *and <transformationSet> elements within the <plugin> elements (using > > the* > > > *existing entries from already-existing exemplar projects as a guide).* > > > * (3) Add appropriate entries for the new exemplar project to the* > > > *createArchetypes.sh and installArchetypes.sh scripts in the > > > hbase-archetype-* > > > *builder subdirectory (using the existing entries as a guide).* > > > > > > *** For additional testing/inspection of an archetype in this > collection: > > > *** > > > *Although integration-testing (which is automatically performed for > each* > > > *archetype during the install process) already performs test generation > > of > > > a* > > > *project from an archetype, it may often be advisable to do further > > manual* > > > *testing of a newly built and installed archetype, particularly to > > examine > > > and* > > > *test a project generated from the archetype (mimicking the end-user > > > experience* > > > *of utilizing the archetype). Upon completion of the install process > > > outlined* > > > *above, all archetypes will have been installed in the local Maven > > > repository* > > > *and can be tested locally by executing the following:* > > > * mvn archetype:generate -DarchetypeCatalog=local* > > > *This displays a numbered list of all locally-installed archetypes for > > the > > > user* > > > *to choose from for generation of a new Maven project.* > > > *=================================================================* > > > *END of notes for contributors and committers to the HBase project* > > > > > > *Footnotes:* > > > *===========================================* > > > *[1] http://maven.apache.org/archetype/index.html > > > <http://maven.apache.org/archetype/index.html>* > > > *[2] http://repo1.maven.org/maven2/archetype-catalog.xml > > > <http://repo1.maven.org/maven2/archetype-catalog.xml>* > > > *[3] http://search.maven.org/ <http://search.maven.org/>* > > > *[4] > > > > > > http://maven.apache.org/archetype/maven-archetype-plugin/generate-mojo.html > > > < > > > http://maven.apache.org/archetype/maven-archetype-plugin/generate-mojo.html > > >* > > > *[5] Prior to archetype creation, each exemplar project's pom.xml is > > > transformed* > > > * as follows to make it into a standalone project: resource > filtering* > > > * replaces ${project.version} with the literal value of the current* > > > * project.version; and xslt tranformation (a) copies <groupId> and > > > <version>* > > > * subelements of <parent> to make them child elements of the root > > > element,* > > > * and (b) removes the <parent> and <description> elements.* > > > *[6] For an explanation of the platform-encoding warning issued during* > > > * maven-archetype-plugin processing, see the first answer to this* > > > * stackoverflow posting: > http://stackoverflow.com/a/24161287/4112172 > > > <http://stackoverflow.com/a/24161287/4112172>* > > > *[7] Prior to archetype installation, each archetype's pom.xml is > > > transformed* > > > * as follows: a <project.build.sourceEncoding> subelement with > value* > > > * 'UTF-8' is added to the <properties> element. This prevents* > > > * platform-encoding warnings from being issued when an end-user > > > generates* > > > * a project from the archetype.* > > > > > > > > > ------------------------------------------------------------------------------- > > > > > > > > -- > // Jonathan Hsieh (shay) > // HBase Tech Lead, Software Engineer, Cloudera > // [email protected] // @jmhsieh >
