Here is the new markdown version of the README rendered on a github test page (much easier to read than the raw text was):
https://github.com/dvimont/test_hbasearchetypes_readme On Fri, Jan 29, 2016 at 6:28 AM, Daniel Vimont <[email protected]> wrote: > Hi Nick, > > Perhaps my choice of the word "infrastructure" is rather poor -- > overstating (even *melodramatizing*!) what is there. > > I'm hopeful that when you take a look at it, that the structures and the > reasons that the structures are there will be apparent. (If not, then my > README text failed one of its initial tests, and needs to be reworked to > provide further clarifications.) > > But, in short, everything that is there, I found necessary to be there in > order to (1) facilitate proper automatic testing of *all* components of an > archetype, and (2) provide automation of archetype creation steps that > would otherwise have to manually be done by a contributor and manually > verified by a committer. (The folks at the Maven project provide some > wonderful archetype plugin tools to accomplish a lot of this, and these are > being leveraged to the hilt; but a few details they don't seem to provide > for had to be provided via simple bash scripts and xslt transformation.) > > At this point, I'll let you take a look at things before I attempt to > elaborate further... > > Thanks, > > Dan > > On Fri, Jan 29, 2016 at 2:06 AM, Nick Dimiduk <[email protected]> wrote: > >> Hi Daniel, >> >> Thanks again for picking this one up and really working it! I'm not clear >> on what new infrastructure is needed. The archetypes would be added as new >> modules to our existing maven structure, built when existing code is >> built, >> tests run when existing code tests are run, and deployed the same. Is that >> not so? Let me spend some time with your patches (I know I keep saying >> that...) and come back for a more coherent conversation. >> >> -n >> >> On Wednesday, January 27, 2016, Daniel Vimont <[email protected]> >> wrote: >> >> > One point to add to this discussion: the intention is to backport this >> to >> > make it available in an upcoming 1.x minor release, so that new >> archetypes >> > are publicly available via the Maven central repository as soon as >> > possible. >> > >> > As Sean brings up over in the HBASE-14877 JIRA entry >> > <https://issues.apache.org/jira/browse/HBASE-14877>, we need to >> determine >> > "what level of 'done' we're looking for before we do that". >> > >> > While I imagine that this will not be anywhere near as involved as the >> > backporting of the new hbase-spark stuff to branch-1 (see HBASE-14160 >> > <https://issues.apache.org/jira/browse/HBASE-14160>), this patch does >> > introduce *new infrastructure* for developing and maintaining new >> artifacts >> > (in this case, Maven archetypes), so it certainly does merit some review >> > before we open up a new JIRA and start adding this functionality to >> > branch-1. >> > >> > On Wed, Jan 27, 2016 at 4:07 PM, Daniel Vimont <[email protected] >> > <javascript:;>> wrote: >> > >> > > Work has been underway for the last month on task HBASE-14877 >> > > <https://issues.apache.org/jira/browse/HBASE-14877>, the first >> subtask >> > of >> > > HBASE-14876 (in Nick's original wording from that JIRA entry: "To help >> > > onboard new users, we should *provide maven archetypes for hbase >> client >> > > applications*"). >> > > >> > > The three originally proposed *Maven archetypes* in HBASE-14876 were >> > > split into three subtasks, for: >> > > >> > > 1. an hbase-client dependent application, >> > > 2. an hbase-shaded-client dependent application, and >> > > 3. a map-reduce job example with hbase-server dependency. >> > > >> > > *(Note that a fourth subtask has since been added -- for an >> hbase-spark >> > > oriented archetype.)* >> > > >> > > *The first subtask (HBASE-14877) provides:* >> > > >> > > *[a] a new infrastructure* for creation and maintenance of >> HBase-related >> > > Maven archetypes, and >> > > [*b] the first archetype* from the list above (to be used by >> end-users to >> > > autogenerate a fully-configured Maven project with hbase-client >> > dependency >> > > and fully-functional sample Java code). >> > > >> > > >> > > *It is item [a] above -- a new infrastructure for >> creation/maintenance of >> > > archetypes -- that we would particularly like for >> contributors/committers >> > > to take a look at, and to offer up comments, questions, suggestions, >> > etc.* >> > > The README text in the new hbase-archetypes subdirectory is intended >> to >> > be >> > > a brief yet comprehensive introduction to how this new infrastructure >> > > works, and how a contributor would add new Maven archetypes in the >> > future. *(It >> > > is intended that it be very simple to contribute new archetypes, >> without >> > > the need for any special expertise with regard to Maven archetypes in >> > > general.)* >> > > >> > > For ease of access, I'll just paste in the README text below. *(Note >> that >> > > this text will soon be reformatted into markdown format, but this is >> the >> > > intended content.)* >> > > >> > > Besides reviewing this text, you can earn extra gratitude-points if >> you: >> > > >> > > - >> > > *set up a sandbox environment, * >> > > - >> > > *install the latest patch (HBASE-14877-v4.patch >> > > < >> > >> https://issues.apache.org/jira/secure/attachment/12784323/HBASE-14877-v4.patch >> > >) >> > > against the HBase master branch, * >> > > - >> > > *follow the instructions at the end of the README (below) to generate >> a >> > > new project from the archetype (emulating the experience of an >> > end-user), >> > > and * >> > > - *review the results of that process, as well!* >> > > >> > > THANKS VERY MUCH, >> > > Dan Vimont >> > > >> > > >> > > >> > > >> > >> ------------------------------------------------------------------------------- >> > > *README.txt* >> > > >> > > >> > >> ------------------------------------------------------------------------------- >> > > *The hbase-archetypes subproject of hbase provides an infrastructure >> for* >> > > *creation and maintenance of Maven archetypes[1] pertinent to HBase. >> > Upon* >> > > *deployment to the archetype catalog[2] of the central Maven >> > > repository[3],* >> > > *these archetypes may be used by end-user developers to autogenerate >> > > completely* >> > > *configured Maven projects (including fully-functioning sample code) >> > > through* >> > > *invocation of the archetype:generate goal of the >> > > maven-archetype-plugin[4].* >> > > >> > > *Notes for contributors and committers to the HBase project* >> > > *==========================================================* >> > > *The hbase-archetypes project contains a separate subproject for each >> > > archetype.* >> > > *The top level components of such a subproject comprise a complete, >> > > standalone* >> > > *exemplar Maven project containing (a) a src directory with sample, >> > fully-* >> > > *functioning code in the ./main and ./test subdirectories, (b) a >> pom.xml >> > > file* >> > > *defining all required dependencies, and (c) any additional resources >> > > required* >> > > *by the exemplar project. (For example, the components of the >> > > hbase-client-* >> > > *project consist of (a) sample code ./src/main/.../HelloHBase.java >> and* >> > > *./src/test/.../TestHelloHBase.java, (b) a pom.xml file establishing >> > > dependency* >> > > *upon hbase-client and test-scope dependency upon hbase-testing-util, >> and >> > > (c) a* >> > > *log4j.properties resource file.)* >> > > >> > > *** Overview of the creation of archetypes during the hbase install >> > > process: *** >> > > *During the mvn install process, all standalone exemplar projects in >> the* >> > > *hbase-archetypes subdirectory are first packaged/tested/installed, >> and >> > > then the* >> > > *following steps are executed in >> hbase-archetypes/hbase-archetype-builder >> > > (via* >> > > *the pom.xml, bash scripts, and xsl templates in that subdirectory):* >> > > * (1) For each exemplar project, resources are copied (via >> > > maven-resources-* >> > > *plugin) and transformed (via xml-maven-plugin xslt functionality) to >> > the* >> > > *exemplar project's ./target/build-archetype subdirectory[5].* >> > > * (2) The script createArchetypes.sh is executed to invoke the >> > > maven-archetype-* >> > > *plugin's create-from-project goal within each exemplar project's* >> > > *./target/build-archetype subdirectory. For each exemplar project, >> this >> > > creates* >> > > *a corresponding Maven archetype in the ./target/build-* >> > > *archetype/target/generate-sources/archetype subdirectory. (Note that >> > this >> > > step* >> > > *always issues two platform-encoding warnings per archetype, due to >> > > hard-wired* >> > > *behavior of the maven-archetype-plugin[6].)* >> > > * (3) The pom.xml file of each newly-created archetype is copied (via >> > > maven-* >> > > *resources-plugin) and transformed (via xml-maven-plugin xslt >> > > functionality)[7].* >> > > * (4) The script installArchetypes.sh is executed to install each >> > > archetype* >> > > *into the local Maven repository, ready for deployment to the central >> > > Maven* >> > > *repository. (Note that installation of an archetype automatically >> > > includes* >> > > *invocation of integration-testing prior to install, which performs a >> > test* >> > > *generation of a project from the archetype.)* >> > > >> > > *** To add a new archetype to the hbase-archetypes collection: *** >> > > * (1) Create a new subdirectory in hbase-archetypes, populated with >> a* >> > > *completely configured Maven project, which will serve as the exemplar >> > > project* >> > > *of the new archetype. (It may be most straightforward to simply copy >> the >> > > src* >> > > *and pom.xml components from one of the existing exemplar projects, >> > > replace the* >> > > *src/main and src/test code, and modify the pom.xml file's >> > <dependencies>,* >> > > *<artifactId>, <name>, and <description> elements.)* >> > > * (2) Modify the hbase-archetype-builder/pom.xml file: (a) add the >> new >> > > exemplar* >> > > *project to the <modules> element, and (b) add appropriate <execution> >> > > elements* >> > > *and <transformationSet> elements within the <plugin> elements (using >> > the* >> > > *existing entries from already-existing exemplar projects as a >> guide).* >> > > * (3) Add appropriate entries for the new exemplar project to the* >> > > *createArchetypes.sh and installArchetypes.sh scripts in the >> > > hbase-archetype-* >> > > *builder subdirectory (using the existing entries as a guide).* >> > > >> > > *** For additional testing/inspection of an archetype in this >> collection: >> > > *** >> > > *Although integration-testing (which is automatically performed for >> each* >> > > *archetype during the install process) already performs test >> generation >> > of >> > > a* >> > > *project from an archetype, it may often be advisable to do further >> > manual* >> > > *testing of a newly built and installed archetype, particularly to >> > examine >> > > and* >> > > *test a project generated from the archetype (mimicking the end-user >> > > experience* >> > > *of utilizing the archetype). Upon completion of the install process >> > > outlined* >> > > *above, all archetypes will have been installed in the local Maven >> > > repository* >> > > *and can be tested locally by executing the following:* >> > > * mvn archetype:generate -DarchetypeCatalog=local* >> > > *This displays a numbered list of all locally-installed archetypes for >> > the >> > > user* >> > > *to choose from for generation of a new Maven project.* >> > > *=================================================================* >> > > *END of notes for contributors and committers to the HBase project* >> > > >> > > *Footnotes:* >> > > *===========================================* >> > > *[1] http://maven.apache.org/archetype/index.html >> > > <http://maven.apache.org/archetype/index.html>* >> > > *[2] http://repo1.maven.org/maven2/archetype-catalog.xml >> > > <http://repo1.maven.org/maven2/archetype-catalog.xml>* >> > > *[3] http://search.maven.org/ <http://search.maven.org/>* >> > > *[4] >> > > >> > >> http://maven.apache.org/archetype/maven-archetype-plugin/generate-mojo.html >> > > < >> > >> http://maven.apache.org/archetype/maven-archetype-plugin/generate-mojo.html >> > >* >> > > *[5] Prior to archetype creation, each exemplar project's pom.xml is >> > > transformed* >> > > * as follows to make it into a standalone project: resource >> filtering* >> > > * replaces ${project.version} with the literal value of the >> current* >> > > * project.version; and xslt tranformation (a) copies <groupId> and >> > > <version>* >> > > * subelements of <parent> to make them child elements of the root >> > > element,* >> > > * and (b) removes the <parent> and <description> elements.* >> > > *[6] For an explanation of the platform-encoding warning issued >> during* >> > > * maven-archetype-plugin processing, see the first answer to this* >> > > * stackoverflow posting: >> http://stackoverflow.com/a/24161287/4112172 >> > > <http://stackoverflow.com/a/24161287/4112172>* >> > > *[7] Prior to archetype installation, each archetype's pom.xml is >> > > transformed* >> > > * as follows: a <project.build.sourceEncoding> subelement with >> value* >> > > * 'UTF-8' is added to the <properties> element. This prevents* >> > > * platform-encoding warnings from being issued when an end-user >> > > generates* >> > > * a project from the archetype.* >> > > >> > > >> > >> ------------------------------------------------------------------------------- >> > > >> > >> > >
