On Fri, Jun 22, 2012 at 11:48 AM, Rohini Palaniswamy < [email protected]> wrote:
> Replying to the dev list instead of the jira as this requires everybody's > attention/thoughts. > > I would prefer all at once for mavenization. All the ivy stuff in ant > will be throwaway in maven. The directory structure would require good > amount of reorganizing (properly modularizing, moving code to standard > directory structure src/main/java, src/test/java, src/test/resources). > Trying to do it incrementally, moving out code and creating sub-projects is > ok, but the effort spent on changing ant scripts for that might be a waste > of effort as it is going to be thrown away. It would be better to have the > build in broken state for few days and complete mavenization faster instead > of trying to keep ant working when we move to maven. Since hcatalog is not > that complex, mavenizing should be easy and quick except for the e2e tests. > Even that can be quickly done with maven ant plugin before completely > moving to maven. > I'm down with this plan. Maybe have a hack day and crank this out? Anyone around SF who wants to work on this? I could host at the new Twitter diggs. If there's interest I can see how hosting people works and we can figure out a date. --travis > > Regards, > Rohini > > > > On Fri, Jun 22, 2012 at 11:14 AM, Travis Crawford < > [email protected]> wrote: > >> This is an automatically generated e-mail. To reply, visit: >> https://reviews.apache.org/r/5496/ >> >> On June 22nd, 2012, 5:21 p.m., *Rohini Palaniswamy* wrote: >> >> >> http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/build.xml<https://reviews.apache.org/r/5496/diff/2/?file=115880#file115880line329> >> (Diff >> revision 2) >> >> 329 >> >> <antcall target="ivy-publish"/> >> >> Do we want to do ivy-publish inside the jar target. Would prefer it to be >> called separately in only apache hudson build. ivy-publish can probably >> depend on jar. All of us run jar in our hosts. Putting ivy-publish here >> might lead to issues. >> >> On June 22nd, 2012, 5:27 p.m., *Travis Crawford* wrote: >> >> So this publishes to the local ivy cache, which is needed because >> hcatalog-pig-adapter depends on hcatalog.jar. Otherwise the subproject >> dependency does not work. >> >> On June 22nd, 2012, 6 p.m., *Rohini Palaniswamy* wrote: >> >> Good then. Thought that it publishes a snapshot jar to the maven repo. >> Should have paid more attention to the "local" keyword there. >> >> What are your thoughts on this general approach of reorganizing the repo to >> match the maven layout, then we can mavenize? Are you okay with this >> incremental approach, or do you think it would be better all at once? >> >> Looking at the current build, I think we also want subprojects for: >> >> * hcatalog-server-extensions.jar >> * hcatalog-core.jar (today just hcatalog.jar; we could keep this just >> hcatalog.jar if people like it) >> >> I don't fully understand the storage-handler stuff, since doesn't that stuff >> belong in Hive? For now I was planning to leave as-is. >> >> Let me know if you want to do this all at once, or incremental. >> >> >> - Travis >> >> On June 22nd, 2012, 3:45 p.m., Travis Crawford wrote: >> Review request for hcatalog. >> By Travis Crawford. >> >> *Updated June 22, 2012, 3:45 p.m.* >> Description >> >> Update HCatalog build to package pig classes as a separate jar. I did not >> update ivy yet, but if the general approach looks good I will update. This >> will let the core hcatalog.jar depends only on stuff needed by all >> processing frameworks; then people that want to use pig can use the pig >> adapter which has the pig dependency. >> >> I believe we'll have more adapters in the future, so I'm trying to make this >> reusable. >> >> For example: >> >> Traviss-iMac:hcatalog travis$ jar -tvf >> hcatalog-pig-adapter/build/hcatalog-pig-adapter-0.5.0-dev.jar >> 0 Thu Jun 21 10:34:18 PDT 2012 META-INF/ >> 107 Thu Jun 21 10:34:16 PDT 2012 META-INF/MANIFEST.MF >> 0 Thu Jun 21 10:34:16 PDT 2012 org/ >> 0 Thu Jun 21 10:34:16 PDT 2012 org/apache/ >> 0 Thu Jun 21 10:34:16 PDT 2012 org/apache/hcatalog/ >> 0 Thu Jun 21 10:34:16 PDT 2012 org/apache/hcatalog/pig/ >> 0 Thu Jun 21 10:34:16 PDT 2012 org/apache/hcatalog/pig/drivers/ >> 4352 Thu Jun 21 10:34:16 PDT 2012 >> org/apache/hcatalog/pig/HCatBaseLoader.class >> 1261 Thu Jun 21 10:34:16 PDT 2012 >> org/apache/hcatalog/pig/HCatBaseStorer$1.class >> 12413 Thu Jun 21 10:34:16 PDT 2012 >> org/apache/hcatalog/pig/HCatBaseStorer.class >> 632 Thu Jun 21 10:34:16 PDT 2012 >> org/apache/hcatalog/pig/HCatLoader$1.class >> 8518 Thu Jun 21 10:34:16 PDT 2012 org/apache/hcatalog/pig/HCatLoader.class >> 6801 Thu Jun 21 10:34:16 PDT 2012 org/apache/hcatalog/pig/HCatStorer.class >> 1019 Thu Jun 21 10:34:16 PDT 2012 >> org/apache/hcatalog/pig/PigHCatUtil$1.class >> 13117 Thu Jun 21 10:34:16 PDT 2012 org/apache/hcatalog/pig/PigHCatUtil.class >> 3711 Thu Jun 21 10:34:16 PDT 2012 >> org/apache/hcatalog/pig/drivers/LoadFuncBasedInputFormat$LoadFuncBasedRecordReader.class >> 2383 Thu Jun 21 10:34:16 PDT 2012 >> org/apache/hcatalog/pig/drivers/LoadFuncBasedInputFormat.class >> 2189 Thu Jun 21 10:34:16 PDT 2012 >> org/apache/hcatalog/pig/drivers/StoreFuncBasedOutputFormat$StoreFuncBasedOutputCommitter.class >> 1775 Thu Jun 21 10:34:16 PDT 2012 >> org/apache/hcatalog/pig/drivers/StoreFuncBasedOutputFormat$StoreFuncBasedRecordWriter.class >> 2647 Thu Jun 21 10:34:16 PDT 2012 >> org/apache/hcatalog/pig/drivers/StoreFuncBasedOutputFormat.class >> Traviss-iMac:hcatalog travis$ >> >> Diffs >> >> - >> >> http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/build-common-new.xml >> (PRE-CREATION) >> - http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/build.xml >> (1352540) >> - >> >> http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/hcatalog-pig-adapter/build.xml >> (PRE-CREATION) >> - >> >> http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/hcatalog-pig-adapter/ivy.xml >> (PRE-CREATION) >> - http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/ivy.xml >> (1352540) >> >> View Diff <https://reviews.apache.org/r/5496/diff/> >> > >
