My personal experience is that the ivy-maven-stuff introduced into the Hadoop build system has tremendously slowed down the Hadoop build process. I am sure that this disadvantage is offet-ed by some advantages that I am not aware of. If you could educate me on the top two advantages that accrued to Hadoop after moving to the new build process, that would be awesome.
thanks a bunch, dhruba On Sat, Feb 13, 2010 at 1:44 AM, Kay Kay <kaykay.uni...@gmail.com> wrote: > On 02/13/2010 01:29 AM, Dhruba Borthakur wrote: > >> From what I understand, the slowness of 'ivy' can be reduced if you can >> fetch dependent jars from local ivy server, isn't it? >> >> > The problem discussed is an artifact of hbase , trying to keep up with the > most recent snapshots of hadoop-core/ hdfs / mapred and hence the ivy > resolution is expensive that every time it hits the mvn repository to check > for the latest snapshot , if any. So the slowness is due to the necessity > to keep up with the dependencies to identify issues early in the cycle. > Specifically this can be attributed to the - changing="true" in all the > ivy.xml-s in hbase, for hadoop artifacts . I am looking to making it a > configurable option to avoid expensive build time. > > > This will not be an issue if this were a hbase release, depending on other > releases of hadoop-core / mapred / common etc. > Both ivy and maven does cache the artifacts locally making the roundtrip > redundant (except for the first time, of course), so this should not be an > issue for people trying to build the release from sources, since it should > be moot by then. > > > > > > thanks, >> dhruba >> >> On Sat, Feb 13, 2010 at 12:25 AM, Kay Kay<kaykay.uni...@gmail.com> >> wrote: >> >> >> >>> Mathias - >>> I have been using Ivy / Maven , interchangeably in different projects >>> for >>> the build management. Both of them clearly have their strong points and >>> drawbacks. Ivy fits great for thrift because of the nature of tasks , >>> involved using some external command-line (thrift generators) etc. As I >>> mentioned before - HBase does not have such cross maven goals / between >>> the >>> hairs as the build lifecycle is pretty straight-forward. >>> In any case - the intention is to get to publish HBase artifacts and >>> maintain a smaller core and encouraging contribs. from the artifacts as >>> opposed to getting into the codebase. >>> Once there are HBase artifacts published , the contrib / plugins for the >>> same would be free to use ivy (with m2compatible="true") / maven as >>> appropriate. >>> >>> Ryan - >>> The slowness is attributed to the 'changing="true" ' in ivy.xml-s for >>> all >>> the hadoop-common / -hdfs / -mapreduce snapshots that we are using. I am >>> facing similar 'slowness' with other mvn hadoop (snapshot) dependencies >>> as >>> well. In retrospective, that should have been made a configurable flag in >>> libraries.properties , to ease things. Hopefully that is sorted out soon. >>> >>> >>> >>> >>> On 02/13/2010 12:10 AM, Ryan Rawson wrote: >>> >>> >>> >>>> Would you mind elaborating more? At the moment, most people do not >>>> build hbase, and the POM/jar/publishing thing is orthogonal - those >>>> who wish to build their own projects with ivy and/or ant are free to >>>> do so and not be impacted by our use of maven. >>>> >>>> We have ivy, but it doesnt integrate with our IDEs and is rather slow >>>> to build and rebuild. >>>> >>>> On Sat, Feb 13, 2010 at 12:03 AM, Mathias Herberts >>>> <mathias.herbe...@gmail.com> wrote: >>>> >>>> >>>> >>>> >>>>> -1 >>>>> >>>>> I think Maven is too complex and will lower the adoption of HBase by >>>>> people today willing to build it. >>>>> >>>>> I would suggest using Ivy for dependency management as was done in >>>>> Thrift. >>>>> >>>>> Mathias. >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>> >>> >> >> >> > > -- Connect to me at http://www.facebook.com/dhruba
