Hello, With Rahul's excellent work in getting maven build issues resolved, we will be able to get back to feature development. A good release point for the project would be to have all the features of HDT and Hadoop-Eclipse projects working.
The big missing piece of this merge is the existing MapReduce functionality of HDT. I have created https://issues.apache.org/jira/browse/HDT-41 to address this issue. I would like to request assignment of HDT-41 and HDT-40 to me, as I have started work on it. I will upload patches as soon as I have them locally working. On a side note, I would like updating of JIRA's components to include 'common', 'zookeeper', 'core' and 'ui'. Regards, Srimanth On Tue, Sep 10, 2013 at 1:58 AM, Rahul Sharma <[email protected]> wrote: > Hi Adam, > > I have made a fix for the issue at hand using tycho, bundle-classpath and > copy-dependencies. Basically during a maven build I am copying the required > dependencies into a folder. These dependencies are referenced in > MANIFEST.MF using Bundle-classpath. The copying takes place before tycho > kicks in and thus these are available for the compiler. This relieves us > from creation of multiple modules which need to be triggered in multiple > profiles. Also we do not need to create any separate bundles as the > required libs are bundles with-in the plugin. In turn we are quite > portable, without sacrificing any configuration like MANIFEST,classpath or > any of the ecllipse settings. > > I have uploaded the updated patch in HDT-39[1], have a look at it and let > me know your thoughts for the same. > > regards > Rahul > > [1] > > https://issues.apache.org/jira/secure/attachment/12602309/0001-HDT-39-Using-maven-to-build-package.patch > > > > On Tue, Sep 10, 2013 at 8:41 AM, Adam Berry <[email protected]> wrote: > > > Hi Rahul, > > > > So I've been thinking about this. I know that we cannot build the > > dependencies wrapper and the IDE plugins in the same reactor build, which > > rules out the multi module approach. I looked at the current patch is up, > > which makes use of profiles to separate this, as profiles are about build > > portability, I'm not sure thats quite right. > > > > Multi module really represents how this should be done, but as we can't > use > > it I propose that we actually make a new maven project to build the > > dependency wrapper plugins, and then document that to build the plugins > > requires two maven commands (at this time). Building HDT can reference a > p2 > > repository output from the dependency build, and this can take a local > > path. > > > > I will work up a path with this approach, and add a comment on HDT-39 to > > this effect, and we'll see if everyone is ok with this. > > > > Cheers, > > Adam > > > > > > On Tue, Sep 3, 2013 at 10:08 PM, Rahul Sharma <[email protected]> > wrote: > > > > > Hi Adam, > > > > > > This is in relation to issue HDT-39[1] of hadoop-eclipse -merging. I > > > configured things using tycho but Srimanth had a different idea of > > things. > > > This is due to the different approach he took for hadoop-eclipse. I > still > > > think the same can be achieved done using tycho. There is nothing > > stopping > > > us to create specific bundles for different versions of hadoop, which > is > > > being done at other Apache projects like servicemix, provisonr etc. > > > > > > Anyways, the idea was to use maven as dumb possible, just for > > dependencies > > > management, and not to use any of tricks of the trade. I tried to dig > on > > > the possible alternatives but I do not think besides tycho there is any > > > other cleaner approach. In all the rest of the approaches we have to > keep > > > eclipse SDK. Apparently maven and eclipse are two separate worlds that > do > > > not converge easily. > > > > > > Let me know thoughts for the same as I think we would like to use maven > > as > > > that would be enable us to things like managing deps, CI, Release etc. > > > > > > regards, > > > Rahul > > > > > > [1] https://issues.apache.org/jira/browse/HDT-39 > > > > > > > > > On Tue, Sep 3, 2013 at 11:26 PM, Adam Berry <[email protected]> wrote: > > > > > > > Hi Rahul, > > > > > > > > What branch are you trying to do this on? > > > > > > > > On master, where Tycho is used, there is a setting in the pom > pointing > > at > > > > the necessary; > > > > > > > > <repositories> > > > > <repository> > > > > <id>juno</id> > > > > <layout>p2</layout> > > > > <url>http://download.eclipse.org/releases/juno</url> > > > > </repository> > > > > </repositories> > > > > > > > > If you are trying to do the same with all local files you will need > the > > > > necessary p2 repositories locally, and the pom configured to use > them. > > I > > > > think just a copy of the core Eclipse SDK repository and the JDT one > > > should > > > > be sufficient. Then you can just use file: urls to point to the local > > > > spots. > > > > > > > > Or are you trying to not use tycho here and do something else? > > > > > > > > Adam > > > > > > > > > > > > On Tue, Sep 3, 2013 at 10:31 AM, Rahul Sharma <[email protected]> > > > wrote: > > > > > > > > > Hi devs, > > > > > > > > > > I tried doing a maven build this via a simple copy process, where > I > > am > > > > > copying the required dependencies and generation the required > > > packaging. > > > > > But there are some issues in the same. Eclipse does not publish > their > > > > > latest artifacts in maven central or any of the other > > repositories[1]. > > > I > > > > > could only find dependencies for 3.6 version of eclipse core [2]. > > > Other > > > > > related dependencies like swt, Jfaces [2] etc are also of older > > > versions, > > > > > available via maven central. > > > > > > > > > > In order to get around the problem I checked how they are doing it > in > > > > tycho > > > > > plugin, as they use the latest versions there. They are packing > their > > > own > > > > > jars and we could use a few(osgi, runtime etc), which can provide > us > > > the > > > > > latest versions. I am not sure of what to with the rest, if we do > it > > in > > > > > this manner. > > > > > > > > > > Much of the problem is due to the eclipse pde > > > > > libs(core,runtime,osgi,emf,jface,swf etc) that are required for the > > > > build. > > > > > I have also tried maven-pde plugin but that requires eclipse path > of > > > the > > > > > machine. I believe it is not actively developed anymore. > > > > > > > > > > Are there any suggestions ? > > > > > > > > > > regards > > > > > Rahul > > > > > > > > > > [1] https://oss.sonatype.org > > > > > [2] http://mvnrepository.com/artifact/org.eclipse.core > > > > > [3] http://mvnrepository.com/artifact/org.eclipse/jface > > > > > > > > > > > > > > >
