Hi Mirko, Kite addition should help the cause but I am not sure if that is what we want to do that next. There have been number of things that have been proposed in the HDT proposal [1] e.g. supporting MRunit, Pig etc, which can give us next direction. Bob also shared some thoughts around product features in the wiki [2]. I think we could take a leaf out of that as well. IMO we should first try to support existing tooling of Hadoop ecosystem eg MRUnit, Crunch, Pig, Oozie etc. Debugger could be a great add-on to the plugin features. Lets throw in our thoughts to determine whats next ?
For new developers I think the best place to start is tacking one's nagging issues. But maybe we can build a few jiras, provided we do something like release planning, that way new developers would have some idea of where to start-off. regards, Rahul [1] http://wiki.apache.org/incubator/HadoopDevelopmentToolsProposal [2] http://wiki.apache.org/hdt/HDTProductExperience On Mon, Aug 4, 2014 at 5:23 PM, Mirko Kämpf <[email protected]> wrote: > Dear Rahul, > > thanks for preparation of the report. I think we should collect our ideas > about the road map of the project. > > New developers can not see where to start with. So I think this is big > barrier. I was thinking about an > answer, but there is still one question: What is the mission of the > project? > > We want to make software development "against a Hadoop cluster" much > simpler. But how do we achieve this? > > A lot of work is done in the Kite-project. They offer abstractions for > datasets and several tools. > > I suggest we should integrate this kind of functionality into Eclipse via > the HDT project. > > Therefore I work on an example already: > > Kite Dataset Browser > ===================== > One registers a bunch ob Kite-Datasets (no matter if HDFS, Hive, or HBase > data) in a Kite-Datastore. This is an entry like a database in the eclipse > plugin. If I click on the store icon, it shows all registered datasets and > a double click retrieves the 50 first records and shows it in a table, > together with existing metadata. > > Starting here, I think it might make sense to collect such stories in JIRA > to enable new developers to find a place to start. > > I also see, there is a need for a "Morphline" editor and debugger plugin > for Eclipse. > And what about an Oozie connector as the high level connector? This would > allow work with many different cluster, just depending on the Oozie version > for job submission. But anyway the problem of Hadoop versions (or profiles > has to be solved first). I think this is the major problem right now. If we > could switch between different profiles we would be ready for the next step > towards a TLP. > > What are your ideas regarding the Kite development? > > Best wishes > Mirko > > > > > 2014-08-04 12:18 GMT+01:00 Rahul Sharma <[email protected]>: > > > Hi Devs, > > > > We have not get a notification but we are due for report [1] this month. > I > > have compiled the following report. Let me know your thoughts. > > > > regards > > Rahul > > > > [1] https://wiki.apache.org/incubator/August2014 > > > > > > > > ------------------------------------------------------------------------------------------------------------------------ > > > > Hadoop Development Tools > > > > Eclipse based tools for developing applications on the Hadoop platform > > > > Hadoop Development Tools has been incubating since 2012-11-09. > > > > Three most important issues to address in the move towards graduation: > > > > 1. Build Community > > 2. Get more activity > > 3. We have to define and implement an effective approach to manage the > > "client connections" to multiple clusters. > > > > Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be > > aware of? > > > > - None > > > > How has the community developed since the last report? > > > > - 133 mails exchanged on mailing list > > > > How has the project developed since the last report? > > > > - Migrated all the existing functionality from master which enabled us > fro > > next release. > > - Released 0.0.2 incubating > > > > Date of last release: > > > > - July'2014 > > > > When were the last committers or PMC members elected? > > > > - November 2013 > > >
