[
https://issues.apache.org/jira/browse/FLINK-838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097916#comment-14097916
]
Artem Tsikiridis edited comment on FLINK-838 at 8/14/14 11:46 PM:
------------------------------------------------------------------
When working on Hadoop's {{JobClient}} api, at a few points the diferences
between hadoop 1.x and 2.x become very visible (different, non-compatible
interfaces etc.) and this does not make Travis happy ( if it is possible to
build the project at all).
I'm wondering what's the best thing to do here.
1) Try to implement both, by going as low-level as possible for the second
version of the API and trying to eventually use mutual code.
2) Support just one of them
3) This might sound a bit stupid, but is there any way to depend on maven
profiles to turn on / off the guilty parts? Or a programmatic way to do
something similar.
Thanks!
was (Author: atsikiridis):
When working on Hadoop's {{JobClient}} api, at a few points the diferences
between hadoop 1.x and 2.x become very visible (different interfaces etc.) and
this does not make Travis happy ( if it is possible to build the project at
all).
I'm wondering what's the best thing to do here.
1) Try to implement both, by going as low-level as possible for the second
version of the API and trying to eventually use mutual code.
2) Support just one of them
3) This might sound a bit stupid, but is there any way to depend on maven
profiles to turn on / off the guilty parts? Or a programmatic way to do
something similar.
Thanks!
> GSoC Summer Project: Implement full Hadoop Compatibility Layer for
> Stratosphere
> -------------------------------------------------------------------------------
>
> Key: FLINK-838
> URL: https://issues.apache.org/jira/browse/FLINK-838
> Project: Flink
> Issue Type: Improvement
> Reporter: GitHub Import
> Labels: github-import
> Fix For: pre-apache
>
>
> This is a meta issue for tracking @atsikiridis progress with implementing a
> full Hadoop Compatibliltiy Layer for Stratosphere.
> Some documentation can be found in the Wiki:
> https://github.com/stratosphere/stratosphere/wiki/%5BGSoC-14%5D-A-Hadoop-abstraction-layer-for-Stratosphere-(Project-Map-and-Notes)
> As well as the project proposal:
> https://github.com/stratosphere/stratosphere/wiki/GSoC-2014-Project-Proposal-Draft-by-Artem-Tsikiridis
> Most importantly, there is the following **schedule**:
> *19 May - 27 June (Midterm)*
> 1) Work on the Hadoop tasks, their Context and the mapping of Hadoop's
> Configuration to the one of Stratosphere. By successfully bridging the Hadoop
> tasks with Stratosphere, we already cover the most basic Hadoop Jobs. This
> can be determined by running some popular Hadoop examples on Stratosphere
> (e.g. WordCount, k-means, join) (4 - 5 weeks)
> 2) Understand how the running of these jobs works (e.g. command line
> interface) for the wrapper. Implement how will the user run them. (1 - 2
> weeks).
> *27 June - 11 August*
> 1) Continue wrapping more "advanced" Hadoop Interfaces (Comparators,
> Partitioners, Distributed Cache etc.) There are quite a few interfaces and it
> will be a challenge to support all of them. (5 full weeks)
> 2) Profiling of the application and optimizations (if applicable)
> *11 August - 18 August*
> Write documentation on code, write a README with care and add more
> unit-tests. (1 week)
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/stratosphere/stratosphere/issues/838
> Created by: [rmetzger|https://github.com/rmetzger]
> Labels: core, enhancement, parent-for-major-feature,
> Milestone: Release 0.7 (unplanned)
> Created at: Tue May 20 10:11:34 CEST 2014
> State: open
--
This message was sent by Atlassian JIRA
(v6.2#6252)
