[
https://issues.apache.org/jira/browse/HADOOP-11929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14565180#comment-14565180
]
Chris Nauroth commented on HADOOP-11929:
----------------------------------------
bq. We could maybe avoid building fuse-dfs, the native mapreduce stuff in
trunk, libhadooppipes, and libwebhdfs unless a file in there had changed. Those
subprojects are truly self-contained so that would work.
I disagree with this statement, at least for the codebase as it stands now.
HDFS-8346 demonstrates that libwebhdfs has a dependency on libhdfs source
files. There also had been an incorrect assumption about being able to call
libhadoop code. fuse-dfs links to native_mini_dfs, so there is a chance that a
change in libhdfs could break it. I'm not sure about libhadooppipes and the
native MR stuff. I haven't looked at them in a while.
A counter-proposal could be to break up some of these unusual dependencies, or
more formally define a "hadoop-native-common" for the sub-projects to
statically link to. For now though, I filed HADOOP-11937 to request full
native builds so that we catch regressions like HDFS-8346 during pre-commit.
> add test-patch plugin points for customizing build layout
> ---------------------------------------------------------
>
> Key: HADOOP-11929
> URL: https://issues.apache.org/jira/browse/HADOOP-11929
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Sean Busbey
> Assignee: Allen Wittenauer
> Priority: Minor
> Attachments: HADOOP-11929.00.patch, HADOOP-11929.01.patch,
> HADOOP-11929.02.patch, HADOOP-11929.03.patch, hadoop.sh
>
>
> Sean Busbey and I had a chat about this at the Bug Bash. Here's the proposal:
> * Introduce the concept of a 'personality module'.
> * There can be only one personality.
> * Personalities provide a single function that takes as input the name of
> the test current being processed
> * This function uses two other built-in functions to define two queues:
> maven module name and profiles to use against those maven module names
> * If something needs to be compiled prior to this test (but not actually
> tested), the personality will be responsible for doing that compilation
> In hadoop, the classic example is hadoop-hdfs needs common compiled with the
> native bits. So prior to the javac tests, the personality would check
> CHANGED_MODULES, see hadoop-hdfs, and compile common w/ -Pnative prior to
> letting test-patch.sh do the work in hadoop-hdfs. Another example is our lack
> of test coverage of various native bits. Since these require profiles to be
> defined prior to compilation, the personality could see that something
> touches native code, set the appropriate profile, and let test-patch.sh be on
> its way.
> One way to think of it is some higher order logic on top of the automated
> 'figure out what modules and what tests to run' functions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
