Hello Colm,
Interesting, I think of this patch as an maintenance upgrade to Knox 1.0.0
build, this should not affect the way Knox works, infact there was just one
line trivial code change in this patch. The rational behind this is to
upgrade Hadoop dependencies that Knox uses to be in-line with the latest
Hadoop release so we can easily leverage new features and improvements
going forward. Also, feature wise 1.0.0 is still close to 0.14.0, this
patch does not add, remove or improve any feature.
About jackson 2.2.2 in root pom, you might be right, this could be an
artifact of my testing, let me try removing it again and see.
The Kerby version rolled with Hadoop 3 has some significant api changes,
which if included would likely have us revisit the SecureClusterTest code.
So the patch uses the version of Kerby that does not need code changes to
our test classes. My aim was to keep Knox code changes as minimum as
possible.
"Beanutils is downgraded from 1.9.3 to 1.9.2."
On the contrary Beanutils was upgraded from 1.9.2 to 1.9.3 [1]
- <version>1.9.2</version>
+ <version>${commons-beanutils-version}</version>
"Also we are defining lower versions of Jackson and Mockito in the
gateway-test-release pom."
Do you mean lower version than before ? I do recall I had trouble with
higher version of jackson with our tests. Through trial and error I found
these were the versions that played nice with our tests.
I decided to stick with these versions since I did not think changing the
Unit tests was a good idea at this point.
Thanks for taking a look at the patch and let me know your thoughts.
Best,
Sandeep
[1]
https://git1-us-west.apache.org/repos/asf?p=knox.git;a=blob;f=pom.xml;h=6bd93962f30fb52fb2273ba5294bae3140eb9119;hb=772bc33d#l125
On Mon, Jan 8, 2018 at 4:52 AM, Colm O hEigeartaigh <cohei...@apache.org>
wrote:
> Hi Sandeep,
>
> I'm wondering if this (major) upgrade is not breaking the idea of trying to
> change as little as possible for 1.0.0 from 0.14.0?
>
> I think "jackson2.version" added to the root pom with version 2.2.2 could
> be removed, as it is not used anywhere. Why do we exclude Kerby from the
> Hadoop pom and then include it as a dependency again?
>
> Also, is it really necessary to "downgrade" various dependencies just
> because they are used by Hadoop? Beanutils is downgraded from 1.9.3 to
> 1.9.2. Also we are defining lower versions of Jackson and Mockito in the
> gateway-test-release pom.
>
> Colm.
>
> On Fri, Jan 5, 2018 at 7:40 PM, <m...@apache.org> wrote:
>
> > Repository: knox
> > Updated Branches:
> > refs/heads/master 6d4756f3d -> 772bc33d4
> >
> >
> > KNOX-1161 - Update hadoop dependencies to hadoop 3
> >
> >
> > Project: http://git-wip-us.apache.org/repos/asf/knox/repo
> > Commit: http://git-wip-us.apache.org/repos/asf/knox/commit/772bc33d
> > Tree: http://git-wip-us.apache.org/repos/asf/knox/tree/772bc33d
> > Diff: http://git-wip-us.apache.org/repos/asf/knox/diff/772bc33d
> >
> > Branch: refs/heads/master
> > Commit: 772bc33d48dd37be5cd098992df3e50db24326f3
> > Parents: 6d4756f
> > Author: Sandeep More <m...@apache.org>
> > Authored: Fri Jan 5 14:38:24 2018 -0500
> > Committer: Sandeep More <m...@apache.org>
> > Committed: Fri Jan 5 14:38:24 2018 -0500
> >
> > ----------------------------------------------------------------------
> > .../filter/PortMappingHelperHandler.java | 2 +-
> > gateway-test-release/pom.xml | 77
> ++++++++++++++++++++
> > pom.xml | 31 +++++++-
> > 3 files changed, 107 insertions(+), 3 deletions(-)
> > ----------------------------------------------------------------------
> >
> >
> > http://git-wip-us.apache.org/repos/asf/knox/blob/772bc33d/
> > gateway-server/src/main/java/org/apache/hadoop/gateway/filter/
> > PortMappingHelperHandler.java
> > ----------------------------------------------------------------------
> > diff --git a/gateway-server/src/main/java/org/apache/hadoop/
> > gateway/filter/PortMappingHelperHandler.java b/gateway-server/src/main/
> > java/org/apache/hadoop/gateway/filter/PortMappingHelperHandler.java
> > index ea3efc4..06d9668 100644
> > --- a/gateway-server/src/main/java/org/apache/hadoop/gateway/filter/
> > PortMappingHelperHandler.java
> > +++ b/gateway-server/src/main/java/org/apache/hadoop/gateway/filter/
> > PortMappingHelperHandler.java
> > @@ -96,7 +96,7 @@ public class PortMappingHelperHandler extends
> > HandlerWrapper {
> > throws IOException, ServletException {
> >
> > String newTarget = target;
> > - String baseURI = baseRequest.getUri().toString();
> > + String baseURI = baseRequest.getRequestURI();
> >
> > // If Port Mapping feature enabled
> > if (config.isGatewayPortMappingEnabled()) {
> >
> > http://git-wip-us.apache.org/repos/asf/knox/blob/772bc33d/
> > gateway-test-release/pom.xml
> > ----------------------------------------------------------------------
> > diff --git a/gateway-test-release/pom.xml b/gateway-test-release/pom.xml
> > index e61e0c8..48def02 100644
> > --- a/gateway-test-release/pom.xml
> > +++ b/gateway-test-release/pom.xml
> > @@ -34,7 +34,62 @@
> > <module>webhdfs-test</module>
> > </modules>
> >
> > + <properties>
> > + <jetty.version>9.3.19.v20170502</jetty.version>
> > + <mockito.version>1.8.4</mockito.version>
> > + <jackson2.version>2.7.8</jackson2.version>
> > + </properties>
> > +
> > +
> > <dependencies>
> > + <!-- Hadoop 3.0 Deps. -->
> > + <dependency>
> > + <groupId>com.fasterxml.jackson.core</groupId>
> > + <artifactId>jackson-databind</artifactId>
> > + <version>${jackson2.version}</version>
> > + </dependency>
> > +
> > + <dependency>
> > + <groupId>org.mockito</groupId>
> > + <artifactId>mockito-all</artifactId>
> > + <version>${mockito.version}</version>
> > + <scope>test</scope>
> > + </dependency>
> > +
> > + <dependency>
> > + <groupId>org.eclipse.jetty</groupId>
> > + <artifactId>jetty-server</artifactId>
> > + <version>${jetty.version}</version>
> > + <exclusions>
> > + <exclusion>
> > + <groupId>org.eclipse.jetty</groupId>
> > + <artifactId>javax.servlet-api</artifactId>
> > + </exclusion>
> > + </exclusions>
> > + </dependency>
> > + <dependency>
> > + <groupId>org.eclipse.jetty</groupId>
> > + <artifactId>jetty-util</artifactId>
> > + <version>${jetty.version}</version>
> > + </dependency>
> > + <dependency>
> > + <groupId>org.eclipse.jetty</groupId>
> > + <artifactId>jetty-servlet</artifactId>
> > + <version>${jetty.version}</version>
> > + </dependency>
> > + <dependency>
> > + <groupId>org.eclipse.jetty</groupId>
> > + <artifactId>jetty-webapp</artifactId>
> > + <version>${jetty.version}</version>
> > + </dependency>
> > + <dependency>
> > + <groupId>org.eclipse.jetty</groupId>
> > + <artifactId>jetty-util-ajax</artifactId>
> > + <version>${jetty.version}</version>
> > + </dependency>
> > +
> > +
> > +
> > <dependency>
> > <groupId>javax.servlet</groupId>
> > <artifactId>javax.servlet-api</artifactId>
> > @@ -118,10 +173,27 @@
> > <groupId>org.apache.directory.server</groupId>
> > <artifactId>apacheds-all</artifactId>
> > </exclusion>
> > +
> > + <exclusion>
> > + <groupId>commons-configuration</groupId>
> > + <artifactId>commons-configuration</artifactId>
> > + </exclusion>
> > +
> > + <exclusion>
> > + <groupId>com.fasterxml.jackson.core</groupId>
> > + <artifactId>jackson-databind</artifactId>
> > + </exclusion>
> > +
> > </exclusions>
> > </dependency>
> >
> > <dependency>
> > + <groupId>commons-configuration</groupId>
> > + <artifactId>commons-configuration</artifactId>
> > + <version>1.10</version>
> > + </dependency>
> > +
> > + <dependency>
> > <groupId>org.hamcrest</groupId>
> > <artifactId>hamcrest-library</artifactId>
> > <scope>test</scope>
> > @@ -152,6 +224,11 @@
> > </dependency>
> >
> > <dependency>
> > + <groupId>org.apache.kerby</groupId>
> > + <artifactId>kerb-simplekdc</artifactId>
> > + </dependency>
> > +
> > + <dependency>
> > <groupId>junit</groupId>
> > <artifactId>junit</artifactId>
> > <scope>test</scope>
> >
> > http://git-wip-us.apache.org/repos/asf/knox/blob/772bc33d/pom.xml
> > ----------------------------------------------------------------------
> > diff --git a/pom.xml b/pom.xml
> > index fd7f62b..6bd9396 100644
> > --- a/pom.xml
> > +++ b/pom.xml
> > @@ -112,7 +112,7 @@
> > <gateway-version>1.0.0-SNAPSHOT</gateway-version>
> > <gateway-group>org.apache.knox</gateway-group>
> > <groovy-version>2.4.6</groovy-version>
> > - <hadoop-version>2.7.3</hadoop-version>
> > + <hadoop-version>3.0.0</hadoop-version>
> > <jackson.version>2.8.10</jackson.version>
> > <jetty-version>9.2.15.v20160210</jetty-version>
> > <surefire-version>2.16</surefire-version>
> > @@ -121,6 +121,9 @@
> > <javax-websocket-version>1.1</javax-websocket-version>
> > <metrics-version>3.1.2</metrics-version>
> > <shiro.version>1.2.6</shiro.version>
> > + <kerb-simplekdc-version>1.0.0-RC2</kerb-simplekdc-version>
> > + <commons-beanutils-version>1.9.3</commons-beanutils-version>
> > + <jackson2.version>2.2.2</jackson2.version>
> > </properties>
> >
> > <licenses>
> > @@ -1015,10 +1018,34 @@
> > <groupId>xmlenc</groupId>
> > <artifactId>xmlenc</artifactId>
> > </exclusion>
> > +
> > + <exclusion>
> > + <groupId>com.sun.jersey</groupId>
> > + <artifactId>jersey-servlet</artifactId>
> > + </exclusion>
> > +
> > + <exclusion>
> > + <groupId>org.apache.kerby</groupId>
> > + <artifactId>kerb-simplekdc</artifactId>
> > + </exclusion>
> > +
> > + <!--
> > + <exclusion>
> > + <groupId>com.fasterxml.jackson.core</groupId>
> > + <artifactId>jackson-databind</artifactId>
> > + </exclusion>
> > + -->
> > +
> > </exclusions>
> > </dependency>
> >
> > <dependency>
> > + <groupId>org.apache.kerby</groupId>
> > + <artifactId>kerb-simplekdc</artifactId>
> > + <version>${kerb-simplekdc-version}</version>
> > + </dependency>
> > +
> > + <dependency>
> > <groupId>com.fasterxml.jackson.core</groupId>
> > <artifactId>jackson-databind</artifactId>
> > <version>${jackson.version}</version>
> > @@ -1057,7 +1084,7 @@
> > <dependency>
> > <groupId>commons-beanutils</groupId>
> > <artifactId>commons-beanutils</artifactId>
> > - <version>1.9.2</version>
> > + <version>${commons-beanutils-version}</version>
> > </dependency>
> > <dependency>
> > <groupId>org.apache.commons</groupId>
> >
> >
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>
