All the issues are fixed now (Thanks Nanda, Tsz-Wo, Mukul)
The only open question is the version of the release (alpha vs non-alpha).
I think there are two kind of alpha releases:
1. Alpha as defined in the semantic versioning specification
(https://semver.org/). Here the alpha is a pre-release version of the
final release (0.2.0-alpha should be followed by an 0.2.0). This is the
schema what is used by hadoop before hadoop 3.0 release.
2. The other alpha is just a label on every release to mark the current
releases as not "production-ready". As I understood ratis had this kind
of alpha label (there was a 0.1.0-alpha but not 0.1.0).
I prefer to follow the semantic versioning. And I can't see the benefit
to release both 0.2.0 and 0.2.0-alpha. I think the current version is
stable enough to get a final number and the 0 as the major version makes
it clear that the project is in an early phase.
So I would use 0.2.0 as the next verison number (but not strongly
against 0.2.0-alpha if this your preference).
What do you think?
Thanks,
Marton
On 05/08/2018 11:45 PM, Tsz Wo Sze wrote:
Agree. Let's fix #2 - #5 for now.
I have a comment on the release version: In the previous release, we
called it 0.1.0-alpha. How about we use 0.2.0-alpha for the next
release?
Tsz-Wo
On Tue, May 8, 2018 at 2:16 PM, Elek, Marton <e...@apache.org> wrote:
Hi,
thank you the test and feedback for all of you.
As a summary. The problems until now:
1. Unit test failures
2. OpenConsensus shading problem: RATIS-237
3. Copyright 2017
4. Missing information about increasing MVN memory.
5. Licence headers in shaded jar files
3/4: are not blockers (IMHO)
2: I think it's a blocker. As some of our unit tests are fragile and
intermittent (1.) Hadoop Ozone could give us additional confidence about the
stability of the release. Also we would like to provide a stable Ratis
version to Ozone, so they should work together.
I would cancel this vote. And initialize a new one with RATIS-237 (+ the
trival fixes).
5. I am not sure if it's blocker or not (as LICENSE.txt is ok), but I think
the shaded sources should not be included in the source distribution. I
think they are just external dependencies similar to other maven
dependencies (just on the source level). I would fix it with removing the
src/main/java of the shaded project from the src release.
1. As I mentioned some of the unit tests seem to be fragile (see
https://builds.apache.org/job/ratis-qbt-master-java8-linux-x86/). I don't
think it's a blocker (but would be great to fix them long-term).
Summary: I cancel this vote and propose to include the fix of 2,3,4,5 in rc1
(hopefully very soon).
But this is just my view. Please let me know what do you think...
Thanks a lot,
Marton
On 05/08/2018 10:08 PM, Tsz Wo Sze wrote:
Thanks Nanda for the explanation.
I seem able to fix the io.opencensus shading problem. Just have
posted a patch on RATIS-237.
Tsz-Wo
On Mon, May 7, 2018 at 8:28 PM, Nandakumar Vadivelu
<nvadiv...@hortonworks.com> wrote:
Hi Tsz-Wo,
True, there is only one version of opencensus in Ratis. Since we do
shading of io.grpc we end up with new shaded class of ContextUtils (same
fully qualified class name, but the reference of io.grpc inside the class is
modified/relocated), we also have non shaded ContextUtils. Both are referred
through Ratis dependency.
The below is dependency tree of Ratis:
[INFO] +-
org.apache.ratis:ratis-proto-shaded:jar:0.1.1-alpha-SNAPSHOT:compile
=========> Modified opencensus packed with proto-shaded
[INFO] | +- com.google.auto.value:auto-value-annotations:jar:1.6:compile
[INFO] | +- com.google.guava:guava:jar:24.1-jre:compile
[INFO] | | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile
[INFO] | | +-
org.checkerframework:checker-compat-qual:jar:2.0.0:compile
[INFO] | | +-
com.google.errorprone:error_prone_annotations:jar:2.1.3:compile
[INFO] | | +- com.google.j2objc:j2objc-annotations:jar:1.1:compile
[INFO] | | \-
org.codehaus.mojo:animal-sniffer-annotations:jar:1.14:compile
[INFO] | \- com.squareup:javapoet:jar:1.10.0:compile
[INFO] +- org.apache.ratis:ratis-common:jar:0.1.1-alpha-SNAPSHOT:compile
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.10:compile
[INFO] | +- org.slf4j:slf4j-log4j12:jar:1.7.10:compile
[INFO] | | \- log4j:log4j:jar:1.2.17:compile
[INFO] | \- io.dropwizard.metrics:metrics-core:jar:3.2.5:compile
[INFO] +- org.apache.ratis:ratis-client:jar:0.1.1-alpha-SNAPSHOT:compile
[INFO] +- org.apache.ratis:ratis-server:jar:0.1.1-alpha-SNAPSHOT:compile
[INFO] +- org.apache.ratis:ratis-netty:jar:0.1.1-alpha-SNAPSHOT:compile
[INFO] | \- org.jctools:jctools-core:jar:2.1.2:compile
[INFO] \- org.apache.ratis:ratis-grpc:jar:0.1.1-alpha-SNAPSHOT:compile
[INFO] +- io.opencensus:opencensus-api:jar:0.12.2:compile
=========> Original jar
[INFO] | \- io.grpc:grpc-context:jar:1.9.0:compile
[INFO] \-
io.opencensus:opencensus-contrib-grpc-metrics:jar:0.12.2:compile
nvadivelu@HW12726 ~/w/t/p/o/t/dependency> javap -classpath
".:./opencensus-api-0.12.2.jar" io.opencensus.trace.unsafe.ContextUtils
Compiled from "ContextUtils.java"
public final class io.opencensus.trace.unsafe.ContextUtils {
public static final io.grpc.Context$Key<io.opencensus.trace.Span>
CONTEXT_SPAN_KEY;
static {};
}
nvadivelu@HW12726 ~/w/t/p/o/t/dependency> javap -classpath
".:./ratis-proto-shaded-0.1.1-alpha-SNAPSHOT.jar"
io.opencensus.trace.unsafe.ContextUtils
Compiled from "ContextUtils.java"
public final class io.opencensus.trace.unsafe.ContextUtils {
public static final
org.apache.ratis.shaded.io.grpc.Context$Key<io.opencensus.trace.Span>
CONTEXT_SPAN_KEY;
static {};
}
-Nanda
On 5/8/18, 4:16 AM, "Tsz Wo Sze" <szets...@gmail.com> wrote:
> The problem is because of two different definition of
io.opencensus.trace.unsafe.ContextUtils class present in the classpath. The
reason for having two different definition of ContextUtils in classpath is
due to the fact that we are shading/relocating io.grpc and not
io.opencensus.
Nanda, in Ratis, there is only one version (0.12.2) of opencensus.
It
seems that another version of opencensus is pulled outside Ratis.
Tsz-Wo
On Mon, May 7, 2018 at 11:58 AM, Nandakumar Vadivelu
<nvadiv...@hortonworks.com> wrote:
> I'm not sure why the file is missing, I can see the attachment in
my "sent items".
>
> Patch content for reference
>
> diff --git a/ratis-grpc/pom.xml b/ratis-grpc/pom.xml
> index a3c3cc1..ebb73cc 100644
> --- a/ratis-grpc/pom.xml
> +++ b/ratis-grpc/pom.xml
> @@ -85,15 +85,5 @@
> <artifactId>jctools-core</artifactId>
> </dependency>
>
> - <dependency>
> - <groupId>io.opencensus</groupId>
> - <artifactId>opencensus-api</artifactId>
> - <version>${io.opencensus.version}</version>
> - </dependency>
> - <dependency>
> - <groupId>io.opencensus</groupId>
> - <artifactId>opencensus-contrib-grpc-metrics</artifactId>
> - <version>${io.opencensus.version}</version>
> - </dependency>
> </dependencies>
> </project>
>
> I will also create a jira under RATIS
>
> -Nanda
>
> On 5/8/18, 12:23 AM, "Ted Yu" <yuzhih...@gmail.com> wrote:
>
> I don't see patch attached.
>
> Please create RATIS JIRA and attach there.
>
> Thanks
>
> On Mon, May 7, 2018 at 10:24 AM, Nandakumar Vadivelu <
> nvadiv...@hortonworks.com> wrote:
>
> > Hi All,
> >
> > Below is the exception which we get with Ratis 0.2.0
> >
> > java.lang.NoSuchFieldError: CONTEXT_SPAN_KEY
> > at
org.apache.ratis.shaded.io.grpc.internal.CensusTracingModule$
> > ServerTracer.filterContext(CensusTracingModule.java:340)
> > at
org.apache.ratis.shaded.io.grpc.internal.StatsTraceContext.
> > serverFilterContext(StatsTraceContext.java:121)
> > at org.apache.ratis.shaded.io.grpc.internal.ServerImpl$
> >
ServerTransportListenerImpl.createContext(ServerImpl.java:482)
> > at org.apache.ratis.shaded.io.grpc.internal.ServerImpl$
> >
ServerTransportListenerImpl.streamCreated(ServerImpl.java:416)
> > <stack trace truncated>
> >
> >
> > The problem is because of two different definition of
> > io.opencensus.trace.unsafe.ContextUtils class present in the
classpath.
> > The reason for having two different definition of
ContextUtils in classpath
> > is due to the fact that we are shading/relocating io.grpc
and not
> > io.opencensus.
> >
> >
> > With shading we are generating the below definition which
ends up in
> > ratis-proto-shaded jar
> >
> > package io.opencensus.trace.unsafe;
> >
> > import org.apache.ratis.shaded.io.grpc.Context;
> > import io.opencensus.trace.Span;
> >
> > /**
> > * Util methods/functionality to interact with the {@link
> > org.apache.ratis.shaded.io.grpc.Context}.
> > *
> > * <p>Users must interact with the current Context via the
public APIs in
> > {@link
> > * io.opencensus.trace.Tracer} and avoid usages of the
{@link
> > #CONTEXT_SPAN_KEY} directly.
> > *
> > * @since 0.5
> > */
> > public final class ContextUtils {
> > // No instance of this class.
> > private ContextUtils() {}
> >
> > /**
> > * The {@link org.apache.ratis.shaded.io.grpc.Context.Key}
used to
> > interact with {@link
org.apache.ratis.shaded.io.grpc.Context}.
> > *
> > * @since 0.5
> > */
> > public static final Context.Key<Span> CONTEXT_SPAN_KEY =
> > Context.key("opencensus-trace-span-key");
> > }
> >
> >
> > Since we have added io.opencensus as direct dependency in
ratis-grpc we
> > get the below definition in io.opencensus:opencensus-api:jar
> >
> > package io.opencensus.trace.unsafe;
> >
> > import io.grpc.Context;
> > import io.opencensus.trace.Span;
> >
> > /**
> > * Util methods/functionality to interact with the {@link
io.grpc.Context}.
> > *
> > * <p>Users must interact with the current Context via the
public APIs in
> > {@link
> > * io.opencensus.trace.Tracer} and avoid usages of the
{@link
> > #CONTEXT_SPAN_KEY} directly.
> > *
> > * @since 0.5
> > */
> > public final class ContextUtils {
> > // No instance of this class.
> > private ContextUtils() {}
> >
> > /**
> > * The {@link io.grpc.Context.Key} used to interact with
{@link
> > io.grpc.Context}.
> > *
> > * @since 0.5
> > */
> > public static final Context.Key<Span> CONTEXT_SPAN_KEY =
> > Context.key("opencensus-trace-span-key");
> > }
> >
> > Both of the jars end up in classpath and ContextUtils is
getting loaded
> > from io.opencensus:opencensus-api:jar which is causing the
problem.
> >
> > The simple fix would be to remove the dependency of
io.opencensus from
> > ratis-grpc/pom.xml (patch for the same is attached in this
mail). This is
> > not a permanent fix, as any application might bring in
> > io.opencensus:opencensus-api:jar of its own.
> >
> > We have to find a way to shade
io.opencensus:opencensus-api:jar.
> >
> > Note: from the comment in ratis-proto-shaded/pom.xml: Cannot
relocate
> > io.opencensus due to AutoValue code generation
> >
> >
> > -Nanda
> >
> > On 5/7/18, 3:28 PM, "Lokesh Jain" <lj...@hortonworks.com>
wrote:
> >
> > We found an issue while updating ozone to Ratis snapshot
build
> > 0.1.1-alpha-4309324-SNAPSHOT (https://issues.apache.org/
> > jira/browse/HDDS-19). I feel we should hold the release till
this is
> > fixed. Please let us know of your opinion as well.
> >
> > Thanks
> > Lokesh
> >
> >
> >
> >
>
>
