Re: [DISCUSS] Migrate to Yetus Interface classification annotations

[Viraj Jasani]

Thanks Masatake for the suggestions. I agree that until Yetus comes to
final conclusion on whether to keep or drop IA/IS annotations for higher
JDK versions (or fix/drop doclet support), we should hold on for now.
Thanks Sean and Akira for providing the context.


On Tue, 28 Sep 2021 at 6:55 PM, Masatake Iwasaki <
iwasak...@oss.nttdata.co.jp> wrote:

> Thanks, Akira and Viraj.
>
> My understanding is that we have options like
>
> 1. migrate org.apache.yetus:audience-annotations to Java >= 9 then
> migrate Hadoop to the new org.apache.yetus:audience-annotations.
>
> 2. "use the Jigsaw feature to export only @Public elements to other
> projects
> and create javadoc by new --show-packages=exported option instead of
> relying on the annotations." as mentioned by Akira[1].
>
> Both require dropping Java 8 support.
>
> If current org.apache.yetus:audience-annotations(:0.13.0) for Java 8 no
> longer evolves,
> migrating to it in short term is not much useful?
>
> [1]
> https://issues.apache.org/jira/browse/HADOOP-15304?focusedCommentId=16418072=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16418072
>
> On 2021/09/28 18:38, Viraj Jasani wrote:
> >> The problem comes from the removal of com.sun.tools.doclets.* packages
> >
> > Agree. Here is the summary
> > <
> https://docs.oracle.com/en/java/javase/11/docs/api/jdk.javadoc/jdk/javadoc/doclet/package-summary.html
> >
> > of the replacement package *jdk.javadoc.doclet*.
> > Here is the migration guide
> > <
> https://docs.oracle.com/en/java/javase/11/docs/api/jdk.javadoc/jdk/javadoc/doclet/package-summary.html#migration
> >
> > for the same.
> >
> > On Tue, Sep 28, 2021 at 1:06 PM Akira Ajisaka 
> wrote:
> >
> >> Hi Masatake,
> >>
> >> The problem comes from the removal of com.sun.tools.doclets.* packages
> in
> >> Java 10.
> >> In Apache Hadoop, I removed the doclet support for filtering javadocs
> when
> >> the environment is Java 10 or upper.
> >> https://issues.apache.org/jira/browse/HADOOP-15304
> >>
> >> Thanks,
> >> Akira
> >>
> >> On Tue, Sep 28, 2021 at 10:27 AM Masatake Iwasaki <
> >> iwasak...@oss.nttdata.co.jp> wrote:
> >>
>  In particular, there has been an outstanding problem with doclet
> >> support
> >>> for filtering javadocs by annotation since JDK9 came out.
> >>>
> >>> Could you give me a pointer to relevant Yetus JIRA or ML thread?
> >>>
> >>> On 2021/09/28 1:17, Sean Busbey wrote:
>  I think consolidating on a common library and tooling for defining API
> >>> expectations for Hadoop would be great.
> 
>  Unfortunately, the Apache Yetus community recently started a
> discussion
> >>> around dropping their maintenance of the audience annotations
> codebase[1]
> >>> due to lack of community interest. In particular, there has been an
> >>> outstanding problem with doclet support for filtering javadocs by
> >>> annotation since JDK9 came out.
> 
>  I think that means a necessary first step here would be to determine
> if
> >>> we have contributors willing to show up over in that project to get
> >> things
> >>> into a good state for future JDK adoption.
> 
> 
> 
>  [1]:
>  https://s.apache.org/ybdl6
>  "[DISCUSS] Drop JDK8; audience-annotations" from d...@yetus.apache.org
> 
> > On Sep 27, 2021, at 2:46 AM, Viraj Jasani 
> wrote:
> >
> > Since the early days, Hadoop has provided Interface classification
> > annotations to represent the scope and stability for downstream
> > applications to select Hadoop APIs carefully. After some time, these
> > annotations (InterfaceAudience and InterfaceStability) have been
> >>> migrated
> > to Apache Yetus. As of today, with increasing number of Hadoop
> >> ecosystem
> > applications using (or starting to use) Yetus stability annotations
> >> for
> > their own downstreamers, we should also consider using IA/IS
> >> annotations
> > provided by *org.apache.yetus.audience *directly in our codebase and
> >>> retire
> > our *org.apache.hadoop.classification* package for the better
> >>> separation of
> > concern and single source.
> >
> > I believe we can go with this migration to maintain compatibility for
> > Hadoop downstreamers:
> >
> > 1. In Hadoop trunk (3.4.0+ releases), replace all usages of
> o.a.h.c
> > stability annotations with o.a.y.a annotations.
> > 2. Deprecate o.a.h.c annotations, and provide deprecation warning
> >>> that
> > we will remove o.a.h.c in 4.0.0 (or 5.0.0) release and the only
> >>> source for
> > these annotations should be o.a.y.a.
> >
> > Any thoughts?
> 
> 
> 
>  -
>  To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>  For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> 
> >>>
> >>> -
> >>> 

Re: [DISCUSS] Migrate to Yetus Interface classification annotations

[Masatake Iwasaki]

Thanks, Akira and Viraj.

My understanding is that we have options like

1. migrate org.apache.yetus:audience-annotations to Java >= 9 then
   migrate Hadoop to the new org.apache.yetus:audience-annotations.

2. "use the Jigsaw feature to export only @Public elements to other projects
   and create javadoc by new --show-packages=exported option instead of
   relying on the annotations." as mentioned by Akira[1].

Both require dropping Java 8 support.

If current org.apache.yetus:audience-annotations(:0.13.0) for Java 8 no longer 
evolves,
migrating to it in short term is not much useful?

[1] 
https://issues.apache.org/jira/browse/HADOOP-15304?focusedCommentId=16418072=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16418072

On 2021/09/28 18:38, Viraj Jasani wrote:

The problem comes from the removal of com.sun.tools.doclets.* packages


Agree. Here is the summary

of the replacement package *jdk.javadoc.doclet*.
Here is the migration guide

for the same.

On Tue, Sep 28, 2021 at 1:06 PM Akira Ajisaka  wrote:


Hi Masatake,

The problem comes from the removal of com.sun.tools.doclets.* packages in
Java 10.
In Apache Hadoop, I removed the doclet support for filtering javadocs when
the environment is Java 10 or upper.
https://issues.apache.org/jira/browse/HADOOP-15304

Thanks,
Akira

On Tue, Sep 28, 2021 at 10:27 AM Masatake Iwasaki <
iwasak...@oss.nttdata.co.jp> wrote:


In particular, there has been an outstanding problem with doclet

support

for filtering javadocs by annotation since JDK9 came out.

Could you give me a pointer to relevant Yetus JIRA or ML thread?

On 2021/09/28 1:17, Sean Busbey wrote:

I think consolidating on a common library and tooling for defining API

expectations for Hadoop would be great.


Unfortunately, the Apache Yetus community recently started a discussion

around dropping their maintenance of the audience annotations codebase[1]
due to lack of community interest. In particular, there has been an
outstanding problem with doclet support for filtering javadocs by
annotation since JDK9 came out.


I think that means a necessary first step here would be to determine if

we have contributors willing to show up over in that project to get

things

into a good state for future JDK adoption.




[1]:
https://s.apache.org/ybdl6
"[DISCUSS] Drop JDK8; audience-annotations" from d...@yetus.apache.org


On Sep 27, 2021, at 2:46 AM, Viraj Jasani  wrote:

Since the early days, Hadoop has provided Interface classification
annotations to represent the scope and stability for downstream
applications to select Hadoop APIs carefully. After some time, these
annotations (InterfaceAudience and InterfaceStability) have been

migrated

to Apache Yetus. As of today, with increasing number of Hadoop

ecosystem

applications using (or starting to use) Yetus stability annotations

for

their own downstreamers, we should also consider using IA/IS

annotations

provided by *org.apache.yetus.audience *directly in our codebase and

retire

our *org.apache.hadoop.classification* package for the better

separation of

concern and single source.

I believe we can go with this migration to maintain compatibility for
Hadoop downstreamers:

1. In Hadoop trunk (3.4.0+ releases), replace all usages of o.a.h.c
stability annotations with o.a.y.a annotations.
2. Deprecate o.a.h.c annotations, and provide deprecation warning

that

we will remove o.a.h.c in 4.0.0 (or 5.0.0) release and the only

source for

these annotations should be o.a.y.a.

Any thoughts?




-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org








-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [DISCUSS] Migrate to Yetus Interface classification annotations

[Viraj Jasani]

> The problem comes from the removal of com.sun.tools.doclets.* packages

Agree. Here is the summary

of the replacement package *jdk.javadoc.doclet*.
Here is the migration guide

for the same.

On Tue, Sep 28, 2021 at 1:06 PM Akira Ajisaka  wrote:

> Hi Masatake,
>
> The problem comes from the removal of com.sun.tools.doclets.* packages in
> Java 10.
> In Apache Hadoop, I removed the doclet support for filtering javadocs when
> the environment is Java 10 or upper.
> https://issues.apache.org/jira/browse/HADOOP-15304
>
> Thanks,
> Akira
>
> On Tue, Sep 28, 2021 at 10:27 AM Masatake Iwasaki <
> iwasak...@oss.nttdata.co.jp> wrote:
>
> > > In particular, there has been an outstanding problem with doclet
> support
> > for filtering javadocs by annotation since JDK9 came out.
> >
> > Could you give me a pointer to relevant Yetus JIRA or ML thread?
> >
> > On 2021/09/28 1:17, Sean Busbey wrote:
> > > I think consolidating on a common library and tooling for defining API
> > expectations for Hadoop would be great.
> > >
> > > Unfortunately, the Apache Yetus community recently started a discussion
> > around dropping their maintenance of the audience annotations codebase[1]
> > due to lack of community interest. In particular, there has been an
> > outstanding problem with doclet support for filtering javadocs by
> > annotation since JDK9 came out.
> > >
> > > I think that means a necessary first step here would be to determine if
> > we have contributors willing to show up over in that project to get
> things
> > into a good state for future JDK adoption.
> > >
> > >
> > >
> > > [1]:
> > > https://s.apache.org/ybdl6
> > > "[DISCUSS] Drop JDK8; audience-annotations" from d...@yetus.apache.org
> > >
> > >> On Sep 27, 2021, at 2:46 AM, Viraj Jasani  wrote:
> > >>
> > >> Since the early days, Hadoop has provided Interface classification
> > >> annotations to represent the scope and stability for downstream
> > >> applications to select Hadoop APIs carefully. After some time, these
> > >> annotations (InterfaceAudience and InterfaceStability) have been
> > migrated
> > >> to Apache Yetus. As of today, with increasing number of Hadoop
> ecosystem
> > >> applications using (or starting to use) Yetus stability annotations
> for
> > >> their own downstreamers, we should also consider using IA/IS
> annotations
> > >> provided by *org.apache.yetus.audience *directly in our codebase and
> > retire
> > >> our *org.apache.hadoop.classification* package for the better
> > separation of
> > >> concern and single source.
> > >>
> > >> I believe we can go with this migration to maintain compatibility for
> > >> Hadoop downstreamers:
> > >>
> > >>1. In Hadoop trunk (3.4.0+ releases), replace all usages of o.a.h.c
> > >>stability annotations with o.a.y.a annotations.
> > >>2. Deprecate o.a.h.c annotations, and provide deprecation warning
> > that
> > >>we will remove o.a.h.c in 4.0.0 (or 5.0.0) release and the only
> > source for
> > >>these annotations should be o.a.y.a.
> > >>
> > >> Any thoughts?
> > >
> > >
> > >
> > > -
> > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> > >
> >
> > -
> > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
> >
> >
>

Re: [DISCUSS] Migrate to Yetus Interface classification annotations

[Akira Ajisaka]

Hi Masatake,

The problem comes from the removal of com.sun.tools.doclets.* packages in
Java 10.
In Apache Hadoop, I removed the doclet support for filtering javadocs when
the environment is Java 10 or upper.
https://issues.apache.org/jira/browse/HADOOP-15304

Thanks,
Akira

On Tue, Sep 28, 2021 at 10:27 AM Masatake Iwasaki <
iwasak...@oss.nttdata.co.jp> wrote:

> > In particular, there has been an outstanding problem with doclet support
> for filtering javadocs by annotation since JDK9 came out.
>
> Could you give me a pointer to relevant Yetus JIRA or ML thread?
>
> On 2021/09/28 1:17, Sean Busbey wrote:
> > I think consolidating on a common library and tooling for defining API
> expectations for Hadoop would be great.
> >
> > Unfortunately, the Apache Yetus community recently started a discussion
> around dropping their maintenance of the audience annotations codebase[1]
> due to lack of community interest. In particular, there has been an
> outstanding problem with doclet support for filtering javadocs by
> annotation since JDK9 came out.
> >
> > I think that means a necessary first step here would be to determine if
> we have contributors willing to show up over in that project to get things
> into a good state for future JDK adoption.
> >
> >
> >
> > [1]:
> > https://s.apache.org/ybdl6
> > "[DISCUSS] Drop JDK8; audience-annotations" from d...@yetus.apache.org
> >
> >> On Sep 27, 2021, at 2:46 AM, Viraj Jasani  wrote:
> >>
> >> Since the early days, Hadoop has provided Interface classification
> >> annotations to represent the scope and stability for downstream
> >> applications to select Hadoop APIs carefully. After some time, these
> >> annotations (InterfaceAudience and InterfaceStability) have been
> migrated
> >> to Apache Yetus. As of today, with increasing number of Hadoop ecosystem
> >> applications using (or starting to use) Yetus stability annotations for
> >> their own downstreamers, we should also consider using IA/IS annotations
> >> provided by *org.apache.yetus.audience *directly in our codebase and
> retire
> >> our *org.apache.hadoop.classification* package for the better
> separation of
> >> concern and single source.
> >>
> >> I believe we can go with this migration to maintain compatibility for
> >> Hadoop downstreamers:
> >>
> >>1. In Hadoop trunk (3.4.0+ releases), replace all usages of o.a.h.c
> >>stability annotations with o.a.y.a annotations.
> >>2. Deprecate o.a.h.c annotations, and provide deprecation warning
> that
> >>we will remove o.a.h.c in 4.0.0 (or 5.0.0) release and the only
> source for
> >>these annotations should be o.a.y.a.
> >>
> >> Any thoughts?
> >
> >
> >
> > -
> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> >
>
> -
> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
>
>

Re: [DISCUSS] Migrate to Yetus Interface classification annotations

[Masatake Iwasaki]

In particular, there has been an outstanding problem with doclet support for 
filtering javadocs by annotation since JDK9 came out.


Could you give me a pointer to relevant Yetus JIRA or ML thread?

On 2021/09/28 1:17, Sean Busbey wrote:

I think consolidating on a common library and tooling for defining API 
expectations for Hadoop would be great.

Unfortunately, the Apache Yetus community recently started a discussion around 
dropping their maintenance of the audience annotations codebase[1] due to lack 
of community interest. In particular, there has been an outstanding problem 
with doclet support for filtering javadocs by annotation since JDK9 came out.

I think that means a necessary first step here would be to determine if we have 
contributors willing to show up over in that project to get things into a good 
state for future JDK adoption.



[1]:
https://s.apache.org/ybdl6
"[DISCUSS] Drop JDK8; audience-annotations" from d...@yetus.apache.org


On Sep 27, 2021, at 2:46 AM, Viraj Jasani  wrote:

Since the early days, Hadoop has provided Interface classification
annotations to represent the scope and stability for downstream
applications to select Hadoop APIs carefully. After some time, these
annotations (InterfaceAudience and InterfaceStability) have been migrated
to Apache Yetus. As of today, with increasing number of Hadoop ecosystem
applications using (or starting to use) Yetus stability annotations for
their own downstreamers, we should also consider using IA/IS annotations
provided by *org.apache.yetus.audience *directly in our codebase and retire
our *org.apache.hadoop.classification* package for the better separation of
concern and single source.

I believe we can go with this migration to maintain compatibility for
Hadoop downstreamers:

   1. In Hadoop trunk (3.4.0+ releases), replace all usages of o.a.h.c
   stability annotations with o.a.y.a annotations.
   2. Deprecate o.a.h.c annotations, and provide deprecation warning that
   we will remove o.a.h.c in 4.0.0 (or 5.0.0) release and the only source for
   these annotations should be o.a.y.a.

Any thoughts?




-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [DISCUSS] Migrate to Yetus Interface classification annotations

[Sean Busbey]

I think consolidating on a common library and tooling for defining API 
expectations for Hadoop would be great.

Unfortunately, the Apache Yetus community recently started a discussion around 
dropping their maintenance of the audience annotations codebase[1] due to lack 
of community interest. In particular, there has been an outstanding problem 
with doclet support for filtering javadocs by annotation since JDK9 came out.

I think that means a necessary first step here would be to determine if we have 
contributors willing to show up over in that project to get things into a good 
state for future JDK adoption.



[1]:
https://s.apache.org/ybdl6
"[DISCUSS] Drop JDK8; audience-annotations" from d...@yetus.apache.org

> On Sep 27, 2021, at 2:46 AM, Viraj Jasani  wrote:
> 
> Since the early days, Hadoop has provided Interface classification
> annotations to represent the scope and stability for downstream
> applications to select Hadoop APIs carefully. After some time, these
> annotations (InterfaceAudience and InterfaceStability) have been migrated
> to Apache Yetus. As of today, with increasing number of Hadoop ecosystem
> applications using (or starting to use) Yetus stability annotations for
> their own downstreamers, we should also consider using IA/IS annotations
> provided by *org.apache.yetus.audience *directly in our codebase and retire
> our *org.apache.hadoop.classification* package for the better separation of
> concern and single source.
> 
> I believe we can go with this migration to maintain compatibility for
> Hadoop downstreamers:
> 
>   1. In Hadoop trunk (3.4.0+ releases), replace all usages of o.a.h.c
>   stability annotations with o.a.y.a annotations.
>   2. Deprecate o.a.h.c annotations, and provide deprecation warning that
>   we will remove o.a.h.c in 4.0.0 (or 5.0.0) release and the only source for
>   these annotations should be o.a.y.a.
> 
> Any thoughts?



-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[DISCUSS] Migrate to Yetus Interface classification annotations

[Viraj Jasani]

Since the early days, Hadoop has provided Interface classification
annotations to represent the scope and stability for downstream
applications to select Hadoop APIs carefully. After some time, these
annotations (InterfaceAudience and InterfaceStability) have been migrated
to Apache Yetus. As of today, with increasing number of Hadoop ecosystem
applications using (or starting to use) Yetus stability annotations for
their own downstreamers, we should also consider using IA/IS annotations
provided by *org.apache.yetus.audience *directly in our codebase and retire
our *org.apache.hadoop.classification* package for the better separation of
concern and single source.

I believe we can go with this migration to maintain compatibility for
Hadoop downstreamers:

   1. In Hadoop trunk (3.4.0+ releases), replace all usages of o.a.h.c
   stability annotations with o.a.y.a annotations.
   2. Deprecate o.a.h.c annotations, and provide deprecation warning that
   we will remove o.a.h.c in 4.0.0 (or 5.0.0) release and the only source for
   these annotations should be o.a.y.a.

Any thoughts?