Re: Redesign of the Aurora UI

[Stephan Erb]

+1 for switching to react. I believe this will reduce maintenance
effort for us in the long term. 

On Fri, 2017-09-22 at 15:19 -0700, David McLaughlin wrote:
> Just a heads up: Facebook have decided to relicense React and Jest as
> MIT:
> 
> https://code.facebook.com/posts/300798627056246
> 
> So I'll be adopting React and dropping the Preact-compat stuff asap.
> The
> effect to our application code is minimal (if any), but tests will
> need to
> be rewritten. So I'll probably hold off on further patches until the
> relicense is complete.
> 
> On Fri, Aug 18, 2017 at 8:13 PM, David McLaughlin <david@dmclaughlin.
> com>
> wrote:
> 
> > Good to know they made the decision. My plan is to move forward
> > with
> > Preact, with the major question how to do unit testing since we
> > can't use
> > jest either. The first patch should land early next week.
> > 
> > On Fri, Aug 18, 2017 at 6:29 PM, Joshua Cohen <jco...@apache.org>
> > wrote:
> > 
> > > FB came back w/ their response to relicense React today:
> > > https://code.facebook.com/posts/112130496157735/explaining-
> > > react-s-license
> > > 
> > > tl;dr: they're not relicensing React. So... if we do go down this
> > > path,
> > > we'll have to investigate alternatives to React (Preact, Vue,
> > > etc.) or
> > > potentially release this as a separate project outside of the ASF
> > > that can
> > > serve as a drop in UI replacement.
> > > 
> > > On Sun, Jul 23, 2017 at 4:48 PM, Erb, Stephan <
> > > stephan@blue-yonder.com>
> > > wrote:
> > > 
> > > > A big +1 from me as well. We have not touched or updated the
> > > > existing UI
> > > > for quite some time, which is a bad sign for code health.
> > > > 
> > > > I would even be OK with a couple of bigger initial code dumps.
> > > > I am not
> > > > really a web-developer, so a working piece of code to play
> > > > around with
> > > > would probably be the fastest way to get up to speed with the
> > > > tech and
> > > 
> > > its
> > > > usage in Aurora.
> > > > 
> > > > Thanks a lot for driving this, David!
> > > > 
> > > > On 21.07.17, 07:00, "Kai Huang" <texasred2...@hotmail.com>
> > > > wrote:
> > > > 
> > > > David - Sure, let's sync on the work when you are ready.
> > > > 
> > > > 
> > > > 
> > > > From: David McLaughlin <dmclaugh...@apache.org>
> > > > Sent: Wednesday, July 19, 2017 14:10
> > > > To: dev@aurora.apache.org
> > > > Subject: Re: Redesign of the Aurora UI
> > > > 
> > > > Thanks for the feedback!
> > > > 
> > > > Joshua - I haven't tried to drop in Preact yet, but I was
> > > > also
> > > > planning to
> > > > throw away the prototype and starting again when it came to
> > > > upstreaming it,
> > > > so as part of that we can just address incompatibilities as
> > > > we go.
> > > 
> > > If
> > > > I was
> > > > to guess, then the only significant impact on my prototype
> > > > would
> > > > probably
> > > > be the reactable plugin I was using (replacement for
> > > > Angular's
> > > > smart-table). But longer term I do have concerns about
> > > > moving away
> > > 
> > > from
> > > > what is a constantly improving and healthy ecosystem around
> > > > React.
> > > 
> > > So
> > > > most
> > > > likely I'll hold off until a decision is made there one way
> > > > or the
> > > > other
> > > > (which should be within a week).
> > > > 
> > > > 
> > > > Kai - I'd be happy to coordinate and collaborate on this
> > > > with
> > > 
> > > others.
> > > > Let
> > > > me try and finish up the CSS/UX of the pages in my
> > > > prototype and
> > > 
> > > from
> > > > there
> > > > we can sync on who does what. Does that sound good?
> > > > 
> > > > 
> > > > 
> > > > 
> > > > On Wed, Jul 19, 2017 at 1:56 PM, Kai Huang <
> > > 
> > >

Re: Redesign of the Aurora UI

[David McLaughlin]

Just a heads up: Facebook have decided to relicense React and Jest as MIT:

https://code.facebook.com/posts/300798627056246

So I'll be adopting React and dropping the Preact-compat stuff asap. The
effect to our application code is minimal (if any), but tests will need to
be rewritten. So I'll probably hold off on further patches until the
relicense is complete.

On Fri, Aug 18, 2017 at 8:13 PM, David McLaughlin <da...@dmclaughlin.com>
wrote:

> Good to know they made the decision. My plan is to move forward with
> Preact, with the major question how to do unit testing since we can't use
> jest either. The first patch should land early next week.
>
> On Fri, Aug 18, 2017 at 6:29 PM, Joshua Cohen <jco...@apache.org> wrote:
>
>> FB came back w/ their response to relicense React today:
>> https://code.facebook.com/posts/112130496157735/explaining-
>> react-s-license
>>
>> tl;dr: they're not relicensing React. So... if we do go down this path,
>> we'll have to investigate alternatives to React (Preact, Vue, etc.) or
>> potentially release this as a separate project outside of the ASF that can
>> serve as a drop in UI replacement.
>>
>> On Sun, Jul 23, 2017 at 4:48 PM, Erb, Stephan <
>> stephan@blue-yonder.com>
>> wrote:
>>
>> > A big +1 from me as well. We have not touched or updated the existing UI
>> > for quite some time, which is a bad sign for code health.
>> >
>> > I would even be OK with a couple of bigger initial code dumps. I am not
>> > really a web-developer, so a working piece of code to play around with
>> > would probably be the fastest way to get up to speed with the tech and
>> its
>> > usage in Aurora.
>> >
>> > Thanks a lot for driving this, David!
>> >
>> > On 21.07.17, 07:00, "Kai Huang" <texasred2...@hotmail.com> wrote:
>> >
>> > David - Sure, let's sync on the work when you are ready.
>> >
>> >
>> > 
>> > From: David McLaughlin <dmclaugh...@apache.org>
>> > Sent: Wednesday, July 19, 2017 14:10
>> > To: dev@aurora.apache.org
>> > Subject: Re: Redesign of the Aurora UI
>> >
>> > Thanks for the feedback!
>> >
>> > Joshua - I haven't tried to drop in Preact yet, but I was also
>> > planning to
>> > throw away the prototype and starting again when it came to
>> > upstreaming it,
>> > so as part of that we can just address incompatibilities as we go.
>> If
>> > I was
>> > to guess, then the only significant impact on my prototype would
>> > probably
>> > be the reactable plugin I was using (replacement for Angular's
>> > smart-table). But longer term I do have concerns about moving away
>> from
>> > what is a constantly improving and healthy ecosystem around React.
>> So
>> > most
>> > likely I'll hold off until a decision is made there one way or the
>> > other
>> > (which should be within a week).
>> >
>> >
>> > Kai - I'd be happy to coordinate and collaborate on this with
>> others.
>> > Let
>> > me try and finish up the CSS/UX of the pages in my prototype and
>> from
>> > there
>> > we can sync on who does what. Does that sound good?
>> >
>> >
>> >
>> >
>> > On Wed, Jul 19, 2017 at 1:56 PM, Kai Huang <
>> texasred2...@hotmail.com>
>> > wrote:
>> >
>> > > Just a few thoughts as an aurora developer and operator:
>> > >
>> > >
>> > > From my experiences with Aurora users, some persisting complaints
>> > are:
>> > >
>> > >   1.  The current UI is not very intuitive for the users to
>> > understand the
>> > > task lifecycle, resource utilization of their job.
>> > >   2.  Often times users are unaware of the new features/changes in
>> > Aurora
>> > > Scheduler/Executor, which leads to a lot of misuse of the system.
>> > >   3.  Users have preferences on the appearance of the
>> > scheduler/thermos UI
>> > > due to special use cases, and ask us to customize for them(or
>> start
>> > to
>> > > write their own UI, which is often not recommended).
>> > >
>> > >
>> > > The other major issue I see in the cur

Re: Redesign of the Aurora UI

[David McLaughlin]

Good to know they made the decision. My plan is to move forward with
Preact, with the major question how to do unit testing since we can't use
jest either. The first patch should land early next week.

On Fri, Aug 18, 2017 at 6:29 PM, Joshua Cohen <jco...@apache.org> wrote:

> FB came back w/ their response to relicense React today:
> https://code.facebook.com/posts/112130496157735/explaining-react-s-license
>
> tl;dr: they're not relicensing React. So... if we do go down this path,
> we'll have to investigate alternatives to React (Preact, Vue, etc.) or
> potentially release this as a separate project outside of the ASF that can
> serve as a drop in UI replacement.
>
> On Sun, Jul 23, 2017 at 4:48 PM, Erb, Stephan <stephan@blue-yonder.com
> >
> wrote:
>
> > A big +1 from me as well. We have not touched or updated the existing UI
> > for quite some time, which is a bad sign for code health.
> >
> > I would even be OK with a couple of bigger initial code dumps. I am not
> > really a web-developer, so a working piece of code to play around with
> > would probably be the fastest way to get up to speed with the tech and
> its
> > usage in Aurora.
> >
> > Thanks a lot for driving this, David!
> >
> > On 21.07.17, 07:00, "Kai Huang" <texasred2...@hotmail.com> wrote:
> >
> > David - Sure, let's sync on the work when you are ready.
> >
> >
> > ________
> > From: David McLaughlin <dmclaugh...@apache.org>
> > Sent: Wednesday, July 19, 2017 14:10
> > To: dev@aurora.apache.org
> > Subject: Re: Redesign of the Aurora UI
> >
> > Thanks for the feedback!
> >
> > Joshua - I haven't tried to drop in Preact yet, but I was also
> > planning to
> > throw away the prototype and starting again when it came to
> > upstreaming it,
> > so as part of that we can just address incompatibilities as we go. If
> > I was
> > to guess, then the only significant impact on my prototype would
> > probably
> > be the reactable plugin I was using (replacement for Angular's
> > smart-table). But longer term I do have concerns about moving away
> from
> > what is a constantly improving and healthy ecosystem around React. So
> > most
> > likely I'll hold off until a decision is made there one way or the
> > other
> > (which should be within a week).
> >
> >
> > Kai - I'd be happy to coordinate and collaborate on this with others.
> > Let
> > me try and finish up the CSS/UX of the pages in my prototype and from
> > there
> > we can sync on who does what. Does that sound good?
> >
> >
> >
> >
> > On Wed, Jul 19, 2017 at 1:56 PM, Kai Huang <texasred2...@hotmail.com
> >
> > wrote:
> >
> > > Just a few thoughts as an aurora developer and operator:
> > >
> > >
> > > From my experiences with Aurora users, some persisting complaints
> > are:
> > >
> > >   1.  The current UI is not very intuitive for the users to
> > understand the
> > > task lifecycle, resource utilization of their job.
> > >   2.  Often times users are unaware of the new features/changes in
> > Aurora
> > > Scheduler/Executor, which leads to a lot of misuse of the system.
> > >   3.  Users have preferences on the appearance of the
> > scheduler/thermos UI
> > > due to special use cases, and ask us to customize for them(or start
> > to
> > > write their own UI, which is often not recommended).
> > >
> > >
> > > The other major issue I see in the current UI is that it's built on
> > an
> > > obsolete tech stack(AngularJS) that has all the binaries and
> > dependencies
> > > in the repo. From a developer's perspective, it's a big burden to
> > > maintain/test the code, and make fast iterations on it.
> > >
> > >
> > > Currently the scheduler UI is readonly, and mainly designed for
> > debugging
> > > purposes. We could have done much better to make the UI more
> > friendly to
> > > the end user, empower them to discover, understand and use all the
> > Aurora
> > > features, and give them more insights into the their jobs, or even
> > the
> > > entire cluster. I love the idea that redesign the UI with a modern
> > stack,
> > > and more importantly, every single part of the application

Re: Redesign of the Aurora UI

[Joshua Cohen]

FB came back w/ their response to relicense React today:
https://code.facebook.com/posts/112130496157735/explaining-react-s-license

tl;dr: they're not relicensing React. So... if we do go down this path,
we'll have to investigate alternatives to React (Preact, Vue, etc.) or
potentially release this as a separate project outside of the ASF that can
serve as a drop in UI replacement.

On Sun, Jul 23, 2017 at 4:48 PM, Erb, Stephan <stephan@blue-yonder.com>
wrote:

> A big +1 from me as well. We have not touched or updated the existing UI
> for quite some time, which is a bad sign for code health.
>
> I would even be OK with a couple of bigger initial code dumps. I am not
> really a web-developer, so a working piece of code to play around with
> would probably be the fastest way to get up to speed with the tech and its
> usage in Aurora.
>
> Thanks a lot for driving this, David!
>
> On 21.07.17, 07:00, "Kai Huang" <texasred2...@hotmail.com> wrote:
>
> David - Sure, let's sync on the work when you are ready.
>
>
> 
> From: David McLaughlin <dmclaugh...@apache.org>
> Sent: Wednesday, July 19, 2017 14:10
> To: dev@aurora.apache.org
> Subject: Re: Redesign of the Aurora UI
>
> Thanks for the feedback!
>
> Joshua - I haven't tried to drop in Preact yet, but I was also
> planning to
> throw away the prototype and starting again when it came to
> upstreaming it,
> so as part of that we can just address incompatibilities as we go. If
> I was
> to guess, then the only significant impact on my prototype would
> probably
> be the reactable plugin I was using (replacement for Angular's
> smart-table). But longer term I do have concerns about moving away from
> what is a constantly improving and healthy ecosystem around React. So
> most
> likely I'll hold off until a decision is made there one way or the
> other
> (which should be within a week).
>
>
> Kai - I'd be happy to coordinate and collaborate on this with others.
> Let
> me try and finish up the CSS/UX of the pages in my prototype and from
> there
> we can sync on who does what. Does that sound good?
>
>
>
>
> On Wed, Jul 19, 2017 at 1:56 PM, Kai Huang <texasred2...@hotmail.com>
> wrote:
>
> > Just a few thoughts as an aurora developer and operator:
> >
> >
> > From my experiences with Aurora users, some persisting complaints
> are:
> >
> >   1.  The current UI is not very intuitive for the users to
> understand the
> > task lifecycle, resource utilization of their job.
> >   2.  Often times users are unaware of the new features/changes in
> Aurora
> > Scheduler/Executor, which leads to a lot of misuse of the system.
> >   3.  Users have preferences on the appearance of the
> scheduler/thermos UI
> > due to special use cases, and ask us to customize for them(or start
> to
> > write their own UI, which is often not recommended).
> >
> >
> > The other major issue I see in the current UI is that it's built on
> an
> > obsolete tech stack(AngularJS) that has all the binaries and
> dependencies
> > in the repo. From a developer's perspective, it's a big burden to
> > maintain/test the code, and make fast iterations on it.
> >
> >
> > Currently the scheduler UI is readonly, and mainly designed for
> debugging
> > purposes. We could have done much better to make the UI more
> friendly to
> > the end user, empower them to discover, understand and use all the
> Aurora
> > features, and give them more insights into the their jobs, or even
> the
> > entire cluster. I love the idea that redesign the UI with a modern
> stack,
> > and more importantly, every single part of the application is a
> module that
> > you could take your own customization.
> >
> >
> > As for the development strategy, I'm in favor of the incremental
> approach
> > that posts one page at a time. The main benefit is that we are
> educating
> > the developers while iterating on it, and this will improve the
> adoption
> > rate in the long term.
> >
> >
> > Overall I'm very interested in this work, and would like to
> collaborate
> > with David to redesign and improve the UI.
> >
> >
> > 
> > From: Joshua Cohen <jco...@apache.org>
> > Sent: Wednesday, July 19, 2017 11:39
> > To: dev@aurora.ap

Re: Redesign of the Aurora UI

[Erb, Stephan]

A big +1 from me as well. We have not touched or updated the existing UI for 
quite some time, which is a bad sign for code health.

I would even be OK with a couple of bigger initial code dumps. I am not really 
a web-developer, so a working piece of code to play around with would probably 
be the fastest way to get up to speed with the tech and its usage in Aurora. 

Thanks a lot for driving this, David!

On 21.07.17, 07:00, "Kai Huang" <texasred2...@hotmail.com> wrote:

David - Sure, let's sync on the work when you are ready.



From: David McLaughlin <dmclaugh...@apache.org>
Sent: Wednesday, July 19, 2017 14:10
To: dev@aurora.apache.org
    Subject: Re: Redesign of the Aurora UI

Thanks for the feedback!

Joshua - I haven't tried to drop in Preact yet, but I was also planning to
throw away the prototype and starting again when it came to upstreaming it,
so as part of that we can just address incompatibilities as we go. If I was
to guess, then the only significant impact on my prototype would probably
be the reactable plugin I was using (replacement for Angular's
smart-table). But longer term I do have concerns about moving away from
what is a constantly improving and healthy ecosystem around React. So most
likely I'll hold off until a decision is made there one way or the other
(which should be within a week).


Kai - I'd be happy to coordinate and collaborate on this with others. Let
me try and finish up the CSS/UX of the pages in my prototype and from there
we can sync on who does what. Does that sound good?




On Wed, Jul 19, 2017 at 1:56 PM, Kai Huang <texasred2...@hotmail.com> wrote:

> Just a few thoughts as an aurora developer and operator:
>
>
> From my experiences with Aurora users, some persisting complaints are:
>
>   1.  The current UI is not very intuitive for the users to understand the
> task lifecycle, resource utilization of their job.
>   2.  Often times users are unaware of the new features/changes in Aurora
> Scheduler/Executor, which leads to a lot of misuse of the system.
>   3.  Users have preferences on the appearance of the scheduler/thermos UI
> due to special use cases, and ask us to customize for them(or start to
> write their own UI, which is often not recommended).
>
>
> The other major issue I see in the current UI is that it's built on an
> obsolete tech stack(AngularJS) that has all the binaries and dependencies
> in the repo. From a developer's perspective, it's a big burden to
> maintain/test the code, and make fast iterations on it.
>
>
> Currently the scheduler UI is readonly, and mainly designed for debugging
> purposes. We could have done much better to make the UI more friendly to
> the end user, empower them to discover, understand and use all the Aurora
> features, and give them more insights into the their jobs, or even the
> entire cluster. I love the idea that redesign the UI with a modern stack,
> and more importantly, every single part of the application is a module 
that
> you could take your own customization.
>
>
> As for the development strategy, I'm in favor of the incremental approach
> that posts one page at a time. The main benefit is that we are educating
> the developers while iterating on it, and this will improve the adoption
> rate in the long term.
>
>
> Overall I'm very interested in this work, and would like to collaborate
> with David to redesign and improve the UI.
>
>
> ____
> From: Joshua Cohen <jco...@apache.org>
> Sent: Wednesday, July 19, 2017 11:39
> To: dev@aurora.apache.org
> Subject: Re: Redesign of the Aurora UI
>
> I think this looks great overall! I'm super excited to see the UI get some
> love (and to set us up for better iteration on the UI going forward). My
> biggest concern, of course, is the current hubbub vis-a-vis Apache and the
> BSD-3+Patents license[1]. Have you tried running this against Preact to
> confirm compatibility/performance? Also note that other Facebook libraries
> have the same license problem (e.g. Immutable, Jest), so unless FB changes
> their patent grant clause, I imagine we'd have to find alternatives to
> those as well. If only we had landed this a week ago, we could've been
> grandfathered in on the license front :(.
>
> As far as the options for landing this, I'll leave that up to more active
> reviewers, but my gut says that smaller reviews will make this easier to
> parse, especially 

Re: Redesign of the Aurora UI

[Kai Huang]

David - Sure, let's sync on the work when you are ready.



From: David McLaughlin <dmclaugh...@apache.org>
Sent: Wednesday, July 19, 2017 14:10
To: dev@aurora.apache.org
Subject: Re: Redesign of the Aurora UI

Thanks for the feedback!

Joshua - I haven't tried to drop in Preact yet, but I was also planning to
throw away the prototype and starting again when it came to upstreaming it,
so as part of that we can just address incompatibilities as we go. If I was
to guess, then the only significant impact on my prototype would probably
be the reactable plugin I was using (replacement for Angular's
smart-table). But longer term I do have concerns about moving away from
what is a constantly improving and healthy ecosystem around React. So most
likely I'll hold off until a decision is made there one way or the other
(which should be within a week).


Kai - I'd be happy to coordinate and collaborate on this with others. Let
me try and finish up the CSS/UX of the pages in my prototype and from there
we can sync on who does what. Does that sound good?




On Wed, Jul 19, 2017 at 1:56 PM, Kai Huang <texasred2...@hotmail.com> wrote:

> Just a few thoughts as an aurora developer and operator:
>
>
> From my experiences with Aurora users, some persisting complaints are:
>
>   1.  The current UI is not very intuitive for the users to understand the
> task lifecycle, resource utilization of their job.
>   2.  Often times users are unaware of the new features/changes in Aurora
> Scheduler/Executor, which leads to a lot of misuse of the system.
>   3.  Users have preferences on the appearance of the scheduler/thermos UI
> due to special use cases, and ask us to customize for them(or start to
> write their own UI, which is often not recommended).
>
>
> The other major issue I see in the current UI is that it's built on an
> obsolete tech stack(AngularJS) that has all the binaries and dependencies
> in the repo. From a developer's perspective, it's a big burden to
> maintain/test the code, and make fast iterations on it.
>
>
> Currently the scheduler UI is readonly, and mainly designed for debugging
> purposes. We could have done much better to make the UI more friendly to
> the end user, empower them to discover, understand and use all the Aurora
> features, and give them more insights into the their jobs, or even the
> entire cluster. I love the idea that redesign the UI with a modern stack,
> and more importantly, every single part of the application is a module that
> you could take your own customization.
>
>
> As for the development strategy, I'm in favor of the incremental approach
> that posts one page at a time. The main benefit is that we are educating
> the developers while iterating on it, and this will improve the adoption
> rate in the long term.
>
>
> Overall I'm very interested in this work, and would like to collaborate
> with David to redesign and improve the UI.
>
>
> ____
> From: Joshua Cohen <jco...@apache.org>
> Sent: Wednesday, July 19, 2017 11:39
> To: dev@aurora.apache.org
> Subject: Re: Redesign of the Aurora UI
>
> I think this looks great overall! I'm super excited to see the UI get some
> love (and to set us up for better iteration on the UI going forward). My
> biggest concern, of course, is the current hubbub vis-a-vis Apache and the
> BSD-3+Patents license[1]. Have you tried running this against Preact to
> confirm compatibility/performance? Also note that other Facebook libraries
> have the same license problem (e.g. Immutable, Jest), so unless FB changes
> their patent grant clause, I imagine we'd have to find alternatives to
> those as well. If only we had landed this a week ago, we could've been
> grandfathered in on the license front :(.
>
> As far as the options for landing this, I'll leave that up to more active
> reviewers, but my gut says that smaller reviews will make this easier to
> parse, especially for those unfamiliar with React. That said, perhaps we
> could go with an alternate method for reviewing here, where people review
> against your fork directly and only when they're comfortable do you post
> the whole patch to reviewboard for what should, by that point, be a rubber
> stamp review?
>
> In general, fantastic work, David!
>
> [1] https://issues.apache.org/jira/browse/LEGAL-319
>
> On Wed, Jul 19, 2017 at 12:55 AM, David McLaughlin <dmclaugh...@apache.org
> >
> wrote:
>
> > Hey all,
> >
> > At Twitter we have had a long-standing desire to be able to put custom
> > widgets and other UX enhancements into the Aurora UI. Recent prototype
> work
> > to do this in a clean way has proved fruitful and I'd like to present
> this
> > approach to the 

Re: Redesign of the Aurora UI

[Santhosh Kumar Shanmugham]

This sounds great and seems like the right step forward to gain engagement
for Aurora from both operator as well as their user's perspective. I am
really excited to see how people start customizing the UI to their
particular situation.

On Wed, Jul 19, 2017 at 2:10 PM, David McLaughlin <dmclaugh...@apache.org>
wrote:

> Thanks for the feedback!
>
> Joshua - I haven't tried to drop in Preact yet, but I was also planning to
> throw away the prototype and starting again when it came to upstreaming it,
> so as part of that we can just address incompatibilities as we go. If I was
> to guess, then the only significant impact on my prototype would probably
> be the reactable plugin I was using (replacement for Angular's
> smart-table). But longer term I do have concerns about moving away from
> what is a constantly improving and healthy ecosystem around React. So most
> likely I'll hold off until a decision is made there one way or the other
> (which should be within a week).
>
>
> Kai - I'd be happy to coordinate and collaborate on this with others. Let
> me try and finish up the CSS/UX of the pages in my prototype and from there
> we can sync on who does what. Does that sound good?
>
>
>
>
> On Wed, Jul 19, 2017 at 1:56 PM, Kai Huang <texasred2...@hotmail.com>
> wrote:
>
> > Just a few thoughts as an aurora developer and operator:
> >
> >
> > From my experiences with Aurora users, some persisting complaints are:
> >
> >   1.  The current UI is not very intuitive for the users to understand
> the
> > task lifecycle, resource utilization of their job.
> >   2.  Often times users are unaware of the new features/changes in Aurora
> > Scheduler/Executor, which leads to a lot of misuse of the system.
> >   3.  Users have preferences on the appearance of the scheduler/thermos
> UI
> > due to special use cases, and ask us to customize for them(or start to
> > write their own UI, which is often not recommended).
> >
> >
> > The other major issue I see in the current UI is that it's built on an
> > obsolete tech stack(AngularJS) that has all the binaries and dependencies
> > in the repo. From a developer's perspective, it's a big burden to
> > maintain/test the code, and make fast iterations on it.
> >
> >
> > Currently the scheduler UI is readonly, and mainly designed for debugging
> > purposes. We could have done much better to make the UI more friendly to
> > the end user, empower them to discover, understand and use all the Aurora
> > features, and give them more insights into the their jobs, or even the
> > entire cluster. I love the idea that redesign the UI with a modern stack,
> > and more importantly, every single part of the application is a module
> that
> > you could take your own customization.
> >
> >
> > As for the development strategy, I'm in favor of the incremental approach
> > that posts one page at a time. The main benefit is that we are educating
> > the developers while iterating on it, and this will improve the adoption
> > rate in the long term.
> >
> >
> > Overall I'm very interested in this work, and would like to collaborate
> > with David to redesign and improve the UI.
> >
> >
> > 
> > From: Joshua Cohen <jco...@apache.org>
> > Sent: Wednesday, July 19, 2017 11:39
> > To: dev@aurora.apache.org
> > Subject: Re: Redesign of the Aurora UI
> >
> > I think this looks great overall! I'm super excited to see the UI get
> some
> > love (and to set us up for better iteration on the UI going forward). My
> > biggest concern, of course, is the current hubbub vis-a-vis Apache and
> the
> > BSD-3+Patents license[1]. Have you tried running this against Preact to
> > confirm compatibility/performance? Also note that other Facebook
> libraries
> > have the same license problem (e.g. Immutable, Jest), so unless FB
> changes
> > their patent grant clause, I imagine we'd have to find alternatives to
> > those as well. If only we had landed this a week ago, we could've been
> > grandfathered in on the license front :(.
> >
> > As far as the options for landing this, I'll leave that up to more active
> > reviewers, but my gut says that smaller reviews will make this easier to
> > parse, especially for those unfamiliar with React. That said, perhaps we
> > could go with an alternate method for reviewing here, where people review
> > against your fork directly and only when they're comfortable do you post
> > the whole patch to reviewboard for what should, by that point, be a
> rubber
> > 

Re: Redesign of the Aurora UI

[David McLaughlin]

Thanks for the feedback!

Joshua - I haven't tried to drop in Preact yet, but I was also planning to
throw away the prototype and starting again when it came to upstreaming it,
so as part of that we can just address incompatibilities as we go. If I was
to guess, then the only significant impact on my prototype would probably
be the reactable plugin I was using (replacement for Angular's
smart-table). But longer term I do have concerns about moving away from
what is a constantly improving and healthy ecosystem around React. So most
likely I'll hold off until a decision is made there one way or the other
(which should be within a week).


Kai - I'd be happy to coordinate and collaborate on this with others. Let
me try and finish up the CSS/UX of the pages in my prototype and from there
we can sync on who does what. Does that sound good?




On Wed, Jul 19, 2017 at 1:56 PM, Kai Huang <texasred2...@hotmail.com> wrote:

> Just a few thoughts as an aurora developer and operator:
>
>
> From my experiences with Aurora users, some persisting complaints are:
>
>   1.  The current UI is not very intuitive for the users to understand the
> task lifecycle, resource utilization of their job.
>   2.  Often times users are unaware of the new features/changes in Aurora
> Scheduler/Executor, which leads to a lot of misuse of the system.
>   3.  Users have preferences on the appearance of the scheduler/thermos UI
> due to special use cases, and ask us to customize for them(or start to
> write their own UI, which is often not recommended).
>
>
> The other major issue I see in the current UI is that it's built on an
> obsolete tech stack(AngularJS) that has all the binaries and dependencies
> in the repo. From a developer's perspective, it's a big burden to
> maintain/test the code, and make fast iterations on it.
>
>
> Currently the scheduler UI is readonly, and mainly designed for debugging
> purposes. We could have done much better to make the UI more friendly to
> the end user, empower them to discover, understand and use all the Aurora
> features, and give them more insights into the their jobs, or even the
> entire cluster. I love the idea that redesign the UI with a modern stack,
> and more importantly, every single part of the application is a module that
> you could take your own customization.
>
>
> As for the development strategy, I'm in favor of the incremental approach
> that posts one page at a time. The main benefit is that we are educating
> the developers while iterating on it, and this will improve the adoption
> rate in the long term.
>
>
> Overall I'm very interested in this work, and would like to collaborate
> with David to redesign and improve the UI.
>
>
> ________
> From: Joshua Cohen <jco...@apache.org>
> Sent: Wednesday, July 19, 2017 11:39
> To: dev@aurora.apache.org
> Subject: Re: Redesign of the Aurora UI
>
> I think this looks great overall! I'm super excited to see the UI get some
> love (and to set us up for better iteration on the UI going forward). My
> biggest concern, of course, is the current hubbub vis-a-vis Apache and the
> BSD-3+Patents license[1]. Have you tried running this against Preact to
> confirm compatibility/performance? Also note that other Facebook libraries
> have the same license problem (e.g. Immutable, Jest), so unless FB changes
> their patent grant clause, I imagine we'd have to find alternatives to
> those as well. If only we had landed this a week ago, we could've been
> grandfathered in on the license front :(.
>
> As far as the options for landing this, I'll leave that up to more active
> reviewers, but my gut says that smaller reviews will make this easier to
> parse, especially for those unfamiliar with React. That said, perhaps we
> could go with an alternate method for reviewing here, where people review
> against your fork directly and only when they're comfortable do you post
> the whole patch to reviewboard for what should, by that point, be a rubber
> stamp review?
>
> In general, fantastic work, David!
>
> [1] https://issues.apache.org/jira/browse/LEGAL-319
>
> On Wed, Jul 19, 2017 at 12:55 AM, David McLaughlin <dmclaugh...@apache.org
> >
> wrote:
>
> > Hey all,
> >
> > At Twitter we have had a long-standing desire to be able to put custom
> > widgets and other UX enhancements into the Aurora UI. Recent prototype
> work
> > to do this in a clean way has proved fruitful and I'd like to present
> this
> > approach to the community and get feedback on the overall approach.
> >
> > The basic approach is simple:
> >
> > 1) Use the node plugin for gradle to bootstrap a modern web development
> > build system using webpack and npm.
> >

Re: Redesign of the Aurora UI

[Kai Huang]

Just a few thoughts as an aurora developer and operator:


>From my experiences with Aurora users, some persisting complaints are:

  1.  The current UI is not very intuitive for the users to understand the task 
lifecycle, resource utilization of their job.
  2.  Often times users are unaware of the new features/changes in Aurora 
Scheduler/Executor, which leads to a lot of misuse of the system.
  3.  Users have preferences on the appearance of the scheduler/thermos UI due 
to special use cases, and ask us to customize for them(or start to write their 
own UI, which is often not recommended).


The other major issue I see in the current UI is that it's built on an obsolete 
tech stack(AngularJS) that has all the binaries and dependencies in the repo. 
From a developer's perspective, it's a big burden to maintain/test the code, 
and make fast iterations on it.


Currently the scheduler UI is readonly, and mainly designed for debugging 
purposes. We could have done much better to make the UI more friendly to the 
end user, empower them to discover, understand and use all the Aurora features, 
and give them more insights into the their jobs, or even the entire cluster. I 
love the idea that redesign the UI with a modern stack, and more importantly, 
every single part of the application is a module that you could take your own 
customization.


As for the development strategy, I'm in favor of the incremental approach that 
posts one page at a time. The main benefit is that we are educating the 
developers while iterating on it, and this will improve the adoption rate in 
the long term.


Overall I'm very interested in this work, and would like to collaborate with 
David to redesign and improve the UI.



From: Joshua Cohen <jco...@apache.org>
Sent: Wednesday, July 19, 2017 11:39
To: dev@aurora.apache.org
Subject: Re: Redesign of the Aurora UI

I think this looks great overall! I'm super excited to see the UI get some
love (and to set us up for better iteration on the UI going forward). My
biggest concern, of course, is the current hubbub vis-a-vis Apache and the
BSD-3+Patents license[1]. Have you tried running this against Preact to
confirm compatibility/performance? Also note that other Facebook libraries
have the same license problem (e.g. Immutable, Jest), so unless FB changes
their patent grant clause, I imagine we'd have to find alternatives to
those as well. If only we had landed this a week ago, we could've been
grandfathered in on the license front :(.

As far as the options for landing this, I'll leave that up to more active
reviewers, but my gut says that smaller reviews will make this easier to
parse, especially for those unfamiliar with React. That said, perhaps we
could go with an alternate method for reviewing here, where people review
against your fork directly and only when they're comfortable do you post
the whole patch to reviewboard for what should, by that point, be a rubber
stamp review?

In general, fantastic work, David!

[1] https://issues.apache.org/jira/browse/LEGAL-319

On Wed, Jul 19, 2017 at 12:55 AM, David McLaughlin <dmclaugh...@apache.org>
wrote:

> Hey all,
>
> At Twitter we have had a long-standing desire to be able to put custom
> widgets and other UX enhancements into the Aurora UI. Recent prototype work
> to do this in a clean way has proved fruitful and I'd like to present this
> approach to the community and get feedback on the overall approach.
>
> The basic approach is simple:
>
> 1) Use the node plugin for gradle to bootstrap a modern web development
> build system using webpack and npm.
> 2) Use a modern JS view library like ReactJS (or Preact depending on the
> Facebook+Patents license issues for Apache projects) to have all UI
> components defined as ES6 modules.
> 3) Use webpack to set up a custom plugin directory that modules should be
> loaded from first. This would allow anyone with custom Scheduler builds to
> also drop in custom JS files to replace selected OSS UI components.
> Example, you could drop a "Navigation.js" into the plugin directory, and
> the build system will use that module instead whenever it sees a
>  used by parent components. Basically: it's a hybrid
> plugin/dependency injection (similar to Guice) approach to customization of
> the UI.
>
> The nice thing here is that since every single part of the application is a
> module you could take your customizations to any level of precision that
> you need. You could replace (or add) key pages or components, or just keep
> it as simple as adding some helper text to one of the pages.
>
> Some use cases this could enable:
>
> * A  widget that you can replace with configuration
> parsing that understands special processes and/or custom executors.
> * Integration with your telemetry stacks to add resource utilization
> feedback or other perfo

Re: Redesign of the Aurora UI

[Joshua Cohen]

I think this looks great overall! I'm super excited to see the UI get some
love (and to set us up for better iteration on the UI going forward). My
biggest concern, of course, is the current hubbub vis-a-vis Apache and the
BSD-3+Patents license[1]. Have you tried running this against Preact to
confirm compatibility/performance? Also note that other Facebook libraries
have the same license problem (e.g. Immutable, Jest), so unless FB changes
their patent grant clause, I imagine we'd have to find alternatives to
those as well. If only we had landed this a week ago, we could've been
grandfathered in on the license front :(.

As far as the options for landing this, I'll leave that up to more active
reviewers, but my gut says that smaller reviews will make this easier to
parse, especially for those unfamiliar with React. That said, perhaps we
could go with an alternate method for reviewing here, where people review
against your fork directly and only when they're comfortable do you post
the whole patch to reviewboard for what should, by that point, be a rubber
stamp review?

In general, fantastic work, David!

[1] https://issues.apache.org/jira/browse/LEGAL-319

On Wed, Jul 19, 2017 at 12:55 AM, David McLaughlin 
wrote:

> Hey all,
>
> At Twitter we have had a long-standing desire to be able to put custom
> widgets and other UX enhancements into the Aurora UI. Recent prototype work
> to do this in a clean way has proved fruitful and I'd like to present this
> approach to the community and get feedback on the overall approach.
>
> The basic approach is simple:
>
> 1) Use the node plugin for gradle to bootstrap a modern web development
> build system using webpack and npm.
> 2) Use a modern JS view library like ReactJS (or Preact depending on the
> Facebook+Patents license issues for Apache projects) to have all UI
> components defined as ES6 modules.
> 3) Use webpack to set up a custom plugin directory that modules should be
> loaded from first. This would allow anyone with custom Scheduler builds to
> also drop in custom JS files to replace selected OSS UI components.
> Example, you could drop a "Navigation.js" into the plugin directory, and
> the build system will use that module instead whenever it sees a
>  used by parent components. Basically: it's a hybrid
> plugin/dependency injection (similar to Guice) approach to customization of
> the UI.
>
> The nice thing here is that since every single part of the application is a
> module you could take your customizations to any level of precision that
> you need. You could replace (or add) key pages or components, or just keep
> it as simple as adding some helper text to one of the pages.
>
> Some use cases this could enable:
>
> * A  widget that you can replace with configuration
> parsing that understands special processes and/or custom executors.
> * Integration with your telemetry stacks to add resource utilization
> feedback or other performance indicators into the job page.
> * Adding custom stats tracking widgets for internal analytics.
>
>
> Some points from my prototype work:
>
> 1) Switching to npm for JS asset management allows us to delete the vast
> majority of our 3rdparty assets that we had to copy into the repo. We can
> most likely extend this to remove all of them, including our CSS frameworks
> like Bootstrap.
> 2) ReactJS was proposed before (by Joshua Cohen) and ironically I was one
> of the main voices against. The argument I used at the time was lack of
> familiarity with the stack within the community, as well as concerns about
> the complexity. So what changed? Almost the entire Aurora team at Twitter
> has ReactJS experience now - from working on our internal Continuous
> Delivery system. I also believe that as a view layer, ReactJS is
> conceptually very simple and that most of the complexity in these modern JS
> applications is in a state layer like Redux. The Aurora UI is currently
> read-only and relatively simple, and we can avoid much of the complexity by
> just using component state instead.
> 3) Unit tests are trivial to add as part of this work.
> 4) I'd like to move to modern, maintainable CSS tooling - SASS - as part of
> this.
>
>
> My prototype is (hastily) published here:
>
> https://github.com/DavidMcLaughlin/aurora/tree/js-build
>
>
> Some highlighted changes:
>
> 1) Bootstrapping the modern UI toolchain from gradle:
> https://github.com/DavidMcLaughlin/aurora/blob/js-build/build.gradle#L452-
> L459
>
> 2) The webpack configuration to enable loading from a plugin directory (and
> SASS):
> https://github.com/DavidMcLaughlin/aurora/blob/js-build/webpack.config.js
>
> 3) The Hello, World of the plugin mechanism:
>
> https://github.com/DavidMcLaughlin/aurora/blob/js-build/src/main/js/
> components/Hello.js
>
> Is replaced by:
>
> https://github.com/DavidMcLaughlin/aurora/blob/
> js-build/plugin/js/components/Hello.js
>
> 4) Unit test example:
>