[Request] Write access to the incubator wiki

[Joo Yeon Kim]

Hi,

Please grant me write access to the incubator wiki:
https://wiki.apache.org/incubator.

My user name is JooYeonKim.

Thank you :)

- Joo Yeon Kim

Re: Apache Policy Quiz

[Justin Mclean]

Hi,

> I think the quiz is a great idea.

Thanks for trying it out.

> I would really like to see the correct
> answers when the provided answer is wrong. Having the hints is very useful,
> and should be kept, but I would still like to also see the correct answers
> instead of just having to pour through documentation…

It now show the wrong answers and the hints (i.e. why they were wrong) on the 
answer page.

Thanks,
Justin
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

[PROPOSAL] Onyx - proposal for Apache Incubation

[Byung-Gon Chun]

Dear Apache Incubator Community,

Please accept the following proposal for presentation and discussion:
https://wiki.apache.org/incubator/OnyxProposal

Onyx is a data processing system that aims to flexibly control the runtime
behaviors of a job to adapt to varying deployment characteristics (e.g.,
harnessing transient resources in datacenters, cross-datacenter deployment,
changing runtime based on job characteristics, etc.). Onyx provides ways to
extend the system’s capabilities and incorporate the extensions to the
flexible job execution.
Onyx translates a user program (e.g., Apache Beam, Apache Spark) into an
Intermediate Representation (IR) DAG, which Onyx optimizes and deploys
based on a deployment policy.

I've attached the proposal below.

Best regards,
Byung-Gon Chun

= OnyxProposal =

== Abstract ==
Onyx is a data processing system for flexible employment with
different execution scenarios for various deployment characteristics
on clusters.

== Proposal ==
Today, there is a wide variety of data processing systems with
different designs for better performance and datacenter efficiency.
They include processing data on specific resource environments and
running jobs with specific attributes. Although each system
successfully solves the problems it targets, most systems are designed
in the way that runtime behaviors are built tightly inside the system
core to hide the complexity of distributed computing. This makes it
hard for a single system to support different deployment
characteristics with different runtime behaviors without substantial
effort.

Onyx is a data processing system that aims to flexibly control the
runtime behaviors of a job to adapt to varying deployment
characteristics. Moreover, it provides a means of extending the
system’s capabilities and incorporating the extensions to the flexible
job execution.

In order to be able to easily modify runtime behaviors to adapt to
varying deployment characteristics, Onyx exposes runtime behaviors to
be flexibly configured and modified at both compile-time and runtime
through a set of high-level graph pass interfaces.

We hope to contribute to the big data processing community by enabling
more flexibility and extensibility in job executions. Furthermore, we
can benefit more together as a community when we work together as a
community to mature the system with more use cases and understanding
of diverse deployment characteristics. The Apache Software Foundation
is the perfect place to achieve these aspirations.

== Background ==
Many data processing systems have distinctive runtime behaviors
optimized and configured for specific deployment characteristics like
different resource environments and for handling special job
attributes.

For example, much research have been conducted to overcome the
challenge of running data processing jobs on cheap, unreliable
transient resources. Likewise, techniques for disaggregating different
types of resources, like memory, CPU and GPU, are being actively
developed to use datacenter resources more efficiently. Many
researchers are also working to run data processing jobs in even more
diverse environments, such as across distant datacenters. Similarly,
for special job attributes, many works take different approaches, such
as runtime optimization, to solve problems like data skew, and to
optimize systems for data processing jobs with small-scale input data.

Although each of the systems performs well with the jobs and in the
environments they target, they perform poorly with unconsidered cases,
and do not consider supporting multiple deployment characteristics on
a single system in their designs.

For an application writer to optimize an application to perform well
on a certain system engraved with its underlying behaviors, it
requires a deep understanding of the system itself, which is an
overhead that often requires a lot of time and effort. Moreover, for a
developer to modify such system behaviors, it requires modifications
of the system core, which requires an even deeper understanding of the
system itself.

With this background, Onyx is designed to represent all of its jobs as
an Intermediate Representation (IR) DAG. In the Onyx compiler, user
applications from various programming models (ex. Apache Beam) are
submitted, transformed to an IR DAG, and optimized/customized for the
deployment characteristics. In the IR DAG optimization phase, the DAG
is modified through a series of compiler “passes” which reshape or
annotate the DAG with an expression of the underlying runtime
behaviors. The IR DAG is then submitted as an execution plan for the
Onyx runtime. The runtime includes the unmodified parts of data
processing in the backbone which is transparently integrated with
configurable components exposed for further extension.

== Rationale ==
Onyx’s vision lies in providing means for flexibly supporting a wide
variety of job execution scenarios for users while facilitating system
developers to extend the 

Re: [Request] Write access to the incubator wiki

[Byung-Gon Chun]

John, thanks for the quick response!


On Fri, Jan 26, 2018 at 7:04 AM, John D. Ament 
wrote:

> I've added you, happy editing!
>
> On Thu, Jan 25, 2018 at 4:36 PM Byung-Gon Chun  wrote:
>
> > Hi,
> >
> > Please grant me write access to the incubator wiki:
> > https://wiki.apache.org/incubator.
> >
> > My user name is Byung-gonChun.
> >
> > Thanks!
> > -Gon
> >
> > --
> > Byung-Gon Chun
> >
>



-- 
Byung-Gon Chun

Re: [Request] Write access to the incubator wiki

[John D. Ament]

I've added you, happy editing!

On Thu, Jan 25, 2018 at 4:36 PM Byung-Gon Chun  wrote:

> Hi,
>
> Please grant me write access to the incubator wiki:
> https://wiki.apache.org/incubator.
>
> My user name is Byung-gonChun.
>
> Thanks!
> -Gon
>
> --
> Byung-Gon Chun
>

[Request] Write access to the incubator wiki

[Byung-Gon Chun]

Hi,

Please grant me write access to the incubator wiki:
https://wiki.apache.org/incubator.

My user name is Byung-gonChun.

Thanks!
-Gon

-- 
Byung-Gon Chun

Re: [VOTE] Apache DataFu 1.3.3 release RC1

[Jakob Homan]

+1 (binding)
- Sigs/asc look good
- NOTICE/LICENSE/DISCLAIMER look good
- Licenses look good
- Tests succeed
- Gradle binaries not included

Good work.
-Jakob

On 24 January 2018 at 13:01, Justin Mclean  wrote:
> Hi,
>
>> Hi, it's been almost 72 hours since the vote was opened.  How many votes do
>> we need for this to pass?  Can other folks take a look if necessary?
>
> I suggest asking your mentor who are IPMC member to vote.
>
> Thanks,
> Justin
>
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Apache Policy Quiz

[Adina Crainiceanu]

Hi,

I think the quiz is a great idea. I would really like to see the correct
answers when the provided answer is wrong. Having the hints is very useful,
and should be kept, but I would still like to also see the correct answers
instead of just having to pour through documentation...

Thanks Justin for your work.
Adina

On Thu, Jan 25, 2018 at 7:34 AM, Justin Mclean  wrote:

> Hi,
>
> >> Maybe let the user choose the level at the start?
> >
> > Also a good idea.
>
> And done - should make it a little easier for people but I may of cranked
> it a little too high. Again feedback welcome.
>
> Thanks,
> Justin
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>


-- 
Dr. Adina Crainiceanu
Associate Professor
Computer Science Department
United States Naval Academy
410-293-6822
ad...@usna.edu
http://www.usna.edu/Users/cs/adina/

Re: Apache Policy Quiz

[Justin Mclean]

Hi,

>> Maybe let the user choose the level at the start?
> 
> Also a good idea.

And done - should make it a little easier for people but I may of cranked it a 
little too high. Again feedback welcome.

Thanks,
Justin
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Apache Policy Quiz

[Justin Mclean]

Hi,

> Maybe let the user choose the level at the start?

Also a good idea.

Thanks,
Justin

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Apache Policy Quiz

[Justin Mclean]

Hi,

> Thanks for having a go, Justin! We need more ways to explain license and
> distribution challenges.

Thanks for giving it a try.,

> I also got just 1 of 5, even on multiple tries on the same questions. Also
> the fact that the correct answer(s) is not shown graphically is confusing.

That was by intention so people read the hints and follow the policy links but 
perhaps that’s too hard.

> i thinks such a quiz should not deliberately cover all the subtleties which
> as we see here we are not in 100% agreement on; that just make people
> conclude that "It's too complicated”.

Good point.

> Which of these need to include/show LICENCE and NOTICE files for an ASF
> release?

Simple but I’m concerned that this may be too simple?

> Which of the below should be included in NOTICE for contributions to Apache
> Foo under Apache License 2.0?

Makes sense.

> Here I left out "Other Apache Projects which code was reused" and "Code
> contributed in software grant" because those can be more complicated

But however they are quite common situations so I think need to be covered.

Thanks,
Justin
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Apache Policy Quiz

[Christofer Dutz]

Maybe let the user choose the level at the start?

I agree that new people could get confused, but being able to increase the 
level could help spread the details. As in the incubator these details are what 
usually result in -1 votes. 

Chris

Am 25.01.18, 11:16 schrieb "Stian Soiland-Reyes" :

Thanks for having a go, Justin! We need more ways to explain license and
distribution challenges.


I also got just 1 of 5, even on multiple tries on the same questions. Also
the fact that the correct answer(s) is not shown graphically is confusing.

i thinks such a quiz should not deliberately cover all the subtleties which
as we see here we are not in 100% agreement on; that just make people
conclude that "It's too complicated".

The quiz should rather cover the obvious bits so that this could be used by
newcomers to the Incubator, rather than try to catch out ASF members.

I would avoid tricky multiple choice, except in obvious multi-things like
"which licenses allowed" (which I thought was a good, more obvious
question).

Other kind of simpler questions could be:


Which of these need to include/show LICENCE and NOTICE files for an ASF
release?


Source code release on www.apache.org/dist
Binary release zip on www.apache.org/dist
Binary JARs in Maven Central
Windows installer at Launchpad.net
OSX disk image in Apple store
Source code release in Debian

Here all of the above is right answer. I left out "Source code repository
in git" because at least in Incubator we allow slight diversions here (and
also repos would include older commits possibly pre-ASF).


Which of the below should be included in NOTICE for contributions to Apache
Foo under Apache License 2.0?

1 Apache Software Foundation
2 Apache Foo
3 Apache Foo PMC members
4 Apache Foo Committers who contributed to release
5 Contributors who submitted patches to Apache Foo
6 NOTICE of Apache-licensed code that was included from outside ASF

Only 1,2,6 are correct.


Here I left out "Other Apache Projects which code was reused" and "Code
contributed in software grant" because those can be more complicated.



On 25 Jan 2018 8:02 am, "Christofer Dutz"  wrote:

> In general I think it's a great Idea.
> I would really like to bring this to my new Apache colleagues in the PLC4X
> project ... this way they could get up to speed with the formal stuff.
>
> But I think the tool does need a little optimization ;-)
>
> I think from 5 questions I got about 1 right ;-)
>
> But I guess it was mainly cause I interpreted the answers as if they are
> aggregated to form a correct answer.
> For example I interpreted the answers "3 +1 votes" and "more +1 than -1"
> as being ANDed to form the correct answer, but Justin told me "more +1 
than
> -1" is correct as you could also have "3 +1 and 10 -1 votes". This was
> not obvious to me and I think if the aggregation of all answers form the
> correct one, It would be more intuitive. I would really like a quiz like
> that that lists up different parts and all that are correct have to be
> clicked. It requires to know the same information, but doesn't make you 
mad
> if you missed some little trick ;-)
>
> Chris
>
> Am 25.01.18, 08:55 schrieb "Justin Mclean" :
>
> Hi,
>
> > I return to my point: "No" was the best answer (tho its qualifiers
> were
> > wrong), and the "Yes" was wrong for that question.
>
> Yes I agree No is the best answer (and is the correct answer is just
> about every single case) and Yes is wrong in that question. Is "Yes but
> only for some common build tools.” correct or not, currently I have it 
down
> as correct, but if it’s not as your saying then the text (and title) at
> [1] I think would need to change. May be best to bring up on legal 
discuss?
>
> Thanks,
> Justin
>
> 1. https://www.apache.org/legal/resolved.html#build-tools
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>
>
>

Re: Apache Policy Quiz

[Stian Soiland-Reyes]

Thanks for having a go, Justin! We need more ways to explain license and
distribution challenges.


I also got just 1 of 5, even on multiple tries on the same questions. Also
the fact that the correct answer(s) is not shown graphically is confusing.

i thinks such a quiz should not deliberately cover all the subtleties which
as we see here we are not in 100% agreement on; that just make people
conclude that "It's too complicated".

The quiz should rather cover the obvious bits so that this could be used by
newcomers to the Incubator, rather than try to catch out ASF members.

I would avoid tricky multiple choice, except in obvious multi-things like
"which licenses allowed" (which I thought was a good, more obvious
question).

Other kind of simpler questions could be:


Which of these need to include/show LICENCE and NOTICE files for an ASF
release?


Source code release on www.apache.org/dist
Binary release zip on www.apache.org/dist
Binary JARs in Maven Central
Windows installer at Launchpad.net
OSX disk image in Apple store
Source code release in Debian

Here all of the above is right answer. I left out "Source code repository
in git" because at least in Incubator we allow slight diversions here (and
also repos would include older commits possibly pre-ASF).


Which of the below should be included in NOTICE for contributions to Apache
Foo under Apache License 2.0?

1 Apache Software Foundation
2 Apache Foo
3 Apache Foo PMC members
4 Apache Foo Committers who contributed to release
5 Contributors who submitted patches to Apache Foo
6 NOTICE of Apache-licensed code that was included from outside ASF

Only 1,2,6 are correct.


Here I left out "Other Apache Projects which code was reused" and "Code
contributed in software grant" because those can be more complicated.



On 25 Jan 2018 8:02 am, "Christofer Dutz"  wrote:

> In general I think it's a great Idea.
> I would really like to bring this to my new Apache colleagues in the PLC4X
> project ... this way they could get up to speed with the formal stuff.
>
> But I think the tool does need a little optimization ;-)
>
> I think from 5 questions I got about 1 right ;-)
>
> But I guess it was mainly cause I interpreted the answers as if they are
> aggregated to form a correct answer.
> For example I interpreted the answers "3 +1 votes" and "more +1 than -1"
> as being ANDed to form the correct answer, but Justin told me "more +1 than
> -1" is correct as you could also have "3 +1 and 10 -1 votes". This was
> not obvious to me and I think if the aggregation of all answers form the
> correct one, It would be more intuitive. I would really like a quiz like
> that that lists up different parts and all that are correct have to be
> clicked. It requires to know the same information, but doesn't make you mad
> if you missed some little trick ;-)
>
> Chris
>
> Am 25.01.18, 08:55 schrieb "Justin Mclean" :
>
> Hi,
>
> > I return to my point: "No" was the best answer (tho its qualifiers
> were
> > wrong), and the "Yes" was wrong for that question.
>
> Yes I agree No is the best answer (and is the correct answer is just
> about every single case) and Yes is wrong in that question. Is "Yes but
> only for some common build tools.” correct or not, currently I have it down
> as correct, but if it’s not as your saying then the text (and title) at
> [1] I think would need to change. May be best to bring up on legal discuss?
>
> Thanks,
> Justin
>
> 1. https://www.apache.org/legal/resolved.html#build-tools
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>
>
>