Re: Proposal

[Michael Wechner]

I agree, but I think we need to give some more concrete guidance how to 
develop an open community and develop in the open, because although it 
is clear to ASF people, I think it is often not so clear for many other 
people


@msacks: Did you already have a look at 
https://incubator.apache.org/cookbook/ ?


HTH

Michael

Am 17.05.22 um 07:56 schrieb Dave Fisher:

Develop an open community and develop in the open. If the community starts to 
grow then come back and ask for guidance in how to make your open source 
community an ASF community.

Did I mention open and community enough?

All the best,
Dave

Sent from my iPhone


On May 16, 2022, at 10:47 PM, m sacks  wrote:

Not sure if this made it:
Just a term of endearment, mot taken to be meant literally.

Sure.

Initially the community would be a private group put together by me.

Then we can discuss building it once others have decided if it’s even a
useful application first?


On Mon, May 16, 2022 at 10:20 PM Daniel Widdis  wrote:

I'm not an ASF warlord or general.  In fact, I don't think such things
exist. It's about community.  Decisions are made by communities.  Warlords,
generals, and benevolent dictators don't fit well.

Related, I don't see anything "community" in your post. You state "I" have
got code, not "we".

You can have the best code in the universe, but if you don't have a
community developing it, it's not really a good fit here.

So tell us less about your code and more about your community developing
it.


On 5/16/22, 10:05 PM, "m sacks"  wrote:

I have some gpt3 based python code to simulate leonardo da vinci as a
chatbot proof of concept. I think it could be useful, but i am not
sure, so
i leave it to the council of ASF warlords and generals to decide if the
code should be incubated?


I have not shared sources as of yet.






-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org




-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Proposal

[Dave Fisher]

Develop an open community and develop in the open. If the community starts to 
grow then come back and ask for guidance in how to make your open source 
community an ASF community.

Did I mention open and community enough?

All the best,
Dave

Sent from my iPhone

> On May 16, 2022, at 10:47 PM, m sacks  wrote:
> 
> Not sure if this made it:
> Just a term of endearment, mot taken to be meant literally.
> 
> Sure.
> 
> Initially the community would be a private group put together by me.
> 
> Then we can discuss building it once others have decided if it’s even a
> useful application first?
> 
>> On Mon, May 16, 2022 at 10:20 PM Daniel Widdis  wrote:
>> 
>> I'm not an ASF warlord or general.  In fact, I don't think such things
>> exist. It's about community.  Decisions are made by communities.  Warlords,
>> generals, and benevolent dictators don't fit well.
>> 
>> Related, I don't see anything "community" in your post. You state "I" have
>> got code, not "we".
>> 
>> You can have the best code in the universe, but if you don't have a
>> community developing it, it's not really a good fit here.
>> 
>> So tell us less about your code and more about your community developing
>> it.
>> 
>> 
>> On 5/16/22, 10:05 PM, "m sacks"  wrote:
>> 
>>I have some gpt3 based python code to simulate leonardo da vinci as a
>>chatbot proof of concept. I think it could be useful, but i am not
>> sure, so
>>i leave it to the council of ASF warlords and generals to decide if the
>>code should be incubated?
>> 
>> 
>>I have not shared sources as of yet.
>> 
>> 
>> 
>> 


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Proposal

[m sacks]

Not sure if this made it:
Just a term of endearment, mot taken to be meant literally.

Sure.

Initially the community would be a private group put together by me.

Then we can discuss building it once others have decided if it’s even a
useful application first?

On Mon, May 16, 2022 at 10:20 PM Daniel Widdis  wrote:

> I'm not an ASF warlord or general.  In fact, I don't think such things
> exist. It's about community.  Decisions are made by communities.  Warlords,
> generals, and benevolent dictators don't fit well.
>
> Related, I don't see anything "community" in your post. You state "I" have
> got code, not "we".
>
> You can have the best code in the universe, but if you don't have a
> community developing it, it's not really a good fit here.
>
> So tell us less about your code and more about your community developing
> it.
>
>
> On 5/16/22, 10:05 PM, "m sacks"  wrote:
>
> I have some gpt3 based python code to simulate leonardo da vinci as a
> chatbot proof of concept. I think it could be useful, but i am not
> sure, so
> i leave it to the council of ASF warlords and generals to decide if the
> code should be incubated?
>
>
> I have not shared sources as of yet.
>
>
>
>

Re: [DISCUSSION] Incubating Proposal of Firestorm

[Saisai Shao]

Thanks Weiwei, let me add you to the mentor list.

Best regards,
Jerry

Weiwei Yang  于2022年5月17日周二 12:18写道：

> +1
> This is an interesting project, I'd be happy to help the project's
> incubation process.
> Let me know if you need my help, Thanks
>
> On Mon, May 16, 2022 at 8:09 PM Saisai Shao 
> wrote:
>
> > Got it, thanks a lot Justin.
> >
> > Best regards,
> > Jerry
> >
> > Justin Mclean  于2022年5月17日周二 10:59写道：
> >
> > > Hi,
> > >
> > > The new project name doesn’t need to be a registered trademark, but you
> > > would still need to pick a name that is now likely to have any
> trademark
> > > issues.
> > >
> > > The project may want to get the ASF to register the mark at a later
> date,
> > > but it wold be best to talk abut that on the trademarks@ list.
> > >
> > > Kind Regards,
> > > Justin
> > >
> > >
> > > -
> > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> > > For additional commands, e-mail: general-h...@incubator.apache.org
> > >
> > >
> >
>

Re: Proposal

[Daniel Widdis]

I'm not an ASF warlord or general.  In fact, I don't think such things exist. 
It's about community.  Decisions are made by communities.  Warlords, generals, 
and benevolent dictators don't fit well.

Related, I don't see anything "community" in your post. You state "I" have got 
code, not "we".

You can have the best code in the universe, but if you don't have a community 
developing it, it's not really a good fit here.

So tell us less about your code and more about your community developing it.


On 5/16/22, 10:05 PM, "m sacks"  wrote:

I have some gpt3 based python code to simulate leonardo da vinci as a
chatbot proof of concept. I think it could be useful, but i am not sure, so
i leave it to the council of ASF warlords and generals to decide if the
code should be incubated?


I have not shared sources as of yet.




-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Proposal

[m sacks]

I have some gpt3 based python code to simulate leonardo da vinci as a
chatbot proof of concept. I think it could be useful, but i am not sure, so
i leave it to the council of ASF warlords and generals to decide if the
code should be incubated?


I have not shared sources as of yet.

On Mon, May 16, 2022 at 9:34 PM m sacks  wrote:

> Test 1
>
> 2
>
> 3
>
> 4
>
> 5
>
> 6
>
> 7
>
> 8
>
> 9
>
> 0
>

Test

[m sacks]

Test 1

2

3

4

5

6

7

8

9

0

Re: [DISCUSSION] Incubating Proposal of Firestorm

[Weiwei Yang]

+1
This is an interesting project, I'd be happy to help the project's
incubation process.
Let me know if you need my help, Thanks

On Mon, May 16, 2022 at 8:09 PM Saisai Shao  wrote:

> Got it, thanks a lot Justin.
>
> Best regards,
> Jerry
>
> Justin Mclean  于2022年5月17日周二 10:59写道：
>
> > Hi,
> >
> > The new project name doesn’t need to be a registered trademark, but you
> > would still need to pick a name that is now likely to have any trademark
> > issues.
> >
> > The project may want to get the ASF to register the mark at a later date,
> > but it wold be best to talk abut that on the trademarks@ list.
> >
> > Kind Regards,
> > Justin
> >
> >
> > -
> > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> > For additional commands, e-mail: general-h...@incubator.apache.org
> >
> >
>

Re: [VOTE] Release Apache Linkis (Incubating) 1.1.1-RC1

[cas...@apache.org]

 Hi Justin,

Thanks for your reminder,

About "how contributions to the project are licensed",I found some
useful information:

According to py4j CONTRIBUTING.rst[1]:
"Py4J does not have an official Contributor License Agreement (CLA),
but it is assumed that as soon as you make a contribution (patch, code
suggestion through any medium, pull requests) to Py4J, you accept that
your code will be redistributed under the current license used by
Py4J, i.e., the new BSD license. This is an irrevocable right to
ensure that developers can use Py4J without the fear of seeing parts
removed in the future."

All code file will be redistributed under the new BSD license.

By the way, we have upgraded the py4j version to py4j-0.10.7-src.zip, and
the backport.py file is removed.

According to the requirements of py4j's LICENSE[2],
We have added the license description of py4j[3] and the license
file[4] in the project source code.

[1]https://github.com/py4j/py4j/blob/master/CONTRIBUTING.rst
[2]https://github.com/py4j/py4j/blob/master/LICENSE.txt
[3]https://github.com/apache/incubator-linkis/blob/dev-1.1.1/LICENSE
[4]
https://github.com/apache/incubator-linkis/blob/dev-1.1.1/licenses/LICENSE-py4j-0.10.9.5-src.txt


Best Regards!

Chen Xia


Justin Mclean  于2022年5月17日周二 06:04写道：

> Hi,
>
> > But the py file is still missing license headers and have different
> > authors[2]. We don't know how to solve this problem.
>
> You may need to look at the history f the files to see if that provide
> some clues or how contributions to the project are licensed. You could also
> contact the project and ask them how the files are licensed.
>
> Kind Regards,
> Justin
>
>
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>

Re: [DISCUSSION] Incubating Proposal of Firestorm

[Saisai Shao]

Got it, thanks a lot Justin.

Best regards,
Jerry

Justin Mclean  于2022年5月17日周二 10:59写道：

> Hi,
>
> The new project name doesn’t need to be a registered trademark, but you
> would still need to pick a name that is now likely to have any trademark
> issues.
>
> The project may want to get the ASF to register the mark at a later date,
> but it wold be best to talk abut that on the trademarks@ list.
>
> Kind Regards,
> Justin
>
>
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>

Re: [DISCUSSION] Incubating Proposal of Firestorm

[Justin Mclean]

Hi,

The new project name doesn’t need to be a registered trademark, but you would 
still need to pick a name that is now likely to have any trademark issues.

The project may want to get the ASF to register the mark at a later date, but 
it wold be best to talk abut that on the trademarks@ list. 

Kind Regards,
Justin


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSSION] Incubating Proposal of Firestorm

[Saisai Shao]

Hi Daniel and Justin，

Sorry about one more questions. Seems like the name "firestorm" has some
conflicts in USPTO searching and we only submitted China trademark
registration.

So about a new name, do we need to register this new "name" before donation
(and transfer to ASF before graduation),  or just a new suitable name
(without registration) is enough for incubation?

Best regards,
Jerry

Saisai Shao  于2022年5月17日周二 10:01写道：

> Thanks Daniel and Justin, let me resolve the naming issue first.
>
> Best regards,
> Jerry
>
> Justin Mclean  于2022年5月17日周二 09:52写道：
>
>> Hi,
>>
>> > Thanks for your reply. The trademark "firestorm" as a software has
>> already
>> > been submitted the registration by Tencent in China, I'm not sure is
>> that
>> > enough or not.
>>
>> If that trademark is approved it would need to be transferred to the ASF
>> before graduation. Even f it was approved there may still be an issue with
>> the name in the USA or other countries.
>>
>> > Is there any guidance in ASF about naming or trademark things? I will
>> > consult this to our company lawyer.
>>
>> Please see [1].
>>
>> Kind Regards,
>> Justin
>>
>> 1. https://incubator.apache.org/guides/names.html
>> -
>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
>> For additional commands, e-mail: general-h...@incubator.apache.org
>>
>>

Re: [DISCUSSION] Incubating Proposal of Firestorm

[Saisai Shao]

Thanks Daniel and Justin, let me resolve the naming issue first.

Best regards,
Jerry

Justin Mclean  于2022年5月17日周二 09:52写道：

> Hi,
>
> > Thanks for your reply. The trademark "firestorm" as a software has
> already
> > been submitted the registration by Tencent in China, I'm not sure is that
> > enough or not.
>
> If that trademark is approved it would need to be transferred to the ASF
> before graduation. Even f it was approved there may still be an issue with
> the name in the USA or other countries.
>
> > Is there any guidance in ASF about naming or trademark things? I will
> > consult this to our company lawyer.
>
> Please see [1].
>
> Kind Regards,
> Justin
>
> 1. https://incubator.apache.org/guides/names.html
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>

Re: [DISCUSSION] Incubating Proposal of Firestorm

[Xun Liu]

Hi,

+1 (binding for me)

As stated in the proposal,
Shuffle is a critical part of the distributed computing engine for
exchanging data between distributed tasks,
and it is important to maintain high performance and continuous stable
operation,
issues that often plague users and developers of big data systems.

When I was asked to be a mentor for Firestorm,
we were excited to learn that Tencent was working to get Firestorm into the
apache incubator for better development,
because Firestorm was a system that had been tested in Tencent's production
environment,
and we prepared the proposal with multiple big data As we prepared our
proposal,
we discussed with several Big Data companies both online and offline about
their respective problems with the Shuffle phase of the distributed system,
and we received good and positive feedback.

We believe that Firestorm can solve real problems and will be well
supported by Big Data users and developers.
I believe that Firestorm can better enrich the apache community's ecosystem.

Best Regards,
Xun Liu


On Tue, May 17, 2022 at 5:58 AM Justin Mclean 
wrote:

> Hi,
>
> Sounds like an interesting project. I also think the name may be
> problematic from a trademark point of view. It might be best to do a
> trademark search as see if there are any issues. Changing the name is
> difficult, but is much much harder if it needs to be done later.
>
> Kind Regards,
> Justin
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>

Re: [DISCUSSION] Incubating Proposal of Firestorm

[Justin Mclean]

Hi,

> Thanks for your reply. The trademark "firestorm" as a software has already
> been submitted the registration by Tencent in China, I'm not sure is that
> enough or not.

If that trademark is approved it would need to be transferred to the ASF before 
graduation. Even f it was approved there may still be an issue with the name in 
the USA or other countries.

> Is there any guidance in ASF about naming or trademark things? I will
> consult this to our company lawyer.

Please see [1].

Kind Regards,
Justin

1. https://incubator.apache.org/guides/names.html
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSSION] Incubating Proposal of Firestorm

[Daniel Widdis]

I trust your lawyer will know better than I will about the trademark.  I simply 
entered "firestorm software" into a search engine and got a product that's not 
yours, available since at least July 2017. 

This site [1] has the general naming guidance and the existence of the other 
software may be of concern with some of the considerations.

1 - https://infra.apache.org/project-names.html


On 5/16/22, 6:43 PM, "Saisai Shao"  wrote:

Hi Daniel and Justin,

Thanks for your reply. The trademark "firestorm" as a software has already
been submitted the registration by Tencent in China, I'm not sure is that
enough or not.

Is there any guidance in ASF about naming or trademark things? I will
consult this to our company lawyer.

Thanks
Jerry

Justin Mclean  于2022年5月17日周二 05:59写道：

> Hi,
>
> Sounds like an interesting project. I also think the name may be
> problematic from a trademark point of view. It might be best to do a
> trademark search as see if there are any issues. Changing the name is
> difficult, but is much much harder if it needs to be done later.
>
> Kind Regards,
> Justin
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>



-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSSION] Incubating Proposal of Firestorm

[Saisai Shao]

Hi Daniel and Justin,

Thanks for your reply. The trademark "firestorm" as a software has already
been submitted the registration by Tencent in China, I'm not sure is that
enough or not.

Is there any guidance in ASF about naming or trademark things? I will
consult this to our company lawyer.

Thanks
Jerry

Justin Mclean  于2022年5月17日周二 05:59写道：

> Hi,
>
> Sounds like an interesting project. I also think the name may be
> problematic from a trademark point of view. It might be best to do a
> trademark search as see if there are any issues. Changing the name is
> difficult, but is much much harder if it needs to be done later.
>
> Kind Regards,
> Justin
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>

Re: [VOTE] Release Apache Linkis (Incubating) 1.1.1-RC1

[Justin Mclean]

Hi,

> But the py file is still missing license headers and have different
> authors[2]. We don't know how to solve this problem.

You may need to look at the history f the files to see if that provide some 
clues or how contributions to the project are licensed. You could also contact 
the project and ask them how the files are licensed.

Kind Regards,
Justin


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSSION] Incubating Proposal of Firestorm

[Justin Mclean]

Hi,

Sounds like an interesting project. I also think the name may be problematic 
from a trademark point of view. It might be best to do a trademark search as 
see if there are any issues. Changing the name is difficult, but is much much 
harder if it needs to be done later.  

Kind Regards,
Justin
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSSION] Incubating Proposal of Firestorm

[Daniel B. Widdis]

+0.9 (non-binding).

I think the project is a great idea.  I think the name is going to run into
a lot of issues with trademarks and pre-existing software products and
would recommend the project consider a new name before incubating.

On Mon, May 16, 2022 at 6:44 AM Jerry Shao  wrote:

> Hi all,
>
> We would like to propose Firestorm[1] as a new Apache incubator project,
> you can find the proposal here [2] for more details.
>
> Firestorm is a high performance, general purpose Remote Shuffle Service for
> distributed compute engines like Apache Spark
> , Apache
> Hadoop MapReduce , Apache Flink
>  and so on. We are aiming to make Firestorm a
> universal shuffle service for distributed compute engines.
>
> Shuffle is the key part for a distributed compute engine to exchange the
> data between distributed tasks, the performance and stability of shuffle
> will directly affect the whole job. Current “local file pull-like shuffle
> style” has several limitations:
>
>1. Current shuffle is hard to support super large workloads, especially
>in a high load environment, the major problem is IO problem (random
> disk IO
>issue, network congestion and timeout).
>2. Current shuffle is hard to deploy on the disaggregated compute
>storage environment, as disk capacity is quite limited on compute nodes.
>3. The constraint of storing shuffle data locally makes it hard to scale
>elastically.
>
> Remote Shuffle Service is the key technology for enterprises to build big
> data platforms, to expand big data applications to disaggregated,
> online-offline hybrid environments, and to solve above problems.
>
> The implementation of Remote Shuffle Service -  “Firestorm”  - is heavily
> adopted in Tencent, and shows its advantages in production. Other
> enterprises also adopted or prepared to adopt Firestorm in their
> environments.
>
> Firestorm’s key idea is brought from Salfish shuffle
> <
> https://www.researchgate.net/publication/262241541_Sailfish_a_framework_for_large_scale_data_processing
> >,
> it has several key design goals:
>
>1. High performance. Firestorm’s performance is close enough to local
>file based shuffle style for small workloads. For large workloads, it is
>far better than the current shuffle style.
>2. Fault tolerance. Firestorm provides high availability for Coordinated
>nodes, and failover for Shuffle nodes.
>3. Pluggable. Firestorm is highly pluggable, which could be suited to
>different compute engines, different backend storages, and different
>wire-protocols.
>
> We believe that Firestorm project will provide the great value for the
> community if it is accepted by the Apache incubator.
>
> I will help this project as champion and many thanks to the 3 mentors:
>
>- Junping du (junping...@apache.org)
>- Xun liu (liu...@apache.org)
>- Zhankun Tang (zt...@apache.org)
>
>
> [1] https://github.com/Tencent/Firestorm
> [2]
> https://cwiki.apache.org/confluence/display/INCUBATOR/FirestormProposal
>
> Best regards,
> Jerry
>


-- 
Dan Widdis

Re: [DISCUSSION] Incubating Proposal of Firestorm

[41108453]

+1， good luck！





-- Original --
From: Jerry Shao https://spark.apache.org/>;, Apache
Hadoop MapReduce ;, Apache Flink
; and so on. We are aiming to make Firestorm a
universal shuffle service for distributed compute engines.

Shuffle is the key part for a distributed compute engine to exchange the
data between distributed tasks, the performance and stability of shuffle
will directly affect the whole job. Current “local file pull-like shuffle
style” has several limitations:

   1. Current shuffle is hard to support super large workloads, 
especially
   in a high load environment, the major problem is IO problem 
(random disk IO
   issue, network congestion and timeout).
   2. Current shuffle is hard to deploy on the disaggregated compute
   storage environment, as disk capacity is quite limited on compute 
nodes.
   3. The constraint of storing shuffle data locally makes it hard to 
scale
   elastically.

Remote Shuffle Service is the key technology for enterprises to build big
data platforms, to expand big data applications to disaggregated,
online-offline hybrid environments, and to solve above problems.

The implementation of Remote Shuffle Service -  “Firestorm”  - is 
heavily
adopted in Tencent, and shows its advantages in production. Other
enterprises also adopted or prepared to adopt Firestorm in their
environments.

Firestorm’s key idea is brought from Salfish shuffle
;,
it has several key design goals:

   1. High performance. Firestorm’s performance is close enough to 
local
   file based shuffle style for small workloads. For large workloads, 
it is
   far better than the current shuffle style.
   2. Fault tolerance. Firestorm provides high availability for 
Coordinated
   nodes, and failover for Shuffle nodes.
   3. Pluggable. Firestorm is highly pluggable, which could be suited 
to
   different compute engines, different backend storages, and 
different
   wire-protocols.

We believe that Firestorm project will provide the great value for the
community if it is accepted by the Apache incubator.

I will help this project as champion and many thanks to the 3 mentors:

   - Junping du (junping...@apache.org)
   - Xun liu (liu...@apache.org)
   - Zhankun Tang (zt...@apache.org)


[1] https://github.com/Tencent/Firestorm
[2] https://cwiki.apache.org/confluence/display/INCUBATOR/FirestormProposal

Best regards,
Jerry

Re: [DISCUSSION] Incubating Proposal of Firestorm

[谭中意]

+1。

On Mon, May 16, 2022 at 10:13 PM Goson zhang  wrote:

> +1
> Good luck!
>
>
> Jerry Shao  于2022年5月16日周一 21:44写道：
>
> > Hi all,
> >
> > We would like to propose Firestorm[1] as a new Apache incubator project,
> > you can find the proposal here [2] for more details.
> >
> > Firestorm is a high performance, general purpose Remote Shuffle Service
> for
> > distributed compute engines like Apache Spark
> > , Apache
> > Hadoop MapReduce , Apache Flink
> >  and so on. We are aiming to make Firestorm a
> > universal shuffle service for distributed compute engines.
> >
> > Shuffle is the key part for a distributed compute engine to exchange the
> > data between distributed tasks, the performance and stability of shuffle
> > will directly affect the whole job. Current “local file pull-like shuffle
> > style” has several limitations:
> >
> >1. Current shuffle is hard to support super large workloads,
> especially
> >in a high load environment, the major problem is IO problem (random
> > disk IO
> >issue, network congestion and timeout).
> >2. Current shuffle is hard to deploy on the disaggregated compute
> >storage environment, as disk capacity is quite limited on compute
> nodes.
> >3. The constraint of storing shuffle data locally makes it hard to
> scale
> >elastically.
> >
> > Remote Shuffle Service is the key technology for enterprises to build big
> > data platforms, to expand big data applications to disaggregated,
> > online-offline hybrid environments, and to solve above problems.
> >
> > The implementation of Remote Shuffle Service -  “Firestorm”  - is heavily
> > adopted in Tencent, and shows its advantages in production. Other
> > enterprises also adopted or prepared to adopt Firestorm in their
> > environments.
> >
> > Firestorm’s key idea is brought from Salfish shuffle
> > <
> >
> https://www.researchgate.net/publication/262241541_Sailfish_a_framework_for_large_scale_data_processing
> > >,
> > it has several key design goals:
> >
> >1. High performance. Firestorm’s performance is close enough to local
> >file based shuffle style for small workloads. For large workloads, it
> is
> >far better than the current shuffle style.
> >2. Fault tolerance. Firestorm provides high availability for
> Coordinated
> >nodes, and failover for Shuffle nodes.
> >3. Pluggable. Firestorm is highly pluggable, which could be suited to
> >different compute engines, different backend storages, and different
> >wire-protocols.
> >
> > We believe that Firestorm project will provide the great value for the
> > community if it is accepted by the Apache incubator.
> >
> > I will help this project as champion and many thanks to the 3 mentors:
> >
> >- Junping du (junping...@apache.org)
> >- Xun liu (liu...@apache.org)
> >- Zhankun Tang (zt...@apache.org)
> >
> >
> > [1] https://github.com/Tencent/Firestorm
> > [2]
> > https://cwiki.apache.org/confluence/display/INCUBATOR/FirestormProposal
> >
> > Best regards,
> > Jerry
> >
>

Re: [DISCUSSION] Incubating Proposal of Firestorm

[Goson zhang]

+1
Good luck!


Jerry Shao  于2022年5月16日周一 21:44写道：

> Hi all,
>
> We would like to propose Firestorm[1] as a new Apache incubator project,
> you can find the proposal here [2] for more details.
>
> Firestorm is a high performance, general purpose Remote Shuffle Service for
> distributed compute engines like Apache Spark
> , Apache
> Hadoop MapReduce , Apache Flink
>  and so on. We are aiming to make Firestorm a
> universal shuffle service for distributed compute engines.
>
> Shuffle is the key part for a distributed compute engine to exchange the
> data between distributed tasks, the performance and stability of shuffle
> will directly affect the whole job. Current “local file pull-like shuffle
> style” has several limitations:
>
>1. Current shuffle is hard to support super large workloads, especially
>in a high load environment, the major problem is IO problem (random
> disk IO
>issue, network congestion and timeout).
>2. Current shuffle is hard to deploy on the disaggregated compute
>storage environment, as disk capacity is quite limited on compute nodes.
>3. The constraint of storing shuffle data locally makes it hard to scale
>elastically.
>
> Remote Shuffle Service is the key technology for enterprises to build big
> data platforms, to expand big data applications to disaggregated,
> online-offline hybrid environments, and to solve above problems.
>
> The implementation of Remote Shuffle Service -  “Firestorm”  - is heavily
> adopted in Tencent, and shows its advantages in production. Other
> enterprises also adopted or prepared to adopt Firestorm in their
> environments.
>
> Firestorm’s key idea is brought from Salfish shuffle
> <
> https://www.researchgate.net/publication/262241541_Sailfish_a_framework_for_large_scale_data_processing
> >,
> it has several key design goals:
>
>1. High performance. Firestorm’s performance is close enough to local
>file based shuffle style for small workloads. For large workloads, it is
>far better than the current shuffle style.
>2. Fault tolerance. Firestorm provides high availability for Coordinated
>nodes, and failover for Shuffle nodes.
>3. Pluggable. Firestorm is highly pluggable, which could be suited to
>different compute engines, different backend storages, and different
>wire-protocols.
>
> We believe that Firestorm project will provide the great value for the
> community if it is accepted by the Apache incubator.
>
> I will help this project as champion and many thanks to the 3 mentors:
>
>- Junping du (junping...@apache.org)
>- Xun liu (liu...@apache.org)
>- Zhankun Tang (zt...@apache.org)
>
>
> [1] https://github.com/Tencent/Firestorm
> [2]
> https://cwiki.apache.org/confluence/display/INCUBATOR/FirestormProposal
>
> Best regards,
> Jerry
>

[DISCUSSION] Incubating Proposal of Firestorm

[Jerry Shao]

Hi all,

We would like to propose Firestorm[1] as a new Apache incubator project,
you can find the proposal here [2] for more details.

Firestorm is a high performance, general purpose Remote Shuffle Service for
distributed compute engines like Apache Spark
, Apache
Hadoop MapReduce , Apache Flink
 and so on. We are aiming to make Firestorm a
universal shuffle service for distributed compute engines.

Shuffle is the key part for a distributed compute engine to exchange the
data between distributed tasks, the performance and stability of shuffle
will directly affect the whole job. Current “local file pull-like shuffle
style” has several limitations:

   1. Current shuffle is hard to support super large workloads, especially
   in a high load environment, the major problem is IO problem (random disk IO
   issue, network congestion and timeout).
   2. Current shuffle is hard to deploy on the disaggregated compute
   storage environment, as disk capacity is quite limited on compute nodes.
   3. The constraint of storing shuffle data locally makes it hard to scale
   elastically.

Remote Shuffle Service is the key technology for enterprises to build big
data platforms, to expand big data applications to disaggregated,
online-offline hybrid environments, and to solve above problems.

The implementation of Remote Shuffle Service -  “Firestorm”  - is heavily
adopted in Tencent, and shows its advantages in production. Other
enterprises also adopted or prepared to adopt Firestorm in their
environments.

Firestorm’s key idea is brought from Salfish shuffle
,
it has several key design goals:

   1. High performance. Firestorm’s performance is close enough to local
   file based shuffle style for small workloads. For large workloads, it is
   far better than the current shuffle style.
   2. Fault tolerance. Firestorm provides high availability for Coordinated
   nodes, and failover for Shuffle nodes.
   3. Pluggable. Firestorm is highly pluggable, which could be suited to
   different compute engines, different backend storages, and different
   wire-protocols.

We believe that Firestorm project will provide the great value for the
community if it is accepted by the Apache incubator.

I will help this project as champion and many thanks to the 3 mentors:

   - Junping du (junping...@apache.org)
   - Xun liu (liu...@apache.org)
   - Zhankun Tang (zt...@apache.org)


[1] https://github.com/Tencent/Firestorm
[2] https://cwiki.apache.org/confluence/display/INCUBATOR/FirestormProposal

Best regards,
Jerry

Re:Re: [DISCUSS] Graduate Apache Doris (Incubating) as TLP

[陈明雨]

[1] and [2] has been renamed. And [3] has been removed.
The website[4] still has some issues waiting to be modified. I will keep touch 
with starrocks.


[1] https://github.com/StarRocks/flink-connector-starrocks
[2] https://github.com/StarRocks/spark-starrocks-connector
[3] https://github.com/StarRocks/apache-orc
[4] https://www.starrocks.com/




--

此致！Best Regards
陈明雨 Mingyu Chen

Email:
chenmin...@apache.org





At 2022-05-16 16:31:07, "Justin Mclean"  wrote:
>HI,
>
>There is also an issue with the naming and use of ASF trademark of this 
>product:
>https://github.com/StarRocks/flink-connector-starrocks
>
>And this:
>https://github.com/StarRocks/apache-orc
>
>And this:
>https://github.com/StarRocks/spark-starrocks-connector
>
>Kind Regards,
>Justin
>-
>To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
>For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSS] Graduate Apache Doris (Incubating) as TLP

[Justin Mclean]

HI,

There is also an issue with the naming and use of ASF trademark of this product:
https://github.com/StarRocks/flink-connector-starrocks

And this:
https://github.com/StarRocks/apache-orc

And this:
https://github.com/StarRocks/spark-starrocks-connector

Kind Regards,
Justin
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSS] Graduate Apache Doris (Incubating) as TLP

[Justin Mclean]

HI,

> I searched their website and found this is a link "trademarks"[1] at the 
> bottom of every page. Does this meet the requirements? 

No as explained before it does not. Most of the answers can be found in the 
trademark FAQ including [1][2][3] There are further issues with pages like [4].

Kind Regards,
Justin

1. https://www.apache.org/foundation/marks/faq/#obvious
2. https://www.apache.org/foundation/marks/faq/#guide
3. https://www.apache.org/foundation/marks/faq/#attribution
4. https://www.starrocks.com/en-US/blog/webinar_220426
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re:Re: [DISCUSS] Graduate Apache Doris (Incubating) as TLP

[陈明雨]

Hi,


> The current StarRock website is still not following ASF trademark / branding 
> policy.


I searched their website and found this is a link "trademarks"[1] at the bottom 
of every page. Does this meet the requirements? 



[1] https://www.starrocks.com/en-US/product/trademarks

--

此致！Best Regards
陈明雨 Mingyu Chen

Email:
chenmin...@apache.org





在 2022-05-16 15:25:20，"Justin Mclean"  写道：
>Hi,
>
>Some are recent (in 2022) but I don’t know how or if the author(s) are related 
>to either project. Taking one at random [1] it was published on 15-5-2022 and 
>looks to me not following ASF trademark policy and potentially misleading 
>users, although It’s hard to say for certain as I can only read the translated 
>text.
>
>> First of all, I have to admit that when the name DorisDB was first used, 
>> starrocks and Doris team did not realize that the name violated Apache 
>> regulations.
>
>Having a similar name was bound to cause confusion with users.
>
>> But after learning about it, the Doris team pushed to complete the renaming 
>> operation, and nearly 9 months have passed since that operation. Since the 
>> name change in September 2021, the Doris team, including starrocks 
>> employees, has strictly followed the Apache trademark and branding guidelines
>
>The issue was brought up in April 2021. The current StarRock website is still 
>not following ASF trademark / branding policy. Please see [2] on why this 
>matters.
>
>Kind Regards,
>Justin
>
>1. https://chowdera.com/2022/135/202205150625040654.html
>2. https://www.apache.org/foundation/marks/responsibility
>
>
>-
>To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
>For additional commands, e-mail: general-h...@incubator.apache.org

Re: [DISCUSS] Graduate Apache Doris (Incubating) as TLP

[Justin Mclean]

Hi,

Some are recent (in 2022) but I don’t know how or if the author(s) are related 
to either project. Taking one at random [1] it was published on 15-5-2022 and 
looks to me not following ASF trademark policy and potentially misleading 
users, although It’s hard to say for certain as I can only read the translated 
text.

> First of all, I have to admit that when the name DorisDB was first used, 
> starrocks and Doris team did not realize that the name violated Apache 
> regulations.

Having a similar name was bound to cause confusion with users.

> But after learning about it, the Doris team pushed to complete the renaming 
> operation, and nearly 9 months have passed since that operation. Since the 
> name change in September 2021, the Doris team, including starrocks employees, 
> has strictly followed the Apache trademark and branding guidelines

The issue was brought up in April 2021. The current StarRock website is still 
not following ASF trademark / branding policy. Please see [2] on why this 
matters.

Kind Regards,
Justin

1. https://chowdera.com/2022/135/202205150625040654.html
2. https://www.apache.org/foundation/marks/responsibility


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [VOTE] Release Apache Linkis (Incubating) 1.1.1-RC1

[cas...@apache.org]

Hi Justin

We found that the new version of py4j has removed the backport.py file, so
after our evaluation, we upgraded the py4j version to
py4j-0.10.7-src.zip[1].

But the py file is still missing license headers and have different
authors[2]. We don't know how to solve this problem.

[1] https://github.com/apache/incubator-linkis/pull/2113
[2] https://github.com/py4j/py4j/tree/0.10.9.5/py4j-python/src/py4j



Best Regards!
Casion


Justin Mclean  于2022年5月13日周五 17:05写道：

> Hi,
>
> >> I confirmed in the py4j-0.10.7-src.zip file, and the backport.py file
> >> clearly states that it is all licensed under the PSF License.
> >>
> >> So can it be considered that py4j-0.10.7-src.zip is licensed under
> >> BDS license, except that backport.py is under PSF license
>
> I would double check the license of the other files as they are missing
> headers and have different authors.
>
> Kind Regards,
> Justin
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>