Re: [PSA] Please read: PR builder now runs test and build in your forked repository

[Yi Wu]

Thanks for the great work, Hyukjin!

On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang  wrote:

> Thanks for the amazing work, Hyukjin!
> I created a PR for trial and it looks well so far:
> https://github.com/apache/spark/pull/32158
>
> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon  wrote:
>
>> Hi all,
>>
>> After https://github.com/apache/spark/pull/32092 merged, now we run the
>> GitHub Actions
>> workflows in your forked repository.
>>
>> In short, please see this example HyukjinKwon#34
>> 
>>
>>1. You create a PR and your repository triggers the workflow. Your PR
>>uses the resources allocated to you for testing.
>>2. Apache Spark repository finds your workflow, and links it in a
>>comment in your PR
>>
>> Please let me know if you guys find any weird behaviour related to this.
>>
>>
>> *What does that mean to contributors?*
>>
>> Please sync your branch to the latest master branch in Apache Spark in
>> order for your forked repository to run the workflow, and
>> for the main repository to detect the workflow.
>>
>>
>> *What does that mean to committers?*
>>
>> Now, GitHub Actions will show a green even when GitHub Actions builds are
>> running (in contributor's forked repository).
>> Please check the build notified by github-actions bot before merging it.
>> There would be a followup work to reflect the status of the forked
>> repository's build to the status of PR.
>>
>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon 님이 작성:
>>
>>> Hi all,
>>>
>>> After https://github.com/apache/spark/pull/32092 merged, now we run the
>>> GitHub Actions
>>> workflows in your forked repository.
>>>
>>> In short, please see this example HyukjinKwon#34
>>> 
>>>
>>>1. You create a PR and your repository triggers the workflow. Your
>>>PR uses the resources allocated to you for testing.
>>>2. Apache Spark repository finds your workflow, and links it in a
>>>comment in your PR
>>>
>>> Please let me know if you guys find any weird behaviour related to this.
>>>
>>>
>>> *What does that mean to contributors?*
>>>
>>> Please sync your branch to the latest master branch in Apache Spark in
>>> order for the main repository to run the workflow and detect it.
>>>
>>>
>>> *What does that mean to committers?*
>>>
>>> Now, GitHub Actions will show a green even when GitHub Actions builds
>>> are running (in contributor's forked repository). Please check the build
>>> notified by github-actions bot before merging it.
>>> There would be a followup work to reflect the status of the forked
>>> repository's build to
>>> the status of PR.
>>>
>>>
>>>

Re: [VOTE] Release Spark 2.4.8 (RC2)

[Kent Yao]

+1 (non-binding)









  



















Kent Yao @ Data Science Center, Hangzhou Research Institute, NetEase Corp.a spark enthusiastkyuubiis a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of Apache Spark.spark-authorizerA Spark SQL extension which provides SQL Standard Authorization for Apache Spark.spark-postgres A library for reading data from and transferring data to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.spark-func-extrasA library that brings excellent and useful functions from various modern database management systems to Apache Spark.















 


On 04/14/2021 13:36，Gengliang Wang wrote： 


+1 (non-binding)On Wed, Apr 14, 2021 at 1:34 PM Jungtaek Lim  wrote:+1 (non-binding)signature OK, extracting tgz files OK, build source without running tests OK.On Tue, Apr 13, 2021 at 5:02 PM Herman van Hovell  wrote:+1On Tue, Apr 13, 2021 at 2:40 AM sarutak  wrote:+1 (non-binding)

> +1
> 
> On Tue, 13 Apr 2021, 02:58 Sean Owen,  wrote:
> 
>> +1 same result as last RC for me.
>> 
>> On Mon, Apr 12, 2021, 12:53 AM Liang-Chi Hsieh 
>> wrote:
>> 
>>> Please vote on releasing the following candidate as Apache Spark
>>> version
>>> 2.4.8.
>>> 
>>> The vote is open until Apr 15th at 9AM PST and passes if a
>>> majority +1 PMC
>>> votes are cast, with a minimum of 3 +1 votes.
>>> 
>>> [ ] +1 Release this package as Apache Spark 2.4.8
>>> [ ] -1 Do not release this package because ...
>>> 
>>> To learn more about Apache Spark, please see
>>> http://spark.apache.org/
>>> 
>>> There are currently no issues targeting 2.4.8 (try project = SPARK
>>> AND
>>> "Target Version/s" = "2.4.8" AND status in (Open, Reopened, "In
>>> Progress"))
>>> 
>>> The tag to be voted on is v2.4.8-rc2 (commit
>>> a0ab27ca6b46b8e5a7ae8bb91e30546082fc551c):
>>> https://github.com/apache/spark/tree/v2.4.8-rc2
>>> 
>>> The release files, including signatures, digests, etc. can be
>>> found at:
>>> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc2-bin/
>>> 
>>> Signatures used for Spark RCs can be found in this file:
>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>> 
>>> The staging repository for this release can be found at:
>>> 
>> 
> https://repository.apache.org/content/repositories/orgapachespark-1373/
>>> 
>>> The documentation corresponding to this release can be found at:
>>> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc2-docs/
>>> 
>>> The list of bug fixes going into 2.4.8 can be found at the
>>> following URL:
>>> https://s.apache.org/spark-v2.4.8-rc2
>>> 
>>> This release is using the release script of the tag v2.4.8-rc2.
>>> 
>>> FAQ
>>> 
>>> =
>>> How can I help test this release?
>>> =
>>> 
>>> If you are a Spark user, you can help us test this release by
>>> taking
>>> an existing Spark workload and running on this release candidate,
>>> then
>>> reporting any regressions.
>>> 
>>> If you're working in PySpark you can set up a virtual env and
>>> install
>>> the current RC and see if anything important breaks, in the
>>> Java/Scala
>>> you can add the staging repository to your projects resolvers and
>>> test
>>> with the RC (make sure to clean up the artifact cache before/after
>>> so
>>> you don't end up building with an out of date RC going forward).
>>> 
>>> ===
>>> What should happen to JIRA tickets still targeting 2.4.8?
>>> ===
>>> 
>>> The current list of open tickets targeted at 2.4.8 can be found
>>> at:
>>> https://issues.apache.org/jira/projects/SPARK and search for
>>> "Target
>>> Version/s" = 2.4.8
>>> 
>>> Committers should look at those and triage. Extremely important
>>> bug
>>> fixes, documentation, and API tweaks that impact compatibility
>>> should
>>> be worked on immediately. Everything else please retarget to an
>>> appropriate release.
>>> 
>>> ==
>>> But my bug isn't fixed?
>>> ==
>>> 
>>> In order to make timely releases, we will typically not hold the
>>> release unless the bug in question is a regression from the
>>> previous
>>> release. That being said, if there is something which is a
>>> regression
>>> that has not been correctly targeted please ping me or a committer
>>> to
>>> help target the issue.
>>> 
>>> --
>>> Sent from:
>>> http://apache-spark-developers-list.1001551.n3.nabble.com/
>>> 
>>> 
>> 
> -
>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

-
To unsubscribe e-mail: dev-unsubscr...@s

Re: [VOTE] Release Spark 2.4.8 (RC2)

[Gengliang Wang]

+1 (non-binding)

On Wed, Apr 14, 2021 at 1:34 PM Jungtaek Lim 
wrote:

> +1 (non-binding)
>
> signature OK, extracting tgz files OK, build source without running tests
> OK.
>
> On Tue, Apr 13, 2021 at 5:02 PM Herman van Hovell 
> wrote:
>
>> +1
>>
>> On Tue, Apr 13, 2021 at 2:40 AM sarutak  wrote:
>>
>>> +1 (non-binding)
>>>
>>> > +1
>>> >
>>> > On Tue, 13 Apr 2021, 02:58 Sean Owen,  wrote:
>>> >
>>> >> +1 same result as last RC for me.
>>> >>
>>> >> On Mon, Apr 12, 2021, 12:53 AM Liang-Chi Hsieh 
>>> >> wrote:
>>> >>
>>> >>> Please vote on releasing the following candidate as Apache Spark
>>> >>> version
>>> >>> 2.4.8.
>>> >>>
>>> >>> The vote is open until Apr 15th at 9AM PST and passes if a
>>> >>> majority +1 PMC
>>> >>> votes are cast, with a minimum of 3 +1 votes.
>>> >>>
>>> >>> [ ] +1 Release this package as Apache Spark 2.4.8
>>> >>> [ ] -1 Do not release this package because ...
>>> >>>
>>> >>> To learn more about Apache Spark, please see
>>> >>> http://spark.apache.org/
>>> >>>
>>> >>> There are currently no issues targeting 2.4.8 (try project = SPARK
>>> >>> AND
>>> >>> "Target Version/s" = "2.4.8" AND status in (Open, Reopened, "In
>>> >>> Progress"))
>>> >>>
>>> >>> The tag to be voted on is v2.4.8-rc2 (commit
>>> >>> a0ab27ca6b46b8e5a7ae8bb91e30546082fc551c):
>>> >>> https://github.com/apache/spark/tree/v2.4.8-rc2
>>> >>>
>>> >>> The release files, including signatures, digests, etc. can be
>>> >>> found at:
>>> >>> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc2-bin/
>>> >>>
>>> >>> Signatures used for Spark RCs can be found in this file:
>>> >>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>> >>>
>>> >>> The staging repository for this release can be found at:
>>> >>>
>>> >>
>>> >
>>> https://repository.apache.org/content/repositories/orgapachespark-1373/
>>> >>>
>>> >>> The documentation corresponding to this release can be found at:
>>> >>> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc2-docs/
>>> >>>
>>> >>> The list of bug fixes going into 2.4.8 can be found at the
>>> >>> following URL:
>>> >>> https://s.apache.org/spark-v2.4.8-rc2
>>> >>>
>>> >>> This release is using the release script of the tag v2.4.8-rc2.
>>> >>>
>>> >>> FAQ
>>> >>>
>>> >>> =
>>> >>> How can I help test this release?
>>> >>> =
>>> >>>
>>> >>> If you are a Spark user, you can help us test this release by
>>> >>> taking
>>> >>> an existing Spark workload and running on this release candidate,
>>> >>> then
>>> >>> reporting any regressions.
>>> >>>
>>> >>> If you're working in PySpark you can set up a virtual env and
>>> >>> install
>>> >>> the current RC and see if anything important breaks, in the
>>> >>> Java/Scala
>>> >>> you can add the staging repository to your projects resolvers and
>>> >>> test
>>> >>> with the RC (make sure to clean up the artifact cache before/after
>>> >>> so
>>> >>> you don't end up building with an out of date RC going forward).
>>> >>>
>>> >>> ===
>>> >>> What should happen to JIRA tickets still targeting 2.4.8?
>>> >>> ===
>>> >>>
>>> >>> The current list of open tickets targeted at 2.4.8 can be found
>>> >>> at:
>>> >>> https://issues.apache.org/jira/projects/SPARK and search for
>>> >>> "Target
>>> >>> Version/s" = 2.4.8
>>> >>>
>>> >>> Committers should look at those and triage. Extremely important
>>> >>> bug
>>> >>> fixes, documentation, and API tweaks that impact compatibility
>>> >>> should
>>> >>> be worked on immediately. Everything else please retarget to an
>>> >>> appropriate release.
>>> >>>
>>> >>> ==
>>> >>> But my bug isn't fixed?
>>> >>> ==
>>> >>>
>>> >>> In order to make timely releases, we will typically not hold the
>>> >>> release unless the bug in question is a regression from the
>>> >>> previous
>>> >>> release. That being said, if there is something which is a
>>> >>> regression
>>> >>> that has not been correctly targeted please ping me or a committer
>>> >>> to
>>> >>> help target the issue.
>>> >>>
>>> >>> --
>>> >>> Sent from:
>>> >>> http://apache-spark-developers-list.1001551.n3.nabble.com/
>>> >>>
>>> >>>
>>> >>
>>> > -
>>> >>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>
>>> -
>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>
>>>

Re: [VOTE] Release Spark 2.4.8 (RC2)

[Jungtaek Lim]

+1 (non-binding)

signature OK, extracting tgz files OK, build source without running tests
OK.

On Tue, Apr 13, 2021 at 5:02 PM Herman van Hovell 
wrote:

> +1
>
> On Tue, Apr 13, 2021 at 2:40 AM sarutak  wrote:
>
>> +1 (non-binding)
>>
>> > +1
>> >
>> > On Tue, 13 Apr 2021, 02:58 Sean Owen,  wrote:
>> >
>> >> +1 same result as last RC for me.
>> >>
>> >> On Mon, Apr 12, 2021, 12:53 AM Liang-Chi Hsieh 
>> >> wrote:
>> >>
>> >>> Please vote on releasing the following candidate as Apache Spark
>> >>> version
>> >>> 2.4.8.
>> >>>
>> >>> The vote is open until Apr 15th at 9AM PST and passes if a
>> >>> majority +1 PMC
>> >>> votes are cast, with a minimum of 3 +1 votes.
>> >>>
>> >>> [ ] +1 Release this package as Apache Spark 2.4.8
>> >>> [ ] -1 Do not release this package because ...
>> >>>
>> >>> To learn more about Apache Spark, please see
>> >>> http://spark.apache.org/
>> >>>
>> >>> There are currently no issues targeting 2.4.8 (try project = SPARK
>> >>> AND
>> >>> "Target Version/s" = "2.4.8" AND status in (Open, Reopened, "In
>> >>> Progress"))
>> >>>
>> >>> The tag to be voted on is v2.4.8-rc2 (commit
>> >>> a0ab27ca6b46b8e5a7ae8bb91e30546082fc551c):
>> >>> https://github.com/apache/spark/tree/v2.4.8-rc2
>> >>>
>> >>> The release files, including signatures, digests, etc. can be
>> >>> found at:
>> >>> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc2-bin/
>> >>>
>> >>> Signatures used for Spark RCs can be found in this file:
>> >>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>> >>>
>> >>> The staging repository for this release can be found at:
>> >>>
>> >>
>> > https://repository.apache.org/content/repositories/orgapachespark-1373/
>> >>>
>> >>> The documentation corresponding to this release can be found at:
>> >>> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc2-docs/
>> >>>
>> >>> The list of bug fixes going into 2.4.8 can be found at the
>> >>> following URL:
>> >>> https://s.apache.org/spark-v2.4.8-rc2
>> >>>
>> >>> This release is using the release script of the tag v2.4.8-rc2.
>> >>>
>> >>> FAQ
>> >>>
>> >>> =
>> >>> How can I help test this release?
>> >>> =
>> >>>
>> >>> If you are a Spark user, you can help us test this release by
>> >>> taking
>> >>> an existing Spark workload and running on this release candidate,
>> >>> then
>> >>> reporting any regressions.
>> >>>
>> >>> If you're working in PySpark you can set up a virtual env and
>> >>> install
>> >>> the current RC and see if anything important breaks, in the
>> >>> Java/Scala
>> >>> you can add the staging repository to your projects resolvers and
>> >>> test
>> >>> with the RC (make sure to clean up the artifact cache before/after
>> >>> so
>> >>> you don't end up building with an out of date RC going forward).
>> >>>
>> >>> ===
>> >>> What should happen to JIRA tickets still targeting 2.4.8?
>> >>> ===
>> >>>
>> >>> The current list of open tickets targeted at 2.4.8 can be found
>> >>> at:
>> >>> https://issues.apache.org/jira/projects/SPARK and search for
>> >>> "Target
>> >>> Version/s" = 2.4.8
>> >>>
>> >>> Committers should look at those and triage. Extremely important
>> >>> bug
>> >>> fixes, documentation, and API tweaks that impact compatibility
>> >>> should
>> >>> be worked on immediately. Everything else please retarget to an
>> >>> appropriate release.
>> >>>
>> >>> ==
>> >>> But my bug isn't fixed?
>> >>> ==
>> >>>
>> >>> In order to make timely releases, we will typically not hold the
>> >>> release unless the bug in question is a regression from the
>> >>> previous
>> >>> release. That being said, if there is something which is a
>> >>> regression
>> >>> that has not been correctly targeted please ping me or a committer
>> >>> to
>> >>> help target the issue.
>> >>>
>> >>> --
>> >>> Sent from:
>> >>> http://apache-spark-developers-list.1001551.n3.nabble.com/
>> >>>
>> >>>
>> >>
>> > -
>> >>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>

Re: [PSA] Please read: PR builder now runs test and build in your forked repository

[Gengliang Wang]

Thanks for the amazing work, Hyukjin!
I created a PR for trial and it looks well so far:
https://github.com/apache/spark/pull/32158

On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon  wrote:

> Hi all,
>
> After https://github.com/apache/spark/pull/32092 merged, now we run the
> GitHub Actions
> workflows in your forked repository.
>
> In short, please see this example HyukjinKwon#34
> 
>
>1. You create a PR and your repository triggers the workflow. Your PR
>uses the resources allocated to you for testing.
>2. Apache Spark repository finds your workflow, and links it in a
>comment in your PR
>
> Please let me know if you guys find any weird behaviour related to this.
>
>
> *What does that mean to contributors?*
>
> Please sync your branch to the latest master branch in Apache Spark in
> order for your forked repository to run the workflow, and
> for the main repository to detect the workflow.
>
>
> *What does that mean to committers?*
>
> Now, GitHub Actions will show a green even when GitHub Actions builds are
> running (in contributor's forked repository).
> Please check the build notified by github-actions bot before merging it.
> There would be a followup work to reflect the status of the forked
> repository's build to the status of PR.
>
> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon 님이 작성:
>
>> Hi all,
>>
>> After https://github.com/apache/spark/pull/32092 merged, now we run the
>> GitHub Actions
>> workflows in your forked repository.
>>
>> In short, please see this example HyukjinKwon#34
>> 
>>
>>1. You create a PR and your repository triggers the workflow. Your PR
>>uses the resources allocated to you for testing.
>>2. Apache Spark repository finds your workflow, and links it in a
>>comment in your PR
>>
>> Please let me know if you guys find any weird behaviour related to this.
>>
>>
>> *What does that mean to contributors?*
>>
>> Please sync your branch to the latest master branch in Apache Spark in
>> order for the main repository to run the workflow and detect it.
>>
>>
>> *What does that mean to committers?*
>>
>> Now, GitHub Actions will show a green even when GitHub Actions builds are
>> running (in contributor's forked repository). Please check the build
>> notified by github-actions bot before merging it.
>> There would be a followup work to reflect the status of the forked
>> repository's build to
>> the status of PR.
>>
>>
>>

Re: [PSA] Please read: PR builder now runs test and build in your forked repository

[Hyukjin Kwon]

Hi all,

After https://github.com/apache/spark/pull/32092 merged, now we run the
GitHub Actions
workflows in your forked repository.

In short, please see this example HyukjinKwon#34


   1. You create a PR and your repository triggers the workflow. Your PR
   uses the resources allocated to you for testing.
   2. Apache Spark repository finds your workflow, and links it in a
   comment in your PR

Please let me know if you guys find any weird behaviour related to this.


*What does that mean to contributors?*

Please sync your branch to the latest master branch in Apache Spark in
order for your forked repository to run the workflow, and
for the main repository to detect the workflow.


*What does that mean to committers?*

Now, GitHub Actions will show a green even when GitHub Actions builds are
running (in contributor's forked repository).
Please check the build notified by github-actions bot before merging it.
There would be a followup work to reflect the status of the forked
repository's build to the status of PR.

2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon 님이 작성:

> Hi all,
>
> After https://github.com/apache/spark/pull/32092 merged, now we run the
> GitHub Actions
> workflows in your forked repository.
>
> In short, please see this example HyukjinKwon#34
> 
>
>1. You create a PR and your repository triggers the workflow. Your PR
>uses the resources allocated to you for testing.
>2. Apache Spark repository finds your workflow, and links it in a
>comment in your PR
>
> Please let me know if you guys find any weird behaviour related to this.
>
>
> *What does that mean to contributors?*
>
> Please sync your branch to the latest master branch in Apache Spark in
> order for the main repository to run the workflow and detect it.
>
>
> *What does that mean to committers?*
>
> Now, GitHub Actions will show a green even when GitHub Actions builds are
> running (in contributor's forked repository). Please check the build
> notified by github-actions bot before merging it.
> There would be a followup work to reflect the status of the forked
> repository's build to
> the status of PR.
>
>
>

[PSA] Please read: PR builder now runs test and build in your forked repository

[Hyukjin Kwon]

Hi all,

After https://github.com/apache/spark/pull/32092 merged, now we run the
GitHub Actions
workflows in your forked repository.

In short, please see this example HyukjinKwon#34


   1. You create a PR and your repository triggers the workflow. Your PR
   uses the resources allocated to you for testing.
   2. Apache Spark repository finds your workflow, and links it in a
   comment in your PR

Please let me know if you guys find any weird behaviour related to this.


*What does that mean to contributors?*

Please sync your branch to the latest master branch in Apache Spark in
order for the main repository to run the workflow and detect it.


*What does that mean to committers?*

Now, GitHub Actions will show a green even when GitHub Actions builds are
running (in contributor's forked repository). Please check the build
notified by github-actions bot before merging it.
There would be a followup work to reflect the status of the forked
repository's build to
the status of PR.

Re: [DISCUSS] Build error message guideline

[Yuming Wang]

Do we have plans to apply these guidelines to assert? For example:

https://github.com/apache/spark/blob/5b478416f8e3fe2f015af1b6c8faa7fe9f15c05d/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala#L136-L138
https://github.com/apache/spark/blob/053dd858d38e6107bc71e0aa3a4954291b74f8c8/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourcePartitioning.scala#L41

On Wed, Apr 14, 2021 at 9:27 AM Hyukjin Kwon  wrote:

> I would just go ahead and create a PR for that. Nothing written there
> looks unreasonable.
> But probably it should be best to wait a couple of days to make sure
> people are happy with it.
>
> 2021년 4월 14일 (수) 오전 6:38, Karen 님이 작성:
>
>> If the proposed guidelines look good, it would be useful to share these
>> guidelines with the wider community. A good landing page for contributors
>> could be https://spark.apache.org/contributing.html. What do you think?
>>
>> Thank you,
>>
>> Karen Feng
>>
>> On Wed, Apr 7, 2021 at 8:19 PM Hyukjin Kwon  wrote:
>>
>>> LGTM (I took a look, and had some offline discussions w/ some
>>> corrections before it came out)
>>>
>>> 2021년 4월 8일 (목) 오전 5:28, Karen 님이 작성:
>>>
 Hi all,

 As discussed in SPIP: Standardize Exception Messages in Spark (
 https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing),
 improving error message quality in Apache Spark involves establishing an
 error message guideline for developers. Error message style guidelines are
 common practice across open-source projects, for example PostgreSQL (
 https://www.postgresql.org/docs/current/error-style-guide.html).

 To move towards the goal of improving error message quality, we would
 like to start building an error message guideline. We have attached a rough
 draft to kick off this discussion:
 https://docs.google.com/document/d/12k4zmaKmmdm6Pk63HS0N1zN1QT-6TihkWaa5CkLmsn8/edit?usp=sharing
 .

 Please let us know what you think should be in the guideline! We look
 forward to building this as a community.

 Thank you,

 Karen Feng

>>>

Re: [DISCUSS] Build error message guideline

[Hyukjin Kwon]

I would just go ahead and create a PR for that. Nothing written there looks
unreasonable.
But probably it should be best to wait a couple of days to make sure people
are happy with it.

2021년 4월 14일 (수) 오전 6:38, Karen 님이 작성:

> If the proposed guidelines look good, it would be useful to share these
> guidelines with the wider community. A good landing page for contributors
> could be https://spark.apache.org/contributing.html. What do you think?
>
> Thank you,
>
> Karen Feng
>
> On Wed, Apr 7, 2021 at 8:19 PM Hyukjin Kwon  wrote:
>
>> LGTM (I took a look, and had some offline discussions w/ some corrections
>> before it came out)
>>
>> 2021년 4월 8일 (목) 오전 5:28, Karen 님이 작성:
>>
>>> Hi all,
>>>
>>> As discussed in SPIP: Standardize Exception Messages in Spark (
>>> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing),
>>> improving error message quality in Apache Spark involves establishing an
>>> error message guideline for developers. Error message style guidelines are
>>> common practice across open-source projects, for example PostgreSQL (
>>> https://www.postgresql.org/docs/current/error-style-guide.html).
>>>
>>> To move towards the goal of improving error message quality, we would
>>> like to start building an error message guideline. We have attached a rough
>>> draft to kick off this discussion:
>>> https://docs.google.com/document/d/12k4zmaKmmdm6Pk63HS0N1zN1QT-6TihkWaa5CkLmsn8/edit?usp=sharing
>>> .
>>>
>>> Please let us know what you think should be in the guideline! We look
>>> forward to building this as a community.
>>>
>>> Thank you,
>>>
>>> Karen Feng
>>>
>>

Re: [DISCUSS] Build error message guideline

[Karen]

If the proposed guidelines look good, it would be useful to share these
guidelines with the wider community. A good landing page for contributors
could be https://spark.apache.org/contributing.html. What do you think?

Thank you,

Karen Feng

On Wed, Apr 7, 2021 at 8:19 PM Hyukjin Kwon  wrote:

> LGTM (I took a look, and had some offline discussions w/ some corrections
> before it came out)
>
> 2021년 4월 8일 (목) 오전 5:28, Karen 님이 작성:
>
>> Hi all,
>>
>> As discussed in SPIP: Standardize Exception Messages in Spark (
>> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing),
>> improving error message quality in Apache Spark involves establishing an
>> error message guideline for developers. Error message style guidelines are
>> common practice across open-source projects, for example PostgreSQL (
>> https://www.postgresql.org/docs/current/error-style-guide.html).
>>
>> To move towards the goal of improving error message quality, we would
>> like to start building an error message guideline. We have attached a rough
>> draft to kick off this discussion:
>> https://docs.google.com/document/d/12k4zmaKmmdm6Pk63HS0N1zN1QT-6TihkWaa5CkLmsn8/edit?usp=sharing
>> .
>>
>> Please let us know what you think should be in the guideline! We look
>> forward to building this as a community.
>>
>> Thank you,
>>
>> Karen Feng
>>
>

Re: [VOTE] Release Spark 2.4.8 (RC2)

[Herman van Hovell]

+1

On Tue, Apr 13, 2021 at 2:40 AM sarutak  wrote:

> +1 (non-binding)
>
> > +1
> >
> > On Tue, 13 Apr 2021, 02:58 Sean Owen,  wrote:
> >
> >> +1 same result as last RC for me.
> >>
> >> On Mon, Apr 12, 2021, 12:53 AM Liang-Chi Hsieh 
> >> wrote:
> >>
> >>> Please vote on releasing the following candidate as Apache Spark
> >>> version
> >>> 2.4.8.
> >>>
> >>> The vote is open until Apr 15th at 9AM PST and passes if a
> >>> majority +1 PMC
> >>> votes are cast, with a minimum of 3 +1 votes.
> >>>
> >>> [ ] +1 Release this package as Apache Spark 2.4.8
> >>> [ ] -1 Do not release this package because ...
> >>>
> >>> To learn more about Apache Spark, please see
> >>> http://spark.apache.org/
> >>>
> >>> There are currently no issues targeting 2.4.8 (try project = SPARK
> >>> AND
> >>> "Target Version/s" = "2.4.8" AND status in (Open, Reopened, "In
> >>> Progress"))
> >>>
> >>> The tag to be voted on is v2.4.8-rc2 (commit
> >>> a0ab27ca6b46b8e5a7ae8bb91e30546082fc551c):
> >>> https://github.com/apache/spark/tree/v2.4.8-rc2
> >>>
> >>> The release files, including signatures, digests, etc. can be
> >>> found at:
> >>> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc2-bin/
> >>>
> >>> Signatures used for Spark RCs can be found in this file:
> >>> https://dist.apache.org/repos/dist/dev/spark/KEYS
> >>>
> >>> The staging repository for this release can be found at:
> >>>
> >>
> > https://repository.apache.org/content/repositories/orgapachespark-1373/
> >>>
> >>> The documentation corresponding to this release can be found at:
> >>> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc2-docs/
> >>>
> >>> The list of bug fixes going into 2.4.8 can be found at the
> >>> following URL:
> >>> https://s.apache.org/spark-v2.4.8-rc2
> >>>
> >>> This release is using the release script of the tag v2.4.8-rc2.
> >>>
> >>> FAQ
> >>>
> >>> =
> >>> How can I help test this release?
> >>> =
> >>>
> >>> If you are a Spark user, you can help us test this release by
> >>> taking
> >>> an existing Spark workload and running on this release candidate,
> >>> then
> >>> reporting any regressions.
> >>>
> >>> If you're working in PySpark you can set up a virtual env and
> >>> install
> >>> the current RC and see if anything important breaks, in the
> >>> Java/Scala
> >>> you can add the staging repository to your projects resolvers and
> >>> test
> >>> with the RC (make sure to clean up the artifact cache before/after
> >>> so
> >>> you don't end up building with an out of date RC going forward).
> >>>
> >>> ===
> >>> What should happen to JIRA tickets still targeting 2.4.8?
> >>> ===
> >>>
> >>> The current list of open tickets targeted at 2.4.8 can be found
> >>> at:
> >>> https://issues.apache.org/jira/projects/SPARK and search for
> >>> "Target
> >>> Version/s" = 2.4.8
> >>>
> >>> Committers should look at those and triage. Extremely important
> >>> bug
> >>> fixes, documentation, and API tweaks that impact compatibility
> >>> should
> >>> be worked on immediately. Everything else please retarget to an
> >>> appropriate release.
> >>>
> >>> ==
> >>> But my bug isn't fixed?
> >>> ==
> >>>
> >>> In order to make timely releases, we will typically not hold the
> >>> release unless the bug in question is a regression from the
> >>> previous
> >>> release. That being said, if there is something which is a
> >>> regression
> >>> that has not been correctly targeted please ping me or a committer
> >>> to
> >>> help target the issue.
> >>>
> >>> --
> >>> Sent from:
> >>> http://apache-spark-developers-list.1001551.n3.nabble.com/
> >>>
> >>>
> >>
> > -
> >>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>