Re: [VOTE] Release Spark 2.4.8 (RC3)

[sarutak]

+1 (non-binding)


+1 (non-binding)

郑瑞峰  于2021年4月29日周四 上午10:50写道：


+1 (non-binding)

Thanks,
ruifengz

-- 原始邮件 --
发件人: "Hyukjin Kwon" ;
发送时间: 2021年4月29日(星期四) 上午8:14
收件人: "Sean Owen";
抄送: "Liang-Chi Hsieh";"dev";
主题: Re: [VOTE] Release Spark 2.4.8 (RC3)

+1

On Thu, 29 Apr 2021, 07:08 Sean Owen,  wrote:


+1 from me too, same result as last time.

On Wed, Apr 28, 2021 at 11:33 AM Liang-Chi Hsieh  
wrote:


Please vote on releasing the following candidate as Apache Spark 
version

2.4.8.

The vote is open until May 4th at 9AM PST and passes if a majority 
+1 PMC

votes are cast, with a minimum of 3 +1 votes.

[ ] +1 Release this package as Apache Spark 2.4.8
[ ] -1 Do not release this package because ...

To learn more about Apache Spark, please see 
http://spark.apache.org/


There are currently no issues targeting 2.4.8 (try project = SPARK 
AND
"Target Version/s" = "2.4.8" AND status in (Open, Reopened, "In 
Progress"))


The tag to be voted on is v2.4.8-rc3 (commit
e89526d2401b3a04719721c923a6f630e555e286):
https://github.com/apache/spark/tree/v2.4.8-rc3

The release files, including signatures, digests, etc. can be found 
at:

https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-bin/

Signatures used for Spark RCs can be found in this file:
https://dist.apache.org/repos/dist/dev/spark/KEYS

The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-1377/

The documentation corresponding to this release can be found at:
https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-docs/

The list of bug fixes going into 2.4.8 can be found at the following 
URL:

https://s.apache.org/spark-v2.4.8-rc3

This release is using the release script of the tag v2.4.8-rc3.

FAQ


=
How can I help test this release?
=

If you are a Spark user, you can help us test this release by taking
an existing Spark workload and running on this release candidate, 
then

reporting any regressions.

If you're working in PySpark you can set up a virtual env and 
install
the current RC and see if anything important breaks, in the 
Java/Scala
you can add the staging repository to your projects resolvers and 
test
with the RC (make sure to clean up the artifact cache before/after 
so

you don't end up building with an out of date RC going forward).

===
What should happen to JIRA tickets still targeting 2.4.8?
===

The current list of open tickets targeted at 2.4.8 can be found at:
https://issues.apache.org/jira/projects/SPARK and search for "Target
Version/s" = 2.4.8

Committers should look at those and triage. Extremely important bug
fixes, documentation, and API tweaks that impact compatibility 
should

be worked on immediately. Everything else please retarget to an
appropriate release.

==
But my bug isn't fixed?
==

In order to make timely releases, we will typically not hold the
release unless the bug in question is a regression from the previous
release. That being said, if there is something which is a 
regression
that has not been correctly targeted please ping me or a committer 
to

help target the issue.



--
Sent from: 
http://apache-spark-developers-list.1001551.n3.nabble.com/


-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org




--

 Kent Yao
@ Data Science Center, Hangzhou Research Institute, NetEase Corp.
a spark enthusiast
kyuubiis a unified multi-tenant JDBC interface for large-scale data
processing and analytics, built on top of Apache Spark.

spark-authorizerA Spark SQL extension which provides SQL Standard
Authorization for Apache Spark.
spark-postgres A library for reading data from and transferring data
to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
itatchiA library that brings useful functions from various modern
database management systems to Apache Spark.

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org


-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE] Release Spark 2.4.8 (RC3)

[Kent Yao]

+1 (non-binding)

郑瑞峰  于2021年4月29日周四 上午10:50写道：
>
> +1 (non-binding)
>
> Thanks,
> ruifengz
>
> -- 原始邮件 --
> 发件人: "Hyukjin Kwon" ;
> 发送时间: 2021年4月29日(星期四) 上午8:14
> 收件人: "Sean Owen";
> 抄送: "Liang-Chi Hsieh";"dev";
> 主题: Re: [VOTE] Release Spark 2.4.8 (RC3)
>
> +1
>
> On Thu, 29 Apr 2021, 07:08 Sean Owen,  wrote:
>>
>> +1 from me too, same result as last time.
>>
>> On Wed, Apr 28, 2021 at 11:33 AM Liang-Chi Hsieh  wrote:
>>>
>>> Please vote on releasing the following candidate as Apache Spark version
>>> 2.4.8.
>>>
>>> The vote is open until May 4th at 9AM PST and passes if a majority +1 PMC
>>> votes are cast, with a minimum of 3 +1 votes.
>>>
>>> [ ] +1 Release this package as Apache Spark 2.4.8
>>> [ ] -1 Do not release this package because ...
>>>
>>> To learn more about Apache Spark, please see http://spark.apache.org/
>>>
>>> There are currently no issues targeting 2.4.8 (try project = SPARK AND
>>> "Target Version/s" = "2.4.8" AND status in (Open, Reopened, "In Progress"))
>>>
>>> The tag to be voted on is v2.4.8-rc3 (commit
>>> e89526d2401b3a04719721c923a6f630e555e286):
>>> https://github.com/apache/spark/tree/v2.4.8-rc3
>>>
>>> The release files, including signatures, digests, etc. can be found at:
>>> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-bin/
>>>
>>> Signatures used for Spark RCs can be found in this file:
>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>
>>> The staging repository for this release can be found at:
>>> https://repository.apache.org/content/repositories/orgapachespark-1377/
>>>
>>> The documentation corresponding to this release can be found at:
>>> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-docs/
>>>
>>> The list of bug fixes going into 2.4.8 can be found at the following URL:
>>> https://s.apache.org/spark-v2.4.8-rc3
>>>
>>> This release is using the release script of the tag v2.4.8-rc3.
>>>
>>> FAQ
>>>
>>>
>>> =
>>> How can I help test this release?
>>> =
>>>
>>> If you are a Spark user, you can help us test this release by taking
>>> an existing Spark workload and running on this release candidate, then
>>> reporting any regressions.
>>>
>>> If you're working in PySpark you can set up a virtual env and install
>>> the current RC and see if anything important breaks, in the Java/Scala
>>> you can add the staging repository to your projects resolvers and test
>>> with the RC (make sure to clean up the artifact cache before/after so
>>> you don't end up building with an out of date RC going forward).
>>>
>>> ===
>>> What should happen to JIRA tickets still targeting 2.4.8?
>>> ===
>>>
>>> The current list of open tickets targeted at 2.4.8 can be found at:
>>> https://issues.apache.org/jira/projects/SPARK and search for "Target
>>> Version/s" = 2.4.8
>>>
>>> Committers should look at those and triage. Extremely important bug
>>> fixes, documentation, and API tweaks that impact compatibility should
>>> be worked on immediately. Everything else please retarget to an
>>> appropriate release.
>>>
>>> ==
>>> But my bug isn't fixed?
>>> ==
>>>
>>> In order to make timely releases, we will typically not hold the
>>> release unless the bug in question is a regression from the previous
>>> release. That being said, if there is something which is a regression
>>> that has not been correctly targeted please ping me or a committer to
>>> help target the issue.
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>>>
>>> -
>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>


-- 

 Kent Yao
@ Data Science Center, Hangzhou Research Institute, NetEase Corp.
a spark enthusiast
kyuubiis a unified multi-tenant JDBC interface for large-scale data
processing and analytics, built on top of Apache Spark.

spark-authorizerA Spark SQL extension which provides SQL Standard
Authorization for Apache Spark.
spark-postgres A library for reading data from and transferring data
to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
itatchiA library that brings useful functions from various modern
database management systems to Apache Spark.

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

回复： [VOTE] Release Spark 2.4.8 (RC3)

[郑瑞峰]

+1 (non-binding)

Thanks,
ruifengz



--原始邮件--
发件人:
"Hyukjin Kwon"  
  
http://spark.apache.org/
 
 There are currently no issues targeting 2.4.8 (try project = SPARK AND
 "Target Version/s" = "2.4.8" AND status in (Open, Reopened, "In Progress"))
 
 The tag to be voted on is v2.4.8-rc3 (commit
 e89526d2401b3a04719721c923a6f630e555e286):
 https://github.com/apache/spark/tree/v2.4.8-rc3
 
 The release files, including signatures, digests, etc. can be found at:
 https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-bin/
 
 Signatures used for Spark RCs can be found in this file:
 https://dist.apache.org/repos/dist/dev/spark/KEYS
 
 The staging repository for this release can be found at:
 https://repository.apache.org/content/repositories/orgapachespark-1377/
 
 The documentation corresponding to this release can be found at:
 https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-docs/
 
 The list of bug fixes going into 2.4.8 can be found at the following URL:
 https://s.apache.org/spark-v2.4.8-rc3
 
 This release is using the release script of the tag v2.4.8-rc3.
 
 FAQ
 
 
 =
 How can I help test this release?
 =
 
 If you are a Spark user, you can help us test this release by taking
 an existing Spark workload and running on this release candidate, then
 reporting any regressions.
 
 If you're working in PySpark you can set up a virtual env and install
 the current RC and see if anything important breaks, in the Java/Scala
 you can add the staging repository to your projects resolvers and test
 with the RC (make sure to clean up the artifact cache before/after so
 you don't end up building with an out of date RC going forward).
 
 ===
 What should happen to JIRA tickets still targeting 2.4.8?
 ===
 
 The current list of open tickets targeted at 2.4.8 can be found at:
 https://issues.apache.org/jira/projects/SPARK and search for "Target
 Version/s" = 2.4.8
 
 Committers should look at those and triage. Extremely important bug
 fixes, documentation, and API tweaks that impact compatibility should
 be worked on immediately. Everything else please retarget to an
 appropriate release.
 
 ==
 But my bug isn't fixed?
 ==
 
 In order to make timely releases, we will typically not hold the
 release unless the bug in question is a regression from the previous
 release. That being said, if there is something which is a regression
 that has not been correctly targeted please ping me or a committer to
 help target the issue. 
 
 
 
 --
 Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
 
 -
 To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE] Release Spark 2.4.8 (RC3)

[Hyukjin Kwon]

+1

On Thu, 29 Apr 2021, 07:08 Sean Owen,  wrote:

> +1 from me too, same result as last time.
>
> On Wed, Apr 28, 2021 at 11:33 AM Liang-Chi Hsieh  wrote:
>
>> Please vote on releasing the following candidate as Apache Spark version
>> 2.4.8.
>>
>> The vote is open until May 4th at 9AM PST and passes if a majority +1 PMC
>> votes are cast, with a minimum of 3 +1 votes.
>>
>> [ ] +1 Release this package as Apache Spark 2.4.8
>> [ ] -1 Do not release this package because ...
>>
>> To learn more about Apache Spark, please see http://spark.apache.org/
>>
>> There are currently no issues targeting 2.4.8 (try project = SPARK AND
>> "Target Version/s" = "2.4.8" AND status in (Open, Reopened, "In
>> Progress"))
>>
>> The tag to be voted on is v2.4.8-rc3 (commit
>> e89526d2401b3a04719721c923a6f630e555e286):
>> https://github.com/apache/spark/tree/v2.4.8-rc3
>>
>> The release files, including signatures, digests, etc. can be found at:
>> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-bin/
>>
>> Signatures used for Spark RCs can be found in this file:
>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>
>> The staging repository for this release can be found at:
>> https://repository.apache.org/content/repositories/orgapachespark-1377/
>>
>> The documentation corresponding to this release can be found at:
>> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-docs/
>>
>> The list of bug fixes going into 2.4.8 can be found at the following URL:
>> https://s.apache.org/spark-v2.4.8-rc3
>>
>> This release is using the release script of the tag v2.4.8-rc3.
>>
>> FAQ
>>
>>
>> =
>> How can I help test this release?
>> =
>>
>> If you are a Spark user, you can help us test this release by taking
>> an existing Spark workload and running on this release candidate, then
>> reporting any regressions.
>>
>> If you're working in PySpark you can set up a virtual env and install
>> the current RC and see if anything important breaks, in the Java/Scala
>> you can add the staging repository to your projects resolvers and test
>> with the RC (make sure to clean up the artifact cache before/after so
>> you don't end up building with an out of date RC going forward).
>>
>> ===
>> What should happen to JIRA tickets still targeting 2.4.8?
>> ===
>>
>> The current list of open tickets targeted at 2.4.8 can be found at:
>> https://issues.apache.org/jira/projects/SPARK and search for "Target
>> Version/s" = 2.4.8
>>
>> Committers should look at those and triage. Extremely important bug
>> fixes, documentation, and API tweaks that impact compatibility should
>> be worked on immediately. Everything else please retarget to an
>> appropriate release.
>>
>> ==
>> But my bug isn't fixed?
>> ==
>>
>> In order to make timely releases, we will typically not hold the
>> release unless the bug in question is a regression from the previous
>> release. That being said, if there is something which is a regression
>> that has not been correctly targeted please ping me or a committer to
>> help target the issue.
>>
>>
>>
>> --
>> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>

Re: [DISCUSS] Add RocksDB StateStore

[Liang-Chi Hsieh]

I am fine with RocksDB state store as built-in state store. Actually the
proposal to have it as external module is to avoid the raised concern in the
previous effort.

The need to have it as experimental doesn't necessarily mean to have it as
external module, I think. They are two things. So I don't think the risk is
highly related to have it as external module or built-in one, except that we
have the state store as default one at the beginning. If it is not a default
one, and we explicitly mention it is an experimental feature, the risk is
not very different between an external module and built-in one. As a
built-in one just makes users easier to try it.

That said even the coming RocksDB state store has been supported for years,
I think it is safer to have it as experimental feature first as it lands to
OSS Spark.

Anyway, I think it is okay to add RocksDB state store among built-in state
stores along with HDFSBasedStateStore.

I also feel that we can just have RocksDB and replace LevelDB with RocksDB.
But this is another story.


Liang-Chi


Jungtaek Lim-2 wrote
> I think adding RocksDB state store to sql/core directly would be
> OK. Personally I also voted "either way is fine with me" against RocksDB
> state store implementation in Spark ecosystem. The overall stance hasn't
> changed, but I'd like to point out that the risk becomes quite lower than
> before, given the fact we can leverage Databricks RocksDB state store
> implementation.
> 
> I feel there were two major reasons to add RocksDB state store to external
> module;
> 
> 1. stability
> 
> Databricks RocksDB state store implementation has been supported for
> years,
> it won't require more time to incubate. We may want to review thoughtfully
> to ensure the open sourced proposal fits to the Apache Spark and still
> retains stability, but this is quite better than the previous targets to
> adopt which may not be tested in production for years.
> 
> That makes me think that we don't have to put it into external and
> consider
> it as experimental.
> 
> 2. dependency
> 
> From Yuanjian's mail, JNI library is the only dependency, which seems fine
> to add by default. We already have LevelDB as one of core dependencies and
> don't concern too much about the JNI library dependency. Probably someone
> might figure out that there are outstanding benefits on replacing LevelDB
> with RocksDB and then RocksDB can even be the one of core dependencies.
> 
> On Tue, Apr 27, 2021 at 6:41 PM Yuanjian Li 

> xyliyuanjian@

>  wrote:
> 
>> Hi all,
>>
>> Following the latest comments in SPARK-34198
>> https://issues.apache.org/jira/browse/SPARK-34198;, Databricks
>> decided
>> to donate the commercial implementation of the RocksDBStateStore.
>> Compared
>> with the original decision, there’s only one topic we want to raise again
>> for discussion: can we directly add the RockDBStateStoreProvider in the
>> sql/core module? This suggestion based on the following reasons:
>>
>>1.
>>
>>The RocksDBStateStore aims to solve the problem of the original
>>HDFSBasedStateStore, which is built-in.
>>2.
>>
>>End users can conveniently set the config to use the new
>>implementation.
>>3.
>>
>>We can set the RocksDB one as the default one in the future.
>>
>>
>> For the consideration of the dependency, I also checked the rocksdbjni we
>> might introduce. As a JNI package
>> https://repo1.maven.org/maven2/org/rocksdb/rocksdbjni/6.2.2/rocksdbjni-6.2.2.pom;,
>> it should not have any dependency conflicts with Apache Spark.
>>
>> Any suggestions are welcome!
>>
>> Best,
>>
>> Yuanjian





--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE] Release Spark 2.4.8 (RC3)

[Sean Owen]

+1 from me too, same result as last time.

On Wed, Apr 28, 2021 at 11:33 AM Liang-Chi Hsieh  wrote:

> Please vote on releasing the following candidate as Apache Spark version
> 2.4.8.
>
> The vote is open until May 4th at 9AM PST and passes if a majority +1 PMC
> votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 2.4.8
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
>
> There are currently no issues targeting 2.4.8 (try project = SPARK AND
> "Target Version/s" = "2.4.8" AND status in (Open, Reopened, "In Progress"))
>
> The tag to be voted on is v2.4.8-rc3 (commit
> e89526d2401b3a04719721c923a6f630e555e286):
> https://github.com/apache/spark/tree/v2.4.8-rc3
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1377/
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-docs/
>
> The list of bug fixes going into 2.4.8 can be found at the following URL:
> https://s.apache.org/spark-v2.4.8-rc3
>
> This release is using the release script of the tag v2.4.8-rc3.
>
> FAQ
>
>
> =
> How can I help test this release?
> =
>
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with an out of date RC going forward).
>
> ===
> What should happen to JIRA tickets still targeting 2.4.8?
> ===
>
> The current list of open tickets targeted at 2.4.8 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 2.4.8
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==
> But my bug isn't fixed?
> ==
>
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something which is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
>
>
>
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: [VOTE] Release Spark 2.4.8 (RC3)

[Holden Karau]

I verified the virtualenv & pyspark installation on OSX & linux works
as expected with the minimum version of Python.

I double checked the Python tagged versions (mostly checking Python
3.8 wasn't listed since Spark 2.x only does up to 3.7). It might be
good to include that as a reminder in the release announcement since
the error message when using in Python 3.8 isn't very clear.

+1 (binding)


On Wed, Apr 28, 2021 at 10:43 AM Dongjoon Hyun  wrote:
>
> +1
>
> Thank you!
>
> Bests,
> Dongjoon.
>
> On Wed, Apr 28, 2021 at 10:32 AM Maxim Gekk  wrote:
>>
>> +1 (non-binding)
>>
>> On Wed, Apr 28, 2021 at 8:12 PM Wenchen Fan  wrote:
>>>
>>> +1 (binding)
>>>
>>> On Thu, Apr 29, 2021 at 1:05 AM DB Tsai  wrote:

 +1 (binding)

 > On Apr 28, 2021, at 9:26 AM, Liang-Chi Hsieh  wrote:
 >
 >
 > Please vote on releasing the following candidate as Apache Spark version
 > 2.4.8.
 >
 > The vote is open until May 4th at 9AM PST and passes if a majority +1 PMC
 > votes are cast, with a minimum of 3 +1 votes.
 >
 > [ ] +1 Release this package as Apache Spark 2.4.8
 > [ ] -1 Do not release this package because ...
 >
 > To learn more about Apache Spark, please see http://spark.apache.org/
 >
 > There are currently no issues targeting 2.4.8 (try project = SPARK AND
 > "Target Version/s" = "2.4.8" AND status in (Open, Reopened, "In 
 > Progress"))
 >
 > The tag to be voted on is v2.4.8-rc3 (commit
 > e89526d2401b3a04719721c923a6f630e555e286):
 > https://github.com/apache/spark/tree/v2.4.8-rc3
 >
 > The release files, including signatures, digests, etc. can be found at:
 > https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-bin/
 >
 > Signatures used for Spark RCs can be found in t
 his file:
 > https://dist.apache.org/repos/dist/dev/spark/KEYS
 >
 > The staging repository for this release can be found at:
 > https://repository.apache.org/content/repositories/orgapachespark-1377/
 >
 > The documentation corresponding to this release can be found at:
 > https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-docs/
 >
 > The list of bug fixes going into 2.4.8 can be found at the following URL:
 > https://s.apache.org/spark-v2.4.8-rc3
 >
 > This release is using the release script of the tag v2.4.8-rc3.
 >
 > FAQ
 >
 >
 > =
 > How can I help test this release?
 > =
 >
 > If you are a Spark user, you can help us test this release by taking
 > an existing Spark workload and running on this release candidate, then
 > reporting any regressions.
 >
 > If you're working in PySpark you can set up a virtual env and install
 > the current RC and see if anything important breaks, in the Java/Scala
 > you can add the stag
 ing repository to your projects resolvers and test
 > with the RC (make sure to clean up the artifact cache before/after so
 > you don't end up building with an out of date RC going forward).
 >
 > ===
 > What should happen to JIRA tickets still targeting 2.4.8?
 > ===
 >
 > The current list of open tickets targeted at 2.4.8 can be found at:
 > https://issues.apache.org/jira/projects/SPARK and search for "Target
 > Version/s" = 2.4.8
 >
 > Committers should look at those and triage. Extremely important bug
 > fixes, documentation, and API tweaks that impact compatibility should
 > be worked on immediately. Everything else please retarget to an
 > appropriate release.
 >
 > ==
 > But my bug isn't fixed?
 > ==
 >
 > In order to make timely releases, we will typically not hold the
 > release unless the bug in question is a regression from the previous
 > release. That being
  said, if there is something which is a regression
 > that has not been correctly targeted please ping me or a committer to
 > help target the issue.
 >
 >
 >
 > --
 > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
 >
 > -
 > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
 >



--
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9
YouTube Live Streams: https://www.youtube.com/user/holdenkarau

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE] Release Spark 2.4.8 (RC3)

[Dongjoon Hyun]

+1

Thank you!

Bests,
Dongjoon.

On Wed, Apr 28, 2021 at 10:32 AM Maxim Gekk 
wrote:

> +1 (non-binding)
>
> On Wed, Apr 28, 2021 at 8:12 PM Wenchen Fan  wrote:
>
>> +1 (binding)
>>
>> On Thu, Apr 29, 2021 at 1:05 AM DB Tsai  wrote:
>>
>>> +1 (binding)
>>>
>>> > On Apr 28, 2021, at 9:26 AM, Liang-Chi Hsieh  wrote:
>>> >
>>> >
>>> > Please vote on releasing the following candidate as Apache Spark
>>> version
>>> > 2.4.8.
>>> >
>>> > The vote is open until May 4th at 9AM PST and passes if a majority +1
>>> PMC
>>> > votes are cast, with a minimum of 3 +1 votes.
>>> >
>>> > [ ] +1 Release this package as Apache Spark 2.4.8
>>> > [ ] -1 Do not release this package because ...
>>> >
>>> > To learn more about Apache Spark, please see http://spark.apache.org/
>>> >
>>> > There are currently no issues targeting 2.4.8 (try project = SPARK AND
>>> > "Target Version/s" = "2.4.8" AND status in (Open, Reopened, "In
>>> Progress"))
>>> >
>>> > The tag to be voted on is v2.4.8-rc3 (commit
>>> > e89526d2401b3a04719721c923a6f630e555e286):
>>> > https://github.com/apache/spark/tree/v2.4.8-rc3
>>> >
>>> > The release files, including signatures, digests, etc. can be found at:
>>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-bin/
>>> >
>>> > Signatures used for Spark RCs can be found in t
>>> his file:
>>> > https://dist.apache.org/repos/dist/dev/spark/KEYS
>>> >
>>> > The staging repository for this release can be found at:
>>> >
>>> https://repository.apache.org/content/repositories/orgapachespark-1377/
>>> >
>>> > The documentation corresponding to this release can be found at:
>>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-docs/
>>> >
>>> > The list of bug fixes going into 2.4.8 can be found at the following
>>> URL:
>>> > https://s.apache.org/spark-v2.4.8-rc3
>>> >
>>> > This release is using the release script of the tag v2.4.8-rc3.
>>> >
>>> > FAQ
>>> >
>>> >
>>> > =
>>> > How can I help test this release?
>>> > =
>>> >
>>> > If you are a Spark user, you can help us test this release by taking
>>> > an existing Spark workload and running on this release candidate, then
>>> > reporting any regressions.
>>> >
>>> > If you're working in PySpark you can set up a virtual env and install
>>> > the current RC and see if anything important breaks, in the Java/Scala
>>> > you can add the stag
>>> ing repository to your projects resolvers and test
>>> > with the RC (make sure to clean up the artifact cache before/after so
>>> > you don't end up building with an out of date RC going forward).
>>> >
>>> > ===
>>> > What should happen to JIRA tickets still targeting 2.4.8?
>>> > ===
>>> >
>>> > The current list of open tickets targeted at 2.4.8 can be found at:
>>> > https://issues.apache.org/jira/projects/SPARK and search for "Target
>>> > Version/s" = 2.4.8
>>> >
>>> > Committers should look at those and triage. Extremely important bug
>>> > fixes, documentation, and API tweaks that impact compatibility should
>>> > be worked on immediately. Everything else please retarget to an
>>> > appropriate release.
>>> >
>>> > ==
>>> > But my bug isn't fixed?
>>> > ==
>>> >
>>> > In order to make timely releases, we will typically not hold the
>>> > release unless the bug in question is a regression from the previous
>>> > release. That being
>>>  said, if there is something which is a regression
>>> > that has not been correctly targeted please ping me or a committer to
>>> > help target the issue.
>>> >
>>> >
>>> >
>>> > --
>>> > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>>> >
>>> > -
>>> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>> >
>>>
>>>

Re: [VOTE] Release Spark 2.4.8 (RC3)

[Maxim Gekk]

+1 (non-binding)

On Wed, Apr 28, 2021 at 8:12 PM Wenchen Fan  wrote:

> +1 (binding)
>
> On Thu, Apr 29, 2021 at 1:05 AM DB Tsai  wrote:
>
>> +1 (binding)
>>
>> > On Apr 28, 2021, at 9:26 AM, Liang-Chi Hsieh  wrote:
>> >
>> >
>> > Please vote on releasing the following candidate as Apache Spark version
>> > 2.4.8.
>> >
>> > The vote is open until May 4th at 9AM PST and passes if a majority +1
>> PMC
>> > votes are cast, with a minimum of 3 +1 votes.
>> >
>> > [ ] +1 Release this package as Apache Spark 2.4.8
>> > [ ] -1 Do not release this package because ...
>> >
>> > To learn more about Apache Spark, please see http://spark.apache.org/
>> >
>> > There are currently no issues targeting 2.4.8 (try project = SPARK AND
>> > "Target Version/s" = "2.4.8" AND status in (Open, Reopened, "In
>> Progress"))
>> >
>> > The tag to be voted on is v2.4.8-rc3 (commit
>> > e89526d2401b3a04719721c923a6f630e555e286):
>> > https://github.com/apache/spark/tree/v2.4.8-rc3
>> >
>> > The release files, including signatures, digests, etc. can be found at:
>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-bin/
>> >
>> > Signatures used for Spark RCs can be found in t
>> his file:
>> > https://dist.apache.org/repos/dist/dev/spark/KEYS
>> >
>> > The staging repository for this release can be found at:
>> > https://repository.apache.org/content/repositories/orgapachespark-1377/
>> >
>> > The documentation corresponding to this release can be found at:
>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-docs/
>> >
>> > The list of bug fixes going into 2.4.8 can be found at the following
>> URL:
>> > https://s.apache.org/spark-v2.4.8-rc3
>> >
>> > This release is using the release script of the tag v2.4.8-rc3.
>> >
>> > FAQ
>> >
>> >
>> > =
>> > How can I help test this release?
>> > =
>> >
>> > If you are a Spark user, you can help us test this release by taking
>> > an existing Spark workload and running on this release candidate, then
>> > reporting any regressions.
>> >
>> > If you're working in PySpark you can set up a virtual env and install
>> > the current RC and see if anything important breaks, in the Java/Scala
>> > you can add the stag
>> ing repository to your projects resolvers and test
>> > with the RC (make sure to clean up the artifact cache before/after so
>> > you don't end up building with an out of date RC going forward).
>> >
>> > ===
>> > What should happen to JIRA tickets still targeting 2.4.8?
>> > ===
>> >
>> > The current list of open tickets targeted at 2.4.8 can be found at:
>> > https://issues.apache.org/jira/projects/SPARK and search for "Target
>> > Version/s" = 2.4.8
>> >
>> > Committers should look at those and triage. Extremely important bug
>> > fixes, documentation, and API tweaks that impact compatibility should
>> > be worked on immediately. Everything else please retarget to an
>> > appropriate release.
>> >
>> > ==
>> > But my bug isn't fixed?
>> > ==
>> >
>> > In order to make timely releases, we will typically not hold the
>> > release unless the bug in question is a regression from the previous
>> > release. That being
>>  said, if there is something which is a regression
>> > that has not been correctly targeted please ping me or a committer to
>> > help target the issue.
>> >
>> >
>> >
>> > --
>> > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>> >
>> > -
>> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>> >
>>
>>

Re: Bintray replacement for spark-packages.org

[Wenchen Fan]

Shall we make new releases for 3.0 and 3.1? So that people don't need to
change the sbt resolver/pom files to work around Bintray sunset. It's also
been a while since the last 3.0 and 3.1 releases.

On Tue, Apr 27, 2021 at 9:02 AM Matthew Powers 
wrote:

> Great job fixing this!!  I just checked and it's working on my end.  Updated
> the resolver
> 
> and sbt test still works just fine.
>
> On Mon, Apr 26, 2021 at 3:31 AM Bo Zhang  wrote:
>
>> Hi Apache Spark devs,
>>
>> As you might know, Bintray, which is the repository service used for
>> spark-packages.org, is in its sunset process. There was a planned
>> brown-out on April 12th
>>  and there will be
>> another one on April 26th
>> , and it will no
>> longer be available from May 1st.
>>
>> We have spun up a new repository service at
>> https://repos.spark-packages.org and it will be the new home for the
>> artifacts on spark-packages.
>>
>> Given the planned Bintray brown-out, this is a good time for us to test
>> the new repository service. To consume artifacts from that, please replace "
>> dl.bintray.com/spark-packages/maven" with "repos.spark-packages.org" in
>> the Maven pom files or sbt build files in your repositories, e.g.:
>> https://github.com/apache/spark/pull/32346
>>
>> We are still working on the release process to the new repository
>> service, and will provide an update here shortly.
>>
>> If you have any questions for using the new repository service, or any
>> general questions for spark-packages, please reach out to
>> feedb...@spark-packages.org.
>>
>> Thanks,
>> Bo
>>
>

Re: [VOTE] Release Spark 2.4.8 (RC3)

[Wenchen Fan]

+1 (binding)

On Thu, Apr 29, 2021 at 1:05 AM DB Tsai  wrote:

> +1 (binding)
>
> > On Apr 28, 2021, at 9:26 AM, Liang-Chi Hsieh  wrote:
> >
> >
> > Please vote on releasing the following candidate as Apache Spark version
> > 2.4.8.
> >
> > The vote is open until May 4th at 9AM PST and passes if a majority +1 PMC
> > votes are cast, with a minimum of 3 +1 votes.
> >
> > [ ] +1 Release this package as Apache Spark 2.4.8
> > [ ] -1 Do not release this package because ...
> >
> > To learn more about Apache Spark, please see http://spark.apache.org/
> >
> > There are currently no issues targeting 2.4.8 (try project = SPARK AND
> > "Target Version/s" = "2.4.8" AND status in (Open, Reopened, "In
> Progress"))
> >
> > The tag to be voted on is v2.4.8-rc3 (commit
> > e89526d2401b3a04719721c923a6f630e555e286):
> > https://github.com/apache/spark/tree/v2.4.8-rc3
> >
> > The release files, including signatures, digests, etc. can be found at:
> > https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-bin/
> >
> > Signatures used for Spark RCs can be found in t
> his file:
> > https://dist.apache.org/repos/dist/dev/spark/KEYS
> >
> > The staging repository for this release can be found at:
> > https://repository.apache.org/content/repositories/orgapachespark-1377/
> >
> > The documentation corresponding to this release can be found at:
> > https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-docs/
> >
> > The list of bug fixes going into 2.4.8 can be found at the following URL:
> > https://s.apache.org/spark-v2.4.8-rc3
> >
> > This release is using the release script of the tag v2.4.8-rc3.
> >
> > FAQ
> >
> >
> > =
> > How can I help test this release?
> > =
> >
> > If you are a Spark user, you can help us test this release by taking
> > an existing Spark workload and running on this release candidate, then
> > reporting any regressions.
> >
> > If you're working in PySpark you can set up a virtual env and install
> > the current RC and see if anything important breaks, in the Java/Scala
> > you can add the stag
> ing repository to your projects resolvers and test
> > with the RC (make sure to clean up the artifact cache before/after so
> > you don't end up building with an out of date RC going forward).
> >
> > ===
> > What should happen to JIRA tickets still targeting 2.4.8?
> > ===
> >
> > The current list of open tickets targeted at 2.4.8 can be found at:
> > https://issues.apache.org/jira/projects/SPARK and search for "Target
> > Version/s" = 2.4.8
> >
> > Committers should look at those and triage. Extremely important bug
> > fixes, documentation, and API tweaks that impact compatibility should
> > be worked on immediately. Everything else please retarget to an
> > appropriate release.
> >
> > ==
> > But my bug isn't fixed?
> > ==
> >
> > In order to make timely releases, we will typically not hold the
> > release unless the bug in question is a regression from the previous
> > release. That being
>  said, if there is something which is a regression
> > that has not been correctly targeted please ping me or a committer to
> > help target the issue.
> >
> >
> >
> > --
> > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
> >
> > -
> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
> >
>
>

Re: [VOTE] Release Spark 2.4.8 (RC3)

[DB Tsai]

+1 (binding)

> On Apr 28, 2021, at 9:26 AM, Liang-Chi Hsieh  wrote:
>
>
> Please vote on releasing the following candidate as Apache Spark version
> 2.4.8.
>
> The vote is open until May 4th at 9AM PST and passes if a majority +1 PMC
> votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 2.4.8
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
>
> There are currently no issues targeting 2.4.8 (try project = SPARK AND
> "Target Version/s" = "2.4.8" AND status in (Open, Reopened, "In Progress"))
>
> The tag to be voted on is v2.4.8-rc3 (commit
> e89526d2401b3a04719721c923a6f630e555e286):
> https://github.com/apache/spark/tree/v2.4.8-rc3
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-bin/
>
> Signatures used for Spark RCs can be found in t
his file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1377/
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-docs/
>
> The list of bug fixes going into 2.4.8 can be found at the following URL:
> https://s.apache.org/spark-v2.4.8-rc3
>
> This release is using the release script of the tag v2.4.8-rc3.
>
> FAQ
>
>
> =
> How can I help test this release?
> =
>
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the stag
ing repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with an out of date RC going forward).
>
> ===
> What should happen to JIRA tickets still targeting 2.4.8?
> ===
>
> The current list of open tickets targeted at 2.4.8 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 2.4.8
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==
> But my bug isn't fixed?
> ==
>
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being
 said, if there is something which is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
>
>
>
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>



publickey - dbtsai@dbtsai.com - 5566b502.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

[VOTE] Release Spark 2.4.8 (RC3)

[Liang-Chi Hsieh]

Please vote on releasing the following candidate as Apache Spark version
2.4.8.

The vote is open until May 4th at 9AM PST and passes if a majority +1 PMC
votes are cast, with a minimum of 3 +1 votes.

[ ] +1 Release this package as Apache Spark 2.4.8
[ ] -1 Do not release this package because ...

To learn more about Apache Spark, please see http://spark.apache.org/

There are currently no issues targeting 2.4.8 (try project = SPARK AND
"Target Version/s" = "2.4.8" AND status in (Open, Reopened, "In Progress"))

The tag to be voted on is v2.4.8-rc3 (commit
e89526d2401b3a04719721c923a6f630e555e286):
https://github.com/apache/spark/tree/v2.4.8-rc3

The release files, including signatures, digests, etc. can be found at:
https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-bin/

Signatures used for Spark RCs can be found in this file:
https://dist.apache.org/repos/dist/dev/spark/KEYS

The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-1377/

The documentation corresponding to this release can be found at:
https://dist.apache.org/repos/dist/dev/spark/v2.4.8-rc3-docs/

The list of bug fixes going into 2.4.8 can be found at the following URL:
https://s.apache.org/spark-v2.4.8-rc3

This release is using the release script of the tag v2.4.8-rc3.

FAQ


=
How can I help test this release?
=

If you are a Spark user, you can help us test this release by taking
an existing Spark workload and running on this release candidate, then
reporting any regressions.

If you're working in PySpark you can set up a virtual env and install
the current RC and see if anything important breaks, in the Java/Scala
you can add the staging repository to your projects resolvers and test
with the RC (make sure to clean up the artifact cache before/after so
you don't end up building with an out of date RC going forward).

===
What should happen to JIRA tickets still targeting 2.4.8?
===

The current list of open tickets targeted at 2.4.8 can be found at:
https://issues.apache.org/jira/projects/SPARK and search for "Target
Version/s" = 2.4.8

Committers should look at those and triage. Extremely important bug
fixes, documentation, and API tweaks that impact compatibility should
be worked on immediately. Everything else please retarget to an
appropriate release.

==
But my bug isn't fixed?
==

In order to make timely releases, we will typically not hold the
release unless the bug in question is a regression from the previous
release. That being said, if there is something which is a regression
that has not been correctly targeted please ping me or a committer to
help target the issue. 



--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

RE: [VOTE] Release Spark 2.4.8 (RC2)

[Nicholas Marion]

Hi,

Looks like both this PR and another PR have been merged into 2.4 branch.
Wondering if we will be seeing an RC3 soon?



   
 Regards,   
   

   
 NICHOLAS T. MARION 
   
 AI and Analytics Development Lead | IzODA CPO  
   

   






 Phone: 1-845-433-5010 | Tie-Line: 293-5010 
IBM 
 E-mail: nmar...@us.ibm.com 

 Find me on: LinkedIn:2455 
South Rd 
 http://www.linkedin.com/in/nicholasmarion Poughkeepie, New York 
12601-5400 


  United 
States 














From:   Liang-Chi Hsieh 
To: dev@spark.apache.org
Date:   04/15/2021 01:07 PM
Subject:[EXTERNAL] Re: [VOTE] Release Spark 2.4.8 (RC2)



Thanks all for voting. Unfortunately, we found a long-standing correctness
bug SPARK-35080 and 2.4 was affected too. That is said we need to drop RC2
in favor of RC3.

The fix is ready for merging at
https://github.com/apache/spark/pull/32179
 .






--
Sent from:
http://apache-spark-developers-list.1001551.n3.nabble.com/


-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org