Re: [DISCUSS] Rename hadoop-3.2/hadoop-2.7 profile to hadoop-3/hadoop-2?

[Chao Sun]

Thanks all for the feedback! Yes I agree that we should target this for
Apache Spark 3.3 release. I'll put this aside for now and pick it up again
after the 3.2 release is finished.

> And maybe the current naming leaves the possibility for a "hadoop-3.5" or
something if that needed to be different.

Yes, that's a good point, although I was under the impression that the
Spark community aims to only support a single Hadoop 3.x profile, in which
case we won't have `hadoop-3` and `hadoop-3.5` in parallel.

Chao


On Thu, Jun 24, 2021 at 10:25 PM Gengliang Wang  wrote:

> +1 for targeting the renaming for Apache Spark 3.3 at the current phase.
>
> On Fri, Jun 25, 2021 at 6:55 AM DB Tsai  wrote:
>
>> +1 on renaming.
>>
>> DB Tsai  |  https://www.dbtsai.com/  |  PGP 42E5B25A8F7A82C1
>>
>> On Jun 24, 2021, at 11:41 AM, Chao Sun  wrote:
>>
>> Hi,
>>
>> As Spark master has upgraded to Hadoop-3.3.1, the current Maven profile
>> name hadoop-3.2 is no longer accurate, and it may confuse Spark users when
>> they realize the actual version is not Hadoop 3.2.x. Therefore, I created
>> https://issues.apache.org/jira/browse/SPARK-33880 to change the profile
>> name to hadoop-3 and hadoop-2 respectively. What do you think? Is this
>> something worth doing as part of Spark 3.2.0 release?
>>
>> Best,
>> Chao
>>
>>
>>

Re: [DISCUSS] Rename hadoop-3.2/hadoop-2.7 profile to hadoop-3/hadoop-2?

[Gengliang Wang]

+1 for targeting the renaming for Apache Spark 3.3 at the current phase.

On Fri, Jun 25, 2021 at 6:55 AM DB Tsai  wrote:

> +1 on renaming.
>
> DB Tsai  |  https://www.dbtsai.com/  |  PGP 42E5B25A8F7A82C1
>
> On Jun 24, 2021, at 11:41 AM, Chao Sun  wrote:
>
> Hi,
>
> As Spark master has upgraded to Hadoop-3.3.1, the current Maven profile
> name hadoop-3.2 is no longer accurate, and it may confuse Spark users when
> they realize the actual version is not Hadoop 3.2.x. Therefore, I created
> https://issues.apache.org/jira/browse/SPARK-33880 to change the profile
> name to hadoop-3 and hadoop-2 respectively. What do you think? Is this
> something worth doing as part of Spark 3.2.0 release?
>
> Best,
> Chao
>
>
>

Re: [DISCUSS] Rename hadoop-3.2/hadoop-2.7 profile to hadoop-3/hadoop-2?

[DB Tsai]

+1 on renaming.

DB Tsai  |  https://www.dbtsai.com/  |  PGP 42E5B25A8F7A82C1

> On Jun 24, 2021, at 11:41 AM, Chao Sun  wrote:
> 
> Hi,
> 
> As Spark master has upgraded to Hadoop-3.3.1, the current Maven profile name 
> hadoop-3.2 is no longer accurate, and it may confuse Spark users when they 
> realize the actual version is not Hadoop 3.2.x. Therefore, I created 
> https://issues.apache.org/jira/browse/SPARK-33880 
>  to change the profile 
> name to hadoop-3 and hadoop-2 respectively. What do you think? Is this 
> something worth doing as part of Spark 3.2.0 release?
> 
> Best,
> Chao

Re: [DISCUSS] Rename hadoop-3.2/hadoop-2.7 profile to hadoop-3/hadoop-2?

[Dongjoon Hyun]

For renaming, I'd target it for Apache Spark 3.3 instead of Apache Spark 3.2
because this is the first release of using Apache Hadoop 3.3.1 and we may
need to revert Apache Hadoop 3.3.1 during RC period.

Dongjoon.

On Thu, Jun 24, 2021 at 12:24 PM Sean Owen  wrote:

> The downside here is that it would break downstream builds that set
> hadoop-3.2 if it's now called hadoop-3. That's not a huge deal. We can
> retain dummy profiles under the old names that do nothing, but that would
> be a quieter 'break'. I suppose this naming is only of importance to
> developers, who might realize that hadoop-3.2 means "hadoop-3.2 or later".
> And maybe the current naming leaves the possibility for a "hadoop-3.5" or
> something if that needed to be different.
>
> I don't feel strongly but would default to leaving it, very slightly.
>
> On Thu, Jun 24, 2021 at 1:42 PM Chao Sun  wrote:
>
>> Hi,
>>
>> As Spark master has upgraded to Hadoop-3.3.1, the current Maven profile
>> name hadoop-3.2 is no longer accurate, and it may confuse Spark users when
>> they realize the actual version is not Hadoop 3.2.x. Therefore, I created
>> https://issues.apache.org/jira/browse/SPARK-33880 to change the profile
>> name to hadoop-3 and hadoop-2 respectively. What do you think? Is this
>> something worth doing as part of Spark 3.2.0 release?
>>
>> Best,
>> Chao
>>
>

Re: [DISCUSS] Rename hadoop-3.2/hadoop-2.7 profile to hadoop-3/hadoop-2?

[Sean Owen]

The downside here is that it would break downstream builds that set
hadoop-3.2 if it's now called hadoop-3. That's not a huge deal. We can
retain dummy profiles under the old names that do nothing, but that would
be a quieter 'break'. I suppose this naming is only of importance to
developers, who might realize that hadoop-3.2 means "hadoop-3.2 or later".
And maybe the current naming leaves the possibility for a "hadoop-3.5" or
something if that needed to be different.

I don't feel strongly but would default to leaving it, very slightly.

On Thu, Jun 24, 2021 at 1:42 PM Chao Sun  wrote:

> Hi,
>
> As Spark master has upgraded to Hadoop-3.3.1, the current Maven profile
> name hadoop-3.2 is no longer accurate, and it may confuse Spark users when
> they realize the actual version is not Hadoop 3.2.x. Therefore, I created
> https://issues.apache.org/jira/browse/SPARK-33880 to change the profile
> name to hadoop-3 and hadoop-2 respectively. What do you think? Is this
> something worth doing as part of Spark 3.2.0 release?
>
> Best,
> Chao
>

[DISCUSS] Rename hadoop-3.2/hadoop-2.7 profile to hadoop-3/hadoop-2?

[Chao Sun]

Hi,

As Spark master has upgraded to Hadoop-3.3.1, the current Maven profile
name hadoop-3.2 is no longer accurate, and it may confuse Spark users when
they realize the actual version is not Hadoop 3.2.x. Therefore, I created
https://issues.apache.org/jira/browse/SPARK-33880 to change the profile
name to hadoop-3 and hadoop-2 respectively. What do you think? Is this
something worth doing as part of Spark 3.2.0 release?

Best,
Chao