Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Allison Wang]

Hi Mridul,

Thanks for reviewing the SPIP. As you correctly pointed out, this proposal
is an internal, Spark engine developer-facing improvement to make error
messages more consistent and user-friendly across the Spark codebase, as
opposed to establishing a public contract with end-users. 

Thanks,
Allison



--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Mridul Muralidharan]

I like the idea of consistent messages; it makes understanding errors
easier.
Having said that, Exception messages themselves are not part of the exposed
contract to users; and are subject to change.
We should leave that flexibility open to spark developers ... I am
currently viewing this proposal as a internal standardization exercise
within Spark codebase, and not as a public contract with users.
Is that aligned with the objectives ? Or are we looking at this as a public
contract to users ?
I am not in favour of the latter.

Regards,
Mridul


On Sun, Oct 25, 2020 at 7:05 PM Xinyi Yu  wrote:

> Hi all,
>
> We like to post a SPIP of Standardize Exception Messages in Spark. Here is
> the document link:
>
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
> <
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing>
>
>
> This SPIP aims to standardize the exception messages in Spark. It has three
> major focuses:
> 1. Group exception messages in dedicated files for easy maintenance and
> auditing.
> 2. Establish an error message guideline for developers.
> 3. Improve error message quality.
>
> Thanks for your time and patience. Looking forward to your feedback!
>
>
>
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Xinyi Yu]

Hi Takeshi, Xiao and all devs,

Thank you for your kind feedback! We are going to initiate a vote for this SPIP 
recently.

On Oct 29, 2020, 6:41 PM -0700, Takeshi Yamamuro , wrote:
> I've already left comments about this idea in 
> https://github.com/apache/spark/pull/29497 though,
> it sounds plausible. +1.
>
> > On Mon, Oct 26, 2020 at 9:04 AM Xinyi Yu  wrote:
> > > Hi all,
> > >
> > > We like to post a SPIP of Standardize Exception Messages in Spark. Here is
> > > the document link:
> > > https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
> > > 
> > >
> > > This SPIP aims to standardize the exception messages in Spark. It has 
> > > three
> > > major focuses:
> > > 1. Group exception messages in dedicated files for easy maintenance and
> > > auditing.
> > > 2. Establish an error message guideline for developers.
> > > 3. Improve error message quality.
> > >
> > > Thanks for your time and patience. Looking forward to your feedback!
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
> > >
> > > -
> > > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
> > >
>
>
> --
> ---
> Takeshi Yamamuro

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Takeshi Yamamuro]

I've already left comments about this idea in
https://github.com/apache/spark/pull/29497 though,
it sounds plausible. +1.

On Mon, Oct 26, 2020 at 9:04 AM Xinyi Yu  wrote:

> Hi all,
>
> We like to post a SPIP of Standardize Exception Messages in Spark. Here is
> the document link:
>
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
> <
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing>
>
>
> This SPIP aims to standardize the exception messages in Spark. It has three
> major focuses:
> 1. Group exception messages in dedicated files for easy maintenance and
> auditing.
> 2. Establish an error message guideline for developers.
> 3. Improve error message quality.
>
> Thanks for your time and patience. Looking forward to your feedback!
>
>
>
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

-- 
---
Takeshi Yamamuro

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Xiao Li]

+1

This is a great proposal to improve the usability of Spark. Make Spark
simple to use!

Xiao

Xinyi Yu  于2020年10月27日周二 下午8:25写道：

> Hi Chang,
>
> It is a script that directly analyzes the source code searching for raw
> "throw new” exception. : ) Hope that give an intuitive overview of current
> exceptions in Spark.
> On Oct 27, 2020, 8:21 PM -0700, Chang Chen , wrote:
>
> hi Xinyi
>
> Just curious, which tool did you use to generate this
> 
>
>
> Xinyi Yu  于2020年10月26日周一 上午8:05写道：
>
>> Hi all,
>>
>> We like to post a SPIP of Standardize Exception Messages in Spark. Here is
>> the document link:
>>
>> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
>> <
>> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
>> >
>>
>> This SPIP aims to standardize the exception messages in Spark. It has
>> three
>> major focuses:
>> 1. Group exception messages in dedicated files for easy maintenance and
>> auditing.
>> 2. Establish an error message guideline for developers.
>> 3. Improve error message quality.
>>
>> Thanks for your time and patience. Looking forward to your feedback!
>>
>>
>>
>> --
>> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Xinyi Yu]

Hi Chang,

It is a script that directly analyzes the source code searching for raw "throw 
new” exception. : ) Hope that give an intuitive overview of current exceptions 
in Spark.
On Oct 27, 2020, 8:21 PM -0700, Chang Chen , wrote:
> hi Xinyi
>
> Just curious, which tool did you use to generate this
>
> > Xinyi Yu  于2020年10月26日周一 上午8:05写道：
> > > Hi all,
> > >
> > > We like to post a SPIP of Standardize Exception Messages in Spark. Here is
> > > the document link:
> > > https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
> > > 
> > >
> > > This SPIP aims to standardize the exception messages in Spark. It has 
> > > three
> > > major focuses:
> > > 1. Group exception messages in dedicated files for easy maintenance and
> > > auditing.
> > > 2. Establish an error message guideline for developers.
> > > 3. Improve error message quality.
> > >
> > > Thanks for your time and patience. Looking forward to your feedback!
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
> > >
> > > -
> > > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
> > >

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Chang Chen]

hi Xinyi

Just curious, which tool did you use to generate this



Xinyi Yu  于2020年10月26日周一 上午8:05写道：

> Hi all,
>
> We like to post a SPIP of Standardize Exception Messages in Spark. Here is
> the document link:
>
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
> <
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing>
>
>
> This SPIP aims to standardize the exception messages in Spark. It has three
> major focuses:
> 1. Group exception messages in dedicated files for easy maintenance and
> auditing.
> 2. Establish an error message guideline for developers.
> 3. Improve error message quality.
>
> Thanks for your time and patience. Looking forward to your feedback!
>
>
>
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Hyukjin Kwon]

Thanks for pointing this out, Nicholas.
This SPIP seems focused on the Scala side, grouping the exception handling
and providing some guidance about error messages.
Yes, I think we can refer to it on the PySpark side. Probably I will follow
up and file some JIRAs based on how this SPIP gose, and run it separately.
BTW, it’s slightly different from SPARK-32194
 because this JIRA
targets to have a set of proper classes instead of, for example, just plain
Exception.


2020년 10월 26일 (월) 오전 10:36, Nicholas Chammas 님이
작성:

> Just want to call out that this SPIP should probably account somehow for
> PySpark and the work being done in SPARK-32082
>  to improve PySpark
> exceptions.
>
> On Sun, Oct 25, 2020 at 8:05 PM Xinyi Yu  wrote:
>
>> Hi all,
>>
>> We like to post a SPIP of Standardize Exception Messages in Spark. Here is
>> the document link:
>>
>> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
>> <
>> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing>
>>
>>
>> This SPIP aims to standardize the exception messages in Spark. It has
>> three
>> major focuses:
>> 1. Group exception messages in dedicated files for easy maintenance and
>> auditing.
>> 2. Establish an error message guideline for developers.
>> 3. Improve error message quality.
>>
>> Thanks for your time and patience. Looking forward to your feedback!
>>
>>
>>
>> --
>> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Nicholas Chammas]

Just want to call out that this SPIP should probably account somehow for
PySpark and the work being done in SPARK-32082
 to improve PySpark
exceptions.

On Sun, Oct 25, 2020 at 8:05 PM Xinyi Yu  wrote:

> Hi all,
>
> We like to post a SPIP of Standardize Exception Messages in Spark. Here is
> the document link:
>
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
> <
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing>
>
>
> This SPIP aims to standardize the exception messages in Spark. It has three
> major focuses:
> 1. Group exception messages in dedicated files for easy maintenance and
> auditing.
> 2. Establish an error message guideline for developers.
> 3. Improve error message quality.
>
> Thanks for your time and patience. Looking forward to your feedback!
>
>
>
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

[DISCUSS][SPIP] Standardize Spark Exception Messages

[Xinyi Yu]

Hi all,

We like to post a SPIP of Standardize Exception Messages in Spark. Here is
the document link: 
https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing

  

This SPIP aims to standardize the exception messages in Spark. It has three
major focuses:
1. Group exception messages in dedicated files for easy maintenance and
auditing.
2. Establish an error message guideline for developers.
3. Improve error message quality.

Thanks for your time and patience. Looking forward to your feedback!



--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org