Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Allison Wang]

Hi Mridul,

Thanks for reviewing the SPIP. As you correctly pointed out, this proposal
is an internal, Spark engine developer-facing improvement to make error
messages more consistent and user-friendly across the Spark codebase, as
opposed to establishing a public contract with end-users. 

Thanks,
Allison



--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Mridul Muralidharan]

I like the idea of consistent messages; it makes understanding errors
easier.
Having said that, Exception messages themselves are not part of the exposed
contract to users; and are subject to change.
We should leave that flexibility open to spark developers ... I am
currently viewing this proposal as a internal standardization exercise
within Spark codebase, and not as a public contract with users.
Is that aligned with the objectives ? Or are we looking at this as a public
contract to users ?
I am not in favour of the latter.

Regards,
Mridul


On Sun, Oct 25, 2020 at 7:05 PM Xinyi Yu  wrote:

> Hi all,
>
> We like to post a SPIP of Standardize Exception Messages in Spark. Here is
> the document link:
>
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
> <
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing>
>
>
> This SPIP aims to standardize the exception messages in Spark. It has three
> major focuses:
> 1. Group exception messages in dedicated files for easy maintenance and
> auditing.
> 2. Establish an error message guideline for developers.
> 3. Improve error message quality.
>
> Thanks for your time and patience. Looking forward to your feedback!
>
>
>
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Xinyi Yu]

Hi Takeshi, Xiao and all devs,

Thank you for your kind feedback! We are going to initiate a vote for this SPIP 
recently.

On Oct 29, 2020, 6:41 PM -0700, Takeshi Yamamuro , wrote:
> I've already left comments about this idea in 
> https://github.com/apache/spark/pull/29497 though,
> it sounds plausible. +1.
>
> > On Mon, Oct 26, 2020 at 9:04 AM Xinyi Yu  wrote:
> > > Hi all,
> > >
> > > We like to post a SPIP of Standardize Exception Messages in Spark. Here is
> > > the document link:
> > > https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
> > > 
> > >
> > > This SPIP aims to standardize the exception messages in Spark. It has 
> > > three
> > > major focuses:
> > > 1. Group exception messages in dedicated files for easy maintenance and
> > > auditing.
> > > 2. Establish an error message guideline for developers.
> > > 3. Improve error message quality.
> > >
> > > Thanks for your time and patience. Looking forward to your feedback!
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
> > >
> > > -
> > > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
> > >
>
>
> --
> ---
> Takeshi Yamamuro

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Takeshi Yamamuro]

I've already left comments about this idea in
https://github.com/apache/spark/pull/29497 though,
it sounds plausible. +1.

On Mon, Oct 26, 2020 at 9:04 AM Xinyi Yu  wrote:

> Hi all,
>
> We like to post a SPIP of Standardize Exception Messages in Spark. Here is
> the document link:
>
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
> <
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing>
>
>
> This SPIP aims to standardize the exception messages in Spark. It has three
> major focuses:
> 1. Group exception messages in dedicated files for easy maintenance and
> auditing.
> 2. Establish an error message guideline for developers.
> 3. Improve error message quality.
>
> Thanks for your time and patience. Looking forward to your feedback!
>
>
>
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

-- 
---
Takeshi Yamamuro

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Xiao Li]

+1

This is a great proposal to improve the usability of Spark. Make Spark
simple to use!

Xiao

Xinyi Yu  于2020年10月27日周二 下午8:25写道：

> Hi Chang,
>
> It is a script that directly analyzes the source code searching for raw
> "throw new” exception. : ) Hope that give an intuitive overview of current
> exceptions in Spark.
> On Oct 27, 2020, 8:21 PM -0700, Chang Chen , wrote:
>
> hi Xinyi
>
> Just curious, which tool did you use to generate this
> 
>
>
> Xinyi Yu  于2020年10月26日周一 上午8:05写道：
>
>> Hi all,
>>
>> We like to post a SPIP of Standardize Exception Messages in Spark. Here is
>> the document link:
>>
>> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
>> <
>> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
>> >
>>
>> This SPIP aims to standardize the exception messages in Spark. It has
>> three
>> major focuses:
>> 1. Group exception messages in dedicated files for easy maintenance and
>> auditing.
>> 2. Establish an error message guideline for developers.
>> 3. Improve error message quality.
>>
>> Thanks for your time and patience. Looking forward to your feedback!
>>
>>
>>
>> --
>> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Xinyi Yu]

Hi Chang,

It is a script that directly analyzes the source code searching for raw "throw 
new” exception. : ) Hope that give an intuitive overview of current exceptions 
in Spark.
On Oct 27, 2020, 8:21 PM -0700, Chang Chen , wrote:
> hi Xinyi
>
> Just curious, which tool did you use to generate this
>
> > Xinyi Yu  于2020年10月26日周一 上午8:05写道：
> > > Hi all,
> > >
> > > We like to post a SPIP of Standardize Exception Messages in Spark. Here is
> > > the document link:
> > > https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
> > > 
> > >
> > > This SPIP aims to standardize the exception messages in Spark. It has 
> > > three
> > > major focuses:
> > > 1. Group exception messages in dedicated files for easy maintenance and
> > > auditing.
> > > 2. Establish an error message guideline for developers.
> > > 3. Improve error message quality.
> > >
> > > Thanks for your time and patience. Looking forward to your feedback!
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
> > >
> > > -
> > > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
> > >

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Chang Chen]

hi Xinyi

Just curious, which tool did you use to generate this



Xinyi Yu  于2020年10月26日周一 上午8:05写道：

> Hi all,
>
> We like to post a SPIP of Standardize Exception Messages in Spark. Here is
> the document link:
>
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
> <
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing>
>
>
> This SPIP aims to standardize the exception messages in Spark. It has three
> major focuses:
> 1. Group exception messages in dedicated files for easy maintenance and
> auditing.
> 2. Establish an error message guideline for developers.
> 3. Improve error message quality.
>
> Thanks for your time and patience. Looking forward to your feedback!
>
>
>
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Hyukjin Kwon]

Thanks for pointing this out, Nicholas.
This SPIP seems focused on the Scala side, grouping the exception handling
and providing some guidance about error messages.
Yes, I think we can refer to it on the PySpark side. Probably I will follow
up and file some JIRAs based on how this SPIP gose, and run it separately.
BTW, it’s slightly different from SPARK-32194
 because this JIRA
targets to have a set of proper classes instead of, for example, just plain
Exception.


2020년 10월 26일 (월) 오전 10:36, Nicholas Chammas 님이
작성:

> Just want to call out that this SPIP should probably account somehow for
> PySpark and the work being done in SPARK-32082
>  to improve PySpark
> exceptions.
>
> On Sun, Oct 25, 2020 at 8:05 PM Xinyi Yu  wrote:
>
>> Hi all,
>>
>> We like to post a SPIP of Standardize Exception Messages in Spark. Here is
>> the document link:
>>
>> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
>> <
>> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing>
>>
>>
>> This SPIP aims to standardize the exception messages in Spark. It has
>> three
>> major focuses:
>> 1. Group exception messages in dedicated files for easy maintenance and
>> auditing.
>> 2. Establish an error message guideline for developers.
>> 3. Improve error message quality.
>>
>> Thanks for your time and patience. Looking forward to your feedback!
>>
>>
>>
>> --
>> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

[Nicholas Chammas]

Just want to call out that this SPIP should probably account somehow for
PySpark and the work being done in SPARK-32082
 to improve PySpark
exceptions.

On Sun, Oct 25, 2020 at 8:05 PM Xinyi Yu  wrote:

> Hi all,
>
> We like to post a SPIP of Standardize Exception Messages in Spark. Here is
> the document link:
>
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing
> <
> https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing>
>
>
> This SPIP aims to standardize the exception messages in Spark. It has three
> major focuses:
> 1. Group exception messages in dedicated files for easy maintenance and
> auditing.
> 2. Establish an error message guideline for developers.
> 3. Improve error message quality.
>
> Thanks for your time and patience. Looking forward to your feedback!
>
>
>
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>