[GitHub] spark pull request #19535: [WIP][PYTHON] Mark/print deprecation warnings as ...

Thu, 19 Oct 2017 02:07:11 -0700 

GitHub user HyukjinKwon opened a pull request:

    https://github.com/apache/spark/pull/19535

    [WIP][PYTHON] Mark/print deprecation warnings as DeprecationWarning for 
deprecated APIs

    ## What changes were proposed in this pull request?
    
    This PR proposes to mark the existing warnings as `DeprecationWarning` and 
print out warnings for deprecated functions.
    
    This could be actually useful for Spark app developers. I use PyCharm and 
this IDE can detect this specific `DeprecationWarning`:
    
    **Before**
    
    <img 
src="https://user-images.githubusercontent.com/6477701/31762664-df68d9f8-b4f6-11e7-8773-f0468f70a2cc.png";
 height="45" />
    
    **After**
    
    <img 
src="https://user-images.githubusercontent.com/6477701/31762662-de4d6868-b4f6-11e7-98dc-3c8446a0c28a.png";
 height="70" />
    
    For console usage, `DeprecationWarning` is usually disabled (see 
https://docs.python.org/2/library/warnings.html#warning-categories and 
https://docs.python.org/3/library/warnings.html#warning-categories):
    
    ```
    >>> import warnings
    >>> filter(lambda f: f[2] == DeprecationWarning, warnings.filters)
    [('ignore', <_sre.SRE_Pattern object at 0x10ba58c00>, <type 
'exceptions.DeprecationWarning'>, <_sre.SRE_Pattern object at 0x10bb04138>, 0), 
('ignore', None, <type 'exceptions.DeprecationWarning'>, None, 0)]
    ```
    
    so, it won't actually mess up the terminal much unless it is intended.
    
    If this is intendedly enabled, it'd should as below:
    
    ```
    >>> import warnings
    >>> warnings.simplefilter('always', DeprecationWarning)
    >>>
    >>> from pyspark.sql import functions
    >>> functions.approxCountDistinct("a")
    .../spark/python/pyspark/sql/functions.py:232: DeprecationWarning: 
Deprecated in 2.1, use approx_count_distinct instead.
      "Deprecated in 2.1, use approx_count_distinct instead.", 
DeprecationWarning)
    ...
    ```
    
    These instances were found by:
    
    ```
    cd python/pyspark
    grep -r "Deprecated" .
    grep -r "deprecated" .
    grep -r "deprecate" .
    ```
    
    ## How was this patch tested?
    
    Manually tested.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/HyukjinKwon/spark deprecated-warning

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/19535.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #19535
    
----
commit 376162a5976877faf87446ac9a2774d01b44e6e3
Author: hyukjinkwon <[email protected]>
Date:   2017-10-19T08:10:03Z

    Mark/print deprecation warnings as DeprecationWarningfor deprecated APIs

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to