GitHub user drewrobb opened a pull request:

    [SPARK-17986] [ML] SQLTransformer should remove temporary tables

    ## What changes were proposed in this pull request?
    A call to the method `SQLTransformer.transform` previously would create a 
temporary table and never delete it. This change adds a call to 
`dropTempView()` that deletes this temporary table before returning the result 
so that the table will not remain in spark's table catalog. Because `tableName` 
is randomized and not exposed, there should be no expected use of this table 
outside of the `transform` method.
    ## How was this patch tested?
    A single new assertion was added to the existing test of the 
`SQLTransformer.transform` method that all temporary tables are removed. 
Without the corresponding code change, this new assertion fails. I am not aware 
of any circumstances in which removing this temporary view would be bad for 
performance or correctness in other ways, but some expertise here would be 
    Please review before 
opening a pull request.

You can merge this pull request into a Git repository by running:

    $ git pull SPARK-17986

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #15526
commit d5c3b419942f1d3b9af265b540a9404d3e8295df
Author: Drew Robb <>
Date:   2016-10-18T03:32:55Z

    SQLTransformer should remove temporary tables


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to