KevinAppelBofa edited a comment on pull request #34693: URL: https://github.com/apache/spark/pull/34693#issuecomment-984676447
@attilapiros Finding in the query where the WITH piece stops, then where the SELECT begins is where I found the place to split. In the test query ``` python query2 = """ WITH DummyCTE AS ( SELECT 1 as DummyCOL ) SELECT * FROM DummyCTE """ ``` This splits into ``` python withClause = """ WITH DummyCTE AS ( SELECT 1 as DummyCOL ) """ query = """ SELECT * FROM DummyCTE """ ``` In the actual query we are running is more complex and is a bunch of chained WITH together, in that one I did the same approach and where the actual WITH part ends to stick that into the withClause and then the rest into the query This same technique works for the temp table query, to split it up where the part generating the temp table goes into the withClause and the rest goes into the query ``` python query3 = """ (SELECT * INTO #Temp1a FROM (SELECT @@VERSION as version) data ) (SELECT * FROM #Temp1a) """ ``` Turns into ``` python withClause = """ (SELECT * INTO #Temp1a FROM (SELECT @@VERSION as version) data ) """ query = """ (SELECT * FROM #Temp1a) """ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
