peter-toth edited a comment on issue #23531: [SPARK-24497][SQL] Support recursive SQL query URL: https://github.com/apache/spark/pull/23531#issuecomment-494917020 I squashed previous commits and rebased it on master. Also modified the implementation and took @maryannxue's suggestion into account: > Since Spark does not offer a stateless physical relation, we need all this hack to reset the state. I'm thinking can we go back to the first stateless construct here, i.e., the logical relation and restart each iteration from there? So instead of clearing up the state of each stateful physical nodes, we can create a clean copy for every iteration. Some missing features compared to PostgreSQL: https://github.com/apache/spark/pull/23531#discussion_r249968739. You can find ~90 tests in `recursion.sql`, most of them are ported from https://github.com/postgres/postgres/blob/master/src/test/regress/sql/with.sql but let me know if you miss something. @maryannxue @mgaido91 @gatorsmile @viirya @cloud-fan @dongjoon-hyun any feedback is very welcome.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
