MaxGekk commented on issue #23471: [SPARK-26550][SQL] New built-in datasource - noop URL: https://github.com/apache/spark/pull/23471#issuecomment-451880098 > Does it work for partitioned data? Could you explain your concerns regarding partitioning, please. Why should it be matter here. Should I test all other sources and bucketing too, following to the logic? > This is because this PR has only a test with range(). The test in the PR checks main goal of the datasources - materialization of all rows. `range()` is not so matter in the test. > It it works, can we have a test for partitioned data? I can write such test but what will it proof taking into account so many variants of building `Dataset`.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
