paleolimbot commented on code in PR #13706:
URL: https://github.com/apache/arrow/pull/13706#discussion_r968572991
##########
r/tests/testthat/test-query-engine.R:
##########
@@ -17,6 +17,76 @@
library(dplyr, warn.conflicts = FALSE)
+test_that("ExecPlanReader does not start evaluating a query", {
+ rbr <- as_record_batch_reader(
+ function(x) stop("This query will error if started"),
+ schema = schema(a = int32())
+ )
+
+ reader <- as_record_batch_reader(as_adq(rbr))
+ expect_identical(reader$PlanStatus(), "PLAN_NOT_STARTED")
+ expect_error(reader$read_table(), "This query will error if started")
+ expect_identical(reader$PlanStatus(), "PLAN_FINISHED")
+})
+
+test_that("ExecPlanReader evaluates nested exec plans lazily", {
+ reader <- as_record_batch_reader(as_adq(arrow_table(a = 1:10)))
+ expect_identical(reader$PlanStatus(), "PLAN_NOT_STARTED")
+
+ head_reader <- head(reader, 4)
+ expect_identical(reader$PlanStatus(), "PLAN_NOT_STARTED")
+
+ expect_equal(
+ head_reader$read_table(),
+ arrow_table(a = 1:4)
+ )
+
+ expect_identical(reader$PlanStatus(), "PLAN_FINISHED")
+})
+
+test_that("ExecPlanReader evaluates head() lazily", {
+ reader <- as_record_batch_reader(as_adq(arrow_table(a = 1:10)))
+ expect_identical(reader$PlanStatus(), "PLAN_NOT_STARTED")
+
+ head_reader <- head(reader, 4)
+ expect_identical(reader$PlanStatus(), "PLAN_NOT_STARTED")
+
+ expect_equal(
+ head_reader$read_table(),
+ arrow_table(a = 1:4)
+ )
+
+ expect_identical(reader$PlanStatus(), "PLAN_FINISHED")
+})
+
+test_that("ExecPlanReader evaluates head() lazily", {
+ # make a 500-row RecordBatchReader
+ reader <- RecordBatchReader$create(
+ batches = rep(
+ list(
+ record_batch(
+ line = c(
+ "this is the RecordBatchReader that never ends",
Review Comment:
Initially it was (you can create a RecordBatchReader from an arbitrary R
function that returns record batches or NULL), but for some reason on CI the
infinite iterator crashed on CI (but not locally on Mac or Ubuntu, where the
`head()` call properly cancels the input). This PR is trying to make it more
clear when `StopProducing()` actually gets called, even if the ExecPlan is
currently rather bad about responding to that request.
I should take this joke out, though, since it's now misleading.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
