vibhatha commented on code in PR #15288:
URL: https://github.com/apache/arrow/pull/15288#discussion_r1070202383
##########
cpp/src/arrow/compute/exec/plan_test.cc:
##########
@@ -322,26 +306,16 @@ void TestSourceSink(
std::string source_factory_name,
std::function<Result<std::vector<ElementType>>(const BatchesWithSchema&)>
to_elements) {
- ASSERT_OK_AND_ASSIGN(auto executor, arrow::internal::ThreadPool::Make(1));
- ExecContext exec_context(default_memory_pool(), executor.get());
- ASSERT_OK_AND_ASSIGN(auto plan, ExecPlan::Make(exec_context));
- AsyncGenerator<std::optional<ExecBatch>> sink_gen;
-
auto exp_batches = MakeBasicBatches();
ASSERT_OK_AND_ASSIGN(auto elements, to_elements(exp_batches));
auto element_it_maker = [&elements]() {
return MakeVectorIterator<ElementType>(elements);
};
-
- ASSERT_OK(Declaration::Sequence({
- {source_factory_name,
- OptionsType{exp_batches.schema,
element_it_maker}},
- {"sink", SinkNodeOptions{&sink_gen}},
- })
- .AddToPlan(plan.get()));
-
- ASSERT_THAT(StartAndCollect(plan.get(), sink_gen),
-
Finishes(ResultWith(UnorderedElementsAreArray(exp_batches.batches))));
+ Declaration plan(source_factory_name,
+ OptionsType{exp_batches.schema, element_it_maker});
+ ASSERT_OK_AND_ASSIGN(auto result,
+ DeclarationToExecBatches(std::move(plan),
/*use_threads=*/false));
Review Comment:
Just for the curiosity I repeated this on the `use_threads=false` setting
(which is the same state this was run before making this change, I guess).
The following was my observation
```bash
/home/asus/github/fork/arrow/cpp/src/arrow/compute/exec/plan_test.cc:887:
Failure
Value of: sink_gen()
Expected: value value value is equal to ExecBatch
# Rows: 4
0: Array[-2147483380,808065407,-681488731,829937304]
1: Array[true,false,false,false]
Actual: 16-byte object <F0-5B 46-2E D1-55 00-00 90-75 47-2E D1-55 00-00>,
whose value (ExecBatch
# Rows: 4
0: Array[-2147483418,113924762,1252144620,-193012790]
1: Array[false,true,true,true]
) doesn't match, whose value ExecBatch
# Rows: 4
0: Array[-2147483418,113924762,1252144620,-193012790]
1: Array[false,true,true,true]
doesn't match
Google Test trace:
/home/asus/github/fork/arrow/cpp/src/arrow/compute/exec/plan_test.cc:866:
single threaded
/home/asus/github/fork/arrow/cpp/src/arrow/compute/exec/plan_test.cc:863:
unslowed
-----------------------------------------------------------------------------------------------------
/home/asus/github/fork/arrow/cpp/src/arrow/compute/exec/plan_test.cc:887:
Failure
Value of: sink_gen()
Expected: value value value is equal to ExecBatch
# Rows: 4
0: Array[-2147483380,808065407,-681488731,829937304]
1: Array[true,false,false,false]
Actual: 16-byte object <00-66 F7-B1 AD-55 00-00 40-DE F7-B1 AD-55 00-00>,
whose value (ExecBatch
# Rows: 4
0: Array[-2147483522,-417100609,768701078,-1848250513]
1: Array[true,false,true,true]
) doesn't match, whose value ExecBatch
# Rows: 4
0: Array[-2147483522,-417100609,768701078,-1848250513]
1: Array[true,false,true,true]
doesn't match
Google Test trace:
/home/asus/github/fork/arrow/cpp/src/arrow/compute/exec/plan_test.cc:866:
parallel
/home/asus/github/fork/arrow/cpp/src/arrow/compute/exec/plan_test.cc:863:
unslowed
-----------------------------------------------------------------------------------------------------
/home/asus/github/fork/arrow/cpp/src/arrow/compute/exec/plan_test.cc:887:
Failure
Value of: sink_gen()
Expected: value value value is equal to ExecBatch
# Rows: 4
0: Array[-2147483380,808065407,-681488731,829937304]
1: Array[true,false,false,false]
Actual: 16-byte object <B0-75 1A-B3 48-56 00-00 30-06 1D-B3 48-56 00-00>,
whose value (ExecBatch
# Rows: 4
0: Array[-2147483522,-417100609,768701078,-1848250513]
1: Array[true,false,true,true]
) doesn't match, whose value ExecBatch
# Rows: 4
0: Array[-2147483522,-417100609,768701078,-1848250513]
1: Array[true,false,true,true]
doesn't match
Google Test trace:
/home/asus/github/fork/arrow/cpp/src/arrow/compute/exec/plan_test.cc:866:
parallel
/home/asus/github/fork/arrow/cpp/src/arrow/compute/exec/plan_test.cc:863:
unslowed
```
For 160 repetitions (bash on 8 runs), I observed 3 failures. Could say 3/8
fails. Do you have a clue why this could be happening? The failing test is
`ExecPlanExecution.StressSourceSinkStopped`. A single run includes all test
cases.
But when run just this case over 100 repetitions, it failed only 2 times. I
suspect a resource allocation issue. Appreciate your thoughts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
