mapleFU commented on PR #38885:
URL: https://github.com/apache/arrow/pull/38885#issuecomment-1855047632
```
/Users/fuxuwei/workspace/CMakeLibs/arrow/cpp/src/arrow/dataset/dataset_writer_test.cc:115:
Failure
Value of: _fut.Wait(::arrow::kDefaultAssertFinishesWaitSeconds)
Actual: false
Expected: true
/Users/fuxuwei/workspace/CMakeLibs/arrow/cpp/src/arrow/result.cc:28:
ValueOrDie called on an error: Invalid: File is too small: 0
/Users/fuxuwei/workspace/CMakeLibs/arrow/cpp/src/arrow/dataset/dataset_writer_test.cc:154:
Failure
Expected: (found) != (actual_paths.end()), actual: 8-byte object <E8-9B
00-53 01-00 00-00> vs 8-byte object <E8-9B 00-53 01-00 00-00>
The file testdir/chunk-63.arrow was not in the list of files visited
Google Test trace:
/Users/fuxuwei/workspace/CMakeLibs/arrow/cpp/src/arrow/dataset/dataset_writer_test.cc:183:
pre_finish
/Users/fuxuwei/workspace/CMakeLibs/arrow/cpp/src/arrow/dataset/dataset_writer_test.cc:154:
Failure
Expected: (found) != (actual_paths.end()), actual: 8-byte object <E8-8B
00-54 01-00 00-00> vs 8-byte object <E8-8B 00-54 01-00 00-00>
The file testdir/chunk-63.arrow was not in the list of files visited
Google Test trace:
/Users/fuxuwei/workspace/CMakeLibs/arrow/cpp/src/arrow/dataset/dataset_writer_test.cc:187:
post_finish
/Users/fuxuwei/workspace/CMakeLibs/arrow/cpp/src/arrow/dataset/dataset_writer_test.cc:162:
Failure
Value of: _st.ok()
Actual: false
Expected: true
'_error_or_value27.status()' failed with Invalid: File is too small: 0
0 libarrow.1500.0.0.dylib 0x00000001082d1ecc
_ZN5arrow4util7CerrLog14PrintBackTraceEv + 44
1 libarrow.1500.0.0.dylib 0x00000001082d1e80
_ZN5arrow4util7CerrLogD2Ev + 184
2 libarrow.1500.0.0.dylib 0x00000001082d1dc0
_ZN5arrow4util7CerrLogD0Ev + 12
3 libarrow.1500.0.0.dylib 0x00000001082d1d5c
_ZN5arrow4util8ArrowLogD1Ev + 48
4 libarrow.1500.0.0.dylib 0x00000001081a7a30
_ZN5arrow8internal17InvalidValueOrDieERKNS_6StatusE + 240
5 arrow-dataset-dataset-writer-test 0x0000000104560ca0
_ZN5arrow7dataset8internal24DatasetWriterTestFixture11ReadAsBatchENSt3__117basic_string_viewIcNS3_11char_traitsIcEEEEPi
+ 4952
6 arrow-dataset-dataset-writer-test 0x0000000104550390
_ZN5arrow7dataset8internal24DatasetWriterTestFixture17AssertCreatedDataERKNSt3__16vectorINS2_12ExpectedFileENS3_9allocatorIS5_EEEEb
+ 192
7 arrow-dataset-dataset-writer-test 0x0000000104551d14
_ZN5arrow7dataset8internal56DatasetWriterTestFixture_MaxRowsOneWriteBackpresure_Test8TestBodyEv
+ 652
8 libarrow_gtest.1.11.0.dylib 0x00000001049d1bdc
_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS_4TestEvEET0_PT_MS4_FS3_vEPKc
+ 92
9 libarrow_gtest.1.11.0.dylib 0x00000001049d1b30
_ZN7testing4Test3RunEv + 444
10 libarrow_gtest.1.11.0.dylib 0x00000001049d329c
_ZN7testing8TestInfo3RunEv + 492
11 libarrow_gtest.1.11.0.dylib 0x00000001049d3e58
_ZN7testing9TestSuite3RunEv + 352
12 libarrow_gtest.1.11.0.dylib 0x00000001049e3c3c
_ZN7testing8internal12UnitTestImpl11RunAllTestsEv + 2120
13 libarrow_gtest.1.11.0.dylib 0x00000001049e32d0
_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS0_12UnitTestImplEbEET0_PT_MS4_FS3_vEPKc
+ 92
14 libarrow_gtest.1.11.0.dylib 0x00000001049e3240
_ZN7testing8UnitTest3RunEv + 124
15 libarrow_gtest_main.1.11.0.dylib 0x0000000104627e9c main + 68
16 dyld 0x00000001046f50f4 start + 520
Process finished with exit code 134 (interrupted by signal 6: SIGABRT)
```
Using diff:
```
// May want to prototype / measure someday pushing the async write down
further
return DeferNotOk(options_.filesystem->io_context().executor()->Submit(
[self = this, batch = std::move(next)]() {
+ // Sleep 1 s
+ std::this_thread::sleep_for(std::chrono::seconds(1));
int64_t rows_to_release = batch->num_rows();
Status status = self->writer_->Write(batch);
self->writer_state_->rows_in_flight_throttle.Release(rows_to_release);
```
This can produce similiar, but when enlarge test time-out, the problem
disappear. I think that maybe just I'm writing too many files, making schedule
writing so slow?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
