kl0u commented on a change in pull request #6957: [FLINK-10627][E2E tests] Test
s3 output for streaming file sink
URL: https://github.com/apache/flink/pull/6957#discussion_r230065738
##########
File path: flink-end-to-end-tests/test-scripts/test_streaming_file_sink.sh
##########
@@ -47,10 +57,32 @@ function wait_for_restart {
# Arguments:
# None
# Returns:
-# None
+# sorted content of part files
###################################
function get_complete_result {
- find "${OUTPUT_PATH}" -type f \( -iname "part-*" \) -exec cat {} + | sort
-g
+ if [ "${OUT_TYPE}" == "s3" ]; then
+ rm -rf $OUTPUT_PATH; mkdir -p $OUTPUT_PATH
+ s3_get_by_prefix ${TEST_DATA_DIR} "${OUT}" "part-"
+ fi
+ find "${OUTPUT_PATH}" -type f \( -iname "part-*" \) -exec cat {} + | sort -g
+}
+
+###################################
+# Get total number of lines in part files.
+#
+# Globals:
+# OUT
+# Arguments:
+# None
+# Returns:
+# line number in part files
+###################################
+function get_number_of_lines {
Review comment:
I would rename it to `get_total_number_of_valid_lines`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
