sarutak opened a new pull request #32715:
URL: https://github.com/apache/spark/pull/32715
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section
is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster
reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class
hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other
DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
This PR proposes to add a feature that logs container output for docker
integration tests.
With this change, if we run test with SBT, we will have like the following
log in `unit-tests.log`.
```
===== CONTAINER LOGS FOR container Id:
3360c98eb28337d8b217fb614e47bf49aafa18a6cb60ecadf3178aee0c663021 =====
21/05/31 20:54:56.433 pool-1-thread-1 INFO PostgresIntegrationSuite: The
files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... UTC
creating configuration files ... ok
running bootstrap script ... ok
sh: locale: not found
2021-05-31 11:54:49.892 UTC [29] WARNING: no usable system locales were
found
performing post-bootstrap initialization ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
waiting for server to start....2021-05-31 11:54:50.284 UTC [34] LOG:
starting PostgreSQL 13.0 on x86_64-pc-linux-musl, compiled by gcc (Alpine
9.3.0) 9.3.0, 64-bit
2021-05-31 11:54:50.287 UTC [34] LOG: listening on Unix socket
"/var/run/postgresql/.s.PGSQL.5432"
2021-05-31 11:54:50.296 UTC [35] LOG: database system was shut down at
2021-05-31 11:54:50 UTC
2021-05-31 11:54:50.301 UTC [34] LOG: database system is ready to accept
connections
done
server started
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
waiting for server to shut down....2021-05-31 11:54:50.363 UTC [34] LOG:
received fast shutdown request
2021-05-31 11:54:50.366 UTC [34] LOG: aborting any active transactions
2021-05-31 11:54:50.368 UTC [34] LOG: background worker "logical
replication launcher" (PID 41) exited with exit code 1
2021-05-31 11:54:50.368 UTC [36] LOG: shutting down
2021-05-31 11:54:50.402 UTC [34] LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
2021-05-31 11:54:50.510 UTC [1] LOG: starting PostgreSQL 13.0 on
x86_64-pc-linux-musl, compiled by gcc (Alpine 9.3.0) 9.3.0, 64-bit
2021-05-31 11:54:50.510 UTC [1] LOG: listening on IPv4 address "0.0.0.0",
port 5432
2021-05-31 11:54:50.510 UTC [1] LOG: listening on IPv6 address "::", port
5432
2021-05-31 11:54:50.517 UTC [1] LOG: listening on Unix socket
"/var/run/postgresql/.s.PGSQL.5432"
2021-05-31 11:54:50.526 UTC [43] LOG: database system was shut down at
2021-05-31 11:54:50 UTC
2021-05-31 11:54:50.531 UTC [1] LOG: database system is ready to accept
connections
2021-05-31 11:54:54.226 UTC [54] ERROR: relation "public.barcopy" does not
exist at character 15
2021-05-31 11:54:54.226 UTC [54] STATEMENT: SELECT 1 FROM public.barcopy
LIMIT 1
2021-05-31 11:54:54.610 UTC [59] ERROR: relation "public.barcopy2" does not
exist at character 15
2021-05-31 11:54:54.610 UTC [59] STATEMENT: SELECT 1 FROM public.barcopy2
LIMIT 1
2021-05-31 11:54:54.934 UTC [63] ERROR: relation "shortfloat" does not
exist at character 15
2021-05-31 11:54:54.934 UTC [63] STATEMENT: SELECT 1 FROM shortfloat LIMIT 1
2021-05-31 11:54:55.675 UTC [75] ERROR: relation "byte_to_smallint_test"
does not exist at character 15
2021-05-31 11:54:55.675 UTC [75] STATEMENT: SELECT 1 FROM
byte_to_smallint_test LIMIT 1
21/05/31 20:54:56.434 pool-1-thread-1 INFO PostgresIntegrationSuite:
===== END OF CONTAINER LOGS FOR container Id:
3360c98eb28337d8b217fb614e47bf49aafa18a6cb60ecadf3178aee0c663021 =====
```
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
If we have container logs, it's useful to debug especially for GA.
### Does this PR introduce _any_ user-facing change?
<!--
Note that it means *any* user-facing change including all aspects such as
the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes
- provide the console output, description and/or an example to show the
behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to
the released Spark versions or within the unreleased branches such as master.
If no, write 'No'.
-->
No.
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some
test cases that check the changes thoroughly including negative and positive
cases if possible.
If it was tested in a way different from regular unit tests, please clarify
how you tested step by step, ideally copy and paste-able, so that other
reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why
it was difficult to add.
-->
I run docker integration tests and got logs. The example shown above is for
`PostgresIntegrationSuite`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
