No dice. Just hangs. No error, nothing in logs that is very helpful.
postgres_1 | Data page checksums are disabled.
postgres_1 |
postgres_1 | fixing permissions on existing directory
/var/lib/postgresql/data ... ok
postgres_1 | creating subdirectories ... ok
postgres_1 | selecting dynamic shared memory implementation ... posix
postgres_1 | selecting default max_connections ... 100
postgres_1 | selecting default shared_buffers ... 128MB
postgres_1 | selecting default time zone ... Etc/UTC
postgres_1 | creating configuration files ... ok
postgres_1 | running bootstrap script ... ok
postgres_1 | performing post-bootstrap initialization ... ok
postgres_1 | syncing data to disk ... ok
postgres_1 |
postgres_1 |
postgres_1 | Success. You can now start the database server using:
postgres_1 |
postgres_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1 |
postgres_1 | initdb: warning: enabling "trust" authentication for
local connections
postgres_1 | You can change this by editing pg_hba.conf or using the
option -A, or
postgres_1 | --auth-local and --auth-host, the next time you run
initdb.
postgres_1 | waiting for server to start....2021-08-16 16:55:38.478
UTC [47] LOG: starting PostgreSQL 13.4 (Debian 13.4-1.pgdg100+1) on
x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres_1 | 2021-08-16 16:55:38.502 UTC [47] LOG: listening on Unix
socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2021-08-16 16:55:38.542 UTC [48] LOG: database system
was shut down at 2021-08-16 16:55:38 UTC
postgres_1 | 2021-08-16 16:55:38.559 UTC [47] LOG: database system
is ready to accept connections
postgres_1 | done
postgres_1 | server started
postgres_1 | CREATE DATABASE
postgres_1 |
postgres_1 |
postgres_1 | /usr/local/bin/docker-entrypoint.sh: ignoring
/docker-entrypoint-initdb.d/*
postgres_1 |
postgres_1 | 2021-08-16 16:55:38.988 UTC [47] LOG: received fast
shutdown request
postgres_1 | waiting for server to shut down....2021-08-16
16:55:39.001 UTC [47] LOG: aborting any active transactions
postgres_1 | 2021-08-16 16:55:39.002 UTC [47] LOG: background worker
"logical replication launcher" (PID 54) exited with exit code 1
postgres_1 | 2021-08-16 16:55:39.002 UTC [49] LOG: shutting down
postgres_1 | 2021-08-16 16:55:39.078 UTC [47] LOG: database system
is shut down
postgres_1 | done
postgres_1 | server stopped
postgres_1 |
postgres_1 | PostgreSQL init process complete; ready for start up.
postgres_1 |
postgres_1 | 2021-08-16 16:55:39.134 UTC [1] LOG: starting
PostgreSQL 13.4 (Debian 13.4-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by
gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres_1 | 2021-08-16 16:55:39.134 UTC [1] LOG: listening on IPv4
address "0.0.0.0", port 5432
postgres_1 | 2021-08-16 16:55:39.135 UTC [1] LOG: listening on IPv6
address "::", port 5432
postgres_1 | 2021-08-16 16:55:39.165 UTC [1] LOG: listening on Unix
socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2021-08-16 16:55:39.205 UTC [75] LOG: database system
was shut down at 2021-08-16 16:55:39 UTC
postgres_1 | 2021-08-16 16:55:39.221 UTC [1] LOG: database system is
ready to accept connections
postgres_1 | 2021-08-16 16:56:25.094 UTC [1] LOG: received fast
shutdown request
postgres_1 | 2021-08-16 16:56:25.112 UTC [1] LOG: aborting any
active transactions
postgres_1 | 2021-08-16 16:56:25.114 UTC [1] LOG: background worker
"logical replication launcher" (PID 81) exited with exit code 1
postgres_1 | 2021-08-16 16:56:25.114 UTC [76] LOG: shutting down
postgres_1 | 2021-08-16 16:56:25.217 UTC [1] LOG: database system is
shut down
postgres_1 |
postgres_1 | PostgreSQL Database directory appears to contain a
database; Skipping initialization
postgres_1 |
postgres_1 | 2021-08-16 16:56:27.235 UTC [1] LOG: starting
PostgreSQL 13.4 (Debian 13.4-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by
gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres_1 | 2021-08-16 16:56:27.235 UTC [1] LOG: listening on IPv4
address "0.0.0.0", port 5432
postgres_1 | 2021-08-16 16:56:27.235 UTC [1] LOG: listening on IPv6
address "::", port 5432
postgres_1 | 2021-08-16 16:56:27.297 UTC [1] LOG: listening on Unix
socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2021-08-16 16:56:27.340 UTC [26] LOG: database system
was shut down at 2021-08-16 16:56:25 UTC
postgres_1 | 2021-08-16 16:56:27.357 UTC [1] LOG: database system is
ready to accept connections
airflow-init_1 | BACKEND=postgresql+psycopg2
airflow-init_1 | DB_HOST=bi-dw-dev.{redacted_host}.com
airflow-init_1 | DB_PORT=5432
airflow-init_1 | BACKEND=postgresql+psycopg2
airflow-init_1 | DB_HOST=bi-dw-dev.{redacted_host}.com
airflow-init_1 | DB_PORT=5432
On Aug 16, 2021, at 12:52 PM, Anthony Joyce
<[email protected]<mailto:[email protected]>> wrote:
Hi Jed,
When I first had this working, I was able too connect to our local database
without password. I set the IP address the docker-airflow was trying to connect
with (I believe it was a 172.18.*.*/16) in pg_hba.conf as trust. However, I
will try logging in via username and password and see if that fixed the issue.
It’s worth a shot.
Not sure why it worked last week. Weird.
Anthony
On Aug 16, 2021, at 12:47 PM, Jed Cunningham
<[email protected]<mailto:[email protected]>> wrote:
You don't often get email from [email protected]<mailto:[email protected]>.
Learn why this is important<http://aka.ms/LearnAboutSenderIdentification>
Your SQL_ALCHEMY_CONN doesn't have a password? Intentional? Did you redact that
and forget to replace it with `{pass}`?
On Mon, Aug 16, 2021 at 9:26 AM Anthony Joyce
<[email protected]<mailto:[email protected]>> wrote:
Sorry, that was on purpose. I redacted the host name and substituted for {host}.
Sent from my iPhone
On Aug 16, 2021, at 11:23 AM, Jarek Potiuk
<[email protected]<mailto:[email protected]>> wrote:
I am almost certain that you have the "{host}" things wrong. It looks like you
are actually trying to connect to "{host}" dns name - I presume you want to
actually put "real" hostname there or at use it from a variable of sorts, but
it seems that this did not happen.
On Mon, Aug 16, 2021 at 4:33 PM Anthony Joyce
<[email protected]<mailto:[email protected]>> wrote:
Hi All,
Thank you for the help I have been provided on my last email. I am stuck yet
again and do not have many areas to turn to so I figured I would email the
listserv again.
Airflow seems to be stuck initializing using docker-compose up airflow-init.
$ docker-compose up airflow-init
airflow-docker_redis_1 is up-to-date
Recreating airflow-docker_postgres_1 ... done
Recreating airflow-docker_airflow-init_1 ... done
Attaching to airflow-docker_airflow-init_1
airflow-init_1 | BACKEND=postgresql+psycopg2
airflow-init_1 | DB_HOST={host}
airflow-init_1 | DB_PORT=5432
I am using an already existing backend from our current local postgresql
database. Weirdly enough, I had this working last Monday. I made a slight
change to my docker-compose.yaml file (added extra_hosts to the yaml, ran it,
got an error than removed it) and tried to reinitialized the container only for
airflow-init to hang.
I stumbled upon this stack overflow page:
https://stackoverflow.com/questions/68415456/docker-compose-up-airflow-init-hangs-no-network-connection-between-containers
and looked at every step this user did. Everything checks out OK with me as
well.
The container builds using the default settings but does not build anymore with
my custom AIRFLOW__CORE__SQL_ALCHEMY_CONN:
postgresql+psycopg2://airflow:@{host}:5432/airflow. Also, my postgresql db is
configured correctly listening on all IP addresses other than localhost. IP’s
added to pg_hba.conf. IPtables looks good with no conflicts in firewall
services. We are just using one from what I can tell.
I’ve uninstalled completely and reinstalled. No change. I even set daemon.json
to not use IPV6…still no change. I feel as if I’ve tried everything with no
luck.
If anyone has experienced this behavior before and offer some insight, that
would be wonderful. Otherwise, I’ll continue to troubleshoot.
Thanks all.
--
+48 660 796 129
