On Thursday, 22 June 2017 18:16:22 UTC+4, Evan Jones wrote: > > The Cloud SQL Proxy logs suggest to me that it may not be using the right > credentials? It is possible that it is trying to use the cluster's "default > service account"? > > It seems to have been a permissions issue. It started working when I gave the cloud-sql service account "Project Owner" rights. The problem with this is that it's too permissive though.
Another question though: Do I really have to use cloudsql proxy to access it from GKE? It seems that whitelisting the IP addresses of the GKE cluster works just as well - In which case I wonder why the documentation recommends the overcomplicated route of setting up cloud sql proxy. > If you use "kubectl exec ... -ti /bin/sh" you should be able to examine > the contents of the credentials file that is being passed to ensure that > the contents match what you expect. You can also try to interactively start > /cloud_sql_proxy with different flags to see if you can get it to start and > print the "expected" log message. > > Good luck I hope that helps! > > > > On Thursday, June 22, 2017 at 9:46:52 AM UTC-4, Traiano Welcome wrote: >> >> Hi Evan >> >> >> On Thu, Jun 22, 2017 at 5:34 PM, Evan Jones <evan....@triggermail.io> >> wrote: >> >>> I know nothing about wordpress, but for what it is worth, we are using >>> this Cloud SQL Proxy container with success. A few notes about the config >>> you posted: >>> >>> >>> * I'm assuming that where you have >>> "-instances=[INSTANCE_CONNECTION_NAME]=tcp:[PORT]" you've replaced this >>> with your Cloud SQL instance name (project:region:cloud_sql_name), and port >>> with 3306, right? >>> >> >> >> I have this: >> >> command: ["/cloud_sql_proxy", "--dir=/cloudsql", >> >> "-instances=lol-staging:europe-west2:lol-staging-001=tcp:3306", >> "-credential_file=/secrets/cloudsql/credentials.json"] >> >> I've redeployed and can see logs now, the wordpress container is failing >> because it's failing connection to SQL via the proxy: >> >> >> MySQL Connection Error: (2006) MySQL server has gone away >> Warning: mysqli::mysqli(): MySQL server has gone away in - on line 10 >> Warning: mysqli::mysqli(): Error while reading greeting packet. >> PID=167 in - on line 10 >> Warning: mysqli::mysqli(): (HY000/2006): MySQL server has gone away >> in - on line 10 >> >> The cloud sql proxy container complains about an account not having >> correct permissions (however I have given it editor permissions): >> >> kubectl logs pods/wordpress-2608214628-9bc9b cloudsql-proxy >> >> >> 2017/06/22 13:40:15 Throttling >> refreshCfg(lol-staging:europe-west2:lol-staging-001): it was only called >> 24.005689746s ago >> 2017/06/22 13:40:15 couldn't connect to >> "lol-staging:europe-west2:lol-staging-001": ensure that the account has >> access to "lol-staging:europe-west2:lol-staging-001" (and make sure there's >> no typo in that name). Error during createEphemeral for >> lol-staging:europe-west2:lol-staging-001: googleapi: Error 403: The client >> is not authorized to make this request., notAuthorized >> 2017/06/22 13:40:18 New connection for >> "lol-staging:europe-west2:lol-staging-001" >> >> >> So I'm now wondering how to better validate which account is being >> referred to here and whether the permissions are correct. >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>> >>> * If you run kubectl logs pod cloudsql-proxy you should see something >>> like the following: >>> >>> 2017/06/21 20:45:06 Listening on (whatever) for (your instance name) >>> 2017/06/21 20:45:06 Ready for new connections >>> >>> * If the cloudsql-proxy container is running, you can run kubectl exec >>> (pod) -c cloudsql-proxy -ti /bin/sh to get a shell and try to verify the >>> cloudsql-proxy configuration, or to start a new instance and make sure it >>> is configured correctly. >>> >>> * We are using this with local Unix socket connections in the /cloudsql >>> directory, but localhost sockets should also work. >>> >>> >>> I'm surprised you don't see any logs from your wordpress container, but >>> I can't help with that part. >>> >>> Good luck! >>> >>> >>> On Thursday, June 22, 2017 at 3:15:26 AM UTC-4, Traiano Welcome wrote: >>>> >>>> Hi Ahmet >>>> >>>> On Thursday, 22 June 2017 02:25:05 UTC+4, Ahmet Alp Balkan wrote: >>>>> >>>>> Can you run "kubectl logs -l app=wordpress"? I am assuming there will >>>>> be some logs from the crashing mysql container. >>>>> >>>>> >>>> Thanks for your response. I get no output from running that command (I >>>> suppose no logs are being generated). I tried both the command, and >>>> running >>>> it in a loop, just in case: >>>> >>>> for i in `seq 1 100`;do echo "checking $i " - $(kubectl logs -l >>>> app=web );done >>>> >>>> No result. >>>> >>>> Just to be clear, wordpress container is in a pod, and the cloud sql >>>> container is a sidecar container connected to pod. >>>> >>>> Google's documentation seems pretty thin on getting this working >>>> properly with GKE containers, and non-existent on troubleshooting >>>> connectivity issues with cloud sql proxy. >>>> >>>> Do you know of an alternative method of getting a container access to >>>> Cloud SQL from GKE ? >>>> >>>> >>>> >>>> On Wed, Jun 21, 2017 at 7:58 AM, Traiano Welcome <tra...@gmail.com> >>>>> wrote: >>>>> >>>>>> I'm following this example of how to get wordpress running on GKE >>>>>> connected to Google Cloud SQL via the Google Cloud SQL Proxy: >>>>>> >>>>>> >>>>>> https://cloud.google.com/sql/docs/mysql/connect-container-engine >>>>>> >>>>>> >>>>>> <https://cloud.google.com/sql/docs/mysql/connect-container-engine> >>>>>> >>>>>> Unfortunately, my wordpress pod is failing with a crashloop error, >>>>>> and it's not clear from the documentation how to dig deeper into the >>>>>> reason >>>>>> for this. Here is a sample of the error: >>>>>> >>>>>> >>>>>> bash-3.2$ kubectl get pods| egrep wordpress >>>>>> wordpress-713960421-v4f49 0/2 CrashLoopBackOff 16 20m >>>>>> >>>>>> (kubectl describe pod ...) >>>>>> >>>>>> 11m 22s 36 kubelet, gke-noon-staging-default- >>>>>> pool-d500b601-dfb6 Warning FailedSync >>>>>> Error syncing pod, skipping: [failed to "StartContainer" for >>>>>> "web" with CrashLoopBackOff: "Back-off 5m0s restarting failed >>>>>> container=web >>>>>> pod=wordpress-713960421-v4f49_default(f64276d2-5660-11e7-a565-42010a9a0023)" >>>>>> , failed to "StartContainer" for "cloudsql-proxy" with >>>>>> CrashLoopBackOff: "Back-off 5m0s restarting failed >>>>>> container=cloudsql-proxy >>>>>> pod=wordpress-713960421-v4f49_default(f64276d2-5660-11e7- >>>>>> a565-42010a9a0023)" >>>>>> ] >>>>>> >>>>>> My questions are: >>>>>> >>>>>> - How to dig further into why the pod is failing to deploy with >>>>>> cloud sql proxy >>>>>> - Has anyone got an example working for using cloud sql proxy >>>>>> with as simple mysql client pod? >>>>>> >>>>>> Here is a description of the deployment (kubectl describe wordpress): >>>>>> >>>>>> >>>>>> https://pastebin.com/DjYM97R7 <https://pastebin.com/DjYM97R7> >>>>>> >>>>>> >>>>>> <https://pastebin.com/DjYM97R7> >>>>>> >>>>>> And a description of the pod (kubectl describe ): >>>>>> >>>>>> >>>>>> <https://pastebin.com/pN7gUZg8> >>>>>> >>>>>> https://pastebin.com/pN7gUZg8 <https://pastebin.com/pN7gUZg8> >>>>>> >>>>>> >>>>>> >>>>>> I deployed the cloud sql proxy and wordpress container separately and >>>>>> found the cloud sql proxy runs fine, but the wordpress blog container >>>>>> fails >>>>>> to startup on kubernetes. >>>>>> Error syncing pod, skipping: >>>>>> >>>>>> failed to "StartContainer" for "wordpress" with CrashLoopBackOff >>>>>> >>>>>> <https://stackoverflow.com/questions/44672411/simple-client-connecting-to-google-cloud-sql-using-cloud-sql-proxy#comment76333823_44672411> >>>>>> >>>>>> Looking at the pod logs for wordpress, it seems wordpress is dying >>>>>> because it cannot connect to the MySQL DB: >>>>>> >>>>>> MySQL Connection Error: (2002) Connection refused – Traiano Welcome >>>>>> <https://stackoverflow.com/users/6052496/traiano-welcome> 3 hours ago >>>>>> <https://stackoverflow.com/questions/44672411/simple-client-connecting-to-google-cloud-sql-using-cloud-sql-proxy#comment76334135_44672411> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Kubernetes user discussion and Q&A" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to kubernetes-use...@googlegroups.com. >>>>>> To post to this group, send email to kubernet...@googlegroups.com. >>>>>> Visit this group at https://groups.google.com/group/kubernetes-users. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "Kubernetes user discussion and Q&A" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/kubernetes-users/2kqHNT1Nbtc/unsubscribe >>> . >>> To unsubscribe from this group and all its topics, send an email to >>> kubernetes-use...@googlegroups.com. >>> To post to this group, send email to kubernet...@googlegroups.com. >>> Visit this group at https://groups.google.com/group/kubernetes-users. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group. To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscr...@googlegroups.com. To post to this group, send email to kubernetes-users@googlegroups.com. Visit this group at https://groups.google.com/group/kubernetes-users. For more options, visit https://groups.google.com/d/optout.
