Hey Matt, any update on this? -- Mateus Caruccio / Master of Puppets GetupCloud.com We make the infrastructure invisible Gartner Cool Vendor 2017
2017-09-28 10:19 GMT-03:00 Matthew Wringe <mwri...@redhat.com>: > Wait, there is another update that we need. That PR probably wont work > properly for you yet. I am investigating > > On Thu, Sep 28, 2017 at 9:06 AM, Matthew Wringe <mwri...@redhat.com> > wrote: > >> The PR is this: https://github.com/openshift/origin-metrics/pull/382 >> >> It was a problem in one of our releases of Hawkular Metrics, but I didn't >> think it made it into the 3.6 release (but it did). >> >> On Thu, Sep 28, 2017 at 8:41 AM, Mateus Caruccio < >> mateus.caruc...@getupcloud.com> wrote: >> >>> Sweet! Would you mind pointing the PR url? >>> Thanks. >>> >>> -- >>> Mateus Caruccio / Master of Puppets >>> GetupCloud.com >>> We make the infrastructure invisible >>> Gartner Cool Vendor 2017 >>> >>> 2017-09-28 9:34 GMT-03:00 Matthew Wringe <mwri...@redhat.com>: >>> >>>> Ah, sorry, this somehow got missed. We have had an issue that slipped >>>> into 3.6.0 that we are currently in progress to fix. The PR has been >>>> submitted and we are waiting for a new image to be built and pushed out. >>>> >>>> On Thu, Sep 28, 2017 at 6:53 AM, Mateus Caruccio < >>>> mateus.caruc...@getupcloud.com> wrote: >>>> >>>>> Nope, no time to debug yet :( >>>>> >>>>> -- >>>>> Mateus Caruccio / Master of Puppets >>>>> GetupCloud.com >>>>> We make the infrastructure invisible >>>>> Gartner Cool Vendor 2017 >>>>> >>>>> 2017-09-28 7:52 GMT-03:00 Andrew Lau <and...@andrewklau.com>: >>>>> >>>>>> Did you find any solution for this? >>>>>> >>>>>> On Fri, 15 Sep 2017 at 01:34 Mateus Caruccio < >>>>>> mateus.caruc...@getupcloud.com> wrote: >>>>>> >>>>>>> Yep, there it is: >>>>>>> >>>>>>> [OSEv3:children] >>>>>>> masters >>>>>>> etcd >>>>>>> nodes >>>>>>> >>>>>>> [OSEv3:vars] >>>>>>> deployment_type=origin >>>>>>> openshift_release=v3.6 >>>>>>> debug_level=1 >>>>>>> openshift_debug_level=1 >>>>>>> openshift_node_debug_level=1 >>>>>>> openshift_master_debug_level=1 >>>>>>> openshift_master_access_token_max_seconds=2419200 >>>>>>> osm_cluster_network_cidr=172.16.0.0/16 >>>>>>> openshift_registry_selector="docker-registry=true" >>>>>>> openshift_hosted_registry_replicas=1 >>>>>>> >>>>>>> openshift_master_cluster_hostname=api-cluster.example.com.br >>>>>>> openshift_master_cluster_public_hostname=api-cluster.example.com.br >>>>>>> osm_default_subdomain=example.com.br >>>>>>> openshift_master_default_subdomain=example.com.br >>>>>>> osm_default_node_selector="role=app" >>>>>>> os_sdn_network_plugin_name=redhat/openshift-ovs-multitenant >>>>>>> openshift_master_identity_providers=[{'name': 'htpasswd_auth', >>>>>>> 'login': 'true', 'challenge': 'true', 'kind': >>>>>>> 'HTPasswdPasswordIdentityProvider', >>>>>>> 'filename': '/etc/origin/master/htpasswd'}] >>>>>>> osm_use_cockpit=false >>>>>>> containerized=False >>>>>>> >>>>>>> openshift_master_cluster_method=native >>>>>>> openshift_master_console_port=443 >>>>>>> openshift_master_api_port=443 >>>>>>> >>>>>>> openshift_master_overwrite_named_certificates=true >>>>>>> openshift_master_named_certificates=[{"certfile":"{{lookup(' >>>>>>> env','PWD')}}/certs/wildcard.example.com.br.crt","keyfile":" >>>>>>> {{lookup('env','PWD')}}/certs/wildcard.example.com.br.key", >>>>>>> "cafile":"{{lookup('env','PWD')}}/certs/wildcard.example.com >>>>>>> .br.int.crt"}] >>>>>>> openshift_master_session_auth_secrets=['F71uoyI/Tkv/LiDH2PiF >>>>>>> KK1o76bLoH10+uE2a'] >>>>>>> openshift_master_session_encryption_secrets=['bjDwQfiy4ksB/3 >>>>>>> qph87BGulYb/GUho6K'] >>>>>>> openshift_master_audit_config={"enabled": true, "auditFilePath": >>>>>>> "/var/log/openshift-audit/openshift-audit.log", >>>>>>> "maximumFileRetentionDays": 30, "maximumFileSizeMegabytes": 500, >>>>>>> "maximumRetainedFiles": 10} >>>>>>> >>>>>>> openshift_ca_cert_expire_days=1825 >>>>>>> openshift_node_cert_expire_days=730 >>>>>>> openshift_master_cert_expire_days=730 >>>>>>> etcd_ca_default_days=1825 >>>>>>> >>>>>>> openshift_hosted_router_create_certificate=false >>>>>>> openshift_hosted_manage_router=true >>>>>>> openshift_router_selector="role=infra" >>>>>>> openshift_hosted_router_replicas=2 >>>>>>> openshift_hosted_router_certificate={"certfile":"{{lookup('e >>>>>>> nv','PWD')}}/certs/wildcard.example.com.br.crt","keyfile":"{ >>>>>>> {lookup('env','PWD')}}/certs/wildcard.example.com.br.key", >>>>>>> "cafile":"{{lookup('env','PWD')}}/certs/wildcard.example.com >>>>>>> .br.int.crt"} >>>>>>> >>>>>>> openshift_hosted_metrics_deploy=true >>>>>>> openshift_hosted_metrics_public_url=https://hawkular-metrics >>>>>>> .example.com.br/hawkular/metrics >>>>>>> >>>>>>> openshift_hosted_logging_deploy=true >>>>>>> openshift_hosted_logging_hostname=kibana.example.com.br >>>>>>> >>>>>>> openshift_install_examples=true >>>>>>> >>>>>>> openshift_node_kubelet_args={'pods-per-core': ['20'], 'max-pods': >>>>>>> ['100'], 'image-gc-high-threshold': ['80'], 'image-gc-low-threshold': >>>>>>> ['50'],'minimum-container-ttl-duration': ['60s'], >>>>>>> 'maximum-dead-containers-per-container': ['1'], >>>>>>> 'maximum-dead-containers': ['15']} >>>>>>> >>>>>>> logrotate_scripts=[{"name": "syslog", "path": >>>>>>> "/var/log/cron\n/var/log/maillog\n/var/log/messages\n/var/log/secure\n/var/log/spooler\n", >>>>>>> "options": ["daily", "rotate 7", "compress", "sharedscripts", >>>>>>> "missingok"], >>>>>>> "scripts": {"postrotate": "/bin/kill -HUP `cat /var/run/syslogd.pid 2> >>>>>>> /dev/null` 2> /dev/null || true"}}] >>>>>>> >>>>>>> openshift_builddefaults_image_labels=[{'name':'builder','val >>>>>>> ue':'true'}] >>>>>>> openshift_builddefaults_nodeselectors={'builder':'true'} >>>>>>> openshift_builddefaults_annotations={'builder':'true'} >>>>>>> openshift_builddefaults_resources_requests_cpu=10m >>>>>>> openshift_builddefaults_resources_requests_memory=128Mi >>>>>>> openshift_builddefaults_resources_limits_cpu=500m >>>>>>> openshift_builddefaults_resources_limits_memory=2Gi >>>>>>> >>>>>>> openshift_upgrade_nodes_serial=1 >>>>>>> openshift_upgrade_nodes_max_fail_percentage=0 >>>>>>> openshift_upgrade_control_plane_nodes_serial=1 >>>>>>> openshift_upgrade_control_plane_nodes_max_fail_percentage=0 >>>>>>> >>>>>>> openshift_disable_check=disk_availability,memory_availability >>>>>>> >>>>>>> [masters] >>>>>>> e001vmov40p42 >>>>>>> e001vmov40p51 >>>>>>> e001vmov40p52 >>>>>>> >>>>>>> [etcd] >>>>>>> e001vmov40p42 >>>>>>> e001vmov40p51 >>>>>>> e001vmov40p52 >>>>>>> >>>>>>> [nodes] >>>>>>> e001vmov40p42 openshift_node_labels="{'role': 'master'}" >>>>>>> e001vmov40p51 openshift_node_labels="{'role': 'master'}" >>>>>>> e001vmov40p52 openshift_node_labels="{'role': 'master'}" >>>>>>> >>>>>>> e001vmov40p45 openshift_node_labels="{'role': 'infra', >>>>>>> 'docker-registry':'true', 'logging':'true'}" >>>>>>> e001vmov40p46 openshift_node_labels="{'role': 'infra', 'metrics': >>>>>>> 'true'}" >>>>>>> >>>>>>> e001vmov40p47 openshift_node_labels="{'role': 'app', 'builder': >>>>>>> 'true'}" >>>>>>> e001vmov40p48 openshift_node_labels="{'role': 'app', 'builder': >>>>>>> 'true'}" >>>>>>> e001vmov40p49 openshift_node_labels="{'role': 'app', 'builder': >>>>>>> 'true'}" >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Mateus Caruccio / Master of Puppets >>>>>>> GetupCloud.com >>>>>>> We make the infrastructure invisible >>>>>>> Gartner Cool Vendor 2017 >>>>>>> >>>>>>> 2017-09-14 10:13 GMT-03:00 Matthew Wringe <mwri...@redhat.com>: >>>>>>> >>>>>>>> We had an issue where it was not possible for normal users to view >>>>>>>> their metrics (but cluster-admin users could). But I didn't think this >>>>>>>> made >>>>>>>> it into any releases. >>>>>>>> >>>>>>>> Would it be possible to attach the inventory file used? >>>>>>>> >>>>>>>> On Thu, Sep 14, 2017 at 8:34 AM, Paul Weil <pw...@redhat.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Including some metrics folks. Matt/Jeff? >>>>>>>>> >>>>>>>>> On Wed, Sep 13, 2017 at 9:44 PM, Mateus Caruccio < >>>>>>>>> mateus.caruc...@getupcloud.com> wrote: >>>>>>>>> >>>>>>>>>> Answering my own question, that "namespace" field on audit log >>>>>>>>>> refers to the unamespaced resource "/oapi/v1/subjectaccessreviews", >>>>>>>>>> not the subject access review object of the request. >>>>>>>>>> >>>>>>>>>> Still, the problem persists... >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Mateus Caruccio / Master of Puppets >>>>>>>>>> GetupCloud.com >>>>>>>>>> We make the infrastructure invisible >>>>>>>>>> Gartner Cool Vendor 2017 >>>>>>>>>> >>>>>>>>>> 2017-09-13 22:39 GMT-03:00 Mateus Caruccio < >>>>>>>>>> mateus.caruc...@getupcloud.com>: >>>>>>>>>> >>>>>>>>>>> Audit logs show this: >>>>>>>>>>> >>>>>>>>>>> 2017-09-13T22:18:43.907186125-03:00 AUDIT: >>>>>>>>>>> id="cf075af6-c8a7-4b3c-8727-4ad2aefa0a49" ip="10.150.10.35" >>>>>>>>>>> method="POST" user="mateus" groups="\"system:authenticated >>>>>>>>>>> :oauth\",\"system:authenticated\"" as="<self>" >>>>>>>>>>> asgroups="<lookup>" namespace="<none>" >>>>>>>>>>> uri="/oapi/v1/subjectaccessrev >>>>>>>>>>> iews" >>>>>>>>>>> 2017-09-13T22:18:43.941696064-03:00 AUDIT: >>>>>>>>>>> id="cf075af6-c8a7-4b3c-8727-4ad2aefa0a49" response="201" >>>>>>>>>>> >>>>>>>>>>> I'm I wrong o that "namespace" field should be not <none>? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Mateus Caruccio / Master of Puppets >>>>>>>>>>> GetupCloud.com >>>>>>>>>>> We make the infrastructure invisible >>>>>>>>>>> Gartner Cool Vendor 2017 >>>>>>>>>>> >>>>>>>>>>> 2017-09-13 20:31 GMT-03:00 Mateus Caruccio < >>>>>>>>>>> mateus.caruc...@getupcloud.com>: >>>>>>>>>>> >>>>>>>>>>>> After a fresh Origin 3.6.0, hawkular returns only 403 Forbiden. >>>>>>>>>>>> What is the auth path used by hawkular and how can I check if >>>>>>>>>>>> it's correct (secrets, servicaccounts, token, etc)? >>>>>>>>>>>> >>>>>>>>>>>> $ oc version >>>>>>>>>>>> oc v3.6.0+c4dd4cf >>>>>>>>>>>> kubernetes v1.6.1+5115d708d7 >>>>>>>>>>>> features: Basic-Auth GSSAPI Kerberos SPNEGO >>>>>>>>>>>> >>>>>>>>>>>> Server <redacted> >>>>>>>>>>>> openshift v3.6.0+c4dd4cf >>>>>>>>>>>> kubernetes v1.6.1+5115d708d7 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> $ oc -n openshift-infra get rc -o yaml | grep image: >>>>>>>>>>>> image: docker.io/openshift/origin-met >>>>>>>>>>>> rics-cassandra:v3.6.0 >>>>>>>>>>>> image: docker.io/openshift/origin-met >>>>>>>>>>>> rics-hawkular-metrics:v3.6.0 >>>>>>>>>>>> image: docker.io/openshift/origin-met >>>>>>>>>>>> rics-heapster:v3.6.0 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> $ oc -n openshift-infra get pods >>>>>>>>>>>> NAME READY STATUS RESTARTS AGE >>>>>>>>>>>> hawkular-cassandra-1-vg250 1/1 Running 0 42m >>>>>>>>>>>> hawkular-metrics-4rkn4 1/1 Running 0 38m >>>>>>>>>>>> heapster-fjg8t 1/1 Running 1 50m >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> $ oadm diagnostics MetricsApiProxy >>>>>>>>>>>> [Note] Determining if client configuration exists for >>>>>>>>>>>> client/cluster diagnostics >>>>>>>>>>>> Info: Successfully read a client config file at >>>>>>>>>>>> '/home/getup/.kube/config' >>>>>>>>>>>> Info: Using context for cluster-admin access: >>>>>>>>>>>> 'default/<redacted>:443/system:admin' >>>>>>>>>>>> >>>>>>>>>>>> [Note] Running diagnostic: MetricsApiProxy >>>>>>>>>>>> Description: Check the integrated heapster metrics can >>>>>>>>>>>> be reached via the API proxy >>>>>>>>>>>> >>>>>>>>>>>> [Note] Summary of diagnostics execution (version >>>>>>>>>>>> v3.6.0+c4dd4cf): >>>>>>>>>>>> [Note] Completed with no errors or warnings seen. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks >>>>>>>>>>>> -- >>>>>>>>>>>> Mateus Caruccio / Master of Puppets >>>>>>>>>>>> GetupCloud.com >>>>>>>>>>>> We make the infrastructure invisible >>>>>>>>>>>> Gartner Cool Vendor 2017 >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> dev mailing list >>>>>>>>>> dev@lists.openshift.redhat.com >>>>>>>>>> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> dev mailing list >>>>>>> dev@lists.openshift.redhat.com >>>>>>> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev >>>>>>> >>>>>> >>>>> >>>> >>> >> >
_______________________________________________ dev mailing list dev@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
