Have you tried to simply restart Traffic Ops? We've seen ours (1.6) not being able to create Certificates after a while.
On Wed, Jan 18, 2017 at 11:10 PM, Nir Sopher <n...@qwilt.com> wrote: > ERROR result for http://ops.nirs-tc1.tc-dev.qwilt.com/api/1.2/cdns/name/ > nirs-tc1-cdn/sslkeys.json is: ...{"message":"No SSL certificates found for > nirs-tc1-cdn"}... > FATAL http://ops.nirs-tc1.tc-dev.qwilt.com/api/1.2/cdns/name/ > nirs-tc1-cdn/sslkeys.json returned HTTP 404! > > > On Thu, Jan 19, 2017 at 12:43 AM, Dave Neuman <neu...@apache.org> wrote: > > > What error are you getting in ORT? > > > > On Wed, Jan 18, 2017 at 11:57 AM, Nir Sopher <n...@qwilt.com> wrote: > > > > > OK. > > > I called the command from traffic op and got the below output, which > > looks > > > ok to me. > > > So now I know that adding a certificate via the "paste" screen works > (and > > > not only say "success"). > > > Still, pulling the configuration via the ort script fails. > > > > > > Regarding the log, no message during the certificate paste. My log cfg > is > > > also paste below. > > > > > > 10x, > > > Nir > > > > > > $ cat /opt/traffic_ops/app/conf/production/log4perl.conf > > > log4perl.rootLogger = ERROR, SCREEN, FILE > > > log4perl.appender.FILE = Log::Log4perl::Appender::File > > > log4perl.appender.FILE.layout = PatternLayout > > > log4perl.appender.FILE.layout.ConversionPattern = [%d{ISO8601}] [%p] > > %m%n > > > log4perl.appender.FILE.filename = /var/log/traffic_ops/traffic_ops.log > > > > > > log4perl.appender.SCREEN = Log::Log4perl::Appender::Screen > > > log4perl.appender.SCREEN.layout = PatternLayout > > > log4perl.appender.SCREEN.layout.ConversionPattern = [%d{ISO8601}] [%p] > > > %m%n > > > > > > > > > > > > $ curl -k "https://admin:admin...@vault-int.nirs-tc1.tc-dev.qwilt.com: > > > 8088/riak/ssl/ynet-images-latest" > > > {"cdn":"nirs-tc1-cdn","deliveryservice":"ynet-images" > > > ,"certificate":{"csr":" > > > LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0NCk1JSUI2REND > > > QVZFQ0FRQXdnYWN4\nQ3pBSkJnTlZCQVlUQWtsTU1ROHdEU > > > VlEVlFRSURBWkpjM0poWld3eEZEQVMNCkJnTlZCQWNNQzBo\ > > > ndlpFaGhjMmhoY205dU1RNHdEQVlEVlFRS0RBVlJkMmxzZERFTE1Ba0dBMVV > > > FQ3d3Q1VVTXgNCk5U\nQXpCZ05WQkFNTUxDb3VlVzVsZEMxc > > > GJXRm5aWE11Ym1seWN5MTBZekV0WTJSdUxuUmpMV1JsZGk1\ > > > namNXeHYNCmRXUXVZMjl0TVIwd0d3WUpLb1pJaHZjTkFRa0JGZzV1YVhKelF > > > IRjNhV3gwTG1OdmJU\nQ0JuekFOQmdrcWhraUcNCjl3MEJBU > > > UVGQUFPQmpRQXdnWWtDZ1lFQTAxVWZnbzZrcEJOMGNQOEV5\ > > > nVXY4MW9WNFB2WlJoM2V5dmViNjBaZnQNCldjblZ0Zk53N1ZJRW52Q1ByU0J > > > 6b25MajI4NGoyUGcv\nQkhQQ3Rudmc2N2N5bXRKT2pJVU4rZ > > > XoyRXkvSUxnUXYNCkdjZFQ0RmErTGZmcXFudUc3Y3gxcDRU\ > > > nR3k2aGpYdFNPZ2R0YklyNFhEajJiWlBIVTVxTFlkak1QSXZXc2M5aGkNCmV > > > QY0NBd0VBQWFBQU1B\nMEdDU3FHU0liM0RRRUJCUVVBQTRHQ > > > kFDRGJQUlFSM1RkNWh1QmtQMUg3V0l4ejdjNU8NCnJsYnpn\ > > > nWHlxcEpjRFg2Q3RJaEd1d1orYkxIa3Y4dXdsMUoyZm5QTWM3TlB4UGxjbXY > > > 0RWU3RXpJQ3dJTzBr\ncTMNClFvdksraEp1MDJLTE1peUp5b > > > HZpT1VEeWlldEtPdEpDNlVKelNhZEpjWjVnSmJzNjNiRk83\ > > > nWmlpbDQ0UmdKaFYNCklBMSsyYUwwU0hmeTY4R2cNCi0tLS0tRU5EIENFUlR > > > JRklDQVRFIFJFUVVF\nU1QtLS0tLQ==","crt":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS > > > 0tLS0tDQpNSUlDeHpDQ0FqQUNDUURvZlNRcTJpcnQ4REFO\ > > > nQmdrcWhraUc5dzBCQVFVRkFEQ0JwekVMTUFrR0ExVUVCaE1DDQpTVXd4RHp > > > BTkJnTlZCQWdNQmts\nemNtRmxiREVVTUJJR0ExVUVCd3dMU > > > 0c5a1NHRnphR0Z5YjI0eERqQU1CZ05WDQpCQW9NQlZGM2FX\ > > > neDBNUXN3Q1FZRFZRUUxEQUpSUXpFMU1ETUdBMVVFQXd3c0tpNTVibVYwTFd > > > sdFlXZGxjeTV1DQph\nWEp6TFhSak1TMWpaRzR1ZEdNdFpHV > > > jJMbU54Ykc5MVpDNWpiMjB4SFRBYkJna3Foa2lHOXcwQkNR\ > > > nRVdEbTVwDQpjbk5BY1hkcGJIUXVZMjl0TUI0WERURTNNREV4TmpFeE5UQTB > > > NbG9YRFRFNE1ERXhO\nakV4TlRBME1sb3dnYWN4DQpDekFKQ > > > mdOVkJBWVRBa2xNTVE4d0RRWURWUVFJREFaSmMzSmhaV3d4\ > > > nRkRBU0JnTlZCQWNNQzBodlpFaGhjMmhoDQpjbTl1TVE0d0RBWURWUVFLREF > > > WUmQybHNkREVMTUFr\nR0ExVUVDd3dDVVVNeE5UQXpCZ05WQ > > > kFNTUxDb3VlVzVsDQpkQzFwYldGblpYTXVibWx5Y3kxMFl6\ > > > nRXRZMlJ1TG5SakxXUmxkaTVqY1d4dmRXUXVZMjl0TVIwd0d3WUpLb1pJDQp > > > odmNOQVFrQkZnNXVh\nWEp6UUhGM2FXeDBMbU52YlRDQm56Q > > > U5CZ2txaGtpRzl3MEJBUUVGQUFPQmpRQXdnWWtDDQpnWUVB\ > > > nMDFVZmdvNmtwQk4wY1A4RXlVdjgxb1Y0UHZaUmgzZXl2ZWI2MFpmdFdjblZ > > > 0Zk53N1ZJRW52Q1By\nU0J6DQpvbkxqMjg0ajJQZy9CSFBDd > > > G52ZzY3Y3ltdEpPaklVTitlejJFeS9JTGdRdkdjZFQ0RmEr\ > > > nTGZmcXFudUc3Y3gxDQpwNFRHeTZoalh0U09nZHRiSXI0WERqMmJaUEhVNXF > > > MWWRqTVBJdldzYzlo\naWVQY0NBd0VBQVRBTkJna3Foa2lHD > > > Qo5dzBCQVFVRkFBT0JnUUJha0tKaTNrN1hOUDljWTZ0K05i\ > > > nT0hNVWJPWVI0WWE2Y2xKN3cyYU1CSTNYdjNZMUcyDQo5K1ZxajA1cDZXaU8 > > > xWVNGWWRBb2QxSnRD\nNDRieUt4NWRBbTNKdnZrUWZNNU8xb > > > 09zNG8yWnhrMXRmZmVqN3NkDQpCSDBKOGdqSkhYbmg0TWFm\ > > > neHhzR09KSXhOSXI3aDA5cTZYUENaTlVVaTROQnRrRzVVM2dsUnB0YWlnPT0 > > > NCi0tLS0tRU5EIENF\nUlRJRklDQVRFLS0tLS0=","key":" > > > LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQ0KTUlJQ1hRSUJBQUtC > > > Z1FEVFZSK0NqcVNr\nRTNSdy93VEpTL3pXaFhnKzlsR0hkN > > > 0s5NXZyUmwrMVp5ZFcxODNEdA0KVWdTZThJK3RJSE9pY3VQ\ > > > nYnppUFkrRDhFYzhLMmUrRHJ0ekthMGs2TWhRMzU3UFlUTDhndUJDOFp4MVB > > > nVnI0dA0KOStxcWU0\nYnR6SFduaE1iTHFHTmUxSTZCMjFza > > > XZoY09QWnRrOGRUbW90aDJNdzhpOWF4ejJHSjQ5d0lEQVFB\ > > > nQg0KQW9HQkFNQmpSL0pGQldGUlRMbnBqMlBweDExTDJISUpMNk9SdHFqbTl > > > BT0d1Yzc1elpKODhw\nczZCWGJrTFFoQQ0KK01RMHIzYlZMU > > > kZDdmF2Qjdzck43NjdtOGlzU3JMWGZWK09MeGlQU2NGMHZk\ > > > nck5Zd1k4YlREMnl5SnpnM0hYcA0KUFVvZDBMQzlzMmdlcW5kRU1ha21BYkJ > > > 2T1ZHNkxKMTF1NXVU\nV1FBdWhPYmg0NzN4QWtFQS9ValN6a > > > jVxUVk2bA0KeVJ2eVh2enM4S0RWVjZCc3k4eHNIaUJjNUg3\ > > > ndEdiL3B3WGZaZ0RDQ0xkaUlBSzdVZ0lmOHZlbDkxNEM1dFB0Zg0KdEhxZEd > > > 5bXJ1d0pCQU5XWktB\nT2dXN0VZVXJ3OWFTdjlKM0Z3dHp4W > > > E9NZURpTnNtbW40OXJ5dmN2bmR6dEVlVA0KOWVybVJsM0N3\ > > > nSE1uZ0ZIS2VYVmJ1dENoWlkvZDZaKy83ZlVDUUZPaUlEbUowbndqSmdycDk > > > zWDEvaWJXZEp1aQ0K\nbFVvV0RmMUVvbWV3b1luSEhPQ05Pb > > > nhoaUJxclRQMHN2VzVUZU5rY3FEam9nR21LTjJmWXROZXJR\ > > > ndEVDUUJWZQ0KM25jR2EwWWJ0ZU5wallVK0xkMFd0dTZObDN1MnVGR2MyaVk > > > 1UzdacXZvKzYvdFdP\nZ3pNK1dObjJxMFNhTmlkNA0KeDVBc > > > lhsU1RZVkwway9STXdxVUNRUUR6SFoyT0JRbHJEdmFyWWIy\ > > > nek1KZkFpMjRmV0lCQ1VTM2tuSmNzZGt3bA0Kc1BseVFZRndDRUMySzh6Y01 > > > DaFVTcVRuZ0NlWWpK\nenJNbXU4Qkp1M1VCNmENCi0tLS0tR > > > U5EIFJTQSBQUklWQVRFIEtFWS0tLS0t"},"version":"5","hostname":"*. > > > ynet-images.nirs-tc1-cdn.tc-dev.qwilt.com","key":"ynet-images"} > > > > > > On Wed, Jan 18, 2017 at 8:01 PM, Dave Neuman <neu...@apache.org> > wrote: > > > > > > > The second curl would be: curl -k " > > > > https://admin:admin...@vault-int.nirs-tc1.tc-dev.qwilt.com:8 > > > > 088/riak/ssl/ynet-images-latest > > > > " > > > > > > > > If that works from your traffic_ops host then it should also work > when > > > you > > > > go into the paste keys screen. > > > > > > > > Turning on Debug logging might also help. You can set > > > log4perl.rootLogger = > > > > ERROR, SCREEN, FILE in traffic_ops/app/conf/production/log4perl.conf > > > > > > > > Try that out and send me what, if anything, you see in the log. > > > > > > > > Thanks, > > > > > > > > Dave > > > > > > > > > > > > On Wed, Jan 18, 2017 at 9:14 AM, Nir Sopher <n...@qwilt.com> wrote: > > > > > > > > > Thanks Dave, > > > > > I am pasting the keys through the Manange SSL Keys -> Paste > Existing > > > Keys > > > > > screen. > > > > > > > > > > Below is the output of the curl commands: > > > > > > > > > > $ curl -k "https://admin:admin123@vault- > > int.nirs-tc1.tc-dev.qwilt.com: > > > > > 8088/buckets/ssl/keys?keys=true" > > > > > {"keys":["ynet-images-5","ynet-images-latest","ynet- > > > > > images-4","ynet-images-3"]} > > > > > > > > > > $ curl -k "https://admin:admin123@vault- > > int.nirs-tc1.tc-dev.qwilt.com: > > > > > 8088/riak/ssl/xmlid-latest" > > > > > not found > > > > > > > > > > Nir > > > > > > > > > > On Wed, Jan 18, 2017 at 4:56 PM, Dave Neuman <neu...@apache.org> > > > wrote: > > > > > > > > > > > That sucks that it still doesn't work :( > > > > > > > > > > > > Lets start with the config. You said you had to set ` > > > > > > listener.https.internal= 0.0.0.0:8088`, we have that configured > > with > > > > the > > > > > > IP > > > > > > of the riak server, but if you can successfully make curl > requests > > > from > > > > > the > > > > > > traffic_ops server, then I guess that is ok. > > > > > > > > > > > > As for the error you are getting...that error is basically saying > > > that > > > > > Riak > > > > > > cannot find the SSL Keys that you are looking for. > > > > > > > > > > > > Which endpoint are you using when you get that error? Are you > > going > > > > > > through the Manange SSL Keys -> Paste Existing Keys screen? Or > are > > > you > > > > > > hitting an API? > > > > > > > > > > > > You should be able to see if the keys exist by running `curl -k > > > > > > "https://admin:password@riakURL:8088/buckets/ssl/keys? > keys=true"` > > > and > > > > > > looking for XMLID-latest in the list of keys; you could also run > > > `curl > > > > -k > > > > > > "https://admin:password@riakURL:8088/riak/ssl/xmlid-latest"` > > > > > > > > > > > > Thanks, > > > > > > Dave > > > > > > > > > > > > On Tue, Jan 17, 2017 at 1:57 PM, Nir Sopher <n...@qwilt.com> > > wrote: > > > > > > > > > > > > > Thank you Dave:) > > > > > > > > > > > > > > Indeed I was using Riak 2.2 with TC 1.7. > > > > > > > I moved now to Riak 2.1.3 (same traffic ops, just replaced the > > > > vault). > > > > > > > I see the same issues. The only change is the added log > messages > > in > > > > > > traffic > > > > > > > ops log during certificate generation: > > > > > > > > > > > > > > [2017-01-17 20:29:58,119] [ERROR] Active Server Severe Error: > > 404 - > > > > > > > vault-int.nirs-tc1.tc-dev.qwilt.com:8088 - not found > > > > > > > > > > > > > > Nir > > > > > > > > > > > > > > On Tue, Jan 17, 2017 at 6:56 PM, Dave Neuman < > neu...@apache.org> > > > > > wrote: > > > > > > > > > > > > > > > Hey Nir, > > > > > > > > I think I can help here. First of all, what version of > Traffic > > > > > Control > > > > > > > are > > > > > > > > you running and which version of Riak are you running? We > have > > > > seen > > > > > > > issues > > > > > > > > using newer versions of Riak with Traffic Control 1.7 and > 1.8. > > > > Those > > > > > > > > issues should be resolved in the next release. For now we > > > > recommend > > > > > > you > > > > > > > > use Riak 2.1.x and not 2.2.x > > > > > > > > > > > > > > > > Once I know that we can start digging deeper. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Dave > > > > > > > > > > > > > > > > On Tue, Jan 17, 2017 at 9:44 AM, Nir Sopher <n...@qwilt.com> > > > > wrote: > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > I am trying to launch a traffic vault and connect it to my > > > > > > traffic-ops > > > > > > > > > server. > > > > > > > > > I followed the instructions in the admin guide > > > > > > > > > <http://traffic-control-cdn.net/docs/latest/admin/traffic_ > > > > > vault.html > > > > > > >, > > > > > > > > > installing riak <http://goog_1273226474>2.2.0-1 > > > > > > > > > <http://s3.amazonaws.com/downloads.basho.com/riak/2.2/ > > > > > > > > > 2.2.0/rhel/6/riak-2.2.0-1.el6.x86_64.rpm> > > > > > > > > > working with a self signed certificate (created via the > > > > > instructions > > > > > > in > > > > > > > > > this > > > > > > > > > <http://www.akadia.com/services/ssh_test_certificate.html> > > > link) > > > > > > > > > > > > > > > > > > I had to deviate from the document in a few places in order > > to > > > > > > > progress: > > > > > > > > > > > > > > > > > > - Replacing the host part in the riak listener > > configuration > > > > > with > > > > > > > > > 0.0.0.0. Using real hostname made riak to fail. e.g. > > > > > > > > > listener.https.internal > > > > > > > > > = 0.0.0.0:8088 > > > > > > > > > - Setting ssl.cacertfile to point at the server.crt (as > > this > > > > is > > > > > a > > > > > > > self > > > > > > > > > signed certificate): ssl.cacertfile = > > > > /etc/riak/certs/server.crt > > > > > > > Note > > > > > > > > > that I assume that this certificate is only used for > > > "traffic > > > > > > vault > > > > > > > > > https" > > > > > > > > > connections. > > > > > > > > > - In traffic ops, I initially set the "tcp port" to > "8098" > > > and > > > > > > > "https > > > > > > > > > port" to "8088". When traffic ops tried to connect the > > vault > > > > it > > > > > > did > > > > > > > it > > > > > > > > > via > > > > > > > > > port "8098", so I changed the "tcp port" to "8088" in > > order > > > > for > > > > > > > https > > > > > > > > > to be > > > > > > > > > used. > > > > > > > > > > > > > > > > > > > > > > > > > > > Validating the installation using curl -kvs "https://admin > > > > > > > > > :password@riakserver:8088/search/query/sslkeys?wt=json& > > > > > q=cdn:mycdn" > > > > > > > > > Produced the below output: > > > > > > > > > < HTTP/1.1 200 OK > > > > > > > > > < Server: MochiWeb/1.1 WebMachine/1.10.9 (cafe not found) > > > > > > > > > < Date: Wed, 11 Jan 2017 12:26:07 GMT > > > > > > > > > < Content-Type: application/json; charset=UTF-8 > > > > > > > > > < Content-Length: 571 > > > > > > > > > < > > > > > > > > > {"responseHeader":{"status":0, > "QTime":176,"params":{"shards" > > :" > > > > > > > > > vault-int.nirs-tc1.tc-dev.qwilt.com:8093/internal_solr/ > > sslkeys > > > > > > > > > ","q":"cdn:nirs-tc1-cdn","wt":"json"," > > > > > > > > > vault-int.nirs-tc1.tc-dev.qwilt.com:8093":"(_yz_pn:62 AND > > > > > > > (_yz_fpn:62)) > > > > > > > > OR > > > > > > > > > _yz_pn:61 OR _yz_pn:58 OR _yz_pn:55 OR _yz_pn:52 OR > _yz_pn:49 > > > OR > > > > > > > > _yz_pn:46 > > > > > > > > > OR _yz_pn:43 OR _yz_pn:40 OR _yz_pn:37 OR _yz_pn:34 OR > > > _yz_pn:31 > > > > OR > > > > > > > > > _yz_pn:28 OR _yz_pn:25 OR _yz_pn:22 OR _yz_pn:19 OR > _yz_pn:16 > > > OR > > > > > > > > _yz_pn:13 > > > > > > > > > OR _yz_pn:10 OR _yz_pn:7 OR _yz_pn:4 OR > > > > > > _yz_pn:1"}},"response":{"numFo > > > > > > > > > und":0,"start":0,"maxScore":0.0,"docs":[]}} > > > > > > > > > * Connection #0 to host vault-int.nirs-tc1.tc-dev. > qwilt.com > > > left > > > > > > > intact > > > > > > > > > * Closing connection # > > > > > > > > > > > > > > > > > > However, when I created a delivery-service and tried to > > > > "generate" > > > > > a > > > > > > > > > certificate via traffic-ops, I got the below message: > > > > > > > > > SSL keys for <ds> could not be created. Response was: > Error > > > > > creating > > > > > > > key > > > > > > > > > and csr. Result is -1 > > > > > > > > > No log message found int traffic_ops log or in the riak > log, > > to > > > > > > explain > > > > > > > > the > > > > > > > > > issue. > > > > > > > > > > > > > > > > > > When pasting a certificate (self signed, including the > "----" > > > > > headers > > > > > > > and > > > > > > > > > footers), the operation succeed. However, when the traffic > > > > servers > > > > > > > tried > > > > > > > > to > > > > > > > > > pull this configuration, I got the below message: > > > > > > > > > ERROR result for > > > > > > > > > http://ops.nirs-tc1.tc-dev.qwilt.com/api/1.2/cdns/name/ > > > > > > > > > nirs-tc1-cdn/sslkeys.json > > > > > > > > > is: ...{"message":"No SSL certificates found for > > > > nirs-tc1-cdn"}... > > > > > > > > > FATAL > > > > > > > > > http://ops.nirs-tc1.tc-dev.qwilt.com/api/1.2/cdns/name/ > > > > > > > > > nirs-tc1-cdn/sslkeys.json > > > > > > > > > returned HTTP 404! > > > > > > > > > > > > > > > > > > Any idea what may cause these issues? > > > > > > > > > Any experience in debugging similar issues? > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Nir > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
