Hi Isuru, I completed the cert renewal part of the feature and the implementation can be found at [1].
At the moment I'm doing some refactoring according to acme protocol best practices. After that I'm hoping to start working on custos integration part of the feature using Java SDK. Basically the idea is to store the updated certificates in Custos. I am using pebble [2] (a lightweight version of letsencrypt boulder CA server) as the CA server for testing purposes. The main reason for using Pebble is that the Lets Encrypt CA server endpoint (staging) is not working as expected in the dev environment for a fake domain. As for the current implementation I'm using a nginx server with Lua module as acme challenge server to store files required for completing the challenge. Lua module is used to embed Lua programming language into nginx server and create files required for acme challenge. (CGI with Nginx and SpringBoot are two other options for this and I'm thinking of do a proper poc/rfc after initial feature implementation is completed) Regards [1]. https://github.com/hasithajayasundara/cert-updater [2]. https://github.com/letsencrypt/pebble On Fri, May 14, 2021 at 2:41 PM Hasitha Jayasundara < [email protected]> wrote: > Hi Isuru, > > Thanks for the explanation. I think I have a clear idea on how to proceed > with the Let's Encrypt part of the implementation now. > > I already started the implementation and am currently designing a solution > for handling the Let's Encrypt challenge. Once that is done I am thinking > of starting the certificates store process in Custos. I think I'll probably > need some setup completed first for making authenticated requests to Custos. > > Found an issue with the certificate store path in the current > implementation and I've sent a fix for that. [1] > > Regards > > [1]. https://github.com/apache/airavata-custos/pull/160 > > On Thu, May 13, 2021 at 9:31 AM Isuru Ranawaka <[email protected]> wrote: > >> Hi Hasitha, >> >> Glad to hear that you have gone through the code. Please feel free to >> submit bug fixes as you see. >> >> We use HarshiCorp vault to store secrets. The paths you mentioned are >> actually created in the vault. Precisely, owner_id is the id of the user >> or >> the service account that owns the certificate. Token is an identifier that >> can be used to retrieve the certificate later. >> >> thanks >> Isuru >> >> On Wed, May 12, 2021 at 11:08 PM Hasitha Jayasundara < >> [email protected]> wrote: >> >> > Hi Isuru, >> > >> > Thanks for the reply. Based on your comment that custos already has an >> > endpoint to store certificates I took some time to go through the Secret >> > Management Service and the proto for Resource Secret Service. >> > >> > According to my observation, I think I can use *POST - >> > /resource-secret-management/v1.0.0/secret/certificate* endpoint to do >> > certificate uploading. (Correct me if I'm wrong) >> > >> > Going through the code I found that certificate is stored in >> > */resourcesecret/<tenant_id>/<owner_id>/ssh/<token>*. The <tenant_id>, >> > <owner_id> and <token> are received through the request body. I am not >> 100% >> > clear about <owner_id> and <token> fields in the context of custos >> secret >> > manager service. Could you please explain a bit about <owner_id> and the >> > <token>? >> > >> > Regards >> > >> > On Wed, May 12, 2021 at 7:30 AM Isuru Ranawaka <[email protected]> >> wrote: >> > >> > > Hi Hasitha, >> > > >> > > We really appreciate your interest in Custos project. Please find the >> > > inline comments. >> > > >> > > On Tue, May 11, 2021 at 9:33 AM Hasitha Jayasundara < >> > > [email protected]> wrote: >> > > >> > > > Hi all, >> > > > >> > > > I'm Hasitha and I'm currently working as a Full Stack Engineer for >> a UK >> > > > based tech company. I graduated from University Of Moratuwa >> Department >> > of >> > > > Electronic and Telecommunication Engineering Sri Lanka and >> completed a >> > > > software engineering internship at a company called WSO2 which is an >> > open >> > > > source technology provider based in Sri Lanka. I also participated >> in >> > > > Google Summer of Code 2017 and successfully completed a project for >> > > > implementing an inbound endpoint and a connector for IBM-MQ. >> > > > >> > > > I'm enthusiastic about the airavata-custos project and would be >> > extremely >> > > > interested in contributing to the project. I've already followed the >> > wiki >> > > > page[1] and the tutorial presented at the Gateways 2020 Conference >> [2] >> > > and >> > > > I think I have a clear understanding about how custos operates in >> the >> > > real >> > > > world. >> > > > >> > > > As a starting point I selected the issue #152 [3] which is to >> > implement a >> > > > cert manager task using Custos. The idea is to automate the cert >> > renewal >> > > > process so that the client's won't need to manually do it by going >> > > through >> > > > Let's Encrypt cert renewal process. The task (cron job) will be >> > > responsible >> > > > for, >> > > > >> > > > - Connecting to Let’s Encrypt APIs and create certificates, >> update >> > > > certificates >> > > > - Saving updated certificates into a given location and where the >> > > server >> > > > application can use it >> > > > >> > > > The overall process can be illustrated as follows. >> > > > >> > > > >> > > > >> > > > - Custos Cert Renewer task (Cron Job) will be executed on >> periodic >> > > > configurable time period >> > > > - Task will communicate with Let's Encrypt using ACME(Automated >> > > > Certificate Management Environments) protocol. (I've already >> tested >> > > this >> > > > with acme4j [4] which is a java client for ACME protocol and it's >> > > > working >> > > > fine without any issues) >> > > > - Task will then obtain the updated certificate and store it in a >> > > given >> > > > path in Custos (probably in a persistent volume) >> > > > - Client applications can query certificates from Custos and add >> > them >> > > to >> > > > client trustore on demand to call a server application >> > > > >> > > > Following I listed out some of the concerns I'm having at the >> moment. >> > > > >> > > > - In order to obtain an updated certificates, >> > > > >> > > > >> > > > 1. we need to have an account in CA with a public and a private >> key >> > > > pair. Do we already have an account in Let's Encrypt for this >> type >> > of >> > > > work >> > > > or Is it possible to create a dedicated account for the cert >> renewer >> > > > task? >> > > > >> > > >> > > yes, we can create a test account in Let's Encrypt. >> > > >> > > 2. we need to provide CNs (domain names) for Let's Encrypt. Where >> do >> > we >> > > > store the domain names of the clients? >> > > > >> > > >> > > I guess you are referring to the server application domains. those >> should >> > > be configurable and can be stored in a configuration file. >> > > >> > > >> > > > - After obtaining the certificate, it's required to save it in >> > Custos >> > > in >> > > > a given path. Do we already have an endpoint to achieve this? >> > > > >> > > >> > > Yes, Custos has a secret management service to save certificates. >> > > >> > > >> > > > - Client applications can query the certificates in Custos. How >> can >> > we >> > > > provide the clients the information about the path to query? >> > > > >> > > >> > > >> > > Client applications can use Custos service accounts to access Custos >> > APIs, >> > > >> > > >> > > > >> > > > Please have a look at the above and let me know your opinion. >> > > > >> > > > Regards >> > > > >> > > > [1]. >> > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/CUSTOS/Custos+Architecture+and+Internals >> > > > [2]. https://youtu.be/CuBvFj194Kg >> > > > [3]. https://github.com/apache/airavata-custos/issues/152 >> > > > [4]. https://github.com/shred/acme4j >> > > > >> > > >> > > >> > > -- >> > > Research Software Engineer >> > > Indiana University, IN >> > > >> > >> >> >> -- >> Research Software Engineer >> Indiana University, IN >> >
