chibenwa commented on a change in pull request #718:
URL: https://github.com/apache/james-project/pull/718#discussion_r738294250
##########
File path: server/apps/distributed-app/helm-chart/james/configs/blob.properties
##########
@@ -0,0 +1,81 @@
+# ============================================= BlobStore Implementation
==================================
Review comment:
If possible I would be happy to find a way not to re-duplicate all the
conf files...
https://docs.harness.io/article/yjkkwi56hl-link-resource-files-or-helm-charts-in-git-repos
I would :+1: linking files from
https://github.com/apache/james-project/tree/master/server/apps/distributed-app/sample-configuration
into the manifest where possible <3
##########
File path:
server/apps/distributed-app/helm-chart/james/configs/cassandra.properties
##########
@@ -0,0 +1,36 @@
+# Configuration file for cassandra mailbox
+# http://james.apache.org/server/config-cassandra.html
+cassandra.nodes=${env:OP_JAMES_CASSANDRA_NODES_URLS}
+
+cassandra.keyspace.create=true
+cassandra.keyspace=${env:OP_JAMES_CASSANDRA_KEYSPACE}
Review comment:
This OP_ prefix in the env do not make sense...
##########
File path:
server/apps/distributed-app/helm-chart/james/configs/usersrepository.xml
##########
@@ -0,0 +1,44 @@
+<?xml version="1.0"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+ -->
+
+<usersrepository name="LocalUsers"
+ class="org.apache.james.user.ldap.ReadOnlyUsersLDAPRepository"
Review comment:
Here we have a dependency on a LDAP.
Please use a simpler CassandraUsersRepository instead to simplify the
setUp...
##########
File path: server/apps/distributed-app/helm-chart/james/doc/misc/checklist.md
##########
@@ -0,0 +1,54 @@
+# Checklist
+
+There is some things that you must have before being able to run James on your
Kubernetes Cluster.
+
+## MUST have
+
+Those are absolutely necessary if you want at least James to start without
crashing.
+
+- [ ] Create a namespace for James deployment.
+- [ ] The right image container used for James
+- [ ] Have an appropriate email domain (Contact your Domain Service Provider)
+- [ ] Deploy Helm RabbitMQ package
+- [ ] Cassandra nodes deployed
+- [ ] Put valid Cassandra keyspaces in the configuration
+- [ ] ElasticSearch nodes deployed
+- [ ] LDAP deployed
+- [ ] S3 deployed
+- [ ] Configure JWT keys, keystores, SPF and DKIM. See document
[here](https://james.apache.org/howTo/)
+- [ ] Adding the proper `mailetcontainer.xml` file with other conf files in
James Helm package
+
+With those you should be able to startup James for first tests without having
it crashing. However it's still not enough to have it fully functional.
+
+## should have
+
+- [ ] Postifx-in deployed in emaily for delivering SMTP messages to James
(avoiding exposing the port 25 for James externally)
Review comment:
Remove mentions of PostFix.
##########
File path: server/apps/distributed-app/helm-chart/james/doc/misc/values.md
##########
@@ -0,0 +1,101 @@
+# Values for Helm James template
+
+Helm James has many visible values to configure before being able to use it
correctly.
+Values are injected into the deployment template as environment variables.
+An example with default values can be found in the `values.yaml` file.
+
+This document presents a quick description of each value field.
+
+## DNS values
+
+| Field | Default value | Description |
+| :----------- | :-------------------------------------------- | :----------- |
+| emailDomain | mail.example.com | The domain
name for your tenant's email addresses |
+| jmapUrl | jmap.example.com | The url for
the ingress jmap endpoint |
+| adminUrl | admin.example.com | The url for
the ingress admin endpoint |
+| smtpHostname | smtp.example.com | The SMTP Ehlo
hostname |
+
+## James values
+
+| Field | Default value
| Description |
+| :----------- |
:---------------------------------------------------------------- |
:----------- |
+| replicaJmapInstanceCount | 1
| The number of pods we deploy for James JMAP instance in the cluster |
+| replicaImapSmtpInstanceCount | 1
| The number of pods we deploy for James IMAP/SMTP instance in the cluster |
+| image | apache/james:distributed-3.7.0
| The James image used for deployment (Cassandra - RabbitMQ - LDAP
distributed James flavor) |
+
+### James environment values
+
+| Field | Default value |
Description |
+| :--------------------------- | :----------------------------------------- |
:----------- |
+| jamesAdminID | [email protected] |
The James admin account (mainly used for imapsync)|
Review comment:
openpaas spotted.
##########
File path: server/apps/distributed-app/helm-chart/james/secrets.sample.yaml
##########
@@ -0,0 +1,40 @@
+james:
+ secret:
+ jwtPublicKey: |-
+ -----BEGIN PUBLIC KEY-----
+ change-me
+ -----END PUBLIC KEY-----
+ jwtPrivateKey: |-
+ -----BEGIN PUBLIC KEY-----
+ change-me
+ -----END PUBLIC KEY-----
+ cassandraUser: change-me
+ cassandraPassword: change-me
+ esUser: change-me
+ esPassword: change-me
+ keystorePassword: change-me
+ dkimPrivateKey: |-
+ -----BEGIN PRIVATE KEY-----
+ change-me
+ -----END PRIVATE KEY-----
+ # A keystore ssl and jwt encoded in base64 with predefined users.
+ keystoreJWTinBase64: ""
+ # Optional
+ remoteDeliveryUsername: "change-me"
+ # Optional
+ remoteDeliveryPassword: "change-me"
+ s3:
+ endpoint: "https://auth.cloud.ovh.net/v3";
+ region: "GRA5"
+ accessKeyId: "change-me"
+ secretKey: "change-me"
+ bucketNameSuffix: "change-me"
+ldap:
+ esn:
+ secret:
+ ldapUser: change-me
+ ldapPassword: change-me
Review comment:
esn spotted. Let's also not set up a LDAP here just yet...
##########
File path:
server/apps/distributed-app/docs/modules/ROOT/pages/run-kubernetes.adoc
##########
@@ -0,0 +1,46 @@
+= Distributed James Server — Run with Kubernetes
+:navtitle: Run with Kubernetes
+
+== Deploy Apache James on Kubernetes with Helm chart.
+
+This chart bootstraps a James mail server on a
link:http://kubernetes.io[Kubernetes]
+cluster using the link:https://helm.sh[Helm] package manager.
+
+Before runnning the Helm chart, James should have particularly access to the
following external products:
+
+* Cassandra
+* ElastiSearch
+* RabbitMQ
+* LDAP
+
+We already provided Helm-chart for James and RabbitMQ in:
+
+https://github.com/apache/james-project/tree/master/server/apps/distributed-app/helm-chart
+
+See the James helm-chart
link:https://github.com/apache/james-project/tree/master/server/apps/distributed-app/helm-chart/james/doc/[documents]
for details about components, configurations and variables.
Review comment:
INO the doc should be in a single place: here ;-)
##########
File path: server/apps/distributed-app/helm-chart/james/configs/imapserver.xml
##########
@@ -0,0 +1,44 @@
+<?xml version="1.0"?>
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+<!--
+http://james.apache.org/server/config-imap4.html
+-->
+<imapservers>
+ <imapserver enabled="${env:OP_JAMES_IMAP_SMTP_ENABLED}">
+ <jmxName>imapserver-ssl</jmxName>
+ <bind>0.0.0.0:993</bind>
+ <connectionBacklog>200</connectionBacklog>
+ <tls socketTLS="true" startTLS="false">
+ <privateKey>file://secrets/tls.key</privateKey>
+ <certificates>file://secrets/tls.crt</certificates>
Review comment:
Indent is crap
##########
File path: server/apps/distributed-app/helm-chart/james/doc/misc/secrets.md
##########
@@ -0,0 +1,66 @@
+# Secrets for James Helm template
+
+Helm James has many secret values to configure before being able to use it
correctly.
+Those secrets are linked to the template through their declaration in
`james-secrets.yaml`.
+A sample with dumb secrets can be found in the `secrets.sample.yaml` file.
+
+This document presents a quick description of each secret field.
+
+For more information about encrypt/descrypt secret with Helm, see:
https://github.com/jkroepke/helm-secrets
+
+## James secret values
+
+| Field | Mandatory or optional | Description |
+| :--------------------- | :-------------------- | :----------- |
+| jwtPublicKey | Mandatory | The JWT public key used for
JWT tokens.|
+| jwtPrivateKey | Mandatory | The JWT private key used
for JWT tokens.|
+| cassandraUser | Mandatory | The user for Cassandra
access |
+| cassandraPassword | Mandatory | The password fort Cassandra
access |
+| esUser | Mandatory | The user for Elasticsearch
access |
+| esPassword | Mandatory | The password for
Elasticsearch access |
+| keystorePassword | Mandatory | The password of the
keystore used for generating JWT keystore |
+| dkimPrivateKey | Mandatory | The private key for the
generated DKIM |
Review comment:
This is optional and dependens on mailetcontainer.xml
##########
File path: server/apps/distributed-app/helm-chart/james/doc/misc/secrets.md
##########
@@ -0,0 +1,66 @@
+# Secrets for James Helm template
+
+Helm James has many secret values to configure before being able to use it
correctly.
+Those secrets are linked to the template through their declaration in
`james-secrets.yaml`.
+A sample with dumb secrets can be found in the `secrets.sample.yaml` file.
+
+This document presents a quick description of each secret field.
+
+For more information about encrypt/descrypt secret with Helm, see:
https://github.com/jkroepke/helm-secrets
+
+## James secret values
+
+| Field | Mandatory or optional | Description |
+| :--------------------- | :-------------------- | :----------- |
+| jwtPublicKey | Mandatory | The JWT public key used for
JWT tokens.|
+| jwtPrivateKey | Mandatory | The JWT private key used
for JWT tokens.|
+| cassandraUser | Mandatory | The user for Cassandra
access |
+| cassandraPassword | Mandatory | The password fort Cassandra
access |
+| esUser | Mandatory | The user for Elasticsearch
access |
+| esPassword | Mandatory | The password for
Elasticsearch access |
+| keystorePassword | Mandatory | The password of the
keystore used for generating JWT keystore |
+| dkimPrivateKey | Mandatory | The private key for the
generated DKIM |
+| keystoreJWTinBase64 | Mandatory | The JWT keystore (encoded
in base64) |
+| remoteDeliveryUsername | Optional | The user for outgoing SMTP
through remote delivery gateway (if configured) |
+| remoteDeliveryPassword | Optional | The password for outgoing
SMTP through remote delivery gateway (if configured) |
Review comment:
Idem this depends of the content of mailetcontainer.xml.
I bet we can remove it.
##########
File path: server/apps/distributed-app/helm-chart/james/doc/misc/checklist.md
##########
@@ -0,0 +1,54 @@
+# Checklist
+
+There is some things that you must have before being able to run James on your
Kubernetes Cluster.
+
+## MUST have
+
+Those are absolutely necessary if you want at least James to start without
crashing.
+
+- [ ] Create a namespace for James deployment.
+- [ ] The right image container used for James
+- [ ] Have an appropriate email domain (Contact your Domain Service Provider)
+- [ ] Deploy Helm RabbitMQ package
+- [ ] Cassandra nodes deployed
+- [ ] Put valid Cassandra keyspaces in the configuration
+- [ ] ElasticSearch nodes deployed
+- [ ] LDAP deployed
+- [ ] S3 deployed
+- [ ] Configure JWT keys, keystores, SPF and DKIM. See document
[here](https://james.apache.org/howTo/)
+- [ ] Adding the proper `mailetcontainer.xml` file with other conf files in
James Helm package
+
+With those you should be able to startup James for first tests without having
it crashing. However it's still not enough to have it fully functional.
+
+## should have
+
+- [ ] Postifx-in deployed in emaily for delivering SMTP messages to James
(avoiding exposing the port 25 for James externally)
+- [ ] Make sure the JWT public key you use is the one defined by ESN
+- [ ] DNS registered entries for JMAP, IMAP, SMTP, Webadmin
+- [ ] change James admin ID to a user we can use for imapsync (to check with
the person responsible of the imapsync)
+- [ ] enabling Ingress routes (if you enabled Ingress before, you need to
disable it to use Ingress routes)
+ - [ ] enabled: true
+ - [ ] create the secret for TLS certificate used by Ingress
+ - [ ] preprod: Usually, we use
[cert-manager](https://github.com/jetstack/cert-manager) to request a Let's
Encrypt Certificate
+ Remember to use the full chain certificate. You can verify that your
certificate works well with `openssl` before adding
+ the name of that secret to your James values.
+ - [ ] prod: Some certificates belonging to the client should exist
already. See that with the team.
+- [ ] Remote delivery
+ - [ ] Get the data regarding the gateway (Postfix usually) address and the
port being used
+ - [ ] See if you need credentials or not.
+ - [ ] Add the allowed IPs for relay concerning outgoing emails (usually
localhost, postfix, other James pods)
+ - [ ] Work closely with the person with rights on the gateway to configure
it correctly for in and outgoing emails
+- [ ] IMAP ports are being defined as NodePort because the load balancer from
OVH has an idle timeout way too short
+and we need longer connections for performance reasons (and the RFC fior IMAP
states also we need 30 minutes for the timeout)
+ - [ ] Ask for a VM where you can install HA proxy (will act as LB for IMAP)
+ - [ ] Install HA proxy
+ - [ ] Redirect traffic towards James nodes IPs and ports
Review comment:
Mentiions to OVH / HAProxy do not make sense in the context of the
Apache project.
##########
File path: server/apps/distributed-app/helm-chart/james/doc/misc/checklist.md
##########
@@ -0,0 +1,54 @@
+# Checklist
+
+There is some things that you must have before being able to run James on your
Kubernetes Cluster.
+
+## MUST have
+
+Those are absolutely necessary if you want at least James to start without
crashing.
+
+- [ ] Create a namespace for James deployment.
+- [ ] The right image container used for James
+- [ ] Have an appropriate email domain (Contact your Domain Service Provider)
+- [ ] Deploy Helm RabbitMQ package
+- [ ] Cassandra nodes deployed
+- [ ] Put valid Cassandra keyspaces in the configuration
+- [ ] ElasticSearch nodes deployed
+- [ ] LDAP deployed
+- [ ] S3 deployed
+- [ ] Configure JWT keys, keystores, SPF and DKIM. See document
[here](https://james.apache.org/howTo/)
+- [ ] Adding the proper `mailetcontainer.xml` file with other conf files in
James Helm package
+
+With those you should be able to startup James for first tests without having
it crashing. However it's still not enough to have it fully functional.
+
+## should have
+
+- [ ] Postifx-in deployed in emaily for delivering SMTP messages to James
(avoiding exposing the port 25 for James externally)
+- [ ] Make sure the JWT public key you use is the one defined by ESN
+- [ ] DNS registered entries for JMAP, IMAP, SMTP, Webadmin
+- [ ] change James admin ID to a user we can use for imapsync (to check with
the person responsible of the imapsync)
+- [ ] enabling Ingress routes (if you enabled Ingress before, you need to
disable it to use Ingress routes)
+ - [ ] enabled: true
+ - [ ] create the secret for TLS certificate used by Ingress
+ - [ ] preprod: Usually, we use
[cert-manager](https://github.com/jetstack/cert-manager) to request a Let's
Encrypt Certificate
+ Remember to use the full chain certificate. You can verify that your
certificate works well with `openssl` before adding
+ the name of that secret to your James values.
+ - [ ] prod: Some certificates belonging to the client should exist
already. See that with the team.
+- [ ] Remote delivery
+ - [ ] Get the data regarding the gateway (Postfix usually) address and the
port being used
+ - [ ] See if you need credentials or not.
+ - [ ] Add the allowed IPs for relay concerning outgoing emails (usually
localhost, postfix, other James pods)
+ - [ ] Work closely with the person with rights on the gateway to configure
it correctly for in and outgoing emails
Review comment:
Let's remove this too
##########
File path:
server/apps/distributed-app/docs/modules/ROOT/pages/run-kubernetes.adoc
##########
@@ -0,0 +1,46 @@
+= Distributed James Server — Run with Kubernetes
+:navtitle: Run with Kubernetes
+
+== Deploy Apache James on Kubernetes with Helm chart.
+
+This chart bootstraps a James mail server on a
link:http://kubernetes.io[Kubernetes]
+cluster using the link:https://helm.sh[Helm] package manager.
+
+Before runnning the Helm chart, James should have particularly access to the
following external products:
+
+* Cassandra
+* ElastiSearch
+* RabbitMQ
+* LDAP
Review comment:
If there are standard Helm charts we can use can we put a link to them?
##########
File path: server/apps/distributed-app/helm-chart/james/configs/logback.xml
##########
@@ -0,0 +1,42 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<configuration scan="true" scanPeriod="30 seconds">
+
+ <contextListener
class="ch.qos.logback.classic.jul.LevelChangePropagator">
+ <resetJUL>true</resetJUL>
+ </contextListener>
+
+ <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
+ <encoder
class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
+ <layout
class="ch.qos.logback.contrib.json.classic.JsonLayout">
+
<timestampFormat>yyyy-MM-dd'T'HH:mm:ss.SSSX</timestampFormat>
+
<timestampFormatTimezoneId>Etc/UTC</timestampFormatTimezoneId>
+
+ <!-- Importance for handling multiple lines log -->
+ <appendLineSeparator>true</appendLineSeparator>
+
+ <jsonFormatter
class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
+ <prettyPrint>false</prettyPrint>
+ </jsonFormatter>
+ </layout>
+ </encoder>
+ </appender>
+
+ <root level="INFO">
+ <appender-ref ref="CONSOLE" />
+ </root>
+
+ <logger name="http.jmap" level="INFO" />
+<!--
+ <logger name="org.apache.james.smtpserver" level="DEBUG" />
+ <logger name="org.apache.mailet.base.MailetPipelineLogging"
level="DEBUG" />
+ <logger name="org.apache.james.mailetcontainer" level="DEBUG" />
+ <logger name="com.datastax.driver.core.QueryLogger.SLOW" level="DEBUG"
/>
+ <logger name="org.apache.james.mailetcontainer" level="DEBUG" />
+ <logger name="org.apache.james" level="DEBUG" />
+ <logger name="org.apache.james.backends.es" level="INFO" />
+ <logger name="com.datastax.driver.core.QueryLogger.SLOW" level="DEBUG"
/>
+ <logger name="com.datastax.driver.core.QueryLogger.ERROR"
level="TRACE" />
+ <logger name="http.jmap" level="DEBUG" />
+-->
Review comment:
Remove comments
##########
File path:
server/apps/distributed-app/helm-chart/james/configs/usersrepository.xml
##########
@@ -0,0 +1,44 @@
+<?xml version="1.0"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+ -->
+
+<usersrepository name="LocalUsers"
+ class="org.apache.james.user.ldap.ReadOnlyUsersLDAPRepository"
+ ldapHost="${env:OP_JAMES_LDAP_URI}"
+ userBase="${env:OP_JAMES_LDAP_BASE_DN}"
+ principal="${env:OP_JAMES_LDAP_PRINCIPAL_DN}"
+ credentials="${env:OP_JAMES_LDAP_BIND_PW}"
+ userIdAttribute="mail"
+ userObjectClass="person"
+ administratorId="${env:OP_JAMES_ADMINISTRATOR_ID}" >
+ <supportsVirtualHosting>true</supportsVirtualHosting>
+</usersrepository>
+
+<!-- <usersrepository name="LocalUsers"
+ class="org.apache.james.user.ldap.CombinedUsersRepository"
Review comment:
This repository do not exist.
Remove the comment?
##########
File path: server/apps/distributed-app/helm-chart/james/doc/misc/checklist.md
##########
@@ -0,0 +1,54 @@
+# Checklist
+
+There is some things that you must have before being able to run James on your
Kubernetes Cluster.
+
+## MUST have
+
+Those are absolutely necessary if you want at least James to start without
crashing.
+
+- [ ] Create a namespace for James deployment.
+- [ ] The right image container used for James
+- [ ] Have an appropriate email domain (Contact your Domain Service Provider)
+- [ ] Deploy Helm RabbitMQ package
+- [ ] Cassandra nodes deployed
+- [ ] Put valid Cassandra keyspaces in the configuration
+- [ ] ElasticSearch nodes deployed
+- [ ] LDAP deployed
+- [ ] S3 deployed
+- [ ] Configure JWT keys, keystores, SPF and DKIM. See document
[here](https://james.apache.org/howTo/)
+- [ ] Adding the proper `mailetcontainer.xml` file with other conf files in
James Helm package
+
+With those you should be able to startup James for first tests without having
it crashing. However it's still not enough to have it fully functional.
+
+## should have
+
+- [ ] Postifx-in deployed in emaily for delivering SMTP messages to James
(avoiding exposing the port 25 for James externally)
+- [ ] Make sure the JWT public key you use is the one defined by ESN
Review comment:
Idem no ESN here
##########
File path: server/apps/distributed-app/helm-chart/james/doc/misc/secrets.md
##########
@@ -0,0 +1,66 @@
+# Secrets for James Helm template
+
+Helm James has many secret values to configure before being able to use it
correctly.
+Those secrets are linked to the template through their declaration in
`james-secrets.yaml`.
+A sample with dumb secrets can be found in the `secrets.sample.yaml` file.
+
+This document presents a quick description of each secret field.
+
+For more information about encrypt/descrypt secret with Helm, see:
https://github.com/jkroepke/helm-secrets
+
+## James secret values
+
+| Field | Mandatory or optional | Description |
+| :--------------------- | :-------------------- | :----------- |
+| jwtPublicKey | Mandatory | The JWT public key used for
JWT tokens.|
+| jwtPrivateKey | Mandatory | The JWT private key used
for JWT tokens.|
+| cassandraUser | Mandatory | The user for Cassandra
access |
+| cassandraPassword | Mandatory | The password fort Cassandra
access |
+| esUser | Mandatory | The user for Elasticsearch
access |
+| esPassword | Mandatory | The password for
Elasticsearch access |
Review comment:
This should not be mandatory as community ES users do not have auth...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
