[GitHub] [lucene-solr-operator] thelabdude commented on a change in pull request #193: Support custom log4j2 config from user-provided ConfigMap
thelabdude commented on a change in pull request #193:
URL:
https://github.com/apache/lucene-solr-operator/pull/193#discussion_r566340100
##
File path: controllers/solrcloud_controller.go
##
@@ -182,44 +182,61 @@ func (r *SolrCloudReconciler) Reconcile(req ctrl.Request)
(ctrl.Result, error) {
}
}
- // Generate ConfigMap unless the user supplied a custom ConfigMap for
solr.xml ... but the provided ConfigMap
- // might be for the Prometheus exporter, so we only care if they
provide a solr.xml in the CM
- solrXmlConfigMapName := instance.ConfigMapName()
- solrXmlMd5 := ""
+ // Generate ConfigMap unless the user supplied a custom ConfigMap for
solr.xml
+ configMapInfo := make(map[string]string)
if instance.Spec.CustomSolrKubeOptions.ConfigMapOptions != nil &&
instance.Spec.CustomSolrKubeOptions.ConfigMapOptions.ProvidedConfigMap != "" {
+ providedConfigMapName :=
instance.Spec.CustomSolrKubeOptions.ConfigMapOptions.ProvidedConfigMap
foundConfigMap := {}
- nn := types.NamespacedName{Name:
instance.Spec.CustomSolrKubeOptions.ConfigMapOptions.ProvidedConfigMap,
Namespace: instance.Namespace}
+ nn := types.NamespacedName{Name: providedConfigMapName,
Namespace: instance.Namespace}
err = r.Get(context.TODO(), nn, foundConfigMap)
if err != nil {
return requeueOrNot, err // if they passed a
providedConfigMap name, then it must exist
}
- // ConfigMap doesn't have to have a solr.xml, but if it does,
then it needs to be valid!
if foundConfigMap.Data != nil {
- solrXml, ok := foundConfigMap.Data["solr.xml"]
- if ok {
+ logXml, hasLogXml :=
foundConfigMap.Data[util.LogXmlFile]
+ solrXml, hasSolrXml :=
foundConfigMap.Data[util.SolrXmlFile]
+
+ // if there's a user-provided config, it must have one
of the expected keys
+ if !hasLogXml && !hasSolrXml {
+ return requeueOrNot, fmt.Errorf("User provided
ConfigMap %s must have one of 'solr.xml' and/or 'log4j2.xml'",
Review comment:
Can you point me to an example elsewhere in the code for what you're
looking for here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr-operator] thelabdude commented on a change in pull request #193: Support custom log4j2 config from user-provided ConfigMap
thelabdude commented on a change in pull request #193:
URL:
https://github.com/apache/lucene-solr-operator/pull/193#discussion_r566236753
##
File path: controllers/suite_test.go
##
@@ -76,6 +78,9 @@ func SetupTestReconcile(inner reconcile.Reconciler)
(reconcile.Reconciler, chan
requests := make(chan reconcile.Request)
fn := reconcile.Func(func(req reconcile.Request) (reconcile.Result,
error) {
result, err := inner.Reconcile(req)
+ if err != nil {
+ fmt.Printf("\n\nReconcile Error: %s\n\n", err)
Review comment:
this is a bit ugly but so useful in tracking down errors during
reconcile that are returned like this:
```
if !strings.Contains(solrXml, "${hostPort:") {
return requeueOrNot,
fmt.Errorf("Custom solr.xml in
ConfigMap %s must contain a placeholder for the 'hostPort' variable, such as
${hostPort:80}",
providedConfigMapName)
}
```
W/o this change, the error isn't reported and the test just fails with
reconcile timeouts. I think it's a useful addition.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr-operator] thelabdude commented on a change in pull request #193: Support custom log4j2 config from user-provided ConfigMap
thelabdude commented on a change in pull request #193:
URL:
https://github.com/apache/lucene-solr-operator/pull/193#discussion_r566235409
##
File path: controllers/util/solr_util.go
##
@@ -327,13 +330,42 @@ func GenerateStatefulSet(solrCloud *solr.SolrCloud,
solrCloudStatus *solr.SolrCl
envVars = append(envVars, customPodOptions.EnvVariables...)
}
+ // Did the user provide a custom log config?
+ if configMapInfo[LogXmlFile] != "" {
+
+ if configMapInfo[LogXmlMd5Annotation] != "" {
+ if podAnnotations == nil {
+ podAnnotations = make(map[string]string, 1)
+ }
+ podAnnotations[LogXmlMd5Annotation] =
configMapInfo[LogXmlMd5Annotation]
+ }
+
+ // cannot use /var/solr as a mountPath, so mount the custom log
config in a sub-dir
+ volName := "log4j2-xml"
+ mountPath := fmt.Sprintf("/var/solr/%s-log-config",
solrCloud.Name)
+ log4jPropsEnvVarPath := fmt.Sprintf("%s/%s", mountPath,
LogXmlFile)
+
+ solrVolumes = append(solrVolumes, corev1.Volume{
Review comment:
Latest commit addresses this issue.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr-operator] thelabdude commented on a change in pull request #193: Support custom log4j2 config from user-provided ConfigMap
thelabdude commented on a change in pull request #193:
URL:
https://github.com/apache/lucene-solr-operator/pull/193#discussion_r566235249
##
File path: controllers/solrcloud_controller.go
##
@@ -182,44 +182,61 @@ func (r *SolrCloudReconciler) Reconcile(req ctrl.Request)
(ctrl.Result, error) {
}
}
- // Generate ConfigMap unless the user supplied a custom ConfigMap for
solr.xml ... but the provided ConfigMap
- // might be for the Prometheus exporter, so we only care if they
provide a solr.xml in the CM
- solrXmlConfigMapName := instance.ConfigMapName()
- solrXmlMd5 := ""
+ // Generate ConfigMap unless the user supplied a custom ConfigMap for
solr.xml
+ configMapInfo := make(map[string]string)
if instance.Spec.CustomSolrKubeOptions.ConfigMapOptions != nil &&
instance.Spec.CustomSolrKubeOptions.ConfigMapOptions.ProvidedConfigMap != "" {
+ providedConfigMapName :=
instance.Spec.CustomSolrKubeOptions.ConfigMapOptions.ProvidedConfigMap
foundConfigMap := {}
- nn := types.NamespacedName{Name:
instance.Spec.CustomSolrKubeOptions.ConfigMapOptions.ProvidedConfigMap,
Namespace: instance.Namespace}
+ nn := types.NamespacedName{Name: providedConfigMapName,
Namespace: instance.Namespace}
err = r.Get(context.TODO(), nn, foundConfigMap)
if err != nil {
return requeueOrNot, err // if they passed a
providedConfigMap name, then it must exist
}
- // ConfigMap doesn't have to have a solr.xml, but if it does,
then it needs to be valid!
if foundConfigMap.Data != nil {
- solrXml, ok := foundConfigMap.Data["solr.xml"]
- if ok {
+ logXml, hasLogXml :=
foundConfigMap.Data[util.LogXmlFile]
+ solrXml, hasSolrXml :=
foundConfigMap.Data[util.SolrXmlFile]
+
+ // if there's a user-provided config, it must have one
of the expected keys
+ if !hasLogXml && !hasSolrXml {
+ return requeueOrNot, fmt.Errorf("User provided
ConfigMap %s must have one of 'solr.xml' and/or 'log4j2.xml'",
+ providedConfigMapName)
+ }
+
+ if hasSolrXml {
+ // make sure the user-provided solr.xml is valid
if !strings.Contains(solrXml, "${hostPort:") {
return requeueOrNot,
fmt.Errorf("Custom solr.xml in
ConfigMap %s must contain a placeholder for the 'hostPort' variable, such as
${hostPort:80}",
-
instance.Spec.CustomSolrKubeOptions.ConfigMapOptions.ProvidedConfigMap)
+ providedConfigMapName)
}
// stored in the pod spec annotations on the
statefulset so that we get a restart when solr.xml changes
- solrXmlMd5 = fmt.Sprintf("%x",
md5.Sum([]byte(solrXml)))
- solrXmlConfigMapName = foundConfigMap.Name
- } else {
- return requeueOrNot, fmt.Errorf("Required
'solr.xml' key not found in provided ConfigMap %s",
-
instance.Spec.CustomSolrKubeOptions.ConfigMapOptions.ProvidedConfigMap)
+ configMapInfo[util.SolrXmlMd5Annotation] =
fmt.Sprintf("%x", md5.Sum([]byte(solrXml)))
+ configMapInfo[util.SolrXmlFile] =
foundConfigMap.Name
+ }
+
+ if hasLogXml {
+ if !strings.Contains(logXml,
"monitorInterval=") {
+ // stored in the pod spec annotations
on the statefulset so that we get a restart when the log config changes
+ configMapInfo[util.LogXmlMd5Annotation]
= fmt.Sprintf("%x", md5.Sum([]byte(logXml)))
+ } // else log4j will automatically refresh for
us, so no restart needed
+ configMapInfo[util.LogXmlFile] =
foundConfigMap.Name
}
+
} else {
- return requeueOrNot, fmt.Errorf("Provided ConfigMap %s
has no data",
-
instance.Spec.CustomSolrKubeOptions.ConfigMapOptions.ProvidedConfigMap)
+ return requeueOrNot, fmt.Errorf("Provided ConfigMap %s
has no data", providedConfigMapName)
}
- } else {
+ }
+
+ if configMapInfo[util.SolrXmlFile] == "" {
Review comment:
ok cool ... I switched to
[GitHub] [lucene-solr-operator] thelabdude commented on a change in pull request #193: Support custom log4j2 config from user-provided ConfigMap
thelabdude commented on a change in pull request #193:
URL:
https://github.com/apache/lucene-solr-operator/pull/193#discussion_r566184234
##
File path: controllers/solrcloud_controller.go
##
@@ -182,44 +182,61 @@ func (r *SolrCloudReconciler) Reconcile(req ctrl.Request)
(ctrl.Result, error) {
}
}
- // Generate ConfigMap unless the user supplied a custom ConfigMap for
solr.xml ... but the provided ConfigMap
- // might be for the Prometheus exporter, so we only care if they
provide a solr.xml in the CM
- solrXmlConfigMapName := instance.ConfigMapName()
- solrXmlMd5 := ""
+ // Generate ConfigMap unless the user supplied a custom ConfigMap for
solr.xml
+ configMapInfo := make(map[string]string)
if instance.Spec.CustomSolrKubeOptions.ConfigMapOptions != nil &&
instance.Spec.CustomSolrKubeOptions.ConfigMapOptions.ProvidedConfigMap != "" {
+ providedConfigMapName :=
instance.Spec.CustomSolrKubeOptions.ConfigMapOptions.ProvidedConfigMap
foundConfigMap := {}
- nn := types.NamespacedName{Name:
instance.Spec.CustomSolrKubeOptions.ConfigMapOptions.ProvidedConfigMap,
Namespace: instance.Namespace}
+ nn := types.NamespacedName{Name: providedConfigMapName,
Namespace: instance.Namespace}
err = r.Get(context.TODO(), nn, foundConfigMap)
if err != nil {
return requeueOrNot, err // if they passed a
providedConfigMap name, then it must exist
}
- // ConfigMap doesn't have to have a solr.xml, but if it does,
then it needs to be valid!
if foundConfigMap.Data != nil {
- solrXml, ok := foundConfigMap.Data["solr.xml"]
- if ok {
+ logXml, hasLogXml :=
foundConfigMap.Data[util.LogXmlFile]
+ solrXml, hasSolrXml :=
foundConfigMap.Data[util.SolrXmlFile]
+
+ // if there's a user-provided config, it must have one
of the expected keys
+ if !hasLogXml && !hasSolrXml {
+ return requeueOrNot, fmt.Errorf("User provided
ConfigMap %s must have one of 'solr.xml' and/or 'log4j2.xml'",
+ providedConfigMapName)
+ }
+
+ if hasSolrXml {
+ // make sure the user-provided solr.xml is valid
if !strings.Contains(solrXml, "${hostPort:") {
return requeueOrNot,
fmt.Errorf("Custom solr.xml in
ConfigMap %s must contain a placeholder for the 'hostPort' variable, such as
${hostPort:80}",
-
instance.Spec.CustomSolrKubeOptions.ConfigMapOptions.ProvidedConfigMap)
+ providedConfigMapName)
}
// stored in the pod spec annotations on the
statefulset so that we get a restart when solr.xml changes
- solrXmlMd5 = fmt.Sprintf("%x",
md5.Sum([]byte(solrXml)))
- solrXmlConfigMapName = foundConfigMap.Name
- } else {
- return requeueOrNot, fmt.Errorf("Required
'solr.xml' key not found in provided ConfigMap %s",
-
instance.Spec.CustomSolrKubeOptions.ConfigMapOptions.ProvidedConfigMap)
+ configMapInfo[util.SolrXmlMd5Annotation] =
fmt.Sprintf("%x", md5.Sum([]byte(solrXml)))
+ configMapInfo[util.SolrXmlFile] =
foundConfigMap.Name
+ }
+
+ if hasLogXml {
+ if !strings.Contains(logXml,
"monitorInterval=") {
+ // stored in the pod spec annotations
on the statefulset so that we get a restart when the log config changes
+ configMapInfo[util.LogXmlMd5Annotation]
= fmt.Sprintf("%x", md5.Sum([]byte(logXml)))
+ } // else log4j will automatically refresh for
us, so no restart needed
+ configMapInfo[util.LogXmlFile] =
foundConfigMap.Name
}
+
} else {
- return requeueOrNot, fmt.Errorf("Provided ConfigMap %s
has no data",
-
instance.Spec.CustomSolrKubeOptions.ConfigMapOptions.ProvidedConfigMap)
+ return requeueOrNot, fmt.Errorf("Provided ConfigMap %s
has no data", providedConfigMapName)
}
- } else {
+ }
+
+ if configMapInfo[util.SolrXmlFile] == "" {
Review comment:
If the user-provided ConfigMap
[GitHub] [lucene-solr-operator] thelabdude commented on a change in pull request #193: Support custom log4j2 config from user-provided ConfigMap
thelabdude commented on a change in pull request #193:
URL:
https://github.com/apache/lucene-solr-operator/pull/193#discussion_r564072280
##
File path: controllers/util/solr_util.go
##
@@ -327,13 +330,42 @@ func GenerateStatefulSet(solrCloud *solr.SolrCloud,
solrCloudStatus *solr.SolrCl
envVars = append(envVars, customPodOptions.EnvVariables...)
}
+ // Did the user provide a custom log config?
+ if configMapInfo[LogXmlFile] != "" {
+
+ if configMapInfo[LogXmlMd5Annotation] != "" {
+ if podAnnotations == nil {
+ podAnnotations = make(map[string]string, 1)
+ }
+ podAnnotations[LogXmlMd5Annotation] =
configMapInfo[LogXmlMd5Annotation]
+ }
+
+ // cannot use /var/solr as a mountPath, so mount the custom log
config in a sub-dir
+ volName := "log4j2-xml"
+ mountPath := fmt.Sprintf("/var/solr/%s-log-config",
solrCloud.Name)
+ log4jPropsEnvVarPath := fmt.Sprintf("%s/%s", mountPath,
LogXmlFile)
+
+ solrVolumes = append(solrVolumes, corev1.Volume{
Review comment:
This is a good catch! K8s allows it and results in a structure like the
following in the STS:
```
volumes:
- configMap:
defaultMode: 420
items:
- key: solr.xml
path: solr.xml
name: dev-custom-solr-xml
name: solr-xml
- configMap:
defaultMode: 420
items:
- key: log4j2.xml
path: log4j2.xml
name: dev-custom-solr-xml
name: log4j2-xml
```
But we certainly should use a single volume with multiple `items` as you
suggest ;-) Will fix it up and add a test for both keys being provided.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr-operator] thelabdude commented on a change in pull request #193: Support custom log4j2 config from user-provided ConfigMap
thelabdude commented on a change in pull request #193:
URL:
https://github.com/apache/lucene-solr-operator/pull/193#discussion_r564072280
##
File path: controllers/util/solr_util.go
##
@@ -327,13 +330,42 @@ func GenerateStatefulSet(solrCloud *solr.SolrCloud,
solrCloudStatus *solr.SolrCl
envVars = append(envVars, customPodOptions.EnvVariables...)
}
+ // Did the user provide a custom log config?
+ if configMapInfo[LogXmlFile] != "" {
+
+ if configMapInfo[LogXmlMd5Annotation] != "" {
+ if podAnnotations == nil {
+ podAnnotations = make(map[string]string, 1)
+ }
+ podAnnotations[LogXmlMd5Annotation] =
configMapInfo[LogXmlMd5Annotation]
+ }
+
+ // cannot use /var/solr as a mountPath, so mount the custom log
config in a sub-dir
+ volName := "log4j2-xml"
+ mountPath := fmt.Sprintf("/var/solr/%s-log-config",
solrCloud.Name)
+ log4jPropsEnvVarPath := fmt.Sprintf("%s/%s", mountPath,
LogXmlFile)
+
+ solrVolumes = append(solrVolumes, corev1.Volume{
Review comment:
This is a good catch! K8s allows it and results in a structure like the
following in the STS:
```
volumes:
- configMap:
defaultMode: 420
items:
- key: solr.xml
path: solr.xml
name: dev-custom-solr-xml
name: solr-xml
- configMap:
defaultMode: 420
items:
- key: log4j2.xml
path: log4j2.xml
name: dev-custom-solr-xml
name: log4j2-xml
```
But we certainly should use a single volume with multiple `items` as you
suggest ;-) Will fix it up and add a test for both keys being provided.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
