Re: Elasticsearch - 5.x
On 3/21/19 2:31 PM, Neale Ferguson wrote:
Are you asking how to erase everything and start over?
- No, just how to get information out of the elasticsearch container to tell me
why it's failing.
Can you share your inventory files with the logging parameters (be sure to redact any sensitive information)?
- The configuration was an all-in-one when it was first created. I added a
compute node shortly after then added the logging.
I don't see anything obviously wrong. I guess I'd have to view the detailed
ansible log (e.g. ansible-playbook -vvv) to see what went wrong.
[nfs]
okcd-master.sinenomine.net
# Create an OSEv3 group that contains the masters, nodes, and etcd groups
[OSEv3:children]
masters
nodes
etcd
nfs
new_nodes
# Set variables common for all OSEv3 hosts
[OSEv3:vars]
openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true',
'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider',}]
openshift_master_htpasswd_users={}
os_firewall_use_firewalld=True
openshift_logging_install_logging=true
openshift_logging_storage_kind=nfs
openshift_logging_storage_access_modes=['ReadWriteOnce']
openshift_logging_storage_nfs_directory=/exports
openshift_logging_storage_nfs_options='*(rw,root_squash)'
openshift_logging_storage_volume_name=logging
openshift_logging_storage_volume_size=200Mi
openshift_logging_storage_labels={'storage': 'logging'}
openshift_logging_kibana_hostname=logging.origin.z
openshift_logging_es_nodeselector={'node-role.kubernetes.io/infra': 'true'}
ansible_ssh_user=root
openshift_deployment_type=origin
oreg_url=docker.io/clefos/origin-${component}:${version}
openshift_examples_modify_imagestreams=true
openshift_web_console_prefix=docker.io/clefos/
openshift_disable_check=disk_availability,docker_storage,memory_availability
openshift_hosted_router_selector='node-role.kubernetes.io/infra=true'
openshift_hosted_manage_registry=true
openshift_enable_unsupported_configurations=True
openshift_hosted_registry_storage_kind=nfs
openshift_hosted_registry_storage_nfs_directory=/exports
openshift_hosted_registry_storage_nfs_options='*(rw,root_squash)'
openshift_hosted_registry_storage_volume_name=registry
openshift_hosted_registry_storage_volume_size=5Gi
openshift_cockpit_deployer_image=docker.io/clefos/cockpit-kubernetes:latest
openshift_console_install=False
# host group for masters
[masters]
okcd-master.sinenomine.net
# host group for etcd
[etcd]
master.example.com
# host group for nodes, includes region info
[nodes]
master.example.com openshift_node_group_name='node-config-all-in-one'
node.example.com openshift_node_group_name='node-config-compute'
# Adding new node to the cluster
[new_nodes]
___
dev mailing list
dev@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
___
dev mailing list
dev@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
Re: Elasticsearch - 5.x
Are you asking how to erase everything and start over?
- No, just how to get information out of the elasticsearch container to tell me
why it's failing.
Can you share your inventory files with the logging parameters (be sure to
redact any sensitive information)?
- The configuration was an all-in-one when it was first created. I added a
compute node shortly after then added the logging.
[nfs]
okcd-master.sinenomine.net
# Create an OSEv3 group that contains the masters, nodes, and etcd groups
[OSEv3:children]
masters
nodes
etcd
nfs
new_nodes
# Set variables common for all OSEv3 hosts
[OSEv3:vars]
openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true',
'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider',}]
openshift_master_htpasswd_users={}
os_firewall_use_firewalld=True
openshift_logging_install_logging=true
openshift_logging_storage_kind=nfs
openshift_logging_storage_access_modes=['ReadWriteOnce']
openshift_logging_storage_nfs_directory=/exports
openshift_logging_storage_nfs_options='*(rw,root_squash)'
openshift_logging_storage_volume_name=logging
openshift_logging_storage_volume_size=200Mi
openshift_logging_storage_labels={'storage': 'logging'}
openshift_logging_kibana_hostname=logging.origin.z
openshift_logging_es_nodeselector={'node-role.kubernetes.io/infra': 'true'}
ansible_ssh_user=root
openshift_deployment_type=origin
oreg_url=docker.io/clefos/origin-${component}:${version}
openshift_examples_modify_imagestreams=true
openshift_web_console_prefix=docker.io/clefos/
openshift_disable_check=disk_availability,docker_storage,memory_availability
openshift_hosted_router_selector='node-role.kubernetes.io/infra=true'
openshift_hosted_manage_registry=true
openshift_enable_unsupported_configurations=True
openshift_hosted_registry_storage_kind=nfs
openshift_hosted_registry_storage_nfs_directory=/exports
openshift_hosted_registry_storage_nfs_options='*(rw,root_squash)'
openshift_hosted_registry_storage_volume_name=registry
openshift_hosted_registry_storage_volume_size=5Gi
openshift_cockpit_deployer_image=docker.io/clefos/cockpit-kubernetes:latest
openshift_console_install=False
# host group for masters
[masters]
okcd-master.sinenomine.net
# host group for etcd
[etcd]
master.example.com
# host group for nodes, includes region info
[nodes]
master.example.com openshift_node_group_name='node-config-all-in-one'
node.example.com openshift_node_group_name='node-config-compute'
# Adding new node to the cluster
[new_nodes]
___
dev mailing list
dev@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
Re: Elasticsearch - 5.x
On 3/21/19 1:56 PM, Neale Ferguson wrote: > - So this leads to the question of how to debug the elasticsearch container not coming up. The console log information in the 1st email came from me running it manually so I could add the -v option to the elasticsearch command. Setting the DEBUG and LOGLEVEL environment variables wasn't illuminating. I guess I need to try and add the -v option to the run.sh script. This simply should not be happening. Did you use openshift-ansible to upgrade from 3.10 to 3.11, or did you deploy logging from scratch on 3.11? - Initially, I created the 3.11 cluster from scratch but without logging. I then added the logging statements to the hosts file and then ran the logging playbook. Are you asking how to erase everything and start over? Can you share your inventory files with the logging parameters (be sure to redact any sensitive information)? ___ dev mailing list dev@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/dev ___ dev mailing list dev@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
Re: Elasticsearch - 5.x
> - So this leads to the question of how to debug the elasticsearch container not coming up. The console log information in the 1st email came from me running it manually so I could add the -v option to the elasticsearch command. Setting the DEBUG and LOGLEVEL environment variables wasn't illuminating. I guess I need to try and add the -v option to the run.sh script. This simply should not be happening. Did you use openshift-ansible to upgrade from 3.10 to 3.11, or did you deploy logging from scratch on 3.11? - Initially, I created the 3.11 cluster from scratch but without logging. I then added the logging statements to the hosts file and then ran the logging playbook. ___ dev mailing list dev@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
Re: Elasticsearch - 5.x
On 3/21/19 1:41 PM, Neale Ferguson wrote: elasticsearch 5 is not tech preview in 3.11 - it is fully supported - and 2.x is gone - Understood. Did that change with the move from 3.10 to 3.11? Yes I must've been scanning another host which was running 3.10 to spot those preview vars. - So this leads to the question of how to debug the elasticsearch container not coming up. The console log information in the 1st email came from me running it manually so I could add the -v option to the elasticsearch command. Setting the DEBUG and LOGLEVEL environment variables wasn't illuminating. I guess I need to try and add the -v option to the run.sh script. This simply should not be happening. Did you use openshift-ansible to upgrade from 3.10 to 3.11, or did you deploy logging from scratch on 3.11? ___ dev mailing list dev@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/dev ___ dev mailing list dev@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
Re: Elasticsearch - 5.x
elasticsearch 5 is not tech preview in 3.11 - it is fully supported - and 2.x is gone - Understood. Did that change with the move from 3.10 to 3.11? I must've been scanning another host which was running 3.10 to spot those preview vars. - So this leads to the question of how to debug the elasticsearch container not coming up. The console log information in the 1st email came from me running it manually so I could add the -v option to the elasticsearch command. Setting the DEBUG and LOGLEVEL environment variables wasn't illuminating. I guess I need to try and add the -v option to the run.sh script. ___ dev mailing list dev@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
Re: Elasticsearch - 5.x
On 3/21/19 1:33 PM, Neale Ferguson wrote: On 3/21/19, 15:33, "Neale Ferguson" wrote: Right. The image is now https://hub.docker.com/r/openshift/origin-logging-elasticsearch5/tags - there are similar changes for origin-logging-curator5, origin-logging-kibana5, origin-logging-fluentd What version of openshift-ansible did you use to deploy logging? 3.11. Strangely, I didn't set the preview option in the hosts file but the only images available for me (the ones that I built) are the "5" series and these are the ones that were started. Elasticsearch 5 is _not_ tech preview in 3.11 - it is fully supported - 2.x is gone ___ dev mailing list dev@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/dev ___ dev mailing list dev@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
Re: Elasticsearch - 5.x
On 3/21/19, 15:33, "Neale Ferguson" wrote: Right. The image is now https://hub.docker.com/r/openshift/origin-logging-elasticsearch5/tags - there are similar changes for origin-logging-curator5, origin-logging-kibana5, origin-logging-fluentd What version of openshift-ansible did you use to deploy logging? 3.11. Strangely, I didn't set the preview option in the hosts file but the only images available for me (the ones that I built) are the "5" series and these are the ones that were started. ___ dev mailing list dev@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
Re: Elasticsearch - 5.x
On 3/21/19 1:23 PM, Neale Ferguson wrote:
If I want to use the techpreview elasticsearch5 images are there any other
changes I need to make as I am seeing this on startup:
Found index level settings on node level configuration.
Since elasticsearch 5.x index level settings can NOT be set on the nodes
configuration like the elasticsearch.yaml, in system properties or command line
arguments.In order to upgrade all indices the settings must be updated via the
/${index}/_settings API. Unless all settings are dynamic all indices must be
closed
in order to apply the upgradeIndices created in the future should use index
templates
to set default values.
Please ensure all required values are updated on all indices by executing:
curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{
"index.number_of_replicas" : "0",
"index.number_of_shards" : "1",
"index.translog.flush_threshold_period" : "5m",
"index.translog.flush_threshold_size" : "256mb",
"index.unassigned.node_left.delayed_timeout" : "2m"
}'
*
[2019-03-21T18:22:36,493][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler]
[logging-es-data-master-1jbe2qib] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException:
java.lang.IllegalArgumentException: node settings must not contain any index
level settings
The Dockerfile is building 5.6.10 but the 3.11.0 image from docker.io/openshift
appears to be based on 2.4.4 (which was changed to 5.x in June 2018).
Right. The image is now https://hub.docker.com/r/openshift/origin-logging-elasticsearch5/tags - there are similar changes for origin-logging-curator5, origin-logging-kibana5,
origin-logging-fluentd
What version of openshift-ansible did you use to deploy logging?
Neale
___
dev mailing list
dev@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
___
dev mailing list
dev@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
