FastNode SlowNode configuration, aging an index based off of time using 3 nodes
1. Edit elasticsearch.yml:
Node1 - node.box_type: Fastnode
Node2 - node.box_type: Fastnode
Node3 - node.box_type: Slownode
2. Configure Template so new indexes are created on FastNodes
curl -XPUT localhost:9200/_template/template_1 -d '
{
"template": "Index_Name_Here_*",
"settings": {
"index.routing.allocation.require.box_type": "FastNode"
}
}
'
All of your data should be going to the designated FastNodes. Now we figure
out how to age the data. The command below contains everything you need to
age the data.
curl –XGET
'http://loggingesms01.apps.appriver.corp:9200/graylog2_40/_settings'
Essentially what you need to do is:
get a list of indexes
loop over the index list looking at the creation date and box_type.
If the box_type matches FastNode and index created date is X days older
than the current timestamp.
Execute these actions
#unwrite protect index
curl -XPOST loggingesms01.apps.appriver.corp:9200/graylog2_40/_settings -d
' {"index.blocks.write" : "false"}'
#drop replicas (up to you, if you have 1 slow node keeping replicas doesn’t
make sence)
curl -XPUT loggingesms01.apps.appriver.corp:9200/graylog2_40/_settings -d '
{"index.routing.allocation.require.box_type" : "SlowNode"} '
# Move index to SlowNode
curl -XPUT loggingesms01.apps.appriver.corp:9200/graylog2_40/_settings -d '
{"index.number_of_replicas" : "0"} '
I ended up using powershell due to familiarity and lack of time to figure
out how to work with json in bash. It ended up being a good choice for us
as we are moving towards using an enterprise task scheduling software which
powershell will work perfectly with.
I attached the ps1 so you can see how exactly I handled things.
I would also like to say that this may not be the "best way" however it
works and seems solid.
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/d0bfda31-9b96-4106-9713-65bd43ef9d8f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
hotcold.ps1
Description: Binary data
