Yeah Otto, pre-0.5.0 (0.4.2) would be ES 2.3 if users were not using
master. ES upgrade is a big piece of this Apache release. Last release was
0.4.2 on Fri Dec 22 2017.
I'm +1 on the idea of an example referencing the ES docs and keeping this
as simple as possible.
* https://archive.apache.org/d
Aren’t people who are on an old version of ES everyone pre 0.5.0? Like all
the metron users?
On June 5, 2018 at 12:31:30, Simon Elliston Ball (
si...@simonellistonball.com) wrote:
Yes, anything using elastic would need the field names changed. That said,
people who are on such an old version (e
Yes, anything using elastic would need the field names changed. That said,
people who are on such an old version (eol) will need to not the bullet with ES
compatibility as some point.
Simon
> On 5 Jun 2018, at 17:17, Otto Fowler wrote:
>
> Are there consequences with Kibana as well? queries
Are there consequences with Kibana as well? queries, visualizations,
templates they may have?
On June 5, 2018 at 12:03:44, Nick Allen (n...@nickallen.org) wrote:
I just don't know if telling users to do a bulk upgrade of their indices is
sufficient enough of an upgrade path. I would expect some
I just don't know if telling users to do a bulk upgrade of their indices is
sufficient enough of an upgrade path. I would expect some to have
downstream processes dependent on those field names, which would also need
to be updated.
Although, we could tell users to do any field name conversions th
Agreed, we should definitely have a clear picture about how to do that,
maybe even a worked example in the use-cases that we can reference. I'm
just saying we don't need to migrate ES docs into Metron, but rather
reference them as much as we possibly can.
On Tue, Jun 5, 2018 at 11:38 AM Otto Fowl
It is still our user list and dev list that will have the burden of talking
folks through that.
On June 5, 2018 at 09:58:32, Casey Stella (ceste...@gmail.com) wrote:
To be clear, I'm not even suggesting that we create any tooling here. I'd
say just a reference to the ES docs and a call-out in Up
I am in favor of removing the `FieldNameConverter` abstraction as an end
state. Although, I don't agree with Simon that we could have just done
that directly without providing a backwards compatible solution as was done
in #1022. There are too many touch points that rely on that conversion and
us
+1 to that. It's a simple problem to solve if you have it, and with a
little docs help I imagine we'll be fine.
On 5 June 2018 at 06:58, Casey Stella wrote:
> To be clear, I'm not even suggesting that we create any tooling here. I'd
> say just a reference to the ES docs and a call-out in Upgrad
To be clear, I'm not even suggesting that we create any tooling here. I'd
say just a reference to the ES docs and a call-out in Upgrading.md would
suffice as long as we have some strong reason to believe it'll work. As
far as I'm concerned, the sooner we're out of the business of transforming
fie
ES does have some docs around how this gets handled in upgrades:
https://www.elastic.co/guide/en/elasticsearch/reference/2.4/dots-in-names.html
Might be worth taking a look to see what conflicts we'd have going from 2.x
to 5.x and figuring out where to go from there.
On Tue, Jun 5, 2018 at 9:46 A
I guess in principal you could use
https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#docs-reindex-change-name
to reindex with the new fields. It wouldn't be hard to script up a bit of
python to help users out with that, or of course to leave that as an
exercise to th
+1 to that, Simon. Do we have a sense of if there are utilities provided
by ES to do this kind of migration transformation easily?
On Tue, Jun 5, 2018 at 9:37 AM Simon Elliston Ball <
si...@simonellistonball.com> wrote:
> I would definitely agree that the transformation should be removed. We hav
I would definitely agree that the transformation should be removed. We have
now however added a complex generic solution in the backend, which is going
to be noop for most people. This was done I believe for the sake of
backward compatibility. I would argue however, that there is no need to
support
I agree completely. I will leave this thread open for a day or two to give
others a chance to weigh in. If no one opposes, I will creates Jiras for
removing field transformations and transforming existing data.
On Tue, Jun 5, 2018 at 8:21 AM, Casey Stella wrote:
> Well, on write it is a transf
Well, on write it is a transformation, on read it's a translation. This is
to say that you're providing a mapping on read to translate field names
given the index you're using. The other approach that I was considering
last night is a field transformation REST call which translates field names
th
Having 2 different patterns for configuring field name transformations on
read vs write is confusing to me. I agree with both of you that
normalizing on '.' and not having to do the translation at all would be
ideal. Like you both suggested, we would need some utility or script to
convert preexis
ES 2.x support officially ended 4 months ago
(https://www.elastic.co/support/eol), so why still support ':' at all?
:) Additionally, 2.x isn't even supported at all on the last 2 Ubuntu
LTS releases (16.04 & 18.05).
Therefor, move everything to use '.' and provide a conversion/upgrade
script
Before we construct a super generic solution, can we get an analysis of all
the places in the UI where we're hard-coding fields? It seems like pulling
the field from the global config is the strategy that we've gone with that
could be expanded upon in https://github.com/apache/metron/pull/1010
(th
We've been dealing with a reoccurring challenge in Metron. It is common
for various fields to contain '.' characters for the purpose of making them
more readable, namespacing, etc. At one point we only supported
Elasticsearch 2.3 which did not allow dots and forced us to use ':'
instead. This li
20 matches
Mail list logo
