Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/622
There was a lot of good discussion on this, but I find it hard to summarize
completely the positions of everyone including @mattf-horton and @cestella.
Here is my attempt in trying to do that. Please correct anything that I have
misstated.
1. Everyone agreed that a ToC (table of contents) is a useful additional
feature for the Profiler. The decodable row key would be needed in addition
to, not instead of, a ToC.
1. In implementing a decodable row key, we do need to plan for future
changes in row key format. This was handled in this PR, but can be improved.
1. The decodable row key feature should be completed **before** a ToC so
that the row keys can be used to generate (or regenerate) a ToC on-demand.
1. There were various suggestions made on how to shorten up the row key
format. Some of those I completed on this PR (like using a murmur hash) and
others (like using shorts instead of ints) I would need to incorporate in a
future PR for a decodable row key.
1. There is a need for a migration tool. A tool that can read the existing
row key format and rewrite the same data using a new format. This tool is
necessary even if it cannot be implemented deterministically with the current
row key format. The tool may not hints from the user like the names of known
profiles.
Once I compile a summary of these changes, I will close this PR. All
enhancements around this will be implemented on new PRs.
---
