[GitHub] metron issue #990: METRON-1520: Add caching for stellar field transformation...

2018-04-23 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/990 +1 Thanks. Looks good. ---

[GitHub] metron issue #990: METRON-1520: Add caching for stellar field transformation...

2018-04-23 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/990 I reran this in full-dev from scratch and it checks out. Let me know if there are any more comments @nickwallen and @ottobackwards ---

[GitHub] metron issue #990: METRON-1520: Add caching for stellar field transformation...

2018-04-23 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/990 Ok, @nickwallen I think I reacted to your comments. I'm going to spin this up one more time since it's churned a bit since I originally submitted. Let me know if I missed anything. ---

[GitHub] metron issue #990: METRON-1520: Add caching for stellar field transformation...

2018-04-12 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/990 Ok, turned off by default and added some more detail to the performance tuning guide. ---

[GitHub] metron issue #990: METRON-1520: Add caching for stellar field transformation...

2018-04-11 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/990 I think that the assumptions that it makes about the ability to cache the output, and apply to every transformation or every function may be a bit optimistic. I would have it off by de

[GitHub] metron issue #990: METRON-1520: Add caching for stellar field transformation...

2018-04-11 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/990 @ottobackwards Sounds good; I'll do that. ---

[GitHub] metron issue #990: METRON-1520: Add caching for stellar field transformation...

2018-04-11 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/990 Yes, the caching semantics are the same as the enrichment topology; namely that we cache all stellar expressions. You can, however, turn caching off in the parsers in the case that you hav

[GitHub] metron issue #990: METRON-1520: Add caching for stellar field transformation...

2018-04-11 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/990 Does this assume, that if you have multiple transformations that all of them should be cached? Does it assume that every stellar function returns the same value with the same input? ---

[GitHub] metron issue #990: METRON-1520: Add caching for stellar field transformation...

2018-04-11 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/990 This PR provides an expression cache for stellar when using the stellar field transformations in parsers. Specifically, we found that without caching we were substantially stymied when performance