For me, I think this work needs to address two situations:
1. Data that is always coming from the same timezone. If a data source always has its timestamp in EST, then we need to add a static offset to each timestamp to make it UTC. Perhaps this could be a variable in the parser configs; I believe some parsers already use a “withTimezone” configuration method. 2. Data that comes in from multiple timezones. This is the trickier case; we want to add or subtract an offset to get the timestamp to UTC, but this offset could be different from record to record. We could compare the log’s timestamp to the system timestamp to get a guess at the log’s timezone, but I’m not sure how reliable and efficient this would be. Let me know what you all think about these two cases. Domenic On 5/27/16, 1:29 PM, "Kumar, Sunny" <[email protected]> wrote: >Hi all > >Here at Capital One we want to standardize the timestamp of each telemetry >tuple to UTC and we think this might be useful for Metron in general too. >If you think so too then we'll be happy to contribute this back to Metron. And >again if so, I would like to have a quick sort-of architecture discussion for >the use-case. >Please let me know what do you guys think? > >Thanks, >Sunny >________________________________________________________ > >The information contained in this e-mail is confidential and/or proprietary to >Capital One and/or its affiliates and may only be used solely in performance >of work or services for Capital One. The information transmitted herewith is >intended only for use by the individual or entity to which it is addressed. If >the reader of this message is not the intended recipient, you are hereby >notified that any review, retransmission, dissemination, distribution, copying >or other use of, or taking of any action in reliance upon this information is >strictly prohibited. If you have received this communication in error, please >contact the sender and delete the material from your computer. ________________________________________________________ The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
