> > 2. Data that comes in from multiple timezones. This is the trickier case; > we want to add or subtract an offset to get the timestamp to UTC, but this > offset could be different from record to record. We could compare the log’s > timestamp to the system timestamp to get a guess at the log’s timezone, but > I’m not sure how reliable and efficient this would be.
I completely agree that this will be a challenge that we should solve. Do you have a specific example that you could bring up? This might help provide some structure around the solution. An alternative approach would be to use another field in the data to indicate the offset. For example, maybe I have a location indicator in the data that can help me get at the right offset. if state = OH then offset = -4 if state = WI then offset = -5 Something a little deeper to think about is whether an enrichment source could be used to solve this problem for more challenging scenarios. Imagine we have a data source with no location indicators, but we do have an IP. We could do a geoip lookup on the IP and then use the geo enriched data to then determine the correct offset as in the example above.
