Re: [jira] [Commented] (MYNEWT-368) Change logging format

[Sterling Hughes]

 In the 16-bit case I don't understand why you would ever reset the index to 0. 
 So long as the value is increasing, you will always have 32k entries prior to 
wrap.   I don't see why you'd care that it starts at a given number.  

Sterling

> On Sep 1, 2016, at 2:28 PM, Ray Suorsa (JIRA)  wrote:
> 
> 
>[ 
> https://issues.apache.org/jira/browse/MYNEWT-368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456671#comment-15456671
>  ] 
> 
> Ray Suorsa commented on MYNEWT-368:
> ---
> 
> Ah yes, the index is doing double duty.  The first one is to try to maintain 
> order when the device logs a lot of lines in a short period of time.  The 
> other, which we trying to use it for also is to deal with a lot of clock 
> drift.  It seems to me that we should not reset the index in that way.
> 
> I am sure we will have to experiment a bit and figure out what sort of bounds 
> we are trying to operate in.
> 
> 
> 
> 
>> Change logging format
>> -
>> 
>>Key: MYNEWT-368
>>URL: https://issues.apache.org/jira/browse/MYNEWT-368
>>Project: Mynewt
>> Issue Type: Bug
>> Components: Newtmgr
>>   Reporter: Sterling Hughes
>>   Assignee: Peter Snyder
>>Fix For: v1_0_0_beta1
>> 
>> 
>> Right now newtmgr has an 8-bit index which goes along with a 64-bit 
>> timestamp.  We are having trouble querying logs when time moves forward, 
>> because of the misordering of log entries.
>> We should change this such that we have a 16-bit log index, which can be 
>> used to query logs remotely.  A result of this will be to make the module an 
>> 8-bit value, which it is already assumed to be.
>> While doing this change, on the FCB implementation, we should also add a 
>> short log entry header (4-bytes), that at least contains the version of the 
>> log format that we are writing, that way when we make changes in the future 
>> to the log format, the code can automatically wipe this sector and reformat 
>> it.
> 
> 
> 
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)

Re: [jira] [Commented] (MYNEWT-368) Change logging format

[Vipul Rahane]

Hello Peter,

Sorry, I did am not copied on the ticket and so I did not receive a 
notification. 

The main reason behind adding an index was indicating the order of entries 
logged in a millisecond. If the timestamp is the same which could be the case 
if we log at a lower level than the app(Communication stack/Drivers/HAL), the 
index indicates the order of entries logged at the same time. OS Ticks 
resolution is 1 ms and our resolution for the logs is 1 microsecond. Events can 
be precisely logged at a microsecond level, although it might not be possible 
in reality. 

Timestamp and index together create a unique indexable entry for the backend. 
Index is considered as a secondary key for searching the log and timestamp is 
considered as the primary key. 

With a 16 bit index, rollover won’t happen that often but how do you track the 
rollover in index ? Logging an entry seems bit of an overkill when the keys 
themselves indicate the behavior. Resetting it to 0 helps by not keeping track 
of the rollover elsewhere. 

Now, even if we generate a log message for the same, it would need to be read 
by someone and interpreted appropriately which is not the case with an index 
and timestamp as that is fairly indicative and easy to manage at the backend. 
This is just my thought behind it. Will and Chris can chime in as well as they 
have been a part of these discussions. Chetna has also played an important role 
in requesting specific behavior for the logs, so I think her understanding is 
important as well. 

Personally, I would keep it as it is but if everybody agrees I don’t have a 
problem either ways. 

Regards,
Vipul Rahane

 
> On Sep 1, 2016, at 10:23 AM, Sterling Hughes  wrote:
> 
> Vipul added this code, so check with him.  Now that this is a larger value, 
> I’m not sure we need to rollover the index personally/special case this.
> 
> On 1 Sep 2016, at 10:21, Peter Snyder (JIRA) wrote:
> 
>>[ 
>> https://issues.apache.org/jira/browse/MYNEWT-368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456063#comment-15456063
>>  ]
>> 
>> Peter Snyder commented on MYNEWT-368:
>> -
>> 
>> Looking at log.c/log_append(), the following lines reset the log index when 
>> there's more than a ms between log entries.
>> 
>> /* Resetting index every millisecond */
>> if (g_log_info.li_timestamp > 1000 + prev_ts) {
>>g_log_info.li_index = 0;
>> }
>> 
>> Was this necessary? If li_index is 16 bits and there's a fewer rollovers, do 
>> we still need it? We could automatically generate a log message when a timer 
>> or index rollover is detected. Thoughts?
>> 
>> 
>>> Change logging format
>>> -
>>> 
>>>Key: MYNEWT-368
>>>URL: https://issues.apache.org/jira/browse/MYNEWT-368
>>>Project: Mynewt
>>> Issue Type: Bug
>>> Components: Newtmgr
>>>   Reporter: Sterling Hughes
>>>   Assignee: Peter Snyder
>>>Fix For: v1_0_0_beta1
>>> 
>>> 
>>> Right now newtmgr has an 8-bit index which goes along with a 64-bit 
>>> timestamp.  We are having trouble querying logs when time moves forward, 
>>> because of the misordering of log entries.
>>> We should change this such that we have a 16-bit log index, which can be 
>>> used to query logs remotely.  A result of this will be to make the module 
>>> an 8-bit value, which it is already assumed to be.
>>> While doing this change, on the FCB implementation, we should also add a 
>>> short log entry header (4-bytes), that at least contains the version of the 
>>> log format that we are writing, that way when we make changes in the future 
>>> to the log format, the code can automatically wipe this sector and reformat 
>>> it.
>> 
>> 
>> 
>> --
>> This message was sent by Atlassian JIRA
>> (v6.3.4#6332)