Hi Daniel, On Thu, Dec 02, 2010 at 10:13:13AM +0100, Daniel Wagner wrote: > Hi Samuel, > > Thanks for taking time to review this patch. > > > Now as for the rest of this code, it seems that you're trying to do the > > following: Save monthly records, plus daily records for at least the last 2 > > months, and then all the records for today. Please correct me if I'm wrong. > > To make things more clear: > > - data file: the full ring buffer which should be processed > - history file: the backlog file > - tempory file: the new backlog file > > It works exactly the other way around. First, all records from today > are written to the tempory file. After that, maximum one record per > day is copied to the tempory file (around 2 months). And then only one > record per month is copied. Yes, I wasn't really talking about the order, but rather about the actual content.
> The timestamps are decreasing from 'begin' to 'end'. > > > I see it a bit differently. I'd like to have monthly records, and then only > > daily records from the last "account_period_offset" up to today. > > That's no problem. I can easily throw away all todays entries but one. > > > So for example, if our period offset is the 18th of each month, our today > > history file would be all the monthly records since we booted our device for > > the first time, and then daily entries from Nov 19 2010 up to today. > > Yep, that is exactly what it is doing right now. Well, it would contains daily entries from September 19 up to yesterday, and then all the records for today, right ? > > I'm not trying to save space here, but to keep this summarizing code simple. > > I'm all for simplifying this beast. > > > Here is how: > > When you want to update your history file, you need to look at your newest > > history entry. Then you go through your valid data entries, and for each of > > them compare its timestamp with the newest history one. If they're on the > > same > > day, you add the actual stats to the history entry and update the timestamp. > > If your current data entry is for the next day, then you add a new daily > > entry > > to your history file, which becomes the newest one. At that point your > > history > > file is just growing, you don't need to create a temporary one yet. > > I am not sure if you really want to modify the existing history > file. In case of a crash you history file is mostlikely corrupted. I > haven't really though what could go wrong there. It might be not that > bad and it is 'easy' to fix corrupted files. No, you're right, we must work on a temporary file, which would be a copy of the exisiting one to begin with. Then we look at the newest entry there and so on. The monthly summary if we go over offset_period would be done on the fly on the temporary file. > > Once you're done updating the existing history file with the data > > file records, you need to check if your latest history record entry > > is strictly newer than your period offset. If it is, then you need a > > new history file in which your going to copy all the monthly > > entries, and then sum up all daily entries from offset period - 1 > > month to offset_period into one into a new monthly entry. And then > > copy the remaining daily entries. > > I see. Instead of writing always a comlete new history file you just > want to write it complety new when a new monthy record has to be > created. Yes, but I now agree with you we must not touch the existing history file first. My idea would be to modify the temporary file by updating the newest record, and then adding new daily entries if needed, then save it as the new history file. The potential monthly summary would be done as a 2nd step. > > I feel this approach is keeping thing a bit simpler, would you agree > > with me here ? > > If I understood you correctly your approach is an optimization that > not always a temporary file is created. No, that was not my point. I was mostly looking at simplifying your summarize() routine. I think my approach would reach that goal. For example it would not need to iterate over both the history file and the data one. Once the current history file is entirely copied over the temporary one, we only look at its newest entry and then interate over the valid data ones. > The existing algorithm though > is still needed. I suggest that we just get 'summarize' in shape and > if it works fine we can still add the optimization (of course with the > 'todays' record change). Having only monthly records and at most 31 daily ones would makes thing a bit simpler at least, yes. Btw, why are the history and data files stored in opposite chronological order? > BTW, do have an idea where offset_period could be configured? I have > hard coded for testing. Is there something like a global config file > for ConnMan? Not yet, but it will eventually come. For now hardcoding this one to 1 is good enough. Cheers, Samuel. > Thanks! > daniel -- Intel Open Source Technology Centre http://oss.intel.com/ _______________________________________________ connman mailing list [email protected] http://lists.connman.net/listinfo/connman
