Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
On Tue, Aug 25, 2009 at 17:42, Martin Langhoffmartin.langh...@gmail.com wrote: On Thu, Aug 20, 2009 at 6:56 PM, Martin Langhoffmartin.langh...@gmail.com wrote: On olpc-sur we're seeing reports of my journal contents disappeared after reboot. (Thread starts at http://lists.laptop.org/pipermail/olpc-sur/2009-August/004177.html ) So, we have a couple of users who could send a sample datastore to a developer. I think Tomeu is interested in diagnosing and fixing the problem within Sugar (great!) -- if so, do you want me to ask for a zipfile of the corrupt datastore on your behalf? If this is easier than getting logs, then yes. It would also be very important for these Uy users to get a rescue my lost journal activity. Anyone interested in helping? I will be happy to answer any questions about this. Regards, Tomeu cheers, m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Sugar-devel mailing list sugar-de...@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel -- «Sugar Labs is anyone who participates in improving and using Sugar. What Sugar Labs does is determined by the participants.» - David Farning ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
On Fri, Aug 21, 2009 at 07:46, Martin Langhoffmartin.langh...@gmail.com wrote: On Fri, Aug 21, 2009 at 12:24 AM, Andrés Ambroisandresambr...@gmail.com wrote: I'll help in any way I can. Would an upload webservice similar to Sacha's for uploading the necessary datastore bits help? We can then distribute a simple cli tool that uploads the data. Thanks! If Uy was using School Servers, this wouldn't be an issue. The DS data would be there :-/ For end users, I don't think a network-based solution is best. A local activity will be a lot more effective. For debugging we might need to get some sample 'broken' datastores (with due care for user privacy, etc...). Will collecting the indexes be enough or do you need the whole datastore? For debugging, better to get the whole thing I guess. Yes, though if we manage to get the logs dir of a machine just after the DS was lost, there's lots of chances we are going to be able to determine the exact cause for the loss. Then we would be in a better position to find the best fix. Regards, Tomeu -- «Sugar Labs is anyone who participates in improving and using Sugar. What Sugar Labs does is determined by the participants.» - David Farning ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
2009/8/21 Tomeu Vizoso to...@sugarlabs.org: Yes, though if we manage to get the logs dir of a machine just after the DS was lost, there's lots of chances we are going to be able to The ones on the ramdisk? :-( m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
2009/8/21 Martin Langhoff martin.langh...@gmail.com: 2009/8/21 Tomeu Vizoso to...@sugarlabs.org: Yes, though if we manage to get the logs dir of a machine just after the DS was lost, there's lots of chances we are going to be able to The ones on the ramdisk? :-( Is ~/.sugar/default/logs in the ramdisk? This would be the first time I hear about it. If it's like that, maybe Andres could modify a XO to have that dir in the nand and wait for it to happen? Regards, Tomeu m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff -- «Sugar Labs is anyone who participates in improving and using Sugar. What Sugar Labs does is determined by the participants.» - David Farning ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
On Friday 21 August 2009 05:08:28 am Martin Langhoff wrote: 2009/8/21 Tomeu Vizoso to...@sugarlabs.org: Yes, though if we manage to get the logs dir of a machine just after the DS was lost, there's lots of chances we are going to be able to The ones on the ramdisk? :-( I think ~/.sugar/default/logs We're holding a jam the 29th, so maybe someone affected could attend, otherwise it'll be very hard for teachers to upload large files (ADSL broadband has a 128 Kbps uplink here in .uy). I'll post a mail in Sur and olpc-uruguay, after I get some sleep. m -- -Andrés ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
On Friday 21 August 2009 05:10:21 am Tomeu Vizoso wrote: 2009/8/21 Martin Langhoff martin.langh...@gmail.com: 2009/8/21 Tomeu Vizoso to...@sugarlabs.org: Yes, though if we manage to get the logs dir of a machine just after the DS was lost, there's lots of chances we are going to be able to The ones on the ramdisk? :-( Is ~/.sugar/default/logs in the ramdisk? This would be the first time I hear about it. If it's like that, maybe Andres could modify a XO to have that dir in the nand and wait for it to happen? Remember that Ceibal XOs have root access locked-down. And I recently found out that since the key-delegation stuff was implemented, we can't request developer keys. Not from OLPC at least, and LATU is not providing that service that I know... I'll also try to flash an XO with 8.2.1 and try to reproduce. Regards, Tomeu m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff -- -Andrés ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
2009/8/21 Tomeu Vizoso to...@sugarlabs.org: 2009/8/21 Martin Langhoff martin.langh...@gmail.com: 2009/8/21 Tomeu Vizoso to...@sugarlabs.org: Yes, though if we manage to get the logs dir of a machine just after the DS was lost, there's lots of chances we are going to be able to The ones on the ramdisk? :-( Is ~/.sugar/default/logs in the ramdisk? This would be the first time I hear about it. OLPC's builds (at least the F7/F9 based series) I understood they were on a ramdisk. Looking at it now, however, it seems that they aren't -- but a restart of X clears it out anyway. So by the time the problem is _seen_, *poof*, logs are gone. Maybe something in the sugar startup clears it out. Dunno. Doesn't seem to be olpc-configure. m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
On Fri, Aug 21, 2009 at 10:36:44AM +0200, Martin Langhoff wrote: 2009/8/21 Tomeu Vizoso to...@sugarlabs.org: 2009/8/21 Martin Langhoff martin.langh...@gmail.com: 2009/8/21 Tomeu Vizoso to...@sugarlabs.org: Yes, though if we manage to get the logs dir of a machine just after the DS was lost, there's lots of chances we are going to be able to The ones on the ramdisk? :-( Is ~/.sugar/default/logs in the ramdisk? This would be the first time I hear about it. OLPC's builds (at least the F7/F9 based series) I understood they were on a ramdisk. Looking at it now, however, it seems that they aren't -- but a restart of X clears it out anyway. So by the time the problem is _seen_, *poof*, logs are gone. You sure that ~/.sugar/default/logs doesn't just get renamed to ~/.sugar/default/logs/N? That's what happens now, but perhaps not on older builds? m Martin pgpPvhO3kuQzs.pgp Description: PGP signature ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
2009/8/21 Martin Langhoff martin.langh...@gmail.com: 2009/8/21 Tomeu Vizoso to...@sugarlabs.org: 2009/8/21 Martin Langhoff martin.langh...@gmail.com: 2009/8/21 Tomeu Vizoso to...@sugarlabs.org: Yes, though if we manage to get the logs dir of a machine just after the DS was lost, there's lots of chances we are going to be able to The ones on the ramdisk? :-( Is ~/.sugar/default/logs in the ramdisk? This would be the first time I hear about it. OLPC's builds (at least the F7/F9 based series) I understood they were on a ramdisk. Looking at it now, however, it seems that they aren't -- but a restart of X clears it out anyway. So by the time the problem is _seen_, *poof*, logs are gone. Maybe something in the sugar startup clears it out. Dunno. Doesn't seem to be olpc-configure. Should be kept in a subdir in there, from: http://git.sugarlabs.org/projects/sugar/repos/mainline/blobs/cbd0509f4bcfe1b3e52f8898b1cb83d522337ab8/src/logsmanager.py#line25 But anyway, the failure to read the DS would be in the first boot where the journal appears empty, not in the boot where the DS index got corrupted. Regards, Tomeu m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff -- «Sugar Labs is anyone who participates in improving and using Sugar. What Sugar Labs does is determined by the participants.» - David Farning ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
2009/8/21 Tomeu Vizoso to...@sugarlabs.org: Should be kept in a subdir in there, from: Damn! I wish I had known about this! So 3 old dirs are kept - excellent. But anyway, the failure to read the DS would be in the first boot where the journal appears empty, not in the boot where the DS index got corrupted. Well, if you can get your hands on the 'corrupt' datastore dir, doesn't it also contain the xapian DB? Just renaming it into 'datastore' should re-produce the error... Maybe not? m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
2009/8/21 Martin Langhoff martin.langh...@gmail.com: 2009/8/21 Tomeu Vizoso to...@sugarlabs.org: Should be kept in a subdir in there, from: Damn! I wish I had known about this! So 3 old dirs are kept - excellent. But anyway, the failure to read the DS would be in the first boot where the journal appears empty, not in the boot where the DS index got corrupted. Well, if you can get your hands on the 'corrupt' datastore dir, doesn't it also contain the xapian DB? Just renaming it into 'datastore' should re-produce the error... Maybe not? I think it should if we know the exact build image on which to place it. So I would say whatever is easier for people to provide. Regards, Tomeu m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff -- «Sugar Labs is anyone who participates in improving and using Sugar. What Sugar Labs does is determined by the participants.» - David Farning ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
On Thu, Aug 20, 2009 at 18:56, Martin Langhoffmartin.langh...@gmail.com wrote: On olpc-sur we're seeing reports of my journal contents disappeared after reboot. (Thread starts at http://lists.laptop.org/pipermail/olpc-sur/2009-August/004177.html ) This reminds me a lot of the issues we saw earlier in development of what became 8.2.x -- if for any reason Sugar doesn't like the Journal contents or indexes, it moves the 'datastore' directory aside, and starts anew. That's my hunch, and I am hoping to hear confirmation from the users reporting the problem. Two pleas for help - A sugar activity that does some data-recovery and freeing up the space. The good news is that we have a json file with the metadata for each Journal entry, so we can either load it back into the Journal (mstone had published a cli script that exports/imports to the Journal?) or create Journal Entry Bundles and save them to a USB stick. I suspect the JEB path might help users more -- if the Journal is choking on some specific metadata, a script that reloads it all in one go will make it choke again. Having files as JEBs allows step-by-step attempts. How do you envision the UI of the tool that will recover the JEBs? There's a good chance that the xapian index got corrupted because the machine lost power while the index on disk was on an inconsistent state. If that's the case, just rebuilding the whole index should work fine. Would be nice to have more details about the failure so we can both fix it and propose the best tool for the job. Maybe a ceibal jam volunteer with some linux knowledge would like to work with us on this? Regards, Tomeu - Diagnosis. If it's something we can pin down, it'll be great to have a fix! This will surely involve getting in touch with teachers on olpc-sur. cheers, m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Sugar-devel mailing list sugar-de...@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel -- «Sugar Labs is anyone who participates in improving and using Sugar. What Sugar Labs does is determined by the participants.» - David Farning ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
On Thu, Aug 20, 2009 at 7:44 PM, Tomeu Vizosoto...@sugarlabs.org wrote: How do you envision the UI of the tool that will recover the JEBs? We don't need many words there :-) all we need is - A listing of Journals available (by date - the numbers after 'datastore' are epoch seconds IIRC) - if there's only one, skip this stage... - A view that looks a bit like the Journal view... we could show an 'action' that is 'inject into journal' (icon of the activity, arrow, journal icon?). There's a good chance that the xapian index got corrupted because the machine lost power while the index on disk was on an inconsistent state. If that's the case, just rebuilding the whole index should work fine. In that case, another option is to not have a UI, and just merge all the datastores present, in one go, no more questions asked. As part of it, files are moved (instead of cp'd). Not sure how to make the above atomic safe. Would be nice to have more details about the failure so we can both fix it and propose the best tool for the job. Maybe a ceibal jam volunteer with some linux knowledge would like to work with us on this? Many are reading olpc-sur -- I hope someone might help. cheers, m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
On Thursday 20 August 2009 06:58:48 pm Martin Langhoff wrote: On Thu, Aug 20, 2009 at 7:44 PM, Tomeu Vizosoto...@sugarlabs.org wrote: Would be nice to have more details about the failure so we can both fix it and propose the best tool for the job. Maybe a ceibal jam volunteer with some linux knowledge would like to work with us on this? Many are reading olpc-sur -- I hope someone might help. I'll help in any way I can. Would an upload webservice similar to Sacha's for uploading the necessary datastore bits help? We can then distribute a simple cli tool that uploads the data. Will collecting the indexes be enough or do you need the whole datastore? cheers, m -- -Andrés ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
On Thu, Aug 20, 2009 at 06:56:17PM +0200, Martin Langhoff wrote: On olpc-sur we're seeing reports of my journal contents disappeared after reboot. (Thread starts at http://lists.laptop.org/pipermail/olpc-sur/2009-August/004177.html ) Hmm. I've two XOs with 8.2.1 being used by children of a friend, and one of them reported a loss event recently ... what do I look for in the filesystem to check for evidence of this? -- James Cameron http://quozl.linux.org.au/ ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] RFH - Journal corruption reports fom 8.2.1 users in Uy
On Fri, Aug 21, 2009 at 12:24 AM, Andrés Ambroisandresambr...@gmail.com wrote: I'll help in any way I can. Would an upload webservice similar to Sacha's for uploading the necessary datastore bits help? We can then distribute a simple cli tool that uploads the data. Thanks! If Uy was using School Servers, this wouldn't be an issue. The DS data would be there :-/ For end users, I don't think a network-based solution is best. A local activity will be a lot more effective. For debugging we might need to get some sample 'broken' datastores (with due care for user privacy, etc...). Will collecting the indexes be enough or do you need the whole datastore? For debugging, better to get the whole thing I guess. m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
