Re: [Sugar-devel] Datastore is broken
I have received a few false warnings. Are you checking the commits related to the failing module or all the commits from the last build? Gonzalo On Fri, Nov 9, 2012 at 10:52 AM, Daniel Narvaez wrote: > On 9 November 2012 09:19, Simon Schampijer wrote: > > Ok, I pushed Martin's fix. Sorry we did regress in the latest commit :/ > > > > Daniel, the buildbot is very valuable, which are the places a developer > > should look at if he wants to get a quick overview. > > > > So, let's say, Fedora should at least build (given the current status of > > Ubuntu) [1] and then check the tests result here [2]? > > > > Thanks, > >Simon > > > > [1] http://buildbot.sugarlabs.org/one_line_per_build > > [2] > > > http://buildbot.sugarlabs.org/builders/fedora-17-32bit/builds/665/steps/shell_3/logs/stdio > > Yup! I tend to use the waterfall rather than one_line_per_build but > that's just a personal preference. > > Just keep in mind that there are a few sources of unreliability at the > moment, so a red build doesn''t necessarily mean there is a bug with > your code. Those are: > > 1 Unreliability of the UI tests. They are timing dependent so it's > hard to get them right. > 2 Network issues. Sometimes bender network goes down, sometimes > freedesktop etc > 3 Random segfaults when buillding gst-ffmpeg > 4 Ubuntu tests are still a work in progress (especially the > crash-on-exit issue on 12.10) > > I think I'm finally getting there for 1, I haven't seen random test > failures on the fedora slaves in a while and anyway I think I > understand the issues involved now, so I'll hopefully have this solved > soon. 2 cannot really be avoided completely, I hope to be able to mark > those builds with something different than a normal red failure, so > that it's immediately clear they don't need developer attention. I > have no idea what to do about 3 other than waiting Fedora 18 is out, > hopefully it doesn't happen too often. I'm working on 4, it will take > some time but it's just work. > > Something else I want to do is to get fix of all the tracebacks > triggered by the UI tests, then we can mark the tracebacks as > warnings, and use buildbot to avoid that kind of regressions too (we > have so many right now that it's difficult to find a new issue in the > logs). > > -- > Daniel Narvaez > ___ > Sugar-devel mailing list > Sugar-devel@lists.sugarlabs.org > http://lists.sugarlabs.org/listinfo/sugar-devel > ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Datastore is broken
On 9 November 2012 09:19, Simon Schampijer wrote: > Ok, I pushed Martin's fix. Sorry we did regress in the latest commit :/ > > Daniel, the buildbot is very valuable, which are the places a developer > should look at if he wants to get a quick overview. > > So, let's say, Fedora should at least build (given the current status of > Ubuntu) [1] and then check the tests result here [2]? > > Thanks, >Simon > > [1] http://buildbot.sugarlabs.org/one_line_per_build > [2] > http://buildbot.sugarlabs.org/builders/fedora-17-32bit/builds/665/steps/shell_3/logs/stdio Yup! I tend to use the waterfall rather than one_line_per_build but that's just a personal preference. Just keep in mind that there are a few sources of unreliability at the moment, so a red build doesn''t necessarily mean there is a bug with your code. Those are: 1 Unreliability of the UI tests. They are timing dependent so it's hard to get them right. 2 Network issues. Sometimes bender network goes down, sometimes freedesktop etc 3 Random segfaults when buillding gst-ffmpeg 4 Ubuntu tests are still a work in progress (especially the crash-on-exit issue on 12.10) I think I'm finally getting there for 1, I haven't seen random test failures on the fedora slaves in a while and anyway I think I understand the issues involved now, so I'll hopefully have this solved soon. 2 cannot really be avoided completely, I hope to be able to mark those builds with something different than a normal red failure, so that it's immediately clear they don't need developer attention. I have no idea what to do about 3 other than waiting Fedora 18 is out, hopefully it doesn't happen too often. I'm working on 4, it will take some time but it's just work. Something else I want to do is to get fix of all the tracebacks triggered by the UI tests, then we can mark the tracebacks as warnings, and use buildbot to avoid that kind of regressions too (we have so many right now that it's difficult to find a new issue in the logs). -- Daniel Narvaez ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Datastore is broken
On 11/08/2012 07:11 PM, Daniel Narvaez wrote: Not sure if it breaks anything but it doesn't sound good, with the latest git: Traceback (most recent call last): File "/home/buildbot/slave/fedora-17-32bit/build/install/lib/python2.7/site-packages/carquinyol/datastore.py", line 180, in _rebuild_index if os.path.exists(index_path): NameError: global name 'index_path' is not defined ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel Ok, I pushed Martin's fix. Sorry we did regress in the latest commit :/ Daniel, the buildbot is very valuable, which are the places a developer should look at if he wants to get a quick overview. So, let's say, Fedora should at least build (given the current status of Ubuntu) [1] and then check the tests result here [2]? Thanks, Simon [1] http://buildbot.sugarlabs.org/one_line_per_build [2] http://buildbot.sugarlabs.org/builders/fedora-17-32bit/builds/665/steps/shell_3/logs/stdio ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Datastore is broken
Thanks for the heads up! Fix attached, m On Thu, Nov 8, 2012 at 1:11 PM, Daniel Narvaez wrote: > Not sure if it breaks anything but it doesn't sound good, with the latest git: > > Traceback (most recent call last): > File > "/home/buildbot/slave/fedora-17-32bit/build/install/lib/python2.7/site-packages/carquinyol/datastore.py", > line 180, in _rebuild_index > if os.path.exists(index_path): > NameError: global name 'index_path' is not defined > ___ > Sugar-devel mailing list > Sugar-devel@lists.sugarlabs.org > http://lists.sugarlabs.org/listinfo/sugar-devel -- martin.langh...@gmail.com mar...@laptop.org -- Software Architect - OLPC - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff 0001-Bring-back-index_path-variable.patch Description: Binary data ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Datastore is broken
Not sure if it breaks anything but it doesn't sound good, with the latest git: Traceback (most recent call last): File "/home/buildbot/slave/fedora-17-32bit/build/install/lib/python2.7/site-packages/carquinyol/datastore.py", line 180, in _rebuild_index if os.path.exists(index_path): NameError: global name 'index_path' is not defined ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Datastore is broken
just rm index_updated On Wed, Nov 7, 2012 at 3:04 PM, Ignacio Rodríguez wrote: > ... > mkdir found.. "index" is a directory.. > > > 2012/11/7 Martin Langhoff > >> Hmmm, mkdir??? I'd suggest >> >> touch /home/user/.sugar/default/datastore/index >> >> >> >> m >> >> On Wed, Nov 7, 2012 at 2:35 PM, Ignacio Rodríguez >> wrote: >> > ¡For repair!: >> > mkdir /home/user/.sugar/default/datastore/index >> > :) >> > >> > >> > 2012/11/7 Martin Langhoff >> >> >> >> On Wed, Nov 7, 2012 at 11:51 AM, Simon Schampijer > > >> >> wrote: >> >> > On 11/07/2012 02:45 PM, Daniel Narvaez wrote: >> >> >> Buildbot tests are failing, apparently because of a datastore issue >> >> >> >> I owe you two a beer. A mighty big beer. Thanks for spotting this >> >> thinko before it landed in an OS build. >> >> >> >> > We fail because we try to access the index at the final path, but at >> >> > this >> >> > point we created it at the temp path only. The following patch does >> work >> >> > for >> >> > me. >> >> >> >> Complete agreement and signoff on your patch. It is a thinko on my >> >> part to be checking the size of the on-disk index, (which after a >> >> reboot in an ENOSPC condition would easily be 0, or some other corrupt >> >> value). The right thing to check is the index on the tmpfs, as Simon's >> >> patch does. >> >> >> >> thanks for the quick patch! >> >> >> >> >> >> >> >> m >> >> -- >> >> martin.langh...@gmail.com >> >> mar...@laptop.org -- Software Architect - OLPC >> >> - ask interesting questions >> >> - don't get distracted with shiny stuff - working code first >> >> - http://wiki.laptop.org/go/User:Martinlanghoff >> > >> > >> > >> > >> > -- >> > Juan Ignacio Rodríguez >> > CeibalJAM! >> > Activity Central >> > >> >> >> >> -- >> mar...@laptop.org -- Software Architect - OLPC >> - ask interesting questions >> - don't get distracted with shiny stuff - working code first >> - http://wiki.laptop.org/go/User:Martinlanghoff >> > > > > -- > Juan Ignacio Rodríguez > CeibalJAM! > Activity Central > > > ___ > Sugar-devel mailing list > Sugar-devel@lists.sugarlabs.org > http://lists.sugarlabs.org/listinfo/sugar-devel > > -- Walter Bender Sugar Labs http://www.sugarlabs.org ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Datastore is broken
... mkdir found.. "index" is a directory.. 2012/11/7 Martin Langhoff > Hmmm, mkdir??? I'd suggest > > touch /home/user/.sugar/default/datastore/index > > > > m > > On Wed, Nov 7, 2012 at 2:35 PM, Ignacio Rodríguez > wrote: > > ¡For repair!: > > mkdir /home/user/.sugar/default/datastore/index > > :) > > > > > > 2012/11/7 Martin Langhoff > >> > >> On Wed, Nov 7, 2012 at 11:51 AM, Simon Schampijer > >> wrote: > >> > On 11/07/2012 02:45 PM, Daniel Narvaez wrote: > >> >> Buildbot tests are failing, apparently because of a datastore issue > >> > >> I owe you two a beer. A mighty big beer. Thanks for spotting this > >> thinko before it landed in an OS build. > >> > >> > We fail because we try to access the index at the final path, but at > >> > this > >> > point we created it at the temp path only. The following patch does > work > >> > for > >> > me. > >> > >> Complete agreement and signoff on your patch. It is a thinko on my > >> part to be checking the size of the on-disk index, (which after a > >> reboot in an ENOSPC condition would easily be 0, or some other corrupt > >> value). The right thing to check is the index on the tmpfs, as Simon's > >> patch does. > >> > >> thanks for the quick patch! > >> > >> > >> > >> m > >> -- > >> martin.langh...@gmail.com > >> mar...@laptop.org -- Software Architect - OLPC > >> - ask interesting questions > >> - don't get distracted with shiny stuff - working code first > >> - http://wiki.laptop.org/go/User:Martinlanghoff > > > > > > > > > > -- > > Juan Ignacio Rodríguez > > CeibalJAM! > > Activity Central > > > > > > -- > mar...@laptop.org -- Software Architect - OLPC > - ask interesting questions > - don't get distracted with shiny stuff - working code first > - http://wiki.laptop.org/go/User:Martinlanghoff > -- Juan Ignacio Rodríguez CeibalJAM! Activity Central ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Datastore is broken
Hmmm, mkdir??? I'd suggest touch /home/user/.sugar/default/datastore/index m On Wed, Nov 7, 2012 at 2:35 PM, Ignacio Rodríguez wrote: > ¡For repair!: > mkdir /home/user/.sugar/default/datastore/index > :) > > > 2012/11/7 Martin Langhoff >> >> On Wed, Nov 7, 2012 at 11:51 AM, Simon Schampijer >> wrote: >> > On 11/07/2012 02:45 PM, Daniel Narvaez wrote: >> >> Buildbot tests are failing, apparently because of a datastore issue >> >> I owe you two a beer. A mighty big beer. Thanks for spotting this >> thinko before it landed in an OS build. >> >> > We fail because we try to access the index at the final path, but at >> > this >> > point we created it at the temp path only. The following patch does work >> > for >> > me. >> >> Complete agreement and signoff on your patch. It is a thinko on my >> part to be checking the size of the on-disk index, (which after a >> reboot in an ENOSPC condition would easily be 0, or some other corrupt >> value). The right thing to check is the index on the tmpfs, as Simon's >> patch does. >> >> thanks for the quick patch! >> >> >> >> m >> -- >> martin.langh...@gmail.com >> mar...@laptop.org -- Software Architect - OLPC >> - ask interesting questions >> - don't get distracted with shiny stuff - working code first >> - http://wiki.laptop.org/go/User:Martinlanghoff > > > > > -- > Juan Ignacio Rodríguez > CeibalJAM! > Activity Central > -- mar...@laptop.org -- Software Architect - OLPC - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Datastore is broken
¡For repair!: mkdir /home/user/.sugar/default/datastore/index :) 2012/11/7 Martin Langhoff > On Wed, Nov 7, 2012 at 11:51 AM, Simon Schampijer > wrote: > > On 11/07/2012 02:45 PM, Daniel Narvaez wrote: > >> Buildbot tests are failing, apparently because of a datastore issue > > I owe you two a beer. A mighty big beer. Thanks for spotting this > thinko before it landed in an OS build. > > > We fail because we try to access the index at the final path, but at this > > point we created it at the temp path only. The following patch does work > for > > me. > > Complete agreement and signoff on your patch. It is a thinko on my > part to be checking the size of the on-disk index, (which after a > reboot in an ENOSPC condition would easily be 0, or some other corrupt > value). The right thing to check is the index on the tmpfs, as Simon's > patch does. > > thanks for the quick patch! > > > > m > -- > martin.langh...@gmail.com > mar...@laptop.org -- Software Architect - OLPC > - ask interesting questions > - don't get distracted with shiny stuff - working code first > - http://wiki.laptop.org/go/User:Martinlanghoff > -- Juan Ignacio Rodríguez CeibalJAM! Activity Central ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Datastore is broken
On Wed, Nov 7, 2012 at 11:51 AM, Simon Schampijer wrote: > On 11/07/2012 02:45 PM, Daniel Narvaez wrote: >> Buildbot tests are failing, apparently because of a datastore issue I owe you two a beer. A mighty big beer. Thanks for spotting this thinko before it landed in an OS build. > We fail because we try to access the index at the final path, but at this > point we created it at the temp path only. The following patch does work for > me. Complete agreement and signoff on your patch. It is a thinko on my part to be checking the size of the on-disk index, (which after a reboot in an ENOSPC condition would easily be 0, or some other corrupt value). The right thing to check is the index on the tmpfs, as Simon's patch does. thanks for the quick patch! m -- martin.langh...@gmail.com mar...@laptop.org -- Software Architect - OLPC - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Datastore is broken
On 11/07/2012 02:45 PM, Daniel Narvaez wrote:
Buildbot tests are failing, apparently because of a datastore issue
http://buildbot.sugarlabs.org/builders/fedora-17-32bit/builds/656/steps/shell_3/logs/testlogs
I verified I get the same issue with a normal run on my laptop, as
long as I use a clean profile. (the shell shows an empty home briefly
and then dies). Ignacio is also seeing something similar.
Going back to 0.97.1 fixes it.
Thanks for the ping. Sorry I did not catch that in my testing, I did not
test with an empty datastore :(
The regression go introduced in the last commit:
http://git.sugarlabs.org/sugar-datastore/mainline/commit/adaf1d0d4bd536e24b7c4df8fd38f80f5b23850f
We fail because we try to access the index at the final path, but at
this point we created it at the temp path only. The following patch does
work for me.
diff --git a/src/carquinyol/datastore.py b/src/carquinyol/datastore.py
index a859dfe..d9bd56b 100644
--- a/src/carquinyol/datastore.py
+++ b/src/carquinyol/datastore.py
@@ -171,8 +171,7 @@ class DataStore(dbus.service.Object):
temp_index_path])
index_du = int(index_du.split('\t')[0])
# disk available, in bytes
-index_path = layoutmanager.get_instance().get_index_path()
-stat = os.statvfs(index_path)
+stat = os.statvfs(temp_index_path)
da = stat.f_bavail * stat.f_bsize
if da > (index_du * 1.2) and da > MIN_INDEX_FREE_BYTES: # 20%
room for growth
logger.warn('Attempting to move tempfs index to disk')
Would be great if Martin as the author of the patch could acknowledge.
Regards,
Simon
___
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel
[Sugar-devel] Datastore is broken
Buildbot tests are failing, apparently because of a datastore issue http://buildbot.sugarlabs.org/builders/fedora-17-32bit/builds/656/steps/shell_3/logs/testlogs I verified I get the same issue with a normal run on my laptop, as long as I use a clean profile. (the shell shows an empty home briefly and then dies). Ignacio is also seeing something similar. Going back to 0.97.1 fixes it. -- Daniel Narvaez ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
