Re: [gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Just a quick note: The server on http://smolt.hartwork.org:45678/ is up again, it was down yesterday. Sebastian
Re: [gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Torsten Veller wrote: > What's the status of the stats project? What's missing? What help is > needed? Hello Torsten, thanks for your interest. Let me quote myself from a recent reply on a similar question: for the quickest summary possible these steps are needed: - make me have and take time for it (soon again as planned) - fix server performance to not take 10 seconds per gentoo-submission (my alchemy code is partly stupid, now i know) - move batch processing upstream (depends on upstream co-op) - get more gentoo people join coding after (http://soc.gentooexperimental.org/projects/stats/issues) > I'd really like to see a system that can answer: > "How often is cpv x installed on arch y (testing/stable flavour)?" That's possible, yes. Please elaborate more on motivation and details. Sebastian
[gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
* Sebastian Pipping : > What's next > === What's the status of the stats project? What's missing? What help is needed? I'd really like to see a system that can answer: "How often is cpv x installed on arch y (testing/stable flavour)?"
[gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Hi, Alexey Shvetsov : > seems smoltSendProfile doesnt work with unicode locales =) > 100%] x11-wm/twm-1.0.4 > Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/smolt/client/sendProfile.py", > line 211, in > """ % excerpts > UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position > 0: ordinal not in range(128) This has been fixed in a debug session over phone by sping yesterday. V-Li -- Christian Faulhammer, Gentoo Lisp project http://www.gentoo.org/proj/en/lisp/>, #gentoo-lisp on FreeNode http://gentoo.faulhammer.org/> signature.asc Description: PGP signature
[gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Hi, Sebastian Pipping : > Christian Faulhammer wrote: > >> I have just committed a patch [1] that could fix it. Please try > >> again with the latest HEAD and let me know how it works for you. > > > > Traceback (most recent call last): > > File > > "/usr/lib/python2.6/site-packages/smolt/client/sendProfile.py", > > line 215, in """ % excerpts UnicodeDecodeError: 'ascii' > > codec can't decode byte 0xc3 in position 3: ordinal not in > > range(128) > > > > Similar. :) > > Is that after the patch? The string should be plain ASCII at that > point already. That puzzles me, are you very sure it's the new code? > There should be a new function "to_ascii" in there. Yes, it is after the patch. The line number also changed. :) > If it's after the patch do you have the time to play with the code > yourself? Is it on a machine I could SSH into? It's kind of hard for > me to debug this with it not happening on my machine. I can > fake-produce it (which I did) but I don't get your scenario, it seems. I only have some basics in Python, but I am willing to play, tell me what to do. Unfortunately I am not able to give out SSH access. V-Li -- Christian Faulhammer, Gentoo Lisp project http://www.gentoo.org/proj/en/lisp/>, #gentoo-lisp on FreeNode http://gentoo.faulhammer.org/> signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Christian Faulhammer wrote: >> I have just committed a patch [1] that could fix it. Please try again >> with the latest HEAD and let me know how it works for you. > > Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/smolt/client/sendProfile.py", > line 215, in """ % excerpts > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 3: > ordinal not in range(128) > > Similar. :) Is that after the patch? The string should be plain ASCII at that point already. That puzzles me, are you very sure it's the new code? There should be a new function "to_ascii" in there. If it's after the patch do you have the time to play with the code yourself? Is it on a machine I could SSH into? It's kind of hard for me to debug this with it not happening on my machine. I can fake-produce it (which I did) but I don't get your scenario, it seems. Sebastian
[gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Hi, Sebastian Pipping : > Alexey Shvetsov wrote: > > Hi all! > > > > seems smoltSendProfile doesnt work with unicode locales =) > > 100%] x11-wm/twm-1.0.4 > > Traceback (most recent call last): > > File > > "/usr/lib/python2.6/site-packages/smolt/client/sendProfile.py", > > line 211, in """ % excerpts > > UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position > > 0: ordinal not in range(128) > > Sorry I wasn't able to address this issue earlier. > > I have just committed a patch [1] that could fix it. Please try again > with the latest HEAD and let me know how it works for you. Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/smolt/client/sendProfile.py", line 215, in """ % excerpts UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 3: ordinal not in range(128) Similar. :) V-Li -- Christian Faulhammer, Gentoo Lisp project http://www.gentoo.org/proj/en/lisp/>, #gentoo-lisp on FreeNode http://gentoo.faulhammer.org/> signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Alexey Shvetsov wrote: > Hi all! > > seems smoltSendProfile doesnt work with unicode locales =) > 100%] x11-wm/twm-1.0.4 > Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/smolt/client/sendProfile.py", > line 211, in > """ % excerpts > UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position > 0: ordinal not in range(128) Sorry I wasn't able to address this issue earlier. I have just committed a patch [1] that could fix it. Please try again with the latest HEAD and let me know how it works for you. If anyone can guide me on how to better address such errors I'm all open for it. Sebastian [1] http://git.goodpoint.de/?p=smolt-gentoo.git;a=commitdiff;h=7592c44f6eef240aaae147ba11ffe85a4156068b
Re: [gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Hi all! seems smoltSendProfile doesnt work with unicode locales =) 100%] x11-wm/twm-1.0.4 Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/smolt/client/sendProfile.py", line 211, in """ % excerpts UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128) 2009/8/26 Sebastian Pipping : > Christian Faulhammer wrote: >> Hi, >> >> Sebastian Pipping : >> >>> Christian Faulhammer wrote: That's a nice starting point to have a look if they aren't installed they are unpopular or because they fail to build (which makes them a candidate for removal). >>> I'm not following - how would we find out about the reason a package >>> is never reported installed? >> >> Own tests? Bugzilla? It is only to get an idea which packages might >> be at their of life. > > I see, good idea. > > > > Sebastian > > > -- Alexey 'Alexxy' Shvetsov Gentoo Linux Dev mailto:alexx...@gmail.com mailto:ale...@gentoo.org mailto:ale...@omrb.pnpi.spb.ru
Re: [gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Christian Faulhammer wrote: > Hi, > > Sebastian Pipping : > >> Christian Faulhammer wrote: >>> That's a nice starting point to have a look if they aren't installed >>> they are unpopular or because they fail to build (which makes them a >>> candidate for removal). >> I'm not following - how would we find out about the reason a package >> is never reported installed? > > Own tests? Bugzilla? It is only to get an idea which packages might > be at their of life. I see, good idea. Sebastian
[gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Hi, Sebastian Pipping : > Christian Faulhammer wrote: > > That's a nice starting point to have a look if they aren't installed > > they are unpopular or because they fail to build (which makes them a > > candidate for removal). > > I'm not following - how would we find out about the reason a package > is never reported installed? Own tests? Bugzilla? It is only to get an idea which packages might be at their of life. V-Li -- Christian Faulhammer, Gentoo Lisp project http://www.gentoo.org/proj/en/lisp/>, #gentoo-lisp on FreeNode http://gentoo.faulhammer.org/> signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Christian Faulhammer wrote: > That's a nice starting point to have a look if they aren't installed > they are unpopular or because they fail to build (which makes them a > candidate for removal). I'm not following - how would we find out about the reason a package is never reported installed? I've been working on the least-installed reporting stuff today. For now I have split it in two halves: (1) a top N least-installed packages table (only looking at packages with one install or more) (2) an plaintext file with a list of _all_ packages with zero installations reported Latter is linked from the report's TOC on the main page, too. Benefits I see with this approach include: - Complete zero-installs data is presented, not excerpts - Still, the report does not explode - Suitable for further machine-processing Small adjustments to this behaviour should be no problem. Sebastian (1) http://smolt.hartwork.org:45678/static/stats/gentoo.html#installed_packages_least_installed_main_tree (2) http://smolt.hartwork.org:45678/static/stats/gentoo_zero_installs_packages.txt
[gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Hi, Sebastian Pipping : > I'm aware that with currently ~13,000 packages in tree and ~4,000 > packages in the Gentoo Smolt database the zero-install list holds > ~7,000 packages. With more submission I expect that number to > decrease strongly in the future. > > With that in mind, what do you think? That's a nice starting point to have a look if they aren't installed they are unpopular or because they fail to build (which makes them a candidate for removal). V-Li -- Christian Faulhammer, Gentoo Lisp project http://www.gentoo.org/proj/en/lisp/>, #gentoo-lisp on FreeNode http://gentoo.faulhammer.org/> signature.asc Description: PGP signature
Re: [gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Nikos Chantziaras wrote: > [..] since [smoltGui] doesn't take a --server parameter [..] Fixed/added. http://git.goodpoint.de/?p=smolt-gentoo.git;a=commitdiff;h=707e98bd454ae416bec7870296ed108549275ecc Sebastian
Re: [gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Nikos Chantziaras wrote: > Seems to work OK now and I just submitted my data to smolt.hardwork.org. Great to hear, thank you. Sebastian
[gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
On 08/22/2009 07:30 PM, Sebastian Pipping wrote: [...] Distribution-specific No data, yet That "No data, yet" indicates you're not running the correct version of the Smolt client. Either that's a plain 1.3.x release or the "master" branch from my repo, not the "gentoo" one. Have you installed app-admin/gentoo-smolt- through portage? It has a blocker on !app-admin/smolt inside and EGIT_BRANCH="gentoo" in the first line. Have you installed it off the overlay through sudo ebuild foo merge or so? Due to stupidity, I had emerged smolt- instead of gentoo-smolt- :P Seems to work OK now and I just submitted my data to smolt.hardwork.org.
Re: [gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Nikos Chantziaras wrote: > (so that smoltGui can actually be > used at all since it doesn't take a --server parameter.) Good catch. Just opened a new task for it here: http://soc.gentooexperimental.org/issues/show/67 >> Before submission you can view all the data you submit. >> Near the bottom should be a long list of package details, right above >> the privacy metrics table. > > No, I don't get anything like that. I see. More on that further down. > I get a *lot* of errors at the > beginning; a *big* stream of: > > ERROR:dbus.proxies:Introspect error on :1.0:/org/freedesktop > /Hal/Manager: dbus.exceptions.DBusException: > org.freedesktop.DBus.Error.AccessDenied: Rejected send message, > 1 matched rules; type="method_call", sender=":1.27" (uid=0 pid=17288 > comm="/usr/bin/python) > interface="org.freedesktop.DBus.Introspectable" member="Introspect" > error name="(unset)" requested_reply=0 destination=":1.0" (uid=0 > pid=1668 comm="/usr/sbin/hald)) > [...several hundreds more snipped...] Sorry to hear. I haven't touched Smolt dbus code though and I cannot do anything about it at the moment. > And at the end: > > Smolt has collected four types of information: > > General > Default run level: 3 > Language: en_US.UTF-8 > OS: Gentoo Base System release 2.0.1 > ... > > Devices > [...snipped...] > > File system-related > [...snipped...] > > Distribution-specific > No data, yet That "No data, yet" indicates you're not running the correct version of the Smolt client. Either that's a plain 1.3.x release or the "master" branch from my repo, not the "gentoo" one. Have you installed app-admin/gentoo-smolt- through portage? It has a blocker on !app-admin/smolt inside and EGIT_BRANCH="gentoo" in the first line. Have you installed it off the overlay through sudo ebuild foo merge or so? Sebastian
[gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
On 08/22/2009 07:06 AM, Sebastian Pipping wrote: Nikos Chantziaras wrote: Done. Seems to work OK. Though there's no info about the scanning of packages and my profile page only lists hardware. I cannot find any new entries in the database. Have you been using --server=http://smolt.hartwork.org:45678/ on submission? I mentioned that in previous threads and didn't think of a need mentioning it again, sorry. Oops, never saw that, sorry. I simply followed the advice of the ebuild: * Call smoltSendProfile as root in order to initialize your profile. and thought that's enough. Perhaps the ebuild should mention the --server parameter, or even better install a config file that sets molt.hartwork.org:45678 as default (so that smoltGui can actually be used at all since it doesn't take a --server parameter.) Before submission you can view all the data you submit. Near the bottom should be a long list of package details, right above the privacy metrics table. No, I don't get anything like that. I get a *lot* of errors at the beginning; a *big* stream of: ERROR:dbus.proxies:Introspect error on :1.0:/org/freedesktop /Hal/Manager: dbus.exceptions.DBusException: org.freedesktop.DBus.Error.AccessDenied: Rejected send message, 1 matched rules; type="method_call", sender=":1.27" (uid=0 pid=17288 comm="/usr/bin/python) interface="org.freedesktop.DBus.Introspectable" member="Introspect" error name="(unset)" requested_reply=0 destination=":1.0" (uid=0 pid=1668 comm="/usr/sbin/hald)) [...several hundreds more snipped...] And at the end: Smolt has collected four types of information: General Default run level: 3 Language: en_US.UTF-8 OS: Gentoo Base System release 2.0.1 ... Devices [...snipped...] File system-related [...snipped...] Distribution-specific No data, yet The "v" answer also doesn't show anything Gentoo related. I don't know if it submitted them nonetheless. My profile is: http://smolt.hartwork.org:45678/client/show_all/pub_55c08510-d9f6-4cb0-832f-263e554e76ec
Re: [gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Jeremy Olexa wrote:
>> - Would zero-install packages be more interesting than
>>almost-zero ones or the other way around?
>
> I don't really understand this question. Does "zero-install" mean that
> they are not installed at all? This isn't really useful, because the
> ommition of a package from the page can mean that no one has it
> installed. ;)
By zero-install I mean a package from the main tree ("gentoo") that none
of the submitters has installed. To detect such packages I plan to
collect the list of currently in-tree packages from rsync and query the
submission database against it.
I'm aware that with currently ~13,000 packages in tree and ~4,000
packages in the Gentoo Smolt database the zero-install list holds ~7,000
packages. With more submission I expect that number to decrease
strongly in the future.
With that in mind, what do you think?
Sebastian
Re: [gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Nikos Chantziaras wrote: > Done. Seems to work OK. Though there's no info about the scanning of > packages and my profile page only lists hardware. I cannot find any new entries in the database. Have you been using --server=http://smolt.hartwork.org:45678/ on submission? I mentioned that in previous threads and didn't think of a need mentioning it again, sorry. Before submission you can view all the data you submit. Near the bottom should be a long list of package details, right above the privacy metrics table. Sebastian
[gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
On 08/22/2009 04:07 AM, Sebastian Pipping wrote: Nikos Chantziaras wrote: What version/edition of Smolt are you trying to commit with? The only available one: app-admin/smolt-1.2 Smolt 1.2 does not have the Gentoo-specific client code you need, yet. Please try again with app-admin/gentoo-smolt- from my "sping" overlay. Done. Seems to work OK. Though there's no info about the scanning of packages and my profile page only lists hardware.
[gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Sebastian Pipping wrote: Hello! Arrivals The third peport table on installed packages "most-unmasked" has just arrived: http://smolt.hartwork.org:45678/static/stats/gentoo.html#installed_packages_most_unmasked Questions = Before adding the forth "least-installed" table, I'd like to take the chance to ask for input from the treecleaner team, as it may be of most interest to you in the future. No need to add treecleaner to CC. I think we are all on this list anyway. ;) - Do you need anything that a dumb linear-with-counter approach cannot offer? Right now, I can't think of anything else actually. - Would zero-install packages be more interesting than almost-zero ones or the other way around? I don't really understand this question. Does "zero-install" mean that they are not installed at all? This isn't really useful, because the ommition of a package from the page can mean that no one has it installed. ;) What's next === Besides before-mentioned table the task Collect FEATURES variables in three sets (conf, defaults, globals), not merged http://soc.gentooexperimental.org/issues/show/58 is next on my list. Sebastian
Re: [gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Nikos Chantziaras wrote: >> What version/edition of Smolt are you trying to commit with? > > The only available one: app-admin/smolt-1.2 Smolt 1.2 does not have the Gentoo-specific client code you need, yet. Please try again with app-admin/gentoo-smolt- from my "sping" overlay. Sebastian
[gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
On 08/22/2009 02:11 AM, Sebastian Pipping wrote: Nikos Chantziaras wrote: Is there something special required to use smolt? I get to a page that tells me this after I submit my profile: Error: Critical: New versions of smolt use a public UUID. Yours is: pub_---- What version/edition of Smolt are you trying to commit with? The only available one: app-admin/smolt-1.2
Re: [gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
Nikos Chantziaras wrote: > Is there something special required to use smolt? I get to a page that > tells me this after I submit my profile: > > Error: Critical: New versions of smolt use a public UUID. Yours is: > pub_---- What version/edition of Smolt are you trying to commit with? Thanks for asking. Sebastian
[gentoo-dev] Re: Gentoo stats server/client @ 2009-08-22
On 08/22/2009 01:55 AM, Sebastian Pipping wrote: Hello! Arrivals The third peport table on installed packages "most-unmasked" has just arrived: http://smolt.hartwork.org:45678/static/stats/gentoo.html#installed_packages_most_unmasked Is there something special required to use smolt? I get to a page that tells me this after I submit my profile: Error: Critical: New versions of smolt use a public UUID. Yours is: pub_----
