Re: GNOME user survey 2011 (v6)
Hi, On Tue, Oct 18, 2011 at 1:51 PM, Olav Vitters wrote: > On Tue, Oct 18, 2011 at 05:15:37PM +0100, Sergey Udaltsov wrote: >> Provocative question: is there any way that some unbiased survey would >> change the emphasis of development from gnome-shell to the fallback mode? >> And increase the configurability and so on.. Or - the current strategy is >> unchangeable (unfalsifiable), regardless? > > See the release notes of 3.2. Feedback is used. > > Note: I care about feedback, not about surveys. That is just one > of many ways to get feedback. I think that has been discussed to death > already. > > Regarding fallback: > At the moment, it seems almost noone is using fallback mode. As such, I > don't think the current efforts made into fallback more will continue > for too long. Usage seems to be minimal. But not a lot of distributions > have GNOME 3 yet, so it is also a bit early to tell. > -- The latest version of Ubuntu (11.10 I guess) allows you to choose between Unity, Gnome Shell and Gnome Classic, which is, up to what I have seen (i dont use ubuntu but a friend of mine does) Gnome 3 with fallback mode on, so we might get some more people using it. Regards, José > Regards, > Olav > ___ > desktop-devel-list mailing list > [email protected] > http://mail.gnome.org/mailman/listinfo/desktop-devel-list > ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Wed, Oct 19, 2011 at 12:38:30PM +1300, John Stowers wrote: > Olav, I suggest you continue to moderate this thread. I predict nothing > good will come of it. Consider it killed. -- Regards, Olav ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Wed, Oct 19, 2011 at 09:58:15AM +0200, Mark wrote: > > False > > > false > > False > > > false > > False > > > false > > False > > > and false > You are false in all false points. The points are all from messages in this > thread so i recommend you to read back on the list since you obviously > missed something. > And i'm not trying to participate. I'm not part of gnome (anymore). I don't see when I say your viewpoints and trying to present them as facts is somehow trying to participate. > All i try to do is lay out the facts and let gnome realize what they see as > a "valid" survey is not possible. It would be in a perfect world but that > isn't the case. As stated before, the purpose is to gather feedback. That can be done in various ways. This survey is really really bad. Especially since loads of warnings have been given before, but no . > You threatening with moderation is really below the belt. I try to be > objective, state facts, be to the point and let gnome realize how realistic > their view of feedback is. I didn't insult anyone in person. Already stated that what you see as facts is just opinions. If I state that your facts aren't facts, and the only thing you do is reply by saying "false"... I am threatening with moderation, I was warning. Up to now, everything in this thread has been said before. Repeating the same discussions by ignoring all what has been said before.. I don't see the need. And yes, I am a sysadmin and hand out warning. Ignoring responses and summarize things as "we don't listen", despite e.g. what what was been said before, despite the changes listed in the release notes that were made based on feedback. I don't see the need. I don't see why someone would think Phoronix survey is not actively trying to influence the results. -- Regards, Olav ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Wed, Oct 19, 2011 at 9:21 AM, Olav Vitters wrote: > On Wed, Oct 19, 2011 at 01:09:18AM +0200, Mark wrote: > > Some facts. > > 1. Gnome wants feedback but any feedback gathered online is "non > > representative". It has to be gathered from a "non biased" site like > > cnn.com-_- guess the person who said that lives in a dream world or > > under a stone > > False > false > > > 2. Gnome gets feedback on _gnome's_own_list_of_questions_ but it's "non > > representative" ... > > False > false > > > 3. It's "useless" > > False > false > > > 4. Feedback should be looked for just by asking people, not on internet. > > False > and false > > > This has been extensively explained by lots of people before. Either > read up on what has been said before, or expect to be moderated. If you > want to participate, please at least read up on what was said before. > Just repeating your own beliefs and saying there facts is going to get > you nowhere. > You are false in all false points. The points are all from messages in this thread so i recommend you to read back on the list since you obviously missed something. And i'm not trying to participate. I'm not part of gnome (anymore). All i try to do is lay out the facts and let gnome realize what they see as a "valid" survey is not possible. It would be in a perfect world but that isn't the case. You threatening with moderation is really below the belt. I try to be objective, state facts, be to the point and let gnome realize how realistic their view of feedback is. I didn't insult anyone in person. > -- > Regards, > Olav > ___ > desktop-devel-list mailing list > [email protected] > http://mail.gnome.org/mailman/listinfo/desktop-devel-list > ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Wed, Oct 19, 2011 at 08:01:53AM +0100, Sergey Udaltsov wrote: > Ok, thanks for the reasonable answer. Don't you think it would make > sense for the GNOME to conduct such review officially? And perhaps > explicitly exclude GNOME developers from participation, to make it > unbiased;) This was already said before. Nothing wrong with doing a survey. But don't ignore the discussion and comments given on that survey. Just read all the previous threads. A lot is just repeating what has been said before. -- Regards, Olav ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Wed, Oct 19, 2011 at 01:09:18AM +0200, Mark wrote: > Some facts. > 1. Gnome wants feedback but any feedback gathered online is "non > representative". It has to be gathered from a "non biased" site like > cnn.com-_- guess the person who said that lives in a dream world or > under a stone False > 2. Gnome gets feedback on _gnome's_own_list_of_questions_ but it's "non > representative" ... False > 3. It's "useless" False > 4. Feedback should be looked for just by asking people, not on internet. False This has been extensively explained by lots of people before. Either read up on what has been said before, or expect to be moderated. If you want to participate, please at least read up on what was said before. Just repeating your own beliefs and saying there facts is going to get you nowhere. -- Regards, Olav ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Wed, Oct 19, 2011 at 1:02 AM, Shaun McCance wrote: > I can't speak for everybody. But as I've said numerous times, > I don't trust any survey that has no control over its sampling. > And preferably, it would be run by people who have some amount > of professional experience. Ok, thanks for the reasonable answer. Don't you think it would make sense for the GNOME to conduct such review officially? And perhaps explicitly exclude GNOME developers from participation, to make it unbiased;) ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
> The metaphore mismatch in two points - the users of GNOME have the power > to fork it (I heard even calls to fork GNOME before 3.0) so if > sufficient number of developers decides to maintain GNOME 2 it may still > live (probably under different name - IANAL and I'm not sure about > trademark etc. details). (Whether the discontent have necessary skills > is of course different matter). > > GNOME is also only a part of stack and I don't think GNOME 2 will stop > working with, say, kernel 3.6 which would include new drivers etc. the > same goes for sane/cups/mesa/... hence it will unlikely have issues with > new HW. I am talking about users, not developers. For the users my point remains, my metaphor is valid. Users do not fork, do not develop. They do not upgrade parts of the stack - they upgrade distros. Have you heard the opinions like "I will stay with existing version of Debian as long as I can - since it runs 2.32, then will switch to XFCE?" I did. Sergey ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On 18/10/2011 18:15, Sergey Udaltsov wrote: >> What's stopping these deprived users from using Gnome 2.X? I don't think >> there's enough developers interested in keeping the 2.X series alive - it >> would be a different matter if people were smashing out the features/patches >> for the 2.X range but as that's not happening I don't see why they don't >> stay with what works for them? > People upgrade distros. They upgrade HW. Would you advise people who > love WinXP and hate Vista(ot Win7) stay with WinXP - considering that > it has issues with new HW? > > Sergey > ___ > desktop-devel-list mailing list > [email protected] > http://mail.gnome.org/mailman/listinfo/desktop-devel-list The metaphore mismatch in two points - the users of GNOME have the power to fork it (I heard even calls to fork GNOME before 3.0) so if sufficient number of developers decides to maintain GNOME 2 it may still live (probably under different name - IANAL and I'm not sure about trademark etc. details). (Whether the discontent have necessary skills is of course different matter). GNOME is also only a part of stack and I don't think GNOME 2 will stop working with, say, kernel 3.6 which would include new drivers etc. the same goes for sane/cups/mesa/... hence it will unlikely have issues with new HW. Regards signature.asc Description: OpenPGP digital signature ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Wed, 2011-10-19 at 00:26 +0100, Sergey Udaltsov wrote: > What I initially asked - and still did not get the answer - what could > be the format of the feedback that could change the policies. Perhaps > reverting some of them. What kind of critical feedback would not be > treated as "useless"? I can't speak for everybody. But as I've said numerous times, I don't trust any survey that has no control over its sampling. And preferably, it would be run by people who have some amount of professional experience. -- Shaun ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Wed, 2011-10-19 at 01:09 +0200, Mark wrote: > I really want to drop in here. > I on purposely say "gnome" instead of "you" to avoid giving the > impression that i attack anyone. Honestly, given your hostile tone, it instead comes off as if you're attacking everybody. I'm going to try to assume you're frustrated, and that you don't mean to offend. > Some facts. > 1. Gnome wants feedback but any feedback gathered online is "non > representative". It has to be gathered from a "non biased" site like > cnn.com -_- guess the person who said that lives in a dream world or > under a stone My objections to the survey are well-documented on this list. I clearly stated multiple times that an open-invite internet survey of any kind cannot control its sampling or control for selection bias in any way. The person who mentioned CNN didn't say that a survey on cnn.com would be scientific. He said *all* web surveys self-select, and that phoronix self-selects moreso. This is simply a product of who reads the sites, and who you want to reach. Most of the people who create GNOME are professional developers, or are working towards becoming professionals. Perhaps surveys ought to be run by professional statisticians, or quantitative social science professionals. And among the many reasons why this survey is clearly biased and poorly run, let's not forget that cherry-picking negative (and overtly hostile) comments before the data is even published is about the least professional thing I've ever seen in a survey. > 2. Gnome gets feedback on _gnome's_own_list_of_questions_ but it's > "non representative" ... The majority of GNOME developers did not participate in creating these questions. Of those that did, most were not supportive of the survey as deployed. To characterize it as "GNOME's own list of questions" is extremely disingenuous. > 3. It's "useless" > 4. Feedback should be looked for just by asking people, not on > internet. > Really, is gnome KIDDING me? How unrealistic can gnome be! And if > phoronix isn't a site for user feedback then what is? For who is Gnome > even targeted? I'm sure if you ask a dozen different GNOME developers, you'll get some different answers. But mostly, I think GNOME targets people who don't want to think about what terms like "operating system", "desktop environment", or "distribution" mean. Many technology enthusiasts enjoy GNOME. In fact, those of us who create GNOME are technology enthusiasts. But we set the bar higher than our own tolerance for technology pain. -- Shaun ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
> > Phoronix and any linux news orientated site would be _perfect_ for a > Gnome survey! If gnome thinks otherwise then keep on living in that > little perfect utopia world of gnome. Reality is way different. Gnome > really seems to be living in some ideal small "everyone loves gnome" > world where they can tap in sites with millions of unbiased users that > all give unbiased objective feedback.. Wake up gnome, not gonna > happen! Read this: http://www.phoronix.com/scan.php?page=news_item&px=MTAwMjY and ask that question again. I think it confirms the worst suspicions of those questioning the usefulness of this survey. What actionable items or lessons do you see in those 'early results'. There is no useful feedback there, all I see is fail and a black mark on the reputation of phoronix. Olav, I suggest you continue to moderate this thread. I predict nothing good will come of it. John ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
> I really want to drop in here. > I on purposely say "gnome" instead of "you" to avoid giving the impression that i attack anyone. Mark, I am afraid that still looks like an attack... While in general I agree with you, I guess the format of your message is not appropriate. It does not make sense to occuse people making decisions. You'll just alienate them. What I initially asked - and still did not get the answer - what could be the format of the feedback that could change the policies. Perhaps reverting some of them. What kind of critical feedback would not be treated as "useless"? ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
> AFAIK the goal was to only maintain it until the very last graphics > chip in use was able to run shell. It's not there as a preference, > it's a fallback mode for unsupported hardware. Plenty of people see it as a preference, but right now on the hardware side there are plenty of chipsets without 3D support or where it's not good enough for Gnome 3. As a starter in recent/currently available chipsets you can add - Some Intel gen chipsets with > 2048 pixel wide displays - All the USB plug in displays - Imagination based hardware and I'm sure there are plenty more. They don't I suspect need fallback mode though, all of the examples I can think of that are current have very fast framebuffer access for pushing bits, usually host memory based (the USB one update is slower but not the draw rates). E/EVAS manages to do pretty much everything Gnome 3 non fallback does effectwise on such chipsets snappily (often faster than Gnome 3 feels on hardware 3D), so really it ought to be a case for the most part of fixing the broke dependancies of Gnome 3 on 3D hardware. You can do drop shadows, shading, scaling of a flat 2D image and the like very fast with the CPU. I do wonder if Gnome 3 had been based on the E canvas whether any of the problem would have occurred in the first place ? Alan ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Tue, Oct 18, 2011 at 6:42 PM, Patryk Zawadzki wrote: > On Tue, Oct 18, 2011 at 6:34 PM, Sergey Udaltsov > wrote: > > Iirc the fallback mode is using new gtk and stuff... why is it obsolete? > > AFAIK the goal was to only maintain it until the very last graphics > chip in use was able to run shell. It's not there as a preference, > it's a fallback mode for unsupported hardware. > > > I was asking looking at the anger and nostalgie expressed on phoronix. > > Phoronix is a tabloid seeking sensation. They feed on flames, FUD and > “scandals” so their readership is far from being average end users. > It's not the crowd you can cater to as whenever you “fix” one > “problem” they will quickly find a new thing to hate. > > -- > Patryk Zawadzki > I solve problems. > I really want to drop in here. I on purposely say "gnome" instead of "you" to avoid giving the impression that i attack anyone. Some facts. 1. Gnome wants feedback but any feedback gathered online is "non representative". It has to be gathered from a "non biased" site like cnn.com-_- guess the person who said that lives in a dream world or under a stone 2. Gnome gets feedback on _gnome's_own_list_of_questions_ but it's "non representative" ... 3. It's "useless" 4. Feedback should be looked for just by asking people, not on internet. Really, is gnome KIDDING me? How unrealistic can gnome be! And if phoronix isn't a site for user feedback then what is? For who is Gnome even targeted? Linux distributions with Desktop Environments are installed by (usually) the more technical people, the same people who infact _do_ visit phoronix simply because it's the only and thus biggest linux news site out there. Phoronix and any linux news orientated site would be _perfect_ for a Gnome survey! If gnome thinks otherwise then keep on living in that little perfect utopia world of gnome. Reality is way different. Gnome really seems to be living in some ideal small "everyone loves gnome" world where they can tap in sites with millions of unbiased users that all give unbiased objective feedback.. Wake up gnome, not gonna happen! Asking unbiased thus people that have never been using gnome is asking first impressions. Not something you would want in your feedback. You would want feedback of users that have been using gnome for a (long) while and are thus able to give real constructive feedback. Exactly the kind of people that visit phoronix and thus a perfect site for asking feedback. What i think is really going on here is that gnome sees the feedback and doesn't like it. Gnome would obviously like to see feedback like "Ohh, good job gnome! btw, gnome shell really rocks!" ... Apparently the vast majority of gnome users (and you can leave ubuntu out since it has it's own interface thus it's own gnome experience) simply don't like what gnome did with the move to Gnome Shell. The people gnome targets don't like gnome anymore and switch to KDE and XFCE. And those are facts! Then we have the settings. Gnome threats people like they are stupid and takes away as much options as possible. Another fact is that most linux users _like_ to have more options so gnome is very much acting like "i know all, you are stupid. Take it or leave it" and isn't listening to it's users _at_all_ Just face it. More options instead of less are wanted by the majority of gnome's user base for years. And what does gnome do as another stubborn move? ... take more options out ... Gnome did the wrong thing by introducing Gnome Shell. It was known way before it was released but gnome was, yet again, extremely stubborn and ignorant to the opinion of the users. Gnome just doesn't listen and that is gonna bite gnome at some point in time. Advanced users just don't like gnome anymore. There might be a few exceptions like for example gnome's own developers ;) So, gnome, please just take this message. Do something about your attitude and listen to your own users! Be friendly towards them, help them and give them a good experience. That does also mean to drop shell in my opinion (and come up with something better). I really do wonder how much people are using gnome (shell, unity and "fallback") now.. I think it's user base declined rapidly with the gnome 3.0 release. XFCE an KDE probably saw usage increases around the same time. I wish there where numbers on that somewhere. Don't take this message the wrong way. I post it with good intentions and for gnome to take it and learn from it. I'm just being fair, open and stating facts. Kind regards, Mark ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Tue, Oct 18, 2011 at 08:20:18PM +0200, Rovanion Luckey wrote: > 2011/10/18 Olav Vitters > > > At the moment, it seems almost noone is using fallback mode. As such, I > > don't think the current efforts made into fallback more will continue > > for too long. Usage seems to be minimal. But not a lot of distributions > > have GNOME 3 yet, so it is also a bit early to tell. > > > > How do you know that almost noone is using the fallback mode? I said that it seems noone is using it, didn't say I know. Also said it is too early to tell properly. I build up impressions by: - reading the amount of questions on 50+ GNOME mailing lists - following various distribution mailing lists (opensuse, Fedora, Mageia) - a few IRC channels (GNOME, Mageia) - sites such as tweakers.net, news.ycombinator.com, lwn.net, slashdot.org, webwereld.net, h-online.com, and a few others - amount of bugs on Bugzilla (GNOME, Mageia) -- Regards, Olav ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
2011/10/18 Olav Vitters > At the moment, it seems almost noone is using fallback mode. As such, I > don't think the current efforts made into fallback more will continue > for too long. Usage seems to be minimal. But not a lot of distributions > have GNOME 3 yet, so it is also a bit early to tell. > How do you know that almost noone is using the fallback mode? -- www.twitter.com/Rovanion ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
> What's stopping these deprived users from using Gnome 2.X? I don't think > there's enough developers interested in keeping the 2.X series alive - it > would be a different matter if people were smashing out the features/patches > for the 2.X range but as that's not happening I don't see why they don't > stay with what works for them? People upgrade distros. They upgrade HW. Would you advise people who love WinXP and hate Vista(ot Win7) stay with WinXP - considering that it has issues with new HW? Sergey ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
> for too long. Usage seems to be minimal. But not a lot of distributions > have GNOME 3 yet, so it is also a bit early to tell. Exactly. Let's wait till all distros outphase gnome 2.x ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
> > > Phoronix is a tabloid seeking sensation. > Agree. But I guess it is not a surprise that some users are crying for good > old gnome2. If gnome could properly estimate the share of those deprived... > would it change anything? What's stopping these deprived users from using Gnome 2.X? I don't think there's enough developers interested in keeping the 2.X series alive - it would be a different matter if people were smashing out the features/patches for the 2.X range but as that's not happening I don't see why they don't stay with what works for them? Regards, Nick ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Tue, Oct 18, 2011 at 05:15:37PM +0100, Sergey Udaltsov wrote: > Provocative question: is there any way that some unbiased survey would > change the emphasis of development from gnome-shell to the fallback mode? > And increase the configurability and so on.. Or - the current strategy is > unchangeable (unfalsifiable), regardless? See the release notes of 3.2. Feedback is used. Note: I care about feedback, not about surveys. That is just one of many ways to get feedback. I think that has been discussed to death already. Regarding fallback: At the moment, it seems almost noone is using fallback mode. As such, I don't think the current efforts made into fallback more will continue for too long. Usage seems to be minimal. But not a lot of distributions have GNOME 3 yet, so it is also a bit early to tell. -- Regards, Olav ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
> AFAIK the goal was to only maintain it until the very last graphics > chip in use was able to run shell. It's not there as a preference, > it's a fallback mode for unsupported hardware. Absolutely! My question was exactly about that - is there theoretical possibility that proper survey would amend those goals. > Phoronix is a tabloid seeking sensation. Agree. But I guess it is not a surprise that some users are crying for good old gnome2. If gnome could properly estimate the share of those deprived... would it change anything? ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Tue, 2011-10-18 at 17:34 +0100, Sergey Udaltsov wrote: > Iirc the fallback mode is using new gtk and stuff... why is it > obsolete? Is this another provocative question? It's a fallback that *by definition* uses older technologies to have something usable on hardware that do not support gnome-shell. gnome-panel might use GTK3, but that doesn't make it on-par with the non-fallback experience in any way... > I was asking looking at the anger and nostalgie expressed on phoronix. I feel like the survey was explicitly intended to provoke such emotions in a specific audience; so I don't think those results are really valuable in choosing the direction GNOME should go. Cosimo ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Tue, Oct 18, 2011 at 6:34 PM, Sergey Udaltsov wrote: > Iirc the fallback mode is using new gtk and stuff... why is it obsolete? AFAIK the goal was to only maintain it until the very last graphics chip in use was able to run shell. It's not there as a preference, it's a fallback mode for unsupported hardware. > I was asking looking at the anger and nostalgie expressed on phoronix. Phoronix is a tabloid seeking sensation. They feed on flames, FUD and “scandals” so their readership is far from being average end users. It's not the crowd you can cater to as whenever you “fix” one “problem” they will quickly find a new thing to hate. -- Patryk Zawadzki I solve problems. ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
Iirc the fallback mode is using new gtk and stuff... why is it obsolete? I was asking looking at the anger and nostalgie expressed on phoronix. On Oct 18, 2011 5:29 p.m., "Cosimo Cecchi" wrote: > On Tue, 2011-10-18 at 17:15 +0100, Sergey Udaltsov wrote: > > > Provocative question: is there any way that some unbiased survey would > > change the emphasis of development from gnome-shell to the fallback > > mode? And increase the configurability and so on.. Or - the current > > strategy is unchangeable (unfalsifiable), regardless? > > How are those two things even connected? How could switching the > development emphasis to work on obsolete technologies help anybody at > all (users/developers/designers and so on)? > > Cosimo > > ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Tue, 2011-10-18 at 17:15 +0100, Sergey Udaltsov wrote: > Provocative question: is there any way that some unbiased survey would > change the emphasis of development from gnome-shell to the fallback > mode? And increase the configurability and so on.. Or - the current > strategy is unchangeable (unfalsifiable), regardless? How are those two things even connected? How could switching the development emphasis to work on obsolete technologies help anybody at all (users/developers/designers and so on)? Cosimo ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
Would anybody have time to prepare some useful survey? Provocative question: is there any way that some unbiased survey would change the emphasis of development from gnome-shell to the fallback mode? And increase the configurability and so on.. Or - the current strategy is unchangeable (unfalsifiable), regardless? Sergey On Oct 18, 2011 5:03 p.m., "Jasper St. Pierre" wrote: > It's useless to me because there's nothing actionable there. The > survey results don't give us anything to do except die in a fire. > -- > Jasper > ___ > desktop-devel-list mailing list > [email protected] > http://mail.gnome.org/mailman/listinfo/desktop-devel-list > ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
It's useless to me because there's nothing actionable there. The survey results don't give us anything to do except die in a fire. -- Jasper ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On 18 October 2011 16:52, Olav Vitters wrote: > Such actions just confirms that the effort was not an honest intention > to gather feedback. Just to confirm own thoughts. I don't think many of us on this list thought the intention of the survey was to highlight areas needing improvement in GNOME. I think it's best if we just let the "Real Users" comment on phoronix without it getting into a huge troll-fest. That way there's no distraction for us developers and designers who are working on GNOME 3.3. Richard. ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Tue, Oct 18, 2011 at 10:35:24AM -0500, Dan Williams wrote: > On Tue, 2011-10-18 at 17:27 +0200, Olav Vitters wrote: > > On Tue, Oct 18, 2011 at 04:54:12PM +0200, Mark wrote: > > > FYI: http://www.phoronix.com/scan.php?page=news_item&px=MTAwMjY > > > > Useless. > > Web surveys are guaranteed to self-select, and I have to imagine a > survey hosted on phoronix self-selects a ton more than one on cnn.com or > msn.com. Useful surveys are ones that go out and find users, instead of > depending on users to find the survey. That and various other things have been mentioned beforehand, and was ignored. Read for instance the "survey" page itself: http://www.phoronix.com/scan.php?page=news_item&px=MTAwMjI "Felipe Contreras and others have been working on a GNOME survey for a while now, but the GNOME developers or the GNOME Foundation weren't interested in endorsing such a survey. But the survey has the blessing of Alan Cox and " totally ignores that most concerns were ignored and the pretty annoying/bad behaviour displayed on the mailing list. Furthermore, if you do a survey, at least try to solicit unbiased feedback. Saying developers "weren't interested" misrepresents what happened. Furthermore, this influences the responses you will get. I don't see why this is added if you care to avoid bias. Then while the survey is running articles are made taking various highly critical (but useless feedback) and this is used to suggest what "GNOME users" think. This while beforehand it was said that an internet survey is not representative. It is still being presented by Phoronix as such. I don't think any of the 600.000 Spanish users saw any notice for a survey. Was specifically warned about beforehand. Such actions just confirms that the effort was not an honest intention to gather feedback. Just to confirm own thoughts. As such, I find it useless. Just waiting for the next misrepresenting article/blog/etc. :P -- Regards, Olav ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Tue, 2011-10-18 at 17:27 +0200, Olav Vitters wrote: > On Tue, Oct 18, 2011 at 04:54:12PM +0200, Mark wrote: > > FYI: http://www.phoronix.com/scan.php?page=news_item&px=MTAwMjY > > Useless. Web surveys are guaranteed to self-select, and I have to imagine a survey hosted on phoronix self-selects a ton more than one on cnn.com or msn.com. Useful surveys are ones that go out and find users, instead of depending on users to find the survey. Dan ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Tue, Oct 18, 2011 at 04:54:12PM +0200, Mark wrote: > FYI: http://www.phoronix.com/scan.php?page=news_item&px=MTAwMjY Useless. -- Regards, Olav ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Tue, Sep 20, 2011 at 12:17 AM, Alan Cox wrote: > > There's been a lot of work done to improve GNOME 3 over the last 6 > > months. A lot of the complaints of GNOME 3.0 have been already > > addressed. Why not just do it after (even more!) distros ship GNOME > > 3.2? > > The first one is probably going to shed more light on what should be > asked than anything else. So why not do it now ? It will also be a basis > upon which you can compare a 2012 survey. > > Alan > ___ > desktop-devel-list mailing list > [email protected] > http://mail.gnome.org/mailman/listinfo/desktop-devel-list > FYI: http://www.phoronix.com/scan.php?page=news_item&px=MTAwMjY ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
> There's been a lot of work done to improve GNOME 3 over the last 6 > months. A lot of the complaints of GNOME 3.0 have been already > addressed. Why not just do it after (even more!) distros ship GNOME > 3.2? The first one is probably going to shed more light on what should be asked than anything else. So why not do it now ? It will also be a basis upon which you can compare a 2012 survey. Alan ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Tue, Sep 20, 2011 at 2:38 AM, Felipe Contreras wrote: > On Mon, Sep 19, 2011 at 11:05 PM, Ionut Biru wrote: [snip] >> In my opinion this survey should be published after gnome 3.2 is presented >> to a larger audience, now that ubuntu 11.10 is going to have it, opensuse >> 12.1 > > That would be what? December? I think that's too far, perhaps for the > 2012 survey. > Why does the survey *have* to be done *right now* at all? I thought the primary aim of this survey was to give useful feedback which would be used to improve GNOME. The first step in evaluating software is to use the latest version — there's no use asking people for feedback about GNOME 3.0 when 3.2 is coming out in a week. There's been a lot of work done to improve GNOME 3 over the last 6 months. A lot of the complaints of GNOME 3.0 have been already addressed. Why not just do it after (even more!) distros ship GNOME 3.2? Cheers, -- ~Nirbheek Chauhan Gentoo GNOME+Mozilla Team ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On 19 September 2011 17:08, Felipe Contreras wrote: > On Mon, Sep 19, 2011 at 11:05 PM, Ionut Biru wrote: >> I didn't participate to this discussion before but i think the survey is >> pointless now because GNOME 3 wasn't presented to users at all. >> >> From the top 10 mainstream distributions, conform distrowatch, only 2 of >> them have gnome 3.0. >> >> In my opinion this survey should be published after gnome 3.2 is presented >> to a larger audience, now that ubuntu 11.10 is going to have it, opensuse >> 12.1 > > That would be what? December? I think that's too far, perhaps for the > 2012 survey. I agree with Ionut. For big distros, GNOME 3 has only been officially released on Fedora & Arch. If you want to get the opinions of normal users, it would be better to wait a few more months to pick up OpenSUSE, Ubuntu, maybe even Debian unstable, and Fedora 16 (of course Fedora 15 had GNOME 3 too), etc. Jeremy ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On Mon, Sep 19, 2011 at 11:05 PM, Ionut Biru wrote: > I didn't participate to this discussion before but i think the survey is > pointless now because GNOME 3 wasn't presented to users at all. > > From the top 10 mainstream distributions, conform distrowatch, only 2 of > them have gnome 3.0. > > In my opinion this survey should be published after gnome 3.2 is presented > to a larger audience, now that ubuntu 11.10 is going to have it, opensuse > 12.1 That would be what? December? I think that's too far, perhaps for the 2012 survey. -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v6)
On 09/19/2011 07:38 PM, Felipe Contreras wrote: Hi, Since I have been effectively banned from desktop devel (my posts take two weeks to be "moderated"), I am sending this mail personally to people that have been active in the development. Michael Larabel has offered to host the survey in the Phoronix site, so I have been able to bring back many questions and not limit it to 10. I have incorporated all the suggestions and haven't had received any more in a while, so I think this is ready to go. I say we should launch it before the weekend, probably on Wednesday. Michael, is there anything else I need to do to help you put it on Phoronix? As usual, it's hosted here: https://gist.github.com/gists/1128166 Cheers. GNOME user survey 2011 === 03. Overall, how satisfied are you with GNOME? === [single choice] * not at all * barely * halfway * mostly * completely I didn't participate to this discussion before but i think the survey is pointless now because GNOME 3 wasn't presented to users at all. From the top 10 mainstream distributions, conform distrowatch, only 2 of them have gnome 3.0. In my opinion this survey should be published after gnome 3.2 is presented to a larger audience, now that ubuntu 11.10 is going to have it, opensuse 12.1 -- Ionuț ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
GNOME user survey 2011 (v6)
Hi, Since I have been effectively banned from desktop devel (my posts take two weeks to be "moderated"), I am sending this mail personally to people that have been active in the development. Michael Larabel has offered to host the survey in the Phoronix site, so I have been able to bring back many questions and not limit it to 10. I have incorporated all the suggestions and haven't had received any more in a while, so I think this is ready to go. I say we should launch it before the weekend, probably on Wednesday. Michael, is there anything else I need to do to help you put it on Phoronix? As usual, it's hosted here: https://gist.github.com/gists/1128166 Cheers. GNOME user survey 2011 === 01. Do you know what GNOME is? === [single choice] * Yes [skip to 03] * No === 02. Which of the following best resemble your desktop? === (click to see the image) [single choice] - Windows http://origin.arstechnica.com/images/windows7/Peek%20-%20Before.png - Mac OS X http://www.guidebookgallery.org/pics/gui/desktop/firstrun/macosx103.png - GNOME 2 http://library.gnome.org/misc/release-notes/2.28/figures/gnome-2.28.png.en - GNOME 3 http://gnome3.org/img/overview-big.png - Unity http://static.arstechnica.com/shell-windows.png - KDE http://www.linuxnov.com/wp-content/uploads/2011/07/KDE-4-7-desktop.jpg - I can't tell === 03. Overall, how satisfied are you with GNOME? === [single choice] * not at all * barely * halfway * mostly * completely === 04. Does GNOME do what you want? === [single choice] * not at all * barely * halfway * mostly * completely === 05. How satisfied are you with GNOME in regards to == [matrix] Columns: not at all / barely / halfway / mostly / completely + ease of use + documentation + language availability + accessibility + community === 06. How are you taking this survey? === [single choice, with other] * Completely on my own * Somebody is pushing for me to do it * I am acting on behalf of somebody else * Other === 07. How old are you? (years) === [numeric] === 08. How long have you been using GNOME? (years) === [numeric] === 09. How many years of experience do you have using computers? === [numeric] === 10. How do you compare your current GNOME version with the version from one year ago? === [single choice] * better * no changes * worse * cannot say === 11. Which GNOME version(s) are you using? === [multiple choice, with other] + 3.2 + 3.0 + 2.x + I don't know + I'm not using it currently + other, please specify === 12. Where do you run GNOME? === [multiple choice, with other] + Desktop + Laptop + Netbook + Tablet === 13. How often do you use a terminal/console? == [single choice] * What is that? * When I have no other option * I can't live without them * Is there anything else? === 14. Have you contributed to the GNOME project? === [single choice] * Yes * No === 15. Have you contacted the GNOME team? === [single choice] * Yes, successfully * Yes, unsuccessfully * No, I don't know how * No, never had the need === 16. Which other desktop environments have you used in recent years? == [multiple choice, with other] + KDE + Unity + XFCE + LXDE + Enlightenment + other (please specify) === 17. Are you using some window arrangement extension on top of GNOME? == (e.g. Compiz + plugins, Awesome TWM + GNOME, etc) [single choice, with other] * No, pure GNOME * GNOME + Compiz window arrangement plugins * I don't know * Other (please specify) === 18. If you could change three things in GNOME, what would they be? === [free form] === 19. Do you have any comments or suggestions for the GNOME team? === [free form] -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v5)
On Sun, Sep 18, 2011 at 7:04 PM, Shaun McCance wrote: > On Tue, 2011-09-06 at 23:38 +0300, Felipe Contreras wrote: >> Here's the next version of the survey. Unfortunately, it seems this is >> not going to be blessed by GNOME, and questionpro.com only allows 10 >> questions in the free version. I haven't found a better free online >> survey, and unless somebody offers hosting for this survey, it would >> have to be limited. > > [snip] > >> What about other kinds of bias? Would the survey be invalidated if we >> missed some group of people? > > Please look up "non-response bias". It's been shown that, in some cases, > the group of respondents (and non-respondents) can be inherently biased. > It's reasonable to suspect that polarized opinions could affect people's > proclivity to respond. People's proclivity to respond is called self-selection bias, and it's tackled with the question "What is the main reason you are taking this survey?", if people respond "Somebody is pushing me", we can identify people without such proclivity and measure the bias of self-selected people. As for non-response bias, I am aware of it, and I am doing all I can to mitigate it. > You cannot even begin to deal with non-response bias unless you know the > non-response rate. And you have no idea what the non-response rate is if > you have an open-invitation survey on the Internet. We don't need to know the exact non-response rate, studies have shown it's not that important[1]. --- As a result of these and other such recent findings, it now seems clear that a low response rate does not guarantee lower survey accuracy and instead simply indicates a risk of lower accuracy. --- Now. Do you actually have a suggestion? [1] http://en.wikipedia.org/wiki/Response_rate -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v5)
On Thu, Sep 15, 2011 at 7:46 PM, Jason D. Clinton wrote: > On Tue, Sep 6, 2011 at 15:38, Felipe Contreras > wrote: >> Unfortunately, it seems this is >> not going to be blessed by GNOME, and questionpro.com only allows 10 >> questions in the free version. I haven't found a better free online >> survey, and unless somebody offers hosting for this survey, it would >> have to be limited. > > Google Docs Spreadsheets has a survey system built in to it called > Forms. It allows anonymous respondents. Actually, it will be hosted on Phoronix. >> One problem raised was the issue of self-selection bias, of course, >> without any suggestions to get rid of it. > > That's false. I gave you two strategies: > >> You can mitigate this problem by >> offering a survey that appears to have nothing to do with the subject >> matter that you're really looking for an answer on so that you get a >> truly random sampling of Linux users. You also must be careful not to >> recruit people to take the survey from communities which will contain >> angry people. For example, going to forums to find people to take a >> survey automatically selectively biases from people who were likely >> there to solve some kind of problem and are so already in a particular >> state of mind.[1] Those "strategies" are unactionable. In this interconnected world one cannot choose who doesn't get a piece of information. > And you still haven't addressed the biased question phrasing in > questions 2 and 3. Yes I have, according to the suggestions proposed. I you have a suggestion, go ahead. -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v5)
On Tue, 2011-09-06 at 23:38 +0300, Felipe Contreras wrote: > Hi, > > Here's the next version of the survey. Unfortunately, it seems this is > not going to be blessed by GNOME, and questionpro.com only allows 10 > questions in the free version. I haven't found a better free online > survey, and unless somebody offers hosting for this survey, it would > have to be limited. [snip] > What about other kinds of bias? Would the survey be invalidated if we > missed some group of people? Please look up "non-response bias". It's been shown that, in some cases, the group of respondents (and non-respondents) can be inherently biased. It's reasonable to suspect that polarized opinions could affect people's proclivity to respond. You cannot even begin to deal with non-response bias unless you know the non-response rate. And you have no idea what the non-response rate is if you have an open-invitation survey on the Internet. -- Shaun ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v5)
On Tue, Sep 6, 2011 at 15:38, Felipe Contreras wrote: > Unfortunately, it seems this is > not going to be blessed by GNOME, and questionpro.com only allows 10 > questions in the free version. I haven't found a better free online > survey, and unless somebody offers hosting for this survey, it would > have to be limited. Google Docs Spreadsheets has a survey system built in to it called Forms. It allows anonymous respondents. > One problem raised was the issue of self-selection bias, of course, > without any suggestions to get rid of it. That's false. I gave you two strategies: > You can mitigate this problem by > offering a survey that appears to have nothing to do with the subject > matter that you're really looking for an answer on so that you get a > truly random sampling of Linux users. You also must be careful not to > recruit people to take the survey from communities which will contain > angry people. For example, going to forums to find people to take a > survey automatically selectively biases from people who were likely > there to solve some kind of problem and are so already in a particular > state of mind.[1] And you still haven't addressed the biased question phrasing in questions 2 and 3. [1] http://mail.gnome.org/archives/desktop-devel-list/2011-August/msg00051.html ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
GNOME user survey 2011 (v5)
Hi, Here's the next version of the survey. Unfortunately, it seems this is not going to be blessed by GNOME, and questionpro.com only allows 10 questions in the free version. I haven't found a better free online survey, and unless somebody offers hosting for this survey, it would have to be limited. Now, there has been a misconception that I am not listening to some feedback, which is not true. From my point of view there have been two kinds of feedback: a) Criticism to improve the survey All these comments have been addressed in one form or another. Unfortunately, I am forced to drop some of the suggested questions to fit the 10 question limit. Further criticism is still welcome. b) Criticism without possibility for improvements Many people in this category have already accepted that they don't think an online user survey is actually possible, so there's really no way to address these. One problem raised was the issue of self-selection bias, of course, without any suggestions to get rid of it. All online surveys have this issue, however, my proposal was actively suggest people to push others to take the survey (those people would not be self-selected), and have a question to identify them. Then we can ignore the self-selected people, or have two analyses. Of course, I didn't get any response. In a private discussion, Matthew Garret pointed out that there's still the issue that some people might not have friends, and thus there wouldn't be people to push them to take the survey. While this is true, this is something not even professionals can overcome, the only thing possible is to make a guess about the amount of people in this category, and thus the amount of bias. What about other kinds of bias? Would the survey be invalidated if we missed some group of people? No. There's a process called poststratification that would allow us to make valid guesses. Yes, it might turn out that in the end we don't have enough statistical power to make any conclusions, but we won't be able to say that until we get the results. Another misconception is that this survey might cause some damage, by drawing the wrong conclusions. As I already explained before, raw data is raw data, and it can't cause damage. What can cause damage is to make decisions based on wrong conclusions, which is why GNOME developers might want to ignore the conclusions. Or different people might try to do different post-hoc power analysis to try to get some consensus on what conclusions are actually valid. I will continue to listen for suggestions for improvement, but I fear GNOME developers don't want to continue improving this survey in this mailing list, so feel free to contact me personally if you are interested in v6 (if needed), or follow this: https://gist.github.com/1128166 Cheers. === 01. Which of the following best resemble your desktop? === (click to see the image) (in case you are not sure what GNOME is) [single choice] - Windows http://origin.arstechnica.com/images/windows7/Peek%20-%20Before.png - Mac OS X http://www.guidebookgallery.org/pics/gui/desktop/firstrun/macosx103.png - GNOME 2 http://library.gnome.org/misc/release-notes/2.28/figures/gnome-2.28.png.en - GNOME 3 http://gnome3.org/img/overview-big.png - Unity http://static.arstechnica.com/shell-windows.png - KDE http://www.linuxnov.com/wp-content/uploads/2011/07/KDE-4-7-desktop.jpg - I can't tell === 02. Overall, how satisfied are you with GNOME? === [single choice] * not at all * barely * halfway * mostly * completely === 03. Does GNOME do what you want? === [single choice] * not at all * barely * halfway * mostly * completely === 04. What is the main reason you are taking this survey? === [single choice, with other] * Somebody is pushing me * I want to provide feedback to the project * I feel I have to * Other === 05. How many years of experience do you have using computers? === [numeric] === 06. How often do you use a terminal/console? == [single choice] * What is that? * When I have no other option * I can't live without them * Is there anything else? === 07. Have you contributed to the GNOME project? === [single choice] * Yes * No === 08. Have you contacted the GNOME team? === [single choice] * Yes, successfully * Yes, unsuccessfully * No, I don't know how * No, never had the need === 09. If you could change three things in GNOME, what would they be? === [free form] === 10. Do you have any comments or suggestions for the GNOME team? === [free form] -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 23:21, Stormy Peters wrote: > Of course, maybe I'm wrong. Perhaps the average user of Linux/GNOME does > know what GNOME is, knows how to contact the GNOME team Maybe this is an interesting point to know. Also from a branding point of view. Is it important (or not) that users know that, this is "GNOME"? That it is free software? That it is made by an international collaborative community? That the community can be contacted? That feedback and contribution is welcome? (shall it be even just a typo in a doc, or a love message) I feel it somehow wrong if a large proportion of users would just vaguely know they are using "linux". ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Sat, Aug 20, 2011 at 6:24 AM, Benjamin Otte wrote: > Felipe Contreras gmail.com> writes: >> >> That doesn't change the fact that everyone understands the word "happy". > http://www.ted.com/talks/daniel_kahneman_the_riddle_of_experience_vs_memory.html Yes, I have seen that video before. However, not once in that video is he suggesting that people are confused about what "happy" means. When somebody says he is "happy" with his vacation, he knows exactly what he means, and so do we. What Kahneman is saying is that we are talking to remembering self, that's all. You can't ask the experiencing self anything, only the remembering self can answer something in a survey. Now, here's the funny part, this is similar to the reason why you need a survey; you need some data, some numbers. Because when you ask the question "Is GNOME improving?" or "Do people like GNOME 3?", all you have are recollections of experiences which can be completely distorted from the reality, only with data you can have some degree of certainty. -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 5:58 PM, Maciej Marcin Piechotka wrote: > On Fri, 2011-08-19 at 16:08 +0300, Felipe Contreras wrote: >> >>> Likewise, >> >>> 'happy' will be thought of differently by different people (a very odd >> >>> word to include in a questionnaire, if you don't mind me saying): >> >> >> >> I think everyone understands the word happy. >> > >> > /ME wipes a mouthful of coffee from my monitor >> > >> > Then you haven't read enough of the survey research literature. >> >> That doesn't change the fact that everyone understands the word "happy". > > Not necessary. Just to give an example - there is strong cultural > influence how do you respond to simple question 'How are you'. In some > cultures it is impolite to answer better then 'so so' and the normal > answer is somehow along lines 'it could be worst, it could be better'. > On the other hand the correct response in English is usually 'great' or > 'fine' (to quote my teacher 'even if your house is burnt and your dog is > terminally ill'). I have been warned to avoid 'standard' 'so so' > response as I will be perceived as either impolite or after some large > disaster because what I really meant was 'great'. > > (Somehow less directly related but also illustrates the problem of > tricky words - in my native language friend means what in English is > understood by close friend and many people whom I would call in English > friend I would call in Polish acquaintance. Even though I know the > difference I am less inclined to call people friends as in my mental > model they are described by word 'acquaintance'). > > Of course this is 'just' cultural bias caused by people not being native > speakers of English. You need to add individual bias. In each case it > adds more and more 'noise' to survey. Yes, words can mean different things depending on the cultural background, but that's not the case for "happy". In my experience all cultures know exactly what you mean when you say you are "happy" with something. If you have evidence to the contrary for this precise word, please say so, otherwise it's just a faulty generalization. -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v2)
On Fri, Aug 19, 2011 at 10:04 PM, Federico Mena Quintero
wrote:
> On Mon, 2011-08-01 at 18:35 +0300, Felipe Contreras wrote:
>
>> What do you think?
>
> Keep in mind that Gnome 3 just hasn't been around for very long. Right
> now Gnome 3 is most likely only being used by technical people, Linux
> enthusiasts, etc. - it has not trickled down to end users yet. We may
> have found the top, obvious problems in Gnome 3, but not the long tail
> of bugs.
Which is why I suggested to wait until 3.2.
> I think the survey would be much more useful with free-form answers,
> instead of fixed options. That way you may be able to eliminate one
> level of indirection ("oh, most people said 'somewhat', now let's make
> another survey to find out why"). While fixed options let you do
> amazing web hackery, free-form answers lead to more insightful results.
From what I can see, you were dealing with a small numbers of
responders, so free-form is excellent, but when you are talking about
thousands of them, I don't think you can do very much.
It might be a good idea to add a bit more of them, but I think the
bulk should be fixed.
> * Being able to publish the raw results really helps; this way other
> people can help you extract statistics. Do ask permission from the
> respondents to post their replies so that other people can study them.
I agree.
> * That survey probably gave too much importance to the system
> administrators themselves - not surprisingly, "better admin tools" was
> the most requested thing. However, the survey *did* get us good insight
> into end-user's problems.
Makes sense.
> It may be very valuable to ask the heads of deployments what they think
> of Gnome 3 so far, even if they haven't evaluated it yet on their users
> - these people have a very good sense of what will work well for people
> in the "real world".
>
> By the way, the main page for the deployments is this (I don't know how
> up-to-date it is):
> https://live.gnome.org/GnomeMarketing/GnomeDeployments
>
> About two years after that survey got published, some people had the
> idea of making it periodic - unfortunately I lost track of them and of
> that effort. You may want to look around for them; they'll have
> interesting thoughts, I'm sure.
While that might be interesting, I am more interested on getting as
many unfiltered results as possible.
--
Felipe Contreras
___
desktop-devel-list mailing list
[email protected]
http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Felipe Contreras gmail.com> writes: > > That doesn't change the fact that everyone understands the word "happy". > http://www.ted.com/talks/daniel_kahneman_the_riddle_of_experience_vs_memory.html ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 11:14 PM, Matthew Garrett wrote: > On Fri, Aug 19, 2011 at 10:26:08PM +0300, Felipe Contreras wrote: >> > Yes, because you have no idea how big the population is. Maybe 10 >> > million is the total population and it's representative. Maybe it's 50% >> > of the population, disproportionately biased towards those of a given >> > prior opinion. You can't know. >> >> Do you have any idea what is the likelihood of that happening? Try >> throwing a dice 10 times and always getting 1-3. Even if the dice is >> rigged, it's very unlikely. It gets exponentially less likely 1 >> million times. > > That's clearly wrong. If you have a bucket of red balls and blue balls > and you draw 10 million balls, and you find that you drew 6 million red > balls and 4 million blue balls, what does that tell you? If you're > sampling randomly it tells you that there are more red balls than blue > balls. If you're subconsciously preferring to pick up red balls then it > tells you nothing. So we need to avoid subconsciously picking red balls, > which means we need to pick users randomly which is something we can't > do with a voluntary survey. Cochran's formulas don't apply here because > you're not picking your sample set at random. That's a very bad example. An example closer to reality would be that color is indeed the bias, but we are not interested in the color, but the size of the balls. After the survey, we find out that overall, red balls are bigger than blue balls. Fortunately we don't care about the proportion of blue vs red balls in the total population, we only care about blue balls, so, we only consider the size of those. In the GNOME case, the color of the balls corresponds to the bias we want to identify; like geekness, and the size is the actual thing we are interested on, which is their happiness. We only care about non geeks (blue balls), as many GNOME people have stated, the real target users are the ones that don't even know what is GNOME. Now, if what you are worried about is the self-selection bias, we can add a new question "Why are you taking this survey?" with the option "Somebody is pushing me", and encourage people to push their relatives/colleagues/friends to fill the survey (just like a "professional" firm would, except "crowd-sourced"). Then, for external validity, you only consider the results of the people that answered "Somebody is pushing me" (they don't have self-selection bias). -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, 19 Aug 2011 21:05:26 +0100 Matthew Garrett wrote: > On Fri, Aug 19, 2011 at 10:53:46PM +0300, Felipe Contreras wrote: > > > If you went back to 1991 and wanted a production-quality kernel within a > > > year, Linux probably wouldn't be your starting point. There'd be a > > > learning process involved with setting up a professional-quality survey > > > team, and the first few attempts would be pretty buggy. We'd get there > > > in time, but until then... > > > > Until then it's better to have nothing? > > It's better to have no data than to have misleading data. It's better to have no desktop than one that might not be production quality ? Same argument, same problem. PS data is never misleading. It's presentation maybe misleading but the data is just bits. I do think the comments on more open and why fill in the box type questions are on the button for the reasons expressed about sample size, randomness and what it would be useful to learn. Or perhaps rerun Federico's survey ? Alan ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 10:26:08PM +0300, Felipe Contreras wrote: > > Yes, because you have no idea how big the population is. Maybe 10 > > million is the total population and it's representative. Maybe it's 50% > > of the population, disproportionately biased towards those of a given > > prior opinion. You can't know. > > Do you have any idea what is the likelihood of that happening? Try > throwing a dice 10 times and always getting 1-3. Even if the dice is > rigged, it's very unlikely. It gets exponentially less likely 1 > million times. That's clearly wrong. If you have a bucket of red balls and blue balls and you draw 10 million balls, and you find that you drew 6 million red balls and 4 million blue balls, what does that tell you? If you're sampling randomly it tells you that there are more red balls than blue balls. If you're subconsciously preferring to pick up red balls then it tells you nothing. So we need to avoid subconsciously picking red balls, which means we need to pick users randomly which is something we can't do with a voluntary survey. Cochran's formulas don't apply here because you're not picking your sample set at random. -- Matthew Garrett | [email protected] ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 9:34 PM, Stormy Peters wrote: > On Fri, Aug 19, 2011 at 12:12 PM, Felipe Contreras > wrote: >> >> I am not being aggressive. All I am asking is for clarification; is >> there *anything* I could do to make the survey more acceptable to you >> guys, or are you opposed to the very idea of having a user survey >> blessed by GNOME? > > Your answers sound aggressive to me but I think that's totally > understandable given all the negative feedback. Perhaps you are not used to straight-forward communication. I'm not trying to aggravate anyone. > I gave my feedback. I'd want the survey to be much more detailed. "What do > you think about this menu option on Cheese" seems like it would give more > feedback than "do you like GNOME?" But I do not have time to help come up > with the questions, so I agree with many folks that say you'll have to take > the feedback you've gotten and move forward. Trying to do that would create a huge survey that most likely most people will not even try to answer. If somebody really detests certain menu option on Cheese, I'm sure they'll let you know in the comments box. Who knows, maybe it turns out the part that most people are not satisfied with is the documentation, those kinds of results might trigger some interesting debate. Or maybe you are right, and we wouldn't not get anything useful, but at least we would have some ideas for the next survey. > Giving feedback does not mean providing alternatives or working on the > project. It's easy to give feedback. It's much harder and more time > consuming to incorporate that feedback. You asked for feedback, you got > some. If you want those people's approval, then you'll probably have to > incorporate that feedback. I have incorporated all the feedback that can be incorporated. The rest is too vague, or not actionable. What do *you* think must absolutely be changed in the survey? > If you aren't planning on incorporating it, then > it's probably best to stop insisting that people need to provide > alternatives if they give negative feedback. Huh? That's a very broad statement. Let's be clear, I have not turned away any feedback. Let's analyze for example the claims by Allan Day: --- > When you do survey research, you have to be certain that one person > understands the questions in the same way that another person does. Generally yes. Is that achievable in all the questions in this survey? Probably not (would love to hear some suggestions otherwise). Which why some other questions are asked to determine the people that might be thinking in other terms. (I already explained that) > You've also got the representativeness problem. Your sample will inevitably > be unrepresentative, probably highly so. Says who? What if we get 10 million answers? That would be such a big chunk of the total population that this problem is not a big deal. Or what if there's a notification app embedded in GNOME 3.2. That would not only maximize the reponders, but also maximize the randomness. Wouldn't it? (I already explained that) > your survey results will be misleading That's very useful. Now, how about some ideas to make the results less misleading? --- What exactly do you want me to do with that feedback? (aside from what I have already done) I am all ears. > Obviously, you don't need everyone's approval to move forward. Rarely does > any project get 100% approval. > > How you move forward, how much feedback you want and how you use that > feedback is up to you. I want the approval of the GNOME community, and I am willing to accept all suggestions for improvement in order to get that. So, what should I do? -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 10:53:46PM +0300, Felipe Contreras wrote: > > If you went back to 1991 and wanted a production-quality kernel within a > > year, Linux probably wouldn't be your starting point. There'd be a > > learning process involved with setting up a professional-quality survey > > team, and the first few attempts would be pretty buggy. We'd get there > > in time, but until then... > > Until then it's better to have nothing? It's better to have no data than to have misleading data. -- Matthew Garrett | [email protected] ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 10:47 PM, Richard Hughes wrote: > On 19 August 2011 20:26, Felipe Contreras wrote: >> ...To me GNOME is hitting >> everything in the room as it's going forward, and saying; I'm fine, I >> know where I'm going... > > To me, the sun is shining through the windows of a freshly redecorated room. Either you are hallucinating, or you are not getting the analogy. -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 9:28 PM, Matthew Garrett wrote: > (Resend: Managed to leave d-d-l off Cc: by accident) > On Fri, Aug 19, 2011 at 06:15:03PM +0100, Alan Cox wrote: >> > Any survey that isn't a carefully controlled randomly selected sample of >> > users doesn't result in learning. It results in data that forms some >> >> You need truely or reasonably random samples for certain kinds of >> activities and analysis in particularly quantitative analysis when you >> want to perform p tests and the like. You don't need it in order to >> learn merely to generate statistical proofs and those are often quite >> useless anyway. Proviing gnome 3 is great/indifferent/sucks doesn't have >> much value. You do not need it for explorative learning. Small children >> do not need to open a statistically valid sample of randomly chosen doors >> to learn about doors ! > > I am all for making it easier for people to give feedback about Gnome, > but presenting it as a survey gives a strong implication that the > results are meaningful as an aggregate rather than as a collection of > anecdotes. If we want to hear form users, let's make it easy for users > to talk to us. A survey isn't the way to achieve that. Again, any better suggestions? I tried many of them back in 2007, and got nowhere, I think a user survey is the best one we've got. >> > sort of rorschach blot. Everyone will see what they want to see. Those >> > who believe that Gnome 3 is a step back will point out that the majority >> > of responses are negative. Those who believe it's a step forward will >> > point out that happy users are going to be far less inclined to respond. >> >> You seem to be assuming the results and that the only question of interest >> is "does gnome 3 suck". > > I'm assuming that the sort of people who are going to go to the effort > of filling out a survey are likely to be closer to the population > discussing things on lwn than the population of usres in general. That > may be entirely untrue! But if we get the opposite results then it still > doesn't tell us anything that's actually true, and it's still an > opportunity to argue the issue rather than focus on making software > better. We most likely are going to be able to identify that bias. Let's make some wild guesses; 50% of the people that use GNOME 3 like it, and 50% don't. Of that amount, 90% seem to be geeks. In the remaining 10%, the people that use GNOME 2 show 80% happiness, and of GNOME 3 it's only 60%. But you still don't think there's any value in there, fair enough. Then we dig through that 40% subset that didn't like GNOME 3 and take a look at their comments, and we find "Very strange", "Can't get used to it", and things like that. At that point we might want to see if they left an email to contact them, and then try to gather more detailed feedback. I think there's a chance that this survey could tell us something that's "actually true". >> > If we want to find out what our users think then the only way to do that >> > is to >> > have professional involvement and a random sample set. >> >> Of course, and the only way to produce a kernel or desktop is to hire >> professionals to do it for you no doubt. > > If you went back to 1991 and wanted a production-quality kernel within a > year, Linux probably wouldn't be your starting point. There'd be a > learning process involved with setting up a professional-quality survey > team, and the first few attempts would be pretty buggy. We'd get there > in time, but until then... Until then it's better to have nothing? -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On 19 August 2011 20:26, Felipe Contreras wrote: > ...To me GNOME is hitting > everything in the room as it's going forward, and saying; I'm fine, I > know where I'm going... To me, the sun is shining through the windows of a freshly redecorated room. If you have specific problems with GNOME, file bugs and discuss things with maintainers and designers. It'll be way more effective at changing things than writing a survey. Richard. ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 9:20 PM, Matthew Garrett wrote: > On Fri, Aug 19, 2011 at 08:14:25PM +0300, Felipe Contreras wrote: >> On Fri, Aug 19, 2011 at 7:24 PM, Matthew Garrett wrote: >> > Any survey that isn't a carefully controlled randomly selected sample of >> > users doesn't result in learning. >> >> Unless the biases are identified, which we are trying to do. > > You can only identify the biases if you already know the population, and > you can only know the population if you've got a random sample set to > begin with. That's not true. You might need that if you want to account for *all* the biases, which nobody can do anyway. What most people do is try to figure them out, chances are you might be missing some of the biases, but hopefully the unidentified misrepresented group won't be that big anyway, and thus wouldn't affect so much the analysis. If it turns out that a significant bias is not identified beforehand, that can be tackled in the next survey in 2012. >> Moreover, I have tried to push the idea to have an automatic >> notification, which would maximize the number of responders, and thus >> increase the randomization. But apparent this idea is not welcome. > > It doesn't help. The people most likely to respond to an irritating > popup that disrupts their work are people who already feel that gnome 3 > is an irritating piece of software that disrupts their work. You can't > get a random sample in-band. It doesn't help? It does randomize the sample more, doesn't it? Maybe it's not perfectly randomized, but nothing can ever be perfect. >> So, ideas to improve the randomization are dismissed, and then you say >> without randomization, the survey is not useful. IOW; you are >> intentionally deadlocking the proposal. > > I am saying that your results aren't useful unless your sample is > random. I don't know of a good way to obtain a representative sample. There's no such thing as 0% random, or 100% random, all we can thrive for is to increase the randomness. And I already explained that non-random samples are already useful if you can identify the biases. >> > Everyone will see what they want to see. Those >> > who believe that Gnome 3 is a step back will point out that the majority >> > of responses are negative. Those who believe it's a step forward will >> > point out that happy users are going to be far less inclined to respond. >> > >> > There's no way whatsoever to determine how representative the responses >> > are, and so there's no way whatsoever to learn anything about the >> > population. All we'd learn is that some users like Gnome 3 and some >> > users don't, and that's something we *already know*. So we'd gain >> > nothing, but we'd guarantee another huge set of arguments which would >> > themselves also tell us nothing. >> >> That's an assumption. What if we get 10 million responses? Would you >> still claim that the results are not representative? > > Yes, because you have no idea how big the population is. Maybe 10 > million is the total population and it's representative. Maybe it's 50% > of the population, disproportionately biased towards those of a given > prior opinion. You can't know. Do you have any idea what is the likelihood of that happening? Try throwing a dice 10 times and always getting 1-3. Even if the dice is rigged, it's very unlikely. It gets exponentially less likely 1 million times. >> I think only *after* getting the results you would be able to say >> anything about it's representativeness. >> >> Something more realistic, say you get at least 300 responses that >> don't have any "geek" bias, that would be more than enough to make >> some statistically significant conclusions. > > It really wouldn't. Yes it would. Check Cochran's formulas. 300 unbiased responses gives you already good statistical power, after a certain point it doesn't matter much what is the total population; 10m, 30m, 1m. The likelihood that would would get 300 unbiased responses all pointing to the wrong direction is almost nothing, in fact a few dozens would do (if they are truly random). There are some simple calculators online: http://www.surveysystem.com/sscalc.htm But yeah, since there's going to be bias, you need more. >> > I disagree. Doing something that sucks more time and energy away from >> > development without actually telling us anything in return is worse than >> > that not happening. Felipe is obviously free to do whatever he wants, >> > but there's no benefit in Gnome itself participating in any way. If we >> > want to find out what our users think then the only way to do that is to >> > have professional involvement and a random sample set. >> >> This is not sucking any time and energy from anybody, I just need >> access to the server that has limesurvey installed, or somebody else >> can do that (can't take that much time), I would contact all the >> relevant news sites and make the relevant posts in social media. All >> that that is needed from GNOME peo
Re: GNOME user survey 2011 (v2)
On Mon, 2011-08-01 at 18:35 +0300, Felipe Contreras wrote:
> What do you think?
Keep in mind that Gnome 3 just hasn't been around for very long. Right
now Gnome 3 is most likely only being used by technical people, Linux
enthusiasts, etc. - it has not trickled down to end users yet. We may
have found the top, obvious problems in Gnome 3, but not the long tail
of bugs.
I think the survey would be much more useful with free-form answers,
instead of fixed options. That way you may be able to eliminate one
level of indirection ("oh, most people said 'somewhat', now let's make
another survey to find out why"). While fixed options let you do
amazing web hackery, free-form answers lead to more insightful results.
Back in 2006 I did a survey of the Gnome deployments (i.e. somewhere
around the middle of the 2.x series). These questions were posted to
various places:
https://live.gnome.org/GnomeMarketing/QuestionsForDeployments
And this is the final report with the results:
http://people.gnome.org/~federico/docs/gnome-deployments-2006/index.html
(At the end of the report there are links to the raw results that people
sent in. Download them and read them; they are short and quite
interesting. Where the links say, "primates.ximian.com", please replace
that with "people.gnome.org" - I can't re-generate the HTML report with
that change just now, unfortunately.)
Let me tell you about some things I learned from that survey:
* Free-form answers work really well. It *will* take you some time to
read them and see how to categorize or weigh them, but it's worth the
effort.
* Being able to publish the raw results really helps; this way other
people can help you extract statistics. Do ask permission from the
respondents to post their replies so that other people can study them.
* That survey probably gave too much importance to the system
administrators themselves - not surprisingly, "better admin tools" was
the most requested thing. However, the survey *did* get us good insight
into end-user's problems.
* Those pie charts in the report make no sense. Apologies for
chartjunk.
It may be very valuable to ask the heads of deployments what they think
of Gnome 3 so far, even if they haven't evaluated it yet on their users
- these people have a very good sense of what will work well for people
in the "real world".
By the way, the main page for the deployments is this (I don't know how
up-to-date it is):
https://live.gnome.org/GnomeMarketing/GnomeDeployments
About two years after that survey got published, some people had the
idea of making it periodic - unfortunately I lost track of them and of
that effort. You may want to look around for them; they'll have
interesting thoughts, I'm sure.
Federico
___
desktop-devel-list mailing list
[email protected]
http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 14:46, Luc Pionchon wrote: > On Fri, Aug 19, 2011 at 13:14, Allan Day wrote: >> >> We already have a wealth of data about peoples' experiences with GNOME 3 > > Allan, this is interesting, what is the main pointer to access this data? > Allan, you may have missed it in this epic thread; I am sincerely interested in reading this data, would you be kind enough to point me at it? Luc ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 21:20, Matthew Garrett wrote: > The people most likely to respond to an irritating popup that disrupts their > work > are people who ... ... do not use GNOME 3. GNOME 3 is designed to reduce distraction and interruption and to put you in control. Our new notifications system subtly presents messages and will save them until you are ready for them, and the GNOME 3 panel has been styled so that it is part of the background, not the foreground. These changes allow you to focus on your creative tasks. I hope this brings a bit of humor in this thread. ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 12:12 PM, Felipe Contreras < [email protected]> wrote: > > I am not being aggressive. All I am asking is for clarification; is > there *anything* I could do to make the survey more acceptable to you > guys, or are you opposed to the very idea of having a user survey > blessed by GNOME? > > Your answers sound aggressive to me but I think that's totally understandable given all the negative feedback. I gave my feedback. I'd want the survey to be much more detailed. "What do you think about this menu option on Cheese" seems like it would give more feedback than "do you like GNOME?" But I do not have time to help come up with the questions, so I agree with many folks that say you'll have to take the feedback you've gotten and move forward. Giving feedback does not mean providing alternatives or working on the project. It's easy to give feedback. It's much harder and more time consuming to incorporate that feedback. You asked for feedback, you got some. If you want those people's approval, then you'll probably have to incorporate that feedback. If you aren't planning on incorporating it, then it's probably best to stop insisting that people need to provide alternatives if they give negative feedback. Obviously, you don't need everyone's approval to move forward. Rarely does any project get 100% approval. How you move forward, how much feedback you want and how you use that feedback is up to you. Good luck! Stormy ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
(Resend: Managed to leave d-d-l off Cc: by accident) On Fri, Aug 19, 2011 at 06:15:03PM +0100, Alan Cox wrote: > > Any survey that isn't a carefully controlled randomly selected sample of > > users doesn't result in learning. It results in data that forms some > > You need truely or reasonably random samples for certain kinds of > activities and analysis in particularly quantitative analysis when you > want to perform p tests and the like. You don't need it in order to > learn merely to generate statistical proofs and those are often quite > useless anyway. Proviing gnome 3 is great/indifferent/sucks doesn't have > much value. You do not need it for explorative learning. Small children > do not need to open a statistically valid sample of randomly chosen doors > to learn about doors ! I am all for making it easier for people to give feedback about Gnome, but presenting it as a survey gives a strong implication that the results are meaningful as an aggregate rather than as a collection of anecdotes. If we want to hear form users, let's make it easy for users to talk to us. A survey isn't the way to achieve that. > > sort of rorschach blot. Everyone will see what they want to see. Those > > who believe that Gnome 3 is a step back will point out that the majority > > of responses are negative. Those who believe it's a step forward will > > point out that happy users are going to be far less inclined to respond. > > You seem to be assuming the results and that the only question of interest > is "does gnome 3 suck". I'm assuming that the sort of people who are going to go to the effort of filling out a survey are likely to be closer to the population discussing things on lwn than the population of usres in general. That may be entirely untrue! But if we get the opposite results then it still doesn't tell us anything that's actually true, and it's still an opportunity to argue the issue rather than focus on making software better. > > If we want to find out what our users think then the only way to do that is > > to > > have professional involvement and a random sample set. > > Of course, and the only way to produce a kernel or desktop is to hire > professionals to do it for you no doubt. If you went back to 1991 and wanted a production-quality kernel within a year, Linux probably wouldn't be your starting point. There'd be a learning process involved with setting up a professional-quality survey team, and the first few attempts would be pretty buggy. We'd get there in time, but until then... -- Matthew Garrett | [email protected] ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 08:14:25PM +0300, Felipe Contreras wrote: > On Fri, Aug 19, 2011 at 7:24 PM, Matthew Garrett wrote: > > Any survey that isn't a carefully controlled randomly selected sample of > > users doesn't result in learning. > > Unless the biases are identified, which we are trying to do. You can only identify the biases if you already know the population, and you can only know the population if you've got a random sample set to begin with. > Moreover, I have tried to push the idea to have an automatic > notification, which would maximize the number of responders, and thus > increase the randomization. But apparent this idea is not welcome. It doesn't help. The people most likely to respond to an irritating popup that disrupts their work are people who already feel that gnome 3 is an irritating piece of software that disrupts their work. You can't get a random sample in-band. > So, ideas to improve the randomization are dismissed, and then you say > without randomization, the survey is not useful. IOW; you are > intentionally deadlocking the proposal. I am saying that your results aren't useful unless your sample is random. I don't know of a good way to obtain a representative sample. > > Everyone will see what they want to see. Those > > who believe that Gnome 3 is a step back will point out that the majority > > of responses are negative. Those who believe it's a step forward will > > point out that happy users are going to be far less inclined to respond. > > > > There's no way whatsoever to determine how representative the responses > > are, and so there's no way whatsoever to learn anything about the > > population. All we'd learn is that some users like Gnome 3 and some > > users don't, and that's something we *already know*. So we'd gain > > nothing, but we'd guarantee another huge set of arguments which would > > themselves also tell us nothing. > > That's an assumption. What if we get 10 million responses? Would you > still claim that the results are not representative? Yes, because you have no idea how big the population is. Maybe 10 million is the total population and it's representative. Maybe it's 50% of the population, disproportionately biased towards those of a given prior opinion. You can't know. > I think only *after* getting the results you would be able to say > anything about it's representativeness. > > Something more realistic, say you get at least 300 responses that > don't have any "geek" bias, that would be more than enough to make > some statistically significant conclusions. It really wouldn't. > > I disagree. Doing something that sucks more time and energy away from > > development without actually telling us anything in return is worse than > > that not happening. Felipe is obviously free to do whatever he wants, > > but there's no benefit in Gnome itself participating in any way. If we > > want to find out what our users think then the only way to do that is to > > have professional involvement and a random sample set. > > This is not sucking any time and energy from anybody, I just need > access to the server that has limesurvey installed, or somebody else > can do that (can't take that much time), I would contact all the > relevant news sites and make the relevant posts in social media. All > that that is needed from GNOME people is a blessing. The sucking of time and energy would come from the argument over the results afterwards. -- Matthew Garrett | [email protected] ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 8:50 PM, Richard Hughes wrote: > On 19 August 2011 18:42, Felipe Contreras wrote: >> Sure, I just wanted to make things clear. In fact, if they cared about >> user feedback, there would be some numbers available somewhere, and I >> wouldn't have to do this. > > We're not asking you to do anything. I am not suggesting you are. > Please just run the poll on your > personal blog and stop getting aggressive with developers on this > mailing list. I am not being aggressive. All I am asking is for clarification; is there *anything* I could do to make the survey more acceptable to you guys, or are you opposed to the very idea of having a user survey blessed by GNOME? -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 10:04 PM, Olav Vitters wrote: > On Thu, Aug 18, 2011 at 08:03:45PM +0300, Felipe Contreras wrote: >> I can only think of one reason why somebody would provide criticism >> without suggestions for improvement... > > 1. Because they cannot think of a good suggestion. Then surely I cannot be blamed for not coming with one either. -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Really ought to stay out of this thread but there is one point that is important to address below. On Fri, Aug 19, 2011 at 7:17 AM, Felipe Contreras wrote: > But again, as I said, if there's no survey on Earth you could trust, > just ignore the results. Results by themselves cannot hurt you. This isn't right. Poorly understood results lead you draw incorrect conclusions which lead you astray. Or at minimum cause the marketing team undue stress trying to explain that the results don't make any sense. Really, I don't understand why anyone would want to go through the trouble if the results aren't *useful*. Now, if you want to do something productive I encourage you to work with the guidance of Allan and others. Jon ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On 19 August 2011 18:42, Felipe Contreras wrote: > Sure, I just wanted to make things clear. In fact, if they cared about > user feedback, there would be some numbers available somewhere, and I > wouldn't have to do this. We're not asking you to do anything. Please just run the poll on your personal blog and stop getting aggressive with developers on this mailing list. Thanks, Richard. ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 8:15 PM, Jonathon Jongsma wrote: > On Fri, 2011-08-19 at 19:42 +0300, Felipe Contreras wrote: >> On Fri, Aug 19, 2011 at 4:56 PM, Richard Hughes wrote: >> > On 19 August 2011 14:13, Felipe Contreras >> > wrote: >> >> Is there anyone in the universe able to create a user survey worthy of >> >> GNOME? Can you convince him of doing so? >> > >> > Do your survey with the questions you want, and come to your own >> > conclusions. Blog about them if you want. You could even convince a >> > distribution to include a popup with a link, although I think that's >> > insane. >> > >> > Just don't tell people that it's from the GNOME project, in any way >> > authorized or blessed by the ruling cabal[1] or developers. I'm pretty >> > sure the majority of the people actually working on GNOME 3.2 don't >> > want a survey at all. >> > >> > Sorry to be blunt. >> >> No, thanks for the direct feedback. So basically you are saying >> there's no way any survey of any quality would be blessed by the GNOME >> community. That certainly clarifies things. > > It seems obvious from most responses here that there are not very many > people within the GNOME community that think that this sort of a survey > would be beneficial, and worry that it may even be counter-productive. > In response to this realization, you have apparently shifted into > outrage mode. You pretend that it is impossible to simultaneously care > about what users while also opposing a user survey that has no hope of > being a representative sample of users. You might say you do, and you might even believe so, but if your actions demonstrate otherwise, perhaps you do not. If the GNOME community really cared about what users have to say, and this survey indeed does not have any hope of having a representative sample of users (I disagree), then wouldn't they take the reins and do it properly? > It is possible for well-meaning people to come to different conclusions > on the best methods for achieving a certain goal. Yes, whenever I have a disagreement on a method to develop some software, I just go ahead and do it that way, and then say; see? this is how it should be done. Saying "you are wrong" is easy, anybody can do that. > It seems that most > people here don't agree with your methods. Please accept the fact that > this does not mean that they hate users, despite your attempts to > conflate the two things. I would, if they went ahead with the "right methods" and got some user feedback, if not in the form of a survey, in any method. > You are free to proceed with your survey on your own. Others are free > to not wish to join you. It's that simple. Can you please stop the > faux outrage? Sure, I just wanted to make things clear. In fact, if they cared about user feedback, there would be some numbers available somewhere, and I wouldn't have to do this. -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 8:25 PM, Stormy Peters wrote: > On Fri, Aug 19, 2011 at 10:40 AM, Felipe Contreras > wrote: >> >> On Fri, Aug 19, 2011 at 4:55 PM, Sam Thursfield wrote: >> > >> > Gathering feedback does not necessarily require an online user survey. >> >> Indeed, do you have a better suggestion? > > There are several other ways to get feedback. > > For example, user testing. I'm sure all the major distributions have done > some user testing. Most large companies have a whole user testing > team/group. And where are the results? Without evidence it's only wishful thinking. > I'm not a user testing expert but it involves giving people (both new and > experienced) tasks to do, watching how they do it (without interfering) and > then asking them about their experience. > > I know people who have successfully used http://www.usertesting.com/ for web > sites. I don't know if a similar, inexpensive option exists for desktop > software or not. Right, so nobody is going to do this. Is there any better suggestion that would actually be implemented? -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 10:40 AM, Felipe Contreras < [email protected]> wrote: > On Fri, Aug 19, 2011 at 4:55 PM, Sam Thursfield wrote: > > > > Gathering feedback does not necessarily require an online user survey. > > Indeed, do you have a better suggestion? > > There are several other ways to get feedback. For example, user testing. I'm sure all the major distributions have done some user testing. Most large companies have a whole user testing team/group. I'm not a user testing expert but it involves giving people (both new and experienced) tasks to do, watching how they do it (without interfering) and then asking them about their experience. I know people who have successfully used http://www.usertesting.com/ for web sites. I don't know if a similar, inexpensive option exists for desktop software or not. Stormy ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
> Any survey that isn't a carefully controlled randomly selected sample of > users doesn't result in learning. It results in data that forms some You need truely or reasonably random samples for certain kinds of activities and analysis in particularly quantitative analysis when you want to perform p tests and the like. You don't need it in order to learn merely to generate statistical proofs and those are often quite useless anyway. Proviing gnome 3 is great/indifferent/sucks doesn't have much value. You do not need it for explorative learning. Small children do not need to open a statistically valid sample of randomly chosen doors to learn about doors ! I for one would not be surprised if a lot of responses were not more positive than some seem to think. There has been time for people to use it and adjust and apply the fixes. Even odder there is no Gnome fork. If as I hear 'Gnome 3 is hated by technical people' and there are enough who care there ought to be a Gnome fork by now. But what do we have - exde, dead, turned into a one page rant and no code Mate - described by phoronix as "The Mate Desktop Environment fork of GNOME2 was started by an Arch Linux user back in June, but it hasn't yet gained too much traction and is mostly just talked about on various forums around the web. " which about sums it up. > sort of rorschach blot. Everyone will see what they want to see. Those > who believe that Gnome 3 is a step back will point out that the majority > of responses are negative. Those who believe it's a step forward will > point out that happy users are going to be far less inclined to respond. You seem to be assuming the results and that the only question of interest is "does gnome 3 suck". > nothing, but we'd guarantee another huge set of arguments which would > themselves also tell us nothing. Those will tell you a lot if someone analyses them. Again you may not be able to do formal mathematical tests on them but so what. > If we want to find out what our users think then the only way to do that is > to > have professional involvement and a random sample set. Of course, and the only way to produce a kernel or desktop is to hire professionals to do it for you no doubt. Alan ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, 2011-08-19 at 19:42 +0300, Felipe Contreras wrote: > On Fri, Aug 19, 2011 at 4:56 PM, Richard Hughes wrote: > > On 19 August 2011 14:13, Felipe Contreras > > wrote: > >> Is there anyone in the universe able to create a user survey worthy of > >> GNOME? Can you convince him of doing so? > > > > Do your survey with the questions you want, and come to your own > > conclusions. Blog about them if you want. You could even convince a > > distribution to include a popup with a link, although I think that's > > insane. > > > > Just don't tell people that it's from the GNOME project, in any way > > authorized or blessed by the ruling cabal[1] or developers. I'm pretty > > sure the majority of the people actually working on GNOME 3.2 don't > > want a survey at all. > > > > Sorry to be blunt. > > No, thanks for the direct feedback. So basically you are saying > there's no way any survey of any quality would be blessed by the GNOME > community. That certainly clarifies things. > It seems obvious from most responses here that there are not very many people within the GNOME community that think that this sort of a survey would be beneficial, and worry that it may even be counter-productive. In response to this realization, you have apparently shifted into outrage mode. You pretend that it is impossible to simultaneously care about what users while also opposing a user survey that has no hope of being a representative sample of users. It is possible for well-meaning people to come to different conclusions on the best methods for achieving a certain goal. It seems that most people here don't agree with your methods. Please accept the fact that this does not mean that they hate users, despite your attempts to conflate the two things. You are free to proceed with your survey on your own. Others are free to not wish to join you. It's that simple. Can you please stop the faux outrage? jonner ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 7:24 PM, Matthew Garrett wrote: > On Thu, Aug 18, 2011 at 10:37:33PM +0100, Alan Cox wrote: > >> Doing nothing achieves nothing, doing something achieves learning. You >> may well not learn what you intended but you will learn something >> including quite possibly how to do future surveys better. > > Any survey that isn't a carefully controlled randomly selected sample of > users doesn't result in learning. Unless the biases are identified, which we are trying to do. Moreover, I have tried to push the idea to have an automatic notification, which would maximize the number of responders, and thus increase the randomization. But apparent this idea is not welcome. So, ideas to improve the randomization are dismissed, and then you say without randomization, the survey is not useful. IOW; you are intentionally deadlocking the proposal. > It results in data that forms some > sort of rorschach blot. It might if you look at it as a whole, but you can try to dissect it. > Everyone will see what they want to see. Those > who believe that Gnome 3 is a step back will point out that the majority > of responses are negative. Those who believe it's a step forward will > point out that happy users are going to be far less inclined to respond. > > There's no way whatsoever to determine how representative the responses > are, and so there's no way whatsoever to learn anything about the > population. All we'd learn is that some users like Gnome 3 and some > users don't, and that's something we *already know*. So we'd gain > nothing, but we'd guarantee another huge set of arguments which would > themselves also tell us nothing. That's an assumption. What if we get 10 million responses? Would you still claim that the results are not representative? I think only *after* getting the results you would be able to say anything about it's representativeness. Something more realistic, say you get at least 300 responses that don't have any "geek" bias, that would be more than enough to make some statistically significant conclusions. >> I'm not saying its necessarily a great approach but it's vastly superior >> to people sitting around picking holes in the idea until it never happens. > > I disagree. Doing something that sucks more time and energy away from > development without actually telling us anything in return is worse than > that not happening. Felipe is obviously free to do whatever he wants, > but there's no benefit in Gnome itself participating in any way. If we > want to find out what our users think then the only way to do that is to > have professional involvement and a random sample set. This is not sucking any time and energy from anybody, I just need access to the server that has limesurvey installed, or somebody else can do that (can't take that much time), I would contact all the relevant news sites and make the relevant posts in social media. All that that is needed from GNOME people is a blessing. -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 10:37:33PM +0100, Alan Cox wrote: > Doing nothing achieves nothing, doing something achieves learning. You > may well not learn what you intended but you will learn something > including quite possibly how to do future surveys better. Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. It results in data that forms some sort of rorschach blot. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. There's no way whatsoever to determine how representative the responses are, and so there's no way whatsoever to learn anything about the population. All we'd learn is that some users like Gnome 3 and some users don't, and that's something we *already know*. So we'd gain nothing, but we'd guarantee another huge set of arguments which would themselves also tell us nothing. > I'm not saying its necessarily a great approach but it's vastly superior > to people sitting around picking holes in the idea until it never happens. I disagree. Doing something that sucks more time and energy away from development without actually telling us anything in return is worse than that not happening. Felipe is obviously free to do whatever he wants, but there's no benefit in Gnome itself participating in any way. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. -- Matthew Garrett | [email protected] ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 4:56 PM, Richard Hughes wrote: > On 19 August 2011 14:13, Felipe Contreras wrote: >> Is there anyone in the universe able to create a user survey worthy of >> GNOME? Can you convince him of doing so? > > Do your survey with the questions you want, and come to your own > conclusions. Blog about them if you want. You could even convince a > distribution to include a popup with a link, although I think that's > insane. > > Just don't tell people that it's from the GNOME project, in any way > authorized or blessed by the ruling cabal[1] or developers. I'm pretty > sure the majority of the people actually working on GNOME 3.2 don't > want a survey at all. > > Sorry to be blunt. No, thanks for the direct feedback. So basically you are saying there's no way any survey of any quality would be blessed by the GNOME community. That certainly clarifies things. -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 4:55 PM, Sam Thursfield wrote: > On Fri, Aug 19, 2011 at 2:13 PM, Felipe Contreras > wrote: >> On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo wrote: >>> On Fri 19 Aug 2011 13:33, Felipe Contreras >>> writes: >>> That's a reasonable alternative. How about "pleased"? Any other people have an opinion? >>> >>> You present yourself as reasonable by adjusting on the small points, >>> but you ignore the feedback of greater importance. >>> >>> My opinion is that you are not the right person to lead an effort to >>> gather feedback on GNOME. >> >> Is there anyone in the universe able to create a user survey worthy of >> GNOME? Can you convince him of doing so? > > Gathering feedback does not necessarily require an online user survey. Indeed, do you have a better suggestion? > As stated, for a project which currently targets, among others, users > who do not care what parts of their operating system can be labelled > "GNOME" a survey is not a very reliable way of gathering feedback. > > Have you ever tried to explain, to a person who doesn't have an > interest in software, what GNOME actually is? Again, do you have a suggestion to get feedback in a more useful way? >> In the meantime, this is the best that we have. I will continue >> listening for constructive feedback, but comments such as "this is not >> good", "you are doing it wrong", "it's impossible", lead to nowhere. >> >> Besides, as Alan Cox said, it doesn't have to be perfect, like >> software, we can learn from the mistakes of the 2011 survey, and make >> a better one for 2012. Can we not? > > I urge you to consider the fact that if the majority of people > subscribed to desktop-devel-list don't have faith in idea of an online > user survey, an online user survey is probably not going to much have > effect on the views of the people who contribute to the discussions on > desktop-devel-list, and since most of the GNOME community read > desktop-devel-list you can probably extend this to all of the other > GNOME mailing lists and IRC channels as well. So the status quo, where there are absolutely no numbers whatsoever is preferred. Any attempt to gather quantifiable feedback is discouraged. IOW; the GNOME community does not care about what users have to say at all. -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 5:42 PM, Josselin Mouette wrote: > Le vendredi 19 août 2011 à 16:08 +0300, Felipe Contreras a écrit : >> It's easy to throw empty criticism. Provide *suggestions*. > > Well, here’s a suggestion: since nobody knows how to address the correct > target population or how to interpret the results, I suggest to spend > our time fixing bugs instead. Yes, because we are absolutely and positively certain that fixing these bugs is exactly what GNOME users want. There is no possibility that they want something else, or that the prioritization is not ideal. By definition, whatever GNOME does, is what the users want, and to suggest otherwise is heresy. -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 5:02 PM, Frederic Muller wrote: > On 08/19/2011 09:13 PM, Felipe Contreras wrote: >> On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo wrote: >>> On Fri 19 Aug 2011 13:33, Felipe Contreras >>> writes: >>> That's a reasonable alternative. How about "pleased"? Any other people have an opinion? >>> >>> You present yourself as reasonable by adjusting on the small points, >>> but you ignore the feedback of greater importance. >>> >>> My opinion is that you are not the right person to lead an effort to >>> gather feedback on GNOME. >> >> Is there anyone in the universe able to create a user survey worthy of >> GNOME? Can you convince him of doing so? >> >> In the meantime, this is the best that we have. I will continue >> listening for constructive feedback, but comments such as "this is not >> good", "you are doing it wrong", "it's impossible", lead to nowhere. >> >> Besides, as Alan Cox said, it doesn't have to be perfect, like >> software, we can learn from the mistakes of the 2011 survey, and make >> a better one for 2012. Can we not? >> > > I actually thought that Andy's feedback was constructive. Really? Then you can take that feedback and translate it into concrete actions, right? To me, that sounds as useful as the advice of some academic that says that no software should ever be deployed without strong static analysis. He might be right, but if he is not offering himself to do the job he is proposing, what's the point? IOW; Talk is cheap, show me the code. > Your approach, > motives and way to handle this discussion are questionable. You have > obviously failed to convince the GNOME community and GNOME developers > that your survey would be useful, and I'm afraid nobody feels like > taking leadership on the project neither. Yeah, I failed at an impossible task, maybe. > A lot of people have already told you that enough feedback has been > gathered at this time. > I doubt you will get much endorsement or help around here anymore (I > could of course be wrong on that last point). > > To tell you the truth I have been involved in trying to run a survey for > GNOME 2 years ago IIRC (and I think it's a "recurrent project" - you can > find some old pages on our wiki) with a group of other people, and we > reached the same conclusion: a survey will not help GNOME to get better. > I obviously had very different motives (and GNOME 3 was not around). I strongly disagree. You can't know that until you actually *try*. Why are you so afraid to try? > So if you want to help GNOME maybe you should discuss further with the > design team and see how to contribute in a positive manner. All I am trying to do is get some user feedback. Without such feedback I doubt any kind of discussion on the design would be fruitful, because it all be dismissed based on assumptions and wishful thinking. Cheers. -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
> Gathering feedback does not necessarily require an online user survey. > As stated, for a project which currently targets, among others, users > who do not care what parts of their operating system can be labelled > "GNOME" a survey is not a very reliable way of gathering feedback. > > Have you ever tried to explain, to a person who doesn't have an > interest in software, what GNOME actually is? Yes - my MBA research was into Linux desktops some years ago and did involve looking at end users attitudes. The quick summary from then would be: Most users used the desktop they got by default (whether because they didn't know to to switch or were never annoyed enough to bother I didn't have time to find out) The managers wanted a system that was a free exact clone of windows look/feel because change was expensive (training, lost time etc) The technies in the organisation often inflicted their personal desktop preference on the entire company. If I wanted to look at the "Gnome 3 is crap" assertion I think I would tackle it a bit differently as so much online updating is going on nowdays. Collect statistics from a few Fedora and other mirror sites, correlate downloads together by IP/time and other evidence, and look at how many of them download which desktops or combination of desktops. Repeat this over time and plot graphs. Distro popularity shifts may also provide some evidence for this. The trouble is while that will tell you about movement and popularity it will not tell you why. So it's a way to evaluate the claim "Gnome 3 is crap loads of people are changing or holding back on updating their desktop" but it's not going to answer useful things. There is a bit of value in knowing if lots of people hate or love Gnome 3, but the real value is knowing how it could be better for users, and counting downloads won't do that. And if real non-technical end users are like the ones I dealt with then asking them probably won't help either. Particularly in the business world to many of them at the time Gnone was 'click on this splodge in the morning to write letters' 'click on that thing in the corner to turn it off'. They are not decision makers either - impress their boss 8) The more interested and technically motivated people on the other hand can tell you stuff, "power users" particularly. They tell you stuff that reflects a particular use and understanding case though. Similarly you can learn an enormous amount by seeing what people are struggling with and what they do to the desktop - eg the various 'how to fix Gnome 3' pages tell you a lot about what people wanted and which is non-obvious for configuration. They are also from people who liked it enough to persevere so made an effort. > I urge you to consider the fact that if the majority of people > subscribed to desktop-devel-list don't have faith in idea of an online > user survey, an online user survey is probably not going to much have > effect on the views of the people who contribute to the discussions on > desktop-devel-list, and since most of the GNOME community read > desktop-devel-list you can probably extend this to all of the other > GNOME mailing lists and IRC channels as well. Some days I think Miguel got the Ximian monkey dead right, except that there should have been three of them. Alan ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, 2011-08-19 at 16:08 +0300, Felipe Contreras wrote: > >>> Likewise, > >>> 'happy' will be thought of differently by different people (a very odd > >>> word to include in a questionnaire, if you don't mind me saying): > >> > >> I think everyone understands the word happy. > > > > /ME wipes a mouthful of coffee from my monitor > > > > Then you haven't read enough of the survey research literature. > > That doesn't change the fact that everyone understands the word "happy". > Not necessary. Just to give an example - there is strong cultural influence how do you respond to simple question 'How are you'. In some cultures it is impolite to answer better then 'so so' and the normal answer is somehow along lines 'it could be worst, it could be better'. On the other hand the correct response in English is usually 'great' or 'fine' (to quote my teacher 'even if your house is burnt and your dog is terminally ill'). I have been warned to avoid 'standard' 'so so' response as I will be perceived as either impolite or after some large disaster because what I really meant was 'great'. (Somehow less directly related but also illustrates the problem of tricky words - in my native language friend means what in English is understood by close friend and many people whom I would call in English friend I would call in Polish acquaintance. Even though I know the difference I am less inclined to call people friends as in my mental model they are described by word 'acquaintance'). Of course this is 'just' cultural bias caused by people not being native speakers of English. You need to add individual bias. In each case it adds more and more 'noise' to survey. > >>> tl;dr version: your survey results will be misleading. > >> > >> No, the results would not be misleading; the *analysis* of the results > >> might. But different people can analyze them in different ways. The > >> important thing is to get *some* results. > > > > It seems bizarre to suggest that research data is valid irrespective > > of how it is gathered. If your questionnaire does not provide valid > > measurements no amount of analysis can compensate. > > You can thrown an analysis saying all this data is crap if that makes > you happier, but this survey won't eat babies. > I would argue that incorrect data (misinformation) is worst case then no data at all. Regards signature.asc Description: This is a digitally signed message part ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Le vendredi 19 août 2011 à 16:08 +0300, Felipe Contreras a écrit : > It's easy to throw empty criticism. Provide *suggestions*. Well, here’s a suggestion: since nobody knows how to address the correct target population or how to interpret the results, I suggest to spend our time fixing bugs instead. Cheers, -- .''`. Josselin Mouette : :' : `. `' `- ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On 08/19/2011 09:13 PM, Felipe Contreras wrote: > On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo wrote: >> On Fri 19 Aug 2011 13:33, Felipe Contreras >> writes: >> >>> That's a reasonable alternative. How about "pleased"? Any other people >>> have an opinion? >> >> You present yourself as reasonable by adjusting on the small points, >> but you ignore the feedback of greater importance. >> >> My opinion is that you are not the right person to lead an effort to >> gather feedback on GNOME. > > Is there anyone in the universe able to create a user survey worthy of > GNOME? Can you convince him of doing so? > > In the meantime, this is the best that we have. I will continue > listening for constructive feedback, but comments such as "this is not > good", "you are doing it wrong", "it's impossible", lead to nowhere. > > Besides, as Alan Cox said, it doesn't have to be perfect, like > software, we can learn from the mistakes of the 2011 survey, and make > a better one for 2012. Can we not? > Hi again, I actually thought that Andy's feedback was constructive. Your approach, motives and way to handle this discussion are questionable. You have obviously failed to convince the GNOME community and GNOME developers that your survey would be useful, and I'm afraid nobody feels like taking leadership on the project neither. A lot of people have already told you that enough feedback has been gathered at this time. I doubt you will get much endorsement or help around here anymore (I could of course be wrong on that last point). To tell you the truth I have been involved in trying to run a survey for GNOME 2 years ago IIRC (and I think it's a "recurrent project" - you can find some old pages on our wiki) with a group of other people, and we reached the same conclusion: a survey will not help GNOME to get better. I obviously had very different motives (and GNOME 3 was not around). So if you want to help GNOME maybe you should discuss further with the design team and see how to contribute in a positive manner. I hope you will find my comments helpful as that's what I am trying to convey (help to someone who seems full of energy to do something to improve GNOME - and we need people like this). All the best. Fred ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On 19 August 2011 14:13, Felipe Contreras wrote: > Is there anyone in the universe able to create a user survey worthy of > GNOME? Can you convince him of doing so? Do your survey with the questions you want, and come to your own conclusions. Blog about them if you want. You could even convince a distribution to include a popup with a link, although I think that's insane. Just don't tell people that it's from the GNOME project, in any way authorized or blessed by the ruling cabal[1] or developers. I'm pretty sure the majority of the people actually working on GNOME 3.2 don't want a survey at all. Sorry to be blunt. Richard. [1] http://ftp.gnome.org/conspiracy/ ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 2:13 PM, Felipe Contreras wrote: > On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo wrote: >> On Fri 19 Aug 2011 13:33, Felipe Contreras >> writes: >> >>> That's a reasonable alternative. How about "pleased"? Any other people >>> have an opinion? >> >> You present yourself as reasonable by adjusting on the small points, >> but you ignore the feedback of greater importance. >> >> My opinion is that you are not the right person to lead an effort to >> gather feedback on GNOME. > > Is there anyone in the universe able to create a user survey worthy of > GNOME? Can you convince him of doing so? Gathering feedback does not necessarily require an online user survey. As stated, for a project which currently targets, among others, users who do not care what parts of their operating system can be labelled "GNOME" a survey is not a very reliable way of gathering feedback. Have you ever tried to explain, to a person who doesn't have an interest in software, what GNOME actually is? > In the meantime, this is the best that we have. I will continue > listening for constructive feedback, but comments such as "this is not > good", "you are doing it wrong", "it's impossible", lead to nowhere. > > Besides, as Alan Cox said, it doesn't have to be perfect, like > software, we can learn from the mistakes of the 2011 survey, and make > a better one for 2012. Can we not? I urge you to consider the fact that if the majority of people subscribed to desktop-devel-list don't have faith in idea of an online user survey, an online user survey is probably not going to much have effect on the views of the people who contribute to the discussions on desktop-devel-list, and since most of the GNOME community read desktop-devel-list you can probably extend this to all of the other GNOME mailing lists and IRC channels as well. Sam ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri 19 Aug 2011 13:33, Felipe Contreras writes: > That's a reasonable alternative. How about "pleased"? Any other people > have an opinion? You present yourself as reasonable by adjusting on the small points, but you ignore the feedback of greater importance. My opinion is that you are not the right person to lead an effort to gather feedback on GNOME. The Git survey, AFAIU, was done _with_ the git developers. This one, if you manage to bully it through, will be _in spite of_ the GNOME developers. It will not have the effect you desire. Andy -- http://wingolog.org/ ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo wrote: > On Fri 19 Aug 2011 13:33, Felipe Contreras > writes: > >> That's a reasonable alternative. How about "pleased"? Any other people >> have an opinion? > > You present yourself as reasonable by adjusting on the small points, > but you ignore the feedback of greater importance. > > My opinion is that you are not the right person to lead an effort to > gather feedback on GNOME. Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? In the meantime, this is the best that we have. I will continue listening for constructive feedback, but comments such as "this is not good", "you are doing it wrong", "it's impossible", lead to nowhere. Besides, as Alan Cox said, it doesn't have to be perfect, like software, we can learn from the mistakes of the 2011 survey, and make a better one for 2012. Can we not? -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 3:17 PM, Allan Day wrote: > Felipe Contreras wrote: >> On Fri, Aug 19, 2011 at 1:14 PM, Allan Day wrote: >>> Felipe Contreras wrote: On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) wrote: > On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras > wrote: > ... >>> Different people will understand the words GNOME/happy/very >>> happy/ecstatic in different ways. Some might think 'GNOME' is their >>> distro (including the lower levels of the stack), >> >> Which is why we ask more question to understand their level of >> "geekness". That should help the make correlations; the people that >> use a terminal all the time more likely know that GNOME is just the >> DE. The people that don't have much experience might be confusing >> GNOME with the distribution. > > 'Geekness' is not the only thing that will affect people's > understandings, and you haven't adequately measured that anyway. Plus > that doesn't do anything to deal with the problem of what people > understand by 'GNOME'. It's easy to throw empty criticism. Provide *suggestions*. >>> Likewise, >>> 'happy' will be thought of differently by different people (a very odd >>> word to include in a questionnaire, if you don't mind me saying): >> >> I think everyone understands the word happy. > > /ME wipes a mouthful of coffee from my monitor > > Then you haven't read enough of the survey research literature. That doesn't change the fact that everyone understands the word "happy". > ... >> In any case, if you have suggestions that don't have these problems, >> feel free to share them. > > My suggestion would be to give up entirely or to rethink the premise > of your research. The latter is what I'd have advised when I was > working as a research consultant, or what I would have told one of my > students when I used to teach this stuff, for that matter. That's not helpful. If you are such a master, surely you can come up with a totally brand new user survey that is order of magnitude better. That would be greatly appreciated. >>> You've also got the representativeness problem. Your sample will >>> inevitably be unrepresentative, probably highly so. Even if 100% of >>> your *unrepresentative sample* tick the unhappy box, that doesn't tell >>> you much about your target population: you might just have sampled >>> every 'unhappy' GNOME user that's out there. >> >> If you can identify the bias, that's not a huge problem. > > So tell me - how will you accurately compensate for the effects of > self-selection bias? What kinds of claims will you make about > representativeness? What would *you* do? >>> tl;dr version: your survey results will be misleading. >> >> No, the results would not be misleading; the *analysis* of the results >> might. But different people can analyze them in different ways. The >> important thing is to get *some* results. > > It seems bizarre to suggest that research data is valid irrespective > of how it is gathered. If your questionnaire does not provide valid > measurements no amount of analysis can compensate. You can thrown an analysis saying all this data is crap if that makes you happier, but this survey won't eat babies. >>> We already have a wealth of data about peoples' experiences with GNOME >>> 3 and are working to address the issues that are being raised. It's >>> great that you want to help, but this survey really won't be useful. >> >> Where? I haven't seen any. > > We've had incredible amounts of feedback; most (if not all) of which > has been read, and which does get taken seriously. I also know that > those of us who are influencing the design of GNOME 3 take a strong > interest in peoples' experiences with it and ask them questions > (that's certainly what I do). There's also a small series of user > tests last I did Christmas, the results of which have been fed into > the development process. Believe me, that is more than enough to be > going on for now. (Some more user testing would be useful at some > point in the future, though.) For a "professor" you should know better. I want the data. Anyway, I am going to ignore your comments, unless you provide some *suggestions* for improvement. -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Felipe Contreras wrote: > On Fri, Aug 19, 2011 at 1:14 PM, Allan Day wrote: >> Felipe Contreras wrote: >>> On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) >>> wrote: On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras wrote: ... >> Different people will understand the words GNOME/happy/very >> happy/ecstatic in different ways. Some might think 'GNOME' is their >> distro (including the lower levels of the stack), > > Which is why we ask more question to understand their level of > "geekness". That should help the make correlations; the people that > use a terminal all the time more likely know that GNOME is just the > DE. The people that don't have much experience might be confusing > GNOME with the distribution. 'Geekness' is not the only thing that will affect people's understandings, and you haven't adequately measured that anyway. Plus that doesn't do anything to deal with the problem of what people understand by 'GNOME'. >> Likewise, >> 'happy' will be thought of differently by different people (a very odd >> word to include in a questionnaire, if you don't mind me saying): > > I think everyone understands the word happy. /ME wipes a mouthful of coffee from my monitor Then you haven't read enough of the survey research literature. ... > In any case, if you have suggestions that don't have these problems, > feel free to share them. My suggestion would be to give up entirely or to rethink the premise of your research. The latter is what I'd have advised when I was working as a research consultant, or what I would have told one of my students when I used to teach this stuff, for that matter. >> You've also got the representativeness problem. Your sample will >> inevitably be unrepresentative, probably highly so. Even if 100% of >> your *unrepresentative sample* tick the unhappy box, that doesn't tell >> you much about your target population: you might just have sampled >> every 'unhappy' GNOME user that's out there. > > If you can identify the bias, that's not a huge problem. So tell me - how will you accurately compensate for the effects of self-selection bias? What kinds of claims will you make about representativeness? >> tl;dr version: your survey results will be misleading. > > No, the results would not be misleading; the *analysis* of the results > might. But different people can analyze them in different ways. The > important thing is to get *some* results. It seems bizarre to suggest that research data is valid irrespective of how it is gathered. If your questionnaire does not provide valid measurements no amount of analysis can compensate. >> We already have a wealth of data about peoples' experiences with GNOME >> 3 and are working to address the issues that are being raised. It's >> great that you want to help, but this survey really won't be useful. > > Where? I haven't seen any. We've had incredible amounts of feedback; most (if not all) of which has been read, and which does get taken seriously. I also know that those of us who are influencing the design of GNOME 3 take a strong interest in peoples' experiences with it and ask them questions (that's certainly what I do). There's also a small series of user tests last I did Christmas, the results of which have been fed into the development process. Believe me, that is more than enough to be going on for now. (Some more user testing would be useful at some point in the future, though.) Allan -- IRC: aday on irc.gnome.org Blog: http://afaikblog.wordpress.com/ ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 2:40 PM, Zeeshan Ali (Khattak) wrote: > On Fri, Aug 19, 2011 at 2:17 PM, Felipe Contreras > wrote: >>> I didn't say this so far because it might sound like I am trying to >>> make a joke but since you still insist on your assertions about the >>> survey, I feel I must say this: How do you know people in general like >>> to participate in surveys? It is my observation that most people do >>> not like to do that, unless they have something to complain about. Now >>> this observation of mine could very well be wrong but how do we know >>> that? Do we do a survey to find out if people like to participate in >>> surveys? >> >> Are you serious? That totally and completely speculative and >> unrealistic. > > What is speculative? I made it very clear that it is *my* observation > and *if* it is correct, the results of this survey may very well be > wrong. Do you have any evidence that suggests that my observations > above are incorrect? Do you even know what speculation means? to make an inference based on inconclusive evidence; to surmise or conjecture [1] You don't have any evidence how often does this happens in real surveys, if at all. It's all based on conjecture. >> Have you ever participated in making a survey? > > No I have not but that does not necessarily mean what I said is > incorrect and could just be ignored by pointing to examples of other > surveys. If other people are ignoring an important issue, doesn't mean > we should do the same. You are again going off-tracks. Let's go back to the point. You said: > What if the results are all wrong and we don't have any way of knowing that? > Would those results still be better than nothing in your opinion? You can ignore the results. Problem solved, is it not? >> I have, as >> I have explained, for the Git survey. In my experience, only the >> people that want to help in some way do spend the amount of time >> required to fill the survey. > > As I have explained to you many times before, git's user-base is > mostly (if not all) geeks and those geeks know where the mailing-list > is and be able to access the survey easily. Still, I am a geek and a > very happy user of git but I didn't even know about the existence of > this survey until you told me. Even then, I didn't care to > participate. I am pretty sure I would have cared to participate if I > had something to complain about its current or planned features. > GNOME's user-base consists of people who do not even know what GNOME > is so many of them will not be able to participate, especially if they > are happy users. > > In short, example of git surveys are quite irrelevant here. So what if that's true? (I don't think so) At least I have a data-point of experience with surveys, you can discard it all you want, but what makes your speculation based on imaginary notions somehow more valid that my experience in real-world scenarios? At best you can say that they are both equally useless (I don't think so). The world is no filled with Zeeshans. Most people fill surveys truthfully. If you think otherwise, you can ignore the results. >> But again, as I said, if there's no survey on Earth you could trust, >> just ignore the results. Results by themselves cannot hurt you. > > In this case those results will really hurt since then you will have > some numbers to back-up your claim of "GNOME 3 is completely > unusable". *If* your motivation for this survey has remained the same, > you'll spread a lot of negative propaganda (which you already did even > when you didn't have any numbers) and many people will just say "Oh, > people don't like this gnome 3 thingie, must be shit" and will stay > away from it. Even if you don't do that, there is many others who will > use this "data" in that way. Aha, so that's what you are afraid. This survey will happen with or without GNOME's blessing. It would be in GNOME's best interest to improve the survey to get more useful results, and so far, I think many people have done so. However, at this point it's clear that you are not interested in improving the survey, all you are doing is making imaginary claims that lead to a dead-end; "all surveys are pointless, because the answers might be lies". There's no way to go forward from there. If you have some *suggestions* how to improve the survey to avoid whatever issues you see, then say so, otherwise I'll not explain any more why this is flawed thinking that leads nowhere. Cheers. [1] http://en.wiktionary.org/wiki/speculate -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 13:14, Allan Day wrote: > > We already have a wealth of data about peoples' experiences with GNOME 3 > Allan, this is interesting, what is the main pointer to access this data? ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 10:12 AM, Luc Pionchon wrote: > On Fri, Aug 19, 2011 at 03:34, Zeeshan Ali (Khattak) > wrote: > > [snip] > >> Maybe they all lied? > > Don't you think it is a bit early to speculate on results? (...) > > > Overall I can see already one clear result, even before the poll has > started: > > We do not know who is using GNOME. > > > > Maybe this needs reflexion No, in the current scenario that is actually a success story. Maybe this will change when GNOME OS becomes a reality. -- Regards, Zeeshan Ali (Khattak) FSF member#5124 ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 14:33, Felipe Contreras wrote: > On Fri, Aug 19, 2011 at 2:25 PM, Patryk Zawadzki > wrote: > > On Fri, Aug 19, 2011 at 1:17 PM, Felipe Contreras > > wrote: > >> Are you serious? That totally and completely speculative and > >> unrealistic. Have you ever participated in making a survey? I have, as > >> I have explained, for the Git survey. In my experience, only the > >> people that want to help in some way do spend the amount of time > >> required to fill the survey. > > > > Could you at least make the answer options less emotional? Like > > exchange "happy" for "satisfied" etc. I don't remember answering > > "ecstatic" in the Git survey but that could be my bad memory. > > That's a reasonable alternative. How about "pleased"? Any other people > have an opinion? > "satisfied" is good > > -- > Felipe Contreras > ___ > desktop-devel-list mailing list > [email protected] > http://mail.gnome.org/mailman/listinfo/desktop-devel-list > ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 2:17 PM, Felipe Contreras wrote: >> I didn't say this so far because it might sound like I am trying to >> make a joke but since you still insist on your assertions about the >> survey, I feel I must say this: How do you know people in general like >> to participate in surveys? It is my observation that most people do >> not like to do that, unless they have something to complain about. Now >> this observation of mine could very well be wrong but how do we know >> that? Do we do a survey to find out if people like to participate in >> surveys? > > Are you serious? That totally and completely speculative and > unrealistic. What is speculative? I made it very clear that it is *my* observation and *if* it is correct, the results of this survey may very well be wrong. Do you have any evidence that suggests that my observations above are incorrect? > Have you ever participated in making a survey? No I have not but that does not necessarily mean what I said is incorrect and could just be ignored by pointing to examples of other surveys. If other people are ignoring an important issue, doesn't mean we should do the same. > I have, as > I have explained, for the Git survey. In my experience, only the > people that want to help in some way do spend the amount of time > required to fill the survey. As I have explained to you many times before, git's user-base is mostly (if not all) geeks and those geeks know where the mailing-list is and be able to access the survey easily. Still, I am a geek and a very happy user of git but I didn't even know about the existence of this survey until you told me. Even then, I didn't care to participate. I am pretty sure I would have cared to participate if I had something to complain about its current or planned features. GNOME's user-base consists of people who do not even know what GNOME is so many of them will not be able to participate, especially if they are happy users. In short, example of git surveys are quite irrelevant here. > But again, as I said, if there's no survey on Earth you could trust, > just ignore the results. Results by themselves cannot hurt you. In this case those results will really hurt since then you will have some numbers to back-up your claim of "GNOME 3 is completely unusable". *If* your motivation for this survey has remained the same, you'll spread a lot of negative propaganda (which you already did even when you didn't have any numbers) and many people will just say "Oh, people don't like this gnome 3 thingie, must be shit" and will stay away from it. Even if you don't do that, there is many others who will use this "data" in that way. -- Regards, Zeeshan Ali (Khattak) FSF member#5124 ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 2:25 PM, Patryk Zawadzki wrote: > On Fri, Aug 19, 2011 at 1:17 PM, Felipe Contreras > wrote: >> Are you serious? That totally and completely speculative and >> unrealistic. Have you ever participated in making a survey? I have, as >> I have explained, for the Git survey. In my experience, only the >> people that want to help in some way do spend the amount of time >> required to fill the survey. > > Could you at least make the answer options less emotional? Like > exchange "happy" for "satisfied" etc. I don't remember answering > "ecstatic" in the Git survey but that could be my bad memory. That's a reasonable alternative. How about "pleased"? Any other people have an opinion? -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 1:17 PM, Felipe Contreras wrote: > Are you serious? That totally and completely speculative and > unrealistic. Have you ever participated in making a survey? I have, as > I have explained, for the Git survey. In my experience, only the > people that want to help in some way do spend the amount of time > required to fill the survey. Could you at least make the answer options less emotional? Like exchange "happy" for "satisfied" etc. I don't remember answering "ecstatic" in the Git survey but that could be my bad memory. -- Patryk Zawadzki I solve problems. ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 1:14 PM, Allan Day wrote: > Felipe Contreras wrote: >> On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) >> wrote: >>> On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras >>> wrote: Nothing is ever perfect, but having at least some results is better than nothing. >>> >>> Since you have repeated this assertion a few times, I must ask: What >>> if the results are all wrong and we don't have any way of knowing >>> that? Would those results still be better than nothing in your >>> opinion? >> >> What do you mean by all wrong? Let's assume that the results show that >> 1000 people are not happy with GNOME. How can that be wrong? 1000 >> people responded that, the results were not somehow altered, or >> boycotted, the results are the results, and that's that. > ... > > 'Wrong' in social research typically means that your results lack > validity: that you think the data is measuring one thing (eg. 'GNOME > users' happiness with GNOME 3') but is in fact measuring something > else. > > When you do survey research, you have to be certain that one person > understands the questions in the same way that another person does. > Looking at your questionnaire, that won't be the case. An example: > >> === 02. Overall, how happy are you with GNOME? === >> (single choice) >> >> * unhappy >> * not so happy >> * happy >> * very happy >> * completely ecstatic > > Different people will understand the words GNOME/happy/very > happy/ecstatic in different ways. Some might think 'GNOME' is their > distro (including the lower levels of the stack), Which is why we ask more question to understand their level of "geekness". That should help the make correlations; the people that use a terminal all the time more likely know that GNOME is just the DE. The people that don't have much experience might be confusing GNOME with the distribution. > Likewise, > 'happy' will be thought of differently by different people (a very odd > word to include in a questionnaire, if you don't mind me saying): I think everyone understands the word happy. That is what is used in Git user survey, and seems to be doing the job just fine. In any case, if you have suggestions that don't have these problems, feel free to share them. > You've also got the representativeness problem. Your sample will > inevitably be unrepresentative, probably highly so. Even if 100% of > your *unrepresentative sample* tick the unhappy box, that doesn't tell > you much about your target population: you might just have sampled > every 'unhappy' GNOME user that's out there. If you can identify the bias, that's not a huge problem. > tl;dr version: your survey results will be misleading. No, the results would not be misleading; the *analysis* of the results might. But different people can analyze them in different ways. The important thing is to get *some* results. > We already have a wealth of data about peoples' experiences with GNOME > 3 and are working to address the issues that are being raised. It's > great that you want to help, but this survey really won't be useful. Where? I haven't seen any. -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Fri, Aug 19, 2011 at 3:34 AM, Zeeshan Ali (Khattak) wrote: > On Fri, Aug 19, 2011 at 2:09 AM, Felipe Contreras > wrote: >> On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) >> wrote: >>> On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras >>> wrote: Nothing is ever perfect, but having at least some results is better than nothing. >>> >>> Since you have repeated this assertion a few times, I must ask: What >>> if the results are all wrong and we don't have any way of knowing >>> that? Would those results still be better than nothing in your >>> opinion? >> >> What do you mean by all wrong? Let's assume that the results show that >> 1000 people are not happy with GNOME. How can that be wrong? > > Maybe they all lied? Maybe people who are satisfied do not want to > or have time to take part in surveys and you only get people who are > not happy into the survey? In which case, the results may show results > that are not correct. i-e a significantly large number of participant > say that they are very unhappy with GNOME but what if that number is > nothing compared to the number of people who are very much satisfied > with GNOME? > > I didn't say this so far because it might sound like I am trying to > make a joke but since you still insist on your assertions about the > survey, I feel I must say this: How do you know people in general like > to participate in surveys? It is my observation that most people do > not like to do that, unless they have something to complain about. Now > this observation of mine could very well be wrong but how do we know > that? Do we do a survey to find out if people like to participate in > surveys? Are you serious? That totally and completely speculative and unrealistic. Have you ever participated in making a survey? I have, as I have explained, for the Git survey. In my experience, only the people that want to help in some way do spend the amount of time required to fill the survey. But again, as I said, if there's no survey on Earth you could trust, just ignore the results. Results by themselves cannot hurt you. -- Felipe Contreras ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
Felipe Contreras wrote: > On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) > wrote: >> On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras >> wrote: >>> >>> Nothing is ever perfect, but having at least some results is better >>> than nothing. >> >> Since you have repeated this assertion a few times, I must ask: What >> if the results are all wrong and we don't have any way of knowing >> that? Would those results still be better than nothing in your >> opinion? > > What do you mean by all wrong? Let's assume that the results show that > 1000 people are not happy with GNOME. How can that be wrong? 1000 > people responded that, the results were not somehow altered, or > boycotted, the results are the results, and that's that. ... 'Wrong' in social research typically means that your results lack validity: that you think the data is measuring one thing (eg. 'GNOME users' happiness with GNOME 3') but is in fact measuring something else. When you do survey research, you have to be certain that one person understands the questions in the same way that another person does. Looking at your questionnaire, that won't be the case. An example: > === 02. Overall, how happy are you with GNOME? === > (single choice) > > * unhappy > * not so happy > * happy > * very happy > * completely ecstatic Different people will understand the words GNOME/happy/very happy/ecstatic in different ways. Some might think 'GNOME' is their distro (including the lower levels of the stack), some that it's their 'shell', others that it's all their GUI software [1]. Likewise, 'happy' will be thought of differently by different people (a very odd word to include in a questionnaire, if you don't mind me saying): there are a vast range of expectations and usage patterns in relation to desktop computers, all of which will affect how people respond. Someone could tick 'unhappy' but by most measures have had a perfectly satisfactory experience. You've also got the representativeness problem. Your sample will inevitably be unrepresentative, probably highly so. Even if 100% of your *unrepresentative sample* tick the unhappy box, that doesn't tell you much about your target population: you might just have sampled every 'unhappy' GNOME user that's out there. tl;dr version: your survey results will be misleading. We already have a wealth of data about peoples' experiences with GNOME 3 and are working to address the issues that are being raised. It's great that you want to help, but this survey really won't be useful. Allan [1] GNOME's place in the stack means that you can't really do satisfaction surveys on it. This is one reason why GNOME is a more difficult research topic than, say, Git. -- IRC: aday on irc.gnome.org Blog: http://afaikblog.wordpress.com/ ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME user survey 2011 (v4)
On Thu, Aug 18, 2011 at 19:53, Stormy Peters wrote: > > All the questions after this assume a knowledge of GNOME and how our > community works. That's fine if you are polling developers. If you are > polling average users, then I think it's not worth asking. Another issue that I don't think has been raised yet (sorry if it has, this thread reached the limits a while ago) is that the changes we make to GNOME aren't intended just for existing GNU/Linux users or people with similar interests. My understanding is that we want GNOME to be useful to people that haven't been attracted yet to any of the existing free desktops (most of the people in this world). Somewhat related, you need to take into account people's natural resistance to change. A factor that isn't equally relevant when surveying future users as it is when surveying present ones. Regards, Tomeu ___ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
