Re: [Haskell-community] 2018 state of Haskell survey results

[Simon Peyton Jones via Haskell-community]

Just wanted to add in: good catch Gershom on identifying the problem, and thank 
you Taylor for working to remove them from the report.

I'd like to add +1 to that.

It's a source of astonishment, and some dismay, to me that anyone would go to 
so much trouble to affect a survey about Haskell.  (Brexit, perhaps, but 
Haskell??)

But many thanks to Gershom and Taylor for dealing with it so professionally.

Simon


From: Haskell-community  On Behalf Of 
Michael Snoyman
Sent: 18 November 2018 19:32
To: Taylor Fausak 
Cc: haskell-community@haskell.org
Subject: Re: [Haskell-community] 2018 state of Haskell survey results

Just wanted to add in: good catch Gershom on identifying the problem, and thank 
you Taylor for working to remove them from the report.


On 18 Nov 2018, at 21:17, Taylor Fausak 
mailto:tay...@fausak.me>> wrote:

Great catch, Gershom! There are indeed about 300 responses that tick all the 
boxes except for disliking the new GHC release schedule. The main thing the 
attacker seemed to be interested in was over-representing Stack and Stackage. 
Also, bizarrely, Java.

That brings the number of bogus responses up to 3,735, which puts the number of 
legitimate responses at 1,361. For context, last year's survey asked far fewer 
questions and had 1,335 responses.


On Sun, Nov 18, 2018, at 1:26 PM, Imants Cekusins wrote:
What if the announcement mentioned a large number of potentially bogus 
responses, explained the grounds for this conclusion, with a new survey 
conducted early next year?

The next survey would then need to be done differently from this one somehow. 
To improve the reliability, some authentication may be necessary.


Maybe Stack, Cabal questions could be grouped as separate distinct surveys, 
conducted by their maintainers through own channels?

Not sure how much value is in exact numbers of users of Stack or Cabal. Both 
groups are large enough. The maintainers of both groups are aware about usage 
stats.

Is either library likely to be influenced by this survey?
___
Haskell-community mailing list
Haskell-community@haskell.org<mailto:Haskell-community@haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

___
Haskell-community mailing list
Haskell-community@haskell.org<mailto:Haskell-community@haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey results

[Richard Eisenberg]

OK. Thanks for sharing some statistics. I'm now convinced as to the 
characterization of the attack. I'm still glad for how the public post 
diplomatically handled this.

> I will also say, though there's speculation about "false flags" and

Oof. That thought never crossed my mind. I can only imagine this is on some 
social media where I don't participate. Every day, I am more and more pleased 
with my non-presence on most social media. :) Besides, just keeping up with 
email is enough of a challenge.

Thanks for the clarification.

Richard

> other silliness floating around that I personally have a very good
> guess as to who did this. There's one well-known troll who has these
> preoccupations and is known for creating serial sockpuppet accounts,
> and is just the right amount of obsessed to do something like this. A
> few of the bogus responses actually had comments, and the comments
> were all written in a voice that was unmistakeable as this troll as
> well. Occam's razor seems to apply.
> 
> Finally, let me add why I don't think this was a "false flag" -- while
> there were enough telltale markers that the fake answers could seem to
> be detected, I don't think this was on purpose. There was _too much_
> effort put into distributions of other choices, etc. If they had
> wanted the fakes to be detected they would have left much stronger
> evidence. Rather, from a forensic standpoint, this seems pretty clear
> to me that the pattern of data is of someone _trying_ to cover their
> tracks, but just making four or five errors which I could assemble
> into a pattern. If they hadn't made those errors -- likely based on
> bad priors about what the organic data would be that theirs would need
> to "mesh" into -- then I think the deception would have been much
> harder to detect.
> 
> --Gershom
> 
>> Given the contention around cabal vs stack, I agree that sociological 
>> concerns suggest that the troll meant to tilt those scales. But I wouldn't 
>> want a public accusation without at least some statistical analysis that 
>> independently supports that conclusion.
>> 
>> In any case, thanks to all for putting this together!
>> 
>> Richard
>> 
>> On Nov 18, 2018, at 4:31 PM, Taylor Fausak  wrote:
>> 
>> Oops, the ordering of the answer choices is manual because some questions 
>> have a natural order while others should just be most to least popular. I've 
>> made another run through to make sure everything is sorted properly. I'll 
>> probably hit publish in the next half hour or so unless there are any 
>> objections.
>> 
>> https://github.com/tfausak/tfausak.github.io/blob/fce97d07c369856d4c05b756c492eb6229a1b5c7/_posts/2018-11-18-2018-state-of-haskell-survey-results.markdown
>> 
>> 
>> On Sun, Nov 18, 2018, at 3:07 PM, Gershom B wrote:
>> 
>> The language extensions section doesn’t appear to be sorted properly. 
>> Outside of that, I think that these results are looking much better and any 
>> effort to find any additional outliers is probably not worth it for the 
>> moment. Thanks for your work on this, and I appreciate you being responsive 
>> and attentive when problems with the data were pointed out. There’s 
>> certainly some interesting and helpful information to be gleaned from this 
>> data.
>> 
>> Cheers,
>> Gershom
>> 
>> 
>> 
>> 
>> On November 18, 2018 at 2:55:10 PM, Taylor Fausak (tay...@fausak.me) wrote:
>> 
>> 
>> 
>> 
>> Ok, I updated the function that checks for bad responses, re-ran the script, 
>> and updated the announcement along with all the assets (charts, tables, and 
>> CSV). Hopefully it's the last time, as I can't justify spending much more 
>> time on this.
>> 
>> https://github.com/tfausak/tfausak.github.io/blob/6f9991758ffeed085c45dd97e4ce6a82a8b1a73f/_posts/2018-11-18-2018-state-of-haskell-survey-results.markdown
>> 
>> 
>> On Sun, Nov 18, 2018, at 2:32 PM, Michael Snoyman wrote:
>> 
>> Just wanted to add in: good catch Gershom on identifying the problem, and 
>> thank you Taylor for working to remove them from the report.
>> 
>> On 18 Nov 2018, at 21:17, Taylor Fausak  wrote:
>> 
>> Great catch, Gershom! There are indeed about 300 responses that tick all the 
>> boxes except for disliking the new GHC release schedule. The main thing the 
>> attacker seemed to be interested in was over-representing Stack and 
>> Stackage. Also, bizarrely, Java.
>> 
>> That brings the number of bogus responses up to 3,735, which puts the number 
>> of legitimate responses at 1,361. For context, last year's survey asked far 
>> fewer questions and had 1,335 responses.
>> 
>> 
>> On Sun, Nov 18, 2018, at 1:26 PM, Imants Cekusins wrote:
>> 
>> What if the announcement mentioned a large number of potentially bogus 
>> responses, explained the grounds for this conclusion, with a new survey 
>> conducted early next year?
>> 
>> The next survey would then need to be done differently from this one 
>> somehow. To improve the reliability, some authentication may be necessary.
>> 
>> 
>> Maybe Stack, 

Re: [Haskell-community] 2018 state of Haskell survey results

[Francesco Ariis]

Hello Richard,

On Sun, Nov 18, 2018 at 11:20:52PM -0500, Richard Eisenberg wrote:
> I have not analyzed the data myself, but I wonder how we jumped to the
> conclusion that the troll was trying to promote Stack. Is there
> statistical data that supports that conclusion? For example, just reading
> this thread, it sounds like the bogus responses also really don't like
> the new release schedule. Maybe the troll wants the old release schedule
> back and was just lazy about programming the tool to vary the
> stack/cabal question answers adequately.

If you filter the results for the (impossible) "linux/mac AND notepad++"
combination, you can check the pattern-of-action of the troll.
Every demographic question is skipped; every "write in" answer is skipped;
all the other questions are filled in with a random value, bar the
"build tools" one and the "release schedule" one, both having a constant
value.
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey results

[Chris Smith]

> For example, just reading this thread, it sounds like the bogus responses
also really don't like the new release schedule. Maybe the troll wants the
old release schedule back and was just lazy about programming the tool to
vary the stack/cabal question answers adequately.

There is another scenario, though, which should caution against making
official statements about motivation.  There was a set of people who worked
very hard while the survey was open to preemptively cast doubt on its
motivation and goals.  It may be that someone was mainly attempting to
sabotage the survey results themselves, rather than taking a side in any
specific dispute.  Of course, had the results been published claiming that
a mere 12% of Haskellers use Cabal, it would have been immediately
dismissed by many people as obviously biased, which would have achieved
that goal, too.

I think Taylor's post handled this well, saying what we know to be true,
that the attack targeted divisive issues, but without drawing unnecessary
conclusions.

On Sun, Nov 18, 2018 at 11:21 PM Richard Eisenberg 
wrote:

> I have not analyzed the data myself, but I wonder how we jumped to the
> conclusion that the troll was trying to promote Stack. Is there statistical
> data that supports that conclusion? For example, just reading this thread,
> it sounds like the bogus responses also really don't like the new release
> schedule. Maybe the troll wants the old release schedule back and was just
> lazy about programming the tool to vary the stack/cabal question answers
> adequately.
>
> Given the contention around cabal vs stack, I agree that sociological
> concerns suggest that the troll meant to tilt those scales. But I wouldn't
> want a public accusation without at least some statistical analysis that
> independently supports that conclusion.
>
> In any case, thanks to all for putting this together!
>
> Richard
>
> On Nov 18, 2018, at 4:31 PM, Taylor Fausak  wrote:
>
> Oops, the ordering of the answer choices is manual because some questions
> have a natural order while others should just be most to least popular.
> I've made another run through to make sure everything is sorted properly.
> I'll probably hit publish in the next half hour or so unless there are any
> objections.
>
>
> https://github.com/tfausak/tfausak.github.io/blob/fce97d07c369856d4c05b756c492eb6229a1b5c7/_posts/2018-11-18-2018-state-of-haskell-survey-results.markdown
>
>
> On Sun, Nov 18, 2018, at 3:07 PM, Gershom B wrote:
>
> The language extensions section doesn’t appear to be sorted properly.
> Outside of that, I think that these results are looking much better and any
> effort to find any additional outliers is probably not worth it for the
> moment. Thanks for your work on this, and I appreciate you being responsive
> and attentive when problems with the data were pointed out. There’s
> certainly some interesting and helpful information to be gleaned from this
> data.
>
> Cheers,
> Gershom
>
>
>
>
> On November 18, 2018 at 2:55:10 PM, Taylor Fausak (tay...@fausak.me)
> wrote:
>
>
>
>
> Ok, I updated the function that checks for bad responses, re-ran the
> script, and updated the announcement along with all the assets (charts,
> tables, and CSV). Hopefully it's the last time, as I can't justify spending
> much more time on this.
>
>
> https://github.com/tfausak/tfausak.github.io/blob/6f9991758ffeed085c45dd97e4ce6a82a8b1a73f/_posts/2018-11-18-2018-state-of-haskell-survey-results.markdown
>
>
> On Sun, Nov 18, 2018, at 2:32 PM, Michael Snoyman wrote:
>
> Just wanted to add in: good catch Gershom on identifying the problem, and
> thank you Taylor for working to remove them from the report.
>
> On 18 Nov 2018, at 21:17, Taylor Fausak  wrote:
>
> Great catch, Gershom! There are indeed about 300 responses that tick all
> the boxes except for disliking the new GHC release schedule. The main thing
> the attacker seemed to be interested in was over-representing Stack and
> Stackage. Also, bizarrely, Java.
>
> That brings the number of bogus responses up to 3,735, which puts the
> number of legitimate responses at 1,361. For context, last year's survey
> asked far fewer questions and had 1,335 responses.
>
>
> On Sun, Nov 18, 2018, at 1:26 PM, Imants Cekusins wrote:
>
> What if the announcement mentioned a large number of potentially bogus
> responses, explained the grounds for this conclusion, with a new survey
> conducted early next year?
>
> The next survey would then need to be done differently from this one
> somehow. To improve the reliability, some authentication may be necessary.
>
>
> Maybe Stack, Cabal questions could be grouped as separate distinct
> surveys, conducted by their maintainers through own channels?
>
> Not sure how much value is in exact numbers of users of Stack or Cabal.
> Both groups are large enough. The maintainers of both groups are aware
> about usage stats.
>
> Is either library likely to be influenced by this survey?
> 

Re: [Haskell-community] 2018 state of Haskell survey results

[Richard Eisenberg]

I have not analyzed the data myself, but I wonder how we jumped to the 
conclusion that the troll was trying to promote Stack. Is there statistical 
data that supports that conclusion? For example, just reading this thread, it 
sounds like the bogus responses also really don't like the new release 
schedule. Maybe the troll wants the old release schedule back and was just lazy 
about programming the tool to vary the stack/cabal question answers adequately.

Given the contention around cabal vs stack, I agree that sociological concerns 
suggest that the troll meant to tilt those scales. But I wouldn't want a public 
accusation without at least some statistical analysis that independently 
supports that conclusion.

In any case, thanks to all for putting this together!

Richard

> On Nov 18, 2018, at 4:31 PM, Taylor Fausak  wrote:
> 
> Oops, the ordering of the answer choices is manual because some questions 
> have a natural order while others should just be most to least popular. I've 
> made another run through to make sure everything is sorted properly. I'll 
> probably hit publish in the next half hour or so unless there are any 
> objections.
> 
> https://github.com/tfausak/tfausak.github.io/blob/fce97d07c369856d4c05b756c492eb6229a1b5c7/_posts/2018-11-18-2018-state-of-haskell-survey-results.markdown
>  
> 
> 
> 
> On Sun, Nov 18, 2018, at 3:07 PM, Gershom B wrote:
>> The language extensions section doesn’t appear to be sorted properly. 
>> Outside of that, I think that these results are looking much better and any 
>> effort to find any additional outliers is probably not worth it for the 
>> moment. Thanks for your work on this, and I appreciate you being responsive 
>> and attentive when problems with the data were pointed out. There’s 
>> certainly some interesting and helpful information to be gleaned from this 
>> data.
>> 
>> Cheers,
>> Gershom
>> 
>> 
>> 
>> 
>> On November 18, 2018 at 2:55:10 PM, Taylor Fausak (tay...@fausak.me 
>> ) wrote:
>> 
>>> 
>>> 
>>> 
>>> Ok, I updated the function that checks for bad responses, re-ran the 
>>> script, and updated the announcement along with all the assets (charts, 
>>> tables, and CSV). Hopefully it's the last time, as I can't justify spending 
>>> much more time on this.
>>> 
>>> https://github.com/tfausak/tfausak.github.io/blob/6f9991758ffeed085c45dd97e4ce6a82a8b1a73f/_posts/2018-11-18-2018-state-of-haskell-survey-results.markdown
>>>  
>>> 
>>> 
>>> 
>>> On Sun, Nov 18, 2018, at 2:32 PM, Michael Snoyman wrote:
 Just wanted to add in: good catch Gershom on identifying the problem, and 
 thank you Taylor for working to remove them from the report.
 
> On 18 Nov 2018, at 21:17, Taylor Fausak  > wrote:
> 
> Great catch, Gershom! There are indeed about 300 responses that tick all 
> the boxes except for disliking the new GHC release schedule. The main 
> thing the attacker seemed to be interested in was over-representing Stack 
> and Stackage. Also, bizarrely, Java. 
> 
> That brings the number of bogus responses up to 3,735, which puts the 
> number of legitimate responses at 1,361. For context, last year's survey 
> asked far fewer questions and had 1,335 responses.
> 
> 
> On Sun, Nov 18, 2018, at 1:26 PM, Imants Cekusins wrote:
>> What if the announcement mentioned a large number of potentially bogus 
>> responses, explained the grounds for this conclusion, with a new survey 
>> conducted early next year?
>> 
>> The next survey would then need to be done differently from this one 
>> somehow. To improve the reliability, some authentication may be 
>> necessary.
>> 
>> 
>> Maybe Stack, Cabal questions could be grouped as separate distinct 
>> surveys, conducted by their maintainers through own channels?
>> 
>> Not sure how much value is in exact numbers of users of Stack or Cabal. 
>> Both groups are large enough. The maintainers of both groups are aware 
>> about usage stats.
>> 
>> Is either library likely to be influenced by this survey?
>> ___
>> Haskell-community mailing list
>> Haskell-community@haskell.org 
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community 
>> 
> 
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org 
> 

Re: [Haskell-community] 2018 state of Haskell survey results

[Gershom B]

The language extensions section doesn’t appear to be sorted properly.
Outside of that, I think that these results are looking much better and any
effort to find any additional outliers is probably not worth it for the
moment. Thanks for your work on this, and I appreciate you being responsive
and attentive when problems with the data were pointed out. There’s
certainly some interesting and helpful information to be gleaned from this
data.

Cheers,
Gershom



On November 18, 2018 at 2:55:10 PM, Taylor Fausak (tay...@fausak.me) wrote:

Ok, I updated the function that checks for bad responses, re-ran the
script, and updated the announcement along with all the assets (charts,
tables, and CSV). Hopefully it's the last time, as I can't justify spending
much more time on this.

https://github.com/tfausak/tfausak.github.io/blob/6f9991758ffeed085c45dd97e4ce6a82a8b1a73f/_posts/2018-11-18-2018-state-of-haskell-survey-results.markdown


On Sun, Nov 18, 2018, at 2:32 PM, Michael Snoyman wrote:

Just wanted to add in: good catch Gershom on identifying the problem, and
thank you Taylor for working to remove them from the report.

On 18 Nov 2018, at 21:17, Taylor Fausak  wrote:

Great catch, Gershom! There are indeed about 300 responses that tick all
the boxes except for disliking the new GHC release schedule. The main thing
the attacker seemed to be interested in was over-representing Stack and
Stackage. Also, bizarrely, Java.

That brings the number of bogus responses up to 3,735, which puts the
number of legitimate responses at 1,361. For context, last year's survey
asked far fewer questions and had 1,335 responses.


On Sun, Nov 18, 2018, at 1:26 PM, Imants Cekusins wrote:

What if the announcement mentioned a large number of potentially bogus
responses, explained the grounds for this conclusion, with a new survey
conducted early next year?

The next survey would then need to be done differently from this one
somehow. To improve the reliability, some authentication may be necessary.


Maybe Stack, Cabal questions could be grouped as separate distinct surveys,
conducted by their maintainers through own channels?

Not sure how much value is in exact numbers of users of Stack or Cabal.
Both groups are large enough. The maintainers of both groups are aware
about usage stats.

Is either library likely to be influenced by this survey?
*___*
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community


___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community


___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey results

[Michael Snoyman]

Just wanted to add in: good catch Gershom on identifying the problem, and thank 
you Taylor for working to remove them from the report.

> On 18 Nov 2018, at 21:17, Taylor Fausak  wrote:
> 
> Great catch, Gershom! There are indeed about 300 responses that tick all the 
> boxes except for disliking the new GHC release schedule. The main thing the 
> attacker seemed to be interested in was over-representing Stack and Stackage. 
> Also, bizarrely, Java. 
> 
> That brings the number of bogus responses up to 3,735, which puts the number 
> of legitimate responses at 1,361. For context, last year's survey asked far 
> fewer questions and had 1,335 responses.
> 
> 
> On Sun, Nov 18, 2018, at 1:26 PM, Imants Cekusins wrote:
>> What if the announcement mentioned a large number of potentially bogus 
>> responses, explained the grounds for this conclusion, with a new survey 
>> conducted early next year?
>> 
>> The next survey would then need to be done differently from this one 
>> somehow. To improve the reliability, some authentication may be necessary.
>> 
>> 
>> Maybe Stack, Cabal questions could be grouped as separate distinct surveys, 
>> conducted by their maintainers through own channels?
>> 
>> Not sure how much value is in exact numbers of users of Stack or Cabal. Both 
>> groups are large enough. The maintainers of both groups are aware about 
>> usage stats.
>> 
>> Is either library likely to be influenced by this survey?
>> ___
>> Haskell-community mailing list
>> Haskell-community@haskell.org 
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community 
>> 
> 
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org 
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community 
> 
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey results

[Taylor Fausak]

Great catch, Gershom! There are indeed about 300 responses that tick all
the boxes except for disliking the new GHC release schedule. The main
thing the attacker seemed to be interested in was over-representing
Stack and Stackage. Also, bizarrely, Java.
That brings the number of bogus responses up to 3,735, which puts the
number of legitimate responses at 1,361. For context, last year's survey
asked far fewer questions and had 1,335 responses.

On Sun, Nov 18, 2018, at 1:26 PM, Imants Cekusins wrote:
> What if the announcement mentioned a large number of potentially bogus
> responses, explained the grounds for this conclusion, with a new
> survey conducted early next year?> 
> The next survey would then need to be done differently from this one
> somehow. To improve the reliability, some authentication may be
> necessary.> 
> 
> Maybe Stack, Cabal questions could be grouped as separate distinct
> surveys, conducted by their maintainers through own channels?> 
> Not sure how much value is in exact numbers of users of Stack or
> Cabal. Both groups are large enough. The maintainers of both groups
> are aware about usage stats.> 
> Is either library likely to be influenced by this survey?
> _
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey results

[Chris Smith]

; The differences continue and defy all probability. Of the “no
> demographics” group, almost everyone dislikes the new release schedule. Of
> the “demographics” group there are answers that like it, were not aware of
> it, or are indifferent, but almost nobody dislikes it. There is naturally a
> difference in proportions of cabal/stack and hackage/stackage responses as
> well.
>
> There are a lot of other things I could point to as well. But, bluntly
> put, I think that some disaffected party or parties wrote a crude script
> and submitted over 3,000 fake responses. Luckily for us, they were not very
> smart, and made some obvious errors, so in this case we can weed out the
> bad responses (although, sadly, losing at least a few real ones as well).
>
> However, assuming  this party isn’t entirely stupid, it doesn’t bode well
> for future surveys as they may get at least slightly less dumb in the
> future if they decide to keep it up :-/
>
> —Gershom
>
>
>
> On November 18, 2018 at 1:10:31 AM, Gershom B (gersh...@gmail.com) wrote:
>
>
>
> This is interesting, but I’m thoroughly confused. Over 2500 people said
> they took last year’s survey, but it only had roughly 1,300 respondants?
>
>
> On Sat, Nov 17, 2018 at 9:56 PM Taylor Fausak  wrote:
>
> Hello! It took a little longer than I expected, but I am nearly ready to
> announce the 2018 state of Haskell survey results. Some community members
> have expressed interest in seeing the announcement post before it's
> published. If you are one of those people, you can see the results here:
> https://github.com/tfausak/tfausak.github.io/blob/7e4937e284a3068add9e9af6b585c8d0215ff360/_posts/2018-11-16-2018-state-of-haskell-survey-results.markdown
>
> If you would like to suggest changes to the announcement post, please
> respond to this email, send me an email directly, or reply to this pull
> request on GitHub: https://github.com/tfausak/tfausak.github.io/pull/148
>
> I plan on publishing the results tomorrow. Once the results are published,
> the post is by no means set in stone. I will happily accept suggestions
> from anyone at any time.
>
> Thank you!
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
>
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
> *___*
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
>
>
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey results

[Gershom B]

cs” group there are answers that like it, were not aware of it, or 
> are indifferent, but almost nobody dislikes it. There is naturally a 
> difference in proportions of cabal/stack and hackage/stackage responses as 
> well.
>
> There are a lot of other things I could point to as well. But, bluntly put, I 
> think that some disaffected party or parties wrote a crude script and 
> submitted over 3,000 fake responses. Luckily for us, they were not very 
> smart, and made some obvious errors, so in this case we can weed out the bad 
> responses (although, sadly, losing at least a few real ones as well).
>
> However, assuming  this party isn’t entirely stupid, it doesn’t bode well for 
> future surveys as they may get at least slightly less dumb in the future if 
> they decide to keep it up :-/
>
> —Gershom
>
>
>
> On November 18, 2018 at 1:10:31 AM, Gershom B (gersh...@gmail.com) wrote:
>
>
>
> This is interesting, but I’m thoroughly confused. Over 2500 people said they 
> took last year’s survey, but it only had roughly 1,300 respondants?
>
>
> On Sat, Nov 17, 2018 at 9:56 PM Taylor Fausak  wrote:
>
> Hello! It took a little longer than I expected, but I am nearly ready to 
> announce the 2018 state of Haskell survey results. Some community members 
> have expressed interest in seeing the announcement post before it's 
> published. If you are one of those people, you can see the results here: 
> https://github.com/tfausak/tfausak.github.io/blob/7e4937e284a3068add9e9af6b585c8d0215ff360/_posts/2018-11-16-2018-state-of-haskell-survey-results.markdown
>
> If you would like to suggest changes to the announcement post, please respond 
> to this email, send me an email directly, or reply to this pull request on 
> GitHub: https://github.com/tfausak/tfausak.github.io/pull/148
>
> I plan on publishing the results tomorrow. Once the results are published, 
> the post is by no means set in stone. I will happily accept suggestions from 
> anyone at any time.
>
> Thank you!
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
>
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
>
>
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey results

[Taylor Fausak]

ge/stackage responses as well.>>>> 
>>>> There are a lot of other things I could point to as well. But,
>>>> bluntly put, I think that some disaffected party or parties wrote a
>>>> crude script and submitted over 3,000 fake responses. Luckily for
>>>> us, they were not very smart, and made some obvious errors, so in
>>>> this case we can weed out the bad responses (although, sadly,
>>>> losing at least a few real ones as well).>>>> 
>>>> However, assuming  this party isn’t entirely stupid, it doesn’t
>>>> bode well for future surveys as they may get at least slightly less
>>>> dumb in the future if they decide to keep it up :-/>>>> 
>>>> —Gershom
>>>> 
>>>> 
>>>> 
>>>> On November 18, 2018 at 1:10:31 AM, Gershom B (gersh...@gmail.com)
>>>> wrote:>>>>> 
>>>>> 
>>>>> This is interesting, but I’m thoroughly confused. Over 2500 people
>>>>> said they took last year’s survey, but it only had roughly 1,300
>>>>> respondants?>>>>> 
>>>>> 
>>>>> On Sat, Nov 17, 2018 at 9:56 PM Taylor Fausak 
>>>>> wrote:
>>>>>> Hello! It took a little longer than I expected, but I am nearly
>>>>>> ready to announce the 2018 state of Haskell survey results. Some
>>>>>> community members have expressed interest in seeing the
>>>>>> announcement post before it's published. If you are one of those
>>>>>> people, you can see the results here:
>>>>>> https://github.com/tfausak/tfausak.github.io/blob/7e4937e284a3068add9e9af6b585c8d0215ff360/_posts/2018-11-16-2018-state-of-haskell-survey-results.markdown
>>>>>>
>>>>>>  If you would like to suggest changes to the announcement post,
>>>>>>  please respond to this email, send me an email directly, or
>>>>>>  reply to this pull request on GitHub:
>>>>>>  https://github.com/tfausak/tfausak.github.io/pull/148
>>>>>>
>>>>>>  I plan on publishing the results tomorrow. Once the results are
>>>>>>  published, the post is by no means set in stone. I will happily
>>>>>>  accept suggestions from anyone at any time.
>>>>>>
>>>>>>  Thank you!
>>>>>>  ___
>>>>>>  Haskell-community mailing list Haskell-community@haskell.org
>>>>>>  http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community>>>>> 
>>>> ___
>>>> Haskell-community mailing list
>>>> Haskell-community@haskell.org
>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community>>> 
>>>> ___
>>> Haskell-community mailing list
>>> Haskell-community@haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>> _
>> Haskell-community mailing list
>> Haskell-community@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
> 

___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey results

[Taylor Fausak]

Thanks for finding those anomalies, Gershom! I'm disappointed that
someone submitted bogus responses, apparently to tip the scales of Cabal
versus Stack. I intend to identify those responses and exclude them from
the results. The work you've done so far will help a great deal in
finding them.
You said that there are about 1,200 responses with demographic
information. That makes sense considering the number of submissions I
got last year. Also, there are 1,185 responses that included an answer
to at least one of the free-response questions. So perhaps whoever wrote
the script didn't bother to put an answer for those types of questions.
Unfortunately I do not have precise submission times or IP address
information about submissions. Beyond what's in the CSV, the only other
thing I have is (some) email addresses.
Fortunately I wrote a script to output all the charts and tables from
the survey responses. Once I've identified the problematic responses, I
should be able to update the script to ignore them and regenerate all
the output.

On Sun, Nov 18, 2018, at 3:40 AM, Chris Smith wrote:
> Sadly, it looks like a Cabal/Stack thing.  Of the responses with a
> country provided, 618 of 1226 claim to use Cabal, and 948 of 1226
> claim to use Stack. Of the responses with no country, only 35 of 3868
> claim to use Cabal, while 3781 of the 3868 claim to use Stack.
> Assuming independence, you'd expect that last number to be about 50,
> meaning there are probably around 3700 fake responses generated just
> to answer "Stack".> 
> To partially answer Simon's question, the flood of no-demographics
> responses started on November 2, around the 750-response point, and
> continued unabated through the close of the survey.  And, indeed,
> looking at just the first 750 responses gives similar distributions
> to what we get by ignoring the no-demographic responses.  For
> example, of the first 750 responses, 359 claim to use Cabal, and 568
> claim to use Stack.> 
> On Sun, Nov 18, 2018 at 2:31 AM Simon Marlow
>  wrote:>> Good spot Gershom. Maybe it would be revealing 
> to look at the times
>> that responses were received for the no-demographics group?>> 
>> On Sun, 18 Nov 2018, 07:17 Gershom B >> I also noticed a number of other bizarre statistical anomolies when
>>> looking at the full results. I know this is a bit much to ask — but
>>> if you could rerun the statistics filtering out people that did not
>>> give demographic information (i.e. country of origin or education,
>>> etc) I think the results will change drastically. By all statistical
>>> logic, this should _not_ be the case, and points to a serious
>>> problem.>>> 
>>> In particular, this drops the results by a huge amount — only 1,200
>>> or so remain. However, the remaining results tend to make a lot more
>>> sense. For example — of the “no demographics” group, there are 713
>>> users who claim to develop with notepad++ but all of these say they
>>> develop on mac and linux, and none on windows — which is impossible,
>>> as notepad++ is a windows program. Further if you drop the “no
>>> demographics” group, then you find that almost everyone uses at
>>> least ghc 8.0.2, while in the “no demographics” group,  a stunning
>>> number of people claim to be on 7.8.3. Even more bizarrely, people
>>> claim to be using the 7.8 series while only having used Haskell for
>>> less than one year. And people claim to have used haskell for “one
>>> week to one month” and also to be advanced and expert users!>>> 
>>> The differences continue and defy all probability. Of the “no
>>> demographics” group, almost everyone dislikes the new release
>>> schedule. Of the “demographics” group there are answers that like
>>> it, were not aware of it, or are indifferent, but almost nobody
>>> dislikes it. There is naturally a difference in proportions of
>>> cabal/stack and hackage/stackage responses as well.>>> 
>>> There are a lot of other things I could point to as well. But,
>>> bluntly put, I think that some disaffected party or parties wrote a
>>> crude script and submitted over 3,000 fake responses. Luckily for
>>> us, they were not very smart, and made some obvious errors, so in
>>> this case we can weed out the bad responses (although, sadly, losing
>>> at least a few real ones as well).>>> 
>>> However, assuming  this party isn’t entirely stupid, it doesn’t bode
>>> well for future surveys as they may get at least slightly less dumb
>>> in the future if they decide to keep it up :-/>>> 
>>> —Gershom
>>> 
>>> 
>>

Re: [Haskell-community] 2018 state of Haskell survey results

[Chris Smith]

Sadly, it looks like a Cabal/Stack thing.  Of the responses with a country
provided, 618 of 1226 claim to use Cabal, and 948 of 1226 claim to use
Stack. Of the responses with no country, only 35 of 3868 claim to use
Cabal, while 3781 of the 3868 claim to use Stack.  Assuming independence,
you'd expect that last number to be about 50, meaning there are probably
around 3700 fake responses generated just to answer "Stack".

To partially answer Simon's question, the flood of no-demographics
responses started on November 2, around the 750-response point, and
continued unabated through the close of the survey.  And, indeed, looking
at just the first 750 responses gives similar distributions to what we get
by ignoring the no-demographic responses.  For example, of the first 750
responses, 359 claim to use Cabal, and 568 claim to use Stack.

On Sun, Nov 18, 2018 at 2:31 AM Simon Marlow  wrote:

> Good spot Gershom. Maybe it would be revealing to look at the times that
> responses were received for the no-demographics group?
>
> On Sun, 18 Nov 2018, 07:17 Gershom B 
>> I also noticed a number of other bizarre statistical anomolies when
>> looking at the full results. I know this is a bit much to ask — but if you
>> could rerun the statistics filtering out people that did not give
>> demographic information (i.e. country of origin or education, etc) I think
>> the results will change drastically. By all statistical logic, this should
>> _not_ be the case, and points to a serious problem.
>>
>> In particular, this drops the results by a huge amount — only 1,200 or so
>> remain. However, the remaining results tend to make a lot more sense. For
>> example — of the “no demographics” group, there are 713 users who claim to
>> develop with notepad++ but all of these say they develop on mac and linux,
>> and none on windows — which is impossible, as notepad++ is a windows
>> program. Further if you drop the “no demographics” group, then you find
>> that almost everyone uses at least ghc 8.0.2, while in the “no
>> demographics” group,  a stunning number of people claim to be on 7.8.3.
>> Even more bizarrely, people claim to be using the 7.8 series while only
>> having used Haskell for less than one year. And people claim to have used
>> haskell for “one week to one month” and also to be advanced and expert
>> users!
>>
>> The differences continue and defy all probability. Of the “no
>> demographics” group, almost everyone dislikes the new release schedule. Of
>> the “demographics” group there are answers that like it, were not aware of
>> it, or are indifferent, but almost nobody dislikes it. There is naturally a
>> difference in proportions of cabal/stack and hackage/stackage responses as
>> well.
>>
>> There are a lot of other things I could point to as well. But, bluntly
>> put, I think that some disaffected party or parties wrote a crude script
>> and submitted over 3,000 fake responses. Luckily for us, they were not very
>> smart, and made some obvious errors, so in this case we can weed out the
>> bad responses (although, sadly, losing at least a few real ones as well).
>>
>> However, assuming  this party isn’t entirely stupid, it doesn’t bode well
>> for future surveys as they may get at least slightly less dumb in the
>> future if they decide to keep it up :-/
>>
>> —Gershom
>>
>>
>> On November 18, 2018 at 1:10:31 AM, Gershom B (gersh...@gmail.com) wrote:
>>
>> This is interesting, but I’m thoroughly confused. Over 2500 people said
>> they took last year’s survey, but it only had roughly 1,300 respondants?
>>
>>
>> On Sat, Nov 17, 2018 at 9:56 PM Taylor Fausak  wrote:
>>
>>> Hello! It took a little longer than I expected, but I am nearly ready to
>>> announce the 2018 state of Haskell survey results. Some community members
>>> have expressed interest in seeing the announcement post before it's
>>> published. If you are one of those people, you can see the results here:
>>> https://github.com/tfausak/tfausak.github.io/blob/7e4937e284a3068add9e9af6b585c8d0215ff360/_posts/2018-11-16-2018-state-of-haskell-survey-results.markdown
>>>
>>> If you would like to suggest changes to the announcement post, please
>>> respond to this email, send me an email directly, or reply to this pull
>>> request on GitHub: https://github.com/tfausak/tfausak.github.io/pull/148
>>>
>>> I plan on publishing the results tomorrow. Once the results are
>>> published, the post is by no means set in stone. I will happily accept
>>> suggestions from anyone at any time.
>>>
>>> Thank you!
>>> __

Re: [Haskell-community] 2018 state of Haskell survey results

[Gershom B]

I also noticed a number of other bizarre statistical anomolies when looking
at the full results. I know this is a bit much to ask — but if you could
rerun the statistics filtering out people that did not give demographic
information (i.e. country of origin or education, etc) I think the results
will change drastically. By all statistical logic, this should _not_ be the
case, and points to a serious problem.

In particular, this drops the results by a huge amount — only 1,200 or so
remain. However, the remaining results tend to make a lot more sense. For
example — of the “no demographics” group, there are 713 users who claim to
develop with notepad++ but all of these say they develop on mac and linux,
and none on windows — which is impossible, as notepad++ is a windows
program. Further if you drop the “no demographics” group, then you find
that almost everyone uses at least ghc 8.0.2, while in the “no
demographics” group,  a stunning number of people claim to be on 7.8.3.
Even more bizarrely, people claim to be using the 7.8 series while only
having used Haskell for less than one year. And people claim to have used
haskell for “one week to one month” and also to be advanced and expert
users!

The differences continue and defy all probability. Of the “no demographics”
group, almost everyone dislikes the new release schedule. Of the
“demographics” group there are answers that like it, were not aware of it,
or are indifferent, but almost nobody dislikes it. There is naturally a
difference in proportions of cabal/stack and hackage/stackage responses as
well.

There are a lot of other things I could point to as well. But, bluntly put,
I think that some disaffected party or parties wrote a crude script and
submitted over 3,000 fake responses. Luckily for us, they were not very
smart, and made some obvious errors, so in this case we can weed out the
bad responses (although, sadly, losing at least a few real ones as well).

However, assuming  this party isn’t entirely stupid, it doesn’t bode well
for future surveys as they may get at least slightly less dumb in the
future if they decide to keep it up :-/

—Gershom


On November 18, 2018 at 1:10:31 AM, Gershom B (gersh...@gmail.com) wrote:

This is interesting, but I’m thoroughly confused. Over 2500 people said
they took last year’s survey, but it only had roughly 1,300 respondants?


On Sat, Nov 17, 2018 at 9:56 PM Taylor Fausak  wrote:

> Hello! It took a little longer than I expected, but I am nearly ready to
> announce the 2018 state of Haskell survey results. Some community members
> have expressed interest in seeing the announcement post before it's
> published. If you are one of those people, you can see the results here:
> https://github.com/tfausak/tfausak.github.io/blob/7e4937e284a3068add9e9af6b585c8d0215ff360/_posts/2018-11-16-2018-state-of-haskell-survey-results.markdown
>
> If you would like to suggest changes to the announcement post, please
> respond to this email, send me an email directly, or reply to this pull
> request on GitHub: https://github.com/tfausak/tfausak.github.io/pull/148
>
> I plan on publishing the results tomorrow. Once the results are published,
> the post is by no means set in stone. I will happily accept suggestions
> from anyone at any time.
>
> Thank you!
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

[Haskell-community] 2018 state of Haskell survey results

[Taylor Fausak]

Hello! It took a little longer than I expected, but I am nearly ready to 
announce the 2018 state of Haskell survey results. Some community members have 
expressed interest in seeing the announcement post before it's published. If 
you are one of those people, you can see the results here: 
https://github.com/tfausak/tfausak.github.io/blob/7e4937e284a3068add9e9af6b585c8d0215ff360/_posts/2018-11-16-2018-state-of-haskell-survey-results.markdown

If you would like to suggest changes to the announcement post, please respond 
to this email, send me an email directly, or reply to this pull request on 
GitHub: https://github.com/tfausak/tfausak.github.io/pull/148

I plan on publishing the results tomorrow. Once the results are published, the 
post is by no means set in stone. I will happily accept suggestions from anyone 
at any time. 

Thank you!
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey

[Taylor Fausak]

I received confirmation from Airtable that they do not support arbitrary markup 
in forms. So I put in separator questions between each of section. 

At this point the survey is ready to publish. I recognize that there are many 
more questions that could be asked, but they’ll have to wait until next year. 
Thank you all for your feedback! I look forward to sharing the results with you 
in a couple weeks. In the meantime, if there’s anything I can do for you, 
please let me know. 

> On Oct 29, 2018, at 7:38 PM, Taylor Fausak  wrote:
> 
> Thanks for the feedback! 
> 
> - I would like to separate the survey into sections, but Airtable does not 
> provide that functionality. I have sent a message to their support asking if 
> I’m just missing it. Worst case scenario I can put some bogus questions in to 
> act as dividers. I’ve put an example of such a divider question at the top of 
> the survey. 
> 
> - The “Add an option” questions allow you to select multiple answers rather 
> than choosing a single one. I’ve updated the questions to make that clearer 
> by adding this help text: “Select all that apply."
> 
> - I’ve added a followup question to the one about GHC’s new release schedule: 
> "Why do you feel the way that you do about the new GHC release schedule?” I’m 
> open to better wording there. 
> 
> - I have added follow up questions of the form “What would you change about 
> X?” where X is the language, compiler, build tool, or package repository. 
> Hopefully that will provide meaningful guidance about how to improve those 
> things without overwhelming the user with questions. 
> 
> - For information about using Haskell at work, I think that is covered by 
> existing questions. Last year’s survey asked if people used Haskell at work, 
> and this year’s added some followup questions to that. Company size is 
> covered by the demographic questions at the end. The only missing piece is 
> asking about the size of the team of Haskell programmers. Is that worth 
> asking about separately? 
> 
> - I have removed “Official” from the title of the survey.
> 
> - I changed the Haskell Prime question to ask about importance rather than 
> interest: “How important do you feel it would be to have a new version of the 
> Haskell standard?” It uses the answer scale from here: 
> https://help.surveymonkey.com/articles/en_US/kb/Likert-Scales 
> 
> 
> - I split academic and commercial conferences in the question about 
> interacting with the Haskell community. 
> 
> - For the question about which type of Haskell software is developed at the 
> respondents company, would it suffice to ask if the software is used 
> internally by other employees and/or externally by customers? Another 
> question already covers the type of software (web, CLI, GUI, …).
> 
> - I like the idea of drilling down into performance bottlenecks. How do you 
> feel about phrasing it like this: “Which performance bottlenecks does your 
> Haskell software typically hit?” With answer choices: CPU, RAM, disk, 
> network, other, none.(I’m not sure what you mean by “bound by serialization.” 
> Can you expand on that?) 
> 
> - I think the way that the software runs is covered by another question about 
> the type of software (web, CLI, GUI, …). Is it worth it to have a separate 
> question? 
> 
> I hope that addresses all the feedback so far. If not, please let me know! 
> Thanks again! 
> 
>> On Oct 29, 2018, at 1:14 PM, Gershom B > > wrote:
>> 
>> HI Taylor.
>> 
>> A few thoughts. First, even with joint sponsorship, I don't think
>> saying "Official" in the name of the survey is a good idea. Everything
>> is "official" from whatever group supports it, but that seems besides
>> the point. I think that the intended meaning here is a bit slippery
>> since it can be interpreted as "approved by some body" but is often
>> used to mean "authoritative" and as we've discussed, you can't really
>> be authoritative with things like this, just "better".  Ok, that said,
>> on to some other points:
>> 
>> "Are you interested in a new version of the Haskell standard?"
>> 
>> Interested is a very vague thing to ask. I'd want something more
>> specific like "how important do you feel it would be to have a new
>> version..."
>> 
>> On "Where do you interact with the Haskell community?" I think that we
>> should distinguish between "conferences (academic)" and "conferences
>> (commercial)" because ICFP and HaskellX, for example, are very
>> different sorts of things.
>> 
>> I'd also like a question, as I mentioned earlier, like "What sort of
>> Haskell software is developed at your company" with options for
>> "in-house" "binaries deployed to customers" and "webapps used by
>> customers" among maybe other options. Also perhaps "is the software
>> you work on A) bound by memory B) bound by processor utilization C)
>> bound by wire/disk speed D) bound by serialization E) not 

Re: [Haskell-community] 2018 state of Haskell survey

[Francesco Ariis]

On Sun, Oct 28, 2018 at 04:33:44PM -0400, Taylor Fausak wrote:
> Thanks for the suggestions!
> 
> - I plan on releasing the results under the ODbL 1.0 license. I'll be
> sure to update the survey to say that. You can read more about the
> license here: https://opendatacommons.org/licenses/odbl/summary/

Excellent choice!
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey

[Taylor Fausak]

Thanks for the suggestions!

- I plan on releasing the results under the ODbL 1.0 license. I'll be sure to 
update the survey to say that. You can read more about the license here: 
https://opendatacommons.org/licenses/odbl/summary/

- The last question asks for an email address. I wouldn't call it mission 
critical, but it will allow me to follow up on confusing or problematic 
responses. Last year I asked for email addresses and 75% of responses included 
them. This year the question remains optional, and it also explains how it will 
be used. I would prefer not to remove it. 

On Sun, Oct 28, 2018, at 4:06 PM, Francesco Ariis wrote:
> Hello Taylor,
> 
> On Sun, Oct 28, 2018 at 02:42:16PM -0400, Taylor Fausak wrote:
> > Please > take a look at the survey to make sure that you're happy
> > with it! Let me know if there are any questions that you would like
> > to be added, removed, or changed. You can view the survey here:
> > https://airtable.com/shr8G4RBPD9T6tnDf
> > You can deliver feedback to me either in this thread or on GitHub:
> > https://github.com/haskellweekly/haskellweekly.github.io/issues/206
> 
> Suggestions:
> - state under which specific one of the "permissive license"s the
>   results will be available;
> - if it not mission critical, axe the last question.
> 
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey

[Francesco Ariis]

Hello Taylor,

On Sun, Oct 28, 2018 at 02:42:16PM -0400, Taylor Fausak wrote:
> Please > take a look at the survey to make sure that you're happy
> with it! Let me know if there are any questions that you would like
> to be added, removed, or changed. You can view the survey here:
> https://airtable.com/shr8G4RBPD9T6tnDf
> You can deliver feedback to me either in this thread or on GitHub:
> https://github.com/haskellweekly/haskellweekly.github.io/issues/206

Suggestions:
- state under which specific one of the "permissive license"s the
  results will be available;
- if it not mission critical, axe the last question.

___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey

[Taylor Fausak]

We're creeping closer to the release date. I spent some time this
weekend tweaking the survey in response to suggestions from this thread
and also from GitHub. I don't plan on making any large changes to the
survey between now and Thursday, except in response to feedback. Please
take a look at the survey to make sure that you're happy with it! Let me
know if there are any questions that you would like to be added,
removed, or changed. You can view the survey here:
https://airtable.com/shr8G4RBPD9T6tnDf
You can deliver feedback to me either in this thread or on GitHub:
https://github.com/haskellweekly/haskellweekly.github.io/issues/206

On Fri, Oct 26, 2018, at 8:57 AM, Taylor Fausak wrote:
>  Hooray! That’s wonderful news! Thank you, Haskell.org committee, for
> supporting the survey.> 
> I plan on releasing this year’s results in the same fashion as
> last year.> 
>> On Oct 26, 2018, at 5:31 AM, Jasper Van der Jeugt
>>  wrote:>> 
>> Hi Taylor,
>> 
>> Yes, we're happy to support it from Haskell.org.
>> 
>> One additional ask from our side would be that the raw results are
>> published as well, but I saw in the issue you're already planning on>> doing 
>> that.
>> 
>> Cheers
>> Jasper
>> 
>> On Thu, Oct 25, 2018 at 11:18:30AM -0400, Taylor Fausak wrote:
>>> We’re one week out from the release of the survey. I plan on
>>> spending this weekend putting the finishing touches on it. Can I
>>> plan on announcing it as the official state of Haskell 2018 survey,
>>> supported by both Haskell Weekly and Haskell.org
>>> ?>>> 
 On Oct 17, 2018, at 7:00 PM, Jasper Van der Jeugt 
 wrote: 
 Hi Taylor,
 
 Just a small comment: I would like to keep the survey open a bit
 longer -- I would suggest two weeks.  This gives us a bit more time
 to push it out twice to as many channels as possible (once at the
 start and a reminder after a week or so).  My intuition is that
 we'll be able to gather significantly more responses that way. 
 Thanks again for organizing this!
 
 Cheers
 Jasper
 
 On Thu, 18 Oct 2018 at 00:55, Taylor Fausak >>> > wrote: Thank you all for the wonderful 
 feedback so far! I greatly
 appreciate all of it. 
 I didn’t mean to be exclusionary with my language before, and I
 thank y’all for correcting me there. “We’re doing this together for
 the benefit of all” is an excellent way to say what I’m shooting
 for here. 
 My goal for the survey is to be useful to many different groups of
 people: the GHC team, library authors, application developers,
 repository maintainers, prospective employees, hiring managers,
 community organizers, and no doubt many more groups that I’m not
 thinking of right now. I want to avoid results that are neat but
 not useful. I also want to avoid results that simply throw fuel
 onto common flame wars. 
 Last year I announced the survey results and provided some
 commentary. I suspect I’ll do something similar this year, although
 reading your comments here makes me want to do less analyzing in
 favor of simply publishing. I am not particularly adept at
 analyzing survey results and am bound to make some rookie mistakes.
 In fact, one of the reasons that I published the results last year
 was so that someone who actually knew what they were doing could
 slice and dice the data. 
 As far as scheduling is concerned, I plan to keep the survey open
 for a week, from November 1st to 7th. Publishing the results should
 happen relatively quickly after that. I slowed myself down last
 year by rendering a bunch of graphs, and even so I published on
 November 15th. 
 It sounds like the Haskell.org committee is broadly in favor of
 backing the upcoming Haskell Weekly survey. Is that correct? In
 either case, what are the next steps? 
> On Oct 16, 2018, at 5:10 PM, Boespflug, Mathieu  > wrote:> 
> Since I was pinged up-thread, might as well chime in. If only
> to say> "I agree": selection bias is what it is. Taylor's efforts to
> come to> this committee are laudable. And really could help mitigate 
> some
> issues we've seen with other surveys. Selection bias isn't
> something> worth agonizing over, provided we're careful to say in the
> analysis of> the results: "We found that X% of the respondents of this
> survey use> Y", not "X% of Haskell devs use Y".
> 
> On Tue, 16 Oct 2018 at 21:02, Simon Peyton Jones via Haskell-
> community> mailto:haskell-
> commun...@haskell.org>> wrote:>> 
>> | Hi Taylor. I like the way you pose things here: "I don't expect
>> | that>> | to remove selection bias, but it will let me (us, really) 
>> say:
>> | We're>> | doing this together for the benefit of all sides". I 

Re: [Haskell-community] 2018 state of Haskell survey

[Taylor Fausak]

 Hooray! That’s wonderful news! Thank you, Haskell.org  
committee, for supporting the survey. 

I plan on releasing this year’s results in the same fashion as last year. 

> On Oct 26, 2018, at 5:31 AM, Jasper Van der Jeugt  wrote:
> 
> Hi Taylor,
> 
> Yes, we're happy to support it from Haskell.org.
> 
> One additional ask from our side would be that the raw results are
> published as well, but I saw in the issue you're already planning on
> doing that.
> 
> Cheers
> Jasper
> 
> On Thu, Oct 25, 2018 at 11:18:30AM -0400, Taylor Fausak wrote:
>> We’re one week out from the release of the survey. I plan on spending this 
>> weekend putting the finishing touches on it. Can I plan on announcing it as 
>> the official state of Haskell 2018 survey, supported by both Haskell Weekly 
>> and Haskell.org ? 
>> 
>>> On Oct 17, 2018, at 7:00 PM, Jasper Van der Jeugt  wrote:
>>> 
>>> Hi Taylor,
>>> 
>>> Just a small comment: I would like to keep the survey open a bit longer -- 
>>> I would suggest two weeks.  This gives us a bit more time to push it out 
>>> twice to as many channels as possible (once at the start and a reminder 
>>> after a week or so).  My intuition is that we'll be able to gather 
>>> significantly more responses that way.
>>> 
>>> Thanks again for organizing this!
>>> 
>>> Cheers
>>> Jasper
>>> 
>>> On Thu, 18 Oct 2018 at 00:55, Taylor Fausak >> > wrote:
>>> Thank you all for the wonderful feedback so far! I greatly appreciate all 
>>> of it. 
>>> 
>>> I didn’t mean to be exclusionary with my language before, and I thank y’all 
>>> for correcting me there. “We’re doing this together for the benefit of all” 
>>> is an excellent way to say what I’m shooting for here. 
>>> 
>>> My goal for the survey is to be useful to many different groups of people: 
>>> the GHC team, library authors, application developers, repository 
>>> maintainers, prospective employees, hiring managers, community organizers, 
>>> and no doubt many more groups that I’m not thinking of right now. I want to 
>>> avoid results that are neat but not useful. I also want to avoid results 
>>> that simply throw fuel onto common flame wars.
>>> 
>>> Last year I announced the survey results and provided some commentary. I 
>>> suspect I’ll do something similar this year, although reading your comments 
>>> here makes me want to do less analyzing in favor of simply publishing. I am 
>>> not particularly adept at analyzing survey results and am bound to make 
>>> some rookie mistakes. In fact, one of the reasons that I published the 
>>> results last year was so that someone who actually knew what they were 
>>> doing could slice and dice the data. 
>>> 
>>> As far as scheduling is concerned, I plan to keep the survey open for a 
>>> week, from November 1st to 7th. Publishing the results should happen 
>>> relatively quickly after that. I slowed myself down last year by rendering 
>>> a bunch of graphs, and even so I published on November 15th. 
>>> 
>>> It sounds like the Haskell.org committee is broadly in favor of backing the 
>>> upcoming Haskell Weekly survey. Is that correct? In either case, what are 
>>> the next steps? 
>>> 
 On Oct 16, 2018, at 5:10 PM, Boespflug, Mathieu >>> > wrote:
 
 Since I was pinged up-thread, might as well chime in. If only to say
 "I agree": selection bias is what it is. Taylor's efforts to come to
 this committee are laudable. And really could help mitigate some
 issues we've seen with other surveys. Selection bias isn't something
 worth agonizing over, provided we're careful to say in the analysis of
 the results: "We found that X% of the respondents of this survey use
 Y", not "X% of Haskell devs use Y".
 
 On Tue, 16 Oct 2018 at 21:02, Simon Peyton Jones via Haskell-community
 mailto:haskell-community@haskell.org>> 
 wrote:
> 
> | Hi Taylor. I like the way you pose things here: "I don't expect that
> | to remove selection bias, but it will let me (us, really) say: We're
> | doing this together for the benefit of all sides". I think that's a
> | better place to start from.
> 
> I like this too -- and like Gershom, I'd delete "sides".  We aspire
> to work together, not on different sides.
> 
> | earlier I've been thinking about a bit, where you wrote: "My goal is
> | for this survey to be *the* authoritative Haskell survey and for the
> | community to broadly accept it's results."
> 
> This sounds a bit too exclusive to me, and implicitly critical of other
> work.  Better to stick to the positives: you simply want the
> opinions of a broad constituency on a broad range of questions.
> 
> | Anyway, this is all a long-winded way of suggesting that it might be
> | good if the purpose of the survey was explicitly set out as trying to
> | inform developers of haskell libraries and tools 

Re: [Haskell-community] 2018 state of Haskell survey

[Jasper Van der Jeugt]

Hi Taylor,

Yes, we're happy to support it from Haskell.org.

One additional ask from our side would be that the raw results are
published as well, but I saw in the issue you're already planning on
doing that.

Cheers
Jasper

On Thu, Oct 25, 2018 at 11:18:30AM -0400, Taylor Fausak wrote:
> We’re one week out from the release of the survey. I plan on spending this 
> weekend putting the finishing touches on it. Can I plan on announcing it as 
> the official state of Haskell 2018 survey, supported by both Haskell Weekly 
> and Haskell.org ? 
> 
> > On Oct 17, 2018, at 7:00 PM, Jasper Van der Jeugt  wrote:
> > 
> > Hi Taylor,
> > 
> > Just a small comment: I would like to keep the survey open a bit longer -- 
> > I would suggest two weeks.  This gives us a bit more time to push it out 
> > twice to as many channels as possible (once at the start and a reminder 
> > after a week or so).  My intuition is that we'll be able to gather 
> > significantly more responses that way.
> > 
> > Thanks again for organizing this!
> > 
> > Cheers
> > Jasper
> > 
> > On Thu, 18 Oct 2018 at 00:55, Taylor Fausak  > > wrote:
> > Thank you all for the wonderful feedback so far! I greatly appreciate all 
> > of it. 
> > 
> > I didn’t mean to be exclusionary with my language before, and I thank y’all 
> > for correcting me there. “We’re doing this together for the benefit of all” 
> > is an excellent way to say what I’m shooting for here. 
> > 
> > My goal for the survey is to be useful to many different groups of people: 
> > the GHC team, library authors, application developers, repository 
> > maintainers, prospective employees, hiring managers, community organizers, 
> > and no doubt many more groups that I’m not thinking of right now. I want to 
> > avoid results that are neat but not useful. I also want to avoid results 
> > that simply throw fuel onto common flame wars.
> > 
> > Last year I announced the survey results and provided some commentary. I 
> > suspect I’ll do something similar this year, although reading your comments 
> > here makes me want to do less analyzing in favor of simply publishing. I am 
> > not particularly adept at analyzing survey results and am bound to make 
> > some rookie mistakes. In fact, one of the reasons that I published the 
> > results last year was so that someone who actually knew what they were 
> > doing could slice and dice the data. 
> > 
> > As far as scheduling is concerned, I plan to keep the survey open for a 
> > week, from November 1st to 7th. Publishing the results should happen 
> > relatively quickly after that. I slowed myself down last year by rendering 
> > a bunch of graphs, and even so I published on November 15th. 
> > 
> > It sounds like the Haskell.org committee is broadly in favor of backing the 
> > upcoming Haskell Weekly survey. Is that correct? In either case, what are 
> > the next steps? 
> > 
> > > On Oct 16, 2018, at 5:10 PM, Boespflug, Mathieu  > > > wrote:
> > > 
> > > Since I was pinged up-thread, might as well chime in. If only to say
> > > "I agree": selection bias is what it is. Taylor's efforts to come to
> > > this committee are laudable. And really could help mitigate some
> > > issues we've seen with other surveys. Selection bias isn't something
> > > worth agonizing over, provided we're careful to say in the analysis of
> > > the results: "We found that X% of the respondents of this survey use
> > > Y", not "X% of Haskell devs use Y".
> > > 
> > > On Tue, 16 Oct 2018 at 21:02, Simon Peyton Jones via Haskell-community
> > > mailto:haskell-community@haskell.org>> 
> > > wrote:
> > >> 
> > >> | Hi Taylor. I like the way you pose things here: "I don't expect that
> > >> | to remove selection bias, but it will let me (us, really) say: We're
> > >> | doing this together for the benefit of all sides". I think that's a
> > >> | better place to start from.
> > >> 
> > >> I like this too -- and like Gershom, I'd delete "sides".  We aspire
> > >> to work together, not on different sides.
> > >> 
> > >> | earlier I've been thinking about a bit, where you wrote: "My goal is
> > >> | for this survey to be *the* authoritative Haskell survey and for the
> > >> | community to broadly accept it's results."
> > >> 
> > >> This sounds a bit too exclusive to me, and implicitly critical of other
> > >> work.  Better to stick to the positives: you simply want the
> > >> opinions of a broad constituency on a broad range of questions.
> > >> 
> > >> | Anyway, this is all a long-winded way of suggesting that it might be
> > >> | good if the purpose of the survey was explicitly set out as trying to
> > >> | inform developers of haskell libraries and tools (and educational
> > >> | materials) regarding the systems their potential users work on and
> > >> | develop, and their habits and practices in doing so, and where they
> > >> | encounter difficulty. That is explicitly as a way of learning rather
> > 

Re: [Haskell-community] 2018 state of Haskell survey

[Taylor Fausak]

We’re one week out from the release of the survey. I plan on spending this 
weekend putting the finishing touches on it. Can I plan on announcing it as the 
official state of Haskell 2018 survey, supported by both Haskell Weekly and 
Haskell.org ? 

> On Oct 17, 2018, at 7:00 PM, Jasper Van der Jeugt  wrote:
> 
> Hi Taylor,
> 
> Just a small comment: I would like to keep the survey open a bit longer -- I 
> would suggest two weeks.  This gives us a bit more time to push it out twice 
> to as many channels as possible (once at the start and a reminder after a 
> week or so).  My intuition is that we'll be able to gather significantly more 
> responses that way.
> 
> Thanks again for organizing this!
> 
> Cheers
> Jasper
> 
> On Thu, 18 Oct 2018 at 00:55, Taylor Fausak  > wrote:
> Thank you all for the wonderful feedback so far! I greatly appreciate all of 
> it. 
> 
> I didn’t mean to be exclusionary with my language before, and I thank y’all 
> for correcting me there. “We’re doing this together for the benefit of all” 
> is an excellent way to say what I’m shooting for here. 
> 
> My goal for the survey is to be useful to many different groups of people: 
> the GHC team, library authors, application developers, repository 
> maintainers, prospective employees, hiring managers, community organizers, 
> and no doubt many more groups that I’m not thinking of right now. I want to 
> avoid results that are neat but not useful. I also want to avoid results that 
> simply throw fuel onto common flame wars.
> 
> Last year I announced the survey results and provided some commentary. I 
> suspect I’ll do something similar this year, although reading your comments 
> here makes me want to do less analyzing in favor of simply publishing. I am 
> not particularly adept at analyzing survey results and am bound to make some 
> rookie mistakes. In fact, one of the reasons that I published the results 
> last year was so that someone who actually knew what they were doing could 
> slice and dice the data. 
> 
> As far as scheduling is concerned, I plan to keep the survey open for a week, 
> from November 1st to 7th. Publishing the results should happen relatively 
> quickly after that. I slowed myself down last year by rendering a bunch of 
> graphs, and even so I published on November 15th. 
> 
> It sounds like the Haskell.org committee is broadly in favor of backing the 
> upcoming Haskell Weekly survey. Is that correct? In either case, what are the 
> next steps? 
> 
> > On Oct 16, 2018, at 5:10 PM, Boespflug, Mathieu  > > wrote:
> > 
> > Since I was pinged up-thread, might as well chime in. If only to say
> > "I agree": selection bias is what it is. Taylor's efforts to come to
> > this committee are laudable. And really could help mitigate some
> > issues we've seen with other surveys. Selection bias isn't something
> > worth agonizing over, provided we're careful to say in the analysis of
> > the results: "We found that X% of the respondents of this survey use
> > Y", not "X% of Haskell devs use Y".
> > 
> > On Tue, 16 Oct 2018 at 21:02, Simon Peyton Jones via Haskell-community
> > mailto:haskell-community@haskell.org>> 
> > wrote:
> >> 
> >> | Hi Taylor. I like the way you pose things here: "I don't expect that
> >> | to remove selection bias, but it will let me (us, really) say: We're
> >> | doing this together for the benefit of all sides". I think that's a
> >> | better place to start from.
> >> 
> >> I like this too -- and like Gershom, I'd delete "sides".  We aspire
> >> to work together, not on different sides.
> >> 
> >> | earlier I've been thinking about a bit, where you wrote: "My goal is
> >> | for this survey to be *the* authoritative Haskell survey and for the
> >> | community to broadly accept it's results."
> >> 
> >> This sounds a bit too exclusive to me, and implicitly critical of other
> >> work.  Better to stick to the positives: you simply want the
> >> opinions of a broad constituency on a broad range of questions.
> >> 
> >> | Anyway, this is all a long-winded way of suggesting that it might be
> >> | good if the purpose of the survey was explicitly set out as trying to
> >> | inform developers of haskell libraries and tools (and educational
> >> | materials) regarding the systems their potential users work on and
> >> | develop, and their habits and practices in doing so, and where they
> >> | encounter difficulty. That is explicitly as a way of learning rather
> >> | than as any sort of horse-race or popularity contest.
> >> 
> >> That sounds good to me -- but again in drafting the goals I'd stick
> >> to the positives, and not speak about horse-races.
> >> 
> >> Simon
> >> ___
> >> Haskell-community mailing list
> >> Haskell-community@haskell.org 
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community 
> >> 

Re: [Haskell-community] 2018 state of Haskell survey

[Taylor Fausak]

Thanks Jasper, that’s a good idea. I’ll run this year’s survey for two weeks, 
from November 1 to November 15. 

> On Oct 17, 2018, at 7:00 PM, Jasper Van der Jeugt  wrote:
> 
> Hi Taylor,
> 
> Just a small comment: I would like to keep the survey open a bit longer -- I 
> would suggest two weeks.  This gives us a bit more time to push it out twice 
> to as many channels as possible (once at the start and a reminder after a 
> week or so).  My intuition is that we'll be able to gather significantly more 
> responses that way.
> 
> Thanks again for organizing this!
> 
> Cheers
> Jasper
> 
> On Thu, 18 Oct 2018 at 00:55, Taylor Fausak  > wrote:
> Thank you all for the wonderful feedback so far! I greatly appreciate all of 
> it. 
> 
> I didn’t mean to be exclusionary with my language before, and I thank y’all 
> for correcting me there. “We’re doing this together for the benefit of all” 
> is an excellent way to say what I’m shooting for here. 
> 
> My goal for the survey is to be useful to many different groups of people: 
> the GHC team, library authors, application developers, repository 
> maintainers, prospective employees, hiring managers, community organizers, 
> and no doubt many more groups that I’m not thinking of right now. I want to 
> avoid results that are neat but not useful. I also want to avoid results that 
> simply throw fuel onto common flame wars.
> 
> Last year I announced the survey results and provided some commentary. I 
> suspect I’ll do something similar this year, although reading your comments 
> here makes me want to do less analyzing in favor of simply publishing. I am 
> not particularly adept at analyzing survey results and am bound to make some 
> rookie mistakes. In fact, one of the reasons that I published the results 
> last year was so that someone who actually knew what they were doing could 
> slice and dice the data. 
> 
> As far as scheduling is concerned, I plan to keep the survey open for a week, 
> from November 1st to 7th. Publishing the results should happen relatively 
> quickly after that. I slowed myself down last year by rendering a bunch of 
> graphs, and even so I published on November 15th. 
> 
> It sounds like the Haskell.org committee is broadly in favor of backing the 
> upcoming Haskell Weekly survey. Is that correct? In either case, what are the 
> next steps? 
> 
> > On Oct 16, 2018, at 5:10 PM, Boespflug, Mathieu  > > wrote:
> > 
> > Since I was pinged up-thread, might as well chime in. If only to say
> > "I agree": selection bias is what it is. Taylor's efforts to come to
> > this committee are laudable. And really could help mitigate some
> > issues we've seen with other surveys. Selection bias isn't something
> > worth agonizing over, provided we're careful to say in the analysis of
> > the results: "We found that X% of the respondents of this survey use
> > Y", not "X% of Haskell devs use Y".
> > 
> > On Tue, 16 Oct 2018 at 21:02, Simon Peyton Jones via Haskell-community
> > mailto:haskell-community@haskell.org>> 
> > wrote:
> >> 
> >> | Hi Taylor. I like the way you pose things here: "I don't expect that
> >> | to remove selection bias, but it will let me (us, really) say: We're
> >> | doing this together for the benefit of all sides". I think that's a
> >> | better place to start from.
> >> 
> >> I like this too -- and like Gershom, I'd delete "sides".  We aspire
> >> to work together, not on different sides.
> >> 
> >> | earlier I've been thinking about a bit, where you wrote: "My goal is
> >> | for this survey to be *the* authoritative Haskell survey and for the
> >> | community to broadly accept it's results."
> >> 
> >> This sounds a bit too exclusive to me, and implicitly critical of other
> >> work.  Better to stick to the positives: you simply want the
> >> opinions of a broad constituency on a broad range of questions.
> >> 
> >> | Anyway, this is all a long-winded way of suggesting that it might be
> >> | good if the purpose of the survey was explicitly set out as trying to
> >> | inform developers of haskell libraries and tools (and educational
> >> | materials) regarding the systems their potential users work on and
> >> | develop, and their habits and practices in doing so, and where they
> >> | encounter difficulty. That is explicitly as a way of learning rather
> >> | than as any sort of horse-race or popularity contest.
> >> 
> >> That sounds good to me -- but again in drafting the goals I'd stick
> >> to the positives, and not speak about horse-races.
> >> 
> >> Simon
> >> ___
> >> Haskell-community mailing list
> >> Haskell-community@haskell.org 
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community 
> >> 
> 
> ___
> Haskell-community mailing list
> 

Re: [Haskell-community] 2018 state of Haskell survey

[Jasper Van der Jeugt]

Hi Taylor,

Just a small comment: I would like to keep the survey open a bit longer --
I would suggest two weeks.  This gives us a bit more time to push it out
twice to as many channels as possible (once at the start and a reminder
after a week or so).  My intuition is that we'll be able to gather
significantly more responses that way.

Thanks again for organizing this!

Cheers
Jasper

On Thu, 18 Oct 2018 at 00:55, Taylor Fausak  wrote:

> Thank you all for the wonderful feedback so far! I greatly appreciate all
> of it.
>
> I didn’t mean to be exclusionary with my language before, and I thank
> y’all for correcting me there. “We’re doing this together for the benefit
> of all” is an excellent way to say what I’m shooting for here.
>
> My goal for the survey is to be useful to many different groups of people:
> the GHC team, library authors, application developers, repository
> maintainers, prospective employees, hiring managers, community organizers,
> and no doubt many more groups that I’m not thinking of right now. I want to
> avoid results that are neat but not useful. I also want to avoid results
> that simply throw fuel onto common flame wars.
>
> Last year I announced the survey results and provided some commentary. I
> suspect I’ll do something similar this year, although reading your comments
> here makes me want to do less analyzing in favor of simply publishing. I am
> not particularly adept at analyzing survey results and am bound to make
> some rookie mistakes. In fact, one of the reasons that I published the
> results last year was so that someone who actually knew what they were
> doing could slice and dice the data.
>
> As far as scheduling is concerned, I plan to keep the survey open for a
> week, from November 1st to 7th. Publishing the results should happen
> relatively quickly after that. I slowed myself down last year by rendering
> a bunch of graphs, and even so I published on November 15th.
>
> It sounds like the Haskell.org committee is broadly in favor of backing
> the upcoming Haskell Weekly survey. Is that correct? In either case, what
> are the next steps?
>
> > On Oct 16, 2018, at 5:10 PM, Boespflug, Mathieu  wrote:
> >
> > Since I was pinged up-thread, might as well chime in. If only to say
> > "I agree": selection bias is what it is. Taylor's efforts to come to
> > this committee are laudable. And really could help mitigate some
> > issues we've seen with other surveys. Selection bias isn't something
> > worth agonizing over, provided we're careful to say in the analysis of
> > the results: "We found that X% of the respondents of this survey use
> > Y", not "X% of Haskell devs use Y".
> >
> > On Tue, 16 Oct 2018 at 21:02, Simon Peyton Jones via Haskell-community
> >  wrote:
> >>
> >> | Hi Taylor. I like the way you pose things here: "I don't expect that
> >> | to remove selection bias, but it will let me (us, really) say: We're
> >> | doing this together for the benefit of all sides". I think that's a
> >> | better place to start from.
> >>
> >> I like this too -- and like Gershom, I'd delete "sides".  We aspire
> >> to work together, not on different sides.
> >>
> >> | earlier I've been thinking about a bit, where you wrote: "My goal is
> >> | for this survey to be *the* authoritative Haskell survey and for the
> >> | community to broadly accept it's results."
> >>
> >> This sounds a bit too exclusive to me, and implicitly critical of other
> >> work.  Better to stick to the positives: you simply want the
> >> opinions of a broad constituency on a broad range of questions.
> >>
> >> | Anyway, this is all a long-winded way of suggesting that it might be
> >> | good if the purpose of the survey was explicitly set out as trying to
> >> | inform developers of haskell libraries and tools (and educational
> >> | materials) regarding the systems their potential users work on and
> >> | develop, and their habits and practices in doing so, and where they
> >> | encounter difficulty. That is explicitly as a way of learning rather
> >> | than as any sort of horse-race or popularity contest.
> >>
> >> That sounds good to me -- but again in drafting the goals I'd stick
> >> to the positives, and not speak about horse-races.
> >>
> >> Simon
> >> ___
> >> Haskell-community mailing list
> >> Haskell-community@haskell.org
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
-- 
Jasper
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey

[Taylor Fausak]

Thank you all for the wonderful feedback so far! I greatly appreciate all of 
it. 

I didn’t mean to be exclusionary with my language before, and I thank y’all for 
correcting me there. “We’re doing this together for the benefit of all” is an 
excellent way to say what I’m shooting for here. 

My goal for the survey is to be useful to many different groups of people: the 
GHC team, library authors, application developers, repository maintainers, 
prospective employees, hiring managers, community organizers, and no doubt many 
more groups that I’m not thinking of right now. I want to avoid results that 
are neat but not useful. I also want to avoid results that simply throw fuel 
onto common flame wars.

Last year I announced the survey results and provided some commentary. I 
suspect I’ll do something similar this year, although reading your comments 
here makes me want to do less analyzing in favor of simply publishing. I am not 
particularly adept at analyzing survey results and am bound to make some rookie 
mistakes. In fact, one of the reasons that I published the results last year 
was so that someone who actually knew what they were doing could slice and dice 
the data. 

As far as scheduling is concerned, I plan to keep the survey open for a week, 
from November 1st to 7th. Publishing the results should happen relatively 
quickly after that. I slowed myself down last year by rendering a bunch of 
graphs, and even so I published on November 15th. 

It sounds like the Haskell.org committee is broadly in favor of backing the 
upcoming Haskell Weekly survey. Is that correct? In either case, what are the 
next steps? 

> On Oct 16, 2018, at 5:10 PM, Boespflug, Mathieu  wrote:
> 
> Since I was pinged up-thread, might as well chime in. If only to say
> "I agree": selection bias is what it is. Taylor's efforts to come to
> this committee are laudable. And really could help mitigate some
> issues we've seen with other surveys. Selection bias isn't something
> worth agonizing over, provided we're careful to say in the analysis of
> the results: "We found that X% of the respondents of this survey use
> Y", not "X% of Haskell devs use Y".
> 
> On Tue, 16 Oct 2018 at 21:02, Simon Peyton Jones via Haskell-community
>  wrote:
>> 
>> | Hi Taylor. I like the way you pose things here: "I don't expect that
>> | to remove selection bias, but it will let me (us, really) say: We're
>> | doing this together for the benefit of all sides". I think that's a
>> | better place to start from.
>> 
>> I like this too -- and like Gershom, I'd delete "sides".  We aspire
>> to work together, not on different sides.
>> 
>> | earlier I've been thinking about a bit, where you wrote: "My goal is
>> | for this survey to be *the* authoritative Haskell survey and for the
>> | community to broadly accept it's results."
>> 
>> This sounds a bit too exclusive to me, and implicitly critical of other
>> work.  Better to stick to the positives: you simply want the
>> opinions of a broad constituency on a broad range of questions.
>> 
>> | Anyway, this is all a long-winded way of suggesting that it might be
>> | good if the purpose of the survey was explicitly set out as trying to
>> | inform developers of haskell libraries and tools (and educational
>> | materials) regarding the systems their potential users work on and
>> | develop, and their habits and practices in doing so, and where they
>> | encounter difficulty. That is explicitly as a way of learning rather
>> | than as any sort of horse-race or popularity contest.
>> 
>> That sounds good to me -- but again in drafting the goals I'd stick
>> to the positives, and not speak about horse-races.
>> 
>> Simon
>> ___
>> Haskell-community mailing list
>> Haskell-community@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey

[Boespflug, Mathieu]

Since I was pinged up-thread, might as well chime in. If only to say
"I agree": selection bias is what it is. Taylor's efforts to come to
this committee are laudable. And really could help mitigate some
issues we've seen with other surveys. Selection bias isn't something
worth agonizing over, provided we're careful to say in the analysis of
the results: "We found that X% of the respondents of this survey use
Y", not "X% of Haskell devs use Y".

On Tue, 16 Oct 2018 at 21:02, Simon Peyton Jones via Haskell-community
 wrote:
>
> | Hi Taylor. I like the way you pose things here: "I don't expect that
> | to remove selection bias, but it will let me (us, really) say: We're
> | doing this together for the benefit of all sides". I think that's a
> | better place to start from.
>
> I like this too -- and like Gershom, I'd delete "sides".  We aspire
> to work together, not on different sides.
>
> | earlier I've been thinking about a bit, where you wrote: "My goal is
> | for this survey to be *the* authoritative Haskell survey and for the
> | community to broadly accept it's results."
>
> This sounds a bit too exclusive to me, and implicitly critical of other
> work.  Better to stick to the positives: you simply want the
> opinions of a broad constituency on a broad range of questions.
>
> | Anyway, this is all a long-winded way of suggesting that it might be
> | good if the purpose of the survey was explicitly set out as trying to
> | inform developers of haskell libraries and tools (and educational
> | materials) regarding the systems their potential users work on and
> | develop, and their habits and practices in doing so, and where they
> | encounter difficulty. That is explicitly as a way of learning rather
> | than as any sort of horse-race or popularity contest.
>
> That sounds good to me -- but again in drafting the goals I'd stick
> to the positives, and not speak about horse-races.
>
> Simon
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey

[Simon Peyton Jones via Haskell-community]

| Hi Taylor. I like the way you pose things here: "I don't expect that
| to remove selection bias, but it will let me (us, really) say: We're
| doing this together for the benefit of all sides". I think that's a
| better place to start from.

I like this too -- and like Gershom, I'd delete "sides".  We aspire
to work together, not on different sides.

| earlier I've been thinking about a bit, where you wrote: "My goal is
| for this survey to be *the* authoritative Haskell survey and for the
| community to broadly accept it's results." 

This sounds a bit too exclusive to me, and implicitly critical of other
work.  Better to stick to the positives: you simply want the
opinions of a broad constituency on a broad range of questions.

| Anyway, this is all a long-winded way of suggesting that it might be
| good if the purpose of the survey was explicitly set out as trying to
| inform developers of haskell libraries and tools (and educational
| materials) regarding the systems their potential users work on and
| develop, and their habits and practices in doing so, and where they
| encounter difficulty. That is explicitly as a way of learning rather
| than as any sort of horse-race or popularity contest.

That sounds good to me -- but again in drafting the goals I'd stick 
to the positives, and not speak about horse-races.

Simon
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey

[Simon Peyton Jones via Haskell-community]

Taylor

On the GHC side, as I say we are going to do a 1-question GHC survey shortly, 
so you don't need to bother about that one.  (I think it'd be too buried as one 
question among many in your survey.)

| - Reaction to the new pace of GHC releases

That would be interesting, yes.

It would be interesting to know people's perceptions of the (relatively new) 
GHC Proposals process.  Do they even know about it?  Do they follow what is 
going on?  Does the greater transparency and opportunity to contribute makes 
them feel a greater sense of ownership?

I also wonder if they feel included or excluded in our shared enterprise of 
making GHC a better tool.  

Simon

| -Original Message-
| From: Taylor Fausak 
| Sent: 16 October 2018 00:43
| To: Simon Peyton Jones ; Gershom B
| ; Mathieu Boespflug ; Ben Gamari
| 
| Cc: haskell-community@haskell.org
| Subject: Re: [Haskell-community] 2018 state of Haskell survey
| 
| Thanks for the kind words, Simon! They mean a lot :)
| 
| I would be happy to include questions that would benefit the GHC team,
| including:
| 
| - Simon's question
| - Reaction to the new pace of GHC releases
| - Target of the GHC team's focus: performance, features, ergonomics, etc.
| - Average wait time before upgrading GHC
| 
| Are there any other questions the GHC team would be interested in asking?
| Perhaps I should ask on a different mailing list.
| 
| In response to Gershom's comments:
| 
| 1. Asking how people heard about the survey is a great idea. Not only would
| it let me identify the best ways to reach people, it could also be useful
| in dealing with selection bias.
| 
| 2. Addressed above.
| 
| 3. In general, distinguishing between work and home is something I would
| love to do for basically every question. Unfortunately I think that would
| balloon the size of the survey. Maybe identifying a few key questions for
| the work/home split would be the best way to go? Build systems, as you
| identified, are certainly one of those key questions. Maybe GHC versions
| used is another?
| 
| 4. I also like Go's survey and have been trying to crib as much as I can
| from it. Questions worth asking:
| - Area of development (web, embedded, etc.)
| - Type of development (server, CLI, desktop, library, etc.)
| - Deploy environments / infrastructure
| - Internal versus external
| 
| 5. Asking about JS solutions for web developers is a great idea! I like the
| choices you've given, and there are a whole slew of JS libraries to include
| as well, such as React or Vue.
| 
| 6. Giving multiple choice answers to the "why did you stop" question (and,
| in fact, as many questions as possible) is awesome and would make the
| results much easier to digest. It also makes things easier to compare
| across time, which could be used to gauge the effectiveness of various
| endeavors.
| 
| Thank you all for your feedback so far! I am very excited about this year's
| survey. I want to include as many useful questions as I can without
| overwhelming respondents. As I continue to develop the survey, I constantly
| ask myself this question: "How would I act on responses to this question?"
| For example, last year's survey asked if people had contributed to an open
| source Haskell project. I suspect I will exclude that question because it's
| not really actionable.
| 
| With regards to Haskell.org sponsorship, I still think that throwing around
| the words "official" and "Haskell.org" would do a lot in terms of
| credibility. I don't expect that to remove selection bias, but it will let
| me (us, really) say: We're doing this together for the benefit of all
| sides. And if people have problems with the survey, I want them to feel
| comfortable trying to fix those problems, even if they're not on my "side".
| 
| On Mon, Oct 15, 2018, at 3:55 AM, Simon Peyton Jones wrote:
| > The GHC team (coincidentally) is cooking up a very short survey
| > that is intended to help guide our strategic priorities.
| >
| > There is only one substantial question:
| >
| > Imagine that you had one developer working on GHC for
| > six months full-time, that you were paying for yourself.
| > What would you ask that person to do?   Keep in mind that
| > the project should be within GHC itself and tractable
| > in a six-month time-frame.
| >
| > It has a rather different purpose to Taylor's proposed survey
| > and FP Complete's, because it is not comprehensive, but instead
| > focuses on a single question about a single artefact.
| >
| > Returning to Taylor's question list, yes, it would be interesting to
| > know whether the increased release tempo is perceived as helpful or
| > unhelpful.  (It's helpful for the development /process/ because it makes
| > reduces the pressure to squeeze "just one more thing" into a release:
| > the next bus will be along in only 6 months.)
| >
| > Incid

Re: [Haskell-community] 2018 state of Haskell survey

[Gershom B]

ng in only 6 months.)
> >
> > Incidentally, I for one think that it's a Good Thing that you included
> > FP Complete's survey in the HWN, Taylor.  First, I think it's a
> > substantial and interesting piece of work -- and /any/ survey is
> > vulnerable to response bias.  Second, I don’t think anyone should expect
> > you as HWN editor to play a role as community censor. Third,
> > deliberately excluding it would in itself be a divisive act in a
> > community that needs less division and more love.
> >
> > You do a fantastic job with HWN.  Please keep doing it!
> >
> > Thanks
> >
> > Simon
> >
> > | -Original Message-
> > | From: Haskell-community  On
> > | Behalf Of Gershom B
> > | Sent: 15 October 2018 01:56
> > | To: tay...@fausak.me; Mathieu Boespflug ; Ben Gamari
> > | 
> > | Cc: haskell-community@haskell.org
> > | Subject: Re: [Haskell-community] 2018 state of Haskell survey
> > |
> > | (cc mathieu boespflug, ben gamari)
> > |
> > | One more thought:
> > |
> > | mathieu, ben -- do you think you would be interested in any questions
> > | on the frequency of ghc releases -- if people appreciate more
> > | frequent, smaller releases, or not, etc?
> > |
> > | I wonder if there are any other questions as well about core libraries
> > | vs. performance vs. "big features" (like type system things) vs. small
> > | ergonomic features etc. that the core ghc team might be interested in
> > | sounding out people on, bearing in mind the necessary limitations of
> > | survey derived data.
> > |
> > | --g
> > | On Sun, Oct 14, 2018 at 8:36 PM Gershom B  wrote:
> > | >
> > | > Hi Taylor.
> > | >
> > | > We're discussing this in the committee. I agree that to the extent
> > | > they can accurately reflect something, language surveys are useful,
> > | > and appreciate that you want to run a useful survey, and certainly
> > | > want to encourage and help you in making it as broad and useful as
> > | > possible. That said, I don't know if slapping a "haskell.org" label on
> > | > the survey will help manage the biggest drivers of selection bias --
> > | > which is not only about who chooses to respond, but about who is
> > | > reached through what mechanisms. (I don't know the relative importance
> > | > of response-bias vs. outreach-bias in general, and would be curious if
> > | > somebody has some good research on that to point to). I honestly don't
> > | > know if we have enough channels _in general_ to do a good survey, no
> > | > matter who runs it or how at all! Regardless of the decision we come
> > | > to, here are a few of my personal thoughts on the questions you have
> > | > thus far, and what could be added:
> > | >
> > | > 1) A question "how did you hear about this survey" -- this could at
> > | > least help to disentangle outreach-bias, or notice it, depending on if
> > | > it induces any correlations.
> > | >
> > | > 2) A question on how long after a new GHC release users upgrade --
> > | > both personally, and at work.
> > | >
> > | > 3) Distinguishing between personal and work build-systems in the
> > | > relevant question.
> > | >
> > | > 4) I think the sorts of questions that make sense to ask in this early
> > | > part can resemble those in the first part of the Go survey:
> > | >
> > | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.go
> > | lang.org%2Fsurvey2017-
> > | resultsdata=02%7C01%7Csimonpj%40microsoft.com%7C9fbbfac1fd9e420bbf2
> > | 008d63238f7c4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6367516175958
> > | 02229sdata=2wB5Ph5%2Be6wZm0CWP7Yzx%2Fxe4dOlKHqNPKZReiJmE50%3Dr
> > | eserved=0 (I especially like the
> > | > questions about the area of development and server vs cli apps vs
> > | > libraries). I also like their questions about what environments teams
> > | > deploy programs to. It would also be worth asking if the apps
> > | > developed are customer-facing or internal.
> > | >
> > | > 5) I think an interesting question would be what preferred js
> > | > solution, if any, teams adopt -- i.e. ghcjs, typescript, purescript,
> > | > raw js, etc.
> > | >
> > | > 6)  for the "why did you stop" question, there are a good range of
> > | > potential multi-choice answers that can be drawn from with the rust
> > | > user survey:
> > | https://na01.safelinks.protection

Re: [Haskell-community] 2018 state of Haskell survey

[Taylor Fausak]

Thanks for the kind words, Simon! They mean a lot :) 

I would be happy to include questions that would benefit the GHC team, 
including:

- Simon's question
- Reaction to the new pace of GHC releases
- Target of the GHC team's focus: performance, features, ergonomics, etc.
- Average wait time before upgrading GHC

Are there any other questions the GHC team would be interested in asking? 
Perhaps I should ask on a different mailing list.

In response to Gershom's comments:

1. Asking how people heard about the survey is a great idea. Not only would it 
let me identify the best ways to reach people, it could also be useful in 
dealing with selection bias. 

2. Addressed above.

3. In general, distinguishing between work and home is something I would love 
to do for basically every question. Unfortunately I think that would balloon 
the size of the survey. Maybe identifying a few key questions for the work/home 
split would be the best way to go? Build systems, as you identified, are 
certainly one of those key questions. Maybe GHC versions used is another? 

4. I also like Go's survey and have been trying to crib as much as I can from 
it. Questions worth asking:
- Area of development (web, embedded, etc.)
- Type of development (server, CLI, desktop, library, etc.)
- Deploy environments / infrastructure
- Internal versus external

5. Asking about JS solutions for web developers is a great idea! I like the 
choices you've given, and there are a whole slew of JS libraries to include as 
well, such as React or Vue. 

6. Giving multiple choice answers to the "why did you stop" question (and, in 
fact, as many questions as possible) is awesome and would make the results much 
easier to digest. It also makes things easier to compare across time, which 
could be used to gauge the effectiveness of various endeavors. 

Thank you all for your feedback so far! I am very excited about this year's 
survey. I want to include as many useful questions as I can without 
overwhelming respondents. As I continue to develop the survey, I constantly ask 
myself this question: "How would I act on responses to this question?" For 
example, last year's survey asked if people had contributed to an open source 
Haskell project. I suspect I will exclude that question because it's not really 
actionable. 

With regards to Haskell.org sponsorship, I still think that throwing around the 
words "official" and "Haskell.org" would do a lot in terms of credibility. I 
don't expect that to remove selection bias, but it will let me (us, really) 
say: We're doing this together for the benefit of all sides. And if people have 
problems with the survey, I want them to feel comfortable trying to fix those 
problems, even if they're not on my "side". 

On Mon, Oct 15, 2018, at 3:55 AM, Simon Peyton Jones wrote:
> The GHC team (coincidentally) is cooking up a very short survey
> that is intended to help guide our strategic priorities.
> 
> There is only one substantial question:
> 
>   Imagine that you had one developer working on GHC for 
>   six months full-time, that you were paying for yourself.
>   What would you ask that person to do?   Keep in mind that
>   the project should be within GHC itself and tractable
>   in a six-month time-frame.
> 
> It has a rather different purpose to Taylor's proposed survey
> and FP Complete's, because it is not comprehensive, but instead
> focuses on a single question about a single artefact.
> 
> Returning to Taylor's question list, yes, it would be interesting to 
> know whether the increased release tempo is perceived as helpful or 
> unhelpful.  (It's helpful for the development /process/ because it makes 
> reduces the pressure to squeeze "just one more thing" into a release: 
> the next bus will be along in only 6 months.)
> 
> Incidentally, I for one think that it's a Good Thing that you included 
> FP Complete's survey in the HWN, Taylor.  First, I think it's a 
> substantial and interesting piece of work -- and /any/ survey is 
> vulnerable to response bias.  Second, I don’t think anyone should expect 
> you as HWN editor to play a role as community censor. Third, 
> deliberately excluding it would in itself be a divisive act in a 
> community that needs less division and more love.
> 
> You do a fantastic job with HWN.  Please keep doing it!
> 
> Thanks
> 
> Simon
> 
> | -Original Message-
> | From: Haskell-community  On
> | Behalf Of Gershom B
> | Sent: 15 October 2018 01:56
> | To: tay...@fausak.me; Mathieu Boespflug ; Ben Gamari
> | 
> | Cc: haskell-community@haskell.org
> | Subject: Re: [Haskell-community] 2018 state of Haskell survey
> | 
> | (cc mathieu boespflug, ben gamari)
> | 
> | One more thought:
> | 
> | mathieu, ben -- do you think you would be i

Re: [Haskell-community] 2018 state of Haskell survey

[Simon Peyton Jones via Haskell-community]

The GHC team (coincidentally) is cooking up a very short survey
that is intended to help guide our strategic priorities.

There is only one substantial question:

Imagine that you had one developer working on GHC for 
six months full-time, that you were paying for yourself.
What would you ask that person to do?   Keep in mind that
the project should be within GHC itself and tractable
in a six-month time-frame.

It has a rather different purpose to Taylor's proposed survey
and FP Complete's, because it is not comprehensive, but instead
focuses on a single question about a single artefact.

Returning to Taylor's question list, yes, it would be interesting to know 
whether the increased release tempo is perceived as helpful or unhelpful.  
(It's helpful for the development /process/ because it makes reduces the 
pressure to squeeze "just one more thing" into a release: the next bus will be 
along in only 6 months.)

Incidentally, I for one think that it's a Good Thing that you included FP 
Complete's survey in the HWN, Taylor.  First, I think it's a substantial and 
interesting piece of work -- and /any/ survey is vulnerable to response bias.  
Second, I don’t think anyone should expect you as HWN editor to play a role as 
community censor. Third, deliberately excluding it would in itself be a 
divisive act in a community that needs less division and more love.

You do a fantastic job with HWN.  Please keep doing it!

Thanks

Simon

| -Original Message-
| From: Haskell-community  On
| Behalf Of Gershom B
| Sent: 15 October 2018 01:56
| To: tay...@fausak.me; Mathieu Boespflug ; Ben Gamari
| 
| Cc: haskell-community@haskell.org
| Subject: Re: [Haskell-community] 2018 state of Haskell survey
| 
| (cc mathieu boespflug, ben gamari)
| 
| One more thought:
| 
| mathieu, ben -- do you think you would be interested in any questions
| on the frequency of ghc releases -- if people appreciate more
| frequent, smaller releases, or not, etc?
| 
| I wonder if there are any other questions as well about core libraries
| vs. performance vs. "big features" (like type system things) vs. small
| ergonomic features etc. that the core ghc team might be interested in
| sounding out people on, bearing in mind the necessary limitations of
| survey derived data.
| 
| --g
| On Sun, Oct 14, 2018 at 8:36 PM Gershom B  wrote:
| >
| > Hi Taylor.
| >
| > We're discussing this in the committee. I agree that to the extent
| > they can accurately reflect something, language surveys are useful,
| > and appreciate that you want to run a useful survey, and certainly
| > want to encourage and help you in making it as broad and useful as
| > possible. That said, I don't know if slapping a "haskell.org" label on
| > the survey will help manage the biggest drivers of selection bias --
| > which is not only about who chooses to respond, but about who is
| > reached through what mechanisms. (I don't know the relative importance
| > of response-bias vs. outreach-bias in general, and would be curious if
| > somebody has some good research on that to point to). I honestly don't
| > know if we have enough channels _in general_ to do a good survey, no
| > matter who runs it or how at all! Regardless of the decision we come
| > to, here are a few of my personal thoughts on the questions you have
| > thus far, and what could be added:
| >
| > 1) A question "how did you hear about this survey" -- this could at
| > least help to disentangle outreach-bias, or notice it, depending on if
| > it induces any correlations.
| >
| > 2) A question on how long after a new GHC release users upgrade --
| > both personally, and at work.
| >
| > 3) Distinguishing between personal and work build-systems in the
| > relevant question.
| >
| > 4) I think the sorts of questions that make sense to ask in this early
| > part can resemble those in the first part of the Go survey:
| >
| https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.go
| lang.org%2Fsurvey2017-
| resultsdata=02%7C01%7Csimonpj%40microsoft.com%7C9fbbfac1fd9e420bbf2
| 008d63238f7c4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6367516175958
| 02229sdata=2wB5Ph5%2Be6wZm0CWP7Yzx%2Fxe4dOlKHqNPKZReiJmE50%3Dr
| eserved=0 (I especially like the
| > questions about the area of development and server vs cli apps vs
| > libraries). I also like their questions about what environments teams
| > deploy programs to. It would also be worth asking if the apps
| > developed are customer-facing or internal.
| >
| > 5) I think an interesting question would be what preferred js
| > solution, if any, teams adopt -- i.e. ghcjs, typescript, purescript,
| > raw js, etc.
| >
| > 6)  for the "why did you stop" question, there are a good range of
| > potential multi-choice answers that can be drawn from

Re: [Haskell-community] 2018 state of Haskell survey

[Gershom B]

(cc mathieu boespflug, ben gamari)

One more thought:

mathieu, ben -- do you think you would be interested in any questions
on the frequency of ghc releases -- if people appreciate more
frequent, smaller releases, or not, etc?

I wonder if there are any other questions as well about core libraries
vs. performance vs. "big features" (like type system things) vs. small
ergonomic features etc. that the core ghc team might be interested in
sounding out people on, bearing in mind the necessary limitations of
survey derived data.

--g
On Sun, Oct 14, 2018 at 8:36 PM Gershom B  wrote:
>
> Hi Taylor.
>
> We're discussing this in the committee. I agree that to the extent
> they can accurately reflect something, language surveys are useful,
> and appreciate that you want to run a useful survey, and certainly
> want to encourage and help you in making it as broad and useful as
> possible. That said, I don't know if slapping a "haskell.org" label on
> the survey will help manage the biggest drivers of selection bias --
> which is not only about who chooses to respond, but about who is
> reached through what mechanisms. (I don't know the relative importance
> of response-bias vs. outreach-bias in general, and would be curious if
> somebody has some good research on that to point to). I honestly don't
> know if we have enough channels _in general_ to do a good survey, no
> matter who runs it or how at all! Regardless of the decision we come
> to, here are a few of my personal thoughts on the questions you have
> thus far, and what could be added:
>
> 1) A question "how did you hear about this survey" -- this could at
> least help to disentangle outreach-bias, or notice it, depending on if
> it induces any correlations.
>
> 2) A question on how long after a new GHC release users upgrade --
> both personally, and at work.
>
> 3) Distinguishing between personal and work build-systems in the
> relevant question.
>
> 4) I think the sorts of questions that make sense to ask in this early
> part can resemble those in the first part of the Go survey:
> https://blog.golang.org/survey2017-results (I especially like the
> questions about the area of development and server vs cli apps vs
> libraries). I also like their questions about what environments teams
> deploy programs to. It would also be worth asking if the apps
> developed are customer-facing or internal.
>
> 5) I think an interesting question would be what preferred js
> solution, if any, teams adopt -- i.e. ghcjs, typescript, purescript,
> raw js, etc.
>
> 6)  for the "why did you stop" question, there are a good range of
> potential multi-choice answers that can be drawn from with the rust
> user survey: 
> https://blog.rust-lang.org/2017/09/05/Rust-2017-Survey-Results.html
>
> Cheers,
> Gershom
>
> On Sun, Oct 14, 2018 at 3:32 PM Taylor Fausak  wrote:
> >
> > I'm not entirely sure what official support would look like. A few things 
> > come to mind:
> >
> > 1. Simply putting "official" somewhere in the title, such as: Official 2018 
> > state of Haskell survey.
> >
> > 2. Putting something about Haskell.org in the description of the survey, 
> > such as: Sponsored by Haskell.org.
> >
> > 3. Announcing the survey through channels that I may not be aware of. Or 
> > helping me announce the survey through various channels (mailing lists, 
> > Reddit, and so on) by mentioning Haskell.org.
> >
> > I'm sure there are more ways that I'm not thinking of.
> >
> > Perhaps it would be better to state my goal and see what, if anything, can 
> > be to achieve that goal? My goal is for this survey to be *the* 
> > authoritative Haskell survey and for the community to broadly accept it 
> > results. In particular I would like to avoid reactions like these to the 
> > recent FP Complete survey:
> >
> > > I browse r/haskell all the time and follow FPco on Twitter, and I wasn't 
> > > aware of this survey. [1]
> >
> > > It should not be surprising to think that fp complete has much better 
> > > outreach to Stackage users than to non Stackage users. [2]
> >
> > > Any survey hosted by FPComplete is biased towards users of stack, for 
> > > reasons that should be self evident. [3]
> >
> > Similar sentiments have been expressed about last year's Haskell Weekly 
> > survey:
> >
> > > Haskell Weekly's reputation is tainted as it appears to be seen as 
> > > partisan (and I tend to agree). [4]
> >
> > > one of those surveys is from FP Complete and one of them is from someone 
> > > who I would consider very partisan in these kind of discussions. [5]
> >
> > > For the love of god stop posting those surveys. They're not convincing 
> > > and obviously flawed. [6]
> >
> > I don't expect to be able to make everyone happy, but I think that 
> > presenting this year's survey as sponsored by both Haskell Weekly and 
> > Haskell.org would go a long way toward making it acceptable to a broad 
> > range of the Haskell community.
> >
> > I hope that helps!
> >
> > [1]: 
> > 

Re: [Haskell-community] 2018 state of Haskell survey

[Gershom B]

Hi Taylor.

We're discussing this in the committee. I agree that to the extent
they can accurately reflect something, language surveys are useful,
and appreciate that you want to run a useful survey, and certainly
want to encourage and help you in making it as broad and useful as
possible. That said, I don't know if slapping a "haskell.org" label on
the survey will help manage the biggest drivers of selection bias --
which is not only about who chooses to respond, but about who is
reached through what mechanisms. (I don't know the relative importance
of response-bias vs. outreach-bias in general, and would be curious if
somebody has some good research on that to point to). I honestly don't
know if we have enough channels _in general_ to do a good survey, no
matter who runs it or how at all! Regardless of the decision we come
to, here are a few of my personal thoughts on the questions you have
thus far, and what could be added:

1) A question "how did you hear about this survey" -- this could at
least help to disentangle outreach-bias, or notice it, depending on if
it induces any correlations.

2) A question on how long after a new GHC release users upgrade --
both personally, and at work.

3) Distinguishing between personal and work build-systems in the
relevant question.

4) I think the sorts of questions that make sense to ask in this early
part can resemble those in the first part of the Go survey:
https://blog.golang.org/survey2017-results (I especially like the
questions about the area of development and server vs cli apps vs
libraries). I also like their questions about what environments teams
deploy programs to. It would also be worth asking if the apps
developed are customer-facing or internal.

5) I think an interesting question would be what preferred js
solution, if any, teams adopt -- i.e. ghcjs, typescript, purescript,
raw js, etc.

6)  for the "why did you stop" question, there are a good range of
potential multi-choice answers that can be drawn from with the rust
user survey: https://blog.rust-lang.org/2017/09/05/Rust-2017-Survey-Results.html

Cheers,
Gershom

On Sun, Oct 14, 2018 at 3:32 PM Taylor Fausak  wrote:
>
> I'm not entirely sure what official support would look like. A few things 
> come to mind:
>
> 1. Simply putting "official" somewhere in the title, such as: Official 2018 
> state of Haskell survey.
>
> 2. Putting something about Haskell.org in the description of the survey, such 
> as: Sponsored by Haskell.org.
>
> 3. Announcing the survey through channels that I may not be aware of. Or 
> helping me announce the survey through various channels (mailing lists, 
> Reddit, and so on) by mentioning Haskell.org.
>
> I'm sure there are more ways that I'm not thinking of.
>
> Perhaps it would be better to state my goal and see what, if anything, can be 
> to achieve that goal? My goal is for this survey to be *the* authoritative 
> Haskell survey and for the community to broadly accept it results. In 
> particular I would like to avoid reactions like these to the recent FP 
> Complete survey:
>
> > I browse r/haskell all the time and follow FPco on Twitter, and I wasn't 
> > aware of this survey. [1]
>
> > It should not be surprising to think that fp complete has much better 
> > outreach to Stackage users than to non Stackage users. [2]
>
> > Any survey hosted by FPComplete is biased towards users of stack, for 
> > reasons that should be self evident. [3]
>
> Similar sentiments have been expressed about last year's Haskell Weekly 
> survey:
>
> > Haskell Weekly's reputation is tainted as it appears to be seen as partisan 
> > (and I tend to agree). [4]
>
> > one of those surveys is from FP Complete and one of them is from someone 
> > who I would consider very partisan in these kind of discussions. [5]
>
> > For the love of god stop posting those surveys. They're not convincing and 
> > obviously flawed. [6]
>
> I don't expect to be able to make everyone happy, but I think that presenting 
> this year's survey as sponsored by both Haskell Weekly and Haskell.org would 
> go a long way toward making it acceptable to a broad range of the Haskell 
> community.
>
> I hope that helps!
>
> [1]: 
> https://np.reddit.com/r/haskell/comments/9mm05d/2018_haskell_survey_results/e7gplya/
> [2]: 
> https://np.reddit.com/r/haskell/comments/8tc8pr/fp_complete_launches_new_blockchain_auditing/e188ftv/?context=3
> [3]: 
> https://np.reddit.com/r/haskell/comments/9mm05d/2018_haskell_survey_results/e7gpfwe/
> [4]: 
> https://np.reddit.com/r/haskell/comments/9mm05d/2018_haskell_survey_results/e7ka8xn/
> [5]: 
> https://np.reddit.com/r/haskell/comments/8tc8pr/fp_complete_launches_new_blockchain_auditing/e187pzq/?context=1
> [6]: 
> https://np.reddit.com/r/haskell/comments/8uw9hw/psa_cabal_breaks_with_yaml0831_on_ghc_710_and/e1lfzgr/?context=1
>
>
> On Sun, Oct 14, 2018, at 2:21 PM, Neil Mitchell wrote:
>
> Hi Taylor,
>
> What does official support look like? I don't think there's anything above 
> and 

Re: [Haskell-community] 2018 state of Haskell survey

[Taylor Fausak]

I'm not entirely sure what official support would look like. A few
things come to mind:
1. Simply putting "official" somewhere in the title, such as: Official
   2018 state of Haskell survey.
2. Putting something about Haskell.org in the description of the survey,
   such as: Sponsored by Haskell.org.
3. Announcing the survey through channels that I may not be aware of. Or
   helping me announce the survey through various channels (mailing
   lists, Reddit, and so on) by mentioning Haskell.org.
I'm sure there are more ways that I'm not thinking of. 

Perhaps it would be better to state my goal and see what, if anything,
can be to achieve that goal? My goal is for this survey to be *the*
authoritative Haskell survey and for the community to broadly accept it
results. In particular I would like to avoid reactions like these to the
recent FP Complete survey:
> I browse r/haskell all the time *and* follow FPco on Twitter, and I
> wasn't aware of this survey. [1]
> It should not be surprising to think that fp complete has much better
> outreach to Stackage users than to non Stackage users. [2]
> Any survey hosted by FPComplete is biased towards users of stack, for
> reasons that should be self evident. [3]
Similar sentiments have been expressed about last year's Haskell
Weekly survey:
> Haskell Weekly's reputation is tainted as it appears to be seen as
> partisan (and I tend to agree). [4]
> one of those surveys is from FP Complete and one of them is from
> someone who I would consider very partisan in these kind of
> discussions. [5]
> For the love of god stop posting those surveys. They're not convincing
> and obviously flawed. [6]
I don't expect to be able to make everyone happy, but I think that
presenting this year's survey as sponsored by both Haskell Weekly and
Haskell.org would go a long way toward making it acceptable to a broad
range of the Haskell community.
I hope that helps! 

[1]: 
https://np.reddit.com/r/haskell/comments/9mm05d/2018_haskell_survey_results/e7gplya/[2]:
 
https://np.reddit.com/r/haskell/comments/8tc8pr/fp_complete_launches_new_blockchain_auditing/e188ftv/?context=3[3]:
 
https://np.reddit.com/r/haskell/comments/9mm05d/2018_haskell_survey_results/e7gpfwe/[4]:
 
https://np.reddit.com/r/haskell/comments/9mm05d/2018_haskell_survey_results/e7ka8xn/[5]:
 
https://np.reddit.com/r/haskell/comments/8tc8pr/fp_complete_launches_new_blockchain_auditing/e187pzq/?context=1[6]:
 
https://np.reddit.com/r/haskell/comments/8uw9hw/psa_cabal_breaks_with_yaml0831_on_ghc_710_and/e1lfzgr/?context=1

On Sun, Oct 14, 2018, at 2:21 PM, Neil Mitchell wrote:
> Hi Taylor,
> 
> What does official support look like? I don't think there's anything
> above and beyond what you're already doing.> 
> Thanks, Neil
> 
> On Sun, 14 Oct 2018 at 3:49 pm, Taylor Fausak
>  wrote:>> (I have CCed  because the 
> haskell-community
>> mailing list seems relatively quiet and I want to make sure this
>> is seen.)>> 
>>  Hello! My name is Taylor Fausak. I run the Haskell Weekly
>>  newsletter. Last year I published a survey [1] for the Haskell
>>  community. I collected and reported [2] on about 1,335 responses. I
>>  plan on publishing another survey this year on the same date,
>>  November 1st. I am developing it in the open again [3] and would
>>  love to hear from any interested parties. Please let me know if you
>>  have any ideas about the survey!>> 
>>  This year I am interested in making the survey official by seeking
>>  support from Haskell.org. Is such a thing possible and desirable?>> 
>>  Thanks for your consideration! I hope to hear from you soon.
>> 
>>  [1]: https://haskellweekly.news/surveys/2017.html
>>  [2]: 
>> https://taylor.fausak.me/2017/11/15/2017-state-of-haskell-survey-results/>>  
>> [3]: https://github.com/haskellweekly/haskellweekly.github.io/issues/206>>  
>> ___
>>  Haskell-community mailing list
>> Haskell-community@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] 2018 state of Haskell survey

[Neil Mitchell]

Hi Taylor,

What does official support look like? I don't think there's anything above
and beyond what you're already doing.

Thanks, Neil

On Sun, 14 Oct 2018 at 3:49 pm, Taylor Fausak  wrote:

> (I have CCed  because the haskell-community
> mailing list seems relatively quiet and I want to make sure this is seen.)
>
> Hello! My name is Taylor Fausak. I run the Haskell Weekly newsletter. Last
> year I published a survey [1] for the Haskell community. I collected and
> reported [2] on about 1,335 responses. I plan on publishing another survey
> this year on the same date, November 1st. I am developing it in the open
> again [3] and would love to hear from any interested parties. Please let me
> know if you have any ideas about the survey!
>
> This year I am interested in making the survey official by seeking support
> from Haskell.org. Is such a thing possible and desirable?
>
> Thanks for your consideration! I hope to hear from you soon.
>
> [1]: https://haskellweekly.news/surveys/2017.html
> [2]:
> https://taylor.fausak.me/2017/11/15/2017-state-of-haskell-survey-results/
> [3]: https://github.com/haskellweekly/haskellweekly.github.io/issues/206
> ___
> Haskell-community mailing list
> Haskell-community@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

[Haskell-community] 2018 state of Haskell survey

[Taylor Fausak]

(I have CCed  because the haskell-community mailing list 
seems relatively quiet and I want to make sure this is seen.)

Hello! My name is Taylor Fausak. I run the Haskell Weekly newsletter. Last year 
I published a survey [1] for the Haskell community. I collected and reported 
[2] on about 1,335 responses. I plan on publishing another survey this year on 
the same date, November 1st. I am developing it in the open again [3] and would 
love to hear from any interested parties. Please let me know if you have any 
ideas about the survey! 

This year I am interested in making the survey official by seeking support from 
Haskell.org. Is such a thing possible and desirable? 

Thanks for your consideration! I hope to hear from you soon.

[1]: https://haskellweekly.news/surveys/2017.html
[2]: https://taylor.fausak.me/2017/11/15/2017-state-of-haskell-survey-results/
[3]: https://github.com/haskellweekly/haskellweekly.github.io/issues/206
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community