Re: [Haskell-community] 2018 state of Haskell survey results

[Chris Smith]

> For example, just reading this thread, it sounds like the bogus responses
also really don't like the new release schedule. Maybe the troll wants the
old release schedule back and was just lazy about programming the tool to
vary the stack/cabal question answers adequately.

There is another scenario, though, which should caution against making
official statements about motivation.  There was a set of people who worked
very hard while the survey was open to preemptively cast doubt on its
motivation and goals.  It may be that someone was mainly attempting to
sabotage the survey results themselves, rather than taking a side in any
specific dispute.  Of course, had the results been published claiming that
a mere 12% of Haskellers use Cabal, it would have been immediately
dismissed by many people as obviously biased, which would have achieved
that goal, too.

I think Taylor's post handled this well, saying what we know to be true,
that the attack targeted divisive issues, but without drawing unnecessary
conclusions.

On Sun, Nov 18, 2018 at 11:21 PM Richard Eisenberg 
wrote:

> I have not analyzed the data myself, but I wonder how we jumped to the
> conclusion that the troll was trying to promote Stack. Is there statistical
> data that supports that conclusion? For example, just reading this thread,
> it sounds like the bogus responses also really don't like the new release
> schedule. Maybe the troll wants the old release schedule back and was just
> lazy about programming the tool to vary the stack/cabal question answers
> adequately.
>
> Given the contention around cabal vs stack, I agree that sociological
> concerns suggest that the troll meant to tilt those scales. But I wouldn't
> want a public accusation without at least some statistical analysis that
> independently supports that conclusion.
>
> In any case, thanks to all for putting this together!
>
> Richard
>
> On Nov 18, 2018, at 4:31 PM, Taylor Fausak  wrote:
>
> Oops, the ordering of the answer choices is manual because some questions
> have a natural order while others should just be most to least popular.
> I've made another run through to make sure everything is sorted properly.
> I'll probably hit publish in the next half hour or so unless there are any
> objections.
>
>
> https://github.com/tfausak/tfausak.github.io/blob/fce97d07c369856d4c05b756c492eb6229a1b5c7/_posts/2018-11-18-2018-state-of-haskell-survey-results.markdown
>
>
> On Sun, Nov 18, 2018, at 3:07 PM, Gershom B wrote:
>
> The language extensions section doesn’t appear to be sorted properly.
> Outside of that, I think that these results are looking much better and any
> effort to find any additional outliers is probably not worth it for the
> moment. Thanks for your work on this, and I appreciate you being responsive
> and attentive when problems with the data were pointed out. There’s
> certainly some interesting and helpful information to be gleaned from this
> data.
>
> Cheers,
> Gershom
>
>
>
>
> On November 18, 2018 at 2:55:10 PM, Taylor Fausak (tay...@fausak.me)
> wrote:
>
>
>
>
> Ok, I updated the function that checks for bad responses, re-ran the
> script, and updated the announcement along with all the assets (charts,
> tables, and CSV). Hopefully it's the last time, as I can't justify spending
> much more time on this.
>
>
> https://github.com/tfausak/tfausak.github.io/blob/6f9991758ffeed085c45dd97e4ce6a82a8b1a73f/_posts/2018-11-18-2018-state-of-haskell-survey-results.markdown
>
>
> On Sun, Nov 18, 2018, at 2:32 PM, Michael Snoyman wrote:
>
> Just wanted to add in: good catch Gershom on identifying the problem, and
> thank you Taylor for working to remove them from the report.
>
> On 18 Nov 2018, at 21:17, Taylor Fausak  wrote:
>
> Great catch, Gershom! There are indeed about 300 responses that tick all
> the boxes except for disliking the new GHC release schedule. The main thing
> the attacker seemed to be interested in was over-representing Stack and
> Stackage. Also, bizarrely, Java.
>
> That brings the number of bogus responses up to 3,735, which puts the
> number of legitimate responses at 1,361. For context, last year's survey
> asked far fewer questions and had 1,335 responses.
>
>
> On Sun, Nov 18, 2018, at 1:26 PM, Imants Cekusins wrote:
>
> What if the announcement mentioned a large number of potentially bogus
> responses, explained the grounds for this conclusion, with a new survey
> conducted early next year?
>
> The next survey would then need to be done differently from this one
> somehow. To improve the reliability, some authentication may be necessary.
>
>
> Maybe Stack, Cabal questions could be grouped as separate distinct
> surveys, conducted by their maintainers through own channels?
>
> Not sure how much value is in exact numbers of users of Stack or Cabal.
> Both groups are large enough. The maintainers of both groups are aware
> about usage stats.
>
> Is either library likely to be influenced by this survey?
> 

Re: [Haskell-community] 2018 state of Haskell survey results

[Chris Smith]

If I could make a suggestion... although this is at the forefront of our
minds right now, I don't think that you want the attempted hack of survey
responses to be THE big news about the survey.  I have no doubt it will
garner lots of attention anyway, and you are certainly right to explain
what happened and what your methodology was; but I think it would be better
to state the legitimate results first... i.e., by saying "This year we
received 1,679 [*] responses, which is quite an improvement.", and waiting
until later to explain about the bogus submissions.  Hopefully, then, more
of the reaction will be around the data this provides, and less around ugly
drama with what seems like only ONE bad actor.

On Sun, Nov 18, 2018 at 10:58 AM Taylor Fausak  wrote:

> I have filtered out the bogus responses and re-generated all the charts
> and tables. You can see the updated results here:
> https://github.com/tfausak/tfausak.github.io/blob/ee29da5bd8389c19763ac2b4dbe27ff5204161f5/_posts/2018-11-16-2018-state-of-haskell-survey-results.markdown
>
> Note that until I post the results on my blog, they are not published.
> Please don't share the preliminary results on social media!
>
>
> On Sun, Nov 18, 2018, at 8:11 AM, Taylor Fausak wrote:
>
> Thanks for finding those anomalies, Gershom! I'm disappointed that someone
> submitted bogus responses, apparently to tip the scales of Cabal versus
> Stack. I intend to identify those responses and exclude them from the
> results. The work you've done so far will help a great deal in finding
> them.
>
> You said that there are about 1,200 responses with demographic
> information. That makes sense considering the number of submissions I got
> last year. Also, there are 1,185 responses that included an answer to at
> least one of the free-response questions. So perhaps whoever wrote the
> script didn't bother to put an answer for those types of questions.
>
> Unfortunately I do not have precise submission times or IP address
> information about submissions. Beyond what's in the CSV, the only other
> thing I have is (some) email addresses.
>
> Fortunately I wrote a script to output all the charts and tables from the
> survey responses. Once I've identified the problematic responses, I should
> be able to update the script to ignore them and regenerate all the output.
>
>
> On Sun, Nov 18, 2018, at 3:40 AM, Chris Smith wrote:
>
> Sadly, it looks like a Cabal/Stack thing.  Of the responses with a country
> provided, 618 of 1226 claim to use Cabal, and 948 of 1226 claim to use
> Stack. Of the responses with no country, only 35 of 3868 claim to use
> Cabal, while 3781 of the 3868 claim to use Stack.  Assuming independence,
> you'd expect that last number to be about 50, meaning there are probably
> around 3700 fake responses generated just to answer "Stack".
>
> To partially answer Simon's question, the flood of no-demographics
> responses started on November 2, around the 750-response point, and
> continued unabated through the close of the survey.  And, indeed, looking
> at just the first 750 responses gives similar distributions to what we get
> by ignoring the no-demographic responses.  For example, of the first 750
> responses, 359 claim to use Cabal, and 568 claim to use Stack.
>
> On Sun, Nov 18, 2018 at 2:31 AM Simon Marlow  wrote:
>
> Good spot Gershom. Maybe it would be revealing to look at the times that
> responses were received for the no-demographics group?
>
> On Sun, 18 Nov 2018, 07:17 Gershom B 
> I also noticed a number of other bizarre statistical anomolies when
> looking at the full results. I know this is a bit much to ask — but if you
> could rerun the statistics filtering out people that did not give
> demographic information (i.e. country of origin or education, etc) I think
> the results will change drastically. By all statistical logic, this should
> _not_ be the case, and points to a serious problem.
>
> In particular, this drops the results by a huge amount — only 1,200 or so
> remain. However, the remaining results tend to make a lot more sense. For
> example — of the “no demographics” group, there are 713 users who claim to
> develop with notepad++ but all of these say they develop on mac and linux,
> and none on windows — which is impossible, as notepad++ is a windows
> program. Further if you drop the “no demographics” group, then you find
> that almost everyone uses at least ghc 8.0.2, while in the “no
> demographics” group,  a stunning number of people claim to be on 7.8.3.
> Even more bizarrely, people claim to be using the 7.8 series while only
> having used Haskell for less than one year. And people claim to have used
> haskell for “one week to one month” and also to be advanced and expert
> users!
>
>

Re: [Haskell-community] 2018 state of Haskell survey results

[Chris Smith]

Sadly, it looks like a Cabal/Stack thing.  Of the responses with a country
provided, 618 of 1226 claim to use Cabal, and 948 of 1226 claim to use
Stack. Of the responses with no country, only 35 of 3868 claim to use
Cabal, while 3781 of the 3868 claim to use Stack.  Assuming independence,
you'd expect that last number to be about 50, meaning there are probably
around 3700 fake responses generated just to answer "Stack".

To partially answer Simon's question, the flood of no-demographics
responses started on November 2, around the 750-response point, and
continued unabated through the close of the survey.  And, indeed, looking
at just the first 750 responses gives similar distributions to what we get
by ignoring the no-demographic responses.  For example, of the first 750
responses, 359 claim to use Cabal, and 568 claim to use Stack.

On Sun, Nov 18, 2018 at 2:31 AM Simon Marlow  wrote:

> Good spot Gershom. Maybe it would be revealing to look at the times that
> responses were received for the no-demographics group?
>
> On Sun, 18 Nov 2018, 07:17 Gershom B 
>> I also noticed a number of other bizarre statistical anomolies when
>> looking at the full results. I know this is a bit much to ask — but if you
>> could rerun the statistics filtering out people that did not give
>> demographic information (i.e. country of origin or education, etc) I think
>> the results will change drastically. By all statistical logic, this should
>> _not_ be the case, and points to a serious problem.
>>
>> In particular, this drops the results by a huge amount — only 1,200 or so
>> remain. However, the remaining results tend to make a lot more sense. For
>> example — of the “no demographics” group, there are 713 users who claim to
>> develop with notepad++ but all of these say they develop on mac and linux,
>> and none on windows — which is impossible, as notepad++ is a windows
>> program. Further if you drop the “no demographics” group, then you find
>> that almost everyone uses at least ghc 8.0.2, while in the “no
>> demographics” group,  a stunning number of people claim to be on 7.8.3.
>> Even more bizarrely, people claim to be using the 7.8 series while only
>> having used Haskell for less than one year. And people claim to have used
>> haskell for “one week to one month” and also to be advanced and expert
>> users!
>>
>> The differences continue and defy all probability. Of the “no
>> demographics” group, almost everyone dislikes the new release schedule. Of
>> the “demographics” group there are answers that like it, were not aware of
>> it, or are indifferent, but almost nobody dislikes it. There is naturally a
>> difference in proportions of cabal/stack and hackage/stackage responses as
>> well.
>>
>> There are a lot of other things I could point to as well. But, bluntly
>> put, I think that some disaffected party or parties wrote a crude script
>> and submitted over 3,000 fake responses. Luckily for us, they were not very
>> smart, and made some obvious errors, so in this case we can weed out the
>> bad responses (although, sadly, losing at least a few real ones as well).
>>
>> However, assuming  this party isn’t entirely stupid, it doesn’t bode well
>> for future surveys as they may get at least slightly less dumb in the
>> future if they decide to keep it up :-/
>>
>> —Gershom
>>
>>
>> On November 18, 2018 at 1:10:31 AM, Gershom B (gersh...@gmail.com) wrote:
>>
>> This is interesting, but I’m thoroughly confused. Over 2500 people said
>> they took last year’s survey, but it only had roughly 1,300 respondants?
>>
>>
>> On Sat, Nov 17, 2018 at 9:56 PM Taylor Fausak  wrote:
>>
>>> Hello! It took a little longer than I expected, but I am nearly ready to
>>> announce the 2018 state of Haskell survey results. Some community members
>>> have expressed interest in seeing the announcement post before it's
>>> published. If you are one of those people, you can see the results here:
>>> https://github.com/tfausak/tfausak.github.io/blob/7e4937e284a3068add9e9af6b585c8d0215ff360/_posts/2018-11-16-2018-state-of-haskell-survey-results.markdown
>>>
>>> If you would like to suggest changes to the announcement post, please
>>> respond to this email, send me an email directly, or reply to this pull
>>> request on GitHub: https://github.com/tfausak/tfausak.github.io/pull/148
>>>
>>> I plan on publishing the results tomorrow. Once the results are
>>> published, the post is by no means set in stone. I will happily accept
>>> suggestions from anyone at any time.
>>>
>>> Thank you!
>>> ___
>>> Haskell-community mailing list
>>> Haskell-community@haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>>>
>> ___
>> Haskell-community mailing list
>> Haskell-community@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>>
> ___
> Haskell-community mailing 

Re: [Haskell-community] Creating a new @haskell.org mailing list?

[Chris Smith]

Any news on this?  Would love to help any way I can, but I am not sure what
to do next.

Thanks,
Chris

On Wed, Oct 24, 2018 at 3:12 PM Gershom B  wrote:

> Sounds good. Ccing Sandy, who has volunteered to start helping with
> mail stuff. Sandy -- do you need any further details in setting this
> up, or do you think it should be straightforward?
>
> -g
> On Wed, Oct 24, 2018 at 10:18 AM Chris Smith  wrote:
> >
> > Good point, Simon.  education@ sounds like a good choice, with the
> understanding that we mean education for the general population, not
> classes in type theory or category theory!
> >
> > Is this a possibility?  Anything else I can do to move this forward?
> >
> > On Mon, Oct 22, 2018 at 11:32 AM Simon Peyton Jones <
> simo...@microsoft.com> wrote:
> >>
> >> Good idea.   “k12” is rather USA specific. What about
> educat...@haskell.org?
> >>
> >>
> >>
> >> Simon
> >>
> >>
> >>
> >> From: Haskell-community  On
> Behalf Of Chris Smith
> >> Sent: 22 October 2018 15:32
> >> To: Haskell-community 
> >> Subject: [Haskell-community] Creating a new @haskell.org mailing list?
> >>
> >>
> >>
> >> Hey,
> >>
> >>
> >>
> >> Is there a process to request a new mailing list on the haskell.org
> domain?
> >>
> >>
> >>
> >> Here's my use case.  About 25 Haskell programmers met at ICFP to
> discuss uses of Haskell in K-12 education (for non-US readers, that means
> before university).  I'm also in touch with another half-dozen people who
> either have done, or are doing, something pre-university with Haskell, but
> could not be at ICFP.  The main result of our conversation was that we
> wanted a common place to discuss, report on our experiences, look for
> productive collaborations and common threads, etc.  There are already a few
> project-specific places, e.g. the codeworld-discuss mailing list for my own
> project, but we were explicitly looking for something general-purpose and
> universal.  It would be great if this could be, say, "k...@haskell.org" or
> something like that.
> >>
> >>
> >>
> >> I'm pretty open in terms of how we'd administer the list.  I'm willing
> to do the work of handling obvious spam bots and things like that.  If
> there's a feeling we'd need something more than that, then let's have that
> discussion.  We explicitly don't want a strict topicality enforcement,
> though.  For example, several people who attended the dinner at ICFP were
> also interested in functional programming for non-majors at the university
> level, or were using Elm and other Haskell-like languages - even a few
> people from the Racket community.  I'd hope to rely on the name of the
> mailing list to keep things a bit focused, but not really police it at all.
> >>
> >>
> >>
> >> Thoughts?
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Chris Smith
> >
> > ___
> > Haskell-community mailing list
> > Haskell-community@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] Creating a new @haskell.org mailing list?

[Chris Smith]

Good point, Simon.  education@ sounds like a good choice, with the
understanding that we mean education for the general population, not
classes in type theory or category theory!

Is this a possibility?  Anything else I can do to move this forward?

On Mon, Oct 22, 2018 at 11:32 AM Simon Peyton Jones 
wrote:

> Good idea.   “k12” is rather USA specific. What about
> educat...@haskell.org?
>
>
>
> Simon
>
>
>
> *From:* Haskell-community  *On
> Behalf Of *Chris Smith
> *Sent:* 22 October 2018 15:32
> *To:* Haskell-community 
> *Subject:* [Haskell-community] Creating a new @haskell.org mailing list?
>
>
>
> Hey,
>
>
>
> Is there a process to request a new mailing list on the haskell.org
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhaskell.org=02%7C01%7Csimonpj%40microsoft.com%7Cb44e74be5c734a964cd608d6382b25a1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636758155306014274=iLYTigePbix7DpkFJt1JBd%2BDj58EytsEiRWzam3kBm4%3D=0>
> domain?
>
>
>
> Here's my use case.  About 25 Haskell programmers met at ICFP to discuss
> uses of Haskell in K-12 education (for non-US readers, that means before
> university).  I'm also in touch with another half-dozen people who either
> have done, or are doing, something pre-university with Haskell, but could
> not be at ICFP.  The main result of our conversation was that we wanted a
> common place to discuss, report on our experiences, look for productive
> collaborations and common threads, etc.  There are already a few
> project-specific places, e.g. the codeworld-discuss mailing list for my own
> project, but we were explicitly looking for something general-purpose and
> universal.  It would be great if this could be, say, "k...@haskell.org" or
> something like that.
>
>
>
> I'm pretty open in terms of how we'd administer the list.  I'm willing to
> do the work of handling obvious spam bots and things like that.  If there's
> a feeling we'd need something more than that, then let's have that
> discussion.  We explicitly don't want a strict topicality enforcement,
> though.  For example, several people who attended the dinner at ICFP were
> also interested in functional programming for non-majors at the university
> level, or were using Elm and other Haskell-like languages - even a few
> people from the Racket community.  I'd hope to rely on the name of the
> mailing list to keep things a bit focused, but not really police it at all.
>
>
>
> Thoughts?
>
>
>
> Thanks,
>
> Chris Smith
>
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

[Haskell-community] Creating a new @haskell.org mailing list?

[Chris Smith]

Hey,

Is there a process to request a new mailing list on the haskell.org domain?

Here's my use case.  About 25 Haskell programmers met at ICFP to discuss
uses of Haskell in K-12 education (for non-US readers, that means before
university).  I'm also in touch with another half-dozen people who either
have done, or are doing, something pre-university with Haskell, but could
not be at ICFP.  The main result of our conversation was that we wanted a
common place to discuss, report on our experiences, look for productive
collaborations and common threads, etc.  There are already a few
project-specific places, e.g. the codeworld-discuss mailing list for my own
project, but we were explicitly looking for something general-purpose and
universal.  It would be great if this could be, say, "k...@haskell.org" or
something like that.

I'm pretty open in terms of how we'd administer the list.  I'm willing to
do the work of handling obvious spam bots and things like that.  If there's
a feeling we'd need something more than that, then let's have that
discussion.  We explicitly don't want a strict topicality enforcement,
though.  For example, several people who attended the dinner at ICFP were
also interested in functional programming for non-majors at the university
level, or were using Elm and other Haskell-like languages - even a few
people from the Racket community.  I'd hope to rely on the name of the
mailing list to keep things a bit focused, but not really police it at all.

Thoughts?

Thanks,
Chris Smith
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] [Haskell-cafe] Standard package file format

[Chris Smith]

I guess the overriding question I have here is: what is the PROBLEM being
solved?  I know of basically no beginners who were confused or intimidated
by the syntax of Cabal's file format.  It's fairly commonplace for
beginners to be confused by the *semantics*: which fields are needed and
what they mean, how package version bounds work, what flags are and how
they interact with dependencies, the relationship between libraries and
executables defined in the same file, etc.  But the syntax?  It's just not
an issue.  I'm not sure what it means to say that people have to "learn"
it, because in introducing dozens of people to building things in Haskell,
I've never seen that learning process even be noticeable, much less an
impediment.

With this in mind, a lot of the statements about these various languages
are not entirely convincing.  That it's a superset of JSON?  It's not clear
why this matters.  A psychological impression of complexity?  Just not
anything I've seen evidence of.  Indeed, aside from the rather painful
many-years-long migration, the *cost* (though certainly not a prohibitive
one) of moving to something like YAML or TOML is that they have a bit
louder syntax, that demands more attention and feels more complex.

There is one substantial disadvantage I'd point out to the Cabal file
format as it stands, and that's that it's pretty non-obvious how to parse
it, so we will always struggle to interact with it from automated tools,
unless those tools are also written in Haskell and can use the Cabal
library.  That's a real concern; pragmatic large-scale build environments
are not tied to specific languages, and include a variety of ad-hoc
third-party tooling that needs to be integrated, and Cabal remains opaque
to them.  But that doesn't seem to be what's motivating this conversation.

On Thu, Sep 15, 2016 at 11:20 PM, Harendra Kumar 
wrote:

> I am starting a new thread for the package file format related discussion.
>
> From a developer's perspective, the major benefit of a standard and widely
> adopted format and is that people can utilize their knowledge acquired from
> elsewhere, they do not have to go through and learn differently looking and
> incomplete documentation of different tools. The benefit of a common config
> specification is that developers can choose tools freely without worrying
> about learning the same concepts presented in different ways.
>
> Multiple formats flying around also create a psychological impression of
> complexity in the ecosystem for newcomers. If we have consistency there are
> better chances of attracting more people to the language ecosystem.
>
> I gather the following from the discussion till now:
>
> * We have cabal, YAML and TOML as potential candidates for a common
> package format which can additionally incorporate the concept of
> snapshots/package collections and potentially more extensions useful across
> build tools.
>
> * cabal has the benefit of incumbency and backward compatibility, it has
> shortcomings which are being addressed but it is still a format which is
> very specific to Haskell ecosystem. It is not a standard and not going to
> become one. We have to always deal with it ourselves and everyone coming to
> Haskell will have to learn it.
>
> * YAML (http://yaml.org/spec/1.2/spec.html) is standard and popular. A
> significant chunk of developer community is already familiar with it. It is
> being used by stack and by hpack as an alternative to cabal format. The
> complaint against it is that the specification/implementation is overly
> complex.
>
> * TOML (https://github.com/toml-lang/toml) is promising, simpler than
> YAML and is being used by a few important projects but is still evolving
> and is not completely stable. On a first glance it looks pretty simple and
> a lot of other tools use a similar config format. It is aiming to become a
> standard and aiming for a wider adoption.
>
> As a next step we can perhaps do an hpack like experiment using the TOML
> format. That way we will have some experience with that as well and get to
> know if there are any potential problems expressing the existing cabal
> files.
>
> More thoughts, opinions on the topic will help create a better
> understanding about it.
>
> -harendra
>
> ___
> Haskell-Cafe mailing list
> To (un)subscribe, modify options or view archives go to:
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> Only members subscribed via the mailman list are allowed to post.
>
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community