Thanks Reynold for reviewing the ASF rules. Albeit the potential issues mentioned, I feel using StackOverflow would be a improvement. And yes, some guidelines/instructions have the potential to improve the questions and the "escalation" process.
On 7 November 2016 at 10:48, <ioannis.deligian...@nomura.com> wrote: > My two cents (As a user/consumer)… > > > > I have been following & using Spark in financial services before version 1 > and before it migrated questions from Google Groups to apache mailing lists > (which was a shame L ). > > > > SO: > > There has been some momentum lately on SO, but as questions were not > “monitored/answered” by Spark experts, the motivation of posting a question > was low and in turn the quality of questions as well. As most of us know, > SO is usually the first place to look for info and can greatly reduce the > need to turn to user/dev groups so it would be great if there was more > attention to it. > > > > Spark mailing lists: > > As the consensus appears to be, questions tend to get lost if not > picked-up within 1-2 days. Re-sending the same question feels “abusive” to > me so would then give up. Provided that a good question takes time, putting > effort in a question that can easily be ignored results to mailing a “bad” > question (see what happens?) or no question at all. As you have probably > observed, a few users will mail a question to “dev” with “…no answers in > user list…” as they incorrectly assume that no-one can answer their > question. > > > > JIRA: > > I find that “issues” are being quite aggressively closed down. I’ve seen > this twice (one I reported myself and found the second ticket while looking > for a solution) and for this reason it doesn’t encourage users spending the > time and effort to use. Personally, I also feel that there is some bias on > what is in-scope and out-of-scope. > > > > My preference would be that SO would be the first place that someone would > post a question. If a few “experts” are found regularly answering > questions, eventually Spark users will start using it more and reduce > “user” load by easily finding previous answers (or SO community marking a > duplicates). The same “experts” can also encourage users to “escalate” to > JIRA, dev/user groups once a question has been properly filtered which is > quite common. > > > > PS. Personally, I would not follow any “bespoke/external” process on SO > E.g. down-voting on SO for any other reason that being a bad question as > per SO rules. > > > > > > *From:* Matei Zaharia [mailto:matei.zaha...@gmail.com] > *Sent:* 07 November 2016 07:45 > *To:* assaf.mendelson > *Cc:* dev@spark.apache.org > > *Subject:* Re: Handling questions in the mailing lists > > > > Even for the mailing list, I'd love to have a short set of instructions on > how to submit your questions (maybe on http://spark.apache.org/ > community.html > <https://urldefense.proofpoint.com/v2/url?u=http-3A__spark.apache.org_community.html&d=DQMFaQ&c=dCBwIlVXJsYZrY6gpNt0LA&r=B8E4n9FrSS85mPCi6Mfs7cyEPQnVrpcQ1zeB-JKws6A&m=Vf-yZoTpLgwZzwUCoQTMr4UFD_R0nx0naxh_SWUHfho&s=MIQDl3ZflIuyNs62JLog9_vi0dD4xyo96x2w7XwGV3w&e=> > or maybe in the welcome email when you subscribe). It would be great if > someone added that. After all, we have such instructions for contributing > PRs, for example. > > > > Matei > > > > On Nov 6, 2016, at 11:09 PM, assaf.mendelson <assaf.mendel...@rsa.com> > wrote: > > > > There are other options as well. For example hosting an answerhub ( > www.answerhub.com > <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.answerhub.com_&d=DQMFaQ&c=dCBwIlVXJsYZrY6gpNt0LA&r=B8E4n9FrSS85mPCi6Mfs7cyEPQnVrpcQ1zeB-JKws6A&m=Vf-yZoTpLgwZzwUCoQTMr4UFD_R0nx0naxh_SWUHfho&s=2oSovyR4k9m576OtymFnf4nQ4Xksk94HX543bDeEVQI&e=>) > or other similar separate Q&A service. > > BTW, I believe the main issue is not how opinionated people are but who is > answering questions. > > Today there are already people asking (and getting answers) on SO > (including myself). The problem is that many people do not go to SO. > > The problem I see is how to “bump” up questions which are not being > answered to someone more likely to be able to answer them. Simple questions > can be answered by many people, many of them even newbies who ran into the > issue themselves. > > The main issue is that the more complex the question, the less people > there are who can answer it and those people’s bandwidth is already clogged > by other questions. > > We could for example try to create tags on SO for “basic questions”, > “medium”, “advanced”. Provide guidelines to ask first on basic, if not > answered after X days then add the medium tag etc. Downvote people who > don’t go by the process. This would mean that committers for example can > look at advanced only tag and have a manageable number of questions they > can help with while others can answer medium and basic. > > > > I agree that some things are not good for SO. Basically stuff which asks > for opinion is such but most cases in the mailing list are either “how do I > solve this bug” or “how do I do X”. Either of those two are good for SO. > > > > > > Assaf. > > > > > > > > *From:* rxin [via Apache Spark Developers List] [mailto:ml-node+[hidden > email]] > *Sent:* Monday, November 07, 2016 8:33 AM > *To:* Mendelson, Assaf > *Subject:* Re: Handling questions in the mailing lists > > > > This is an excellent point. If we do go ahead and feature SO as a way for > users to ask questions more prominently, as someone who knows SO very well, > would you be willing to help write a short guideline (ideally the shorter > the better, which makes it hard) to direct what goes to user@ and what > goes to SO? > > > > > > On Sun, Nov 6, 2016 at 9:54 PM, Maciej Szymkiewicz <[hidden email]> wrote: > > Damn, I always thought that mailing list is only for nice and welcoming > people and there is nothing to do for me here >:) > > To be serious though, there are many questions on the users list which > would fit just fine on SO but it is not true in general. There are dozens > of questions which are to broad, opinion based, ask for external resources > and so on. If you want to direct users to SO you have to help them to > decide if it is the right channel. Otherwise it will just create a really > bad experience for both seeking help and active answerers. Former ones will > be downvoted and bashed, latter ones will have to deal with handling all > the junk and the number of active Spark users with moderation privileges is > really low (with only Massg and me being able to directly close duplicates). > > Believe me, I've seen this before. > > On 11/07/2016 05:08 AM, Reynold Xin wrote: > > You have substantially underestimated how opinionated people can be on > mailing lists too :) > > On Sunday, November 6, 2016, Maciej Szymkiewicz <[hidden email]> wrote: > > You have to remember that Stack Overflow crowd (like me) is highly > opinionated, so many questions, which could be just fine on the mailing > list, will be quickly downvoted and / or closed as off-topic. Just > saying... > > -- > > Best, > > Maciej > > > > On 11/07/2016 04:03 AM, Reynold Xin wrote: > > OK I've checked on the ASF member list (which is private so there is no > public archive). > > > > It is not against any ASF rule to recommend StackOverflow as a place for > users to ask questions. I don't think we can or should delete the existing > user@spark list either, but we can certainly make SO more visible than it > is. > > > > > > > > On Wed, Nov 2, 2016 at 10:21 AM, Reynold Xin <[hidden email]> wrote: > > Actually after talking with more ASF members, I believe the only policy is > that development decisions have to be made and announced on ASF properties > (dev list or jira), but user questions don't have to. > > > > I'm going to double check this. If it is true, I would actually recommend > us moving entirely over the Q&A part of the user list to stackoverflow, or > at least make that the recommended way rather than the existing user list > which is not very scalable. > > > > On Wednesday, November 2, 2016, Nicholas Chammas <[hidden email]> wrote: > > We’ve discussed several times upgrading our communication tools, as far > back as 2014 and maybe even before that too. The bottom line is that we > can’t due to ASF rules requiring the use of ASF-managed mailing lists. > > For some history, see this discussion: > > 1. https://mail-archives.apache.org/mod_mbox/spark-user/201412.mbox/% > 3CCAOhmDzfL2COdysV8r5hZN8f=NqXM=f=oY5NO2dHWJ_kVEoP+Ng@...%3E > <https://urldefense.proofpoint.com/v2/url?u=https-3A__mail-2Darchives.apache.org_mod-5Fmbox_spark-2Duser_201412.mbox_-253CCAOhmDzfL2COdysV8r5hZN8f-3DNqXM-3Df-3DoY5NO2dHWJ-5FkVEoP-2BNg-40mail.gmail.com-253E&d=DQMFaQ&c=dCBwIlVXJsYZrY6gpNt0LA&r=B8E4n9FrSS85mPCi6Mfs7cyEPQnVrpcQ1zeB-JKws6A&m=Vf-yZoTpLgwZzwUCoQTMr4UFD_R0nx0naxh_SWUHfho&s=fILmWaylBzYeV5-XRmdm75cBbKG57kiU81cArNLLbdA&e=> > > 2. https://mail-archives.apache.org/mod_mbox/spark-user/201501.mbox/% > 3CCAOhmDzec1JdsXQq3dDwAv7eLnzRidSkrsKKG0xKw=TKTxY_sYw@...%3E > <https://urldefense.proofpoint.com/v2/url?u=https-3A__mail-2Darchives.apache.org_mod-5Fmbox_spark-2Duser_201501.mbox_-253CCAOhmDzec1JdsXQq3dDwAv7eLnzRidSkrsKKG0xKw-3DTKTxY-5FsYw-40mail.gmail.com-253E&d=DQMFaQ&c=dCBwIlVXJsYZrY6gpNt0LA&r=B8E4n9FrSS85mPCi6Mfs7cyEPQnVrpcQ1zeB-JKws6A&m=Vf-yZoTpLgwZzwUCoQTMr4UFD_R0nx0naxh_SWUHfho&s=_snNLu3ds5DSqCrMJ30_tq_qhaCPD6I72Sc25p0idmY&e=> > > (It’s ironic that it’s difficult to follow the past discussion on why we > can’t change our official communication tools due to those very tools…) > > Nick > > > > > > On Wed, Nov 2, 2016 at 12:24 PM Ricardo Almeida <[hidden email]> wrote: > > I fell Assaf point is quite relevant if we want to move this project > forward from the Spark user perspective (as I do). In fact, we're still > using 20th century tools (mailing lists) with some add-ons (like Stack > Overflow). > > > > As usually, Sean and Cody's contributions are very to the point. > > I fell it is indeed a matter of of culture (hard to enforce) and tools > (much easier). Isn't it? > > > > On 2 November 2016 at 16:36, Cody Koeninger <[hidden email]> wrote: > > So concrete things people could do > > - users could tag subject lines appropriately to the component they're > asking about > > - contributors could monitor user@ for tags relating to components > they've worked on. > I'd be surprised if my miss rate for any mailing list questions > well-labeled as Kafka was higher than 5% > > - committers could be more aggressive about soliciting and merging PRs > to improve documentation. > It's a lot easier to answer even poorly-asked questions with a link to > relevant docs. > > > On Wed, Nov 2, 2016 at 7:39 AM, Sean Owen <[hidden email]> wrote: > > There's already reviews@ and issues@. dev@ is for project development > itself > > and I think is OK. You're suggesting splitting up user@ and I sympathize > > with the motivation. Experience tells me that we'll have a beginner@ > that's > > then totally ignored, and people will quickly learn to post to advanced@ > to > > get attention, and we'll be back where we started. Putting it in JIRA > > doesn't help. I don't think this a problem that is merely down to lack of > > process. It actually requires cultivating a culture change on the > community > > list. > > > > On Wed, Nov 2, 2016 at 12:11 PM Mendelson, Assaf <[hidden email]> > > wrote: > >> > >> What I am suggesting is basically to fix that. > >> > >> For example, we might say that mailing list A is only for voting, > mailing > >> list B is only for PR and have something like stack overflow for > developer > >> questions (I would even go as far as to have beginner, intermediate and > >> advanced mailing list for users and beginner/advanced for dev). > >> > >> > >> > >> This can easily be done using stack overflow tags, however, that would > >> probably be harder to manage. > >> > >> Maybe using special jira tags and manage it in jira? > >> > >> > >> > >> Anyway as I said, the main issue is not user questions (except maybe > >> advanced ones) but more for dev questions. It is so easy to get lost in > the > >> chatter that it makes it very hard for people to learn spark internals… > >> > >> Assaf. > >> > >> > >> > >> From: Sean Owen [mailto:[hidden email]] > >> Sent: Wednesday, November 02, 2016 2:07 PM > >> To: Mendelson, Assaf; [hidden email] > >> Subject: Re: Handling questions in the mailing lists > >> > >> > >> > >> I think that unfortunately mailing lists don't scale well. This one has > >> thousands of subscribers with different interests and levels of > experience. > >> For any given person, most messages will be irrelevant. I also find > that a > >> lot of questions on user@ are not well-asked, aren't an SSCCE > >> (http://sscce.org/ > <https://urldefense.proofpoint.com/v2/url?u=http-3A__sscce.org_&d=DQMFaQ&c=dCBwIlVXJsYZrY6gpNt0LA&r=B8E4n9FrSS85mPCi6Mfs7cyEPQnVrpcQ1zeB-JKws6A&m=Vf-yZoTpLgwZzwUCoQTMr4UFD_R0nx0naxh_SWUHfho&s=BhbHzFmq52GPq1-vnJpo1WYJSivxTYV2DTqLE2lcomU&e=>), > not something most people are going to bother replying > >> to even if they could answer. I almost entirely ignore user@ because > there > >> are higher-priority channels like PRs to deal with, that already have > >> hundreds of messages per day. This is why little of it gets an answer > -- too > >> noisy. > >> > >> > >> > >> We have to have official mailing lists, in any event, to have some > >> official channel for things like votes and announcements. It's not > wrong to > >> ask questions on user@ of course, but a lot of the questions I see > could > >> have been answered with research of existing docs or looking at the > code. I > >> think that given the scale of the list, it's not wrong to assert that > this > >> is sort of a prerequisite for asking thousands of people to answer one's > >> question. But we can't enforce that. > >> > >> > >> > >> The situation will get better to the extent people ask better questions, > >> help other people ask better questions, and answer good questions. I'd > >> encourage anyone feeling this way to try to help along those dimensions. > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> On Wed, Nov 2, 2016 at 11:32 AM assaf.mendelson <[hidden email]> > >> wrote: > >> > >> Hi, > >> > >> I know this is a little off topic but I wanted to raise an issue about > >> handling questions in the mailing list (this is true both for the user > >> mailing list and the dev but since there are other options such as stack > >> overflow for user questions, this is more problematic in dev). > >> > >> Let’s say I ask a question (as I recently did). Unfortunately this was > >> during spark summit in Europe so probably people were busy. In any case > no > >> one answered. > >> > >> The problem is, that if no one answers very soon, the question will > almost > >> certainly remain unanswered because new messages will simply drown it. > >> > >> > >> > >> This is a common issue not just for questions but for any comment or > idea > >> which is not immediately picked up. > >> > >> > >> > >> I believe we should have a method of handling this. > >> > >> Generally, I would say these types of things belong in stack overflow, > >> after all, the way it is built is perfect for this. More seasoned spark > >> contributors and committers can periodically check out unanswered > questions > >> and answer them. > >> > >> The problem is that stack overflow (as well as other targets such as the > >> databricks forums) tend to have a more user based orientation. This > means > >> that any spark internal question will almost certainly remain > unanswered. > >> > >> > >> > >> I was wondering if we could come up with a solution for this. > >> > >> > >> > >> Assaf. > >> > >> > >> > >> > >> > >> ________________________________ > >> > >> View this message in context: Handling questions in the mailing lists > >> Sent from the Apache Spark Developers List mailing list archive at > >> Nabble.com > <https://urldefense.proofpoint.com/v2/url?u=http-3A__nabble.com&d=DQMFaQ&c=dCBwIlVXJsYZrY6gpNt0LA&r=B8E4n9FrSS85mPCi6Mfs7cyEPQnVrpcQ1zeB-JKws6A&m=Vf-yZoTpLgwZzwUCoQTMr4UFD_R0nx0naxh_SWUHfho&s=DOSXyWQ25VrvEJ61e9vezaFFqQ6ERTNkf2btm8y3JEA&e=> > . > > --------------------------------------------------------------------- > To unsubscribe e-mail: [hidden email]rg > > > > > > > > > > > > > > ------------------------------ > > *If you reply to this email, your message will be added to the discussion > below:* > > http://apache-spark-developers-list.1001551.n3. > nabble.com/Handling-questions-in-the-mailing-lists-tp19690p19757.html > <https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dspark-2Ddevelopers-2Dlist.1001551.n3.nabble.com_Handling-2Dquestions-2Din-2Dthe-2Dmailing-2Dlists-2Dtp19690p19757.html&d=DQMFaQ&c=dCBwIlVXJsYZrY6gpNt0LA&r=B8E4n9FrSS85mPCi6Mfs7cyEPQnVrpcQ1zeB-JKws6A&m=Vf-yZoTpLgwZzwUCoQTMr4UFD_R0nx0naxh_SWUHfho&s=ZX7C3jXG0WAWbui6GXRkT15WDj5s6Yb9U_uCYr0p7Ew&e=> > > To start a new topic under Apache Spark Developers List, email [hidden > email] > To unsubscribe from Apache Spark Developers List, click here. > NAML > <https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dspark-2Ddevelopers-2Dlist.1001551.n3.nabble.com_template_NamlServlet.jtp-3Fmacro-3Dmacro-5Fviewer-26id-3Dinstant-5Fhtml-2521nabble-253Aemail.naml-26base-3Dnabble.naml.namespaces.BasicNamespace-2Dnabble.view.web.template.NabbleNamespace-2Dnabble.view.web.template.NodeNamespace-26breadcrumbs-3Dnotify-5Fsubscribers-2521nabble-253Aemail.naml-2Dinstant-5Femails-2521nabble-253Aemail.naml-2Dsend-5Finstant-5Femail-2521nabble-253Aemail.naml&d=DQMFaQ&c=dCBwIlVXJsYZrY6gpNt0LA&r=B8E4n9FrSS85mPCi6Mfs7cyEPQnVrpcQ1zeB-JKws6A&m=Vf-yZoTpLgwZzwUCoQTMr4UFD_R0nx0naxh_SWUHfho&s=ZSjjNhkCGzaVsA9UhbpYXaKujat6vR7r6SdBFmyzWdc&e=> > > > ------------------------------ > > View this message in context: RE: Handling questions in the mailing lists > <https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dspark-2Ddevelopers-2Dlist.1001551.n3.nabble.com_Handling-2Dquestions-2Din-2Dthe-2Dmailing-2Dlists-2Dtp19690p19758.html&d=DQMFaQ&c=dCBwIlVXJsYZrY6gpNt0LA&r=B8E4n9FrSS85mPCi6Mfs7cyEPQnVrpcQ1zeB-JKws6A&m=Vf-yZoTpLgwZzwUCoQTMr4UFD_R0nx0naxh_SWUHfho&s=ElcVsDry_U8-DCD_Awaa6kExNKJo0gQ6Dpbp0HNMUUI&e=> > Sent from the Apache Spark Developers List mailing list archive > <https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dspark-2Ddevelopers-2Dlist.1001551.n3.nabble.com_&d=DQMFaQ&c=dCBwIlVXJsYZrY6gpNt0LA&r=B8E4n9FrSS85mPCi6Mfs7cyEPQnVrpcQ1zeB-JKws6A&m=Vf-yZoTpLgwZzwUCoQTMr4UFD_R0nx0naxh_SWUHfho&s=dXRHr7SeGMH3ksDwIya1xGEJGhVRQU4TYLf3dun_L5k&e=> > at Nabble.com > <https://urldefense.proofpoint.com/v2/url?u=http-3A__nabble.com&d=DQMFaQ&c=dCBwIlVXJsYZrY6gpNt0LA&r=B8E4n9FrSS85mPCi6Mfs7cyEPQnVrpcQ1zeB-JKws6A&m=Vf-yZoTpLgwZzwUCoQTMr4UFD_R0nx0naxh_SWUHfho&s=DOSXyWQ25VrvEJ61e9vezaFFqQ6ERTNkf2btm8y3JEA&e=> > . > > > > This e-mail (including any attachments) is private and confidential, may > contain proprietary or privileged information and is intended for the named > recipient(s) only. Unintended recipients are strictly prohibited from > taking action on the basis of information in this e-mail and must contact > the sender immediately, delete this e-mail (and all attachments) and > destroy any hard copies. Nomura will not accept responsibility or liability > for the accuracy or completeness of, or the presence of any virus or > disabling code in, this e-mail. If verification is sought please request a > hard copy. Any reference to the terms of executed transactions should be > treated as preliminary only and subject to formal written confirmation by > Nomura. Nomura reserves the right to retain, monitor and intercept e-mail > communications through its networks (subject to and in accordance with > applicable laws). No confidentiality or privilege is waived or lost by > Nomura by any mistransmission of this e-mail. Any reference to "Nomura" is > a reference to any entity in the Nomura Holdings, Inc. group. Please read > our Electronic Communications Legal Notice which forms part of this e-mail: > http://www.Nomura.com/email_disclaimer.htm >
