Solving it socially is of course ideal. We do already likewise point everyone to http://spark.apache.org/contributing.html . I think this is about what to do in extreme cases. It's a minor point of workflow, but, seems like there's no particular need to let anyone reopen any issue.
Speaking to the particular issue, maybe the error can be improved, but it already shows ClosureCleaner calling ensureSerializable and saying "task not serializable" and the object in question. That wasn't quite the issue though, but rather that it was a) an extended question about how batches are processed mixed with b) basic misapprehensions about Spark and c) unacceptable tone for an OSS community from start to finish. Closing it is the right resolution for several reasons, and I don't see that a better exception message would have done anything. On Thu, Oct 5, 2017 at 11:38 AM Steve Loughran <ste...@hortonworks.com> wrote: > It's a losing battle which you need to deal with socially rather than > through the tooling...if the user is unhappy then at best they don't use > the code, don't contribute in future, at worse: they keep filing the same > JIRA. I'll add a comment > > In hadoop we've ended up creating a wiki page added as a link when closing > things as an invalid, usually with a polite let down "sorry" > > https://wiki.apache.org/hadoop/InvalidJiraIssues > > and a video to go into detail > > https://www.youtube.com/watch?v=NaJlRk5aTRQ > > > There's something to consider though: how can error reporting be improved? > Especially for people new to a system? > > > Serialization errors are ubiquitous in spark when you call RDDs with > unserializable data; the first thing people learn when they start writing > them is "this stack trace means I'm invoking something which can't go over > the wire". So: how to help people over the hump there? catch & wrap with a > pointer to some docs. For networking, feel free to use > org.apache.hadoop.net.NetUtils#wrapException , where the links to wiki > entries are lists of "common causes of these networking issues" are a > checklist for everyone; the real facts of hostnames and ports are there for > tracking things down. The core Java io networking errors are without > meaningful information, so it's up to the layers above to fix. > > > > On 5 Oct 2017, at 03:51, Sean Owen <so...@cloudera.com> wrote: > > Although I assume we could get an account suspended if it started opening > spam issues, yes we default to letting anyone open issues, and potentially > abusing it. That much is the right default and I don't see any policy tweak > that stops that. > > I see several INFRA tickets asking to *allow* the Closed -> Reopened > transition, which suggests it's not the default. > https://issues.apache.org/jira/browse/INFRA-11857?jql=project%20%3D%20INFRA%20AND%20text%20~%20%22reopen%20JIRA%22 > > I'm accustomed to Closed being a final state that nobody can reopen as a > matter of workflow -- the idea being that anything else should be a new > discussion if the current issue was deemed formally done. > > Spark pretty much leaves all issues in "Resolved" status which can still > be reopened, and I think that's right. Although I'd like to limit all > reopening to committers, it isn't that important. > > Being able to move a JIRA to Closed permanently seems useful, as it > doesn't interfere with any normal workflow, doesn't actually prevent a new > issue from succeeding it in normal usage, and gives another tool to limit a > specific kind of abuse. > > On Thu, Oct 5, 2017 at 3:28 AM Dongjoon Hyun <dongjoon.h...@gmail.com> > wrote: > >> It can stop reopening, but new JIRA issues with duplicate content will be >> created intentionally instead. >> >> Is that policy (privileged reopening) used in other Apache communities >> for that purpose? >> >> >> On Wed, Oct 4, 2017 at 7:06 PM, Sean Owen <so...@cloudera.com> wrote: >> >>> We have this problem occasionally, where a disgruntled user continually >>> reopens an issue after it's closed. >>> >>> https://issues.apache.org/jira/browse/SPARK-21999 >>> >>> (Feel free to comment on this one if anyone disagrees) >>> >>> Regardless of that particular JIRA, I'd like to disable to Closed -> >>> Reopened transition for non-committers: >>> https://issues.apache.org/jira/browse/INFRA-15221 >>> >>> >> >
