Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/14673
Thanks @srowen based on that I'll just open it to the 2.0 branch then
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14673
Ah forgot to comment on the substance: normally we probably wouldn't
back-port a feature to a maintenance branch. This is pretty much a feature. If
it's low-risk and someone wants to make the case
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14673
You would also have to check out branch-2.0 and branch from there in order
to work off the right base. From there you can cherry-pick your commit from
master and resolve conflicts. Then there's
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/14673
ok, thanks, I'll open those pr's once I can cleanly merge the patch
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/14673
when you do the pr to apache/spark just pick branch-2.0 instead of master.
I see things going into branch-1.6 also but I haven't seen any discussion on
the list about doing another release so we
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/14673
@tgravescs would would I go about opening a pr for older branches? I've
only done work on the latest code before. Also is the 1.6.x branch still
updating or is 2.0.x and 2.1 the only updating
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/14673
I merged this into 2.1.0, if you are wanting this in older ones can you put
up PR for those, cherry-pick to branch-2.0 wasn't clean.
---
If your project is set up for it, you can reply to this
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/14673
+1, I'll merge shortly
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/14673
sorry @ajbozarth somehow my mail didn't show me you updated it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/14673
@srowen @vanzin or @JoshRosen could one of you take a look at this? I've
already addressed @tgravescs feedback, and would love to see this go through
soon if it looks good to you, thanks
---
If
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/14673
secondary sort in this case is whatever order the collection is stored in,
since `LinkedHashMap` keeps an order (essentially by `tasked`, but really by
insertion order) `tasked` becomes the
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/14673
code changes look good, does it change the secondary sort of taskId in a
bad way? Looking at your json files it looks like they were in order before
and now they aren't.
---
If your project is
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14673
**[Test build #64022 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64022/consoleFull)**
for PR 14673 at commit
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/14673
Pushed my latest updates including the change to using `ConfigEntry` and
updating the test cases (using `LinkedHashMap` changes the secondary sort to
`taskId`)
---
If your project is set up for
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14673
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63971/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14673
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14673
**[Test build #63971 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63971/consoleFull)**
for PR 14673 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14673
**[Test build #63971 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63971/consoleFull)**
for PR 14673 at commit
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/14673
ran into a odd test failure in HistoryServer I still haven't figured out,
I'll push an update to fix it tomorrow
---
If your project is set up for it, you can reply to this email and have your
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/14673
@tgravescs I've addressed some issues: I changed the default max to 10
and removed the on/off. I also updated the `taskData` to be a `LinkedHashMap`
instead of a `HashMap` which allows for
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/14673
yes the queue filling up and dropping events is a separate issue there are
other jiras filed for it.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14673
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14673
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63874/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14673
**[Test build #63874 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63874/consoleFull)**
for PR 14673 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14673
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14673
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63876/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14673
**[Test build #63876 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63876/consoleFull)**
for PR 14673 at commit
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/14673
@vanzin and @tgravescs here's my update of pr #12990 by @tankkyo if you wan
to take a look, I raised a few discussion points above
---
If your project is set up for it, you can reply to this
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/14673
Pushed a update to the text so it's clearer in case of the above bug:
**
for PR 14673 at commit
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/14673
Found a odd behavior while testing. When a application encounters an issue
(in my case my task queue filled up and SparkListenerBus was stopped) it can
cause the tasks HashMap size to be larger
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14673
**[Test build #63874 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63874/consoleFull)**
for PR 14673 at commit
32 matches
Mail list logo
