[GitHub] roadan closed pull request #20: PySpark fixes for YARN and Mesos

roadan closed pull request #20: PySpark fixes for YARN and Mesos URL: https://github.com/apache/incubator-amaterasu/pull/20 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull

Re: Speeding up Anaconda deployment on executors

I am kinda inclined towards a solution that could be performed at setup time. Let's try to explore in that direction. If we manage to nail this at setup time, it means that the runtime will be lightning fast (compared to what it is now :)) Cheers, Nadav On 28 May 2018 at 00:17, Nadav Har Tzvi w

Speeding up Anaconda deployment on executors

Hey everyone, So we have this issue, Anaconda takes forever to deploy on the executors, whether it is YARN or Mesos. Let's first discuss why is it like this right now. First, let's see for each platform, how Apache Amaterasu interacts with the underlying platform, in regard to what smallest inde

Re: AMATERASU-24

Hi Yaniv, Nadiv and all, Not sure if PR is the best way to initiate a discussion on the code but I just managed to raise one based on my forked remote branch. https://github.com/apache/incubator-amaterasu/pull/22 https://github.com/arunma/incubator-amaterasu/tree/AMATERASU-24-FrameworkRefactor I

[jira] [Commented] (AMATERASU-24) Refactor Spark out of Amaterasu executor to it's own project

[ https://issues.apache.org/jira/browse/AMATERASU-24?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16491973#comment-16491973 ] ASF GitHub Bot commented on AMATERASU-24: - arunma opened a new pull request #22:

Re: Amaterasu PyCon il Presentation

Looks great !! I suppose this will be accompanied by a Demo. On Sun, May 27, 2018 at 5:49 PM Yaniv Rodenski wrote: > Hi All, > > @Nadav is giving an intro to Amaterasu @ PyCon Israel next week and we > thought everyone here might like to have the deck > < > https://drive.google.com/file/d/1rGL

[GitHub] arunma opened a new pull request #22: AMATERASU-24 Refactor Spark out of Amaterasu executor to it's own pro…

arunma opened a new pull request #22: AMATERASU-24 Refactor Spark out of Amaterasu executor to it's own pro… URL: https://github.com/apache/incubator-amaterasu/pull/22 …ject This is an automated message from the Apache Git Se

Amaterasu PyCon il Presentation

Hi All, @Nadav is giving an intro to Amaterasu @ PyCon Israel next week and we thought everyone here might like to have the deck and may be present somewhere :) Cheers, -- Yaniv Rodenski +61 477 778 405 ya...@

Re: AMATERASU-24

Hi Nadav, Absolutely. This sounds fantastic. I am really keen to understand more on this and would be happy to help where I can to move this forward. Yes, It would be great if we could have a different discussion thread on this (or preferably a meeting). Let me quickly push a branch for your re

Re: AMATERASU-24

Hi Yaniv, Makes perfect sense. Non-JVM frameworks is something I hadn't considered. I haven't done something like this in the past and would require your guidance. Would it be okay if we have a discussion over a meeting? Two modules - Yes. I have made changes in that direction but have kept both