Re: Initial setup of Amaterasu - Unable to run

2017-09-30 Thread Yaniv Rodenski
Hi Arun, I think you are hitting a bug that we’ve fixed but was in a pending PR, I've just merged the PR. Try to do a git pull and run again, let us know if it solves the problem. Cheers, Yaniv On Sat, 30 Sep 2017 at 7:36 pm, Arun Manivannan wrote: > Hi, > > I am trying to

Re: Initial setup of Amaterasu - Unable to run

2017-09-30 Thread Yaniv Rodenski
ibly sorry if these questions doesn't make much sense in the > context of the project. I would just like to know if I had misunderstood > the purpose of the project. I absolutely realise that the project is just > incubating and it's too much to ask for all the bells and whistles o

Re: Write access to the incubator-amaterasu @ GitHub

2017-11-26 Thread Yaniv Rodenski
fré <j...@nanthrax.net> wrote: > Hi, > > we can't see image on the mailing list. Can you send the actual message ? > > As we are on gitbox, github or github should work. > > Regards > JB > > On 11/26/2017 02:40 AM, Yaniv Rodenski wrote: > >> Hi Mentors, &

Write access to the incubator-amaterasu @ GitHub

2017-11-25 Thread Yaniv Rodenski
Hi Mentors, I'm trying to merge a PR via GitHub and seeing this message:[image: Inline image 1] It looks like that while I do have access to gitbox I have no write access to the GitHub mirror. How can I get it? Cheers, -- Yaniv Rodenski

Re: Next steps - Planning

2017-11-22 Thread Yaniv Rodenski
Hi All, Just one last reminder, if you can answer the poll we would really like to have as many of us at the meeting as possible Cheers Yaniv On Sat, Nov 18, 2017 at 11:16 AM, Yaniv Rodenski <ya...@shinto.io> wrote: > Hi All, > > As you can probably see, I've started migrating

Re: podling report

2017-12-13 Thread Yaniv Rodenski
utes reported back to the list, and most ... > > > > Ok to push to the report. > > > -- Yaniv Rodenski

Re: Unable to run the sample job

2017-11-01 Thread Yaniv Rodenski
0.0.0.0:8000} > 2017-10-31 10:53:37.732:INFO:oejsh.ContextHandler:Thread-73: Stopped > o.e.j.s.ServletContextHandler@8c3b9d{/,file:/home/shad/apps/ > apache-amaterasu-0.2.0-incubating/dist/,UNAVAILABLE} > I1031 10:53:37.733896 36249 sched.cpp:2021] Asked to stop the driver > I1031 10:5

Re: Podling Report Reminder - December 2017

2017-12-10 Thread Yaniv Rodenski
ittle before > >> this page is created from a template. > >> > >> Mentors > >> --- > >> > >> Mentors should review reports for their project(s) and sign them off on > >> the Incubator wiki page. Signing off reports shows that you are > >> following the project - projects that are not signed may raise alarms > >> for the Incubator PMC. > >> > >> Incubator PMC > >> > > > > -- > Jean-Baptiste Onofré > jbono...@apache.org > http://blog.nanthrax.net > Talend - http://www.talend.com > -- Yaniv Rodenski

podling report

2017-12-11 Thread Yaniv Rodenski
r its first release in the incubator Date of last release: N/A When were the last committers or PMC members elected? N/A " If there are no objections I will need help updating the wiki as I still don't have access. Cheers, -- Yaniv Rodenski

Re: Write access to the incubator-amaterasu @ GitHub

2017-12-11 Thread Yaniv Rodenski
ake sure you've linked your GitHub and ASF accounts at > https://gitbox.apache.org/. > > On Sun, Nov 26, 2017 at 12:47 AM, Yaniv Rodenski <ya...@shinto.io> wrote: > > > Hi JB, > > > > I get the following in GitHub: > > This branch has no conflicts with t

Re: Apache Amaterasu planning meeting notes - actions required

2017-12-10 Thread Yaniv Rodenski
Hi John, I believe @JB is looking into our permissions issue. Regards, Yaniv On Mon, Dec 11, 2017 at 1:25 AM, John D. Ament <johndam...@apache.org> wrote: > Yaniv, > > Did you ever receive a response to this? > > John > > On 2017-11-26 05:53, Yaniv Rodenski <ya..

Re: Write access to the incubator-amaterasu @ GitHub

2017-12-11 Thread Yaniv Rodenski
ter is empty, so I populated it from the > project status page (modulo Karel, who doesn't seem to have an account). > This might solve the problem -- can you try again? > > On Mon, Dec 11, 2017 at 4:00 AM, Yaniv Rodenski <ya...@shinto.io> wrote: > > > Hi Davor, >

podling report

2017-10-30 Thread Yaniv Rodenski
t box * One new pull request was issued Date of last release: N/A When were the last committers or PMC members elected? N/A " If there are no objections I will update the wiki. Cheers, -- Yaniv Rodenski

Re: Unable to run the sample job

2017-10-30 Thread Yaniv Rodenski
h(Iterator.scala:891) > at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) > at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) > at scala.collection.AbstractIterable.foreach(Iterable.scala:54) > at > org.apache.amaterasu.leader.mesos.schedulers.JobScheduler. > resourceOffers(JobScheduler.scala:128) > I1030 14:00:12.991056 16624 sched.cpp:2055] Asked to abort the driver > I1030 14:00:12.991195 16624 sched.cpp:1233] Aborting framework > e72b9609-4b7f-4509-b1a4-bd2055d674aa-0002 > > Thanks, > Shad > -- Yaniv Rodenski

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #2

2018-05-06 Thread Yaniv Rodenski
No worries. This will take a number of tries -- completely normal. > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

License questions for RC3

2018-05-14 Thread Yaniv Rodenski
has a BSD license. Is that an issue? if so, an alternative approach would be to install it via pip during Amaterasu's setup stage, is that a valid solution? Thanks in advance, -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Mail archive test

2018-06-14 Thread Yaniv Rodenski
Hi All, I'm just checking going directly via dev@amaterasu and not dev@incubator.amaterasu since I can't see the result email in the archive yet -- Yaniv Rodenski

[RESULT][VOTE] Amaterasu release 0.2.0-incubating, release candidate #3

2018-06-14 Thread Yaniv Rodenski
. I will start the voting on the @general mailing list ASAP Thanks everyone! Yaniv Rodenski

Vote at the general@ mailing list

2018-06-17 Thread Yaniv Rodenski
Hi Mentors, Just a quick etiquette question, as we are over 72 hours into the vote and only JB has voted, what (and when) would be the best way to request more votes? I understand people are busy, when do we consider a vote to be "forgotten" and in need of a reminder? Cheers, -- Yani

Re: AMATERASU-24

2018-05-26 Thread Yaniv Rodenski
ation > master can use it. > > Is it okay if I rename the Miniconda install file to miniconda-install > using the "wget -O". The reason why this change is proposed is to avoid > hardcoding the conda version inside the code and possibly pull it away into > amaterasu.properties

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #3

2018-05-30 Thread Yaniv Rodenski
sion > > > > > > > > > 0.2.0-incubating, as follows > > > > > > > > > > > > > > > > > > [ ] +1, Approve the release > > > > > > > > > [ ] -1, Do not approve the release (please provide specific > > > > > comments) > > > > > > > > > > > > > > > > > > The complete staging area is available for your review, > which > > > > > > includes: > > > > > > > > > > > > > > > > > > * JIRA release notes [1], > > > > > > > > > * the official Apache source release to be deployed to > > > > > > dist.apache.org > > > > > > > > > [2], > > > > > > > > > which is signed with the key with fingerprint [3], > > > > > > > > > * source code tag "version-0.2.0-incubating-rc3" [4], > > > > > > > > > * Java artifacts were built with Gradle 3.1 and > > OpenJDK/Oracle > > > > JDK > > > > > > > > > 1.8.0_151 > > > > > > > > > > > > > > > > > > The vote will be open for at least 72 hours. It is adopted > by > > > > > > majority > > > > > > > > > approval, with at least 3 PMC affirmative votes. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Yaniv > > > > > > > > > > > > > > > > > > [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa > ? > > > > > > > > > projectId=12321521=12342793 > > > > > > > > > [2] https://dist.apache.org/repos/ > > > dist/dev/incubator/amaterasu/ > > > > > > > 0.2.0rc3/ > > > > > > > > > [3] > > > > > https://dist.apache.org/repos/dist/dev/incubator/amaterasu/KEYS > > > > > > > > > [4] https://github.com/apache/incubator-amaterasu/tags > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Amaterasu PyCon il Presentation

2018-05-27 Thread Yaniv Rodenski
Hi All, @Nadav is giving an intro to Amaterasu @ PyCon Israel next week and we thought everyone here might like to have the deck <https://drive.google.com/file/d/1rGLBPw3hkpd0ZVQrJGZWqHf6gtBk3Ztv/view?usp=sharing> and may be present somewhere :) Cheers, -- Yaniv Rodenski +61 477 778

[VOTE] Amaterasu release 0.2.0-incubating, release candidate #3

2018-05-28 Thread Yaniv Rodenski
Hi everyone, We have fixed the legal issues, as well as a bug found by @Nadav please review and vote on the release candidate #3 for the version 0.2.0-incubating, as follows [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging

Re: Speeding up Anaconda deployment on executors

2018-05-29 Thread Yaniv Rodenski
ition it gives us > the nifty Conda package manager. > However, it doesn't mean that it has to stay that way. If the need or > reason arises, we may need to also support pip and support using the native > Python version (instead of the one supplied by Anaconda). > > I want to discuss the possible solutions to this. Please feel free to bring > up your ideas. > > Cheers, > Nadav > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #3

2018-06-03 Thread Yaniv Rodenski
gt; > Disabled > >> > > > physical memory check (yarn.nodemanager.pmem-check-enabled) and > ran > >> > jobs > >> > > > successfully. > >> > > > > >> > > > > >> > > > > >> > > > &g

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #3

2018-06-04 Thread Yaniv Rodenski
> Cheers, > Arun > > On Sun, Jun 3, 2018 at 6:37 PM Yaniv Rodenski wrote: > > > Hi Arun, > > > > Fair point, but I think it can be configured via env/[env_name]/spark.yml > > am I wrong? > > Anyway, ideally I think we should try and have per

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #3

2018-06-01 Thread Yaniv Rodenski
One thing to clarify, the tar.gz is the source code as I've seen this is released in other projects. if that is not needed I'm happy to drop it from future releases. On Fri, Jun 1, 2018 at 5:01 PM, Yaniv Rodenski wrote: > Thanks Davor, > > We will fix those for the next release. &

[RESULT][VOTE] Amaterasu release 0.2.0-incubating, release candidate #4

2018-06-30 Thread Yaniv Rodenski
I'm happy to announce that we have unanimously approved this release. There are 8 approving votes, 3 of which are binding: * Nadav Har Tzvi * Kirupa Devarajan * Guy Peleg * Arun Manivannan * Eyal Ben Ivri IPMC votes * Davor Bonaci * Olivier Lamy * Jean-Baptiste Onofré There are no

[VOTE] Release Apache Amaterasu (incubating) 0.2.0 (rc4)

2018-06-23 Thread Yaniv Rodenski
Hi everyone, After cancelling the vote in the general@ list we've fixed the following: * Headers added to all non-code files where applicable as remarked by Davor * Sources now match the released version + no keys are present * The gradle-wrapper.jar was removed and instructions ware added to the

Re: Cancelling the vote in the @general mailing list

2018-06-20 Thread Yaniv Rodenski
is public domain, there are > typically options as well. I suggest understanding the genesis of software > in question. > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Cancelling the vote in the @general mailing list

2018-06-19 Thread Yaniv Rodenski
have one. what can we do in that case? Thanks in advance -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #2

2018-04-30 Thread Yaniv Rodenski
; Hi guys, > > > > I'm just back from vacation. I will review the release. > > > > Regards > > JB > > > > On 04/29/2018 12:44 PM, Yaniv Rodenski wrote: > > > Hi Mentors, > > > > > > In order to progress with the vote, can any of you

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #2

2018-04-29 Thread Yaniv Rodenski
previous issues were resolved , in > > addition we need short feedback loop from users. > > > > On Thu, Apr 26, 2018, 08:48 Kirupa Devarajan <kirupagara...@gmail.com> > > wrote: > > > > > +1 - Test and Build successful for release candidate #2 >

Moving forward

2017-10-22 Thread Yaniv Rodenski
? Also we are very close to version 0.2.0-incubating. not a lot of open tasks, but I think we should also get our JIRA in order, I can port everything we have on our old trello, but maybe a hangouts sync to go over the open tasks is a good way to push things forward? Cheers, -- Yaniv Rodenski

Re: Moving forward

2017-10-28 Thread Yaniv Rodenski
gt; Actually, after I am done :) > > > On 26 Oct 2017, at 2:48, Yaniv Rodenski <ya...@shinto.io> wrote: > > > > OK so GitHub doesn't allow forking empty branches, so I think I'll just > > push the shintoio master to the amaterasu-incubator master as Olivier > > sug

[DISCUSS] Monthly report and getting ready to release 0.2.0 incubating

2018-02-10 Thread Yaniv Rodenski
wiki looks like the user I've re-created recently got deleted again :( Cheers, -- Yaniv Rodenski

Re: [DISCUSS] release procedure

2018-02-18 Thread Yaniv Rodenski
stated that the release > >requires the approval of 3 IPMC members, is that correct? > > > > 3 PPMC should approve before presenting to Incubator. 3 IPMC members then > to release. > -- Yaniv Rodenski

Podling report

2018-01-02 Thread Yaniv Rodenski
need help updating the wiki looks like the user I've re-created recently got deleted again :( Cheers, -- Yaniv Rodenski

Re: Next steps?

2017-12-26 Thread Yaniv Rodenski
t; mentor attention. > > > > Otherwise, the report is due 1/3. Yaniv, do you have everything you need > > there? > > > > Thanks! > > > > Davor > > > -- Yaniv Rodenski

[DISCUSS] podling report

2018-07-06 Thread Yaniv Rodenski
Hi All, Sorry for doing this late again, but I propose the following report to be submitted: " Apache Amaterasu is a framework providing configuration management and deployment for Big Data Pipelines. It provides the following capabilities: Continuous integration tools to package pipelines and

Re: [VOTE] Release Apache Amaterasu (incubating) 0.2.0 (rc4)

2018-07-10 Thread Yaniv Rodenski
mod=license=2=57>/resources < > > http://localhost:8081/repo/?mod=license=2=58>/codegen.py > > 2. incubator-amaterasu-version-0.2.0-incubating-rc4/executor/ > > src/test/resources/py4j-0.10.4-src.zip > > 3. incubator-amaterasu-version-0.2.0-incubating-rc4/executor/ > >

[DISCUSS] release version 0.2.0-incubating

2018-04-05 Thread Yaniv Rodenski
would like to proceed with the release in the next couple of days. Cheers, -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

[VOTE] Amaterasu release 0.2.0-incubating, release candidate #2

2018-04-24 Thread Yaniv Rodenski
nd OpenJDK/Oracle JDK 1.8.0_151 The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Thanks, Yaniv Rodenski [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12321521=12342793 [2] https://dist.apache.org/repo

[DISCUSS] podling report

2018-04-02 Thread Yaniv Rodenski
butor communities 3. Prepare documentation Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of? None How has the community developed since the last report? * Two conference talks are scheduled in the near future, Voxxed Days Melbourne (Yaniv Rodenski) and SDP in Tel

Re: Project status

2018-03-20 Thread Yaniv Rodenski
> project, and graduation from Incubator seems hard right now. > > I know this may be difficult, but I think it is better to talk about these > things openly and investigate what can be done and/or whether the project > would be better served as a standalone project in GitHub. &

Re: Project status

2018-03-20 Thread Yaniv Rodenski
erm work is done by one more contributor on his local fork (Nadav > > Har-Tzvi) > > > > I think this is the crux of the problem. Why is longer-term work going on > in a local fork? > > > > * We should be presenting more, and growing the community more which is > &

Re: [DISCUSS] Dependencies resolution and action level dependencies

2018-10-23 Thread Yaniv Rodenski
ach the YARN implementation. Cheers, Yaniv On Tue, Oct 23, 2018 at 9:51 PM Yariv Triffon wrote: > Hi Yaniv, > i'm good to grab the task of moving Mesos to use Docker containers. > > Cheers, > Yariv > > On Tue, Oct 23, 2018 at 5:13 PM Yaniv Rodenski wrote: > > > Th

Re: AMATERASU-44

2018-10-10 Thread Yaniv Rodenski
fore making the > change. > > > Cheers, > Arun > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Re: [DISCUSS] podling report

2018-10-03 Thread Yaniv Rodenski
Hi All, I've submitted the report with all the remarks applied. Yaniv On Thu, Oct 4, 2018 at 6:11 AM Justin Mclean wrote: > Hi, > > The report has not been submitted, please do so as soon as you. > > Thanks, > Justin > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Re: Podling Report Reminder - April 2019

2019-03-31 Thread Yaniv Rodenski
page is created from a template. > > > > Mentors > > --- > > > > Mentors should review reports for their project(s) and sign them off on > > the Incubator wiki page. Signing off reports shows that you are > > following the project - projects that are not signed may raise alarms > > for the Incubator PMC. > > > > Incubator PMC > > > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Re: [Discuss] datasets input file in user's repository

2019-03-18 Thread Yaniv Rodenski
sets. > In that case, how will the user submit datasets configuration? Is it > another file next to the maki.yaml? Is it a file that resides in the > environment, e.g. next to the env.yaml? Is it not even a file on its own > but a part of the env.yaml? > Ideas, anyone? > >

Re: Podling Report Reminder - July 2019

2019-06-23 Thread Yaniv Rodenski
fluence/display/INCUBATOR/July2019 > > > > Note: This is manually populated. You may need to wait a little before > > this page is created from a template. > > > > Note: The format of the report has changed to use markdown. > > > > Mentors > > --- > > > > Mentors should review reports for their project(s) and sign them off on > > the Incubator wiki page. Signing off reports shows that you are > > following the project - projects that are not signed may raise alarms > > for the Incubator PMC. > > > > Incubator PMC > > > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

[DISCUSS] podling report - Amaterasu

2019-07-01 Thread Yaniv Rodenski
Hi All, I propose the following report to be submitted. Amaterasu Apache Amaterasu is a framework providing continuous deployment for Big Data pipelines. Amaterasu has been incubating since 2017-09-07. Three most

[jira] [Updated] (AMATERASU-6) Long running actions/pipelines

2017-11-26 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-6?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-6: --- Description: As a pipeline developer, I can define pipelines and actions to be long

[jira] [Updated] (AMATERASU-7) Scheduled pipelines/actions

2017-11-26 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-7?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-7: --- Description: As a pipeline developer, I can define a schedule in the maki.yml for my

[jira] [Updated] (AMATERASU-7) Scheduled pipelines/actions

2017-11-26 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-7?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-7: --- Description: As a pipeline developer, I can define a schedule in the maki.yml for my

[jira] [Created] (AMATERASU-11) Monitoring UI

2017-11-17 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-11: --- Summary: Monitoring UI Key: AMATERASU-11 URL: https://issues.apache.org/jira/browse/AMATERASU-11 Project: AMATERASU Issue Type: New Feature

[jira] [Updated] (AMATERASU-9) Nested pipelines

2017-11-17 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-9?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-9: --- Issue Type: Improvement (was: Bug) > Nested pipeli

[jira] [Created] (AMATERASU-12) Spark framework - support for spark streaming

2017-11-17 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-12: --- Summary: Spark framework - support for spark streaming Key: AMATERASU-12 URL: https://issues.apache.org/jira/browse/AMATERASU-12 Project: AMATERASU

[jira] [Created] (AMATERASU-23) Add executor classpath from framework configuration

2018-05-14 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-23: --- Summary: Add executor classpath from framework configuration Key: AMATERASU-23 URL: https://issues.apache.org/jira/browse/AMATERASU-23 Project: AMATERASU

[jira] [Created] (AMATERASU-24) Refactor Spark out of Amaterasu executor to it's own project

2018-05-15 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-24: --- Summary: Refactor Spark out of Amaterasu executor to it's own project Key: AMATERASU-24 URL: https://issues.apache.org/jira/browse/AMATERASU-24 Project

[jira] [Assigned] (AMATERASU-33) Upgrade Gradle version to 4.8

2018-06-09 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-33?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-33: --- Assignee: Yaniv Rodenski > Upgrade Gradle version to

[jira] [Created] (AMATERASU-33) Upgrade Gradle version to 4.8

2018-06-09 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-33: --- Summary: Upgrade Gradle version to 4.8 Key: AMATERASU-33 URL: https://issues.apache.org/jira/browse/AMATERASU-33 Project: AMATERASU Issue Type

[jira] [Assigned] (AMATERASU-23) Add executor classpath from framework configuration

2018-06-09 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-23?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-23: --- Assignee: Yaniv Rodenski > Add executor classpath from framework configurat

[jira] [Updated] (AMATERASU-23) Add executor classpath from framework configuration - Mesos

2018-06-18 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-23?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-23: Summary: Add executor classpath from framework configuration - Mesos (was: Add executor

[jira] [Created] (AMATERASU-36) Refactoring - leader: extract projects per resource manager + common

2018-06-18 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-36: --- Summary: Refactoring - leader: extract projects per resource manager + common Key: AMATERASU-36 URL: https://issues.apache.org/jira/browse/AMATERASU-36 Project

[jira] [Created] (AMATERASU-35) Generate the container command from the framework providere

2018-06-17 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-35: --- Summary: Generate the container command from the framework providere Key: AMATERASU-35 URL: https://issues.apache.org/jira/browse/AMATERASU-35 Project

[jira] [Closed] (AMATERASU-21) Fix Spark scala tests

2018-06-17 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-21?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski closed AMATERASU-21. --- Resolution: Fixed > Fix Spark scala tests > - > >

[jira] [Resolved] (AMATERASU-26) Pipeline tasks (sub-Yarn jobs) runs as "yarn" user instead of inhering the user in which the amaterasu job was submitted

2018-06-08 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-26?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski resolved AMATERASU-26. - Resolution: Fixed > Pipeline tasks (sub-Yarn jobs) runs as "yarn"

[jira] [Updated] (AMATERASU-13) make ama_context available in SparkSQL

2017-12-25 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-13?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-13: Issue Type: Improvement (was: Bug) > make ama_context available in Spark

[jira] [Updated] (AMATERASU-13) make ama_context available in SparkSQL

2017-12-25 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-13?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-13: Description: SparkSQL actions runner should support accessing datasets via ama_context

[jira] [Created] (AMATERASU-13) make ama_context available in SparkSQL

2017-12-25 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-13: --- Summary: make ama_context available in SparkSQL Key: AMATERASU-13 URL: https://issues.apache.org/jira/browse/AMATERASU-13 Project: AMATERASU Issue

[jira] [Assigned] (AMATERASU-17) SDK - add framework distribution capability

2018-01-23 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-17?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-17: --- Assignee: Yaniv Rodenski > SDK - add framework distribution capabil

[jira] [Assigned] (AMATERASU-15) Implement executor notifications for YARN

2018-01-23 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-15?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-15: --- Assignee: Eyal Ben Ivri > Implement executor notifications for Y

[jira] [Assigned] (AMATERASU-3) Scaffolding based on maki.yml

2018-01-23 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-3: -- Assignee: Nadav Har Tzvi > Scaffolding based on maki.

[jira] [Assigned] (AMATERASU-1) Amaterasu CLI

2018-01-23 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-1: -- Assignee: Nadav Har Tzvi > Amaterasu CLI > - > >

[jira] [Assigned] (AMATERASU-4) Run an Amaterasu pipeline

2018-01-23 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-4: -- Assignee: Nadav Har Tzvi > Run an Amaterasu pipel

[jira] [Assigned] (AMATERASU-13) make ama_context available in SparkSQL

2018-02-11 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-13?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-13: --- Assignee: Yaniv Rodenski > make ama_context available in Spark

[jira] [Resolved] (AMATERASU-15) Implement executor notifications for YARN

2018-02-10 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-15?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski resolved AMATERASU-15. - Resolution: Fixed > Implement executor notifications for Y

[jira] [Updated] (AMATERASU-7) Scheduled pipelines/actions

2018-02-18 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-7?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-7: --- Affects Version/s: 0.3.0-incubating > Scheduled pipelines/acti

[jira] [Updated] (AMATERASU-14) Make SparkSQL exports explicit

2018-02-18 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-14: Affects Version/s: 0.2.0-incubating > Make SparkSQL exports expli

[jira] [Updated] (AMATERASU-9) Nested pipelines

2018-02-18 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-9?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-9: --- Affects Version/s: 0.3.0-incubating > Nested pipeli

[jira] [Updated] (AMATERASU-12) Spark framework - support for spark streaming

2018-02-18 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-12?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-12: Affects Version/s: 0.3.0-incubating > Spark framework - support for spark stream

[jira] [Updated] (AMATERASU-13) make ama_context available in SparkSQL

2018-02-18 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-13?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-13: Affects Version/s: 0.2.0-incubating > make ama_context available in Spark

[jira] [Updated] (AMATERASU-18) Containers are not influenced by framework configuration

2018-02-24 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-18?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-18: Summary: Containers are not influenced by framework configuration (was: Containers

[jira] [Updated] (AMATERASU-18) Containers are not influenced by framework configuration

2018-02-24 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-18?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-18: Description: Currently, the containers sizing (YARN and Mesos) is derived from

[jira] [Assigned] (AMATERASU-18) Containers are not influenced by framework configuration

2018-02-24 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-18?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-18: --- Assignee: Eyal Ben Ivri > Containers are not influenced by framework configurat

[jira] [Updated] (AMATERASU-18) Containers are not influenced by framework configuration

2018-02-24 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-18?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-18: Fix Version/s: 0.2.0-incubating > Containers are not influenced by framew

[jira] [Created] (AMATERASU-19) Setup travis-ci release pipeline

2018-02-24 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-19: --- Summary: Setup travis-ci release pipeline Key: AMATERASU-19 URL: https://issues.apache.org/jira/browse/AMATERASU-19 Project: AMATERASU Issue Type

[jira] [Resolved] (AMATERASU-16) YARN - Deploy Amaterasu binaries only if they do not exist on HDFS

2018-02-22 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-16?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski resolved AMATERASU-16. - Resolution: Done > YARN - Deploy Amaterasu binaries only if they do not exist on H

[jira] [Updated] (AMATERASU-8) Support for Jar/External actions

2018-02-22 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-8?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-8: --- Fix Version/s: 0.2.1-incubating > Support for Jar/External acti

[jira] [Updated] (AMATERASU-15) Implement executor notifications for YARN

2018-02-22 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-15?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-15: Fix Version/s: 0.2.0-incubating > Implement executor notifications for Y

[jira] [Updated] (AMATERASU-6) Long running actions/pipelines

2018-02-22 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-6?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-6: --- Fix Version/s: 0.3.0-incubating > Long running actions/pipeli

[jira] [Updated] (AMATERASU-13) make ama_context available in SparkSQL

2018-02-22 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-13?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-13: Fix Version/s: 0.2.0-incubating > make ama_context available in Spark

[jira] [Updated] (AMATERASU-10) Action level configuration

2018-02-22 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-10?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-10: Fix Version/s: 0.3.0-incubating > Action level configurat

[jira] [Updated] (AMATERASU-11) Monitoring UI

2018-02-22 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-11?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-11: Fix Version/s: 0.3.0-incubating > Monitoring UI > - > >

[jira] [Updated] (AMATERASU-12) Spark framework - support for spark streaming

2018-02-22 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-12?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-12: Fix Version/s: 0.3.0-incubating > Spark framework - support for spark stream

[jira] [Updated] (AMATERASU-17) SDK - add framework distribution capability

2018-02-22 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-17?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-17: Fix Version/s: 0.2.0-incubating > SDK - add framework distribution capabil

[jira] [Updated] (AMATERASU-9) Nested pipelines

2018-02-22 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-9?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-9: --- Fix Version/s: 0.3.0-incubating > Nested pipeli

[jira] [Updated] (AMATERASU-14) Make SparkSQL exports explicit

2018-02-22 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-14: Fix Version/s: 0.2.0-incubating > Make SparkSQL exports expli

  1   2   >