Re: Initial setup of Amaterasu - Unable to run

2017-09-30 Thread Yaniv Rodenski
Hi Arun, I think you are hitting a bug that we’ve fixed but was in a pending PR, I've just merged the PR. Try to do a git pull and run again, let us know if it solves the problem. Cheers, Yaniv On Sat, 30 Sep 2017 at 7:36 pm, Arun Manivannan wrote: > Hi, > > I am trying to

Re: Initial setup of Amaterasu - Unable to run

2017-09-30 Thread Yaniv Rodenski
ibly sorry if these questions doesn't make much sense in the > context of the project. I would just like to know if I had misunderstood > the purpose of the project. I absolutely realise that the project is just > incubating and it's too much to ask for all the bells and whistles o

Re: Write access to the incubator-amaterasu @ GitHub

2017-11-26 Thread Yaniv Rodenski
fré <j...@nanthrax.net> wrote: > Hi, > > we can't see image on the mailing list. Can you send the actual message ? > > As we are on gitbox, github or github should work. > > Regards > JB > > On 11/26/2017 02:40 AM, Yaniv Rodenski wrote: > >> Hi Mentors, &

Write access to the incubator-amaterasu @ GitHub

2017-11-25 Thread Yaniv Rodenski
Hi Mentors, I'm trying to merge a PR via GitHub and seeing this message:[image: Inline image 1] It looks like that while I do have access to gitbox I have no write access to the GitHub mirror. How can I get it? Cheers, -- Yaniv Rodenski

Apache Amaterasu planning meeting notes - actions required

2017-11-26 Thread Yaniv Rodenski
ameworks and try to bring them to integrate with Amaterasu. Cheers, -- Yaniv Rodenski

Re: Next steps - Planning

2017-11-22 Thread Yaniv Rodenski
Hi All, Just one last reminder, if you can answer the poll we would really like to have as many of us at the meeting as possible Cheers Yaniv On Sat, Nov 18, 2017 at 11:16 AM, Yaniv Rodenski <ya...@shinto.io> wrote: > Hi All, > > As you can probably see, I've started migrating

Re: podling report

2017-12-13 Thread Yaniv Rodenski
utes reported back to the list, and most ... > > > > Ok to push to the report. > > > -- Yaniv Rodenski

Next steps - Planning

2017-11-17 Thread Yaniv Rodenski
Hi All, As you can probably see, I've started migrating tasks from our old Trello to JIRA. A few things to note: We currently have two tasks that are pending for version 0.2.0-incubating and started on JIRA - YARN support and the completion of SparkSQL. I don't see the need to migrate those, but

Re: Unable to run the sample job

2017-11-01 Thread Yaniv Rodenski
0.0.0.0:8000} > 2017-10-31 10:53:37.732:INFO:oejsh.ContextHandler:Thread-73: Stopped > o.e.j.s.ServletContextHandler@8c3b9d{/,file:/home/shad/apps/ > apache-amaterasu-0.2.0-incubating/dist/,UNAVAILABLE} > I1031 10:53:37.733896 36249 sched.cpp:2021] Asked to stop the driver > I1031 10:5

Re: Podling Report Reminder - December 2017

2017-12-10 Thread Yaniv Rodenski
ittle before > >> this page is created from a template. > >> > >> Mentors > >> --- > >> > >> Mentors should review reports for their project(s) and sign them off on > >> the Incubator wiki page. Signing off reports shows that you are > >> following the project - projects that are not signed may raise alarms > >> for the Incubator PMC. > >> > >> Incubator PMC > >> > > > > -- > Jean-Baptiste Onofré > jbono...@apache.org > http://blog.nanthrax.net > Talend - http://www.talend.com > -- Yaniv Rodenski

podling report

2017-12-11 Thread Yaniv Rodenski
r its first release in the incubator Date of last release: N/A When were the last committers or PMC members elected? N/A " If there are no objections I will need help updating the wiki as I still don't have access. Cheers, -- Yaniv Rodenski

Re: Write access to the incubator-amaterasu @ GitHub

2017-12-11 Thread Yaniv Rodenski
ake sure you've linked your GitHub and ASF accounts at > https://gitbox.apache.org/. > > On Sun, Nov 26, 2017 at 12:47 AM, Yaniv Rodenski <ya...@shinto.io> wrote: > > > Hi JB, > > > > I get the following in GitHub: > > This branch has no conflicts with t

Re: Apache Amaterasu planning meeting notes - actions required

2017-12-10 Thread Yaniv Rodenski
Hi John, I believe @JB is looking into our permissions issue. Regards, Yaniv On Mon, Dec 11, 2017 at 1:25 AM, John D. Ament <johndam...@apache.org> wrote: > Yaniv, > > Did you ever receive a response to this? > > John > > On 2017-11-26 05:53, Yaniv Rodenski <ya..

Re: Write access to the incubator-amaterasu @ GitHub

2017-12-11 Thread Yaniv Rodenski
ter is empty, so I populated it from the > project status page (modulo Karel, who doesn't seem to have an account). > This might solve the problem -- can you try again? > > On Mon, Dec 11, 2017 at 4:00 AM, Yaniv Rodenski <ya...@shinto.io> wrote: > > > Hi Davor, >

podling report

2017-10-30 Thread Yaniv Rodenski
t box * One new pull request was issued Date of last release: N/A When were the last committers or PMC members elected? N/A " If there are no objections I will update the wiki. Cheers, -- Yaniv Rodenski

Re: Unable to run the sample job

2017-10-30 Thread Yaniv Rodenski
h(Iterator.scala:891) > at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) > at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) > at scala.collection.AbstractIterable.foreach(Iterable.scala:54) > at > org.apache.amaterasu.leader.mesos.schedulers.JobScheduler. > resourceOffers(JobScheduler.scala:128) > I1030 14:00:12.991056 16624 sched.cpp:2055] Asked to abort the driver > I1030 14:00:12.991195 16624 sched.cpp:1233] Aborting framework > e72b9609-4b7f-4509-b1a4-bd2055d674aa-0002 > > Thanks, > Shad > -- Yaniv Rodenski

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #2

2018-05-06 Thread Yaniv Rodenski
No worries. This will take a number of tries -- completely normal. > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

License questions for RC3

2018-05-14 Thread Yaniv Rodenski
has a BSD license. Is that an issue? if so, an alternative approach would be to install it via pip during Amaterasu's setup stage, is that a valid solution? Thanks in advance, -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Re: License questions for RC3

2018-05-14 Thread Yaniv Rodenski
n > the two, please pick the approach that you prefer. (There's a likely small > difference in crafting the LICENSE/NOTICE files between these two cases.) > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Mail archive test

2018-06-14 Thread Yaniv Rodenski
Hi All, I'm just checking going directly via dev@amaterasu and not dev@incubator.amaterasu since I can't see the result email in the archive yet -- Yaniv Rodenski

[RESULT][VOTE] Amaterasu release 0.2.0-incubating, release candidate #3

2018-06-14 Thread Yaniv Rodenski
. I will start the voting on the @general mailing list ASAP Thanks everyone! Yaniv Rodenski

Vote at the general@ mailing list

2018-06-17 Thread Yaniv Rodenski
Hi Mentors, Just a quick etiquette question, as we are over 72 hours into the vote and only JB has voted, what (and when) would be the best way to request more votes? I understand people are busy, when do we consider a vote to be "forgotten" and in need of a reminder? Cheers, -- Yani

Re: AMATERASU-24

2018-05-26 Thread Yaniv Rodenski
ation > master can use it. > > Is it okay if I rename the Miniconda install file to miniconda-install > using the "wget -O". The reason why this change is proposed is to avoid > hardcoding the conda version inside the code and possibly pull it away into > amaterasu.properties

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #3

2018-05-30 Thread Yaniv Rodenski
sion > > > > > > > > > 0.2.0-incubating, as follows > > > > > > > > > > > > > > > > > > [ ] +1, Approve the release > > > > > > > > > [ ] -1, Do not approve the release (please provide specific > > > > > comments) > > > > > > > > > > > > > > > > > > The complete staging area is available for your review, > which > > > > > > includes: > > > > > > > > > > > > > > > > > > * JIRA release notes [1], > > > > > > > > > * the official Apache source release to be deployed to > > > > > > dist.apache.org > > > > > > > > > [2], > > > > > > > > > which is signed with the key with fingerprint [3], > > > > > > > > > * source code tag "version-0.2.0-incubating-rc3" [4], > > > > > > > > > * Java artifacts were built with Gradle 3.1 and > > OpenJDK/Oracle > > > > JDK > > > > > > > > > 1.8.0_151 > > > > > > > > > > > > > > > > > > The vote will be open for at least 72 hours. It is adopted > by > > > > > > majority > > > > > > > > > approval, with at least 3 PMC affirmative votes. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Yaniv > > > > > > > > > > > > > > > > > > [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa > ? > > > > > > > > > projectId=12321521=12342793 > > > > > > > > > [2] https://dist.apache.org/repos/ > > > dist/dev/incubator/amaterasu/ > > > > > > > 0.2.0rc3/ > > > > > > > > > [3] > > > > > https://dist.apache.org/repos/dist/dev/incubator/amaterasu/KEYS > > > > > > > > > [4] https://github.com/apache/incubator-amaterasu/tags > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Amaterasu PyCon il Presentation

2018-05-27 Thread Yaniv Rodenski
Hi All, @Nadav is giving an intro to Amaterasu @ PyCon Israel next week and we thought everyone here might like to have the deck <https://drive.google.com/file/d/1rGLBPw3hkpd0ZVQrJGZWqHf6gtBk3Ztv/view?usp=sharing> and may be present somewhere :) Cheers, -- Yaniv Rodenski +61 477 778

[VOTE] Amaterasu release 0.2.0-incubating, release candidate #3

2018-05-28 Thread Yaniv Rodenski
Hi everyone, We have fixed the legal issues, as well as a bug found by @Nadav please review and vote on the release candidate #3 for the version 0.2.0-incubating, as follows [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging

Re: Speeding up Anaconda deployment on executors

2018-05-29 Thread Yaniv Rodenski
ition it gives us > the nifty Conda package manager. > However, it doesn't mean that it has to stay that way. If the need or > reason arises, we may need to also support pip and support using the native > Python version (instead of the one supplied by Anaconda). > > I want to discuss the possible solutions to this. Please feel free to bring > up your ideas. > > Cheers, > Nadav > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #3

2018-06-03 Thread Yaniv Rodenski
gt; > Disabled > >> > > > physical memory check (yarn.nodemanager.pmem-check-enabled) and > ran > >> > jobs > >> > > > successfully. > >> > > > > >> > > > > >> > > > > >> > > > &g

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #3

2018-06-04 Thread Yaniv Rodenski
> Cheers, > Arun > > On Sun, Jun 3, 2018 at 6:37 PM Yaniv Rodenski wrote: > > > Hi Arun, > > > > Fair point, but I think it can be configured via env/[env_name]/spark.yml > > am I wrong? > > Anyway, ideally I think we should try and have per

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #3

2018-06-01 Thread Yaniv Rodenski
One thing to clarify, the tar.gz is the source code as I've seen this is released in other projects. if that is not needed I'm happy to drop it from future releases. On Fri, Jun 1, 2018 at 5:01 PM, Yaniv Rodenski wrote: > Thanks Davor, > > We will fix those for the next release. &

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #3

2018-06-02 Thread Yaniv Rodenski
(which are in the -source.tar.gz file) The main question is if we do or do not need to release both. Cheers, Yaniv On Sat, Jun 2, 2018 at 1:13 AM, Davor Bonaci wrote: > No problem with .tar.gz. But, you don't need .tar then. > > On Fri, Jun 1, 2018 at 12:04 AM, Yaniv Rodenski wrote:

[RESULT][VOTE] Amaterasu release 0.2.0-incubating, release candidate #4

2018-06-30 Thread Yaniv Rodenski
I'm happy to announce that we have unanimously approved this release. There are 8 approving votes, 3 of which are binding: * Nadav Har Tzvi * Kirupa Devarajan * Guy Peleg * Arun Manivannan * Eyal Ben Ivri IPMC votes * Davor Bonaci * Olivier Lamy * Jean-Baptiste Onofré There are no

[VOTE] Release Apache Amaterasu (incubating) 0.2.0 (rc4)

2018-06-23 Thread Yaniv Rodenski
Hi everyone, After cancelling the vote in the general@ list we've fixed the following: * Headers added to all non-code files where applicable as remarked by Davor * Sources now match the released version + no keys are present * The gradle-wrapper.jar was removed and instructions ware added to the

Re: Cancelling the vote in the @general mailing list

2018-06-20 Thread Yaniv Rodenski
is public domain, there are > typically options as well. I suggest understanding the genesis of software > in question. > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Cancelling the vote in the @general mailing list

2018-06-19 Thread Yaniv Rodenski
have one. what can we do in that case? Thanks in advance -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #2

2018-04-30 Thread Yaniv Rodenski
; Hi guys, > > > > I'm just back from vacation. I will review the release. > > > > Regards > > JB > > > > On 04/29/2018 12:44 PM, Yaniv Rodenski wrote: > > > Hi Mentors, > > > > > > In order to progress with the vote, can any of you

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #2

2018-04-29 Thread Yaniv Rodenski
previous issues were resolved , in > > addition we need short feedback loop from users. > > > > On Thu, Apr 26, 2018, 08:48 Kirupa Devarajan <kirupagara...@gmail.com> > > wrote: > > > > > +1 - Test and Build successful for release candidate #2 >

Moving forward

2017-10-22 Thread Yaniv Rodenski
? Also we are very close to version 0.2.0-incubating. not a lot of open tasks, but I think we should also get our JIRA in order, I can port everything we have on our old trello, but maybe a hangouts sync to go over the open tasks is a good way to push things forward? Cheers, -- Yaniv Rodenski

Re: Moving forward

2017-10-28 Thread Yaniv Rodenski
gt; Actually, after I am done :) > > > On 26 Oct 2017, at 2:48, Yaniv Rodenski <ya...@shinto.io> wrote: > > > > OK so GitHub doesn't allow forking empty branches, so I think I'll just > > push the shintoio master to the amaterasu-incubator master as Olivier > > sug

Re: ama-start and JobLauncher stuck (Solution found) + Running Spark jobs across several repo

2017-12-31 Thread Yaniv Rodenski
he file that instantiates the Spark session, context etc has to be > explicitly given as a ".scala" file (we then use the IMain interpreter to > inject the AmaContext?) > > Now, with two repositories in play and only the binaries/repository name > given for the repo, would it be a good idea to achieve the AmaContext > insertion using a compiler plugin? I am pretty sure this has been > discussed before and it would be great if you could share your views on > this. I can come up with a POC PR of sorts if I get some ideas. > > Best Regards, > Arun > > -- Yaniv Rodenski

[DISCUSS] Monthly report and getting ready to release 0.2.0 incubating

2018-02-10 Thread Yaniv Rodenski
wiki looks like the user I've re-created recently got deleted again :( Cheers, -- Yaniv Rodenski

[DISCUSS] release procedure

2018-02-16 Thread Yaniv Rodenski
contributors to do the same. Checklist to proceed to the finalization step: 1. Release announced on the dev@ mailing list. 2. A blog post published, if applicable. 3. Release recorded in reporter.apache.org. 4. Release announced on social media. 5. Completion declared on the ge

Re: [DISCUSS] release procedure

2018-02-18 Thread Yaniv Rodenski
stated that the release > >requires the approval of 3 IPMC members, is that correct? > > > > 3 PPMC should approve before presenting to Incubator. 3 IPMC members then > to release. > -- Yaniv Rodenski

Podling report

2018-01-02 Thread Yaniv Rodenski
need help updating the wiki looks like the user I've re-created recently got deleted again :( Cheers, -- Yaniv Rodenski

Re: Next steps?

2017-12-26 Thread Yaniv Rodenski
t; mentor attention. > > > > Otherwise, the report is due 1/3. Yaniv, do you have everything you need > > there? > > > > Thanks! > > > > Davor > > > -- Yaniv Rodenski

Re: Next steps?

2017-12-27 Thread Yaniv Rodenski
> > Let's start with Jira and wiki. Can you check about the wiki first ? Do > you need any help for the report content ? > > Regards > JB > > > On 27/12/2017 06:44, Yaniv Rodenski wrote: > >> Excellent, >> >> Here are the open items on my end: >>

[DISCUSS] podling report

2018-07-06 Thread Yaniv Rodenski
Hi All, Sorry for doing this late again, but I propose the following report to be submitted: " Apache Amaterasu is a framework providing configuration management and deployment for Big Data Pipelines. It provides the following capabilities: Continuous integration tools to package pipelines and

Re: [VOTE] Release Apache Amaterasu (incubating) 0.2.0 (rc4)

2018-07-10 Thread Yaniv Rodenski
mod=license=2=57>/resources < > > http://localhost:8081/repo/?mod=license=2=58>/codegen.py > > 2. incubator-amaterasu-version-0.2.0-incubating-rc4/executor/ > > src/test/resources/py4j-0.10.4-src.zip > > 3. incubator-amaterasu-version-0.2.0-incubating-rc4/executor/ > >

[ANNOUNCE] Apache Amaterasu (incubating) 0.2.0-incubating released

2018-07-11 Thread Yaniv Rodenski
of the Amaterasu team, Yaniv Rodenski

[DISCUSS] release version 0.2.0-incubating

2018-04-05 Thread Yaniv Rodenski
would like to proceed with the release in the next couple of days. Cheers, -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #1

2018-04-16 Thread Yaniv Rodenski
ava.net.BindException: > Address already in use > > This is caused when there is another instance of Zookeeper running on the > port 2181. If this zookeeper instance is shutdown, then the tests run > sucessfully > > Regards, > Shad > > On Tue, Apr 17, 2018 at 9:58 AM, Yan

Release question

2018-04-15 Thread Yaniv Rodenski
folders for us and grunt committers rights to deploy there. Hope to start voting soon! -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Re: Release question

2018-04-15 Thread Yaniv Rodenski
n Mon, 16 Apr 2018 at 09:50, Yaniv Rodenski <ya...@shinto.io> wrote: > > > Hi Mentors, > > > > As you know we are trying to release an RC of Amaterasu, and I have a > quick > > question regarding the dist.apache.org setup. > > the question is am I sup

Re: YARN deployment

2018-04-24 Thread Yaniv Rodenski
e (reconciliation > table). > > > For now, we can safely assume that there's no data shared between these > dataframes. > > Greatly appreciate your response on the YARN job submission. > > Cheers, > Arun > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

[VOTE] Amaterasu release 0.2.0-incubating, release candidate #2

2018-04-24 Thread Yaniv Rodenski
nd OpenJDK/Oracle JDK 1.8.0_151 The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Thanks, Yaniv Rodenski [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12321521=12342793 [2] https://dist.apache.org/repo

Re: [VOTE] Amaterasu release 0.2.0-incubating, release candidate #1

2018-04-16 Thread Yaniv Rodenski
a:111) > at > org.apache.curator.test.TestingZooKeeperMain.runFromConfig( > TestingZooKeeperMain.java:73) > > at > org.apache.curator.test.TestingZooKeeperServer$1.run( > TestingZooKeeperServer.java:148) > > 3. I received the above error also when testing in the

[VOTE] Amaterasu release 0.2.0-incubating, release candidate #1

2018-04-16 Thread Yaniv Rodenski
ll be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Thanks, Yaniv Rodenski [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12321521=12342793 [2] https://dist.apache.org/repos/dist/dev/incubator/amaterasu/0.2.0rc1/

[DISCUSS] podling report

2018-04-02 Thread Yaniv Rodenski
butor communities 3. Prepare documentation Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of? None How has the community developed since the last report? * Two conference talks are scheduled in the near future, Voxxed Days Melbourne (Yaniv Rodenski) and SDP in Tel

Re: Project status

2018-03-20 Thread Yaniv Rodenski
> project, and graduation from Incubator seems hard right now. > > I know this may be difficult, but I think it is better to talk about these > things openly and investigate what can be done and/or whether the project > would be better served as a standalone project in GitHub. &

Re: Project status

2018-03-20 Thread Yaniv Rodenski
erm work is done by one more contributor on his local fork (Nadav > > Har-Tzvi) > > > > I think this is the crux of the problem. Why is longer-term work going on > in a local fork? > > > > * We should be presenting more, and growing the community more which is > &

Re: [DISCUSS] Dependencies resolution and action level dependencies

2018-10-22 Thread Yaniv Rodenski
tartup time per action. The more dependencies >you have, the longer it takes to resolve and install them. Actions will > no >longer get any unnecessary dependencies. > > > What do you think? Does it make sense? > > Cheers, > Nadav > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Re: [DISCUSS] Dependencies resolution and action level dependencies

2018-10-23 Thread Yaniv Rodenski
encies > > Regards, > Kirupa > > On Tue., 23 Oct. 2018, 11:04 am Yaniv Rodenski, wrote: > > > Hi Nadav, > > > > It does make sense, in fact, we actually have action level resources > > already, however they are limited to the configuration files for the > &g

Re: Project status

2018-09-20 Thread Yaniv Rodenski
Hi All, First of all, sorry for the slow response, I've been traveling lately and this is the first opportunity I had to sit down and write my thoughts. Also, thanks @Davor, I think it's important that we deal with the state Amaterasu fell into past the 0.2.0 release. I think the current

[DISCUSS] podling report

2018-10-02 Thread Yaniv Rodenski
Hi All I would like to submit the following report to the board. Please note that I have raised @Davor's concern under Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of? Please review my comment there as well. I have also removed item 1 (Prepare for the first

Re: AMATERASU-44

2018-10-10 Thread Yaniv Rodenski
fore making the > change. > > > Cheers, > Arun > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Re: Project status

2018-10-02 Thread Yaniv Rodenski
mmunity, and before joining the Incubator. > >>> > >>> It is really, really, really hard to build the community before early > >> user > >>> fit. > >>> > >>> I think that the next few months are more about staying in the > incubating > >>>> or not, it is do-or-die for Amaterasu. We need to fix the situation > so I > >>>> wouldn't rush in this situation to consider retiring quite yet. > >>>> > >>> > >>> I'm totally fine with leaving things as-is for a few more months. But, > I > >>> don't think it is realistic to expect changes to the degree necessary > to > >>> graduate. As a result, I think you can use *your time* better. > >>> > >>> * * * > >>> > >>> I'm really sorry to be the messenger of bad news. I don't want to paint > >>> Amaterasu (or your work) in any bad way. I do want you and the project > to > >>> be most successful as possible. > >>> > >>> Please note that I don't gain anything by driving this conversation. In > >>> fact, I lose a lot. Time is a precious resource of everyone -- and I'd > >> like > >>> to make sure that the time *you* have for the project is spent in a way > >>> that is likely to yield results, not trying to achieve various ASF > goals > >>> that may not achievable. (If this is not obvious now, I trust that over > >>> time direct feedback and tough conversations will be appreciated, > instead > >>> of letting you waste time for something unrealistic -- which is what > most > >>> people in my shoes would do.) > >>> > >> > > > > -- > Jean-Baptiste Onofré > jbono...@apache.org > http://blog.nanthrax.net > Talend - http://www.talend.com > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Re: [DISCUSS] podling report

2018-10-03 Thread Yaniv Rodenski
Hi All, I've submitted the report with all the remarks applied. Yaniv On Thu, Oct 4, 2018 at 6:11 AM Justin Mclean wrote: > Hi, > > The report has not been submitted, please do so as soon as you. > > Thanks, > Justin > -- Yaniv Rodenski +61 477 778 405 ya...@shinto.io

Re: [DISCUSS] Dependencies resolution and action level dependencies

2018-10-23 Thread Yaniv Rodenski
ach the YARN implementation. Cheers, Yaniv On Tue, Oct 23, 2018 at 9:51 PM Yariv Triffon wrote: > Hi Yaniv, > i'm good to grab the task of moving Mesos to use Docker containers. > > Cheers, > Yariv > > On Tue, Oct 23, 2018 at 5:13 PM Yaniv Rodenski wrote: > > > Th

[jira] [Updated] (AMATERASU-6) Long running actions/pipelines

2017-11-26 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-6?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-6: --- Description: As a pipeline developer, I can define pipelines and actions to be long

[jira] [Updated] (AMATERASU-7) Scheduled pipelines/actions

2017-11-26 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-7?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-7: --- Description: As a pipeline developer, I can define a schedule in the maki.yml for my

[jira] [Updated] (AMATERASU-7) Scheduled pipelines/actions

2017-11-26 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-7?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-7: --- Description: As a pipeline developer, I can define a schedule in the maki.yml for my

[jira] [Created] (AMATERASU-6) Long running actions/pipelines

2017-11-16 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-6: -- Summary: Long running actions/pipelines Key: AMATERASU-6 URL: https://issues.apache.org/jira/browse/AMATERASU-6 Project: AMATERASU Issue Type: New

[jira] [Created] (AMATERASU-7) Scheduled pipelines/actions

2017-11-16 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-7: -- Summary: Scheduled pipelines/actions Key: AMATERASU-7 URL: https://issues.apache.org/jira/browse/AMATERASU-7 Project: AMATERASU Issue Type: New Feature

[jira] [Created] (AMATERASU-11) Monitoring UI

2017-11-17 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-11: --- Summary: Monitoring UI Key: AMATERASU-11 URL: https://issues.apache.org/jira/browse/AMATERASU-11 Project: AMATERASU Issue Type: New Feature

[jira] [Updated] (AMATERASU-9) Nested pipelines

2017-11-17 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-9?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-9: --- Issue Type: Improvement (was: Bug) > Nested pipeli

[jira] [Created] (AMATERASU-12) Spark framework - support for spark streaming

2017-11-17 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-12: --- Summary: Spark framework - support for spark streaming Key: AMATERASU-12 URL: https://issues.apache.org/jira/browse/AMATERASU-12 Project: AMATERASU

[jira] [Assigned] (AMATERASU-7) Scheduled pipelines/actions

2018-05-14 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-7?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-7: -- Assignee: Yaniv Rodenski > Scheduled pipelines/acti

[jira] [Created] (AMATERASU-23) Add executor classpath from framework configuration

2018-05-14 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-23: --- Summary: Add executor classpath from framework configuration Key: AMATERASU-23 URL: https://issues.apache.org/jira/browse/AMATERASU-23 Project: AMATERASU

[jira] [Created] (AMATERASU-24) Refactor Spark out of Amaterasu executor to it's own project

2018-05-15 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-24: --- Summary: Refactor Spark out of Amaterasu executor to it's own project Key: AMATERASU-24 URL: https://issues.apache.org/jira/browse/AMATERASU-24 Project

[jira] [Assigned] (AMATERASU-33) Upgrade Gradle version to 4.8

2018-06-09 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-33?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-33: --- Assignee: Yaniv Rodenski > Upgrade Gradle version to

[jira] [Created] (AMATERASU-33) Upgrade Gradle version to 4.8

2018-06-09 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-33: --- Summary: Upgrade Gradle version to 4.8 Key: AMATERASU-33 URL: https://issues.apache.org/jira/browse/AMATERASU-33 Project: AMATERASU Issue Type

[jira] [Assigned] (AMATERASU-23) Add executor classpath from framework configuration

2018-06-09 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-23?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-23: --- Assignee: Yaniv Rodenski > Add executor classpath from framework configurat

[jira] [Updated] (AMATERASU-23) Add executor classpath from framework configuration - Mesos

2018-06-18 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-23?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-23: Summary: Add executor classpath from framework configuration - Mesos (was: Add executor

[jira] [Created] (AMATERASU-36) Refactoring - leader: extract projects per resource manager + common

2018-06-18 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-36: --- Summary: Refactoring - leader: extract projects per resource manager + common Key: AMATERASU-36 URL: https://issues.apache.org/jira/browse/AMATERASU-36 Project

[jira] [Created] (AMATERASU-35) Generate the container command from the framework providere

2018-06-17 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-35: --- Summary: Generate the container command from the framework providere Key: AMATERASU-35 URL: https://issues.apache.org/jira/browse/AMATERASU-35 Project

[jira] [Closed] (AMATERASU-21) Fix Spark scala tests

2018-06-17 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-21?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski closed AMATERASU-21. --- Resolution: Fixed > Fix Spark scala tests > - > >

[jira] [Resolved] (AMATERASU-26) Pipeline tasks (sub-Yarn jobs) runs as "yarn" user instead of inhering the user in which the amaterasu job was submitted

2018-06-08 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-26?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski resolved AMATERASU-26. - Resolution: Fixed > Pipeline tasks (sub-Yarn jobs) runs as "yarn"

[jira] [Created] (AMATERASU-37) Fix support for ssh keys auth

2018-06-30 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-37: --- Summary: Fix support for ssh keys auth Key: AMATERASU-37 URL: https://issues.apache.org/jira/browse/AMATERASU-37 Project: AMATERASU Issue Type

[jira] [Commented] (AMATERASU-4) Run an Amaterasu pipeline

2017-10-28 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16223364#comment-16223364 ] Yaniv Rodenski commented on AMATERASU-4: Hi Shad, I've just moved the code from https

[jira] [Updated] (AMATERASU-13) make ama_context available in SparkSQL

2017-12-25 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-13?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-13: Issue Type: Improvement (was: Bug) > make ama_context available in Spark

[jira] [Updated] (AMATERASU-13) make ama_context available in SparkSQL

2017-12-25 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-13?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-13: Description: SparkSQL actions runner should support accessing datasets via ama_context

[jira] [Updated] (AMATERASU-13) make ama_context available in SparkSQL

2017-12-25 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-13?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski updated AMATERASU-13: Description: SparkSQL actions runner should support accessing datasets via ama_context

[jira] [Created] (AMATERASU-14) Make SparkSQL exports explicit

2017-12-25 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-14: --- Summary: Make SparkSQL exports explicit Key: AMATERASU-14 URL: https://issues.apache.org/jira/browse/AMATERASU-14 Project: AMATERASU Issue Type

[jira] [Created] (AMATERASU-13) make ama_context available in SparkSQL

2017-12-25 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-13: --- Summary: make ama_context available in SparkSQL Key: AMATERASU-13 URL: https://issues.apache.org/jira/browse/AMATERASU-13 Project: AMATERASU Issue

[jira] [Created] (AMATERASU-16) YARN - Deploy Amaterasu binaries only if they do not exist on HDFS

2018-01-16 Thread Yaniv Rodenski (JIRA)
Yaniv Rodenski created AMATERASU-16: --- Summary: YARN - Deploy Amaterasu binaries only if they do not exist on HDFS Key: AMATERASU-16 URL: https://issues.apache.org/jira/browse/AMATERASU-16 Project

[jira] [Assigned] (AMATERASU-17) SDK - add framework distribution capability

2018-01-23 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-17?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-17: --- Assignee: Yaniv Rodenski > SDK - add framework distribution capabil

[jira] [Assigned] (AMATERASU-15) Implement executor notifications for YARN

2018-01-23 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-15?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-15: --- Assignee: Eyal Ben Ivri > Implement executor notifications for Y

[jira] [Assigned] (AMATERASU-2) Initialization of an Amaterasu job repository via the CLI

2018-01-23 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-2: -- Assignee: Nadav Har Tzvi > Initialization of an Amaterasu job repository via the

[jira] [Assigned] (AMATERASU-3) Scaffolding based on maki.yml

2018-01-23 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-3: -- Assignee: Nadav Har Tzvi > Scaffolding based on maki.

[jira] [Assigned] (AMATERASU-1) Amaterasu CLI

2018-01-23 Thread Yaniv Rodenski (JIRA)
[ https://issues.apache.org/jira/browse/AMATERASU-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaniv Rodenski reassigned AMATERASU-1: -- Assignee: Nadav Har Tzvi > Amaterasu CLI > - > >

  1   2   >