Please do let me know if some other clarification is needed. Thanks, Vaibhav Garg +91-9505020924 vaibhavgar...@gmail.com LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
On Sat, Feb 29, 2020 at 10:31 AM Vaibhav Garg <vaibhavgar...@gmail.com> wrote: > OK. Here is what I have inside my configuration in order to start NewRelic > at the time of the container start: > > task.opts=-Xmx2048m -XX:+UseConcMarkSweepGC > *-javaagent:/usr/home/vaibhav/newrelic/newrelic.jar* > > Now, I have specified the absolute path of the NewRelic jar. There are 2 > challenges that I see with this approach: > > 1. The NewRelic jar and config would have to be present on all the Yarn > slave nodes since that container can start at any of the Yarn nodes. > 2. A single Yarn cluster can be used to run multiple Samza applications, > so I would have to keep configuration for each application at each of the > Yarn slave nodes. > > In order to fix this, I am bundling NewRelic inside my tar file: > > // download new relic dependencies > task downloadNewrelic(type: Download) { > mkdir "newrelic" > src " > https://download.newrelic.com/newrelic/java-agent/newrelic-agent/current/newrelic-java.zip > " > dest file("newrelic") > } > > task buildTar(dependsOn: [build, downloadNewrelic], type: Tar) { > // unzip new relic dependencies and add to the archive > into(".") { > from zipTree(file("newrelic/newrelic-java.zip")) > } > } > > With this build configuration, I have an additional *newrelic* directory > inside my archive. Here is the directory structure: > my-job-name.tar.gz > bin > config > lib > newrelic > newrelic.jar > newrelic.yml > ... > > What I would like to achieve is to: > - Remove hardcoding the *javaagent* inside *task.opts* in the container > config. > - Specify the path of NewRelic agent in *task.opts* relative to where the > container files are present on the Yarn slave node. > > I hope I was clearer this time. Please let me know if any other > clarification is needed. > > Thanks, > Vaibhav Garg > +91-9505020924 > vaibhavgar...@gmail.com > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> > > > On Sat, Feb 29, 2020 at 5:22 AM Yi Pan <nickpa...@gmail.com> wrote: > >> Hi, Vaibhav, >> >> Not quite sure whether I understand your ask. How do you package and >> distribute NewRelic jar in the YARN cluster? It would be much helpful if >> you can share your job configure that worked and the places that you would >> like to change. >> >> Best, >> >> -Yi >> >> On Fri, Feb 28, 2020 at 11:55 AM Vaibhav Garg <vaibhavgar...@gmail.com> >> wrote: >> >> > OK. I have been able to use NewRelic by specifying -javaagent option in >> > task.opts. >> > >> > However, I had to give the absolute path to the NewRelic jar file in >> order >> > to make it work. >> > >> > How can I give a path that is relative to the container since I would >> like >> > to integrate NewRelic dependencies along with other container files? >> > >> > Thanks, >> > Vaibhav Garg >> > +91-9505020924 >> > vaibhavgar...@gmail.com >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> >> > >> > >> > On Sat, Feb 29, 2020 at 12:41 AM Vaibhav Garg <vaibhavgar...@gmail.com> >> > wrote: >> > >> > > Yes. That's exactly what I am looking for. >> > > >> > > Let me try it out and I will get back to you if I face any issues. >> > > >> > > Thanks again >> > > >> > > Vaibhav Garg >> > > +91-9505020924 >> > > vaibhavgar...@gmail.com >> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> >> > > >> > > >> > > On Fri, Feb 28, 2020 at 1:23 PM Yi Pan <nickpa...@gmail.com> wrote: >> > > >> > >> Hi, Vaibhav, >> > >> >> > >> Check the description of task.opts in the configuration doc here: >> > >> >> > >> >> > >> http://samza.apache.org/learn/documentation/latest/jobs/samza-configurations.html >> > >> >> > >> Is this what you are looking for? >> > >> >> > >> On Thu, Feb 27, 2020 at 9:54 PM Vaibhav Garg < >> vaibhavgar...@gmail.com> >> > >> wrote: >> > >> >> > >> > Hi, >> > >> > >> > >> > Any help here would be great. >> > >> > >> > >> > Thanks, >> > >> > Vaibhav Garg >> > >> > +91-9505020924 >> > >> > vaibhavgar...@gmail.com >> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> >> > >> > >> > >> > >> > >> > On Thu, Feb 27, 2020 at 10:20 AM Vaibhav Garg < >> > vaibhavgar...@gmail.com> >> > >> > wrote: >> > >> > >> > >> > > Dear Bharath, >> > >> > > >> > >> > > Thanks for your reply. I now realize that I put a very ambiguous >> > >> question >> > >> > > to the community. Here is another attempt: >> > >> > > >> > >> > > I have set up a Yarn cluster that is configured to run >> containers of >> > >> > > multiple Samza jobs. >> > >> > > >> > >> > > I would like to treat all the containers of the job as a single >> > >> > > application in New Relic (as expected). >> > >> > > >> > >> > > Since the containers of the application can be killed on one Yarn >> > node >> > >> > and >> > >> > > can start on another Yarn node, I would have to specify New Relic >> > >> > > environment settings at the container start. >> > >> > > >> > >> > > Now, I am not sure how to specify java agent (to include New >> Relic >> > >> java >> > >> > > agent) and pass additional arguments such as New Relic >> environment >> > >> every >> > >> > > time the container starts. >> > >> > > >> > >> > > Please let me know if my understanding is wrong or some other >> > >> > > clarification is needed. >> > >> > > >> > >> > > Thanks in advance, >> > >> > > Vaibhav Garg >> > >> > > +91-9505020924 >> > >> > > vaibhavgar...@gmail.com >> > >> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> >> > >> > > >> > >> > > >> > >> > > On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara Subramanian < >> > >> > > codin.mart...@gmail.com> wrote: >> > >> > > >> > >> > >> Hi, >> > >> > >> >> > >> > >> I am not sure I fully understand the ask. >> > >> > >> >> > >> > >> IIUC, Samza doesn't have native integration with New relic. >> > However, >> > >> you >> > >> > >> should still be able to integrate your application with New >> relic >> > on >> > >> > your >> > >> > >> end without native support. >> > >> > >> If you are particularly looking to integrate native Samza >> metrics >> > w/ >> > >> New >> > >> > >> relic, you might need to implement your own custom metrics >> > reporter. >> > >> You >> > >> > >> can find more details here >> > >> > >> < >> > >> > >> >> > >> > >> > >> >> > >> https://samza.apache.org/learn/documentation/latest/operations/monitoring.html#customreporter >> > >> > >> > >> > >> > >> >> > >> > >> Thanks, >> > >> > >> Bharath >> > >> > >> >> > >> > >> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg < >> > >> vaibhavgar...@gmail.com> >> > >> > >> wrote: >> > >> > >> >> > >> > >> > Hi, >> > >> > >> > >> > >> > >> > I want to integrate New Relic in my Samza jobs. Can anyone >> help >> > >> with >> > >> > >> this, >> > >> > >> > please? >> > >> > >> > >> > >> > >> > Thanks, >> > >> > >> > Vaibhav Garg >> > >> > >> > +91-9505020924 >> > >> > >> > vaibhavgar...@gmail.com >> > >> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> >> > >> > >> > >> > >> > >> >> > >> > > >> > >> > >> > >> >> > > >> > >> >