Ok. That worked for me. Thank you, Bharath, Yi Pan, and Jagadish for your help.
Thanks, Vaibhav Garg +91-9505020924 vaibhavgar...@gmail.com LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> On Sun, Mar 1, 2020 at 12:56 PM Vaibhav Garg <vaibhavgar...@gmail.com> wrote: > Ok. I will try that. > > Thanks, > Vaibhav Garg > +91-9505020924 > vaibhavgar...@gmail.com > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> > > > On Sun, Mar 1, 2020 at 12:31 PM Jagadish Venkatraman < > jagadish1...@gmail.com> wrote: > >> Hi Vaibhav, >> >> Can you try ./__package/newrelic/newrelic.jar? >> >> If that doesn't work, I'd recommend doing the following: >> 1. Print the current working directory of your Samza container JVM >> 2. Use (1) to determine a relative reference to the directory path within >> your package >> >> Cheers >> Jagadish >> >> On Saturday, February 29, 2020, Vaibhav Garg <vaibhavgar...@gmail.com> >> wrote: >> >> > I would be really grateful if someone could help me with this. >> > >> > Thanks, >> > Vaibhav Garg >> > +91-9505020924 >> > vaibhavgar...@gmail.com >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> >> > >> > >> > On Sat, Feb 29, 2020 at 9:02 PM Vaibhav Garg <vaibhavgar...@gmail.com> >> > wrote: >> > >> > > Please do let me know if some other clarification is needed. >> > > >> > > Thanks, >> > > Vaibhav Garg >> > > +91-9505020924 >> > > vaibhavgar...@gmail.com >> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> >> > > >> > > >> > > On Sat, Feb 29, 2020 at 10:31 AM Vaibhav Garg < >> vaibhavgar...@gmail.com> >> > > wrote: >> > > >> > >> OK. Here is what I have inside my configuration in order to start >> > >> NewRelic at the time of the container start: >> > >> >> > >> task.opts=-Xmx2048m -XX:+UseConcMarkSweepGC >> > >> *-javaagent:/usr/home/vaibhav/newrelic/newrelic.jar* >> > >> >> > >> Now, I have specified the absolute path of the NewRelic jar. There >> are 2 >> > >> challenges that I see with this approach: >> > >> >> > >> 1. The NewRelic jar and config would have to be present on all the >> Yarn >> > >> slave nodes since that container can start at any of the Yarn nodes. >> > >> 2. A single Yarn cluster can be used to run multiple Samza >> applications, >> > >> so I would have to keep configuration for each application at each of >> > the >> > >> Yarn slave nodes. >> > >> >> > >> In order to fix this, I am bundling NewRelic inside my tar file: >> > >> >> > >> // download new relic dependencies >> > >> task downloadNewrelic(type: Download) { >> > >> mkdir "newrelic" >> > >> src " >> > >> https://download.newrelic.com/newrelic/java-agent/newrelic- >> > agent/current/newrelic-java.zip >> > >> " >> > >> dest file("newrelic") >> > >> } >> > >> >> > >> task buildTar(dependsOn: [build, downloadNewrelic], type: Tar) { >> > >> // unzip new relic dependencies and add to the archive >> > >> into(".") { >> > >> from zipTree(file("newrelic/newrelic-java.zip")) >> > >> } >> > >> } >> > >> >> > >> With this build configuration, I have an additional *newrelic* >> directory >> > >> inside my archive. Here is the directory structure: >> > >> my-job-name.tar.gz >> > >> bin >> > >> config >> > >> lib >> > >> newrelic >> > >> newrelic.jar >> > >> newrelic.yml >> > >> ... >> > >> >> > >> What I would like to achieve is to: >> > >> - Remove hardcoding the *javaagent* inside *task.opts* in the >> container >> > >> config. >> > >> - Specify the path of NewRelic agent in *task.opts* relative to where >> > >> the container files are present on the Yarn slave node. >> > >> >> > >> I hope I was clearer this time. Please let me know if any other >> > >> clarification is needed. >> > >> >> > >> Thanks, >> > >> Vaibhav Garg >> > >> +91-9505020924 >> > >> vaibhavgar...@gmail.com >> > >> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> >> > >> >> > >> >> > >> On Sat, Feb 29, 2020 at 5:22 AM Yi Pan <nickpa...@gmail.com> wrote: >> > >> >> > >>> Hi, Vaibhav, >> > >>> >> > >>> Not quite sure whether I understand your ask. How do you package and >> > >>> distribute NewRelic jar in the YARN cluster? It would be much >> helpful >> > if >> > >>> you can share your job configure that worked and the places that you >> > >>> would >> > >>> like to change. >> > >>> >> > >>> Best, >> > >>> >> > >>> -Yi >> > >>> >> > >>> On Fri, Feb 28, 2020 at 11:55 AM Vaibhav Garg < >> vaibhavgar...@gmail.com >> > > >> > >>> wrote: >> > >>> >> > >>> > OK. I have been able to use NewRelic by specifying -javaagent >> option >> > in >> > >>> > task.opts. >> > >>> > >> > >>> > However, I had to give the absolute path to the NewRelic jar file >> in >> > >>> order >> > >>> > to make it work. >> > >>> > >> > >>> > How can I give a path that is relative to the container since I >> would >> > >>> like >> > >>> > to integrate NewRelic dependencies along with other container >> files? >> > >>> > >> > >>> > Thanks, >> > >>> > Vaibhav Garg >> > >>> > +91-9505020924 >> > >>> > vaibhavgar...@gmail.com >> > >>> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> >> > >>> > >> > >>> > >> > >>> > On Sat, Feb 29, 2020 at 12:41 AM Vaibhav Garg < >> > vaibhavgar...@gmail.com >> > >>> > >> > >>> > wrote: >> > >>> > >> > >>> > > Yes. That's exactly what I am looking for. >> > >>> > > >> > >>> > > Let me try it out and I will get back to you if I face any >> issues. >> > >>> > > >> > >>> > > Thanks again >> > >>> > > >> > >>> > > Vaibhav Garg >> > >>> > > +91-9505020924 >> > >>> > > vaibhavgar...@gmail.com >> > >>> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> >> > >>> > > >> > >>> > > >> > >>> > > On Fri, Feb 28, 2020 at 1:23 PM Yi Pan <nickpa...@gmail.com> >> > wrote: >> > >>> > > >> > >>> > >> Hi, Vaibhav, >> > >>> > >> >> > >>> > >> Check the description of task.opts in the configuration doc >> here: >> > >>> > >> >> > >>> > >> >> > >>> > >> > >>> http://samza.apache.org/learn/documentation/latest/jobs/ >> > samza-configurations.html >> > >>> > >> >> > >>> > >> Is this what you are looking for? >> > >>> > >> >> > >>> > >> On Thu, Feb 27, 2020 at 9:54 PM Vaibhav Garg < >> > >>> vaibhavgar...@gmail.com> >> > >>> > >> wrote: >> > >>> > >> >> > >>> > >> > Hi, >> > >>> > >> > >> > >>> > >> > Any help here would be great. >> > >>> > >> > >> > >>> > >> > Thanks, >> > >>> > >> > Vaibhav Garg >> > >>> > >> > +91-9505020924 >> > >>> > >> > vaibhavgar...@gmail.com >> > >>> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> >> > >>> > >> > >> > >>> > >> > >> > >>> > >> > On Thu, Feb 27, 2020 at 10:20 AM Vaibhav Garg < >> > >>> > vaibhavgar...@gmail.com> >> > >>> > >> > wrote: >> > >>> > >> > >> > >>> > >> > > Dear Bharath, >> > >>> > >> > > >> > >>> > >> > > Thanks for your reply. I now realize that I put a very >> > ambiguous >> > >>> > >> question >> > >>> > >> > > to the community. Here is another attempt: >> > >>> > >> > > >> > >>> > >> > > I have set up a Yarn cluster that is configured to run >> > >>> containers of >> > >>> > >> > > multiple Samza jobs. >> > >>> > >> > > >> > >>> > >> > > I would like to treat all the containers of the job as a >> > single >> > >>> > >> > > application in New Relic (as expected). >> > >>> > >> > > >> > >>> > >> > > Since the containers of the application can be killed on >> one >> > >>> Yarn >> > >>> > node >> > >>> > >> > and >> > >>> > >> > > can start on another Yarn node, I would have to specify New >> > >>> Relic >> > >>> > >> > > environment settings at the container start. >> > >>> > >> > > >> > >>> > >> > > Now, I am not sure how to specify java agent (to include >> New >> > >>> Relic >> > >>> > >> java >> > >>> > >> > > agent) and pass additional arguments such as New Relic >> > >>> environment >> > >>> > >> every >> > >>> > >> > > time the container starts. >> > >>> > >> > > >> > >>> > >> > > Please let me know if my understanding is wrong or some >> other >> > >>> > >> > > clarification is needed. >> > >>> > >> > > >> > >>> > >> > > Thanks in advance, >> > >>> > >> > > Vaibhav Garg >> > >>> > >> > > +91-9505020924 >> > >>> > >> > > vaibhavgar...@gmail.com >> > >>> > >> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> >> > >>> > >> > > >> > >>> > >> > > >> > >>> > >> > > On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara Subramanian >> < >> > >>> > >> > > codin.mart...@gmail.com> wrote: >> > >>> > >> > > >> > >>> > >> > >> Hi, >> > >>> > >> > >> >> > >>> > >> > >> I am not sure I fully understand the ask. >> > >>> > >> > >> >> > >>> > >> > >> IIUC, Samza doesn't have native integration with New >> relic. >> > >>> > However, >> > >>> > >> you >> > >>> > >> > >> should still be able to integrate your application with >> New >> > >>> relic >> > >>> > on >> > >>> > >> > your >> > >>> > >> > >> end without native support. >> > >>> > >> > >> If you are particularly looking to integrate native Samza >> > >>> metrics >> > >>> > w/ >> > >>> > >> New >> > >>> > >> > >> relic, you might need to implement your own custom metrics >> > >>> > reporter. >> > >>> > >> You >> > >>> > >> > >> can find more details here >> > >>> > >> > >> < >> > >>> > >> > >> >> > >>> > >> > >> > >>> > >> >> > >>> > >> > >>> https://samza.apache.org/learn/documentation/latest/ >> > operations/monitoring.html#customreporter >> > >>> > >> > >> > >> > >>> > >> > >> >> > >>> > >> > >> Thanks, >> > >>> > >> > >> Bharath >> > >>> > >> > >> >> > >>> > >> > >> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg < >> > >>> > >> vaibhavgar...@gmail.com> >> > >>> > >> > >> wrote: >> > >>> > >> > >> >> > >>> > >> > >> > Hi, >> > >>> > >> > >> > >> > >>> > >> > >> > I want to integrate New Relic in my Samza jobs. Can >> anyone >> > >>> help >> > >>> > >> with >> > >>> > >> > >> this, >> > >>> > >> > >> > please? >> > >>> > >> > >> > >> > >>> > >> > >> > Thanks, >> > >>> > >> > >> > Vaibhav Garg >> > >>> > >> > >> > +91-9505020924 >> > >>> > >> > >> > vaibhavgar...@gmail.com >> > >>> > >> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> >> > >>> > >> > >> > >> > >>> > >> > >> >> > >>> > >> > > >> > >>> > >> > >> > >>> > >> >> > >>> > > >> > >>> > >> > >>> >> > >> >> > >> >> >> -- >> Jagadish >> >
