Hi Debraj Manna, The app-def in previous email is just an example where you can configure node labels. Yarn node labels <https://hadoop.apache.org/docs/r2.7.3/hadoop-yarn/hadoop-yarn-site/NodeLabel.html> is a general feature (not specific to Samza), and it depends on the configuration system your system uses. The example uses xml format to configure Samza job, but Samza as a framework, it does not restrict configuration format. Please let us know if you have further questions, and we should detail the documents in OSS to describe the usage of certain features.
Best, Yang On Tue, Dec 17, 2019 at 9:58 PM Debraj Manna <subharaj.ma...@gmail.com> wrote: > Thanks, Yang for replaying. > > Yes, my use case is almost similar. > > Can you let me know which app-def you are referring to? I am not able to > locate yarn.am.container.label in samza-configurations > < > http://samza.apache.org/learn/documentation/latest/jobs/samza-configurations.html > > > . > Is there any samza project whose code I can refer to regarding the usage of > these configurations? > > On Wed, Dec 18, 2019 at 7:42 AM Yang Zhang <zhan3...@umn.edu> wrote: > > > Hello Debraj, > > > > We do not have a formal documentation in open source to describe how yarn > > node label is used in general. In contrast, we have an example of using > > yarn node label to specify Samza container to run over "HDD" rather than > > default "SSD" nodes. Please take a look at the following guide and let us > > know whether it can be applied for your use case. Thank you for reporting > > this issue! > > =================================================Step-by-step guide > > > > > > 1. > > > > Add the *yarn.container.label *and* yarn.am.container.label* to the > > job's *app-def* if not already present. The default of an empty string > > will keep the current default behavior of using SSD nodes. > > <?xml version="1.0" encoding="UTF-8"?> > > <application xmlns="urn:com:linkedin:ns:configuration:definition:1.0" > > name="my-application" version=""> > > <configuration-definition> > > <property name="yarn.am.container.label" default="" /> <!-- > the > > label used for launching the application master --> > > <property name="yarn.container.label" default="" /> <!-- the > > label used for other containers --> > > </configuration-definition> > > </application> > > 2. If you had to modify your *app-def* in step 1, you will need to do > a > > trigger-build to get the change to take effect. > > 3. > > > > Add the label to *application.src* for your job. The *hdd* label will > > assign your containers to machines with spinning disks instead of > solid > > state disks. > > <?xml version="1.0" encoding="UTF-8"?> > > <application xmlns="urn:com:linkedin:ns:configuration:source:1.0" > > name="my-application"> > > <configuration-source> > > <property name="yarn.container.label" value="hdd" /> > > <property name="yarn.am.container.label" value="hdd" /> > > </configuration-source> > > </application> > > > > > > 4. > > > > Deploy. > > > > ================================================= > > > > > > Best, > > > > Yang > > > > On Tue, Dec 17, 2019 at 10:13 AM Debraj Manna <subharaj.ma...@gmail.com> > > wrote: > > > > > Hi > > > > > > I am seeing running samza with yarn node label is resolved in 0.12. > > > > > > > > > > > > https://issues.apache.org/jira/browse/SAMZA-1013?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aall-tabpanel > > > > > > But I am not able to locate the relevant documentation in samza-yarn > > > documentation > > > < > > https://samza.apache.org/learn/documentation/latest/deployment/yarn.html > > > > > > > > Can someone point me to the relevant documentation? > > > > > >
