Thanks, Bill. Well I have started my foray into the the thrift API today. And I think I am stuck with some thrift configs.
Does it matter if I use thrift v0.9.0 on the client side to talk with aurora using thrift 0.9.1? Are they compatible? I couldn't find any changelog or compatibility statement on the thrift project site. Since Aurora v0.12 uses thrift version 0.9.1, and the debian repos have 0.9.0, I had to compile the thrift compiler v0.9.1 from source. However, when I try to generate golang code, I think I hit a compiler bug: krish@krish:/tmp > thrift --gen go api.thrift ./gen-go//api/ttypes.go:2623:6: missing ',' in composite literal ./gen-go//api/ttypes.go:2624:19: expected '==', found '=' WARNING - Running 'gofmt -w ./gen-go//api/ttypes.go' failed. I can modify the golang code by hand, but I would like to play it safe and use the working compiler from the debian repos. Also, when I use thrift v0.9.0, and try to integrate code into a test golang app, it fails to find "thriftlib/api" package. Anyone faced a similar error and gone past it? I have already done a `go get git.apache.org/thrift.git/lib/go/thrift/...` -- κρισhναν On Wed, Mar 16, 2016 at 10:30 PM, Bill Farner <[email protected]> wrote: > Regarding documentation - Maxim is correct that there isn't much in the > way of independent/holistic docs for the thrift API. There is, however, > scant javadoc-style documentation within the IDL spec itself: > https://github.com/apache/aurora/blob/master/api/src/main/thrift/org/apache/aurora/gen/api.thrift > > If you are looking to use the thrift API directly, the most difficult API > method will be defining the ExecutorConfig.data value when calling > createJob. Please don't hesitate to ask for assistance if you get to that > point! > > On Wed, Mar 16, 2016 at 9:19 AM, Maxim Khutornenko <[email protected]> > wrote: > >> 1. All APIs require thrift inputs of the structs specified, and return >>> thrift values only in Response.result field. >> >> Correct. There is also 'details' field that may have additional messages >> (of error or informational nature) >> >> 2. Is there a set of examples in the documentation to help understand >>> Thrift API better? >> >> The thrift API is largely undocumented. There is an effort to bring up a >> fully supported REST API that will presumably get documented and become >> much easier to use. It's mostly in flux now. >> >> 3. createJob(JobDescription desc, Lock lock): >> >> This is the API to use when you a brand new service or adhoc (batch) job >> created. The JobDescription is populated from the .aurora config. You may >> want to trace "aurora job create" client command implementation to see how >> it happens. >> >> 4. What is the Lock object? I see that some APIs require locking and some >>> don't. For example, createJob needs a Lock object as parameter, & I am >>> assuming that it is required so that one does not create multiple jobs with >>> the same JobKey. >> >> Ignore this object as it's an echo of the old client updater. It's now >> deprecated and will be removed soon. You can pass NULL for now. >> >> 5. addInstances(AddInstancesConfig cfg, Lock lock): >> >> Another echo of the client updater but this time it's got a second life. >> Check out its new signature and comments in the api.thrift. It's >> essentially a "scale-out" API that can add instances to the existing job >> without changing the underlying task assumptions. >> >> 6. getPendingResult(TaskQuery taskquery): >> >> It's actually 'getPendingReason' and is currently used exclusively by the >> UI to get the reason for a task PENDING state. >> >> 7. setQuota & getQuota for setting user level quotas. >> >> This is to set role-level quota. Currently only required for tasks with >> 'production=True'. Search through our docs for more details. >> >> 8. killTasks to kill all running instances of a job in the cluster. >> >> It's quite versatile and can be used to kill some or all instances of the >> job. >> >> 9. startJobUpdate(JobUpdateRequest request, string message): >> >> Your observations are correct. This is the main API to change your >> service job in any way (including adding, removing or modifying instances). >> >> An aurora scheduling question is if I start a job with 5 instances, and >>> there are resources available to run only 4 of them, does the entire job >>> block, or only the 5th instance of the job blocks? >> >> Scheduler will try to schedule as many instances as it can. Those that >> will not find resources will remain in PENDING state until more resources >> are available. In your particular example only the 5th will remain PENDING. >> >> >> On Wed, Mar 16, 2016 at 5:54 AM, Krish <[email protected]> wrote: >> >>> Hi, >>> I was going through the Aurora Thrift API to determine how to add new >>> jobs. >>> I am using aurora v0.12 released last month and have upgraded to mesos >>> v0.25 accordingly. >>> >>> Below is a summary of my (very limited) understanding of some APIs, & >>> would help it if someone can point out flaws in my understanding: >>> >>> 1. All APIs require thrift inputs of the structs specified, and >>> return thrift values only in Response.result field. >>> >>> 2. Is there a set of examples in the documentation to help >>> understand Thrift API better? >>> >>> 3. createJob(JobDescription desc, Lock lock): >>> This is basically the API to replace the Aurora DSL/.aurora files >>> for job configuration. >>> >>> 4. What is the Lock object? I see that some APIs require locking and >>> some don't. For example, createJob needs a Lock object as parameter, & I >>> am >>> assuming that it is required so that one does not create multiple jobs >>> with >>> the same JobKey. >>> >>> 5. addInstances(AddInstancesConfig cfg, Lock lock): >>> By the naming convention, it seems this is used to increase the >>> number of instances of a job. It will not result in stopping of current >>> instances of the job. >>> >>> My second explanation for this API: Since it uses a set of >>> instanceIds, this is used for adding already running job in slaves to the >>> internal data structures of Aurora to track the job. >>> >>> 6. getPendingResult(TaskQuery taskquery): >>> Return the reason (in string) about why the job is PENDING. For >>> example: insufficient CPU. >>> >>> 7. setQuota & getQuota for setting user level quotas. >>> >>> 8. killTasks to kill all running instances of a job in the cluster. >>> >>> 9. startJobUpdate(JobUpdateRequest request, string message): >>> Used for updating jobs with the new TaskConfig specified. Can be >>> used if resource requirement changes. For example: If I wanted aurora to >>> update the version of container used for a job using TaskConfig.Container >>> attribute. >>> >>> >>> An aurora scheduling question is if I start a job with 5 instances, and >>> there are resources available to run only 4 of them, does the entire job >>> block, or only the 5th instance of the job blocks? >>> >>> Thanks! >>> >>> -- >>> κρισhναν >>> >> >> >
