I've used the latest thrift to generate go code, and then manually created executor config which works and is able to launch jobs.
On Thu, 17 Mar 2016, 1:55 p.m. Jake Farrell, <[email protected]> wrote: > Hi Krish > We are using Thrift with go for all our api calls to Aurora, would > recommend you use Thrift 0.9.3 to interact with the api. > > happy to help answer any questions you might have > > -Jake > > On Thu, Mar 17, 2016 at 9:43 AM, Krish <[email protected]> wrote: > >> Thanks, Bill. >> >> Well I have started my foray into the the thrift API today. And I think I >> am stuck with some thrift configs. >> >> Does it matter if I use thrift v0.9.0 on the client side to talk with >> aurora using thrift 0.9.1? Are they compatible? I couldn't find any >> changelog or compatibility statement on the thrift project site. >> >> >> Since Aurora v0.12 uses thrift version 0.9.1, and the debian repos have >> 0.9.0, I had to compile the thrift compiler v0.9.1 from source. However, >> when I try to generate golang code, I think I hit a compiler bug: >> krish@krish:/tmp >> > thrift --gen go api.thrift >> ./gen-go//api/ttypes.go:2623:6: missing ',' in composite literal >> ./gen-go//api/ttypes.go:2624:19: expected '==', found '=' >> WARNING - Running 'gofmt -w ./gen-go//api/ttypes.go' failed. >> >> I can modify the golang code by hand, but I would like to play it safe >> and use the working compiler from the debian repos. >> >> Also, when I use thrift v0.9.0, and try to integrate code into a test >> golang app, it fails to find "thriftlib/api" package. Anyone faced a >> similar error and gone past it? >> I have already done a `go get >> git.apache.org/thrift.git/lib/go/thrift/...` >> <http://git.apache.org/thrift.git/lib/go/thrift/...> >> >> >> >> >> -- >> κρισhναν >> >> On Wed, Mar 16, 2016 at 10:30 PM, Bill Farner <[email protected]> wrote: >> >>> Regarding documentation - Maxim is correct that there isn't much in the >>> way of independent/holistic docs for the thrift API. There is, however, >>> scant javadoc-style documentation within the IDL spec itself: >>> https://github.com/apache/aurora/blob/master/api/src/main/thrift/org/apache/aurora/gen/api.thrift >>> >>> If you are looking to use the thrift API directly, the most difficult >>> API method will be defining the ExecutorConfig.data value when calling >>> createJob. Please don't hesitate to ask for assistance if you get to that >>> point! >>> >>> On Wed, Mar 16, 2016 at 9:19 AM, Maxim Khutornenko <[email protected]> >>> wrote: >>> >>>> 1. All APIs require thrift inputs of the structs specified, and return >>>>> thrift values only in Response.result field. >>>> >>>> Correct. There is also 'details' field that may have additional >>>> messages (of error or informational nature) >>>> >>>> 2. Is there a set of examples in the documentation to help understand >>>>> Thrift API better? >>>> >>>> The thrift API is largely undocumented. There is an effort to bring up >>>> a fully supported REST API that will presumably get documented and become >>>> much easier to use. It's mostly in flux now. >>>> >>>> 3. createJob(JobDescription desc, Lock lock): >>>> >>>> This is the API to use when you a brand new service or adhoc (batch) >>>> job created. The JobDescription is populated from the .aurora config. You >>>> may want to trace "aurora job create" client command implementation to see >>>> how it happens. >>>> >>>> 4. What is the Lock object? I see that some APIs require locking and >>>>> some don't. For example, createJob needs a Lock object as parameter, & I >>>>> am >>>>> assuming that it is required so that one does not create multiple jobs >>>>> with >>>>> the same JobKey. >>>> >>>> Ignore this object as it's an echo of the old client updater. It's now >>>> deprecated and will be removed soon. You can pass NULL for now. >>>> >>>> 5. addInstances(AddInstancesConfig cfg, Lock lock): >>>> >>>> Another echo of the client updater but this time it's got a second >>>> life. Check out its new signature and comments in the api.thrift. It's >>>> essentially a "scale-out" API that can add instances to the existing job >>>> without changing the underlying task assumptions. >>>> >>>> 6. getPendingResult(TaskQuery taskquery): >>>> >>>> It's actually 'getPendingReason' and is currently used exclusively by >>>> the UI to get the reason for a task PENDING state. >>>> >>>> 7. setQuota & getQuota for setting user level quotas. >>>> >>>> This is to set role-level quota. Currently only required for tasks with >>>> 'production=True'. Search through our docs for more details. >>>> >>>> 8. killTasks to kill all running instances of a job in the cluster. >>>> >>>> It's quite versatile and can be used to kill some or all instances of >>>> the job. >>>> >>>> 9. startJobUpdate(JobUpdateRequest request, string message): >>>> >>>> Your observations are correct. This is the main API to change your >>>> service job in any way (including adding, removing or modifying instances). >>>> >>>> An aurora scheduling question is if I start a job with 5 instances, and >>>>> there are resources available to run only 4 of them, does the entire job >>>>> block, or only the 5th instance of the job blocks? >>>> >>>> Scheduler will try to schedule as many instances as it can. Those that >>>> will not find resources will remain in PENDING state until more resources >>>> are available. In your particular example only the 5th will remain PENDING. >>>> >>>> >>>> On Wed, Mar 16, 2016 at 5:54 AM, Krish <[email protected]> >>>> wrote: >>>> >>>>> Hi, >>>>> I was going through the Aurora Thrift API to determine how to add new >>>>> jobs. >>>>> I am using aurora v0.12 released last month and have upgraded to mesos >>>>> v0.25 accordingly. >>>>> >>>>> Below is a summary of my (very limited) understanding of some APIs, & >>>>> would help it if someone can point out flaws in my understanding: >>>>> >>>>> 1. All APIs require thrift inputs of the structs specified, and >>>>> return thrift values only in Response.result field. >>>>> >>>>> 2. Is there a set of examples in the documentation to help >>>>> understand Thrift API better? >>>>> >>>>> 3. createJob(JobDescription desc, Lock lock): >>>>> This is basically the API to replace the Aurora DSL/.aurora files >>>>> for job configuration. >>>>> >>>>> 4. What is the Lock object? I see that some APIs require locking >>>>> and some don't. For example, createJob needs a Lock object as >>>>> parameter, & >>>>> I am assuming that it is required so that one does not create multiple >>>>> jobs >>>>> with the same JobKey. >>>>> >>>>> 5. addInstances(AddInstancesConfig cfg, Lock lock): >>>>> By the naming convention, it seems this is used to increase the >>>>> number of instances of a job. It will not result in stopping of current >>>>> instances of the job. >>>>> >>>>> My second explanation for this API: Since it uses a set of >>>>> instanceIds, this is used for adding already running job in slaves to >>>>> the >>>>> internal data structures of Aurora to track the job. >>>>> >>>>> 6. getPendingResult(TaskQuery taskquery): >>>>> Return the reason (in string) about why the job is PENDING. For >>>>> example: insufficient CPU. >>>>> >>>>> 7. setQuota & getQuota for setting user level quotas. >>>>> >>>>> 8. killTasks to kill all running instances of a job in the cluster. >>>>> >>>>> 9. startJobUpdate(JobUpdateRequest request, string message): >>>>> Used for updating jobs with the new TaskConfig specified. Can be >>>>> used if resource requirement changes. For example: If I wanted aurora >>>>> to >>>>> update the version of container used for a job using >>>>> TaskConfig.Container >>>>> attribute. >>>>> >>>>> >>>>> An aurora scheduling question is if I start a job with 5 instances, >>>>> and there are resources available to run only 4 of them, does the entire >>>>> job block, or only the 5th instance of the job blocks? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> κρισhναν >>>>> >>>> >>>> >>> >> >
