If you'd rather not build from source, there's also a docker image for the thrift compiler: https://hub.docker.com/r/thrift/thrift-compiler/
On Thu, Mar 17, 2016 at 9:32 AM, Krish <[email protected]> wrote: > Jake/Chris, > Thanks for the info. > When I try to install thrift v0.9.3 from source, I get an error as follows > while running `make check`: > ... > ... > [junit] Running org.apache.thrift.protocol.TestTProtocolUtil > [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time > elapsed: 0.062 sec > [junit] Running org.apache.thrift.protocol.TestTSimpleJSONProtocol > [junit] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time > elapsed: 0.046 sec > > BUILD FAILED > /tmp/thrift-0.9.3/lib/java/build.xml:202: Test > org.apache.thrift.protocol.TestTSimpleJSONProtocol failed > > Total time: 17 seconds > make[3]: *** [check-local] Error 1 > make[3]: Leaving directory `/tmp/thrift-0.9.3/lib/java' > make[2]: *** [check-am] Error 2 > make[2]: Leaving directory `/tmp/thrift-0.9.3/lib/java' > make[1]: *** [check-recursive] Error 1 > make[1]: Leaving directory `/tmp/thrift-0.9.3/lib' > make: *** [check-recursive] Error 1 > > > > > -- > κρισhναν > > On Thu, Mar 17, 2016 at 7:32 PM, Chris Bannister <[email protected]> > wrote: > >> I've used the latest thrift to generate go code, and then manually >> created executor config which works and is able to launch jobs. >> >> On Thu, 17 Mar 2016, 1:55 p.m. Jake Farrell, <[email protected]> wrote: >> >>> Hi Krish >>> We are using Thrift with go for all our api calls to Aurora, would >>> recommend you use Thrift 0.9.3 to interact with the api. >>> >>> happy to help answer any questions you might have >>> >>> -Jake >>> >>> On Thu, Mar 17, 2016 at 9:43 AM, Krish <[email protected]> >>> wrote: >>> >>>> Thanks, Bill. >>>> >>>> Well I have started my foray into the the thrift API today. And I think >>>> I am stuck with some thrift configs. >>>> >>>> Does it matter if I use thrift v0.9.0 on the client side to talk with >>>> aurora using thrift 0.9.1? Are they compatible? I couldn't find any >>>> changelog or compatibility statement on the thrift project site. >>>> >>>> >>>> Since Aurora v0.12 uses thrift version 0.9.1, and the debian repos have >>>> 0.9.0, I had to compile the thrift compiler v0.9.1 from source. However, >>>> when I try to generate golang code, I think I hit a compiler bug: >>>> krish@krish:/tmp >>>> > thrift --gen go api.thrift >>>> ./gen-go//api/ttypes.go:2623:6: missing ',' in composite literal >>>> ./gen-go//api/ttypes.go:2624:19: expected '==', found '=' >>>> WARNING - Running 'gofmt -w ./gen-go//api/ttypes.go' failed. >>>> >>>> I can modify the golang code by hand, but I would like to play it safe >>>> and use the working compiler from the debian repos. >>>> >>>> Also, when I use thrift v0.9.0, and try to integrate code into a test >>>> golang app, it fails to find "thriftlib/api" package. Anyone faced a >>>> similar error and gone past it? >>>> I have already done a `go get >>>> git.apache.org/thrift.git/lib/go/thrift/...` >>>> <http://git.apache.org/thrift.git/lib/go/thrift/...> >>>> >>>> >>>> >>>> >>>> -- >>>> κρισhναν >>>> >>>> On Wed, Mar 16, 2016 at 10:30 PM, Bill Farner <[email protected]> >>>> wrote: >>>> >>>>> Regarding documentation - Maxim is correct that there isn't much in >>>>> the way of independent/holistic docs for the thrift API. There is, >>>>> however, scant javadoc-style documentation within the IDL spec itself: >>>>> https://github.com/apache/aurora/blob/master/api/src/main/thrift/org/apache/aurora/gen/api.thrift >>>>> >>>>> If you are looking to use the thrift API directly, the most difficult >>>>> API method will be defining the ExecutorConfig.data value when calling >>>>> createJob. Please don't hesitate to ask for assistance if you get to that >>>>> point! >>>>> >>>>> On Wed, Mar 16, 2016 at 9:19 AM, Maxim Khutornenko <[email protected]> >>>>> wrote: >>>>> >>>>>> 1. All APIs require thrift inputs of the structs specified, and >>>>>>> return thrift values only in Response.result field. >>>>>> >>>>>> Correct. There is also 'details' field that may have additional >>>>>> messages (of error or informational nature) >>>>>> >>>>>> 2. Is there a set of examples in the documentation to help understand >>>>>>> Thrift API better? >>>>>> >>>>>> The thrift API is largely undocumented. There is an effort to bring >>>>>> up a fully supported REST API that will presumably get documented and >>>>>> become much easier to use. It's mostly in flux now. >>>>>> >>>>>> 3. createJob(JobDescription desc, Lock lock): >>>>>> >>>>>> This is the API to use when you a brand new service or adhoc (batch) >>>>>> job created. The JobDescription is populated from the .aurora config. You >>>>>> may want to trace "aurora job create" client command implementation to >>>>>> see >>>>>> how it happens. >>>>>> >>>>>> 4. What is the Lock object? I see that some APIs require locking and >>>>>>> some don't. For example, createJob needs a Lock object as parameter, & >>>>>>> I am >>>>>>> assuming that it is required so that one does not create multiple jobs >>>>>>> with >>>>>>> the same JobKey. >>>>>> >>>>>> Ignore this object as it's an echo of the old client updater. It's >>>>>> now deprecated and will be removed soon. You can pass NULL for now. >>>>>> >>>>>> 5. addInstances(AddInstancesConfig cfg, Lock lock): >>>>>> >>>>>> Another echo of the client updater but this time it's got a second >>>>>> life. Check out its new signature and comments in the api.thrift. It's >>>>>> essentially a "scale-out" API that can add instances to the existing job >>>>>> without changing the underlying task assumptions. >>>>>> >>>>>> 6. getPendingResult(TaskQuery taskquery): >>>>>> >>>>>> It's actually 'getPendingReason' and is currently used exclusively by >>>>>> the UI to get the reason for a task PENDING state. >>>>>> >>>>>> 7. setQuota & getQuota for setting user level quotas. >>>>>> >>>>>> This is to set role-level quota. Currently only required for tasks >>>>>> with 'production=True'. Search through our docs for more details. >>>>>> >>>>>> 8. killTasks to kill all running instances of a job in the cluster. >>>>>> >>>>>> It's quite versatile and can be used to kill some or all instances of >>>>>> the job. >>>>>> >>>>>> 9. startJobUpdate(JobUpdateRequest request, string message): >>>>>> >>>>>> Your observations are correct. This is the main API to change your >>>>>> service job in any way (including adding, removing or modifying >>>>>> instances). >>>>>> >>>>>> An aurora scheduling question is if I start a job with 5 instances, >>>>>>> and there are resources available to run only 4 of them, does the entire >>>>>>> job block, or only the 5th instance of the job blocks? >>>>>> >>>>>> Scheduler will try to schedule as many instances as it can. Those >>>>>> that will not find resources will remain in PENDING state until more >>>>>> resources are available. In your particular example only the 5th will >>>>>> remain PENDING. >>>>>> >>>>>> >>>>>> On Wed, Mar 16, 2016 at 5:54 AM, Krish <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> I was going through the Aurora Thrift API to determine how to add >>>>>>> new jobs. >>>>>>> I am using aurora v0.12 released last month and have upgraded to >>>>>>> mesos v0.25 accordingly. >>>>>>> >>>>>>> Below is a summary of my (very limited) understanding of some APIs, >>>>>>> & would help it if someone can point out flaws in my understanding: >>>>>>> >>>>>>> 1. All APIs require thrift inputs of the structs specified, and >>>>>>> return thrift values only in Response.result field. >>>>>>> >>>>>>> 2. Is there a set of examples in the documentation to help >>>>>>> understand Thrift API better? >>>>>>> >>>>>>> 3. createJob(JobDescription desc, Lock lock): >>>>>>> This is basically the API to replace the Aurora DSL/.aurora >>>>>>> files for job configuration. >>>>>>> >>>>>>> 4. What is the Lock object? I see that some APIs require locking >>>>>>> and some don't. For example, createJob needs a Lock object as >>>>>>> parameter, & >>>>>>> I am assuming that it is required so that one does not create >>>>>>> multiple jobs >>>>>>> with the same JobKey. >>>>>>> >>>>>>> 5. addInstances(AddInstancesConfig cfg, Lock lock): >>>>>>> By the naming convention, it seems this is used to increase the >>>>>>> number of instances of a job. It will not result in stopping of >>>>>>> current >>>>>>> instances of the job. >>>>>>> >>>>>>> My second explanation for this API: Since it uses a set of >>>>>>> instanceIds, this is used for adding already running job in slaves >>>>>>> to the >>>>>>> internal data structures of Aurora to track the job. >>>>>>> >>>>>>> 6. getPendingResult(TaskQuery taskquery): >>>>>>> Return the reason (in string) about why the job is PENDING. For >>>>>>> example: insufficient CPU. >>>>>>> >>>>>>> 7. setQuota & getQuota for setting user level quotas. >>>>>>> >>>>>>> 8. killTasks to kill all running instances of a job in the >>>>>>> cluster. >>>>>>> >>>>>>> 9. startJobUpdate(JobUpdateRequest request, string message): >>>>>>> Used for updating jobs with the new TaskConfig specified. Can be >>>>>>> used if resource requirement changes. For example: If I wanted >>>>>>> aurora to >>>>>>> update the version of container used for a job using >>>>>>> TaskConfig.Container >>>>>>> attribute. >>>>>>> >>>>>>> >>>>>>> An aurora scheduling question is if I start a job with 5 instances, >>>>>>> and there are resources available to run only 4 of them, does the entire >>>>>>> job block, or only the 5th instance of the job blocks? >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> -- >>>>>>> κρισhναν >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >
