Hi Jihoon , Thanks for clarifying to me I was trying to get results already
submitted queries from . That was my mistake :( .
I tried step by step from through rest api . Everything is fine except
step 5 I need little bit more detail for this step .
Can you explan step 5 ? How can I get full result set data ? Currently i
assume i get forst 100 records .. Am I wrong ?
In the documentation
"GET /queries/{query-id}/result/{cache-id}
It will return result set that requested. This result set encoded by
RowStoreEncoder and represented in binary form. Clients have a
responsibility for decoding result set. More information can be found in
org.apache.tajo.storage.RowStoreUtil.RowStoreDecoder. Also, this REST
response will return additional HTTP header entries, Conent-Type and
X-Tajo-Digest. X-Tajo-Digest will set the checksum of result set, not a
http message. REST service will use the SHA-1 hash function as a checksum
function. set database as current database in session values.
*Query Patameters*
Parameter Name
Parameter Type
Description
offsetIntegerIt is the index of the result set record. It must be the
positive integer.countInteger
It is the number of the result set record return after the start record.
This is required.
default value is 100
Step 1 Get Session Id I get Succeed
curl -i -H Accept:application/json -X POST
http://192.168.0.120:26880/rest/sessions -H Content-Type:application/json
-d '{ "userName": "tajo-user", "databaseName": "tpc_h10m" }'
Result : {
"id": "07d17e6a-da58-4905-869c-f7ced7f239ed", "resultCode": "OK",
"variables": { "USERNAME": "tajo-user", "SESSION_ID":
"07d17e6a-da58-4905-869c-f7ced7f239ed", "SESSION_LAST_ACCESS_TIME":
"1445174906111", "CURRENT_DATABASE": "tpc_h10m" } }
Step 2 Post Your Query with session Id Succeed
curl -i -H Accept:application/json -H
X-Tajo-Session:07d17e6a-da58-4905-869c-f7ced7f239ed -X POST
http://192.168.0.120:26880/rest/queries -H Content-Type:application/json -d
'{ "query": "select * from customer" }'
{ "resultCode": "OK", "query": "select * from customer", "uri": "
http://192.168.0.120:26880/rest/queries/q_1445080186694_0006"; }
Step 3 - Get result status Succeed
curl -i -H Accept:application/json -H
X-Tajo-Session:07d17e6a-da58-4905-869c-f7ced7f239ed -X GET
http://192.168.0.120:26880/rest/queries/q_1445080186694_0006
{ "context": { "keyVals": { "USERNAME": "tajo-user", "default_space_uri":
"file:/home/tajo/tajo/data/tajo/warehouse", "SESSION_ID":
"07d17e6a-da58-4905-869c-f7ced7f239ed", "default_space_root_uri": "file:///",
"SESSION_LAST_ACCESS_TIME": "1445174994879", "CURRENT_DATABASE": "tpc_h10m"
} }, "sql": "select * from customer", "queryState": "QUERY_SUCCEEDED",
"progress": 0, "startTime": 1445174994897, "finishTime": 1445174994897,
"queryMasterPort": 0, "queryMasterClientPort": 0, "queryMasterInfoPort": 0,
"queryIdStr": "q_1445080186694_0006"
Step 4 - Get Result Set Id Succeed
curl -i -H Accept:application/json -H
X-Tajo-Session:07d17e6a-da58-4905-869c-f7ced7f239ed -X GET
http://192.168.0.120:26880/rest/queries/q_1445080186694_0006/result
{ "resultCode": "OK", "schema": { "fields": [ { "name":
"tpc_h10m.customer.c_custkey", "typeDesc": { "dataType": { "type": "INT8" }
} }, { "name": "tpc_h10m.customer.c_name", "typeDesc": { "dataType": {
"type": "TEXT" } } }, { "name": "tpc_h10m.customer.c_address", "typeDesc":
{ "dataType": { "type": "TEXT" } } }, { "name":
"tpc_h10m.customer.c_nationkey", "typeDesc": { "dataType": { "type": "INT8"
} } }, { "name": "tpc_h10m.customer.c_phone", "typeDesc": { "dataType": {
"type": "TEXT" } } }, { "name": "tpc_h10m.customer.c_acctbal", "typeDesc":
{ "dataType": { "type": "FLOAT8" } } }, { "name":
"tpc_h10m.customer.c_mktsegment", "typeDesc": { "dataType": { "type": "TEXT"
} } }, { "name": "tpc_h10m.customer.c_comment", "typeDesc": { "dataType": {
"type": "TEXT" } } } ], "fieldsByQualifiedName": {
"tpc_h10m.customer.c_nationkey": 3, "tpc_h10m.customer.c_mktsegment": 6,
"tpc_h10m.customer.c_name": 1, "tpc_h10m.customer.c_phone": 4,
"tpc_h10m.customer.c_comment": 7, "tpc_h10m.customer.c_custkey": 0,
"tpc_h10m.customer.c_acctbal": 5, "tpc_h10m.customer.c_address": 2 },
"fieldsByName": { "c_custkey": [ 0 ], "c_phone": [ 4 ], "c_acctbal": [ 5 ],
"c_mktsegment": [ 6 ], "c_address": [ 2 ], "c_nationkey": [ 3 ], "c_name":
[ 1 ], "c_comment": [ 7 ] } }, "bytesNum": 0, "resultset": { "id":
1596878999304630500, "link": "
http://192.168.0.120:26880/rest/queries/q_1445080186694_0006/result/1596878999304630493
" } }
Step 5 - Get Result Set Seems OK
curl -i -H Accept:*/* -H
X-Tajo-Session:07d17e6a-da58-4905-869c-f7ced7f239ed -X GET
http://192.168.0.120:26880/rest/queries/q_1445080186694_0006/result/1596878999304630493
200, OK
X-Tajo-Offset: 100
X-Tajo-Digest: F1W/xGYXLGIZ5jO8c7XM7EK4T+Y=
X-Tajo-EOS: false
X-Tajo-Count: 100
Content-Type: application/octet-stream
Result Set is
"�eCustomer#000000101sMmL2rNeHDltovSm Y12-514-298-3699@�.�\)
MACHINERY1 sleep. pending packages detect slyly ironic
pack�fCustomer#000000102 UAtflJ06
fn9zBfKjInkQZlWtqaA29-324-978-8538@��\)BUILDINGPously
regular dependencies nag among the furiously express dinos. blithely
final�gCustomer#000000103 8KIsQX4LJ7QMsj6DrtFtXu0nUEdV,8a
19-216-107-2107@���ffffBUILDINGkfuriously
pending notornis boost slyly around the blithely ironic ideas? final, even
instructions cajole fl�hCustomer#000000104 9mcCK L7rt0SwiYtrbO88DiZS7U d7M
20-966-284-8065��c
=p�� FURNITURE rate carefully slyly special
pla�iCustomer#0000001054iSJe4L SPjg7kJj98Yz3z0B
20-793-553-6417@����\ MACHINERY5l pains cajole even accounts. quietly
final instructi�jCustomer#000000106 xGCOEAUjUNG11-751-989-4627@���
=p� MACHINERYflose slyly. ironic accounts along the evenly regular
theodolites wake about the special, final gifts.
�kCustomer#000000107 Zwg64UZ,q7GRqo3zm7P1tZIRshBDz25-336-529-9919@��L����
AUTOMOBILEbcounts cajole slyly. regular requests wake. furiously regular
deposits about the blithely final fo�lCustomer#000000108
GPoeEvpKo115-908-619-7526@��\(�BUILDINGCrefully ironic deposits sleep.
regular, unusual requests wake
slyly�mCustomer#000000109#OOOkYBgCMzgMQXUmkocoLb56rfrdWp2NE2c26-992-422-8153��`�����BUILDINGqes.
fluffily final dependencies sleep along the blithely even pinto beans.
final deposits haggle furiously furiou�nCustomer#000000110mymPfgphaYXNYtk
20-893-536-2069@�&�p��"
Thanks ,
Emin
On Sun, Oct 18, 2015 at 1:55 PM, Jihoon Son <[email protected]> wrote:
> Maybe you are trying to get the result of already submitted queries via
> tsql or web ui. If so, you cannot get the result using REST api.
>
> Honestly, this is not a problem of REST api. The reason is that you have
> only one chance to get the result for each query. Tajo stores query results
> on hdfs, and once you get a result, tajo automatically removes it from
> hdfs. So, if you submitted your queries using tsql or web ui which get the
> query result immediately, the results of those queries have already been
> removed.
>
> Or, if you submitted queries using REST api and cannot get query results,
> would you share which steps you went through more detail?
> Also, tajo master's log will be very helpful. You can find it from
> ${tajo_home}/logs/tajo-${user.name}-master-xxx.log.
>
> Thanks,
> Jihoon
>
> 2015년 10월 18일 (일) 오후 9:37, Emin TÜYLÜCE <[email protected]>님이 작성:
>
>> Hi Jihoon ,
>>
>> curl --header "X-Tajo-Session: e65d9652-f7c6-4047-ba89-2cadc9c19938"
>> http://192.168.0.120:26880/rest/queries/q_1445080186694_0004/result/7754785746404232624
>>
>>
>> I tried this and still no result but no error messages
>>
>> Thanks ,
>>
>> Emin
>>
>> On Sun, Oct 18, 2015 at 1:26 PM, Jihoon Son <[email protected]> wrote:
>>
>>> Oh, I mean, you also need to add 'X-Tajo-Session' to the header of your
>>> http request like the former 'get query result' api.
>>> Here is an example.
>>> $ curl --header "X-Tajo-Session: 2d6f80f2-8189-4c0f-8f23-578f4fa52096"
>>> http://192.168.0.120:26880/rest/queries/q_1445080186694_0004/result/7754785746404232624
>>>
>>> Thanks,
>>> Jihoon
>>>
>>> 2015년 10월 18일 (일) 오후 9:09, Emin TÜYLÜCE <[email protected]>님이 작성:
>>>
>>>> Thanks Jihoon , How I should send session Id ? Could you give me an
>>>> example please ? Because in the documentation that part is not clear .
>>>>
>>>> Thanks ,
>>>>
>>>> Emin
>>>>
>>>> On Sun, Oct 18, 2015 at 2:18 AM, Jihoon Son <[email protected]> wrote:
>>>>
>>>>> This message also means that your session id is missed.
>>>>> It seems that our error message is very unkind to users. I opened a
>>>>> jira ticket to improve it (
>>>>> https://issues.apache.org/jira/browse/TAJO-1937).
>>>>>
>>>>> Thanks!
>>>>> Jihoon
>>>>>
>>>>> 2015년 10월 18일 (일) 오전 3:25, Emin TÜYLÜCE <[email protected]>님이 작성:
>>>>>
>>>>>> Hi Jihoon / Char thank you very much now I am getting similar
>>>>>> querydetails . Now I want to get query result dataset,
>>>>>> { ..... "bytesNum": 0, "resultset": { "id": 7754785746404232000,
>>>>>> "link": "
>>>>>> http://192.168.0.120:26880/rest/queries/q_1445080186694_0004/result/7754785746404232624
>>>>>> " } }
>>>>>>
>>>>>> First When I use link in json result with curl I am getting
>>>>>> "curl
>>>>>> http://192.168.0.120:26880/rest/queries/q_1445080186694_0004/result/7754785746404232624
>>>>>>
>>>>>> MessageBodyWriter not found for media type=application/octet-stream,
>>>>>> type=class org.apache.tajo.ws.rs.responses.ExceptionResponse,
>>>>>> genericType=class org.apache.tajo.ws.rs.responses.ExceptionResponse."
>>>>>>
>>>>>> Whats is the best way to get result dataset
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> Emin
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sat, Oct 17, 2015 at 5:11 PM, CharSyam <[email protected]> wrote:
>>>>>>
>>>>>>> Tajo Rest Api needs X-Tajo-Session header.
>>>>>>> Currently, If session header doesn't exis, It will return NPE.
>>>>>>> I will soon change error type from internal error to bad request.
>>>>>>> Thanks.
>>>>>>>
>>>>>>> 2015-10-18 0:42 GMT+09:00 Jihoon Son <[email protected]>:
>>>>>>>
>>>>>>>> Emin, this problem is due to the missing session id in the request
>>>>>>>> header.
>>>>>>>> You need to explicitly add a session id for Tajo to your http
>>>>>>>> request header like below.
>>>>>>>>
>>>>>>>> $ curl --header "X-Tajo-Session:
>>>>>>>> 2d6f80f2-8189-4c0f-8f23-578f4fa52096"
>>>>>>>> http://localhost:26880/rest/queries/q_1445094169341_0001/result
>>>>>>>>
>>>>>>>> {"resultCode":"OK","schema":{"fields":[{"name":"?count","typeDesc":{"dataType":{"type":"INT8"}}}],"fieldsByQualifiedName":{"?count":0},"fieldsByName":{"?count":[0]}},"bytesNum":0,"resultset":{"id":3713217288559068505,"link":"
>>>>>>>> http://localhost:26880/rest/queries/q_1445091590251_0001/result/3713217288559068505
>>>>>>>> "}}
>>>>>>>>
>>>>>>>> You can create a session using a REST api. Would you test again
>>>>>>>> with a session id?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Jihoon
>>>>>>>>
>>>>>>>> 2015년 10월 17일 (토) 오후 11:48, Jihoon Son <[email protected]>님이 작성:
>>>>>>>>
>>>>>>>>> Hi Emin, thanks for reporting. This is definitely our bug.
>>>>>>>>> I created a Jira ticket to fix this problem (
>>>>>>>>> https://issues.apache.org/jira/browse/TAJO-1936).
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>> Jihoon
>>>>>>>>>
>>>>>>>>> 2015년 10월 17일 (토) 오후 8:52, Emin TÜYLÜCE <[email protected]>님이 작성:
>>>>>>>>>
>>>>>>>>>> Hi Jihoon ,
>>>>>>>>>>
>>>>>>>>>> .
>>>>>>>>>> When i call any of my executed quries with query id to access
>>>>>>>>>> results I am getting 500, Internal Server Error
>>>>>>>>>> This is my url
>>>>>>>>>> http://192.168.0.120:26880/rest/queries/q_1445080186694_0001/result
>>>>>>>>>> my reference document is
>>>>>>>>>> https://issues.apache.org/jira/secure/attachment/12698879/TajoClientRestAPIDesign_v0.1.pdf
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks ,
>>>>>>>>>>
>>>>>>>>>> Emin
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Oct 14, 2015 at 1:59 AM, Jihoon Son <[email protected]>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Since we moved to java 8 from 0.12.0, it is strange that 0.11.1
>>>>>>>>>>> doesn't work well with java 7.
>>>>>>>>>>> I'll investigate it.
>>>>>>>>>>> Thanks!
>>>>>>>>>>> 2015년 10월 14일 (수) 오전 6:41, Emin TÜYLÜCE <[email protected]>님이
>>>>>>>>>>> 작성:
>>>>>>>>>>>
>>>>>>>>>>>> After Upgrade Java 8 everything works fine .
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks .
>>>>>>>>>>>> Emin
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Oct 13, 2015 at 8:15 PM, Emin TÜYLÜCE <
>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi , I extract "tajo-0.11.1-SNAPSHOT.tar.gz" and I copied
>>>>>>>>>>>>> hadoop folders from my previous extract . when i run Tajo i am
>>>>>>>>>>>>> getting
>>>>>>>>>>>>> error message in web frontend of Tajo.
>>>>>>>>>>>>>
>>>>>>>>>>>>> "HTTP ERROR: 500
>>>>>>>>>>>>>
>>>>>>>>>>>>> PWC6033: Unable to compile class for JSP
>>>>>>>>>>>>>
>>>>>>>>>>>>> RequestURI=/
>>>>>>>>>>>>> Caused by:
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.jasper.JasperException: PWC6033: Unable to compile
>>>>>>>>>>>>> class for JSP
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.apache.jasper.compiler.DefaultErrorHandler.javacError(DefaultErrorHandler.java:123)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.apache.jasper.compiler.ErrorDispatcher.javacError(ErrorDispatcher.java:296)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:376)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.apache.jasper.compiler.Compiler.compile(Compiler.java:437)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:608)"
>>>>>>>>>>>>>
>>>>>>>>>>>>> error message .
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks ,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Emin
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sat, Oct 10, 2015 at 3:06 PM, Emin TÜYLÜCE <
>>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi , I used standalone desktop package of Tajo , I will try
>>>>>>>>>>>>>> nightly build.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Emin
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sat, Oct 10, 2015 at 2:34 PM, Jihoon Son <
>>>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Seems strange. I also tested getting query list using rest
>>>>>>>>>>>>>>> client, and found it works well with my laptop.
>>>>>>>>>>>>>>> When did you build your snapshot version? Would you try the
>>>>>>>>>>>>>>> nightly build version (
>>>>>>>>>>>>>>> https://builds.apache.org/view/All/job/Tajo-0.11.0-nightly/lastSuccessfulBuild/artifact/tajo-dist/target/tajo-0.11.1-SNAPSHOT.tar.gz
>>>>>>>>>>>>>>> )?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In addition, please check if the session is expired. You can
>>>>>>>>>>>>>>> see in Tajo master's log. The default log path is
>>>>>>>>>>>>>>> ${TAJO_HOME}/logs/tajo-${
>>>>>>>>>>>>>>> user.name}-master-xxx.log.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>> Jihoon
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2015년 10월 10일 (토) 오후 9:32, Emin TÜYLÜCE <[email protected]>님이
>>>>>>>>>>>>>>> 작성:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi ,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I am using Tajo 0.11.0-SNAPSHOT . I wanted to use Tajo
>>>>>>>>>>>>>>>> rest api . In my try Databases rest functions are working very
>>>>>>>>>>>>>>>> well but
>>>>>>>>>>>>>>>> when I try query related functions (Get Post ) in rest client
>>>>>>>>>>>>>>>> I am getting
>>>>>>>>>>>>>>>> 404 messages .
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This is my Url : http://192.168.0.120:26880/rest/queries
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks ,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Emin
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>
