Re: CMIS paging query result sample code

Fri, 17 Nov 2017 01:59:09 -0800 

Hi Wentao,
OpenCMIS does pretty much what you have done, but behind the scenes. It sets the maxItems and skipCount parameters when it executes a query. Once a batch is processed, it fires the same query again with another skipCount. 


Re 1) Do you have a cluster of servers? If so, the other calls might 
have been processed by another node.



Re 2) The server needed 34 seconds of the 52 seconds to compile the 
response. That leaves us with 18 seconds, which is still a lot of time. 
OpenCMIS is usually pretty fast. Do you have an idea how big the 
responses are?



Re 3) Depending on how paging is implemented on the server side, the 
server might have to internally recalculate the first page again before 
it can serve the second page. That’s the case for many repositories. 
Every page gets slower than the page before. That’s why you should make 
the pages as big as possible if you want process all query results.



- Florian




Thanks Florian. It works.

Could you help to provide a little more details on how the  paging
works between OpenCMIS client with CMIS server on the backend? Here
are some logs I get from FileNet CMIS for FileNet access logs.

2017-11-16      14:03:09        0.02    GET
/fncmis/resources/Service?repositoryId=os1  200
2017-11-16      14:03:39        0.013   GET
/fncmis/resources/Service?repositoryId= os1  200
2017-11-16      14:03:39        0.003   GET
/fncmis/resources/Service?repositoryId= os1  200

2017-11-16      14:04:13        34.107  POST    
/fncmis/resources/os1/Query 201

2017-11-16      14:05:05        0.028   GET
/fncmis/resources/Service?repositoryId= os1  200
2017-11-16      14:05:05        0.004   GET
/fncmis/resources/Service?repositoryId= os1  200

2017-11-16      14:05:55        50.104  POST    
/fncmis/resources/os1/Query 201


There were total 6661 items back and the page size was 1500, from the
log I noticed a few things that I am not quite understand:

1. There were only 2 queries been triggered, I thought it should at
least had 5 (6661/1500)?
2. There were 52 seconds (14:05:05 - 14:04:13) between the first query
call and the subsequent calls, I guess it took a while for OpenCMIS
client to process something?  52 seconds seems like a bit longer than
expect.
3. The second query took longer than first query, but I guess it is a
question for IBM CMIS expert to answer.

Wentao

-----Original Message-----
From: Florian Müller [mailto:f...@apache.org]
Sent: Thursday, November 16, 2017 1:32 PM
To: dev@chemistry.apache.org; Lu, Wentao
Subject: Re: CMIS paging query result sample code

Hi Wentao,

If you are processing all query results, it's actually much simpler
than you think.

Please see:
https://chemistry.apache.org/docs/cmis-samples/samples/operation-context/index.html#list-batch-size

Use an Operation Context, set the maxItemsPerPage and just iterate
over the query results:



OperationContext oc = 
OperationContextUtils.createMinimumOperationContext();

oc.setMaxItemsPerPage(500);


ItemIterable<QueryResult>  results = session.query(queryString, false, 
oc);


for (QueryResult result : results) {
  System.out.println(result.getPropertyValueByQueryName("cmis:name"));
}



- Florian



Hi,

Is there a sample code for OpenCMIS to paging query result?  We have a
query that will return large result set, we want to use paging query
to avoid timeout if return all dataset in one query from backend
FileNet repository, is there a sample code we can reference?  I
googled but get confused sample code especially after the first query
call,

     while (results.getHasMoreItems()){
         PAGE_NUM ++;

//Should we do another session.query() call or just use the 
results.skipTo() from the first query result?
         ItemIterable<QueryResult>  tmp_results = 
session.query(queryString, false).skipTo(PAGE_NUM * 
PAGE_SIZE).getPage(PAGE_SIZE);   //this line or next line?

         ItemIterable<QueryResult>  tmp_results =
results.skipTo(PAGE_NUM * PAGE_SIZE).getPage(PAGE_SIZE);

        for (QueryResult result : tmp_results) {

          
System.out.println(result.getPropertyValueByQueryName("cmis:name"));

          resultCount++;
        }
    }

Here is the complete code snip
     String queryString = "SELECT cmis:name FROM InformationSystem";

     int PAGE_NUM = 0;
     int PAGE_SIZE = 500;


     ItemIterable<QueryResult> results = session.query(queryString, 
false).skipTo(PAGE_NUM * PAGE_SIZE).getPage(PAGE_SIZE);

     int resultCount = 0;
     for (QueryResult result : results) {

          
System.out.println(result.getPropertyValueByQueryName("cmis:name"));

          resultCount ++;
     }
     System.out.println("resultCount="+resultCount);

     while (results.getHasMoreItems()){
         PAGE_NUM ++;

//Should we do another session.query() call or just use the 
results.skipTo() from the first query result?
         ItemIterable<QueryResult>  tmp_results = 
session.query(queryString, false).skipTo(PAGE_NUM * 
PAGE_SIZE).getPage(PAGE_SIZE);

         ItemIterable<QueryResult>  tmp_results =
results.skipTo(PAGE_NUM * PAGE_SIZE).getPage(PAGE_SIZE);

        for (QueryResult result : tmp_results) {

          
System.out.println(result.getPropertyValueByQueryName("cmis:name"));

          resultCount++;
        }
        System.out.println("resultCount="+resultCount);
    }



Thanks
Wentao
________________________________

This email and its attachments are intended solely for the personal 
use of the individual or entity named above. Any use of this 
communication by an unintended recipient is strictly prohibited. If 
you have received this email in error, any publication, use, 
reproduction, disclosure or dissemination of its contents is strictly 
prohibited. Please immediately delete this message and its attachments 
from your computer and servers. We would also appreciate if you would 
contact us by a collect call or return email to notify us of this 
error. Thank you for your cooperation.

-BCHydroDisclaimerID5.2.8.1541























					Reply via email to