liangyuanpeng commented on issue #5262: pulsar SQL (Presto) can only query data 
less than 100000 
URL: https://github.com/apache/pulsar/issues/5262#issuecomment-541121599
 
 
   There are may be other reasons.
   
   I can query 100000+ data
   
   ``pulsar-admin persistent stats-internal``  result
   
   ```
   [root@localhost apache-pulsar-2.4.1-normal]# bin/pulsar-admin persistent 
stats-internal persistent://public/default/topic-3
   
   {
     "entriesAddedCounter" : 102062,
     "numberOfEntries" : 102062,
     "totalSize" : 7623081,
     "currentLedgerEntries" : 102062,
     "currentLedgerSize" : 7623081,
     "lastLedgerCreatedTimestamp" : "2019-10-11T04:23:27.017+08:00",
     "waitingCursorsCount" : 0,
     "pendingAddEntriesCount" : 0,
     "lastConfirmedEntry" : "6:102061",
     "state" : "LedgerOpened",
     "ledgers" : [ {
       "ledgerId" : 6,
       "entries" : 0,
       "size" : 0,
       "offloaded" : false
     } ],
     "cursors" : { }
   }
   ```
   
   ``sql run`` result
   
   ```
   presto> SELECT COUNT(1) FROM pulsar."public/default"."topic-3";
    _col0  
   --------
    102307 
   (1 row)
   
   Query 20191010_202608_00012_zu36h, FINISHED, 1 node
   Splits: 19 total, 19 done (100.00%)
   1:10 [102K rows, 7.26MB] [1.47K rows/s, 107KB/s]
   ```
   
   and my test code
   
   ```
   Producer<byte[]> producer = 
client.newProducer().topic("persistent://public/default/topic-3").create();
   
           producer.sendAsync("My message".getBytes());
   
           ExecutorService executorService = Executors.newFixedThreadPool(100);
   
           int count = 102400;
           for(int i=0;i<count;i++){
               executorService.execute(new Runnable() {
                   @Override
                   public void run() {
                       String msg = ("My message"+System.currentTimeMillis()+ 
RandomUtils.nextInt(1,10000));
                       System.out.println(msg);
                       try {
                           producer.send(msg.getBytes());
                       } catch (PulsarClientException e) {
                           System.out.println(e.getMessage());
                       }
                   }
               });
           }
   ```
   
   
   pulsar 2.4.1

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Reply via email to