liangyuanpeng commented on issue #5262: pulsar SQL (Presto) can only query data less than 100000 URL: https://github.com/apache/pulsar/issues/5262#issuecomment-541121599 There are may be other reasons. I can query 100000+ data ``pulsar-admin persistent stats-internal`` result ``` [root@localhost apache-pulsar-2.4.1-normal]# bin/pulsar-admin persistent stats-internal persistent://public/default/topic-3 { "entriesAddedCounter" : 102062, "numberOfEntries" : 102062, "totalSize" : 7623081, "currentLedgerEntries" : 102062, "currentLedgerSize" : 7623081, "lastLedgerCreatedTimestamp" : "2019-10-11T04:23:27.017+08:00", "waitingCursorsCount" : 0, "pendingAddEntriesCount" : 0, "lastConfirmedEntry" : "6:102061", "state" : "LedgerOpened", "ledgers" : [ { "ledgerId" : 6, "entries" : 0, "size" : 0, "offloaded" : false } ], "cursors" : { } } ``` ``sql run`` result ``` presto> SELECT COUNT(1) FROM pulsar."public/default"."topic-3"; _col0 -------- 102307 (1 row) Query 20191010_202608_00012_zu36h, FINISHED, 1 node Splits: 19 total, 19 done (100.00%) 1:10 [102K rows, 7.26MB] [1.47K rows/s, 107KB/s] ``` and my test code ``` Producer<byte[]> producer = client.newProducer().topic("persistent://public/default/topic-3").create(); producer.sendAsync("My message".getBytes()); ExecutorService executorService = Executors.newFixedThreadPool(100); int count = 102400; for(int i=0;i<count;i++){ executorService.execute(new Runnable() { @Override public void run() { String msg = ("My message"+System.currentTimeMillis()+ RandomUtils.nextInt(1,10000)); System.out.println(msg); try { producer.send(msg.getBytes()); } catch (PulsarClientException e) { System.out.println(e.getMessage()); } } }); } ``` pulsar 2.4.1
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
