EmmyMiao87 opened a new issue #2069: Multi Distinct hang while single distinct is executed quickly URL: https://github.com/apache/incubator-doris/issues/2069 **Describe the bug** select count(distinct cuid),count(distinct ext8) from baiduboxapp_event_new where event_day=20191016 and event='c_query' and upstream='na' limit 10; This sql hang. The data size: more then 300G **To Reproduce** 1. create table Table schema: CREATE TABLE `baiduboxapp_event_new` ( `event` varchar(1024) NULL COMMENT "", `event_day` int(11) NULL COMMENT "", `event_hour` varchar(1024) NULL COMMENT "", `event_time` varchar(1024) NULL COMMENT "", `cuid` varchar(1024) NULL COMMENT "", `country` varchar(1024) NULL COMMENT "", `province` varchar(1024) NULL COMMENT "", `city` varchar(1024) NULL COMMENT "", `os` varchar(1024) NULL COMMENT "", `browser` varchar(1024) NULL COMMENT "", `network` varchar(1024) NULL COMMENT "", `version` varchar(1024) NULL COMMENT "", `logid` largeint(40) NULL COMMENT "", `query` varchar(1024) NULL COMMENT "", `sa` varchar(1024) NULL COMMENT "", `upstream` varchar(1024) NULL COMMENT "", `ext1` varchar(30000) NULL COMMENT "", `ext2` varchar(1024) NULL COMMENT "", `ext3` varchar(4096) NULL COMMENT "", `ext4` varchar(1024) NULL COMMENT "", `ext5` varchar(1024) NULL COMMENT "", `ext6` varchar(1024) NULL COMMENT "", `ext7` varchar(1024) NULL COMMENT "", `ext8` varchar(1024) NULL COMMENT "", `ext9` varchar(1024) NULL COMMENT "", `ext10` varchar(1024) NULL COMMENT "", `ext11` varchar(1024) NULL COMMENT "", `ext12` varchar(1024) NULL COMMENT "", `ext13` varchar(1024) NULL COMMENT "", `ext14` varchar(1024) NULL COMMENT "", `ext15` varchar(1024) NULL COMMENT "" ) ENGINE=OLAP DISTRIBUTED BY HASH(`cuid`) BUCKETS 1 PROPERTIES ( "storage_type" = "COLUMN", "bloom_filter_columns" = "cuid, logid", ); 2. load data with 300G in one partition 3. select multi distinct 4. select single distinct is very quickly **Expected behavior** Return values **Screenshots** I1024 20:59:09.580003 9314 plan_fragment_executor.cpp:265] Open(): fragment_instance_id=cc6a173cc7b4428c-89e52e92a83cddcb I1024 20:59:09.584149 9525 internal_service.cpp:149] exec plan fragment, fragment_instance_id=cc6a173cc7b4428c-89e52e92a83cddc4, coord=TNetworkAddress(hostname=10.157.86.14, port=9020), backend=2 I1024 20:59:09.584197 9525 plan_fragment_executor.cpp:74] Prepare(): query_id=cc6a173cc7b4428c-89e52e92a83cddc2 fragment_instance_id=cc6a173cc7b4428c-89e52e92a83cddc4 backend_num=2 I1024 20:59:09.584269 9525 plan_fragment_executor.cpp:131] Using query memory limit: 2.00 GB I1024 20:59:09.584970 9315 plan_fragment_executor.cpp:265] Open(): fragment_instance_id=cc6a173cc7b4428c-89e52e92a83cddc4 I1024 20:59:14.462715 9423 olap_engine.cpp:586] get root path info cost: 20 ms. tablet counter: 8891 I1024 20:59:14.642843 9424 olap_engine.cpp:1708] begin to process report all tablets info. I1024 20:59:14.672446 9424 olap_engine.cpp:1752] success to process report all tablets info. tablet_num=8891 I1024 20:59:17.062338 10227 heartbeat_server.cpp:56] get heartbeat from FE.host:10.157.86.4, port:9020, cluster id:1100092276, counter:37 E1024 20:59:18.566884 9532 baidu_rpc_protocol.cpp:115] body_size=68059115 from 10.157.86.13:50044 is too large E1024 20:59:18.572157 9532 input_messenger.cpp:76] A message from 10.157.86.13:50044(protocol=baidu_std) is bigger than 67108864 bytes, the connection will be closed. Set max_body_size to allow bigger messages W1024 20:59:18.572239 9532 input_messenger.cpp:248] Close Socket{id=8589934849 fd=344 addr=10.157.86.13:50044:8060} (0x2c59e200): too big data E1024 20:59:21.089999 9496 baidu_rpc_protocol.cpp:115] **body_size=70824875 from 10.157.86.6:41676 is too large** E1024 20:59:21.090839 9496 input_messenger.cpp:76] A message from 10.157.86.6:41676(protocol=baidu_std) **is bigger than 67108864 bytes, the connection will be closed. Set max_body_size to allow bigger messages** W1024 20:59:21.090876 9496 input_messenger.cpp:248] Close Socket{id=129 fd=375 addr=10.157.86.6:41676:8060} (0x2c576200): too big data W1024 20:59:21.091259 9492 socket.cpp:1676] Fail to keep-write into Socket{id=385 fd=374 addr=10.157.86.6:8060:41676} (0x2c5c2200): Connection reset by peer [104] W1024 20:59:21.100333 9315 data_stream_sender.cpp:131] **failed to send brpc batch**, error=Host is down, error_text=[E1014]Got EOF of Socket{id=385 fd=374 addr=10.157.86.6:8060:41676} (0x0x2c5c2200) [R1][E112]Not connected to 10.157.86.6:8060 yet, server_id=385 [R2][E112]Not connected to 10.157.86.6:8060 yet, server_id=385 [R3][E112]Not connected to 10.157.86.6:8060 yet, server_id=385 I1024 20:59:21.203899 9532 socket.cpp:2260] Checking Socket{id=385 addr=10.157.86.6:8060} (0x2c5c2200) I1024 20:59:21.222337 9492 socket.cpp:2322] Revived Socket{id=385 addr=10.157.86.6:8060} (0x2c5c2200) I1024 20:59:21.222357 9315 plan_fragment_executor.cpp:587] Fragment cc6a173cc7b4428c-89e52e92a83cddc4:(Active: 11s519ms, % non-child: 0.00%)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
