wangbo opened a new issue #5267:
URL: https://github.com/apache/incubator-doris/issues/5267


   **Question**
   When I upgrade doris be from 0.12 to 0.13-11,
   I found be starts very slowly;
   LOG is as below:
   BE start to load tablet time:
   ```
   I0107 20:26:00.874502 35609 data_dir.cpp:670] load rowset from meta 
finished, data dir: /data6/olap
   I0107 20:26:00.874577 35609 data_dir.cpp:675] begin loading tablet from meta
   ```
   BE finish to load table time:
   ```
   I0107 20:40:56.520390 35165 storage_engine.cpp:105] success to init storage 
engine.
   I0107 20:40:53.236734 35609 data_dir.cpp:714] load tablet from meta 
finished, loaded tablet: 225803, error tablet: 0, path: /data6/olap
   ```
   It costs about 14min.
   
   Use perf tool to sample time-consuming, result is as below:
   
![image](https://user-images.githubusercontent.com/12951030/105129228-95c8a300-5b1f-11eb-9378-9c1b3b3cc849.png)
   ```
   -   76.55%     0.23%  palo_be          palo_be             [.] 
doris::TabletManager::load_tablet_from_meta                                     
                                   ▒
      - 76.33% doris::TabletManager::load_tablet_from_meta                      
                                                                                
                     ▒
         - 23.34% __GI___access                                                 
                                                                                
                     ▒
            - 22.47% system_call_fastpath                                       
                                                                                
                     ▒
               - 22.43% sys_access                                              
                                                                                
                     ▒
                  - 22.33% sys_faccessat                                        
                                                                                
                     ▒
                     - 21.51% user_path_at                                      
                                                                                
                     ▒
                        - 21.48% user_path_at_empty                             
                                                                                
                     ▒
                           - 21.22% filename_lookup                             
                                                                                
                     ▒
                              - 21.09% path_lookupat                            
                                                                                
                     ▒
                                 + 11.59% lookup_slow                           
                                                                                
                     ▒
                                 + 8.71% link_path_walk                         
                                                                                
                     ▒
         + 17.83% doris::TabletMeta::create_from_file                           
                                                                                
                     ▒
         + 10.79% pthread_once                                                  
                                                                                
                     ▒
         + 8.42% google::protobuf::MessageLite::ParseFromCodedStream            
                                                                                
                     ▒
         + 8.16% doris::Tablet::_contains_rowset                                
                                                                                
                     ▒
         + 5.28% doris::TabletManager::_add_tablet_unlocked                     
                                                                                
                     ▒
         + 0.81% pthread_rwlock_wrlock           
   ```
   The main sys call in ```load_tablet_from_meta``` is path check.
   
   After I removed path check when start be, LOG as below
   The time which BE begins to load tablet meta:
   ```
   I0111 17:06:43.250362 33369 data_dir.cpp:675] begin loading tablet from meta
   ```
   The time which BE load tablet meta finished:
   ```
   I0111 17:07:37.121706 33371 data_dir.cpp:714] load tablet from meta 
finished, loaded tablet: 224436, error tablet: 0, path: /data8/olap
   ```
   It costs about one minute;
   
   By the way, the BE has about 1.75 million tablet
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]



---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to