wulishann33 opened a new issue, #37503: URL: https://github.com/apache/doris/issues/37503
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version Doris version：2.0 beta Python version : 3.10 ### What's Wrong? I am using the LOAD DATA INFILE method to import local csv files, looping through local directory files and importing them one by one. During the import process, when the data volume of the table reaches 45G the following error occurs: `Traceback (most recent call last): File "C:\weather-data-processing\venv\lib\site-packages\mysql\connector\connection_cext.py", line 705, in cmd_query self._cmysql.query( _mysql_connector.MySQLInterfaceError: Unexpected exception: submit task failed, queue size is full: Mysql Load The above exception was the direct cause of the following exception: Traceback (most recent call last): File "C:\weather-data-processing\getWeatherSanple\process\import_data_test.py", line 48, in <module> cursor.execute(sql) File "C:\weather-data-processing\venv\lib\site-packages\mysql\connector\cursor_cext.py", line 357, in execute result = self._connection.cmd_query( File "C:\weather-data-processing\venv\lib\site-packages\mysql\connector\opentelemetry\context_propagation.py", line 97, in wrapper return method(cnx, *args, **kwargs) File "C:\weather-data-processing\venv\lib\site-packages\mysql\connector\connection_cext.py", line 713, in cmd_query raise get_mysql_exception( mysql.connector.errors.DatabaseError: 1105 (HY000): Unexpected exception: submit task failed, queue size is full: Mysql Load` I had searched all issues and didn't see any similar problems. ### What You Expected? I want to know why the queue is full and how to solve it. I manually closed the connection every time I created it, and the default expiration time of the queue should be 8 hours. If the queue is blocked, it should have expired. This problem has occurred for three days and the source of the problem has not been found yet. ### How to Reproduce? def insert_dataframe_to_doris(df, connection, file_path): connection = reconnect_if_needed(connection) cursor = connection.cursor() df = preprocess_dataframe(df) df.to_csv(file_path, index=False) try: file_path = file_path.replace('\\', '\\\\') sql = rf""" LOAD DATA LOCAL INFILE '{file_path}' INTO TABLE {doris_table} COLUMNS TERMINATED BY ',' LINES TERMINATED BY '\n' IGNORE 1 LINES """ cursor.execute(sql) except Exception as e: logging.error(f"Error inserting row {file_path}: {e}") save_progress(file_path, 0) return False connection.commit() return True ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
