lightning-L commented on issue #4876: URL: https://github.com/apache/kyuubi/issues/4876#issuecomment-1576109317
@huangzhir I checked the code again. https://github.com/apache/kyuubi/blob/master/kyuubi-server/src/main/scala/org/apache/kyuubi/session/KyuubiSessionImpl.scala#L199 The `waitForEngineLaunched()` is executed when operation is not LaunchEngine. And LaunchEngine is running asynchronously by default. https://github.com/apache/kyuubi/blob/master/kyuubi-server/src/main/scala/org/apache/kyuubi/operation/LaunchEngine.scala#L65 And this is the unit-test.log from my local, see `======test:` key word I have added locally for debugging: ``` 22:14:52.670 ScalaTest-run INFO BackendServiceMetricSuite: ======test: server started 22:14:52.702 ScalaTest-run-running-BackendServiceMetricSuite INFO BackendServiceMetricSuite: ===== TEST OUTPUT FOR o.a.k.server.BackendServiceMetricSuite: 'backend service metric test' ===== 22:14:53.020 KyuubiTBinaryFrontendHandler-Pool: Thread-45 INFO KyuubiTBinaryFrontendService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V10 22:14:53.029 KyuubiTBinaryFrontendHandler-Pool: Thread-45 INFO KyuubiSessionManager: Opening session for [email protected] 22:14:53.090 KyuubiTBinaryFrontendHandler-Pool: Thread-45 INFO OperationLog: Creating operation log file /Users/tiliao/Documents/Code/incubator-kyuubi/server_operation_logs/47af9c1f-e720-4c93-8531-b8cd1006fde6/b48c7bbe-04f4-443b-b8ba-0e5a3cb9d5e8 22:14:53.093 KyuubiTBinaryFrontendHandler-Pool: Thread-45 INFO KyuubiSessionManager: tiliao's session with SessionHandle [47af9c1f-e720-4c93-8531-b8cd1006fde6] is opened, current opening sessions 1 22:14:53.093 KyuubiSessionManager-exec-pool: Thread-46 INFO LaunchEngine: Processing tiliao's query[b48c7bbe-04f4-443b-b8ba-0e5a3cb9d5e8]: PENDING_STATE -> RUNNING_STATE, statement: LaunchEngine 22:14:53.094 KyuubiTBinaryFrontendHandler-Pool: Thread-45 INFO MetricsSystem: =====test: update timer, key: kyuubi.backend_service.open_session, duration: 64872263, unit: NANOSECONDS 22:14:53.112 KyuubiSessionManager-exec-pool: Thread-46 INFO CuratorFrameworkImpl: Starting 22:14:53.112 KyuubiSessionManager-exec-pool: Thread-46 INFO ZooKeeper: Initiating client connection, connectString=192.168.0.103:60669 sessionTimeout=60000 watcher=org.apache.kyuubi.shaded.curator.ConnectionState@7a8912ab 22:14:53.115 KyuubiSessionManager-exec-pool: Thread-46-SendThread(192.168.0.103:60669) INFO ClientCnxn: Opening socket connection to server 192.168.0.103/192.168.0.103:60669. Will not attempt to authenticate using SASL (unknown error) 22:14:53.118 KyuubiSessionManager-exec-pool: Thread-46-SendThread(192.168.0.103:60669) INFO ClientCnxn: Socket connection established to 192.168.0.103/192.168.0.103:60669, initiating session 22:14:53.118 NIOServerCxn.Factory:/192.168.0.103:0 INFO NIOServerCnxnFactory: Accepted socket connection from /192.168.0.103:60739 22:14:53.119 NIOServerCxn.Factory:/192.168.0.103:0 INFO ZooKeeperServer: Client attempting to establish new session at /192.168.0.103:60739 22:14:53.124 ScalaTest-run-running-BackendServiceMetricSuite INFO KyuubiConnection: Starting to get launch engine log. 22:14:53.130 KyuubiSessionManager-exec-pool: Thread-46 INFO HadoopCredentialsManager: Scheduling renewal in 0 ms. ... ... 22:14:53.429 KyuubiSessionManager-exec-pool: Thread-46 INFO EngineRef: Launching engine: /Users/tiliao/Documents/Software/spark-3.2.1-bin-hadoop2.7/bin/spark-submit \ --class org.apache.kyuubi.engine.spark.SparkSQLEngine \ --conf spark.hive.server2.thrift.resultset.default.fetch.size=1000 \ --conf spark.kyuubi.client.ipAddress=127.0.0.1 \ --conf spark.kyuubi.client.version=1.8.0-SNAPSHOT \ --conf spark.kyuubi.engine.credentials=SERUUwAAAA== \ --conf spark.kyuubi.engine.submit.time=1685888093401 \ --conf spark.kyuubi.frontend.protocols=THRIFT_BINARY \ --conf spark.kyuubi.ha.addresses=192.168.0.103:60669 \ --conf spark.kyuubi.ha.engine.ref.id=47af9c1f-e720-4c93-8531-b8cd1006fde6 \ --conf spark.kyuubi.ha.namespace=/kyuubi_1.8.0-SNAPSHOT_USER_SPARK_SQL/tiliao/default \ --conf spark.kyuubi.ha.zookeeper.auth.type=NONE \ --conf spark.kyuubi.ha.zookeeper.connection.retry.policy=ONE_TIME \ --conf spark.kyuubi.metrics.json.interval=PT0.1S \ --conf spark.kyuubi.metrics.json.location=/var/folders/hk/7j9sqdtn55j3cq_gv5qvp5pm39d49n/T/kyuubi-3969fed7-fa68-4fc3-b5ad-10c612223064 \ --conf spark.kyuubi.metrics.reporters=JSON \ --conf spark.kyuubi.server.ipAddress=192.168.0.103 \ --conf spark.kyuubi.session.connection.url=192.168.0.103:60727 \ --conf spark.kyuubi.session.engine.check.interval=PT1S \ --conf spark.kyuubi.session.engine.idle.timeout=PT5S \ --conf spark.kyuubi.session.real.user=tiliao \ --conf spark.kyuubi.zookeeper.embedded.client.port=0 \ --conf spark.kyuubi.zookeeper.embedded.data.dir=/var/folders/hk/7j9sqdtn55j3cq_gv5qvp5pm39d49n/T/kyuubi-308cd5b3-9142-4978-be3f-a4a4bdba10a1 \ --conf spark.app.name=kyuubi_USER_SPARK_SQL_tiliao_default_47af9c1f-e720-4c93-8531-b8cd1006fde6 \ --conf spark.kubernetes.driver.label.kyuubi-unique-tag=47af9c1f-e720-4c93-8531-b8cd1006fde6 \ --conf spark.sql.catalogImplementation=in-memory \ --conf spark.ui.enabled=false \ --conf spark.yarn.tags=KYUUBI,47af9c1f-e720-4c93-8531-b8cd1006fde6 \ --proxy-user tiliao /Users/tiliao/Documents/Code/incubator-kyuubi/externals/kyuubi-spark-sql-engine/target/kyuubi-spark-sql-engine_2.12-1.8.0-SNAPSHOT.jar 22:14:53.436 KyuubiSessionManager-exec-pool: Thread-46 INFO ProcBuilder: Logging to /var/folders/hk/7j9sqdtn55j3cq_gv5qvp5pm39d49n/T/tiliao-32142424-39a5-493c-8392-ef6705daa3b4/kyuubi-spark-sql-engine.log.0 22:14:58.163 KyuubiTBinaryFrontendHandler-Pool: Thread-45 INFO MetricsSystem: =====test: update timer, key: kyuubi.backend_service.get_operation_status, duration: 5002672658, unit: NANOSECONDS 22:14:58.242 KyuubiTBinaryFrontendHandler-Pool: Thread-45 INFO MetricsSystem: =====test: update timer, key: kyuubi.backend_service.fetch_results, duration: 29478618, unit: NANOSECONDS ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
