simhadri-g commented on PR #6452: URL: https://github.com/apache/hive/pull/6452#issuecomment-4529926918
Hi @ayushtkn , This is really awesome ! I tried out the hive Kubernetes operator using the helm chart you have published here:https://hub.docker.com/r/ayushtkn/hive-operator After fixing minor issue on my end of installing helm and enabling Kubernetes in my docker desktop , all the ozone, zookeeper, postgres and hive pods did come up seamlessly. That was awesome! <img width="1483" height="880" alt="image" src="https://github.com/user-attachments/assets/810ccc73-1bd7-4f7b-b956-66eae63862da"; /> Once all the pods were up, I was able to launch hive queries on llap demons via beeline . I even tested it with my s3 buckets and i did not face any major issue. You can find the full terminal logs below. I do have a few comments in readme and a few comments on improvements but i think we can handle them in a new jira. ``` PS I:\simhadrig\hive> helm install hive oci://registry-1.docker.io/ayushtkn/hive-operator --version 4.3.0-SNAPSHOT --set operator.image.repository="ayushtkn/hive" --set operator.image.tag="operator-4.3.0-SNAPSHOT" --set cluster.image="ayushtkn/hive:4.3.0-SNAPSHOT" --set cluster.database.type=postgres --set cluster.database.url="jdbc:postgresql://postgres-postgresql:5432/metastore" --set cluster.database.driver="org.postgresql.Driver" --set cluster.database.username=hive --set cluster.database.passwordSecretRef.name=hive-db-secret --set cluster.database.passwordSecretRef.key=password --set cluster.database.driverJarUrl="https://repo1.maven.org/maven2/org/postgresql/postgresql/42.7.5/postgresql-42.7.5.jar"; --set cluster.zookeeper.quorum="zookeeper:2181" --set cluster.storage.coreSiteOverrides."fs\.defaultFS"="s3a://hive" --set cluster.storage.coreSiteOverrides."fs\.s3a\.endpoint"="http://ozone-s3g-rest:9878"; --set-string cluster.storage.coreSiteOverri des."fs\.s3a\.path\.style\.access"=true --set 'cluster.storage.envVars[0].name=HADOOP_OPTIONAL_TOOLS' --set 'cluster.storage.envVars[0].value=hadoop-aws' --set 'cluster.storage.envVars[1].name=AWS_ACCESS_KEY_ID' --set 'cluster.storage.envVars[1].value=ozone' --set 'cluster.storage.envVars[2].name=AWS_SECRET_ACCESS_KEY' --set 'cluster.storage.envVars[2].value=ozone' Pulled: registry-1.docker.io/ayushtkn/hive-operator:4.3.0-SNAPSHOT Digest: sha256:031447a5bd55a0fefec2d242d6dcb03c4e57e8f3e9a0e937f64f28263b496832 NAME: hive LAST DEPLOYED: Mon May 25 02:01:30 2026 NAMESPACE: default STATUS: deployed REVISION: 1 DESCRIPTION: Install complete TEST SUITE: None PS I:\simhadrig\hive> kubectl wait --for=condition=Ready hivecluster/hive --timeout=300s PS I:\simhadrig\hive> kubectl exec -it deployment/hive-hiveserver2 -- beeline -u "jdbc:hive2://hive-hiveserver2:10000/" SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/hive/lib/log4j-slf4j-impl-2.25.3.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Connecting to jdbc:hive2://hive-hiveserver2:10000/ Connected to: Apache Hive (version 4.3.0-SNAPSHOT) Driver: Hive JDBC (version 4.3.0-SNAPSHOT) Transaction isolation: TRANSACTION_REPEATABLE_READ Beeline version 4.3.0-SNAPSHOT by Apache Hive 0: jdbc:hive2://hive-hiveserver2:10000/> create table hive_example(a string, b int) partitioned by(c int); > alter table hive_example add partition(c=1); > insert into hive_example partition(c=1) values('a', 1), ('a', 2),('b',3); > select count(distinct a) from hive_example; > select sum(b) from hive_example; INFO : Compiling command(queryId=hive_20260524203703_59b12949-84ff-4ee3-86cc-70ad36d3002f): create table hive_example(a string, b int) partitioned by(c int) INFO : HMS client filtering is enabled. INFO : Resolved metastore uris: [thrift://hive-metastore:9083] INFO : Trying to connect to metastore with URI (thrift://hive-metastore:9083) in binary transport mode INFO : Opened a connection to metastore, URI (thrift://hive-metastore:9083) current connections: 6 INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20260524203703_59b12949-84ff-4ee3-86cc-70ad36d3002f); Time taken: 8.76 seconds INFO : Concurrency mode is disabled, not creating a lock manager INFO : Executing command(queryId=hive_20260524203703_59b12949-84ff-4ee3-86cc-70ad36d3002f): create table hive_example(a string, b int) partitioned by(c int) INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20260524203703_59b12949-84ff-4ee3-86cc-70ad36d3002f); Time taken: 15.567 seconds No rows affected (26.192 seconds) INFO : Compiling command(queryId=hive_20260524203730_aca893f3-184b-46db-9625-9ceec743dbe2): alter table hive_example add partition(c=1) INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20260524203730_aca893f3-184b-46db-9625-9ceec743dbe2); Time taken: 0.865 seconds INFO : Concurrency mode is disabled, not creating a lock manager INFO : Executing command(queryId=hive_20260524203730_aca893f3-184b-46db-9625-9ceec743dbe2): alter table hive_example add partition(c=1) INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20260524203730_aca893f3-184b-46db-9625-9ceec743dbe2); Time taken: 4.181 seconds No rows affected (5.184 seconds) INFO : Compiling command(queryId=hive_20260524203735_40867723-378d-4da4-bd32-c30c582bcf0d): insert into hive_example partition(c=1) values('a', 1), ('a', 2),('b',3) INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:col1, type:string, comment:null), FieldSchema(name:col2, type:int, comment:null)], properties:null) INFO : Completed compiling command(queryId=hive_20260524203735_40867723-378d-4da4-bd32-c30c582bcf0d); Time taken: 17.279 seconds INFO : Concurrency mode is disabled, not creating a lock manager INFO : Executing command(queryId=hive_20260524203735_40867723-378d-4da4-bd32-c30c582bcf0d): insert into hive_example partition(c=1) values('a', 1), ('a', 2),('b',3) INFO : Query ID = hive_20260524203735_40867723-378d-4da4-bd32-c30c582bcf0d INFO : Total jobs = 1 INFO : Launching Job 1 out of 1 INFO : Starting task [Stage-1:MAPRED] in serial mode INFO : Subscribed to counters: [] for queryId: hive_20260524203735_40867723-378d-4da4-bd32-c30c582bcf0d INFO : Tez session hasn't been created yet. Opening session INFO : Dag name: insert into hive_exam...... ('a', 2),('b',3) (Stage-1) INFO : HS2 Host: [hive-hiveserver2-757b7b5987-257nv], Query ID: [hive_20260524203735_40867723-378d-4da4-bd32-c30c582bcf0d], Dag ID: [dag_1779654818589_0000_1], DAG App ID: [application_1779654818589_0000], DAG App address: [hive-tezam-0.hive-tezam.default.svc.cluster.local] INFO : Status: Running (Executing on YARN cluster with App id application_1779654818589_0000) ---------------------------------------------------------------------------------------------- VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED ---------------------------------------------------------------------------------------------- Map 1 .......... llap SUCCEEDED 1 1 0 0 0 0 Reducer 2 ...... llap SUCCEEDED 1 1 0 0 0 0 ---------------------------------------------------------------------------------------------- VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 35.02 s ---------------------------------------------------------------------------------------------- INFO : Status: DAG finished successfully in 33.12 seconds INFO : DAG ID: dag_1779654818589_0000_1 INFO : INFO : Query Execution Summary INFO : ---------------------------------------------------------------------------------------------- INFO : OPERATION DURATION INFO : ---------------------------------------------------------------------------------------------- INFO : Compile Query 17.28s INFO : Prepare Plan 9.60s INFO : Get Query Coordinator (AM) 0.26s INFO : Submit Plan 4.99s INFO : Start DAG 0.65s INFO : Run DAG 33.12s INFO : ---------------------------------------------------------------------------------------------- INFO : INFO : Task Execution Summary INFO : ---------------------------------------------------------------------------------------------- INFO : VERTICES DURATION(ms) CPU_TIME(ms) GC_TIME(ms) INPUT_RECORDS OUTPUT_RECORDS INFO : ---------------------------------------------------------------------------------------------- INFO : Map 1 24727.00 0 0 3 1 INFO : Reducer 2 2006.00 0 0 1 0 INFO : ---------------------------------------------------------------------------------------------- INFO : FileSystem Counters Summary INFO : INFO : Scheme: S3A INFO : ---------------------------------------------------------------------------------------------- INFO : VERTICES BYTES_READ READ_OPS LARGE_READ_OPS BYTES_WRITTEN WRITE_OPS INFO : ---------------------------------------------------------------------------------------------- INFO : Map 1 0B 6 0 12B 4 INFO : Reducer 2 0B 6 0 187B 4 INFO : ---------------------------------------------------------------------------------------------- INFO : INFO : Scheme: FILE INFO : ---------------------------------------------------------------------------------------------- INFO : VERTICES BYTES_READ READ_OPS LARGE_READ_OPS BYTES_WRITTEN WRITE_OPS INFO : ---------------------------------------------------------------------------------------------- INFO : Map 1 64B 0 0 114B 0 INFO : Reducer 2 0B 0 0 0B 0 INFO : ---------------------------------------------------------------------------------------------- INFO : INFO : org.apache.tez.common.counters.DAGCounter: INFO : NUM_SUCCEEDED_TASKS: 2 INFO : TOTAL_LAUNCHED_TASKS: 2 INFO : DURATION_SUCCEEDED_TASKS_MILLIS: 26324 INFO : RACK_LOCAL_TASKS: 1 INFO : AM_CPU_MILLISECONDS: 25650 INFO : WALL_CLOCK_MILLIS: 26324 INFO : AM_GC_TIME_MILLIS: 511 INFO : INITIAL_HELD_CONTAINERS: 0 INFO : TOTAL_CONTAINERS_USED: 2 INFO : TOTAL_CONTAINER_LAUNCH_COUNT: 2 INFO : TOTAL_CONTAINER_RELEASE_COUNT: 2 INFO : NODE_USED_COUNT: 1 INFO : NODE_TOTAL_COUNT: 2 INFO : File System Counters: INFO : FILE_BYTES_READ: 64 INFO : FILE_BYTES_WRITTEN: 114 INFO : FILE_READ_OPS: 0 INFO : FILE_LARGE_READ_OPS: 0 INFO : FILE_WRITE_OPS: 0 INFO : S3A_BYTES_READ: 0 INFO : S3A_BYTES_WRITTEN: 199 INFO : S3A_READ_OPS: 12 INFO : S3A_LARGE_READ_OPS: 0 INFO : S3A_WRITE_OPS: 8 INFO : org.apache.tez.common.counters.TaskCounter: INFO : REDUCE_INPUT_GROUPS: 1 INFO : REDUCE_INPUT_RECORDS: 1 INFO : COMBINE_INPUT_RECORDS: 0 INFO : SPILLED_RECORDS: 2 INFO : NUM_SHUFFLED_INPUTS: 1 INFO : NUM_SKIPPED_INPUTS: 0 INFO : NUM_FAILED_SHUFFLE_INPUTS: 0 INFO : MERGED_MAP_OUTPUTS: 1 INFO : INPUT_RECORDS_PROCESSED: 4 INFO : INPUT_SPLIT_LENGTH_BYTES: 1 INFO : OUTPUT_RECORDS: 1 INFO : OUTPUT_LARGE_RECORDS: 0 INFO : OUTPUT_BYTES: 70 INFO : OUTPUT_BYTES_WITH_OVERHEAD: 78 INFO : OUTPUT_BYTES_PHYSICAL: 82 INFO : ADDITIONAL_SPILLS_BYTES_WRITTEN: 0 INFO : ADDITIONAL_SPILLS_BYTES_READ: 82 INFO : ADDITIONAL_SPILL_COUNT: 0 INFO : SHUFFLE_CHUNK_COUNT: 1 INFO : SHUFFLE_BYTES: 82 INFO : SHUFFLE_BYTES_DECOMPRESSED: 78 INFO : SHUFFLE_BYTES_TO_MEM: 0 INFO : SHUFFLE_BYTES_TO_DISK: 0 INFO : SHUFFLE_BYTES_DISK_DIRECT: 82 INFO : NUM_MEM_TO_DISK_MERGES: 0 INFO : NUM_DISK_TO_DISK_MERGES: 0 INFO : SHUFFLE_PHASE_TIME: 158 INFO : MERGE_PHASE_TIME: 197 INFO : FIRST_EVENT_RECEIVED: 87 INFO : LAST_EVENT_RECEIVED: 87 INFO : HIVE: INFO : CREATED_FILES: 2 INFO : DESERIALIZE_ERRORS: 0 INFO : RECORDS_IN_Map_1: 3 INFO : RECORDS_OUT_0: 1 INFO : RECORDS_OUT_1_default.hive_example: 3 INFO : RECORDS_OUT_INTERMEDIATE_Map_1: 1 INFO : RECORDS_OUT_INTERMEDIATE_Reducer_2: 0 INFO : RECORDS_OUT_OPERATOR_FS_15: 1 INFO : RECORDS_OUT_OPERATOR_FS_4: 3 INFO : RECORDS_OUT_OPERATOR_GBY_13: 1 INFO : RECORDS_OUT_OPERATOR_GBY_7: 1 INFO : RECORDS_OUT_OPERATOR_MAP_0: 0 INFO : RECORDS_OUT_OPERATOR_RS_8: 1 INFO : RECORDS_OUT_OPERATOR_SEL_1: 1 INFO : RECORDS_OUT_OPERATOR_SEL_14: 1 INFO : RECORDS_OUT_OPERATOR_SEL_3: 3 INFO : RECORDS_OUT_OPERATOR_SEL_6: 3 INFO : RECORDS_OUT_OPERATOR_TS_0: 1 INFO : RECORDS_OUT_OPERATOR_UDTF_2: 3 INFO : TOTAL_TABLE_ROWS_WRITTEN: 3 INFO : Shuffle Errors: INFO : BAD_ID: 0 INFO : CONNECTION: 0 INFO : IO_ERROR: 0 INFO : WRONG_LENGTH: 0 INFO : WRONG_MAP: 0 INFO : WRONG_REDUCE: 0 ---------------------------------------------------------------------------------------------- VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED ---------------------------------------------------------------------------------------------- INFO : Starting task [Stage-2:DEPENDENCY_COLLECTION] in serial mode 0 0 0 INFO : Starting task [Stage-0:MOVE] in serial mode 1 0 0 0 0 INFO : Loading data to table default.hive_example partition (c=1) from s3a://hive/hive/warehouse/hive_example/c=1/.hive-staging_hive_2026-05-24_20-37-35_688_3563499212412533844-1/-ext-10000.02 s ---------------------------------------------------------------------------------------------- VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED ---------------------------------------------------------------------------------------------- Map 1 .......... llap SUCCEEDED 1 1 0 0 0 0 Reducer 2 ...... llap SUCCEEDED 1 1 0 0 0 0 ---------------------------------------------------------------------------------------------- VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 45.67 s ---------------------------------------------------------------------------------------------- INFO : Partition {c=1} stats: [numFiles=1, numRows=0, totalSize=12, rawDataSize=0, numFilesErasureCoded=0] INFO : StatsTask took 2980 INFO : Completed executing command(queryId=hive_20260524203735_40867723-378d-4da4-bd32-c30c582bcf0d); Time taken: 61.102 seconds 3 rows affected (78.489 seconds) INFO : Compiling command(queryId=hive_20260524203854_d1018576-27ac-4a0f-8cbf-01bab307db0e): select count(distinct a) from hive_example INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null) INFO : Completed compiling command(queryId=hive_20260524203854_d1018576-27ac-4a0f-8cbf-01bab307db0e); Time taken: 7.146 seconds INFO : Concurrency mode is disabled, not creating a lock manager INFO : Executing command(queryId=hive_20260524203854_d1018576-27ac-4a0f-8cbf-01bab307db0e): select count(distinct a) from hive_example INFO : Query ID = hive_20260524203854_d1018576-27ac-4a0f-8cbf-01bab307db0e INFO : Total jobs = 1 INFO : Launching Job 1 out of 1 INFO : Starting task [Stage-1:MAPRED] in serial mode INFO : Subscribed to counters: [] for queryId: hive_20260524203854_d1018576-27ac-4a0f-8cbf-01bab307db0e INFO : Session is already open INFO : Dag name: select count(distinct a) from hive_example (Stage-1) INFO : HS2 Host: [hive-hiveserver2-757b7b5987-257nv], Query ID: [hive_20260524203854_d1018576-27ac-4a0f-8cbf-01bab307db0e], Dag ID: [dag_1779654818589_0000_2], DAG App ID: [application_1779654818589_0000], DAG App address: [hive-tezam-0.hive-tezam.default.svc.cluster.local] INFO : Status: Running (Executing on YARN cluster with App id application_1779654818589_0000) ---------------------------------------------------------------------------------------------- VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED ---------------------------------------------------------------------------------------------- Map 1 .......... llap SUCCEEDED 1 1 0 0 0 0 Reducer 2 ...... llap SUCCEEDED 1 1 0 0 0 0 Reducer 3 ...... llap SUCCEEDED 1 1 0 0 0 0 ---------------------------------------------------------------------------------------------- VERTICES: 03/03 [==========================>>] 100% ELAPSED TIME: 10.48 s ---------------------------------------------------------------------------------------------- INFO : Status: DAG finished successfully in 9.92 seconds INFO : DAG ID: dag_1779654818589_0000_2 INFO : INFO : Query Execution Summary INFO : ---------------------------------------------------------------------------------------------- INFO : OPERATION DURATION INFO : ---------------------------------------------------------------------------------------------- INFO : Compile Query 7.15s INFO : Prepare Plan 0.67s INFO : Get Query Coordinator (AM) 0.00s INFO : Submit Plan 1.71s INFO : Start DAG 0.57s INFO : Run DAG 9.96s INFO : ---------------------------------------------------------------------------------------------- INFO : INFO : Task Execution Summary INFO : ---------------------------------------------------------------------------------------------- INFO : VERTICES DURATION(ms) CPU_TIME(ms) GC_TIME(ms) INPUT_RECORDS OUTPUT_RECORDS INFO : ---------------------------------------------------------------------------------------------- INFO : Map 1 5866.00 0 0 3 2 INFO : Reducer 2 562.00 0 0 2 1 INFO : Reducer 3 1007.00 0 0 1 0 INFO : ---------------------------------------------------------------------------------------------- INFO : FileSystem Counters Summary INFO : INFO : Scheme: S3A INFO : ---------------------------------------------------------------------------------------------- INFO : VERTICES BYTES_READ READ_OPS LARGE_READ_OPS BYTES_WRITTEN WRITE_OPS INFO : ---------------------------------------------------------------------------------------------- INFO : Map 1 12B 1 0 0B 0 INFO : Reducer 2 0B 0 0 0B 0 INFO : Reducer 3 0B 6 0 101B 4 INFO : ---------------------------------------------------------------------------------------------- INFO : INFO : Scheme: FILE INFO : ---------------------------------------------------------------------------------------------- INFO : VERTICES BYTES_READ READ_OPS LARGE_READ_OPS BYTES_WRITTEN WRITE_OPS INFO : ---------------------------------------------------------------------------------------------- INFO : Map 1 64B 0 0 54B 0 INFO : Reducer 2 0B 0 0 0B 0 INFO : Reducer 3 0B 0 0 0B 0 INFO : ---------------------------------------------------------------------------------------------- INFO : INFO : org.apache.tez.common.counters.DAGCounter: INFO : NUM_SUCCEEDED_TASKS: 3 INFO : TOTAL_LAUNCHED_TASKS: 3 INFO : DURATION_SUCCEEDED_TASKS_MILLIS: 7206 INFO : DATA_LOCAL_TASKS: 1 INFO : AM_CPU_MILLISECONDS: 8120 INFO : WALL_CLOCK_MILLIS: 7206 INFO : AM_GC_TIME_MILLIS: 0 INFO : INITIAL_HELD_CONTAINERS: 0 INFO : TOTAL_CONTAINERS_USED: 3 INFO : TOTAL_CONTAINER_LAUNCH_COUNT: 3 INFO : TOTAL_CONTAINER_RELEASE_COUNT: 3 INFO : NODE_USED_COUNT: 1 INFO : NODE_TOTAL_COUNT: 2 INFO : File System Counters: INFO : FILE_BYTES_READ: 64 INFO : FILE_BYTES_WRITTEN: 54 INFO : FILE_READ_OPS: 0 INFO : FILE_LARGE_READ_OPS: 0 INFO : FILE_WRITE_OPS: 0 INFO : S3A_BYTES_READ: 12 INFO : S3A_BYTES_WRITTEN: 101 INFO : S3A_READ_OPS: 7 INFO : S3A_LARGE_READ_OPS: 0 INFO : S3A_WRITE_OPS: 4 INFO : org.apache.tez.common.counters.TaskCounter: INFO : REDUCE_INPUT_GROUPS: 2 INFO : REDUCE_INPUT_RECORDS: 2 INFO : COMBINE_INPUT_RECORDS: 0 INFO : SPILLED_RECORDS: 4 INFO : NUM_SHUFFLED_INPUTS: 1 INFO : NUM_SKIPPED_INPUTS: 0 INFO : NUM_FAILED_SHUFFLE_INPUTS: 0 INFO : MERGED_MAP_OUTPUTS: 1 INFO : INPUT_RECORDS_PROCESSED: 2 INFO : INPUT_SPLIT_LENGTH_BYTES: 12 INFO : OUTPUT_RECORDS: 3 INFO : APPROXIMATE_INPUT_RECORDS: 1 INFO : OUTPUT_LARGE_RECORDS: 0 INFO : OUTPUT_BYTES: 11 INFO : OUTPUT_BYTES_WITH_OVERHEAD: 29 INFO : OUTPUT_BYTES_PHYSICAL: 61 INFO : ADDITIONAL_SPILLS_BYTES_WRITTEN: 0 INFO : ADDITIONAL_SPILLS_BYTES_READ: 22 INFO : ADDITIONAL_SPILL_COUNT: 0 INFO : SHUFFLE_CHUNK_COUNT: 1 INFO : SHUFFLE_BYTES: 22 INFO : SHUFFLE_BYTES_DECOMPRESSED: 18 INFO : SHUFFLE_BYTES_TO_MEM: 0 INFO : SHUFFLE_BYTES_TO_DISK: 0 INFO : SHUFFLE_BYTES_DISK_DIRECT: 22 INFO : NUM_MEM_TO_DISK_MERGES: 0 INFO : NUM_DISK_TO_DISK_MERGES: 0 INFO : SHUFFLE_PHASE_TIME: 97 INFO : MERGE_PHASE_TIME: 76 INFO : FIRST_EVENT_RECEIVED: 69 INFO : LAST_EVENT_RECEIVED: 69 INFO : DATA_BYTES_VIA_EVENT: 15 INFO : HIVE: INFO : CREATED_FILES: 1 INFO : DESERIALIZE_ERRORS: 0 INFO : RECORDS_IN_Map_1: 3 INFO : RECORDS_OUT_0: 1 INFO : RECORDS_OUT_INTERMEDIATE_Map_1: 2 INFO : RECORDS_OUT_INTERMEDIATE_Reducer_2: 1 INFO : RECORDS_OUT_INTERMEDIATE_Reducer_3: 0 INFO : RECORDS_OUT_OPERATOR_FS_19: 1 INFO : RECORDS_OUT_OPERATOR_GBY_13: 2 INFO : RECORDS_OUT_OPERATOR_GBY_15: 2 INFO : RECORDS_OUT_OPERATOR_GBY_16: 1 INFO : RECORDS_OUT_OPERATOR_GBY_18: 1 INFO : RECORDS_OUT_OPERATOR_MAP_0: 0 INFO : RECORDS_OUT_OPERATOR_RS_14: 2 INFO : RECORDS_OUT_OPERATOR_RS_17: 1 INFO : RECORDS_OUT_OPERATOR_SEL_12: 3 INFO : RECORDS_OUT_OPERATOR_TS_0: 3 INFO : Shuffle Errors: INFO : BAD_ID: 0 INFO : CONNECTION: 0 INFO : IO_ERROR: 0 INFO : WRONG_LENGTH: 0 INFO : WRONG_MAP: 0 INFO : WRONG_REDUCE: 0 INFO : org.apache.hadoop.hive.llap.LlapThreadLocalStatistics$LlapExecutorCounters: INFO : EXECUTOR_CPU_NS: 4180180300 INFO : EXECUTOR_USER_NS: 3170000000 INFO : org.apache.hadoop.hive.llap.counters.LlapIOCounters: INFO : CACHE_MISS_BYTES: 12 INFO : CONSUMER_TIME_NS: 2885588 INFO : DECODE_TIME_NS: 47502348 INFO : IO_CPU_NS: 805223800 INFO : IO_USER_NS: 740000000 INFO : NUM_DECODED_BATCHES: 1 INFO : NUM_VECTOR_BATCHES: 1 INFO : ROWS_EMITTED: 3 INFO : TOTAL_IO_TIME_NS: 1167676778 INFO : org.apache.hadoop.hive.llap.counters.LlapWmCounters: ---------------------------------------------------------------------------------------------- VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED ---------------------------------------------------------------------------------------------- Map 1 .......... llap SUCCEEDED 1 1 0 0 0 0 Reducer 2 ...... llap SUCCEEDED 1 1 0 0 0 0 Reducer 3 ...... llap SUCCEEDED 1 1 0 0 0 0 ---------------------------------------------------------------------------------------------- VERTICES: 03/03 [==========================>>] 100% ELAPSED TIME: 11.02 s ---------------------------------------------------------------------------------------------- INFO : Completed executing command(queryId=hive_20260524203854_d1018576-27ac-4a0f-8cbf-01bab307db0e); Time taken: 13.939 seconds +------+ | _c0 | +------+ | 2 | +------+ 1 row selected (21.814 seconds) INFO : Compiling command(queryId=hive_20260524203916_aecfe59c-a918-4c32-9ac0-4f39c808e11d): select sum(b) from hive_example INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null) INFO : Completed compiling command(queryId=hive_20260524203916_aecfe59c-a918-4c32-9ac0-4f39c808e11d); Time taken: 4.025 seconds INFO : Concurrency mode is disabled, not creating a lock manager INFO : Executing command(queryId=hive_20260524203916_aecfe59c-a918-4c32-9ac0-4f39c808e11d): select sum(b) from hive_example INFO : Query ID = hive_20260524203916_aecfe59c-a918-4c32-9ac0-4f39c808e11d INFO : Total jobs = 1 INFO : Launching Job 1 out of 1 INFO : Starting task [Stage-1:MAPRED] in serial mode INFO : Subscribed to counters: [] for queryId: hive_20260524203916_aecfe59c-a918-4c32-9ac0-4f39c808e11d INFO : Session is already open INFO : Dag name: select sum(b) from hive_example (Stage-1) INFO : HS2 Host: [hive-hiveserver2-757b7b5987-257nv], Query ID: [hive_20260524203916_aecfe59c-a918-4c32-9ac0-4f39c808e11d], Dag ID: [dag_1779654818589_0000_3], DAG App ID: [application_1779654818589_0000], DAG App address: [hive-tezam-0.hive-tezam.default.svc.cluster.local] INFO : Status: Running (Executing on YARN cluster with App id application_1779654818589_0000) ---------------------------------------------------------------------------------------------- VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED ---------------------------------------------------------------------------------------------- Map 1 .......... llap SUCCEEDED 1 1 0 0 0 0 Reducer 2 ...... llap SUCCEEDED 1 1 0 0 0 0 ---------------------------------------------------------------------------------------------- VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 20.78 s ---------------------------------------------------------------------------------------------- INFO : Status: DAG finished successfully in 20.35 seconds INFO : DAG ID: dag_1779654818589_0000_3 INFO : INFO : Query Execution Summary INFO : ---------------------------------------------------------------------------------------------- INFO : OPERATION DURATION INFO : ---------------------------------------------------------------------------------------------- INFO : Compile Query 4.03s INFO : Prepare Plan 0.51s INFO : Get Query Coordinator (AM) 0.00s INFO : Submit Plan 0.25s INFO : Start DAG 0.13s INFO : Run DAG 20.35s INFO : ---------------------------------------------------------------------------------------------- INFO : INFO : Task Execution Summary INFO : ---------------------------------------------------------------------------------------------- INFO : VERTICES DURATION(ms) CPU_TIME(ms) GC_TIME(ms) INPUT_RECORDS OUTPUT_RECORDS INFO : ---------------------------------------------------------------------------------------------- INFO : Map 1 525.00 0 0 3 1 INFO : Reducer 2 18743.00 0 0 1 0 INFO : ---------------------------------------------------------------------------------------------- INFO : FileSystem Counters Summary INFO : INFO : Scheme: S3A INFO : ---------------------------------------------------------------------------------------------- INFO : VERTICES BYTES_READ READ_OPS LARGE_READ_OPS BYTES_WRITTEN WRITE_OPS INFO : ---------------------------------------------------------------------------------------------- INFO : Map 1 12B 2 0 0B 0 INFO : Reducer 2 0B 6 0 101B 4 INFO : ---------------------------------------------------------------------------------------------- INFO : INFO : Scheme: FILE INFO : ---------------------------------------------------------------------------------------------- INFO : VERTICES BYTES_READ READ_OPS LARGE_READ_OPS BYTES_WRITTEN WRITE_OPS INFO : ---------------------------------------------------------------------------------------------- INFO : Map 1 0B 0 0 0B 0 INFO : Reducer 2 0B 0 0 0B 0 INFO : ---------------------------------------------------------------------------------------------- INFO : INFO : org.apache.tez.common.counters.DAGCounter: INFO : NUM_SUCCEEDED_TASKS: 2 INFO : TOTAL_LAUNCHED_TASKS: 2 INFO : DURATION_SUCCEEDED_TASKS_MILLIS: 19017 INFO : DATA_LOCAL_TASKS: 1 INFO : AM_CPU_MILLISECONDS: 9160 INFO : WALL_CLOCK_MILLIS: 19017 INFO : AM_GC_TIME_MILLIS: 373 INFO : INITIAL_HELD_CONTAINERS: 0 INFO : TOTAL_CONTAINERS_USED: 2 INFO : TOTAL_CONTAINER_LAUNCH_COUNT: 2 INFO : TOTAL_CONTAINER_RELEASE_COUNT: 2 INFO : NODE_USED_COUNT: 2 INFO : NODE_TOTAL_COUNT: 2 INFO : File System Counters: INFO : FILE_BYTES_READ: 0 INFO : FILE_BYTES_WRITTEN: 0 INFO : FILE_READ_OPS: 0 INFO : FILE_LARGE_READ_OPS: 0 INFO : FILE_WRITE_OPS: 0 INFO : S3A_BYTES_READ: 12 INFO : S3A_BYTES_WRITTEN: 101 INFO : S3A_READ_OPS: 8 INFO : S3A_LARGE_READ_OPS: 0 INFO : S3A_WRITE_OPS: 4 INFO : org.apache.tez.common.counters.TaskCounter: INFO : SPILLED_RECORDS: 0 INFO : NUM_SHUFFLED_INPUTS: 0 INFO : NUM_FAILED_SHUFFLE_INPUTS: 0 INFO : INPUT_RECORDS_PROCESSED: 2 INFO : INPUT_SPLIT_LENGTH_BYTES: 12 INFO : OUTPUT_RECORDS: 1 INFO : APPROXIMATE_INPUT_RECORDS: 1 INFO : OUTPUT_LARGE_RECORDS: 0 INFO : OUTPUT_BYTES: 3 INFO : OUTPUT_BYTES_WITH_OVERHEAD: 11 INFO : OUTPUT_BYTES_PHYSICAL: 39 INFO : ADDITIONAL_SPILLS_BYTES_WRITTEN: 0 INFO : ADDITIONAL_SPILLS_BYTES_READ: 0 INFO : ADDITIONAL_SPILL_COUNT: 0 INFO : SHUFFLE_BYTES: 0 INFO : SHUFFLE_BYTES_DECOMPRESSED: 0 INFO : SHUFFLE_BYTES_TO_MEM: 0 INFO : SHUFFLE_BYTES_TO_DISK: 0 INFO : SHUFFLE_BYTES_DISK_DIRECT: 0 INFO : SHUFFLE_PHASE_TIME: 108 INFO : FIRST_EVENT_RECEIVED: 107 INFO : LAST_EVENT_RECEIVED: 107 INFO : DATA_BYTES_VIA_EVENT: 15 INFO : HIVE: INFO : CREATED_FILES: 1 INFO : DESERIALIZE_ERRORS: 0 INFO : RECORDS_IN_Map_1: 3 INFO : RECORDS_OUT_0: 1 INFO : RECORDS_OUT_INTERMEDIATE_Map_1: 1 INFO : RECORDS_OUT_INTERMEDIATE_Reducer_2: 0 INFO : RECORDS_OUT_OPERATOR_FS_11: 1 INFO : RECORDS_OUT_OPERATOR_GBY_10: 1 INFO : RECORDS_OUT_OPERATOR_GBY_8: 1 INFO : RECORDS_OUT_OPERATOR_MAP_0: 0 INFO : RECORDS_OUT_OPERATOR_RS_9: 1 INFO : RECORDS_OUT_OPERATOR_SEL_7: 3 INFO : RECORDS_OUT_OPERATOR_TS_0: 3 INFO : org.apache.hadoop.hive.llap.LlapThreadLocalStatistics$LlapExecutorCounters: INFO : EXECUTOR_CPU_NS: 10931285300 INFO : EXECUTOR_USER_NS: 9650000000 INFO : org.apache.hadoop.hive.llap.counters.LlapIOCounters: INFO : CACHE_MISS_BYTES: 12 INFO : CONSUMER_TIME_NS: 31652541 INFO : DECODE_TIME_NS: 6697414 INFO : IO_CPU_NS: 147236800 INFO : IO_USER_NS: 120000000 INFO : NUM_DECODED_BATCHES: 1 INFO : NUM_VECTOR_BATCHES: 1 INFO : ROWS_EMITTED: 3 INFO : TOTAL_IO_TIME_NS: 262142261 INFO : org.apache.hadoop.hive.llap.counters.LlapWmCounters: INFO : GUARANTEED_QUEUED_NS: 0 INFO : GUARANTEED_RUNNING_NS: 0 INFO : SPECULATIVE_QUEUED_NS: 1262906425 INFO : SPECULATIVE_RUNNING_NS: 16623777320 INFO : org.apache.hadoop.hive.ql.exec.tez.HiveInputCounters: INFO : GROUPED_INPUT_SPLITS_Map_1: 1 INFO : INPUT_DIRECTORIES_Map_1: 1 INFO : INPUT_FILES_Map_1: 1 INFO : RAW_INPUT_SPLITS_Map_1: 1 INFO : Completed executing command(queryId=hive_20260524203916_aecfe59c-a918-4c32-9ac0-4f39c808e11d); Time taken: 21.61 seconds +------+ | _c0 | +------+ | 6 | +------+ 1 row selected (25.961 seconds) 0: jdbc:hive2://hive-hiveserver2:10000/> org.jline.reader.UserInterruptException Closing: 0: jdbc:hive2://hive-hiveserver2:10000/ command terminated with exit code 2 PS I:\simhadrig\hive> ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
