yousj created ZEPPELIN-6023:
---
Summary: The Hive JDBC Interpreter executes the same sql task for
many times and the sql returned data may not be parsed out of the result set.
Key: ZEPPELIN-6023
URL: https://issues.apache.org/jira/browse/ZEPPELIN-6023
Project: Zeppelin
Issue Type: Bug
Components: hive
Affects Versions: 0.10.1
Environment: This is my hive jdbc interpreter configuration:
!image-2024-05-31-15-36-21-357.png!
Below are the different results obtained by running zeppelin directly several
times:
!image-2024-05-31-15-38-11-200.png!
!image-2024-05-31-15-40-04-522.png!
Reporter: yousj
Attachments: image-2024-05-31-15-35-20-870.png,
image-2024-05-31-15-36-21-357.png, image-2024-05-31-15-38-11-200.png,
image-2024-05-31-15-40-04-522.png
The Hive JDBC Interpreter executes the same sql task for many times and the sql
returned data may not be parsed out of the result set. What is the cause of
this situation? I don't know if there is any configuration that causes the
result set to be confused or if this is a bug. How can I obtain the result of
the sql execution? So far I'm getting results by trying to execute multiple
times
Here is a demo of the test:
{code:java}
package com.example.demo;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.zeppelin.client.*;
import java.util.*;
public class ZeppelinDemo {
/**
* CREATE TABLE `yangjw_hive_ods.ods_test_202405291427`(
* `id` bigint COMMENT 'id',
* `t_name` string COMMENT '姓名',
* `t_address` string COMMENT '地址',
* `t_create_time` string COMMENT '创建时间',
* `t_update_time` string COMMENT '更新时间',
* `t_ext01` string COMMENT '扩展字段01',
* `t_ext02` string COMMENT '扩展字段02',
* `t_ext03` string COMMENT '扩展字段03')
* COMMENT 'ods_test_202405291427';
*
* INSERT INTO yangjw_hive_ods.ods_test_202405291427 VALUES
(4,'zhaoliu','hnzz','2024-05-30 10:01:01','2024-05-30 10:01:01','123','','');
* INSERT INTO yangjw_hive_ods.ods_test_202405291427 VALUES
(1,'zhangsan','hnzz','2024-05-29 10:01:01','2024-05-29 10:01:01','','','');
* INSERT INTO yangjw_hive_ods.ods_test_202405291427 VALUES
(2,'lisi','hnzz','2024-05-29 10:01:01','2024-05-29 10:01:01','','','');
* INSERT INTO yangjw_hive_ods.ods_test_202405291427 VALUES
(3,'wangwu','hnzz','2024-05-29 10:01:01','2024-05-29 10:01:01','','','');
*/
private static final String tableName = "
yangjw_hive_ods.ods_test_202405291427 ";
private static final String sql01 = " SELECT COUNT(1) AS num FROM " +
tableName;
private static final String sql02 = " SELECT COUNT(1) AS num FROM " +
tableName + " WHERE t_name IS NULL OR concat(t_name, '') = '';";
public static void main(String[] args) throws Exception {
List> resultList = new ArrayList<>();
List execTimeList = new ArrayList<>();
for (int i = 0; i < 10; i++) {
ClientConfig clientConfig = new ClientConfig("my_zeppelin_url");
ZeppelinClient zeppelinClient = new ZeppelinClient(clientConfig);
String taskId = "1795363739585282051";
String taskInstanceId = UUID.randomUUID().toString();
String notePath = String.format("auditTask/%s/%s", taskId,
taskInstanceId);
String zeppelinId = "hive-jdbc-tyc";
String noteId = zeppelinClient.createNote(notePath, zeppelinId);
zeppelinClient.addParagraph(noteId, "sql01", sql01);
zeppelinClient.addParagraph(noteId, "sql02", sql02);
NoteResult noteResult = zeppelinClient.executeNote(noteId);
List paragraphResultList =
noteResult.getParagraphResultList();
for (ParagraphResult paragraphResult : paragraphResultList) {
if (paragraphResult.getStatus().isCompleted()) {
resultList.add(paragraphResult.getResults());
}
}
}
ObjectMapper objectMapper = new ObjectMapper();
System.out.println(objectMapper.writeValueAsString(resultList));
}
} {code}
Run it multiple times and you may get different results :
{code:java}
[
[
{
"type": "TEXT",
"data": "INFO : Compiling
command(queryId=hive_20240531144933_087992f6-3eb6-477b-9b2d-efd0097788c1):
SELECT COUNT(1) AS num FROM yangjw_hive_ods.ods_test_202405291427 \nINFO :
Semantic Analysis Completed (retrial = false)\nINFO : Returning Hive schema:
Schema(fieldSchemas:[FieldSchema(name:num, type:bigint, comment:null)],
properties:null)\nINFO : Completed compiling
command(queryId=hive_20240531144933_087992f6-3eb6-477b-9b2d-efd0097788c1); Time
taken: 0.254 seconds\nINFO : Executing
command(queryId=hive_20240531144933_087992f6-3eb6-477b-9b2d-efd0097788c1):
SELECT COUNT(1) AS num FROM yangjw_hive_ods.ods_test_202405291427 \nINFO :
Query ID =
