Xiangdong Huang created IOTDB-367:
-------------------------------------
Summary: Aggregation query performance for the data in memory
Key: IOTDB-367
URL: https://issues.apache.org/jira/browse/IOTDB-367
Project: Apache IoTDB
Issue Type: Improvement
Reporter: Xiangdong Huang
A user follows and modifies the JDBCExample and then reports a performance
problem:
1. Two series are created.
!https://mail.google.com/mail/u/0?ui=2&ik=dfcbf3c637&attid=0.1&permmsgid=msg-f:1652861281026133792&th=16f02493ee8f4f20&view=fimg&sz=s0-l75-ft&attbid=ANGjdJ_ohwmWuAixzEuDLdub7ai1AiTLlKZ3afZcP-k3w6jvqNhV6hrB9kwBHY6LPu7VAAeqh_bWicRmpFNBgoLcb_SJE727SNupshW10JxWyP7q7mEVEW6xlspxAkw&disp=emb|width=2098,height=194!
2. the user runs aggregation queries while keeping writing data.
3. The user finds that the aggregation query latency increases along with the
number of data points increase. When the number of points is increased to 20
million, the query latency is about tens of seconds.
4. But, once the data is flushed on disk, the aggregation speed is back to
several milliseconds.
I think it is a design problem because we have no summary info for the data in
memory.
Codes:
{code:java}
// code placeholder
import java.sql.*;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;
public class JDBCExample {
/**
* Before executing a SQL statement with a Statement object, you need to
create a Statement object using the createStatement() method of the Connection
object.
* After creating a Statement object, you can use its execute() method to
execute a SQL statement
* Finally, remember to close the 'statement' and 'connection' objects by
using their close() method
* For statements with query results, we can use the getResultSet() method
of the Statement object to get the result set.
*/
public static void main(String[] args) throws SQLException, ParseException {
Connection connection = getConnection();
if (connection == null) {
System.out.println("get connection defeat");
return;
}
int[] days={1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20};
int i=days.length;
while (i-->0){
AddInThread addInThread=new AddInThread(getConnection(),days[i]);
addInThread.start();
}
}
public static Connection getConnection() {
// JDBC driver name and database URL
String driver = "org.apache.iotdb.jdbc.IoTDBDriver";
String url = "jdbc:iotdb://192.168.5.244:6667/";
// Database credentials
String username = "root";
String password = "root";
Connection connection = null;
try {
Class.forName(driver);
connection = DriverManager.getConnection(url, username, password);
} catch (ClassNotFoundException e) {
e.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
}
return connection;
}
/**
* This is an example of outputting the results in the ResultSet
*/
private static void outputResult(ResultSet resultSet) throws SQLException {
if (resultSet != null) {
System.out.println("--------------------------");
final ResultSetMetaData metaData = resultSet.getMetaData();
final int columnCount = metaData.getColumnCount();
for (int i = 0; i < columnCount; i++) {
System.out.print(metaData.getColumnLabel(i + 1) + " ");
}
System.out.println();
while (resultSet.next()) {
for (int i = 1; ; i++) {
System.out.print(resultSet.getString(i));
if (i < columnCount) {
System.out.print(", ");
} else {
System.out.println();
break;
}
}
}
System.out.println("--------------------------\n");
}
}
}
class AddInThread extends Thread {
final Connection connection;
final int day;
final String datetimeText;
SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss:SSS");
AddInThread(Connection connection,int days) {
this.connection = connection;
day = days;
datetimeText="2019-11-"+days+" 10:34:59:000";
}
@Override
public void run() {
try {
Date datetime = format.parse(datetimeText);
long times=datetime.getTime();
Statement statement = connection.createStatement();
Integer i = 100;
while (i-- >0) {
statement.execute("insert into
root.ch.baby.d01(timestamp,status) values("+times+++",true)");
}
statement.close();
connection.close();
} catch (ParseException e) {
e.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
{code}
The problem is reported original from [https://www.oschina.net/p/iotdb]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
