[
https://issues.apache.org/jira/browse/KYLIN-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15195028#comment-15195028
]
Hao Chen commented on KYLIN-1434:
---------------------------------
Generic TimeSeries Rowkey design, referring to Eagle & OpenTSDB:
{code}
/job/${timestamp:8}${fieldValueHash:4}* N
{code}
*
https://github.com/apache/incubator-eagle/blob/master/eagle-core/eagle-query/eagle-entity-base/src/main/java/org/apache/eagle/log/entity/RowkeyBuilder.java#L105-L139
Generic Rowkey filter:
* https://github.com/OpenTSDB/opentsdb/blob/master/src/core/TsdbQuery.java
*
https://github.com/apache/incubator-eagle/blob/master/eagle-core/eagle-query/eagle-entity-base/src/main/java/org/apache/eagle/log/entity/filter/HBaseFilterBuilder.java#L488-L543
{code}
// <tag1:4><value1:4> ... <tagn:4><valuen:4>
StringBuilder sb = new StringBuilder();
sb.append("(?s)");
sb.append("^(?:.{").append(headerLength).append("})");
sb.append("(?:.{").append(8).append("})*"); // for any number of tags
for (Map.Entry<Integer, List<Integer>> entry : tagHash.entrySet()) {
try {
addId(sb, ByteUtil.intToBytes(entry.getKey()));
List<Integer> hashValues = entry.getValue();
sb.append("(?:");
boolean first = true;
for(Integer value : hashValues){
if(!first){
sb.append('|');
}
final byte[] id = ByteUtil.intToBytes(value);
buf.append("\\Q");
boolean backslash = false;
for (final byte b : id) {
buf.append((char) (b & 0xFF));
if (b == 'E' && backslash) { // If we saw a `\' and now
we have a `E'.
// So we just terminated the quoted section because we
just added \E
// to `buf'. So let's put a litteral \E now and start
quoting again.
buf.append("\\\\E\\Q");
} else {
backslash = b == '\\';
}
}
buf.append("\\E");
first = false;
}
sb.append(")");
sb.append("(?:.{").append(8).append("})*"); // for any number
of tags
} catch (Exception ex) {
LOG.error("constructing regex error", ex);
}
}
sb.append("$");
RegexStringComparator regexStringComparator = new RegexStringComparator(binary);
regexStringComparator.setCharset(_charset);
RowFilter filter = new RowFilter(CompareOp.EQUAL, regexStringComparator);
{code}
where the tag fields could be any filtering fields like job status/type/cube,
etc.
> Kylin Job Monitor API: /kylin/api/jobs is too slow
> --------------------------------------------------
>
> Key: KYLIN-1434
> URL: https://issues.apache.org/jira/browse/KYLIN-1434
> Project: Kylin
> Issue Type: Bug
> Affects Versions: v1.4.0
> Reporter: Hao Chen
>
> The API request for Job Monitor page like:
> {code}/kylin/api/jobs?limit=15&offset=15{code} takes more than 11 seconds for
> fetching only 15 row records (25.1 KB), which is too slow.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
