[
https://issues.apache.org/jira/browse/TAJO-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371108#comment-14371108
]
Dongjoon Hyun commented on TAJO-1430:
-------------------------------------
Thank you for your reminding, [~hyunsik]. I agree with you in that point. I
love your kind guide, always. I suggest that we create another issue for that
because, as you told, it is heavier issue that can not be included in 0.10.1.
As you know, this issue is only dependent on only SQLAnalyzer. Actually, you
can think this as a faster SQLAnalyzer using caching.
Finally, let me allow to introduce my two main reasons for that.
First, this issue is based on global semiconductor's use cases. There exist
Statement(not PreparedStatement) use cases with 20~30MB real queries. This
patch just caches queries whose parsing time is greater than 30 seconds, not
for all general queries. As you see in the example, in real sites, Tajo
customer experience is not good. TajoCli shows a few seconds, but the clock
wall shows over 30 seconds.
Second, current TajoPreparedStatement class just makes 'SQL String' as a result
from the prepared statements. I'm sure that you want me to modify this class
only. In fact, I desired to improve both Statement and PreparedStatement cases.
If I patches TajoPreparedStatement, the effect is very limited and cannot
fullfill my customers.
Last, but not least, you can remove later this feature very easily.
> Implement Query Parsing Result Caching
> --------------------------------------
>
> Key: TAJO-1430
> URL: https://issues.apache.org/jira/browse/TAJO-1430
> Project: Tajo
> Issue Type: New Feature
> Components: parser
> Affects Versions: 0.10.0
> Reporter: Dongjoon Hyun
> Assignee: Dongjoon Hyun
> Fix For: 0.10.1
>
> Attachments: TAJO-1430.patch, long.sql, middle.sql, wide_table.sql
>
>
> There are wide tables with many many columns. Moveover, BI tools generate
> very complex queries whose size is several MB. Although Tajo executes those
> queries very fast in a few seconds, the total time of UX is slow.
> To become a fastest Hadoop DW, we need this following feature.
> {code:sql}
> time tsql -f middle.sql > /dev/null
> real 0m19.058s
> user 0m2.148s
> sys 0m0.268s
> time tsql -f ~/tajo/middle.sql > /dev/null
> real 0m18.496s
> user 0m2.119s
> sys 0m0.240s
> $ time ./tsql -f ~/tajo/long.sql > /dev/null
> real 0m36.974s
> user 0m2.305s
> sys 0m0.272s
> $ time ./tsql -f ~/tajo/long.sql > /dev/null
> real 0m4.103s
> user 0m2.237s
> sys 0m0.249s
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
