[
https://issues.apache.org/jira/browse/HIVE-18051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16249928#comment-16249928
]
Vihang Karajgaonkar edited comment on HIVE-18051 at 11/13/17 6:19 PM:
----------------------------------------------------------------------
Thanks [~kgyrtkirk] for reporting this. I think its a great idea, the current
setup of the qtest is pretty heavy and most of the tests don't really need all
the set up tables. Having lightweight setup where needed would help us
reducing the execution time. We should be careful not to batch the heavy setup
qtests with the lighter ones since that might overshadow the benefits.
[~pvary] has spent some time with the beeline driver and he may have some
inputs of using beeline to run qtests regarding stability.
was (Author: vihangk1):
Thanks [~kgyrtkirk] for reporting this. I think its a great idea, the current
setup of the qtest is pretty heavy and most of the tests don't really need all
the set up tables. Having lightweight setup where needed would help us
reducing the execution time. We should be careful not to batch the heavy setup
qtests with the lighter ones since that might overshadow the benefits.
> qfiles: dataset support
> -----------------------
>
> Key: HIVE-18051
> URL: https://issues.apache.org/jira/browse/HIVE-18051
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
> Reporter: Zoltan Haindrich
>
> it would be great to have some kind of test dataset support; currently there
> is the {{q_test_init.sql}} which is quite large; and I'm often override it
> with an invalid string; because I write independent qtests most of the time -
> and the load of {{src}} and other tables are just a waste of time for me ;
> not to mention that the loading of those tables may also trigger breakpoints
> - which is a bit annoying.
> Most of the tests are "only" using the {{src}} table and possibly 2 others;
> however the main init script contains a bunch of tables - meanwhile there are
> quite few other tests which could possibly also benefit from a more general
> feature; for example the creation of {{bucket_small}} is present in 20 q
> files.
> the proposal would be to enable the qfiles to be annotated with metadata like
> datasets:
> {code}
> --! qt:dataset:src,bucket_small
> {code}
> proposal for storing a dataset:
> * the loader script would be at: {{data/datasets/__NAME__/load.hive.sql}}
> * the table data could be stored under that location
> a draft about this; and other qfiles related ideas:
> https://docs.google.com/document/d/1KtcIx8ggL9LxDintFuJo8NQuvNWkmtvv_ekbWrTLNGc/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
