Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/13251 )
Change subject: IMPALA-8369 (part 4): Hive 3: fixes for functional dataset loading ...................................................................... Patch Set 3: (1 comment) What happened to the test that uses hive_index? Is it disabled for Hive 3? http://gerrit.cloudera.org:8080/#/c/13251/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13251/3//COMMIT_MSG@17 PS3, Line 17: - adds the ability to specify version restrictions for tables to load. : The restrictions use the Python "requirements.txt" syntax. This new : functionality is used to skip creating a hive "INDEX" table on Hive 3, : where this functionality has been removed. I have mixed feelings about something this flexible. For dataload, we usually know that some test is going to use this table. Any flexibility we introduce in what gets loaded on particular versions requires equivalent changes to tests. If we aren't careful, we would end up with mismatches between dataload and tests, and flexibility makes the system harder to reason about (and harder to test the combinations). It could be that we add this and use it once for Hive 3, which lines up with USE_CDP_HIVE, and then never use it again. Then everything is fine. But if we use it for other things beyond Hive, I think it gets more error prone. An alternative is to tie this specifically to USE_CDP_HIVE (or reintroduce a MINICLUSTER_PROFILE). With a fixed set of configurations, we just run tests on those configurations and everything naturally stays valid. -- To view, visit http://gerrit.cloudera.org:8080/13251 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic34930dc064da3136dde4e01a011d14db6a74ecd Gerrit-Change-Number: 13251 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Todd Lipcon <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Wed, 08 May 2019 15:53:32 +0000 Gerrit-HasComments: Yes
