[Impala-ASF-CR] Release note updates for Impala 2.8
John Russell has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 7: (10 comments) Addressed all comments. Dealt with some gerrit glitches (had to squash 2 commits that appeared to both have the same change ID, one of which had gone into the wrong private branch.) http://gerrit.cloudera.org:8080/#/c/5668/7/docs/shared/impala_common.xml File docs/shared/impala_common.xml: PS7, Line 3297: CDH > Please do not add new CDH references. Not adding, this one was already here AFAIK. http://gerrit.cloudera.org:8080/#/c/5668/6/docs/topics/impala_incompatible_changes.xml File docs/topics/impala_incompatible_changes.xml: Line 71: such as COMPRESSION, DEFAULT, and ENCODING, that > They are usually done within seconds of each other. I am occasionally held up for some time resolving a merge conflict or otherwise dealing with gerrit glitches. http://gerrit.cloudera.org:8080/#/c/5668/7/docs/topics/impala_incompatible_changes.xml File docs/topics/impala_incompatible_changes.xml: Line 63: > You can remove this. It was only a temporary bug in 2.8. We never shipped a OK. I had an item mentioning the uppercasing under 2.7. That might have been an assumption on my part because of the timing of when this came to light. I'll remove it under 2.7 as well. http://gerrit.cloudera.org:8080/#/c/5668/3/docs/topics/impala_new_features.xml File docs/topics/impala_new_features.xml: PS3, Line 53: / CDH 5.10.x > I do not see a response to his yet. Done. I am doing a bunch of replacements with substitution variables in these titles, although not 100% of the titles in this file. Back in the Impala 1.3 / 1.2 days, there was some discontinuity around the release numbering (e.g. new features going out in .1 maintenance releases) and I'll leave that cleanup for a subsequent CR. PS3, Line 53: / CDH 5.10.x > Please remove. Done Line 84: The MT_DOP query option enables > The line breaks here are sometimes around 80 characters, and sometimes over Holding off on making any big changes to line length in individual CRs, saving that for a global change later. PS3, Line 100: IMPALA-2522 > This should be IMPALA-2521. IMPALA-2522 is an Epic that covers much more t Done http://gerrit.cloudera.org:8080/#/c/5668/7/docs/topics/impala_new_features.xml File docs/topics/impala_new_features.xml: Line 65: > We should definitely also highlight IMPALA-1430 which fixes the slow COMPUT Done. I'll do some follow-on work to link to the specific JIRAs rather than just summarizing each improvement. (I'll probably save that for a follow-on CR.) Line 130: format TIMESTAMP values, such as the result > A naked now() function is a particularly bad example, because now() is alre Done PS7, Line 724: > New spaces at end of lines Done. Puzzling because I'm avoiding reformatting existing lines. Trailing spaces are usually removed by my pre-commit hook but I haven't tested that hook with the ASF repo. -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 7 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs Gerrit-Reviewer: Silvius Rus Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
Alex Behm has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 7: (2 comments) http://gerrit.cloudera.org:8080/#/c/5668/7/docs/topics/impala_new_features.xml File docs/topics/impala_new_features.xml: Line 65: We should definitely also highlight IMPALA-1430 which fixes the slow COMPUTE STATS issue with TIMESTAMP. COMPUTE STATS is now codegen'd even if there are TIMESTAMP columns. The JIRA is not yet marked as fixed because there is some follow-on work, but the COMPUTE STATS issue specifically has been fixed. Line 130: format TIMESTAMP values, such as the result A naked now() function is a particularly bad example, because now() is already constant within a query, so does not benefit from this improvement. A better example is something like: WHERE date_col = to_date(now() - interval 1 day) the right-hand size expression is somewhat complicated and expensive to evaluate for every row. Constant folding makes this significantly faster. -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 7 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs Gerrit-Reviewer: Silvius Rus Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
Alex Behm has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/5668/7/docs/topics/impala_incompatible_changes.xml File docs/topics/impala_incompatible_changes.xml: Line 63: You can remove this. It was only a temporary bug in 2.8. We never shipped a version that had DESCRIBE FORMATTED in uppercase. -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 7 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs Gerrit-Reviewer: Silvius Rus Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
Greg Rahn has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/5668/3/docs/topics/impala_new_features.xml File docs/topics/impala_new_features.xml: PS3, Line 100: IMPALA-2522 This should be IMPALA-2521. IMPALA-2522 is an Epic that covers much more than just this. -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs Gerrit-Reviewer: Silvius Rus Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
Jim Apple has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/5668/6/docs/topics/impala_incompatible_changes.xml File docs/topics/impala_incompatible_changes.xml: Line 71: and query options (such as V_CPU_CORES) remain but do not have any effect. > OK. I wasn't sure in the race condition between "Reply" and "upload new pat They are usually done within seconds of each other. -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs Gerrit-Reviewer: Silvius Rus Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
Jim Apple has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/5668/7/docs/shared/impala_common.xml File docs/shared/impala_common.xml: PS7, Line 3297: CDH Please do not add new CDH references. http://gerrit.cloudera.org:8080/#/c/5668/7/docs/topics/impala_new_features.xml File docs/topics/impala_new_features.xml: PS7, Line 724: New spaces at end of lines -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs Gerrit-Reviewer: Silvius Rus Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
John Russell has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/5668/6/docs/topics/impala_incompatible_changes.xml File docs/topics/impala_incompatible_changes.xml: Line 71: and query options (such as V_CPU_CORES) remain but do not have any effect. > It is difficult to review changes when the author replies "Done" before the OK. I wasn't sure in the race condition between "Reply" and "upload new patch set" which should come first. In a different review I got a request not to upload the patch set before the responses. -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs Gerrit-Reviewer: Silvius Rus Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
John Russell has uploaded a new patch set (#7). Change subject: Release note updates for Impala 2.8 .. Release note updates for Impala 2.8 First cut at 'new features' topic. Includes the Incompatible Changes subtopic for Impala 2.8. Also did some cleanup throughout the Incompatible Changes page: - Took out references to Cloudera release numbers from titles. - Suppressed the display of ancient subtopics from the Impala beta days, which are intertwined with things like what version of Cloudera Manager was supported. Patch set 3: More on MT_DOP for COMPUTE STATS. Address comments from Greg and MJ. Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed --- M docs/topics/impala_incompatible_changes.xml M docs/topics/impala_new_features.xml 2 files changed, 457 insertions(+), 177 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/5668/7 -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 7 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs Gerrit-Reviewer: Silvius Rus
[Impala-ASF-CR] Release note updates for Impala 2.8
John Russell has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 3: (5 comments) Only minor changes (trailing spaces) to come in next patch set. Right now I'm stuck at "failed to push some refs", so that might not be today. http://gerrit.cloudera.org:8080/#/c/5668/3/docs/topics/impala_incompatible_changes.xml File docs/topics/impala_incompatible_changes.xml: Line 56: > I do not see an answer to this yet. These are all from JIRAs that are marked 'fixed in 2.8.0' and/or coding work completed back in December, before 2.8.0 was branched off. Is there an additional round of checking you are thinking of? The current behavior is as described elsewhere in the docs. The previous behavior (at least for the Kudu-related bullets) is from the impala_kudu fork. PS3, Line 58: Impala Incompatible Changes Introduced in Impala 2.8.x > There was something I saw relating to cross-reference links, but I think th Done Line 1523: > My experience with the CHANGES files in the root directories of open-source Let's continue this discussion as part of the general cleanup of Cloudera-specific info. That is, if we preserve some historical info as-is, we'll have to adjust expectations a little bit for how many instances of Cloudera / CDH / Cloudera Manager remain in the source files. http://gerrit.cloudera.org:8080/#/c/5668/6/docs/topics/impala_incompatible_changes.xml File docs/topics/impala_incompatible_changes.xml: Line 71: such as COMPRESSION, DEFAULT, and ENCODING, that > new spaces at end of lines Done. I have a pre-commit hook that suppresses these but haven't adapted it yet for the Apache repo. http://gerrit.cloudera.org:8080/#/c/5668/6/docs/topics/impala_new_features.xml File docs/topics/impala_new_features.xml: PS6, Line 105: , and each partition is onl > Do we describe the behavior in more detail someplace else? Yes, in the MT_DOP page linked from this bullet point. -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs Gerrit-Reviewer: Silvius Rus Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
Jim Apple has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 6: (4 comments) http://gerrit.cloudera.org:8080/#/c/5668/3/docs/topics/impala_incompatible_changes.xml File docs/topics/impala_incompatible_changes.xml: Line 56: > Have you checked that these changes are all in the 2.8.0 release candidate I do not see an answer to this yet. Line 1523 > The beta releases are so intertwined with CDH and CM release numbers, I wan My experience with the CHANGES files in the root directories of open-source projects is that the usually accrete forever. http://gerrit.cloudera.org:8080/#/c/5668/6/docs/topics/impala_incompatible_changes.xml File docs/topics/impala_incompatible_changes.xml: Line 71: and query options (such as V_CPU_CORES) remain but do not have any effect. new spaces at end of lines http://gerrit.cloudera.org:8080/#/c/5668/3/docs/topics/impala_new_features.xml File docs/topics/impala_new_features.xml: PS3, Line 53: / CDH 5.10.x > Please remove. I do not see a response to his yet. -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs Gerrit-Reviewer: Silvius Rus Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
Silvius Rus has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/5668/6/docs/topics/impala_new_features.xml File docs/topics/impala_new_features.xml: PS6, Line 105: number of Impala operations Do we describe the behavior in more detail someplace else? -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs Gerrit-Reviewer: Silvius Rus Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
John Russell has uploaded a new patch set (#6). Change subject: Release note updates for Impala 2.8 .. Release note updates for Impala 2.8 First cut at 'new features' topic. Includes the Incompatible Changes subtopic for Impala 2.8. Also did some cleanup throughout the Incompatible Changes page: - Took out references to Cloudera release numbers from titles. - Suppressed the display of ancient subtopics from the Impala beta days, which are intertwined with things like what version of Cloudera Manager was supported. Patch set 3: More on MT_DOP for COMPUTE STATS. Address comments from Greg and MJ. Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed --- M docs/shared/impala_common.xml M docs/topics/impala_incompatible_changes.xml M docs/topics/impala_new_features.xml M docs/topics/impala_reservation_request_timeout.xml M docs/topics/impala_resource_management.xml M docs/topics/impala_v_cpu_cores.xml 6 files changed, 499 insertions(+), 338 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/5668/6 -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs
[Impala-ASF-CR] Release note updates for Impala 2.8
John Russell has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 5: (9 comments) Finished Mostafa comments. Patch set might be delayed because of a merge conflict. (Accidentally posted this latest patch set to a different gerrit first.) http://gerrit.cloudera.org:8080/#/c/5668/5/docs/topics/impala_new_features.xml File docs/topics/impala_new_features.xml: PS5, Line 59: > Talked with Mostafa. Subsequent comments from me are based on our chat. Done PS5, Line 65: : The COMPUTE STATS statement can : take advantage of multithreading. : > Fold this item into the longer one below on the MT_DOP option. Done. I'm sure I did this also in another gerrit review so it'll probably need conflict resolution later. PS5, Line 79: native code generation > Group all the codegen-related items together. (Except the filtering one, wh Done PS5, Line 95: large number : of columns > Especially if the table has TIMESTAMP columns. Done PS5, Line 112: A new hint, SORTBY > Why does the Fix Version field of IMPALA-4163 say Impala 2.9 not 2.8? Verif Done PS5, Line 120: new configuration setting > Explain how to use this (requires safety valve). Done PS5, Line 121: network overhead > The improvement is mainly reduced load on the catalog server. Done PS5, Line 140: > Mention IMPALA-4302 i.e. better IN list scalability. Done PS5, Line 141: > Mention IMPALA-1286 i.e. extracting common conjuncts from disjunctions. Done -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
John Russell has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 5: (9 comments) Annotated with suggestions from Mostafa. http://gerrit.cloudera.org:8080/#/c/5668/5/docs/topics/impala_new_features.xml File docs/topics/impala_new_features.xml: PS5, Line 59: Talked with Mostafa. Subsequent comments from me are based on our chat. PS5, Line 65: : The COMPUTE STATS statement can : take advantage of multithreading. : Fold this item into the longer one below on the MT_DOP option. PS5, Line 79: native code generation Group all the codegen-related items together. (Except the filtering one, which combines codegen + other improvements.) PS5, Line 95: large number : of columns Especially if the table has TIMESTAMP columns. PS5, Line 112: A new hint, SORTBY Why does the Fix Version field of IMPALA-4163 say Impala 2.9 not 2.8? Verify this hint is supposed to be included and documented. PS5, Line 120: new configuration setting Explain how to use this (requires safety valve). I'll defer that suggestion until a subsequent doc review. Probably will involve touching impala_scalability.xml. PS5, Line 121: network overhead The improvement is mainly reduced load on the catalog server. PS5, Line 140: Mention IMPALA-4302 i.e. better IN list scalability. PS5, Line 141: Mention IMPALA-1286 i.e. extracting common conjuncts from disjunctions. -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
John Russell has uploaded a new patch set (#4). Change subject: Release note updates for Impala 2.8 .. Release note updates for Impala 2.8 First cut at 'new features' topic. Includes the Incompatible Changes subtopic for Impala 2.8. Also did some cleanup throughout the Incompatible Changes page: - Took out references to Cloudera release numbers from titles. - Suppressed the display of ancient subtopics from the Impala beta days, which are intertwined with things like what version of Cloudera Manager was supported. Patch set 3: More on MT_DOP for COMPUTE STATS. Address comments from Greg and MJ. Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed --- M docs/topics/impala_incompatible_changes.xml M docs/topics/impala_new_features.xml 2 files changed, 457 insertions(+), 177 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/5668/4 -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs
[Impala-ASF-CR] Release note updates for Impala 2.8
John Russell has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 3: (16 comments) http://gerrit.cloudera.org:8080/#/c/5668/1//COMMIT_MSG Commit Message: Line 28: Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed > I won't have time to do a thorough review here but this commit message need Done http://gerrit.cloudera.org:8080/#/c/5668/1/docs/topics/impala_incompatible_changes.xml File docs/topics/impala_incompatible_changes.xml: PS1, Line 91: now > typo Done PS1, Line 111: There was a period during early experimental versions of Impala + Kudu where : non-primary-key olumns had the NOT NULL I don't think this is correct. AFAIK the default behavior is the same and w I clarified the wording a bit. PS1, Line 111: There was a period during early experimental versions of Impala + Kudu where : non-primary-key olumns had the NOT NULL The GA behavior is columns are nullable by default, unless they are part of Done http://gerrit.cloudera.org:8080/#/c/5668/3/docs/topics/impala_incompatible_changes.xml File docs/topics/impala_incompatible_changes.xml: PS3, Line 58: Impala Incompatible Changes Introduced in Impala 2.8.x > This wording is surprising to me. Why is the first word needed at all? There was something I saw relating to cross-reference links, but I think that was more applicable in Cloudera-specific docs. I removed the leading "Impala" globally. Line 65: They were output in uppercase by mistake, but only for a single Impala release (Impala 2.7). > It makes changes easier to review when the lines are 90 characters or fewer Good point. I'm going to wait and do that globally as a separate gerrit review. I suggest for entirely new text like this, go into gerrit settings and set 'left side' to 'hide'. PS3, Line 111: > space at end of line Done Line 112: non-primary-key olumns had the NOT NULL attribute by default. > "olumns" Done Line 1523: > Why hide this? The beta releases are so intertwined with CDH and CM release numbers, I want to make a blanket rule to hide or remove such stale historical info. It is more of a Cloudera convention to let release notes accrete forever. Contrast with Apache Kudu - https://kudu.apache.org/docs/release_notes.html - where only the changes for the current release are listed in the docs for that release. http://gerrit.cloudera.org:8080/#/c/5668/1/docs/topics/impala_new_features.xml File docs/topics/impala_new_features.xml: PS1, Line 66: The COMPUTE STATS statement can : take advantage of multithreading. > Not sure if you want to be more specific here as the default for Parquet ta Done PS1, Line 90: > move left? You mean indentation-wise? For the XML tagging in the docs, I use Berkeley-style indenting, e.g. some text is the equivalent of { some C code } PS1, Line 91: > : : > SORTBY adds ordering for non-partition key columns to better support the ef Done PS1, Line 103: that use dynamic > move to p? Good catch. I use even where optional inside list items, and prefer to apply attributes on the inner tags, because there's the possibility of reusing those paragraphs verbatim in the detailed writeup of each feature. PS1, Line 238: STAMP, DECIMAL< > needed? Clunky wording, I'll reword just to emphasize not to worry about inserting data in small batches like we warn people about for other kinds of tables. PS1, Line 253: of the Kudu data makes it more efficient than with HDFS > dimitris should review I'll mention that Sentry can be bypassed and then have Dimitris review the subsequent patch set. PS1, Line 290: : Is this purely a CM feature? If so I won't mention it in the upstream docs. -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
Jim Apple has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 3: (8 comments) http://gerrit.cloudera.org:8080/#/c/5668/3/docs/topics/impala_incompatible_changes.xml File docs/topics/impala_incompatible_changes.xml: Line 56: Have you checked that these changes are all in the 2.8.0 release candidate 1 that the IPMC is voting on right now? PS3, Line 58: Impala Incompatible Changes Introduced in Impala 2.8.x This wording is surprising to me. Why is the first word needed at all? Line 65: They were output in uppercase by mistake, but only for a single Impala release (Impala 2.7). It makes changes easier to review when the lines are 90 characters or fewer. PS3, Line 111: space at end of line Line 112: non-primary-key olumns had the NOT NULL attribute by default. "olumns" Line 1523: Why hide this? http://gerrit.cloudera.org:8080/#/c/5668/3/docs/topics/impala_new_features.xml File docs/topics/impala_new_features.xml: PS3, Line 53: / CDH 5.10.x Please remove. Line 84: The MT_DOP query option enables The line breaks here are sometimes around 80 characters, and sometimes over 90. -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
John Russell has uploaded a new patch set (#3). Change subject: Release note updates for Impala 2.8 .. Release note updates for Impala 2.8 First cut at 'new features' topic. Includes the Incompatible Changes subtopic for Impala 2.8. Also did some cleanup throughout the Incompatible Changes page: - Took out references to Cloudera release numbers from titles. - Suppressed the display of ancient subtopics from the Impala beta days, which are intertwined with things like what version of Cloudera Manager was supported. Patch set 3: More on MT_DOP for COMPUTE STATS. Address comments from Greg and MJ. Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed --- M docs/topics/impala_incompatible_changes.xml M docs/topics/impala_new_features.xml 2 files changed, 437 insertions(+), 53 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/5668/3 -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Laurel Hale Gerrit-Reviewer: Matthew Jacobs
[Impala-ASF-CR] Release note updates for Impala 2.8
John Russell has abandoned this change. Change subject: Release note updates for Impala 2.8 .. Abandoned OK, got the CLUSTERED hint into the 5668 code review. -- To view, visit http://gerrit.cloudera.org:8080/5644 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: abandon Gerrit-Change-Id: I03144b423c4d698e87dd335914a8b7c0ff030496 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: John Russell Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] Release note updates for Impala 2.8
Michael Brown has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 3: > I don't know if this CR becomes inaccessible after being abandoned. Reviews are still accessible after being abandoned. https://gerrit.cloudera.org/#/q/project:Impala-ASF+status:abandoned -- To view, visit http://gerrit.cloudera.org:8080/5644 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I03144b423c4d698e87dd335914a8b7c0ff030496 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: John Russell Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] Release note updates for Impala 2.8
John Russell has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 3: Code-Review-1 I need to rescue patch set #3 which is missing from 5668. I'll do that before abandoning this one because I don't know if this CR becomes inaccessible after being abandoned. -- To view, visit http://gerrit.cloudera.org:8080/5644 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I03144b423c4d698e87dd335914a8b7c0ff030496 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: John Russell Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] Release note updates for Impala 2.8
Tim Armstrong has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 3: Is this a duplicate of https://gerrit.cloudera.org/#/c/5668/? -- To view, visit http://gerrit.cloudera.org:8080/5644 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I03144b423c4d698e87dd335914a8b7c0ff030496 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] Release note updates for Impala 2.8
Jim Apple has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/5668/1//COMMIT_MSG Commit Message: Line 28: Change-Id: I03144b423c4d698e87dd335914a8b7c0ff030496 I won't have time to do a thorough review here but this commit message needs exactly one Change-Id. When I'm rebasing, I usually fixup, rather than squash, which removes the later message, including its change Id -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Anonymous Coward #250 Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Matthew Jacobs Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
Greg Rahn has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/5668/1/docs/topics/impala_incompatible_changes.xml File docs/topics/impala_incompatible_changes.xml: PS1, Line 111: This is a change from early experimental versions of Impala + Kudu, where columns : had the NOT NULL attribute by default. > I don't think this is correct. AFAIK the default behavior is the same and w The GA behavior is columns are nullable by default, unless they are part of the PK. There was a change that inverted this behavior for a time but now impala explicitly implements the expected RDBMS behavior. Perhaps that is where the confusion comes from. http://gerrit.cloudera.org:8080/#/c/5668/1/docs/topics/impala_new_features.xml File docs/topics/impala_new_features.xml: PS1, Line 66: The COMPUTE STATS statement can : take advantage of multithreading. Not sure if you want to be more specific here as the default for Parquet tables will now be MT_DOP=4. PS1, Line 91: allows Impala INSERT operations : that use dynamic partitioning to process a high number of : partitions in a single statement. SORTBY adds ordering for non-partition key columns to better support the effectiveness of min/max data elimination techniques (still a WIP). CLUSTERED adds ordering for the partition key columns such that there is only a single writer for any given partition thus reducing the overall memory needed when inserting into many partitions. Both of these are mentioned in the description of IMPALA-2522. -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Anonymous Coward #250 Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Matthew Jacobs Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
Matthew Jacobs has posted comments on this change. Change subject: Release note updates for Impala 2.8 .. Patch Set 1: (7 comments) http://gerrit.cloudera.org:8080/#/c/5668/1/docs/topics/impala_incompatible_changes.xml File docs/topics/impala_incompatible_changes.xml: PS1, Line 91: not typo PS1, Line 111: This is a change from early experimental versions of Impala + Kudu, where columns : had the NOT NULL attribute by default. I don't think this is correct. AFAIK the default behavior is the same and we have nullable cols by default http://gerrit.cloudera.org:8080/#/c/5668/1/docs/topics/impala_new_features.xml File docs/topics/impala_new_features.xml: PS1, Line 90: ) move left? PS1, Line 103: rev="IMPALA-1788" move to p? PS1, Line 238: involving a small number of rows needed? PS1, Line 247: lineage metadata is not generated for : UPDATE and DELETE operations on Kudu tables. does insert work for kudu tables? check with dimitris PS1, Line 253: Currently, Kudu tables have limited support for Sentry: dimitris should review we may call out that access thru other engines to kudu doesnt respect this -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John RussellGerrit-Reviewer: Ambreen Kazi Gerrit-Reviewer: Anonymous Coward #250 Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Matthew Jacobs Gerrit-HasComments: Yes
[Impala-ASF-CR] Release note updates for Impala 2.8
John Russell has uploaded a new change for review. http://gerrit.cloudera.org:8080/5668 Change subject: Release note updates for Impala 2.8 .. Release note updates for Impala 2.8 First cut at 'new features' topic. Commented out one link to SCRATCH_LIMIT query option - topic doesn't exist yet. Filled in JIRA # and name for sortby() hint. Made Incompatible Changes subtopic for Impala 2.8. Also did some cleanup throughout the Incompatible Changes page: - Took out references to Cloudera release numbers from titles. - Suppressed the display of ancient subtopics from the Impala beta days, which are intertwined with things like what version of Cloudera Manager was supported. Change-Id: I03144b423c4d698e87dd335914a8b7c0ff030496 Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed --- M docs/topics/impala_incompatible_changes.xml M docs/topics/impala_new_features.xml 2 files changed, 422 insertions(+), 32 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/5668/1 -- To view, visit http://gerrit.cloudera.org:8080/5668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7c47f422e509cec6d3eb8aaa82294b584f393aed Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: John Russell