[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3466/ -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 3 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Nov 2018 11:09:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Hello David Knupp, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11938 to look at the new patch set (#4). Change subject: IMPALA-7585: support LDAP in run-workload.py .. IMPALA-7585: support LDAP in run-workload.py This patch just threads through the user, password, and ssl settings all the way back to the ImpalaBeeswaxClient. Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba --- M bin/run-workload.py M tests/performance/query_exec_functions.py M tests/performance/query_executor.py M tests/performance/workload_runner.py 4 files changed, 21 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/38/11938/4 -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 4 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Jim Apple has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 4: PS4: fix broken test "bin/run-workload.py -w tpch --num_clients=2 --query_names=TPCH-Q1 --table_format=text/none --exec_options=disable_codegen:False" -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 4 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Comment-Date: Fri, 16 Nov 2018 13:45:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Jim Apple has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 4: This fix to the broken test was relatively minor, but I wanted to give you another chance to review it before GVDing it again. -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 4 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Comment-Date: Fri, 16 Nov 2018 14:06:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1382/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 4 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Comment-Date: Fri, 16 Nov 2018 14:21:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3467/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 4 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Comment-Date: Fri, 16 Nov 2018 14:40:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7801: Remove toSql() from ParseNode interface.
Andrew Sherman has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11942 Change subject: IMPALA-7801: Remove toSql() from ParseNode interface. .. IMPALA-7801: Remove toSql() from ParseNode interface. In IMPALA-5821 the method "toSql(ToSqlOptions)" was added to ParseNode, to allow options to be passed when generating SQL from a parse tree. This change was suggested as part of the review of IMPALA-5821. The old "toSql()" method is currently implemented everywhere by calling toSql(ToSqlOptions.DEFAULT), but this is fragile as this convention could be accidentally ignored, To make the code more maintainable, remove the old "toSql()" method and have all callers call the new method instead. In addition, similarly replace "toSql()" in a few more associated classes: AnalyticWindow, LimitElement, Boundary, OrderByElement, SelectListItem. TESTING: No new tests are added as there are no functional changes. All end to end tests run clean. Change-Id: I17025901838e9ffd753894a8087170123f9d8b33 --- M fe/src/main/java/org/apache/impala/analysis/AdminFnStmt.java M fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java M fe/src/main/java/org/apache/impala/analysis/AggregateInfoBase.java M fe/src/main/java/org/apache/impala/analysis/AlterTableAddDropRangePartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableDropPartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableSetLocationStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterViewStmt.java M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java M fe/src/main/java/org/apache/impala/analysis/AnalyticInfo.java M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/main/java/org/apache/impala/analysis/CaseExpr.java M fe/src/main/java/org/apache/impala/analysis/CastExpr.java M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java M fe/src/main/java/org/apache/impala/analysis/CompoundPredicate.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/main/java/org/apache/impala/analysis/ExprSubstitutionMap.java M fe/src/main/java/org/apache/impala/analysis/ExtractFromExpr.java M fe/src/main/java/org/apache/impala/analysis/FromClause.java M fe/src/main/java/org/apache/impala/analysis/FunctionArgs.java M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java M fe/src/main/java/org/apache/impala/analysis/HdfsCachingOp.java M fe/src/main/java/org/apache/impala/analysis/InPredicate.java M fe/src/main/java/org/apache/impala/analysis/IsNotEmptyPredicate.java M fe/src/main/java/org/apache/impala/analysis/KuduPartitionParam.java M fe/src/main/java/org/apache/impala/analysis/LikePredicate.java M fe/src/main/java/org/apache/impala/analysis/LimitElement.java M fe/src/main/java/org/apache/impala/analysis/LiteralExpr.java M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java M fe/src/main/java/org/apache/impala/analysis/NumericLiteral.java M fe/src/main/java/org/apache/impala/analysis/OrderByElement.java M fe/src/main/java/org/apache/impala/analysis/ParseNode.java M fe/src/main/java/org/apache/impala/analysis/PartitionDef.java M fe/src/main/java/org/apache/impala/analysis/PartitionKeyValue.java M fe/src/main/java/org/apache/impala/analysis/PartitionSet.java M fe/src/main/java/org/apache/impala/analysis/PartitionSpec.java M fe/src/main/java/org/apache/impala/analysis/PartitionSpecBase.java M fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java M fe/src/main/java/org/apache/impala/analysis/QueryStmt.java M fe/src/main/java/org/apache/impala/analysis/RangePartition.java M fe/src/main/java/org/apache/impala/analysis/SelectListItem.java M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/StatementBase.java M fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java M fe/src/main/java/org/apache/impala/analysis/Subquery.java M fe/src/main/java/org/apache/impala/analysis/TableRef.java M fe/src/main/java/org/apache/impala/analysis/TableSampleClause.java M fe/src/main/java/org/apache/impala/analysis/TimestampArithmeticExpr.java M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java M fe/src/main/java/org/apache/impala/analysis/TypeDef.java M fe/src/main/java/org/apache/impala/analysis/UnionStmt.java M fe/src/main/java/org/apache/impala/analysis/WithClause.java M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java M fe/src/main/java/org/apache/impal
[Impala-ASF-CR] IMPALA-7801: Remove toSql() from ParseNode interface.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11942 ) Change subject: IMPALA-7801: Remove toSql() from ParseNode interface. .. Patch Set 1: (4 comments) http://gerrit.cloudera.org:8080/#/c/11942/1/fe/src/main/java/org/apache/impala/analysis/AlterTableSetLocationStmt.java File fe/src/main/java/org/apache/impala/analysis/AlterTableSetLocationStmt.java: http://gerrit.cloudera.org:8080/#/c/11942/1/fe/src/main/java/org/apache/impala/analysis/AlterTableSetLocationStmt.java@108 PS1, Line 108: + "please uncache before changing the location using: ALTER TABLE %s %s " line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/11942/1/fe/src/main/java/org/apache/impala/analysis/QueryStmt.java File fe/src/main/java/org/apache/impala/analysis/QueryStmt.java: http://gerrit.cloudera.org:8080/#/c/11942/1/fe/src/main/java/org/apache/impala/analysis/QueryStmt.java@205 PS1, Line 205: + "correlated with an outer block as well as an uncorrelated one '%s':\n%s", line too long (92 > 90) http://gerrit.cloudera.org:8080/#/c/11942/1/fe/src/main/java/org/apache/impala/analysis/RangePartition.java File fe/src/main/java/org/apache/impala/analysis/RangePartition.java: http://gerrit.cloudera.org:8080/#/c/11942/1/fe/src/main/java/org/apache/impala/analysis/RangePartition.java@202 PS1, Line 202: + "(type: %s) is not type compatible with partitioning column '%s' (type: %s).", line too long (94 > 90) http://gerrit.cloudera.org:8080/#/c/11942/1/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java File fe/src/main/java/org/apache/impala/analysis/SelectStmt.java: http://gerrit.cloudera.org:8080/#/c/11942/1/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java@276 PS1, Line 276: + "Star exprs only expand to scalar-typed columns because complex-typed exprs " line too long (91 > 90) -- To view, visit http://gerrit.cloudera.org:8080/11942 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I17025901838e9ffd753894a8087170123f9d8b33 Gerrit-Change-Number: 11942 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 16 Nov 2018 17:54:14 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7801: Remove toSql() from ParseNode interface.
Andrew Sherman has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/11942 ) Change subject: IMPALA-7801: Remove toSql() from ParseNode interface. .. IMPALA-7801: Remove toSql() from ParseNode interface. In IMPALA-5821 the method "toSql(ToSqlOptions)" was added to ParseNode, to allow options to be passed when generating SQL from a parse tree. This change was suggested as part of the review of IMPALA-5821. The old "toSql()" method is currently implemented everywhere by calling toSql(ToSqlOptions.DEFAULT), but this is fragile as this convention could be accidentally ignored, To make the code more maintainable, remove the old "toSql()" method and have all callers call the new method instead. In addition, similarly replace "toSql()" in a few more associated classes: AnalyticWindow, LimitElement, Boundary, OrderByElement, SelectListItem. TESTING: No new tests are added as there are no functional changes. All end to end tests run clean. Change-Id: I17025901838e9ffd753894a8087170123f9d8b33 --- M fe/src/main/java/org/apache/impala/analysis/AdminFnStmt.java M fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java M fe/src/main/java/org/apache/impala/analysis/AggregateInfoBase.java M fe/src/main/java/org/apache/impala/analysis/AlterTableAddDropRangePartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableDropPartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableSetLocationStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterViewStmt.java M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java M fe/src/main/java/org/apache/impala/analysis/AnalyticInfo.java M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/main/java/org/apache/impala/analysis/CaseExpr.java M fe/src/main/java/org/apache/impala/analysis/CastExpr.java M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java M fe/src/main/java/org/apache/impala/analysis/CompoundPredicate.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/main/java/org/apache/impala/analysis/ExprSubstitutionMap.java M fe/src/main/java/org/apache/impala/analysis/ExtractFromExpr.java M fe/src/main/java/org/apache/impala/analysis/FromClause.java M fe/src/main/java/org/apache/impala/analysis/FunctionArgs.java M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java M fe/src/main/java/org/apache/impala/analysis/HdfsCachingOp.java M fe/src/main/java/org/apache/impala/analysis/InPredicate.java M fe/src/main/java/org/apache/impala/analysis/IsNotEmptyPredicate.java M fe/src/main/java/org/apache/impala/analysis/KuduPartitionParam.java M fe/src/main/java/org/apache/impala/analysis/LikePredicate.java M fe/src/main/java/org/apache/impala/analysis/LimitElement.java M fe/src/main/java/org/apache/impala/analysis/LiteralExpr.java M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java M fe/src/main/java/org/apache/impala/analysis/NumericLiteral.java M fe/src/main/java/org/apache/impala/analysis/OrderByElement.java M fe/src/main/java/org/apache/impala/analysis/ParseNode.java M fe/src/main/java/org/apache/impala/analysis/PartitionDef.java M fe/src/main/java/org/apache/impala/analysis/PartitionKeyValue.java M fe/src/main/java/org/apache/impala/analysis/PartitionSet.java M fe/src/main/java/org/apache/impala/analysis/PartitionSpec.java M fe/src/main/java/org/apache/impala/analysis/PartitionSpecBase.java M fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java M fe/src/main/java/org/apache/impala/analysis/QueryStmt.java M fe/src/main/java/org/apache/impala/analysis/RangePartition.java M fe/src/main/java/org/apache/impala/analysis/SelectListItem.java M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/StatementBase.java M fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java M fe/src/main/java/org/apache/impala/analysis/Subquery.java M fe/src/main/java/org/apache/impala/analysis/TableRef.java M fe/src/main/java/org/apache/impala/analysis/TableSampleClause.java M fe/src/main/java/org/apache/impala/analysis/TimestampArithmeticExpr.java M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java M fe/src/main/java/org/apache/impala/analysis/TypeDef.java M fe/src/main/java/org/apache/impala/analysis/UnionStmt.java M fe/src/main/java/org/apache/impala/analysis/WithClause.java M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java M fe/src/main/java/org/apache/impala
[Impala-ASF-CR] IMPALA-7801: Remove toSql() from ParseNode interface.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11942 ) Change subject: IMPALA-7801: Remove toSql() from ParseNode interface. .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1383/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11942 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I17025901838e9ffd753894a8087170123f9d8b33 Gerrit-Change-Number: 11942 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 16 Nov 2018 18:30:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7801: Remove toSql() from ParseNode interface.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11942 ) Change subject: IMPALA-7801: Remove toSql() from ParseNode interface. .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1384/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11942 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I17025901838e9ffd753894a8087170123f9d8b33 Gerrit-Change-Number: 11942 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 16 Nov 2018 18:57:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
David Knupp has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 4: > Patch Set 4: > > This fix to the broken test was relatively minor, but I wanted to give you > another chance to review it before GVDing it again. OK, I'll look shortly. -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 4 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Comment-Date: Fri, 16 Nov 2018 19:36:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Hello David Knupp, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11938 to look at the new patch set (#5). Change subject: IMPALA-7585: support LDAP in run-workload.py .. IMPALA-7585: support LDAP in run-workload.py This patch just threads through the user, password, and ssl settings all the way back to the ImpalaBeeswaxClient. Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba --- M bin/run-workload.py M tests/performance/query_exec_functions.py M tests/performance/query_executor.py M tests/performance/workload_runner.py 4 files changed, 21 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/38/11938/5 -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 5 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 4: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3467/ -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 4 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Comment-Date: Fri, 16 Nov 2018 20:06:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3468/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 5 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Comment-Date: Fri, 16 Nov 2018 20:11:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 5: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/1385/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 5 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Comment-Date: Fri, 16 Nov 2018 20:27:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1386/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 5 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Comment-Date: Fri, 16 Nov 2018 21:12:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7233: [DOCS] Support for IANA timezone database
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11946 Change subject: IMPALA-7233: [DOCS] Support for IANA timezone database .. IMPALA-7233: [DOCS] Support for IANA timezone database - Updated the timezone section - Added the sections on customizing timezone db and aliases Change-Id: Id400cda5a1be321063d17e0ee6337e92a5da732a --- M docs/topics/impala_timestamp.xml 1 file changed, 278 insertions(+), 281 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/11946/1 -- To view, visit http://gerrit.cloudera.org:8080/11946 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id400cda5a1be321063d17e0ee6337e92a5da732a Gerrit-Change-Number: 11946 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-7233: [DOCS] Support for IANA timezone database
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11946 ) Change subject: IMPALA-7233: [DOCS] Support for IANA timezone database .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/152/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11946 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id400cda5a1be321063d17e0ee6337e92a5da732a Gerrit-Change-Number: 11946 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Nov 2018 21:24:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7233: [DOCS] Support for IANA timezone database
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11946 ) Change subject: IMPALA-7233: [DOCS] Support for IANA timezone database .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/152/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/11946 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id400cda5a1be321063d17e0ee6337e92a5da732a Gerrit-Change-Number: 11946 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Nov 2018 21:34:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Hello Lars Volker, Zoltan Borok-Nagy, Csaba Ringhofer, Alex Behm, Mostafa Mokhtar, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/8319 to look at the new patch set (#23). Change subject: IMPALA-4123: Columnar decoding in Parquet .. IMPALA-4123: Columnar decoding in Parquet The idea is to optimise the common case where there are long runs of NULL or non-NULL values (i.e. the def level is repeated). We can detect this cheaply by keying the decoding loop in the column reader off the state of the def level RLE decoder - if there's a long run of repeated levels, we can skip checking the def level for every value. We still fall back to decoding, caching and reading value-by-value a batch of def levels whenever the next def level is not in a repeated run. We still use the old approach for decoding rep levels. There might be some benefit to using the same approach for rep levels *if* repeated def and rep level runs line up. These changes should unlock further optimizations because more time is spent in simple kernel functions, e.g. UnpackAndDecode32Values() for dictionary decompression, which is very optimisable using SIMD etc. Snappy decompression now seems to be the main CPU bottleneck for decoding snappy-compressed Parquet. Perf: Running TPC-H scale factor 60 on uncompressed and snappy parquet both showed a ~4% speedup overall. Microbenchmarks on uncompressed parquet show scans only doing dictionary decoding on uncompressed Parquet is ~75% faster: set mt_dop=1; select min(l_returnflag) from lineitem; Testing: We have alltypes agg with a mix of null and non-null. Many tables have long runs of non-null values. Added new test data and coverage: * a test table manynulls with long runs of null values. * a large CHAR test table * missing coverage for materialising pos slot in flattened nested types scan. * Extended dict test to test longer runs. * A larger version of complextypestbl with interesting collection shapes - NULL collections, empty collections, etc, particularly runs of collections with the same shape. * Test interaction of timestamp validation with conversion * Ran code coverage build to confirm all code paths are tested * ASAN and exhaustive runs. Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef --- M be/src/exec/hdfs-parquet-scanner.cc M be/src/exec/parquet-column-readers.cc M be/src/exec/parquet-column-readers.h M be/src/exec/parquet-common.h M be/src/runtime/tuple.cc M be/src/runtime/tuple.h M be/src/util/bit-packing.h M be/src/util/bit-packing.inline.h M be/src/util/bit-stream-utils.h M be/src/util/bit-stream-utils.inline.h M be/src/util/dict-encoding.h M be/src/util/dict-test.cc A be/src/util/mem-util.h M be/src/util/rle-encoding.h M testdata/bin/generate-schema-statements.py M testdata/data/README A testdata/data/out_of_range_timestamp2_hive_211.parquet A testdata/data/out_of_range_timestamp_hive_211.parquet M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/chars.test M testdata/workloads/functional-query/queries/QueryTest/nested-types-scanner-position.test M testdata/workloads/functional-query/queries/QueryTest/nested-types-tpch.test A testdata/workloads/functional-query/queries/QueryTest/out-of-range-timestamp-local-tz-conversion.test A testdata/workloads/functional-query/queries/QueryTest/scanners-many-nulls.test M tests/custom_cluster/test_hive_parquet_timestamp_conversion.py M tests/query_test/test_scanners.py 27 files changed, 883 insertions(+), 133 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/8319/23 -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 23 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 22: Code-Review+1 I filed IMPALA-7862 and updated the comments -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 22 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Nov 2018 21:47:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 23: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 23 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Nov 2018 21:50:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 24: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 24 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Nov 2018 21:52:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 24: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3470/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 24 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Nov 2018 21:52:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7801: Remove toSql() from ParseNode interface.
Paul Rogers has posted comments on this change. (
http://gerrit.cloudera.org:8080/11942 )
Change subject: IMPALA-7801: Remove toSql() from ParseNode interface.
..
Patch Set 2:
(6 comments)
Thanks much for the improvement. Some general comments sprinkled among the
files.
http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/analysis/Expr.java
File fe/src/main/java/org/apache/impala/analysis/Expr.java:
http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/analysis/Expr.java@395
PS2, Line 395: String sql = toSql(DEFAULT);
Let's think about the meaning of the options for expressions. Expr nodes won't
have the state to provide un-rewritten SQL: a node is what it is, and does not
know if it should be considered rewritten or not. The rewritten state is
something that the outer non-expr node must provide.
Expr nodes can decide whether to include or exclude implicit casts.
This, then, raises and interesting issue. The ToSqlOptions enum is an enum: we
get to pick one value. But, some of the options overlap: I may want to include
implicit casts as one choice, and may want source or rewritten SQL on the other
hand.
I wonder, can we make this simpler somehow?
http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/analysis/Expr.java@1300
PS2, Line 1300: name, (printExpr) ? " '" + toSql(DEFAULT) + "'" : "",
type_.toString()));
This comment applies in a zillion other places. Our general rule for error
messages should be that we show the user's original SQL without rewrites or
implicit casts. We have hundreds of such usages and we have to check that new
messages use the correct form. Can we standardize these somehow?
One thought is to rename the option from DEFAULT to something like ORIGINAL or
SOURCE. But, this means we still have to sprinkle this option across many error
messages. This is, really, exposing implementation in too many places.
As it turns out, ParseNode is an interface, with many direct decedents. Does it
make sense to introduce a new base class, AbstractParseNode, that has a
userSql() (or sourceSql()) method that we use in error messages?
Recognizing that expressions are different than statements, maybe:
ParseNode
|- Expr (root of all expressions)
|- AbstractStmt (root of all statements)
The statement base would provide the source/rewritten options. Exprs only
provide the implicit cast or not options.
http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/analysis/ParseNode.java
File fe/src/main/java/org/apache/impala/analysis/ParseNode.java:
http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/analysis/ParseNode.java@33
PS2, Line 33: String toSql(ToSqlOptions options);
A common practice in Java is to include the enum definition in the interface
file if the enum is used only by the interface (and its implementations).
http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/analysis/QueryStmt.java
File fe/src/main/java/org/apache/impala/analysis/QueryStmt.java:
http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/analysis/QueryStmt.java@217
PS2, Line 217: "OFFSET requires an ORDER BY clause: " +
limitElement_.toSql(DEFAULT).trim());
I wonder if we can solve a mess here with exceptions also. Rather than building
message (very inconsistently) in each message, maybe have a builder:
throw new AnalysisException
.builder("OFFSET requires an ORDER BY clause")
.expr(limitElement_)
.build();
The builder would have methods for expressions (which will call the proper
toSql version), for statements, for additional context, and so on.
This would avoid the issue of exposing the magic toSql(DEFAULT) incantation in
zillions of places.
http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/planner/SortNode.java
File fe/src/main/java/org/apache/impala/planner/SortNode.java:
http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/planner/SortNode.java@44
PS2, Line 44: import static org.apache.impala.analysis.ToSqlOptions.DEFAULT;
Two suggestions.
1. Rename this as ToSqlOption (or ToSqlFormat) since we can pass only one
option.
2. Import the enum, not the static value. Reference it with the enum name:
ToSqlFormat.DEFAULT (or, based on earlier suggestions, ToSqlFormat.SOURCE).
http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/planner/SortNode.java@214
PS2, Line 214: output.append(info_.getSortExprs().get(i).toSql(DEFAULT)
+ " ");
This is an example of the point made elsewhere. It SEEMS we are asking the
expression to give the DEFAULT (source) SQL. But, an expression can't make that
decision. if we want the source SQL, we'd have to have cached it.
I'm working on a project to do just that in a more standardized form. (
[Impala-ASF-CR] IMPALA-7811: optionally count JVM heap towards process mem limit
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10928 ) Change subject: IMPALA-7811: optionally count JVM heap towards process mem limit .. Patch Set 8: Hey Pooja, Bikram won't be able to review this for a couple of weeks but maybe this will be interesting for you to look at. I can talk through the details of the memory accounting in person if that would be helpful (it would be good to understand in any case) -- To view, visit http://gerrit.cloudera.org:8080/10928 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39dd715882a32fc986755d573bd46f0fd9eefbfc Gerrit-Change-Number: 10928 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 16 Nov 2018 22:22:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 23: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1387/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 23 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 16 Nov 2018 22:22:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11948 ) Change subject: IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/153/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11948 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I88f615cf23f406035e544e68adacdd0393f69ab3 Gerrit-Change-Number: 11948 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Nov 2018 23:22:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11948 Change subject: IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme .. IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme Change-Id: I88f615cf23f406035e544e68adacdd0393f69ab3 --- M docs/topics/impala_adls.xml 1 file changed, 40 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/11948/1 -- To view, visit http://gerrit.cloudera.org:8080/11948 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I88f615cf23f406035e544e68adacdd0393f69ab3 Gerrit-Change-Number: 11948 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11948 ) Change subject: IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/153/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/11948 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I88f615cf23f406035e544e68adacdd0393f69ab3 Gerrit-Change-Number: 11948 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Fri, 16 Nov 2018 23:32:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/11948 ) Change subject: IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/11948/1/docs/topics/impala_adls.xml File docs/topics/impala_adls.xml: http://gerrit.cloudera.org:8080/#/c/11948/1/docs/topics/impala_adls.xml@283 PS1, Line 283: abfs://container@account.dfs.core.windows.net/path/fileorabfss://container@account.dfs.core.windows.net/path/file I realize this is the output of a tool, but if we can break up this long line, I would appreciate it. http://gerrit.cloudera.org:8080/#/c/11948/1/docs/topics/impala_adls.xml@298 PS1, Line 298: abfss:// "abfs://" You have abfss:// twice. -- To view, visit http://gerrit.cloudera.org:8080/11948 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I88f615cf23f406035e544e68adacdd0393f69ab3 Gerrit-Change-Number: 11948 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Fri, 16 Nov 2018 23:29:59 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme
Hello mackror...@apache.org, Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11948 to look at the new patch set (#2). Change subject: IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme .. IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme Change-Id: I88f615cf23f406035e544e68adacdd0393f69ab3 --- M docs/topics/impala_adls.xml 1 file changed, 43 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/11948/2 -- To view, visit http://gerrit.cloudera.org:8080/11948 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I88f615cf23f406035e544e68adacdd0393f69ab3 Gerrit-Change-Number: 11948 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11948 ) Change subject: IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/154/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11948 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I88f615cf23f406035e544e68adacdd0393f69ab3 Gerrit-Change-Number: 11948 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Fri, 16 Nov 2018 23:40:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/11948 ) Change subject: IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/11948/1/docs/topics/impala_adls.xml File docs/topics/impala_adls.xml: http://gerrit.cloudera.org:8080/#/c/11948/1/docs/topics/impala_adls.xml@283 PS1, Line 283: abfs://container@account.dfs.core.windows.net/path/fileorabfss://container@account.dfs.core.windows.net/path/file > I realize this is the output of a tool, but if we can break up this long li Fixed. Still look very long in XML, but the output HTML looks fine. http://gerrit.cloudera.org:8080/#/c/11948/1/docs/topics/impala_adls.xml@298 PS1, Line 298: abfss:// > "abfs://" Done -- To view, visit http://gerrit.cloudera.org:8080/11948 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I88f615cf23f406035e544e68adacdd0393f69ab3 Gerrit-Change-Number: 11948 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Fri, 16 Nov 2018 23:41:03 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/11948 ) Change subject: IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme .. Patch Set 2: Code-Review+2 Thanks, looks good. -- To view, visit http://gerrit.cloudera.org:8080/11948 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I88f615cf23f406035e544e68adacdd0393f69ab3 Gerrit-Change-Number: 11948 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Fri, 16 Nov 2018 23:45:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7801: Remove toSql() from ParseNode interface.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/11942 ) Change subject: IMPALA-7801: Remove toSql() from ParseNode interface. .. Patch Set 2: (6 comments) Thanks Paul for the interesting review and discussion http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/analysis/Expr.java File fe/src/main/java/org/apache/impala/analysis/Expr.java: http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/analysis/Expr.java@395 PS2, Line 395: String sql = toSql(DEFAULT); > Let's think about the meaning of the options for expressions. Expr nodes wo I understand the point about Expr nodes but there is no simple solution. The ToSqlOption(s) does allow more complex behaviors by having different methods that can be called on a ToSqlOption object. Csaba also suggested having some sort of context object to be passed as well. Maybe that is closer to what you want. In this change I am just trying to remove the default toSql() methods. http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/analysis/Expr.java@1300 PS2, Line 1300: name, (printExpr) ? " '" + toSql(DEFAULT) + "'" : "", type_.toString())); > This comment applies in a zillion other places. Our general rule for error In this change, as requested by Vuk, I am removing the default toSql() method. He thought, and I think I agree, that forcing a ToSqlOption object parameter makes things more consistent. If we don't want that then maybe I should abandon this change. http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/analysis/ParseNode.java File fe/src/main/java/org/apache/impala/analysis/ParseNode.java: http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/analysis/ParseNode.java@33 PS2, Line 33: String toSql(ToSqlOptions options); > A common practice in Java is to include the enum definition in the interfac Nice suggestion, done. http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/analysis/QueryStmt.java File fe/src/main/java/org/apache/impala/analysis/QueryStmt.java: http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/analysis/QueryStmt.java@217 PS2, Line 217: "OFFSET requires an ORDER BY clause: " + limitElement_.toSql(DEFAULT).trim()); > I wonder if we can solve a mess here with exceptions also. Rather than buil Nice idea, but IMHO beyond the scope of this change. http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/planner/SortNode.java File fe/src/main/java/org/apache/impala/planner/SortNode.java: http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/planner/SortNode.java@44 PS2, Line 44: import static org.apache.impala.analysis.ToSqlOptions.DEFAULT; > Two suggestions. Rename is done in next patch. I am not sure I agree about importing the enum (i.e. not a static import of the needed enums). I think it is tidier to have just ORIGINAL in the code. As it is (now) a common idiom I believe people will get used to reading this correctly. http://gerrit.cloudera.org:8080/#/c/11942/2/fe/src/main/java/org/apache/impala/planner/SortNode.java@214 PS2, Line 214: output.append(info_.getSortExprs().get(i).toSql(DEFAULT) + " "); > This is an example of the point made elsewhere. It SEEMS we are asking the OK, thanks for explaining this. I will add another followup jira which will say something like: IMPALA-7801 changed the code so that all ParseNode trees can be asked to create sql by calling toSql(SqlOption.ORIGINAL) or toSql(SqlOption.REWRITTEN). Although an Expr node produces output when toSql(SqlOption.ORIGINAL) is called, it is a lie in that it is not the original sql. The current Expr may have replaced a tree of the original Expr objects during query rewrite. The replaced objects are gone, so we cannot reconstruct the "original" sql. A possible solution is to observe that the select-like ParseNodes and Expr ParseNodes have different semantics and to change the interface of toSql() to express this. -- To view, visit http://gerrit.cloudera.org:8080/11942 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I17025901838e9ffd753894a8087170123f9d8b33 Gerrit-Change-Number: 11942 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 16 Nov 2018 23:46:37 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7801: Remove toSql() from ParseNode interface.
Andrew Sherman has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/11942 ) Change subject: IMPALA-7801: Remove toSql() from ParseNode interface. .. IMPALA-7801: Remove toSql() from ParseNode interface. In IMPALA-5821 the method "toSql(ToSqlOptions)" was added to ParseNode, to allow options to be passed when generating SQL from a parse tree. This change was suggested as part of the review of IMPALA-5821. The old "toSql()" method is currently implemented everywhere by calling toSql(ToSqlOptions.DEFAULT), but this is fragile as this convention could be accidentally ignored, To make the code more maintainable, remove the old "toSql()" method and have all callers call the new method instead. In addition, similarly replace "toSql()" in a few more associated classes: AnalyticWindow, LimitElement, Boundary, OrderByElement, SelectListItem. Finally, rename ToSqlOptions to ToSqlOption anbd noive it to be part of the ParseNode interface, and, in ToSqlOption, rename DEFAULT to ORIGINAL. TESTING: No new tests are added as there are no functional changes. All end to end tests run clean. Change-Id: I17025901838e9ffd753894a8087170123f9d8b33 --- M fe/src/main/java/org/apache/impala/analysis/AdminFnStmt.java M fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java M fe/src/main/java/org/apache/impala/analysis/AggregateInfoBase.java M fe/src/main/java/org/apache/impala/analysis/AlterTableAddDropRangePartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableAddPartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableAlterColStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableDropPartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableSetLocationStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterViewStmt.java M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java M fe/src/main/java/org/apache/impala/analysis/AnalyticInfo.java M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java M fe/src/main/java/org/apache/impala/analysis/BetweenPredicate.java M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/main/java/org/apache/impala/analysis/BoolLiteral.java M fe/src/main/java/org/apache/impala/analysis/CaseExpr.java M fe/src/main/java/org/apache/impala/analysis/CastExpr.java M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java M fe/src/main/java/org/apache/impala/analysis/CompoundPredicate.java M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateDataSrcStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateDbStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateDropRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateViewStmt.java M fe/src/main/java/org/apache/impala/analysis/DeleteStmt.java M fe/src/main/java/org/apache/impala/analysis/DescribeDbStmt.java M fe/src/main/java/org/apache/impala/analysis/DescribeTableStmt.java M fe/src/main/java/org/apache/impala/analysis/DropDataSrcStmt.java M fe/src/main/java/org/apache/impala/analysis/DropDbStmt.java M fe/src/main/java/org/apache/impala/analysis/DropFunctionStmt.java M fe/src/main/java/org/apache/impala/analysis/DropStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/DropTableOrViewStmt.java M fe/src/main/java/org/apache/impala/analysis/ExistsPredicate.java M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/main/java/org/apache/impala/analysis/ExprSubstitutionMap.java M fe/src/main/java/org/apache/impala/analysis/ExtractFromExpr.java M fe/src/main/java/org/apache/impala/analysis/FromClause.java M fe/src/main/java/org/apache/impala/analysis/FunctionArgs.java M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokeRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/HdfsCachingOp.java M fe/src/main/java/org/apache/impala/analysis/InPredicate.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/analysis/IsNotEmptyPredicate.java M fe/src/main/java/org/apache/impala/analysis/
[Impala-ASF-CR] IMPALA-7801: Remove toSql() from ParseNode interface.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11942 ) Change subject: IMPALA-7801: Remove toSql() from ParseNode interface. .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/11942/3/fe/src/test/java/org/apache/impala/analysis/ParserTest.java File fe/src/test/java/org/apache/impala/analysis/ParserTest.java: http://gerrit.cloudera.org:8080/#/c/11942/3/fe/src/test/java/org/apache/impala/analysis/ParserTest.java@2310 PS3, Line 2310: + "NULL ENCODING PLAIN_ENCODING COMPRESSION LZ4 ORIGINAL 10 BLOCK_SIZE 1024", line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/11942/3/fe/src/test/java/org/apache/impala/analysis/ParserTest.java@3277 PS3, Line 3277: "Expected: AND, AS, BETWEEN, ORIGINAL, DIV, FROM, ILIKE, IN, IREGEXP, IS, LIKE, " + line too long (91 > 90) -- To view, visit http://gerrit.cloudera.org:8080/11942 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I17025901838e9ffd753894a8087170123f9d8b33 Gerrit-Change-Number: 11942 Gerrit-PatchSet: 3 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 16 Nov 2018 23:48:45 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11948 ) Change subject: IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/154/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/11948 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I88f615cf23f406035e544e68adacdd0393f69ab3 Gerrit-Change-Number: 11948 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Fri, 16 Nov 2018 23:58:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 5 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Comment-Date: Sat, 17 Nov 2018 00:10:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7801: Remove toSql() from ParseNode interface.
Andrew Sherman has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/11942 ) Change subject: IMPALA-7801: Remove toSql() from ParseNode interface. .. IMPALA-7801: Remove toSql() from ParseNode interface. In IMPALA-5821 the method "toSql(ToSqlOptions)" was added to ParseNode, to allow options to be passed when generating SQL from a parse tree. This change was suggested as part of the review of IMPALA-5821. The old "toSql()" method is currently implemented everywhere by calling toSql(ToSqlOptions.DEFAULT), but this is fragile as this convention could be accidentally ignored, To make the code more maintainable, remove the old "toSql()" method and have all callers call the new method instead. In addition, similarly replace "toSql()" in a few more associated classes: AnalyticWindow, LimitElement, Boundary, OrderByElement, SelectListItem. Finally, rename ToSqlOptions to ToSqlOption anbd noive it to be part of the ParseNode interface, and, in ToSqlOption, rename DEFAULT to ORIGINAL. TESTING: No new tests are added as there are no functional changes. All end to end tests run clean. Change-Id: I17025901838e9ffd753894a8087170123f9d8b33 --- M fe/src/main/java/org/apache/impala/analysis/AdminFnStmt.java M fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java M fe/src/main/java/org/apache/impala/analysis/AggregateInfoBase.java M fe/src/main/java/org/apache/impala/analysis/AlterTableAddDropRangePartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableAddPartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableDropPartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableSetLocationStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterViewStmt.java M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java M fe/src/main/java/org/apache/impala/analysis/AnalyticInfo.java M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java M fe/src/main/java/org/apache/impala/analysis/BetweenPredicate.java M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/main/java/org/apache/impala/analysis/BoolLiteral.java M fe/src/main/java/org/apache/impala/analysis/CaseExpr.java M fe/src/main/java/org/apache/impala/analysis/CastExpr.java M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java M fe/src/main/java/org/apache/impala/analysis/CompoundPredicate.java M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateDataSrcStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateDbStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateDropRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateViewStmt.java M fe/src/main/java/org/apache/impala/analysis/DeleteStmt.java M fe/src/main/java/org/apache/impala/analysis/DescribeDbStmt.java M fe/src/main/java/org/apache/impala/analysis/DescribeTableStmt.java M fe/src/main/java/org/apache/impala/analysis/DropDataSrcStmt.java M fe/src/main/java/org/apache/impala/analysis/DropDbStmt.java M fe/src/main/java/org/apache/impala/analysis/DropFunctionStmt.java M fe/src/main/java/org/apache/impala/analysis/DropStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/DropTableOrViewStmt.java M fe/src/main/java/org/apache/impala/analysis/ExistsPredicate.java M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/main/java/org/apache/impala/analysis/ExprSubstitutionMap.java M fe/src/main/java/org/apache/impala/analysis/ExtractFromExpr.java M fe/src/main/java/org/apache/impala/analysis/FromClause.java M fe/src/main/java/org/apache/impala/analysis/FunctionArgs.java M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokeRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/HdfsCachingOp.java M fe/src/main/java/org/apache/impala/analysis/InPredicate.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/analysis/IsNotEmptyPredicate.java M fe/src/main/java/org/apache/impala/analysis/IsNullPredicate.java M fe/src/main/java/org/apache/impala/analysis/KuduPar
[Impala-ASF-CR] IMPALA-7801: Remove toSql() from ParseNode interface.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11942 ) Change subject: IMPALA-7801: Remove toSql() from ParseNode interface. .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1388/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11942 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I17025901838e9ffd753894a8087170123f9d8b33 Gerrit-Change-Number: 11942 Gerrit-PatchSet: 3 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Sat, 17 Nov 2018 00:20:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7801: Remove toSql() from ParseNode interface.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/11942 ) Change subject: IMPALA-7801: Remove toSql() from ParseNode interface. .. Patch Set 4: reviewers should ignore patch set 3 which is very wrong -- To view, visit http://gerrit.cloudera.org:8080/11942 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I17025901838e9ffd753894a8087170123f9d8b33 Gerrit-Change-Number: 11942 Gerrit-PatchSet: 4 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Sat, 17 Nov 2018 00:49:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7801: Remove toSql() from ParseNode interface.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11942 ) Change subject: IMPALA-7801: Remove toSql() from ParseNode interface. .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1389/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11942 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I17025901838e9ffd753894a8087170123f9d8b33 Gerrit-Change-Number: 11942 Gerrit-PatchSet: 4 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Sat, 17 Nov 2018 00:46:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. IMPALA-4123: Columnar decoding in Parquet The idea is to optimise the common case where there are long runs of NULL or non-NULL values (i.e. the def level is repeated). We can detect this cheaply by keying the decoding loop in the column reader off the state of the def level RLE decoder - if there's a long run of repeated levels, we can skip checking the def level for every value. We still fall back to decoding, caching and reading value-by-value a batch of def levels whenever the next def level is not in a repeated run. We still use the old approach for decoding rep levels. There might be some benefit to using the same approach for rep levels *if* repeated def and rep level runs line up. These changes should unlock further optimizations because more time is spent in simple kernel functions, e.g. UnpackAndDecode32Values() for dictionary decompression, which is very optimisable using SIMD etc. Snappy decompression now seems to be the main CPU bottleneck for decoding snappy-compressed Parquet. Perf: Running TPC-H scale factor 60 on uncompressed and snappy parquet both showed a ~4% speedup overall. Microbenchmarks on uncompressed parquet show scans only doing dictionary decoding on uncompressed Parquet is ~75% faster: set mt_dop=1; select min(l_returnflag) from lineitem; Testing: We have alltypes agg with a mix of null and non-null. Many tables have long runs of non-null values. Added new test data and coverage: * a test table manynulls with long runs of null values. * a large CHAR test table * missing coverage for materialising pos slot in flattened nested types scan. * Extended dict test to test longer runs. * A larger version of complextypestbl with interesting collection shapes - NULL collections, empty collections, etc, particularly runs of collections with the same shape. * Test interaction of timestamp validation with conversion * Ran code coverage build to confirm all code paths are tested * ASAN and exhaustive runs. Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Reviewed-on: http://gerrit.cloudera.org:8080/8319 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/exec/hdfs-parquet-scanner.cc M be/src/exec/parquet-column-readers.cc M be/src/exec/parquet-column-readers.h M be/src/exec/parquet-common.h M be/src/runtime/tuple.cc M be/src/runtime/tuple.h M be/src/util/bit-packing.h M be/src/util/bit-packing.inline.h M be/src/util/bit-stream-utils.h M be/src/util/bit-stream-utils.inline.h M be/src/util/dict-encoding.h M be/src/util/dict-test.cc A be/src/util/mem-util.h M be/src/util/rle-encoding.h M testdata/bin/generate-schema-statements.py M testdata/data/README A testdata/data/out_of_range_timestamp2_hive_211.parquet A testdata/data/out_of_range_timestamp_hive_211.parquet M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/chars.test M testdata/workloads/functional-query/queries/QueryTest/nested-types-scanner-position.test M testdata/workloads/functional-query/queries/QueryTest/nested-types-tpch.test A testdata/workloads/functional-query/queries/QueryTest/out-of-range-timestamp-local-tz-conversion.test A testdata/workloads/functional-query/queries/QueryTest/scanners-many-nulls.test M tests/custom_cluster/test_hive_parquet_timestamp_conversion.py M tests/query_test/test_scanners.py 27 files changed, 883 insertions(+), 133 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 25 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 24: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 24 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Sat, 17 Nov 2018 01:48:03 + Gerrit-HasComments: No
