[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16196 ) Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. IMPALA-9882: Import KLL functionality from Apache DataSketches First, I updated our existing snapshot of DataSketches to the following commit: c67d92faad3827932ca3b5d864222e64977f2c20 "Merge pull request #166 from gaborkaszab/const_cast" This affects files originated from kll/ and common/ directories of the DataSketches repo. Then I copied all the files needed for KLL into our snapshot directory. You can find the original Apache DataSketches files here: https://github.com/apache/incubator-datasketches-cpp This new snapshot however, broke the interface we used for serializing hll_union objects with dropping serialize_compact(). As a solution I had to make changes to the serialization and merging phases of the union operator by not serializing hll_union itself but the underlying hll_sketch instead. Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Reviewed-on: http://gerrit.cloudera.org:8080/16196 Reviewed-by: Gabor Kaszab Tested-by: Impala Public Jenkins --- M be/src/exprs/CMakeLists.txt M be/src/exprs/aggregate-functions-ir.cc M be/src/exprs/datasketches-test.cc M be/src/thirdparty/datasketches/AuxHashMap-internal.hpp D be/src/thirdparty/datasketches/CommonUtil.hpp M be/src/thirdparty/datasketches/CompositeInterpolationXTable-internal.hpp M be/src/thirdparty/datasketches/CompositeInterpolationXTable.hpp M be/src/thirdparty/datasketches/CouponHashSet-internal.hpp M be/src/thirdparty/datasketches/CouponList-internal.hpp M be/src/thirdparty/datasketches/Hll4Array-internal.hpp M be/src/thirdparty/datasketches/HllArray-internal.hpp M be/src/thirdparty/datasketches/HllSketch-internal.hpp M be/src/thirdparty/datasketches/HllSketchImplFactory.hpp M be/src/thirdparty/datasketches/HllUnion-internal.hpp M be/src/thirdparty/datasketches/HllUtil.hpp M be/src/thirdparty/datasketches/MurmurHash3.h M be/src/thirdparty/datasketches/README.md A be/src/thirdparty/datasketches/bounds_binomial_proportions.hpp A be/src/thirdparty/datasketches/common_defs.hpp A be/src/thirdparty/datasketches/count_zeros.hpp M be/src/thirdparty/datasketches/hll.hpp A be/src/thirdparty/datasketches/kll_helper.hpp A be/src/thirdparty/datasketches/kll_helper_impl.hpp A be/src/thirdparty/datasketches/kll_quantile_calculator.hpp A be/src/thirdparty/datasketches/kll_quantile_calculator_impl.hpp A be/src/thirdparty/datasketches/kll_sketch.hpp A be/src/thirdparty/datasketches/kll_sketch_impl.hpp A be/src/thirdparty/datasketches/memory_operations.hpp A be/src/thirdparty/datasketches/serde.hpp 29 files changed, 3,283 insertions(+), 348 deletions(-) Approvals: Gabor Kaszab: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 9 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16196 ) Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. Patch Set 8: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 8 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 27 Jul 2020 12:46:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16196 ) Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/6710/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 8 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 27 Jul 2020 07:57:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16196 ) Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. Patch Set 8: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6183/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 8 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 27 Jul 2020 07:32:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16196 ) Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. Patch Set 8: Code-Review+2 (4 comments) Carry +2 from Csaba. http://gerrit.cloudera.org:8080/#/c/16196/6/be/src/thirdparty/datasketches/README.md File be/src/thirdparty/datasketches/README.md: http://gerrit.cloudera.org:8080/#/c/16196/6/be/src/thirdparty/datasketches/README.md@2 PS6, Line 2: s > typo Done http://gerrit.cloudera.org:8080/#/c/16196/6/be/src/thirdparty/datasketches/README.md@4 PS6, Line 4: the following folders > I would prefer to move this to a separate list, where all lines contain a d Done http://gerrit.cloudera.org:8080/#/c/16196/6/be/src/thirdparty/datasketches/README.md@6 PS6, Line 6: kll/include/ > nit: . or ... ? Done http://gerrit.cloudera.org:8080/#/c/16196/6/be/src/thirdparty/datasketches/README.md@14 PS6, Line 14: Browse the source files here: > nit: extra line(s) Done -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 8 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 27 Jul 2020 07:31:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Hello Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16196 to look at the new patch set (#8). Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. IMPALA-9882: Import KLL functionality from Apache DataSketches First, I updated our existing snapshot of DataSketches to the following commit: c67d92faad3827932ca3b5d864222e64977f2c20 "Merge pull request #166 from gaborkaszab/const_cast" This affects files originated from kll/ and common/ directories of the DataSketches repo. Then I copied all the files needed for KLL into our snapshot directory. You can find the original Apache DataSketches files here: https://github.com/apache/incubator-datasketches-cpp This new snapshot however, broke the interface we used for serializing hll_union objects with dropping serialize_compact(). As a solution I had to make changes to the serialization and merging phases of the union operator by not serializing hll_union itself but the underlying hll_sketch instead. Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 --- M be/src/exprs/CMakeLists.txt M be/src/exprs/aggregate-functions-ir.cc M be/src/exprs/datasketches-test.cc M be/src/thirdparty/datasketches/AuxHashMap-internal.hpp D be/src/thirdparty/datasketches/CommonUtil.hpp M be/src/thirdparty/datasketches/CompositeInterpolationXTable-internal.hpp M be/src/thirdparty/datasketches/CompositeInterpolationXTable.hpp M be/src/thirdparty/datasketches/CouponHashSet-internal.hpp M be/src/thirdparty/datasketches/CouponList-internal.hpp M be/src/thirdparty/datasketches/Hll4Array-internal.hpp M be/src/thirdparty/datasketches/HllArray-internal.hpp M be/src/thirdparty/datasketches/HllSketch-internal.hpp M be/src/thirdparty/datasketches/HllSketchImplFactory.hpp M be/src/thirdparty/datasketches/HllUnion-internal.hpp M be/src/thirdparty/datasketches/HllUtil.hpp M be/src/thirdparty/datasketches/MurmurHash3.h M be/src/thirdparty/datasketches/README.md A be/src/thirdparty/datasketches/bounds_binomial_proportions.hpp A be/src/thirdparty/datasketches/common_defs.hpp A be/src/thirdparty/datasketches/count_zeros.hpp M be/src/thirdparty/datasketches/hll.hpp A be/src/thirdparty/datasketches/kll_helper.hpp A be/src/thirdparty/datasketches/kll_helper_impl.hpp A be/src/thirdparty/datasketches/kll_quantile_calculator.hpp A be/src/thirdparty/datasketches/kll_quantile_calculator_impl.hpp A be/src/thirdparty/datasketches/kll_sketch.hpp A be/src/thirdparty/datasketches/kll_sketch_impl.hpp A be/src/thirdparty/datasketches/memory_operations.hpp A be/src/thirdparty/datasketches/serde.hpp 29 files changed, 3,283 insertions(+), 348 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/16196/8 -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 8 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16196 ) Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. Patch Set 6: Code-Review+2 (4 comments) I have only nits about the README http://gerrit.cloudera.org:8080/#/c/16196/6/be/src/thirdparty/datasketches/README.md File be/src/thirdparty/datasketches/README.md: http://gerrit.cloudera.org:8080/#/c/16196/6/be/src/thirdparty/datasketches/README.md@2 PS6, Line 2: a typo http://gerrit.cloudera.org:8080/#/c/16196/6/be/src/thirdparty/datasketches/README.md@4 PS6, Line 4: hll/, kll/ and common/ I would prefer to move this to a separate list, where all lines contain a directory. http://gerrit.cloudera.org:8080/#/c/16196/6/be/src/thirdparty/datasketches/README.md@6 PS6, Line 6: compile them without rewriting the include paths in the files themselves.. nit: . or ... ? http://gerrit.cloudera.org:8080/#/c/16196/6/be/src/thirdparty/datasketches/README.md@14 PS6, Line 14: nit: extra line(s) -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 6 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 24 Jul 2020 13:43:17 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16196 ) Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/6699/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 6 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 23 Jul 2020 07:03:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Hello Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16196 to look at the new patch set (#6). Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. IMPALA-9882: Import KLL functionality from Apache DataSketches First, I updated our existing snapshot of DataSketches to the following commit: c67d92faad3827932ca3b5d864222e64977f2c20 "Merge pull request #166 from gaborkaszab/const_cast" This affects files originated from kll/ and common/ directories of the DataSketches repo. Then I copied all the files needed for KLL into our snapshot directory. You can find the original Apache DataSketches files here: https://github.com/apache/incubator-datasketches-cpp This new snapshot however, broke the interface we used for serializing hll_union objects with dropping serialize_compact(). As a solution I had to make changes to the serialization and merging phases of the union operator by not serializing hll_union itself but the underlying hll_sketch instead. Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 --- M be/src/exprs/CMakeLists.txt M be/src/exprs/aggregate-functions-ir.cc M be/src/exprs/datasketches-test.cc M be/src/thirdparty/datasketches/AuxHashMap-internal.hpp D be/src/thirdparty/datasketches/CommonUtil.hpp M be/src/thirdparty/datasketches/CompositeInterpolationXTable-internal.hpp M be/src/thirdparty/datasketches/CompositeInterpolationXTable.hpp M be/src/thirdparty/datasketches/CouponHashSet-internal.hpp M be/src/thirdparty/datasketches/CouponList-internal.hpp M be/src/thirdparty/datasketches/Hll4Array-internal.hpp M be/src/thirdparty/datasketches/HllArray-internal.hpp M be/src/thirdparty/datasketches/HllSketch-internal.hpp M be/src/thirdparty/datasketches/HllSketchImplFactory.hpp M be/src/thirdparty/datasketches/HllUnion-internal.hpp M be/src/thirdparty/datasketches/HllUtil.hpp M be/src/thirdparty/datasketches/MurmurHash3.h M be/src/thirdparty/datasketches/README.md A be/src/thirdparty/datasketches/bounds_binomial_proportions.hpp A be/src/thirdparty/datasketches/common_defs.hpp A be/src/thirdparty/datasketches/count_zeros.hpp M be/src/thirdparty/datasketches/hll.hpp A be/src/thirdparty/datasketches/kll_helper.hpp A be/src/thirdparty/datasketches/kll_helper_impl.hpp A be/src/thirdparty/datasketches/kll_quantile_calculator.hpp A be/src/thirdparty/datasketches/kll_quantile_calculator_impl.hpp A be/src/thirdparty/datasketches/kll_sketch.hpp A be/src/thirdparty/datasketches/kll_sketch_impl.hpp A be/src/thirdparty/datasketches/memory_operations.hpp A be/src/thirdparty/datasketches/serde.hpp 29 files changed, 3,280 insertions(+), 347 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/16196/6 -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 6 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16196 ) Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/6688/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 5 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 22 Jul 2020 13:44:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Hello Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16196 to look at the new patch set (#5). Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. IMPALA-9882: Import KLL functionality from Apache DataSketches First, I updated our existing snapshot of DataSketches to the following commit: dddc149209902f72b71109f1a098e58d6d4761ee "Merge pull request #159 from apache/workflow_update" This affects files originated from hll/ and common/ directories of the DataSketches repo. Then I copied all the files needed for KLL into our snapshot directory. You can find the original Apache DataSketches files here: https://github.com/apache/incubator-datasketches-cpp This new snapshot however, broke the interface we used for serializing hll_union objects with dropping serialize_compact(). As a solution I had to make changes to the serialization and merging phases of the union operator by not serializing hll_union itself but the underlying hll_sketch instead. Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 --- M be/src/exprs/CMakeLists.txt M be/src/exprs/aggregate-functions-ir.cc M be/src/exprs/datasketches-test.cc M be/src/thirdparty/datasketches/AuxHashMap-internal.hpp D be/src/thirdparty/datasketches/CommonUtil.hpp M be/src/thirdparty/datasketches/CompositeInterpolationXTable-internal.hpp M be/src/thirdparty/datasketches/CompositeInterpolationXTable.hpp M be/src/thirdparty/datasketches/CouponHashSet-internal.hpp M be/src/thirdparty/datasketches/CouponList-internal.hpp M be/src/thirdparty/datasketches/Hll4Array-internal.hpp M be/src/thirdparty/datasketches/HllArray-internal.hpp M be/src/thirdparty/datasketches/HllSketch-internal.hpp M be/src/thirdparty/datasketches/HllSketchImplFactory.hpp M be/src/thirdparty/datasketches/HllUnion-internal.hpp M be/src/thirdparty/datasketches/HllUtil.hpp M be/src/thirdparty/datasketches/MurmurHash3.h M be/src/thirdparty/datasketches/README.md A be/src/thirdparty/datasketches/bounds_binomial_proportions.hpp A be/src/thirdparty/datasketches/common_defs.hpp A be/src/thirdparty/datasketches/count_zeros.hpp M be/src/thirdparty/datasketches/hll.hpp A be/src/thirdparty/datasketches/kll_helper.hpp A be/src/thirdparty/datasketches/kll_helper_impl.hpp A be/src/thirdparty/datasketches/kll_quantile_calculator.hpp A be/src/thirdparty/datasketches/kll_quantile_calculator_impl.hpp A be/src/thirdparty/datasketches/kll_sketch.hpp A be/src/thirdparty/datasketches/kll_sketch_impl.hpp A be/src/thirdparty/datasketches/memory_operations.hpp A be/src/thirdparty/datasketches/serde.hpp 29 files changed, 3,280 insertions(+), 347 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/16196/5 -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 5 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16196 ) Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/6680/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 4 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 21 Jul 2020 14:49:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Hello Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16196 to look at the new patch set (#4). Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. IMPALA-9882: Import KLL functionality from Apache DataSketches First, I updated our existing snapshot of DataSketches to the following commit: dddc149209902f72b71109f1a098e58d6d4761ee "Merge pull request #159 from apache/workflow_update" This affects files originated from hll/ and common/ directories of the DataSketches repo. Then I copied all the files needed for KLL into our snapshot directory. You can find the original Apache DataSketches files here: https://github.com/apache/incubator-datasketches-cpp This new snapshot however, broke the interface we used for serializing hll_union objects with dropping serialize_compact(). As a solution I had to make changes to the serialization and merging phases of the union operator by not serializing hll_union itself but the underlying hll_sketch instead. Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 --- M be/src/exprs/CMakeLists.txt M be/src/exprs/aggregate-functions-ir.cc M be/src/exprs/datasketches-test.cc M be/src/thirdparty/datasketches/AuxHashMap-internal.hpp D be/src/thirdparty/datasketches/CommonUtil.hpp M be/src/thirdparty/datasketches/CompositeInterpolationXTable-internal.hpp M be/src/thirdparty/datasketches/CompositeInterpolationXTable.hpp M be/src/thirdparty/datasketches/CouponHashSet-internal.hpp M be/src/thirdparty/datasketches/CouponList-internal.hpp M be/src/thirdparty/datasketches/Hll4Array-internal.hpp M be/src/thirdparty/datasketches/HllArray-internal.hpp M be/src/thirdparty/datasketches/HllSketch-internal.hpp M be/src/thirdparty/datasketches/HllSketchImplFactory.hpp M be/src/thirdparty/datasketches/HllUnion-internal.hpp M be/src/thirdparty/datasketches/HllUtil.hpp M be/src/thirdparty/datasketches/MurmurHash3.h M be/src/thirdparty/datasketches/README.md A be/src/thirdparty/datasketches/bounds_binomial_proportions.hpp A be/src/thirdparty/datasketches/common_defs.hpp A be/src/thirdparty/datasketches/count_zeros.hpp M be/src/thirdparty/datasketches/hll.hpp A be/src/thirdparty/datasketches/kll_helper.hpp A be/src/thirdparty/datasketches/kll_helper_impl.hpp A be/src/thirdparty/datasketches/kll_quantile_calculator.hpp A be/src/thirdparty/datasketches/kll_quantile_calculator_impl.hpp A be/src/thirdparty/datasketches/kll_sketch.hpp A be/src/thirdparty/datasketches/kll_sketch_impl.hpp A be/src/thirdparty/datasketches/memory_operations.hpp A be/src/thirdparty/datasketches/serde.hpp 29 files changed, 3,280 insertions(+), 347 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/16196/4 -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 4 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16196 ) Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. Patch Set 3: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/6634/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 3 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 17 Jul 2020 13:28:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16196 ) Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. Patch Set 3: In PS3 I've put this patch on top of the ds_hll_union() patch and turned out that the new snapshot I brought in here brokes the interface of hll_union I used in the parent patch. I had to make some adjustments. -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 3 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 17 Jul 2020 13:05:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Hello Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16196 to look at the new patch set (#3). Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. IMPALA-9882: Import KLL functionality from Apache DataSketches First, I updated our existing snapshot of DataSketches to the following commit: dddc149209902f72b71109f1a098e58d6d4761ee "Merge pull request #159 from apache/workflow_update" This affects files originated from hll/ and common/ directories of the DataSketches repo. Then I copied all the files needed for KLL into our snapshot directory. You can find the original Apache DataSketches files here: https://github.com/apache/incubator-datasketches-cpp This new snapshot however, broke the interface we used for serializing hll_union objects with dropping serialize_compact(). As a solution I had to make changes to the serialization and merging phases of the union operator by not serializing hll_union itself but the underlying hll_sketch instead. Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 --- M be/src/exprs/CMakeLists.txt M be/src/exprs/aggregate-functions-ir.cc M be/src/exprs/datasketches-test.cc M be/src/thirdparty/datasketches/AuxHashMap-internal.hpp D be/src/thirdparty/datasketches/CommonUtil.hpp M be/src/thirdparty/datasketches/CompositeInterpolationXTable-internal.hpp M be/src/thirdparty/datasketches/CompositeInterpolationXTable.hpp M be/src/thirdparty/datasketches/CouponHashSet-internal.hpp M be/src/thirdparty/datasketches/CouponList-internal.hpp M be/src/thirdparty/datasketches/Hll4Array-internal.hpp M be/src/thirdparty/datasketches/HllArray-internal.hpp M be/src/thirdparty/datasketches/HllSketch-internal.hpp M be/src/thirdparty/datasketches/HllSketchImplFactory.hpp M be/src/thirdparty/datasketches/HllUnion-internal.hpp M be/src/thirdparty/datasketches/HllUtil.hpp M be/src/thirdparty/datasketches/MurmurHash3.h M be/src/thirdparty/datasketches/README.md A be/src/thirdparty/datasketches/bounds_binomial_proportions.hpp A be/src/thirdparty/datasketches/common_defs.hpp A be/src/thirdparty/datasketches/count_zeros.hpp M be/src/thirdparty/datasketches/hll.hpp A be/src/thirdparty/datasketches/kll_helper.hpp A be/src/thirdparty/datasketches/kll_helper_impl.hpp A be/src/thirdparty/datasketches/kll_quantile_calculator.hpp A be/src/thirdparty/datasketches/kll_quantile_calculator_impl.hpp A be/src/thirdparty/datasketches/kll_sketch.hpp A be/src/thirdparty/datasketches/kll_sketch_impl.hpp A be/src/thirdparty/datasketches/memory_operations.hpp A be/src/thirdparty/datasketches/serde.hpp 29 files changed, 3,280 insertions(+), 347 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/16196/3 -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 3 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16196 ) Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. Patch Set 1: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/6592/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 1 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 14 Jul 2020 15:12:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9882: Import KLL functionality from Apache DataSketches
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16196 to look at the new patch set (#2). Change subject: IMPALA-9882: Import KLL functionality from Apache DataSketches .. IMPALA-9882: Import KLL functionality from Apache DataSketches First, I updated our existing snapshot of DataSketches to the following commit: dddc149209902f72b71109f1a098e58d6d4761ee "Merge pull request #159 from apache/workflow_update" This affects files originated from hll/ and common/ directories of the DataSketches repo. Then I copied all the files needed for KLL into our snapshot directory. You can find the original Apache DataSketches files here: https://github.com/apache/incubator-datasketches-cpp Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 --- M be/src/exprs/CMakeLists.txt M be/src/exprs/datasketches-test.cc M be/src/thirdparty/datasketches/AuxHashMap-internal.hpp D be/src/thirdparty/datasketches/CommonUtil.hpp M be/src/thirdparty/datasketches/CompositeInterpolationXTable-internal.hpp M be/src/thirdparty/datasketches/CompositeInterpolationXTable.hpp M be/src/thirdparty/datasketches/CouponHashSet-internal.hpp M be/src/thirdparty/datasketches/CouponList-internal.hpp M be/src/thirdparty/datasketches/Hll4Array-internal.hpp M be/src/thirdparty/datasketches/HllArray-internal.hpp M be/src/thirdparty/datasketches/HllSketch-internal.hpp M be/src/thirdparty/datasketches/HllSketchImplFactory.hpp M be/src/thirdparty/datasketches/HllUnion-internal.hpp M be/src/thirdparty/datasketches/HllUtil.hpp M be/src/thirdparty/datasketches/MurmurHash3.h M be/src/thirdparty/datasketches/README.md A be/src/thirdparty/datasketches/bounds_binomial_proportions.hpp A be/src/thirdparty/datasketches/common_defs.hpp A be/src/thirdparty/datasketches/count_zeros.hpp M be/src/thirdparty/datasketches/hll.hpp A be/src/thirdparty/datasketches/kll_helper.hpp A be/src/thirdparty/datasketches/kll_helper_impl.hpp A be/src/thirdparty/datasketches/kll_quantile_calculator.hpp A be/src/thirdparty/datasketches/kll_quantile_calculator_impl.hpp A be/src/thirdparty/datasketches/kll_sketch.hpp A be/src/thirdparty/datasketches/kll_sketch_impl.hpp A be/src/thirdparty/datasketches/memory_operations.hpp A be/src/thirdparty/datasketches/serde.hpp 28 files changed, 3,255 insertions(+), 333 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/16196/2 -- To view, visit http://gerrit.cloudera.org:8080/16196 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I848488d5145c808109bd50aecfbf3ef83f981943 Gerrit-Change-Number: 16196 Gerrit-PatchSet: 2 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
