[Impala-ASF-CR](2.x) IMPALA-7174: fix test cancellation for RELEASE builds
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12504 ) Change subject: IMPALA-7174: fix test_cancellation for RELEASE builds .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12504 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: comment Gerrit-Change-Id: I41da7b5ac58a468a8ed11969906f63df6d4b Gerrit-Change-Number: 12504 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 16 Feb 2019 07:29:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8193: Fix python 2.6 issue in junit prune notrun.py
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12479 ) Change subject: IMPALA-8193: Fix python 2.6 issue in junit_prune_notrun.py .. IMPALA-8193: Fix python 2.6 issue in junit_prune_notrun.py Python 2.6's ElementTree.write() does not have an xml_declaration argument, so junitxml_prune_notrun.py fails on python 2.6. This fixes junitxml_prune_notrun.py by using minidom to write the output. This mirrors how bin/generate_junitxml.py outputs XML. Verified that tests now pass on python 2.6 and python 2.7 does not change. Change-Id: I9ef8fb77b1ac8c51e3dfb6b04690ae9ccc490d62 Reviewed-on: http://gerrit.cloudera.org:8080/12479 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M lib/python/impala_py_lib/jenkins/junitxml_prune_notrun.py 1 file changed, 9 insertions(+), 3 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/12479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I9ef8fb77b1ac8c51e3dfb6b04690ae9ccc490d62 Gerrit-Change-Number: 12479 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8193: Fix python 2.6 issue in junit prune notrun.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12479 ) Change subject: IMPALA-8193: Fix python 2.6 issue in junit_prune_notrun.py .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9ef8fb77b1ac8c51e3dfb6b04690ae9ccc490d62 Gerrit-Change-Number: 12479 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 16 Feb 2019 04:47:59 + Gerrit-HasComments: No
[Impala-ASF-CR](2.x) IMPALA-7174: fix test cancellation for RELEASE builds
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12504 ) Change subject: IMPALA-7174: fix test_cancellation for RELEASE builds .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3787/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12504 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: comment Gerrit-Change-Id: I41da7b5ac58a468a8ed11969906f63df6d4b Gerrit-Change-Number: 12504 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 16 Feb 2019 03:30:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7976 : Add a flag to disable sync using events at a table level
Impala Public Jenkins has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/12365 )
Change subject: IMPALA-7976 : Add a flag to disable sync using events at a
table level
..
IMPALA-7976 : Add a flag to disable sync using events at a table level
This change adds the ability to disable event processing at a table or
database level. If the user adds a property with the key
"impala.disableHmsSync" in the table or database properties, event
processing can be turned on/off at the respective level. The value of
the property is string value of Booleans ("true" or "false"). By default
event processing is enabled when the global config
hms_event_polling_interval_s is set to a non-zero value. The value of
property determines if the event processing needs to be disabled for a
particular table or database. So for example, if
"impala.disableHmsSync" is set to "true" then events for that
table/database are ignored. If the property is unset or a value of
"false" is used, the event processing is enabled (although it is not
necessary to explicitly enable event processing, given the global flag).
When both table and db level properties are set, the table level
property takes precedence. If the table level property is not set, then
database level property is used to evaluate if the event needs to be
processed or not.
Users can issue alter table command to change the value of this
property. When the flag value is changed, the alter table event
generated is always processed and subsequent events are either ignored
or processed based on the new value of table flag and database flag. In
theory, a database level property can also be changed. But Impala
doesn't currently support alter database operations. Users who may wish
to change the database level property should use alter database command
from Apache Hive (or any other metastore client)
When the flag is turned OFF and the previous value of ON or unset, it is
possible that the table does not exist in catalog anymore. For example,
when the table was created the event sync was disabled. Hence the create
event was skipped. Now when the user re-enables the sync again on such
table, event processor has no idea how many events were skipped during
this interval. Event processor errs on the side of caution and changes
the state to NEEDS_INVALIDATE in such case.
Testing done: Added new unit tests which execute all supported DDL
operations with all possible combination of db and table flags. Another
test executes alter table operations for changing flags for all combinations
of transitions at table level.
Change-Id: If116edba51bae139bc003be858b3214c3f0104e3
Reviewed-on: http://gerrit.cloudera.org:8080/12365
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
---
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
D fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventUtils.java
A fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M
fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
A
fe/src/main/java/org/apache/impala/catalog/events/MetastoreNotificationNeedsInvalidateException.java
M
fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java
6 files changed, 1,651 insertions(+), 752 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/12365
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: If116edba51bae139bc003be858b3214c3f0104e3
Gerrit-Change-Number: 12365
Gerrit-PatchSet: 13
Gerrit-Owner: Vihang Karajgaonkar
Gerrit-Reviewer: Bharath Krishna
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Paul Rogers
Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR](2.x) IMPALA-7174: fix test cancellation for RELEASE builds
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12504 ) Change subject: IMPALA-7174: fix test_cancellation for RELEASE builds .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2145/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12504 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: comment Gerrit-Change-Id: I41da7b5ac58a468a8ed11969906f63df6d4b Gerrit-Change-Number: 12504 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 16 Feb 2019 01:09:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7976 : Add a flag to disable sync using events at a table level
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12365 ) Change subject: IMPALA-7976 : Add a flag to disable sync using events at a table level .. Patch Set 12: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12365 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If116edba51bae139bc003be858b3214c3f0104e3 Gerrit-Change-Number: 12365 Gerrit-PatchSet: 12 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Bharath Krishna Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Sat, 16 Feb 2019 01:08:11 + Gerrit-HasComments: No
[Impala-ASF-CR](2.x) IMPALA-8204: checkout to the right commit of Impala-lzo in buildall.sh
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/12495 ) Change subject: IMPALA-8204: checkout to the right commit of Impala-lzo in buildall.sh .. Patch Set 2: (1 comment) > Patch Set 2: > > > I still think we should write down the depended Impala-lzo version > > somewhere. Once I want to build an older version of Impala, I don't > > need to guess which Impala-lzo commit should I check out to. > > it's in bin/bootstrap_development.sh, no? bootstrap_development.sh just chooses branches. I think we can also checkout to the right commit(version) there. But we need to source impala-config.sh beffore it. Two year agos when I wanted to build Impala-2.5-cdh5.7.3, I encountered build errors from Impala-lzo too. At that time Impala-2.5 was also a fairly old version, so I can just guessing the right commit of Impala-lzo based on the committed time. > > > > > > There's a benifit to do the checkout in buildall.sh. In the future > > when I want to build an older version (but >2.13, >3.2), I just > > need to checkout to the right tag in Impala and then run > > buildall.sh. It'll bootstrap dependencies including the Impala-lzo > > to the corresponding versions. > > > > If you're developing Impala-lzo, you can first commit it locally, > > and then change the depended commit id in impala-config-branch.sh. > > I think this is the same with other dependencies. For example, if > > we want to bump the version of ORC lib, we need to upgrade it in > > native-toolchain project and then change the corresponding version > > in impala-config.sh. > > I understand your argument, but I don't think it's acceptable for a "build" > script to change branches of git repos. We could treat impala-lzo like we > treat the toolchain (i.e., checked out in bootstrap_toolchain.sh), and then > I'd sort of see it as a "build artifact" which can be overridden rather than > my source code. Sure. I'm ok to move it to bootstrap_toolchain.sh. > (BTW, if you choose to cherrypick the num_unqueued_files that drove this > mess, we'll be back to using the same impala-lzo code base in both > branches...) Sure. That would fix the mess quickly! Let's at least pin the Impala-lzo version(tag or commit id) to ease builds of older version from now on. http://gerrit.cloudera.org:8080/#/c/12495/2/bin/impala-config-branch.sh File bin/impala-config-branch.sh: http://gerrit.cloudera.org:8080/#/c/12495/2/bin/impala-config-branch.sh@26 PS2, Line 26: IMPALA_LZO_VERSION=8a984e0 > Do you want to use a long git hash or a tag or a branch name instead? The s Sure. Creating tags for each Impala version may be better but that may be tedious for your management. I change it to long git hash. -- To view, visit http://gerrit.cloudera.org:8080/12495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: comment Gerrit-Change-Id: I5e4b53d695de04d31d39b6909572907944713ba4 Gerrit-Change-Number: 12495 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Sat, 16 Feb 2019 01:00:34 + Gerrit-HasComments: Yes
[Impala-ASF-CR](2.x) IMPALA-8204: checkout to the right commit of Impala-lzo in buildall.sh
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/12495 ) Change subject: IMPALA-8204: checkout to the right commit of Impala-lzo in buildall.sh .. Patch Set 2: > I still think we should write down the depended Impala-lzo version > somewhere. Once I want to build an older version of Impala, I don't > need to guess which Impala-lzo commit should I check out to. it's in bin/bootstrap_development.sh, no? > > There's a benifit to do the checkout in buildall.sh. In the future > when I want to build an older version (but >2.13, >3.2), I just > need to checkout to the right tag in Impala and then run > buildall.sh. It'll bootstrap dependencies including the Impala-lzo > to the corresponding versions. > > If you're developing Impala-lzo, you can first commit it locally, > and then change the depended commit id in impala-config-branch.sh. > I think this is the same with other dependencies. For example, if > we want to bump the version of ORC lib, we need to upgrade it in > native-toolchain project and then change the corresponding version > in impala-config.sh. I understand your argument, but I don't think it's acceptable for a "build" script to change branches of git repos. We could treat impala-lzo like we treat the toolchain (i.e., checked out in bootstrap_toolchain.sh), and then I'd sort of see it as a "build artifact" which can be overridden rather than my source code. (BTW, if you choose to cherrypick the num_unqueued_files that drove this mess, we'll be back to using the same impala-lzo code base in both branches...) -- To view, visit http://gerrit.cloudera.org:8080/12495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: comment Gerrit-Change-Id: I5e4b53d695de04d31d39b6909572907944713ba4 Gerrit-Change-Number: 12495 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Sat, 16 Feb 2019 00:34:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8193: Fix python 2.6 issue in junit prune notrun.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12479 ) Change subject: IMPALA-8193: Fix python 2.6 issue in junit_prune_notrun.py .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9ef8fb77b1ac8c51e3dfb6b04690ae9ccc490d62 Gerrit-Change-Number: 12479 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 16 Feb 2019 00:30:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8193: Fix python 2.6 issue in junit prune notrun.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12479 ) Change subject: IMPALA-8193: Fix python 2.6 issue in junit_prune_notrun.py .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3786/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9ef8fb77b1ac8c51e3dfb6b04690ae9ccc490d62 Gerrit-Change-Number: 12479 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 16 Feb 2019 00:30:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8193: Fix python 2.6 issue in junit prune notrun.py
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/12479 ) Change subject: IMPALA-8193: Fix python 2.6 issue in junit_prune_notrun.py .. Patch Set 1: Code-Review+2 Sure. -- To view, visit http://gerrit.cloudera.org:8080/12479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9ef8fb77b1ac8c51e3dfb6b04690ae9ccc490d62 Gerrit-Change-Number: 12479 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 16 Feb 2019 00:29:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8207: Fix query loading for perf and stress tests
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12503 ) Change subject: IMPALA-8207: Fix query loading for perf and stress tests .. Patch Set 2: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/12503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id1966d6ca8babdda07d47e089b75ba06d0318c0d Gerrit-Change-Number: 12503 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 16 Feb 2019 00:21:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8207: Fix query loading for perf and stress tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12503 ) Change subject: IMPALA-8207: Fix query loading for perf and stress tests .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2144/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id1966d6ca8babdda07d47e089b75ba06d0318c0d Gerrit-Change-Number: 12503 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 16 Feb 2019 00:22:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8191: Wait for additional breakpad processes during test
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/12501 ) Change subject: IMPALA-8191: Wait for additional breakpad processes during test .. Patch Set 1: (3 comments) Ok. Thanks for explaining. I'm fine with the tactical fix, but I think doing process.get_pids() is probably easy enough and will be less confusing. http://gerrit.cloudera.org:8080/#/c/12501/1/tests/common/impala_cluster.py File tests/common/impala_cluster.py: http://gerrit.cloudera.org:8080/#/c/12501/1/tests/common/impala_cluster.py@321 PS1, Line 321: if set(self.cmd) == set(process.cmdline): Do you know why this is set? http://gerrit.cloudera.org:8080/#/c/12501/1/tests/common/impala_cluster.py@322 PS1, Line 322: return pid Instead of returning pid here, you could collect all matches, and then either pick the better one (parent) or fail? Would that have made this more obvious? http://gerrit.cloudera.org:8080/#/c/12501/1/tests/custom_cluster/test_breakpad.py File tests/custom_cluster/test_breakpad.py: http://gerrit.cloudera.org:8080/#/c/12501/1/tests/custom_cluster/test_breakpad.py@88 PS1, Line 88: def wait_for_all_processes_dead(self, processes, timeout=300): : # For every process in the list we might see the original Impala process plus a forked : # off child that is writing the minidump. To catch both we go through the list twice. : for process in processes * 2: > These are Process objects (https://github.com/apache/impala/blob/master/tes OMG. I would never have thought that "Process" is really "output of pgrep". How would you feel about changing this to process.get_pids() here, so that it's more obvious that a process can get represented multiple times in this case, and we have an answer? -- To view, visit http://gerrit.cloudera.org:8080/12501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia4dcc5fecb9b5f38ae1504aae40f099837cf1bca Gerrit-Change-Number: 12501 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Sat, 16 Feb 2019 00:27:02 + Gerrit-HasComments: Yes
[Impala-ASF-CR](2.x) IMPALA-7174: fix test cancellation for RELEASE builds
Hello Impala Public Jenkins, I'd like you to do a code review. Please visit http://gerrit.cloudera.org:8080/12504 to review the following change. Change subject: IMPALA-7174: fix test_cancellation for RELEASE builds .. IMPALA-7174: fix test_cancellation for RELEASE builds The test was DOA when run against a release build because the debug actions that it depends on were disabled. The fix is to enable the debug actions for release builds, similar to other debug actions. I assume the original motivation of the NDEBUG checks was to avoid adding overhead to release builds. The cost is minimised by quickly checking whether the string is empty before proceeding with any further work. Also remove wonky exception handling - the test was swallowing exceptions but we don't expect that code to throw exceptions. Testing: Looped the test on a release build. Change-Id: I41da7b5ac58a468a8ed11969906f63df6d4b Reviewed-on: http://gerrit.cloudera.org:8080/10722 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/scheduling/admission-controller.cc M be/src/service/client-request-state.cc M be/src/util/debug-util.cc M be/src/util/debug-util.h M tests/custom_cluster/test_admission_controller.py 5 files changed, 14 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/12504/1 -- To view, visit http://gerrit.cloudera.org:8080/12504 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: newchange Gerrit-Change-Id: I41da7b5ac58a468a8ed11969906f63df6d4b Gerrit-Change-Number: 12504 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR](2.x) IMPALA-8204: checkout to the right commit of Impala-lzo in buildall.sh
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/12495 ) Change subject: IMPALA-8204: checkout to the right commit of Impala-lzo in buildall.sh .. Patch Set 2: I still think we should write down the depended Impala-lzo version somewhere. Once I want to build an older version of Impala, I don't need to guess which Impala-lzo commit should I check out to. There's a benifit to do the checkout in buildall.sh. In the future when I want to build an older version (but >2.13, >3.2), I just need to checkout to the right tag in Impala and then run buildall.sh. It'll bootstrap dependencies including the Impala-lzo to the corresponding versions. If you're developing Impala-lzo, you can first commit it locally, and then change the depended commit id in impala-config-branch.sh. I think this is the same with other dependencies. For example, if we want to bump the version of ORC lib, we need to upgrade it in native-toolchain project and then change the corresponding version in impala-config.sh. -- To view, visit http://gerrit.cloudera.org:8080/12495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: comment Gerrit-Change-Id: I5e4b53d695de04d31d39b6909572907944713ba4 Gerrit-Change-Number: 12495 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Sat, 16 Feb 2019 00:21:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8207: Fix query loading for perf and stress tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12503 ) Change subject: IMPALA-8207: Fix query loading for perf and stress tests .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2143/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id1966d6ca8babdda07d47e089b75ba06d0318c0d Gerrit-Change-Number: 12503 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 16 Feb 2019 00:21:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8207: Fix query loading for perf and stress tests
Hello David Knupp, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12503 to look at the new patch set (#2). Change subject: IMPALA-8207: Fix query loading for perf and stress tests .. IMPALA-8207: Fix query loading for perf and stress tests Problems with perf queries (run-workload.py): - TPCH picks up stress test specific queries (TPCH-AGG1/2/3) - TPCDS picks up queries that were intended just to validate that data was loaded properly but that aren't interesting from a perf perspective (TPCDS-COUNT-) - TPCDS picks up both decimal_v1 and decimal_v2 queries. This is mostly harmless as for queries with matching names only one gets run but it causes some queries with mismatched names to be run twice (TPCDS-Q39-1/2 vs. TPCDS-Q39.1/2) Problems with stress queries (concurrent_select.py): - TPCDS fails to pick up Q22A as it does not use the decimal_v2 queries, even though decimal_v2 is the default now. This problem is exacerbated by the fact that the two scripts have different code paths for selecting the queries, so in the past changes that were made to one path were not always made to the other. This patch merges the two paths to reduce code duplication and prevent these sorts of issues in the future, and fixes the above issues. One complication is that historically the stress test has used query names in the form 'q1' whereas the perf test has used query names in the form 'TPCH-Q1'. This patch standardizes on using 'TPCH-Q1'. Testing: - Added a test that checks that the perf tests pick up the expected number of queries. - Manually ran the scripts and verified that the correct queries are selected. Change-Id: Id1966d6ca8babdda07d47e089b75ba06d0318c0d --- M testdata/workloads/tpcds/queries/tpcds-decimal_v2-q39-1.test M testdata/workloads/tpcds/queries/tpcds-decimal_v2-q39-2.test A tests/infra/test_perf_infra.py M tests/infra/test_stress_infra.py M tests/performance/workload.py M tests/stress/concurrent_select.py M tests/util/parse_util.py M tests/util/test_file_parser.py 8 files changed, 107 insertions(+), 63 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/03/12503/2 -- To view, visit http://gerrit.cloudera.org:8080/12503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id1966d6ca8babdda07d47e089b75ba06d0318c0d Gerrit-Change-Number: 12503 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8207: Fix query loading for perf and stress tests
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/12503 ) Change subject: IMPALA-8207: Fix query loading for perf and stress tests .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/12503/1/tests/infra/test_perf_infra.py File tests/infra/test_perf_infra.py: http://gerrit.cloudera.org:8080/#/c/12503/1/tests/infra/test_perf_infra.py@30 PS1, Line 30: > flake8: E302 expected 2 blank lines, found 1 Done http://gerrit.cloudera.org:8080/#/c/12503/1/tests/stress/concurrent_select.py File tests/stress/concurrent_select.py: http://gerrit.cloudera.org:8080/#/c/12503/1/tests/stress/concurrent_select.py@1310 PS1, Line 1310: > flake8: E502 the backslash is redundant between brackets Done -- To view, visit http://gerrit.cloudera.org:8080/12503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id1966d6ca8babdda07d47e089b75ba06d0318c0d Gerrit-Change-Number: 12503 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 15 Feb 2019 23:42:00 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8207: Fix query loading for perf and stress tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12503 ) Change subject: IMPALA-8207: Fix query loading for perf and stress tests .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/12503/1/tests/infra/test_perf_infra.py File tests/infra/test_perf_infra.py: http://gerrit.cloudera.org:8080/#/c/12503/1/tests/infra/test_perf_infra.py@30 PS1, Line 30: class TestPerfInfra(ImpalaTestSuite): flake8: E302 expected 2 blank lines, found 1 http://gerrit.cloudera.org:8080/#/c/12503/1/tests/stress/concurrent_select.py File tests/stress/concurrent_select.py: http://gerrit.cloudera.org:8080/#/c/12503/1/tests/stress/concurrent_select.py@1310 PS1, Line 1310: \ flake8: E502 the backslash is redundant between brackets -- To view, visit http://gerrit.cloudera.org:8080/12503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id1966d6ca8babdda07d47e089b75ba06d0318c0d Gerrit-Change-Number: 12503 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 15 Feb 2019 23:37:06 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8207: Fix query loading for perf and stress tests
Thomas Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12503 Change subject: IMPALA-8207: Fix query loading for perf and stress tests .. IMPALA-8207: Fix query loading for perf and stress tests Problems with perf queries (run-workload.py): - TPCH picks up stress test specific queries (TPCH-AGG1/2/3) - TPCDS picks up queries that were intended just to validate that data was loaded properly but that aren't interesting from a perf perspective (TPCDS-COUNT-) - TPCDS picks up both decimal_v1 and decimal_v2 queries. This is mostly harmless as for queries with matching names only one gets run but it causes some queries with mismatched names to be run twice (TPCDS-Q39-1/2 vs. TPCDS-Q39.1/2) Problems with stress queries (concurrent_select.py): - TPCDS fails to pick up Q22A as it does not use the decimal_v2 queries, even though decimal_v2 is the default now. This problem is exacerbated by the fact that the two scripts have different code paths for selecting the queries, so in the past changes that were made to one path were not always made to the other. This patch merges the two paths to reduce code duplication and prevent these sorts of issues in the future, and fixes the above issues. One complication is that historically the stress test has used query names in the form 'q1' whereas the perf test has used query names in the form 'TPCH-Q1'. This patch standardizes on using 'TPCH-Q1'. Testing: - Added a test that checks that the perf tests pick up the expected number of queries. - Manually ran the scripts and verified that the correct queries are selected. Change-Id: Id1966d6ca8babdda07d47e089b75ba06d0318c0d --- M testdata/workloads/tpcds/queries/tpcds-decimal_v2-q39-1.test M testdata/workloads/tpcds/queries/tpcds-decimal_v2-q39-2.test A tests/infra/test_perf_infra.py M tests/infra/test_stress_infra.py M tests/performance/workload.py M tests/stress/concurrent_select.py M tests/util/parse_util.py M tests/util/test_file_parser.py 8 files changed, 106 insertions(+), 63 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/03/12503/1 -- To view, visit http://gerrit.cloudera.org:8080/12503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id1966d6ca8babdda07d47e089b75ba06d0318c0d Gerrit-Change-Number: 12503 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall
[Impala-ASF-CR] IMPALA-8191: Wait for additional breakpad processes during test
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/12501 ) Change subject: IMPALA-8191: Wait for additional breakpad processes during test .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/12501/1/tests/custom_cluster/test_breakpad.py File tests/custom_cluster/test_breakpad.py: http://gerrit.cloudera.org:8080/#/c/12501/1/tests/custom_cluster/test_breakpad.py@88 PS1, Line 88: def wait_for_all_processes_dead(self, processes, timeout=300): : # For every process in the list we might see the original Impala process plus a forked : # off child that is writing the minidump. To catch both we go through the list twice. : for process in processes * 2: > I don't understand this. processes here is a list of pids, and we wait() on These are Process objects (https://github.com/apache/impala/blob/master/tests/common/impala_cluster.py#L280). They store their command and look up their PID by iterating through all PIDs on the system comparing them to their cmd. I think the idea behind this is that processes can restart, but of course we could track the expected state including its PID more explicitly. Let me know if you'd like me to explain what these are in the comment and/or file a Jira to improve the Process class to know its PID. -- To view, visit http://gerrit.cloudera.org:8080/12501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia4dcc5fecb9b5f38ae1504aae40f099837cf1bca Gerrit-Change-Number: 12501 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Fri, 15 Feb 2019 23:10:30 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8191: Wait for additional breakpad processes during test
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/12501 ) Change subject: IMPALA-8191: Wait for additional breakpad processes during test .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/12501/1/tests/custom_cluster/test_breakpad.py File tests/custom_cluster/test_breakpad.py: http://gerrit.cloudera.org:8080/#/c/12501/1/tests/custom_cluster/test_breakpad.py@88 PS1, Line 88: def wait_for_all_processes_dead(self, processes, timeout=300): : # For every process in the list we might see the original Impala process plus a forked : # off child that is writing the minidump. To catch both we go through the list twice. : for process in processes * 2: I don't understand this. processes here is a list of pids, and we wait() on each of them. Why does waiting on them multiple times help things? It sounded from your description in the commit message is that the issue is where we enumarate which process to wait for (the impalad or its fork), but that's not what this code is doing. I'm sure I'm confused; let me know where! -- To view, visit http://gerrit.cloudera.org:8080/12501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia4dcc5fecb9b5f38ae1504aae40f099837cf1bca Gerrit-Change-Number: 12501 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Fri, 15 Feb 2019 23:00:54 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8203: [DOCS] Clarified a few points in DISABLE CODEGEN
Hello Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12496 to look at the new patch set (#2). Change subject: IMPALA-8203: [DOCS] Clarified a few points in DISABLE_CODEGEN .. IMPALA-8203: [DOCS] Clarified a few points in DISABLE_CODEGEN Change-Id: Ic10fd66d8a83d10c0900e93a00241d532902a759 --- M docs/topics/impala_disable_codegen.xml 1 file changed, 38 insertions(+), 12 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/12496/2 -- To view, visit http://gerrit.cloudera.org:8080/12496 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic10fd66d8a83d10c0900e93a00241d532902a759 Gerrit-Change-Number: 12496 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8203: [DOCS] Clarified a few points in DISABLE CODEGEN
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12496 ) Change subject: IMPALA-8203: [DOCS] Clarified a few points in DISABLE_CODEGEN .. IMPALA-8203: [DOCS] Clarified a few points in DISABLE_CODEGEN Change-Id: Ic10fd66d8a83d10c0900e93a00241d532902a759 Reviewed-on: http://gerrit.cloudera.org:8080/12496 Tested-by: Impala Public Jenkins Reviewed-by: Alex Rodoni --- M docs/topics/impala_disable_codegen.xml 1 file changed, 38 insertions(+), 12 deletions(-) Approvals: Impala Public Jenkins: Verified Alex Rodoni: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/12496 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ic10fd66d8a83d10c0900e93a00241d532902a759 Gerrit-Change-Number: 12496 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8203: [DOCS] Clarified a few points in DISABLE CODEGEN
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/12496 ) Change subject: IMPALA-8203: [DOCS] Clarified a few points in DISABLE_CODEGEN .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12496 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic10fd66d8a83d10c0900e93a00241d532902a759 Gerrit-Change-Number: 12496 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 15 Feb 2019 21:31:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8108: Impala query returns TIMESTAMP values in different types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12498 ) Change subject: IMPALA-8108: Impala query returns TIMESTAMP values in different types .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3784/ -- To view, visit http://gerrit.cloudera.org:8080/12498 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iad9a4659ff6d5353329e9d062886ee80296b5b43 Gerrit-Change-Number: 12498 Gerrit-PatchSet: 3 Gerrit-Owner: Robbie Zhang Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 15 Feb 2019 21:31:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8099: Update the build scripts to support Apache Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12469 ) Change subject: IMPALA-8099: Update the build scripts to support Apache Ranger .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I249cd64d74518946829e8588ed33d5ac454ffa7b Gerrit-Change-Number: 12469 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anonymous Coward (402) Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Fri, 15 Feb 2019 21:28:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8099: Update the build scripts to support Apache Ranger
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12469 ) Change subject: IMPALA-8099: Update the build scripts to support Apache Ranger .. IMPALA-8099: Update the build scripts to support Apache Ranger This patch updates the build scripts to suport Apache Ranger: - Download Apache Ranger - Setup Apache Ranger database - Create Apache Ranger configuration files - Start/stop Apache Ranger Testing: - Ran ./buildall.sh -format on a clean repository and was able to start Ranger without any problem. - Ran test-with-docker Change-Id: I249cd64d74518946829e8588ed33d5ac454ffa7b Reviewed-on: http://gerrit.cloudera.org:8080/12469 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M bin/bootstrap_toolchain.py M bin/create-test-configuration.sh M bin/impala-config.sh M buildall.sh M testdata/bin/kill-all.sh A testdata/bin/kill-ranger-server.sh M testdata/bin/run-all.sh A testdata/bin/run-ranger-server.sh A testdata/cluster/ranger/install.properties.template A testdata/cluster/ranger/java_home.sh A testdata/cluster/ranger/ranger-admin-default-site.xml.template A testdata/cluster/ranger/ranger-admin-env-logdir.sh A testdata/cluster/ranger/ranger-admin-env-piddir.sh A testdata/cluster/ranger/ranger-admin-site.xml.template A testdata/cluster/ranger/security-applicationContext.xml 15 files changed, 1,262 insertions(+), 30 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/12469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I249cd64d74518946829e8588ed33d5ac454ffa7b Gerrit-Change-Number: 12469 Gerrit-PatchSet: 5 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anonymous Coward (402) Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger
[Impala-ASF-CR] [DOCS] Copy edit a part of impala timestamp.xml
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12502 ) Change subject: [DOCS] Copy edit a part of impala_timestamp.xml .. [DOCS] Copy edit a part of impala_timestamp.xml Change-Id: I140ca64692982c7409653e01e9d089a730e5e787 Reviewed-on: http://gerrit.cloudera.org:8080/12502 Reviewed-by: Alex Rodoni Tested-by: Impala Public Jenkins --- M docs/topics/impala_timestamp.xml 1 file changed, 12 insertions(+), 9 deletions(-) Approvals: Alex Rodoni: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/12502 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I140ca64692982c7409653e01e9d089a730e5e787 Gerrit-Change-Number: 12502 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7976 : Add a flag to disable sync using events at a table level
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12365 ) Change subject: IMPALA-7976 : Add a flag to disable sync using events at a table level .. Patch Set 12: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3785/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12365 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If116edba51bae139bc003be858b3214c3f0104e3 Gerrit-Change-Number: 12365 Gerrit-PatchSet: 12 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Bharath Krishna Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 15 Feb 2019 21:14:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7976 : Add a flag to disable sync using events at a table level
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12365 ) Change subject: IMPALA-7976 : Add a flag to disable sync using events at a table level .. Patch Set 12: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12365 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If116edba51bae139bc003be858b3214c3f0104e3 Gerrit-Change-Number: 12365 Gerrit-PatchSet: 12 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Bharath Krishna Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 15 Feb 2019 21:14:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7976 : Add a flag to disable sync using events at a table level
Paul Rogers has posted comments on this change. ( http://gerrit.cloudera.org:8080/12365 ) Change subject: IMPALA-7976 : Add a flag to disable sync using events at a table level .. Patch Set 11: Code-Review+2 LGTM. -- To view, visit http://gerrit.cloudera.org:8080/12365 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If116edba51bae139bc003be858b3214c3f0104e3 Gerrit-Change-Number: 12365 Gerrit-PatchSet: 11 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Bharath Krishna Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 15 Feb 2019 21:13:44 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Copy edit a part of impala timestamp.xml
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12502 ) Change subject: [DOCS] Copy edit a part of impala_timestamp.xml .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/221/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/12502 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I140ca64692982c7409653e01e9d089a730e5e787 Gerrit-Change-Number: 12502 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 15 Feb 2019 21:12:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8191: Wait for additional breakpad processes during test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12501 ) Change subject: IMPALA-8191: Wait for additional breakpad processes during test .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2142/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia4dcc5fecb9b5f38ae1504aae40f099837cf1bca Gerrit-Change-Number: 12501 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 15 Feb 2019 21:12:01 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Copy edit a part of impala timestamp.xml
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12502 ) Change subject: [DOCS] Copy edit a part of impala_timestamp.xml .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/221/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/12502 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I140ca64692982c7409653e01e9d089a730e5e787 Gerrit-Change-Number: 12502 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 15 Feb 2019 21:05:09 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Copy edit a part of impala timestamp.xml
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/12502 ) Change subject: [DOCS] Copy edit a part of impala_timestamp.xml .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12502 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I140ca64692982c7409653e01e9d089a730e5e787 Gerrit-Change-Number: 12502 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 15 Feb 2019 21:05:12 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Copy edit a part of impala timestamp.xml
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12502 Change subject: [DOCS] Copy edit a part of impala_timestamp.xml .. [DOCS] Copy edit a part of impala_timestamp.xml Change-Id: I140ca64692982c7409653e01e9d089a730e5e787 --- M docs/topics/impala_timestamp.xml 1 file changed, 12 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/02/12502/1 -- To view, visit http://gerrit.cloudera.org:8080/12502 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I140ca64692982c7409653e01e9d089a730e5e787 Gerrit-Change-Number: 12502 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-8191: Wait for additional breakpad processes during test
Lars Volker has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12501 Change subject: IMPALA-8191: Wait for additional breakpad processes during test .. IMPALA-8191: Wait for additional breakpad processes during test The Breakpad signal handler forks off a process to write a minidump. During the breakpad tests we send signals to the Impala daemons and then wait for all processes to go away. Prior to this change we did this by waiting on the PID returned by process.get_pid(). It is determined by iterating over psutil.get_pid_list() which is an ordered list of PIDs running on the system. We return the first process in the list with a matching command line. In cases where the PID space rolled over, this could have been the forked off breakpad process and we'd wait on that one. During the subsequent check that all processes are indeed gone, we could then pick up the original Impala daemon that had forked off to write the minidump and was still in the process of shutting down. To fix this, we wait for every process twice. Processes are identified by their command and iterating through them twice makes sure we catch both the original daemon and it's breakpad child. This change also contains improvements to the logging of processes in our tests. This should make it easier to identify similar issues in the future. Testing: I ran the breakpad tests in exhaustive mode. I didn't try to exercise it around a PID roll-over, but we shouldn't see the issue in IMPALA-8191 again. Change-Id: Ia4dcc5fecb9b5f38ae1504aae40f099837cf1bca --- M tests/common/impala_cluster.py M tests/custom_cluster/test_breakpad.py 2 files changed, 27 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/12501/1 -- To view, visit http://gerrit.cloudera.org:8080/12501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia4dcc5fecb9b5f38ae1504aae40f099837cf1bca Gerrit-Change-Number: 12501 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker
[Impala-ASF-CR] IMPALA-8203: [DOCS] Clarified a few points in DISABLE CODEGEN
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/12496 ) Change subject: IMPALA-8203: [DOCS] Clarified a few points in DISABLE_CODEGEN .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/12496/1/docs/topics/impala_disable_codegen.xml File docs/topics/impala_disable_codegen.xml: http://gerrit.cloudera.org:8080/#/c/12496/1/docs/topics/impala_disable_codegen.xml@48 PS1, Line 48: disabled > I think the word "disabled" can be removed here. Done -- To view, visit http://gerrit.cloudera.org:8080/12496 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic10fd66d8a83d10c0900e93a00241d532902a759 Gerrit-Change-Number: 12496 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 15 Feb 2019 19:05:45 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8203: [DOCS] Clarified a few points in DISABLE CODEGEN
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12496 ) Change subject: IMPALA-8203: [DOCS] Clarified a few points in DISABLE_CODEGEN .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/220/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/12496 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic10fd66d8a83d10c0900e93a00241d532902a759 Gerrit-Change-Number: 12496 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 15 Feb 2019 19:10:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8203: [DOCS] Clarified a few points in DISABLE CODEGEN
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12496 ) Change subject: IMPALA-8203: [DOCS] Clarified a few points in DISABLE_CODEGEN .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/220/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/12496 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic10fd66d8a83d10c0900e93a00241d532902a759 Gerrit-Change-Number: 12496 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 15 Feb 2019 19:05:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8108: Impala query returns TIMESTAMP values in different types
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/12498 ) Change subject: IMPALA-8108: Impala query returns TIMESTAMP values in different types .. Patch Set 3: (3 comments) Some high level concerns: this seems like a potentially breaking change, as some workloads may depend on the current way of printing timestamps without subseconds. Adding a flag to change this behavior would be a relatively easy way to avoid breaking things by default. TimestampValue's ToString() function is used in a lot of places internally. If the goal is to return the results differently to the user, then it is enough to change the way Impala returns timestamps in HS2/Beeswax. Or do you also want to change the way cast(timestamp_column as string) works? I guess that not every existing test was run, as I would expect a lot of them to fail because of this change. I started a gerrit-verify-dryrun build on jenkins to verify this. http://gerrit.cloudera.org:8080/#/c/12498/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12498/3//COMMIT_MSG@10 PS3, Line 10: : seconds if they are all zeros. nit: long line, commit message lines are capped at 72 characters http://gerrit.cloudera.org:8080/#/c/12498/3/be/src/runtime/timestamp-value.cc File be/src/runtime/timestamp-value.cc: http://gerrit.cloudera.org:8080/#/c/12498/3/be/src/runtime/timestamp-value.cc@211 PS3, Line 211: BOOST_DATE_TIME_POSIX_TIME_STD_CONFIG What is the role of this config? Generally we assume that Impala works with nanosec precision and lot of tests would break in a different build. If this will be always defined and the #else path is not tested, then I would prefer a compile error if it is not defined. e.g. #ifndef BOOST_DATE_TIME_POSIX_TIME_STD_CONFIG #error "BOOST_DATE_TIME_POSIX_TIME_STD_CONFIG should be defined." #endif http://gerrit.cloudera.org:8080/#/c/12498/3/testdata/workloads/functional-query/queries/QueryTest/timestamps.test File testdata/workloads/functional-query/queries/QueryTest/timestamps.test: http://gerrit.cloudera.org:8080/#/c/12498/3/testdata/workloads/functional-query/queries/QueryTest/timestamps.test@3 PS3, Line 3: nit: trailing spaces -- To view, visit http://gerrit.cloudera.org:8080/12498 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iad9a4659ff6d5353329e9d062886ee80296b5b43 Gerrit-Change-Number: 12498 Gerrit-PatchSet: 3 Gerrit-Owner: Robbie Zhang Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 15 Feb 2019 17:44:03 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8108: Impala query returns TIMESTAMP values in different types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12498 ) Change subject: IMPALA-8108: Impala query returns TIMESTAMP values in different types .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3784/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/12498 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iad9a4659ff6d5353329e9d062886ee80296b5b43 Gerrit-Change-Number: 12498 Gerrit-PatchSet: 3 Gerrit-Owner: Robbie Zhang Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 15 Feb 2019 17:30:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8204: checkout to the right commit of Impala-lzo in buildall.sh
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/12497 ) Change subject: IMPALA-8204: checkout to the right commit of Impala-lzo in buildall.sh .. Patch Set 3: Please see my reviews in https://gerrit.cloudera.org/#/c/12495/. I don't think it's appropriate for buildall to move git checkouts around. -- To view, visit http://gerrit.cloudera.org:8080/12497 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5e4b53d695de04d31d39b6909572907944713ba4 Gerrit-Change-Number: 12497 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Fri, 15 Feb 2019 17:46:38 + Gerrit-HasComments: No
[Impala-ASF-CR](2.x) IMPALA-8204: checkout to the right commit of Impala-lzo in buildall.sh
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/12495 ) Change subject: IMPALA-8204: checkout to the right commit of Impala-lzo in buildall.sh .. Patch Set 2: (2 comments) I don't think this approach is wise. We have two places where we check out Impala-lzo: a) bin/bootstrap_system.sh: git clone --branch master https://github.com/cloudera/impala-lzo.git "$IMPALA_LZO_HOME" b) Jenkins jobs. I think it's ok to move a git branch when doing the checkout, but it's not so nice to move the branch when you're doing buildall. (For example, if I were to need to make a change to impala-lzo, this would confuse me incredibly, since buildall would be undoing my changes!) http://gerrit.cloudera.org:8080/#/c/12495/2/bin/impala-config-branch.sh File bin/impala-config-branch.sh: http://gerrit.cloudera.org:8080/#/c/12495/2/bin/impala-config-branch.sh@26 PS2, Line 26: IMPALA_LZO_VERSION=8a984e0 Do you want to use a long git hash or a tag or a branch name instead? The short git hash is sort of the worst of all worlds, since it's not guaranteed to work if there's a collision. http://gerrit.cloudera.org:8080/#/c/12495/2/buildall.sh File buildall.sh: http://gerrit.cloudera.org:8080/#/c/12495/2/buildall.sh@337 PS2, Line 337: # Checkout to the right version of Impala-lzo. IMPALA_LZO_VERSION is set in : # bin/impala-config-branch.sh : if [[ -e "$IMPALA_LZO" ]]; then : echo "Checkout to commit $IMPALA_LZO_VERSION for Impala-lzo repo in $IMPALA_LZO" : (pushd "$IMPALA_LZO" && git checkout "$IMPALA_LZO_VERSION") : fi I don't think it's appropriate for buildall.sh to move a repo around like this. -- To view, visit http://gerrit.cloudera.org:8080/12495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: comment Gerrit-Change-Id: I5e4b53d695de04d31d39b6909572907944713ba4 Gerrit-Change-Number: 12495 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Fri, 15 Feb 2019 17:45:49 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8203: [DOCS] Clarified a few points in DISABLE CODEGEN
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12496 ) Change subject: IMPALA-8203: [DOCS] Clarified a few points in DISABLE_CODEGEN .. Patch Set 1: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/12496/1/docs/topics/impala_disable_codegen.xml File docs/topics/impala_disable_codegen.xml: http://gerrit.cloudera.org:8080/#/c/12496/1/docs/topics/impala_disable_codegen.xml@48 PS1, Line 48: disabled I think the word "disabled" can be removed here. -- To view, visit http://gerrit.cloudera.org:8080/12496 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic10fd66d8a83d10c0900e93a00241d532902a759 Gerrit-Change-Number: 12496 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 15 Feb 2019 17:24:36 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8099: Update the build scripts to support Apache Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12469 ) Change subject: IMPALA-8099: Update the build scripts to support Apache Ranger .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I249cd64d74518946829e8588ed33d5ac454ffa7b Gerrit-Change-Number: 12469 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anonymous Coward (402) Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Fri, 15 Feb 2019 17:15:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8099: Update the build scripts to support Apache Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12469 ) Change subject: IMPALA-8099: Update the build scripts to support Apache Ranger .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3783/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I249cd64d74518946829e8588ed33d5ac454ffa7b Gerrit-Change-Number: 12469 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anonymous Coward (402) Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Fri, 15 Feb 2019 17:15:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8099: Update the build scripts to support Apache Ranger
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/12469 ) Change subject: IMPALA-8099: Update the build scripts to support Apache Ranger .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I249cd64d74518946829e8588ed33d5ac454ffa7b Gerrit-Change-Number: 12469 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anonymous Coward (402) Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Fri, 15 Feb 2019 17:13:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7368: Add initial support for DATE type
Csaba Ringhofer has posted comments on this change. (
http://gerrit.cloudera.org:8080/12481 )
Change subject: IMPALA-7368: Add initial support for DATE type
..
Patch Set 3:
(9 comments)
http://gerrit.cloudera.org:8080/#/c/12481/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/12481/3//COMMIT_MSG@22
PS3, Line 22: - from BIGINT, INT, SMALLINT, TINYINT to DATE. The source
value is
: interpreted as a number of days since the epoch.
: - from DATE to BIGINT, INT, SMALLINT, TINYINT. The resulting
: integer is the number of days since epoch.
: - from DOUBLE, FLOAT, DECIMAL to DATE. The source value's
: fractional part is ignored, the integer part is interpreted
as a
: number of days since epoch.
: - from DATE to DOUBLE, FLOAT, DECIMAL. The resulting value is
the
: number of days since epoch.
Hive does not support these casts (I checked with Hive 2.1.1), are you sure
that it is useful to support them in Impala? My first impression is that these
casts are more confusing than useful.
http://gerrit.cloudera.org:8080/#/c/12481/3/be/src/udf/udf.h
File be/src/udf/udf.h:
http://gerrit.cloudera.org:8080/#/c/12481/3/be/src/udf/udf.h@586
PS3, Line 586: /// Date value represented as days since epoch 1970-01-01.
The valid range and the fact that the value is interpreted as proleptic
Gregorian could be also mentioned here.
http://gerrit.cloudera.org:8080/#/c/12481/3/be/src/util/string-parser-test.cc
File be/src/util/string-parser-test.cc:
http://gerrit.cloudera.org:8080/#/c/12481/3/be/src/util/string-parser-test.cc@526
PS3, Line 526: TestDateValue("-01-01", DateValue(0, 1, 1),
StringParser::PARSE_SUCCESS);
: TestDateValue("-12-31", DateValue(, 12, 31),
StringParser::PARSE_SUCCESS);
Please add the "wrong side" of the edge values:
-0001-12-31 (Hive had a bug with these kinds of values)
1-01-01
http://gerrit.cloudera.org:8080/#/c/12481/3/be/src/util/string-parser-test.cc@531
PS3, Line 531: TestDateValue("2-11-10", invalid_date,
StringParser::PARSE_FAILURE);
Please add tests where month/day have the correct number of characters but the
value is invalid, e.g 2018-00-10, 2018-13-10, 2018-01-0, 2018-01-32, 2019-02-39
http://gerrit.cloudera.org:8080/#/c/12481/3/common/thrift/Exprs.thrift
File common/thrift/Exprs.thrift:
http://gerrit.cloudera.org:8080/#/c/12481/3/common/thrift/Exprs.thrift@56
PS3, Line 56: // string representation of date formatted as -MM-dd.
: 1: required string value;
Why do you use string instead of i32 to represent the date? TTimestampLiteral
uses binary representation instead of string.
http://gerrit.cloudera.org:8080/#/c/12481/3/fe/src/main/java/org/apache/impala/analysis/DateLiteral.java
File fe/src/main/java/org/apache/impala/analysis/DateLiteral.java:
http://gerrit.cloudera.org:8080/#/c/12481/3/fe/src/main/java/org/apache/impala/analysis/DateLiteral.java@32
PS3, Line 32: yyy
Isn't it ?
http://gerrit.cloudera.org:8080/#/c/12481/3/fe/src/main/java/org/apache/impala/analysis/StringLiteral.java
File fe/src/main/java/org/apache/impala/analysis/StringLiteral.java:
http://gerrit.cloudera.org:8080/#/c/12481/3/fe/src/main/java/org/apache/impala/analysis/StringLiteral.java@155
PS3, Line 155: isDateType
By implementing the Date type the name of this function became very misleading.
Can you rename it to something like isDateOrTimeType()?
http://gerrit.cloudera.org:8080/#/c/12481/3/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java:
http://gerrit.cloudera.org:8080/#/c/12481/3/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java@194
PS3, Line 194: AnalyzesOk("select cast (cast ('1970-10-10 10:00:00.123' as
timestamp) as date)");
What will happen if some writes
select cast('1970-10-10 10:00:00.123' as date)
? Will it return NULL without warning?
http://gerrit.cloudera.org:8080/#/c/12481/3/fe/src/test/java/org/apache/impala/analysis/ExprRewriterTest.java
File fe/src/test/java/org/apache/impala/analysis/ExprRewriterTest.java:
http://gerrit.cloudera.org:8080/#/c/12481/3/fe/src/test/java/org/apache/impala/analysis/ExprRewriterTest.java@368
PS3, Line 368: assertToSqlWithImplicitCasts(ctx, "select * from
functional.alltypes, "
: + "functional.date_tbl where timestamp_col = date_col",
: "SELECT * FROM functional.alltypes, functional.date_tbl "
: + "WHERE timestamp_col = CAST(date_col AS TIMESTAMP)");
I saw some weirdness around this in Hive, it seems to convert everything to
string:
select cast("1970-01-01" as timestamp) > cast("1970-01-01" as date);
return True
What happened is that the appended 00:00:00 part made the timestamp version
[Impala-ASF-CR] Revert "Update toolchain to support ubuntu 18.04"
Lars Volker has abandoned this change. ( http://gerrit.cloudera.org:8080/12499 ) Change subject: Revert "Update toolchain to support ubuntu 18.04" .. Abandoned Builds now -- To view, visit http://gerrit.cloudera.org:8080/12499 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: Ib8b9dd9ab2564a70bc7f01530e4d6445153f77e1 Gerrit-Change-Number: 12499 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: hector.aco...@cloudera.com
[Impala-ASF-CR] Revert "Update toolchain to support ubuntu 18.04"
hector.aco...@cloudera.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/12499 ) Change subject: Revert "Update toolchain to support ubuntu 18.04" .. Patch Set 1: Code-Review-1 Added ubuntu 14.04 to 43961c5c-7ece-489c-a6e3-ec4cbe7ef4b5-b23c19a002. -- To view, visit http://gerrit.cloudera.org:8080/12499 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8b9dd9ab2564a70bc7f01530e4d6445153f77e1 Gerrit-Change-Number: 12499 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: hector.aco...@cloudera.com Gerrit-Comment-Date: Fri, 15 Feb 2019 16:22:21 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "Update toolchain to support ubuntu 18.04"
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12499 ) Change subject: Revert "Update toolchain to support ubuntu 18.04" .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2141/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12499 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8b9dd9ab2564a70bc7f01530e4d6445153f77e1 Gerrit-Change-Number: 12499 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Fri, 15 Feb 2019 15:27:22 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "Update toolchain to support ubuntu 18.04"
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/12499 ) Change subject: Revert "Update toolchain to support ubuntu 18.04" .. Patch Set 1: Code-Review+1 I checked the S3 bucket too: the binary tarballs for Ubuntu14 are not there. I recall that the toolchain changes for supporting Ubuntu18 actually contain code that builds those bits for Ubuntu14 (https://gerrit.cloudera.org/c/12285/ and https://gerrit.cloudera.org/c/12286/), so I assume this was just a local build that went bad. IMPALA-7273 says we still support Ubuntu 14 with toolchain bits, so let's fix that first in the toolchain, then we can revive this patch. -- To view, visit http://gerrit.cloudera.org:8080/12499 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8b9dd9ab2564a70bc7f01530e4d6445153f77e1 Gerrit-Change-Number: 12499 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Fri, 15 Feb 2019 14:48:34 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "Update toolchain to support ubuntu 18.04"
Lars Volker has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12499 Change subject: Revert "Update toolchain to support ubuntu 18.04" .. Revert "Update toolchain to support ubuntu 18.04" This reverts commit f8c9ef48419966840b6026a7d81cfa11a3540a17, which broke the Ubuntu 14.04 developer builds. Change-Id: Ib8b9dd9ab2564a70bc7f01530e4d6445153f77e1 --- M bin/bootstrap_toolchain.py M bin/impala-config.sh 2 files changed, 8 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/12499/1 -- To view, visit http://gerrit.cloudera.org:8080/12499 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib8b9dd9ab2564a70bc7f01530e4d6445153f77e1 Gerrit-Change-Number: 12499 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker
[Impala-ASF-CR] IMPALA-7368: Add initial support for DATE type
Csaba Ringhofer has posted comments on this change. (
http://gerrit.cloudera.org:8080/12481 )
Change subject: IMPALA-7368: Add initial support for DATE type
..
Patch Set 3:
(4 comments)
I ran through the non-test cpp code.
http://gerrit.cloudera.org:8080/#/c/12481/3/be/src/exec/hdfs-scan-node-base.cc
File be/src/exec/hdfs-scan-node-base.cc:
http://gerrit.cloudera.org:8080/#/c/12481/3/be/src/exec/hdfs-scan-node-base.cc@644
PS3, Line 644: // TODO: Remove this block once DATE type is supported accross
all file formats.
: if (has_materialized_date_slot_ && partition->file_format() !=
THdfsFileFormat::TEXT) {
: context->ClearStreams();
: return Status("DATE type is only supported with TEXT file
format.");
: }
I think that this logic should be implemented in frontend, and the backend
should only contains DCHECKs.
http://gerrit.cloudera.org:8080/#/c/12481/3/be/src/exprs/cast-functions-ir.cc
File be/src/exprs/cast-functions-ir.cc:
http://gerrit.cloudera.org:8080/#/c/12481/3/be/src/exprs/cast-functions-ir.cc@310
PS3, Line 310: int year, month, day;
: if (UNLIKELY(!dv.ToYearMonthDay(, , ))) return
TimestampVal::null();
: if (year < 1400 || year > ) return TimestampVal::null();
:
: const boost::gregorian::date d(year, month, day);
: const boost::posix_time::time_duration t(0, 0, 0, 0);
: TimestampValue tv(d, t);
performance: this could be done much faster, as both DateVal and
TimestampValue's date_ are "number of days since some epoch", so the conversion
should be simply checking a range + add the difference between the to epoch.
http://gerrit.cloudera.org:8080/#/c/12481/3/be/src/exprs/cast-functions-ir.cc@330
PS3, Line 330: if (val.is_null) return DateVal::null();
: TimestampValue tv = TimestampValue::FromTimestampVal(val);
: if (UNLIKELY(!tv.HasDate())) return DateVal::null();
:
: const boost::gregorian::date d = tv.date();
: const DateValue dv(d.year(), d.month(), d.day());
: return dv.ToDateVal();
same as in line 310: this could be done with a simple addition. There is a
function that does exactly this conversion in
https://gerrit.cloudera.org/#/c/12247/8/be/src/runtime/timestamp-value.inline.h
( int64_t TimestampValue::DaysSinceUnixEpoch() )
http://gerrit.cloudera.org:8080/#/c/12481/3/be/src/exprs/slot-ref.cc
File be/src/exprs/slot-ref.cc:
http://gerrit.cloudera.org:8080/#/c/12481/3/be/src/exprs/slot-ref.cc@485
PS3, Line 485: const DateValue
dv(*reinterpret_cast(t->GetSlot(slot_offset_))
Why don't we reinterpret_cast to DateValue* directly? This would avoid the
unnecessary range check in DateValue's constructor.
--
To view, visit http://gerrit.cloudera.org:8080/12481
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iea8155ef09557e0afa2f8b2d0b2dc9d0896dc30f
Gerrit-Change-Number: 12481
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Jeges
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Fri, 15 Feb 2019 13:29:31 +
Gerrit-HasComments: Yes
[Impala-ASF-CR](2.x) IMPALA-8204: checkout to the right commit of Impala-lzo in buildall.sh
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12495 ) Change subject: IMPALA-8204: checkout to the right commit of Impala-lzo in buildall.sh .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: comment Gerrit-Change-Id: I5e4b53d695de04d31d39b6909572907944713ba4 Gerrit-Change-Number: 12495 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 15 Feb 2019 10:46:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8204: checkout to the right commit of Impala-lzo in buildall.sh
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12497 ) Change subject: IMPALA-8204: checkout to the right commit of Impala-lzo in buildall.sh .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12497 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5e4b53d695de04d31d39b6909572907944713ba4 Gerrit-Change-Number: 12497 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 15 Feb 2019 10:39:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8108: Impala query returns TIMESTAMP values in different types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12498 ) Change subject: IMPALA-8108: Impala query returns TIMESTAMP values in different types .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2140/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12498 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iad9a4659ff6d5353329e9d062886ee80296b5b43 Gerrit-Change-Number: 12498 Gerrit-PatchSet: 3 Gerrit-Owner: Robbie Zhang Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 15 Feb 2019 09:01:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8108: Impala query returns TIMESTAMP values in different types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12498 ) Change subject: IMPALA-8108: Impala query returns TIMESTAMP values in different types .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2138/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12498 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iad9a4659ff6d5353329e9d062886ee80296b5b43 Gerrit-Change-Number: 12498 Gerrit-PatchSet: 1 Gerrit-Owner: Robbie Zhang Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 15 Feb 2019 08:55:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8108: Impala query returns TIMESTAMP values in different types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12498 ) Change subject: IMPALA-8108: Impala query returns TIMESTAMP values in different types .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2139/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12498 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iad9a4659ff6d5353329e9d062886ee80296b5b43 Gerrit-Change-Number: 12498 Gerrit-PatchSet: 2 Gerrit-Owner: Robbie Zhang Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 15 Feb 2019 08:53:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8108: Impala query returns TIMESTAMP values in different types
Hello Greg Rahn, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12498 to look at the new patch set (#3). Change subject: IMPALA-8108: Impala query returns TIMESTAMP values in different types .. IMPALA-8108: Impala query returns TIMESTAMP values in different types Impala uses function boost::posix_time::to_simple_string(time_duration) to convert timestamp to a string and to_simple_string() remove fractional seconds if they are all zeros. In order to output timestamp values in the same format, we add padding zeros if where isn't fractional seconds. Change-Id: Iad9a4659ff6d5353329e9d062886ee80296b5b43 --- M be/src/runtime/timestamp-value.cc A testdata/workloads/functional-query/queries/QueryTest/timestamps.test A tests/query_test/test_timestamps.py 3 files changed, 65 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/98/12498/3 -- To view, visit http://gerrit.cloudera.org:8080/12498 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iad9a4659ff6d5353329e9d062886ee80296b5b43 Gerrit-Change-Number: 12498 Gerrit-PatchSet: 3 Gerrit-Owner: Robbie Zhang Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8108: Impala query returns TIMESTAMP values in different types
Hello Greg Rahn, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12498 to look at the new patch set (#2). Change subject: IMPALA-8108: Impala query returns TIMESTAMP values in different types .. IMPALA-8108: Impala query returns TIMESTAMP values in different types Impala uses function boost::posix_time::to_simple_string(time_duration) to convert timestamp to a string and to_simple_string() remove fractional seconds if they are all zeros. In order to output timestamp values in the same format, we add padding zeros if where isn't fractional seconds. Change-Id: Iad9a4659ff6d5353329e9d062886ee80296b5b43 --- M be/src/runtime/timestamp-value.cc A testdata/workloads/functional-query/queries/QueryTest/timestamps.test A tests/query_test/test_timestamps.py 3 files changed, 64 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/98/12498/2 -- To view, visit http://gerrit.cloudera.org:8080/12498 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iad9a4659ff6d5353329e9d062886ee80296b5b43 Gerrit-Change-Number: 12498 Gerrit-PatchSet: 2 Gerrit-Owner: Robbie Zhang Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8108: Impala query returns TIMESTAMP values in different types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12498 ) Change subject: IMPALA-8108: Impala query returns TIMESTAMP values in different types .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/12498/2/tests/query_test/test_timestamps.py File tests/query_test/test_timestamps.py: http://gerrit.cloudera.org:8080/#/c/12498/2/tests/query_test/test_timestamps.py@20 PS2, Line 20: class TestTimestampFormats(ImpalaTestSuite): flake8: E302 expected 2 blank lines, found 1 -- To view, visit http://gerrit.cloudera.org:8080/12498 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iad9a4659ff6d5353329e9d062886ee80296b5b43 Gerrit-Change-Number: 12498 Gerrit-PatchSet: 2 Gerrit-Owner: Robbie Zhang Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 15 Feb 2019 08:21:15 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8108: Impala query returns TIMESTAMP values in different types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12498 ) Change subject: IMPALA-8108: Impala query returns TIMESTAMP values in different types .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/12498/1/tests/query_test/test_timestamps.py File tests/query_test/test_timestamps.py: http://gerrit.cloudera.org:8080/#/c/12498/1/tests/query_test/test_timestamps.py@18 PS1, Line 18: import pytest flake8: F401 'pytest' imported but unused http://gerrit.cloudera.org:8080/#/c/12498/1/tests/query_test/test_timestamps.py@21 PS1, Line 21: from tests.util.filesystem_utils import get_fs_path flake8: F401 'tests.util.filesystem_utils.get_fs_path' imported but unused http://gerrit.cloudera.org:8080/#/c/12498/1/tests/query_test/test_timestamps.py@23 PS1, Line 23: class TestTimestampFormats(ImpalaTestSuite): flake8: E302 expected 2 blank lines, found 1 -- To view, visit http://gerrit.cloudera.org:8080/12498 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iad9a4659ff6d5353329e9d062886ee80296b5b43 Gerrit-Change-Number: 12498 Gerrit-PatchSet: 1 Gerrit-Owner: Robbie Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 15 Feb 2019 08:08:30 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8108: Impala query returns TIMESTAMP values in different types
Robbie Zhang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12498 Change subject: IMPALA-8108: Impala query returns TIMESTAMP values in different types .. IMPALA-8108: Impala query returns TIMESTAMP values in different types Impala uses function boost::posix_time::to_simple_string(time_duration) to convert timestamp to a string and to_simple_string() remove fractional seconds if they are all zeros. In order to output timestamp values in the same format, we add padding zeros if where isn't fractional seconds. Change-Id: Iad9a4659ff6d5353329e9d062886ee80296b5b43 --- M be/src/runtime/timestamp-value.cc A testdata/workloads/functional-query/queries/QueryTest/timestamps.test A tests/query_test/test_timestamps.py 3 files changed, 66 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/98/12498/1 -- To view, visit http://gerrit.cloudera.org:8080/12498 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Iad9a4659ff6d5353329e9d062886ee80296b5b43 Gerrit-Change-Number: 12498 Gerrit-PatchSet: 1 Gerrit-Owner: Robbie Zhang
