[jira] [Commented] (HIVE-21456) Hive Metastore HTTP Thrift
[
https://issues.apache.org/jira/browse/HIVE-21456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794134#comment-16794134
]
Hive QA commented on HIVE-21456:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962672/HIVE-21456.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15833 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_groupby_reduce]
(batchId=61)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16538/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16538/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16538/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962672 - PreCommit-HIVE-Build
> Hive Metastore HTTP Thrift
> --
>
> Key: HIVE-21456
> URL: https://issues.apache.org/jira/browse/HIVE-21456
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Standalone Metastore
>Reporter: Amit Khanna
>Assignee: Amit Khanna
>Priority: Major
> Attachments: HIVE-21456.patch
>
>
> Hive Metastore currently doesn't have support for HTTP transport because of
> which it is not possible to access it via Knox. Adding support for Thrift
> over HTTP transport will allow the clients to access via Knox
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21456) Hive Metastore HTTP Thrift
[
https://issues.apache.org/jira/browse/HIVE-21456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794117#comment-16794117
]
Hive QA commented on HIVE-21456:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
14s{color} | {color:blue} standalone-metastore/metastore-server in master has
179 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 13 new + 0 unchanged - 0 fixed = 13 total (was 0) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 10 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16538/dev-support/hive-personality.sh
|
| git revision | master / 8d4300a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16538/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16538/yetus/whitespace-eol.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16538/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Hive Metastore HTTP Thrift
> --
>
> Key: HIVE-21456
> URL: https://issues.apache.org/jira/browse/HIVE-21456
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Standalone Metastore
>Reporter: Amit Khanna
>Assignee: Amit Khanna
>Priority: Major
> Attachments: HIVE-21456.patch
>
>
> Hive Metastore currently doesn't have support for HTTP transport because of
> which it is not possible to access it via Knox. Adding support for Thrift
> over HTTP transport will allow the clients to access via Knox
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21457) Perf optimizations in ORC split-generation
[
https://issues.apache.org/jira/browse/HIVE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794111#comment-16794111
]
Hive QA commented on HIVE-21457:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962675/HIVE-21457.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15833 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16537/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16537/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16537/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962675 - PreCommit-HIVE-Build
> Perf optimizations in ORC split-generation
> --
>
> Key: HIVE-21457
> URL: https://issues.apache.org/jira/browse/HIVE-21457
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Minor
> Attachments: HIVE-21457.1.patch
>
>
> Minor split generation optimizations
> * Reuse vectorization checks
> * Reuse isAcid checks
> * Reuse filesystem objects
> * Improved logging (log at top-level instead of inside the thread pool)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21399) Adjust hive.map.aggr.hash.min.reduction statically depending on group by statistics
[ https://issues.apache.org/jira/browse/HIVE-21399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21399: --- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks [~gopalv] > Adjust hive.map.aggr.hash.min.reduction statically depending on group by > statistics > --- > > Key: HIVE-21399 > URL: https://issues.apache.org/jira/browse/HIVE-21399 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21399.01.patch, HIVE-21399.02.patch, > HIVE-21399.03.patch, HIVE-21399.04.patch, HIVE-21399.05.patch, > HIVE-21399.06.patch, HIVE-21399.07.patch, HIVE-21399.08.patch > > > Currently, the value is set statically from config variable. If stats are > available, we could try to adjust this value at optimization time to favor > turning off hash aggregation earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21458) ACID: Optimize AcidUtils$MetaDataFile.isRawFormat
[
https://issues.apache.org/jira/browse/HIVE-21458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-21458:
Labels: Transactions-Performance (was: )
> ACID: Optimize AcidUtils$MetaDataFile.isRawFormat
> --
>
> Key: HIVE-21458
> URL: https://issues.apache.org/jira/browse/HIVE-21458
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.1
>Reporter: Vaibhav Gumashta
>Priority: Major
> Labels: Transactions-Performance
> Attachments: async-prof-pid-1-cpu-1.svg
>
>
> In the transactional subsystems, in several places we check to see if a data
> file has ROW__ID fields or not. Every time we do that (even within the
> context of the same query), we open a Reader for that file/split. We could
> optimize this by caching or perhaps checking once, and saving our result for
> later. Also, perhaps we don't need to do this for every split. An example
> call stack:
> {code}
> OrcFile.createReader(Path, OrcFile$ReaderOptions) line: 105
> AcidUtils$MetaDataFile.isRawFormatFile(Path, FileSystem) line: 2026
> AcidUtils$MetaDataFile.isRawFormat(Path, FileSystem) line: 2022
> AcidUtils.parsedDelta(Path, String, FileSystem) line: 1007
> OrcRawRecordMerger$TransactionMetaData.findWriteIDForSynthetcRowIDs(Path,
> Path, Configuration) line: 1231
> OrcRawRecordMerger.discoverOriginalKeyBounds(Reader, int, Reader$Options,
> Configuration, OrcRawRecordMerger$Options) line: 722
> OrcRawRecordMerger.(Configuration, boolean, Reader, boolean, int,
> ValidWriteIdList, Reader$Options, Path[], OrcRawRecordMerger$Options) line:
> 1022
> OrcInputFormat.getReader(InputSplit, Options) line: 2108
> OrcInputFormat.getRecordReader(InputSplit, JobConf, Reporter) line: 2006
> FetchOperator$FetchInputFormatSplit.getRecordReader(JobConf) line: 776
> FetchOperator.getRecordReader() line: 344
> FetchOperator.getNextRow() line: 540
> FetchOperator.pushRow() line: 509
> FetchTask.fetch(List) line: 146
> {code}
> Here, for each split we'll make that check.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21444) Additional tests for materialized view rewriting
[ https://issues.apache.org/jira/browse/HIVE-21444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21444: --- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master, thanks [~ashutoshc] > Additional tests for materialized view rewriting > > > Key: HIVE-21444 > URL: https://issues.apache.org/jira/browse/HIVE-21444 > Project: Hive > Issue Type: Test > Components: CBO, Materialized views >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21444.patch, HIVE-21444.patch, HIVE-21444.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (HIVE-21392) Misconfigurations of DataNucleus log in log4j.properties
[ https://issues.apache.org/jira/browse/HIVE-21392?focusedWorklogId=214163=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-214163 ] ASF GitHub Bot logged work on HIVE-21392: - Author: ASF GitHub Bot Created on: 16/Mar/19 03:02 Start Date: 16/Mar/19 03:02 Worklog Time Spent: 10m Work Description: rmsmani commented on issue #571: HIVE-21392 Fix misconfigurations of DataNucleus log in log4j.properties URL: https://github.com/apache/hive/pull/571#issuecomment-473493360 @coder-chenzhi If the test case is failing other than your test cases, it may be due to falky test cases. Only way to solve this resubmit the patch again. Add some test case for this change This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 214163) Time Spent: 1h 20m (was: 1h 10m) > Misconfigurations of DataNucleus log in log4j.properties > > > Key: HIVE-21392 > URL: https://issues.apache.org/jira/browse/HIVE-21392 > Project: Hive > Issue Type: Improvement > Components: Logging >Affects Versions: 2.0.0 >Reporter: Chen Zhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21392.02.patch, HIVE-21392.03.patch, > HIVE-21392.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > In the patch of > [HIVE-12020|https://issues.apache.org/jira/browse/HIVE-12020], we changed the > DataNucleus related logging configuration from nine fine-grained loggers with > three coarse-grained loggers (DataNucleus, Datastore and JPOX). As Prasanth > Jayachandran > [explain|https://issues.apache.org/jira/browse/HIVE-12020?focusedCommentId=15025612=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15025612], > these three loggers are the top-level logger in DataNucleus, so that we > don't need to specify other loggers for DataNucleus. However, according to > the > [documents|http://www.datanucleus.org/products/accessplatform/logging.html] > and [source > codes|https://github.com/datanucleus/datanucleus-core/blob/master/src/main/java/org/datanucleus/util/NucleusLogger.java#L108] > of DataNucleus, the top-level logger in DataNucleus is `DataNucleus`. > Therefore, we just need to keep the right one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21457) Perf optimizations in ORC split-generation
[
https://issues.apache.org/jira/browse/HIVE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794102#comment-16794102
]
Hive QA commented on HIVE-21457:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
14s{color} | {color:blue} ql in master has 2257 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 18 new + 505 unchanged - 4
fixed = 523 total (was 509) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16537/dev-support/hive-personality.sh
|
| git revision | master / 6a33d26 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16537/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16537/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Perf optimizations in ORC split-generation
> --
>
> Key: HIVE-21457
> URL: https://issues.apache.org/jira/browse/HIVE-21457
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Minor
> Attachments: HIVE-21457.1.patch
>
>
> Minor split generation optimizations
> * Reuse vectorization checks
> * Reuse isAcid checks
> * Reuse filesystem objects
> * Improved logging (log at top-level instead of inside the thread pool)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (HIVE-21392) Misconfigurations of DataNucleus log in log4j.properties
[ https://issues.apache.org/jira/browse/HIVE-21392?focusedWorklogId=214128=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-214128 ] ASF GitHub Bot logged work on HIVE-21392: - Author: ASF GitHub Bot Created on: 16/Mar/19 00:57 Start Date: 16/Mar/19 00:57 Worklog Time Spent: 10m Work Description: coder-chenzhi commented on issue #556: HIVE-21392 Fix misconfigurations of DataNucleus log in log4j.properties URL: https://github.com/apache/hive/pull/556#issuecomment-473483540 Hi, @rmsmani I have resolved the error in the patch and create another (PR)[https://github.com/apache/hive/pull/571]. The new test report in JIRA shows that a test case has failed, but I can't figure out why my patch will influence that test case. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 214128) Time Spent: 1h (was: 50m) > Misconfigurations of DataNucleus log in log4j.properties > > > Key: HIVE-21392 > URL: https://issues.apache.org/jira/browse/HIVE-21392 > Project: Hive > Issue Type: Improvement > Components: Logging >Affects Versions: 2.0.0 >Reporter: Chen Zhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21392.02.patch, HIVE-21392.03.patch, > HIVE-21392.patch > > Time Spent: 1h > Remaining Estimate: 0h > > In the patch of > [HIVE-12020|https://issues.apache.org/jira/browse/HIVE-12020], we changed the > DataNucleus related logging configuration from nine fine-grained loggers with > three coarse-grained loggers (DataNucleus, Datastore and JPOX). As Prasanth > Jayachandran > [explain|https://issues.apache.org/jira/browse/HIVE-12020?focusedCommentId=15025612=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15025612], > these three loggers are the top-level logger in DataNucleus, so that we > don't need to specify other loggers for DataNucleus. However, according to > the > [documents|http://www.datanucleus.org/products/accessplatform/logging.html] > and [source > codes|https://github.com/datanucleus/datanucleus-core/blob/master/src/main/java/org/datanucleus/util/NucleusLogger.java#L108] > of DataNucleus, the top-level logger in DataNucleus is `DataNucleus`. > Therefore, we just need to keep the right one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21444) Additional tests for materialized view rewriting
[ https://issues.apache.org/jira/browse/HIVE-21444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794099#comment-16794099 ] Ashutosh Chauhan commented on HIVE-21444: - +1 > Additional tests for materialized view rewriting > > > Key: HIVE-21444 > URL: https://issues.apache.org/jira/browse/HIVE-21444 > Project: Hive > Issue Type: Test > Components: CBO, Materialized views >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21444.patch, HIVE-21444.patch, HIVE-21444.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21399) Adjust hive.map.aggr.hash.min.reduction statically depending on group by statistics
[
https://issues.apache.org/jira/browse/HIVE-21399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794094#comment-16794094
]
Hive QA commented on HIVE-21399:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962668/HIVE-21399.08.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15833 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16536/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16536/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16536/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962668 - PreCommit-HIVE-Build
> Adjust hive.map.aggr.hash.min.reduction statically depending on group by
> statistics
> ---
>
> Key: HIVE-21399
> URL: https://issues.apache.org/jira/browse/HIVE-21399
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21399.01.patch, HIVE-21399.02.patch,
> HIVE-21399.03.patch, HIVE-21399.04.patch, HIVE-21399.05.patch,
> HIVE-21399.06.patch, HIVE-21399.07.patch, HIVE-21399.08.patch
>
>
> Currently, the value is set statically from config variable. If stats are
> available, we could try to adjust this value at optimization time to favor
> turning off hash aggregation earlier.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21399) Adjust hive.map.aggr.hash.min.reduction statically depending on group by statistics
[
https://issues.apache.org/jira/browse/HIVE-21399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794085#comment-16794085
]
Hive QA commented on HIVE-21399:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
8s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
12s{color} | {color:blue} ql in master has 2257 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} contrib in master has 10 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
55s{color} | {color:red} ql: The patch generated 5 new + 700 unchanged - 2
fixed = 705 total (was 702) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16536/dev-support/hive-personality.sh
|
| git revision | master / 6a33d26 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16536/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql accumulo-handler contrib hbase-handler
itests/hive-blobstore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16536/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Adjust hive.map.aggr.hash.min.reduction statically depending on group by
> statistics
> ---
>
> Key: HIVE-21399
> URL: https://issues.apache.org/jira/browse/HIVE-21399
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21399.01.patch,
[jira] [Commented] (HIVE-21458) ACID: Optimize AcidUtils$MetaDataFile.isRawFormat
[
https://issues.apache.org/jira/browse/HIVE-21458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794071#comment-16794071
]
Prasanth Jayachandran commented on HIVE-21458:
--
This profile shows how expensive isRawFormat check is.
> ACID: Optimize AcidUtils$MetaDataFile.isRawFormat
> --
>
> Key: HIVE-21458
> URL: https://issues.apache.org/jira/browse/HIVE-21458
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.1
>Reporter: Vaibhav Gumashta
>Priority: Major
> Attachments: async-prof-pid-1-cpu-1.svg
>
>
> In the transactional subsystems, in several places we check to see if a data
> file has ROW__ID fields or not. Every time we do that (even within the
> context of the same query), we open a Reader for that file/split. We could
> optimize this by caching or perhaps checking once, and saving our result for
> later. Also, perhaps we don't need to do this for every split. An example
> call stack:
> {code}
> OrcFile.createReader(Path, OrcFile$ReaderOptions) line: 105
> AcidUtils$MetaDataFile.isRawFormatFile(Path, FileSystem) line: 2026
> AcidUtils$MetaDataFile.isRawFormat(Path, FileSystem) line: 2022
> AcidUtils.parsedDelta(Path, String, FileSystem) line: 1007
> OrcRawRecordMerger$TransactionMetaData.findWriteIDForSynthetcRowIDs(Path,
> Path, Configuration) line: 1231
> OrcRawRecordMerger.discoverOriginalKeyBounds(Reader, int, Reader$Options,
> Configuration, OrcRawRecordMerger$Options) line: 722
> OrcRawRecordMerger.(Configuration, boolean, Reader, boolean, int,
> ValidWriteIdList, Reader$Options, Path[], OrcRawRecordMerger$Options) line:
> 1022
> OrcInputFormat.getReader(InputSplit, Options) line: 2108
> OrcInputFormat.getRecordReader(InputSplit, JobConf, Reporter) line: 2006
> FetchOperator$FetchInputFormatSplit.getRecordReader(JobConf) line: 776
> FetchOperator.getRecordReader() line: 344
> FetchOperator.getNextRow() line: 540
> FetchOperator.pushRow() line: 509
> FetchTask.fetch(List) line: 146
> {code}
> Here, for each split we'll make that check.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21458) ACID: Optimize AcidUtils$MetaDataFile.isRawFormat
[
https://issues.apache.org/jira/browse/HIVE-21458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran reassigned HIVE-21458:
Assignee: (was: Prasanth Jayachandran)
> ACID: Optimize AcidUtils$MetaDataFile.isRawFormat
> --
>
> Key: HIVE-21458
> URL: https://issues.apache.org/jira/browse/HIVE-21458
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.1
>Reporter: Vaibhav Gumashta
>Priority: Major
> Attachments: async-prof-pid-1-cpu-1.svg
>
>
> In the transactional subsystems, in several places we check to see if a data
> file has ROW__ID fields or not. Every time we do that (even within the
> context of the same query), we open a Reader for that file/split. We could
> optimize this by caching or perhaps checking once, and saving our result for
> later. Also, perhaps we don't need to do this for every split. An example
> call stack:
> {code}
> OrcFile.createReader(Path, OrcFile$ReaderOptions) line: 105
> AcidUtils$MetaDataFile.isRawFormatFile(Path, FileSystem) line: 2026
> AcidUtils$MetaDataFile.isRawFormat(Path, FileSystem) line: 2022
> AcidUtils.parsedDelta(Path, String, FileSystem) line: 1007
> OrcRawRecordMerger$TransactionMetaData.findWriteIDForSynthetcRowIDs(Path,
> Path, Configuration) line: 1231
> OrcRawRecordMerger.discoverOriginalKeyBounds(Reader, int, Reader$Options,
> Configuration, OrcRawRecordMerger$Options) line: 722
> OrcRawRecordMerger.(Configuration, boolean, Reader, boolean, int,
> ValidWriteIdList, Reader$Options, Path[], OrcRawRecordMerger$Options) line:
> 1022
> OrcInputFormat.getReader(InputSplit, Options) line: 2108
> OrcInputFormat.getRecordReader(InputSplit, JobConf, Reporter) line: 2006
> FetchOperator$FetchInputFormatSplit.getRecordReader(JobConf) line: 776
> FetchOperator.getRecordReader() line: 344
> FetchOperator.getNextRow() line: 540
> FetchOperator.pushRow() line: 509
> FetchTask.fetch(List) line: 146
> {code}
> Here, for each split we'll make that check.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21458) ACID: Optimize AcidUtils$MetaDataFile.isRawFormat
[
https://issues.apache.org/jira/browse/HIVE-21458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran reassigned HIVE-21458:
Assignee: Prasanth Jayachandran
> ACID: Optimize AcidUtils$MetaDataFile.isRawFormat
> --
>
> Key: HIVE-21458
> URL: https://issues.apache.org/jira/browse/HIVE-21458
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.1
>Reporter: Vaibhav Gumashta
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: async-prof-pid-1-cpu-1.svg
>
>
> In the transactional subsystems, in several places we check to see if a data
> file has ROW__ID fields or not. Every time we do that (even within the
> context of the same query), we open a Reader for that file/split. We could
> optimize this by caching or perhaps checking once, and saving our result for
> later. Also, perhaps we don't need to do this for every split. An example
> call stack:
> {code}
> OrcFile.createReader(Path, OrcFile$ReaderOptions) line: 105
> AcidUtils$MetaDataFile.isRawFormatFile(Path, FileSystem) line: 2026
> AcidUtils$MetaDataFile.isRawFormat(Path, FileSystem) line: 2022
> AcidUtils.parsedDelta(Path, String, FileSystem) line: 1007
> OrcRawRecordMerger$TransactionMetaData.findWriteIDForSynthetcRowIDs(Path,
> Path, Configuration) line: 1231
> OrcRawRecordMerger.discoverOriginalKeyBounds(Reader, int, Reader$Options,
> Configuration, OrcRawRecordMerger$Options) line: 722
> OrcRawRecordMerger.(Configuration, boolean, Reader, boolean, int,
> ValidWriteIdList, Reader$Options, Path[], OrcRawRecordMerger$Options) line:
> 1022
> OrcInputFormat.getReader(InputSplit, Options) line: 2108
> OrcInputFormat.getRecordReader(InputSplit, JobConf, Reporter) line: 2006
> FetchOperator$FetchInputFormatSplit.getRecordReader(JobConf) line: 776
> FetchOperator.getRecordReader() line: 344
> FetchOperator.getNextRow() line: 540
> FetchOperator.pushRow() line: 509
> FetchTask.fetch(List) line: 146
> {code}
> Here, for each split we'll make that check.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21458) ACID: Optimize AcidUtils$MetaDataFile.isRawFormat
[
https://issues.apache.org/jira/browse/HIVE-21458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-21458:
-
Attachment: async-prof-pid-1-cpu-1.svg
> ACID: Optimize AcidUtils$MetaDataFile.isRawFormat
> --
>
> Key: HIVE-21458
> URL: https://issues.apache.org/jira/browse/HIVE-21458
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.1
>Reporter: Vaibhav Gumashta
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: async-prof-pid-1-cpu-1.svg
>
>
> In the transactional subsystems, in several places we check to see if a data
> file has ROW__ID fields or not. Every time we do that (even within the
> context of the same query), we open a Reader for that file/split. We could
> optimize this by caching or perhaps checking once, and saving our result for
> later. Also, perhaps we don't need to do this for every split. An example
> call stack:
> {code}
> OrcFile.createReader(Path, OrcFile$ReaderOptions) line: 105
> AcidUtils$MetaDataFile.isRawFormatFile(Path, FileSystem) line: 2026
> AcidUtils$MetaDataFile.isRawFormat(Path, FileSystem) line: 2022
> AcidUtils.parsedDelta(Path, String, FileSystem) line: 1007
> OrcRawRecordMerger$TransactionMetaData.findWriteIDForSynthetcRowIDs(Path,
> Path, Configuration) line: 1231
> OrcRawRecordMerger.discoverOriginalKeyBounds(Reader, int, Reader$Options,
> Configuration, OrcRawRecordMerger$Options) line: 722
> OrcRawRecordMerger.(Configuration, boolean, Reader, boolean, int,
> ValidWriteIdList, Reader$Options, Path[], OrcRawRecordMerger$Options) line:
> 1022
> OrcInputFormat.getReader(InputSplit, Options) line: 2108
> OrcInputFormat.getRecordReader(InputSplit, JobConf, Reporter) line: 2006
> FetchOperator$FetchInputFormatSplit.getRecordReader(JobConf) line: 776
> FetchOperator.getRecordReader() line: 344
> FetchOperator.getNextRow() line: 540
> FetchOperator.pushRow() line: 509
> FetchTask.fetch(List) line: 146
> {code}
> Here, for each split we'll make that check.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21444) Additional tests for materialized view rewriting
[
https://issues.apache.org/jira/browse/HIVE-21444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794069#comment-16794069
]
Hive QA commented on HIVE-21444:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962666/HIVE-21444.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15833 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16535/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16535/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16535/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962666 - PreCommit-HIVE-Build
> Additional tests for materialized view rewriting
>
>
> Key: HIVE-21444
> URL: https://issues.apache.org/jira/browse/HIVE-21444
> Project: Hive
> Issue Type: Test
> Components: CBO, Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21444.patch, HIVE-21444.patch, HIVE-21444.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (HIVE-21392) Misconfigurations of DataNucleus log in log4j.properties
[ https://issues.apache.org/jira/browse/HIVE-21392?focusedWorklogId=214129=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-214129 ] ASF GitHub Bot logged work on HIVE-21392: - Author: ASF GitHub Bot Created on: 16/Mar/19 00:58 Start Date: 16/Mar/19 00:58 Worklog Time Spent: 10m Work Description: coder-chenzhi commented on issue #556: HIVE-21392 Fix misconfigurations of DataNucleus log in log4j.properties URL: https://github.com/apache/hive/pull/556#issuecomment-473483540 Hi, @rmsmani I have resolved the error in the patch and create another [PR](https://github.com/apache/hive/pull/571). The new test report in JIRA shows that a test case has failed, but I can't figure out why my patch will influence that test case. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 214129) Time Spent: 1h 10m (was: 1h) > Misconfigurations of DataNucleus log in log4j.properties > > > Key: HIVE-21392 > URL: https://issues.apache.org/jira/browse/HIVE-21392 > Project: Hive > Issue Type: Improvement > Components: Logging >Affects Versions: 2.0.0 >Reporter: Chen Zhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21392.02.patch, HIVE-21392.03.patch, > HIVE-21392.patch > > Time Spent: 1h 10m > Remaining Estimate: 0h > > In the patch of > [HIVE-12020|https://issues.apache.org/jira/browse/HIVE-12020], we changed the > DataNucleus related logging configuration from nine fine-grained loggers with > three coarse-grained loggers (DataNucleus, Datastore and JPOX). As Prasanth > Jayachandran > [explain|https://issues.apache.org/jira/browse/HIVE-12020?focusedCommentId=15025612=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15025612], > these three loggers are the top-level logger in DataNucleus, so that we > don't need to specify other loggers for DataNucleus. However, according to > the > [documents|http://www.datanucleus.org/products/accessplatform/logging.html] > and [source > codes|https://github.com/datanucleus/datanucleus-core/blob/master/src/main/java/org/datanucleus/util/NucleusLogger.java#L108] > of DataNucleus, the top-level logger in DataNucleus is `DataNucleus`. > Therefore, we just need to keep the right one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21369) LLAP: Logging is expensive in encoded reader path
[
https://issues.apache.org/jira/browse/HIVE-21369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-21369:
-
Attachment: HIVE-21369.2.patch
> LLAP: Logging is expensive in encoded reader path
> -
>
> Key: HIVE-21369
> URL: https://issues.apache.org/jira/browse/HIVE-21369
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Nita Dembla
>Priority: Major
> Attachments: HIVE-21369.2.patch, HIVE-21369.patch, HIVE-21369.patch.2
>
>
> There should be no INFO logging in EncodedReaderImpl. Stringifying of disk
> ranges is expensive in core read path.
> {code:java}
> 2019-03-01T17:55:56.322852142Z 2019-03-01T17:55:56,306 INFO
> [IO-Elevator-Thread-3
> (hive_20190301175546_a279f33c-4f2b-4cd5-8695-57bc8b042a61)]
> encoded.EncodedReaderImpl: Disk ranges after cache (found everything true;
> file [-3693547618692831801, 1551190876000, 1047660824], base offset
> 792920167): [{start: 887940 end: 1003508 cache buffer: 0x5165f83d(1)},
> {start: 1003508 end: 1119078 cache buffer: 0xb63cac3(1)}, {start: 1119078
> end: 1234745 cache buffer: 0x41a724fa(1)}, {start: 1234745 end: 1350261 cache
> buffer: 0x2f71bc38(1)}, {start: 1350261 end: 1465752 cache buffer:
> 0x2c38e1bb(1)}, {start: 1465752 end: 1581231 cache buffer: 0x5827982(1)},
> {start: 1581231 end: 1696885 cache buffer: 0x75a6773c(1)}, {start: 1696885
> end: 1812492 cache buffer: 0x2ed060f9(1)},{start: 1812492 end: 1928086 cache
> buffer: 0x20b2c8aa(1)}, {start: 1928086 end: 2043588 cache buffer:
> 0x6559aacb(1)}, {start: 2043588 end: 2159089 cache buffer: 0x569c85e1(1)},
> {start: 2159089 end: 2274725 cache buffer: 0x25a88dd0(1)}, {start: 2274725
> end: 2390228 cache buffer: 0x738b7e87(1)}, {start: 2390228 end: 2505715 cache
> buffer: 0x26edafa0(1)}, {start: 2505715 end: 2621322 cache buffer:
> 0x69db7752(1)}, {start: 2621322 end: 2736844 cache b{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (HIVE-21382) Group by keys reduction optimization - keys are not reduced in query23
[
https://issues.apache.org/jira/browse/HIVE-21382?focusedWorklogId=214123=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-214123
]
ASF GitHub Bot logged work on HIVE-21382:
-
Author: ASF GitHub Bot
Created on: 16/Mar/19 00:22
Start Date: 16/Mar/19 00:22
Worklog Time Spent: 10m
Work Description: asfgit commented on pull request #567: HIVE-21382:
Group by keys reduction optimization - keys are not reduced in query23
URL: https://github.com/apache/hive/pull/567
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 214123)
Time Spent: 3h 10m (was: 3h)
> Group by keys reduction optimization - keys are not reduced in query23
> --
>
> Key: HIVE-21382
> URL: https://issues.apache.org/jira/browse/HIVE-21382
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21382.1.patch, HIVE-21382.2.patch,
> HIVE-21382.2.patch, HIVE-21382.3.patch, HIVE-21382.4.patch,
> HIVE-21382.5.patch, HIVE-21382.6.patch
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> {code:sql}
> explain cbo with frequent_ss_items as
> (select substr(i_item_desc,1,30) itemdesc,i_item_sk item_sk,d_date
> solddate,count(*) cnt
> from store_sales
> ,date_dim
> ,item
> where ss_sold_date_sk = d_date_sk
> and ss_item_sk = i_item_sk
> and d_year in (1999,1999+1,1999+2,1999+3)
> group by substr(i_item_desc,1,30),i_item_sk,d_date
> having count(*) >4)
> select sum(sales)
> from ((select cs_quantity*cs_list_price sales
>from catalog_sales
>,date_dim
>where d_year = 1999
> and d_moy = 1
> and cs_sold_date_sk = d_date_sk
> and cs_item_sk in (select item_sk from frequent_ss_items))) subq
> limit 100;
> {code}
> {code:sql}
> HiveSortLimit(fetch=[100])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{}], agg#0=[sum($0)])
> HiveProject(sales=[*(CAST($2):DECIMAL(10, 0), $3)])
> HiveSemiJoin(condition=[=($1, $5)], joinType=[inner])
> HiveJoin(condition=[=($0, $4)], joinType=[inner], algorithm=[none],
> cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(cs_sold_date_sk=[$0], cs_item_sk=[$15],
> cs_quantity=[$18], cs_list_price=[$20])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints, catalog_sales]],
> table:alias=[catalog_sales])
> HiveProject(d_date_sk=[$0])
> HiveFilter(condition=[AND(=($6, 1999), =($8, 1))])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$1])
> HiveFilter(condition=[>($3, 4)])
> HiveProject(substr=[$2], i_item_sk=[$1], d_date=[$0], $f3=[$3])
> HiveAggregate(group=[{3, 4, 5}], agg#0=[count()])
> HiveJoin(condition=[=($1, $4)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveJoin(condition=[=($0, $2)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(ss_sold_date_sk=[$0], ss_item_sk=[$2])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints,
> store_sales]], table:alias=[store_sales])
> HiveProject(d_date_sk=[$0], d_date=[$2])
> HiveFilter(condition=[IN($6, 1999, 2000, 2001, 2002)])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$0], substr=[substr($4, 1, 30)])
> HiveTableScan(table=[[perf_constraints, item]],
> table:alias=[item])
> {code}
> Right side of HiveSemiJoin has an aggregate which could be reduce to have
> only {{i_item_sk}} as group by key since {{i_item_sk}} is primary key.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21457) Perf optimizations in ORC split-generation
[ https://issues.apache.org/jira/browse/HIVE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-21457: - Status: Patch Available (was: Open) > Perf optimizations in ORC split-generation > -- > > Key: HIVE-21457 > URL: https://issues.apache.org/jira/browse/HIVE-21457 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-21457.1.patch > > > Minor split generation optimizations > * Reuse vectorization checks > * Reuse isAcid checks > * Reuse filesystem objects > * Improved logging (log at top-level instead of inside the thread pool) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-16924) Support distinct in presence of Group By
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-16924:
--
Status: Open (was: Patch Available)
> Support distinct in presence of Group By
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch, HIVE-16924.07.patch, HIVE-16924.08.patch,
> HIVE-16924.09.patch, HIVE-16924.10.patch, HIVE-16924.11.patch,
> HIVE-16924.12.patch, HIVE-16924.13.patch, HIVE-16924.14.patch,
> HIVE-16924.15.patch, HIVE-16924.16.patch, HIVE-16924.17.patch,
> HIVE-16924.18.patch, HIVE-16924.19.patch, HIVE-16924.20.patch,
> HIVE-16924.21.patch, HIVE-16924.22.patch, HIVE-16924.23.patch,
> HIVE-16924.24.patch, HIVE-16924.25.patch
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21457) Perf optimizations in ORC split-generation
[ https://issues.apache.org/jira/browse/HIVE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-21457: - Attachment: HIVE-21457.1.patch > Perf optimizations in ORC split-generation > -- > > Key: HIVE-21457 > URL: https://issues.apache.org/jira/browse/HIVE-21457 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-21457.1.patch > > > Minor split generation optimizations > * Reuse vectorization checks > * Reuse isAcid checks > * Reuse filesystem objects > * Improved logging (log at top-level instead of inside the thread pool) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21457) Perf optimizations in ORC split-generation
[ https://issues.apache.org/jira/browse/HIVE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-21457: - Attachment: (was: HIVE-21457.1.patch) > Perf optimizations in ORC split-generation > -- > > Key: HIVE-21457 > URL: https://issues.apache.org/jira/browse/HIVE-21457 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-21457.1.patch > > > Minor split generation optimizations > * Reuse vectorization checks > * Reuse isAcid checks > * Reuse filesystem objects > * Improved logging (log at top-level instead of inside the thread pool) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21457) Perf optimizations in ORC split-generation
[ https://issues.apache.org/jira/browse/HIVE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-21457: - Attachment: (was: HIVE-21457.1.patch) > Perf optimizations in ORC split-generation > -- > > Key: HIVE-21457 > URL: https://issues.apache.org/jira/browse/HIVE-21457 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-21457.1.patch > > > Minor split generation optimizations > * Reuse vectorization checks > * Reuse isAcid checks > * Reuse filesystem objects > * Improved logging (log at top-level instead of inside the thread pool) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21457) Perf optimizations in ORC split-generation
[ https://issues.apache.org/jira/browse/HIVE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-21457: - Attachment: HIVE-21457.1.patch > Perf optimizations in ORC split-generation > -- > > Key: HIVE-21457 > URL: https://issues.apache.org/jira/browse/HIVE-21457 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-21457.1.patch > > > Minor split generation optimizations > * Reuse vectorization checks > * Reuse isAcid checks > * Reuse filesystem objects > * Improved logging (log at top-level instead of inside the thread pool) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-16924) Support distinct in presence of Group By
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-16924:
--
Attachment: HIVE-16924.26.patch
> Support distinct in presence of Group By
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch, HIVE-16924.07.patch, HIVE-16924.08.patch,
> HIVE-16924.09.patch, HIVE-16924.10.patch, HIVE-16924.11.patch,
> HIVE-16924.12.patch, HIVE-16924.13.patch, HIVE-16924.14.patch,
> HIVE-16924.15.patch, HIVE-16924.16.patch, HIVE-16924.17.patch,
> HIVE-16924.18.patch, HIVE-16924.19.patch, HIVE-16924.20.patch,
> HIVE-16924.21.patch, HIVE-16924.22.patch, HIVE-16924.23.patch,
> HIVE-16924.24.patch, HIVE-16924.25.patch, HIVE-16924.26.patch
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16924) Support distinct in presence of Group By
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-16924:
--
Status: Patch Available (was: Open)
> Support distinct in presence of Group By
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch, HIVE-16924.07.patch, HIVE-16924.08.patch,
> HIVE-16924.09.patch, HIVE-16924.10.patch, HIVE-16924.11.patch,
> HIVE-16924.12.patch, HIVE-16924.13.patch, HIVE-16924.14.patch,
> HIVE-16924.15.patch, HIVE-16924.16.patch, HIVE-16924.17.patch,
> HIVE-16924.18.patch, HIVE-16924.19.patch, HIVE-16924.20.patch,
> HIVE-16924.21.patch, HIVE-16924.22.patch, HIVE-16924.23.patch,
> HIVE-16924.24.patch, HIVE-16924.25.patch, HIVE-16924.26.patch
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21457) Perf optimizations in ORC split-generation
[ https://issues.apache.org/jira/browse/HIVE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794057#comment-16794057 ] Prasanth Jayachandran commented on HIVE-21457: -- [~gopalv] can you please take a look? > Perf optimizations in ORC split-generation > -- > > Key: HIVE-21457 > URL: https://issues.apache.org/jira/browse/HIVE-21457 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-21457.1.patch > > > Minor split generation optimizations > * Reuse vectorization checks > * Reuse isAcid checks > * Reuse filesystem objects > * Improved logging (log at top-level instead of inside the thread pool) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21456) Hive Metastore HTTP Thrift
[ https://issues.apache.org/jira/browse/HIVE-21456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Khanna updated HIVE-21456: --- Attachment: HIVE-21456.patch Status: Patch Available (was: Open) > Hive Metastore HTTP Thrift > -- > > Key: HIVE-21456 > URL: https://issues.apache.org/jira/browse/HIVE-21456 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore >Reporter: Amit Khanna >Assignee: Amit Khanna >Priority: Major > Attachments: HIVE-21456.patch > > > Hive Metastore currently doesn't have support for HTTP transport because of > which it is not possible to access it via Knox. Adding support for Thrift > over HTTP transport will allow the clients to access via Knox -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21382) Group by keys reduction optimization - keys are not reduced in query23
[
https://issues.apache.org/jira/browse/HIVE-21382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21382:
---
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
Pushed to master. Thanks for reviewing [~jcamachorodriguez]
> Group by keys reduction optimization - keys are not reduced in query23
> --
>
> Key: HIVE-21382
> URL: https://issues.apache.org/jira/browse/HIVE-21382
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21382.1.patch, HIVE-21382.2.patch,
> HIVE-21382.2.patch, HIVE-21382.3.patch, HIVE-21382.4.patch,
> HIVE-21382.5.patch, HIVE-21382.6.patch
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> {code:sql}
> explain cbo with frequent_ss_items as
> (select substr(i_item_desc,1,30) itemdesc,i_item_sk item_sk,d_date
> solddate,count(*) cnt
> from store_sales
> ,date_dim
> ,item
> where ss_sold_date_sk = d_date_sk
> and ss_item_sk = i_item_sk
> and d_year in (1999,1999+1,1999+2,1999+3)
> group by substr(i_item_desc,1,30),i_item_sk,d_date
> having count(*) >4)
> select sum(sales)
> from ((select cs_quantity*cs_list_price sales
>from catalog_sales
>,date_dim
>where d_year = 1999
> and d_moy = 1
> and cs_sold_date_sk = d_date_sk
> and cs_item_sk in (select item_sk from frequent_ss_items))) subq
> limit 100;
> {code}
> {code:sql}
> HiveSortLimit(fetch=[100])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{}], agg#0=[sum($0)])
> HiveProject(sales=[*(CAST($2):DECIMAL(10, 0), $3)])
> HiveSemiJoin(condition=[=($1, $5)], joinType=[inner])
> HiveJoin(condition=[=($0, $4)], joinType=[inner], algorithm=[none],
> cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(cs_sold_date_sk=[$0], cs_item_sk=[$15],
> cs_quantity=[$18], cs_list_price=[$20])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints, catalog_sales]],
> table:alias=[catalog_sales])
> HiveProject(d_date_sk=[$0])
> HiveFilter(condition=[AND(=($6, 1999), =($8, 1))])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$1])
> HiveFilter(condition=[>($3, 4)])
> HiveProject(substr=[$2], i_item_sk=[$1], d_date=[$0], $f3=[$3])
> HiveAggregate(group=[{3, 4, 5}], agg#0=[count()])
> HiveJoin(condition=[=($1, $4)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveJoin(condition=[=($0, $2)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(ss_sold_date_sk=[$0], ss_item_sk=[$2])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints,
> store_sales]], table:alias=[store_sales])
> HiveProject(d_date_sk=[$0], d_date=[$2])
> HiveFilter(condition=[IN($6, 1999, 2000, 2001, 2002)])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$0], substr=[substr($4, 1, 30)])
> HiveTableScan(table=[[perf_constraints, item]],
> table:alias=[item])
> {code}
> Right side of HiveSemiJoin has an aggregate which could be reduce to have
> only {{i_item_sk}} as group by key since {{i_item_sk}} is primary key.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21399) Adjust hive.map.aggr.hash.min.reduction statically depending on group by statistics
[
https://issues.apache.org/jira/browse/HIVE-21399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794058#comment-16794058
]
Gopal V commented on HIVE-21399:
The patch LGTM - +1
A significant number of them are 0.0 or close to
{code}
$ grep "minRed.*:" HIVE-21399.08.patch | grep -v 0.99 | sed "s/^+ *//" |sort |
uniq -c
2198 minReductionHashAggr: 0.0
2 minReductionHashAggr: 0.0077519417
3 minReductionHashAggr: 0.05582
10 minReductionHashAggr: 0.1669
1 minReductionHashAggr: 0.278
34 minReductionHashAggr: 0.333
1 minReductionHashAggr: 0.46745563
5 minReductionHashAggr: 0.5
1 minReductionHashAggr: 0.536193
1 minReductionHashAggr: 0.5666
1 minReductionHashAggr: 0.5879121
4 minReductionHashAggr: 0.666
1 minReductionHashAggr: 0.76018816
{code}
I'll look through all the cases in some more detail - for example, the
{{0.0077519417}} value seems to be related to a compute-stats query.
> Adjust hive.map.aggr.hash.min.reduction statically depending on group by
> statistics
> ---
>
> Key: HIVE-21399
> URL: https://issues.apache.org/jira/browse/HIVE-21399
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21399.01.patch, HIVE-21399.02.patch,
> HIVE-21399.03.patch, HIVE-21399.04.patch, HIVE-21399.05.patch,
> HIVE-21399.06.patch, HIVE-21399.07.patch, HIVE-21399.08.patch
>
>
> Currently, the value is set statically from config variable. If stats are
> available, we could try to adjust this value at optimization time to favor
> turning off hash aggregation earlier.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21457) Perf optimizations in ORC split-generation
[ https://issues.apache.org/jira/browse/HIVE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-21457: - Attachment: HIVE-21457.1.patch > Perf optimizations in ORC split-generation > -- > > Key: HIVE-21457 > URL: https://issues.apache.org/jira/browse/HIVE-21457 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-21457.1.patch > > > Minor split generation optimizations > * Reuse vectorization checks > * Reuse isAcid checks > * Reuse filesystem objects > * Improved logging (log at top-level instead of inside the thread pool) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21399) Adjust hive.map.aggr.hash.min.reduction statically depending on group by statistics
[ https://issues.apache.org/jira/browse/HIVE-21399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21399: --- Attachment: HIVE-21399.08.patch > Adjust hive.map.aggr.hash.min.reduction statically depending on group by > statistics > --- > > Key: HIVE-21399 > URL: https://issues.apache.org/jira/browse/HIVE-21399 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21399.01.patch, HIVE-21399.02.patch, > HIVE-21399.03.patch, HIVE-21399.04.patch, HIVE-21399.05.patch, > HIVE-21399.06.patch, HIVE-21399.07.patch, HIVE-21399.08.patch > > > Currently, the value is set statically from config variable. If stats are > available, we could try to adjust this value at optimization time to favor > turning off hash aggregation earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21399) Adjust hive.map.aggr.hash.min.reduction statically depending on group by statistics
[ https://issues.apache.org/jira/browse/HIVE-21399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794053#comment-16794053 ] Jesus Camacho Rodriguez commented on HIVE-21399: [~ashutoshc], [~gopalv], could you take a look? Thanks > Adjust hive.map.aggr.hash.min.reduction statically depending on group by > statistics > --- > > Key: HIVE-21399 > URL: https://issues.apache.org/jira/browse/HIVE-21399 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21399.01.patch, HIVE-21399.02.patch, > HIVE-21399.03.patch, HIVE-21399.04.patch, HIVE-21399.05.patch, > HIVE-21399.06.patch, HIVE-21399.07.patch, HIVE-21399.08.patch > > > Currently, the value is set statically from config variable. If stats are > available, we could try to adjust this value at optimization time to favor > turning off hash aggregation earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17357) Plugin jars are not properly added for LocalHiveSparkClient
[
https://issues.apache.org/jira/browse/HIVE-17357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794046#comment-16794046
]
t oo commented on HIVE-17357:
-
can you clarify this issue? does it mean before the fix there was no way for
HiveonSpark (with spark.master in non local mode) to use custom
serdes/udfs?[https://cwiki.apache.org//confluence/display/Hive/Hive+on+Spark:+Getting+Started]
does not mention how to register custom serde/udf jars/classes. For example if
i want to query a {{'com.uber.hoodie.hadoop.HoodieInputFormat' (this class
relies on parquet) table the docs dont say where to place the jar.}}
> Plugin jars are not properly added for LocalHiveSparkClient
> ---
>
> Key: HIVE-17357
> URL: https://issues.apache.org/jira/browse/HIVE-17357
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 3.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-17357.1.patch
>
>
> I forgot to include the same change for LocalHiveSparkClient.java in
> HIVE-17336. We need to make the same change as HIVE-17336 in
> LocalHiveSparkClient class to include plugin jars. Maybe we should have a
> common base class for both LocalHiveSparkClient and RemoteHiveSparkClient to
> have some common functions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21444) Additional tests for materialized view rewriting
[
https://issues.apache.org/jira/browse/HIVE-21444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794047#comment-16794047
]
Hive QA commented on HIVE-21444:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
54s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16535/dev-support/hive-personality.sh
|
| git revision | master / 8c44806 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16535/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Additional tests for materialized view rewriting
>
>
> Key: HIVE-21444
> URL: https://issues.apache.org/jira/browse/HIVE-21444
> Project: Hive
> Issue Type: Test
> Components: CBO, Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21444.patch, HIVE-21444.patch, HIVE-21444.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17336) Missing class 'org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat' from Hive on Spark when inserting into hbase based table
[
https://issues.apache.org/jira/browse/HIVE-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794045#comment-16794045
]
t oo commented on HIVE-17336:
-
can you clarify this issue? does it mean before the fix there was no way for
HiveonSpark (with spark.master in non local mode) to use custom
serdes/udfs?[https://cwiki.apache.org//confluence/display/Hive/Hive+on+Spark:+Getting+Started]
does not mention how to register custom serde/udf jars/classes. For example if
i want to query a {{'com.uber.hoodie.hadoop.HoodieInputFormat' (this class
relies on parquet) table the docs dont say where to place the jar.}}
> Missing class 'org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat' from
> Hive on Spark when inserting into hbase based table
> ---
>
> Key: HIVE-17336
> URL: https://issues.apache.org/jira/browse/HIVE-17336
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 3.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-17336.1.patch
>
>
> When inserting into a hbase based table from hive on spark, the following
> exception is thrown
> {noformat}
> Error while processing statement: FAILED: Execution Error, return code 3 from
> org.apache.hadoop.hive.ql.exec.spark.SparkTask.
> org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find
> class: org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat
> Serialization trace:
> inputFileFormatClass (org.apache.hadoop.hive.ql.plan.TableDesc)
> tableInfo (org.apache.hadoop.hive.ql.plan.FileSinkDesc)
> conf (org.apache.hadoop.hive.ql.exec.FileSinkOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> invertedWorkGraph (org.apache.hadoop.hive.ql.plan.SparkWork)
> at
> org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:156)
> at
> org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670)
> at
> org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClass(SerializationUtilities.java:183)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.DefaultSerializers$ClassSerializer.read(DefaultSerializers.java:326)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.DefaultSerializers$ClassSerializer.read(DefaultSerializers.java:314)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:759)
> at
> org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObjectOrNull(SerializationUtilities.java:201)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:708)
> at
> org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:216)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:708)
> at
> org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:216)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:790)
> at
> org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClassAndObject(SerializationUtilities.java:178)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:708)
> at
> org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:216)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> at
> org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
>
[jira] [Commented] (HIVE-21444) Additional tests for materialized view rewriting
[ https://issues.apache.org/jira/browse/HIVE-21444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794039#comment-16794039 ] Jesus Camacho Rodriguez commented on HIVE-21444: [~vgarg], could you take a look? It is just two additional tests. Thanks > Additional tests for materialized view rewriting > > > Key: HIVE-21444 > URL: https://issues.apache.org/jira/browse/HIVE-21444 > Project: Hive > Issue Type: Test > Components: CBO, Materialized views >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21444.patch, HIVE-21444.patch, HIVE-21444.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21444) Additional tests for materialized view rewriting
[ https://issues.apache.org/jira/browse/HIVE-21444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21444: --- Attachment: HIVE-21444.patch > Additional tests for materialized view rewriting > > > Key: HIVE-21444 > URL: https://issues.apache.org/jira/browse/HIVE-21444 > Project: Hive > Issue Type: Test > Components: CBO, Materialized views >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21444.patch, HIVE-21444.patch, HIVE-21444.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21385) Allow disabling pushdown of non-splittable computation to JDBC sources
[ https://issues.apache.org/jira/browse/HIVE-21385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21385: --- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master. > Allow disabling pushdown of non-splittable computation to JDBC sources > -- > > Key: HIVE-21385 > URL: https://issues.apache.org/jira/browse/HIVE-21385 > Project: Hive > Issue Type: Improvement > Components: CBO, StorageHandler >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21385.01.patch, HIVE-21385.01.patch, > HIVE-21385.02.patch, HIVE-21385.02.patch, HIVE-21385.patch > > > Until pushdown is cost-based decision, we will be able to enable / disable > pushdown of operators that prevent reading results from the JDBC connection > in parallel. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21445) Support range check for DECIMAL type in stats annotation
[ https://issues.apache.org/jira/browse/HIVE-21445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21445: --- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master. > Support range check for DECIMAL type in stats annotation > > > Key: HIVE-21445 > URL: https://issues.apache.org/jira/browse/HIVE-21445 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer, Statistics >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21445.01.patch, HIVE-21445.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-16924) Support distinct in presence of Group By
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794022#comment-16794022
]
Hive QA commented on HIVE-16924:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
53s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
15s{color} | {color:blue} ql in master has 2257 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 8 new + 639 unchanged - 13
fixed = 647 total (was 652) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
7s{color} | {color:red} root: The patch generated 8 new + 647 unchanged - 13
fixed = 655 total (was 660) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
28s{color} | {color:green} ql generated 0 new + 2255 unchanged - 2 fixed = 2255
total (was 2257) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 66m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16534/dev-support/hive-personality.sh
|
| git revision | master / 677eef8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16534/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16534/yetus/diff-checkstyle-root.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16534/yetus/whitespace-eol.txt
|
| modules | C: ql . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16534/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support distinct in presence of Group By
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch, HIVE-16924.07.patch, HIVE-16924.08.patch,
> HIVE-16924.09.patch, HIVE-16924.10.patch, HIVE-16924.11.patch,
> HIVE-16924.12.patch,
[jira] [Commented] (HIVE-21385) Allow disabling pushdown of non-splittable computation to JDBC sources
[ https://issues.apache.org/jira/browse/HIVE-21385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794026#comment-16794026 ] Ashutosh Chauhan commented on HIVE-21385: - +1 > Allow disabling pushdown of non-splittable computation to JDBC sources > -- > > Key: HIVE-21385 > URL: https://issues.apache.org/jira/browse/HIVE-21385 > Project: Hive > Issue Type: Improvement > Components: CBO, StorageHandler >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21385.01.patch, HIVE-21385.01.patch, > HIVE-21385.02.patch, HIVE-21385.02.patch, HIVE-21385.patch > > > Until pushdown is cost-based decision, we will be able to enable / disable > pushdown of operators that prevent reading results from the JDBC connection > in parallel. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-16924) Support distinct in presence of Group By
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794021#comment-16794021
]
Hive QA commented on HIVE-16924:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962650/HIVE-16924.25.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15832 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[vector_outer_join3]
(batchId=191)
org.apache.hive.hcatalog.mapreduce.TestHCatMutableNonPartitioned.testHCatNonPartitionedTable[6]
(batchId=214)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16534/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16534/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16534/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962650 - PreCommit-HIVE-Build
> Support distinct in presence of Group By
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch, HIVE-16924.07.patch, HIVE-16924.08.patch,
> HIVE-16924.09.patch, HIVE-16924.10.patch, HIVE-16924.11.patch,
> HIVE-16924.12.patch, HIVE-16924.13.patch, HIVE-16924.14.patch,
> HIVE-16924.15.patch, HIVE-16924.16.patch, HIVE-16924.17.patch,
> HIVE-16924.18.patch, HIVE-16924.19.patch, HIVE-16924.20.patch,
> HIVE-16924.21.patch, HIVE-16924.22.patch, HIVE-16924.23.patch,
> HIVE-16924.24.patch, HIVE-16924.25.patch
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21458) ACID: Optimize AcidUtils$MetaDataFile.isRawFormat
[
https://issues.apache.org/jira/browse/HIVE-21458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-21458:
Description:
In the transactional subsystems, in several places we check to see if a data
file has ROW__ID fields or not. Every time we do that (even within the context
of the same query), we open a Reader for that file/split. We could optimize
this by caching or perhaps checking once, and saving our result for later.
Also, perhaps we don't need to do this for every split. An example call stack:
{code}
OrcFile.createReader(Path, OrcFile$ReaderOptions) line: 105
AcidUtils$MetaDataFile.isRawFormatFile(Path, FileSystem) line: 2026
AcidUtils$MetaDataFile.isRawFormat(Path, FileSystem) line: 2022
AcidUtils.parsedDelta(Path, String, FileSystem) line: 1007
OrcRawRecordMerger$TransactionMetaData.findWriteIDForSynthetcRowIDs(Path, Path,
Configuration) line: 1231
OrcRawRecordMerger.discoverOriginalKeyBounds(Reader, int, Reader$Options,
Configuration, OrcRawRecordMerger$Options) line: 722
OrcRawRecordMerger.(Configuration, boolean, Reader, boolean, int,
ValidWriteIdList, Reader$Options, Path[], OrcRawRecordMerger$Options) line:
1022
OrcInputFormat.getReader(InputSplit, Options) line: 2108
OrcInputFormat.getRecordReader(InputSplit, JobConf, Reporter) line: 2006
FetchOperator$FetchInputFormatSplit.getRecordReader(JobConf) line: 776
FetchOperator.getRecordReader() line: 344
FetchOperator.getNextRow() line: 540
FetchOperator.pushRow() line: 509
FetchTask.fetch(List) line: 146
{code}
Here, for each split we'll make that check.
was:
In the transactional subsystems, in several places we check to see if a data
file has ROW__ID fields or not. Every time we do that (even within the context
of the same query), we open a Reader for that file/split. We could optimize
this by caching. Also, perhaps we don't need to do this for every split. An
example call stack:
{code}
OrcFile.createReader(Path, OrcFile$ReaderOptions) line: 105
AcidUtils$MetaDataFile.isRawFormatFile(Path, FileSystem) line: 2026
AcidUtils$MetaDataFile.isRawFormat(Path, FileSystem) line: 2022
AcidUtils.parsedDelta(Path, String, FileSystem) line: 1007
OrcRawRecordMerger$TransactionMetaData.findWriteIDForSynthetcRowIDs(Path, Path,
Configuration) line: 1231
OrcRawRecordMerger.discoverOriginalKeyBounds(Reader, int, Reader$Options,
Configuration, OrcRawRecordMerger$Options) line: 722
OrcRawRecordMerger.(Configuration, boolean, Reader, boolean, int,
ValidWriteIdList, Reader$Options, Path[], OrcRawRecordMerger$Options) line:
1022
OrcInputFormat.getReader(InputSplit, Options) line: 2108
OrcInputFormat.getRecordReader(InputSplit, JobConf, Reporter) line: 2006
FetchOperator$FetchInputFormatSplit.getRecordReader(JobConf) line: 776
FetchOperator.getRecordReader() line: 344
FetchOperator.getNextRow() line: 540
FetchOperator.pushRow() line: 509
FetchTask.fetch(List) line: 146
{code}
Here, for each split we'll make that check.
> ACID: Optimize AcidUtils$MetaDataFile.isRawFormat
> --
>
> Key: HIVE-21458
> URL: https://issues.apache.org/jira/browse/HIVE-21458
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.1
>Reporter: Vaibhav Gumashta
>Priority: Major
>
> In the transactional subsystems, in several places we check to see if a data
> file has ROW__ID fields or not. Every time we do that (even within the
> context of the same query), we open a Reader for that file/split. We could
> optimize this by caching or perhaps checking once, and saving our result for
> later. Also, perhaps we don't need to do this for every split. An example
> call stack:
> {code}
> OrcFile.createReader(Path, OrcFile$ReaderOptions) line: 105
> AcidUtils$MetaDataFile.isRawFormatFile(Path, FileSystem) line: 2026
> AcidUtils$MetaDataFile.isRawFormat(Path, FileSystem) line: 2022
> AcidUtils.parsedDelta(Path, String, FileSystem) line: 1007
> OrcRawRecordMerger$TransactionMetaData.findWriteIDForSynthetcRowIDs(Path,
> Path, Configuration) line: 1231
> OrcRawRecordMerger.discoverOriginalKeyBounds(Reader, int, Reader$Options,
> Configuration, OrcRawRecordMerger$Options) line: 722
> OrcRawRecordMerger.(Configuration, boolean, Reader, boolean, int,
> ValidWriteIdList, Reader$Options, Path[], OrcRawRecordMerger$Options) line:
> 1022
> OrcInputFormat.getReader(InputSplit, Options) line: 2108
> OrcInputFormat.getRecordReader(InputSplit, JobConf, Reporter) line: 2006
> FetchOperator$FetchInputFormatSplit.getRecordReader(JobConf) line: 776
> FetchOperator.getRecordReader() line: 344
> FetchOperator.getNextRow() line:
[jira] [Updated] (HIVE-21458) ACID: Optimize AcidUtils$MetaDataFile.isRawFormat
[
https://issues.apache.org/jira/browse/HIVE-21458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-21458:
Summary: ACID: Optimize AcidUtils$MetaDataFile.isRawFormat (was: ACID:
Optimize AcidUtils$MetaDataFile.isRawFormat check by caching the split reader)
> ACID: Optimize AcidUtils$MetaDataFile.isRawFormat
> --
>
> Key: HIVE-21458
> URL: https://issues.apache.org/jira/browse/HIVE-21458
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.1
>Reporter: Vaibhav Gumashta
>Priority: Major
>
> In the transactional subsystems, in several places we check to see if a data
> file has ROW__ID fields or not. Every time we do that (even within the
> context of the same query), we open a Reader for that file/split. We could
> optimize this by caching. Also, perhaps we don't need to do this for every
> split. An example call stack:
> {code}
> OrcFile.createReader(Path, OrcFile$ReaderOptions) line: 105
> AcidUtils$MetaDataFile.isRawFormatFile(Path, FileSystem) line: 2026
> AcidUtils$MetaDataFile.isRawFormat(Path, FileSystem) line: 2022
> AcidUtils.parsedDelta(Path, String, FileSystem) line: 1007
> OrcRawRecordMerger$TransactionMetaData.findWriteIDForSynthetcRowIDs(Path,
> Path, Configuration) line: 1231
> OrcRawRecordMerger.discoverOriginalKeyBounds(Reader, int, Reader$Options,
> Configuration, OrcRawRecordMerger$Options) line: 722
> OrcRawRecordMerger.(Configuration, boolean, Reader, boolean, int,
> ValidWriteIdList, Reader$Options, Path[], OrcRawRecordMerger$Options) line:
> 1022
> OrcInputFormat.getReader(InputSplit, Options) line: 2108
> OrcInputFormat.getRecordReader(InputSplit, JobConf, Reporter) line: 2006
> FetchOperator$FetchInputFormatSplit.getRecordReader(JobConf) line: 776
> FetchOperator.getRecordReader() line: 344
> FetchOperator.getNextRow() line: 540
> FetchOperator.pushRow() line: 509
> FetchTask.fetch(List) line: 146
> {code}
> Here, for each split we'll make that check.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21382) Group by keys reduction optimization - keys are not reduced in query23
[
https://issues.apache.org/jira/browse/HIVE-21382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793983#comment-16793983
]
Hive QA commented on HIVE-21382:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962639/HIVE-21382.6.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15832 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16533/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16533/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16533/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962639 - PreCommit-HIVE-Build
> Group by keys reduction optimization - keys are not reduced in query23
> --
>
> Key: HIVE-21382
> URL: https://issues.apache.org/jira/browse/HIVE-21382
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21382.1.patch, HIVE-21382.2.patch,
> HIVE-21382.2.patch, HIVE-21382.3.patch, HIVE-21382.4.patch,
> HIVE-21382.5.patch, HIVE-21382.6.patch
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> {code:sql}
> explain cbo with frequent_ss_items as
> (select substr(i_item_desc,1,30) itemdesc,i_item_sk item_sk,d_date
> solddate,count(*) cnt
> from store_sales
> ,date_dim
> ,item
> where ss_sold_date_sk = d_date_sk
> and ss_item_sk = i_item_sk
> and d_year in (1999,1999+1,1999+2,1999+3)
> group by substr(i_item_desc,1,30),i_item_sk,d_date
> having count(*) >4)
> select sum(sales)
> from ((select cs_quantity*cs_list_price sales
>from catalog_sales
>,date_dim
>where d_year = 1999
> and d_moy = 1
> and cs_sold_date_sk = d_date_sk
> and cs_item_sk in (select item_sk from frequent_ss_items))) subq
> limit 100;
> {code}
> {code:sql}
> HiveSortLimit(fetch=[100])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{}], agg#0=[sum($0)])
> HiveProject(sales=[*(CAST($2):DECIMAL(10, 0), $3)])
> HiveSemiJoin(condition=[=($1, $5)], joinType=[inner])
> HiveJoin(condition=[=($0, $4)], joinType=[inner], algorithm=[none],
> cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(cs_sold_date_sk=[$0], cs_item_sk=[$15],
> cs_quantity=[$18], cs_list_price=[$20])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints, catalog_sales]],
> table:alias=[catalog_sales])
> HiveProject(d_date_sk=[$0])
> HiveFilter(condition=[AND(=($6, 1999), =($8, 1))])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$1])
> HiveFilter(condition=[>($3, 4)])
> HiveProject(substr=[$2], i_item_sk=[$1], d_date=[$0], $f3=[$3])
> HiveAggregate(group=[{3, 4, 5}], agg#0=[count()])
> HiveJoin(condition=[=($1, $4)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveJoin(condition=[=($0, $2)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(ss_sold_date_sk=[$0], ss_item_sk=[$2])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints,
> store_sales]], table:alias=[store_sales])
> HiveProject(d_date_sk=[$0], d_date=[$2])
> HiveFilter(condition=[IN($6, 1999, 2000, 2001, 2002)])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$0], substr=[substr($4, 1, 30)])
> HiveTableScan(table=[[perf_constraints, item]],
> table:alias=[item])
> {code}
> Right side of HiveSemiJoin has an aggregate which could be reduce to have
> only {{i_item_sk}} as group by key since {{i_item_sk}} is primary key.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21382) Group by keys reduction optimization - keys are not reduced in query23
[
https://issues.apache.org/jira/browse/HIVE-21382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793952#comment-16793952
]
Hive QA commented on HIVE-21382:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
20s{color} | {color:blue} ql in master has 2257 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16533/dev-support/hive-personality.sh
|
| git revision | master / 677eef8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16533/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Group by keys reduction optimization - keys are not reduced in query23
> --
>
> Key: HIVE-21382
> URL: https://issues.apache.org/jira/browse/HIVE-21382
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21382.1.patch, HIVE-21382.2.patch,
> HIVE-21382.2.patch, HIVE-21382.3.patch, HIVE-21382.4.patch,
> HIVE-21382.5.patch, HIVE-21382.6.patch
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> {code:sql}
> explain cbo with frequent_ss_items as
> (select substr(i_item_desc,1,30) itemdesc,i_item_sk item_sk,d_date
> solddate,count(*) cnt
> from store_sales
> ,date_dim
> ,item
> where ss_sold_date_sk = d_date_sk
> and ss_item_sk = i_item_sk
> and d_year in (1999,1999+1,1999+2,1999+3)
> group by substr(i_item_desc,1,30),i_item_sk,d_date
> having count(*) >4)
> select sum(sales)
> from ((select cs_quantity*cs_list_price sales
>from catalog_sales
>,date_dim
>where d_year = 1999
> and d_moy = 1
> and cs_sold_date_sk = d_date_sk
> and cs_item_sk in (select item_sk from frequent_ss_items))) subq
> limit 100;
> {code}
> {code:sql}
> HiveSortLimit(fetch=[100])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{}], agg#0=[sum($0)])
> HiveProject(sales=[*(CAST($2):DECIMAL(10, 0), $3)])
> HiveSemiJoin(condition=[=($1, $5)], joinType=[inner])
> HiveJoin(condition=[=($0, $4)], joinType=[inner],
[jira] [Commented] (HIVE-21369) LLAP: Logging is expensive in encoded reader path
[
https://issues.apache.org/jira/browse/HIVE-21369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793939#comment-16793939
]
Hive QA commented on HIVE-21369:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962634/HIVE-21369.patch.2
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15817 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
[intersect_all.q,unionDistinct_1.q,table_nonprintable.q,orc_llap_counters1.q,mm_cttas.q,whroot_external1.q,global_limit.q,cte_2.q,rcfile_createas1.q,dynamic_partition_pruning_2.q,intersect_merge.q,results_cache_diff_fs.q,cttl.q,parallel_colstats.q,load_hdfs_file_with_space_in_the_name.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16532/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16532/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16532/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962634 - PreCommit-HIVE-Build
> LLAP: Logging is expensive in encoded reader path
> -
>
> Key: HIVE-21369
> URL: https://issues.apache.org/jira/browse/HIVE-21369
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Nita Dembla
>Priority: Major
> Attachments: HIVE-21369.patch, HIVE-21369.patch.2
>
>
> There should be no INFO logging in EncodedReaderImpl. Stringifying of disk
> ranges is expensive in core read path.
> {code:java}
> 2019-03-01T17:55:56.322852142Z 2019-03-01T17:55:56,306 INFO
> [IO-Elevator-Thread-3
> (hive_20190301175546_a279f33c-4f2b-4cd5-8695-57bc8b042a61)]
> encoded.EncodedReaderImpl: Disk ranges after cache (found everything true;
> file [-3693547618692831801, 1551190876000, 1047660824], base offset
> 792920167): [{start: 887940 end: 1003508 cache buffer: 0x5165f83d(1)},
> {start: 1003508 end: 1119078 cache buffer: 0xb63cac3(1)}, {start: 1119078
> end: 1234745 cache buffer: 0x41a724fa(1)}, {start: 1234745 end: 1350261 cache
> buffer: 0x2f71bc38(1)}, {start: 1350261 end: 1465752 cache buffer:
> 0x2c38e1bb(1)}, {start: 1465752 end: 1581231 cache buffer: 0x5827982(1)},
> {start: 1581231 end: 1696885 cache buffer: 0x75a6773c(1)}, {start: 1696885
> end: 1812492 cache buffer: 0x2ed060f9(1)},{start: 1812492 end: 1928086 cache
> buffer: 0x20b2c8aa(1)}, {start: 1928086 end: 2043588 cache buffer:
> 0x6559aacb(1)}, {start: 2043588 end: 2159089 cache buffer: 0x569c85e1(1)},
> {start: 2159089 end: 2274725 cache buffer: 0x25a88dd0(1)}, {start: 2274725
> end: 2390228 cache buffer: 0x738b7e87(1)}, {start: 2390228 end: 2505715 cache
> buffer: 0x26edafa0(1)}, {start: 2505715 end: 2621322 cache buffer:
> 0x69db7752(1)}, {start: 2621322 end: 2736844 cache b{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21001) Upgrade to calcite-1.19
[ https://issues.apache.org/jira/browse/HIVE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21001: --- Summary: Upgrade to calcite-1.19 (was: Upgrade to calcite-1.18) > Upgrade to calcite-1.19 > --- > > Key: HIVE-21001 > URL: https://issues.apache.org/jira/browse/HIVE-21001 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21001.01.patch, HIVE-21001.01.patch, > HIVE-21001.02.patch, HIVE-21001.03.patch, HIVE-21001.04.patch, > HIVE-21001.05.patch, HIVE-21001.06.patch, HIVE-21001.06.patch, > HIVE-21001.07.patch, HIVE-21001.08.patch, HIVE-21001.08.patch, > HIVE-21001.08.patch, HIVE-21001.09.patch, HIVE-21001.09.patch, > HIVE-21001.09.patch, HIVE-21001.10.patch, HIVE-21001.11.patch, > HIVE-21001.12.patch, HIVE-21001.13.patch, HIVE-21001.15.patch, > HIVE-21001.16.patch, HIVE-21001.17.patch, HIVE-21001.18.patch, > HIVE-21001.18.patch, HIVE-21001.19.patch, HIVE-21001.20.patch, > HIVE-21001.21.patch, HIVE-21001.22.patch, HIVE-21001.22.patch, > HIVE-21001.22.patch, HIVE-21001.23.patch, HIVE-21001.24.patch, > HIVE-21001.26.patch, HIVE-21001.26.patch, HIVE-21001.26.patch, > HIVE-21001.26.patch, HIVE-21001.26.patch, HIVE-21001.27.patch, > HIVE-21001.28.patch, HIVE-21001.29.patch, HIVE-21001.29.patch, > HIVE-21001.30.patch, HIVE-21001.31.patch, HIVE-21001.32.patch, > HIVE-21001.34.patch, HIVE-21001.35.patch, HIVE-21001.36.patch, > HIVE-21001.37.patch, HIVE-21001.38.patch, HIVE-21001.39.patch, > HIVE-21001.40.patch, HIVE-21001.41.patch, HIVE-21001.42.patch, > HIVE-21001.43.patch > > > XLEAR LIBRARY CACHE -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-16924) Support distinct in presence of Group By
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-16924:
--
Attachment: HIVE-16924.25.patch
> Support distinct in presence of Group By
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch, HIVE-16924.07.patch, HIVE-16924.08.patch,
> HIVE-16924.09.patch, HIVE-16924.10.patch, HIVE-16924.11.patch,
> HIVE-16924.12.patch, HIVE-16924.13.patch, HIVE-16924.14.patch,
> HIVE-16924.15.patch, HIVE-16924.16.patch, HIVE-16924.17.patch,
> HIVE-16924.18.patch, HIVE-16924.19.patch, HIVE-16924.20.patch,
> HIVE-16924.21.patch, HIVE-16924.22.patch, HIVE-16924.23.patch,
> HIVE-16924.24.patch, HIVE-16924.25.patch
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16924) Support distinct in presence of Group By
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-16924:
--
Status: Open (was: Patch Available)
> Support distinct in presence of Group By
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch, HIVE-16924.07.patch, HIVE-16924.08.patch,
> HIVE-16924.09.patch, HIVE-16924.10.patch, HIVE-16924.11.patch,
> HIVE-16924.12.patch, HIVE-16924.13.patch, HIVE-16924.14.patch,
> HIVE-16924.15.patch, HIVE-16924.16.patch, HIVE-16924.17.patch,
> HIVE-16924.18.patch, HIVE-16924.19.patch, HIVE-16924.20.patch,
> HIVE-16924.21.patch, HIVE-16924.22.patch, HIVE-16924.23.patch,
> HIVE-16924.24.patch, HIVE-16924.25.patch
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16924) Support distinct in presence of Group By
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-16924:
--
Status: Patch Available (was: Open)
> Support distinct in presence of Group By
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch, HIVE-16924.07.patch, HIVE-16924.08.patch,
> HIVE-16924.09.patch, HIVE-16924.10.patch, HIVE-16924.11.patch,
> HIVE-16924.12.patch, HIVE-16924.13.patch, HIVE-16924.14.patch,
> HIVE-16924.15.patch, HIVE-16924.16.patch, HIVE-16924.17.patch,
> HIVE-16924.18.patch, HIVE-16924.19.patch, HIVE-16924.20.patch,
> HIVE-16924.21.patch, HIVE-16924.22.patch, HIVE-16924.23.patch,
> HIVE-16924.24.patch, HIVE-16924.25.patch
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21369) LLAP: Logging is expensive in encoded reader path
[
https://issues.apache.org/jira/browse/HIVE-21369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793911#comment-16793911
]
Hive QA commented on HIVE-21369:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
17s{color} | {color:blue} ql in master has 2257 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16532/dev-support/hive-personality.sh
|
| git revision | master / 677eef8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16532/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> LLAP: Logging is expensive in encoded reader path
> -
>
> Key: HIVE-21369
> URL: https://issues.apache.org/jira/browse/HIVE-21369
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Nita Dembla
>Priority: Major
> Attachments: HIVE-21369.patch, HIVE-21369.patch.2
>
>
> There should be no INFO logging in EncodedReaderImpl. Stringifying of disk
> ranges is expensive in core read path.
> {code:java}
> 2019-03-01T17:55:56.322852142Z 2019-03-01T17:55:56,306 INFO
> [IO-Elevator-Thread-3
> (hive_20190301175546_a279f33c-4f2b-4cd5-8695-57bc8b042a61)]
> encoded.EncodedReaderImpl: Disk ranges after cache (found everything true;
> file [-3693547618692831801, 1551190876000, 1047660824], base offset
> 792920167): [{start: 887940 end: 1003508 cache buffer: 0x5165f83d(1)},
> {start: 1003508 end: 1119078 cache buffer: 0xb63cac3(1)}, {start: 1119078
> end: 1234745 cache buffer: 0x41a724fa(1)}, {start: 1234745 end: 1350261 cache
> buffer: 0x2f71bc38(1)}, {start: 1350261 end: 1465752 cache buffer:
> 0x2c38e1bb(1)}, {start: 1465752 end: 1581231 cache buffer: 0x5827982(1)},
> {start: 1581231 end: 1696885 cache buffer: 0x75a6773c(1)}, {start: 1696885
> end: 1812492 cache buffer: 0x2ed060f9(1)},{start: 1812492 end: 1928086 cache
> buffer: 0x20b2c8aa(1)}, {start: 1928086 end: 2043588 cache buffer:
> 0x6559aacb(1)}, {start: 2043588 end: 2159089 cache buffer: 0x569c85e1(1)},
> {start: 2159089 end: 2274725 cache buffer: 0x25a88dd0(1)}, {start: 2274725
> end: 2390228 cache
[jira] [Commented] (HIVE-16924) Support distinct in presence of Group By
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793892#comment-16793892
]
Hive QA commented on HIVE-16924:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962618/HIVE-16924.24.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15802 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=169)
[semijoin_reddedup.q,sysdb.q,dpp.q,tez_join.q,vectorization_limit.q,vectorized_rcfile_columnar.q,vector_reuse_scratchcols.q,schema_evol_orc_acid_table_llap_io.q,delete_where_non_partitioned.q,partialdhj.q,schema_evol_orc_acid_table.q,vector_when_case_null.q,orc_merge_incompat_schema.q,vectorization_11.q,schema_evol_text_vecrow_table_llap_io.q,vector_reduce2.q,vector_interval_mapjoin.q,schema_evol_orc_acidvec_table_update_llap_io.q,tez_joins_explain.q,vector_windowing_order_null.q,vector_decimal_trailing.q,tez_union.q,vector_aggregate_9.q,vector_groupby_grouping_sets_limit.q,approx_distinct.q,materialized_view_rewrite_ssb.q,results_cache_invalidation.q,default_constraint.q,offset_limit.q,results_cache_invalidation2.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16531/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16531/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16531/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962618 - PreCommit-HIVE-Build
> Support distinct in presence of Group By
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch, HIVE-16924.07.patch, HIVE-16924.08.patch,
> HIVE-16924.09.patch, HIVE-16924.10.patch, HIVE-16924.11.patch,
> HIVE-16924.12.patch, HIVE-16924.13.patch, HIVE-16924.14.patch,
> HIVE-16924.15.patch, HIVE-16924.16.patch, HIVE-16924.17.patch,
> HIVE-16924.18.patch, HIVE-16924.19.patch, HIVE-16924.20.patch,
> HIVE-16924.21.patch, HIVE-16924.22.patch, HIVE-16924.23.patch,
> HIVE-16924.24.patch
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21457) Perf optimizations in ORC split-generation
[ https://issues.apache.org/jira/browse/HIVE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-21457: - Summary: Perf optimizations in ORC split-generation (was: Perf optimizations in split-generation) > Perf optimizations in ORC split-generation > -- > > Key: HIVE-21457 > URL: https://issues.apache.org/jira/browse/HIVE-21457 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Minor > > Minor split generation optimizations > * Reuse vectorization checks > * Reuse isAcid checks > * Reuse filesystem objects > * Improved logging (log at top-level instead of inside the thread pool) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-16924) Support distinct in presence of Group By
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793889#comment-16793889
]
Hive QA commented on HIVE-16924:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
15s{color} | {color:blue} ql in master has 2257 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
48s{color} | {color:red} ql: The patch generated 8 new + 639 unchanged - 13
fixed = 647 total (was 652) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
4s{color} | {color:red} root: The patch generated 8 new + 647 unchanged - 13
fixed = 655 total (was 660) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
27s{color} | {color:green} ql generated 0 new + 2255 unchanged - 2 fixed = 2255
total (was 2257) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 66m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16531/dev-support/hive-personality.sh
|
| git revision | master / 677eef8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16531/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16531/yetus/diff-checkstyle-root.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16531/yetus/whitespace-eol.txt
|
| modules | C: ql . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16531/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support distinct in presence of Group By
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch, HIVE-16924.07.patch, HIVE-16924.08.patch,
> HIVE-16924.09.patch, HIVE-16924.10.patch, HIVE-16924.11.patch,
> HIVE-16924.12.patch,
[jira] [Assigned] (HIVE-21457) Perf optimizations in split-generation
[ https://issues.apache.org/jira/browse/HIVE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-21457: > Perf optimizations in split-generation > -- > > Key: HIVE-21457 > URL: https://issues.apache.org/jira/browse/HIVE-21457 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Minor > > Minor split generation optimizations > * Reuse vectorization checks > * Reuse isAcid checks > * Reuse filesystem objects > * Improved logging (log at top-level instead of inside the thread pool) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-15406) Consider vectorizing the new 'trunc' function
[
https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793846#comment-16793846
]
Hive QA commented on HIVE-15406:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962614/HIVE-15406.05.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15833 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16530/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16530/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16530/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962614 - PreCommit-HIVE-Build
> Consider vectorizing the new 'trunc' function
> -
>
> Key: HIVE-15406
> URL: https://issues.apache.org/jira/browse/HIVE-15406
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 4.0.0
>Reporter: Matt McCline
>Assignee: Laszlo Bodor
>Priority: Critical
> Attachments: HIVE-15406.01.patch, HIVE-15406.02.patch,
> HIVE-15406.03.patch, HIVE-15406.04.patch, HIVE-15406.05.patch
>
>
> Rounding function 'trunc' added by HIVE-14582.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21382) Group by keys reduction optimization - keys are not reduced in query23
[
https://issues.apache.org/jira/browse/HIVE-21382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21382:
---
Attachment: HIVE-21382.6.patch
> Group by keys reduction optimization - keys are not reduced in query23
> --
>
> Key: HIVE-21382
> URL: https://issues.apache.org/jira/browse/HIVE-21382
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21382.1.patch, HIVE-21382.2.patch,
> HIVE-21382.2.patch, HIVE-21382.3.patch, HIVE-21382.4.patch,
> HIVE-21382.5.patch, HIVE-21382.6.patch
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> {code:sql}
> explain cbo with frequent_ss_items as
> (select substr(i_item_desc,1,30) itemdesc,i_item_sk item_sk,d_date
> solddate,count(*) cnt
> from store_sales
> ,date_dim
> ,item
> where ss_sold_date_sk = d_date_sk
> and ss_item_sk = i_item_sk
> and d_year in (1999,1999+1,1999+2,1999+3)
> group by substr(i_item_desc,1,30),i_item_sk,d_date
> having count(*) >4)
> select sum(sales)
> from ((select cs_quantity*cs_list_price sales
>from catalog_sales
>,date_dim
>where d_year = 1999
> and d_moy = 1
> and cs_sold_date_sk = d_date_sk
> and cs_item_sk in (select item_sk from frequent_ss_items))) subq
> limit 100;
> {code}
> {code:sql}
> HiveSortLimit(fetch=[100])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{}], agg#0=[sum($0)])
> HiveProject(sales=[*(CAST($2):DECIMAL(10, 0), $3)])
> HiveSemiJoin(condition=[=($1, $5)], joinType=[inner])
> HiveJoin(condition=[=($0, $4)], joinType=[inner], algorithm=[none],
> cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(cs_sold_date_sk=[$0], cs_item_sk=[$15],
> cs_quantity=[$18], cs_list_price=[$20])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints, catalog_sales]],
> table:alias=[catalog_sales])
> HiveProject(d_date_sk=[$0])
> HiveFilter(condition=[AND(=($6, 1999), =($8, 1))])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$1])
> HiveFilter(condition=[>($3, 4)])
> HiveProject(substr=[$2], i_item_sk=[$1], d_date=[$0], $f3=[$3])
> HiveAggregate(group=[{3, 4, 5}], agg#0=[count()])
> HiveJoin(condition=[=($1, $4)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveJoin(condition=[=($0, $2)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(ss_sold_date_sk=[$0], ss_item_sk=[$2])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints,
> store_sales]], table:alias=[store_sales])
> HiveProject(d_date_sk=[$0], d_date=[$2])
> HiveFilter(condition=[IN($6, 1999, 2000, 2001, 2002)])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$0], substr=[substr($4, 1, 30)])
> HiveTableScan(table=[[perf_constraints, item]],
> table:alias=[item])
> {code}
> Right side of HiveSemiJoin has an aggregate which could be reduce to have
> only {{i_item_sk}} as group by key since {{i_item_sk}} is primary key.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21382) Group by keys reduction optimization - keys are not reduced in query23
[
https://issues.apache.org/jira/browse/HIVE-21382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21382:
---
Status: Patch Available (was: Open)
> Group by keys reduction optimization - keys are not reduced in query23
> --
>
> Key: HIVE-21382
> URL: https://issues.apache.org/jira/browse/HIVE-21382
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21382.1.patch, HIVE-21382.2.patch,
> HIVE-21382.2.patch, HIVE-21382.3.patch, HIVE-21382.4.patch,
> HIVE-21382.5.patch, HIVE-21382.6.patch
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> {code:sql}
> explain cbo with frequent_ss_items as
> (select substr(i_item_desc,1,30) itemdesc,i_item_sk item_sk,d_date
> solddate,count(*) cnt
> from store_sales
> ,date_dim
> ,item
> where ss_sold_date_sk = d_date_sk
> and ss_item_sk = i_item_sk
> and d_year in (1999,1999+1,1999+2,1999+3)
> group by substr(i_item_desc,1,30),i_item_sk,d_date
> having count(*) >4)
> select sum(sales)
> from ((select cs_quantity*cs_list_price sales
>from catalog_sales
>,date_dim
>where d_year = 1999
> and d_moy = 1
> and cs_sold_date_sk = d_date_sk
> and cs_item_sk in (select item_sk from frequent_ss_items))) subq
> limit 100;
> {code}
> {code:sql}
> HiveSortLimit(fetch=[100])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{}], agg#0=[sum($0)])
> HiveProject(sales=[*(CAST($2):DECIMAL(10, 0), $3)])
> HiveSemiJoin(condition=[=($1, $5)], joinType=[inner])
> HiveJoin(condition=[=($0, $4)], joinType=[inner], algorithm=[none],
> cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(cs_sold_date_sk=[$0], cs_item_sk=[$15],
> cs_quantity=[$18], cs_list_price=[$20])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints, catalog_sales]],
> table:alias=[catalog_sales])
> HiveProject(d_date_sk=[$0])
> HiveFilter(condition=[AND(=($6, 1999), =($8, 1))])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$1])
> HiveFilter(condition=[>($3, 4)])
> HiveProject(substr=[$2], i_item_sk=[$1], d_date=[$0], $f3=[$3])
> HiveAggregate(group=[{3, 4, 5}], agg#0=[count()])
> HiveJoin(condition=[=($1, $4)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveJoin(condition=[=($0, $2)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(ss_sold_date_sk=[$0], ss_item_sk=[$2])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints,
> store_sales]], table:alias=[store_sales])
> HiveProject(d_date_sk=[$0], d_date=[$2])
> HiveFilter(condition=[IN($6, 1999, 2000, 2001, 2002)])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$0], substr=[substr($4, 1, 30)])
> HiveTableScan(table=[[perf_constraints, item]],
> table:alias=[item])
> {code}
> Right side of HiveSemiJoin has an aggregate which could be reduce to have
> only {{i_item_sk}} as group by key since {{i_item_sk}} is primary key.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21382) Group by keys reduction optimization - keys are not reduced in query23
[
https://issues.apache.org/jira/browse/HIVE-21382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21382:
---
Status: Open (was: Patch Available)
> Group by keys reduction optimization - keys are not reduced in query23
> --
>
> Key: HIVE-21382
> URL: https://issues.apache.org/jira/browse/HIVE-21382
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21382.1.patch, HIVE-21382.2.patch,
> HIVE-21382.2.patch, HIVE-21382.3.patch, HIVE-21382.4.patch,
> HIVE-21382.5.patch, HIVE-21382.6.patch
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> {code:sql}
> explain cbo with frequent_ss_items as
> (select substr(i_item_desc,1,30) itemdesc,i_item_sk item_sk,d_date
> solddate,count(*) cnt
> from store_sales
> ,date_dim
> ,item
> where ss_sold_date_sk = d_date_sk
> and ss_item_sk = i_item_sk
> and d_year in (1999,1999+1,1999+2,1999+3)
> group by substr(i_item_desc,1,30),i_item_sk,d_date
> having count(*) >4)
> select sum(sales)
> from ((select cs_quantity*cs_list_price sales
>from catalog_sales
>,date_dim
>where d_year = 1999
> and d_moy = 1
> and cs_sold_date_sk = d_date_sk
> and cs_item_sk in (select item_sk from frequent_ss_items))) subq
> limit 100;
> {code}
> {code:sql}
> HiveSortLimit(fetch=[100])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{}], agg#0=[sum($0)])
> HiveProject(sales=[*(CAST($2):DECIMAL(10, 0), $3)])
> HiveSemiJoin(condition=[=($1, $5)], joinType=[inner])
> HiveJoin(condition=[=($0, $4)], joinType=[inner], algorithm=[none],
> cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(cs_sold_date_sk=[$0], cs_item_sk=[$15],
> cs_quantity=[$18], cs_list_price=[$20])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints, catalog_sales]],
> table:alias=[catalog_sales])
> HiveProject(d_date_sk=[$0])
> HiveFilter(condition=[AND(=($6, 1999), =($8, 1))])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$1])
> HiveFilter(condition=[>($3, 4)])
> HiveProject(substr=[$2], i_item_sk=[$1], d_date=[$0], $f3=[$3])
> HiveAggregate(group=[{3, 4, 5}], agg#0=[count()])
> HiveJoin(condition=[=($1, $4)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveJoin(condition=[=($0, $2)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(ss_sold_date_sk=[$0], ss_item_sk=[$2])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints,
> store_sales]], table:alias=[store_sales])
> HiveProject(d_date_sk=[$0], d_date=[$2])
> HiveFilter(condition=[IN($6, 1999, 2000, 2001, 2002)])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$0], substr=[substr($4, 1, 30)])
> HiveTableScan(table=[[perf_constraints, item]],
> table:alias=[item])
> {code}
> Right side of HiveSemiJoin has an aggregate which could be reduce to have
> only {{i_item_sk}} as group by key since {{i_item_sk}} is primary key.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-15406) Consider vectorizing the new 'trunc' function
[
https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793801#comment-16793801
]
Hive QA commented on HIVE-15406:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
12s{color} | {color:blue} ql in master has 2257 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 16 new + 4 unchanged - 1 fixed
= 20 total (was 5) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 7 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16530/dev-support/hive-personality.sh
|
| git revision | master / 677eef8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16530/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16530/yetus/whitespace-eol.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16530/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16530/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Consider vectorizing the new 'trunc' function
> -
>
> Key: HIVE-15406
> URL: https://issues.apache.org/jira/browse/HIVE-15406
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 4.0.0
>Reporter: Matt McCline
>Assignee: Laszlo Bodor
>Priority: Critical
> Attachments: HIVE-15406.01.patch, HIVE-15406.02.patch,
> HIVE-15406.03.patch, HIVE-15406.04.patch, HIVE-15406.05.patch
>
>
> Rounding function 'trunc' added by HIVE-14582.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21456) Hive Metastore HTTP Thrift
[ https://issues.apache.org/jira/browse/HIVE-21456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Khanna reassigned HIVE-21456: -- > Hive Metastore HTTP Thrift > -- > > Key: HIVE-21456 > URL: https://issues.apache.org/jira/browse/HIVE-21456 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore >Reporter: Amit Khanna >Assignee: Amit Khanna >Priority: Major > > Hive Metastore currently doesn't have support for HTTP transport because of > which it is not possible to access it via Knox. Adding support for Thrift > over HTTP transport will allow the clients to access via Knox -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21283) Create Synonym mid for substr, position for locate
[
https://issues.apache.org/jira/browse/HIVE-21283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793776#comment-16793776
]
Hive QA commented on HIVE-21283:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962602/HIVE.21283.05.PATCH
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 38 failed/errored test(s), 15834 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mid] (batchId=17)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.otherCatalog[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartAlreadyExists[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartEmptyDB[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartEmptyPartName[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartEmptyTable[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartInvalidPartName[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartLessPartValues[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartMorePartValues[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartMultiplePartitions[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartNonExistingDB[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartNonExistingTable[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartNullDB[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartNullPartName[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartNullTable[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartToExternalTable[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartToTableWithoutPartCols[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartToView[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartWrongColumnInPartName[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPart[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionAlreadyExists[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionBogusCatalog[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionByNameBogusCatalog[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionEmptyDB[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionEmptyPartValues[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionEmptyTable[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionLessPartValues[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionMorePartValues[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionMultiplePartitions[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionNonExistingDB[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionNonExistingTable[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionNullDB[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionNullPartValues[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionNullTable[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionToExternalTable[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionToTableWithoutPartCols[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartitionToView[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestAppendPartitions.testAppendPartition[Remote]
(batchId=222)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16529/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16529/console
[jira] [Commented] (HIVE-21283) Create Synonym mid for substr, position for locate
[
https://issues.apache.org/jira/browse/HIVE-21283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793751#comment-16793751
]
Hive QA commented on HIVE-21283:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
9s{color} | {color:blue} ql in master has 2257 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16529/dev-support/hive-personality.sh
|
| git revision | master / 677eef8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16529/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Create Synonym mid for substr, position for locate
>
>
> Key: HIVE-21283
> URL: https://issues.apache.org/jira/browse/HIVE-21283
> Project: Hive
> Issue Type: New Feature
>Reporter: Mani M
>Assignee: Mani M
>Priority: Minor
> Labels: UDF, pull-request-available, todoc4.0
> Fix For: 4.0.0
>
> Attachments: HIVE.21283.03.PATCH, HIVE.21283.04.PATCH,
> HIVE.21283.05.PATCH, HIVE.21283.2.PATCH, HIVE.21283.PATCH
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Create new synonym for the existing function
>
> Mid for substr
> postiion for locate
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21454) Tez default configs get overwritten by MR default configs
[ https://issues.apache.org/jira/browse/HIVE-21454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Syed Shameerur Rahman updated HIVE-21454: - Description: Due to changes done in HIVE-17781 Tez default configs such as tez.counters.max which has a default value of 1200 gets overwritten by mapreduce.job.counters.max which has a default value of 120 cc [~mithun] was:Due to changes done in HIVE-17781 Tez default configs such as tez.counters.max which has a default value of 1200 gets overwritten by mapreduce.job.counters.max which has a default value of 120 > Tez default configs get overwritten by MR default configs > - > > Key: HIVE-21454 > URL: https://issues.apache.org/jira/browse/HIVE-21454 > Project: Hive > Issue Type: Bug >Reporter: Syed Shameerur Rahman >Priority: Major > > Due to changes done in HIVE-17781 Tez default configs such as > tez.counters.max which has a default value of 1200 gets overwritten by > mapreduce.job.counters.max which has a default value of 120 > cc [~mithun] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21369) LLAP: Logging is expensive in encoded reader path
[
https://issues.apache.org/jira/browse/HIVE-21369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nita Dembla updated HIVE-21369:
---
Attachment: HIVE-21369.patch.2
> LLAP: Logging is expensive in encoded reader path
> -
>
> Key: HIVE-21369
> URL: https://issues.apache.org/jira/browse/HIVE-21369
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Nita Dembla
>Priority: Major
> Attachments: HIVE-21369.patch, HIVE-21369.patch.2
>
>
> There should be no INFO logging in EncodedReaderImpl. Stringifying of disk
> ranges is expensive in core read path.
> {code:java}
> 2019-03-01T17:55:56.322852142Z 2019-03-01T17:55:56,306 INFO
> [IO-Elevator-Thread-3
> (hive_20190301175546_a279f33c-4f2b-4cd5-8695-57bc8b042a61)]
> encoded.EncodedReaderImpl: Disk ranges after cache (found everything true;
> file [-3693547618692831801, 1551190876000, 1047660824], base offset
> 792920167): [{start: 887940 end: 1003508 cache buffer: 0x5165f83d(1)},
> {start: 1003508 end: 1119078 cache buffer: 0xb63cac3(1)}, {start: 1119078
> end: 1234745 cache buffer: 0x41a724fa(1)}, {start: 1234745 end: 1350261 cache
> buffer: 0x2f71bc38(1)}, {start: 1350261 end: 1465752 cache buffer:
> 0x2c38e1bb(1)}, {start: 1465752 end: 1581231 cache buffer: 0x5827982(1)},
> {start: 1581231 end: 1696885 cache buffer: 0x75a6773c(1)}, {start: 1696885
> end: 1812492 cache buffer: 0x2ed060f9(1)},{start: 1812492 end: 1928086 cache
> buffer: 0x20b2c8aa(1)}, {start: 1928086 end: 2043588 cache buffer:
> 0x6559aacb(1)}, {start: 2043588 end: 2159089 cache buffer: 0x569c85e1(1)},
> {start: 2159089 end: 2274725 cache buffer: 0x25a88dd0(1)}, {start: 2274725
> end: 2390228 cache buffer: 0x738b7e87(1)}, {start: 2390228 end: 2505715 cache
> buffer: 0x26edafa0(1)}, {start: 2505715 end: 2621322 cache buffer:
> 0x69db7752(1)}, {start: 2621322 end: 2736844 cache b{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21453) HPL/SQL can not SELECT Date and Timestamp type value into variable
[
https://issues.apache.org/jira/browse/HIVE-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793733#comment-16793733
]
Hive QA commented on HIVE-21453:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962601/HIVE-21453.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15832 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16528/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16528/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16528/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962601 - PreCommit-HIVE-Build
> HPL/SQL can not SELECT Date and Timestamp type value into variable
> ---
>
> Key: HIVE-21453
> URL: https://issues.apache.org/jira/browse/HIVE-21453
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Affects Versions: 3.1.1
> Environment: Centos 7.1, Hive 3.1.1
>Reporter: DingQiang Liu
>Assignee: DingQiang Liu
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21453.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HPL/SQL forgot Date and Timestamp types when SELECT INTO variables. for
> example, current implement will set null, not correct value, to variables
> for following case, select_into3.sql:
> declare v_date date;
> declare v_timestamp timestamp(17, 3);
> select
> cast('2019-02-20 12:23:45.678' as date),
> cast('2019-02-20 12:23:45.678' as timestamp)
> into
> v_date,
> v_timestamp
> from src limit 1;
> print 'date: ' || v_date;
> print 'timestamp: ' || v_timestamp;
> The result when running : bin/hplsql -f select_into3.sql --trace
> Configuration file: file:/hive/conf/hplsql-site.xml
> Parser tree: (program (block (stmt (declare_stmt declare (declare_stmt_item
> (declare_var_item (ident v_date) (dtype date) (stmt (semicolon_stmt ;))
> (stmt (declare_stmt declare (declare_stmt_item (declare_var_item (ident
> v_timestamp) (dtype timestamp) (dtype_len ( 17 , 3 )) (stmt
> (semicolon_stmt ;)) (stmt (select_stmt (fullselect_stmt (fullselect_stmt_item
> (subselect_stmt select (select_list (select_list_item (expr (expr_spec_func
> cast ( (expr (expr_atom (string '2019-02-20 12:23:45.678'))) as (dtype date)
> , (select_list_item (expr (expr_spec_func cast ( (expr (expr_atom
> (string '2019-02-20 12:23:45.678'))) as (dtype timestamp) ) (into_clause
> into (ident v_date) , (ident v_timestamp)) (from_clause from
> (from_table_clause (from_table_name_clause (table_name (ident src)
> (select_options (select_options_item limit (expr (expr_atom (int_number
> 1)) (stmt (semicolon_stmt ;)) (stmt (print_stmt print (expr
> (expr_concat (expr_concat_item (expr_atom (string 'date: '))) ||
> (expr_concat_item (expr_atom (ident v_date))) (stmt (semicolon_stmt ;))
> (stmt (print_stmt print (expr (expr_concat (expr_concat_item (expr_atom
> (string 'timestamp: '))) || (expr_concat_item (expr_atom (ident
> v_timestamp))) (stmt (semicolon_stmt ;))) )
> Ln:1 DECLARE v_date date
> Ln:2 DECLARE v_timestamp timestamp
> Ln:4 SELECT
> Ln:4 select
> cast('2019-02-20 12:23:45.678' as date), cast('2019-02-20 12:23:45.678' as
> timestamp)
> from src LIMIT 1
> Open connection: jdbc:vertica://v001:5433/test (256 ms)
> Starting query
> Query executed successfully (55 ms)
> Ln:4 SELECT completed successfully
> Ln:4 SELECT INTO statement executed
> Ln:4 COLUMN: ?column?, Date
> Ln:4 SET v_date = null
> Ln:4 COLUMN: ?column?, Timestamp
> Ln:4 SET v_timestamp = null
> Ln:12 PRINT
> date:
> Ln:13 PRINT
> timestamp:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21453) HPL/SQL can not SELECT Date and Timestamp type value into variable
[
https://issues.apache.org/jira/browse/HIVE-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793686#comment-16793686
]
Hive QA commented on HIVE-21453:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
46s{color} | {color:blue} hplsql in master has 157 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} hplsql: The patch generated 4 new + 266 unchanged - 0
fixed = 270 total (was 266) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16528/dev-support/hive-personality.sh
|
| git revision | master / bc79adf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16528/yetus/diff-checkstyle-hplsql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16528/yetus/whitespace-eol.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16528/yetus/patch-asflicense-problems.txt
|
| modules | C: hplsql U: hplsql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16528/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HPL/SQL can not SELECT Date and Timestamp type value into variable
> ---
>
> Key: HIVE-21453
> URL: https://issues.apache.org/jira/browse/HIVE-21453
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Affects Versions: 3.1.1
> Environment: Centos 7.1, Hive 3.1.1
>Reporter: DingQiang Liu
>Assignee: DingQiang Liu
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21453.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HPL/SQL forgot Date and Timestamp types when SELECT INTO variables. for
> example, current implement will set null, not correct value, to variables
> for following case, select_into3.sql:
> declare v_date date;
> declare v_timestamp timestamp(17, 3);
> select
> cast('2019-02-20 12:23:45.678' as date),
> cast('2019-02-20 12:23:45.678' as timestamp)
> into
> v_date,
> v_timestamp
> from src limit 1;
> print 'date: ' || v_date;
> print 'timestamp: ' || v_timestamp;
> The result when running : bin/hplsql -f select_into3.sql --trace
> Configuration file: file:/hive/conf/hplsql-site.xml
>
[jira] [Commented] (HIVE-21430) INSERT into a dynamically partitioned table with hive.stats.autogather = false throws a MetaException
[
https://issues.apache.org/jira/browse/HIVE-21430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793673#comment-16793673
]
Hive QA commented on HIVE-21430:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962590/HIVE-21430.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15833 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16527/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16527/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16527/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962590 - PreCommit-HIVE-Build
> INSERT into a dynamically partitioned table with hive.stats.autogather =
> false throws a MetaException
> -
>
> Key: HIVE-21430
> URL: https://issues.apache.org/jira/browse/HIVE-21430
> Project: Hive
> Issue Type: Bug
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21430.01.patch, metaexception_repro.patch,
> org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread-output.txt
>
> Original Estimate: 48h
> Time Spent: 10m
> Remaining Estimate: 47h 50m
>
> When the test TestStatsUpdaterThread#testTxnDynamicPartitions added in the
> attached patch is run it throws exception (full logs attached.)
> org.apache.hadoop.hive.metastore.api.MetaException: Cannot change stats state
> for a transactional table default.simple_stats without providing the
> transactional write state for verification (new write ID 5, valid write IDs
> null; current state \{"BASIC_STATS":"true","COLUMN_STATS":{"s":"true"}}; new
> state null
> at
> org.apache.hadoop.hive.metastore.ObjectStore.alterPartitionNoTxn(ObjectStore.java:4328)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16924) Support distinct in presence of Group By
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-16924:
--
Attachment: HIVE-16924.24.patch
> Support distinct in presence of Group By
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch, HIVE-16924.07.patch, HIVE-16924.08.patch,
> HIVE-16924.09.patch, HIVE-16924.10.patch, HIVE-16924.11.patch,
> HIVE-16924.12.patch, HIVE-16924.13.patch, HIVE-16924.14.patch,
> HIVE-16924.15.patch, HIVE-16924.16.patch, HIVE-16924.17.patch,
> HIVE-16924.18.patch, HIVE-16924.19.patch, HIVE-16924.20.patch,
> HIVE-16924.21.patch, HIVE-16924.22.patch, HIVE-16924.23.patch,
> HIVE-16924.24.patch
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16924) Support distinct in presence of Group By
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-16924:
--
Status: Patch Available (was: Open)
> Support distinct in presence of Group By
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch, HIVE-16924.07.patch, HIVE-16924.08.patch,
> HIVE-16924.09.patch, HIVE-16924.10.patch, HIVE-16924.11.patch,
> HIVE-16924.12.patch, HIVE-16924.13.patch, HIVE-16924.14.patch,
> HIVE-16924.15.patch, HIVE-16924.16.patch, HIVE-16924.17.patch,
> HIVE-16924.18.patch, HIVE-16924.19.patch, HIVE-16924.20.patch,
> HIVE-16924.21.patch, HIVE-16924.22.patch, HIVE-16924.23.patch,
> HIVE-16924.24.patch
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16924) Support distinct in presence of Group By
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-16924:
--
Status: Open (was: Patch Available)
> Support distinct in presence of Group By
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch, HIVE-16924.07.patch, HIVE-16924.08.patch,
> HIVE-16924.09.patch, HIVE-16924.10.patch, HIVE-16924.11.patch,
> HIVE-16924.12.patch, HIVE-16924.13.patch, HIVE-16924.14.patch,
> HIVE-16924.15.patch, HIVE-16924.16.patch, HIVE-16924.17.patch,
> HIVE-16924.18.patch, HIVE-16924.19.patch, HIVE-16924.20.patch,
> HIVE-16924.21.patch, HIVE-16924.22.patch, HIVE-16924.23.patch
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21430) INSERT into a dynamically partitioned table with hive.stats.autogather = false throws a MetaException
[
https://issues.apache.org/jira/browse/HIVE-21430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793640#comment-16793640
]
Hive QA commented on HIVE-21430:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
13s{color} | {color:blue} ql in master has 2257 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16527/dev-support/hive-personality.sh
|
| git revision | master / bc79adf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16527/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16527/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> INSERT into a dynamically partitioned table with hive.stats.autogather =
> false throws a MetaException
> -
>
> Key: HIVE-21430
> URL: https://issues.apache.org/jira/browse/HIVE-21430
> Project: Hive
> Issue Type: Bug
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21430.01.patch, metaexception_repro.patch,
> org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread-output.txt
>
> Original Estimate: 48h
> Time Spent: 10m
> Remaining Estimate: 47h 50m
>
> When the test TestStatsUpdaterThread#testTxnDynamicPartitions added in the
> attached patch is run it throws exception (full logs attached.)
> org.apache.hadoop.hive.metastore.api.MetaException: Cannot change stats state
> for a transactional table default.simple_stats without providing the
> transactional write state for verification (new write ID 5, valid write IDs
> null; current state \{"BASIC_STATS":"true","COLUMN_STATS":{"s":"true"}}; new
> state null
> at
> org.apache.hadoop.hive.metastore.ObjectStore.alterPartitionNoTxn(ObjectStore.java:4328)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16924) Support distinct in presence of Group By
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793605#comment-16793605
]
Hive QA commented on HIVE-16924:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962591/HIVE-16924.23.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 15795 tests
executed
*Failed tests:*
{noformat}
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=230)
TestObjectStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestPartitionProjectionEvaluator - did not produce a TEST-*.xml file (likely
timed out) (batchId=230)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions
(batchId=263)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=263)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16526/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16526/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16526/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962591 - PreCommit-HIVE-Build
> Support distinct in presence of Group By
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch, HIVE-16924.07.patch, HIVE-16924.08.patch,
> HIVE-16924.09.patch, HIVE-16924.10.patch, HIVE-16924.11.patch,
> HIVE-16924.12.patch, HIVE-16924.13.patch, HIVE-16924.14.patch,
> HIVE-16924.15.patch, HIVE-16924.16.patch, HIVE-16924.17.patch,
> HIVE-16924.18.patch, HIVE-16924.19.patch, HIVE-16924.20.patch,
> HIVE-16924.21.patch, HIVE-16924.22.patch, HIVE-16924.23.patch
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16924) Support distinct in presence of Group By
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793603#comment-16793603
]
Hive QA commented on HIVE-16924:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
8s{color} | {color:blue} ql in master has 2257 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 8 new + 639 unchanged - 13
fixed = 647 total (was 652) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
0s{color} | {color:red} root: The patch generated 8 new + 647 unchanged - 13
fixed = 655 total (was 660) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
23s{color} | {color:green} ql generated 0 new + 2255 unchanged - 2 fixed = 2255
total (was 2257) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 63m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16526/dev-support/hive-personality.sh
|
| git revision | master / bc79adf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16526/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16526/yetus/diff-checkstyle-root.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16526/yetus/whitespace-eol.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16526/yetus/patch-asflicense-problems.txt
|
| modules | C: ql . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16526/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support distinct in presence of Group By
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch, HIVE-16924.07.patch, HIVE-16924.08.patch,
[jira] [Commented] (HIVE-21423) Do not check for whitespace issues in generated code
[
https://issues.apache.org/jira/browse/HIVE-21423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793573#comment-16793573
]
Hive QA commented on HIVE-21423:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962592/HIVE-21423.04.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15832 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_groupby_reduce]
(batchId=61)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16525/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16525/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16525/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962592 - PreCommit-HIVE-Build
> Do not check for whitespace issues in generated code
>
>
> Key: HIVE-21423
> URL: https://issues.apache.org/jira/browse/HIVE-21423
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21423.01.patch, HIVE-21423.02.patch,
> HIVE-21423.03.patch, HIVE-21423.04.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-15406) Consider vectorizing the new 'trunc' function
[ https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-15406: Attachment: HIVE-15406.05.patch > Consider vectorizing the new 'trunc' function > - > > Key: HIVE-15406 > URL: https://issues.apache.org/jira/browse/HIVE-15406 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 4.0.0 >Reporter: Matt McCline >Assignee: Laszlo Bodor >Priority: Critical > Attachments: HIVE-15406.01.patch, HIVE-15406.02.patch, > HIVE-15406.03.patch, HIVE-15406.04.patch, HIVE-15406.05.patch > > > Rounding function 'trunc' added by HIVE-14582. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-15406) Consider vectorizing the new 'trunc' function
[ https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793578#comment-16793578 ] Laszlo Bodor commented on HIVE-15406: - failure is not related, reuploading 05.patch > Consider vectorizing the new 'trunc' function > - > > Key: HIVE-15406 > URL: https://issues.apache.org/jira/browse/HIVE-15406 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 4.0.0 >Reporter: Matt McCline >Assignee: Laszlo Bodor >Priority: Critical > Attachments: HIVE-15406.01.patch, HIVE-15406.02.patch, > HIVE-15406.03.patch, HIVE-15406.04.patch, HIVE-15406.05.patch > > > Rounding function 'trunc' added by HIVE-14582. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-15406) Consider vectorizing the new 'trunc' function
[ https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-15406: Attachment: (was: HIVE-15406.05.patch) > Consider vectorizing the new 'trunc' function > - > > Key: HIVE-15406 > URL: https://issues.apache.org/jira/browse/HIVE-15406 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 4.0.0 >Reporter: Matt McCline >Assignee: Laszlo Bodor >Priority: Critical > Attachments: HIVE-15406.01.patch, HIVE-15406.02.patch, > HIVE-15406.03.patch, HIVE-15406.04.patch > > > Rounding function 'trunc' added by HIVE-14582. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21283) Create Synonym mid for substr, position for locate
[ https://issues.apache.org/jira/browse/HIVE-21283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mani M updated HIVE-21283: -- Status: Patch Available (was: In Progress) > Create Synonym mid for substr, position for locate > > > Key: HIVE-21283 > URL: https://issues.apache.org/jira/browse/HIVE-21283 > Project: Hive > Issue Type: New Feature >Reporter: Mani M >Assignee: Mani M >Priority: Minor > Labels: UDF, pull-request-available, todoc4.0 > Fix For: 4.0.0 > > Attachments: HIVE.21283.03.PATCH, HIVE.21283.04.PATCH, > HIVE.21283.05.PATCH, HIVE.21283.2.PATCH, HIVE.21283.PATCH > > Time Spent: 1.5h > Remaining Estimate: 0h > > Create new synonym for the existing function > > Mid for substr > postiion for locate -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21283) Create Synonym mid for substr, position for locate
[ https://issues.apache.org/jira/browse/HIVE-21283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mani M updated HIVE-21283: -- Attachment: HIVE.21283.05.PATCH > Create Synonym mid for substr, position for locate > > > Key: HIVE-21283 > URL: https://issues.apache.org/jira/browse/HIVE-21283 > Project: Hive > Issue Type: New Feature >Reporter: Mani M >Assignee: Mani M >Priority: Minor > Labels: UDF, pull-request-available, todoc4.0 > Fix For: 4.0.0 > > Attachments: HIVE.21283.03.PATCH, HIVE.21283.04.PATCH, > HIVE.21283.05.PATCH, HIVE.21283.2.PATCH, HIVE.21283.PATCH > > Time Spent: 1.5h > Remaining Estimate: 0h > > Create new synonym for the existing function > > Mid for substr > postiion for locate -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21283) Create Synonym mid for substr, position for locate
[ https://issues.apache.org/jira/browse/HIVE-21283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mani M updated HIVE-21283: -- Status: In Progress (was: Patch Available) > Create Synonym mid for substr, position for locate > > > Key: HIVE-21283 > URL: https://issues.apache.org/jira/browse/HIVE-21283 > Project: Hive > Issue Type: New Feature >Reporter: Mani M >Assignee: Mani M >Priority: Minor > Labels: UDF, pull-request-available, todoc4.0 > Fix For: 4.0.0 > > Attachments: HIVE.21283.03.PATCH, HIVE.21283.04.PATCH, > HIVE.21283.2.PATCH, HIVE.21283.PATCH > > Time Spent: 1.5h > Remaining Estimate: 0h > > Create new synonym for the existing function > > Mid for substr > postiion for locate -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21392) Misconfigurations of DataNucleus log in log4j.properties
[
https://issues.apache.org/jira/browse/HIVE-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793531#comment-16793531
]
Hive QA commented on HIVE-21392:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962563/HIVE-21392.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15832 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.mapreduce.TestHCatPartitioned.testHCatPartitionedTable[2]
(batchId=209)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16523/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16523/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16523/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962563 - PreCommit-HIVE-Build
> Misconfigurations of DataNucleus log in log4j.properties
>
>
> Key: HIVE-21392
> URL: https://issues.apache.org/jira/browse/HIVE-21392
> Project: Hive
> Issue Type: Improvement
> Components: Logging
>Affects Versions: 2.0.0
>Reporter: Chen Zhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21392.02.patch, HIVE-21392.03.patch,
> HIVE-21392.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> In the patch of
> [HIVE-12020|https://issues.apache.org/jira/browse/HIVE-12020], we changed the
> DataNucleus related logging configuration from nine fine-grained loggers with
> three coarse-grained loggers (DataNucleus, Datastore and JPOX). As Prasanth
> Jayachandran
> [explain|https://issues.apache.org/jira/browse/HIVE-12020?focusedCommentId=15025612=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15025612],
> these three loggers are the top-level logger in DataNucleus, so that we
> don't need to specify other loggers for DataNucleus. However, according to
> the
> [documents|http://www.datanucleus.org/products/accessplatform/logging.html]
> and [source
> codes|https://github.com/datanucleus/datanucleus-core/blob/master/src/main/java/org/datanucleus/util/NucleusLogger.java#L108]
> of DataNucleus, the top-level logger in DataNucleus is `DataNucleus`.
> Therefore, we just need to keep the right one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21423) Do not check for whitespace issues in generated code
[ https://issues.apache.org/jira/browse/HIVE-21423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793536#comment-16793536 ] Hive QA commented on HIVE-21423: (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at http://104.198.109.242/logs//PreCommit-HIVE-Build-16525/yetus.txt in case of problems. > Do not check for whitespace issues in generated code > > > Key: HIVE-21423 > URL: https://issues.apache.org/jira/browse/HIVE-21423 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21423.01.patch, HIVE-21423.02.patch, > HIVE-21423.03.patch, HIVE-21423.04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (HIVE-21453) HPL/SQL can not SELECT Date and Timestamp type value into variable
[
https://issues.apache.org/jira/browse/HIVE-21453?focusedWorklogId=213687=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-213687
]
ASF GitHub Bot logged work on HIVE-21453:
-
Author: ASF GitHub Bot
Created on: 15/Mar/19 10:47
Start Date: 15/Mar/19 10:47
Worklog Time Spent: 10m
Work Description: dingqiangliu commented on issue #566: HIVE-21453:
HPL/SQL can not SELECT Date and Timestamp type value INTO variable
URL: https://github.com/apache/hive/pull/566#issuecomment-473240651
@rmsmani thanks for your patience, patch submitted.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 213687)
Time Spent: 0.5h (was: 20m)
> HPL/SQL can not SELECT Date and Timestamp type value into variable
> ---
>
> Key: HIVE-21453
> URL: https://issues.apache.org/jira/browse/HIVE-21453
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Affects Versions: 3.1.1
> Environment: Centos 7.1, Hive 3.1.1
>Reporter: DingQiang Liu
>Assignee: DingQiang Liu
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21453.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HPL/SQL forgot Date and Timestamp types when SELECT INTO variables. for
> example, current implement will set null, not correct value, to variables
> for following case, select_into3.sql:
> declare v_date date;
> declare v_timestamp timestamp(17, 3);
> select
> cast('2019-02-20 12:23:45.678' as date),
> cast('2019-02-20 12:23:45.678' as timestamp)
> into
> v_date,
> v_timestamp
> from src limit 1;
> print 'date: ' || v_date;
> print 'timestamp: ' || v_timestamp;
> The result when running : bin/hplsql -f select_into3.sql --trace
> Configuration file: file:/hive/conf/hplsql-site.xml
> Parser tree: (program (block (stmt (declare_stmt declare (declare_stmt_item
> (declare_var_item (ident v_date) (dtype date) (stmt (semicolon_stmt ;))
> (stmt (declare_stmt declare (declare_stmt_item (declare_var_item (ident
> v_timestamp) (dtype timestamp) (dtype_len ( 17 , 3 )) (stmt
> (semicolon_stmt ;)) (stmt (select_stmt (fullselect_stmt (fullselect_stmt_item
> (subselect_stmt select (select_list (select_list_item (expr (expr_spec_func
> cast ( (expr (expr_atom (string '2019-02-20 12:23:45.678'))) as (dtype date)
> , (select_list_item (expr (expr_spec_func cast ( (expr (expr_atom
> (string '2019-02-20 12:23:45.678'))) as (dtype timestamp) ) (into_clause
> into (ident v_date) , (ident v_timestamp)) (from_clause from
> (from_table_clause (from_table_name_clause (table_name (ident src)
> (select_options (select_options_item limit (expr (expr_atom (int_number
> 1)) (stmt (semicolon_stmt ;)) (stmt (print_stmt print (expr
> (expr_concat (expr_concat_item (expr_atom (string 'date: '))) ||
> (expr_concat_item (expr_atom (ident v_date))) (stmt (semicolon_stmt ;))
> (stmt (print_stmt print (expr (expr_concat (expr_concat_item (expr_atom
> (string 'timestamp: '))) || (expr_concat_item (expr_atom (ident
> v_timestamp))) (stmt (semicolon_stmt ;))) )
> Ln:1 DECLARE v_date date
> Ln:2 DECLARE v_timestamp timestamp
> Ln:4 SELECT
> Ln:4 select
> cast('2019-02-20 12:23:45.678' as date), cast('2019-02-20 12:23:45.678' as
> timestamp)
> from src LIMIT 1
> Open connection: jdbc:vertica://v001:5433/test (256 ms)
> Starting query
> Query executed successfully (55 ms)
> Ln:4 SELECT completed successfully
> Ln:4 SELECT INTO statement executed
> Ln:4 COLUMN: ?column?, Date
> Ln:4 SET v_date = null
> Ln:4 COLUMN: ?column?, Timestamp
> Ln:4 SET v_timestamp = null
> Ln:12 PRINT
> date:
> Ln:13 PRINT
> timestamp:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21392) Misconfigurations of DataNucleus log in log4j.properties
[
https://issues.apache.org/jira/browse/HIVE-21392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793533#comment-16793533
]
Hive QA commented on HIVE-21392:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962563/HIVE-21392.03.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16524/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16524/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16524/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12962563/HIVE-21392.03.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962563 - PreCommit-HIVE-Build
> Misconfigurations of DataNucleus log in log4j.properties
>
>
> Key: HIVE-21392
> URL: https://issues.apache.org/jira/browse/HIVE-21392
> Project: Hive
> Issue Type: Improvement
> Components: Logging
>Affects Versions: 2.0.0
>Reporter: Chen Zhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21392.02.patch, HIVE-21392.03.patch,
> HIVE-21392.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> In the patch of
> [HIVE-12020|https://issues.apache.org/jira/browse/HIVE-12020], we changed the
> DataNucleus related logging configuration from nine fine-grained loggers with
> three coarse-grained loggers (DataNucleus, Datastore and JPOX). As Prasanth
> Jayachandran
> [explain|https://issues.apache.org/jira/browse/HIVE-12020?focusedCommentId=15025612=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15025612],
> these three loggers are the top-level logger in DataNucleus, so that we
> don't need to specify other loggers for DataNucleus. However, according to
> the
> [documents|http://www.datanucleus.org/products/accessplatform/logging.html]
> and [source
> codes|https://github.com/datanucleus/datanucleus-core/blob/master/src/main/java/org/datanucleus/util/NucleusLogger.java#L108]
> of DataNucleus, the top-level logger in DataNucleus is `DataNucleus`.
> Therefore, we just need to keep the right one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21453) HPL/SQL can not SELECT Date and Timestamp type value into variable
[
https://issues.apache.org/jira/browse/HIVE-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DingQiang Liu updated HIVE-21453:
-
Release Note: Resolved the issue that HPL/SQL can not SELECT Date and
Timestamp type values INTO variables
Attachment: HIVE-21453.patch
Target Version/s: 4.0.0
Status: Patch Available (was: Open)
> HPL/SQL can not SELECT Date and Timestamp type value into variable
> ---
>
> Key: HIVE-21453
> URL: https://issues.apache.org/jira/browse/HIVE-21453
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Affects Versions: 3.1.1
> Environment: Centos 7.1, Hive 3.1.1
>Reporter: DingQiang Liu
>Assignee: DingQiang Liu
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21453.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> HPL/SQL forgot Date and Timestamp types when SELECT INTO variables. for
> example, current implement will set null, not correct value, to variables
> for following case, select_into3.sql:
> declare v_date date;
> declare v_timestamp timestamp(17, 3);
> select
> cast('2019-02-20 12:23:45.678' as date),
> cast('2019-02-20 12:23:45.678' as timestamp)
> into
> v_date,
> v_timestamp
> from src limit 1;
> print 'date: ' || v_date;
> print 'timestamp: ' || v_timestamp;
> The result when running : bin/hplsql -f select_into3.sql --trace
> Configuration file: file:/hive/conf/hplsql-site.xml
> Parser tree: (program (block (stmt (declare_stmt declare (declare_stmt_item
> (declare_var_item (ident v_date) (dtype date) (stmt (semicolon_stmt ;))
> (stmt (declare_stmt declare (declare_stmt_item (declare_var_item (ident
> v_timestamp) (dtype timestamp) (dtype_len ( 17 , 3 )) (stmt
> (semicolon_stmt ;)) (stmt (select_stmt (fullselect_stmt (fullselect_stmt_item
> (subselect_stmt select (select_list (select_list_item (expr (expr_spec_func
> cast ( (expr (expr_atom (string '2019-02-20 12:23:45.678'))) as (dtype date)
> , (select_list_item (expr (expr_spec_func cast ( (expr (expr_atom
> (string '2019-02-20 12:23:45.678'))) as (dtype timestamp) ) (into_clause
> into (ident v_date) , (ident v_timestamp)) (from_clause from
> (from_table_clause (from_table_name_clause (table_name (ident src)
> (select_options (select_options_item limit (expr (expr_atom (int_number
> 1)) (stmt (semicolon_stmt ;)) (stmt (print_stmt print (expr
> (expr_concat (expr_concat_item (expr_atom (string 'date: '))) ||
> (expr_concat_item (expr_atom (ident v_date))) (stmt (semicolon_stmt ;))
> (stmt (print_stmt print (expr (expr_concat (expr_concat_item (expr_atom
> (string 'timestamp: '))) || (expr_concat_item (expr_atom (ident
> v_timestamp))) (stmt (semicolon_stmt ;))) )
> Ln:1 DECLARE v_date date
> Ln:2 DECLARE v_timestamp timestamp
> Ln:4 SELECT
> Ln:4 select
> cast('2019-02-20 12:23:45.678' as date), cast('2019-02-20 12:23:45.678' as
> timestamp)
> from src LIMIT 1
> Open connection: jdbc:vertica://v001:5433/test (256 ms)
> Starting query
> Query executed successfully (55 ms)
> Ln:4 SELECT completed successfully
> Ln:4 SELECT INTO statement executed
> Ln:4 COLUMN: ?column?, Date
> Ln:4 SET v_date = null
> Ln:4 COLUMN: ?column?, Timestamp
> Ln:4 SET v_timestamp = null
> Ln:12 PRINT
> date:
> Ln:13 PRINT
> timestamp:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21453) HPL/SQL can not SELECT Date and Timestamp type value into variable
[
https://issues.apache.org/jira/browse/HIVE-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DingQiang Liu updated HIVE-21453:
-
Attachment: (was: HIVE-21453.patch)
> HPL/SQL can not SELECT Date and Timestamp type value into variable
> ---
>
> Key: HIVE-21453
> URL: https://issues.apache.org/jira/browse/HIVE-21453
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Affects Versions: 3.1.1
> Environment: Centos 7.1, Hive 3.1.1
>Reporter: DingQiang Liu
>Assignee: DingQiang Liu
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21453.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> HPL/SQL forgot Date and Timestamp types when SELECT INTO variables. for
> example, current implement will set null, not correct value, to variables
> for following case, select_into3.sql:
> declare v_date date;
> declare v_timestamp timestamp(17, 3);
> select
> cast('2019-02-20 12:23:45.678' as date),
> cast('2019-02-20 12:23:45.678' as timestamp)
> into
> v_date,
> v_timestamp
> from src limit 1;
> print 'date: ' || v_date;
> print 'timestamp: ' || v_timestamp;
> The result when running : bin/hplsql -f select_into3.sql --trace
> Configuration file: file:/hive/conf/hplsql-site.xml
> Parser tree: (program (block (stmt (declare_stmt declare (declare_stmt_item
> (declare_var_item (ident v_date) (dtype date) (stmt (semicolon_stmt ;))
> (stmt (declare_stmt declare (declare_stmt_item (declare_var_item (ident
> v_timestamp) (dtype timestamp) (dtype_len ( 17 , 3 )) (stmt
> (semicolon_stmt ;)) (stmt (select_stmt (fullselect_stmt (fullselect_stmt_item
> (subselect_stmt select (select_list (select_list_item (expr (expr_spec_func
> cast ( (expr (expr_atom (string '2019-02-20 12:23:45.678'))) as (dtype date)
> , (select_list_item (expr (expr_spec_func cast ( (expr (expr_atom
> (string '2019-02-20 12:23:45.678'))) as (dtype timestamp) ) (into_clause
> into (ident v_date) , (ident v_timestamp)) (from_clause from
> (from_table_clause (from_table_name_clause (table_name (ident src)
> (select_options (select_options_item limit (expr (expr_atom (int_number
> 1)) (stmt (semicolon_stmt ;)) (stmt (print_stmt print (expr
> (expr_concat (expr_concat_item (expr_atom (string 'date: '))) ||
> (expr_concat_item (expr_atom (ident v_date))) (stmt (semicolon_stmt ;))
> (stmt (print_stmt print (expr (expr_concat (expr_concat_item (expr_atom
> (string 'timestamp: '))) || (expr_concat_item (expr_atom (ident
> v_timestamp))) (stmt (semicolon_stmt ;))) )
> Ln:1 DECLARE v_date date
> Ln:2 DECLARE v_timestamp timestamp
> Ln:4 SELECT
> Ln:4 select
> cast('2019-02-20 12:23:45.678' as date), cast('2019-02-20 12:23:45.678' as
> timestamp)
> from src LIMIT 1
> Open connection: jdbc:vertica://v001:5433/test (256 ms)
> Starting query
> Query executed successfully (55 ms)
> Ln:4 SELECT completed successfully
> Ln:4 SELECT INTO statement executed
> Ln:4 COLUMN: ?column?, Date
> Ln:4 SET v_date = null
> Ln:4 COLUMN: ?column?, Timestamp
> Ln:4 SET v_timestamp = null
> Ln:12 PRINT
> date:
> Ln:13 PRINT
> timestamp:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (HIVE-21453) HPL/SQL can not SELECT Date and Timestamp type value into variable
[
https://issues.apache.org/jira/browse/HIVE-21453?focusedWorklogId=213682=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-213682
]
ASF GitHub Bot logged work on HIVE-21453:
-
Author: ASF GitHub Bot
Created on: 15/Mar/19 10:40
Start Date: 15/Mar/19 10:40
Worklog Time Spent: 10m
Work Description: rmsmani commented on issue #566: HIVE-21453: HPL/SQL
can not SELECT Date and Timestamp type value INTO variable
URL: https://github.com/apache/hive/pull/566#issuecomment-473238518
@dingqiangliu
In Jira there will be a button **Submit Patch**
Once the patch is submitted, the build will get triggered automatically and
the results will be published to the JIRA ticket automatically.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 213682)
Time Spent: 20m (was: 10m)
> HPL/SQL can not SELECT Date and Timestamp type value into variable
> ---
>
> Key: HIVE-21453
> URL: https://issues.apache.org/jira/browse/HIVE-21453
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Affects Versions: 3.1.1
> Environment: Centos 7.1, Hive 3.1.1
>Reporter: DingQiang Liu
>Assignee: DingQiang Liu
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21453.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> HPL/SQL forgot Date and Timestamp types when SELECT INTO variables. for
> example, current implement will set null, not correct value, to variables
> for following case, select_into3.sql:
> declare v_date date;
> declare v_timestamp timestamp(17, 3);
> select
> cast('2019-02-20 12:23:45.678' as date),
> cast('2019-02-20 12:23:45.678' as timestamp)
> into
> v_date,
> v_timestamp
> from src limit 1;
> print 'date: ' || v_date;
> print 'timestamp: ' || v_timestamp;
> The result when running : bin/hplsql -f select_into3.sql --trace
> Configuration file: file:/hive/conf/hplsql-site.xml
> Parser tree: (program (block (stmt (declare_stmt declare (declare_stmt_item
> (declare_var_item (ident v_date) (dtype date) (stmt (semicolon_stmt ;))
> (stmt (declare_stmt declare (declare_stmt_item (declare_var_item (ident
> v_timestamp) (dtype timestamp) (dtype_len ( 17 , 3 )) (stmt
> (semicolon_stmt ;)) (stmt (select_stmt (fullselect_stmt (fullselect_stmt_item
> (subselect_stmt select (select_list (select_list_item (expr (expr_spec_func
> cast ( (expr (expr_atom (string '2019-02-20 12:23:45.678'))) as (dtype date)
> , (select_list_item (expr (expr_spec_func cast ( (expr (expr_atom
> (string '2019-02-20 12:23:45.678'))) as (dtype timestamp) ) (into_clause
> into (ident v_date) , (ident v_timestamp)) (from_clause from
> (from_table_clause (from_table_name_clause (table_name (ident src)
> (select_options (select_options_item limit (expr (expr_atom (int_number
> 1)) (stmt (semicolon_stmt ;)) (stmt (print_stmt print (expr
> (expr_concat (expr_concat_item (expr_atom (string 'date: '))) ||
> (expr_concat_item (expr_atom (ident v_date))) (stmt (semicolon_stmt ;))
> (stmt (print_stmt print (expr (expr_concat (expr_concat_item (expr_atom
> (string 'timestamp: '))) || (expr_concat_item (expr_atom (ident
> v_timestamp))) (stmt (semicolon_stmt ;))) )
> Ln:1 DECLARE v_date date
> Ln:2 DECLARE v_timestamp timestamp
> Ln:4 SELECT
> Ln:4 select
> cast('2019-02-20 12:23:45.678' as date), cast('2019-02-20 12:23:45.678' as
> timestamp)
> from src LIMIT 1
> Open connection: jdbc:vertica://v001:5433/test (256 ms)
> Starting query
> Query executed successfully (55 ms)
> Ln:4 SELECT completed successfully
> Ln:4 SELECT INTO statement executed
> Ln:4 COLUMN: ?column?, Date
> Ln:4 SET v_date = null
> Ln:4 COLUMN: ?column?, Timestamp
> Ln:4 SET v_timestamp = null
> Ln:12 PRINT
> date:
> Ln:13 PRINT
> timestamp:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HIVE-21453) HPL/SQL can not SELECT Date and Timestamp type value into variable
[
https://issues.apache.org/jira/browse/HIVE-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DingQiang Liu updated HIVE-21453:
-
Comment: was deleted
(was: PR is here: https://github.com/apache/hive/pull/566 )
> HPL/SQL can not SELECT Date and Timestamp type value into variable
> ---
>
> Key: HIVE-21453
> URL: https://issues.apache.org/jira/browse/HIVE-21453
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Affects Versions: 3.1.1
> Environment: Centos 7.1, Hive 3.1.1
>Reporter: DingQiang Liu
>Assignee: DingQiang Liu
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21453.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HPL/SQL forgot Date and Timestamp types when SELECT INTO variables. for
> example, current implement will set null, not correct value, to variables
> for following case, select_into3.sql:
> declare v_date date;
> declare v_timestamp timestamp(17, 3);
> select
> cast('2019-02-20 12:23:45.678' as date),
> cast('2019-02-20 12:23:45.678' as timestamp)
> into
> v_date,
> v_timestamp
> from src limit 1;
> print 'date: ' || v_date;
> print 'timestamp: ' || v_timestamp;
> The result when running : bin/hplsql -f select_into3.sql --trace
> Configuration file: file:/hive/conf/hplsql-site.xml
> Parser tree: (program (block (stmt (declare_stmt declare (declare_stmt_item
> (declare_var_item (ident v_date) (dtype date) (stmt (semicolon_stmt ;))
> (stmt (declare_stmt declare (declare_stmt_item (declare_var_item (ident
> v_timestamp) (dtype timestamp) (dtype_len ( 17 , 3 )) (stmt
> (semicolon_stmt ;)) (stmt (select_stmt (fullselect_stmt (fullselect_stmt_item
> (subselect_stmt select (select_list (select_list_item (expr (expr_spec_func
> cast ( (expr (expr_atom (string '2019-02-20 12:23:45.678'))) as (dtype date)
> , (select_list_item (expr (expr_spec_func cast ( (expr (expr_atom
> (string '2019-02-20 12:23:45.678'))) as (dtype timestamp) ) (into_clause
> into (ident v_date) , (ident v_timestamp)) (from_clause from
> (from_table_clause (from_table_name_clause (table_name (ident src)
> (select_options (select_options_item limit (expr (expr_atom (int_number
> 1)) (stmt (semicolon_stmt ;)) (stmt (print_stmt print (expr
> (expr_concat (expr_concat_item (expr_atom (string 'date: '))) ||
> (expr_concat_item (expr_atom (ident v_date))) (stmt (semicolon_stmt ;))
> (stmt (print_stmt print (expr (expr_concat (expr_concat_item (expr_atom
> (string 'timestamp: '))) || (expr_concat_item (expr_atom (ident
> v_timestamp))) (stmt (semicolon_stmt ;))) )
> Ln:1 DECLARE v_date date
> Ln:2 DECLARE v_timestamp timestamp
> Ln:4 SELECT
> Ln:4 select
> cast('2019-02-20 12:23:45.678' as date), cast('2019-02-20 12:23:45.678' as
> timestamp)
> from src LIMIT 1
> Open connection: jdbc:vertica://v001:5433/test (256 ms)
> Starting query
> Query executed successfully (55 ms)
> Ln:4 SELECT completed successfully
> Ln:4 SELECT INTO statement executed
> Ln:4 COLUMN: ?column?, Date
> Ln:4 SET v_date = null
> Ln:4 COLUMN: ?column?, Timestamp
> Ln:4 SET v_timestamp = null
> Ln:12 PRINT
> date:
> Ln:13 PRINT
> timestamp:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21453) HPL/SQL can not SELECT Date and Timestamp type value into variable
[
https://issues.apache.org/jira/browse/HIVE-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DingQiang Liu updated HIVE-21453:
-
Attachment: HIVE-21453.patch
> HPL/SQL can not SELECT Date and Timestamp type value into variable
> ---
>
> Key: HIVE-21453
> URL: https://issues.apache.org/jira/browse/HIVE-21453
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Affects Versions: 3.1.1
> Environment: Centos 7.1, Hive 3.1.1
>Reporter: DingQiang Liu
>Assignee: DingQiang Liu
>Priority: Minor
> Attachments: HIVE-21453.patch
>
>
> HPL/SQL forgot Date and Timestamp types when SELECT INTO variables. for
> example, current implement will set null, not correct value, to variables
> for following case, select_into3.sql:
> declare v_date date;
> declare v_timestamp timestamp(17, 3);
> select
> cast('2019-02-20 12:23:45.678' as date),
> cast('2019-02-20 12:23:45.678' as timestamp)
> into
> v_date,
> v_timestamp
> from src limit 1;
> print 'date: ' || v_date;
> print 'timestamp: ' || v_timestamp;
> The result when running : bin/hplsql -f select_into3.sql --trace
> Configuration file: file:/hive/conf/hplsql-site.xml
> Parser tree: (program (block (stmt (declare_stmt declare (declare_stmt_item
> (declare_var_item (ident v_date) (dtype date) (stmt (semicolon_stmt ;))
> (stmt (declare_stmt declare (declare_stmt_item (declare_var_item (ident
> v_timestamp) (dtype timestamp) (dtype_len ( 17 , 3 )) (stmt
> (semicolon_stmt ;)) (stmt (select_stmt (fullselect_stmt (fullselect_stmt_item
> (subselect_stmt select (select_list (select_list_item (expr (expr_spec_func
> cast ( (expr (expr_atom (string '2019-02-20 12:23:45.678'))) as (dtype date)
> , (select_list_item (expr (expr_spec_func cast ( (expr (expr_atom
> (string '2019-02-20 12:23:45.678'))) as (dtype timestamp) ) (into_clause
> into (ident v_date) , (ident v_timestamp)) (from_clause from
> (from_table_clause (from_table_name_clause (table_name (ident src)
> (select_options (select_options_item limit (expr (expr_atom (int_number
> 1)) (stmt (semicolon_stmt ;)) (stmt (print_stmt print (expr
> (expr_concat (expr_concat_item (expr_atom (string 'date: '))) ||
> (expr_concat_item (expr_atom (ident v_date))) (stmt (semicolon_stmt ;))
> (stmt (print_stmt print (expr (expr_concat (expr_concat_item (expr_atom
> (string 'timestamp: '))) || (expr_concat_item (expr_atom (ident
> v_timestamp))) (stmt (semicolon_stmt ;))) )
> Ln:1 DECLARE v_date date
> Ln:2 DECLARE v_timestamp timestamp
> Ln:4 SELECT
> Ln:4 select
> cast('2019-02-20 12:23:45.678' as date), cast('2019-02-20 12:23:45.678' as
> timestamp)
> from src LIMIT 1
> Open connection: jdbc:vertica://v001:5433/test (256 ms)
> Starting query
> Query executed successfully (55 ms)
> Ln:4 SELECT completed successfully
> Ln:4 SELECT INTO statement executed
> Ln:4 COLUMN: ?column?, Date
> Ln:4 SET v_date = null
> Ln:4 COLUMN: ?column?, Timestamp
> Ln:4 SET v_timestamp = null
> Ln:12 PRINT
> date:
> Ln:13 PRINT
> timestamp:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21453) HPL/SQL can not SELECT Date and Timestamp type value into variable
[
https://issues.apache.org/jira/browse/HIVE-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793519#comment-16793519
]
DingQiang Liu commented on HIVE-21453:
--
PR is here: https://github.com/apache/hive/pull/566
> HPL/SQL can not SELECT Date and Timestamp type value into variable
> ---
>
> Key: HIVE-21453
> URL: https://issues.apache.org/jira/browse/HIVE-21453
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Affects Versions: 3.1.1
> Environment: Centos 7.1, Hive 3.1.1
>Reporter: DingQiang Liu
>Assignee: DingQiang Liu
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21453.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HPL/SQL forgot Date and Timestamp types when SELECT INTO variables. for
> example, current implement will set null, not correct value, to variables
> for following case, select_into3.sql:
> declare v_date date;
> declare v_timestamp timestamp(17, 3);
> select
> cast('2019-02-20 12:23:45.678' as date),
> cast('2019-02-20 12:23:45.678' as timestamp)
> into
> v_date,
> v_timestamp
> from src limit 1;
> print 'date: ' || v_date;
> print 'timestamp: ' || v_timestamp;
> The result when running : bin/hplsql -f select_into3.sql --trace
> Configuration file: file:/hive/conf/hplsql-site.xml
> Parser tree: (program (block (stmt (declare_stmt declare (declare_stmt_item
> (declare_var_item (ident v_date) (dtype date) (stmt (semicolon_stmt ;))
> (stmt (declare_stmt declare (declare_stmt_item (declare_var_item (ident
> v_timestamp) (dtype timestamp) (dtype_len ( 17 , 3 )) (stmt
> (semicolon_stmt ;)) (stmt (select_stmt (fullselect_stmt (fullselect_stmt_item
> (subselect_stmt select (select_list (select_list_item (expr (expr_spec_func
> cast ( (expr (expr_atom (string '2019-02-20 12:23:45.678'))) as (dtype date)
> , (select_list_item (expr (expr_spec_func cast ( (expr (expr_atom
> (string '2019-02-20 12:23:45.678'))) as (dtype timestamp) ) (into_clause
> into (ident v_date) , (ident v_timestamp)) (from_clause from
> (from_table_clause (from_table_name_clause (table_name (ident src)
> (select_options (select_options_item limit (expr (expr_atom (int_number
> 1)) (stmt (semicolon_stmt ;)) (stmt (print_stmt print (expr
> (expr_concat (expr_concat_item (expr_atom (string 'date: '))) ||
> (expr_concat_item (expr_atom (ident v_date))) (stmt (semicolon_stmt ;))
> (stmt (print_stmt print (expr (expr_concat (expr_concat_item (expr_atom
> (string 'timestamp: '))) || (expr_concat_item (expr_atom (ident
> v_timestamp))) (stmt (semicolon_stmt ;))) )
> Ln:1 DECLARE v_date date
> Ln:2 DECLARE v_timestamp timestamp
> Ln:4 SELECT
> Ln:4 select
> cast('2019-02-20 12:23:45.678' as date), cast('2019-02-20 12:23:45.678' as
> timestamp)
> from src LIMIT 1
> Open connection: jdbc:vertica://v001:5433/test (256 ms)
> Starting query
> Query executed successfully (55 ms)
> Ln:4 SELECT completed successfully
> Ln:4 SELECT INTO statement executed
> Ln:4 COLUMN: ?column?, Date
> Ln:4 SET v_date = null
> Ln:4 COLUMN: ?column?, Timestamp
> Ln:4 SET v_timestamp = null
> Ln:12 PRINT
> date:
> Ln:13 PRINT
> timestamp:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (HIVE-21453) HPL/SQL can not SELECT Date and Timestamp type value into variable
[
https://issues.apache.org/jira/browse/HIVE-21453?focusedWorklogId=213676=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-213676
]
ASF GitHub Bot logged work on HIVE-21453:
-
Author: ASF GitHub Bot
Created on: 15/Mar/19 10:25
Start Date: 15/Mar/19 10:25
Worklog Time Spent: 10m
Work Description: dingqiangliu commented on issue #566: HIVE-21453:
HPL/SQL can not SELECT Date and Timestamp type value INTO variable
URL: https://github.com/apache/hive/pull/566#issuecomment-473234237
thank you @rmsmani !
it's done, please review it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 213676)
Time Spent: 10m
Remaining Estimate: 0h
> HPL/SQL can not SELECT Date and Timestamp type value into variable
> ---
>
> Key: HIVE-21453
> URL: https://issues.apache.org/jira/browse/HIVE-21453
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Affects Versions: 3.1.1
> Environment: Centos 7.1, Hive 3.1.1
>Reporter: DingQiang Liu
>Assignee: DingQiang Liu
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21453.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HPL/SQL forgot Date and Timestamp types when SELECT INTO variables. for
> example, current implement will set null, not correct value, to variables
> for following case, select_into3.sql:
> declare v_date date;
> declare v_timestamp timestamp(17, 3);
> select
> cast('2019-02-20 12:23:45.678' as date),
> cast('2019-02-20 12:23:45.678' as timestamp)
> into
> v_date,
> v_timestamp
> from src limit 1;
> print 'date: ' || v_date;
> print 'timestamp: ' || v_timestamp;
> The result when running : bin/hplsql -f select_into3.sql --trace
> Configuration file: file:/hive/conf/hplsql-site.xml
> Parser tree: (program (block (stmt (declare_stmt declare (declare_stmt_item
> (declare_var_item (ident v_date) (dtype date) (stmt (semicolon_stmt ;))
> (stmt (declare_stmt declare (declare_stmt_item (declare_var_item (ident
> v_timestamp) (dtype timestamp) (dtype_len ( 17 , 3 )) (stmt
> (semicolon_stmt ;)) (stmt (select_stmt (fullselect_stmt (fullselect_stmt_item
> (subselect_stmt select (select_list (select_list_item (expr (expr_spec_func
> cast ( (expr (expr_atom (string '2019-02-20 12:23:45.678'))) as (dtype date)
> , (select_list_item (expr (expr_spec_func cast ( (expr (expr_atom
> (string '2019-02-20 12:23:45.678'))) as (dtype timestamp) ) (into_clause
> into (ident v_date) , (ident v_timestamp)) (from_clause from
> (from_table_clause (from_table_name_clause (table_name (ident src)
> (select_options (select_options_item limit (expr (expr_atom (int_number
> 1)) (stmt (semicolon_stmt ;)) (stmt (print_stmt print (expr
> (expr_concat (expr_concat_item (expr_atom (string 'date: '))) ||
> (expr_concat_item (expr_atom (ident v_date))) (stmt (semicolon_stmt ;))
> (stmt (print_stmt print (expr (expr_concat (expr_concat_item (expr_atom
> (string 'timestamp: '))) || (expr_concat_item (expr_atom (ident
> v_timestamp))) (stmt (semicolon_stmt ;))) )
> Ln:1 DECLARE v_date date
> Ln:2 DECLARE v_timestamp timestamp
> Ln:4 SELECT
> Ln:4 select
> cast('2019-02-20 12:23:45.678' as date), cast('2019-02-20 12:23:45.678' as
> timestamp)
> from src LIMIT 1
> Open connection: jdbc:vertica://v001:5433/test (256 ms)
> Starting query
> Query executed successfully (55 ms)
> Ln:4 SELECT completed successfully
> Ln:4 SELECT INTO statement executed
> Ln:4 COLUMN: ?column?, Date
> Ln:4 SET v_date = null
> Ln:4 COLUMN: ?column?, Timestamp
> Ln:4 SET v_timestamp = null
> Ln:12 PRINT
> date:
> Ln:13 PRINT
> timestamp:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21453) HPL/SQL can not SELECT Date and Timestamp type value into variable
[
https://issues.apache.org/jira/browse/HIVE-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-21453:
--
Labels: pull-request-available (was: )
> HPL/SQL can not SELECT Date and Timestamp type value into variable
> ---
>
> Key: HIVE-21453
> URL: https://issues.apache.org/jira/browse/HIVE-21453
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Affects Versions: 3.1.1
> Environment: Centos 7.1, Hive 3.1.1
>Reporter: DingQiang Liu
>Assignee: DingQiang Liu
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21453.patch
>
>
> HPL/SQL forgot Date and Timestamp types when SELECT INTO variables. for
> example, current implement will set null, not correct value, to variables
> for following case, select_into3.sql:
> declare v_date date;
> declare v_timestamp timestamp(17, 3);
> select
> cast('2019-02-20 12:23:45.678' as date),
> cast('2019-02-20 12:23:45.678' as timestamp)
> into
> v_date,
> v_timestamp
> from src limit 1;
> print 'date: ' || v_date;
> print 'timestamp: ' || v_timestamp;
> The result when running : bin/hplsql -f select_into3.sql --trace
> Configuration file: file:/hive/conf/hplsql-site.xml
> Parser tree: (program (block (stmt (declare_stmt declare (declare_stmt_item
> (declare_var_item (ident v_date) (dtype date) (stmt (semicolon_stmt ;))
> (stmt (declare_stmt declare (declare_stmt_item (declare_var_item (ident
> v_timestamp) (dtype timestamp) (dtype_len ( 17 , 3 )) (stmt
> (semicolon_stmt ;)) (stmt (select_stmt (fullselect_stmt (fullselect_stmt_item
> (subselect_stmt select (select_list (select_list_item (expr (expr_spec_func
> cast ( (expr (expr_atom (string '2019-02-20 12:23:45.678'))) as (dtype date)
> , (select_list_item (expr (expr_spec_func cast ( (expr (expr_atom
> (string '2019-02-20 12:23:45.678'))) as (dtype timestamp) ) (into_clause
> into (ident v_date) , (ident v_timestamp)) (from_clause from
> (from_table_clause (from_table_name_clause (table_name (ident src)
> (select_options (select_options_item limit (expr (expr_atom (int_number
> 1)) (stmt (semicolon_stmt ;)) (stmt (print_stmt print (expr
> (expr_concat (expr_concat_item (expr_atom (string 'date: '))) ||
> (expr_concat_item (expr_atom (ident v_date))) (stmt (semicolon_stmt ;))
> (stmt (print_stmt print (expr (expr_concat (expr_concat_item (expr_atom
> (string 'timestamp: '))) || (expr_concat_item (expr_atom (ident
> v_timestamp))) (stmt (semicolon_stmt ;))) )
> Ln:1 DECLARE v_date date
> Ln:2 DECLARE v_timestamp timestamp
> Ln:4 SELECT
> Ln:4 select
> cast('2019-02-20 12:23:45.678' as date), cast('2019-02-20 12:23:45.678' as
> timestamp)
> from src LIMIT 1
> Open connection: jdbc:vertica://v001:5433/test (256 ms)
> Starting query
> Query executed successfully (55 ms)
> Ln:4 SELECT completed successfully
> Ln:4 SELECT INTO statement executed
> Ln:4 COLUMN: ?column?, Date
> Ln:4 SET v_date = null
> Ln:4 COLUMN: ?column?, Timestamp
> Ln:4 SET v_timestamp = null
> Ln:12 PRINT
> date:
> Ln:13 PRINT
> timestamp:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21453) HPL/SQL can not SELECT Date and Timestamp type value into variable
[
https://issues.apache.org/jira/browse/HIVE-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793516#comment-16793516
]
DingQiang Liu commented on HIVE-21453:
--
patch attached, including query hplsql/src/test/queries/db/select_into3.sql and
output hplsql/src/test/results/db/select_into3.out.txt for test case
> HPL/SQL can not SELECT Date and Timestamp type value into variable
> ---
>
> Key: HIVE-21453
> URL: https://issues.apache.org/jira/browse/HIVE-21453
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Affects Versions: 3.1.1
> Environment: Centos 7.1, Hive 3.1.1
>Reporter: DingQiang Liu
>Assignee: DingQiang Liu
>Priority: Minor
> Attachments: HIVE-21453.patch
>
>
> HPL/SQL forgot Date and Timestamp types when SELECT INTO variables. for
> example, current implement will set null, not correct value, to variables
> for following case, select_into3.sql:
> declare v_date date;
> declare v_timestamp timestamp(17, 3);
> select
> cast('2019-02-20 12:23:45.678' as date),
> cast('2019-02-20 12:23:45.678' as timestamp)
> into
> v_date,
> v_timestamp
> from src limit 1;
> print 'date: ' || v_date;
> print 'timestamp: ' || v_timestamp;
> The result when running : bin/hplsql -f select_into3.sql --trace
> Configuration file: file:/hive/conf/hplsql-site.xml
> Parser tree: (program (block (stmt (declare_stmt declare (declare_stmt_item
> (declare_var_item (ident v_date) (dtype date) (stmt (semicolon_stmt ;))
> (stmt (declare_stmt declare (declare_stmt_item (declare_var_item (ident
> v_timestamp) (dtype timestamp) (dtype_len ( 17 , 3 )) (stmt
> (semicolon_stmt ;)) (stmt (select_stmt (fullselect_stmt (fullselect_stmt_item
> (subselect_stmt select (select_list (select_list_item (expr (expr_spec_func
> cast ( (expr (expr_atom (string '2019-02-20 12:23:45.678'))) as (dtype date)
> , (select_list_item (expr (expr_spec_func cast ( (expr (expr_atom
> (string '2019-02-20 12:23:45.678'))) as (dtype timestamp) ) (into_clause
> into (ident v_date) , (ident v_timestamp)) (from_clause from
> (from_table_clause (from_table_name_clause (table_name (ident src)
> (select_options (select_options_item limit (expr (expr_atom (int_number
> 1)) (stmt (semicolon_stmt ;)) (stmt (print_stmt print (expr
> (expr_concat (expr_concat_item (expr_atom (string 'date: '))) ||
> (expr_concat_item (expr_atom (ident v_date))) (stmt (semicolon_stmt ;))
> (stmt (print_stmt print (expr (expr_concat (expr_concat_item (expr_atom
> (string 'timestamp: '))) || (expr_concat_item (expr_atom (ident
> v_timestamp))) (stmt (semicolon_stmt ;))) )
> Ln:1 DECLARE v_date date
> Ln:2 DECLARE v_timestamp timestamp
> Ln:4 SELECT
> Ln:4 select
> cast('2019-02-20 12:23:45.678' as date), cast('2019-02-20 12:23:45.678' as
> timestamp)
> from src LIMIT 1
> Open connection: jdbc:vertica://v001:5433/test (256 ms)
> Starting query
> Query executed successfully (55 ms)
> Ln:4 SELECT completed successfully
> Ln:4 SELECT INTO statement executed
> Ln:4 COLUMN: ?column?, Date
> Ln:4 SET v_date = null
> Ln:4 COLUMN: ?column?, Timestamp
> Ln:4 SET v_timestamp = null
> Ln:12 PRINT
> date:
> Ln:13 PRINT
> timestamp:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21453) HPL/SQL can not SELECT Date and Timestamp type value into variable
[
https://issues.apache.org/jira/browse/HIVE-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DingQiang Liu reassigned HIVE-21453:
> HPL/SQL can not SELECT Date and Timestamp type value into variable
> ---
>
> Key: HIVE-21453
> URL: https://issues.apache.org/jira/browse/HIVE-21453
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Affects Versions: 3.1.1
> Environment: Centos 7.1, Hive 3.1.1
>Reporter: DingQiang Liu
>Assignee: DingQiang Liu
>Priority: Minor
>
> HPL/SQL forgot Date and Timestamp types when SELECT INTO variables. for
> example, current implement will set null, not correct value, to variables
> for following case, select_into3.sql:
> declare v_date date;
> declare v_timestamp timestamp(17, 3);
> select
> cast('2019-02-20 12:23:45.678' as date),
> cast('2019-02-20 12:23:45.678' as timestamp)
> into
> v_date,
> v_timestamp
> from src limit 1;
> print 'date: ' || v_date;
> print 'timestamp: ' || v_timestamp;
> The result when running : bin/hplsql -f select_into3.sql --trace
> Configuration file: file:/hive/conf/hplsql-site.xml
> Parser tree: (program (block (stmt (declare_stmt declare (declare_stmt_item
> (declare_var_item (ident v_date) (dtype date) (stmt (semicolon_stmt ;))
> (stmt (declare_stmt declare (declare_stmt_item (declare_var_item (ident
> v_timestamp) (dtype timestamp) (dtype_len ( 17 , 3 )) (stmt
> (semicolon_stmt ;)) (stmt (select_stmt (fullselect_stmt (fullselect_stmt_item
> (subselect_stmt select (select_list (select_list_item (expr (expr_spec_func
> cast ( (expr (expr_atom (string '2019-02-20 12:23:45.678'))) as (dtype date)
> , (select_list_item (expr (expr_spec_func cast ( (expr (expr_atom
> (string '2019-02-20 12:23:45.678'))) as (dtype timestamp) ) (into_clause
> into (ident v_date) , (ident v_timestamp)) (from_clause from
> (from_table_clause (from_table_name_clause (table_name (ident src)
> (select_options (select_options_item limit (expr (expr_atom (int_number
> 1)) (stmt (semicolon_stmt ;)) (stmt (print_stmt print (expr
> (expr_concat (expr_concat_item (expr_atom (string 'date: '))) ||
> (expr_concat_item (expr_atom (ident v_date))) (stmt (semicolon_stmt ;))
> (stmt (print_stmt print (expr (expr_concat (expr_concat_item (expr_atom
> (string 'timestamp: '))) || (expr_concat_item (expr_atom (ident
> v_timestamp))) (stmt (semicolon_stmt ;))) )
> Ln:1 DECLARE v_date date
> Ln:2 DECLARE v_timestamp timestamp
> Ln:4 SELECT
> Ln:4 select
> cast('2019-02-20 12:23:45.678' as date), cast('2019-02-20 12:23:45.678' as
> timestamp)
> from src LIMIT 1
> Open connection: jdbc:vertica://v001:5433/test (256 ms)
> Starting query
> Query executed successfully (55 ms)
> Ln:4 SELECT completed successfully
> Ln:4 SELECT INTO statement executed
> Ln:4 COLUMN: ?column?, Date
> Ln:4 SET v_date = null
> Ln:4 COLUMN: ?column?, Timestamp
> Ln:4 SET v_timestamp = null
> Ln:12 PRINT
> date:
> Ln:13 PRINT
> timestamp:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21430) INSERT into a dynamically partitioned table with hive.stats.autogather = false throws a MetaException
[
https://issues.apache.org/jira/browse/HIVE-21430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Bapat updated HIVE-21430:
--
Fix Version/s: 4.0.0
Attachment: HIVE-21430.01.patch
Status: Patch Available (was: Open)
loadDynamicPartitions is not passing a valid writeId list while altering
multiple partitions. It's
also fetching table snapshot separately for each of the partitions. Instead
fetch the snapshot onceto be used for all partitions. Use the same snapshot to
alter partitions.
> INSERT into a dynamically partitioned table with hive.stats.autogather =
> false throws a MetaException
> -
>
> Key: HIVE-21430
> URL: https://issues.apache.org/jira/browse/HIVE-21430
> Project: Hive
> Issue Type: Bug
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21430.01.patch, metaexception_repro.patch,
> org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread-output.txt
>
> Original Estimate: 48h
> Time Spent: 10m
> Remaining Estimate: 47h 50m
>
> When the test TestStatsUpdaterThread#testTxnDynamicPartitions added in the
> attached patch is run it throws exception (full logs attached.)
> org.apache.hadoop.hive.metastore.api.MetaException: Cannot change stats state
> for a transactional table default.simple_stats without providing the
> transactional write state for verification (new write ID 5, valid write IDs
> null; current state \{"BASIC_STATS":"true","COLUMN_STATS":{"s":"true"}}; new
> state null
> at
> org.apache.hadoop.hive.metastore.ObjectStore.alterPartitionNoTxn(ObjectStore.java:4328)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
