[jira] [Commented] (HIVE-19892) Disable query results cache for for HiveServer2 doAs=true
[
https://issues.apache.org/jira/browse/HIVE-19892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515002#comment-16515002
]
Hive QA commented on HIVE-19892:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927912/HIVE-19892.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11842/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11842/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11842/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12927912/HIVE-19892.2.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927912 - PreCommit-HIVE-Build
> Disable query results cache for for HiveServer2 doAs=true
> -
>
> Key: HIVE-19892
> URL: https://issues.apache.org/jira/browse/HIVE-19892
> Project: Hive
> Issue Type: Sub-task
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19892.1.patch, HIVE-19892.2.patch
>
>
> If running HS2 with doAs=true, the temp query results directory will have
> ownership/permissions based on the doAs user. A subsequent query running as a
> different user may not be able to access this query results directory.
> Results caching will have to be disabled in this case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file
[
https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515001#comment-16515001
]
Hive QA commented on HIVE-18118:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927900/HIVE-18118.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11841/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11841/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11841/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-06-17 05:28:01.554
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-11841/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-06-17 05:28:01.558
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 3eaca1f HIVE-19366: Vectorization causing
TestStreaming.testStreamBucketingMatchesRegularBucketing to fail (Prasanth
Jayachandran reviewed by Eugene Koifman)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 3eaca1f HIVE-19366: Vectorization causing
TestStreaming.testStreamBucketingMatchesRegularBucketing to fail (Prasanth
Jayachandran reviewed by Eugene Koifman)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-06-17 05:28:02.775
+ rm -rf ../yetus_PreCommit-HIVE-Build-11841
+ mkdir ../yetus_PreCommit-HIVE-Build-11841
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-11841
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-11841/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/test/results/clientpositive/llap/materialized_view_create.q.out:48
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/llap/materialized_view_create.q.out'
cleanly.
error: patch failed:
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_5.q.out:400
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_5.q.out'
with conflicts.
error: patch failed:
ql/src/test/results/clientpositive/llap/materialized_view_describe.q.out:71
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/llap/materialized_view_describe.q.out'
cleanly.
error: patch failed: ql/src/test/results/clientpositive/llap/mm_exim.q.out:311
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/llap/mm_exim.q.out' with
conflicts.
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:20: trailing whitespace.
numFilesErasureCoded0
/data/hiveptest/working/scratch/build.patch:32: trailing whitespace.
numFilesErasureCoded0
/data/hiveptest/working/scratch/build.patch:40: trailing whitespace.
numFilesErasureCoded0
/data/hiveptest/working/scratch/build.patch:52: trailing whitespace.
numFilesErasureCoded0
/data/hiveptest/working/scratch/build.patch:60: trailing whitespace.
numFilesErasureCoded0
error: patch failed:
ql/src/test/results/clientpositive/llap/materialized_view_create.q.out:48
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/llap/materialized_view_create.q.out'
cleanly.
error: patch failed:
[jira] [Commented] (HIVE-19753) Strict managed tables mode in Hive
[
https://issues.apache.org/jira/browse/HIVE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515000#comment-16515000
]
Hive QA commented on HIVE-19753:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927899/HIVE-19753.4.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11840/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11840/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11840/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12927899/HIVE-19753.4.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927899 - PreCommit-HIVE-Build
> Strict managed tables mode in Hive
> --
>
> Key: HIVE-19753
> URL: https://issues.apache.org/jira/browse/HIVE-19753
> Project: Hive
> Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19753.1.patch, HIVE-19753.2.patch,
> HIVE-19753.3.patch, HIVE-19753.4.patch
>
>
> Create a mode in Hive which enforces that all managed tables are
> transactional (both full or insert-only tables allowed). Non-transactional
> tables, as well as non-native tables, must be created as external tables when
> this mode is enabled.
> The idea would be that in strict managed tables mode all of the data written
> to managed tables would have been done through Hive.
> The mode would be enabled using config setting hive.strict.managed.tables.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-15190) Field names are not preserved in ORC files written with ACID
[
https://issues.apache.org/jira/browse/HIVE-15190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514999#comment-16514999
]
Hive QA commented on HIVE-15190:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927894/HIVE-15190.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 14541 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_nullscan]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=81)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_vectorization_original]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization_acid]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_3]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_4]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_5]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_rebuild_dummy]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[acid_vectorization_original_tez]
(batchId=106)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.testTableProperties[0]
(batchId=234)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.testTableProperties[1]
(batchId=234)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11839/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11839/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11839/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927894 - PreCommit-HIVE-Build
> Field names are not preserved in ORC files written with ACID
>
>
> Key: HIVE-15190
> URL: https://issues.apache.org/jira/browse/HIVE-15190
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0, 2.2.0, 3.1.0, 4.0.0
>Reporter: Anthony Hsu
>Assignee: Prasanth Jayachandran
>Priority: Critical
> Attachments: HIVE-15190.1.patch, HIVE-15190.2.patch,
> HIVE-15190.3.patch
>
>
> To repro:
> {noformat}
> drop table if exists orc_nonacid;
> drop table if exists orc_acid;
> create table orc_nonacid (a int) clustered by (a) into 2 buckets stored as
> orc;
> create table orc_acid (a int) clustered by (a) into 2 buckets stored as orc
> TBLPROPERTIES('transactional'='true');
> insert into table orc_nonacid values(1), (2);
> insert into table orc_acid values(1), (2);
> {noformat}
> Running {{hive --service orcfiledump }} on the files created by the
> {{insert}} statements above, you'll see that for {{orc_nonacid}}, the files
> have schema {{struct}} whereas for {{orc_acid}}, the files have schema
> {{struct>}}.
> The last field {{row}} should have schema {{struct}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-15190) Field names are not preserved in ORC files written with ACID
[
https://issues.apache.org/jira/browse/HIVE-15190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514989#comment-16514989
]
Hive QA commented on HIVE-15190:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
3s{color} | {color:blue} ql in master has 2281 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 13 new + 660 unchanged - 0
fixed = 673 total (was 660) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
56s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11839/dev-support/hive-personality.sh
|
| git revision | master / 3eaca1f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11839/yetus/diff-checkstyle-ql.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11839/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11839/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Field names are not preserved in ORC files written with ACID
>
>
> Key: HIVE-15190
> URL: https://issues.apache.org/jira/browse/HIVE-15190
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0, 2.2.0, 3.1.0, 4.0.0
>Reporter: Anthony Hsu
>Assignee: Prasanth Jayachandran
>Priority: Critical
> Attachments: HIVE-15190.1.patch, HIVE-15190.2.patch,
> HIVE-15190.3.patch
>
>
> To repro:
> {noformat}
> drop table if exists orc_nonacid;
> drop table if exists orc_acid;
> create table orc_nonacid (a int) clustered by (a) into 2 buckets stored as
> orc;
> create table orc_acid (a int) clustered by (a) into 2 buckets stored as orc
> TBLPROPERTIES('transactional'='true');
> insert into table orc_nonacid values(1), (2);
> insert into table orc_acid values(1), (2);
> {noformat}
> Running {{hive --service orcfiledump }} on the files created by the
> {{insert}} statements above, you'll see that for {{orc_nonacid}}, the files
> have schema {{struct}} whereas for {{orc_acid}}, the files have schema
> {{struct>}}.
> The last field {{row}} should have schema {{struct}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19812) Disable external table replication by default via a configuration property
[
https://issues.apache.org/jira/browse/HIVE-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera updated HIVE-19812:
---
Attachment: HIVE-19812.05.patch
> Disable external table replication by default via a configuration property
> --
>
> Key: HIVE-19812
> URL: https://issues.apache.org/jira/browse/HIVE-19812
> Project: Hive
> Issue Type: Task
> Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19812.01.patch, HIVE-19812.02.patch,
> HIVE-19812.03.patch, HIVE-19812.04.patch, HIVE-19812.05.patch
>
>
> use a hive config property to allow external table replication. set this
> property by default to prevent external table replication.
> for metadata only hive repl always export metadata for external tables.
>
> REPL_DUMP_EXTERNAL_TABLES("hive.repl.dump.include.external.tables", false,
> "Indicates if repl dump should include information about external tables. It
> should be \n"
> + "used in conjunction with 'hive.repl.dump.metadata.only' set to false. if
> 'hive.repl.dump.metadata.only' \n"
> + " is set to true then this config parameter has no effect as external table
> meta data is flushed \n"
> + " always by default.")
> This should be done for only replication dump and not for export
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-14249) Add simple materialized views with manual rebuilds
[ https://issues.apache.org/jira/browse/HIVE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-14249: -- Labels: TODOC2.3 (was: TODOC2.2) > Add simple materialized views with manual rebuilds > -- > > Key: HIVE-14249 > URL: https://issues.apache.org/jira/browse/HIVE-14249 > Project: Hive > Issue Type: New Feature > Components: Materialized views, Parser >Reporter: Alan Gates >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: TODOC2.3 > Fix For: 2.3.0 > > Attachments: HIVE-10459.2.patch, HIVE-14249.05.patch, > HIVE-14249.06.patch > > > This patch is a start at implementing simple views. It doesn't have enough > testing yet (e.g. there's no negative testing). And I know it fails in the > partitioned case. I suspect things like security and locking don't work > properly yet either. But I'm posting it as a starting point. > In this initial patch I'm just handling simple materialized views with manual > rebuilds. In later JIRAs we can add features such as allowing the optimizer > to rewrite queries to use materialized views rather than tables named in the > queries, giving the optimizer the ability to determine when a materialized > view is stale, etc. > Also, I didn't rebase this patch against trunk after the migration from > svn->git so it may not apply cleanly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19898) Disable TransactionalValidationListener when the table is not in the Hive catalog
[
https://issues.apache.org/jira/browse/HIVE-19898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514984#comment-16514984
]
Hive QA commented on HIVE-19898:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927893/HIVE-19898.1.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14541 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11838/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11838/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11838/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927893 - PreCommit-HIVE-Build
> Disable TransactionalValidationListener when the table is not in the Hive
> catalog
> -
>
> Key: HIVE-19898
> URL: https://issues.apache.org/jira/browse/HIVE-19898
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Standalone Metastore, Transactions
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19898.1.patch
>
>
> The TransactionalValidationListener does validation of tables specified as
> transactional tables, as well as enforcing create.as.acid. While this can be
> useful to Hive, this may not be useful to other catalogs which do not support
> transactional tables, and would not benefit from being automatically tagged
> as a transactional table. This should be changed so the
> TransactionalValidationListener does not run for non-hive catalogs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera reassigned HIVE-19924: -- > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 3.1.0, 4.0.0 > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn tagged with the Query_id. > * To get around the additional failure point as mentioned above. The jobs > can be tagged with an additional unique tag_id provided by Beacon in the WITH > clause in repl load command to be used to tag distcp jobs ). Enhance the kill > api to take the tag as input and kill jobs associated with that tag. Problem > here is how do we validate the association of the tag with a hive query id to > make sure this api is not used to kill jobs run by other components, however > we can provide this capability to only admins and should be ok in that case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19103) Nested structure Projection Push Down in Hive with ORC
[ https://issues.apache.org/jira/browse/HIVE-19103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Sharma updated HIVE-19103: - Attachment: HIVE-19103.2.patch Status: Patch Available (was: Open) > Nested structure Projection Push Down in Hive with ORC > -- > > Key: HIVE-19103 > URL: https://issues.apache.org/jira/browse/HIVE-19103 > Project: Hive > Issue Type: Improvement > Components: Hive, ORC >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-19103.2.patch, HIVE-19103.patch > > > Reading required columns only in nested structure schema > Example - > *Current state* - > Schema - struct,g:string>> > Query - select c.e.f from t where c.e.f > 10; > Current state - read entire c struct from the file and then filter because > "hive.io.file.readcolumn.ids" is referred due to which all the children > column are select to read from the file. > Conf - > _hive.io.file.readcolumn.ids = "2" > hive.io.file.readNestedColumn.paths = "c.e.f"_ > Result - > boolean[ ] include = [true,false,false,true,true,true,true,true] > *Expected state* - > Schema - struct,g:string>> > Query - select c.e.f from t where c.e.f > 10; > Expected state - instead of reading entire c struct from the file just read > only the f column by referring the " hive.io.file.readNestedColumn.paths". > Conf - > _hive.io.file.readcolumn.ids = "2" > hive.io.file.readNestedColumn.paths = "c.e.f"_ > Result - > boolean[ ] include = [true,false,false,true,false,true,true,false] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19103) Nested structure Projection Push Down in Hive with ORC
[ https://issues.apache.org/jira/browse/HIVE-19103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Sharma updated HIVE-19103: - Status: Open (was: Patch Available) > Nested structure Projection Push Down in Hive with ORC > -- > > Key: HIVE-19103 > URL: https://issues.apache.org/jira/browse/HIVE-19103 > Project: Hive > Issue Type: Improvement > Components: Hive, ORC >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-19103.patch > > > Reading required columns only in nested structure schema > Example - > *Current state* - > Schema - struct,g:string>> > Query - select c.e.f from t where c.e.f > 10; > Current state - read entire c struct from the file and then filter because > "hive.io.file.readcolumn.ids" is referred due to which all the children > column are select to read from the file. > Conf - > _hive.io.file.readcolumn.ids = "2" > hive.io.file.readNestedColumn.paths = "c.e.f"_ > Result - > boolean[ ] include = [true,false,false,true,true,true,true,true] > *Expected state* - > Schema - struct,g:string>> > Query - select c.e.f from t where c.e.f > 10; > Expected state - instead of reading entire c struct from the file just read > only the f column by referring the " hive.io.file.readNestedColumn.paths". > Conf - > _hive.io.file.readcolumn.ids = "2" > hive.io.file.readNestedColumn.paths = "c.e.f"_ > Result - > boolean[ ] include = [true,false,false,true,false,true,true,false] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19366) Vectorization causing TestStreaming.testStreamBucketingMatchesRegularBucketing to fail
[ https://issues.apache.org/jira/browse/HIVE-19366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19366: - Resolution: Fixed Fix Version/s: 4.0.0 3.1.0 Status: Resolved (was: Patch Available) Committed to branch-3 and master. > Vectorization causing > TestStreaming.testStreamBucketingMatchesRegularBucketing to fail > -- > > Key: HIVE-19366 > URL: https://issues.apache.org/jira/browse/HIVE-19366 > Project: Hive > Issue Type: Sub-task > Components: Streaming >Affects Versions: 3.1.0, 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19366.1.patch > > > Disabled vectorization for > TestStreaming#testStreamBucketingMatchesRegularBucketing test case in > HIVE-19211 as it is giving incorrect results (the issue is mostly related to > wrong table directory location which returns 0 splits). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19898) Disable TransactionalValidationListener when the table is not in the Hive catalog
[
https://issues.apache.org/jira/browse/HIVE-19898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514982#comment-16514982
]
Hive QA commented on HIVE-19898:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
2s{color} | {color:blue} standalone-metastore in master has 227 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
36s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} itests/hive-unit: The patch generated 7 new + 0
unchanged - 0 fixed = 7 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} standalone-metastore: The patch generated 1 new + 20
unchanged - 0 fixed = 21 total (was 20) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11838/dev-support/hive-personality.sh
|
| git revision | master / b100483 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11838/yetus/patch-mvninstall-itests_hive-unit.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11838/yetus/diff-checkstyle-itests_hive-unit.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11838/yetus/diff-checkstyle-standalone-metastore.txt
|
| modules | C: itests/hive-unit standalone-metastore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11838/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Disable TransactionalValidationListener when the table is not in the Hive
> catalog
> -
>
> Key: HIVE-19898
> URL: https://issues.apache.org/jira/browse/HIVE-19898
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Standalone Metastore, Transactions
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19898.1.patch
>
>
> The
[jira] [Commented] (HIVE-19899) Support stored as JsonFile
[
https://issues.apache.org/jira/browse/HIVE-19899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514966#comment-16514966
]
Hive QA commented on HIVE-19899:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927930/HIVE-19899.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11837/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11837/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11837/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12927930/HIVE-19899.3.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927930 - PreCommit-HIVE-Build
> Support stored as JsonFile
> ---
>
> Key: HIVE-19899
> URL: https://issues.apache.org/jira/browse/HIVE-19899
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 3.0.0
> Environment: This is to add "stored as jsonfile" support for json
> file format.
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-19899.1.patch, HIVE-19899.2.patch,
> HIVE-19899.3.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19711) Refactor Hive Schema Tool
[
https://issues.apache.org/jira/browse/HIVE-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514965#comment-16514965
]
Hive QA commented on HIVE-19711:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927883/HIVE-19711.05.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11836/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11836/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11836/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12927883/HIVE-19711.05.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927883 - PreCommit-HIVE-Build
> Refactor Hive Schema Tool
> -
>
> Key: HIVE-19711
> URL: https://issues.apache.org/jira/browse/HIVE-19711
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-19711.01.patch, HIVE-19711.02.patch,
> HIVE-19711.03.patch, HIVE-19711.04.patch, HIVE-19711.05.patch
>
>
> HiveSchemaTool is an 1500 lines long class trying to do everything It shold
> be cut into multiple classes doing smaller components.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19777) NPE in TezSessionState
[
https://issues.apache.org/jira/browse/HIVE-19777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514964#comment-16514964
]
Hive QA commented on HIVE-19777:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927873/HIVE-19777.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14539 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11835/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11835/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11835/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927873 - PreCommit-HIVE-Build
> NPE in TezSessionState
> --
>
> Key: HIVE-19777
> URL: https://issues.apache.org/jira/browse/HIVE-19777
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Reporter: Jason Dere
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19777.patch
>
>
> Encountered while running "insert into table values (..)"
> Looks like it is due to the fact that TezSessionState.close() sets console to
> null at the start of the method, and then calls getSession() which attempts
> to log to console.
> {noformat}
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.getSession(TezSessionState.java:711)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.close(TezSessionState.java:646)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.closeIfNotDefault(TezSessionPoolManager.java:353)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:467)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.WorkloadManagerFederation.getUnmanagedSession(WorkloadManagerFederation.java:66)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.WorkloadManagerFederation.getSession(WorkloadManagerFederation.java:38)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:184)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:205)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2497)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2149)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1826)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1569)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1563)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:218)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> ~[hive-cli-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> ~[hive-cli-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> ~[hive-cli-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
> ~[hive-cli-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> ~[hive-cli-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
> ~[hive-cli-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_121]
> at
>
[jira] [Commented] (HIVE-19777) NPE in TezSessionState
[
https://issues.apache.org/jira/browse/HIVE-19777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514960#comment-16514960
]
Hive QA commented on HIVE-19777:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
2s{color} | {color:blue} ql in master has 2281 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 3 new + 27 unchanged - 0 fixed
= 30 total (was 27) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11835/dev-support/hive-personality.sh
|
| git revision | master / b100483 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11835/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11835/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> NPE in TezSessionState
> --
>
> Key: HIVE-19777
> URL: https://issues.apache.org/jira/browse/HIVE-19777
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Reporter: Jason Dere
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19777.patch
>
>
> Encountered while running "insert into table values (..)"
> Looks like it is due to the fact that TezSessionState.close() sets console to
> null at the start of the method, and then calls getSession() which attempts
> to log to console.
> {noformat}
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.getSession(TezSessionState.java:711)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.close(TezSessionState.java:646)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.closeIfNotDefault(TezSessionPoolManager.java:353)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:467)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.WorkloadManagerFederation.getUnmanagedSession(WorkloadManagerFederation.java:66)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
>
[jira] [Commented] (HIVE-19899) Support stored as JsonFile
[
https://issues.apache.org/jira/browse/HIVE-19899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514959#comment-16514959
]
Hive QA commented on HIVE-19899:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927930/HIVE-19899.3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 30 failed/errored test(s), 14588 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=258)
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testMergeOnTezEdges
(batchId=291)
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testMapNullKey[6]
(batchId=200)
org.apache.hive.hcatalog.pig.TestHCatStorer.testBagNStruct[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testDateCharTypes[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testDynamicPartitioningMultiPartColsInDataNoSpec[6]
(batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testDynamicPartitioningMultiPartColsInDataPartialSpec[6]
(batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testDynamicPartitioningMultiPartColsNoDataInDataNoSpec[6]
(batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testEmptyStore[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testMultiPartColsInData[6]
(batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testNoAlias[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testPartColsInData[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testPartitionPublish[6]
(batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testStoreFuncAllSimpleTypes[6]
(batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testStoreFuncSimple[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testStoreInPartiitonedTbl[6]
(batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testStoreMultiTables[6]
(batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testStoreWithNoCtorArgs[6]
(batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testStoreWithNoSchema[6]
(batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteChar[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteDate2[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteDate3[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteDate[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteDecimalXY[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteDecimalX[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteDecimal[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteSmallint[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteTimestamp[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteTinyint[6] (batchId=199)
org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteVarchar[6] (batchId=199)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11834/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11834/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11834/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 30 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927930 - PreCommit-HIVE-Build
> Support stored as JsonFile
> ---
>
> Key: HIVE-19899
> URL: https://issues.apache.org/jira/browse/HIVE-19899
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 3.0.0
> Environment: This is to add "stored as jsonfile" support for json
> file format.
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-19899.1.patch, HIVE-19899.2.patch,
> HIVE-19899.3.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19899) Support stored as JsonFile
[
https://issues.apache.org/jira/browse/HIVE-19899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514948#comment-16514948
]
Hive QA commented on HIVE-19899:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
3s{color} | {color:blue} ql in master has 2281 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 1 new + 3 unchanged - 0 fixed
= 4 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11834/dev-support/hive-personality.sh
|
| git revision | master / b100483 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11834/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11834/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support stored as JsonFile
> ---
>
> Key: HIVE-19899
> URL: https://issues.apache.org/jira/browse/HIVE-19899
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 3.0.0
> Environment: This is to add "stored as jsonfile" support for json
> file format.
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-19899.1.patch, HIVE-19899.2.patch,
> HIVE-19899.3.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-16829) A multiline/pretty-print JSON Data Record Reader
[
https://issues.apache.org/jira/browse/HIVE-16829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514928#comment-16514928
]
Alex S edited comment on HIVE-16829 at 6/17/18 12:57 AM:
-
As a work around you can change a record delimiter from \n to some other value
so hive doesn't get confused by getting \n within the formatted json record.
You will also need to change record delimiter in your json files accordingly.
You can use command below in CLI to change the delimiter value:
set textinputformat.record.delimiter='';
Why would we need a separate RecordReader?
was (Author: alse):
{{As a work around you can change a record delimiter from \n to some other
value so hive doesn't get confused by getting \n within the formatted json
record. You will also need to change record delimiter in your json files
accordingly. }}
{{You can use command below in CLI to change the delimiter value:}}
{{set textinputformat.record.delimiter='';}}
{{Why would we need a separate RecordReader? }}
> A multiline/pretty-print JSON Data Record Reader
>
>
> Key: HIVE-16829
> URL: https://issues.apache.org/jira/browse/HIVE-16829
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Reporter: Ujjawal Nayak
>Priority: Trivial
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> We already have 'org.apache.hive.hcatalog.data.JsonSerDe' which can
> serialize/deserialize a JSON Record. But it fails when it gets a JSON record
> which is in pretty-print format. And we do not have a RecordReader which can
> read this multiline JSON structure.
> We should create a RecordReader which can create a record from pretty-print
> format of JSON.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19891) inserting into external tables with custom partition directories may cause data loss
[
https://issues.apache.org/jira/browse/HIVE-19891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514944#comment-16514944
]
Hive QA commented on HIVE-19891:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927764/HIVE-19891.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 14540 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[concatenate_inherit_table_location]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[external_insert]
(batchId=16)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11833/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11833/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11833/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927764 - PreCommit-HIVE-Build
> inserting into external tables with custom partition directories may cause
> data loss
>
>
> Key: HIVE-19891
> URL: https://issues.apache.org/jira/browse/HIVE-19891
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19891.patch
>
>
> tbl1 is just used as a prop to create data, could be an existing directory
> for an external table.
> Due to weird behavior of LoadTableDesc (some ancient code for overriding old
> partition path), custom partition path is overwritten after the query and the
> data in it ceases being a part of the table (can be seen in desc formatted
> output with masking commented out in QTestUtil)
> This affects branch-1 too, so it's pretty old.
> {noformat}drop table tbl1;
> CREATE TABLE tbl1 (index int, value int ) PARTITIONED BY ( created_date
> string );
> insert into tbl1 partition(created_date='2018-02-01') VALUES (2, 2);
> CREATE external TABLE tbl2 (index int, value int ) PARTITIONED BY (
> created_date string );
> ALTER TABLE tbl2 ADD PARTITION(created_date='2018-02-01');
> ALTER TABLE tbl2 PARTITION(created_date='2018-02-01') SET LOCATION
> 'file:/Users/sergey/git/hivegit/itests/qtest/target/warehouse/tbl1/created_date=2018-02-01';
> select * from tbl2;
> describe formatted tbl2 partition(created_date='2018-02-01');
> insert into tbl2 partition(created_date='2018-02-01') VALUES (1, 1);
> select * from tbl2;
> describe formatted tbl2 partition(created_date='2018-02-01');
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19923) Follow up of HIVE-19615, use UnaryFunction instead of prefix
[
https://issues.apache.org/jira/browse/HIVE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514942#comment-16514942
]
Ashutosh Chauhan commented on HIVE-19923:
-
+1
> Follow up of HIVE-19615, use UnaryFunction instead of prefix
>
>
> Key: HIVE-19923
> URL: https://issues.apache.org/jira/browse/HIVE-19923
> Project: Hive
> Issue Type: Sub-task
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19923.patch
>
>
> Correct usage of Druid isnull function is {code} isnull(exp){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19876) Multiple fixes for Driver.isValidTxnListState
[
https://issues.apache.org/jira/browse/HIVE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-19876:
---
Resolution: Fixed
Fix Version/s: 4.0.0
3.1.0
Status: Resolved (was: Patch Available)
Pushed to master, branch-3. Thanks [~ekoifman]
> Multiple fixes for Driver.isValidTxnListState
> -
>
> Key: HIVE-19876
> URL: https://issues.apache.org/jira/browse/HIVE-19876
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Eugene Koifman
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19876.06.patch
>
>
> 1) Only locks for tables should be checked.
> 2) Check on whether {{txnWriteIdList}} is null or not needs to be fixed: it
> should be done on the configuration property value, as the object
> {{txnWriteIdList}} will never be null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19876) Multiple fixes for Driver.isValidTxnListState
[
https://issues.apache.org/jira/browse/HIVE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514933#comment-16514933
]
Eugene Koifman commented on HIVE-19876:
---
+1 patch 6.
Incidentally, acid (i.e. DbTxnManager always uses DbLockManager) ignores
hive.lock.manager setting so
isValidTxnListState() does not need any logic for other LMs. This is just FYI,
for future.
> Multiple fixes for Driver.isValidTxnListState
> -
>
> Key: HIVE-19876
> URL: https://issues.apache.org/jira/browse/HIVE-19876
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Eugene Koifman
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19876.06.patch
>
>
> 1) Only locks for tables should be checked.
> 2) Check on whether {{txnWriteIdList}} is null or not needs to be fixed: it
> should be done on the configuration property value, as the object
> {{txnWriteIdList}} will never be null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19891) inserting into external tables with custom partition directories may cause data loss
[
https://issues.apache.org/jira/browse/HIVE-19891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514930#comment-16514930
]
Hive QA commented on HIVE-19891:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
7s{color} | {color:blue} ql in master has 2282 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 3 new + 267 unchanged - 0
fixed = 270 total (was 267) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11833/dev-support/hive-personality.sh
|
| git revision | master / dd51259 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11833/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11833/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> inserting into external tables with custom partition directories may cause
> data loss
>
>
> Key: HIVE-19891
> URL: https://issues.apache.org/jira/browse/HIVE-19891
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19891.patch
>
>
> tbl1 is just used as a prop to create data, could be an existing directory
> for an external table.
> Due to weird behavior of LoadTableDesc (some ancient code for overriding old
> partition path), custom partition path is overwritten after the query and the
> data in it ceases being a part of the table (can be seen in desc formatted
> output with masking commented out in QTestUtil)
> This affects branch-1 too, so it's pretty old.
> {noformat}drop table tbl1;
> CREATE TABLE tbl1 (index int, value int ) PARTITIONED BY ( created_date
> string );
> insert into tbl1 partition(created_date='2018-02-01') VALUES (2, 2);
> CREATE external TABLE tbl2 (index int, value int ) PARTITIONED BY (
> created_date string );
> ALTER TABLE tbl2 ADD PARTITION(created_date='2018-02-01');
> ALTER TABLE tbl2 PARTITION(created_date='2018-02-01') SET LOCATION
> 'file:/Users/sergey/git/hivegit/itests/qtest/target/warehouse/tbl1/created_date=2018-02-01';
> select * from tbl2;
> describe formatted tbl2 partition(created_date='2018-02-01');
> insert into tbl2
[jira] [Commented] (HIVE-16829) A multiline/pretty-print JSON Data Record Reader
[
https://issues.apache.org/jira/browse/HIVE-16829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514928#comment-16514928
]
Alex S commented on HIVE-16829:
---
{{As a work around you can change a record delimiter from \n to some other
value so hive doesn't get confused by getting \n within the formatted json
record. You will also need to change record delimiter in your json files
accordingly. }}
{{You can use command below in CLI to change the delimiter value:}}
{{set textinputformat.record.delimiter='';}}
{{Why would we need a separate RecordReader? }}
> A multiline/pretty-print JSON Data Record Reader
>
>
> Key: HIVE-16829
> URL: https://issues.apache.org/jira/browse/HIVE-16829
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Reporter: Ujjawal Nayak
>Priority: Trivial
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> We already have 'org.apache.hive.hcatalog.data.JsonSerDe' which can
> serialize/deserialize a JSON Record. But it fails when it gets a JSON record
> which is in pretty-print format. And we do not have a RecordReader which can
> read this multiline JSON structure.
> We should create a RecordReader which can create a record from pretty-print
> format of JSON.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19903) Disable temporary insert-only transactional table
[ https://issues.apache.org/jira/browse/HIVE-19903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514925#comment-16514925 ] Steve Yeom commented on HIVE-19903: --- Hi [~jdere] please look at the patch 02. I have tested CREATE TABLE LIKE case it is already clear so we do not need a test. Also ran the two tests mentioned by Eric without failures. 23:05:10.393 [summary] :TestJdbcWithMiniLlapArrow.testComplexQuery 23:05:10.393 [summary] :TestJdbcWithMiniLlapRow.testComplexQuery > Disable temporary insert-only transactional table > - > > Key: HIVE-19903 > URL: https://issues.apache.org/jira/browse/HIVE-19903 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19903.01.patch, HIVE-19903.02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19903) Disable temporary insert-only transactional table
[ https://issues.apache.org/jira/browse/HIVE-19903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom updated HIVE-19903: -- Attachment: HIVE-19903.02.patch > Disable temporary insert-only transactional table > - > > Key: HIVE-19903 > URL: https://issues.apache.org/jira/browse/HIVE-19903 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19903.01.patch, HIVE-19903.02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19812) Disable external table replication by default via a configuration property
[
https://issues.apache.org/jira/browse/HIVE-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514923#comment-16514923
]
Hive QA commented on HIVE-19812:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927850/HIVE-19812.04.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14543 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[exim_13_nonnative_import]
(batchId=96)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11832/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11832/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11832/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927850 - PreCommit-HIVE-Build
> Disable external table replication by default via a configuration property
> --
>
> Key: HIVE-19812
> URL: https://issues.apache.org/jira/browse/HIVE-19812
> Project: Hive
> Issue Type: Task
> Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19812.01.patch, HIVE-19812.02.patch,
> HIVE-19812.03.patch, HIVE-19812.04.patch
>
>
> use a hive config property to allow external table replication. set this
> property by default to prevent external table replication.
> for metadata only hive repl always export metadata for external tables.
>
> REPL_DUMP_EXTERNAL_TABLES("hive.repl.dump.include.external.tables", false,
> "Indicates if repl dump should include information about external tables. It
> should be \n"
> + "used in conjunction with 'hive.repl.dump.metadata.only' set to false. if
> 'hive.repl.dump.metadata.only' \n"
> + " is set to true then this config parameter has no effect as external table
> meta data is flushed \n"
> + " always by default.")
> This should be done for only replication dump and not for export
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19812) Disable external table replication by default via a configuration property
[
https://issues.apache.org/jira/browse/HIVE-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514916#comment-16514916
]
Hive QA commented on HIVE-19812:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
2s{color} | {color:blue} ql in master has 2282 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
37s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} itests/hive-unit: The patch generated 2 new + 110
unchanged - 0 fixed = 112 total (was 110) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11832/dev-support/hive-personality.sh
|
| git revision | master / dd51259 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11832/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: common itests/hive-unit ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11832/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Disable external table replication by default via a configuration property
> --
>
> Key: HIVE-19812
> URL: https://issues.apache.org/jira/browse/HIVE-19812
> Project: Hive
> Issue Type: Task
> Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19812.01.patch, HIVE-19812.02.patch,
> HIVE-19812.03.patch, HIVE-19812.04.patch
>
>
> use a hive config property to allow external table replication. set this
> property by default to prevent external table replication.
> for metadata only hive repl always export metadata for
[jira] [Updated] (HIVE-12342) Set default value of hive.optimize.index.filter to true
[ https://issues.apache.org/jira/browse/HIVE-12342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Kryvenko updated HIVE-12342: - Attachment: HIVE-12342.17.patch > Set default value of hive.optimize.index.filter to true > --- > > Key: HIVE-12342 > URL: https://issues.apache.org/jira/browse/HIVE-12342 > Project: Hive > Issue Type: Task > Components: Configuration >Reporter: Ashutosh Chauhan >Assignee: Igor Kryvenko >Priority: Major > Attachments: HIVE-12342.05.patch, HIVE-12342.06.patch, > HIVE-12342.07.patch, HIVE-12342.08.patch, HIVE-12342.09.patch, > HIVE-12342.1.patch, HIVE-12342.10.patch, HIVE-12342.11.patch, > HIVE-12342.12.patch, HIVE-12342.13.patch, HIVE-12342.14.patch, > HIVE-12342.15.patch, HIVE-12342.16.patch, HIVE-12342.17.patch, > HIVE-12342.2.patch, HIVE-12342.3.patch, HIVE-12342.4.patch, HIVE-12342.patch > > > This configuration governs ppd for storage layer. When applicable, it will > always help. It should be on by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19569) alter table db1.t1 rename db2.t2 generates MetaStoreEventListener.onDropTable()
[
https://issues.apache.org/jira/browse/HIVE-19569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514909#comment-16514909
]
Hive QA commented on HIVE-19569:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927849/HIVE-19569.04.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14539 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11831/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11831/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11831/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927849 - PreCommit-HIVE-Build
> alter table db1.t1 rename db2.t2 generates
> MetaStoreEventListener.onDropTable()
> ---
>
> Key: HIVE-19569
> URL: https://issues.apache.org/jira/browse/HIVE-19569
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Standalone Metastore, Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-19569.01.patch, HIVE-19569.02.patch,
> HIVE-19569.03.patch, HIVE-19569.04.patch
>
>
> When renaming a table within the same DB, this operation causes
> {{MetaStoreEventListener.onAlterTable()}} to fire but when changing DB name
> for a table it causes {{MetaStoreEventListener.onDropTable()}} +
> {{MetaStoreEventListener.onCreateTable()}}.
> The files from original table are moved to new table location.
> This creates confusing semantics since any logic in {{onDropTable()}} doesn't
> know about the larger context, i.e. that there will be a matching
> {{onCreateTable()}}.
> In particular, this causes a problem for Acid tables since files moved from
> old table use WriteIDs that are not meaningful with the context of new table.
> Current implementation is due to replication. This should ideally be changed
> to raise a "not supported" error for tables that are marked for replication.
> cc [~sankarh]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19569) alter table db1.t1 rename db2.t2 generates MetaStoreEventListener.onDropTable()
[
https://issues.apache.org/jira/browse/HIVE-19569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514902#comment-16514902
]
Hive QA commented on HIVE-19569:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
6s{color} | {color:blue} ql in master has 2282 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
17s{color} | {color:blue} standalone-metastore in master has 227 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11831/dev-support/hive-personality.sh
|
| git revision | master / dd51259 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql standalone-metastore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11831/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> alter table db1.t1 rename db2.t2 generates
> MetaStoreEventListener.onDropTable()
> ---
>
> Key: HIVE-19569
> URL: https://issues.apache.org/jira/browse/HIVE-19569
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Standalone Metastore, Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-19569.01.patch, HIVE-19569.02.patch,
> HIVE-19569.03.patch, HIVE-19569.04.patch
>
>
> When renaming a table within the same DB, this operation causes
> {{MetaStoreEventListener.onAlterTable()}} to fire but when changing DB name
> for a table it causes {{MetaStoreEventListener.onDropTable()}} +
> {{MetaStoreEventListener.onCreateTable()}}.
> The files from original table are moved to new table location.
> This creates confusing semantics since any logic in {{onDropTable()}} doesn't
> know about the larger context, i.e. that there will be a matching
> {{onCreateTable()}}.
> In particular, this causes a problem
[jira] [Commented] (HIVE-19876) Multiple fixes for Driver.isValidTxnListState
[
https://issues.apache.org/jira/browse/HIVE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514890#comment-16514890
]
Hive QA commented on HIVE-19876:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927927/HIVE-19876.06.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11830/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11830/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11830/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12927927/HIVE-19876.06.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927927 - PreCommit-HIVE-Build
> Multiple fixes for Driver.isValidTxnListState
> -
>
> Key: HIVE-19876
> URL: https://issues.apache.org/jira/browse/HIVE-19876
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Eugene Koifman
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19876.06.patch
>
>
> 1) Only locks for tables should be checked.
> 2) Check on whether {{txnWriteIdList}} is null or not needs to be fixed: it
> should be done on the configuration property value, as the object
> {{txnWriteIdList}} will never be null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19876) Multiple fixes for Driver.isValidTxnListState
[
https://issues.apache.org/jira/browse/HIVE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514889#comment-16514889
]
Hive QA commented on HIVE-19876:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927927/HIVE-19876.06.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11829/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11829/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11829/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12927927/HIVE-19876.06.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927927 - PreCommit-HIVE-Build
> Multiple fixes for Driver.isValidTxnListState
> -
>
> Key: HIVE-19876
> URL: https://issues.apache.org/jira/browse/HIVE-19876
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Eugene Koifman
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19876.06.patch
>
>
> 1) Only locks for tables should be checked.
> 2) Check on whether {{txnWriteIdList}} is null or not needs to be fixed: it
> should be done on the configuration property value, as the object
> {{txnWriteIdList}} will never be null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-15976) Support CURRENT_CATALOG and CURRENT_SCHEMA
[
https://issues.apache.org/jira/browse/HIVE-15976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514888#comment-16514888
]
Hive QA commented on HIVE-15976:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927840/HIVE-15976.06.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14540 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=258)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11828/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11828/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11828/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927840 - PreCommit-HIVE-Build
> Support CURRENT_CATALOG and CURRENT_SCHEMA
> --
>
> Key: HIVE-15976
> URL: https://issues.apache.org/jira/browse/HIVE-15976
> Project: Hive
> Issue Type: Sub-task
> Components: SQL
>Reporter: Carter Shanklin
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-15976.01.patch, HIVE-15976.02.patch,
> HIVE-15976.03.patch, HIVE-15976.04.patch, HIVE-15976.05.patch,
> HIVE-15976.06.patch
>
>
> Support these keywords for querying the current catalog and schema. SQL
> reference: section 6.4
> *oracle*
> CREATE TABLE CURRENT_SCHEMA (col VARCHAR2(1)); -- ok
> SELECT CURRENT_SCHEMA FROM DUAL; -- error, ORA-00904: "CURRENT_SCHEMA":
> invalid identifier
> SELECT CURRENT_SCHEMA() FROM DUAL; -- error, ORA-00904: "CURRENT_SCHEMA":
> invalid identifier
> *postgres*
> CREATE TABLE CURRENT_SCHEMA (col VARCHAR(1)); -- error: syntax error at or
> near "CURRENT_SCHEMA"
> SELECT CURRENT_SCHEMA; -- ok, "public"
> SELECT CURRENT_SCHEMA(); -- ok, "public"
> *mysql*
> CREATE TABLE CURRENT_SCHEMA (col VARCHAR(1)); -- ok
> SELECT CURRENT_SCHEMA; -- error, Unknown column 'CURRENT_SCHEMA' in 'field
> list'
> SELECT CURRENT_SCHEMA(); -- error, FUNCTION db_9_e28e6f.CURRENT_SCHEMA does
> not exist
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19920) Schematool fails in embedded mode when auth is on
[ https://issues.apache.org/jira/browse/HIVE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514886#comment-16514886 ] Thejas M Nair commented on HIVE-19920: -- +1 > Schematool fails in embedded mode when auth is on > - > > Key: HIVE-19920 > URL: https://issues.apache.org/jira/browse/HIVE-19920 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19920.1.patch, HIVE-19920.2.patch > > > This is a follow up of HIVE-19775. We need to override more properties in > embedded hs2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19920) Schematool fails in embedded mode when auth is on
[ https://issues.apache.org/jira/browse/HIVE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514883#comment-16514883 ] Daniel Dai commented on HIVE-19920: --- Thanks for pointing out, I missed it. > Schematool fails in embedded mode when auth is on > - > > Key: HIVE-19920 > URL: https://issues.apache.org/jira/browse/HIVE-19920 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19920.1.patch, HIVE-19920.2.patch > > > This is a follow up of HIVE-19775. We need to override more properties in > embedded hs2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19920) Schematool fails in embedded mode when auth is on
[ https://issues.apache.org/jira/browse/HIVE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-19920: -- Attachment: HIVE-19920.2.patch > Schematool fails in embedded mode when auth is on > - > > Key: HIVE-19920 > URL: https://issues.apache.org/jira/browse/HIVE-19920 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19920.1.patch, HIVE-19920.2.patch > > > This is a follow up of HIVE-19775. We need to override more properties in > embedded hs2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-15976) Support CURRENT_CATALOG and CURRENT_SCHEMA
[
https://issues.apache.org/jira/browse/HIVE-15976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514881#comment-16514881
]
Hive QA commented on HIVE-15976:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
4s{color} | {color:blue} ql in master has 2282 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 5 new + 72 unchanged - 0 fixed
= 77 total (was 72) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11828/dev-support/hive-personality.sh
|
| git revision | master / dd51259 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11828/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11828/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support CURRENT_CATALOG and CURRENT_SCHEMA
> --
>
> Key: HIVE-15976
> URL: https://issues.apache.org/jira/browse/HIVE-15976
> Project: Hive
> Issue Type: Sub-task
> Components: SQL
>Reporter: Carter Shanklin
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-15976.01.patch, HIVE-15976.02.patch,
> HIVE-15976.03.patch, HIVE-15976.04.patch, HIVE-15976.05.patch,
> HIVE-15976.06.patch
>
>
> Support these keywords for querying the current catalog and schema. SQL
> reference: section 6.4
> *oracle*
> CREATE TABLE CURRENT_SCHEMA (col VARCHAR2(1)); -- ok
> SELECT CURRENT_SCHEMA FROM DUAL; -- error, ORA-00904: "CURRENT_SCHEMA":
> invalid identifier
> SELECT CURRENT_SCHEMA() FROM DUAL; -- error, ORA-00904: "CURRENT_SCHEMA":
> invalid identifier
> *postgres*
> CREATE TABLE CURRENT_SCHEMA (col VARCHAR(1)); -- error: syntax error at or
> near "CURRENT_SCHEMA"
> SELECT CURRENT_SCHEMA; -- ok, "public"
> SELECT CURRENT_SCHEMA(); -- ok, "public"
> *mysql*
> CREATE TABLE CURRENT_SCHEMA (col VARCHAR(1)); -- ok
> SELECT CURRENT_SCHEMA; -- error, Unknown column 'CURRENT_SCHEMA' in 'field
> list'
> SELECT CURRENT_SCHEMA(); -- error, FUNCTION db_9_e28e6f.CURRENT_SCHEMA does
> not exist
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19739) Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded data/metadata.
[
https://issues.apache.org/jira/browse/HIVE-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514878#comment-16514878
]
Hive QA commented on HIVE-19739:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927830/HIVE-19739.01-branch-3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 14344 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=256)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=256)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=256)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=256)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=256)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testManagedPaths
(batchId=233)
org.apache.hive.jdbc.TestJdbcGenericUDTFGetSplits.testGenericUDTFOrderBySplitCount1
(batchId=242)
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
(batchId=242)
org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=308)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11827/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11827/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11827/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927830 - PreCommit-HIVE-Build
> Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded
> data/metadata.
> -
>
> Key: HIVE-19739
> URL: https://issues.apache.org/jira/browse/HIVE-19739
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: DR, pull-request-available, replication
> Fix For: 4.0.0
>
> Attachments: HIVE-19739.01-branch-3.patch, HIVE-19739.01.patch,
> HIVE-19739.02.patch, HIVE-19739.03.patch, HIVE-19739.04.patch
>
>
> Currently. bootstrap REPL LOAD have added checkpoint identifiers in
> DB/table/partition object properties once the data/metadata related to the
> object is successfully loaded.
> If the Db exist and is not empty, then currently we are throwing exception.
> But need to support it for the retry scenario after a failure.
> If there is a retry of bootstrap load using the same dump, then instead of
> throwing error, we should check if any of the tables/partitions are
> completely loaded using the checkpoint identifiers. If yes, then skip it or
> else drop/create them again.
> If the bootstrap load is performed using different dump, then it should throw
> exception.
> Allow bootstrap on empty Db only if ckpt property is not set. Also, if
> bootstrap load is completed on the target Db, then shouldn't allow bootstrap
> retry at all.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19920) Schematool fails in embedded mode when auth is on
[ https://issues.apache.org/jira/browse/HIVE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514877#comment-16514877 ] Thejas M Nair commented on HIVE-19920: -- It should actually be - hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdConfOnlyAuthorizerFactory But the URL doesn't have the "hive.security.authorization.manager=" part. > Schematool fails in embedded mode when auth is on > - > > Key: HIVE-19920 > URL: https://issues.apache.org/jira/browse/HIVE-19920 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19920.1.patch > > > This is a follow up of HIVE-19775. We need to override more properties in > embedded hs2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19629) Enable Decimal64 reader after orc version upgrade
[ https://issues.apache.org/jira/browse/HIVE-19629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19629: - Attachment: HIVE-19629-branch-3.patch > Enable Decimal64 reader after orc version upgrade > - > > Key: HIVE-19629 > URL: https://issues.apache.org/jira/browse/HIVE-19629 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19629-branch-3.patch, HIVE-19629.1.patch, > HIVE-19629.10.patch, HIVE-19629.11.patch, HIVE-19629.12.patch, > HIVE-19629.13.patch, HIVE-19629.14.patch, HIVE-19629.15.patch, > HIVE-19629.16.patch, HIVE-19629.2.patch, HIVE-19629.3.patch, > HIVE-19629.4.patch, HIVE-19629.5.patch, HIVE-19629.6.patch, > HIVE-19629.7.patch, HIVE-19629.8.patch, HIVE-19629.9.patch > > > ORC 1.5.0 supports new fast decimal 64 reader. New VRB has to be created for > making use of decimal 64 column vectors. Also LLAP IO will need a new reader > to reader from long stream to decimal 64. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19629) Enable Decimal64 reader after orc version upgrade
[ https://issues.apache.org/jira/browse/HIVE-19629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514876#comment-16514876 ] Prasanth Jayachandran commented on HIVE-19629: -- Committed to master. Will put up a patch for branch-3. > Enable Decimal64 reader after orc version upgrade > - > > Key: HIVE-19629 > URL: https://issues.apache.org/jira/browse/HIVE-19629 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19629.1.patch, HIVE-19629.10.patch, > HIVE-19629.11.patch, HIVE-19629.12.patch, HIVE-19629.13.patch, > HIVE-19629.14.patch, HIVE-19629.15.patch, HIVE-19629.16.patch, > HIVE-19629.2.patch, HIVE-19629.3.patch, HIVE-19629.4.patch, > HIVE-19629.5.patch, HIVE-19629.6.patch, HIVE-19629.7.patch, > HIVE-19629.8.patch, HIVE-19629.9.patch > > > ORC 1.5.0 supports new fast decimal 64 reader. New VRB has to be created for > making use of decimal 64 column vectors. Also LLAP IO will need a new reader > to reader from long stream to decimal 64. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19629) Enable Decimal64 reader after orc version upgrade
[ https://issues.apache.org/jira/browse/HIVE-19629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19629: - Fix Version/s: 4.0.0 > Enable Decimal64 reader after orc version upgrade > - > > Key: HIVE-19629 > URL: https://issues.apache.org/jira/browse/HIVE-19629 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19629.1.patch, HIVE-19629.10.patch, > HIVE-19629.11.patch, HIVE-19629.12.patch, HIVE-19629.13.patch, > HIVE-19629.14.patch, HIVE-19629.15.patch, HIVE-19629.16.patch, > HIVE-19629.2.patch, HIVE-19629.3.patch, HIVE-19629.4.patch, > HIVE-19629.5.patch, HIVE-19629.6.patch, HIVE-19629.7.patch, > HIVE-19629.8.patch, HIVE-19629.9.patch > > > ORC 1.5.0 supports new fast decimal 64 reader. New VRB has to be created for > making use of decimal 64 column vectors. Also LLAP IO will need a new reader > to reader from long stream to decimal 64. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19923) Follow up of HIVE-19615, use UnaryFunction instead of prefix
[
https://issues.apache.org/jira/browse/HIVE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-19923:
--
Component/s: Druid integration
> Follow up of HIVE-19615, use UnaryFunction instead of prefix
>
>
> Key: HIVE-19923
> URL: https://issues.apache.org/jira/browse/HIVE-19923
> Project: Hive
> Issue Type: Sub-task
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19923.patch
>
>
> Correct usage of Druid isnull function is {code} isnull(exp){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19923) Follow up of HIVE-19615, use UnaryFunction instead of prefix
[
https://issues.apache.org/jira/browse/HIVE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-19923:
--
Attachment: HIVE-19923.patch
> Follow up of HIVE-19615, use UnaryFunction instead of prefix
>
>
> Key: HIVE-19923
> URL: https://issues.apache.org/jira/browse/HIVE-19923
> Project: Hive
> Issue Type: Sub-task
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19923.patch
>
>
> Correct usage of Druid isnull function is {code} isnull(exp){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19923) Follow up of HIVE-19615, use UnaryFunction instead of prefix
[
https://issues.apache.org/jira/browse/HIVE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-19923:
--
Attachment: (was: HIVE-19923.patch)
> Follow up of HIVE-19615, use UnaryFunction instead of prefix
>
>
> Key: HIVE-19923
> URL: https://issues.apache.org/jira/browse/HIVE-19923
> Project: Hive
> Issue Type: Sub-task
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19923.patch
>
>
> Correct usage of Druid isnull function is {code} isnull(exp){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19923) Follow up of HIVE-19615, use UnaryFunction instead of prefix
[
https://issues.apache.org/jira/browse/HIVE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-19923:
--
Status: Patch Available (was: Open)
> Follow up of HIVE-19615, use UnaryFunction instead of prefix
>
>
> Key: HIVE-19923
> URL: https://issues.apache.org/jira/browse/HIVE-19923
> Project: Hive
> Issue Type: Sub-task
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19923.patch
>
>
> Correct usage of Druid isnull function is {code} isnull(exp){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19853) Arrow serializer needs to create a TimeStampMicroTZVector instead of TimeStampMicroVector
[ https://issues.apache.org/jira/browse/HIVE-19853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514855#comment-16514855 ] Eric Wohlstadter commented on HIVE-19853: - The test failure was previously reported here, I assume it is unrelated: https://issues.apache.org/jira/browse/HIVE-19922 [https://builds.apache.org/job/PreCommit-HIVE-Build/11826/testReport/] > Arrow serializer needs to create a TimeStampMicroTZVector instead of > TimeStampMicroVector > - > > Key: HIVE-19853 > URL: https://issues.apache.org/jira/browse/HIVE-19853 > Project: Hive > Issue Type: Bug >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-19853.1.patch, HIVE-19853.2.patch > > > HIVE-19723 changed nanosecond to microsecond in Arrow serialization. However, > it needs to be microsecond with time zone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19923) Follow up of HIVE-19615, use UnaryFunction instead of prefix
[
https://issues.apache.org/jira/browse/HIVE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514852#comment-16514852
]
Nishant Bangarwa commented on HIVE-19923:
-
+1
> Follow up of HIVE-19615, use UnaryFunction instead of prefix
>
>
> Key: HIVE-19923
> URL: https://issues.apache.org/jira/browse/HIVE-19923
> Project: Hive
> Issue Type: Sub-task
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19923.patch
>
>
> Correct usage of Druid isnull function is {code} isnull(exp){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19923) Follow up of HIVE-19615, use UnaryFunction instead of prefix
[
https://issues.apache.org/jira/browse/HIVE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514840#comment-16514840
]
slim bouguerra commented on HIVE-19923:
---
CC [~ashutoshc]
> Follow up of HIVE-19615, use UnaryFunction instead of prefix
>
>
> Key: HIVE-19923
> URL: https://issues.apache.org/jira/browse/HIVE-19923
> Project: Hive
> Issue Type: Sub-task
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19923.patch
>
>
> Correct usage of Druid isnull function is {code} isnull(exp){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19923) Follow up of HIVE-19615, use UnaryFunction instead of prefix
[
https://issues.apache.org/jira/browse/HIVE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra reassigned HIVE-19923:
-
Assignee: Oleksandr Shulgin
> Follow up of HIVE-19615, use UnaryFunction instead of prefix
>
>
> Key: HIVE-19923
> URL: https://issues.apache.org/jira/browse/HIVE-19923
> Project: Hive
> Issue Type: Sub-task
>Reporter: slim bouguerra
>Assignee: Oleksandr Shulgin
>Priority: Major
> Attachments: HIVE-19923.patch
>
>
> Correct usage of Druid isnull function is {code} isnull(exp){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19923) Follow up of HIVE-19615, use UnaryFunction instead of prefix
[
https://issues.apache.org/jira/browse/HIVE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-19923:
--
Attachment: HIVE-19923.patch
> Follow up of HIVE-19615, use UnaryFunction instead of prefix
>
>
> Key: HIVE-19923
> URL: https://issues.apache.org/jira/browse/HIVE-19923
> Project: Hive
> Issue Type: Sub-task
>Reporter: slim bouguerra
>Priority: Major
> Attachments: HIVE-19923.patch
>
>
> Correct usage of Druid isnull function is {code} isnull(exp){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19923) Follow up of HIVE-19615, use UnaryFunction instead of prefix
[
https://issues.apache.org/jira/browse/HIVE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra reassigned HIVE-19923:
-
Assignee: slim bouguerra (was: Oleksandr Shulgin)
> Follow up of HIVE-19615, use UnaryFunction instead of prefix
>
>
> Key: HIVE-19923
> URL: https://issues.apache.org/jira/browse/HIVE-19923
> Project: Hive
> Issue Type: Sub-task
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19923.patch
>
>
> Correct usage of Druid isnull function is {code} isnull(exp){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19739) Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded data/metadata.
[
https://issues.apache.org/jira/browse/HIVE-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514835#comment-16514835
]
Hive QA commented on HIVE-19739:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 12s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-11827/patches/PreCommit-HIVE-Build-11827.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11827/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded
> data/metadata.
> -
>
> Key: HIVE-19739
> URL: https://issues.apache.org/jira/browse/HIVE-19739
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: DR, pull-request-available, replication
> Fix For: 4.0.0
>
> Attachments: HIVE-19739.01-branch-3.patch, HIVE-19739.01.patch,
> HIVE-19739.02.patch, HIVE-19739.03.patch, HIVE-19739.04.patch
>
>
> Currently. bootstrap REPL LOAD have added checkpoint identifiers in
> DB/table/partition object properties once the data/metadata related to the
> object is successfully loaded.
> If the Db exist and is not empty, then currently we are throwing exception.
> But need to support it for the retry scenario after a failure.
> If there is a retry of bootstrap load using the same dump, then instead of
> throwing error, we should check if any of the tables/partitions are
> completely loaded using the checkpoint identifiers. If yes, then skip it or
> else drop/create them again.
> If the bootstrap load is performed using different dump, then it should throw
> exception.
> Allow bootstrap on empty Db only if ckpt property is not set. Also, if
> bootstrap load is completed on the target Db, then shouldn't allow bootstrap
> retry at all.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19853) Arrow serializer needs to create a TimeStampMicroTZVector instead of TimeStampMicroVector
[
https://issues.apache.org/jira/browse/HIVE-19853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514832#comment-16514832
]
Hive QA commented on HIVE-19853:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927835/HIVE-19853.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14537 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=258)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11826/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11826/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11826/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927835 - PreCommit-HIVE-Build
> Arrow serializer needs to create a TimeStampMicroTZVector instead of
> TimeStampMicroVector
> -
>
> Key: HIVE-19853
> URL: https://issues.apache.org/jira/browse/HIVE-19853
> Project: Hive
> Issue Type: Bug
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-19853.1.patch, HIVE-19853.2.patch
>
>
> HIVE-19723 changed nanosecond to microsecond in Arrow serialization. However,
> it needs to be microsecond with time zone.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19695) Year Month Day extraction functions need to add an implicit cast for column that are String types
[
https://issues.apache.org/jira/browse/HIVE-19695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-19695:
Resolution: Fixed
Fix Version/s: 3.1.0
Status: Resolved (was: Patch Available)
Pushed to master and branch-3. Thanks, Slim!
> Year Month Day extraction functions need to add an implicit cast for column
> that are String types
> -
>
> Key: HIVE-19695
> URL: https://issues.apache.org/jira/browse/HIVE-19695
> Project: Hive
> Issue Type: Bug
> Components: Druid integration, Query Planning
>Affects Versions: 3.0.0
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19695.1.patch, HIVE-19695.2.patch,
> HIVE-19695.patch, HIVE-19695.patch
>
>
> To avoid surprising/wrong results, Hive Query plan shall add an explicit cast
> over non date/timestamp column type when user try to extract Year/Month/Hour
> etc..
> This is an example of misleading results.
> {code}
> create table test_base_table(`timecolumn` timestamp, `date_c` string,
> `timestamp_c` string, `metric_c` double);
> insert into test_base_table values ('2015-03-08 00:00:00', '2015-03-10',
> '2015-03-08 00:00:00', 5.0);
> CREATE TABLE druid_test_table
> STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
> TBLPROPERTIES ("druid.segment.granularity" = "DAY")
> AS select
> cast(`timecolumn` as timestamp with local time zone) as `__time`, `date_c`,
> `timestamp_c`, `metric_c` FROM test_base_table;
> select
> year(date_c), month(date_c),day(date_c), hour(date_c),
> year(timestamp_c), month(timestamp_c),day(timestamp_c), hour(timestamp_c)
> from druid_test_table;
> {code}
> will return the following wrong results:
> {code}
> PREHOOK: query: select
> year(date_c), month(date_c),day(date_c), hour(date_c),
> year(timestamp_c), month(timestamp_c),day(timestamp_c), hour(timestamp_c)
> from druid_test_table
> PREHOOK: type: QUERY
> PREHOOK: Input: default@druid_test_table
> A masked pattern was here
> POSTHOOK: query: select
> year(date_c), month(date_c),day(date_c), hour(date_c),
> year(timestamp_c), month(timestamp_c),day(timestamp_c), hour(timestamp_c)
> from druid_test_table
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@druid_test_table
> A masked pattern was here
> 1969 12 31 16 196912 31 16
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19853) Arrow serializer needs to create a TimeStampMicroTZVector instead of TimeStampMicroVector
[
https://issues.apache.org/jira/browse/HIVE-19853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514805#comment-16514805
]
Hive QA commented on HIVE-19853:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
14s{color} | {color:blue} ql in master has 2277 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
47s{color} | {color:red} ql: The patch generated 43 new + 520 unchanged - 62
fixed = 563 total (was 582) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11826/dev-support/hive-personality.sh
|
| git revision | master / 5ba634a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11826/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11826/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Arrow serializer needs to create a TimeStampMicroTZVector instead of
> TimeStampMicroVector
> -
>
> Key: HIVE-19853
> URL: https://issues.apache.org/jira/browse/HIVE-19853
> Project: Hive
> Issue Type: Bug
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-19853.1.patch, HIVE-19853.2.patch
>
>
> HIVE-19723 changed nanosecond to microsecond in Arrow serialization. However,
> it needs to be microsecond with time zone.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19922) TestMiniDruidKafkaCliDriver[druidkafkamini_basic started] to fail
[ https://issues.apache.org/jira/browse/HIVE-19922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-19922: -- Status: Patch Available (was: Open) Addressing a revert for [~daijy]'s last comment to check if this helps, or not > TestMiniDruidKafkaCliDriver[druidkafkamini_basic started] to fail > - > > Key: HIVE-19922 > URL: https://issues.apache.org/jira/browse/HIVE-19922 > Project: Hive > Issue Type: Bug >Reporter: Peter Vary >Priority: Major > Attachments: HIVE-19922.patch > > > Consistently failing in the last 4 runs. > See: > [https://builds.apache.org/job/PreCommit-HIVE-Build/11824/testReport/org.apache.hadoop.hive.cli/TestMiniDruidKafkaCliDriver/testCliDriver_druidkafkamini_basic_/history/] > Can not reproduce the failure locally :( > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19922) TestMiniDruidKafkaCliDriver[druidkafkamini_basic started] to fail
[ https://issues.apache.org/jira/browse/HIVE-19922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-19922: -- Attachment: HIVE-19922.patch > TestMiniDruidKafkaCliDriver[druidkafkamini_basic started] to fail > - > > Key: HIVE-19922 > URL: https://issues.apache.org/jira/browse/HIVE-19922 > Project: Hive > Issue Type: Bug >Reporter: Peter Vary >Priority: Major > Attachments: HIVE-19922.patch > > > Consistently failing in the last 4 runs. > See: > [https://builds.apache.org/job/PreCommit-HIVE-Build/11824/testReport/org.apache.hadoop.hive.cli/TestMiniDruidKafkaCliDriver/testCliDriver_druidkafkamini_basic_/history/] > Can not reproduce the failure locally :( > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19922) TestMiniDruidKafkaCliDriver[druidkafkamini_basic started] to fail
[ https://issues.apache.org/jira/browse/HIVE-19922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514799#comment-16514799 ] Peter Vary commented on HIVE-19922: --- [~daijy]: Could you please take a look at this? Based on timing your commit caused this, although I do not have idea how can it be... > TestMiniDruidKafkaCliDriver[druidkafkamini_basic started] to fail > - > > Key: HIVE-19922 > URL: https://issues.apache.org/jira/browse/HIVE-19922 > Project: Hive > Issue Type: Bug >Reporter: Peter Vary >Priority: Major > Attachments: HIVE-19922.patch > > > Consistently failing in the last 4 runs. > See: > [https://builds.apache.org/job/PreCommit-HIVE-Build/11824/testReport/org.apache.hadoop.hive.cli/TestMiniDruidKafkaCliDriver/testCliDriver_druidkafkamini_basic_/history/] > Can not reproduce the failure locally :( > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19695) Year Month Day extraction functions need to add an implicit cast for column that are String types
[
https://issues.apache.org/jira/browse/HIVE-19695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514792#comment-16514792
]
Hive QA commented on HIVE-19695:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927834/HIVE-19695.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 14537 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_extractTime]
(batchId=258)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=184)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=184)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11825/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11825/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11825/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927834 - PreCommit-HIVE-Build
> Year Month Day extraction functions need to add an implicit cast for column
> that are String types
> -
>
> Key: HIVE-19695
> URL: https://issues.apache.org/jira/browse/HIVE-19695
> Project: Hive
> Issue Type: Bug
> Components: Druid integration, Query Planning
>Affects Versions: 3.0.0
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19695.1.patch, HIVE-19695.2.patch,
> HIVE-19695.patch, HIVE-19695.patch
>
>
> To avoid surprising/wrong results, Hive Query plan shall add an explicit cast
> over non date/timestamp column type when user try to extract Year/Month/Hour
> etc..
> This is an example of misleading results.
> {code}
> create table test_base_table(`timecolumn` timestamp, `date_c` string,
> `timestamp_c` string, `metric_c` double);
> insert into test_base_table values ('2015-03-08 00:00:00', '2015-03-10',
> '2015-03-08 00:00:00', 5.0);
> CREATE TABLE druid_test_table
> STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
> TBLPROPERTIES ("druid.segment.granularity" = "DAY")
> AS select
> cast(`timecolumn` as timestamp with local time zone) as `__time`, `date_c`,
> `timestamp_c`, `metric_c` FROM test_base_table;
> select
> year(date_c), month(date_c),day(date_c), hour(date_c),
> year(timestamp_c), month(timestamp_c),day(timestamp_c), hour(timestamp_c)
> from druid_test_table;
> {code}
> will return the following wrong results:
> {code}
> PREHOOK: query: select
> year(date_c), month(date_c),day(date_c), hour(date_c),
> year(timestamp_c), month(timestamp_c),day(timestamp_c), hour(timestamp_c)
> from druid_test_table
> PREHOOK: type: QUERY
> PREHOOK: Input: default@druid_test_table
> A masked pattern was here
> POSTHOOK: query: select
> year(date_c), month(date_c),day(date_c), hour(date_c),
> year(timestamp_c), month(timestamp_c),day(timestamp_c), hour(timestamp_c)
> from druid_test_table
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@druid_test_table
> A masked pattern was here
> 1969 12 31 16 196912 31 16
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19921) Fix perf duration and queue name in HiveProtoLoggingHook
[ https://issues.apache.org/jira/browse/HIVE-19921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Jaiprakash updated HIVE-19921: - Status: Patch Available (was: Open) > Fix perf duration and queue name in HiveProtoLoggingHook > > > Key: HIVE-19921 > URL: https://issues.apache.org/jira/browse/HIVE-19921 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash >Priority: Major > Attachments: HIVE-19921.01.patch > > > The perf log should return duration instead of end time. > The queue name should be llap queue for llap queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19921) Fix perf duration and queue name in HiveProtoLoggingHook
[ https://issues.apache.org/jira/browse/HIVE-19921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Jaiprakash updated HIVE-19921: - Attachment: (was: HIVE-19921.01.patch) > Fix perf duration and queue name in HiveProtoLoggingHook > > > Key: HIVE-19921 > URL: https://issues.apache.org/jira/browse/HIVE-19921 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash >Priority: Major > Attachments: HIVE-19921.01.patch > > > The perf log should return duration instead of end time. > The queue name should be llap queue for llap queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19921) Fix perf duration and queue name in HiveProtoLoggingHook
[ https://issues.apache.org/jira/browse/HIVE-19921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Jaiprakash updated HIVE-19921: - Attachment: HIVE-19921.01.patch > Fix perf duration and queue name in HiveProtoLoggingHook > > > Key: HIVE-19921 > URL: https://issues.apache.org/jira/browse/HIVE-19921 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash >Priority: Major > Attachments: HIVE-19921.01.patch > > > The perf log should return duration instead of end time. > The queue name should be llap queue for llap queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19921) Fix perf duration and queue name in HiveProtoLoggingHook
[ https://issues.apache.org/jira/browse/HIVE-19921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514782#comment-16514782 ] Harish Jaiprakash commented on HIVE-19921: -- [~anishek], Please review. https://reviews.apache.org/r/67619/ > Fix perf duration and queue name in HiveProtoLoggingHook > > > Key: HIVE-19921 > URL: https://issues.apache.org/jira/browse/HIVE-19921 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash >Priority: Major > Attachments: HIVE-19921.01.patch > > > The perf log should return duration instead of end time. > The queue name should be llap queue for llap queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19921) Fix perf duration and queue name in HiveProtoLoggingHook
[ https://issues.apache.org/jira/browse/HIVE-19921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Jaiprakash updated HIVE-19921: - Description: The perf log should return duration instead of end time. The queue name should be llap queue for llap queries. was: The perf log return end time instead of duration. The queue name should be llap queue for llap queries. > Fix perf duration and queue name in HiveProtoLoggingHook > > > Key: HIVE-19921 > URL: https://issues.apache.org/jira/browse/HIVE-19921 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash >Priority: Major > Attachments: HIVE-19921.01.patch > > > The perf log should return duration instead of end time. > The queue name should be llap queue for llap queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19921) Fix perf duration and queue name in HiveProtoLoggingHook
[ https://issues.apache.org/jira/browse/HIVE-19921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Jaiprakash updated HIVE-19921: - Component/s: HiveServer2 > Fix perf duration and queue name in HiveProtoLoggingHook > > > Key: HIVE-19921 > URL: https://issues.apache.org/jira/browse/HIVE-19921 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash >Priority: Major > Attachments: HIVE-19921.01.patch > > > The perf log return end time instead of duration. > The queue name should be llap queue for llap queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19921) Fix perf duration and queue name in HiveProtoLoggingHook
[ https://issues.apache.org/jira/browse/HIVE-19921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Jaiprakash updated HIVE-19921: - Attachment: HIVE-19921.01.patch > Fix perf duration and queue name in HiveProtoLoggingHook > > > Key: HIVE-19921 > URL: https://issues.apache.org/jira/browse/HIVE-19921 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash >Priority: Major > Attachments: HIVE-19921.01.patch > > > The perf log return end time instead of duration. > The queue name should be llap queue for llap queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19695) Year Month Day extraction functions need to add an implicit cast for column that are String types
[
https://issues.apache.org/jira/browse/HIVE-19695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514778#comment-16514778
]
Hive QA commented on HIVE-19695:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
9s{color} | {color:blue} ql in master has 2277 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11825/dev-support/hive-personality.sh
|
| git revision | master / 5ba634a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11825/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Year Month Day extraction functions need to add an implicit cast for column
> that are String types
> -
>
> Key: HIVE-19695
> URL: https://issues.apache.org/jira/browse/HIVE-19695
> Project: Hive
> Issue Type: Bug
> Components: Druid integration, Query Planning
>Affects Versions: 3.0.0
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19695.1.patch, HIVE-19695.2.patch,
> HIVE-19695.patch, HIVE-19695.patch
>
>
> To avoid surprising/wrong results, Hive Query plan shall add an explicit cast
> over non date/timestamp column type when user try to extract Year/Month/Hour
> etc..
> This is an example of misleading results.
> {code}
> create table test_base_table(`timecolumn` timestamp, `date_c` string,
> `timestamp_c` string, `metric_c` double);
> insert into test_base_table values ('2015-03-08 00:00:00', '2015-03-10',
> '2015-03-08 00:00:00', 5.0);
> CREATE TABLE druid_test_table
> STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
> TBLPROPERTIES ("druid.segment.granularity" = "DAY")
> AS select
> cast(`timecolumn` as timestamp with local time zone) as `__time`, `date_c`,
> `timestamp_c`, `metric_c` FROM test_base_table;
> select
> year(date_c), month(date_c),day(date_c), hour(date_c),
> year(timestamp_c), month(timestamp_c),day(timestamp_c), hour(timestamp_c)
> from druid_test_table;
> {code}
> will return the following wrong results:
> {code}
> PREHOOK: query: select
> year(date_c), month(date_c),day(date_c), hour(date_c),
>
[jira] [Assigned] (HIVE-19921) Fix perf duration and queue name in HiveProtoLoggingHook
[ https://issues.apache.org/jira/browse/HIVE-19921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Jaiprakash reassigned HIVE-19921: > Fix perf duration and queue name in HiveProtoLoggingHook > > > Key: HIVE-19921 > URL: https://issues.apache.org/jira/browse/HIVE-19921 > Project: Hive > Issue Type: Bug >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash >Priority: Major > > The perf log return end time instead of duration. > The queue name should be llap queue for llap queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19870) HCatalog dynamic partition query can fail, if the table path is managed by Sentry
[
https://issues.apache.org/jira/browse/HIVE-19870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514776#comment-16514776
]
Hive QA commented on HIVE-19870:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927952/HIVE-19870.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14537 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=258)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11824/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11824/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11824/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927952 - PreCommit-HIVE-Build
> HCatalog dynamic partition query can fail, if the table path is managed by
> Sentry
> -
>
> Key: HIVE-19870
> URL: https://issues.apache.org/jira/browse/HIVE-19870
> Project: Hive
> Issue Type: Bug
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19870.2.patch, HIVE-19870.3.patch,
> HIVE-19870.4.patch, HIVE-19870.patch
>
>
> The central issue is that HCatalog is assuming it needs to handle the storage
> based authorization features. When a job completes, in HCatalog's file
> committing phase it tries to manually set the permissions for the table for
> authorization's sake, which makes it go against auto-authorization managment
> features provided by Sentry.
> The offending code is specifically at
> [https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L373-L374]
> and
> [https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L380-L385]
> and
> [https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L952-L954]
> where you can notice that _after_ it moves the files created by the job into
> their respective partition directories under the final table destination, it
> goes onto trying to perform chmod/chgrp operations which will fail out
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[
https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514765#comment-16514765
]
Hive QA commented on HIVE-17852:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 84m
57s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
38s{color} | {color:red} branch/itests/hive-unit cannot run setBugDatabaseInfo
from findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
21s{color} | {color:red} branch/metastore cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
2s{color} | {color:red} branch/ql cannot run setBugDatabaseInfo from findbugs
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
45s{color} | {color:red} branch/standalone-metastore cannot run
setBugDatabaseInfo from findbugs {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 14m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 49m
59s{color} | {color:red} root: The patch generated 1352 new + 248722 unchanged
- 1485 fixed = 250074 total (was 250207) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
43s{color} | {color:red} itests/hive-unit: The patch generated 150 new + 12313
unchanged - 157 fixed = 12463 total (was 12470) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 28m
16s{color} | {color:red} ql: The patch generated 733 new + 128073 unchanged -
839 fixed = 128806 total (was 128912) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 3m
42s{color} | {color:red} standalone-metastore: The patch generated 471 new +
19529 unchanged - 491 fixed = 2 total (was 20020) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
53s{color} | {color:red} patch/itests/hive-unit cannot run setBugDatabaseInfo
from findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
36s{color} | {color:red} patch/metastore cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 5m
2s{color} | {color:red} patch/ql cannot run setBugDatabaseInfo from findbugs
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
4s{color} | {color:red} patch/standalone-metastore cannot run
setBugDatabaseInfo from findbugs {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 5m
54s{color} | {color:red} root generated 2 new + 367 unchanged - 2 fixed = 369
total (was 369) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
57s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} |
[jira] [Commented] (HIVE-19870) HCatalog dynamic partition query can fail, if the table path is managed by Sentry
[
https://issues.apache.org/jira/browse/HIVE-19870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514762#comment-16514762
]
Hive QA commented on HIVE-19870:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
49s{color} | {color:blue} hcatalog/core in master has 33 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} hcatalog/core: The patch generated 0 new + 41
unchanged - 9 fixed = 41 total (was 50) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11824/dev-support/hive-personality.sh
|
| git revision | master / 5ba634a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: hcatalog/core U: hcatalog/core |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11824/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HCatalog dynamic partition query can fail, if the table path is managed by
> Sentry
> -
>
> Key: HIVE-19870
> URL: https://issues.apache.org/jira/browse/HIVE-19870
> Project: Hive
> Issue Type: Bug
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19870.2.patch, HIVE-19870.3.patch,
> HIVE-19870.4.patch, HIVE-19870.patch
>
>
> The central issue is that HCatalog is assuming it needs to handle the storage
> based authorization features. When a job completes, in HCatalog's file
> committing phase it tries to manually set the permissions for the table for
> authorization's sake, which makes it go against auto-authorization managment
> features provided by Sentry.
> The offending code is specifically at
> [https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L373-L374]
> and
> [https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L380-L385]
> and
> [https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L952-L954]
> where you can notice that _after_ it moves the files created by the job into
> their respective partition directories under the final table destination, it
> goes onto trying to perform chmod/chgrp operations which will
[jira] [Commented] (HIVE-19584) Dictionary encoding for string types
[
https://issues.apache.org/jira/browse/HIVE-19584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514759#comment-16514759
]
Hive QA commented on HIVE-19584:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927824/HIVE-19584.6.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14539 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=258)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11823/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11823/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11823/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927824 - PreCommit-HIVE-Build
> Dictionary encoding for string types
>
>
> Key: HIVE-19584
> URL: https://issues.apache.org/jira/browse/HIVE-19584
> Project: Hive
> Issue Type: Sub-task
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-19584.1.patch, HIVE-19584.2.patch,
> HIVE-19584.3.patch, HIVE-19584.4.patch, HIVE-19584.5.patch, HIVE-19584.6.patch
>
>
> Apache Arrow supports dictionary encoding for some data types. So implement
> dictionary encoding for string types in Arrow SerDe.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19734) Beeline: When beeline-site.xml is present, beeline does not honor -n (username) and -p (password) arguments
[ https://issues.apache.org/jira/browse/HIVE-19734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-19734: - Attachment: HIVE-19734.2-branch-3.patch > Beeline: When beeline-site.xml is present, beeline does not honor -n > (username) and -p (password) arguments > --- > > Key: HIVE-19734 > URL: https://issues.apache.org/jira/browse/HIVE-19734 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Thejas M Nair >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19734.1.patch, HIVE-19734.2-branch-3.patch, > HIVE-19734.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19734) Beeline: When beeline-site.xml is present, beeline does not honor -n (username) and -p (password) arguments
[ https://issues.apache.org/jira/browse/HIVE-19734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-19734: - Attachment: (was: HIVE-19801.1-branch-3.patch) > Beeline: When beeline-site.xml is present, beeline does not honor -n > (username) and -p (password) arguments > --- > > Key: HIVE-19734 > URL: https://issues.apache.org/jira/browse/HIVE-19734 > Project: Hive > Issue Type: Bug > Components: Beeline >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Thejas M Nair >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19734.1.patch, HIVE-19734.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19920) Schematool fails in embedded mode when auth is on
[ https://issues.apache.org/jira/browse/HIVE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514752#comment-16514752 ] Thejas M Nair commented on HIVE-19920: -- +1 > Schematool fails in embedded mode when auth is on > - > > Key: HIVE-19920 > URL: https://issues.apache.org/jira/browse/HIVE-19920 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19920.1.patch > > > This is a follow up of HIVE-19775. We need to override more properties in > embedded hs2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19584) Dictionary encoding for string types
[
https://issues.apache.org/jira/browse/HIVE-19584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514751#comment-16514751
]
Hive QA commented on HIVE-19584:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
49s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
46s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m
50s{color} | {color:red} ql in master failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
46s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 5m
0s{color} | {color:blue} ql in master has 2277 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m
17s{color} | {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m
19s{color} | {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 19s{color}
| {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
13s{color} | {color:red} ql: The patch generated 249 new + 515 unchanged - 132
fixed = 764 total (was 647) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
54s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11823/dev-support/hive-personality.sh
|
| git revision | master / 5ba634a |
| Default Java | 1.8.0_111 |
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11823/yetus/branch-compile-ql.txt
|
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11823/yetus/patch-mvninstall-ql.txt
|
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11823/yetus/patch-compile-ql.txt
|
| javac |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11823/yetus/patch-compile-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11823/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11823/yetus/patch-findbugs-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11823/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Dictionary encoding for string types
>
>
> Key: HIVE-19584
> URL: https://issues.apache.org/jira/browse/HIVE-19584
> Project: Hive
> Issue Type: Sub-task
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-19584.1.patch, HIVE-19584.2.patch,
> HIVE-19584.3.patch, HIVE-19584.4.patch, HIVE-19584.5.patch, HIVE-19584.6.patch
>
>
> Apache Arrow supports dictionary encoding for some data types. So
[jira] [Commented] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[
https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514739#comment-16514739
]
Hive QA commented on HIVE-17852:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927822/HIVE-17852.09.patch
{color:green}SUCCESS:{color} +1 due to 34 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 14534 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=258)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testShowLocksFilterOptions
(batchId=306)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
(batchId=244)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitions
(batchId=244)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert
(batchId=244)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=244)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
(batchId=244)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
(batchId=244)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
(batchId=244)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
(batchId=244)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
(batchId=244)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11822/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11822/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11822/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927822 - PreCommit-HIVE-Build
> remove support for list bucketing "stored as directories" in 3.0
>
>
> Key: HIVE-17852
> URL: https://issues.apache.org/jira/browse/HIVE-17852
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Laszlo Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-17852.01.patch, HIVE-17852.02.patch,
> HIVE-17852.03.patch, HIVE-17852.04.patch, HIVE-17852.05.patch,
> HIVE-17852.06.patch, HIVE-17852.07.patch, HIVE-17852.08.patch,
> HIVE-17852.09.patch
>
>
> From the email thread:
> 1) LB, when stored as directories, adds a lot of low-level complexity to Hive
> tables that has to be accounted for in many places in the code where the
> files are written or modified - from FSOP to ACID/replication/export.
> 2) While working on some FSOP code I noticed that some of that logic is
> broken - e.g. the duplicate file removal from tasks, a pretty fundamental
> correctness feature in Hive, may be broken. LB also doesn’t appear to be
> compatible with e.g. regular bucketing.
> 3) The feature hasn’t seen development activity in a while; it also doesn’t
> appear to be used a lot.
> Keeping with the theme of cleaning up “legacy” code for 3.0, I was proposing
> we remove it.
> (2) also suggested that, if needed, it might be easier to implement similar
> functionality by adding some flexibility to partitions (which LB directories
> look like anyway); that would also keep the logic on a higher level of
> abstraction (split generation, partition pruning) as opposed to many
> low-level places like FSOP, etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19711) Refactor Hive Schema Tool
[
https://issues.apache.org/jira/browse/HIVE-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514715#comment-16514715
]
Hive QA commented on HIVE-19711:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927883/HIVE-19711.05.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14537 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=258)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11821/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11821/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11821/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927883 - PreCommit-HIVE-Build
> Refactor Hive Schema Tool
> -
>
> Key: HIVE-19711
> URL: https://issues.apache.org/jira/browse/HIVE-19711
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-19711.01.patch, HIVE-19711.02.patch,
> HIVE-19711.03.patch, HIVE-19711.04.patch, HIVE-19711.05.patch
>
>
> HiveSchemaTool is an 1500 lines long class trying to do everything It shold
> be cut into multiple classes doing smaller components.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19890) ACID: Inherit bucket-id from original ROW_ID for delete deltas
[ https://issues.apache.org/jira/browse/HIVE-19890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-19890: --- Attachment: HIVE-19890.1.patch > ACID: Inherit bucket-id from original ROW_ID for delete deltas > -- > > Key: HIVE-19890 > URL: https://issues.apache.org/jira/browse/HIVE-19890 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-19890.1.patch > > > The ACID delete deltas for unbucketed tables are written to arbitrary files, > which should instead be shuffled using the bucket-id instead of hash(ROW__ID). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19890) ACID: Inherit bucket-id from original ROW_ID for delete deltas
[ https://issues.apache.org/jira/browse/HIVE-19890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-19890: --- Status: Patch Available (was: Open) > ACID: Inherit bucket-id from original ROW_ID for delete deltas > -- > > Key: HIVE-19890 > URL: https://issues.apache.org/jira/browse/HIVE-19890 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-19890.1.patch > > > The ACID delete deltas for unbucketed tables are written to arbitrary files, > which should instead be shuffled using the bucket-id instead of hash(ROW__ID). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19725) Add ability to dump non-native tables in replication metadata dump
[ https://issues.apache.org/jira/browse/HIVE-19725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-19725: --- Attachment: HIVE-19725.07.patch > Add ability to dump non-native tables in replication metadata dump > -- > > Key: HIVE-19725 > URL: https://issues.apache.org/jira/browse/HIVE-19725 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.0.0, 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: Repl, pull-request-available > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19725.01.patch, HIVE-19725.02.patch, > HIVE-19725.03.patch, HIVE-19725.04.patch, HIVE-19725.05.patch, > HIVE-19725.06-branch-3.patch, HIVE-19725.07.patch > > > if hive.repl.dump.metadata.only is set to true, allow dumping non native > tables also. > Data dump for non-native tables should never be allowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19711) Refactor Hive Schema Tool
[
https://issues.apache.org/jira/browse/HIVE-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514714#comment-16514714
]
Hive QA commented on HIVE-19711:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} beeline in master has 70 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
48s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
15s{color} | {color:red} root: The patch generated 22 new + 238 unchanged - 93
fixed = 260 total (was 331) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} beeline: The patch generated 21 new + 72 unchanged -
63 fixed = 93 total (was 135) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 166
unchanged - 30 fixed = 167 total (was 196) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
42s{color} | {color:red} beeline generated 5 new + 51 unchanged - 19 fixed = 56
total (was 70) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 52m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:beeline |
| | org.apache.hive.beeline.BeeLineOpts.getInitFiles() may expose internal
representation by returning BeeLineOpts.initFiles At BeeLineOpts.java:by
returning BeeLineOpts.initFiles At BeeLineOpts.java:[line 469] |
| | org.apache.hive.beeline.BeeLineOpts.setInitFiles(String[]) may expose
internal representation by storing an externally mutable object into
BeeLineOpts.initFiles At BeeLineOpts.java:by storing an externally mutable
object into BeeLineOpts.initFiles At BeeLineOpts.java:[line 473] |
| | org.apache.hive.beeline.BeeLineOpts.env should be package protected At
BeeLineOpts.java: At BeeLineOpts.java:[line 143] |
| |
org.apache.hive.beeline.schematool.HiveSchemaToolTaskAlterCatalog.execute()
passes a nonconstant String to an execute or addBatch method on an SQL
statement At HiveSchemaToolTaskAlterCatalog.java:to an execute or addBatch
method on an SQL statement At HiveSchemaToolTaskAlterCatalog.java:[line 72] |
| | Found reliance on default encoding in
org.apache.hive.beeline.schematool.HiveSchemaToolTaskValidate.findCreateTable(String,
List):in
org.apache.hive.beeline.schematool.HiveSchemaToolTaskValidate.findCreateTable(String,
List): new java.io.FileReader(String) At
HiveSchemaToolTaskValidate.java:[line 285] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle
[jira] [Comment Edited] (HIVE-19916) master-txnstats branch - integrate with HIVE-19382
[ https://issues.apache.org/jira/browse/HIVE-19916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514679#comment-16514679 ] Steve Yeom edited comment on HIVE-19916 at 6/16/18 7:43 AM: I think I did not clearly mention. Sorry and thank you for your great help Sergey. 1. The fact is the write id lists for the query for conf is generated and saved into "conf" at acquireLocks() right at the start of execution. 2. Sergey may be right in saying that the current patch may access the Metastore. I will double check. 3. For stats updaters, the current or original patch gets a writeId list from "conf". The reason why I should take conf-version-writeId-list for a write entity table is that at acquireLocks() we get a new write id (after +1) for a write entity table of the query. 4. But in case of StatsOptimizer I need to get a writeId list for a read entity table and use it. But "conf" may not have a writeIdList for the read entity table. So I think I have obtained a writeId list for a read entity table based on the global transaction database snapshot that is already taken at the start of query plan optimization. But I think this can be the final version for the transaction since I think we can determine commit level of a table write with this version. I.e., with this version, we can decide open->committed write instance, open->aborted instance, new open write instance when the equivalence method for the patch is invoked, But I will double check. P.S.: I thought the current patch of HIVE-19382 is for checking and regenerating write entity table write id lists for I guess UPDATE/DELETE conflict resolution at TxnHandler.commitTxns(). Again only for write id lists for write entity tables. So this does not affect read entity tables. Hey [~jcamachorodriguez] please let me know if my understanding on the current temporary patch is not correct. I may add more after checking.. was (Author: steveyeom2017): I think I did not clearly mention. Sorry and thank you for your great help Sergey. 1. The fact is the write id lists for the query for conf is generated and saved into "conf" at acquireLocks() right at the start of execution. 2. Sergey may be right in saying that the current patch may access the Metastore. I will double check. 3. For stats updaters, the current or original patch gets a writeId list from "conf". The reason why I should take conf-version-writeId-list for a write entity table is that at acquireLocks() we get a new write id (after +1) for a write entity table of the query. 4. But in case of StatsOptimizer I need to get a writeId list for a read entity table and use it. But "conf" may not have a writeIdList for the read entity table. So I think I have obtained a writeId list for a read entity table based on the global transaction database snapshot that is already taken at the start of query plan optimization. But I think this can be the final version for the transaction since I think we can determine commit level of a table write with this version. I.e., with this version, we can decide open->committed write instance, open->aborted instance, new open write instance when the equivalence method for the patch is invoked, But I will double check. P.S.: I thought the current patch of HIVE-19382 is for checking and regenerating write entity table write id lists for I guess UPDATE/DELETE conflict resolution at TxnHandler.commitTxns(). Again only for write id lists for write entity tables. So this does not affect read entity tables. I may add more after checking.. > master-txnstats branch - integrate with HIVE-19382 > -- > > Key: HIVE-19916 > URL: https://issues.apache.org/jira/browse/HIVE-19916 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Steve Yeom >Priority: Major > > There's some code in original txn stats patch that may go to metastore to get > write Ids. This code should not go to metastore, it should fail instead. > HIVE-19382 should ensure that we have correct IDs already present during > optimizer - they are using by e.g. materialized view optimizer, so they > should be there; if they are not present, some integration might be needed so > that txn stats optimizations also have access to those write Ids. > cc [~jcamachorodriguez] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19916) master-txnstats branch - integrate with HIVE-19382
[ https://issues.apache.org/jira/browse/HIVE-19916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514679#comment-16514679 ] Steve Yeom edited comment on HIVE-19916 at 6/16/18 7:35 AM: I think I did not clearly mention. Sorry and thank you for your great help Sergey. 1. The fact is the write id lists for the query for conf is generated and saved into "conf" at acquireLocks() right at the start of execution. 2. Sergey may be right in saying that the current patch may access the Metastore. I will double check. 3. For stats updaters, the current or original patch gets a writeId list from "conf". The reason why I should take conf-version-writeId-list for a write entity table is that at acquireLocks() we get a new write id (after +1) for a write entity table of the query. 4. But in case of StatsOptimizer I need to get a writeId list for a read entity table and use it. But "conf" may not have a writeIdList for the read entity table. So I think I have obtained a writeId list for a read entity table based on the global transaction database snapshot that is already taken at the start of query plan optimization. But I think this can be the final version for the transaction since I think we can determine commit level of a table write with this version. I.e., with this version, we can decide open->committed write instance, open->aborted instance, new open write instance when the equivalence method for the patch is invoked, But I will double check. P.S.: I thought the current patch of HIVE-19382 is for checking and regenerating write entity table write id lists for I guess UPDATE/DELETE conflict resolution at TxnHandler.commitTxns(). Again only for write id lists for write entity tables. So this does not affect read entity tables. I may add more after checking.. was (Author: steveyeom2017): I think I did not clearly mentioned. Sorry Sergey. The current or original patch gets conf version writeId list. In other words, it uses already taken global transaction snspshot before starting query optimization and get writeIdList based on the transaction snapshot like the case of MV and cache result invalidation. The same way! If needed, I can provide more details. Thank you Sergey for your great help! > master-txnstats branch - integrate with HIVE-19382 > -- > > Key: HIVE-19916 > URL: https://issues.apache.org/jira/browse/HIVE-19916 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Steve Yeom >Priority: Major > > There's some code in original txn stats patch that may go to metastore to get > write Ids. This code should not go to metastore, it should fail instead. > HIVE-19382 should ensure that we have correct IDs already present during > optimizer - they are using by e.g. materialized view optimizer, so they > should be there; if they are not present, some integration might be needed so > that txn stats optimizations also have access to those write Ids. > cc [~jcamachorodriguez] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19629) Enable Decimal64 reader after orc version upgrade
[
https://issues.apache.org/jira/browse/HIVE-19629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514706#comment-16514706
]
Hive QA commented on HIVE-19629:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927938/HIVE-19629.16.patch
{color:green}SUCCESS:{color} +1 due to 31 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14539 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11820/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11820/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11820/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927938 - PreCommit-HIVE-Build
> Enable Decimal64 reader after orc version upgrade
> -
>
> Key: HIVE-19629
> URL: https://issues.apache.org/jira/browse/HIVE-19629
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19629.1.patch, HIVE-19629.10.patch,
> HIVE-19629.11.patch, HIVE-19629.12.patch, HIVE-19629.13.patch,
> HIVE-19629.14.patch, HIVE-19629.15.patch, HIVE-19629.16.patch,
> HIVE-19629.2.patch, HIVE-19629.3.patch, HIVE-19629.4.patch,
> HIVE-19629.5.patch, HIVE-19629.6.patch, HIVE-19629.7.patch,
> HIVE-19629.8.patch, HIVE-19629.9.patch
>
>
> ORC 1.5.0 supports new fast decimal 64 reader. New VRB has to be created for
> making use of decimal 64 column vectors. Also LLAP IO will need a new reader
> to reader from long stream to decimal 64.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19629) Enable Decimal64 reader after orc version upgrade
[
https://issues.apache.org/jira/browse/HIVE-19629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514697#comment-16514697
]
Hive QA commented on HIVE-19629:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} llap-server in master has 83 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
55s{color} | {color:blue} ql in master has 2277 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
20s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} llap-server: The patch generated 26 new + 265
unchanged - 10 fixed = 291 total (was 275) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
2s{color} | {color:red} ql: The patch generated 82 new + 1946 unchanged - 6
fixed = 2028 total (was 1952) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
9s{color} | {color:red} ql generated 5 new + 2277 unchanged - 0 fixed = 2282
total (was 2277) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 56s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| |
org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatchCtx.setRowDataTypePhysicalVariations(DataTypePhysicalVariation[])
may expose internal representation by storing an externally mutable object
into VectorizedRowBatchCtx.rowDataTypePhysicalVariations At
VectorizedRowBatchCtx.java:by storing an externally mutable object into
VectorizedRowBatchCtx.rowDataTypePhysicalVariations At
VectorizedRowBatchCtx.java:[line 168] |
| | org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.getArgDescs()
may expose internal representation by returning VectorUDFAdaptor.argDescs At
VectorUDFAdaptor.java:by returning VectorUDFAdaptor.argDescs At
VectorUDFAdaptor.java:[line 240] |
| |
org.apache.hadoop.hive.ql.exec.vector.udf.VectorUDFAdaptor.setArgDescs(VectorUDFArgDesc[])
may expose internal representation by storing an externally mutable object
into VectorUDFAdaptor.argDescs At VectorUDFAdaptor.java:by storing an
externally mutable object into VectorUDFAdaptor.argDescs At
VectorUDFAdaptor.java:[line 244] |
| | Switch statement found in
org.apache.hadoop.hive.ql.io.orc.WriterImpl.setColumn(int, ColumnVector,
ObjectInspector, Object) where default case is missing At
WriterImpl.java:ColumnVector, ObjectInspector, Object) where default case is
missing At WriterImpl.java:[lines 138-227] |
| | Self assignment of physicalContext in
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.resolve(PhysicalContext)
At
