[jira] [Commented] (HIVE-17824) msck repair table should drop the missing partitions from metastore
[
https://issues.apache.org/jira/browse/HIVE-17824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427986#comment-16427986
]
Hive QA commented on HIVE-17824:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
51s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 50 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10031/dev-support/hive-personality.sh
|
| git revision | master / eb40ea5 |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10031/yetus/patch-asflicense-problems.txt
|
| modules | C: itests/util ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10031/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> msck repair table should drop the missing partitions from metastore
> ---
>
> Key: HIVE-17824
> URL: https://issues.apache.org/jira/browse/HIVE-17824
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-17824.1.patch, HIVE-17824.2.patch
>
>
> {{msck repair table }} is often used in environments where the new
> partitions are loaded as directories on HDFS or S3 and users want to create
> the missing partitions in bulk. However, currently it only supports addition
> of missing partitions. If there are any partitions which are present in
> metastore but not on the FileSystem, it should also delete them so that it
> truly repairs the table metadata.
> We should be careful not to break backwards compatibility so we should either
> introduce a new config or keyword to add support to delete unnecessary
> partitions from the metastore. This way users who want the old behavior can
> easily turn it off.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-12369) Native Vector GroupBy
[
https://issues.apache.org/jira/browse/HIVE-12369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427975#comment-16427975
]
Hive QA commented on HIVE-12369:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12917609/HIVE-12369.091.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 324 failed/errored test(s), 13579 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestCopyUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=252)
TestExportImport - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=252)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=96)
[jira] [Commented] (HIVE-16944) schematool -dbType hive should give some more feedback/assistance
[
https://issues.apache.org/jira/browse/HIVE-16944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427971#comment-16427971
]
Bharathkrishna Guruvayoor Murali commented on HIVE-16944:
-
Thanks [~pvary] for looking at the patch.
As of now, the options URL and driver are only used when dbType is hive along
with metaDbType, which are used only if upgrading the system catalog for Hive.
I have added the options metaDbType, URL and driver to the confluence page.
> schematool -dbType hive should give some more feedback/assistance
> -
>
> Key: HIVE-16944
> URL: https://issues.apache.org/jira/browse/HIVE-16944
> Project: Hive
> Issue Type: Bug
>Reporter: Carter Shanklin
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Attachments: HIVE-16944.1.patch
>
>
> Given the other ways schematool is used, the most obvious guess I would have
> for initializing the Hive schema is:
> {code}
> schematool -metaDbType mysql -dbType hive -initSchema
> {code}
> Unfortunately that fails with this NPE:
> {code}
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getDbCommandParser(HiveSchemaHelper.java:570)
> at
> org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getDbCommandParser(HiveSchemaHelper.java:564)
> at
> org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getDbCommandParser(HiveSchemaHelper.java:560)
> at
> org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper$HiveCommandParser.(HiveSchemaHelper.java:373)
> at
> org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getDbCommandParser(HiveSchemaHelper.java:573)
> at
> org.apache.hive.beeline.HiveSchemaTool.getDbCommandParser(HiveSchemaTool.java:165)
> at
> org.apache.hive.beeline.HiveSchemaTool.(HiveSchemaTool.java:101)
> at org.apache.hive.beeline.HiveSchemaTool.(HiveSchemaTool.java:90)
> at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1166)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
> {code}
> Two additional arguments are needed:
> -url jdbc:hive2://localhost:1/default -driver
> org.apache.hive.jdbc.HiveDriver
> If the user does not supply these for dbType hive, schematool should detect
> and error out appropriately, plus give an example of what it's looking for.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18651) Expose additional Spark metrics
[ https://issues.apache.org/jira/browse/HIVE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18651: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master, thanks Vihang for the review! > Expose additional Spark metrics > --- > > Key: HIVE-18651 > URL: https://issues.apache.org/jira/browse/HIVE-18651 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18651.1.patch, HIVE-18651.2.patch, > HIVE-18651.3.patch > > > There have been multiples additional metrics that get collected via Spark > (such as executor CPU time spent). We should expose them in HoS. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19102) Vectorization: Suppress known Q file bugs
[ https://issues.apache.org/jira/browse/HIVE-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19102: Status: Patch Available (was: In Progress) > Vectorization: Suppress known Q file bugs > - > > Key: HIVE-19102 > URL: https://issues.apache.org/jira/browse/HIVE-19102 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19102.01.patch, HIVE-19102.02.patch, > HIVE-19102.03.patch > > > There are known bugs recently found and reported that occur when > vectorization is turn on in Q files. Until those bugs are fixed, add SET > statements to the top of the Q files that suppress vectorization. > Change a few EXPLAIN VECTORIZATION to EXPLAIN where it isn't needed and > creates an unnecessary Q file difference during enable by default experiments. > * +TestCliDriver+ > * > ** +Execution Failures+ > *** *input_lazyserde.q* > HIVE-19088 > *** *input_lazyserde2.q* > HIVE-19088, too. > *** *nested_column_pruning.q* > HIVE-19016 > *** *parquet_map_of_arrays_of_ints.q* > HIVE-19015 > *** *parquet_map_of_maps.q* > HIVE-19015, too. > *** *parquet_nested_complex.q* > HIVE-19016 > * > ** +Wrong Results+ > *** *delete_orig_table.q* > HIVE-19109 > *** *escape_crlf.q* > HIVE-19118 > *** *offset_limit_global_optimizer.q* > Added ORDER BY clauses to fix different vectorization intermediate > results due to LIMIT clause. > *** *parquet_ppd_decimal.q* > HIVE-19108 > *** *udf_context_aware.q* > Test detects vectorization and output changes. > Added "set hive.test.vectorized.execution.enabled.override=disable;" > *** *vector_udf3.q* > Test detects vectorization and output changes. > Added "set hive.test.vectorized.execution.enabled.override=disable;" > * +TestContribCliDriver+ > * > ** +Wrong Results+ > *** *udf_example_arraymapstruct.q* > HIVE-19110 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19102) Vectorization: Suppress known Q file bugs
[ https://issues.apache.org/jira/browse/HIVE-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19102: Attachment: HIVE-19102.03.patch > Vectorization: Suppress known Q file bugs > - > > Key: HIVE-19102 > URL: https://issues.apache.org/jira/browse/HIVE-19102 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19102.01.patch, HIVE-19102.02.patch, > HIVE-19102.03.patch > > > There are known bugs recently found and reported that occur when > vectorization is turn on in Q files. Until those bugs are fixed, add SET > statements to the top of the Q files that suppress vectorization. > Change a few EXPLAIN VECTORIZATION to EXPLAIN where it isn't needed and > creates an unnecessary Q file difference during enable by default experiments. > * +TestCliDriver+ > * > ** +Execution Failures+ > *** *input_lazyserde.q* > HIVE-19088 > *** *input_lazyserde2.q* > HIVE-19088, too. > *** *nested_column_pruning.q* > HIVE-19016 > *** *parquet_map_of_arrays_of_ints.q* > HIVE-19015 > *** *parquet_map_of_maps.q* > HIVE-19015, too. > *** *parquet_nested_complex.q* > HIVE-19016 > * > ** +Wrong Results+ > *** *delete_orig_table.q* > HIVE-19109 > *** *escape_crlf.q* > HIVE-19118 > *** *offset_limit_global_optimizer.q* > Added ORDER BY clauses to fix different vectorization intermediate > results due to LIMIT clause. > *** *parquet_ppd_decimal.q* > HIVE-19108 > *** *udf_context_aware.q* > Test detects vectorization and output changes. > Added "set hive.test.vectorized.execution.enabled.override=disable;" > *** *vector_udf3.q* > Test detects vectorization and output changes. > Added "set hive.test.vectorized.execution.enabled.override=disable;" > * +TestContribCliDriver+ > * > ** +Wrong Results+ > *** *udf_example_arraymapstruct.q* > HIVE-19110 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19102) Vectorization: Suppress known Q file bugs
[ https://issues.apache.org/jira/browse/HIVE-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19102: Status: In Progress (was: Patch Available) > Vectorization: Suppress known Q file bugs > - > > Key: HIVE-19102 > URL: https://issues.apache.org/jira/browse/HIVE-19102 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19102.01.patch, HIVE-19102.02.patch > > > There are known bugs recently found and reported that occur when > vectorization is turn on in Q files. Until those bugs are fixed, add SET > statements to the top of the Q files that suppress vectorization. > Change a few EXPLAIN VECTORIZATION to EXPLAIN where it isn't needed and > creates an unnecessary Q file difference during enable by default experiments. > * +TestCliDriver+ > * > ** +Execution Failures+ > *** *input_lazyserde.q* > HIVE-19088 > *** *input_lazyserde2.q* > HIVE-19088, too. > *** *nested_column_pruning.q* > HIVE-19016 > *** *parquet_map_of_arrays_of_ints.q* > HIVE-19015 > *** *parquet_map_of_maps.q* > HIVE-19015, too. > *** *parquet_nested_complex.q* > HIVE-19016 > * > ** +Wrong Results+ > *** *delete_orig_table.q* > HIVE-19109 > *** *escape_crlf.q* > HIVE-19118 > *** *offset_limit_global_optimizer.q* > Added ORDER BY clauses to fix different vectorization intermediate > results due to LIMIT clause. > *** *parquet_ppd_decimal.q* > HIVE-19108 > *** *udf_context_aware.q* > Test detects vectorization and output changes. > Added "set hive.test.vectorized.execution.enabled.override=disable;" > *** *vector_udf3.q* > Test detects vectorization and output changes. > Added "set hive.test.vectorized.execution.enabled.override=disable;" > * +TestContribCliDriver+ > * > ** +Wrong Results+ > *** *udf_example_arraymapstruct.q* > HIVE-19110 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-12369) Native Vector GroupBy
[
https://issues.apache.org/jira/browse/HIVE-12369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427906#comment-16427906
]
Hive QA commented on HIVE-12369:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 229 new + 336 unchanged - 22
fixed = 565 total (was 358) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
16s{color} | {color:red} The patch generated 50 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10030/dev-support/hive-personality.sh
|
| git revision | master / 2f802e9 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10030/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10030/yetus/patch-asflicense-problems.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10030/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Native Vector GroupBy
> -
>
> Key: HIVE-12369
> URL: https://issues.apache.org/jira/browse/HIVE-12369
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-12369.01.patch, HIVE-12369.02.patch,
> HIVE-12369.05.patch, HIVE-12369.06.patch, HIVE-12369.091.patch
>
>
> Implement Native Vector GroupBy using fast hash table technology developed
> for Native Vector MapJoin, etc.
> Patch is currently limited to a single Long key with a single COUNT
> aggregation. Or, a single Long key and no aggregation also known as
> duplicate reduction.
> 3 new classes introduces that stored the count in the slot table and don't
> allocate hash elements:
> {noformat}
> COUNT(column) VectorGroupByHashLongKeyCountColumnOperator
> COUNT(key) VectorGroupByHashLongKeyCountKeyOperator
> COUNT(*) VectorGroupByHashLongKeyCountStarOperator
> {noformat}
> And the duplicate reduction operator a single Long key:
> {noformat}
> VectorGroupByHashLongKeyDuplicateReductionOperator
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18910) Migrate to Murmur hash for shuffle and bucketing
[
https://issues.apache.org/jira/browse/HIVE-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427884#comment-16427884
]
Hive QA commented on HIVE-18910:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12917641/HIVE-18910.23.patch
{color:green}SUCCESS:{color} +1 due to 10 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 208 failed/errored test(s), 13579 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestCopyUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=252)
TestExportImport - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=252)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Commented] (HIVE-18525) Add explain plan to Hive on Spark Web UI
[
https://issues.apache.org/jira/browse/HIVE-18525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427862#comment-16427862
]
Sahil Takiar commented on HIVE-18525:
-
[~aihuaxu] thanks for taking a look. I think the test failures are actually
unrelated. A Hive QA run on HIVE-19033 at a similar time showed an almost
identical list of test failures, with the exception of {{TestCliDriver}}
{{auto_sortmerge_join_2.q}} which I ran locally without any issues, so probably
another flaky test.
> Add explain plan to Hive on Spark Web UI
>
>
> Key: HIVE-18525
> URL: https://issues.apache.org/jira/browse/HIVE-18525
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18525.1.patch, HIVE-18525.2.patch,
> HIVE-18525.3.patch, HIVE-18525.4.patch, Job-Page-Collapsed.png,
> Job-Page-Expanded.png, Map-Explain-Plan.png, Reduce-Explain-Plan.png
>
>
> More of an investigation JIRA. The Spark UI has a "long description" of each
> stage in the Spark DAG. Typically one stage in the Spark DAG corresponds to
> either a {{MapWork}} or {{ReduceWork}} object. It would be useful if the long
> description contained the explain plan of the corresponding work object.
> I'm not sure how much additional overhead this would introduce. If not the
> full explain plan, then maybe a modified one that just lists out all the
> operator tree along with each operator name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18910) Migrate to Murmur hash for shuffle and bucketing
[
https://issues.apache.org/jira/browse/HIVE-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427864#comment-16427864
]
Hive QA commented on HIVE-18910:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
36s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
20s{color} | {color:red} streaming in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
55s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} storage-api: The patch generated 3 new + 97 unchanged
- 3 fixed = 100 total (was 100) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
21s{color} | {color:red} serde: The patch generated 150 new + 214 unchanged - 3
fixed = 364 total (was 217) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} hcatalog/streaming: The patch generated 1 new + 33
unchanged - 0 fixed = 34 total (was 33) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
1s{color} | {color:red} ql: The patch generated 26 new + 1267 unchanged - 3
fixed = 1293 total (was 1270) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch 248 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 51 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10029/dev-support/hive-personality.sh
|
| git revision | master / 2f802e9 |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10029/yetus/patch-mvninstall-hcatalog_streaming.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10029/yetus/patch-mvninstall-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10029/yetus/diff-checkstyle-storage-api.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10029/yetus/diff-checkstyle-serde.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10029/yetus/diff-checkstyle-hcatalog_streaming.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10029/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10029/yetus/whitespace-tabs.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10029/yetus/patch-asflicense-problems.txt
|
| modules | C: storage-api serde hbase-handler hcatalog/streaming itests
itests/hive-blobstore ql standalone-metastore U: . |
| Console output |
[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables
[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427857#comment-16427857 ] Sergey Shelukhin commented on HIVE-19124: - [~ekoifman] can you take a look? thnx > implement a basic major compactor for MM tables > --- > > Key: HIVE-19124 > URL: https://issues.apache.org/jira/browse/HIVE-19124 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Labels: mm-gap-2 > Attachments: HIVE-19124.patch > > > For now, it will run a query directly and only major compactions will be > supported. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19033) Provide an option to purge LLAP IO cache
[ https://issues.apache.org/jira/browse/HIVE-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19033: - Attachment: HIVE-19033.8.patch > Provide an option to purge LLAP IO cache > > > Key: HIVE-19033 > URL: https://issues.apache.org/jira/browse/HIVE-19033 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19033.1.patch, HIVE-19033.2.patch, > HIVE-19033.3.patch, HIVE-19033.4.patch, HIVE-19033.5.patch, > HIVE-19033.6.patch, HIVE-19033.7.patch, HIVE-19033.8.patch, HIVE-19033.8.patch > > > Provide an API endpoint that will trigger purging of LLAP IO cache. Also CLI > tool to invoke the endpoint of all LLAP daemons. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19033) Provide an option to purge LLAP IO cache
[ https://issues.apache.org/jira/browse/HIVE-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427855#comment-16427855 ] Prasanth Jayachandran commented on HIVE-19033: -- Rebased patch to fix build failure. > Provide an option to purge LLAP IO cache > > > Key: HIVE-19033 > URL: https://issues.apache.org/jira/browse/HIVE-19033 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19033.1.patch, HIVE-19033.2.patch, > HIVE-19033.3.patch, HIVE-19033.4.patch, HIVE-19033.5.patch, > HIVE-19033.6.patch, HIVE-19033.7.patch, HIVE-19033.8.patch, HIVE-19033.8.patch > > > Provide an API endpoint that will trigger purging of LLAP IO cache. Also CLI > tool to invoke the endpoint of all LLAP daemons. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19124) implement a basic major compactor for MM tables
[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19124: Status: Patch Available (was: Open) > implement a basic major compactor for MM tables > --- > > Key: HIVE-19124 > URL: https://issues.apache.org/jira/browse/HIVE-19124 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Labels: mm-gap-2 > Attachments: HIVE-19124.patch > > > For now, it will run a query directly and only major compactions will be > supported. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables
[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427853#comment-16427853 ] Sergey Shelukhin commented on HIVE-19124: - Probably also needs a config setting to be able to turn it off if smth goes wrong. > implement a basic major compactor for MM tables > --- > > Key: HIVE-19124 > URL: https://issues.apache.org/jira/browse/HIVE-19124 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Labels: mm-gap-2 > Attachments: HIVE-19124.patch > > > For now, it will run a query directly and only major compactions will be > supported. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables
[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427849#comment-16427849 ] Sergey Shelukhin commented on HIVE-19124: - A query based compactor with tests. Probably needs to be tested in a real cluster, to make sure nothing is messed up w.r.t. state in real metastore. I'll look into that. > implement a basic major compactor for MM tables > --- > > Key: HIVE-19124 > URL: https://issues.apache.org/jira/browse/HIVE-19124 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Labels: mm-gap-2 > Attachments: HIVE-19124.patch > > > For now, it will run a query directly and only major compactions will be > supported. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19124) implement a basic major compactor for MM tables
[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19124: Attachment: HIVE-19124.patch > implement a basic major compactor for MM tables > --- > > Key: HIVE-19124 > URL: https://issues.apache.org/jira/browse/HIVE-19124 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Labels: mm-gap-2 > Attachments: HIVE-19124.patch > > > For now, it will run a query directly and only major compactions will be > supported. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19033) Provide an option to purge LLAP IO cache
[
https://issues.apache.org/jira/browse/HIVE-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427847#comment-16427847
]
Hive QA commented on HIVE-19033:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12917600/HIVE-19033.8.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10028/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10028/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10028/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-04-06 01:45:34.099
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-10028/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-04-06 01:45:34.101
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 2f802e9 HIVE-19085: FastHiveDecimal abs(0) sets sign to +ve
(Gopal V, reviewed by Matt McCline)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 2f802e9 HIVE-19085: FastHiveDecimal abs(0) sets sign to +ve
(Gopal V, reviewed by Matt McCline)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-04-06 01:45:34.614
+ rm -rf ../yetus_PreCommit-HIVE-Build-10028
+ mkdir ../yetus_PreCommit-HIVE-Build-10028
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-10028
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-10028/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java: does not
exist in index
error:
a/itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestActivePassiveHA.java:
does not exist in index
error:
a/itests/hive-unit/src/test/java/org/apache/hive/jdbc/authorization/TestJdbcWithSQLAuthorization.java:
does not exist in index
error: a/llap-client/src/java/org/apache/hadoop/hive/llap/io/api/LlapIo.java:
does not exist in index
error:
a/llap-common/src/gen/protobuf/gen-java/org/apache/hadoop/hive/llap/daemon/rpc/LlapDaemonProtocolProtos.java:
does not exist in index
error:
a/llap-common/src/java/org/apache/hadoop/hive/llap/impl/LlapManagementProtocolClientImpl.java:
does not exist in index
error:
a/llap-common/src/java/org/apache/hadoop/hive/llap/protocol/LlapManagementProtocolPB.java:
does not exist in index
error: a/llap-common/src/protobuf/LlapDaemonProtocol.proto: does not exist in
index
error:
a/llap-server/src/java/org/apache/hadoop/hive/llap/cache/CacheContentsTracker.java:
does not exist in index
error:
a/llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCachePolicy.java:
does not exist in index
error:
a/llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelFifoCachePolicy.java:
does not exist in index
error:
a/llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelLrfuCachePolicy.java:
does not exist in index
error:
a/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/LlapProtocolServerImpl.java:
does not exist in index
error:
a/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/services/impl/LlapIoMemoryServlet.java:
does not exist in index
error:
a/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapIoImpl.java:
does not exist in index
error:
a/llap-server/src/test/org/apache/hadoop/hive/llap/cache/TestLowLevelCacheImpl.java:
does not exist in index
error:
a/llap-server/src/test/org/apache/hadoop/hive/llap/cache/TestOrcMetadataCache.java:
does not exist in index
error:
[jira] [Commented] (HIVE-18840) CachedStore: Prioritize loading of recently accessed tables during prewarm
[
https://issues.apache.org/jira/browse/HIVE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427846#comment-16427846
]
Hive QA commented on HIVE-18840:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12917591/HIVE-18840.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10026/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10026/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10026/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-04-06 01:41:52.193
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-10026/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-04-06 01:41:52.195
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
105cc65..2f802e9 master -> origin/master
+ git reset --hard HEAD
HEAD is now at 105cc65 HIVE-19031 Mark duplicate configs in HiveConf as
deprecated (Alan Gates, reviewed by Thejas Nair)
+ git clean -f -d
Removing ${project.basedir}/
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 2f802e9 HIVE-19085: FastHiveDecimal abs(0) sets sign to +ve
(Gopal V, reviewed by Matt McCline)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-04-06 01:41:56.496
+ rm -rf ../yetus_PreCommit-HIVE-Build-10026
+ mkdir ../yetus_PreCommit-HIVE-Build-10026
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-10026
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-10026/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java:
does not exist in index
Going to apply patch with: git apply -p1
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc7925382271077889119.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc7925382271077889119.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
[ERROR] COMPILATION ERROR :
[ERROR]
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java:[269,18]
method shouldCacheTable in class
org.apache.hadoop.hive.metastore.cache.CachedStore cannot be applied to given
types;
required: java.lang.String,java.lang.String,java.lang.String
found: java.lang.String,java.lang.String
reason: actual and formal argument lists differ in length
[ERROR]
[jira] [Commented] (HIVE-19102) Vectorization: Suppress known Q file bugs
[
https://issues.apache.org/jira/browse/HIVE-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427844#comment-16427844
]
Hive QA commented on HIVE-19102:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12917731/HIVE-19102.02.patch
{color:green}SUCCESS:{color} +1 due to 15 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 102 failed/errored test(s), 13172 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestCopyUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=252)
TestExportImport - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=252)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Assigned] (HIVE-19124) implement a basic major compactor for MM tables
[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-19124: --- > implement a basic major compactor for MM tables > --- > > Key: HIVE-19124 > URL: https://issues.apache.org/jira/browse/HIVE-19124 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Labels: mm-gap-2 > > For now, it will run a query directly and only major compactions will be > supported. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18739) Add support for Export from Acid table
[ https://issues.apache.org/jira/browse/HIVE-18739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427813#comment-16427813 ] Eugene Koifman commented on HIVE-18739: --- patch 13 for test - includes support (mostly) for Import - doesn't address security issue yet > Add support for Export from Acid table > -- > > Key: HIVE-18739 > URL: https://issues.apache.org/jira/browse/HIVE-18739 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Major > Attachments: HIVE-18739.01.patch, HIVE-18739.04.patch, > HIVE-18739.04.patch, HIVE-18739.06.patch, HIVE-18739.08.patch, > HIVE-18739.09.patch, HIVE-18739.10.patch, HIVE-18739.11.patch, > HIVE-18739.12.patch, HIVE-18739.13.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18739) Add support for Export from Acid table
[ https://issues.apache.org/jira/browse/HIVE-18739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-18739: -- Attachment: HIVE-18739.13.patch > Add support for Export from Acid table > -- > > Key: HIVE-18739 > URL: https://issues.apache.org/jira/browse/HIVE-18739 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Major > Attachments: HIVE-18739.01.patch, HIVE-18739.04.patch, > HIVE-18739.04.patch, HIVE-18739.06.patch, HIVE-18739.08.patch, > HIVE-18739.09.patch, HIVE-18739.10.patch, HIVE-18739.11.patch, > HIVE-18739.12.patch, HIVE-18739.13.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19117) hiveserver2 org.apache.thrift.transport.TTransportException error when running 2nd query after minute of inactivity
[
https://issues.apache.org/jira/browse/HIVE-19117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427810#comment-16427810
]
t oo commented on HIVE-19117:
-
[~gopalv] Even with ;http.header.Connection=close it is still an issue
> hiveserver2 org.apache.thrift.transport.TTransportException error when
> running 2nd query after minute of inactivity
> ---
>
> Key: HIVE-19117
> URL: https://issues.apache.org/jira/browse/HIVE-19117
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2, Metastore, Thrift API
>Affects Versions: 2.1.1
> Environment: * Hive 2.1.1 with hive.server2.transport.mode set to
> binary (sample JDBC string is jdbc:hive2://remotehost:1/default)
> * Hadoop 2.8.3
> * Metastore using MySQL
> * Java 8
>Reporter: t oo
>Priority: Blocker
>
> I make a JDBC connection from my SQL tool (ie Squirrel SQL, Oracle SQL
> Developer) to HiveServer2 (running on remote server) with port 1.
> I am able to run some queries successfully. I then do something else (not in
> the SQL tool) for 1-2minutes and then return to my SQL tool and attempt to
> run a query but I get this error:
> {code:java}
> org.apache.thrift.transport.TTransportException: java.net.SocketException:
> Software caused connection abort: socket write error{code}
> If I now disconnect and reconnect in my SQL tool I can run queries again. But
> does anyone know what HiveServer2 settings I should change to prevent the
> error? I assume something in hive-site.xml
> From the hiveserver2 logs below, can see an exact 1 minute gap from 30th min
> to 31stmin where the disconnect happens.
> {code:java}
> 2018-04-05T03:30:41,706 INFO [HiveServer2-Handler-Pool: Thread-36]
> session.SessionState: Resetting thread name to HiveServer2-Handler-Pool:
> Thread-36
> 2018-04-05T03:30:41,712 INFO [HiveServer2-Handler-Pool: Thread-36]
> session.SessionState: Updating thread name to
> c81ec0f9-7a9d-46b6-9708-e7d78520a48a HiveServer2-Handler-Pool: Thread-36
> 2018-04-05T03:30:41,712 INFO [HiveServer2-Handler-Pool: Thread-36]
> session.SessionState: Resetting thread name to HiveServer2-Handler-Pool:
> Thread-36
> 2018-04-05T03:30:41,718 INFO [HiveServer2-Handler-Pool: Thread-36]
> session.SessionState: Updating thread name to
> c81ec0f9-7a9d-46b6-9708-e7d78520a48a HiveServer2-Handler-Pool: Thread-36
> 2018-04-05T03:30:41,719 INFO [HiveServer2-Handler-Pool: Thread-36]
> session.SessionState: Resetting thread name to HiveServer2-Handler-Pool:
> Thread-36
> 2018-04-05T03:31:41,232 INFO [HiveServer2-Handler-Pool: Thread-36]
> thrift.ThriftCLIService: Session disconnected without closing properly.
> 2018-04-05T03:31:41,233 INFO [HiveServer2-Handler-Pool: Thread-36]
> thrift.ThriftCLIService: Closing the session: SessionHandle
> [c81ec0f9-7a9d-46b6-9708-e7d78520a48a]
> 2018-04-05T03:31:41,233 INFO [HiveServer2-Handler-Pool: Thread-36]
> service.CompositeService: Session closed, SessionHandle
> [c81ec0f9-7a9d-46b6-9708-e7d78520a48a], current sessions:0
> 2018-04-05T03:31:41,233 INFO [HiveServer2-Handler-Pool: Thread-36]
> session.SessionState: Updating thread name to
> c81ec0f9-7a9d-46b6-9708-e7d78520a48a HiveServer2-Handler-Pool: Thread-36
> 2018-04-05T03:31:41,233 INFO [HiveServer2-Handler-Pool: Thread-36]
> session.SessionState: Resetting thread name to HiveServer2-Handler-Pool:
> Thread-36
> 2018-04-05T03:31:41,233 INFO [HiveServer2-Handler-Pool: Thread-36]
> session.SessionState: Updating thread name to
> c81ec0f9-7a9d-46b6-9708-e7d78520a48a HiveServer2-Handler-Pool: Thread-36
> 2018-04-05T03:31:41,233 INFO [HiveServer2-Handler-Pool: Thread-36]
> session.HiveSessionImpl: Operation log session directory is deleted:
> /var/hive/hs2log/tmp/c81ec0f9-7a9d-46b6-9708-e7d78520a48a
> 2018-04-05T03:31:41,233 INFO [HiveServer2-Handler-Pool: Thread-36]
> session.SessionState: Resetting thread name to HiveServer2-Handler-Pool:
> Thread-36
> 2018-04-05T03:31:41,236 INFO [HiveServer2-Handler-Pool: Thread-36]
> session.SessionState: Deleted directory:
> /var/hive/scratch/tmp/anonymous/c81ec0f9-7a9d-46b6-9708-e7d78520a48a on fs
> with scheme file
> 2018-04-05T03:31:41,236 INFO [HiveServer2-Handler-Pool: Thread-36]
> session.SessionState: Deleted directory:
> /var/hive/ec2-user/c81ec0f9-7a9d-46b6-9708-e7d78520a48a on fs with scheme file
> 2018-04-05T03:31:41,236 INFO [HiveServer2-Handler-Pool: Thread-36]
> hive.metastore: Closed a connection to metastore, current connections: 1{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19085) FastHiveDecimal abs(0) sets sign to +ve
[
https://issues.apache.org/jira/browse/HIVE-19085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-19085:
---
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Pushed to master, thanks [~mmccline]
> FastHiveDecimal abs(0) sets sign to +ve
> ---
>
> Key: HIVE-19085
> URL: https://issues.apache.org/jira/browse/HIVE-19085
> Project: Hive
> Issue Type: Bug
> Components: Hive, Vectorization
>Affects Versions: 2.3.2
>Reporter: ACOSS
>Assignee: Gopal V
>Priority: Minor
> Labels: fastdecimal, vectorization
> Fix For: 3.0.0
>
> Attachments: HIVE-19085.1.patch
>
>
> Hi,
> We use parquet's table to store the result of others query. Some query use
> the function abs. If the function "abs" take 0 (type decimal) in input, then
> the insert in the parquet's table failed
>
> +Scenario:+
> create table test (col1 decimal(10,2)) stored as parquet;
> insert into test values(0);
> insert into test select abs(col1) from test;
>
> +Result;+
> The insert query crash with the error:
>
> 2018-03-30 17:39:02,123 FATAL [IPC Server handler 2 on 35885]
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
> attempt_1522311557218_0002_m_00_0 - exited : java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row \{"col1":0}
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:169)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row \{"col1":0}
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:562)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:160)
> ... 8 more
> Caused by: java.lang.RuntimeException: Unexpected #3
> at
> org.apache.hadoop.hive.common.type.FastHiveDecimalImpl.fastBigIntegerBytesUnscaled(FastHiveDecimalImpl.java:2550)
> at
> org.apache.hadoop.hive.common.type.FastHiveDecimalImpl.fastBigIntegerBytesScaled(FastHiveDecimalImpl.java:2806)
> at
> org.apache.hadoop.hive.common.type.FastHiveDecimal.fastBigIntegerBytesScaled(FastHiveDecimal.java:295)
> at
> org.apache.hadoop.hive.common.type.HiveDecimal.bigIntegerBytesScaled(HiveDecimal.java:712)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$DecimalDataWriter.decimalToBinary(DataWritableWriter.java:521)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$DecimalDataWriter.write(DataWritableWriter.java:514)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$GroupDataWriter.write(DataWritableWriter.java:204)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$MessageDataWriter.write(DataWritableWriter.java:220)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter.write(DataWritableWriter.java:91)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.write(DataWritableWriteSupport.java:59)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.write(DataWritableWriteSupport.java:31)
> at
> org.apache.parquet.hadoop.InternalParquetRecordWriter.write(InternalParquetRecordWriter.java:121)
> at
> org.apache.parquet.hadoop.ParquetRecordWriter.write(ParquetRecordWriter.java:123)
> at
> org.apache.parquet.hadoop.ParquetRecordWriter.write(ParquetRecordWriter.java:42)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.write(ParquetRecordWriterWrapper.java:112)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.write(ParquetRecordWriterWrapper.java:125)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:762)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:148)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:547)
>
> Problem is probably due to the fastABS method.
[jira] [Updated] (HIVE-19085) FastHiveDecimal abs(0) sets sign to +ve
[
https://issues.apache.org/jira/browse/HIVE-19085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-19085:
---
Summary: FastHiveDecimal abs(0) sets sign to +ve (was: HIVE:
FastHiveDecimal abs(0) sets sign to +ve)
> FastHiveDecimal abs(0) sets sign to +ve
> ---
>
> Key: HIVE-19085
> URL: https://issues.apache.org/jira/browse/HIVE-19085
> Project: Hive
> Issue Type: Bug
> Components: Hive, Vectorization
>Affects Versions: 2.3.2
>Reporter: ACOSS
>Assignee: Gopal V
>Priority: Minor
> Labels: fastdecimal, vectorization
> Attachments: HIVE-19085.1.patch
>
>
> Hi,
> We use parquet's table to store the result of others query. Some query use
> the function abs. If the function "abs" take 0 (type decimal) in input, then
> the insert in the parquet's table failed
>
> +Scenario:+
> create table test (col1 decimal(10,2)) stored as parquet;
> insert into test values(0);
> insert into test select abs(col1) from test;
>
> +Result;+
> The insert query crash with the error:
>
> 2018-03-30 17:39:02,123 FATAL [IPC Server handler 2 on 35885]
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
> attempt_1522311557218_0002_m_00_0 - exited : java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row \{"col1":0}
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:169)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row \{"col1":0}
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:562)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:160)
> ... 8 more
> Caused by: java.lang.RuntimeException: Unexpected #3
> at
> org.apache.hadoop.hive.common.type.FastHiveDecimalImpl.fastBigIntegerBytesUnscaled(FastHiveDecimalImpl.java:2550)
> at
> org.apache.hadoop.hive.common.type.FastHiveDecimalImpl.fastBigIntegerBytesScaled(FastHiveDecimalImpl.java:2806)
> at
> org.apache.hadoop.hive.common.type.FastHiveDecimal.fastBigIntegerBytesScaled(FastHiveDecimal.java:295)
> at
> org.apache.hadoop.hive.common.type.HiveDecimal.bigIntegerBytesScaled(HiveDecimal.java:712)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$DecimalDataWriter.decimalToBinary(DataWritableWriter.java:521)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$DecimalDataWriter.write(DataWritableWriter.java:514)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$GroupDataWriter.write(DataWritableWriter.java:204)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$MessageDataWriter.write(DataWritableWriter.java:220)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter.write(DataWritableWriter.java:91)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.write(DataWritableWriteSupport.java:59)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.write(DataWritableWriteSupport.java:31)
> at
> org.apache.parquet.hadoop.InternalParquetRecordWriter.write(InternalParquetRecordWriter.java:121)
> at
> org.apache.parquet.hadoop.ParquetRecordWriter.write(ParquetRecordWriter.java:123)
> at
> org.apache.parquet.hadoop.ParquetRecordWriter.write(ParquetRecordWriter.java:42)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.write(ParquetRecordWriterWrapper.java:112)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.write(ParquetRecordWriterWrapper.java:125)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:762)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:148)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:547)
>
> Problem is probably due to the fastABS method. This method force "fastSignum"
> to 1 even when the decimal is
[jira] [Commented] (HIVE-19102) Vectorization: Suppress known Q file bugs
[
https://issues.apache.org/jira/browse/HIVE-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427786#comment-16427786
]
Hive QA commented on HIVE-19102:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
56s{color} | {color:blue} Maven dependency ordering for branch {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 50 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 2m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10025/dev-support/hive-personality.sh
|
| git revision | master / 105cc65 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10025/yetus/patch-asflicense-problems.txt
|
| modules | C: contrib ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10025/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: Suppress known Q file bugs
> -
>
> Key: HIVE-19102
> URL: https://issues.apache.org/jira/browse/HIVE-19102
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-19102.01.patch, HIVE-19102.02.patch
>
>
> There are known bugs recently found and reported that occur when
> vectorization is turn on in Q files. Until those bugs are fixed, add SET
> statements to the top of the Q files that suppress vectorization.
> Change a few EXPLAIN VECTORIZATION to EXPLAIN where it isn't needed and
> creates an unnecessary Q file difference during enable by default experiments.
> * +TestCliDriver+
> *
> ** +Execution Failures+
> *** *input_lazyserde.q*
> HIVE-19088
> *** *input_lazyserde2.q*
> HIVE-19088, too.
> *** *nested_column_pruning.q*
> HIVE-19016
> *** *parquet_map_of_arrays_of_ints.q*
> HIVE-19015
> *** *parquet_map_of_maps.q*
> HIVE-19015, too.
> *** *parquet_nested_complex.q*
> HIVE-19016
> *
> ** +Wrong Results+
> *** *delete_orig_table.q*
> HIVE-19109
> *** *escape_crlf.q*
> HIVE-19118
> *** *offset_limit_global_optimizer.q*
> Added ORDER BY clauses to fix different vectorization intermediate
> results due to LIMIT clause.
> *** *parquet_ppd_decimal.q*
> HIVE-19108
> *** *udf_context_aware.q*
> Test detects vectorization and output changes.
> Added "set hive.test.vectorized.execution.enabled.override=disable;"
> *** *vector_udf3.q*
> Test detects vectorization and output changes.
> Added "set hive.test.vectorized.execution.enabled.override=disable;"
> * +TestContribCliDriver+
> *
> ** +Wrong Results+
> *** *udf_example_arraymapstruct.q*
> HIVE-19110
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18783) ALTER TABLE post-commit listener does not include the transactional listener responses
[
https://issues.apache.org/jira/browse/HIVE-18783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427779#comment-16427779
]
Hive QA commented on HIVE-18783:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12917565/HIVE-18783.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 187 failed/errored test(s), 13579 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestCopyUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=252)
TestExportImport - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=252)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Updated] (HIVE-19031) Mark duplicate configs in HiveConf as deprecated
[ https://issues.apache.org/jira/browse/HIVE-19031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-19031: -- Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Patch 3 committed. Thanks Thejas for the review. > Mark duplicate configs in HiveConf as deprecated > > > Key: HIVE-19031 > URL: https://issues.apache.org/jira/browse/HIVE-19031 > Project: Hive > Issue Type: Sub-task > Components: Configuration, Standalone Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Blocker > Fix For: 3.0.0 > > Attachments: HIVE-19031.2.patch, HIVE-19031.3.patch, HIVE-19031.patch > > > There are a number of configuration values that were copied from HiveConf to > MetastoreConf. They have been left in HiveConf for backwards compatibility. > But they need to be marked as deprecated so that users know to use the new > values in MetastoreConf. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19031) Mark duplicate configs in HiveConf as deprecated
[ https://issues.apache.org/jira/browse/HIVE-19031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427762#comment-16427762 ] Alan Gates commented on HIVE-19031: --- In a local run of patch 3 I see the following test issues: ERRORS TestAcidOnTez.testGetSplitsLocks TestAppendPartitions.testAppendPartitionEmptyPartValues TestAppendPartitions.testAppendPartitionEmptyPartValues TestAppendPartitions.testAppendPartitionNullPartValues TestAppendPartitions.testAppendPartitionNullPartValues TestJdbcWithLocalClusterSpark.testSparkQuery TestJdbcWithLocalClusterSpark.testTempTable TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd TestMTQueries.testMTQueries1 TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery TestNegativeCliDriver.alter_notnull_constraint_violation TestNegativeCliDriver.insert_into_acid_notnull TestNegativeCliDriver.insert_into_notnull_constraint TestNegativeCliDriver.insert_multi_into_notnull TestNegativeCliDriver.insert_overwrite_notnull_constraint TestNegativeCliDriver.update_notnull_constraint FAILURES: TestActivePassiveHA.testManualFailover TestBlobstoreCliDriver.testCliDriver TestBlobstoreCliDriver.testCliDriver TestBlobstoreCliDriver.testCliDriver TestCliDriver.acid_table_stats TestCliDriver.auto_sortmerge_join_2 TestCliDriver.avro_alter_table_update_columns TestCliDriver.avrotblsjoin TestCliDriver.dbtxnmgr_showlocks TestCliDriver.orc_merge10 TestCliDriver.orc_schema_evolution_float TestCliDriver.parquet_ppd_multifiles TestCliDriver.schema_evol_par_vec_table_dictionary_encoding TestCliDriver.schema_evol_par_vec_table_non_dictionary_encoding TestCliDriver.selectindate TestCliDriver.statsoptimizer TestCliDriver.vector_bround TestCliDriver.vector_case_when_1 TestCliDriver.vector_coalesce_2 TestCliDriver.vector_coalesce_3 TestCliDriver.vector_interval_1 TestCliDriver.vectorized_parquet_types TestMetastoreVersion.testMetastoreVersion TestMetastoreVersion.testVersionMatching TestMiniDruidCliDriver.druidkafkamini_basic TestMiniLlapCliDriver.llap_smb TestMiniTezCliDriver.explainanalyze_5 TestNegativeCliDriver.nopart_insert TestNegativeCliDriver.subquery_subquery_chain TestRpc.testServerPort TestSessionState.testCreatePath TestSessionState.testCreatePath TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill I see all the same issues when I run the tests against master. > Mark duplicate configs in HiveConf as deprecated > > > Key: HIVE-19031 > URL: https://issues.apache.org/jira/browse/HIVE-19031 > Project: Hive > Issue Type: Sub-task > Components: Configuration, Standalone Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Blocker > Attachments: HIVE-19031.2.patch, HIVE-19031.3.patch, HIVE-19031.patch > > > There are a number of configuration values that were copied from HiveConf to > MetastoreConf. They have been left in HiveConf for backwards compatibility. > But they need to be marked as deprecated so that users know to use the new > values in MetastoreConf. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18857) Store default value text instead of default value expression in metastore
[ https://issues.apache.org/jira/browse/HIVE-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427754#comment-16427754 ] Ashutosh Chauhan commented on HIVE-18857: - +1 pending tests > Store default value text instead of default value expression in metastore > - > > Key: HIVE-18857 > URL: https://issues.apache.org/jira/browse/HIVE-18857 > Project: Hive > Issue Type: Improvement > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-18857.1.patch > > > Currently for default value an expression is generated and serialized to > store in metastore. It should be improved to rather serialize the default > value itself instead of expression and store that in metastore. This will > have the following benefits: > * It will make metastore schema upgrade safe. e.g. if a UDF function name is > changed hive wouldn't be able to parse back the expression for this UDF which > was serialized in earlier version. > * It will make metastore schema for default constraint hive agnostic. Other > databases would be able to use the value as it is. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19031) Mark duplicate configs in HiveConf as deprecated
[ https://issues.apache.org/jira/browse/HIVE-19031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-19031: -- Attachment: HIVE-19031.3.patch > Mark duplicate configs in HiveConf as deprecated > > > Key: HIVE-19031 > URL: https://issues.apache.org/jira/browse/HIVE-19031 > Project: Hive > Issue Type: Sub-task > Components: Configuration, Standalone Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Blocker > Attachments: HIVE-19031.2.patch, HIVE-19031.3.patch, HIVE-19031.patch > > > There are a number of configuration values that were copied from HiveConf to > MetastoreConf. They have been left in HiveConf for backwards compatibility. > But they need to be marked as deprecated so that users know to use the new > values in MetastoreConf. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427721#comment-16427721
]
Eugene Koifman commented on HIVE-17647:
---
yes, that would be useful
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.01.patch, HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427715#comment-16427715
]
Sergey Shelukhin commented on HIVE-17647:
-
Perhaps we need separate jira to only mask warehouse path if the path is under
it, and not the entire path.
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.01.patch, HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427711#comment-16427711
]
Sergey Shelukhin commented on HIVE-17647:
-
Looks like HDFS paths get replaced partially, whereas local fs paths get
replaced fully (in the default or LlapLocal driver).
In any case, these rows are useless for verifying the file name :)
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.01.patch, HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19085) HIVE: FastHiveDecimal abs(0) sets sign to +ve
[
https://issues.apache.org/jira/browse/HIVE-19085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-19085:
---
Summary: HIVE: FastHiveDecimal abs(0) sets sign to +ve (was: HIVE:
impossible to insert abs(0) in a decimal column in a parquet's table)
> HIVE: FastHiveDecimal abs(0) sets sign to +ve
> -
>
> Key: HIVE-19085
> URL: https://issues.apache.org/jira/browse/HIVE-19085
> Project: Hive
> Issue Type: Bug
> Components: Hive, Vectorization
>Affects Versions: 2.3.2
>Reporter: ACOSS
>Assignee: Gopal V
>Priority: Minor
> Labels: fastdecimal, vectorization
> Attachments: HIVE-19085.1.patch
>
>
> Hi,
> We use parquet's table to store the result of others query. Some query use
> the function abs. If the function "abs" take 0 (type decimal) in input, then
> the insert in the parquet's table failed
>
> +Scenario:+
> create table test (col1 decimal(10,2)) stored as parquet;
> insert into test values(0);
> insert into test select abs(col1) from test;
>
> +Result;+
> The insert query crash with the error:
>
> 2018-03-30 17:39:02,123 FATAL [IPC Server handler 2 on 35885]
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
> attempt_1522311557218_0002_m_00_0 - exited : java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row \{"col1":0}
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:169)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row \{"col1":0}
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:562)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:160)
> ... 8 more
> Caused by: java.lang.RuntimeException: Unexpected #3
> at
> org.apache.hadoop.hive.common.type.FastHiveDecimalImpl.fastBigIntegerBytesUnscaled(FastHiveDecimalImpl.java:2550)
> at
> org.apache.hadoop.hive.common.type.FastHiveDecimalImpl.fastBigIntegerBytesScaled(FastHiveDecimalImpl.java:2806)
> at
> org.apache.hadoop.hive.common.type.FastHiveDecimal.fastBigIntegerBytesScaled(FastHiveDecimal.java:295)
> at
> org.apache.hadoop.hive.common.type.HiveDecimal.bigIntegerBytesScaled(HiveDecimal.java:712)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$DecimalDataWriter.decimalToBinary(DataWritableWriter.java:521)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$DecimalDataWriter.write(DataWritableWriter.java:514)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$GroupDataWriter.write(DataWritableWriter.java:204)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$MessageDataWriter.write(DataWritableWriter.java:220)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter.write(DataWritableWriter.java:91)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.write(DataWritableWriteSupport.java:59)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.write(DataWritableWriteSupport.java:31)
> at
> org.apache.parquet.hadoop.InternalParquetRecordWriter.write(InternalParquetRecordWriter.java:121)
> at
> org.apache.parquet.hadoop.ParquetRecordWriter.write(ParquetRecordWriter.java:123)
> at
> org.apache.parquet.hadoop.ParquetRecordWriter.write(ParquetRecordWriter.java:42)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.write(ParquetRecordWriterWrapper.java:112)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.write(ParquetRecordWriterWrapper.java:125)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:762)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:148)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:547)
>
> Problem is probably due to the fastABS method. This method
[jira] [Commented] (HIVE-18783) ALTER TABLE post-commit listener does not include the transactional listener responses
[
https://issues.apache.org/jira/browse/HIVE-18783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427706#comment-16427706
]
Hive QA commented on HIVE-18783:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 50 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10023/dev-support/hive-personality.sh
|
| git revision | master / 7845089 |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10023/yetus/patch-asflicense-problems.txt
|
| modules | C: itests/hcatalog-unit standalone-metastore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10023/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ALTER TABLE post-commit listener does not include the transactional listener
> responses
> ---
>
> Key: HIVE-18783
> URL: https://issues.apache.org/jira/browse/HIVE-18783
> Project: Hive
> Issue Type: Bug
>Reporter: Na Li
>Assignee: Sergio Peña
>Priority: Major
> Attachments: HIVE-18783.1.patch, HIVE-18783.2.patch,
> HIVE-18783.3.patch
>
>
> in HiveMetaStore, alter_table_core does NOT call transactional listener, and
> the notification ID corresponding to the alter table event is NOT set in the
> event parameters.
> {code}
> + alter_table_core
>
> try {
> Table oldt = this.get_table_core(dbname, name);
> this.firePreEvent(new PreAlterTableEvent(oldt, newTable, this));
> this.alterHandler.alterTable(this.getMS(), this.wh, dbname, name,
> newTable, envContext, this);
> success = true;
> if (!this.listeners.isEmpty()) {
> MetaStoreListenerNotifier.notifyEvent(this.listeners,
> EventType.ALTER_TABLE, new AlterTableEvent(oldt, newTable, true, this),
> envContext);
> }
> } catch (NoSuchObjectException var12) {
> ex = var12;
> throw new InvalidOperationException(var12.getMessage());
> } catch (Exception var13) {
> ex = var13;
> if (var13
[jira] [Comment Edited] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427700#comment-16427700
]
Eugene Koifman edited comment on HIVE-17647 at 4/5/18 10:41 PM:
if you look at acid_vectorization_original_tez.q.out, the same query shows
rows
{noformat}
PREHOOK: query: select distinct 7 as seven, INPUT__FILE__NAME from
over10k_orc_bucketed
PREHOOK: type: QUERY
PREHOOK: Input: default@over10k_orc_bucketed
PREHOOK: Output: hdfs://### HDFS PATH ###
POSTHOOK: query: select distinct 7 as seven, INPUT__FILE__NAME from
over10k_orc_bucketed
POSTHOOK: type: QUERY
POSTHOOK: Input: default@over10k_orc_bucketed
POSTHOOK: Output: hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
{noformat}
Either way, removing rows from output is not really "masking"
was (Author: ekoifman):
if you look at acid_vectorization_original_tez.q.out, the same query shows
rows
{noformat}
PREHOOK: query: select distinct 7 as seven, INPUT__FILE__NAME from
over10k_orc_bucketed
PREHOOK: type: QUERY
PREHOOK: Input: default@over10k_orc_bucketed
PREHOOK: Output: hdfs://### HDFS PATH ###
POSTHOOK: query: select distinct 7 as seven, INPUT__FILE__NAME from
over10k_orc_bucketed
POSTHOOK: type: QUERY
POSTHOOK: Input: default@over10k_orc_bucketed
POSTHOOK: Output: hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
{noformat}
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.01.patch, HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19116) Vectorization: Vector Map data type doesn't keep the order of the key/values pairs as read
[ https://issues.apache.org/jira/browse/HIVE-19116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427699#comment-16427699 ] Deepak Jaiswal commented on HIVE-19116: --- +1 lgtm. > Vectorization: Vector Map data type doesn't keep the order of the key/values > pairs as read > -- > > Key: HIVE-19116 > URL: https://issues.apache.org/jira/browse/HIVE-19116 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19116.01.patch > > > The VectorExtractRow class does not preserve the order of the key/value pairs > when going from MapColumnVector to a Map object. This causes Q file > differences in tests with the MAP data type making it seem like we are > getting Wrong Results (well, actually we are). > When LazyMap class (for example) adds key/value pairs to its "map" it uses a > LinkedHashSet to preserve insert order. > FYI: [~teddy.choi] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427700#comment-16427700
]
Eugene Koifman commented on HIVE-17647:
---
if you look at acid_vectorization_original_tez.q.out, the same query shows
rows
{noformat}
PREHOOK: query: select distinct 7 as seven, INPUT__FILE__NAME from
over10k_orc_bucketed
PREHOOK: type: QUERY
PREHOOK: Input: default@over10k_orc_bucketed
PREHOOK: Output: hdfs://### HDFS PATH ###
POSTHOOK: query: select distinct 7 as seven, INPUT__FILE__NAME from
over10k_orc_bucketed
POSTHOOK: type: QUERY
POSTHOOK: Input: default@over10k_orc_bucketed
POSTHOOK: Output: hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
7 hdfs://### HDFS PATH ###
{noformat}
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.01.patch, HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19031) Mark duplicate configs in HiveConf as deprecated
[
https://issues.apache.org/jira/browse/HIVE-19031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427685#comment-16427685
]
Hive QA commented on HIVE-19031:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12917563/HIVE-19031.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10022/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10022/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10022/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-04-05 22:19:58.640
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-10022/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-04-05 22:19:58.654
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
e4fb7bf..7845089 master -> origin/master
+ git reset --hard HEAD
HEAD is now at e4fb7bf HIVE-18909: Metrics for results cache (Jason Dere,
reviewed by GopalV)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 7845089 HIVE-18990 : Hive doesn't close Tez session properly
(Igor Kryvenko via Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-04-05 22:20:05.394
+ rm -rf ../yetus_PreCommit-HIVE-Build-10022
+ mkdir ../yetus_PreCommit-HIVE-Build-10022
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-10022
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-10022/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java:59
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java' cleanly.
Going to apply patch with: git apply -p0
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java:59
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java' cleanly.
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc1609899243246397696.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc1609899243246397696.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
log4j:WARN No appenders could be found for logger (DataNucleus.Persistence).
log4j:WARN Please initialize the log4j system properly.
DataNucleus Enhancer (version 4.1.17) for API "JDO"
DataNucleus Enhancer completed with success for 39 classes.
[ERROR] COMPILATION ERROR :
[jira] [Commented] (HIVE-18878) Lower MoveTask Lock Logging to Debug
[
https://issues.apache.org/jira/browse/HIVE-18878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427679#comment-16427679
]
Hive QA commented on HIVE-18878:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12917558/HIVE-18878.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 152 failed/errored test(s), 13579 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestCopyUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=252)
TestExportImport - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=252)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Commented] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427640#comment-16427640
]
Sergey Shelukhin commented on HIVE-17647:
-
Ah, nm. This seems to stem from the fact that out files cannot contain file
paths, they get removed.
[~ekoifman] if you check the output in acid_vectorization_original.q.out, the
filename query also returns no results :)
{noformat}
PREHOOK: query: select distinct 7 as seven, INPUT__FILE__NAME from
over10k_orc_bucketed
PREHOOK: type: QUERY
PREHOOK: Input: default@over10k_orc_bucketed
A masked pattern was here
POSTHOOK: query: select distinct 7 as seven, INPUT__FILE__NAME from
over10k_orc_bucketed
{noformat}
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.01.patch, HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18905) HS2: SASL auth loads HiveConf for every JDBC call
[
https://issues.apache.org/jira/browse/HIVE-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko updated HIVE-18905:
-
Attachment: HIVE-18905.04.patch
> HS2: SASL auth loads HiveConf for every JDBC call
> -
>
> Key: HIVE-18905
> URL: https://issues.apache.org/jira/browse/HIVE-18905
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Igor Kryvenko
>Priority: Minor
> Attachments: HIVE-18905.01.patch, HIVE-18905.03.patch,
> HIVE-18905.04.patch, HIVE-18905.patch
>
>
> SASL authentication filter does a new HiveConf() for no good reason.
> {code}
> public static PasswdAuthenticationProvider
> getAuthenticationProvider(AuthMethods authMethod)
> throws AuthenticationException {
> return getAuthenticationProvider(authMethod, new HiveConf());
> }
> {code}
> The session HiveConf is not needed to do this operation & it can't be changed
> after the HS2 starts up (today).
> {code}
> org.apache.hadoop.hive.conf.HiveConf.() HiveConf.java:4404
> org.apache.hive.service.auth.AuthenticationProviderFactory.getAuthenticationProvider(AuthenticationProviderFactory$AuthMethods)
> AuthenticationProviderFactory.java:61
> org.apache.hive.service.auth.PlainSaslHelper$PlainServerCallbackHandler.handle(Callback[])
> PlainSaslHelper.java:106
> org.apache.hive.service.auth.PlainSaslServer.evaluateResponse(byte[])
> PlainSaslServer.java:103
> org.apache.thrift.transport.TSaslTransport$SaslParticipant.evaluateChallengeOrResponse(byte[])
> TSaslTransport.java:539
> org.apache.thrift.transport.TSaslTransport.open() TSaslTransport.java:283
> org.apache.thrift.transport.TSaslServerTransport.open()
> TSaslServerTransport.java:41
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TTransport)
> TSaslServerTransport.java:216
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run()
> TThreadPoolServer.java:269
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker)
> ThreadPoolExecutor.java:1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run()
> ThreadPoolExecutor.java:617
> java.lang.Thread.run() Thread.java:745
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18857) Store default value text instead of default value expression in metastore
[ https://issues.apache.org/jira/browse/HIVE-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18857: --- Status: Patch Available (was: Open) > Store default value text instead of default value expression in metastore > - > > Key: HIVE-18857 > URL: https://issues.apache.org/jira/browse/HIVE-18857 > Project: Hive > Issue Type: Improvement > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-18857.1.patch > > > Currently for default value an expression is generated and serialized to > store in metastore. It should be improved to rather serialize the default > value itself instead of expression and store that in metastore. This will > have the following benefits: > * It will make metastore schema upgrade safe. e.g. if a UDF function name is > changed hive wouldn't be able to parse back the expression for this UDF which > was serialized in earlier version. > * It will make metastore schema for default constraint hive agnostic. Other > databases would be able to use the value as it is. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18857) Store default value text instead of default value expression in metastore
[ https://issues.apache.org/jira/browse/HIVE-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18857: --- Attachment: HIVE-18857.1.patch > Store default value text instead of default value expression in metastore > - > > Key: HIVE-18857 > URL: https://issues.apache.org/jira/browse/HIVE-18857 > Project: Hive > Issue Type: Improvement > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-18857.1.patch > > > Currently for default value an expression is generated and serialized to > store in metastore. It should be improved to rather serialize the default > value itself instead of expression and store that in metastore. This will > have the following benefits: > * It will make metastore schema upgrade safe. e.g. if a UDF function name is > changed hive wouldn't be able to parse back the expression for this UDF which > was serialized in earlier version. > * It will make metastore schema for default constraint hive agnostic. Other > databases would be able to use the value as it is. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19121) Fix HiveSchemaTool validation for databases that don't support schema
[ https://issues.apache.org/jira/browse/HIVE-19121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427599#comment-16427599 ] Ashutosh Chauhan commented on HIVE-19121: - +1 > Fix HiveSchemaTool validation for databases that don't support schema > - > > Key: HIVE-19121 > URL: https://issues.apache.org/jira/browse/HIVE-19121 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Minor > Fix For: 3.0.0 > > Attachments: HIVE-19121.patch > > > HiveSchemaTool -validate fails for those databases that don't support schemas. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18990) Hive doesn't close Tez session properly
[ https://issues.apache.org/jira/browse/HIVE-18990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427596#comment-16427596 ] Igor Kryvenko commented on HIVE-18990: -- [~ashutoshc] Thanks for review. > Hive doesn't close Tez session properly > --- > > Key: HIVE-18990 > URL: https://issues.apache.org/jira/browse/HIVE-18990 > Project: Hive > Issue Type: Bug >Reporter: Igor Kryvenko >Assignee: Igor Kryvenko >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18990.01.patch, HIVE-18990.02.patch, > HIVE-18990.03.patch > > > Hive doesn't close Tez session properly if AM isn't ready for accepting DAG. > *STR* > This can be easily reproduced using the following steps: > *1) configure cluster on Tez;* > *2) create file test.hql* > cat ~/test.hql > show databases; > *3) run the job* > $ hive --hiveconf hive.root.logger=DEBUG,console --hiveconf > hive.execution.engine=tez -f ~/test.hql > If we login into Yarn UI, we will see that jobs status is failed even it > finished successfully. > It happens because hive creates tez session by default. And if query finished > very quickly, we can't close tez session properly because AM isn't ready for > accepting any requests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18878) Lower MoveTask Lock Logging to Debug
[
https://issues.apache.org/jira/browse/HIVE-18878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427600#comment-16427600
]
Hive QA commented on HIVE-18878:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} ql: The patch generated 0 new + 42 unchanged - 1
fixed = 42 total (was 43) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
17s{color} | {color:red} The patch generated 50 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10021/dev-support/hive-personality.sh
|
| git revision | master / e4fb7bf |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10021/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10021/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Lower MoveTask Lock Logging to Debug
>
>
> Key: HIVE-18878
> URL: https://issues.apache.org/jira/browse/HIVE-18878
> Project: Hive
> Issue Type: Improvement
> Components: Locking
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: Igor Kryvenko
>Priority: Minor
> Labels: noob
> Attachments: HIVE-18878.1.patch, HIVE-18878.2.patch, HIVE-18878.patch
>
>
> https://github.com/apache/hive/blob/cbb9233a3b39ab8489d777fc76f0758c49b69bef/ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java#L233-L234
> {code}
> LOG.info("about to release lock for output: {} lock: {}", output,
> lock.getHiveLockObject().getName());
> try {
> lockMgr.unlock(lock);
> } catch (LockException le) {
> {code}
> Change this to _debug_ level logging. The lock manager's 'unlock' method
> should have additional logging available. Knowing that the unlock is about
> to take place is only helpful for debugging, not for normal operation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427597#comment-16427597
]
Sergey Shelukhin commented on HIVE-17647:
-
Actually I hit this issue even with non MM tables locally.
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.01.patch, HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18990) Hive doesn't close Tez session properly
[ https://issues.apache.org/jira/browse/HIVE-18990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-18990: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks, Igor! > Hive doesn't close Tez session properly > --- > > Key: HIVE-18990 > URL: https://issues.apache.org/jira/browse/HIVE-18990 > Project: Hive > Issue Type: Bug >Reporter: Igor Kryvenko >Assignee: Igor Kryvenko >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18990.01.patch, HIVE-18990.02.patch, > HIVE-18990.03.patch > > > Hive doesn't close Tez session properly if AM isn't ready for accepting DAG. > *STR* > This can be easily reproduced using the following steps: > *1) configure cluster on Tez;* > *2) create file test.hql* > cat ~/test.hql > show databases; > *3) run the job* > $ hive --hiveconf hive.root.logger=DEBUG,console --hiveconf > hive.execution.engine=tez -f ~/test.hql > If we login into Yarn UI, we will see that jobs status is failed even it > finished successfully. > It happens because hive creates tez session by default. And if query finished > very quickly, we can't close tez session properly because AM isn't ready for > accepting any requests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18725) Improve error handling for subqueries if there is wrong column reference
[
https://issues.apache.org/jira/browse/HIVE-18725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427590#comment-16427590
]
Vineet Garg commented on HIVE-18725:
+1 pending tests.
> Improve error handling for subqueries if there is wrong column reference
>
>
> Key: HIVE-18725
> URL: https://issues.apache.org/jira/browse/HIVE-18725
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-18725.01.patch, HIVE-18725.02.patch,
> HIVE-18725.03.patch, HIVE-18725.04.patch, HIVE-18725.05.patch
>
>
> If there is a column reference within subquery which doesn't exist Hive
> throws misleading error message.
> e.g.
> {code:sql}
> select * from table1 where table1.col1 IN (select col2 from table2 where
> table2.col1=table1.non_existing_column) and table1.col1 IN (select 4);
> {code}
> The above query, assuming table1 doesn't have non_existing_column, will throw
> following misleading error:
> {noformat}
> FAILED: SemanticException Line 0:-1 Unsupported SubQuery Expression 'col1':
> Only 1 SubQuery expression is supported.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18990) Hive doesn't close Tez session properly
[ https://issues.apache.org/jira/browse/HIVE-18990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427587#comment-16427587 ] Ashutosh Chauhan commented on HIVE-18990: - +1 ideally session should only be created when needed (e.g., show database doesn't need it) > Hive doesn't close Tez session properly > --- > > Key: HIVE-18990 > URL: https://issues.apache.org/jira/browse/HIVE-18990 > Project: Hive > Issue Type: Bug >Reporter: Igor Kryvenko >Assignee: Igor Kryvenko >Priority: Major > Attachments: HIVE-18990.01.patch, HIVE-18990.02.patch, > HIVE-18990.03.patch > > > Hive doesn't close Tez session properly if AM isn't ready for accepting DAG. > *STR* > This can be easily reproduced using the following steps: > *1) configure cluster on Tez;* > *2) create file test.hql* > cat ~/test.hql > show databases; > *3) run the job* > $ hive --hiveconf hive.root.logger=DEBUG,console --hiveconf > hive.execution.engine=tez -f ~/test.hql > If we login into Yarn UI, we will see that jobs status is failed even it > finished successfully. > It happens because hive creates tez session by default. And if query finished > very quickly, we can't close tez session properly because AM isn't ready for > accepting any requests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18725) Improve error handling for subqueries if there is wrong column reference
[
https://issues.apache.org/jira/browse/HIVE-18725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko updated HIVE-18725:
-
Attachment: HIVE-18725.05.patch
> Improve error handling for subqueries if there is wrong column reference
>
>
> Key: HIVE-18725
> URL: https://issues.apache.org/jira/browse/HIVE-18725
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-18725.01.patch, HIVE-18725.02.patch,
> HIVE-18725.03.patch, HIVE-18725.04.patch, HIVE-18725.05.patch
>
>
> If there is a column reference within subquery which doesn't exist Hive
> throws misleading error message.
> e.g.
> {code:sql}
> select * from table1 where table1.col1 IN (select col2 from table2 where
> table2.col1=table1.non_existing_column) and table1.col1 IN (select 4);
> {code}
> The above query, assuming table1 doesn't have non_existing_column, will throw
> following misleading error:
> {noformat}
> FAILED: SemanticException Line 0:-1 Unsupported SubQuery Expression 'col1':
> Only 1 SubQuery expression is supported.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HIVE-18950) DESCRIBE EXTENDED missing details of default constraint
[
https://issues.apache.org/jira/browse/HIVE-18950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg resolved HIVE-18950.
Resolution: Won't Fix
This is fixed by HIVE-18953
> DESCRIBE EXTENDED missing details of default constraint
> ---
>
> Key: HIVE-18950
> URL: https://issues.apache.org/jira/browse/HIVE-18950
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Aswathy Chellammal Sreekumar
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
> Describe extended output is missing default constraint details
> {noformat}
> 0: jdbc:hive2://ctr-e138-1518143905142-95188-> create table t1(j int
> constraint c1 default 4);
> INFO : Compiling
> command(queryId=hive_20180313202851_de315f0e-4064-467d-9dcc-f8dd7f737318):
> create table t1(j int constraint c1 default 4)
> INFO : Semantic Analysis Completed
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling
> command(queryId=hive_20180313202851_de315f0e-4064-467d-9dcc-f8dd7f737318);
> Time taken: 0.015 seconds
> INFO : Executing
> command(queryId=hive_20180313202851_de315f0e-4064-467d-9dcc-f8dd7f737318):
> create table t1(j int constraint c1 default 4)
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing
> command(queryId=hive_20180313202851_de315f0e-4064-467d-9dcc-f8dd7f737318);
> Time taken: 0.048 seconds
> INFO : OK
> No rows affected (0.087 seconds)
> {noformat}
> {noformat}
> 0: jdbc:hive2://ctr-e138-1518143905142-95188-> DESCRIBE EXTENDED t1;
> INFO : Compiling
> command(queryId=hive_20180313215805_0596cea8-918c-46f7-bd9a-8611972eb3cc):
> DESCRIBE EXTENDED t1
> INFO : Semantic Analysis Completed
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:col_name, type:string, comment:from
> deserializer), FieldSchema(name:data_type, type:string, comment:from
> deserializer), FieldSchema(name:comment, type:string, comment:from
> deserializer)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20180313215805_0596cea8-918c-46f7-bd9a-8611972eb3cc);
> Time taken: 0.029 seconds
> INFO : Executing
> command(queryId=hive_20180313215805_0596cea8-918c-46f7-bd9a-8611972eb3cc):
> DESCRIBE EXTENDED t1
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing
> command(queryId=hive_20180313215805_0596cea8-918c-46f7-bd9a-8611972eb3cc);
> Time taken: 0.03 seconds
> INFO : OK
> +-++--+
> | col_name | data_type
> | comment |
> +-++--+
> | j | int
> | |
> | | NULL
> | NULL |
> | Detailed Table Information | Table(tableName:t1, dbName:default,
> owner:hrt_qa, createTime:1520972931, lastAccessTime:0, retention:0,
> sd:StorageDescriptor(cols:[FieldSchema(name:j, type:int, comment:null)],
> location:hdfs://mycluster/apps/hive/warehouse/t1,
> inputFormat:org.apache.hadoop.mapred.TextInputFormat,
> outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat,
> compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null,
> serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe,
> parameters:{serialization.format=1}), bucketCols:[], sortCols:[],
> parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[],
> skewedColValueLocationMaps:{}), storedAsSubDirectories:false),
> partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0,
> transactional_properties=insert_only,
> COLUMN_STATS_ACCURATE={\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"j\":\"true\"}},
> numFiles=0, transient_lastDdlTime=1520972931, transactional=true},
> viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE,
> rewriteEnabled:false) | |
> +-++--+
> 3 rows selected (0.099 seconds){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427564#comment-16427564
]
Sergey Shelukhin commented on HIVE-17647:
-
Added the test. The no-results issue happens in llap and non-llap test, looks
like a MM table bug. I'll investigate more and file a JIRA.
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.01.patch, HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-17647:
Attachment: HIVE-17647.01.patch
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.01.patch, HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19123) TestNegativeCliDriver nopart_insert failing
[ https://issues.apache.org/jira/browse/HIVE-19123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19123: --- Attachment: HIVE-19123.1.patch > TestNegativeCliDriver nopart_insert failing > --- > > Key: HIVE-19123 > URL: https://issues.apache.org/jira/browse/HIVE-19123 > Project: Hive > Issue Type: Test >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19123.1.patch > > > Looks like HIVE-19083 caused this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19123) TestNegativeCliDriver nopart_insert failing
[ https://issues.apache.org/jira/browse/HIVE-19123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19123: --- Status: Patch Available (was: Open) > TestNegativeCliDriver nopart_insert failing > --- > > Key: HIVE-19123 > URL: https://issues.apache.org/jira/browse/HIVE-19123 > Project: Hive > Issue Type: Test >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19123.1.patch > > > Looks like HIVE-19083 caused this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19123) TestNegativeCliDriver nopart_insert failing
[ https://issues.apache.org/jira/browse/HIVE-19123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-19123: -- > TestNegativeCliDriver nopart_insert failing > --- > > Key: HIVE-19123 > URL: https://issues.apache.org/jira/browse/HIVE-19123 > Project: Hive > Issue Type: Test >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19123) TestNegativeCliDriver nopart_insert failing
[ https://issues.apache.org/jira/browse/HIVE-19123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19123: --- Description: Looks like HIVE-19083 caused this. > TestNegativeCliDriver nopart_insert failing > --- > > Key: HIVE-19123 > URL: https://issues.apache.org/jira/browse/HIVE-19123 > Project: Hive > Issue Type: Test >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > > Looks like HIVE-19083 caused this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-12192) Hive should carry out timestamp computations in UTC
[
https://issues.apache.org/jira/browse/HIVE-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427555#comment-16427555
]
Hive QA commented on HIVE-12192:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12894251/HIVE-12192.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10020/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10020/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10020/
Messages:
{noformat}
This message was trimmed, see log for full details
error: patch failed:
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorTypeCasts.java:22
Falling back to three-way merge...
Applied patch to
'ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorTypeCasts.java'
with conflicts.
error: patch failed:
serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java:20
Falling back to three-way merge...
Applied patch to
'serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java' with
conflicts.
error: patch failed:
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java:23
Falling back to three-way merge...
Applied patch to
'serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java'
cleanly.
error: patch failed:
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java:285
Falling back to three-way merge...
Applied patch to
'storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java'
with conflicts.
Going to apply patch with: git apply -p0
error: patch failed:
common/src/java/org/apache/hive/common/util/TimestampParser.java:18
Falling back to three-way merge...
Applied patch to
'common/src/java/org/apache/hive/common/util/TimestampParser.java' with
conflicts.
error: patch failed:
druid-handler/src/java/org/apache/hadoop/hive/druid/serde/DruidSerDe.java:510
Falling back to three-way merge...
Applied patch to
'druid-handler/src/java/org/apache/hadoop/hive/druid/serde/DruidSerDe.java'
with conflicts.
error: patch failed:
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java:348
Falling back to three-way merge...
Applied patch to
'hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java'
with conflicts.
error: patch failed:
hcatalog/core/src/main/java/org/apache/hive/hcatalog/data/JsonSerDe.java:21
Falling back to three-way merge...
Applied patch to
'hcatalog/core/src/main/java/org/apache/hive/hcatalog/data/JsonSerDe.java'
cleanly.
error: patch failed:
ql/src/gen/vectorization/ExpressionTemplates/DateColumnArithmeticTimestampColumn.txt:18
Falling back to three-way merge...
Applied patch to
'ql/src/gen/vectorization/ExpressionTemplates/DateColumnArithmeticTimestampColumn.txt'
with conflicts.
error: patch failed:
ql/src/gen/vectorization/ExpressionTemplates/DateColumnArithmeticTimestampScalar.txt:18
Falling back to three-way merge...
Applied patch to
'ql/src/gen/vectorization/ExpressionTemplates/DateColumnArithmeticTimestampScalar.txt'
with conflicts.
error: patch failed:
ql/src/gen/vectorization/ExpressionTemplates/DateScalarArithmeticTimestampColumn.txt:18
Falling back to three-way merge...
Applied patch to
'ql/src/gen/vectorization/ExpressionTemplates/DateScalarArithmeticTimestampColumn.txt'
with conflicts.
error: patch failed:
ql/src/gen/vectorization/ExpressionTemplates/IntervalYearMonthColumnArithmeticTimestampScalar.txt:18
Falling back to three-way merge...
Applied patch to
'ql/src/gen/vectorization/ExpressionTemplates/IntervalYearMonthColumnArithmeticTimestampScalar.txt'
with conflicts.
error: patch failed:
ql/src/gen/vectorization/ExpressionTemplates/IntervalYearMonthScalarArithmeticTimestampColumn.txt:18
Falling back to three-way merge...
Applied patch to
'ql/src/gen/vectorization/ExpressionTemplates/IntervalYearMonthScalarArithmeticTimestampColumn.txt'
with conflicts.
error: patch failed:
ql/src/gen/vectorization/ExpressionTemplates/LongDoubleColumnCompareTimestampScalar.txt:18
Falling back to three-way merge...
Applied patch to
'ql/src/gen/vectorization/ExpressionTemplates/LongDoubleColumnCompareTimestampScalar.txt'
with conflicts.
error: patch failed:
ql/src/gen/vectorization/ExpressionTemplates/TimestampColumnArithmeticDateColumn.txt:47
Falling back to three-way merge...
Applied patch to
'ql/src/gen/vectorization/ExpressionTemplates/TimestampColumnArithmeticDateColumn.txt'
with conflicts.
error: patch failed:
ql/src/gen/vectorization/ExpressionTemplates/TimestampColumnArithmeticDateScalar.txt:18
Falling back to three-way merge...
Applied patch to
[jira] [Commented] (HIVE-19105) HIVE-18781 broke WarehouseInstance
[
https://issues.apache.org/jira/browse/HIVE-19105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427551#comment-16427551
]
Hive QA commented on HIVE-19105:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12917548/HIVE-19105.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 142 failed/errored test(s), 13647 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=252)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=252)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Commented] (HIVE-18783) ALTER TABLE post-commit listener does not include the transactional listener responses
[
https://issues.apache.org/jira/browse/HIVE-18783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427545#comment-16427545
]
Vihang Karajgaonkar commented on HIVE-18783:
+1
> ALTER TABLE post-commit listener does not include the transactional listener
> responses
> ---
>
> Key: HIVE-18783
> URL: https://issues.apache.org/jira/browse/HIVE-18783
> Project: Hive
> Issue Type: Bug
>Reporter: Na Li
>Assignee: Sergio Peña
>Priority: Major
> Attachments: HIVE-18783.1.patch, HIVE-18783.2.patch,
> HIVE-18783.3.patch
>
>
> in HiveMetaStore, alter_table_core does NOT call transactional listener, and
> the notification ID corresponding to the alter table event is NOT set in the
> event parameters.
> {code}
> + alter_table_core
>
> try {
> Table oldt = this.get_table_core(dbname, name);
> this.firePreEvent(new PreAlterTableEvent(oldt, newTable, this));
> this.alterHandler.alterTable(this.getMS(), this.wh, dbname, name,
> newTable, envContext, this);
> success = true;
> if (!this.listeners.isEmpty()) {
> MetaStoreListenerNotifier.notifyEvent(this.listeners,
> EventType.ALTER_TABLE, new AlterTableEvent(oldt, newTable, true, this),
> envContext);
> }
> } catch (NoSuchObjectException var12) {
> ex = var12;
> throw new InvalidOperationException(var12.getMessage());
> } catch (Exception var13) {
> ex = var13;
> if (var13 instanceof MetaException) {
> throw (MetaException)var13;
> }
> if (var13 instanceof InvalidOperationException) {
> throw (InvalidOperationException)var13;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427541#comment-16427541
]
Eugene Koifman edited comment on HIVE-17647 at 4/5/18 8:29 PM:
---
hmm. I use this in all Acid UTs (though not q files), e.g. TestTxnLoadData.
using virtual columns (except ROW__ID) will disable Vectorization but surely
should return data
acid_vectorization_original.q has queries like
select ROW__ID, t, si, i from over10k_orc_bucketed where b = 4294967363 and t <
100 order by ROW__ID;
which is using a VC.
was (Author: ekoifman):
hmm. I use this in all Acid UTs (though not q files), e.g. TestTxnLoadData.
using virtual columns (except ROW__ID) will disable Vectorization but surely
should return data
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427541#comment-16427541
]
Eugene Koifman commented on HIVE-17647:
---
hmm. I use this in all Acid UTs (though not q files), e.g. TestTxnLoadData.
using virtual columns (except ROW__ID) will disable Vectorization but surely
should return data
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427526#comment-16427526
]
Sergey Shelukhin commented on HIVE-17647:
-
Heh, adding that to queries causes them to return no result. Looks like another
bug...
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17970) MM LOAD DATA with OVERWRITE doesn't use base_n directory concept
[
https://issues.apache.org/jira/browse/HIVE-17970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427484#comment-16427484
]
Sergey Shelukhin commented on HIVE-17970:
-
Fixing the HCat build issue... not sure why it didn't trigger locally. Maybe
maven is smart enough to exclude HCat from my local build based on usefulness
heuristic.
> MM LOAD DATA with OVERWRITE doesn't use base_n directory concept
>
>
> Key: HIVE-17970
> URL: https://issues.apache.org/jira/browse/HIVE-17970
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17970.01.patch, HIVE-17970.patch
>
>
> Judging by
> {code:java}
> Hive.loadTable(Path loadPath, String tableName, LoadFileType loadFileType,
> boolean isSrcLocal,
> boolean isSkewedStoreAsSubdir, boolean isAcid, boolean
> hasFollowingStatsTask,
> Long txnId, int stmtId, boolean isMmTable)
> {code}
> LOAD DATA with OVERWRITE will delete all existing data then write new data
> into the table. This logic makes sense for non-acid tables but for Acid/MM
> it should work like INSERT OVERWRITE statement and write new data to base_n/.
> This way the lock manager can be used to either get an X lock for IOW and
> thus block all readers or let it run with SemiShared and let readers continue
> and make the system more concurrent.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17970) MM LOAD DATA with OVERWRITE doesn't use base_n directory concept
[
https://issues.apache.org/jira/browse/HIVE-17970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-17970:
Attachment: HIVE-17970.01.patch
> MM LOAD DATA with OVERWRITE doesn't use base_n directory concept
>
>
> Key: HIVE-17970
> URL: https://issues.apache.org/jira/browse/HIVE-17970
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17970.01.patch, HIVE-17970.patch
>
>
> Judging by
> {code:java}
> Hive.loadTable(Path loadPath, String tableName, LoadFileType loadFileType,
> boolean isSrcLocal,
> boolean isSkewedStoreAsSubdir, boolean isAcid, boolean
> hasFollowingStatsTask,
> Long txnId, int stmtId, boolean isMmTable)
> {code}
> LOAD DATA with OVERWRITE will delete all existing data then write new data
> into the table. This logic makes sense for non-acid tables but for Acid/MM
> it should work like INSERT OVERWRITE statement and write new data to base_n/.
> This way the lock manager can be used to either get an X lock for IOW and
> thus block all readers or let it run with SemiShared and let readers continue
> and make the system more concurrent.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19105) HIVE-18781 broke WarehouseInstance
[
https://issues.apache.org/jira/browse/HIVE-19105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427475#comment-16427475
]
Hive QA commented on HIVE-19105:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 50 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10019/dev-support/hive-personality.sh
|
| git revision | master / e4fb7bf |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10019/yetus/patch-asflicense-problems.txt
|
| modules | C: itests/hive-unit U: itests/hive-unit |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10019/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HIVE-18781 broke WarehouseInstance
> --
>
> Key: HIVE-19105
> URL: https://issues.apache.org/jira/browse/HIVE-19105
> Project: Hive
> Issue Type: Bug
> Components: repl, Tests
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19105.patch
>
>
> HIVE-18781 removed the line
> {{hiveConf.set(HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY.varname, "false");}}
> from WarehouseInstance. This causes tests such as
> TestReplicationOnHDFSEncryptedZones and TestExportImport to hang as they are
> trying to connect to a non-existent ZooKeeper instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427463#comment-16427463
]
Eugene Koifman commented on HIVE-17647:
---
select *, INPUT__FILE__NAME from T
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427455#comment-16427455
]
Sergey Shelukhin commented on HIVE-17647:
-
I don't think it's actually possible to verify... do ACID q tests verify the
directory structure?
Will see if lock test can be added.
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19014) utilize YARN-8028 (queue ACL check) in Hive Tez session pool
[ https://issues.apache.org/jira/browse/HIVE-19014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427451#comment-16427451 ] Sergey Shelukhin commented on HIVE-19014: - Fixing a stupid config type issue. > utilize YARN-8028 (queue ACL check) in Hive Tez session pool > > > Key: HIVE-19014 > URL: https://issues.apache.org/jira/browse/HIVE-19014 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19014.01.patch, HIVE-19014.02.patch, > HIVE-19014.03.patch, HIVE-19014.04.patch, HIVE-19014.05.patch, > HIVE-19014.06.patch, HIVE-19014.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19014) utilize YARN-8028 (queue ACL check) in Hive Tez session pool
[ https://issues.apache.org/jira/browse/HIVE-19014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19014: Attachment: (was: HIVE-19014.06.patch) > utilize YARN-8028 (queue ACL check) in Hive Tez session pool > > > Key: HIVE-19014 > URL: https://issues.apache.org/jira/browse/HIVE-19014 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19014.01.patch, HIVE-19014.02.patch, > HIVE-19014.03.patch, HIVE-19014.04.patch, HIVE-19014.05.patch, > HIVE-19014.06.patch, HIVE-19014.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19014) utilize YARN-8028 (queue ACL check) in Hive Tez session pool
[ https://issues.apache.org/jira/browse/HIVE-19014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19014: Attachment: HIVE-19014.06.patch > utilize YARN-8028 (queue ACL check) in Hive Tez session pool > > > Key: HIVE-19014 > URL: https://issues.apache.org/jira/browse/HIVE-19014 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19014.01.patch, HIVE-19014.02.patch, > HIVE-19014.03.patch, HIVE-19014.04.patch, HIVE-19014.05.patch, > HIVE-19014.06.patch, HIVE-19014.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19014) utilize YARN-8028 (queue ACL check) in Hive Tez session pool
[ https://issues.apache.org/jira/browse/HIVE-19014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19014: Attachment: HIVE-19014.06.patch > utilize YARN-8028 (queue ACL check) in Hive Tez session pool > > > Key: HIVE-19014 > URL: https://issues.apache.org/jira/browse/HIVE-19014 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19014.01.patch, HIVE-19014.02.patch, > HIVE-19014.03.patch, HIVE-19014.04.patch, HIVE-19014.05.patch, > HIVE-19014.06.patch, HIVE-19014.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-12719) As a hive user, I am facing issues using permanent UDAF's.
[
https://issues.apache.org/jira/browse/HIVE-12719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427440#comment-16427440
]
Hive QA commented on HIVE-12719:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901512/HIVE-12719.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 160 failed/errored test(s), 13579 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestCopyUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=252)
TestExportImport - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=252)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=96)
[jira] [Updated] (HIVE-19085) HIVE: impossible to insert abs(0) in a decimal column in a parquet's table
[
https://issues.apache.org/jira/browse/HIVE-19085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-19085:
---
Fix Version/s: (was: 3.0.0)
> HIVE: impossible to insert abs(0) in a decimal column in a parquet's table
> ---
>
> Key: HIVE-19085
> URL: https://issues.apache.org/jira/browse/HIVE-19085
> Project: Hive
> Issue Type: Bug
> Components: Hive, Vectorization
>Affects Versions: 2.3.2
>Reporter: ACOSS
>Assignee: Gopal V
>Priority: Minor
> Labels: fastdecimal, vectorization
> Attachments: HIVE-19085.1.patch
>
>
> Hi,
> We use parquet's table to store the result of others query. Some query use
> the function abs. If the function "abs" take 0 (type decimal) in input, then
> the insert in the parquet's table failed
>
> +Scenario:+
> create table test (col1 decimal(10,2)) stored as parquet;
> insert into test values(0);
> insert into test select abs(col1) from test;
>
> +Result;+
> The insert query crash with the error:
>
> 2018-03-30 17:39:02,123 FATAL [IPC Server handler 2 on 35885]
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
> attempt_1522311557218_0002_m_00_0 - exited : java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row \{"col1":0}
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:169)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row \{"col1":0}
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:562)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:160)
> ... 8 more
> Caused by: java.lang.RuntimeException: Unexpected #3
> at
> org.apache.hadoop.hive.common.type.FastHiveDecimalImpl.fastBigIntegerBytesUnscaled(FastHiveDecimalImpl.java:2550)
> at
> org.apache.hadoop.hive.common.type.FastHiveDecimalImpl.fastBigIntegerBytesScaled(FastHiveDecimalImpl.java:2806)
> at
> org.apache.hadoop.hive.common.type.FastHiveDecimal.fastBigIntegerBytesScaled(FastHiveDecimal.java:295)
> at
> org.apache.hadoop.hive.common.type.HiveDecimal.bigIntegerBytesScaled(HiveDecimal.java:712)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$DecimalDataWriter.decimalToBinary(DataWritableWriter.java:521)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$DecimalDataWriter.write(DataWritableWriter.java:514)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$GroupDataWriter.write(DataWritableWriter.java:204)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$MessageDataWriter.write(DataWritableWriter.java:220)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter.write(DataWritableWriter.java:91)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.write(DataWritableWriteSupport.java:59)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.write(DataWritableWriteSupport.java:31)
> at
> org.apache.parquet.hadoop.InternalParquetRecordWriter.write(InternalParquetRecordWriter.java:121)
> at
> org.apache.parquet.hadoop.ParquetRecordWriter.write(ParquetRecordWriter.java:123)
> at
> org.apache.parquet.hadoop.ParquetRecordWriter.write(ParquetRecordWriter.java:42)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.write(ParquetRecordWriterWrapper.java:112)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.write(ParquetRecordWriterWrapper.java:125)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:762)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:148)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:547)
>
> Problem is probably due to the fastABS method. This method force "fastSignum"
> to 1 even when the decimal is
[jira] [Updated] (HIVE-19085) HIVE: impossible to insert abs(0) in a decimal column in a parquet's table
[
https://issues.apache.org/jira/browse/HIVE-19085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-19085:
---
Fix Version/s: 3.0.0
> HIVE: impossible to insert abs(0) in a decimal column in a parquet's table
> ---
>
> Key: HIVE-19085
> URL: https://issues.apache.org/jira/browse/HIVE-19085
> Project: Hive
> Issue Type: Bug
> Components: Hive, Vectorization
>Affects Versions: 2.3.2
>Reporter: ACOSS
>Assignee: Gopal V
>Priority: Minor
> Labels: fastdecimal, vectorization
> Attachments: HIVE-19085.1.patch
>
>
> Hi,
> We use parquet's table to store the result of others query. Some query use
> the function abs. If the function "abs" take 0 (type decimal) in input, then
> the insert in the parquet's table failed
>
> +Scenario:+
> create table test (col1 decimal(10,2)) stored as parquet;
> insert into test values(0);
> insert into test select abs(col1) from test;
>
> +Result;+
> The insert query crash with the error:
>
> 2018-03-30 17:39:02,123 FATAL [IPC Server handler 2 on 35885]
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
> attempt_1522311557218_0002_m_00_0 - exited : java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row \{"col1":0}
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:169)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row \{"col1":0}
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:562)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:160)
> ... 8 more
> Caused by: java.lang.RuntimeException: Unexpected #3
> at
> org.apache.hadoop.hive.common.type.FastHiveDecimalImpl.fastBigIntegerBytesUnscaled(FastHiveDecimalImpl.java:2550)
> at
> org.apache.hadoop.hive.common.type.FastHiveDecimalImpl.fastBigIntegerBytesScaled(FastHiveDecimalImpl.java:2806)
> at
> org.apache.hadoop.hive.common.type.FastHiveDecimal.fastBigIntegerBytesScaled(FastHiveDecimal.java:295)
> at
> org.apache.hadoop.hive.common.type.HiveDecimal.bigIntegerBytesScaled(HiveDecimal.java:712)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$DecimalDataWriter.decimalToBinary(DataWritableWriter.java:521)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$DecimalDataWriter.write(DataWritableWriter.java:514)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$GroupDataWriter.write(DataWritableWriter.java:204)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$MessageDataWriter.write(DataWritableWriter.java:220)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter.write(DataWritableWriter.java:91)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.write(DataWritableWriteSupport.java:59)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.write(DataWritableWriteSupport.java:31)
> at
> org.apache.parquet.hadoop.InternalParquetRecordWriter.write(InternalParquetRecordWriter.java:121)
> at
> org.apache.parquet.hadoop.ParquetRecordWriter.write(ParquetRecordWriter.java:123)
> at
> org.apache.parquet.hadoop.ParquetRecordWriter.write(ParquetRecordWriter.java:42)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.write(ParquetRecordWriterWrapper.java:112)
> at
> org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.write(ParquetRecordWriterWrapper.java:125)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:762)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:148)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:547)
>
> Problem is probably due to the fastABS method. This method force "fastSignum"
> to 1 even when the decimal is 0 (in this
[jira] [Updated] (HIVE-19121) Fix HiveSchemaTool validation for databases that don't support schema
[ https://issues.apache.org/jira/browse/HIVE-19121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19121: Status: Patch Available (was: Open) > Fix HiveSchemaTool validation for databases that don't support schema > - > > Key: HIVE-19121 > URL: https://issues.apache.org/jira/browse/HIVE-19121 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Minor > Fix For: 3.0.0 > > Attachments: HIVE-19121.patch > > > HiveSchemaTool -validate fails for those databases that don't support schemas. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427398#comment-16427398
]
Eugene Koifman edited comment on HIVE-17647 at 4/5/18 6:29 PM:
---
In DDLTask
{noformat}
if (writeId == null) {
throw new HiveException("Internal error - write ID not set for MM
conversion");
}
could this include table name or path
{noformat}
Could you add a test to check that X lock is actually acquired?
(TestDbTxnManager2) I'm not sure {{ddlWork.setNeedLock(true);}} will do
anything. LM relies on WriteEntity with WriteType.DD_EXCLUSIVE to get X lock...
Also, I don't any .q tests check that the data lands in the right place, i.e.
into a delta. that would be useful.
was (Author: ekoifman):
In DDLTask
{noformat}
if (writeId == null) {
throw new HiveException("Internal error - write ID not set for MM
conversion");
}
could this include table name or path
{noformat}
Could you add a test to check that X lock is actually acquired?
(TestDbTxnManager2) I'm not sure {{ddlWork.setNeedLock(true);}} will do
anything. LM relies on WriteEntity with WriteType.DD_EXCLUSIVE to get X lock...
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17647) DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
[
https://issues.apache.org/jira/browse/HIVE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427398#comment-16427398
]
Eugene Koifman commented on HIVE-17647:
---
In DDLTask
{noformat}
if (writeId == null) {
throw new HiveException("Internal error - write ID not set for MM
conversion");
}
could this include table name or path
{noformat}
Could you add a test to check that X lock is actually acquired?
(TestDbTxnManager2) I'm not sure {{ddlWork.setNeedLock(true);}} will do
anything. LM relies on WriteEntity with WriteType.DD_EXCLUSIVE to get X lock...
> DDLTask.generateAddMmTasks(Table tbl) and other random code should not start
> transactions
> -
>
> Key: HIVE-17647
> URL: https://issues.apache.org/jira/browse/HIVE-17647
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Sergey Shelukhin
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17647.patch
>
>
> This method (and other places) have
> {noformat}
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
> this should throw if there is no open transaction. It should never open one.
> In general the logic seems suspect. Looks like the intent is to move all
> existing files into a delta_x_x/ when a plain table is converted to MM table.
> This seems like something that needs to be done from under an Exclusive lock
> to prevent concurrent Insert operations writing data under table/partition
> root. But this is too late to acquire locks which should be done from the
> Driver.acquireLocks() (or else have deadlock detector since acquiring them
> here would bread all-or-nothing lock acquisition semantics currently required
> w/o deadlock detector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19083) Make partition clause optional for INSERT
[ https://issues.apache.org/jira/browse/HIVE-19083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427384#comment-16427384 ] Vineet Garg commented on HIVE-19083: Thanks for pointing it out [~ekoifman]. I have updated the wiki. > Make partition clause optional for INSERT > - > > Key: HIVE-19083 > URL: https://issues.apache.org/jira/browse/HIVE-19083 > Project: Hive > Issue Type: Sub-task > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19083.1.patch, HIVE-19083.2.patch, > HIVE-19083.3.patch, HIVE-19083.4.patch > > > Partition clause should be optional for > * INSERT INTO VALUES > * INSERT OVERWRITE > * INSERT SELECT -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18990) Hive doesn't close Tez session properly
[ https://issues.apache.org/jira/browse/HIVE-18990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427369#comment-16427369 ] Igor Kryvenko commented on HIVE-18990: -- [~ashutoshc] Could you review please? > Hive doesn't close Tez session properly > --- > > Key: HIVE-18990 > URL: https://issues.apache.org/jira/browse/HIVE-18990 > Project: Hive > Issue Type: Bug >Reporter: Igor Kryvenko >Assignee: Igor Kryvenko >Priority: Major > Attachments: HIVE-18990.01.patch, HIVE-18990.02.patch, > HIVE-18990.03.patch > > > Hive doesn't close Tez session properly if AM isn't ready for accepting DAG. > *STR* > This can be easily reproduced using the following steps: > *1) configure cluster on Tez;* > *2) create file test.hql* > cat ~/test.hql > show databases; > *3) run the job* > $ hive --hiveconf hive.root.logger=DEBUG,console --hiveconf > hive.execution.engine=tez -f ~/test.hql > If we login into Yarn UI, we will see that jobs status is failed even it > finished successfully. > It happens because hive creates tez session by default. And if query finished > very quickly, we can't close tez session properly because AM isn't ready for > accepting any requests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19097) related equals and in operators may cause inaccurate stats estimations
[
https://issues.apache.org/jira/browse/HIVE-19097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427358#comment-16427358
]
Zoltan Haindrich commented on HIVE-19097:
-
Thank you [~jcamachorodriguez], I've not known that part in
HivePointLookupOptimizerRule.java; for now I probably continue with the Calcite
way fix - I'm currently trying to add it similarily like
[CaseShuttle|https://github.com/kgyrtkirk/calcite/blob/8847efda4f7f43bf707ebc2db379c8b29870690a/core/src/main/java/org/apache/calcite/rel/rules/ReduceExpressionsRule.java#L563]
is handled - this also gives me some usefull insights into Calcite :)
> related equals and in operators may cause inaccurate stats estimations
> --
>
> Key: HIVE-19097
> URL: https://issues.apache.org/jira/browse/HIVE-19097
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19097.01.patch
>
>
> tpcds#74 is optimized in a way that for date_dim the condition contains IN
> and = for the same column
> {code:java}
> | Map Operator Tree: |
> | TableScan |
> | alias: date_dim |
> | filterExpr: (((d_year) IN (2001, 2002) and (d_year =
> 2002) and d_date_sk is not null) or ((d_year) IN (2001, 2002) and (d_year =
> 2001) and d_date_sk is not null)) (type: boolean) |
> | Statistics: Num rows: 73049 Data size: 876588 Basic
> stats: COMPLETE Column stats: COMPLETE |
> | Filter Operator |
> | predicate: ((d_year) IN (2001, 2002) and (d_year =
> 2002) and d_date_sk is not null) (type: boolean) |
> | Statistics: Num rows: 4 Data size: 48 Basic stats:
> COMPLETE Column stats: COMPLETE |
> {code}
> the "real" row count will be 365
> for separate {{IN}} and {{=}} the estimation is very good; but if both are
> present it becomes (very) underestimated.
> {code:java}
> set hive.query.results.cache.enabled=false;
> drop table if exists t1;
> drop table if exists t8;
> create table t1 (a integer,b integer);
> create table t8 like t1;
> insert into t1 values (1,1),(2,2),(3,3),(4,4),(5,5);
> insert into t8
> select * from t1 union all select * from t1 union all select * from t1 union
> all select * from t1 union all
> select * from t1 union all select * from t1 union all select * from t1 union
> all select * from t1
> ;
> analyze table t1 compute statistics for columns;
> analyze table t8 compute statistics for columns;
> explain analyze select sum(a) from t8 where b in (2,3) group by b;
> explain analyze select sum(a) from t8 where b=2 group by b;
> explain analyze select sum(a) from t1 where b in (2,3) and b=2 group by b;
> explain analyze select sum(a) from t8 where b in (2,3) and b=2 group by b;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18909) Metrics for results cache
[ https://issues.apache.org/jira/browse/HIVE-18909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-18909: -- Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed to master > Metrics for results cache > - > > Key: HIVE-18909 > URL: https://issues.apache.org/jira/browse/HIVE-18909 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Labels: Metrics > Fix For: 3.0.0 > > Attachments: HIVE-18909.1.patch, HIVE-18909.2.patch, > HIVE-18909.3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19104) When test MetaStore is started with retry the instances should be independent
[
https://issues.apache.org/jira/browse/HIVE-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427316#comment-16427316
]
Peter Vary commented on HIVE-19104:
---
Sure. Good catch! Will fix this.
Thanks, [~aihuaxu]!
> When test MetaStore is started with retry the instances should be independent
> -
>
> Key: HIVE-19104
> URL: https://issues.apache.org/jira/browse/HIVE-19104
> Project: Hive
> Issue Type: Improvement
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19104.patch
>
>
> When multiple MetaStore instances are started with
> {{MetaStoreTestUtils.startMetaStoreWithRetry}} currently they use the same
> JDBC url, and warehouse directory. This can cause problem in the tests
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19121) Fix HiveSchemaTool validation for databases that don't support schema
[ https://issues.apache.org/jira/browse/HIVE-19121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-19121: -- Description: HiveSchemaTool -validate fails for those databases that don't support schemas. > Fix HiveSchemaTool validation for databases that don't support schema > - > > Key: HIVE-19121 > URL: https://issues.apache.org/jira/browse/HIVE-19121 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Minor > Fix For: 3.0.0 > > Attachments: HIVE-19121.patch > > > HiveSchemaTool -validate fails for those databases that don't support schemas. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19121) Fix HiveSchemaTool validation for databases that don't support schema
[ https://issues.apache.org/jira/browse/HIVE-19121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427309#comment-16427309 ] Miklos Gergely commented on HIVE-19121: --- [~ashutoshc], please review and merge it in. > Fix HiveSchemaTool validation for databases that don't support schema > - > > Key: HIVE-19121 > URL: https://issues.apache.org/jira/browse/HIVE-19121 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Minor > Fix For: 3.0.0 > > Attachments: HIVE-19121.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19104) When test MetaStore is started with retry the instances should be independent
[
https://issues.apache.org/jira/browse/HIVE-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427266#comment-16427266
]
Aihua Xu edited comment on HIVE-19104 at 4/5/18 5:40 PM:
-
[~pvary] Here postfixedJdbcUrl should be postfixedWarehouseDir right?
LOG.info("MetaStore warehouse root dir ({}) is created", postfixedJdbcUrl);
was (Author: aihuaxu):
[~pvary] Should we use new postfixedWarehouseDir as well?
> When test MetaStore is started with retry the instances should be independent
> -
>
> Key: HIVE-19104
> URL: https://issues.apache.org/jira/browse/HIVE-19104
> Project: Hive
> Issue Type: Improvement
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19104.patch
>
>
> When multiple MetaStore instances are started with
> {{MetaStoreTestUtils.startMetaStoreWithRetry}} currently they use the same
> JDBC url, and warehouse directory. This can cause problem in the tests
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19121) Fix HiveSchemaTool validation for databases that don't support schema
[ https://issues.apache.org/jira/browse/HIVE-19121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-19121: -- Attachment: HIVE-19121.patch > Fix HiveSchemaTool validation for databases that don't support schema > - > > Key: HIVE-19121 > URL: https://issues.apache.org/jira/browse/HIVE-19121 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Minor > Fix For: 3.0.0 > > Attachments: HIVE-19121.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19121) Fix HiveSchemaTool validation for databases that don't support schema
[ https://issues.apache.org/jira/browse/HIVE-19121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely reassigned HIVE-19121: - > Fix HiveSchemaTool validation for databases that don't support schema > - > > Key: HIVE-19121 > URL: https://issues.apache.org/jira/browse/HIVE-19121 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Minor > Fix For: 3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-12719) As a hive user, I am facing issues using permanent UDAF's.
[
https://issues.apache.org/jira/browse/HIVE-12719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427306#comment-16427306
]
Hive QA commented on HIVE-12719:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 50 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10018/dev-support/hive-personality.sh
|
| git revision | master / dc5a943 |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10018/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10018/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> As a hive user, I am facing issues using permanent UDAF's.
> --
>
> Key: HIVE-12719
> URL: https://issues.apache.org/jira/browse/HIVE-12719
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.2.1
>Reporter: Surbhit
>Assignee: Ganesha Shreedhara
>Priority: Major
> Attachments: HIVE-12719.patch
>
>
> Currently function name for the permanent function is getting formed as
> dbName.WINDOW_FUNC_PREFIX+functionName where as function registry has
> function name as WINDOW_FUNC_PREFIX+dbName.functionName. This is leading to
> invalid function error when we use permanent function with window function.
> Fix has been done such that we form the permanent function name rightly as
> WINDOW_FUNC_PREFIX+dbName+.+functionName so that it matches with the name in
> function registry. The functionality for built-in/temporary function would
> remain the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18986) Table rename will run java.lang.StackOverflowError in dataNucleus if the table contains large number of columns
[ https://issues.apache.org/jira/browse/HIVE-18986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-18986: Status: In Progress (was: Patch Available) > Table rename will run java.lang.StackOverflowError in dataNucleus if the > table contains large number of columns > --- > > Key: HIVE-18986 > URL: https://issues.apache.org/jira/browse/HIVE-18986 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18986.1.patch, HIVE-18986.2.patch, > HIVE-18986.3.patch > > > If the table contains a lot of columns e.g, 5k, simple table rename would > fail with the following stack trace. The issue is datanucleus can't handle > the query with lots of colName='c1' && colName='c2' && ... . > > 2018-03-13 17:19:52,770 INFO > org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-5-thread-200]: > ugi=anonymous ip=10.17.100.135 cmd=source:10.17.100.135 alter_table: > db=default tbl=fgv_full_var_pivoted02 newtbl=fgv_full_var_pivoted 2018-03-13 > 17:20:00,495 ERROR org.apache.hadoop.hive.metastore.RetryingHMSHandler: > [pool-5-thread-200]: java.lang.StackOverflowError at > org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:330) at > org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:339) at > org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:339) at > org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:339) at > org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:339) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18986) Table rename will run java.lang.StackOverflowError in dataNucleus if the table contains large number of columns
[ https://issues.apache.org/jira/browse/HIVE-18986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-18986: Status: Patch Available (was: In Progress) > Table rename will run java.lang.StackOverflowError in dataNucleus if the > table contains large number of columns > --- > > Key: HIVE-18986 > URL: https://issues.apache.org/jira/browse/HIVE-18986 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18986.1.patch, HIVE-18986.2.patch, > HIVE-18986.3.patch > > > If the table contains a lot of columns e.g, 5k, simple table rename would > fail with the following stack trace. The issue is datanucleus can't handle > the query with lots of colName='c1' && colName='c2' && ... . > > 2018-03-13 17:19:52,770 INFO > org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-5-thread-200]: > ugi=anonymous ip=10.17.100.135 cmd=source:10.17.100.135 alter_table: > db=default tbl=fgv_full_var_pivoted02 newtbl=fgv_full_var_pivoted 2018-03-13 > 17:20:00,495 ERROR org.apache.hadoop.hive.metastore.RetryingHMSHandler: > [pool-5-thread-200]: java.lang.StackOverflowError at > org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:330) at > org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:339) at > org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:339) at > org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:339) at > org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:339) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17193) HoS: don't combine map works that are targets of different DPPs
[
https://issues.apache.org/jira/browse/HIVE-17193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427278#comment-16427278
]
Hive QA commented on HIVE-17193:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12917528/HIVE-17193.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 141 failed/errored test(s), 13580 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestCopyUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=252)
TestExportImport - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=246)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=252)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Assigned] (HIVE-19120) catalog not properly set for some tables in SQL upgrade scripts
[ https://issues.apache.org/jira/browse/HIVE-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates reassigned HIVE-19120: - > catalog not properly set for some tables in SQL upgrade scripts > --- > > Key: HIVE-19120 > URL: https://issues.apache.org/jira/browse/HIVE-19120 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Blocker > Fix For: 3.0.0 > > > A catalog column is added to the PARTITION_EVENTS and NOTIFICATION_LOG but > the upgrade scripts do not include an UPDATE statement to set this to the > default value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19104) When test MetaStore is started with retry the instances should be independent
[
https://issues.apache.org/jira/browse/HIVE-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427266#comment-16427266
]
Aihua Xu commented on HIVE-19104:
-
[~pvary] Should we use new postfixedWarehouseDir as well?
> When test MetaStore is started with retry the instances should be independent
> -
>
> Key: HIVE-19104
> URL: https://issues.apache.org/jira/browse/HIVE-19104
> Project: Hive
> Issue Type: Improvement
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19104.patch
>
>
> When multiple MetaStore instances are started with
> {{MetaStoreTestUtils.startMetaStoreWithRetry}} currently they use the same
> JDBC url, and warehouse directory. This can cause problem in the tests
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19083) Make partition clause optional for INSERT
[ https://issues.apache.org/jira/browse/HIVE-19083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427262#comment-16427262 ] Eugene Koifman commented on HIVE-19083: --- [~vgarg] could you update the wiki on this? https://cwiki.apache.org/confluence/display/hive/languagemanual+dml#LanguageManualDML-InsertingdataintoHiveTablesfromqueries https://cwiki.apache.org/confluence/display/hive/languagemanual+dml#LanguageManualDML-InsertingvaluesintotablesfromSQL have all examples using PARTITION clause > Make partition clause optional for INSERT > - > > Key: HIVE-19083 > URL: https://issues.apache.org/jira/browse/HIVE-19083 > Project: Hive > Issue Type: Sub-task > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19083.1.patch, HIVE-19083.2.patch, > HIVE-19083.3.patch, HIVE-19083.4.patch > > > Partition clause should be optional for > * INSERT INTO VALUES > * INSERT OVERWRITE > * INSERT SELECT -- This message was sent by Atlassian JIRA (v7.6.3#76005)
