[jira] [Commented] (HIVE-14117) HS2 UI: List of recent queries shows most recent query last
[ https://issues.apache.org/jira/browse/HIVE-14117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420625#comment-15420625 ] Lefty Leverenz commented on HIVE-14117: --- [~hagleitn]: a gentle nudge to update errata.txt for the typo in this commit's JIRA number. > HS2 UI: List of recent queries shows most recent query last > --- > > Key: HIVE-14117 > URL: https://issues.apache.org/jira/browse/HIVE-14117 > Project: Hive > Issue Type: Bug >Reporter: Gunther Hagleitner >Assignee: Gunther Hagleitner > Fix For: 2.2.0 > > Attachments: HIVE-14117.1.patch, HIVE-14117.1.patch > > > It's more useful to see the latest one first in your "last n queries" view. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14412) Add a timezone-aware timestamp
[
https://issues.apache.org/jira/browse/HIVE-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420621#comment-15420621
]
Hive QA commented on HIVE-14412:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12823648/HIVE-14412.1.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10476 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[mapjoin_mapjoin]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[tez_join_hash]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hadoop.hive.serde2.objectinspector.TestStandardObjectInspectors.testStandardPrimitiveObjectInspector
org.apache.hive.hcatalog.listener.TestMsgBusConnection.testConnection
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/884/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/884/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-884/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12823648 - PreCommit-HIVE-MASTER-Build
> Add a timezone-aware timestamp
> --
>
> Key: HIVE-14412
> URL: https://issues.apache.org/jira/browse/HIVE-14412
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-14412.1.patch, HIVE-14412.1.patch,
> HIVE-14412.1.patch
>
>
> Java's Timestamp stores the time elapsed since the epoch. While it's by
> itself unambiguous, ambiguity comes when we parse a string into timestamp, or
> convert a timestamp to string, causing problems like HIVE-14305.
> To solve the issue, I think we should make timestamp aware of timezone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14480) ORC ETLSplitStrategy should use thread pool when computing splits
[ https://issues.apache.org/jira/browse/HIVE-14480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420603#comment-15420603 ] Rajesh Balamohan commented on HIVE-14480: - Thanks [~sershe]. Please let me know if anything needs to be added for the commit as Jenkin's has completed the test cases. > ORC ETLSplitStrategy should use thread pool when computing splits > - > > Key: HIVE-14480 > URL: https://issues.apache.org/jira/browse/HIVE-14480 > Project: Hive > Issue Type: Bug >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-14480.1.patch, HIVE-14480.2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14412) Add a timezone-aware timestamp
[ https://issues.apache.org/jira/browse/HIVE-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-14412: -- Attachment: HIVE-14412.1.patch Thank you [~spena]. But I still can't reproduce it locally with the same maven and JDK. Use the same path to try again. > Add a timezone-aware timestamp > -- > > Key: HIVE-14412 > URL: https://issues.apache.org/jira/browse/HIVE-14412 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Rui Li >Assignee: Rui Li > Attachments: HIVE-14412.1.patch, HIVE-14412.1.patch, > HIVE-14412.1.patch > > > Java's Timestamp stores the time elapsed since the epoch. While it's by > itself unambiguous, ambiguity comes when we parse a string into timestamp, or > convert a timestamp to string, causing problems like HIVE-14305. > To solve the issue, I think we should make timestamp aware of timezone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14527) Schema evolution tests are not running in TestCliDriver
[
https://issues.apache.org/jira/browse/HIVE-14527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420562#comment-15420562
]
Hive QA commented on HIVE-14527:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12823639/HIVE-14527.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10441 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hive.hcatalog.listener.TestMsgBusConnection.testConnection
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/883/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/883/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-883/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12823639 - PreCommit-HIVE-MASTER-Build
> Schema evolution tests are not running in TestCliDriver
> ---
>
> Key: HIVE-14527
> URL: https://issues.apache.org/jira/browse/HIVE-14527
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Affects Versions: 2.2.0
>Reporter: Matt McCline
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14527.1.patch, HIVE-14527.2.patch
>
>
> HIVE-14376 broke something that makes schema evolution tests being excluded
> from TestCliDriver test suite.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14527) Schema evolution tests are not running in TestCliDriver
[ https://issues.apache.org/jira/browse/HIVE-14527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14527: - Attachment: HIVE-14527.2.patch Rebased patch > Schema evolution tests are not running in TestCliDriver > --- > > Key: HIVE-14527 > URL: https://issues.apache.org/jira/browse/HIVE-14527 > Project: Hive > Issue Type: Sub-task > Components: Test >Affects Versions: 2.2.0 >Reporter: Matt McCline >Assignee: Prasanth Jayachandran > Attachments: HIVE-14527.1.patch, HIVE-14527.2.patch > > > HIVE-14376 broke something that makes schema evolution tests being excluded > from TestCliDriver test suite. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14482) Add and drop table partition is not audit logged in HMS
[
https://issues.apache.org/jira/browse/HIVE-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Lin updated HIVE-14482:
Summary: Add and drop table partition is not audit logged in HMS (was:
Drop table partition is not audit logged in HMS)
> Add and drop table partition is not audit logged in HMS
> ---
>
> Key: HIVE-14482
> URL: https://issues.apache.org/jira/browse/HIVE-14482
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 2.1.0
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Minor
>
> When running:
> {code}
> ALTER TABLE test DROP PARTITION (b=140);
> {code}
> I only see the following in the HMS log:
> {code}
> 2016-08-08 23:12:34,081 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
> [pool-4-thread-2]: from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,082 INFO org.apache.hadoop.hive.metastore.HiveMetaStore:
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,082 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]:
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default
> tbl=test
> 2016-08-08 23:12:34,094 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
> [pool-4-thread-2]: end=1470723154094 duration=13
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2
> retryCount=0 error=false>
> 2016-08-08 23:12:34,095 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
> [pool-4-thread-2]: from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,095 INFO org.apache.hadoop.hive.metastore.HiveMetaStore:
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_partitions_by_expr :
> db=default tbl=test
> 2016-08-08 23:12:34,096 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]:
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_partitions_by_expr
> : db=default tbl=test
> 2016-08-08 23:12:34,112 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
> [pool-4-thread-2]: start=1470723154095 end=1470723154112 duration=17
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2
> retryCount=0 error=false>
> 2016-08-08 23:12:34,172 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
> [pool-4-thread-2]: from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,173 INFO org.apache.hadoop.hive.metastore.HiveMetaStore:
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,173 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]:
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default
> tbl=test
> 2016-08-08 23:12:34,186 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
> [pool-4-thread-2]: end=1470723154186 duration=14
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2
> retryCount=0 error=false>
> 2016-08-08 23:12:34,186 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
> [pool-4-thread-2]: from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,187 INFO org.apache.hadoop.hive.metastore.HiveMetaStore:
> [pool-4-thread-2]: 2: source:xx.xx.xxx.xxx get_table : db=default tbl=test
> 2016-08-08 23:12:34,187 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-4-thread-2]:
> ugi=hive ip=xx.xx.xxx.xxxcmd=source:xx.xx.xxx.xxx get_table : db=default
> tbl=test
> 2016-08-08 23:12:34,199 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
> [pool-4-thread-2]: end=1470723154199 duration=13
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=2
> retryCount=0 error=false>
> 2016-08-08 23:12:34,203 INFO org.apache.hadoop.hive.ql.log.PerfLogger:
> [pool-4-thread-2]: from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 2016-08-08 23:12:34,215 INFO org.apache.hadoop.hive.metastore.ObjectStore:
> [pool-4-thread-2]: JDO filter pushdown cannot be used: Filtering is supported
> only on partition keys of type string
> 2016-08-08 23:12:34,226 ERROR org.apache.hadoop.hdfs.KeyProviderCache:
> [pool-4-thread-2]: Could not find uri with key
> [dfs.encryption.key.provider.uri] to create a keyProvider !!
> 2016-08-08 23:12:34,239 INFO org.apache.hadoop.hive.metastore.HiveMetaStore:
> [pool-4-thread-2]: dropPartition() will move partition-directories to
> trash-directory.
> 2016-08-08 23:12:34,239 INFO hive.metastore.hivemetastoressimpl:
> [pool-4-thread-2]: deleting
> hdfs://:8020/user/hive/warehouse/default/test/b=140
> 2016-08-08 23:12:34,247 INFO org.apache.hadoop.fs.TrashPolicyDefault:
> [pool-4-thread-2]: Moved:
> 'hdfs://:8020/user/hive/warehouse/default/test/b=140' to trash at:
> hdfs://:8020/user/hive/.Trash/Current/user/hive/warehouse/default/test/b=140
> 2016-08-08
[jira] [Commented] (HIVE-14537) Please add HMS audit logs for ADD and CHANGE COLUMN operations
[
https://issues.apache.org/jira/browse/HIVE-14537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420528#comment-15420528
]
Eric Lin commented on HIVE-14537:
-
This one seems a bit more complicated than HIVE-14482, so I created this
separately.
> Please add HMS audit logs for ADD and CHANGE COLUMN operations
> --
>
> Key: HIVE-14537
> URL: https://issues.apache.org/jira/browse/HIVE-14537
> Project: Hive
> Issue Type: Improvement
>Reporter: Eric Lin
>Priority: Minor
>
> Currently if you ALTER TABLE test ADD COLUMNS (c int), the only audit log we
> can see is:
> {code}
> 2016-08-09T13:29:56,411 INFO [pool-6-thread-2]: metastore.HiveMetaStore
> (HiveMetaStore.java:logInfo(754)) - 2: source:127.0.0.1 alter_table:
> db=default tbl=test newtbl=test
> {code}
> This is not enough to tell which columns are added or changed. It would be
> useful to add such information.
> Ideally we could see:
> {code}
> 2016-08-09T13:29:56,411 INFO [pool-6-thread-2]: metastore.HiveMetaStore
> (HiveMetaStore.java:logInfo(754)) - 2: source:127.0.0.1 alter_table:
> db=default tbl=test newtbl=test newCol=c[int]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14299) Log serialized plan size
[ https://issues.apache.org/jira/browse/HIVE-14299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14299: - Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Thanks for the review! Pushed to master. > Log serialized plan size > - > > Key: HIVE-14299 > URL: https://issues.apache.org/jira/browse/HIVE-14299 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Minor > Fix For: 2.2.0 > > Attachments: HIVE-14299.1.patch > > > It will be good to log the size of the serialized plan. This can help > identifying cases where large objects are accidentally serialized. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14426) Extensive logging on info level in WebHCat
[
https://issues.apache.org/jira/browse/HIVE-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420247#comment-15420247
]
Hive QA commented on HIVE-14426:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12823611/HIVE-14426.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10454 tests
executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver-dynamic_partition_pruning.q-vector_char_mapjoin1.q-unionDistinct_2.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[tez_join_hash]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler
org.apache.hive.hcatalog.listener.TestMsgBusConnection.testConnection
org.apache.hive.spark.client.TestSparkClient.testJobSubmission
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/882/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/882/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-882/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12823611 - PreCommit-HIVE-MASTER-Build
> Extensive logging on info level in WebHCat
> --
>
> Key: HIVE-14426
> URL: https://issues.apache.org/jira/browse/HIVE-14426
> Project: Hive
> Issue Type: Bug
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Minor
> Attachments: HIVE-14426.2.patch, HIVE-14426.3.patch,
> HIVE-14426.4.patch, HIVE-14426.patch
>
>
> There is an extensive logging in WebHCat at info level, and even some
> sensitive information could be logged
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14035) Enable predicate pushdown to delta files created by ACID Transactions
[ https://issues.apache.org/jira/browse/HIVE-14035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420238#comment-15420238 ] Lefty Leverenz commented on HIVE-14035: --- Here's how you can get wiki edit privileges: * [About This Wiki -- How to get permission to edit | https://cwiki.apache.org/confluence/display/Hive/AboutThisWiki#AboutThisWiki-Howtogetpermissiontoedit] > Enable predicate pushdown to delta files created by ACID Transactions > - > > Key: HIVE-14035 > URL: https://issues.apache.org/jira/browse/HIVE-14035 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Saket Saurabh >Assignee: Saket Saurabh > Labels: TODOC2.2 > Fix For: 2.2.0 > > Attachments: Design.Document.Improving ACID performance in > Hive.01.docx, Design.Document.Improving ACID performance in Hive.02.docx, > HIVE-14035.02.patch, HIVE-14035.03.patch, HIVE-14035.04.patch, > HIVE-14035.05.patch, HIVE-14035.06.patch, HIVE-14035.07.patch, > HIVE-14035.08.patch, HIVE-14035.09.patch, HIVE-14035.10.patch, > HIVE-14035.11.patch, HIVE-14035.12.patch, HIVE-14035.13.patch, > HIVE-14035.14.patch, HIVE-14035.15.patch, HIVE-14035.16.patch, > HIVE-14035.17.patch, HIVE-14035.patch > > > In current Hive version, delta files created by ACID transactions do not > allow predicate pushdown if they contain any update/delete events. This is > done to preserve correctness when following a multi-version approach during > event collapsing, where an update event overwrites an existing insert event. > This JIRA proposes to split an update event into a combination of a delete > event followed by a new insert event, that can enable predicate push down to > all delta files without breaking correctness. To support backward > compatibility for this feature, this JIRA also proposes to add some sort of > versioning to ACID that can allow different versions of ACID transactions to > co-exist together. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
