[jira] [Commented] (HIVE-14412) Add a timezone-aware timestamp
[
https://issues.apache.org/jira/browse/HIVE-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528517#comment-15528517
]
Rui Li commented on HIVE-14412:
---
[~jdere], thank you for the comments and trying this out.
bq. Is there a way to convert TimestampTZ to a different TZ?
We don't have such a UDF yet. But it's easy to implement -
{{TimestampTZ::setOffsetInMin}} should serve the purpose.
bq. the time zone should not be used for comparison
I thought about this. One difficulty is we need to serialize TimestampTZ into
BytesWritable in BinarySortableSerDe. The timezone needs to be serialized into
the BytesWritable. I'm not sure how to keep it from being used for comparison.
bq. Is there way to keep non-numeric timezones, like ‘America/Los_Angeles’
rather than ‘GMT-08:00'?
I chose the GMT format because it saves space, and is guaranteed to be
understood by TimeZone. Whether a non-numeric timezone ID is recognizable
depends on local environment, and may vary among different nodes in a cluster.
One shortcoming of the GMT format is it's difficult to support functions that
modify the TimestampTZ, e.g. {{date_add}}. Because for example,
‘America/Los_Angeles’ maps to different GMT formats in different time of the
year, due to DST. So I think we should recommend the user to use the GMT format
in the first place. Is this acceptable?
> Add a timezone-aware timestamp
> --
>
> Key: HIVE-14412
> URL: https://issues.apache.org/jira/browse/HIVE-14412
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-14412.1.patch, HIVE-14412.2.patch,
> HIVE-14412.3.patch, HIVE-14412.4.patch, HIVE-14412.5.patch,
> HIVE-14412.6.patch, HIVE-14412.7.patch, HIVE-14412.8.patch
>
>
> Java's Timestamp stores the time elapsed since the epoch. While it's by
> itself unambiguous, ambiguity comes when we parse a string into timestamp, or
> convert a timestamp to string, causing problems like HIVE-14305.
> To solve the issue, I think we should make timestamp aware of timezone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7224) Set incremental printing to true by default in Beeline
[ https://issues.apache.org/jira/browse/HIVE-7224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528498#comment-15528498 ] Lefty Leverenz commented on HIVE-7224: -- Doc note: The change of default for --incremental needs to be documented in the wiki for Hive 2.2.0. * [HiveServer2 Clients -- Beeline Command Options | https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-BeelineCommandOptions] Added a TODOC2.2 label. > Set incremental printing to true by default in Beeline > -- > > Key: HIVE-7224 > URL: https://issues.apache.org/jira/browse/HIVE-7224 > Project: Hive > Issue Type: Bug > Components: Beeline, Clients, JDBC >Affects Versions: 0.13.0, 1.0.0, 1.2.0, 1.1.0 >Reporter: Vaibhav Gumashta >Assignee: Sahil Takiar > Labels: TODOC2.2 > Fix For: 2.2.0 > > Attachments: HIVE-7224.1.patch, HIVE-7224.2.patch, HIVE-7224.2.patch, > HIVE-7224.3.patch, HIVE-7224.4.patch, HIVE-7224.5.patch > > > See HIVE-7221. > By default beeline tries to buffer the entire output relation before printing > it on stdout. This can cause OOM when the output relation is large. However, > beeline has the option of incremental prints. We should keep that as the > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7224) Set incremental printing to true by default in Beeline
[ https://issues.apache.org/jira/browse/HIVE-7224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-7224: - Labels: TODOC2.2 (was: ) > Set incremental printing to true by default in Beeline > -- > > Key: HIVE-7224 > URL: https://issues.apache.org/jira/browse/HIVE-7224 > Project: Hive > Issue Type: Bug > Components: Beeline, Clients, JDBC >Affects Versions: 0.13.0, 1.0.0, 1.2.0, 1.1.0 >Reporter: Vaibhav Gumashta >Assignee: Sahil Takiar > Labels: TODOC2.2 > Fix For: 2.2.0 > > Attachments: HIVE-7224.1.patch, HIVE-7224.2.patch, HIVE-7224.2.patch, > HIVE-7224.3.patch, HIVE-7224.4.patch, HIVE-7224.5.patch > > > See HIVE-7221. > By default beeline tries to buffer the entire output relation before printing > it on stdout. This can cause OOM when the output relation is large. However, > beeline has the option of incremental prints. We should keep that as the > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14029) Update Spark version to 2.0.0
[ https://issues.apache.org/jira/browse/HIVE-14029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528490#comment-15528490 ] Ferdinand Xu commented on HIVE-14029: - Thanks [~leftylev] for the remind. It should be updated in WIKI. Shall we add a new section for Hive on Spark 2.0? > Update Spark version to 2.0.0 > - > > Key: HIVE-14029 > URL: https://issues.apache.org/jira/browse/HIVE-14029 > Project: Hive > Issue Type: Bug >Reporter: Ferdinand Xu >Assignee: Ferdinand Xu > Fix For: 2.2.0 > > Attachments: HIVE-14029.1.patch, HIVE-14029.2.patch, > HIVE-14029.3.patch, HIVE-14029.4.patch, HIVE-14029.5.patch, > HIVE-14029.6.patch, HIVE-14029.7.patch, HIVE-14029.8.patch, HIVE-14029.patch > > > There are quite some new optimizations in Spark 2.0.0. We need to bump up > Spark to 2.0.0 to benefit those performance improvements. > To update Spark version to 2.0.0, the following changes are required: > * Spark API updates: > ** SparkShuffler#call return Iterator instead of Iterable > ** SparkListener -> JavaSparkListener > ** InputMetrics constructor doesn’t accept readMethod > ** Method remoteBlocksFetched and localBlocksFetched in ShuffleReadMetrics > return long type instead of integer > * Dependency upgrade: > ** Jackson: 2.4.2 -> 2.6.5 > ** Netty version: 4.0.23.Final -> 4.0.29.Final > ** Scala binary version: 2.10 -> 2.11 > ** Scala version: 2.10.4 -> 2.11.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14029) Update Spark version to 2.0.0
[ https://issues.apache.org/jira/browse/HIVE-14029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528484#comment-15528484 ] Lefty Leverenz commented on HIVE-14029: --- Should this be documented in the wiki? * [Hive on Spark: Getting Started | https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started] > Update Spark version to 2.0.0 > - > > Key: HIVE-14029 > URL: https://issues.apache.org/jira/browse/HIVE-14029 > Project: Hive > Issue Type: Bug >Reporter: Ferdinand Xu >Assignee: Ferdinand Xu > Fix For: 2.2.0 > > Attachments: HIVE-14029.1.patch, HIVE-14029.2.patch, > HIVE-14029.3.patch, HIVE-14029.4.patch, HIVE-14029.5.patch, > HIVE-14029.6.patch, HIVE-14029.7.patch, HIVE-14029.8.patch, HIVE-14029.patch > > > There are quite some new optimizations in Spark 2.0.0. We need to bump up > Spark to 2.0.0 to benefit those performance improvements. > To update Spark version to 2.0.0, the following changes are required: > * Spark API updates: > ** SparkShuffler#call return Iterator instead of Iterable > ** SparkListener -> JavaSparkListener > ** InputMetrics constructor doesn’t accept readMethod > ** Method remoteBlocksFetched and localBlocksFetched in ShuffleReadMetrics > return long type instead of integer > * Dependency upgrade: > ** Jackson: 2.4.2 -> 2.6.5 > ** Netty version: 4.0.23.Final -> 4.0.29.Final > ** Scala binary version: 2.10 -> 2.11 > ** Scala version: 2.10.4 -> 2.11.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-14100) current_user() returns invalid information
[ https://issues.apache.org/jira/browse/HIVE-14100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-14100: Assignee: Prasanth Jayachandran (was: Peter Vary) > current_user() returns invalid information > -- > > Key: HIVE-14100 > URL: https://issues.apache.org/jira/browse/HIVE-14100 > Project: Hive > Issue Type: Bug > Components: Authentication, Beeline >Reporter: Peter Vary >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-14100.2.patch, HIVE-14100.2.patch, > HIVE-14100.2.patch, HIVE-14100.patch > > > Using HadoopDeaultAuthenticator the current_user() returns the username of > the unix user running hiveservice2. > Using SessionStateAuthenticator the current_user returns the username which > is provided when the connection started. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14100) current_user() returns invalid information
[ https://issues.apache.org/jira/browse/HIVE-14100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14100: - Status: Open (was: Patch Available) > current_user() returns invalid information > -- > > Key: HIVE-14100 > URL: https://issues.apache.org/jira/browse/HIVE-14100 > Project: Hive > Issue Type: Bug > Components: Authentication, Beeline >Reporter: Peter Vary >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-14100.2.patch, HIVE-14100.2.patch, > HIVE-14100.2.patch, HIVE-14100.patch > > > Using HadoopDeaultAuthenticator the current_user() returns the username of > the unix user running hiveservice2. > Using SessionStateAuthenticator the current_user returns the username which > is provided when the connection started. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14100) current_user() returns invalid information
[ https://issues.apache.org/jira/browse/HIVE-14100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14100: - Status: Patch Available (was: Open) > current_user() returns invalid information > -- > > Key: HIVE-14100 > URL: https://issues.apache.org/jira/browse/HIVE-14100 > Project: Hive > Issue Type: Bug > Components: Authentication, Beeline >Reporter: Peter Vary >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-14100.2.patch, HIVE-14100.2.patch, > HIVE-14100.2.patch, HIVE-14100.patch > > > Using HadoopDeaultAuthenticator the current_user() returns the username of > the unix user running hiveservice2. > Using SessionStateAuthenticator the current_user returns the username which > is provided when the connection started. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14100) current_user() returns invalid information
[ https://issues.apache.org/jira/browse/HIVE-14100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14100: - Attachment: HIVE-14100.2.patch Restarted ptest. Reuploading the patch for precommit tests. > current_user() returns invalid information > -- > > Key: HIVE-14100 > URL: https://issues.apache.org/jira/browse/HIVE-14100 > Project: Hive > Issue Type: Bug > Components: Authentication, Beeline >Reporter: Peter Vary >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-14100.2.patch, HIVE-14100.2.patch, > HIVE-14100.2.patch, HIVE-14100.patch > > > Using HadoopDeaultAuthenticator the current_user() returns the username of > the unix user running hiveservice2. > Using SessionStateAuthenticator the current_user returns the username which > is provided when the connection started. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14100) current_user() returns invalid information
[ https://issues.apache.org/jira/browse/HIVE-14100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14100: - Assignee: Peter Vary (was: Prasanth Jayachandran) > current_user() returns invalid information > -- > > Key: HIVE-14100 > URL: https://issues.apache.org/jira/browse/HIVE-14100 > Project: Hive > Issue Type: Bug > Components: Authentication, Beeline >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Minor > Attachments: HIVE-14100.2.patch, HIVE-14100.2.patch, > HIVE-14100.2.patch, HIVE-14100.patch > > > Using HadoopDeaultAuthenticator the current_user() returns the username of > the unix user running hiveservice2. > Using SessionStateAuthenticator the current_user returns the username which > is provided when the connection started. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14830) Move a majority of the MiniLlapCliDriver tests to use an inline AM
[ https://issues.apache.org/jira/browse/HIVE-14830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14830: - Attachment: HIVE-14830.01.patch Restarted ptest. Reuploading the patch for precommit tests. > Move a majority of the MiniLlapCliDriver tests to use an inline AM > -- > > Key: HIVE-14830 > URL: https://issues.apache.org/jira/browse/HIVE-14830 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Prasanth Jayachandran > Attachments: HIVE-14830.01.patch, HIVE-14830.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14830) Move a majority of the MiniLlapCliDriver tests to use an inline AM
[ https://issues.apache.org/jira/browse/HIVE-14830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14830: - Status: Patch Available (was: Open) > Move a majority of the MiniLlapCliDriver tests to use an inline AM > -- > > Key: HIVE-14830 > URL: https://issues.apache.org/jira/browse/HIVE-14830 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Prasanth Jayachandran > Attachments: HIVE-14830.01.patch, HIVE-14830.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14830) Move a majority of the MiniLlapCliDriver tests to use an inline AM
[ https://issues.apache.org/jira/browse/HIVE-14830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14830: - Status: Open (was: Patch Available) > Move a majority of the MiniLlapCliDriver tests to use an inline AM > -- > > Key: HIVE-14830 > URL: https://issues.apache.org/jira/browse/HIVE-14830 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-14830.01.patch, HIVE-14830.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14830) Move a majority of the MiniLlapCliDriver tests to use an inline AM
[ https://issues.apache.org/jira/browse/HIVE-14830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14830: - Assignee: Siddharth Seth (was: Prasanth Jayachandran) > Move a majority of the MiniLlapCliDriver tests to use an inline AM > -- > > Key: HIVE-14830 > URL: https://issues.apache.org/jira/browse/HIVE-14830 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-14830.01.patch, HIVE-14830.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-14830) Move a majority of the MiniLlapCliDriver tests to use an inline AM
[ https://issues.apache.org/jira/browse/HIVE-14830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-14830: Assignee: Prasanth Jayachandran (was: Siddharth Seth) > Move a majority of the MiniLlapCliDriver tests to use an inline AM > -- > > Key: HIVE-14830 > URL: https://issues.apache.org/jira/browse/HIVE-14830 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Prasanth Jayachandran > Attachments: HIVE-14830.01.patch, HIVE-14830.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14848) S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
[
https://issues.apache.org/jira/browse/HIVE-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-14848:
-
Attachment: HIVE-14848.1.patch
Restarted ptest. Reuploading the patch for precommit tests.
> S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
>
>
> Key: HIVE-14848
> URL: https://issues.apache.org/jira/browse/HIVE-14848
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.2.0
>Reporter: Sergio Peña
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14848.1.patch, HIVE-14848.1.patch
>
>
> When S3 credentials are included in hive-site.xml, then MR jobs that need to
> read
> data from S3 cannot use them because S3 values are stripped from the Job
> configuration
> object before submitting the MR job.
> {noformat}
> @Override
> public void initialize(HiveConf conf, QueryPlan queryPlan, DriverContext
> driverContext) {
> ...
> conf.stripHiddenConfigurations(job);
> this.jobExecHelper = new HadoopJobExecHelper(job, console, this, this);
> }
> {noformat}
> We should use a MR property {{mapreduce.job.redacted-properties}} instead to
> hide this list on the MR side (such as history server UI) to allow MR run the
> job without issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14848) S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
[
https://issues.apache.org/jira/browse/HIVE-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-14848:
-
Status: Patch Available (was: Open)
> S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
>
>
> Key: HIVE-14848
> URL: https://issues.apache.org/jira/browse/HIVE-14848
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.2.0
>Reporter: Sergio Peña
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14848.1.patch, HIVE-14848.1.patch
>
>
> When S3 credentials are included in hive-site.xml, then MR jobs that need to
> read
> data from S3 cannot use them because S3 values are stripped from the Job
> configuration
> object before submitting the MR job.
> {noformat}
> @Override
> public void initialize(HiveConf conf, QueryPlan queryPlan, DriverContext
> driverContext) {
> ...
> conf.stripHiddenConfigurations(job);
> this.jobExecHelper = new HadoopJobExecHelper(job, console, this, this);
> }
> {noformat}
> We should use a MR property {{mapreduce.job.redacted-properties}} instead to
> hide this list on the MR side (such as history server UI) to allow MR run the
> job without issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14848) S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
[
https://issues.apache.org/jira/browse/HIVE-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-14848:
-
Assignee: Prasanth Jayachandran (was: Sergio Peña)
Status: Open (was: Patch Available)
> S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
>
>
> Key: HIVE-14848
> URL: https://issues.apache.org/jira/browse/HIVE-14848
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.2.0
>Reporter: Sergio Peña
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-14848.1.patch, HIVE-14848.1.patch
>
>
> When S3 credentials are included in hive-site.xml, then MR jobs that need to
> read
> data from S3 cannot use them because S3 values are stripped from the Job
> configuration
> object before submitting the MR job.
> {noformat}
> @Override
> public void initialize(HiveConf conf, QueryPlan queryPlan, DriverContext
> driverContext) {
> ...
> conf.stripHiddenConfigurations(job);
> this.jobExecHelper = new HadoopJobExecHelper(job, console, this, this);
> }
> {noformat}
> We should use a MR property {{mapreduce.job.redacted-properties}} instead to
> hide this list on the MR side (such as history server UI) to allow MR run the
> job without issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14848) S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
[
https://issues.apache.org/jira/browse/HIVE-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-14848:
-
Assignee: Sergio Peña (was: Prasanth Jayachandran)
> S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
>
>
> Key: HIVE-14848
> URL: https://issues.apache.org/jira/browse/HIVE-14848
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.2.0
>Reporter: Sergio Peña
>Assignee: Sergio Peña
> Attachments: HIVE-14848.1.patch, HIVE-14848.1.patch
>
>
> When S3 credentials are included in hive-site.xml, then MR jobs that need to
> read
> data from S3 cannot use them because S3 values are stripped from the Job
> configuration
> object before submitting the MR job.
> {noformat}
> @Override
> public void initialize(HiveConf conf, QueryPlan queryPlan, DriverContext
> driverContext) {
> ...
> conf.stripHiddenConfigurations(job);
> this.jobExecHelper = new HadoopJobExecHelper(job, console, this, this);
> }
> {noformat}
> We should use a MR property {{mapreduce.job.redacted-properties}} instead to
> hide this list on the MR side (such as history server UI) to allow MR run the
> job without issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14824) Separate fstype from cluster type in QTestUtil
[ https://issues.apache.org/jira/browse/HIVE-14824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14824: - Status: Patch Available (was: Open) > Separate fstype from cluster type in QTestUtil > -- > > Key: HIVE-14824 > URL: https://issues.apache.org/jira/browse/HIVE-14824 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Prasanth Jayachandran > Attachments: HIVE-14824.01.patch, HIVE-14824.02.patch, > HIVE-14824.02.patch > > > QTestUtil cluster type encodes the file system. e.g. > MiniClusterType.encrypted means mr + encrypted hdfs, spark means file://, mr > means hdfs etc. > These can be separated out. e.g. To add tests for tez against encrypted, and > llap against encrypted - I'd need to introduce 2 new cluster types. > Instead it's better to separate the storage into it's own types. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14824) Separate fstype from cluster type in QTestUtil
[ https://issues.apache.org/jira/browse/HIVE-14824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14824: - Assignee: Siddharth Seth (was: Prasanth Jayachandran) > Separate fstype from cluster type in QTestUtil > -- > > Key: HIVE-14824 > URL: https://issues.apache.org/jira/browse/HIVE-14824 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-14824.01.patch, HIVE-14824.02.patch, > HIVE-14824.02.patch > > > QTestUtil cluster type encodes the file system. e.g. > MiniClusterType.encrypted means mr + encrypted hdfs, spark means file://, mr > means hdfs etc. > These can be separated out. e.g. To add tests for tez against encrypted, and > llap against encrypted - I'd need to introduce 2 new cluster types. > Instead it's better to separate the storage into it's own types. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13098) Add a strict check for when the decimal gets converted to null due to insufficient width
[
https://issues.apache.org/jira/browse/HIVE-13098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528392#comment-15528392
]
Hive QA commented on HIVE-13098:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12830622/HIVE-13098.WIP.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1325/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1325/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1325/
Messages:
{noformat}
This message was trimmed, see log for full details
[copy] Copying 15 files to
/data/hive-ptest/working/apache-github-source-source/itests/custom-udfs/udf-vectorized-badexample/target/tmp/conf
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @
udf-vectorized-badexample ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @
udf-vectorized-badexample ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ udf-vectorized-badexample
---
[INFO] Building jar:
/data/hive-ptest/working/apache-github-source-source/itests/custom-udfs/udf-vectorized-badexample/target/udf-vectorized-badexample-2.2.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @
udf-vectorized-badexample ---
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @
udf-vectorized-badexample ---
[INFO] Installing
/data/hive-ptest/working/apache-github-source-source/itests/custom-udfs/udf-vectorized-badexample/target/udf-vectorized-badexample-2.2.0-SNAPSHOT.jar
to
/data/hive-ptest/working/maven/org/apache/hive/hive-it-custom-udfs/udf-vectorized-badexample/2.2.0-SNAPSHOT/udf-vectorized-badexample-2.2.0-SNAPSHOT.jar
[INFO] Installing
/data/hive-ptest/working/apache-github-source-source/itests/custom-udfs/udf-vectorized-badexample/pom.xml
to
/data/hive-ptest/working/maven/org/apache/hive/hive-it-custom-udfs/udf-vectorized-badexample/2.2.0-SNAPSHOT/udf-vectorized-badexample-2.2.0-SNAPSHOT.pom
[INFO]
[INFO]
[INFO] Building Hive Integration - HCatalog Unit Tests 2.2.0-SNAPSHOT
[INFO]
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-hcatalog-it-unit
---
[INFO] Deleting
/data/hive-ptest/working/apache-github-source-source/itests/hcatalog-unit/target
[INFO] Deleting
/data/hive-ptest/working/apache-github-source-source/itests/hcatalog-unit
(includes = [datanucleus.log, derby.log], excludes = [])
[INFO]
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-no-snapshots) @
hive-hcatalog-it-unit ---
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (download-spark) @ hive-hcatalog-it-unit
---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @
hive-hcatalog-it-unit ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
hive-hcatalog-it-unit ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/data/hive-ptest/working/apache-github-source-source/itests/hcatalog-unit/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @
hive-hcatalog-it-unit ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @
hive-hcatalog-it-unit ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @
hive-hcatalog-it-unit ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/data/hive-ptest/working/apache-github-source-source/itests/hcatalog-unit/src/test/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @
hive-hcatalog-it-unit ---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/itests/hcatalog-unit/target/tmp
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/itests/hcatalog-unit/target/warehouse
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/itests/hcatalog-unit/target/tmp/conf
[copy] Copying 15 files to
/data/hive-ptest/working/apache-github-source-source/itests/hcatalog-unit/target/tmp/conf
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @
hive-hcatalog-it-unit ---
[INFO] Compiling 8 source files to
[jira] [Updated] (HIVE-14824) Separate fstype from cluster type in QTestUtil
[ https://issues.apache.org/jira/browse/HIVE-14824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14824: - Assignee: Prasanth Jayachandran (was: Siddharth Seth) Status: Open (was: Patch Available) > Separate fstype from cluster type in QTestUtil > -- > > Key: HIVE-14824 > URL: https://issues.apache.org/jira/browse/HIVE-14824 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Prasanth Jayachandran > Attachments: HIVE-14824.01.patch, HIVE-14824.02.patch, > HIVE-14824.02.patch > > > QTestUtil cluster type encodes the file system. e.g. > MiniClusterType.encrypted means mr + encrypted hdfs, spark means file://, mr > means hdfs etc. > These can be separated out. e.g. To add tests for tez against encrypted, and > llap against encrypted - I'd need to introduce 2 new cluster types. > Instead it's better to separate the storage into it's own types. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14824) Separate fstype from cluster type in QTestUtil
[ https://issues.apache.org/jira/browse/HIVE-14824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14824: - Attachment: HIVE-14824.02.patch Restarted ptest. Reuploading the patch for precommit tests. > Separate fstype from cluster type in QTestUtil > -- > > Key: HIVE-14824 > URL: https://issues.apache.org/jira/browse/HIVE-14824 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Prasanth Jayachandran > Attachments: HIVE-14824.01.patch, HIVE-14824.02.patch, > HIVE-14824.02.patch > > > QTestUtil cluster type encodes the file system. e.g. > MiniClusterType.encrypted means mr + encrypted hdfs, spark means file://, mr > means hdfs etc. > These can be separated out. e.g. To add tests for tez against encrypted, and > llap against encrypted - I'd need to introduce 2 new cluster types. > Instead it's better to separate the storage into it's own types. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14824) Separate fstype from cluster type in QTestUtil
[ https://issues.apache.org/jira/browse/HIVE-14824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14824: - Assignee: Siddharth Seth (was: Prasanth Jayachandran) > Separate fstype from cluster type in QTestUtil > -- > > Key: HIVE-14824 > URL: https://issues.apache.org/jira/browse/HIVE-14824 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-14824.01.patch, HIVE-14824.02.patch, > HIVE-14824.02.patch > > > QTestUtil cluster type encodes the file system. e.g. > MiniClusterType.encrypted means mr + encrypted hdfs, spark means file://, mr > means hdfs etc. > These can be separated out. e.g. To add tests for tez against encrypted, and > llap against encrypted - I'd need to introduce 2 new cluster types. > Instead it's better to separate the storage into it's own types. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-14824) Separate fstype from cluster type in QTestUtil
[ https://issues.apache.org/jira/browse/HIVE-14824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-14824: Assignee: Prasanth Jayachandran (was: Siddharth Seth) > Separate fstype from cluster type in QTestUtil > -- > > Key: HIVE-14824 > URL: https://issues.apache.org/jira/browse/HIVE-14824 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Prasanth Jayachandran > Attachments: HIVE-14824.01.patch, HIVE-14824.02.patch, > HIVE-14824.02.patch > > > QTestUtil cluster type encodes the file system. e.g. > MiniClusterType.encrypted means mr + encrypted hdfs, spark means file://, mr > means hdfs etc. > These can be separated out. e.g. To add tests for tez against encrypted, and > llap against encrypted - I'd need to introduce 2 new cluster types. > Instead it's better to separate the storage into it's own types. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14558) Add support for listing views similar to "show tables"
[ https://issues.apache.org/jira/browse/HIVE-14558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14558: - Assignee: Naveen Gangam (was: Prasanth Jayachandran) > Add support for listing views similar to "show tables" > -- > > Key: HIVE-14558 > URL: https://issues.apache.org/jira/browse/HIVE-14558 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam > Attachments: HIVE-14558.1.patch, HIVE-14558.2.patch, > HIVE-14558.4.patch, HIVE-14558.4.patch > > > Users have been asking for such feature where they can get a lists of views > separately. > So perhaps a syntax similar to "show tables" command? > show views [in/from ] [] > Does it make sense to add such command? or is it not worth the effort? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14558) Add support for listing views similar to "show tables"
[ https://issues.apache.org/jira/browse/HIVE-14558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14558: - Assignee: Prasanth Jayachandran (was: Naveen Gangam) Status: Open (was: Patch Available) > Add support for listing views similar to "show tables" > -- > > Key: HIVE-14558 > URL: https://issues.apache.org/jira/browse/HIVE-14558 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.0.0 >Reporter: Naveen Gangam >Assignee: Prasanth Jayachandran > Attachments: HIVE-14558.1.patch, HIVE-14558.2.patch, > HIVE-14558.4.patch, HIVE-14558.4.patch > > > Users have been asking for such feature where they can get a lists of views > separately. > So perhaps a syntax similar to "show tables" command? > show views [in/from ] [] > Does it make sense to add such command? or is it not worth the effort? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14558) Add support for listing views similar to "show tables"
[ https://issues.apache.org/jira/browse/HIVE-14558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14558: - Status: Patch Available (was: Open) > Add support for listing views similar to "show tables" > -- > > Key: HIVE-14558 > URL: https://issues.apache.org/jira/browse/HIVE-14558 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.0.0 >Reporter: Naveen Gangam >Assignee: Prasanth Jayachandran > Attachments: HIVE-14558.1.patch, HIVE-14558.2.patch, > HIVE-14558.4.patch, HIVE-14558.4.patch > > > Users have been asking for such feature where they can get a lists of views > separately. > So perhaps a syntax similar to "show tables" command? > show views [in/from ] [] > Does it make sense to add such command? or is it not worth the effort? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14558) Add support for listing views similar to "show tables"
[ https://issues.apache.org/jira/browse/HIVE-14558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14558: - Attachment: HIVE-14558.4.patch Restarted ptest. Reuploading the patch for precommit tests. > Add support for listing views similar to "show tables" > -- > > Key: HIVE-14558 > URL: https://issues.apache.org/jira/browse/HIVE-14558 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.0.0 >Reporter: Naveen Gangam >Assignee: Prasanth Jayachandran > Attachments: HIVE-14558.1.patch, HIVE-14558.2.patch, > HIVE-14558.4.patch, HIVE-14558.4.patch > > > Users have been asking for such feature where they can get a lists of views > separately. > So perhaps a syntax similar to "show tables" command? > show views [in/from ] [] > Does it make sense to add such command? or is it not worth the effort? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HIVE-14835) Improve ptest2 build time
[ https://issues.apache.org/jira/browse/HIVE-14835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reopened HIVE-14835: -- ptest is broken. Reverting this patch for now. > Improve ptest2 build time > - > > Key: HIVE-14835 > URL: https://issues.apache.org/jira/browse/HIVE-14835 > Project: Hive > Issue Type: Sub-task > Components: Testing Infrastructure >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Fix For: 2.2.0 > > Attachments: HIVE-14835.1.patch > > > NO PRECOMMIT TESTS > 2 things can be improved > 1) ptest2 always downloads jars for compiling its own directory which takes > about 1m30s which should take only 5s with cache jars. The reason for that is > maven.repo.local is pointing to a path under WORKSPACE which will be cleaned > by jenkins for every run. > 2) For hive build we can make use of parallel build and quite the output of > build which should shave off another 15-30s. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13894) Fix more json related JDK8 test failures Part 2
[ https://issues.apache.org/jira/browse/HIVE-13894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-13894: --- Target Version/s: (was: 2.1.1) > Fix more json related JDK8 test failures Part 2 > --- > > Key: HIVE-13894 > URL: https://issues.apache.org/jira/browse/HIVE-13894 > Project: Hive > Issue Type: Sub-task >Reporter: Mohit Sabharwal >Assignee: Mohit Sabharwal > Fix For: 2.2.0 > > Attachments: HIVE-13894.patch > > > After merge of java8 branch to master, some more json ordering related > failures -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13549) Remove jdk version specific out files from Hive2
[ https://issues.apache.org/jira/browse/HIVE-13549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-13549: --- Target Version/s: (was: 2.1.1) > Remove jdk version specific out files from Hive2 > > > Key: HIVE-13549 > URL: https://issues.apache.org/jira/browse/HIVE-13549 > Project: Hive > Issue Type: Sub-task >Reporter: Mohit Sabharwal >Assignee: Mohit Sabharwal > Fix For: java8, 2.2.0 > > Attachments: HIVE-13549-1-java8.patch, HIVE-13549-java8.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13860) Fix more json related JDK8 test failures
[ https://issues.apache.org/jira/browse/HIVE-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-13860: --- Target Version/s: (was: 2.1.1) > Fix more json related JDK8 test failures > > > Key: HIVE-13860 > URL: https://issues.apache.org/jira/browse/HIVE-13860 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Mohit Sabharwal >Assignee: Mohit Sabharwal > Fix For: java8, 2.2.0 > > Attachments: HIVE-13860-java8.patch, HIVE-13860-java8.patch, > HIVE-13860-java8.patch, HIVE-13860-java8.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13894) Fix more json related JDK8 test failures Part 2
[ https://issues.apache.org/jira/browse/HIVE-13894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-13894: --- Target Version/s: 2.1.1 > Fix more json related JDK8 test failures Part 2 > --- > > Key: HIVE-13894 > URL: https://issues.apache.org/jira/browse/HIVE-13894 > Project: Hive > Issue Type: Sub-task >Reporter: Mohit Sabharwal >Assignee: Mohit Sabharwal > Fix For: 2.2.0 > > Attachments: HIVE-13894.patch > > > After merge of java8 branch to master, some more json ordering related > failures -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13549) Remove jdk version specific out files from Hive2
[ https://issues.apache.org/jira/browse/HIVE-13549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-13549: --- Target Version/s: 2.1.1 > Remove jdk version specific out files from Hive2 > > > Key: HIVE-13549 > URL: https://issues.apache.org/jira/browse/HIVE-13549 > Project: Hive > Issue Type: Sub-task >Reporter: Mohit Sabharwal >Assignee: Mohit Sabharwal > Fix For: java8, 2.2.0 > > Attachments: HIVE-13549-1-java8.patch, HIVE-13549-java8.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13860) Fix more json related JDK8 test failures
[ https://issues.apache.org/jira/browse/HIVE-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-13860: --- Target Version/s: 2.1.1 > Fix more json related JDK8 test failures > > > Key: HIVE-13860 > URL: https://issues.apache.org/jira/browse/HIVE-13860 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Mohit Sabharwal >Assignee: Mohit Sabharwal > Fix For: java8, 2.2.0 > > Attachments: HIVE-13860-java8.patch, HIVE-13860-java8.patch, > HIVE-13860-java8.patch, HIVE-13860-java8.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13749) Memory leak in Hive Metastore
[ https://issues.apache.org/jira/browse/HIVE-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-13749: --- Target Version/s: 2.1.1 > Memory leak in Hive Metastore > - > > Key: HIVE-13749 > URL: https://issues.apache.org/jira/browse/HIVE-13749 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 1.1.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam > Fix For: 2.2.0 > > Attachments: HIVE-13749.1.patch, HIVE-13749.patch, Top_Consumers7.html > > > Looking a heap dump of 10GB, a large number of Configuration objects(> 66k > instances) are being retained. These objects along with its retained set is > occupying about 95% of the heap space. This leads to HMS crashes every few > days. > I will attach an exported snapshot from the eclipse MAT. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14743) ArrayIndexOutOfBoundsException - HBASE-backed views' query with JOINs
[
https://issues.apache.org/jira/browse/HIVE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14743:
---
Target Version/s: 2.1.1
> ArrayIndexOutOfBoundsException - HBASE-backed views' query with JOINs
> -
>
> Key: HIVE-14743
> URL: https://issues.apache.org/jira/browse/HIVE-14743
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
>Affects Versions: 1.0.0
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
> Fix For: 2.2.0
>
> Attachments: HIVE-14743.1.patch, HIVE-14743.2.patch
>
>
> The stack:
> {noformat}
> 2016-09-13T09:38:49,972 ERROR [186b4545-65b5-4bfc-bc8e-3e14e251bb12 main]
> exec.Task: Job Submission failed with exception
> 'java.lang.ArrayIndexOutOfBoundsException(1)'
> java.lang.ArrayIndexOutOfBoundsException: 1
> at
> org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat.createFilterScan(HiveHBaseTableInputFormat.java:224)
> at
> org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat.getSplitsInternal(HiveHBaseTableInputFormat.java:492)
> at
> org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat.getSplits(HiveHBaseTableInputFormat.java:449)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:370)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:466)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getCombineSplits(CombineHiveInputFormat.java:356)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:546)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:329)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:320)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570)
> {noformat}
> Repro:
> {noformat}
> CREATE TABLE HBASE_TABLE_TEST_1(
> cvalue string ,
> pk string,
> ccount int )
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.hbase.HBaseSerDe'
> STORED BY
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
> 'hbase.columns.mapping'='cf:val,:key,cf2:count',
> 'hbase.scan.cache'='500',
> 'hbase.scan.cacheblocks'='false',
> 'serialization.format'='1')
> TBLPROPERTIES (
> 'hbase.table.name'='hbase_table_test_1',
> 'serialization.null.format'='' );
> CREATE VIEW VIEW_HBASE_TABLE_TEST_1 AS SELECT
> hbase_table_test_1.cvalue,hbase_table_test_1.pk,hbase_table_test_1.ccount
> FROM hbase_table_test_1 WHERE hbase_table_test_1.ccount IS NOT NULL;
> CREATE TABLE HBASE_TABLE_TEST_2(
> cvalue string ,
> pk string ,
>ccount int )
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.hbase.HBaseSerDe'
> STORED BY
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
> 'hbase.columns.mapping'='cf:val,:key,cf2:count',
> 'hbase.scan.cache'='500',
> 'hbase.scan.cacheblocks'='false',
> 'serialization.format'='1')
> TBLPROPERTIES (
> 'hbase.table.name'='hbase_table_test_2',
> 'serialization.null.format'='');
> CREATE VIEW VIEW_HBASE_TABLE_TEST_2 AS SELECT
> hbase_table_test_2.cvalue,hbase_table_test_2.pk,hbase_table_test_2.ccount
> FROM hbase_table_test_2 WHERE hbase_table_test_2.pk >='3-h-0' AND
> hbase_table_test_2.pk <= '3-h-g' AND hbase_table_test_2.ccount IS NOT
> NULL;
> set hive.auto.convert.join=false;
> SELECT p.cvalue cvalue
> FROM `VIEW_HBASE_TABLE_TEST_1` `p`
> LEFT OUTER JOIN `VIEW_HBASE_TABLE_TEST_2` `A1`
> ON `p`.cvalue = `A1`.cvalue
> LEFT OUTER JOIN `VIEW_HBASE_TABLE_TEST_1` `A2`
> ON `p`.cvalue = `A2`.cvalue;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14137) Hive on Spark throws FileAlreadyExistsException for jobs with multiple empty tables
[
https://issues.apache.org/jira/browse/HIVE-14137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14137:
---
Target Version/s: 2.1.1
> Hive on Spark throws FileAlreadyExistsException for jobs with multiple empty
> tables
> ---
>
> Key: HIVE-14137
> URL: https://issues.apache.org/jira/browse/HIVE-14137
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Fix For: 2.2.0
>
> Attachments: HIVE-14137.1.patch, HIVE-14137.2.patch,
> HIVE-14137.3.patch, HIVE-14137.4.patch, HIVE-14137.5.patch,
> HIVE-14137.6.patch, HIVE-14137.patch
>
>
> The following queries:
> {code}
> -- Setup
> drop table if exists empty1;
> create table empty1 (col1 bigint) stored as parquet tblproperties
> ('parquet.compress'='snappy');
> drop table if exists empty2;
> create table empty2 (col1 bigint, col2 bigint) stored as parquet
> tblproperties ('parquet.compress'='snappy');
> drop table if exists empty3;
> create table empty3 (col1 bigint) stored as parquet tblproperties
> ('parquet.compress'='snappy');
> -- All empty HDFS directories.
> -- Fails with [08S01]: Error while processing statement: FAILED: Execution
> Error, return code 3 from org.apache.hadoop.hive.ql.exec.spark.SparkTask.
> select empty1.col1
> from empty1
> inner join empty2
> on empty2.col1 = empty1.col1
> inner join empty3
> on empty3.col1 = empty2.col2;
> -- Two empty HDFS directories.
> -- Create an empty file in HDFS.
> insert into empty1 select * from empty1 where false;
> -- Same query fails with [08S01]: Error while processing statement: FAILED:
> Execution Error, return code 3 from
> org.apache.hadoop.hive.ql.exec.spark.SparkTask.
> select empty1.col1
> from empty1
> inner join empty2
> on empty2.col1 = empty1.col1
> inner join empty3
> on empty3.col1 = empty2.col2;
> -- One empty HDFS directory.
> -- Create an empty file in HDFS.
> insert into empty2 select * from empty2 where false;
> -- Same query succeeds.
> select empty1.col1
> from empty1
> inner join empty2
> on empty2.col1 = empty1.col1
> inner join empty3
> on empty3.col1 = empty2.col2;
> {code}
> Will result in the following exception:
> {code}
> org.apache.hadoop.fs.FileAlreadyExistsException:
> /tmp/hive/hive/1f3837aa-9407-4780-92b1-42a66d205139/hive_2016-06-24_15-45-23_206_79177714958655528-2/-mr-10004/0/emptyFile
> for client 172.26.14.151 already exists
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:2784)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2676)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2561)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:593)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.create(AuthorizationProviderProxyClientProtocol.java:111)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:393)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
> at
> org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:1902)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1738)
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1663)
> at
>
[jira] [Updated] (HIVE-14519) Multi insert query bug
[
https://issues.apache.org/jira/browse/HIVE-14519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14519:
---
Target Version/s: 2.1.1
> Multi insert query bug
> --
>
> Key: HIVE-14519
> URL: https://issues.apache.org/jira/browse/HIVE-14519
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
> Fix For: 2.2.0
>
> Attachments: HIVE-14519.1.patch
>
>
> When running multi-insert queries, when one of the query is not returning
> results, the other query is not returning the right result.
> For example:
> After following query, there is no value in /tmp/emp/dir3/00_0
> {noformat}
> From (select * from src) a
> insert overwrite directory '/tmp/emp/dir1/'
> select key, value
> insert overwrite directory '/tmp/emp/dir2/'
> select 'header'
> where 1=2
> insert overwrite directory '/tmp/emp/dir3/'
> select key, value
> where key = 100;
> {noformat}
> where clause in the second insert should not affect the third insert.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14715) Hive throws NumberFormatException with query with Null value
[ https://issues.apache.org/jira/browse/HIVE-14715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-14715: --- Target Version/s: 2.1.1 > Hive throws NumberFormatException with query with Null value > > > Key: HIVE-14715 > URL: https://issues.apache.org/jira/browse/HIVE-14715 > Project: Hive > Issue Type: Bug >Reporter: Yongzhi Chen >Assignee: Yongzhi Chen > Fix For: 2.2.0 > > Attachments: HIVE-14715.1.patch, HIVE-14715.2.patch > > > The java.lang.NumberFormatException will throw with following reproduce: > set hive.cbo.enable=false; > CREATE TABLE `paqtest`( > `c1` int, > `s1` string, > `s2` string, > `bn1` bigint) > ROW FORMAT SERDE > 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' > STORED AS INPUTFORMAT > 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' > OUTPUTFORMAT > 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'; > insert into paqtest values (58, '', 'ABC', 0); > SELECT > 'PM' AS cy, > c1, > NULL AS iused, > NULL AS itp, > s2, > NULL AS cvg, > NULL AS acavg, > sum(bn1) AS cca > FROM paqtest > WHERE (s1 IS NULL OR length(s1) = 0) > GROUP BY 'Pricing mismatch', c1, NULL, NULL, s2, NULL, NULL; > The stack like following: > java.lang.NumberFormatException: ABC > GroupByOperator.process(Object, int) line: 773 > ExecReducer.reduce(Object, Iterator, OutputCollector, Reporter) line: 236 > ReduceTask.runOldReducer(JobConf, TaskUmbilicalProtocol, TaskReporter, > RawKeyValueIterator, RawComparator, Class, Class) > line: 444 > ReduceTask.run(JobConf, TaskUmbilicalProtocol) line: 392 > LocalJobRunner$Job$ReduceTaskRunnable.run() line: 319 > Executors$RunnableAdapter.call() line: 471 > It works fine when hive.cbo.enable = true -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14538) beeline throws exceptions with parsing hive config when using !sh statement
[
https://issues.apache.org/jira/browse/HIVE-14538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14538:
---
Target Version/s: 2.1.1
> beeline throws exceptions with parsing hive config when using !sh statement
> ---
>
> Key: HIVE-14538
> URL: https://issues.apache.org/jira/browse/HIVE-14538
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
> Fix For: 2.2.0
>
> Attachments: HIVE-14538.1.patch, HIVE-14538.2.patch
>
>
> When beeline has a connection to a server, in some env it has following
> problem:
> {noformat}
> 0: jdbc:hive2://localhost> !verbose
> verbose: on
> 0: jdbc:hive2://localhost> !sh id
> java.lang.ArrayIndexOutOfBoundsException: 1
> at org.apache.hive.beeline.Commands.addConf(Commands.java:758)
> at org.apache.hive.beeline.Commands.getHiveConf(Commands.java:704)
> at org.apache.hive.beeline.Commands.sh(Commands.java:1002)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:52)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1081)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:917)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:845)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:482)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:465)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> 0: jdbc:hive2://localhost> !sh echo hello
> java.lang.ArrayIndexOutOfBoundsException: 1
> at org.apache.hive.beeline.Commands.addConf(Commands.java:758)
> at org.apache.hive.beeline.Commands.getHiveConf(Commands.java:704)
> at org.apache.hive.beeline.Commands.sh(Commands.java:1002)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:52)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1081)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:917)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:845)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:482)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:465)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> 0: jdbc:hive2://localhost>
> {noformat}
> Also it breaks if there is no connection established:
> {noformat}
> beeline> !sh id
> java.lang.NullPointerException
> at org.apache.hive.beeline.BeeLine.createStatement(BeeLine.java:1897)
> at org.apache.hive.beeline.Commands.getConfInternal(Commands.java:724)
> at org.apache.hive.beeline.Commands.getHiveConf(Commands.java:702)
> at org.apache.hive.beeline.Commands.sh(Commands.java:1002)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:52)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1081)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:917)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:845)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:482)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:465)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
[jira] [Updated] (HIVE-14215) Displaying inconsistent CPU usage data with MR execution engine
[
https://issues.apache.org/jira/browse/HIVE-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14215:
---
Target Version/s: 2.1.1
> Displaying inconsistent CPU usage data with MR execution engine
> ---
>
> Key: HIVE-14215
> URL: https://issues.apache.org/jira/browse/HIVE-14215
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Minor
> Fix For: 2.2.0
>
> Attachments: HIVE-14215.patch
>
>
> If the MR task is finished after printing the cumulative CPU time then there
> is the possibility to print inconsistent CPU usage information.
> Correct one:
> {noformat}
> 2016-07-12 11:31:42,961 Stage-3 map = 0%, reduce = 0%
> 2016-07-12 11:31:48,237 Stage-3 map = 100%, reduce = 0%, Cumulative CPU 2.5
> sec
> MapReduce Total cumulative CPU time: 2 seconds 500 msec
> Ended Job = job_1468321038188_0003
> MapReduce Jobs Launched:
> Stage-Stage-3: Map: 1 Cumulative CPU: 2.5 sec HDFS Read: 5864 HDFS Write:
> 103 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 500 msec
> {noformat}
> One type of inconsistent data (easily reproducible one):
> {noformat}
> 2016-07-12 11:39:00,540 Stage-3 map = 0%, reduce = 0%
> Ended Job = job_1468321038188_0004
> MapReduce Jobs Launched:
> Stage-Stage-3: Map: 1 Cumulative CPU: 2.51 sec HDFS Read: 5864 HDFS
> Write: 103 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 510 msec
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14296) Session count is not decremented when HS2 clients do not shutdown cleanly.
[
https://issues.apache.org/jira/browse/HIVE-14296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14296:
---
Target Version/s: 2.1.1
> Session count is not decremented when HS2 clients do not shutdown cleanly.
> --
>
> Key: HIVE-14296
> URL: https://issues.apache.org/jira/browse/HIVE-14296
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 2.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
> Fix For: 2.2.0
>
> Attachments: HIVE-14296.2.patch, HIVE-14296.patch
>
>
> When a JDBC client like beeline abruptly disconnects from HS2, the session
> gets closed on the serverside but the session count reported in the logs is
> incorrect. It never gets decremented.
> For example, I created 6 connections from the same instance of beeline to HS2.
> {code}
> 2016-07-20T15:05:17,987 INFO [HiveServer2-Handler-Pool: Thread-40]
> thrift.ThriftCLIService: Opened a session SessionHandle
> [28b225ee-204f-4b3e-b4fd-0039ef8e276e], current sessions: 1
> .
> 2016-07-20T15:05:24,239 INFO [HiveServer2-Handler-Pool: Thread-45]
> thrift.ThriftCLIService: Opened a session SessionHandle
> [1d267de8-ff9a-4e76-ac5c-e82c871588e7], current sessions: 2
> .
> 2016-07-20T15:05:25,710 INFO [HiveServer2-Handler-Pool: Thread-50]
> thrift.ThriftCLIService: Opened a session SessionHandle
> [04d53deb-8965-464b-aa3f-7042304cfb54], current sessions: 3
> .
> 2016-07-20T15:05:26,795 INFO [HiveServer2-Handler-Pool: Thread-55]
> thrift.ThriftCLIService: Opened a session SessionHandle
> [b4bb8b86-74e1-4e3c-babb-674d34ad1caf], current sessions: 4
> 2016-07-20T15:05:28,160 INFO [HiveServer2-Handler-Pool: Thread-60]
> thrift.ThriftCLIService: Opened a session SessionHandle
> [6d3c3ed9-fadb-4673-8c15-3315b7e2995d], current sessions: 5
> .
> 2016-07-20T15:05:29,136 INFO [HiveServer2-Handler-Pool: Thread-65]
> thrift.ThriftCLIService: Opened a session SessionHandle
> [88b630c0-f272-427d-8263-febfef8d], current sessions: 6
> {code}
> When I CNTRL-C the beeline process, in the HS2 logs I see
> {code}
> 2016-07-20T15:11:37,858 INFO [HiveServer2-Handler-Pool: Thread-55]
> thrift.ThriftCLIService: Session disconnected without closing properly.
> 2016-07-20T15:11:37,858 INFO [HiveServer2-Handler-Pool: Thread-40]
> thrift.ThriftCLIService: Session disconnected without closing properly.
> 2016-07-20T15:11:37,858 INFO [HiveServer2-Handler-Pool: Thread-65]
> thrift.ThriftCLIService: Session disconnected without closing properly.
> 2016-07-20T15:11:37,858 INFO [HiveServer2-Handler-Pool: Thread-60]
> thrift.ThriftCLIService: Session disconnected without closing properly.
> 2016-07-20T15:11:37,859 INFO [HiveServer2-Handler-Pool: Thread-50]
> thrift.ThriftCLIService: Session disconnected without closing properly.
> 2016-07-20T15:11:37,859 INFO [HiveServer2-Handler-Pool: Thread-45]
> thrift.ThriftCLIService: Session disconnected without closing properly.
> 2016-07-20T15:11:37,859 INFO [HiveServer2-Handler-Pool: Thread-55]
> thrift.ThriftCLIService: Closing the session: SessionHandle
> [b4bb8b86-74e1-4e3c-babb-674d34ad1caf]
> 2016-07-20T15:11:37,859 INFO [HiveServer2-Handler-Pool: Thread-40]
> thrift.ThriftCLIService: Closing the session: SessionHandle
> [28b225ee-204f-4b3e-b4fd-0039ef8e276e]
> 2016-07-20T15:11:37,859 INFO [HiveServer2-Handler-Pool: Thread-65]
> thrift.ThriftCLIService: Closing the session: SessionHandle
> [88b630c0-f272-427d-8263-febfef8d]
> 2016-07-20T15:11:37,859 INFO [HiveServer2-Handler-Pool: Thread-60]
> thrift.ThriftCLIService: Closing the session: SessionHandle
> [6d3c3ed9-fadb-4673-8c15-3315b7e2995d]
> 2016-07-20T15:11:37,859 INFO [HiveServer2-Handler-Pool: Thread-45]
> thrift.ThriftCLIService: Closing the session: SessionHandle
> [1d267de8-ff9a-4e76-ac5c-e82c871588e7]
> 2016-07-20T15:11:37,859 INFO [HiveServer2-Handler-Pool: Thread-50]
> thrift.ThriftCLIService: Closing the session: SessionHandle
> [04d53deb-8965-464b-aa3f-7042304cfb54]
> {code}
> The next time I connect to HS2 via beeline, I see
> {code}
> 2016-07-20T15:14:33,679 INFO [HiveServer2-Handler-Pool: Thread-50]
> thrift.ThriftCLIService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V8
> 2016-07-20T15:14:33,710 INFO [HiveServer2-Handler-Pool: Thread-50]
> session.SessionState: Created HDFS directory:
> /tmp/hive/hive/d47759e8-df3a-4504-9f28-99ff5247352c
> 2016-07-20T15:14:33,725 INFO [HiveServer2-Handler-Pool: Thread-50]
> session.SessionState: Created local directory:
> /var/folders/_3/0w477k4j5bjd6h967rw4vflwgp/T/ngangam/d47759e8-df3a-4504-9f28-99ff5247352c
> 2016-07-20T15:14:33,735 INFO [HiveServer2-Handler-Pool: Thread-50]
> session.SessionState: Created HDFS directory:
>
[jira] [Updated] (HIVE-14090) JDOExceptions thrown by the Metastore have their full stack trace returned to clients
[
https://issues.apache.org/jira/browse/HIVE-14090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14090:
---
Target Version/s: 2.1.1
> JDOExceptions thrown by the Metastore have their full stack trace returned to
> clients
> -
>
> Key: HIVE-14090
> URL: https://issues.apache.org/jira/browse/HIVE-14090
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0, 2.1.0
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Fix For: 2.2.0
>
> Attachments: HIVE-14090.1.patch, HIVE-14090.patch
>
>
> When user try to create any database or table with a name longer than 128
> characters:
> {code}
> create database
> test_longname_looonglooonglooonglooonglooonglooonglooonglooonglooonglooonglooongNametableFAIL;
> {code}
> It dumps the full exception stack-trace in a non-user-friendly message. The
> lends to relatively negative user-experience for Beeline users who hit this
> exception, they are generally not interested in the full stack-trace.
> The formatted stack-trace is below:
> {code}
> Error while processing statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask.
> MetaException(message:javax.jdo.JDOFatalUserException: Attempt to store value
> "test_longname_looonglooonglooonglooonglooonglooonglooonglooonglooonglooonglooongnametablefail2"
> in column "`NAME`" that has maximum length of 128. Please correct your data!
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:528)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:569)
> at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
> at com.sun.proxy.$Proxy10.createDatabase(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database_core(HiveMetaStore.java:923)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database(HiveMetaStore.java:962)
> at sun.reflect.GeneratedMethodAccessor30.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:138)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
> at com.sun.proxy.$Proxy12.create_database(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_database.getResult(ThriftHiveMetastore.java:8863)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_database.getResult(ThriftHiveMetastore.java:8847)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:707)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:702)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:702)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745) NestedThrowablesStackTrace: Attempt
> to store value
> "test_longname_looonglooonglooonglooonglooonglooonglooonglooonglooonglooonglooongnametablefail2"
> in column "`NAME`" that has maximum length of 128. Please correct your data!
> org.datanucleus.exceptions.NucleusUserException: Attempt to store value
>
[jira] [Updated] (HIVE-14226) Invalid check on an ASTNode#toStringTree in CalcitePlanner
[ https://issues.apache.org/jira/browse/HIVE-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-14226: --- Target Version/s: 2.1.1 > Invalid check on an ASTNode#toStringTree in CalcitePlanner > -- > > Key: HIVE-14226 > URL: https://issues.apache.org/jira/browse/HIVE-14226 > Project: Hive > Issue Type: Bug >Reporter: Barna Zsombor Klara >Assignee: Barna Zsombor Klara >Priority: Minor > Fix For: 2.2.0 > > Attachments: HIVE-14226.patch > > > The patch for HIVE-12590 has modified the string tree representation of an > ASTNode, so that everything except string literals are now lowercase. > However CalcitePlanner#genSelectLogicalPlan is still checking the occurrence > of an uppercase TOK_FUNCTIONDI constant in the string tree. > (Line 2997 in CalcitePlanner on master) > I think this check will never evaluate to true, so we should either be using > a lowercase String, or remove the check if it is unnecessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13836) DbNotifications giving an error = Invalid state. Transaction has already started
[
https://issues.apache.org/jira/browse/HIVE-13836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-13836:
---
Target Version/s: 2.1.1
> DbNotifications giving an error = Invalid state. Transaction has already
> started
>
>
> Key: HIVE-13836
> URL: https://issues.apache.org/jira/browse/HIVE-13836
> Project: Hive
> Issue Type: Bug
>Reporter: Nachiket Vaidya
>Assignee: Nachiket Vaidya
>Priority: Critical
> Labels: patch-available
> Fix For: 2.2.0
>
> Attachments: HIVE-13836.2.patch, HIVE-13836.patch
>
>
> I used pyhs2 python client to create tables/partitions in hive. I was working
> fine until I moved to multithreaded scripts which created 8 connections and
> ran DDL queries concurrently.
> I got the error as
> {noformat}
> 2016-05-04 17:49:26,226 ERROR
> org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-4-thread-194]:
> HMSHandler Fatal error: Invalid state. Transaction has already started
> org.datanucleus.transaction.NucleusTransactionException: Invalid state.
> Transaction has already started
> at
> org.datanucleus.transaction.TransactionManager.begin(TransactionManager.java:47)
> at org.datanucleus.TransactionImpl.begin(TransactionImpl.java:131)
> at
> org.datanucleus.api.jdo.JDOTransaction.internalBegin(JDOTransaction.java:88)
> at
> org.datanucleus.api.jdo.JDOTransaction.begin(JDOTransaction.java:80)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.openTransaction(ObjectStore.java:463)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.addNotificationEvent(ObjectStore.java:7522)
> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
> at com.sun.proxy.$Proxy10.addNotificationEvent(Unknown Source)
> at
> org.apache.hive.hcatalog.listener.DbNotificationListener.enqueue(DbNotificationListener.java:261)
> at
> org.apache.hive.hcatalog.listener.DbNotificationListener.onCreateTable(DbNotificationListener.java:123)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1483)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1502)
> at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:138)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
> at
> com.sun.proxy.$Proxy14.create_table_with_environment_context(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:9267)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14768) Add a new UDTF ExplodeByNumber
[ https://issues.apache.org/jira/browse/HIVE-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528255#comment-15528255 ] Pengcheng Xiong commented on HIVE-14768: Sure, will do, thanks. > Add a new UDTF ExplodeByNumber > -- > > Key: HIVE-14768 > URL: https://issues.apache.org/jira/browse/HIVE-14768 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-14768.01.patch > > > For intersect all and except all implementation purpose. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14768) Add a new UDTF ExplodeByNumber
[ https://issues.apache.org/jira/browse/HIVE-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528242#comment-15528242 ] Ashutosh Chauhan commented on HIVE-14768: - * I think replicate_rows() is a better name. * We should add a check in initialize() to make sure first argument is of long type. * In your test case, can you add another row. > Add a new UDTF ExplodeByNumber > -- > > Key: HIVE-14768 > URL: https://issues.apache.org/jira/browse/HIVE-14768 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-14768.01.patch > > > For intersect all and except all implementation purpose. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13098) Add a strict check for when the decimal gets converted to null due to insufficient width
[ https://issues.apache.org/jira/browse/HIVE-13098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13098: Attachment: HIVE-13098.WIP.patch The epic WIP patch... still need to take care of some paths. Propagating config to all decimals is difficult thanks to massive static use in Hive... > Add a strict check for when the decimal gets converted to null due to > insufficient width > > > Key: HIVE-13098 > URL: https://issues.apache.org/jira/browse/HIVE-13098 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13098.WIP.patch > > > When e.g. 99 is selected as decimal(5,0), the result is null. This can be > problematic, esp. if the data is written to a table and lost without the user > realizing it. There should be an option to error out in such cases instead; > it should probably be on by default and the error message should instruct the > user on how to disable it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13098) Add a strict check for when the decimal gets converted to null due to insufficient width
[ https://issues.apache.org/jira/browse/HIVE-13098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13098: Status: Patch Available (was: Open) > Add a strict check for when the decimal gets converted to null due to > insufficient width > > > Key: HIVE-13098 > URL: https://issues.apache.org/jira/browse/HIVE-13098 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13098.WIP.patch > > > When e.g. 99 is selected as decimal(5,0), the result is null. This can be > problematic, esp. if the data is written to a table and lost without the user > realizing it. There should be an option to error out in such cases instead; > it should probably be on by default and the error message should instruct the > user on how to disable it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14848) S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
[
https://issues.apache.org/jira/browse/HIVE-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528068#comment-15528068
]
Chaoyu Tang commented on HIVE-14848:
[~spena] Is the mapreduce.job.redacted-properties newly introduced by
MAPREDUCE-6741 only available in Hadoop 2.9 and above? If so, it is not
applicable to current Hive.
> S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
>
>
> Key: HIVE-14848
> URL: https://issues.apache.org/jira/browse/HIVE-14848
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.2.0
>Reporter: Sergio Peña
>Assignee: Sergio Peña
> Attachments: HIVE-14848.1.patch
>
>
> When S3 credentials are included in hive-site.xml, then MR jobs that need to
> read
> data from S3 cannot use them because S3 values are stripped from the Job
> configuration
> object before submitting the MR job.
> {noformat}
> @Override
> public void initialize(HiveConf conf, QueryPlan queryPlan, DriverContext
> driverContext) {
> ...
> conf.stripHiddenConfigurations(job);
> this.jobExecHelper = new HadoopJobExecHelper(job, console, this, this);
> }
> {noformat}
> We should use a MR property {{mapreduce.job.redacted-properties}} instead to
> hide this list on the MR side (such as history server UI) to allow MR run the
> job without issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14029) Update Spark version to 2.0.0
[ https://issues.apache.org/jira/browse/HIVE-14029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528017#comment-15528017 ] ASF GitHub Bot commented on HIVE-14029: --- Github user winningsix closed the pull request at: https://github.com/apache/hive/pull/103 > Update Spark version to 2.0.0 > - > > Key: HIVE-14029 > URL: https://issues.apache.org/jira/browse/HIVE-14029 > Project: Hive > Issue Type: Bug >Reporter: Ferdinand Xu >Assignee: Ferdinand Xu > Fix For: 2.2.0 > > Attachments: HIVE-14029.1.patch, HIVE-14029.2.patch, > HIVE-14029.3.patch, HIVE-14029.4.patch, HIVE-14029.5.patch, > HIVE-14029.6.patch, HIVE-14029.7.patch, HIVE-14029.8.patch, HIVE-14029.patch > > > There are quite some new optimizations in Spark 2.0.0. We need to bump up > Spark to 2.0.0 to benefit those performance improvements. > To update Spark version to 2.0.0, the following changes are required: > * Spark API updates: > ** SparkShuffler#call return Iterator instead of Iterable > ** SparkListener -> JavaSparkListener > ** InputMetrics constructor doesn’t accept readMethod > ** Method remoteBlocksFetched and localBlocksFetched in ShuffleReadMetrics > return long type instead of integer > * Dependency upgrade: > ** Jackson: 2.4.2 -> 2.6.5 > ** Netty version: 4.0.23.Final -> 4.0.29.Final > ** Scala binary version: 2.10 -> 2.11 > ** Scala version: 2.10.4 -> 2.11.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14029) Update Spark version to 2.0.0
[ https://issues.apache.org/jira/browse/HIVE-14029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferdinand Xu updated HIVE-14029: Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed to the master. [~aihuaxu] [~szehon] [~xuefuz] [~lirui] [~spena] [~stakiar] Thank you for the reviews. > Update Spark version to 2.0.0 > - > > Key: HIVE-14029 > URL: https://issues.apache.org/jira/browse/HIVE-14029 > Project: Hive > Issue Type: Bug >Reporter: Ferdinand Xu >Assignee: Ferdinand Xu > Fix For: 2.2.0 > > Attachments: HIVE-14029.1.patch, HIVE-14029.2.patch, > HIVE-14029.3.patch, HIVE-14029.4.patch, HIVE-14029.5.patch, > HIVE-14029.6.patch, HIVE-14029.7.patch, HIVE-14029.8.patch, HIVE-14029.patch > > > There are quite some new optimizations in Spark 2.0.0. We need to bump up > Spark to 2.0.0 to benefit those performance improvements. > To update Spark version to 2.0.0, the following changes are required: > * Spark API updates: > ** SparkShuffler#call return Iterator instead of Iterable > ** SparkListener -> JavaSparkListener > ** InputMetrics constructor doesn’t accept readMethod > ** Method remoteBlocksFetched and localBlocksFetched in ShuffleReadMetrics > return long type instead of integer > * Dependency upgrade: > ** Jackson: 2.4.2 -> 2.6.5 > ** Netty version: 4.0.23.Final -> 4.0.29.Final > ** Scala binary version: 2.10 -> 2.11 > ** Scala version: 2.10.4 -> 2.11.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14100) current_user() returns invalid information
[ https://issues.apache.org/jira/browse/HIVE-14100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14100: - Status: Patch Available (was: Open) > current_user() returns invalid information > -- > > Key: HIVE-14100 > URL: https://issues.apache.org/jira/browse/HIVE-14100 > Project: Hive > Issue Type: Bug > Components: Authentication, Beeline >Reporter: Peter Vary >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-14100.2.patch, HIVE-14100.2.patch, HIVE-14100.patch > > > Using HadoopDeaultAuthenticator the current_user() returns the username of > the unix user running hiveservice2. > Using SessionStateAuthenticator the current_user returns the username which > is provided when the connection started. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14100) current_user() returns invalid information
[ https://issues.apache.org/jira/browse/HIVE-14100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14100: - Assignee: Peter Vary (was: Prasanth Jayachandran) > current_user() returns invalid information > -- > > Key: HIVE-14100 > URL: https://issues.apache.org/jira/browse/HIVE-14100 > Project: Hive > Issue Type: Bug > Components: Authentication, Beeline >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Minor > Attachments: HIVE-14100.2.patch, HIVE-14100.2.patch, HIVE-14100.patch > > > Using HadoopDeaultAuthenticator the current_user() returns the username of > the unix user running hiveservice2. > Using SessionStateAuthenticator the current_user returns the username which > is provided when the connection started. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-14100) current_user() returns invalid information
[ https://issues.apache.org/jira/browse/HIVE-14100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-14100: Assignee: Prasanth Jayachandran (was: Peter Vary) > current_user() returns invalid information > -- > > Key: HIVE-14100 > URL: https://issues.apache.org/jira/browse/HIVE-14100 > Project: Hive > Issue Type: Bug > Components: Authentication, Beeline >Reporter: Peter Vary >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-14100.2.patch, HIVE-14100.2.patch, HIVE-14100.patch > > > Using HadoopDeaultAuthenticator the current_user() returns the username of > the unix user running hiveservice2. > Using SessionStateAuthenticator the current_user returns the username which > is provided when the connection started. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14100) current_user() returns invalid information
[ https://issues.apache.org/jira/browse/HIVE-14100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14100: - Assignee: Peter Vary (was: Prasanth Jayachandran) > current_user() returns invalid information > -- > > Key: HIVE-14100 > URL: https://issues.apache.org/jira/browse/HIVE-14100 > Project: Hive > Issue Type: Bug > Components: Authentication, Beeline >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Minor > Attachments: HIVE-14100.2.patch, HIVE-14100.2.patch, HIVE-14100.patch > > > Using HadoopDeaultAuthenticator the current_user() returns the username of > the unix user running hiveservice2. > Using SessionStateAuthenticator the current_user returns the username which > is provided when the connection started. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14100) current_user() returns invalid information
[ https://issues.apache.org/jira/browse/HIVE-14100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14100: - Attachment: HIVE-14100.2.patch I restarted jenkins and this patch got kicked out. Reuploading it again for precommit test. > current_user() returns invalid information > -- > > Key: HIVE-14100 > URL: https://issues.apache.org/jira/browse/HIVE-14100 > Project: Hive > Issue Type: Bug > Components: Authentication, Beeline >Reporter: Peter Vary >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-14100.2.patch, HIVE-14100.2.patch, HIVE-14100.patch > > > Using HadoopDeaultAuthenticator the current_user() returns the username of > the unix user running hiveservice2. > Using SessionStateAuthenticator the current_user returns the username which > is provided when the connection started. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14100) current_user() returns invalid information
[ https://issues.apache.org/jira/browse/HIVE-14100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-14100: - Status: Open (was: Patch Available) > current_user() returns invalid information > -- > > Key: HIVE-14100 > URL: https://issues.apache.org/jira/browse/HIVE-14100 > Project: Hive > Issue Type: Bug > Components: Authentication, Beeline >Reporter: Peter Vary >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-14100.2.patch, HIVE-14100.2.patch, HIVE-14100.patch > > > Using HadoopDeaultAuthenticator the current_user() returns the username of > the unix user running hiveservice2. > Using SessionStateAuthenticator the current_user returns the username which > is provided when the connection started. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-14100) current_user() returns invalid information
[ https://issues.apache.org/jira/browse/HIVE-14100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-14100: Assignee: Prasanth Jayachandran (was: Peter Vary) > current_user() returns invalid information > -- > > Key: HIVE-14100 > URL: https://issues.apache.org/jira/browse/HIVE-14100 > Project: Hive > Issue Type: Bug > Components: Authentication, Beeline >Reporter: Peter Vary >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-14100.2.patch, HIVE-14100.patch > > > Using HadoopDeaultAuthenticator the current_user() returns the username of > the unix user running hiveservice2. > Using SessionStateAuthenticator the current_user returns the username which > is provided when the connection started. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14822) Add support for credential provider for jobs launched from Hiveserver2
[ https://issues.apache.org/jira/browse/HIVE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-14822: --- Attachment: HIVE-14822.01.patch > Add support for credential provider for jobs launched from Hiveserver2 > -- > > Key: HIVE-14822 > URL: https://issues.apache.org/jira/browse/HIVE-14822 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar > Attachments: HIVE-14822.01.patch > > > When using encrypted passwords via the Hadoop Credential Provider, > HiveServer2 currently does not correctly forward enough information to the > job configuration for jobs to read those secrets. If your job needs to access > any secrets, like S3 credentials, then there's no convenient and secure way > to configure this today. > You could specify the decryption key in files like mapred-site.xml that > HiveServer2 uses, but this would place the encryption password on local disk > in plaintext, which can be a security concern. > To solve this problem, HiveServer2 should modify job configuration to include > the environment variable settings needed to decrypt the passwords. > Specifically, it will need to modify: > * For MR2 jobs: > ** yarn.app.mapreduce.am.admin.user.env > ** mapreduce.admin.user.env > * For Spark jobs: > ** spark.yarn.appMasterEnv.HADOOP_CREDSTORE_PASSWORD > ** spark.executorEnv.HADOOP_CREDSTORE_PASSWORD > HiveServer2 can get the decryption password from its own environment, the > same way it does for its own credential provider store today. > Additionally, it can be desirable for HiveServer2 to have a separate > encrypted password file than what is used by the job. HiveServer2 may have > secrets that the job should not have, such as the metastore database password > or the password to decrypt its private SSL certificate. It is also best > practices to have separate passwords on separate files. To facilitate this, > Hive will also accept: > * A configuration for a path to a credential store to use for jobs. This > should already be uploaded in HDFS. (hive.server2.job.keystore.location or a > better name) If this is not specified, then HS2 will simply use the value of > hadoop.security.credential.provider.path. > * An environment variable for the password to decrypt the credential store > (HIVE_JOB_KEYSTORE_PASSWORD or better). If this is not specified, then HS2 > will simply use the standard environment variable for decrypting the Hadoop > Credential Provider. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14830) Move a majority of the MiniLlapCliDriver tests to use an inline AM
[ https://issues.apache.org/jira/browse/HIVE-14830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527864#comment-15527864 ] Prasanth Jayachandran commented on HIVE-14830: -- setUseLocalFs seems redundant? We can implicitly set localFs to true when setUseLocalMode is set to true. > Move a majority of the MiniLlapCliDriver tests to use an inline AM > -- > > Key: HIVE-14830 > URL: https://issues.apache.org/jira/browse/HIVE-14830 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-14830.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14830) Move a majority of the MiniLlapCliDriver tests to use an inline AM
[ https://issues.apache.org/jira/browse/HIVE-14830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-14830: -- Attachment: HIVE-14830.01.patch The patch moves a majority of the MiniLllap tests to run with an inline AM, and against localFs instead of HDFS. The ones left behind for MiniLlap fail in this mode - mainly because of direct references to HDFS, validation of HDFS counters, or in some cases diffs (many of which seemed invalid). cc [~prasanth_j] for review. This will take longer to run on jenkins at the moment, since there's no batching yet, and I cannot change ptest batching configs for this without breaking all other runs. > Move a majority of the MiniLlapCliDriver tests to use an inline AM > -- > > Key: HIVE-14830 > URL: https://issues.apache.org/jira/browse/HIVE-14830 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-14830.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14830) Move a majority of the MiniLlapCliDriver tests to use an inline AM
[ https://issues.apache.org/jira/browse/HIVE-14830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-14830: -- Status: Patch Available (was: Open) > Move a majority of the MiniLlapCliDriver tests to use an inline AM > -- > > Key: HIVE-14830 > URL: https://issues.apache.org/jira/browse/HIVE-14830 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-14830.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14099) Hive security authorization can be disabled by users
[
https://issues.apache.org/jira/browse/HIVE-14099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527700#comment-15527700
]
Hive QA commented on HIVE-14099:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12830528/HIVE-14099.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10644 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_join_part_col_char]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.testMetaDataCounts
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1318/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1318/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1318/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12830528 - PreCommit-HIVE-Build
> Hive security authorization can be disabled by users
>
>
> Key: HIVE-14099
> URL: https://issues.apache.org/jira/browse/HIVE-14099
> Project: Hive
> Issue Type: Improvement
> Components: Authorization
>Affects Versions: 0.13.1
>Reporter: Prashant Kumar Singh
>Assignee: Aihua Xu
> Attachments: HIVE-14099.1.patch
>
>
> In case we enables :
> hive.security.authorization.enabled=true in hive-site.xml
> this setting can be disabled by users at their hive prompt. There should be
> hardcoded setting in the configs.
> The other thing is once we enable authorization, the tables that got created
> before enabling looses access as they don't have authorization defined. How
> this situation can be tackled in hive.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-7224) Set incremental printing to true by default in Beeline
[ https://issues.apache.org/jira/browse/HIVE-7224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-7224: -- Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) > Set incremental printing to true by default in Beeline > -- > > Key: HIVE-7224 > URL: https://issues.apache.org/jira/browse/HIVE-7224 > Project: Hive > Issue Type: Bug > Components: Beeline, Clients, JDBC >Affects Versions: 0.13.0, 1.0.0, 1.2.0, 1.1.0 >Reporter: Vaibhav Gumashta >Assignee: Sahil Takiar > Fix For: 2.2.0 > > Attachments: HIVE-7224.1.patch, HIVE-7224.2.patch, HIVE-7224.2.patch, > HIVE-7224.3.patch, HIVE-7224.4.patch, HIVE-7224.5.patch > > > See HIVE-7221. > By default beeline tries to buffer the entire output relation before printing > it on stdout. This can cause OOM when the output relation is large. However, > beeline has the option of incremental prints. We should keep that as the > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7224) Set incremental printing to true by default in Beeline
[ https://issues.apache.org/jira/browse/HIVE-7224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527645#comment-15527645 ] Sergio Peña commented on HIVE-7224: --- Sure. +1 > Set incremental printing to true by default in Beeline > -- > > Key: HIVE-7224 > URL: https://issues.apache.org/jira/browse/HIVE-7224 > Project: Hive > Issue Type: Bug > Components: Beeline, Clients, JDBC >Affects Versions: 0.13.0, 1.0.0, 1.2.0, 1.1.0 >Reporter: Vaibhav Gumashta >Assignee: Sahil Takiar > Attachments: HIVE-7224.1.patch, HIVE-7224.2.patch, HIVE-7224.2.patch, > HIVE-7224.3.patch, HIVE-7224.4.patch, HIVE-7224.5.patch > > > See HIVE-7221. > By default beeline tries to buffer the entire output relation before printing > it on stdout. This can cause OOM when the output relation is large. However, > beeline has the option of incremental prints. We should keep that as the > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7224) Set incremental printing to true by default in Beeline
[ https://issues.apache.org/jira/browse/HIVE-7224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527624#comment-15527624 ] Sahil Takiar commented on HIVE-7224: [~spena] can this be merged? The change simply sets Beeline's incremental property to true. There were a few subtasks that needed to be completed before this could be merged, but they are all resolved now. > Set incremental printing to true by default in Beeline > -- > > Key: HIVE-7224 > URL: https://issues.apache.org/jira/browse/HIVE-7224 > Project: Hive > Issue Type: Bug > Components: Beeline, Clients, JDBC >Affects Versions: 0.13.0, 1.0.0, 1.2.0, 1.1.0 >Reporter: Vaibhav Gumashta >Assignee: Sahil Takiar > Attachments: HIVE-7224.1.patch, HIVE-7224.2.patch, HIVE-7224.2.patch, > HIVE-7224.3.patch, HIVE-7224.4.patch, HIVE-7224.5.patch > > > See HIVE-7221. > By default beeline tries to buffer the entire output relation before printing > it on stdout. This can cause OOM when the output relation is large. However, > beeline has the option of incremental prints. We should keep that as the > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13913) LLAP: introduce backpressure to recordreader
[
https://issues.apache.org/jira/browse/HIVE-13913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527496#comment-15527496
]
Hive QA commented on HIVE-13913:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12811254/HIVE-13913.03.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1317/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1317/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1317/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-09-27 21:41:22.959
+ [[ -n /usr/java/jdk1.8.0_25 ]]
+ export JAVA_HOME=/usr/java/jdk1.8.0_25
+ JAVA_HOME=/usr/java/jdk1.8.0_25
+ export
PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+
PATH=/usr/java/jdk1.8.0_25/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1317/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-09-27 21:41:22.961
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
warning: unable to access '/home/sseth/.config/git/attributes': Permission
denied
warning: unable to access '/home/sseth/.config/git/attributes': Permission
denied
warning: unable to access '/home/sseth/.config/git/attributes': Permission
denied
HEAD is now at d3b88f6 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git clean -f -d
warning: unable to access '/home/sseth/.config/git/ignore': Permission denied
+ git checkout master
warning: unable to access '/home/sseth/.config/git/attributes': Permission
denied
warning: unable to access '/home/sseth/.config/git/ignore': Permission denied
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
warning: unable to access '/home/sseth/.config/git/attributes': Permission
denied
warning: unable to access '/home/sseth/.config/git/attributes': Permission
denied
HEAD is now at d3b88f6 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-09-27 21:41:23.958
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
warning: unable to access '/home/sseth/.config/git/attributes': Permission
denied
error: patch failed:
llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapInputFormat.java:20
error:
llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapInputFormat.java:
patch does not apply
error: patch failed:
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/OrcEncodedDataReader.java:156
error:
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/OrcEncodedDataReader.java:
patch does not apply
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12811254 - PreCommit-HIVE-Build
> LLAP: introduce backpressure to recordreader
>
>
> Key: HIVE-13913
> URL: https://issues.apache.org/jira/browse/HIVE-13913
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-13913.01.patch, HIVE-13913.02.patch,
> HIVE-13913.03.patch, HIVE-13913.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14843) HIVE-14751 introduced ambiguity in grammar
[
https://issues.apache.org/jira/browse/HIVE-14843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527447#comment-15527447
]
Pengcheng Xiong commented on HIVE-14843:
[~jcamachorodriguez] could u compile and confirm that the ambiguity is gone? If
so, +1. Thanks for your patch.
> HIVE-14751 introduced ambiguity in grammar
> --
>
> Key: HIVE-14843
> URL: https://issues.apache.org/jira/browse/HIVE-14843
> Project: Hive
> Issue Type: Bug
> Components: Parser
>Affects Versions: 2.2.0
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14843.01.patch
>
>
> As reported by [~ekoifman] and tested by myself, your patch introduced
> ambiguity into the grammar.
> {code}
> warning(200): IdentifiersParser.g:327:5:
> Decision can match input such as "KW_DAY KW_TO KW_SECOND" using multiple
> alternatives: 2, 5
> As a result, alternative(s) 5 were disabled for that input
> warning(200): IdentifiersParser.g:327:5:
> Decision can match input such as "KW_YEAR KW_TO KW_MONTH" using multiple
> alternatives: 1, 3
> As a result, alternative(s) 3 were disabled for that input
> {code}
> Could you please take another look? Thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14843) HIVE-14751 introduced ambiguity in grammar
[
https://issues.apache.org/jira/browse/HIVE-14843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527439#comment-15527439
]
Hive QA commented on HIVE-14843:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12830516/HIVE-14843.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10590 tests
executed
*Failed tests:*
{noformat}
173_UTBatch_hcatalog__hcatalog-pig-adapter_8_tests - did not produce a
TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_join_part_col_char]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.testMetaDataCounts
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1316/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1316/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1316/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12830516 - PreCommit-HIVE-Build
> HIVE-14751 introduced ambiguity in grammar
> --
>
> Key: HIVE-14843
> URL: https://issues.apache.org/jira/browse/HIVE-14843
> Project: Hive
> Issue Type: Bug
> Components: Parser
>Affects Versions: 2.2.0
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14843.01.patch
>
>
> As reported by [~ekoifman] and tested by myself, your patch introduced
> ambiguity into the grammar.
> {code}
> warning(200): IdentifiersParser.g:327:5:
> Decision can match input such as "KW_DAY KW_TO KW_SECOND" using multiple
> alternatives: 2, 5
> As a result, alternative(s) 5 were disabled for that input
> warning(200): IdentifiersParser.g:327:5:
> Decision can match input such as "KW_YEAR KW_TO KW_MONTH" using multiple
> alternatives: 1, 3
> As a result, alternative(s) 3 were disabled for that input
> {code}
> Could you please take another look? Thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14412) Add a timezone-aware timestamp
[
https://issues.apache.org/jira/browse/HIVE-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527345#comment-15527345
]
Jason Dere commented on HIVE-14412:
---
Thanks for the patch. Had a few questions on the conversions:
Date to TimestampTZ: Converted to -mm-dd 00:00:00, in the local TZ
date ‘1970-01-01’ => timestamp_tz ‘1970-01-01 00:00:00 GMT-08:00'
Timestamp to TimestampTZ: Converted to -mm-dd HH:MM:SS, in local TZ
timestamp '1970-01-01 00:00:00’ => timestamp_tz '1970-01-01 00:00:00 GMT-08:00'
TimestampTZ to Date:
timestamp_tz “1970-01-01 00:00:00 GMT-05:00” => date “1970-01-01"
TimestampTZ to Timestamp:
timestamp_tz“1970-01-01 00:00:00 GMT-05:00” => timestamp “1970-01-01 00:00:00”
I think this matches the SQL spec conversion behavior, but would be good for
someone to double check - [~alangates] or anyone else?
{quote}
For the convenience of users, whenever a datetime value with time zone is to be
implicitly derived from one without (for example, in a simple assignment
operation), SQL assumes the value without time zone to be local, subtracts the
current default time zone displacement of the SQL-session from it to give UTC,
and associates that time zone displacement with the result.
Conversely, whenever a datetime value without time zone is to be implicitly
derived from one with, SQL assumes the value with time zone to be UTC, adds the
time zone displacement to it to give local time, and the result, without any
time zone displacement, is local.
{quote}
Is there a way to convert TimestampTZ to a different TZ?
“1970-01-01 00:00:00 GMT” => “1969-12-31 16:00:00 GMT-08:00”
Comparison (equality, mim/max). Here both the spec, as well as Oracle
(https://docs.oracle.com/cd/B19306_01/server.102/b14225/ch4datetime.htm#i1006081),
mention that the time zone should not be used for comparison, just the instant
in UTC.
“1970-01-01 00:00:00 GMT” == “1969-12-31 16:00:00”
Is there way to keep non-numeric timezones, like ‘America/Los_Angeles’ rather
than ‘GMT-08:00'?
> Add a timezone-aware timestamp
> --
>
> Key: HIVE-14412
> URL: https://issues.apache.org/jira/browse/HIVE-14412
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-14412.1.patch, HIVE-14412.2.patch,
> HIVE-14412.3.patch, HIVE-14412.4.patch, HIVE-14412.5.patch,
> HIVE-14412.6.patch, HIVE-14412.7.patch, HIVE-14412.8.patch
>
>
> Java's Timestamp stores the time elapsed since the epoch. While it's by
> itself unambiguous, ambiguity comes when we parse a string into timestamp, or
> convert a timestamp to string, causing problems like HIVE-14305.
> To solve the issue, I think we should make timestamp aware of timezone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14848) S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
[
https://issues.apache.org/jira/browse/HIVE-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527300#comment-15527300
]
Sergio Peña commented on HIVE-14848:
[~lskuff] FYI
> S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
>
>
> Key: HIVE-14848
> URL: https://issues.apache.org/jira/browse/HIVE-14848
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.2.0
>Reporter: Sergio Peña
>Assignee: Sergio Peña
> Attachments: HIVE-14848.1.patch
>
>
> When S3 credentials are included in hive-site.xml, then MR jobs that need to
> read
> data from S3 cannot use them because S3 values are stripped from the Job
> configuration
> object before submitting the MR job.
> {noformat}
> @Override
> public void initialize(HiveConf conf, QueryPlan queryPlan, DriverContext
> driverContext) {
> ...
> conf.stripHiddenConfigurations(job);
> this.jobExecHelper = new HadoopJobExecHelper(job, console, this, this);
> }
> {noformat}
> We should use a MR property {{mapreduce.job.redacted-properties}} instead to
> hide this list on the MR side (such as history server UI) to allow MR run the
> job without issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14848) S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
[
https://issues.apache.org/jira/browse/HIVE-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14848:
---
Attachment: HIVE-14848.1.patch
> S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
>
>
> Key: HIVE-14848
> URL: https://issues.apache.org/jira/browse/HIVE-14848
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.2.0
>Reporter: Sergio Peña
>Assignee: Sergio Peña
> Attachments: HIVE-14848.1.patch
>
>
> When S3 credentials are included in hive-site.xml, then MR jobs that need to
> read
> data from S3 cannot use them because S3 values are stripped from the Job
> configuration
> object before submitting the MR job.
> {noformat}
> @Override
> public void initialize(HiveConf conf, QueryPlan queryPlan, DriverContext
> driverContext) {
> ...
> conf.stripHiddenConfigurations(job);
> this.jobExecHelper = new HadoopJobExecHelper(job, console, this, this);
> }
> {noformat}
> We should use a MR property {{mapreduce.job.redacted-properties}} instead to
> hide this list on the MR side (such as history server UI) to allow MR run the
> job without issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14848) S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
[
https://issues.apache.org/jira/browse/HIVE-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14848:
---
Status: Patch Available (was: Open)
> S3 creds added to a hidden list by HIVE-14588 are not working on MR jobs
>
>
> Key: HIVE-14848
> URL: https://issues.apache.org/jira/browse/HIVE-14848
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.2.0
>Reporter: Sergio Peña
>Assignee: Sergio Peña
> Attachments: HIVE-14848.1.patch
>
>
> When S3 credentials are included in hive-site.xml, then MR jobs that need to
> read
> data from S3 cannot use them because S3 values are stripped from the Job
> configuration
> object before submitting the MR job.
> {noformat}
> @Override
> public void initialize(HiveConf conf, QueryPlan queryPlan, DriverContext
> driverContext) {
> ...
> conf.stripHiddenConfigurations(job);
> this.jobExecHelper = new HadoopJobExecHelper(job, console, this, this);
> }
> {noformat}
> We should use a MR property {{mapreduce.job.redacted-properties}} instead to
> hide this list on the MR side (such as history server UI) to allow MR run the
> job without issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14778) document threading model of Streaming API
[ https://issues.apache.org/jira/browse/HIVE-14778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527166#comment-15527166 ] Eugene Koifman commented on HIVE-14778: --- It depends on what you mean by multiple threads. TransactionBatches are made from StreamingConnection objects. So the model is that if you want to write to the same HiveEndPoint in parallel, you create different StreamingConnection objects but from any given StreamingConnection you open/close TransactionBatches sequentially. (The exception is that you can heartbeat any given TransactionBatch using a separate thread). This seems like a reasonable model. For example, JDBC (usually) has the same model. You can create any number of connections but operations on a given Connection are expected to be sequential. > document threading model of Streaming API > - > > Key: HIVE-14778 > URL: https://issues.apache.org/jira/browse/HIVE-14778 > Project: Hive > Issue Type: Bug > Components: HCatalog, Transactions >Affects Versions: 0.14.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-14778.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > The model is not obvious and needs to be documented properly. > A StreamingConnection internally maintains 2 MetaStoreClient objects (each > has 1 Thrift client for actual RPC). Let's call them "primary" and > "heartbeat". Each TransactionBatch created from a given StreamingConnection, > gets a reference to both of these MetaStoreClients. > So the model is that there is at most 1 outstanding (not closed) > TransactionBatch for any given StreamingConnection and for any given > TransactionBatch there can be at most 2 threads accessing it concurrently. 1 > thread calling TransactionBatch.heartbeat() (and nothing else) and the other > calling all other methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13316) Upgrade to Calcite 1.9
[
https://issues.apache.org/jira/browse/HIVE-13316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527161#comment-15527161
]
Hive QA commented on HIVE-13316:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12830510/HIVE-13316.02.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 61 failed/errored test(s), 10048 tests
executed
*Failed tests:*
{noformat}
238_UTBatch_itests__hcatalog-unit_6_tests - did not produce a TEST-*.xml file
TestCliDriver-alter_char2.q-parquet_array_of_unannotated_primitives.q-vector_decimal_5.q-and-27-more
- did not produce a TEST-*.xml file
TestCliDriver-cbo_rp_lineage2.q-bool_literal.q-udf_lower.q-and-27-more - did
not produce a TEST-*.xml file
TestCliDriver-cbo_windowing.q-touch.q-nonreserved_keywords_input37.q-and-27-more
- did not produce a TEST-*.xml file
TestCliDriver-groupby_grouping_sets5.q-autogen_colalias.q-confirm_initial_tbl_stats.q-and-27-more
- did not produce a TEST-*.xml file
TestCliDriver-groupby_map_ppr.q-unionall_unbalancedppd.q-bucket3.q-and-27-more
- did not produce a TEST-*.xml file
TestCliDriver-groupby_sort_6.q-smb_mapjoin_2.q-index_compact_1.q-and-27-more -
did not produce a TEST-*.xml file
TestCliDriver-input31.q-cbo_rp_join0.q-udf_ceiling.q-and-27-more - did not
produce a TEST-*.xml file
TestCliDriver-insert_values_partitioned.q-insert_values_non_partitioned.q-parquet_types.q-and-27-more
- did not produce a TEST-*.xml file
TestCliDriver-join13.q-update_orig_table.q-smb_mapjoin_21.q-and-27-more - did
not produce a TEST-*.xml file
TestCliDriver-leadlag_queries.q-bucketmapjoin_negative.q-alter_table_location.q-and-27-more
- did not produce a TEST-*.xml file
TestCliDriver-sample2.q-multi_insert_with_join2.q-groupby7.q-and-27-more - did
not produce a TEST-*.xml file
TestCliDriver-tez_dynpart_hashjoin_3.q-authorization_revoke_table_priv.q-udf_bitwise_xor.q-and-27-more
- did not produce a TEST-*.xml file
TestCliDriver-union_top_level.q-ql_rewrite_gbtoidx_cbo_2.q-ppd_join3.q-and-27-more
- did not produce a TEST-*.xml file
TestCliDriver-varchar_serde.q-exim_11_managed_external.q-cbo_rp_udf_percentile.q-and-27-more
- did not produce a TEST-*.xml file
TestCliDriver-vectorized_parquet.q-describe_xpath.q-show_columns.q-and-27-more
- did not produce a TEST-*.xml file
TestHBaseCliDriver-hbase_ppd_key_range.q-hbase_binary_external_table_queries.q-hbase_pushdown.q-and-2-more
- did not produce a TEST-*.xml file
TestMiniLlapCliDriver-vector_partition_diff_num_cols.q-vectorized_rcfile_columnar.q-union8.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniLlapCliDriver-vectorization_16.q-vector_acid3.q-tez_union_dynamic_partition.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniLlapCliDriver-vectorized_parquet.q-enforce_order.q-orc_merge9.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-stats12.q-skewjoin_union_remove_2.q-union_top_level.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_select]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_SortUnionTransposeRule]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_subq_not_in]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explain_logical]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[limit0]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[limit_join_transpose]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[limit_pushdown3]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[limit_pushdown]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[lineage3]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mergejoin]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[offset_limit_ppd_optimizer]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_udf_col]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_notin]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_notin_having]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf5]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_hour]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_minute]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_join_part_col_char]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_12]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_limit]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[cbo_subq_not_in]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_4]
[jira] [Commented] (HIVE-14775) Investigate IOException usage in Metrics APIs
[ https://issues.apache.org/jira/browse/HIVE-14775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527063#comment-15527063 ] Mohit Sabharwal commented on HIVE-14775: A bunch of JMXExceptions can get thrown by MetricsMBeanImpl#getAttribute here: https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/common/metrics/MetricsMBeanImpl.java#L50-L51 AttributeNotFoundException, MBeanException, ReflectionException are all types of JMXException > Investigate IOException usage in Metrics APIs > - > > Key: HIVE-14775 > URL: https://issues.apache.org/jira/browse/HIVE-14775 > Project: Hive > Issue Type: Sub-task > Components: Hive, HiveServer2, Metastore >Reporter: Barna Zsombor Klara >Assignee: Barna Zsombor Klara > > A large number of metrics APIs seem to declare to throw IOExceptions > needlessly. (incrementCounter, decrementCounter etc.) > This is not only misleading but it fills up the code with unnecessary catch > blocks never to be reached. > We should investigate if these exceptions are thrown at all, and remove them > if it is truly unused. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14775) Investigate IOException usage in Metrics APIs
[ https://issues.apache.org/jira/browse/HIVE-14775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527043#comment-15527043 ] Szehon Ho commented on HIVE-14775: -- Yea definitely appreciate the cleanup, never had time to investigate. Do we know what scenario lead to JMXException? I did have only some minor comments, left on RB > Investigate IOException usage in Metrics APIs > - > > Key: HIVE-14775 > URL: https://issues.apache.org/jira/browse/HIVE-14775 > Project: Hive > Issue Type: Sub-task > Components: Hive, HiveServer2, Metastore >Reporter: Barna Zsombor Klara >Assignee: Barna Zsombor Klara > > A large number of metrics APIs seem to declare to throw IOExceptions > needlessly. (incrementCounter, decrementCounter etc.) > This is not only misleading but it fills up the code with unnecessary catch > blocks never to be reached. > We should investigate if these exceptions are thrown at all, and remove them > if it is truly unused. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-14835) Improve ptest2 build time
[ https://issues.apache.org/jira/browse/HIVE-14835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran resolved HIVE-14835. -- Resolution: Fixed Fix Version/s: 2.2.0 I am not sure if any changes has to be made in ptest cluster. I will monitor the next run to see if both changes are taking effect. Patch committed to master. Thanks [~spena] for the review! > Improve ptest2 build time > - > > Key: HIVE-14835 > URL: https://issues.apache.org/jira/browse/HIVE-14835 > Project: Hive > Issue Type: Sub-task > Components: Testing Infrastructure >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Fix For: 2.2.0 > > Attachments: HIVE-14835.1.patch > > > NO PRECOMMIT TESTS > 2 things can be improved > 1) ptest2 always downloads jars for compiling its own directory which takes > about 1m30s which should take only 5s with cache jars. The reason for that is > maven.repo.local is pointing to a path under WORKSPACE which will be cleaned > by jenkins for every run. > 2) For hive build we can make use of parallel build and quite the output of > build which should shave off another 15-30s. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HIVE-14835) Improve ptest2 build time
[ https://issues.apache.org/jira/browse/HIVE-14835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-14835 started by Prasanth Jayachandran. > Improve ptest2 build time > - > > Key: HIVE-14835 > URL: https://issues.apache.org/jira/browse/HIVE-14835 > Project: Hive > Issue Type: Sub-task > Components: Testing Infrastructure >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-14835.1.patch > > > NO PRECOMMIT TESTS > 2 things can be improved > 1) ptest2 always downloads jars for compiling its own directory which takes > about 1m30s which should take only 5s with cache jars. The reason for that is > maven.repo.local is pointing to a path under WORKSPACE which will be cleaned > by jenkins for every run. > 2) For hive build we can make use of parallel build and quite the output of > build which should shave off another 15-30s. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14558) Add support for listing views similar to "show tables"
[ https://issues.apache.org/jira/browse/HIVE-14558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-14558: - Attachment: HIVE-14558.4.patch Skipping patch index .3 and attaching .4 directly. This is to match the version attaching to the reviewboard. In the new patch, 1) Repurpose ShowTablesDesc to incorporate a type for the tables. This can also be used for "show views" where there the type becomes "TableType.VIRTUAL_VIEW. Deleted the ShowViewsDesc class that was added as part of this fix. 2) Using TableType instead of String for the type of the tables being fetched. I will upload a new patch with those changes. > Add support for listing views similar to "show tables" > -- > > Key: HIVE-14558 > URL: https://issues.apache.org/jira/browse/HIVE-14558 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam > Attachments: HIVE-14558.1.patch, HIVE-14558.2.patch, > HIVE-14558.4.patch > > > Users have been asking for such feature where they can get a lists of views > separately. > So perhaps a syntax similar to "show tables" command? > show views [in/from ] [] > Does it make sense to add such command? or is it not worth the effort? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14558) Add support for listing views similar to "show tables"
[ https://issues.apache.org/jira/browse/HIVE-14558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-14558: - Status: Patch Available (was: Open) > Add support for listing views similar to "show tables" > -- > > Key: HIVE-14558 > URL: https://issues.apache.org/jira/browse/HIVE-14558 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam > Attachments: HIVE-14558.1.patch, HIVE-14558.2.patch, > HIVE-14558.4.patch > > > Users have been asking for such feature where they can get a lists of views > separately. > So perhaps a syntax similar to "show tables" command? > show views [in/from ] [] > Does it make sense to add such command? or is it not worth the effort? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14558) Add support for listing views similar to "show tables"
[ https://issues.apache.org/jira/browse/HIVE-14558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-14558: - Status: Open (was: Patch Available) Got some additional feedback. I will attach a new patch with the summary of the changes. > Add support for listing views similar to "show tables" > -- > > Key: HIVE-14558 > URL: https://issues.apache.org/jira/browse/HIVE-14558 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam > Attachments: HIVE-14558.1.patch, HIVE-14558.2.patch > > > Users have been asking for such feature where they can get a lists of views > separately. > So perhaps a syntax similar to "show tables" command? > show views [in/from ] [] > Does it make sense to add such command? or is it not worth the effort? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14100) current_user() returns invalid information
[
https://issues.apache.org/jira/browse/HIVE-14100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526902#comment-15526902
]
Mohit Sabharwal commented on HIVE-14100:
Thanks, [~pvary]!
Could you fix the description to say SessionStateUserAuthenticator instead of
SessionStateAuthenticator ?
Also, I'm confused here, isn't the SessionStateUserAuthenticator authenticator
passed as the {{hive.security.authenticator.manager}} config when HS2 is
started in hiveServer2.cmd ? HIVE-9143 is getting the user from
SessionState.getUserFromAuthenticator, so isn't that udf already using
SessionStateUserAuthenticator ?
> current_user() returns invalid information
> --
>
> Key: HIVE-14100
> URL: https://issues.apache.org/jira/browse/HIVE-14100
> Project: Hive
> Issue Type: Bug
> Components: Authentication, Beeline
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Minor
> Attachments: HIVE-14100.2.patch, HIVE-14100.patch
>
>
> Using HadoopDeaultAuthenticator the current_user() returns the username of
> the unix user running hiveservice2.
> Using SessionStateAuthenticator the current_user returns the username which
> is provided when the connection started.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14099) Hive security authorization can be disabled by users
[ https://issues.apache.org/jira/browse/HIVE-14099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526874#comment-15526874 ] Aihua Xu commented on HIVE-14099: - Of course, the admin can configure hive.conf.restricted.list even without this patch. But seems it's reasonable to add them as default ones. > Hive security authorization can be disabled by users > > > Key: HIVE-14099 > URL: https://issues.apache.org/jira/browse/HIVE-14099 > Project: Hive > Issue Type: Improvement > Components: Authorization >Affects Versions: 0.13.1 >Reporter: Prashant Kumar Singh >Assignee: Aihua Xu > Attachments: HIVE-14099.1.patch > > > In case we enables : > hive.security.authorization.enabled=true in hive-site.xml > this setting can be disabled by users at their hive prompt. There should be > hardcoded setting in the configs. > The other thing is once we enable authorization, the tables that got created > before enabling looses access as they don't have authorization defined. How > this situation can be tackled in hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14099) Hive security authorization can be disabled by users
[ https://issues.apache.org/jira/browse/HIVE-14099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-14099: Status: Patch Available (was: Open) Patch-1: added the some security configurations to the default hive.conf.restricted.list. > Hive security authorization can be disabled by users > > > Key: HIVE-14099 > URL: https://issues.apache.org/jira/browse/HIVE-14099 > Project: Hive > Issue Type: Improvement > Components: Authorization >Affects Versions: 0.13.1 >Reporter: Prashant Kumar Singh >Assignee: Aihua Xu > Attachments: HIVE-14099.1.patch > > > In case we enables : > hive.security.authorization.enabled=true in hive-site.xml > this setting can be disabled by users at their hive prompt. There should be > hardcoded setting in the configs. > The other thing is once we enable authorization, the tables that got created > before enabling looses access as they don't have authorization defined. How > this situation can be tackled in hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14099) Hive security authorization can be disabled by users
[ https://issues.apache.org/jira/browse/HIVE-14099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-14099: Attachment: HIVE-14099.1.patch > Hive security authorization can be disabled by users > > > Key: HIVE-14099 > URL: https://issues.apache.org/jira/browse/HIVE-14099 > Project: Hive > Issue Type: Improvement > Components: Authorization >Affects Versions: 0.13.1 >Reporter: Prashant Kumar Singh >Assignee: Aihua Xu > Attachments: HIVE-14099.1.patch > > > In case we enables : > hive.security.authorization.enabled=true in hive-site.xml > this setting can be disabled by users at their hive prompt. There should be > hardcoded setting in the configs. > The other thing is once we enable authorization, the tables that got created > before enabling looses access as they don't have authorization defined. How > this situation can be tackled in hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-14099) Hive security authorization can be disabled by users
[ https://issues.apache.org/jira/browse/HIVE-14099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu reassigned HIVE-14099: --- Assignee: Aihua Xu > Hive security authorization can be disabled by users > > > Key: HIVE-14099 > URL: https://issues.apache.org/jira/browse/HIVE-14099 > Project: Hive > Issue Type: Improvement > Components: Authorization >Affects Versions: 0.13.1 >Reporter: Prashant Kumar Singh >Assignee: Aihua Xu > > In case we enables : > hive.security.authorization.enabled=true in hive-site.xml > this setting can be disabled by users at their hive prompt. There should be > hardcoded setting in the configs. > The other thing is once we enable authorization, the tables that got created > before enabling looses access as they don't have authorization defined. How > this situation can be tackled in hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14791) LLAP: Use FQDN for all communication
[
https://issues.apache.org/jira/browse/HIVE-14791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-14791:
Assignee: (was: Sergey Shelukhin)
> LLAP: Use FQDN for all communication
>
>
> Key: HIVE-14791
> URL: https://issues.apache.org/jira/browse/HIVE-14791
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 2.2.0
>Reporter: Gopal V
> Fix For: 2.2.0
>
>
> {code}
> llap-client/src/java/org/apache/hadoop/hive/llap/registry/impl/LlapFixedRegistryImpl.java:
> + socketAddress.getHostName());
> llap-client/src/java/org/apache/hadoop/hive/llap/registry/impl/LlapFixedRegistryImpl.java:
> host = socketAddress.getHostName();
> llap-common/src/java/org/apache/hadoop/hive/llap/metrics/MetricsUtils.java:
> public static String getHostName() {
> llap-common/src/java/org/apache/hadoop/hive/llap/metrics/MetricsUtils.java:
>return InetAddress.getLocalHost().getHostName();
> llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java:
> String name = address.getHostName();
> llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java:
> builder.setAmHost(address.getHostName());
> llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/AMReporter.java:
>nodeId = LlapNodeId.getInstance(localAddress.get().getHostName(),
> localAddress.get().getPort());
> llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/ContainerRunnerImpl.java:
> localAddress.get().getHostName(), vertex.getDagName(),
> qIdProto.getDagIndex(),
> llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/ContainerRunnerImpl.java:
> new ExecutionContextImpl(localAddress.get().getHostName()), env,
> llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/LlapDaemon.java:
>String hostName = MetricsUtils.getHostName();
> llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/LlapProtocolServerImpl.java:
> .setBindAddress(addr.getHostName())
> llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/TaskRunnerCallable.java:
> request.getContainerIdString(), executionContext.getHostName(),
> vertex.getDagName(),
> llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapIoImpl.java:
>String displayName = "LlapDaemonCacheMetrics-" +
> MetricsUtils.getHostName();
> llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapIoImpl.java:
>displayName = "LlapDaemonIOMetrics-" + MetricsUtils.getHostName();
> llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/TestLlapDaemonProtocolServerImpl.java:
> new LlapProtocolClientImpl(new Configuration(),
> serverAddr.getHostName(),
> llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskCommunicator.java:
> builder.setAmHost(getAddress().getHostName());
> llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskSchedulerService.java:
> String displayName = "LlapTaskSchedulerMetrics-" +
> MetricsUtils.getHostName();
> {code}
> In systems where the hostnames do not match FQDN, calling the
> getCanonicalHostName() will allow for resolution of the hostname when
> accessing from a different base domain.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14778) document threading model of Streaming API
[
https://issues.apache.org/jira/browse/HIVE-14778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526802#comment-15526802
]
Alan Gates commented on HIVE-14778:
---
These changes appear to say that the streaming is single threaded. I don't
think that's what you mean, but I want to make sure I understand what you're
saying, which I think is the following:
{quote}
A single HiveEndPoint object cannot support having more than one
TransactionBatch open and being committed to at the same time. Also it does
not properly support multiple threads committing in parallel, even inside one
TransactionBatch. However, it does support multiple threads as long as the
commits are serialized.
{quote}
Is that correct?
> document threading model of Streaming API
> -
>
> Key: HIVE-14778
> URL: https://issues.apache.org/jira/browse/HIVE-14778
> Project: Hive
> Issue Type: Bug
> Components: HCatalog, Transactions
>Affects Versions: 0.14.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-14778.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The model is not obvious and needs to be documented properly.
> A StreamingConnection internally maintains 2 MetaStoreClient objects (each
> has 1 Thrift client for actual RPC). Let's call them "primary" and
> "heartbeat". Each TransactionBatch created from a given StreamingConnection,
> gets a reference to both of these MetaStoreClients.
> So the model is that there is at most 1 outstanding (not closed)
> TransactionBatch for any given StreamingConnection and for any given
> TransactionBatch there can be at most 2 threads accessing it concurrently. 1
> thread calling TransactionBatch.heartbeat() (and nothing else) and the other
> calling all other methods.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14775) Investigate IOException usage in Metrics APIs
[ https://issues.apache.org/jira/browse/HIVE-14775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526800#comment-15526800 ] Mohit Sabharwal commented on HIVE-14775: Thanks, [~zsombor.klara], this looks like a useful cleanup! Couple minor comments on RB. Looks ok to trickle JMException all the way to the top instead of wrapping it inside IOException since MetricsMBean extends a JMX interface. cc [~szehon] in case he has any comments > Investigate IOException usage in Metrics APIs > - > > Key: HIVE-14775 > URL: https://issues.apache.org/jira/browse/HIVE-14775 > Project: Hive > Issue Type: Sub-task > Components: Hive, HiveServer2, Metastore >Reporter: Barna Zsombor Klara >Assignee: Barna Zsombor Klara > > A large number of metrics APIs seem to declare to throw IOExceptions > needlessly. (incrementCounter, decrementCounter etc.) > This is not only misleading but it fills up the code with unnecessary catch > blocks never to be reached. > We should investigate if these exceptions are thrown at all, and remove them > if it is truly unused. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13913) LLAP: introduce backpressure to recordreader
[ https://issues.apache.org/jira/browse/HIVE-13913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526801#comment-15526801 ] Sergey Shelukhin commented on HIVE-13913: - [~gopalv] this jira is still there btw ;) > LLAP: introduce backpressure to recordreader > > > Key: HIVE-13913 > URL: https://issues.apache.org/jira/browse/HIVE-13913 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13913.01.patch, HIVE-13913.02.patch, > HIVE-13913.03.patch, HIVE-13913.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9423) HiveServer2: Provide the user with different error messages depending on the Thrift client exception code
[
https://issues.apache.org/jira/browse/HIVE-9423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526618#comment-15526618
]
Peter Vary commented on HIVE-9423:
--
Hi [~vgumashta],
Ok - I kept the description to document that there is no hangs any more. I
would be happy to help in that jira - maybe I was wrong when I thought that
having hive.server2.thrift.max.worker.threads,
hive.server2.thrift.min.worker.threads,
hive.server2.thrift.exponential.backoff.slot.length and
hive.server2.thrift.login.timeout to control the Thrift server connection
settings is enough.
Since 0.9.2 the Thrift code respect the configurations above - no hangs, and
restarts are needed -, and with this patch I try to provide a meaningful error
message.
If you have something even better, I would be happy to help.
Thanks to checking out this jira,
Peter
> HiveServer2: Provide the user with different error messages depending on the
> Thrift client exception code
> -
>
> Key: HIVE-9423
> URL: https://issues.apache.org/jira/browse/HIVE-9423
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 0.12.0, 0.13.0, 0.14.0, 0.15.0
>Reporter: Vaibhav Gumashta
>Assignee: Peter Vary
> Attachments: HIVE-9423.2.patch, HIVE-9423.3.patch, HIVE-9423.4.patch,
> HIVE-9423.5.patch, HIVE-9423.patch
>
>
> An example of where it is needed: it has been reported that when # of client
> connections is greater than {{hive.server2.thrift.max.worker.threads}},
> HiveServer2 stops accepting new connections and ends up having to be
> restarted. This should be handled more gracefully by the server and the JDBC
> driver, so that the end user gets aware of the problem and can take
> appropriate steps (either close existing connections or bump of the config
> value or use multiple server instances with dynamic service discovery
> enabled). Similarly, we should also review the behaviour of background thread
> pool to have a well defined behavior on the the pool getting exhausted.
> Ideally implementing some form of general admission control will be a better
> solution, so that we do not accept new work unless sufficient resources are
> available and display graceful degradation under overload.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14550) HiveServer2: enable ThriftJDBCBinarySerde use by default
[ https://issues.apache.org/jira/browse/HIVE-14550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526586#comment-15526586 ] Vaibhav Gumashta commented on HIVE-14550: - [~ziyangz] The failed tests might need investigation > HiveServer2: enable ThriftJDBCBinarySerde use by default > > > Key: HIVE-14550 > URL: https://issues.apache.org/jira/browse/HIVE-14550 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC, ODBC >Affects Versions: 2.1.0 >Reporter: Vaibhav Gumashta >Assignee: Ziyang Zhao > Attachments: HIVE-14550.1.patch > > > We've covered all items in HIVE-12427 and created HIVE-14549 for part2 of the > effort. Before closing the umbrella jira, we should enable this feature by > default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12222) Define port range in property for RPCServer
[
https://issues.apache.org/jira/browse/HIVE-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526582#comment-15526582
]
Xuefu Zhang commented on HIVE-1:
+1
> Define port range in property for RPCServer
> ---
>
> Key: HIVE-1
> URL: https://issues.apache.org/jira/browse/HIVE-1
> Project: Hive
> Issue Type: Improvement
> Components: CLI, Spark
>Affects Versions: 1.2.1
> Environment: Apache Hadoop 2.7.0
> Apache Hive 1.2.1
> Apache Spark 1.5.1
>Reporter: Andrew Lee
>Assignee: Aihua Xu
> Attachments: HIVE-1.1.patch, HIVE-1.2.patch,
> HIVE-1.3.patch
>
>
> Creating this JIRA after discussin with Xuefu on the dev mailing list. Would
> need some help to review and update the fields in this JIRA ticket, thanks.
> I notice that in
> ./spark-client/src/main/java/org/apache/hive/spark/client/rpc/RpcServer.java
> The port number is assigned with 0 which means it will be a random port every
> time when the RPC Server is created to talk to Spark in the same session.
> Because of this, this is causing problems to configure firewall between the
> HiveCLI RPC Server and Spark due to unpredictable port numbers here. In other
> word, users need to open all hive ports range
> from Data Node => HiveCLI (edge node).
> {code}
> this.channel = new ServerBootstrap()
> .group(group)
> .channel(NioServerSocketChannel.class)
> .childHandler(new ChannelInitializer() {
> @Override
> public void initChannel(SocketChannel ch) throws Exception {
> SaslServerHandler saslHandler = new SaslServerHandler(config);
> final Rpc newRpc = Rpc.createServer(saslHandler, config, ch,
> group);
> saslHandler.rpc = newRpc;
> Runnable cancelTask = new Runnable() {
> @Override
> public void run() {
> LOG.warn("Timed out waiting for hello from client.");
> newRpc.close();
> }
> };
> saslHandler.cancelTask = group.schedule(cancelTask,
> RpcServer.this.config.getServerConnectTimeoutMs(),
> TimeUnit.MILLISECONDS);
> }
> })
> {code}
> 2 Main reasons.
> - Most users (what I see and encounter) use HiveCLI as a command line tool,
> and in order to use that, they need to login to the edge node (via SSH). Now,
> here comes the interesting part.
> Could be true or not, but this is what I observe and encounter from time to
> time. Most users will abuse the resource on that edge node (increasing
> HADOOP_HEAPSIZE, dumping output to local disk, running huge python workflow,
> etc), this may cause the HS2 process to run into OOME, choke and die, etc.
> various resource issues including others like login, etc.
> - Analyst connects to Hive via HS2 + ODBC. So HS2 needs to be highly
> available. This makes sense to run it on the gateway node or a service node
> and separated from the HiveCLI.
> The logs are located in different location, monitoring and auditing is easier
> to run HS2 with a daemon user account, etc. so we don't want users to run
> HiveCLI where HS2 is running.
> It's better to isolate the resource this way to avoid any memory, file
> handlers, disk space, issues.
> From a security standpoint,
> - Since users can login to edge node (via SSH), the security on the edge node
> needs to be fortified and enhanced. Therefore, all the FW comes in and
> auditing.
> - Regulation/compliance for auditing is another requirement to monitor all
> traffic, specifying ports and locking down the ports makes it easier since we
> can focus
> on a range to monitor and audit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14847) HiveServer2: Implement some admission control mechanism for graceful degradation when resources are exhausted
[
https://issues.apache.org/jira/browse/HIVE-14847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-14847:
Assignee: (was: Peter Vary)
> HiveServer2: Implement some admission control mechanism for graceful
> degradation when resources are exhausted
> -
>
> Key: HIVE-14847
> URL: https://issues.apache.org/jira/browse/HIVE-14847
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 0.12.0, 0.13.0, 0.14.0, 0.15.0
>Reporter: Vaibhav Gumashta
>
> An example of where it is needed: it has been reported that when # of client
> connections is greater than {{hive.server2.thrift.max.worker.threads}},
> HiveServer2 stops accepting new connections and ends up having to be
> restarted. This should be handled more gracefully by the server and the JDBC
> driver, so that the end user gets aware of the problem and can take
> appropriate steps (either close existing connections or bump of the config
> value or use multiple server instances with dynamic service discovery
> enabled). Similarly, we should also review the behaviour of background thread
> pool to have a well defined behavior on the the pool getting exhausted.
> Ideally implementing some form of general admission control will be a better
> solution, so that we do not accept new work unless sufficient resources are
> available and display graceful degradation under overload.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14146) Column comments with "\n" character "corrupts" table metadata
[
https://issues.apache.org/jira/browse/HIVE-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526568#comment-15526568
]
Hive QA commented on HIVE-14146:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12830464/HIVE-14146.11.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10645 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_join_part_col_char]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.testMetaDataCounts
org.apache.hadoop.hive.thrift.TestHadoopAuthBridge23.testDelegationTokenSharedStore
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
org.apache.hive.spark.client.TestSparkClient.testJobSubmission
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1314/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1314/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1314/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12830464 - PreCommit-HIVE-Build
> Column comments with "\n" character "corrupts" table metadata
> -
>
> Key: HIVE-14146
> URL: https://issues.apache.org/jira/browse/HIVE-14146
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 2.2.0
>Reporter: Peter Vary
>Assignee: Peter Vary
> Attachments: HIVE-14146.10.patch, HIVE-14146.11.patch,
> HIVE-14146.2.patch, HIVE-14146.3.patch, HIVE-14146.4.patch,
> HIVE-14146.5.patch, HIVE-14146.6.patch, HIVE-14146.7.patch,
> HIVE-14146.8.patch, HIVE-14146.9.patch, HIVE-14146.patch, changes
>
>
> Create a table with the following(noting the \n in the COMMENT):
> {noformat}
> CREATE TABLE commtest(first_nm string COMMENT 'Indicates First name\nof an
> individual’);
> {noformat}
> Describe shows that now the metadata is messed up:
> {noformat}
> beeline> describe commtest;
> +---++---+--+
> | col_name | data_type |comment|
> +---++---+--+
> | first_nm | string | Indicates First name |
> | of an individual | NULL | NULL |
> +---++---+--+
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
