[jira] [Created] (HIVE-24458) Allow access to SArgs without converting to disjunctive normal form
Owen O'Malley created HIVE-24458: Summary: Allow access to SArgs without converting to disjunctive normal form Key: HIVE-24458 URL: https://issues.apache.org/jira/browse/HIVE-24458 Project: Hive Issue Type: Improvement Reporter: Owen O'Malley Assignee: Owen O'Malley For some use cases, it is useful to have access to the SArg expression in a non-normalized form. Currently, the SArg only provides the fully normalized expression. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24457) WorkloadManager.scheduleWork should use FileSystem.getLocal(conf) to delete files
Hao Zhong created HIVE-24457:
Summary: WorkloadManager.scheduleWork should use
FileSystem.getLocal(conf) to delete files
Key: HIVE-24457
URL: https://issues.apache.org/jira/browse/HIVE-24457
Project: Hive
Issue Type: Bug
Reporter: Hao Zhong
WorkloadManager.scheduleWork deletes files as follows:
{code:java}
private void scheduleWork(WmThreadSyncWork context) {
...
// 4. Delete unneeded directories that were replaced by other ones via reopen.
for (final Path path : context.pathsToDelete) {
LOG.info("Deleting {}", path);
workPool.submit(() -> {
try {
path.getFileSystem(conf).delete(path, true);
} catch (Exception ex) {
LOG.error("Failed to delete an old path; ignoring " +
ex.getMessage());
}
});
}
context.pathsToDelete.clear();
}{code}
It should call FileSystem.getLocal(conf).
A previous bug report complains a similar problem:
https://issues.apache.org/jira/browse/HIVE-8056
Its buggy code is:
{code:java}
private void dropSessionPaths(Configuration conf) throws IOException {
if (hdfsSessionPath != null) {
hdfsSessionPath.getFileSystem(conf).delete(hdfsSessionPath, true);
}
if (localSessionPath != null) {
localSessionPath.getFileSystem(conf).delete(localSessionPath, true);
}
{code}
Its fixed code is:
{code:java}
private void dropSessionPaths(Configuration conf) throws IOException {
if (hdfsSessionPath != null) {
hdfsSessionPath.getFileSystem(conf).delete(hdfsSessionPath, true);
}
if (localSessionPath != null) {
FileSystem.getLocal(conf).delete(localSessionPath, true);
}
}
{code}
BTW, HIVE-8056 warns that the bug is flaky. This bug can also be flaky.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24456) Column masking/hashing function in hive should use SH512 if FIPS mode is enabled
Sai Hemanth Gantasala created HIVE-24456: Summary: Column masking/hashing function in hive should use SH512 if FIPS mode is enabled Key: HIVE-24456 URL: https://issues.apache.org/jira/browse/HIVE-24456 Project: Hive Issue Type: Wish Components: HiveServer2 Reporter: Sai Hemanth Gantasala Assignee: Sai Hemanth Gantasala hive-site.xml should have the following property to indicate that FIPS mode is enabled. hive.masking.algo sha256 If this property is present, then GenericUDFMaskHash should use SHA512 instead of SHA256 encoding for column masking. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Commit Message Mistake
Yeap, that's exactly what I had in mind :) Maybe the community should reconsider the rules for force pushing. On Mon, Nov 30, 2020 at 5:25 PM László Bodor wrote: > Hi! > > Amend means a force push in general (right?), which is something we are not > about to do. However, until no newer commits come in (as in this case, when > you immediately noticed the problem), we should consider amending/force > pushing. Not sure if this will be accepted/supported in the community, just > thinking aloud. > > Regards, > Laszlo Bodor > > > On Mon, 30 Nov 2020 at 16:59, Stamatis Zampetakis > wrote: > > > Hello, > > > > Personal preferences but I think it would be better to allow such > > amendments instead of maintaining a separate file/commits for this > purpose. > > Cleaner history makes various operations easier (git bisect for instance) > > and the danger of losing commits is rather low (with all the commit > > notification emails in-place). > > > > Best, > > Stamatis > > > > On Mon, Nov 30, 2020 at 4:03 PM David wrote: > > > > > Hey Peter, > > > > > > Ya, yes. The list of shame. Ha. > > > > > > Updated. Thanks. > > > > > > On Mon, Nov 30, 2020 at 9:50 AM Peter Vary > > > > wrote: > > > > > > > Hi David, > > > > > > > > You are not the first and most probably not the last to do such a > > > mistake.. > > > > You should check the errata.txt for fixing these mistakes. > > > > > > > > Thanks, > > > > Peter > > > > > > > > > On Nov 30, 2020, at 15:33, David wrote: > > > > > > > > > > Hello Team, > > > > > > > > > > I just merged PR #1703 to the master branch via GitHub. However, > > when > > > I > > > > > went to edit the commit message, I accidentally hit the "Enter" key > > > > instead > > > > > of the "Shift" key and the commit was merged with an improper > commit > > > > > message. > > > > > > > > > > It should read: > > > > > > > > > > HIVE-24423: Improve DbNotificationListener Thread (David Mollitor > > > > reviewed > > > > > by Naveen Gangam, Miklos Gergely) > > > > > > > > > > https://github.com/apache/hive/pull/1703 > > > > > > > > > > What's the best course of action here? Does someone have the power > > to > > > > > amend the message? > > > > > > > > > > Again, my apologies. Thanks. > > > > > > > > > > > > > >
[jira] [Created] (HIVE-24455) Fix broken junit framework in storage-api
Owen O'Malley created HIVE-24455: Summary: Fix broken junit framework in storage-api Key: HIVE-24455 URL: https://issues.apache.org/jira/browse/HIVE-24455 Project: Hive Issue Type: Bug Reporter: Owen O'Malley Assignee: Owen O'Malley The use of junit is broken in storage-api. It results in no tests being found. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24454) Protobuf Mesage Incompatibility Detector
junwen yang created HIVE-24454:
--
Summary: Protobuf Mesage Incompatibility Detector
Key: HIVE-24454
URL: https://issues.apache.org/jira/browse/HIVE-24454
Project: Hive
Issue Type: Bug
Reporter: junwen yang
Attachments: hive_proto_incompatibility.txt
We have created a static checker which keeps track of the proto file change,
and detect potential incompatibility:
# Add/delete required field, which is the case reported in HBASE-25238.
# The tag number of a field has been changed, as described in HDFS-9788. Also,
the [protobuf guidelines
|https://developers.google.com/protocol-buffers/docs/proto]suggests _each field
in the message definition has a *unique number*. These numbers are used to
identify your fields in the [message binary
format|https://developers.google.com/protocol-buffers/docs/encoding], and
should not be changed once your message type is in use_.
# A required field has been changed to optional, or an optional field has
been changed to required. According to the guidelines in [protobuf official
website|https://developers.google.com/protocol-buffers/docs/proto], _*Required
Is Forever* You should be very careful about marking fields as {{required}}. If
at some point you wish to stop writing or sending a required field, it will be
problematic to change the field to an optional field - old readers will
consider messages without this field to be incomplete and may reject or drop
them unintentionally. You should consider writing application-specific custom
validation routines for your buffers instead._
We have applied our checker on the frequently maintained HIVE versions:
rel/release-1.2.2, rel/release-2.1.0, rel/release-2.1.1, rel/release-2.2.0,
rel/release-2.3.0, rel/release-2.3.1, rel/release-2.3.2, rel/release-2.3.3,
rel/release-2.3.4, rel/release-2.3.5, rel/release-2.3.6, rel/release-2.3.7,
rel/release-3.0.0, rel/release-3.1.0, rel/release-3.1.1, rel/release-3.1.2, and
we found 260 potential problems as attached.
The results reported by our checker got confirmation from developers in HBASE,
which can be found here HBASE-25340.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24453) DirectSQL error when parsing create_time value for database
Jesus Camacho Rodriguez created HIVE-24453:
--
Summary: DirectSQL error when parsing create_time value for
database
Key: HIVE-24453
URL: https://issues.apache.org/jira/browse/HIVE-24453
Project: Hive
Issue Type: Bug
Components: Metastore
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
HIVE-21077 introduced a {{create_time}} field for {{DBS}} table in HMS.
Although the value for that field is always set after that patch, the value
could be null if the database was created before the feature went in. DirectSQL
should check for null value before parsing the integer, otherwise we hit an
exception and fallback to ORM path:
{noformat}
2020-11-28 09:06:05,414 WARN org.apache.hadoop.hive.metastore.ObjectStore:
[pool-8-thread-194]: Falling back to ORM path due to direct SQL failure (this
is not an error): null at
org.apache.hadoop.hive.metastore.MetastoreDirectSqlUtils.extractSqlInt(MetastoreDirectSqlUtils.java:251)
at
org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getDatabase(MetaStoreDirectSql.java:420)
at
org.apache.hadoop.hive.metastore.ObjectStore$1.getSqlResult(ObjectStore.java:839)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24452) Add a generic JDBC implementation that can be used to other JDBC DBs
Naveen Gangam created HIVE-24452: Summary: Add a generic JDBC implementation that can be used to other JDBC DBs Key: HIVE-24452 URL: https://issues.apache.org/jira/browse/HIVE-24452 Project: Hive Issue Type: Sub-task Reporter: Naveen Gangam Currently, we added a custom provider for each of the JDBC DBs supported by hive (MySQL, Postgres, MSSQL(pending), Oracle(pending) and Derby (pending)). But if there are other JDBC providers we want to add support for, adding a generic JDBC provider would be useful that hive can default to. This means 1) We have to support means to indicate that a connector is for a JDBC datasource. So maybe add a property in DCPROPERTIES on connector to indicate that the datasource supports JDBC. 2) If there is no custom connector for a data source, use the GenericJDBCDatasource connector that is to be added as part of this jira. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24451) Add schema changes for MSSQL
Naveen Gangam created HIVE-24451: Summary: Add schema changes for MSSQL Key: HIVE-24451 URL: https://issues.apache.org/jira/browse/HIVE-24451 Project: Hive Issue Type: Sub-task Reporter: Naveen Gangam The current patch does not include schema changes for MSSQL backend. This should be right after the initial commit. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24450) DbNotificationListener Request Notification IDs in Batches
David Mollitor created HIVE-24450: - Summary: DbNotificationListener Request Notification IDs in Batches Key: HIVE-24450 URL: https://issues.apache.org/jira/browse/HIVE-24450 Project: Hive Issue Type: Improvement Reporter: David Mollitor Assignee: David Mollitor Every time a new notification event is logged into the database, the sequence number for the ID of the even is incremented by one. It is very standard in database design to instead request a block of IDs for each fetch from the database. The sequence numbers are then handed out locally until the block of IDs is exhausted. This allows for fewer database round-trips and transactions, at the expense of perhaps burning a few IDs. Burning of IDs happens when the server is restarted in the middle of a block of sequence IDs. That is, if the HMS requests a block of 10 ids, and only three have been assigned, after the restart, the HMS will request another block of 10, burning (wasting) 7 IDs. As long as the blocks are not too small, and restarts are infrequent, then few IDs are lost. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Commit Message Mistake
Hi! Amend means a force push in general (right?), which is something we are not about to do. However, until no newer commits come in (as in this case, when you immediately noticed the problem), we should consider amending/force pushing. Not sure if this will be accepted/supported in the community, just thinking aloud. Regards, Laszlo Bodor On Mon, 30 Nov 2020 at 16:59, Stamatis Zampetakis wrote: > Hello, > > Personal preferences but I think it would be better to allow such > amendments instead of maintaining a separate file/commits for this purpose. > Cleaner history makes various operations easier (git bisect for instance) > and the danger of losing commits is rather low (with all the commit > notification emails in-place). > > Best, > Stamatis > > On Mon, Nov 30, 2020 at 4:03 PM David wrote: > > > Hey Peter, > > > > Ya, yes. The list of shame. Ha. > > > > Updated. Thanks. > > > > On Mon, Nov 30, 2020 at 9:50 AM Peter Vary > > wrote: > > > > > Hi David, > > > > > > You are not the first and most probably not the last to do such a > > mistake.. > > > You should check the errata.txt for fixing these mistakes. > > > > > > Thanks, > > > Peter > > > > > > > On Nov 30, 2020, at 15:33, David wrote: > > > > > > > > Hello Team, > > > > > > > > I just merged PR #1703 to the master branch via GitHub. However, > when > > I > > > > went to edit the commit message, I accidentally hit the "Enter" key > > > instead > > > > of the "Shift" key and the commit was merged with an improper commit > > > > message. > > > > > > > > It should read: > > > > > > > > HIVE-24423: Improve DbNotificationListener Thread (David Mollitor > > > reviewed > > > > by Naveen Gangam, Miklos Gergely) > > > > > > > > https://github.com/apache/hive/pull/1703 > > > > > > > > What's the best course of action here? Does someone have the power > to > > > > amend the message? > > > > > > > > Again, my apologies. Thanks. > > > > > > > > >
Re: Commit Message Mistake
Hello, Personal preferences but I think it would be better to allow such amendments instead of maintaining a separate file/commits for this purpose. Cleaner history makes various operations easier (git bisect for instance) and the danger of losing commits is rather low (with all the commit notification emails in-place). Best, Stamatis On Mon, Nov 30, 2020 at 4:03 PM David wrote: > Hey Peter, > > Ya, yes. The list of shame. Ha. > > Updated. Thanks. > > On Mon, Nov 30, 2020 at 9:50 AM Peter Vary > wrote: > > > Hi David, > > > > You are not the first and most probably not the last to do such a > mistake.. > > You should check the errata.txt for fixing these mistakes. > > > > Thanks, > > Peter > > > > > On Nov 30, 2020, at 15:33, David wrote: > > > > > > Hello Team, > > > > > > I just merged PR #1703 to the master branch via GitHub. However, when > I > > > went to edit the commit message, I accidentally hit the "Enter" key > > instead > > > of the "Shift" key and the commit was merged with an improper commit > > > message. > > > > > > It should read: > > > > > > HIVE-24423: Improve DbNotificationListener Thread (David Mollitor > > reviewed > > > by Naveen Gangam, Miklos Gergely) > > > > > > https://github.com/apache/hive/pull/1703 > > > > > > What's the best course of action here? Does someone have the power to > > > amend the message? > > > > > > Again, my apologies. Thanks. > > > > >
[jira] [Created] (HIVE-24449) Implement connector provider for Derby DB
Naveen Gangam created HIVE-24449: Summary: Implement connector provider for Derby DB Key: HIVE-24449 URL: https://issues.apache.org/jira/browse/HIVE-24449 Project: Hive Issue Type: Sub-task Components: Hive Affects Versions: 4.0.0 Reporter: Naveen Gangam Assignee: Naveen Gangam Provide an implementation of Connector provider for Derby DB. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24448) Support case-sensitivity for tables in REMOTE database.
Naveen Gangam created HIVE-24448:
Summary: Support case-sensitivity for tables in REMOTE database.
Key: HIVE-24448
URL: https://issues.apache.org/jira/browse/HIVE-24448
Project: Hive
Issue Type: Sub-task
Components: HiveServer2
Affects Versions: 4.0.0
Reporter: Naveen Gangam
Hive tables are case-insensitive. So any case specified in user queries are
converted to lower case for query planning and all of the HMS metadata is also
persisted as lower case names.
However, with REMOTE data sources, certain data source will support
case-sensitivity for tables.
So HiveServer2 query planner needs to preserve user-provided case to be used
with HMS APIs, for HMS to be able to fetch the metadata from a remote data
source.
We now see something like this
{noformat}
2020-11-25T16:45:36,402 WARN [HiveServer2-Handler-Pool: Thread-76]
thrift.ThriftCLIService: Error executing statement:
org.apache.hive.service.cli.HiveSQLException: Error while compiling statement:
FAILED: RuntimeException
MetaException(message:org.apache.hadoop.hive.serde2.SerDeException
org.apache.hive.storage.jdbc.exception.HiveJdbcDatabaseAccessException: Error
while trying to get column names: Table 'hive1.txns' doesn't exist)
at
org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:365)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:206)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:262)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.service.cli.operation.Operation.run(Operation.java:277)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:560)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:545)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at sun.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) ~[?:?]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_231]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_231]
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.8.0_231]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_231]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
~[hadoop-common-3.1.0.jar:?]
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at com.sun.proxy.$Proxy43.executeStatementAsync(Unknown Source) ~[?:?]
at
org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:315)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:571)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1550)
~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1530)
~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:313)
~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_231]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_231]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_231]
Caused by: java.lang.RuntimeException:
[jira] [Created] (HIVE-24447) Move create/drop/alter table to the provider interface
Naveen Gangam created HIVE-24447: Summary: Move create/drop/alter table to the provider interface Key: HIVE-24447 URL: https://issues.apache.org/jira/browse/HIVE-24447 Project: Hive Issue Type: Sub-task Components: Hive Affects Versions: 4.0.0 Reporter: Naveen Gangam Assignee: Naveen Gangam The support for such operations on a table in a REMOTE database will be left to the discretion of the providers to support/implement. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Commit Message Mistake
Hey Peter, Ya, yes. The list of shame. Ha. Updated. Thanks. On Mon, Nov 30, 2020 at 9:50 AM Peter Vary wrote: > Hi David, > > You are not the first and most probably not the last to do such a mistake.. > You should check the errata.txt for fixing these mistakes. > > Thanks, > Peter > > > On Nov 30, 2020, at 15:33, David wrote: > > > > Hello Team, > > > > I just merged PR #1703 to the master branch via GitHub. However, when I > > went to edit the commit message, I accidentally hit the "Enter" key > instead > > of the "Shift" key and the commit was merged with an improper commit > > message. > > > > It should read: > > > > HIVE-24423: Improve DbNotificationListener Thread (David Mollitor > reviewed > > by Naveen Gangam, Miklos Gergely) > > > > https://github.com/apache/hive/pull/1703 > > > > What's the best course of action here? Does someone have the power to > > amend the message? > > > > Again, my apologies. Thanks. > >
Re: Commit Message Mistake
Hi David, You are not the first and most probably not the last to do such a mistake.. You should check the errata.txt for fixing these mistakes. Thanks, Peter > On Nov 30, 2020, at 15:33, David wrote: > > Hello Team, > > I just merged PR #1703 to the master branch via GitHub. However, when I > went to edit the commit message, I accidentally hit the "Enter" key instead > of the "Shift" key and the commit was merged with an improper commit > message. > > It should read: > > HIVE-24423: Improve DbNotificationListener Thread (David Mollitor reviewed > by Naveen Gangam, Miklos Gergely) > > https://github.com/apache/hive/pull/1703 > > What's the best course of action here? Does someone have the power to > amend the message? > > Again, my apologies. Thanks.
Commit Message Mistake
Hello Team, I just merged PR #1703 to the master branch via GitHub. However, when I went to edit the commit message, I accidentally hit the "Enter" key instead of the "Shift" key and the commit was merged with an improper commit message. It should read: HIVE-24423: Improve DbNotificationListener Thread (David Mollitor reviewed by Naveen Gangam, Miklos Gergely) https://github.com/apache/hive/pull/1703 What's the best course of action here? Does someone have the power to amend the message? Again, my apologies. Thanks.
[jira] [Created] (HIVE-24446) Materialized View plan remove explicit cast from query
Krisztian Kasa created HIVE-24446:
-
Summary: Materialized View plan remove explicit cast from query
Key: HIVE-24446
URL: https://issues.apache.org/jira/browse/HIVE-24446
Project: Hive
Issue Type: Bug
Reporter: Krisztian Kasa
Assignee: Krisztian Kasa
{code}
create materialized view mv_tv_view_data_av2 stored as orc TBLPROPERTIES
('transactional'='true') as
select
total_views `total_views`,
sum(cast(1.5 as decimal(9,4))) over (order by total_views) as quartile,
program
from tv_view_data;
{code}
{code}
LogicalProject(quartile=[CAST($0):DECIMAL(12, 1)], total=[$1])
HiveTableScan(table=[[arc_view, mv_tv_view_data_av1]],
table:alias=[mv_tv_view_data_av1])
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24445) Non blocking DROP table implementation
Zoltan Chovan created HIVE-24445: Summary: Non blocking DROP table implementation Key: HIVE-24445 URL: https://issues.apache.org/jira/browse/HIVE-24445 Project: Hive Issue Type: New Feature Components: Hive Reporter: Zoltan Chovan Assignee: Zoltan Chovan Implement a way to execute drop table operations in a way that doesn't have to wait for currently running read operations to be finished. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24444) compactor.Cleaner should not set state "mark cleaned" if there are obsolete files in the FS
Karen Coppage created HIVE-2: Summary: compactor.Cleaner should not set state "mark cleaned" if there are obsolete files in the FS Key: HIVE-2 URL: https://issues.apache.org/jira/browse/HIVE-2 Project: Hive Issue Type: Bug Reporter: Karen Coppage Assignee: Karen Coppage This is an improvement on HIVE-24314, in which markCleaned() is called only if +any+ files are deleted by the cleaner. This could cause a problem in the following case: Say for table_1 compaction1 cleaning was blocked by an open txn, and compaction is run again on the same table (compaction2). Both compaction1 and compaction2 could be in "ready for cleaning" at the same time. By this time the blocking open txn could be committed. When the cleaner runs, one of compaction1 and compaction2 will remain in the "ready for cleaning" state: Say compaction2 is picked up by the cleaner first. The Cleaner deletes all obsolete files. Then compaction1 is picked up by the cleaner; the cleaner doesn't remove any files and compaction1 will stay in the queue in a "ready for cleaning" state. HIVE-24291 already solves this issue but if it isn't usable (for example if HMS schema changes are out the question) then HIVE-24314 + this change will fix the issue of the Cleaner not removing all obsolete files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
