[jira] [Work logged] (HIVE-26032) Upgrade cron-utils to 9.1.6
[ https://issues.apache.org/jira/browse/HIVE-26032?focusedWorklogId=749815=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749815 ] ASF GitHub Bot logged work on HIVE-26032: - Author: ASF GitHub Bot Created on: 30/Mar/22 03:11 Start Date: 30/Mar/22 03:11 Worklog Time Spent: 10m Work Description: wangyum closed pull request #3099: URL: https://github.com/apache/hive/pull/3099 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 749815) Time Spent: 20m (was: 10m) > Upgrade cron-utils to 9.1.6 > --- > > Key: HIVE-26032 > URL: https://issues.apache.org/jira/browse/HIVE-26032 > Project: Hive > Issue Type: Task > Components: Hive >Affects Versions: 4.0.0 >Reporter: Yuming Wang >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > To fix [CVE-2021-41269|https://nvd.nist.gov/vuln/detail/CVE-2021-41269] issue. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-26032) Upgrade cron-utils to 9.1.6
[ https://issues.apache.org/jira/browse/HIVE-26032?focusedWorklogId=749816=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749816 ] ASF GitHub Bot logged work on HIVE-26032: - Author: ASF GitHub Bot Created on: 30/Mar/22 03:11 Start Date: 30/Mar/22 03:11 Worklog Time Spent: 10m Work Description: wangyum opened a new pull request #3099: URL: https://github.com/apache/hive/pull/3099 ### What changes were proposed in this pull request? Upgrade cron-utils to 9.1.6. ### Why are the changes needed? Fix [CVE-2021-41269](https://nvd.nist.gov/vuln/detail/CVE-2021-41269) issue. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Jenkins -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 749816) Time Spent: 0.5h (was: 20m) > Upgrade cron-utils to 9.1.6 > --- > > Key: HIVE-26032 > URL: https://issues.apache.org/jira/browse/HIVE-26032 > Project: Hive > Issue Type: Task > Components: Hive >Affects Versions: 4.0.0 >Reporter: Yuming Wang >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > To fix [CVE-2021-41269|https://nvd.nist.gov/vuln/detail/CVE-2021-41269] issue. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-26030) Backport HIVE-21498 to branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-26030?focusedWorklogId=749814=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749814 ] ASF GitHub Bot logged work on HIVE-26030: - Author: ASF GitHub Bot Created on: 30/Mar/22 03:10 Start Date: 30/Mar/22 03:10 Worklog Time Spent: 10m Work Description: wangyum closed pull request #3098: URL: https://github.com/apache/hive/pull/3098 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 749814) Time Spent: 40m (was: 0.5h) > Backport HIVE-21498 to branch-2.3 > - > > Key: HIVE-26030 > URL: https://issues.apache.org/jira/browse/HIVE-26030 > Project: Hive > Issue Type: Task > Components: Thrift API >Affects Versions: 2.3.9 >Reporter: Yuming Wang >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-26030) Backport HIVE-21498 to branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-26030?focusedWorklogId=749813=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749813 ] ASF GitHub Bot logged work on HIVE-26030: - Author: ASF GitHub Bot Created on: 30/Mar/22 03:10 Start Date: 30/Mar/22 03:10 Worklog Time Spent: 10m Work Description: wangyum opened a new pull request #3098: URL: https://github.com/apache/hive/pull/3098 ### What changes were proposed in this pull request? Backport HIVE-21498 to branch-2.3. ### Why are the changes needed? Address CVE-2020-13949. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Local test. ``` mvn clean install -DskipTests=true -Pthriftif -Dthrift.home=/usr/local -Pdist ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 749813) Time Spent: 0.5h (was: 20m) > Backport HIVE-21498 to branch-2.3 > - > > Key: HIVE-26030 > URL: https://issues.apache.org/jira/browse/HIVE-26030 > Project: Hive > Issue Type: Task > Components: Thrift API >Affects Versions: 2.3.9 >Reporter: Yuming Wang >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HIVE-26027) [Hive] SemanticException Invalid path with ofs/o3fs
[
https://issues.apache.org/jira/browse/HIVE-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naveen Gangam resolved HIVE-26027.
--
Resolution: Information Provided
> [Hive] SemanticException Invalid path with ofs/o3fs
> ---
>
> Key: HIVE-26027
> URL: https://issues.apache.org/jira/browse/HIVE-26027
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: All Versions
>Reporter: Soumitra Sulav
>Assignee: Naveen Gangam
>Priority: Major
>
> Import and export queries fail with ozone schemes i.e. ofs/o3fs.
> *Error stacktrace (also observed with o3fs) :*
> {code:java}
> > export table test to 'ofs://ozone1/ozonestage/hivetest/tmp/testtemptable/';
> going to print operations logs
> printed operations logs
> Getting log thread is interrupted, since query is done!
> Error: Error while compiling statement: FAILED: SemanticException Invalid
> path only the following file systems accepted for export/import :
> hdfs,pfile,file,s3,s3a,adl,abfs,abfss,gs (state=42000,code=4)
> org.apache.hive.service.cli.HiveSQLException: Error while compiling
> statement: FAILED: SemanticException Invalid path only the following file
> systems accepted for export/import : hdfs,pfile,file,s3,s3a,adl,abfs,abfss,gs
> at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:357)
> at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:343)
> at
> org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:334)
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:275)
> at org.apache.hive.beeline.Commands.executeInternal(Commands.java:1007)
> at org.apache.hive.beeline.Commands.execute(Commands.java:1217)
> at org.apache.hive.beeline.Commands.sql(Commands.java:1146)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1499)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1357)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1136)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1084)
> at
> org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:546)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:528)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> Caused by: org.apache.hive.service.cli.HiveSQLException: Error while
> compiling statement: FAILED: SemanticException Invalid path only the
> following file systems accepted for export/import :
> hdfs,pfile,file,s3,s3a,adl,abfs,abfss,gs
> at
> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:356)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:207)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:261)
> at org.apache.hive.service.cli.operation.Operation.run(Operation.java:272)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:549)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:535)
> at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:315)
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:567)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at org.apache.thrift.server.TServlet.doPost(TServlet.java:83)
> at
> org.apache.hive.service.cli.thrift.ThriftHttpServlet.doPost(ThriftHttpServlet.java:237)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
>
[jira] [Commented] (HIVE-26027) [Hive] SemanticException Invalid path with ofs/o3fs
[
https://issues.apache.org/jira/browse/HIVE-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17514280#comment-17514280
]
Naveen Gangam commented on HIVE-26027:
--
The value of supported FS prefixes comes from a hive configuration property
"_*hive.exim.uri.scheme.whitelist*_". The default value for this property is
"_hdfs,pfile,file,s3,s3a,adl,abfs,abfss,gs_". So if you need to get ofs to
work, please add this to the whitelisted configuration.
> [Hive] SemanticException Invalid path with ofs/o3fs
> ---
>
> Key: HIVE-26027
> URL: https://issues.apache.org/jira/browse/HIVE-26027
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: All Versions
>Reporter: Soumitra Sulav
>Assignee: Naveen Gangam
>Priority: Major
>
> Import and export queries fail with ozone schemes i.e. ofs/o3fs.
> *Error stacktrace (also observed with o3fs) :*
> {code:java}
> > export table test to 'ofs://ozone1/ozonestage/hivetest/tmp/testtemptable/';
> going to print operations logs
> printed operations logs
> Getting log thread is interrupted, since query is done!
> Error: Error while compiling statement: FAILED: SemanticException Invalid
> path only the following file systems accepted for export/import :
> hdfs,pfile,file,s3,s3a,adl,abfs,abfss,gs (state=42000,code=4)
> org.apache.hive.service.cli.HiveSQLException: Error while compiling
> statement: FAILED: SemanticException Invalid path only the following file
> systems accepted for export/import : hdfs,pfile,file,s3,s3a,adl,abfs,abfss,gs
> at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:357)
> at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:343)
> at
> org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:334)
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:275)
> at org.apache.hive.beeline.Commands.executeInternal(Commands.java:1007)
> at org.apache.hive.beeline.Commands.execute(Commands.java:1217)
> at org.apache.hive.beeline.Commands.sql(Commands.java:1146)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1499)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1357)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1136)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1084)
> at
> org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:546)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:528)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> Caused by: org.apache.hive.service.cli.HiveSQLException: Error while
> compiling statement: FAILED: SemanticException Invalid path only the
> following file systems accepted for export/import :
> hdfs,pfile,file,s3,s3a,adl,abfs,abfss,gs
> at
> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:356)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:207)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:261)
> at org.apache.hive.service.cli.operation.Operation.run(Operation.java:272)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:549)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:535)
> at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:315)
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:567)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at org.apache.thrift.server.TServlet.doPost(TServlet.java:83)
> at
> org.apache.hive.service.cli.thrift.ThriftHttpServlet.doPost(ThriftHttpServlet.java:237)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
> at
>
[jira] [Assigned] (HIVE-26027) [Hive] SemanticException Invalid path with ofs/o3fs
[
https://issues.apache.org/jira/browse/HIVE-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naveen Gangam reassigned HIVE-26027:
Assignee: Naveen Gangam
> [Hive] SemanticException Invalid path with ofs/o3fs
> ---
>
> Key: HIVE-26027
> URL: https://issues.apache.org/jira/browse/HIVE-26027
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: All Versions
>Reporter: Soumitra Sulav
>Assignee: Naveen Gangam
>Priority: Major
>
> Import and export queries fail with ozone schemes i.e. ofs/o3fs.
> *Error stacktrace (also observed with o3fs) :*
> {code:java}
> > export table test to 'ofs://ozone1/ozonestage/hivetest/tmp/testtemptable/';
> going to print operations logs
> printed operations logs
> Getting log thread is interrupted, since query is done!
> Error: Error while compiling statement: FAILED: SemanticException Invalid
> path only the following file systems accepted for export/import :
> hdfs,pfile,file,s3,s3a,adl,abfs,abfss,gs (state=42000,code=4)
> org.apache.hive.service.cli.HiveSQLException: Error while compiling
> statement: FAILED: SemanticException Invalid path only the following file
> systems accepted for export/import : hdfs,pfile,file,s3,s3a,adl,abfs,abfss,gs
> at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:357)
> at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:343)
> at
> org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:334)
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:275)
> at org.apache.hive.beeline.Commands.executeInternal(Commands.java:1007)
> at org.apache.hive.beeline.Commands.execute(Commands.java:1217)
> at org.apache.hive.beeline.Commands.sql(Commands.java:1146)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1499)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1357)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1136)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1084)
> at
> org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:546)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:528)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> Caused by: org.apache.hive.service.cli.HiveSQLException: Error while
> compiling statement: FAILED: SemanticException Invalid path only the
> following file systems accepted for export/import :
> hdfs,pfile,file,s3,s3a,adl,abfs,abfss,gs
> at
> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:356)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:207)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:261)
> at org.apache.hive.service.cli.operation.Operation.run(Operation.java:272)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:549)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:535)
> at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:315)
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:567)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at org.apache.thrift.server.TServlet.doPost(TServlet.java:83)
> at
> org.apache.hive.service.cli.thrift.ThriftHttpServlet.doPost(ThriftHttpServlet.java:237)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at
[jira] [Work logged] (HIVE-26086) TestIcebergCliDriver must have an object cache
[
https://issues.apache.org/jira/browse/HIVE-26086?focusedWorklogId=749313=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749313
]
ASF GitHub Bot logged work on HIVE-26086:
-
Author: ASF GitHub Bot
Created on: 29/Mar/22 13:01
Start Date: 29/Mar/22 13:01
Worklog Time Spent: 10m
Work Description: abstractdog opened a new pull request #3149:
URL: https://github.com/apache/hive/pull/3149
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 749313)
Remaining Estimate: 0h
Time Spent: 10m
> TestIcebergCliDriver must have an object cache
> --
>
> Key: HIVE-26086
> URL: https://issues.apache.org/jira/browse/HIVE-26086
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Similarly to TestMiniTezCliDriver, otherwise limit queries fail:
> {code}
> // create a new object cache for tez-based tests which rely on that
> ObjectCache.setupObjectRegistry(new ObjectRegistryImpl());
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HIVE-26086) TestIcebergCliDriver must have an object cache
[
https://issues.apache.org/jira/browse/HIVE-26086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-26086:
--
Labels: pull-request-available (was: )
> TestIcebergCliDriver must have an object cache
> --
>
> Key: HIVE-26086
> URL: https://issues.apache.org/jira/browse/HIVE-26086
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Similarly to TestMiniTezCliDriver, otherwise limit queries fail:
> {code}
> // create a new object cache for tez-based tests which rely on that
> ObjectCache.setupObjectRegistry(new ObjectRegistryImpl());
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Assigned] (HIVE-26086) TestIcebergCliDriver must have an object cache
[ https://issues.apache.org/jira/browse/HIVE-26086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned HIVE-26086: --- Assignee: László Bodor > TestIcebergCliDriver must have an object cache > -- > > Key: HIVE-26086 > URL: https://issues.apache.org/jira/browse/HIVE-26086 > Project: Hive > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HIVE-26086) TestIcebergCliDriver must have an object cache
[
https://issues.apache.org/jira/browse/HIVE-26086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-26086:
Description:
Similarly to TestMiniTezCliDriver, otherwise limit queries fail:
{code}
// create a new object cache for tez-based tests which rely on that
ObjectCache.setupObjectRegistry(new ObjectRegistryImpl());
{code}
> TestIcebergCliDriver must have an object cache
> --
>
> Key: HIVE-26086
> URL: https://issues.apache.org/jira/browse/HIVE-26086
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
>
> Similarly to TestMiniTezCliDriver, otherwise limit queries fail:
> {code}
> // create a new object cache for tez-based tests which rely on that
> ObjectCache.setupObjectRegistry(new ObjectRegistryImpl());
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-26082) Upgrade DataNucleus dependency to 5.2.8
[ https://issues.apache.org/jira/browse/HIVE-26082?focusedWorklogId=749234=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749234 ] ASF GitHub Bot logged work on HIVE-26082: - Author: ASF GitHub Bot Created on: 29/Mar/22 09:36 Start Date: 29/Mar/22 09:36 Worklog Time Spent: 10m Work Description: ashish-kumar-sharma commented on pull request #3148: URL: https://github.com/apache/hive/pull/3148#issuecomment-1081649461 @nrg4878 I haven't run any HMS microbenchmarks. I am more interested in (https://github.com/datanucleus/datanucleus-rdbms/issues/353) one of the bug which get solve as part of 5.2.5. Yes I can also run some benchmark as well as part of this activity. Could uoplease point me to some document which can help in the same? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 749234) Time Spent: 0.5h (was: 20m) > Upgrade DataNucleus dependency to 5.2.8 > --- > > Key: HIVE-26082 > URL: https://issues.apache.org/jira/browse/HIVE-26082 > Project: Hive > Issue Type: Task >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Minor > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Upgrade > datanucleus-api-jdo 5.2.4 to 5.2.8 > datanucleus-core 5.2.4 to 5.2.10 > datanucleus-rdbms 5.2.4 to 5.2.10 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-21456) Hive Metastore Thrift over HTTP
[
https://issues.apache.org/jira/browse/HIVE-21456?focusedWorklogId=749229=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749229
]
ASF GitHub Bot logged work on HIVE-21456:
-
Author: ASF GitHub Bot
Created on: 29/Mar/22 09:28
Start Date: 29/Mar/22 09:28
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #3105:
URL: https://github.com/apache/hive/pull/3105#discussion_r837259862
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HmsThriftHttpServlet.java
##
@@ -0,0 +1,116 @@
+/* * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.metastore;
+
+import java.io.IOException;
+import java.security.PrivilegedExceptionAction;
+import java.util.Enumeration;
+
+import javax.servlet.ServletException;
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+import org.apache.hadoop.hive.metastore.utils.MetaStoreUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.thrift.TProcessor;
+import org.apache.thrift.protocol.TProtocolFactory;
+import org.apache.thrift.server.TServlet;
+
+public class HmsThriftHttpServlet extends TServlet {
+
+ private static final Logger LOG = LoggerFactory
+ .getLogger(HmsThriftHttpServlet.class);
+
+ private static final String X_USER = MetaStoreUtils.USER_NAME_HTTP_HEADER;
+
+ private final boolean isSecurityEnabled;
+
+ public HmsThriftHttpServlet(TProcessor processor,
+ TProtocolFactory inProtocolFactory, TProtocolFactory outProtocolFactory)
{
+super(processor, inProtocolFactory, outProtocolFactory);
+// This should ideally be reveiving an instance of the Configuration which
is used for the check
+isSecurityEnabled = UserGroupInformation.isSecurityEnabled();
+ }
+
+ public HmsThriftHttpServlet(TProcessor processor,
+ TProtocolFactory protocolFactory) {
+super(processor, protocolFactory);
+isSecurityEnabled = UserGroupInformation.isSecurityEnabled();
+ }
+
+ @Override
+ protected void doPost(HttpServletRequest request,
+ HttpServletResponse response) throws ServletException, IOException {
+
+Enumeration headerNames = request.getHeaderNames();
+if (LOG.isDebugEnabled()) {
+ LOG.debug(" Logging headers in request");
Review comment:
nit: Extra space?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 749229)
Time Spent: 2h 50m (was: 2h 40m)
> Hive Metastore Thrift over HTTP
> ---
>
> Key: HIVE-21456
> URL: https://issues.apache.org/jira/browse/HIVE-21456
> Project: Hive
> Issue Type: New Feature
> Components: Metastore, Standalone Metastore
>Reporter: Amit Khanna
>Assignee: Sourabh Goyal
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21456.2.patch, HIVE-21456.3.patch,
> HIVE-21456.4.patch, HIVE-21456.patch
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Hive Metastore currently doesn't have support for HTTP transport because of
> which it is not possible to access it via Knox. Adding support for Thrift
> over HTTP transport will allow the clients to access via Knox
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-21456) Hive Metastore Thrift over HTTP
[
https://issues.apache.org/jira/browse/HIVE-21456?focusedWorklogId=749227=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749227
]
ASF GitHub Bot logged work on HIVE-21456:
-
Author: ASF GitHub Bot
Created on: 29/Mar/22 09:24
Start Date: 29/Mar/22 09:24
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #3105:
URL: https://github.com/apache/hive/pull/3105#discussion_r837256170
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##
@@ -343,21 +366,162 @@ public static void startMetaStore(int port,
HadoopThriftAuthBridge bridge,
startMetaStore(port, bridge, conf, false, null);
}
- /**
- * Start Metastore based on a passed {@link HadoopThriftAuthBridge}.
- *
- * @param port The port on which the Thrift server will start to serve
- * @param bridge
- * @param conf Configuration overrides
- * @param startMetaStoreThreads Start the background threads (initiator,
cleaner, statsupdater, etc.)
- * @param startedBackgroundThreads If startMetaStoreThreads is true, this
AtomicBoolean will be switched to true,
- * when all of the background threads are scheduled. Useful for testing
purposes to wait
- * until the MetaStore is fully initialized.
- * @throws Throwable
- */
- public static void startMetaStore(int port, HadoopThriftAuthBridge bridge,
- Configuration conf, boolean startMetaStoreThreads, AtomicBoolean
startedBackgroundThreads) throws Throwable {
-isMetaStoreRemote = true;
+ public static boolean isThriftServerRunning() {
+return thriftServer != null && thriftServer.isRunning();
+ }
+
+ // TODO: Is it worth trying to use a server that supports HTTP/2?
+ // Does the Thrift http client support this?
+
+ public static ThriftServer startHttpMetastore(int port, Configuration conf)
+ throws Exception {
+LOG.info("Attempting to start http metastore server on port: {}", port);
+
+// This check is likely pointless, especially with the current state of
the http
+// servlet which respects whatever comes in. Putting this in place for the
moment
+// only to enable testing on an otherwise secure cluster.
+LOG.info(" Checking if security is enabled");
+if (UserGroupInformation.isSecurityEnabled()) {
+ LOG.info("Logging in via keytab while starting HTTP metastore");
+ // Handle renewal
+ String kerberosName =
SecurityUtil.getServerPrincipal(MetastoreConf.getVar(conf,
ConfVars.KERBEROS_PRINCIPAL), "0.0.0.0");
+ String keyTabFile = MetastoreConf.getVar(conf,
ConfVars.KERBEROS_KEYTAB_FILE);
+ UserGroupInformation.loginUserFromKeytab(kerberosName, keyTabFile);
+} else {
+ LOG.info("Security is not enabled. Not logging in via keytab");
+}
+
+// TODO Bunch of http specific variables need to be defined here.
+long maxMessageSize = MetastoreConf.getLongVar(conf,
ConfVars.SERVER_MAX_MESSAGE_SIZE);
+int minWorkerThreads = MetastoreConf.getIntVar(conf,
ConfVars.SERVER_MIN_THREADS);
+int maxWorkerThreads = MetastoreConf.getIntVar(conf,
ConfVars.SERVER_MAX_THREADS);
+
+boolean useCompactProtocol = MetastoreConf.getBoolVar(conf,
ConfVars.USE_THRIFT_COMPACT_PROTOCOL);
+
+// Server thread pool
+// Start with minWorkerThreads, expand till maxWorkerThreads and reject
+// subsequent requests
+String threadPoolName = "HiveServer2-HttpHandler-Pool";
+ExecutorService executorService = new ThreadPoolExecutor(
+minWorkerThreads, maxWorkerThreads, 60, TimeUnit.SECONDS,
+new SynchronousQueue<>());
+
+ExecutorThreadPool threadPool = new
ExecutorThreadPool((ThreadPoolExecutor) executorService);
+
+// HTTP Server
+org.eclipse.jetty.server.Server server = new Server(threadPool);
+server.setStopAtShutdown(true);
+
+ServerConnector connector;
+
+final HttpConfiguration httpServerConf = new HttpConfiguration();
+// TODO: Read from Configuration
+httpServerConf.setRequestHeaderSize(
+MetastoreConf.getIntVar(conf,
ConfVars.METASTORE_THRIFT_HTTP_REQUEST_HEADER_SIZE));
+httpServerConf.setResponseHeaderSize(
+MetastoreConf.getIntVar(conf,
ConfVars.METASTORE_THRIFT_HTTP_RESPONSE_HEADER_SIZE));
+
+final HttpConnectionFactory http = new
HttpConnectionFactory(httpServerConf);
+
+boolean useSsl = MetastoreConf.getBoolVar(conf, ConfVars.USE_SSL);
+String schemeName = useSsl ? "https" : "http";
+if (useSsl) {
+ String keyStorePath = MetastoreConf.getVar(conf,
ConfVars.SSL_KEYSTORE_PATH).trim();
+ if (keyStorePath.isEmpty()) {
+throw new
IllegalArgumentException(ConfVars.SSL_KEYSTORE_PATH.toString()
++ " Not configured for SSL connection");
+ }
+ String keyStorePassword =
+ MetastoreConf.getPassword(conf,
[jira] [Commented] (HIVE-26075) hive metastore connection leaking when hiveserver2 kerberos enable and hive.server2.enable.doAs set to true
[ https://issues.apache.org/jira/browse/HIVE-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17513950#comment-17513950 ] Stamatis Zampetakis commented on HIVE-26075: If the problem appears on master then definitely raise a PR. We don't actively maintain older branches so it might not be easy to get this in there. > hive metastore connection leaking when hiveserver2 kerberos enable and > hive.server2.enable.doAs set to true > > > Key: HIVE-26075 > URL: https://issues.apache.org/jira/browse/HIVE-26075 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: All Versions >Reporter: liuguanghua >Assignee: liuguanghua >Priority: Major > Attachments: HIVE-26075.patch > > > (1)When hadoop cluster kerberos is enable > (2) HiveServer2 config hive.server2.enable.doAs is set true > After a beeline scripte has been executed, hivemetastore connection is > created are in ESTABLISHED state and never closed. > If we submit a lot of task to hiveserver2 ,this will result in hive metastore > thrift thread(default is 1000) full ,thus new task will fail. > > HiveServer2 use ThreadLocal to store multithreading metastore > connection,the application should call Hive.closeCurrent() to close > connection after task finished. > > When HiveServer2 impersonate is enable (hive.server2.enable.doAs is set > true), the ugi will create proxy user via > UserGroupInformation.createProxyUser( > owner, UserGroupInformation.getLoginUser()),the old metastore client is never > closed. > > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-21456) Hive Metastore Thrift over HTTP
[
https://issues.apache.org/jira/browse/HIVE-21456?focusedWorklogId=749223=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749223
]
ASF GitHub Bot logged work on HIVE-21456:
-
Author: ASF GitHub Bot
Created on: 29/Mar/22 09:21
Start Date: 29/Mar/22 09:21
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #3105:
URL: https://github.com/apache/hive/pull/3105#discussion_r837253159
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##
@@ -343,21 +366,162 @@ public static void startMetaStore(int port,
HadoopThriftAuthBridge bridge,
startMetaStore(port, bridge, conf, false, null);
}
- /**
- * Start Metastore based on a passed {@link HadoopThriftAuthBridge}.
- *
- * @param port The port on which the Thrift server will start to serve
- * @param bridge
- * @param conf Configuration overrides
- * @param startMetaStoreThreads Start the background threads (initiator,
cleaner, statsupdater, etc.)
- * @param startedBackgroundThreads If startMetaStoreThreads is true, this
AtomicBoolean will be switched to true,
- * when all of the background threads are scheduled. Useful for testing
purposes to wait
- * until the MetaStore is fully initialized.
- * @throws Throwable
- */
- public static void startMetaStore(int port, HadoopThriftAuthBridge bridge,
- Configuration conf, boolean startMetaStoreThreads, AtomicBoolean
startedBackgroundThreads) throws Throwable {
-isMetaStoreRemote = true;
+ public static boolean isThriftServerRunning() {
+return thriftServer != null && thriftServer.isRunning();
+ }
+
+ // TODO: Is it worth trying to use a server that supports HTTP/2?
+ // Does the Thrift http client support this?
+
+ public static ThriftServer startHttpMetastore(int port, Configuration conf)
+ throws Exception {
+LOG.info("Attempting to start http metastore server on port: {}", port);
Review comment:
Do we disable the TRACE request for our server?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 749223)
Time Spent: 2.5h (was: 2h 20m)
> Hive Metastore Thrift over HTTP
> ---
>
> Key: HIVE-21456
> URL: https://issues.apache.org/jira/browse/HIVE-21456
> Project: Hive
> Issue Type: New Feature
> Components: Metastore, Standalone Metastore
>Reporter: Amit Khanna
>Assignee: Sourabh Goyal
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21456.2.patch, HIVE-21456.3.patch,
> HIVE-21456.4.patch, HIVE-21456.patch
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Hive Metastore currently doesn't have support for HTTP transport because of
> which it is not possible to access it via Knox. Adding support for Thrift
> over HTTP transport will allow the clients to access via Knox
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-21456) Hive Metastore Thrift over HTTP
[
https://issues.apache.org/jira/browse/HIVE-21456?focusedWorklogId=749221=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749221
]
ASF GitHub Bot logged work on HIVE-21456:
-
Author: ASF GitHub Bot
Created on: 29/Mar/22 09:18
Start Date: 29/Mar/22 09:18
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #3105:
URL: https://github.com/apache/hive/pull/3105#discussion_r837249852
##
File path:
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
##
@@ -1356,6 +1356,13 @@ public static ConfVars getMetaConf(String name) {
"Comma-separated list of tasks that will be started in separate
threads. These will be" +
" started only when the metastore is running as a separate
service. They must " +
"implement " + METASTORE_TASK_THREAD_CLASS),
+THRIFT_TRANSPORT_MODE("metastore.server.thrift.transport.mode",
Review comment:
We should consider which of these should kept secret, and which of these
could be changed on session level
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 749221)
Time Spent: 2h 20m (was: 2h 10m)
> Hive Metastore Thrift over HTTP
> ---
>
> Key: HIVE-21456
> URL: https://issues.apache.org/jira/browse/HIVE-21456
> Project: Hive
> Issue Type: New Feature
> Components: Metastore, Standalone Metastore
>Reporter: Amit Khanna
>Assignee: Sourabh Goyal
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21456.2.patch, HIVE-21456.3.patch,
> HIVE-21456.4.patch, HIVE-21456.patch
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Hive Metastore currently doesn't have support for HTTP transport because of
> which it is not possible to access it via Knox. Adding support for Thrift
> over HTTP transport will allow the clients to access via Knox
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-21456) Hive Metastore Thrift over HTTP
[
https://issues.apache.org/jira/browse/HIVE-21456?focusedWorklogId=749219=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749219
]
ASF GitHub Bot logged work on HIVE-21456:
-
Author: ASF GitHub Bot
Created on: 29/Mar/22 09:14
Start Date: 29/Mar/22 09:14
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #3105:
URL: https://github.com/apache/hive/pull/3105#discussion_r837245803
##
File path:
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##
@@ -803,6 +765,114 @@ private void open() throws MetaException {
snapshotActiveConf();
}
+ private String[] getHttpUrlAndUser(URI store) throws MetaException {
+boolean useSSL = MetastoreConf.getBoolVar(conf, ConfVars.USE_SSL);
+String path = MetaStoreUtils.getHttpPath(MetastoreConf.getVar(conf,
ConfVars.THRIFT_HTTP_PATH));
+String httpUrl = (useSSL ? "https://; : "http://;) + store.getHost() + ":"
+ store.getPort() + path;
+String httpUser = MetastoreConf.getVar(conf,
ConfVars.METASTORE_CLIENT_PLAIN_USERNAME);
Review comment:
we probably want to make sure that this config can only be changed by an
admin
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 749219)
Time Spent: 2h 10m (was: 2h)
> Hive Metastore Thrift over HTTP
> ---
>
> Key: HIVE-21456
> URL: https://issues.apache.org/jira/browse/HIVE-21456
> Project: Hive
> Issue Type: New Feature
> Components: Metastore, Standalone Metastore
>Reporter: Amit Khanna
>Assignee: Sourabh Goyal
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21456.2.patch, HIVE-21456.3.patch,
> HIVE-21456.4.patch, HIVE-21456.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Hive Metastore currently doesn't have support for HTTP transport because of
> which it is not possible to access it via Knox. Adding support for Thrift
> over HTTP transport will allow the clients to access via Knox
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-26059) Eventually clean compactions in "refused" state from compaction history
[ https://issues.apache.org/jira/browse/HIVE-26059?focusedWorklogId=749217=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749217 ] ASF GitHub Bot logged work on HIVE-26059: - Author: ASF GitHub Bot Created on: 29/Mar/22 09:11 Start Date: 29/Mar/22 09:11 Worklog Time Spent: 10m Work Description: deniskuzZ merged pull request #3127: URL: https://github.com/apache/hive/pull/3127 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 749217) Time Spent: 1h 20m (was: 1h 10m) > Eventually clean compactions in "refused" state from compaction history > --- > > Key: HIVE-26059 > URL: https://issues.apache.org/jira/browse/HIVE-26059 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: László Végh >Assignee: László Végh >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > Compactions in states succeeded, failed, and did not initiate have a > retention threshold (example: > metastore.compactor.history.retention.succeeded) and are purged from > COMPLETED_COMPACTIONS if the number of compactions in this state per > partition/unpartitioned table passes the threshold. This keeps the size of > COMPLETED_COMPACTIONS in check. > We should also purge refused compactions from COMPLETED_COMPACTIONS. > See: > CompactionTxnHandler#purgeCompactionHistory > ! Also: REFUSED_RESPONSE should be added to > org.apache.hadoop.hive.metastore.txn.TxnStore#COMPACTION_STATES so that > metrics will be collected about it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-26082) Upgrade DataNucleus dependency to 5.2.8
[ https://issues.apache.org/jira/browse/HIVE-26082?focusedWorklogId=749216=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749216 ] ASF GitHub Bot logged work on HIVE-26082: - Author: ASF GitHub Bot Created on: 29/Mar/22 09:08 Start Date: 29/Mar/22 09:08 Worklog Time Spent: 10m Work Description: nrg4878 commented on pull request #3148: URL: https://github.com/apache/hive/pull/3148#issuecomment-1081621160 @ashish-kumar-sharma Have you run any HMS microbenchmarks to see if there is any performance impact with the new JDO binaries? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 749216) Time Spent: 20m (was: 10m) > Upgrade DataNucleus dependency to 5.2.8 > --- > > Key: HIVE-26082 > URL: https://issues.apache.org/jira/browse/HIVE-26082 > Project: Hive > Issue Type: Task >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade > datanucleus-api-jdo 5.2.4 to 5.2.8 > datanucleus-core 5.2.4 to 5.2.10 > datanucleus-rdbms 5.2.4 to 5.2.10 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (HIVE-26077) Implement CTAS for Iceberg tables with partition spec
[
https://issues.apache.org/jira/browse/HIVE-26077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17513940#comment-17513940
]
Peter Vary edited comment on HIVE-26077 at 3/29/22, 9:01 AM:
-
Pushed to master.
Thanks for the review [~Marton Bod]!
was (Author: pvary):
Pushed to master.
Thanks for the review [~Marton Bod]
> Implement CTAS for Iceberg tables with partition spec
> -
>
> Key: HIVE-26077
> URL: https://issues.apache.org/jira/browse/HIVE-26077
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> We enabled CTAS for Iceberg tables, but currently only for unpartitioned
> tables and tables where the partitioning schema uses the original HIVE
> {{PARTITIONED BY}} syntax.
> We would like to use CTAS with the new syntax, like:
> {code}
> CREATE TABLE t PARTITIONED BY SPEC(day(ts)) AS SELECT ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-26077) Implement CTAS for Iceberg tables with partition spec
[
https://issues.apache.org/jira/browse/HIVE-26077?focusedWorklogId=749213=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749213
]
ASF GitHub Bot logged work on HIVE-26077:
-
Author: ASF GitHub Bot
Created on: 29/Mar/22 09:01
Start Date: 29/Mar/22 09:01
Worklog Time Spent: 10m
Work Description: pvary merged pull request #3147:
URL: https://github.com/apache/hive/pull/3147
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 749213)
Time Spent: 40m (was: 0.5h)
> Implement CTAS for Iceberg tables with partition spec
> -
>
> Key: HIVE-26077
> URL: https://issues.apache.org/jira/browse/HIVE-26077
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> We enabled CTAS for Iceberg tables, but currently only for unpartitioned
> tables and tables where the partitioning schema uses the original HIVE
> {{PARTITIONED BY}} syntax.
> We would like to use CTAS with the new syntax, like:
> {code}
> CREATE TABLE t PARTITIONED BY SPEC(day(ts)) AS SELECT ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (HIVE-26077) Implement CTAS for Iceberg tables with partition spec
[
https://issues.apache.org/jira/browse/HIVE-26077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary resolved HIVE-26077.
---
Fix Version/s: 4.0.0
Resolution: Fixed
Pushed to master.
Thanks for the review [~Marton Bod]
> Implement CTAS for Iceberg tables with partition spec
> -
>
> Key: HIVE-26077
> URL: https://issues.apache.org/jira/browse/HIVE-26077
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> We enabled CTAS for Iceberg tables, but currently only for unpartitioned
> tables and tables where the partitioning schema uses the original HIVE
> {{PARTITIONED BY}} syntax.
> We would like to use CTAS with the new syntax, like:
> {code}
> CREATE TABLE t PARTITIONED BY SPEC(day(ts)) AS SELECT ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-26077) Implement CTAS for Iceberg tables with partition spec
[
https://issues.apache.org/jira/browse/HIVE-26077?focusedWorklogId=749212=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749212
]
ASF GitHub Bot logged work on HIVE-26077:
-
Author: ASF GitHub Bot
Created on: 29/Mar/22 09:01
Start Date: 29/Mar/22 09:01
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #3147:
URL: https://github.com/apache/hive/pull/3147#discussion_r837233568
##
File path:
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/HiveIcebergTestUtils.java
##
@@ -299,12 +300,14 @@ public static void validateDataWithSQL(TestHiveShell
shell, String tableName, Li
Assert.assertEquals(record.size(), row.length);
for (int j = 0; j < record.size(); ++j) {
Object field = record.get(j);
-if (field instanceof LocalDateTime) {
+if (field == null) {
+ Assert.assertNull(row[j]);
+} else if (field instanceof LocalDateTime) {
Assert.assertEquals(((LocalDateTime)
field).toInstant(ZoneOffset.UTC).toEpochMilli(),
TimestampUtils.stringToTimestamp((String)
row[j]).toEpochMilli());
} else if (field instanceof OffsetDateTime) {
Assert.assertEquals(((OffsetDateTime)
field).toInstant().toEpochMilli(),
- TimestampTZUtil.parse((String) row[j]).toEpochMilli());
+ TimestampTZUtil.parse((String) row[j],
ZoneId.systemDefault()).toEpochMilli());
Review comment:
Yeah. Without that, if the string does not contain the TZ, then
`TimestampTZUtil.parse` does not know which TZ it should use in parsing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 749212)
Time Spent: 0.5h (was: 20m)
> Implement CTAS for Iceberg tables with partition spec
> -
>
> Key: HIVE-26077
> URL: https://issues.apache.org/jira/browse/HIVE-26077
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> We enabled CTAS for Iceberg tables, but currently only for unpartitioned
> tables and tables where the partitioning schema uses the original HIVE
> {{PARTITIONED BY}} syntax.
> We would like to use CTAS with the new syntax, like:
> {code}
> CREATE TABLE t PARTITIONED BY SPEC(day(ts)) AS SELECT ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Assigned] (HIVE-26084) Oracle metastore init tests are flaky
[
https://issues.apache.org/jira/browse/HIVE-26084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary reassigned HIVE-26084:
-
> Oracle metastore init tests are flaky
> -
>
> Key: HIVE-26084
> URL: https://issues.apache.org/jira/browse/HIVE-26084
> Project: Hive
> Issue Type: Test
>Reporter: Peter Vary
>Assignee: Stamatis Zampetakis
>Priority: Major
>
> After HIVE-26022 we started to run the oracle metastore init tests, but they
> seem to be flaky.
> I see this issue quite often:
> http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3147/1/pipeline/551
> We might have to increase the timeout, or use another oracle image for more
> consistent tests.
> The error in the logs for future reference
> {code}
> [2022-03-28T14:10:07.804Z] + echo 127.0.0.1 dev_oracle
> [2022-03-28T14:10:07.804Z] + sudo tee -a /etc/hosts
> [2022-03-28T14:10:07.804Z] 127.0.0.1 dev_oracle
> [2022-03-28T14:10:07.804Z] + . /etc/profile.d/confs.sh
> [2022-03-28T14:10:07.804Z] ++ export MAVEN_OPTS=-Xmx2g
> [2022-03-28T14:10:07.804Z] ++ MAVEN_OPTS=-Xmx2g
> [2022-03-28T14:10:07.804Z] ++ export HADOOP_CONF_DIR=/etc/hadoop
> [2022-03-28T14:10:07.804Z] ++ HADOOP_CONF_DIR=/etc/hadoop
> [2022-03-28T14:10:07.804Z] ++ export HADOOP_LOG_DIR=/data/log
> [2022-03-28T14:10:07.804Z] ++ HADOOP_LOG_DIR=/data/log
> [2022-03-28T14:10:07.804Z] ++ export
> 'HADOOP_CLASSPATH=/etc/tez/:/active/tez/lib/*:/active/tez/*:/apps/lib/*'
> [2022-03-28T14:10:07.804Z] ++
> HADOOP_CLASSPATH='/etc/tez/:/active/tez/lib/*:/active/tez/*:/apps/lib/*'
> [2022-03-28T14:10:07.804Z] ++ export HIVE_CONF_DIR=/etc/hive/
> [2022-03-28T14:10:07.804Z] ++ HIVE_CONF_DIR=/etc/hive/
> [2022-03-28T14:10:07.804Z] ++ export
> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/active/hive/bin:/active/hadoop/bin:/active/eclipse/:/active/maven/bin/:/active/protobuf/bin:/active/visualvm/bin:/active/kubebuilder/bin:/active/idea/bin
> [2022-03-28T14:10:07.804Z] ++
> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/active/hive/bin:/active/hadoop/bin:/active/eclipse/:/active/maven/bin/:/active/protobuf/bin:/active/visualvm/bin:/active/kubebuilder/bin:/active/idea/bin
> [2022-03-28T14:10:07.804Z] ++ . /etc/profile.d/java.sh
> [2022-03-28T14:10:07.804Z] +++ export JAVA_HOME=/usr/lib/jvm/zulu-8-amd64/
> [2022-03-28T14:10:07.804Z] +++ JAVA_HOME=/usr/lib/jvm/zulu-8-amd64/
> [2022-03-28T14:10:07.804Z] + sw hive-dev
> /home/jenkins/agent/workspace/hive-precommit_PR-3147
> [2022-03-28T14:10:07.804Z] @ activating:
> /home/jenkins/agent/workspace/hive-precommit_PR-3147/packaging/target/apache-hive-4.0.0-alpha-1-SNAPSHOT-bin/apache-hive-4.0.0-alpha-1-SNAPSHOT-bin/
> for hive
> [2022-03-28T14:10:07.804Z] + ping -c2 dev_oracle
> [2022-03-28T14:10:07.804Z] PING dev_oracle (127.0.0.1) 56(84) bytes of data.
> [2022-03-28T14:10:07.804Z] 64 bytes from localhost (127.0.0.1): icmp_seq=1
> ttl=64 time=0.082 ms
> [2022-03-28T14:10:08.795Z] 64 bytes from localhost (127.0.0.1): icmp_seq=2
> ttl=64 time=0.087 ms
> [2022-03-28T14:10:08.795Z]
> [2022-03-28T14:10:08.795Z] --- dev_oracle ping statistics ---
> [2022-03-28T14:10:08.795Z] 2 packets transmitted, 2 received, 0% packet loss,
> time 51ms
> [2022-03-28T14:10:08.795Z] rtt min/avg/max/mdev = 0.082/0.084/0.087/0.009 ms
> [2022-03-28T14:10:08.795Z] + export DOCKER_NETWORK=host
> [2022-03-28T14:10:08.795Z] + DOCKER_NETWORK=host
> [2022-03-28T14:10:08.795Z] + export DBNAME=metastore
> [2022-03-28T14:10:08.795Z] + DBNAME=metastore
> [2022-03-28T14:10:08.795Z] + reinit_metastore oracle
> [2022-03-28T14:10:08.795Z] @ initializing: oracle
> [2022-03-28T14:10:08.795Z] metastore database name: metastore
> [2022-03-28T14:10:09.135Z] @ starting dev_oracle...
> [2022-03-28T14:10:09.445Z] Unable to find image
> 'quay.io/maksymbilenko/oracle-12c:latest' locally
> [2022-03-28T14:10:10.407Z] latest: Pulling from maksymbilenko/oracle-12c
> [2022-03-28T14:10:10.407Z] 8ba884070f61: Pulling fs layer
> [2022-03-28T14:10:10.407Z] ef9513b81046: Pulling fs layer
> [2022-03-28T14:10:10.407Z] 6f1de349e202: Pulling fs layer
> [2022-03-28T14:10:10.407Z] 5376ebfa0fa3: Pulling fs layer
> [2022-03-28T14:10:10.407Z] 5f632c3633d2: Pulling fs layer
> [2022-03-28T14:10:10.407Z] 3e74293031d2: Pulling fs layer
> [2022-03-28T14:10:10.407Z] 5376ebfa0fa3: Waiting
> [2022-03-28T14:10:10.407Z] 5f632c3633d2: Waiting
> [2022-03-28T14:10:10.407Z] 3e74293031d2: Waiting
> [2022-03-28T14:10:10.407Z] 6f1de349e202: Download complete
> [2022-03-28T14:10:11.365Z] ef9513b81046: Download complete
> [2022-03-28T14:10:11.365Z] 5f632c3633d2: Verifying Checksum
> [2022-03-28T14:10:11.365Z] 5f632c3633d2: Download complete
> [2022-03-28T14:10:11.365Z] 8ba884070f61: Verifying Checksum
> [2022-03-28T14:10:11.365Z] 8ba884070f61: Download complete
>
[jira] [Work logged] (HIVE-26077) Implement CTAS for Iceberg tables with partition spec
[
https://issues.apache.org/jira/browse/HIVE-26077?focusedWorklogId=749196=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749196
]
ASF GitHub Bot logged work on HIVE-26077:
-
Author: ASF GitHub Bot
Created on: 29/Mar/22 08:48
Start Date: 29/Mar/22 08:48
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #3147:
URL: https://github.com/apache/hive/pull/3147#discussion_r837220839
##
File path:
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/HiveIcebergTestUtils.java
##
@@ -299,12 +300,14 @@ public static void validateDataWithSQL(TestHiveShell
shell, String tableName, Li
Assert.assertEquals(record.size(), row.length);
for (int j = 0; j < record.size(); ++j) {
Object field = record.get(j);
-if (field instanceof LocalDateTime) {
+if (field == null) {
+ Assert.assertNull(row[j]);
+} else if (field instanceof LocalDateTime) {
Assert.assertEquals(((LocalDateTime)
field).toInstant(ZoneOffset.UTC).toEpochMilli(),
TimestampUtils.stringToTimestamp((String)
row[j]).toEpochMilli());
} else if (field instanceof OffsetDateTime) {
Assert.assertEquals(((OffsetDateTime)
field).toInstant().toEpochMilli(),
- TimestampTZUtil.parse((String) row[j]).toEpochMilli());
+ TimestampTZUtil.parse((String) row[j],
ZoneId.systemDefault()).toEpochMilli());
Review comment:
Did you get a failure without specifying the the zone?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 749196)
Time Spent: 20m (was: 10m)
> Implement CTAS for Iceberg tables with partition spec
> -
>
> Key: HIVE-26077
> URL: https://issues.apache.org/jira/browse/HIVE-26077
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We enabled CTAS for Iceberg tables, but currently only for unpartitioned
> tables and tables where the partitioning schema uses the original HIVE
> {{PARTITIONED BY}} syntax.
> We would like to use CTAS with the new syntax, like:
> {code}
> CREATE TABLE t PARTITIONED BY SPEC(day(ts)) AS SELECT ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-26081) Upgrade ant to 1.10.9
[ https://issues.apache.org/jira/browse/HIVE-26081?focusedWorklogId=749188=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749188 ] ASF GitHub Bot logged work on HIVE-26081: - Author: ASF GitHub Bot Created on: 29/Mar/22 08:25 Start Date: 29/Mar/22 08:25 Worklog Time Spent: 10m Work Description: sankarh merged pull request #3146: URL: https://github.com/apache/hive/pull/3146 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 749188) Time Spent: 20m (was: 10m) > Upgrade ant to 1.10.9 > - > > Key: HIVE-26081 > URL: https://issues.apache.org/jira/browse/HIVE-26081 > Project: Hive > Issue Type: Task >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade org.apache.ant:ant from 1.9.1 to 1.10.9 to fix the vulnerability > CVE-2020-11979 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HIVE-26081) Upgrade ant to 1.10.9
[ https://issues.apache.org/jira/browse/HIVE-26081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan resolved HIVE-26081. - Fix Version/s: 4.0.0 Resolution: Fixed > Upgrade ant to 1.10.9 > - > > Key: HIVE-26081 > URL: https://issues.apache.org/jira/browse/HIVE-26081 > Project: Hive > Issue Type: Task >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade org.apache.ant:ant from 1.9.1 to 1.10.9 to fix the vulnerability > CVE-2020-11979 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HIVE-26078) Upgrade gson to 2.8.9
[ https://issues.apache.org/jira/browse/HIVE-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan resolved HIVE-26078. - Resolution: Fixed > Upgrade gson to 2.8.9 > - > > Key: HIVE-26078 > URL: https://issues.apache.org/jira/browse/HIVE-26078 > Project: Hive > Issue Type: Task >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade to version com.google.code.gson:gson:2.8.9 to avoid WS-2021-0419 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HIVE-26078) Upgrade gson to 2.8.9
[ https://issues.apache.org/jira/browse/HIVE-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-26078: Fix Version/s: 4.0.0 > Upgrade gson to 2.8.9 > - > > Key: HIVE-26078 > URL: https://issues.apache.org/jira/browse/HIVE-26078 > Project: Hive > Issue Type: Task >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade to version com.google.code.gson:gson:2.8.9 to avoid WS-2021-0419 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-26078) Upgrade gson to 2.8.9
[ https://issues.apache.org/jira/browse/HIVE-26078?focusedWorklogId=749185=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749185 ] ASF GitHub Bot logged work on HIVE-26078: - Author: ASF GitHub Bot Created on: 29/Mar/22 08:22 Start Date: 29/Mar/22 08:22 Worklog Time Spent: 10m Work Description: sankarh merged pull request #3143: URL: https://github.com/apache/hive/pull/3143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 749185) Time Spent: 20m (was: 10m) > Upgrade gson to 2.8.9 > - > > Key: HIVE-26078 > URL: https://issues.apache.org/jira/browse/HIVE-26078 > Project: Hive > Issue Type: Task >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade to version com.google.code.gson:gson:2.8.9 to avoid WS-2021-0419 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HIVE-25492) Major query-based compaction is skipped if partition is empty
[ https://issues.apache.org/jira/browse/HIVE-25492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Sinkovits reassigned HIVE-25492: -- Assignee: Antal Sinkovits > Major query-based compaction is skipped if partition is empty > - > > Key: HIVE-25492 > URL: https://issues.apache.org/jira/browse/HIVE-25492 > Project: Hive > Issue Type: Bug >Reporter: Karen Coppage >Assignee: Antal Sinkovits >Priority: Major > > Currently if the result of query-based compaction is an empty base, delta, or > delete delta, the empty directory is deleted. > This is because of minor compaction – if there are only deltas to compact, > then no compacted delete delta should be created (only a compacted delta). In > the same way, if there are only delete deltas to compact, then no compacted > delta should be created (only a compacted delete delta). > There is an issue with major compaction. If all the data in the partition has > been deleted, then we should get an empty base directory after compaction. > Instead, the empty base directory is deleted because it's empty and > compaction claims to succeed but we end up with the same deltas/delete deltas > we started with – basically compaction does not run. > Where to start? MajorQueryCompactor#commitCompaction -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work started] (HIVE-25492) Major query-based compaction is skipped if partition is empty
[ https://issues.apache.org/jira/browse/HIVE-25492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-25492 started by Antal Sinkovits. -- > Major query-based compaction is skipped if partition is empty > - > > Key: HIVE-25492 > URL: https://issues.apache.org/jira/browse/HIVE-25492 > Project: Hive > Issue Type: Bug >Reporter: Karen Coppage >Assignee: Antal Sinkovits >Priority: Major > > Currently if the result of query-based compaction is an empty base, delta, or > delete delta, the empty directory is deleted. > This is because of minor compaction – if there are only deltas to compact, > then no compacted delete delta should be created (only a compacted delta). In > the same way, if there are only delete deltas to compact, then no compacted > delta should be created (only a compacted delete delta). > There is an issue with major compaction. If all the data in the partition has > been deleted, then we should get an empty base directory after compaction. > Instead, the empty base directory is deleted because it's empty and > compaction claims to succeed but we end up with the same deltas/delete deltas > we started with – basically compaction does not run. > Where to start? MajorQueryCompactor#commitCompaction -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HIVE-26083) Fix flaky TestParseDriver.testExoticSJSSubQuery
[ https://issues.apache.org/jira/browse/HIVE-26083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17513862#comment-17513862 ] Peter Vary commented on HIVE-26083: --- CC: [~kgyrtkirk] as he might know more about the test, and why it is timing out > Fix flaky TestParseDriver.testExoticSJSSubQuery > --- > > Key: HIVE-26083 > URL: https://issues.apache.org/jira/browse/HIVE-26083 > Project: Hive > Issue Type: Task >Reporter: Peter Vary >Priority: Major > > The test is failing from time to time. > http://ci.hive.apache.org/job/hive-flaky-check/550 > http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3132/6/tests -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-26082) Upgrade DataNucleus dependency to 5.2.8
[ https://issues.apache.org/jira/browse/HIVE-26082?focusedWorklogId=749135=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749135 ] ASF GitHub Bot logged work on HIVE-26082: - Author: ASF GitHub Bot Created on: 29/Mar/22 06:36 Start Date: 29/Mar/22 06:36 Worklog Time Spent: 10m Work Description: ashish-kumar-sharma opened a new pull request #3148: URL: https://github.com/apache/hive/pull/3148 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 749135) Remaining Estimate: 0h Time Spent: 10m > Upgrade DataNucleus dependency to 5.2.8 > --- > > Key: HIVE-26082 > URL: https://issues.apache.org/jira/browse/HIVE-26082 > Project: Hive > Issue Type: Task >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade > datanucleus-api-jdo 5.2.4 to 5.2.8 > datanucleus-core 5.2.4 to 5.2.10 > datanucleus-rdbms 5.2.4 to 5.2.10 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HIVE-26082) Upgrade DataNucleus dependency to 5.2.8
[ https://issues.apache.org/jira/browse/HIVE-26082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-26082: -- Labels: pull-request-available (was: ) > Upgrade DataNucleus dependency to 5.2.8 > --- > > Key: HIVE-26082 > URL: https://issues.apache.org/jira/browse/HIVE-26082 > Project: Hive > Issue Type: Task >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade > datanucleus-api-jdo 5.2.4 to 5.2.8 > datanucleus-core 5.2.4 to 5.2.10 > datanucleus-rdbms 5.2.4 to 5.2.10 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HIVE-26075) hive metastore connection leaking when hiveserver2 kerberos enable and hive.server2.enable.doAs set to true
[ https://issues.apache.org/jira/browse/HIVE-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17513842#comment-17513842 ] liuguanghua commented on HIVE-26075: Thank you very much.I will pull requests on GitHub > hive metastore connection leaking when hiveserver2 kerberos enable and > hive.server2.enable.doAs set to true > > > Key: HIVE-26075 > URL: https://issues.apache.org/jira/browse/HIVE-26075 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: All Versions >Reporter: liuguanghua >Assignee: liuguanghua >Priority: Major > Attachments: HIVE-26075.patch > > > (1)When hadoop cluster kerberos is enable > (2) HiveServer2 config hive.server2.enable.doAs is set true > After a beeline scripte has been executed, hivemetastore connection is > created are in ESTABLISHED state and never closed. > If we submit a lot of task to hiveserver2 ,this will result in hive metastore > thrift thread(default is 1000) full ,thus new task will fail. > > HiveServer2 use ThreadLocal to store multithreading metastore > connection,the application should call Hive.closeCurrent() to close > connection after task finished. > > When HiveServer2 impersonate is enable (hive.server2.enable.doAs is set > true), the ugi will create proxy user via > UserGroupInformation.createProxyUser( > owner, UserGroupInformation.getLoginUser()),the old metastore client is never > closed. > > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HIVE-26075) hive metastore connection leaking when hiveserver2 kerberos enable and hive.server2.enable.doAs set to true
[ https://issues.apache.org/jira/browse/HIVE-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuguanghua updated HIVE-26075: --- Affects Version/s: All Versions > hive metastore connection leaking when hiveserver2 kerberos enable and > hive.server2.enable.doAs set to true > > > Key: HIVE-26075 > URL: https://issues.apache.org/jira/browse/HIVE-26075 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: All Versions >Reporter: liuguanghua >Assignee: liuguanghua >Priority: Major > Attachments: HIVE-26075.patch > > > (1)When hadoop cluster kerberos is enable > (2) HiveServer2 config hive.server2.enable.doAs is set true > After a beeline scripte has been executed, hivemetastore connection is > created are in ESTABLISHED state and never closed. > If we submit a lot of task to hiveserver2 ,this will result in hive metastore > thrift thread(default is 1000) full ,thus new task will fail. > > HiveServer2 use ThreadLocal to store multithreading metastore > connection,the application should call Hive.closeCurrent() to close > connection after task finished. > > When HiveServer2 impersonate is enable (hive.server2.enable.doAs is set > true), the ugi will create proxy user via > UserGroupInformation.createProxyUser( > owner, UserGroupInformation.getLoginUser()),the old metastore client is never > closed. > > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work started] (HIVE-26082) Upgrade DataNucleus dependency to 5.2.8
[ https://issues.apache.org/jira/browse/HIVE-26082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-26082 started by Ashish Sharma. > Upgrade DataNucleus dependency to 5.2.8 > --- > > Key: HIVE-26082 > URL: https://issues.apache.org/jira/browse/HIVE-26082 > Project: Hive > Issue Type: Task >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Major > > Upgrade > datanucleus-api-jdo 5.2.4 to 5.2.8 > datanucleus-core 5.2.4 to 5.2.10 > datanucleus-rdbms 5.2.4 to 5.2.10 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HIVE-26082) Upgrade DataNucleus dependency to 5.2.8
[ https://issues.apache.org/jira/browse/HIVE-26082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Sharma updated HIVE-26082: - Priority: Minor (was: Major) > Upgrade DataNucleus dependency to 5.2.8 > --- > > Key: HIVE-26082 > URL: https://issues.apache.org/jira/browse/HIVE-26082 > Project: Hive > Issue Type: Task >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Minor > > Upgrade > datanucleus-api-jdo 5.2.4 to 5.2.8 > datanucleus-core 5.2.4 to 5.2.10 > datanucleus-rdbms 5.2.4 to 5.2.10 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HIVE-26082) Upgrade DataNucleus dependency to 5.2.8
[ https://issues.apache.org/jira/browse/HIVE-26082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Sharma updated HIVE-26082: - Description: Upgrade datanucleus-api-jdo 5.2.4 to 5.2.8 datanucleus-core 5.2.4 to 5.2.10 datanucleus-rdbms 5.2.4 to 5.2.10 was:Upgrade 5.2.4 to 5.2.6 > Upgrade DataNucleus dependency to 5.2.8 > --- > > Key: HIVE-26082 > URL: https://issues.apache.org/jira/browse/HIVE-26082 > Project: Hive > Issue Type: Task >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Major > > Upgrade > datanucleus-api-jdo 5.2.4 to 5.2.8 > datanucleus-core 5.2.4 to 5.2.10 > datanucleus-rdbms 5.2.4 to 5.2.10 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HIVE-26082) Upgrade DataNucleus dependency to 5.2.8
[ https://issues.apache.org/jira/browse/HIVE-26082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Sharma updated HIVE-26082: - Summary: Upgrade DataNucleus dependency to 5.2.8 (was: Upgrade DataNucleus dependency to 5.2.6) > Upgrade DataNucleus dependency to 5.2.8 > --- > > Key: HIVE-26082 > URL: https://issues.apache.org/jira/browse/HIVE-26082 > Project: Hive > Issue Type: Task >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Major > > Upgrade 5.2.4 to 5.2.6 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (HIVE-26075) hive metastore connection leaking when hiveserver2 kerberos enable and hive.server2.enable.doAs set to true
[ https://issues.apache.org/jira/browse/HIVE-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17513840#comment-17513840 ] liuguanghua edited comment on HIVE-26075 at 3/29/22, 6:28 AM: -- I have tested hive version that is 1.2.2 and 2.3.3. Both of them has the same problem was (Author: liuguanghua): I have tested hive version that is 1.2.2 and 2.3.3. > hive metastore connection leaking when hiveserver2 kerberos enable and > hive.server2.enable.doAs set to true > > > Key: HIVE-26075 > URL: https://issues.apache.org/jira/browse/HIVE-26075 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: liuguanghua >Assignee: liuguanghua >Priority: Major > Attachments: HIVE-26075.patch > > > (1)When hadoop cluster kerberos is enable > (2) HiveServer2 config hive.server2.enable.doAs is set true > After a beeline scripte has been executed, hivemetastore connection is > created are in ESTABLISHED state and never closed. > If we submit a lot of task to hiveserver2 ,this will result in hive metastore > thrift thread(default is 1000) full ,thus new task will fail. > > HiveServer2 use ThreadLocal to store multithreading metastore > connection,the application should call Hive.closeCurrent() to close > connection after task finished. > > When HiveServer2 impersonate is enable (hive.server2.enable.doAs is set > true), the ugi will create proxy user via > UserGroupInformation.createProxyUser( > owner, UserGroupInformation.getLoginUser()),the old metastore client is never > closed. > > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HIVE-26075) hive metastore connection leaking when hiveserver2 kerberos enable and hive.server2.enable.doAs set to true
[ https://issues.apache.org/jira/browse/HIVE-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17513840#comment-17513840 ] liuguanghua commented on HIVE-26075: I have tested hive version that is 1.2.2 and 2.3.3. > hive metastore connection leaking when hiveserver2 kerberos enable and > hive.server2.enable.doAs set to true > > > Key: HIVE-26075 > URL: https://issues.apache.org/jira/browse/HIVE-26075 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: liuguanghua >Assignee: liuguanghua >Priority: Major > Attachments: HIVE-26075.patch > > > (1)When hadoop cluster kerberos is enable > (2) HiveServer2 config hive.server2.enable.doAs is set true > After a beeline scripte has been executed, hivemetastore connection is > created are in ESTABLISHED state and never closed. > If we submit a lot of task to hiveserver2 ,this will result in hive metastore > thrift thread(default is 1000) full ,thus new task will fail. > > HiveServer2 use ThreadLocal to store multithreading metastore > connection,the application should call Hive.closeCurrent() to close > connection after task finished. > > When HiveServer2 impersonate is enable (hive.server2.enable.doAs is set > true), the ugi will create proxy user via > UserGroupInformation.createProxyUser( > owner, UserGroupInformation.getLoginUser()),the old metastore client is never > closed. > > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HIVE-26082) Upgrade DataNucleus dependency to 5.2.6
[ https://issues.apache.org/jira/browse/HIVE-26082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Sharma reassigned HIVE-26082: > Upgrade DataNucleus dependency to 5.2.6 > --- > > Key: HIVE-26082 > URL: https://issues.apache.org/jira/browse/HIVE-26082 > Project: Hive > Issue Type: Task >Reporter: Ashish Sharma >Assignee: Ashish Sharma >Priority: Major > > Upgrade 5.2.4 to 5.2.6 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HIVE-25516) ITestDbTxnManager is broken after HIVE-24120
[ https://issues.apache.org/jira/browse/HIVE-25516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Sharma resolved HIVE-25516. -- Resolution: Resolved > ITestDbTxnManager is broken after HIVE-24120 > > > Key: HIVE-25516 > URL: https://issues.apache.org/jira/browse/HIVE-25516 > Project: Hive > Issue Type: Task >Reporter: Denys Kuzmenko >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
