[jira] [Created] (HIVE-24087) FK side join elimination in presence of PK-FK constraint
Vineet Garg created HIVE-24087:
--
Summary: FK side join elimination in presence of PK-FK constraint
Key: HIVE-24087
URL: https://issues.apache.org/jira/browse/HIVE-24087
Project: Hive
Issue Type: Improvement
Components: Query Planning
Reporter: Vineet Garg
Assignee: Vineet Garg
If there is PK-FK join FK join could be eliminated by removing FK side if
following conditions are met
* There is no row filtering on FK side.
* No columns from FK side is required after JOIN.
* FK join columns are guranteed to be unique (have group by)
* FK join columns are guranteed to be NOT NULL (either IS NOT NULL filter or
constraint)
*Example*
{code:sql}
EXPLAIN
SELECT customer_removal_n0.*
FROM customer_removal_n0
JOIN
(SELECT lo_custkey
FROM lineorder_removal_n0
WHERE lo_custkey IS NOT NULL
GROUP BY lo_custkey) fkSide ON fkSide.lo_custkey =
customer_removal_n0.c_custkey;
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24086) CTAS with HMS translation enabled returns empty results.
Naveen Gangam created HIVE-24086: Summary: CTAS with HMS translation enabled returns empty results. Key: HIVE-24086 URL: https://issues.apache.org/jira/browse/HIVE-24086 Project: Hive Issue Type: Bug Components: Hive Reporter: Naveen Gangam Assignee: Naveen Gangam when you execute something like create table ctas_table as select * from mgd_table; if mgd_table is a managed table, the hive query planner creates a plan with ctas_table as a managed table, so the location is set to something in the managed warehouse directory. However with HMS translation enabled, non-acid MANAGED tables are converted to EXTERNAL with purge set to true. So the table location for this table is altered to be in the external warehouse directory. But after the table creation, the rest of the query executes but the data is copied to the location set in the query plan. As a result when you execute a select from ctas_table, it will not return any results because that location is empty. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Your project's website
I committed the change to master, thanks Zoltan and John for looking at the patch. I filed https://issues.apache.org/jira/browse/INFRA-20776 to move the site. .. Owen On Thu, Aug 27, 2020 at 4:06 PM Zoltan Haindrich wrote: > Hey Owen! > > It looks great! > Thank you for modernizing the site...we will be able to change the site > more easily after this change > +1 merging it > > cheers, > Zoltan > > > On 8/26/20 5:52 PM, Owen O'Malley wrote: > > Is there any more feedback or concerns on the proposed site? > > > > If not, I'll go ahead and check it in and file the jira for Apache Infra > to > > move the hosting for our site. > > > > .. Owen > > > > On Tue, Aug 18, 2020 at 2:05 PM Owen O'Malley > > wrote: > > > >> All, > >> I realized that I could mark my fork of the Hive repository as > having > >> github pages to effectively publish them. It took some changes, but now > it > >> is live on http://omalley.github.io/hive . > >> > >> On Tue, Aug 18, 2020 at 6:14 AM Andrew Wetmore > wrote: > >> > >>> Good progress! We will need an official PMC statement that Infra can > >>> close down the project presence on the CMS. > >>> > >>> Thanks! > >>> > >>> Andrew > >>> > >>> On Tue, Aug 18, 2020 at 4:09 AM Owen O'Malley > >>> wrote: > >>> > All, > > I ported the current site to github pages and pushed it to > https://github.com/apache/hive/pull/1410 . > > Take a look. It will certainly be much easier to maintain. > > .. Owen > > On Mon, Aug 17, 2020 at 6:26 AM Andrew Wetmore > wrote: > > > Hi: > > > > I wrote to you two weeks ago about your project website, but have not > > seen > > a response. Whom should I contact directly? I am trying to see > whether > > your > > project still uses the Apache CMS and, if so, what your plans are to > > migrate off it. > > > > Thanks in advance for your help. > > > > Andrew > > > > On Wed, Aug 5, 2020 at 9:17 AM Andrew Wetmore > > wrote: > > > >> Hi: > >> > >> I am part of the Infrastructure team, and am writing to ask whether > > your > >> project is still using the Apache CMS for your project website. As > you > >> know, the CMS is reaching end-of-life, and we need projects to move > > their > >> websites onto a different option within the next few weeks. > >> > >> There are several alternatives available, including those listed on > > this > >> page [1] on managing project websites. Infra is assembling a Wiki > > page [2] > >> on migrating a website from the CMS, and is looking forward to > helping > >> projects with this transition. > >> > >> Please let me know whether your site is still on the Apache CMS and, > > if > >> so, who will be the project point-of-contact with Infra for the > > migration. > >> > >> Thank you! > >> > >> > >> > >> > >> [1] https://infra.apache.org/project-site.html > >> > >> [2] > >> > > > https://cwiki.apache.org/confluence/display/INFRA/Migrate+your+project+website+from+the+Apache+CMS > >> > >> > >> > >> -- > >> Andrew Wetmore > >> Technical Writer-Editor > >> Infra > >> *Apache Software Foundation* > >> andr...@apache.org > >> > >> > >> < > > > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > > > > Virus-free. > >> www.avast.com > >> < > > > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > >> > >> <#m_2078696594329065464_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> > >> > > > > > > -- > > Andrew Wetmore > > Technical Writer-Editor > > Infra > > *Apache Software Foundation* > > andr...@apache.org > > > > >>> > >>> -- > >>> Andrew Wetmore > >>> Technical Writer-Editor > >>> Infra > >>> *Apache Software Foundation* > >>> andr...@apache.org > >>> > >> > > >
Re: Your project's website
Hey Owen! It looks great! Thank you for modernizing the site...we will be able to change the site more easily after this change +1 merging it cheers, Zoltan On 8/26/20 5:52 PM, Owen O'Malley wrote: Is there any more feedback or concerns on the proposed site? If not, I'll go ahead and check it in and file the jira for Apache Infra to move the hosting for our site. .. Owen On Tue, Aug 18, 2020 at 2:05 PM Owen O'Malley wrote: All, I realized that I could mark my fork of the Hive repository as having github pages to effectively publish them. It took some changes, but now it is live on http://omalley.github.io/hive . On Tue, Aug 18, 2020 at 6:14 AM Andrew Wetmore wrote: Good progress! We will need an official PMC statement that Infra can close down the project presence on the CMS. Thanks! Andrew On Tue, Aug 18, 2020 at 4:09 AM Owen O'Malley wrote: All, I ported the current site to github pages and pushed it to https://github.com/apache/hive/pull/1410 . Take a look. It will certainly be much easier to maintain. .. Owen On Mon, Aug 17, 2020 at 6:26 AM Andrew Wetmore wrote: Hi: I wrote to you two weeks ago about your project website, but have not seen a response. Whom should I contact directly? I am trying to see whether your project still uses the Apache CMS and, if so, what your plans are to migrate off it. Thanks in advance for your help. Andrew On Wed, Aug 5, 2020 at 9:17 AM Andrew Wetmore wrote: Hi: I am part of the Infrastructure team, and am writing to ask whether your project is still using the Apache CMS for your project website. As you know, the CMS is reaching end-of-life, and we need projects to move their websites onto a different option within the next few weeks. There are several alternatives available, including those listed on this page [1] on managing project websites. Infra is assembling a Wiki page [2] on migrating a website from the CMS, and is looking forward to helping projects with this transition. Please let me know whether your site is still on the Apache CMS and, if so, who will be the project point-of-contact with Infra for the migration. Thank you! [1] https://infra.apache.org/project-site.html [2] https://cwiki.apache.org/confluence/display/INFRA/Migrate+your+project+website+from+the+Apache+CMS -- Andrew Wetmore Technical Writer-Editor Infra *Apache Software Foundation* andr...@apache.org < https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virus-free. www.avast.com < https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#m_2078696594329065464_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> -- Andrew Wetmore Technical Writer-Editor Infra *Apache Software Foundation* andr...@apache.org -- Andrew Wetmore Technical Writer-Editor Infra *Apache Software Foundation* andr...@apache.org
[jira] [Created] (HIVE-24085) Optimize build times of druid related modules
László Bodor created HIVE-24085: --- Summary: Optimize build times of druid related modules Key: HIVE-24085 URL: https://issues.apache.org/jira/browse/HIVE-24085 Project: Hive Issue Type: Improvement Reporter: László Bodor -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24084) Enhance cost model to push down more Aggregates
Zoltan Haindrich created HIVE-24084: --- Summary: Enhance cost model to push down more Aggregates Key: HIVE-24084 URL: https://issues.apache.org/jira/browse/HIVE-24084 Project: Hive Issue Type: Improvement Reporter: Zoltan Haindrich Assignee: Zoltan Haindrich -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24083) hcatalog error in Hadoop 3.3.0: authentication type needed
Javier J. Salmeron Garcia created HIVE-24083: Summary: hcatalog error in Hadoop 3.3.0: authentication type needed Key: HIVE-24083 URL: https://issues.apache.org/jira/browse/HIVE-24083 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 3.1.2 Reporter: Javier J. Salmeron Garcia Using Hive 3.1.2, webhcat fails to start in Hadoop 3.3.0 with the following error: ``` javax.servlet.ServletException: Authentication type must be specified: simple|kerberos ``` I tried in Hadoop 3.2.1 with the exact settings and it starts without issues: ``` webhcat: /tmp/hadoop-3.2.1//bin/hadoop jar /opt/bitnami/hadoop/hive/hcatalog/sbin/../share/webhcat/svr/lib/hive-webhcat-3.1.2.jar org.apache.hive.hcatalog.templeton.Main webhcat: starting ... started. webhcat: done ``` I can provide more logs if needed. Detected authentication settings: ``` hadoop.http.authentication.simple.anonymous.allowed=true hadoop.http.authentication.type=simple hadoop.security.authentication=simple ipc.client.fallback-to-simple-auth-allowed=false yarn.timeline-service.http-authentication.simple.anonymous.allowed=true yarn.timeline-service.http-authentication.type=simple ``` -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24082) Expose information whether AcidUtils.ParsedDelta contains statementId
Piotr Findeisen created HIVE-24082:
--
Summary: Expose information whether AcidUtils.ParsedDelta contains
statementId
Key: HIVE-24082
URL: https://issues.apache.org/jira/browse/HIVE-24082
Project: Hive
Issue Type: Improvement
Reporter: Piotr Findeisen
In [Presto|https://prestosql.io] we support reading ORC ACID tables by
leveraging AcidUtils rather than duplicate the file name parsing logic in our
code.
To do this fully correctly, we need information whether
{{org.apache.hadoop.hive.ql.io.AcidUtils.ParsedDelta}} contains {{statementId}}
information or not.
Currently, a getter of that property does not allow us to access this
information.
[https://github.com/apache/hive/blob/468907eab36f78df3e14a24005153c9a23d62555/ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java#L804-L806]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24081) Enable pre-materializing CTEs referenced in scalar subqueries
Krisztian Kasa created HIVE-24081:
-
Summary: Enable pre-materializing CTEs referenced in scalar
subqueries
Key: HIVE-24081
URL: https://issues.apache.org/jira/browse/HIVE-24081
Project: Hive
Issue Type: Improvement
Components: Query Planning
Reporter: Krisztian Kasa
Assignee: Krisztian Kasa
HIVE-11752 introduces materializing CTE based on config
{code}
hive.optimize.cte.materialize.threshold
{code}
Goal of this jira is
* extending the implementation to support materializing CTE's referenced in
scalar subqueries
* add a config to materialize CTEs with aggregate output only
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24080) CBO failed: Array and Map are not support when convert calcite types to hive types
范宜臻 created HIVE-24080: -- Summary: CBO failed: Array and Map are not support when convert calcite types to hive types Key: HIVE-24080 URL: https://issues.apache.org/jira/browse/HIVE-24080 Project: Hive Issue Type: Bug Components: CBO, Logical Optimizer Affects Versions: 2.3.4, 3.0.0 Reporter: 范宜臻 -- This message was sent by Atlassian Jira (v8.3.4#803005)
