[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=458668&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-458668 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 14/Jul/20 14:28 Start Date: 14/Jul/20 14:28 Worklog Time Spent: 10m Work Description: belugabehr closed pull request #1118: URL: https://github.com/apache/hive/pull/1118 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 458668) Time Spent: 2h 40m (was: 2.5h) > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: David Mollitor >Priority: Critical > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 2h 40m > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=458669&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-458669 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 14/Jul/20 14:28 Start Date: 14/Jul/20 14:28 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #1118: URL: https://github.com/apache/hive/pull/1118#issuecomment-658213248 Complete and merged through manual patch process https://issues.apache.org/jira/browse/HIVE-23363 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 458669) Time Spent: 2h 50m (was: 2h 40m) > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: David Mollitor >Priority: Critical > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 2h 50m > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=457190&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-457190 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 10/Jul/20 15:03 Start Date: 10/Jul/20 15:03 Worklog Time Spent: 10m Work Description: belugabehr edited a comment on pull request #1118: URL: https://github.com/apache/hive/pull/1118#issuecomment-656722071 > Foreign Keys > > So we now have given the datastore control over the cascade deletion strategy for objects stored in these tables. Please be aware that JDO provides Dependent Fields as a way of allowing cascade deletion. The difference here is that Dependent Fields is controlled by DataNucleus, whereas foreign key delete actions are controlled by the datastore (assuming the datastore supports it even) ``` http://www.datanucleus.org/products/accessplatform/jdo/mapping.html#fk This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 457190) Time Spent: 2.5h (was: 2h 20m) > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: David Mollitor >Priority: Critical > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 2.5h > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=457189&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-457189 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 10/Jul/20 15:02 Start Date: 10/Jul/20 15:02 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #1118: URL: https://github.com/apache/hive/pull/1118#issuecomment-656722071 ``` Foreign Keys So we now have given the datastore control over the cascade deletion strategy for objects stored in these tables. Please be aware that JDO provides Dependent Fields as a way of allowing cascade deletion. The difference here is that Dependent Fields is controlled by DataNucleus, whereas foreign key delete actions are controlled by the datastore (assuming the datastore supports it even) ``` http://www.datanucleus.org/products/accessplatform/jdo/mapping.html#fk This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 457189) Time Spent: 2h 20m (was: 2h 10m) > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: David Mollitor >Priority: Critical > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[
https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=457186&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-457186
]
ASF GitHub Bot logged work on HIVE-23363:
-
Author: ASF GitHub Bot
Created on: 10/Jul/20 14:54
Start Date: 10/Jul/20 14:54
Worklog Time Spent: 10m
Work Description: belugabehr commented on pull request #1118:
URL: https://github.com/apache/hive/pull/1118#issuecomment-656718317
```
ALTER TABLE "APP"."COLUMNS_V2" ADD CONSTRAINT "COLUMNS_V2_FK1" FOREIGN KEY
("CD_ID") REFERENCES "APP"."CDS" ("CD_ID") ON DELETE NO ACTION ON UPDATE NO
ACTION;
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 457186)
Time Spent: 2h 10m (was: 2h)
> Upgrade DataNucleus dependency to 5.2
> -
>
> Key: HIVE-23363
> URL: https://issues.apache.org/jira/browse/HIVE-23363
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Zoltan Chovan
>Assignee: David Mollitor
>Priority: Critical
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-23363.2.patch, HIVE-23363.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been
> retired:
> [http://www.datanucleus.org/documentation/products.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=455546&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-455546 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 07/Jul/20 15:22 Start Date: 07/Jul/20 15:22 Worklog Time Spent: 10m Work Description: belugabehr opened a new pull request #1118: URL: https://github.com/apache/hive/pull/1118 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 455546) Time Spent: 2h (was: 1h 50m) > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 2h > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=455543&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-455543 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 07/Jul/20 15:21 Start Date: 07/Jul/20 15:21 Worklog Time Spent: 10m Work Description: belugabehr closed pull request #1118: URL: https://github.com/apache/hive/pull/1118 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 455543) Time Spent: 1h 50m (was: 1h 40m) > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=453612&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-453612 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 01/Jul/20 19:14 Start Date: 01/Jul/20 19:14 Worklog Time Spent: 10m Work Description: belugabehr edited a comment on pull request #1118: URL: https://github.com/apache/hive/pull/1118#issuecomment-652546609 @ashutoshc Let me see if I can address all of your questions with some background and context. It took me a long time to get these changes to pass the unit tests. So, these mappings, in some respect, don't really matter. When HMS is started, users use the `schema-tool` to create the HMS schema for real. Some of these mappings in the `jdo` file (like indexes) are only applied when unit testing because the unit tests build the schema via DN and `datanucleus.schema.autoCreateAll`. For unit testing, the database backend is Apache Derby. I changed the name of the index to match the Derby schema more closely. In trying to debug these various errors, I was very confused at first about it complaining about "COLUMNS_PK". https://github.com/apache/hive/blob/4942a7c0b4be3a5b0c889a89b903e9a70c57d494/standalone-metastore/metastore-server/src/main/sql/derby/hive-schema-4.0.0.derby.sql#L364 With that said, when I upgraded to DN 5.x, the unit tests would not pass. I narrowed the issue down to this one table definition. I tried several iterations to get success, but this is the one that worked. I derived this solution by closely examining the docs on this topic. It has an example that very closely aligns with this use case: http://www.datanucleus.org/products/accessplatform/jdo/mapping.html#embedded_collection It is a bit of a wonder looking at the existing JDO definition how this ever worked. ``` ``` This is not correct, this should be a compound primary key of CD_ID *and* COLUMN_NAME. This exact scenario is covered in the second half of: http://www.datanucleus.org/products/accessplatform/jdo/mapping.html#embedded_collection In the official schema (hive-schema-4.0.0.derby.sql), the primary key is enforced by the `SQL110922153006740` index. As things currently stand, the COLUMN_NAME definition in the `jdo` file says that the COLUMN_NAME is not defined to be non-null. This caused an error with Derby as it didn't allow creating a PRIMARY KEY on a field that could be null. So, putting it all together, I came to the current solution. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 453612) Time Spent: 1h 40m (was: 1.5h) > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=453611&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-453611 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 01/Jul/20 19:11 Start Date: 01/Jul/20 19:11 Worklog Time Spent: 10m Work Description: belugabehr edited a comment on pull request #1118: URL: https://github.com/apache/hive/pull/1118#issuecomment-652546609 @ashutoshc Let me see if I can address all of your questions with some background and context. It took me a long time to get these changes to pass the unit tests. So, these mappings, in some respect, don't really matter. When HMS is started, users use the `schema-tool` to create the HMS schema for real. Some of these mappings in the `jdo` file (like indexes) are only applied when unit testing because the unit tests build the schema via DN and `datanucleus.schema.autoCreateAll`. For unit testing, the database backend is Apache Derby. I changed the name of the index to match the Derby schema more closely. In trying to debug these various errors, I was very confused at first about it complaining about "COLUMNS_PK". https://github.com/apache/hive/blob/4942a7c0b4be3a5b0c889a89b903e9a70c57d494/standalone-metastore/metastore-server/src/main/sql/derby/hive-schema-4.0.0.derby.sql#L364 With that said, when I upgraded to DN 5.x, the unit tests would not pass. I narrowed the issue down to this one table definition. I tried several iterations to get success, but this is the one that worked. I derived this solution by closely examining the docs on this topic. It has an example that very closely aligns with this use case: http://www.datanucleus.org/products/accessplatform/jdo/mapping.html#embedded_collection It is a bit of a wonder looking at the existing JDO definition how this ever worked. ``` ``` This is not correct, this should be a compound primary key of CD_ID *and* COLUMN_NAME. This is enforced by `SQL110922153006740` in the full schema. As things currently stand, the COLUMN_NAME definition in the `jdo` file says that the COLUMN_NAME is not defined to be non-null. This caused an error with Derby as it didn't allow creating a PRIMARY KEY on a field that could be null. So, putting it all together, I came to the current solution. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 453611) Time Spent: 1.5h (was: 1h 20m) > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=453610&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-453610 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 01/Jul/20 19:10 Start Date: 01/Jul/20 19:10 Worklog Time Spent: 10m Work Description: belugabehr commented on a change in pull request #1118: URL: https://github.com/apache/hive/pull/1118#discussion_r448563837 ## File path: standalone-metastore/metastore-server/src/main/resources/package.jdo ## @@ -345,20 +345,20 @@ - Review comment: Just following the directions here: http://www.datanucleus.org/products/accessplatform/jdo/mapping.html#embedded_collection This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 453610) Time Spent: 1h 20m (was: 1h 10m) > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=453602&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-453602 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 01/Jul/20 19:04 Start Date: 01/Jul/20 19:04 Worklog Time Spent: 10m Work Description: belugabehr commented on a change in pull request #1118: URL: https://github.com/apache/hive/pull/1118#discussion_r448560561 ## File path: standalone-metastore/metastore-server/src/main/resources/package.jdo ## @@ -345,20 +345,20 @@ - Review comment: Changed to: ``` ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 453602) Time Spent: 1h 10m (was: 1h) > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=453599&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-453599 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 01/Jul/20 18:59 Start Date: 01/Jul/20 18:59 Worklog Time Spent: 10m Work Description: belugabehr commented on a change in pull request #1118: URL: https://github.com/apache/hive/pull/1118#discussion_r448558398 ## File path: standalone-metastore/metastore-server/src/main/resources/package.jdo ## @@ -345,20 +345,20 @@ - + - Review comment: ``` If a foreign-key is specified (in MetaData) for the relation field then leave any deletion to the datastore to perform ``` I don't see any such relationship cascading relationship defined in the schema for Derby or MySQL, so DN should be doing it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 453599) Time Spent: 1h (was: 50m) > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=453544&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-453544 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 01/Jul/20 17:18 Start Date: 01/Jul/20 17:18 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #1118: URL: https://github.com/apache/hive/pull/1118#issuecomment-652546609 @ashutoshc Let me see if I can address all of your questions with some background and context. It took me a long time to get these changes to pass the unit tests. So, these mappings, in some respect, don't really matter. When HMS is started, users use the `schema-tool` to create the HMS schema for real. Some of these mappings in the `jdo` file (like indexes) are only applied when unit testing because the unit tests build the schema via DN and `datanucleus.schema.autoCreateAll`. For unit testing, the database backend is Apache Derby. I changed the name of the index to match the Derby schema more closely. In trying to debug these various errors, I was very confused at first about it complaining about "COLUMNS_PK". https://github.com/apache/hive/blob/4942a7c0b4be3a5b0c889a89b903e9a70c57d494/standalone-metastore/metastore-server/src/main/sql/derby/hive-schema-4.0.0.derby.sql#L364 With that said, when I upgraded to DN 5.x, the unit tests would not pass. I narrowed the issue down to this one table definition. I tried several iterations to get success, but this is the one that worked. I derived this solution by closely examining the docs on this topic. It has an example that very closely aligns with this use case: http://www.datanucleus.org/products/accessplatform/jpa/mapping.html#embedded_collection It is a bit of a wonder looking at the existing JDO definition how this ever worked. ``` ``` This is not correct, this should be a compound primary key of CD_ID *and* COLUMN_NAME. This is enforced by `SQL110922153006740` in the full schema. As things currently stand, the COLUMN_NAME definition in the `jdo` file says that the COLUMN_NAME is not defined to be non-null. This caused an error with Derby as it didn't allow creating a PRIMARY KEY on a field that could be null. So, putting it all together, I came to the current solution. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 453544) Time Spent: 50m (was: 40m) > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 50m > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=453508&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-453508 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 01/Jul/20 16:31 Start Date: 01/Jul/20 16:31 Worklog Time Spent: 10m Work Description: ashutoshc commented on a change in pull request #1118: URL: https://github.com/apache/hive/pull/1118#discussion_r448480980 ## File path: standalone-metastore/metastore-server/src/main/resources/package.jdo ## @@ -345,20 +345,20 @@ - Review comment: Can you describe the need for this change? ## File path: standalone-metastore/metastore-server/src/main/resources/package.jdo ## @@ -345,20 +345,20 @@ - + - - Review comment: Any reason to change the name here? ## File path: standalone-metastore/metastore-server/src/main/resources/package.jdo ## @@ -345,20 +345,20 @@ - + - Review comment: We do want cascade-delete here. Any reason to remove it? ## File path: standalone-metastore/metastore-server/src/main/resources/package.jdo ## @@ -345,20 +345,20 @@ - + - - + + Review comment: This probably is fine to do. Though, was it necessary? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 453508) Time Spent: 40m (was: 0.5h) > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=446101&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-446101 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 15/Jun/20 20:36 Start Date: 15/Jun/20 20:36 Worklog Time Spent: 10m Work Description: belugabehr opened a new pull request #1118: URL: https://github.com/apache/hive/pull/1118 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 446101) Time Spent: 0.5h (was: 20m) > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=446099&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-446099 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 15/Jun/20 20:35 Start Date: 15/Jun/20 20:35 Worklog Time Spent: 10m Work Description: belugabehr closed pull request #1118: URL: https://github.com/apache/hive/pull/1118 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 446099) Time Spent: 20m (was: 10m) > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23363) Upgrade DataNucleus dependency to 5.2
[ https://issues.apache.org/jira/browse/HIVE-23363?focusedWorklogId=446026&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-446026 ] ASF GitHub Bot logged work on HIVE-23363: - Author: ASF GitHub Bot Created on: 15/Jun/20 18:58 Start Date: 15/Jun/20 18:58 Worklog Time Spent: 10m Work Description: belugabehr opened a new pull request #1118: URL: https://github.com/apache/hive/pull/1118 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 446026) Remaining Estimate: 0h Time Spent: 10m > Upgrade DataNucleus dependency to 5.2 > - > > Key: HIVE-23363 > URL: https://issues.apache.org/jira/browse/HIVE-23363 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Critical > Attachments: HIVE-23363.2.patch, HIVE-23363.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade Datanucleus from 4.2 to 5.2 as based on it's docs 4.2 has been > retired: > [http://www.datanucleus.org/documentation/products.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
