Re: [PR] [WIP] [hive]
sonarcloud[bot] commented on PR #5283: URL: https://github.com/apache/hive/pull/5283#issuecomment-2154222466 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5283) **Quality Gate passed** Issues  [0 New issues](https://sonarcloud.io/project/issues?id=apache_hive=5283=false=true)  [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive=5283=WONTFIX) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5283=false=true)  No data about Coverage  No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5283) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] [hive]
sonarcloud[bot] commented on PR #5283: URL: https://github.com/apache/hive/pull/5283#issuecomment-2153725992 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5283) **Quality Gate passed** Issues  [0 New issues](https://sonarcloud.io/project/issues?id=apache_hive=5283=false=true)  [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive=5283=WONTFIX) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5283=false=true)  No data about Coverage  No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5283) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-28302: Let SUM UDF return NULL when all rows have non-numeric texts [hive]
sonarcloud[bot] commented on PR #5283: URL: https://github.com/apache/hive/pull/5283#issuecomment-2152483416 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5283) **Quality Gate passed** Issues  [0 New issues](https://sonarcloud.io/project/issues?id=apache_hive=5283=false=true)  [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive=5283=WONTFIX) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5283=false=true)  No data about Coverage  No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5283) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
[PR] [WIP] HIVE-28302: Let SUM UDF return NULL when all rows have non-numeric texts [hive]
okumin opened a new pull request, #5283: URL: https://github.com/apache/hive/pull/5283 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### Is the change a dependency upgrade? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27847: Casting NUMERIC types to TIMESTAMP is prohibited [hive]
sonarcloud[bot] commented on PR #5278: URL: https://github.com/apache/hive/pull/5278#issuecomment-2148036035 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5278) **Quality Gate passed** Issues  [26 New issues](https://sonarcloud.io/project/issues?id=apache_hive=5278=false=true)  [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive=5278=WONTFIX) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5278=false=true)  No data about Coverage  No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5278) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27847: Casting NUMERIC types to TIMESTAMP is prohibited [hive]
sonarcloud[bot] commented on PR #5278: URL: https://github.com/apache/hive/pull/5278#issuecomment-2147469447 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5278) **Quality Gate passed** Issues  [1 New issue](https://sonarcloud.io/project/issues?id=apache_hive=5278=false=true)  [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive=5278=WONTFIX) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5278=false=true)  No data about Coverage  No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5278) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
[PR] [WIP] HIVE-27847: Casting NUMERIC types to TIMESTAMP is prohibited [hive]
okumin opened a new pull request, #5278: URL: https://github.com/apache/hive/pull/5278 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### Is the change a dependency upgrade? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
github-actions[bot] closed pull request #4380: WIP URL: https://github.com/apache/hive/pull/4380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
github-actions[bot] commented on PR #4380: URL: https://github.com/apache/hive/pull/4380#issuecomment-2019145658 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP. [hive]
ayushtkn closed pull request #5120: WIP. URL: https://github.com/apache/hive/pull/5120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
okumin commented on code in PR #5106:
URL: https://github.com/apache/hive/pull/5106#discussion_r1510501245
##
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:
##
@@ -1600,26 +1600,35 @@ Table materializeCTE(String cteName, CTEClause cte)
throws HiveException {
LOG.info("{} will be materialized into {}", cteName, location);
cte.source = analyzer;
-ctx.addMaterializedTable(cteName, table,
getMaterializedTableStats(analyzer.getSinkOp(), table));
+ctx.addMaterializedTable(cteName, table,
getMaterializedTableStats(analyzer.getSinkOp()));
return table;
}
- static Statistics getMaterializedTableStats(Operator sinkOp, Table table)
{
+ protected Statistics getMaterializedTableStats(Operator sinkOp) {
final Statistics tableStats = sinkOp.getStatistics().clone();
-final List sourceColStatsList = tableStats.getColumnStats();
-final List colNames =
table.getCols().stream().map(FieldSchema::getName).collect(Collectors.toList());
-if (sourceColStatsList.size() != colNames.size()) {
- throw new IllegalStateException(String.format(
- "The size of col stats must be equal to that of schema. Stats = %s,
Schema = %s",
- sourceColStatsList, colNames));
-}
-final List colStatsList = new
ArrayList<>(sourceColStatsList.size());
-for (int i = 0; i < sourceColStatsList.size(); i++) {
- final ColStatistics colStats = sourceColStatsList.get(i);
- // FileSinkOperator stores column stats with internal names such as
"_col1"
- colStats.setColumnName(colNames.get(i));
- colStatsList.add(colStats);
+if (tableStats.getColumnStatsState() == Statistics.State.NONE ||
sinkOp.getNumParent() == 0) {
+ return tableStats;
+}
+
+final List parentColumnNames =
sinkOp.getParentOperators().get(0).getSchema().getColumnNames();
Review Comment:
While I was debugging with a remote debugger, I found the order of
`Statistics#getColumnStats` is non-deterministic as the underlying data
structure is HashMap. That's why I modified how to map column names.
-
https://github.com/apache/hive/blob/rel/release-4.0.0-beta-1/ql/src/java/org/apache/hadoop/hive/ql/plan/Statistics.java#L327C33-L327C44
-
https://github.com/apache/hive/blob/rel/release-4.0.0-beta-1/ql/src/java/org/apache/hadoop/hive/ql/plan/Statistics.java#L211
##
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:
##
@@ -1600,26 +1600,35 @@ Table materializeCTE(String cteName, CTEClause cte)
throws HiveException {
LOG.info("{} will be materialized into {}", cteName, location);
cte.source = analyzer;
-ctx.addMaterializedTable(cteName, table,
getMaterializedTableStats(analyzer.getSinkOp(), table));
+ctx.addMaterializedTable(cteName, table,
getMaterializedTableStats(analyzer.getSinkOp()));
return table;
}
- static Statistics getMaterializedTableStats(Operator sinkOp, Table table)
{
+ protected Statistics getMaterializedTableStats(Operator sinkOp) {
final Statistics tableStats = sinkOp.getStatistics().clone();
-final List sourceColStatsList = tableStats.getColumnStats();
-final List colNames =
table.getCols().stream().map(FieldSchema::getName).collect(Collectors.toList());
-if (sourceColStatsList.size() != colNames.size()) {
- throw new IllegalStateException(String.format(
- "The size of col stats must be equal to that of schema. Stats = %s,
Schema = %s",
- sourceColStatsList, colNames));
-}
-final List colStatsList = new
ArrayList<>(sourceColStatsList.size());
-for (int i = 0; i < sourceColStatsList.size(); i++) {
- final ColStatistics colStats = sourceColStatsList.get(i);
- // FileSinkOperator stores column stats with internal names such as
"_col1"
- colStats.setColumnName(colNames.get(i));
- colStatsList.add(colStats);
+if (tableStats.getColumnStatsState() == Statistics.State.NONE ||
sinkOp.getNumParent() == 0) {
+ return tableStats;
Review Comment:
This is the main change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #5106: URL: https://github.com/apache/hive/pull/5106#issuecomment-1971387895 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5106) **Quality Gate passed** Issues  [11 New issues](https://sonarcloud.io/project/issues?id=apache_hive=5106=false=true)  [0 Accepted issues](https://sonarcloud.io/component_measures?id=apache_hive=5106=new_accepted_issues=list) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5106=false=true)  No data about Coverage  No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5106) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #5106: URL: https://github.com/apache/hive/pull/5106#issuecomment-1970851254 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5106) **Quality Gate passed** Issues  [5 New issues](https://sonarcloud.io/project/issues?id=apache_hive=5106=false=true)  [0 Accepted issues](https://sonarcloud.io/component_measures?id=apache_hive=5106=new_accepted_issues=list) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5106=false=true)  No data about Coverage  No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5106) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
[PR] WIP [hive]
okumin opened a new pull request, #5106: URL: https://github.com/apache/hive/pull/5106 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### Is the change a dependency upgrade? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] [hive]
sonarcloud[bot] commented on PR #5091: URL: https://github.com/apache/hive/pull/5091#issuecomment-1956184141 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5091) **Quality Gate passed** Issues  [2 New issues](https://sonarcloud.io/project/issues?id=apache_hive=5091=false=true) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5091=false=true)  No data about Coverage  No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5091) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] [hive]
sonarcloud[bot] commented on PR #5091: URL: https://github.com/apache/hive/pull/5091#issuecomment-1953834645 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5091) **Quality Gate passed** Issues  [2 New issues](https://sonarcloud.io/project/issues?id=apache_hive=5091=false=true) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5091=false=true)  No data about Coverage  No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5091) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] [hive]
sonarcloud[bot] commented on PR #5091: URL: https://github.com/apache/hive/pull/5091#issuecomment-1953464793 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5091) **Quality Gate passed** Issues  [2 New issues](https://sonarcloud.io/project/issues?id=apache_hive=5091=false=true) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5091=false=true)  No data about Coverage  No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5091) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
[PR] [WIP] [hive]
okumin opened a new pull request, #5091: URL: https://github.com/apache/hive/pull/5091 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### Is the change a dependency upgrade? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP - HIVE-27767 : Copy more data into HIVE_LOCKS for better supportability [hive]
github-actions[bot] closed pull request #4775: WIP - HIVE-27767 : Copy more data into HIVE_LOCKS for better supportability URL: https://github.com/apache/hive/pull/4775 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-24167: TPC-DS query 14 fails while generating plan for the filter [hive]
okumin commented on code in PR #5037:
URL: https://github.com/apache/hive/pull/5037#discussion_r1470541604
##
ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapper.java:
##
@@ -200,54 +199,30 @@ public void merge(Object o1, Object o2) {
}
private void link(Object o1, Object o2, boolean mayMerge) {
-
-Set keySet = Collections.newSetFromMap(new IdentityHashMap());
-keySet.add(o1);
-keySet.add(o2);
-keySet.add(getKeyFor(o1));
-keySet.add(getKeyFor(o2));
-
-Set mGroups = Collections.newSetFromMap(new
IdentityHashMap());
-
-for (Object object : keySet) {
- EquivGroup group = objectMap.get(object);
- if (group != null) {
-mGroups.add(group);
- }
-}
-if (mGroups.size() > 1) {
+final EquivGroup group1 = objectMap.get(o1);
+final EquivGroup group2 = objectMap.get(o2);
+if (group1 != null && group2 != null && group1 != group2) {
Review Comment:
I made some refactoring and will resolve the current conversations once.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-24167: TPC-DS query 14 fails while generating plan for the filter [hive]
sonarcloud[bot] commented on PR #5037: URL: https://github.com/apache/hive/pull/5037#issuecomment-1914780935 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5037) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [1 New issue](https://sonarcloud.io/project/issues?id=apache_hive=5037=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5037=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5037) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-24167: TPC-DS query 14 fails while generating plan for the filter [hive]
sonarcloud[bot] commented on PR #5037: URL: https://github.com/apache/hive/pull/5037#issuecomment-1914313579 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5037) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [1 New issue](https://sonarcloud.io/project/issues?id=apache_hive=5037=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5037=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5037) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP - HIVE-27767 : Copy more data into HIVE_LOCKS for better supportability [hive]
github-actions[bot] commented on PR #4775: URL: https://github.com/apache/hive/pull/4775#issuecomment-1913776758 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-24167: TPC-DS query 14 fails while generating plan for the filter [hive]
sonarcloud[bot] commented on PR #5037: URL: https://github.com/apache/hive/pull/5037#issuecomment-1913134667 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5037) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [5 New issues](https://sonarcloud.io/project/issues?id=apache_hive=5037=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5037=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5037) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-24167: TPC-DS query 14 fails while generating plan for the filter [hive]
sonarcloud[bot] commented on PR #5037: URL: https://github.com/apache/hive/pull/5037#issuecomment-1911974645 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5037) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [4 New issues](https://sonarcloud.io/project/issues?id=apache_hive=5037=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5037=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5037) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-24167: TPC-DS query 14 fails while generating plan for the filter [hive]
sonarcloud[bot] commented on PR #5037: URL: https://github.com/apache/hive/pull/5037#issuecomment-1911395609 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5037) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [8 New issues](https://sonarcloud.io/project/issues?id=apache_hive=5037=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5037=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5037) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-24167: TPC-DS query 14 fails while generating plan for the filter [hive]
okumin commented on code in PR #5037:
URL: https://github.com/apache/hive/pull/5037#discussion_r1467141935
##
ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapper.java:
##
@@ -200,54 +199,30 @@ public void merge(Object o1, Object o2) {
}
private void link(Object o1, Object o2, boolean mayMerge) {
-
-Set keySet = Collections.newSetFromMap(new IdentityHashMap());
-keySet.add(o1);
-keySet.add(o2);
-keySet.add(getKeyFor(o1));
-keySet.add(getKeyFor(o2));
-
-Set mGroups = Collections.newSetFromMap(new
IdentityHashMap());
-
-for (Object object : keySet) {
- EquivGroup group = objectMap.get(object);
- if (group != null) {
-mGroups.add(group);
- }
-}
-if (mGroups.size() > 1) {
+final EquivGroup group1 = objectMap.get(o1);
+final EquivGroup group2 = objectMap.get(o2);
+if (group1 != null && group2 != null && group1 != group2) {
Review Comment:
Thanks for your review but you may not need to review this PR yet.
I'm still investigating the issue and trying several approaches. I created
this PR to trigger CI as I can't know which test cases could be affected
because of the complexity of SemanticAnalyzer. It turned out that the current
approach would break something else, and I am trying another approach.
See also https://gist.github.com/okumin/b111fe0a911507bdf6a7204f49b9cb72 to
trace the current findings and my activity logs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-24167: TPC-DS query 14 fails while generating plan for the filter [hive]
aturoczy commented on code in PR #5037:
URL: https://github.com/apache/hive/pull/5037#discussion_r1466719874
##
ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapper.java:
##
@@ -200,54 +199,30 @@ public void merge(Object o1, Object o2) {
}
private void link(Object o1, Object o2, boolean mayMerge) {
-
-Set keySet = Collections.newSetFromMap(new IdentityHashMap());
-keySet.add(o1);
-keySet.add(o2);
-keySet.add(getKeyFor(o1));
-keySet.add(getKeyFor(o2));
-
-Set mGroups = Collections.newSetFromMap(new
IdentityHashMap());
-
-for (Object object : keySet) {
- EquivGroup group = objectMap.get(object);
- if (group != null) {
-mGroups.add(group);
- }
-}
-if (mGroups.size() > 1) {
+final EquivGroup group1 = objectMap.get(o1);
+final EquivGroup group2 = objectMap.get(o2);
+if (group1 != null && group2 != null && group1 != group2) {
Review Comment:
I would change the logic a bit. In my mind, from Clean Code perspective if
you check the true path it is easier to process and maintain the code.
i.e:
if (group1 == null || group2 == null || group1 == group2)
return;
##
ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapper.java:
##
@@ -200,54 +199,30 @@ public void merge(Object o1, Object o2) {
}
private void link(Object o1, Object o2, boolean mayMerge) {
-
-Set keySet = Collections.newSetFromMap(new IdentityHashMap());
-keySet.add(o1);
-keySet.add(o2);
-keySet.add(getKeyFor(o1));
-keySet.add(getKeyFor(o2));
-
-Set mGroups = Collections.newSetFromMap(new
IdentityHashMap());
-
-for (Object object : keySet) {
- EquivGroup group = objectMap.get(object);
- if (group != null) {
-mGroups.add(group);
- }
-}
-if (mGroups.size() > 1) {
+final EquivGroup group1 = objectMap.get(o1);
+final EquivGroup group2 = objectMap.get(o2);
+if (group1 != null && group2 != null && group1 != group2) {
if (!mayMerge) {
throw new RuntimeException("equivalence mapping violation");
}
EquivGroup newGrp = new EquivGroup();
newGrp.add(o1);
newGrp.add(o2);
- for (EquivGroup g : mGroups) {
-for (Object o : g.members) {
- newGrp.add(o);
-}
- }
+ group1.members.forEach(newGrp::add);
+ group2.members.forEach(newGrp::add);
groups.add(newGrp);
- groups.removeAll(mGroups);
+ groups.remove(group1);
+ groups.remove(group2);
} else {
- EquivGroup targetGroup = mGroups.isEmpty() ? new EquivGroup() :
mGroups.iterator().next();
+ EquivGroup targetGroup = group1 != null ? group1 : (group2 != null ?
group2 : new EquivGroup());
Review Comment:
The logic is great, but my eyes hurted with this amount of ? : operator :)
##
ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapper.java:
##
@@ -200,54 +199,30 @@ public void merge(Object o1, Object o2) {
}
private void link(Object o1, Object o2, boolean mayMerge) {
-
-Set keySet = Collections.newSetFromMap(new IdentityHashMap());
-keySet.add(o1);
-keySet.add(o2);
-keySet.add(getKeyFor(o1));
-keySet.add(getKeyFor(o2));
-
-Set mGroups = Collections.newSetFromMap(new
IdentityHashMap());
-
-for (Object object : keySet) {
- EquivGroup group = objectMap.get(object);
- if (group != null) {
-mGroups.add(group);
- }
-}
-if (mGroups.size() > 1) {
+final EquivGroup group1 = objectMap.get(o1);
+final EquivGroup group2 = objectMap.get(o2);
+if (group1 != null && group2 != null && group1 != group2) {
if (!mayMerge) {
throw new RuntimeException("equivalence mapping violation");
Review Comment:
If we touch this change please the exception message to Uppercase the first
word
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-24167: TPC-DS query 14 fails while generating plan for the filter [hive]
sonarcloud[bot] commented on PR #5037: URL: https://github.com/apache/hive/pull/5037#issuecomment-1909840390 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5037) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [1 New issue](https://sonarcloud.io/project/issues?id=apache_hive=5037=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5037=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5037) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
[PR] [WIP] HIVE-24167: TPC-DS query 14 fails while generating plan for the filter [hive]
okumin opened a new pull request, #5037: URL: https://github.com/apache/hive/pull/5037 Don't review this yet ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### Is the change a dependency upgrade? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-28000: Fix scenarios where 'not in' gives incorrect results due. [hive]
sonarcloud[bot] commented on PR #5007: URL: https://github.com/apache/hive/pull/5007#issuecomment-1909353216 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5007) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [2 New issues](https://sonarcloud.io/project/issues?id=apache_hive=5007=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5007=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5007) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-28000: Fix scenarios where 'not in' gives incorrect results due. [hive]
sonarcloud[bot] commented on PR #5007: URL: https://github.com/apache/hive/pull/5007#issuecomment-1909128158 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5007) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [2 New issues](https://sonarcloud.io/project/issues?id=apache_hive=5007=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5007=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5007) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #4380: URL: https://github.com/apache/hive/pull/4380#issuecomment-1909019511 ## [](https://sonarcloud.io/dashboard?id=apache_hive=4380) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [18 New issues](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=4380) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-28000: Fix scenarios where 'not in' gives incorrect results due. [hive]
sonarcloud[bot] commented on PR #5007: URL: https://github.com/apache/hive/pull/5007#issuecomment-1907188045 ## [](https://sonarcloud.io/dashboard?id=apache_hive=5007) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [2 New issues](https://sonarcloud.io/project/issues?id=apache_hive=5007=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=5007=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=5007) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #4380: URL: https://github.com/apache/hive/pull/4380#issuecomment-1879495743 ## [](https://sonarcloud.io/dashboard?id=apache_hive=4380) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [18 New issues](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=4380) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #4380: URL: https://github.com/apache/hive/pull/4380#issuecomment-1879352803 ## [](https://sonarcloud.io/dashboard?id=apache_hive=4380) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [25 New issues](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=4380) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #4380: URL: https://github.com/apache/hive/pull/4380#issuecomment-1879149051 ## [](https://sonarcloud.io/dashboard?id=apache_hive=4380) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [18 New issues](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=4380) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #4380: URL: https://github.com/apache/hive/pull/4380#issuecomment-1878824768 ## [](https://sonarcloud.io/dashboard?id=apache_hive=4380) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [25 New issues](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=4380) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #4380: URL: https://github.com/apache/hive/pull/4380#issuecomment-1878173500 ## [](https://sonarcloud.io/dashboard?id=apache_hive=4380) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [25 New issues](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=4380) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #4380: URL: https://github.com/apache/hive/pull/4380#issuecomment-1878000186 ## [](https://sonarcloud.io/dashboard?id=apache_hive=4380) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [25 New issues](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=4380) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #4380: URL: https://github.com/apache/hive/pull/4380#issuecomment-1877828081 ## [](https://sonarcloud.io/dashboard?id=apache_hive=4380) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [18 New issues](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=4380) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #4380: URL: https://github.com/apache/hive/pull/4380#issuecomment-1877663487 ## [](https://sonarcloud.io/dashboard?id=apache_hive=4380) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [18 New issues](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=4380) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #4380: URL: https://github.com/apache/hive/pull/4380#issuecomment-1877061520 ## [](https://sonarcloud.io/dashboard?id=apache_hive=4380) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [18 New issues](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=4380) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27775: DirectSQL and JDO results are different when fetching par… [hive]
sonarcloud[bot] commented on PR #4959: URL: https://github.com/apache/hive/pull/4959#issuecomment-1865447599 ## [](https://sonarcloud.io/dashboard?id=apache_hive=4959) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [14 New issues](https://sonarcloud.io/project/issues?id=apache_hive=4959=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4959=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=4959) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27775: DirectSQL and JDO results are different when fetching par… [hive]
dengzhhu653 commented on code in PR #4959:
URL: https://github.com/apache/hive/pull/4959#discussion_r1433391759
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/DatabaseProduct.java:
##
@@ -263,7 +265,9 @@ protected String toDate(String tableValue) {
protected String toTimestamp(String tableValue) {
if (isORACLE()) {
- return "TO_TIMESTAMP(" + tableValue + ", '-MM-DD HH:mm:ss')";
+ return "TO_TIMESTAMP(" + tableValue + ", '-MM-DD HH:mi:ss')";
Review Comment:
We don't need it here, I have tested against Oracle19
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27775: DirectSQL and JDO results are different when fetching par… [hive]
dengzhhu653 commented on code in PR #4959:
URL: https://github.com/apache/hive/pull/4959#discussion_r1433391031
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java:
##
@@ -1341,29 +1339,34 @@ public void visit(LeafNode node) throws MetaException {
return;
}
+ String nodeValue0 = "?";
// if Filter.g does date parsing for quoted strings, we'd need to verify
there's no
// type mismatch when string col is filtered by a string that looks like
date.
- if (colType == FilterType.Date && valType == FilterType.String) {
-// Filter.g cannot parse a quoted date; try to parse date here too.
+ if (colType == FilterType.Date) {
try {
- nodeValue = MetaStoreUtils.convertStringToDate((String)nodeValue);
+ nodeValue = dbType.convertDateValue(nodeValue);
valType = FilterType.Date;
-} catch (Exception pe) { // do nothing, handled below - types will
mismatch
+ if (dbType.isPOSTGRES() || dbType.isORACLE()) {
+nodeValue0 = "date '" + nodeValue + "'";
+nodeValue = null;
+ }
+} catch (Exception e) { // do nothing, handled below - types will
mismatch
+}
+ } else if (colType == FilterType.Timestamp) {
+if (dbType.isDERBY() || dbType.isMYSQL()) {
Review Comment:
For Derby or MySQL, we don't push the filter down directly, as the
`partition_timestamp3.q` gets
`java.lang.RuntimeException:
org.apache.hadoop.hive.ql.parse.SemanticException: MetaException(message:Lists
are not the same size: SQL 2, ORM 1)` on Derby and `Different results from SQL
and ORM, see log for details` on MySQL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27775: DirectSQL and JDO results are different when fetching par… [hive]
dengzhhu653 commented on code in PR #4959:
URL: https://github.com/apache/hive/pull/4959#discussion_r1432091301
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/DatabaseProduct.java:
##
@@ -263,7 +265,9 @@ protected String toDate(String tableValue) {
protected String toTimestamp(String tableValue) {
if (isORACLE()) {
- return "TO_TIMESTAMP(" + tableValue + ", '-MM-DD HH:mm:ss')";
+ return "TO_TIMESTAMP(" + tableValue + ", '-MM-DD HH:mi:ss')";
Review Comment:
I haven't tested against Oracle yet, let me try
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27775: DirectSQL and JDO results are different when fetching par… [hive]
sonarcloud[bot] commented on PR #4959: URL: https://github.com/apache/hive/pull/4959#issuecomment-1863142754 ## [](https://sonarcloud.io/dashboard?id=apache_hive=4959) **Quality Gate passed** The SonarCloud Quality Gate passed, but some issues were introduced. [8 New issues](https://sonarcloud.io/project/issues?id=apache_hive=4959=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4959=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=4959) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27775: DirectSQL and JDO results are different when fetching par… [hive]
aturoczy commented on code in PR #4959:
URL: https://github.com/apache/hive/pull/4959#discussion_r1431364088
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/DatabaseProduct.java:
##
@@ -263,7 +265,9 @@ protected String toDate(String tableValue) {
protected String toTimestamp(String tableValue) {
if (isORACLE()) {
- return "TO_TIMESTAMP(" + tableValue + ", '-MM-DD HH:mm:ss')";
+ return "TO_TIMESTAMP(" + tableValue + ", '-MM-DD HH:mi:ss')";
Review Comment:
Timezone information does not need here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
[PR] [WIP] HIVE-27775: DirectSQL and JDO results are different when fetching par… [hive]
dengzhhu653 opened a new pull request, #4959:
URL: https://github.com/apache/hive/pull/4959
…titions by timestamp in DST shift
### What changes were proposed in this pull request?
### Why are the changes needed?
On JDO path, we select the matched partitions by the partition name, which
is timezone agnostic, however on DirectSQL,
we do a cast partition value to timestamp, then select by the local time,
for example:
```
cast("FILTER0"."PART_KEY_VAL" as TIMESTAMP) else null end) as TIMESTAMP) =
<'2023-03-26 03:30:00'>
```
This could lead to different result as described in the Jira.
### Does this PR introduce _any_ user-facing change?
No
### Is the change a dependency upgrade?
No
### How was this patch tested?
mvn test -Dtest=TestMiniLlapLocalCliDriver -Dtest.output.overwrite=true
-Dqfile=partition_timestamp3.q -Dtest.metastore.db=postgres|derby|mssql|mysql
TODO: Test against Oracle
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27957: Better error message for STORED BY [hive]
sonarcloud[bot] commented on PR #4954: URL: https://github.com/apache/hive/pull/4954#issuecomment-1858749372 ## [](https://sonarcloud.io/dashboard?id=apache_hive=4954) **Quality Gate passed** Kudos, no new issues were introduced! [0 New issues](https://sonarcloud.io/project/issues?id=apache_hive=4954=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4954=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=4954) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
[PR] [WIP] HIVE-27957: Better error message for STORED BY [hive]
okumin opened a new pull request, #4954:
URL: https://github.com/apache/hive/pull/4954
### What changes were proposed in this pull request?
This PR would change an error message thrown when a user gives a wrong
storage format in STORED BY.
### Why are the changes needed?
The current messages are less informative. I got confused while I was
testing Iceberg and non-Iceberg tables in turns.
```
0: jdbc:hive2://hive-hiveserver2:1/defaul> CREATE TABLE test (a STRING)
STORED BY ORC TBLPROPERTIES ("orc.compress"="ZSTD");
Error: Error while compiling statement: FAILED: SemanticException Cannot
find class '' (state=42000,code=4)
0: jdbc:hive2://hive-hiveserver2:1/defaul> CREATE TABLE test (a STRING)
STORED BY DEFAULT;
Error: Error while compiling statement: FAILED: NullPointerException null
(state=42000,code=4)
```
### Does this PR introduce _any_ user-facing change?
It changes only error messages.
### Is the change a dependency upgrade?
No.
### How was this patch tested?
I added test cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27950: STACK UDTF returns wrong results when # of argument is no… [hive]
sonarcloud[bot] commented on PR #4938: URL: https://github.com/apache/hive/pull/4938#issuecomment-1853037841 ## [](https://sonarcloud.io/dashboard?id=apache_hive=4938) **Quality Gate passed** Kudos, no new issues were introduced! [0 New issues](https://sonarcloud.io/project/issues?id=apache_hive=4938=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4938=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive=4938) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27950: STACK UDTF returns wrong results when # of argument is no… [hive]
sonarcloud[bot] commented on PR #4938: URL: https://github.com/apache/hive/pull/4938#issuecomment-1851312859 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4938) [](https://sonarcloud.io/project/issues?id=apache_hive=4938=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4938=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4938=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4938=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4938=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4938=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4938=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4938=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4938=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4938=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4938=false=CODE_SMELL) [1 Code Smell](https://sonarcloud.io/project/issues?id=apache_hive=4938=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4938=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4938=duplicated_lines_density=list) No Duplication information  The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarsource.com/sonarcloud/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27950: STACK UDTF returns wrong results when # of argument is no… [hive]
okumin commented on code in PR #4938:
URL: https://github.com/apache/hive/pull/4938#discussion_r1423354230
##
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFStack.java:
##
@@ -63,11 +64,16 @@ public StructObjectInspector initialize(ObjectInspector[]
args)
}
if (!(args[0] instanceof ConstantObjectInspector)) {
throw new UDFArgumentException(
+ "The first argument to STACK() must be a constant.");
+}
+final Object value = ((ConstantObjectInspector)
args[0]).getWritableConstantValue();
+if (!(value instanceof IntWritable)) {
Review Comment:
I added a validation since the current implementation throws a
ClassCastException. I can split PRs if we don't want to fix it in this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #4380: URL: https://github.com/apache/hive/pull/4380#issuecomment-1848429894 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4380) [](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=CODE_SMELL) [8 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4380=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4380=duplicated_lines_density=list) No Duplication information  The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarsource.com/sonarcloud/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
github-actions[bot] commented on PR #4380: URL: https://github.com/apache/hive/pull/4380#issuecomment-1839787048 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
github-actions[bot] closed pull request #4724: WIP URL: https://github.com/apache/hive/pull/4724 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27905: SPLIT throws ClassCastException [hive]
sonarcloud[bot] commented on PR #4896: URL: https://github.com/apache/hive/pull/4896#issuecomment-1825137297 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4896) [](https://sonarcloud.io/project/issues?id=apache_hive=4896=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4896=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4896=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4896=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4896=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4896=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4896=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4896=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4896=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4896=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4896=false=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4896=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4896=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4896=duplicated_lines_density=list) No Duplication information  The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarsource.com/sonarcloud/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
ayushtkn merged PR #4887: URL: https://github.com/apache/hive/pull/4887 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #4887: URL: https://github.com/apache/hive/pull/4887#issuecomment-1822765953 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4887) [](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4887=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4887=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4887=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4887=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4887=duplicated_lines_density=list) No Duplication information  The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarsource.com/sonarcloud/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #4887: URL: https://github.com/apache/hive/pull/4887#issuecomment-1821785767 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4887) [](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4887=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4887=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4887=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4887=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4887=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4887=duplicated_lines_density=list) No Duplication information  The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarsource.com/sonarcloud/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
[PR] WIP [hive]
ayushtkn opened a new pull request, #4887: URL: https://github.com/apache/hive/pull/4887 Just random stuff -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
github-actions[bot] commented on PR #4724: URL: https://github.com/apache/hive/pull/4724#issuecomment-1820016369 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-22390: Remove JODA Time [hive]
github-actions[bot] closed pull request #4652: [WIP] HIVE-22390: Remove JODA Time URL: https://github.com/apache/hive/pull/4652 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
sonarcloud[bot] commented on PR #4820: URL: https://github.com/apache/hive/pull/4820#issuecomment-1798362059 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4820) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [11 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4820=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4820=duplicated_lines_density=list) No Duplication information  The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
dengzhhu653 commented on code in PR #4820:
URL: https://github.com/apache/hive/pull/4820#discussion_r1384573860
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java:
##
@@ -908,23 +909,12 @@ public List alterPartitions(final RawStore
msdb, Warehouse wh, final
}
msdb.alterPartitions(catName, dbname, name, partValsList, new_parts,
writeId, writeIdList);
- Iterator oldPartsIt = oldParts.iterator();
- for (Partition newPart : new_parts) {
-Partition oldPart;
-if (oldPartsIt.hasNext()) {
- oldPart = oldPartsIt.next();
-} else {
- throw new InvalidOperationException("Missing old partition
corresponding to new partition " +
- "when invoking MetaStoreEventListener for alterPartitions
event.");
-}
-if (transactionalListeners != null &&
!transactionalListeners.isEmpty()) {
- MetaStoreListenerNotifier.notifyEvent(transactionalListeners,
EventMessage.EventType.ALTER_PARTITION,
- new AlterPartitionEvent(oldPart, newPart, tbl, false, true,
newPart.getWriteId(), handler),
- environmentContext);
-}
+ if (transactionalListeners != null && !transactionalListeners.isEmpty())
{
+MetaStoreListenerNotifier.notifyEvent(transactionalListeners,
EventMessage.EventType.ALTER_PARTITIONS,
Review Comment:
A new property "hive.metastore.alterPartitions.notification.v2.enabled" to
address the breaking changes, we may need to recycle this property in the future
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
saihemanth-cloudera commented on code in PR #4820:
URL: https://github.com/apache/hive/pull/4820#discussion_r1384331395
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java:
##
@@ -908,23 +909,12 @@ public List alterPartitions(final RawStore
msdb, Warehouse wh, final
}
msdb.alterPartitions(catName, dbname, name, partValsList, new_parts,
writeId, writeIdList);
- Iterator oldPartsIt = oldParts.iterator();
- for (Partition newPart : new_parts) {
-Partition oldPart;
-if (oldPartsIt.hasNext()) {
- oldPart = oldPartsIt.next();
-} else {
- throw new InvalidOperationException("Missing old partition
corresponding to new partition " +
- "when invoking MetaStoreEventListener for alterPartitions
event.");
-}
-if (transactionalListeners != null &&
!transactionalListeners.isEmpty()) {
- MetaStoreListenerNotifier.notifyEvent(transactionalListeners,
EventMessage.EventType.ALTER_PARTITION,
- new AlterPartitionEvent(oldPart, newPart, tbl, false, true,
newPart.getWriteId(), handler),
- environmentContext);
-}
+ if (transactionalListeners != null && !transactionalListeners.isEmpty())
{
+MetaStoreListenerNotifier.notifyEvent(transactionalListeners,
EventMessage.EventType.ALTER_PARTITIONS,
Review Comment:
Should we make this configurable? This is breaking interoperability.
Consumers of the notification log should update their code to consume this new
event. If it is configurable, older clients can process the ALTER_PARTITION
events and whenever they are ready to consume newer event ALTER_PARTITIONS,
then can flip the config.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-22390: Remove JODA Time [hive]
github-actions[bot] commented on PR #4652: URL: https://github.com/apache/hive/pull/4652#issuecomment-1789866218 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
sonarcloud[bot] commented on PR #4820: URL: https://github.com/apache/hive/pull/4820#issuecomment-1780406752 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4820) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [10 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4820=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4820=duplicated_lines_density=list) No Duplication information  The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
dengzhhu653 commented on code in PR #4820: URL: https://github.com/apache/hive/pull/4820#discussion_r1372492570 ## standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/events/AlterPartitionsEvent.java: ## Review Comment: The partitions in this class come from here: https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java#L853 Ideally it might contain the same two partitions in the list, but we also alter them twice, so I think we should generate the same number of logs. I think we can rename the class to `AlterPartitionListEvent`, what do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
dengzhhu653 commented on code in PR #4820:
URL: https://github.com/apache/hive/pull/4820#discussion_r1372484523
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/events/AlterPartitionsEvent.java:
##
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.metastore.events;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.hadoop.classification.InterfaceAudience;
+import org.apache.hadoop.classification.InterfaceStability;
+import org.apache.hadoop.hive.metastore.IHMSHandler;
+import org.apache.hadoop.hive.metastore.api.Partition;
+import org.apache.hadoop.hive.metastore.api.Table;
+
+import static java.util.Objects.requireNonNull;
+
+@InterfaceAudience.Public
+@InterfaceStability.Evolving
+public class AlterPartitionsEvent extends ListenerEvent {
+
+ private final List old_parts;
+ private final List new_parts;
+ private final Table table;
+ private final boolean isTruncateOp;
+
+ public AlterPartitionsEvent(List old_parts, List
new_parts, Table table,
+ boolean isTruncateOp, boolean status, IHMSHandler handler) {
+super(status, handler);
+this.old_parts = requireNonNull(old_parts, "old_parts is null");
+this.new_parts = requireNonNull(new_parts, "new_parts is null");
+this.table = requireNonNull(table, "table is null");
+this.isTruncateOp = isTruncateOp;
+ }
+
+ /**
+ * @return The table.
+ */
+ public Table getTable() {
+return table;
+ }
+
+ /**
+ * @return Iterator for new partitions.
+ */
+ public Iterator> getNewPartsIterator(int batchSize) {
+return iteratePartitions(new_parts, batchSize);
+ }
+
Review Comment:
The partition in `new_parts` might have different write ids, I would prefer
there is only one write id per a single notification event, so the function
groups the partitions by write id first, then iterate the group, for each
iteration on a group, there is a maximum number(e.g, the `batchSize`) of the
partitions returned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27825: Better error message for an empty quoted identifier [hive]
sonarcloud[bot] commented on PR #4829: URL: https://github.com/apache/hive/pull/4829#issuecomment-1779642366 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4829) [](https://sonarcloud.io/project/issues?id=apache_hive=4829=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4829=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4829=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4829=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4829=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4829=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4829=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4829=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4829=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4829=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4829=false=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4829=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4829=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4829=duplicated_lines_density=list) No Duplication information  The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
henrib commented on code in PR #4820:
URL: https://github.com/apache/hive/pull/4820#discussion_r1371746550
##
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/events/TestAlterPartitionsEvent.java:
##
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.metastore.events;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.IntStream;
+
+import org.apache.hadoop.hive.metastore.annotation.MetastoreUnitTest;
+import org.apache.hadoop.hive.metastore.api.Partition;
+import org.apache.hadoop.hive.metastore.api.Table;
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+@RunWith(Parameterized.class)
+@Category(MetastoreUnitTest.class)
+public class TestAlterPartitionsEvent {
+
+ private AlterPartitionsEvent event;
+ private List expectedParts;
+ private Map> writeIdToParts;
+ private int batchSize;
+ private int expectedBatch;
+
+ @Parameterized.Parameters
+ public static Collection getIteratorToTest() {
+List parts = new ArrayList<>();
+IntStream.range(0, 10).forEach(i -> {
+ Partition partition = new Partition();
+ partition.setValues(Arrays.asList(i + "", "part" + i));
+ partition.setWriteId(i < 5 ? 1 : 2);
+ parts.add(partition);
+});
+Partition partition = new Partition();
+partition.setValues(Arrays.asList(10 + "", "part" + 10));
+partition.setWriteId(3);
+parts.add(partition);
+
+AlterPartitionsEvent event = new AlterPartitionsEvent(parts, parts, new
Table(), false, true, null);
+Collection params = new ArrayList<>();
+params.add(new Object[]{event, 1000, parts, 3});
+params.add(new Object[]{event, 5, parts, 3});
+params.add(new Object[]{event, 3, parts, 5});
+params.add(new Object[]{event, 2, parts, 7});
+params.add(new Object[]{event, 1, parts, 11});
+params.add(new Object[]{event, -1, parts, 3});
+return params;
+ }
+
+ public TestAlterPartitionsEvent(AlterPartitionsEvent event, int batchSize,
+ List expectedParts, int expectedBatch) {
+this.event = event;
+this.batchSize = batchSize;
+this.expectedParts = expectedParts;
+this.expectedBatch = expectedBatch;
+this.writeIdToParts = new HashMap<>();
+expectedParts.stream().forEach(partition -> {
Review Comment:
computeIfAbsent(...)
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/events/AlterPartitionsEvent.java:
##
Review Comment:
After the fact but naming a class as a plural of another is asking for
trouble (AlterPartitionEvent vs AlterPartitionsEvent) since they are very
similar. AlterPartitionSetEvent for instance could be considered. Incidentally,
shouldn't these be sets, the list of partitions ? Can the same partition appear
twice in an event ? Shouldn't we see Iterator> ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
henrib commented on code in PR #4820:
URL: https://github.com/apache/hive/pull/4820#discussion_r1371729751
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/events/AlterPartitionsEvent.java:
##
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.metastore.events;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.hadoop.classification.InterfaceAudience;
+import org.apache.hadoop.classification.InterfaceStability;
+import org.apache.hadoop.hive.metastore.IHMSHandler;
+import org.apache.hadoop.hive.metastore.api.Partition;
+import org.apache.hadoop.hive.metastore.api.Table;
+
+import static java.util.Objects.requireNonNull;
+
+@InterfaceAudience.Public
+@InterfaceStability.Evolving
+public class AlterPartitionsEvent extends ListenerEvent {
+
+ private final List old_parts;
+ private final List new_parts;
+ private final Table table;
+ private final boolean isTruncateOp;
+
+ public AlterPartitionsEvent(List old_parts, List
new_parts, Table table,
+ boolean isTruncateOp, boolean status, IHMSHandler handler) {
+super(status, handler);
+this.old_parts = requireNonNull(old_parts, "old_parts is null");
+this.new_parts = requireNonNull(new_parts, "new_parts is null");
+this.table = requireNonNull(table, "table is null");
+this.isTruncateOp = isTruncateOp;
+ }
+
+ /**
+ * @return The table.
+ */
+ public Table getTable() {
+return table;
+ }
+
+ /**
+ * @return Iterator for new partitions.
+ */
+ public Iterator> getNewPartsIterator(int batchSize) {
+return iteratePartitions(new_parts, batchSize);
+ }
+
Review Comment:
A comment would be nice to understand why the inner hash map on
partition.getWriteid is necessary. Since we end up chunking the union of values
(aka group partitions by batch size) independently of the key, I fail to see
what we gain from it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
henrib commented on code in PR #4820:
URL: https://github.com/apache/hive/pull/4820#discussion_r1371643012
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/events/AlterPartitionsEvent.java:
##
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.metastore.events;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.hadoop.classification.InterfaceAudience;
+import org.apache.hadoop.classification.InterfaceStability;
+import org.apache.hadoop.hive.metastore.IHMSHandler;
+import org.apache.hadoop.hive.metastore.api.Partition;
+import org.apache.hadoop.hive.metastore.api.Table;
+
+import static java.util.Objects.requireNonNull;
+
+@InterfaceAudience.Public
+@InterfaceStability.Evolving
+public class AlterPartitionsEvent extends ListenerEvent {
+
+ private final List old_parts;
+ private final List new_parts;
+ private final Table table;
+ private final boolean isTruncateOp;
+
+ public AlterPartitionsEvent(List old_parts, List
new_parts, Table table,
+ boolean isTruncateOp, boolean status, IHMSHandler handler) {
+super(status, handler);
+this.old_parts = requireNonNull(old_parts, "old_parts is null");
+this.new_parts = requireNonNull(new_parts, "new_parts is null");
+this.table = requireNonNull(table, "table is null");
+this.isTruncateOp = isTruncateOp;
+ }
+
+ /**
+ * @return The table.
+ */
+ public Table getTable() {
+return table;
+ }
+
+ /**
+ * @return Iterator for new partitions.
+ */
+ public Iterator> getNewPartsIterator(int batchSize) {
+return iteratePartitions(new_parts, batchSize);
+ }
+
+ private Iterator> iteratePartitions(List
partitions,
+ final int batchSize) {
+Map> writeIdToParts = new HashMap<>();
+partitions.forEach(partition -> {
Review Comment:
Following lines could be simplified with:
writeIdToParts.computeIfAbsent(writeId, k -> new
ArrayList<>()).add(partition);
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
sonarcloud[bot] commented on PR #4820: URL: https://github.com/apache/hive/pull/4820#issuecomment-1778765168 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4820) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [10 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4820=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4820=duplicated_lines_density=list) No Duplication information  The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
dengzhhu653 commented on code in PR #4820:
URL: https://github.com/apache/hive/pull/4820#discussion_r1371079076
##
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/events/AlterPartitionsHandler.java:
##
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.parse.repl.dump.events;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hive.common.TableName;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.metastore.Warehouse;
+import org.apache.hadoop.hive.metastore.api.NotificationEvent;
+import org.apache.hadoop.hive.metastore.messaging.AlterPartitionsMessage;
+import org.apache.hadoop.hive.ql.metadata.Partition;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.parse.EximUtil;
+import org.apache.hadoop.hive.ql.parse.repl.DumpType;
+import org.apache.hadoop.hive.ql.parse.repl.dump.Utils;
+import org.apache.hadoop.hive.ql.parse.repl.load.DumpMetaData;
+
+public class AlterPartitionsHandler extends
AbstractEventHandler {
+ private final TableName tableName;
+ private final List> part_vals;
+ // the isTruncateOp is always false now
+ private final boolean isTruncateOp;
Review Comment:
Removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
dengzhhu653 commented on code in PR #4820:
URL: https://github.com/apache/hive/pull/4820#discussion_r1371078988
##
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/load/message/TableHandler.java:
##
@@ -113,9 +114,16 @@ private Tuple extract(Context context) throws
SemanticException {
writeId = alterTableMessage.getWriteId();
break;
case EVENT_ALTER_PARTITION:
-AlterPartitionMessage msg =
deserializer.getAlterPartitionMessage(context.dmd.getPayload());
-tableType = msg.getTableObj().getTableType();
-writeId = msg.getWriteId();
+try {
+ AlterPartitionMessage msg =
deserializer.getAlterPartitionMessage(context.dmd.getPayload());
+ tableType = msg.getTableObj().getTableType();
+ writeId = msg.getWriteId();
+} catch (Exception e) {
Review Comment:
There is a `deSerializeGenericString` function to decompress the string, I
try to inspect the json message to get the type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
dengzhhu653 commented on code in PR #4820:
URL: https://github.com/apache/hive/pull/4820#discussion_r1371064470
##
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/load/message/TableHandler.java:
##
@@ -113,9 +114,16 @@ private Tuple extract(Context context) throws
SemanticException {
writeId = alterTableMessage.getWriteId();
break;
case EVENT_ALTER_PARTITION:
-AlterPartitionMessage msg =
deserializer.getAlterPartitionMessage(context.dmd.getPayload());
-tableType = msg.getTableObj().getTableType();
-writeId = msg.getWriteId();
+try {
+ AlterPartitionMessage msg =
deserializer.getAlterPartitionMessage(context.dmd.getPayload());
+ tableType = msg.getTableObj().getTableType();
+ writeId = msg.getWriteId();
+} catch (Exception e) {
Review Comment:
I was thinking introduce another DumpType(EVENT_ALTER_PARTITIONS), it was a
bit risky as I don't find where the Repl load consumes such type, so I would
prefer using the old DumpType `EVENT_ALTER_PARTITION`.
From the `context.dmd.getPayload()`, an idea is to inspect the json message
to detect the event type, but it looks difficult when the message is Gzip
compressed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
jfsii commented on code in PR #4820: URL: https://github.com/apache/hive/pull/4820#discussion_r1370748708 ## standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/events/TestAlterPartitionsEvent.java: ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * Review Comment: remove the < p > and the one below (I think they are unneeded) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
jfsii commented on code in PR #4820:
URL: https://github.com/apache/hive/pull/4820#discussion_r1370641965
##
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/events/AlterPartitionsHandler.java:
##
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.parse.repl.dump.events;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hive.common.TableName;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.metastore.Warehouse;
+import org.apache.hadoop.hive.metastore.api.NotificationEvent;
+import org.apache.hadoop.hive.metastore.messaging.AlterPartitionsMessage;
+import org.apache.hadoop.hive.ql.metadata.Partition;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.parse.EximUtil;
+import org.apache.hadoop.hive.ql.parse.repl.DumpType;
+import org.apache.hadoop.hive.ql.parse.repl.dump.Utils;
+import org.apache.hadoop.hive.ql.parse.repl.load.DumpMetaData;
+
+public class AlterPartitionsHandler extends
AbstractEventHandler {
+ private final TableName tableName;
+ private final List> part_vals;
+ // the isTruncateOp is always false now
+ private final boolean isTruncateOp;
Review Comment:
Is there a follow up JIRA for this? I would just remove this until it is
implemented.
##
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/load/message/TableHandler.java:
##
@@ -113,9 +114,16 @@ private Tuple extract(Context context) throws
SemanticException {
writeId = alterTableMessage.getWriteId();
break;
case EVENT_ALTER_PARTITION:
-AlterPartitionMessage msg =
deserializer.getAlterPartitionMessage(context.dmd.getPayload());
-tableType = msg.getTableObj().getTableType();
-writeId = msg.getWriteId();
+try {
+ AlterPartitionMessage msg =
deserializer.getAlterPartitionMessage(context.dmd.getPayload());
+ tableType = msg.getTableObj().getTableType();
+ writeId = msg.getWriteId();
+} catch (Exception e) {
Review Comment:
Is there a better way to detect this? This looks like control flow via
exceptions which is typically inefficient and sometimes confusing.
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/messaging/AlterPartitionsMessage.java:
##
@@ -0,0 +1,64 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.metastore.messaging;
+
+import java.util.List;
+
+import org.apache.hadoop.hive.metastore.api.Partition;
+
+public abstract class AlterPartitionsMessage extends EventMessage {
+
+ protected AlterPartitionsMessage() {
+super(EventType.ALTER_PARTITIONS);
+ }
+
+ public abstract String getTable();
+
+ public abstract String getTableType();
+
+ public abstract boolean getIsTruncateOp();
+
+ public abstract Long getWriteId();
+
+ public abstract List getPartitionKeys();
+
+ public abstract List> getPartitionValues();
+
+ public abstract List getPartitionsAfter();
+
+ @Override
+ public EventMessage checkValid() {
+if (getTable() == null) {
+ throw new IllegalStateException("Table name unset.");
+}
+if (getPartitionKeys() == null || getPartitionKeys().isEmpty()) {
+ throw new IllegalStateException("Partition keys unset");
+}
+if (getPartitionValues() == null ||
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
sonarcloud[bot] commented on PR #4820: URL: https://github.com/apache/hive/pull/4820#issuecomment-1777306078 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4820) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [3 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [11 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4820=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4820=duplicated_lines_density=list) No Duplication information  The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
nrg4878 commented on PR #4820: URL: https://github.com/apache/hive/pull/4820#issuecomment-1777241427 @henrib Can you please review? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
nrg4878 commented on PR #4820: URL: https://github.com/apache/hive/pull/4820#issuecomment-1777229482 @jfsii Can you please review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
sonarcloud[bot] commented on PR #4820: URL: https://github.com/apache/hive/pull/4820#issuecomment-1776850465 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4820) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [3 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [9 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4820=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4820=duplicated_lines_density=list) No Duplication information  The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] [WIP] HIVE-27746: Hive Metastore should send single AlterPartitionEvent wit… [hive]
sonarcloud[bot] commented on PR #4820: URL: https://github.com/apache/hive/pull/4820#issuecomment-1774706046 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4820) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [3 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4820=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [9 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4820=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4820=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4820=duplicated_lines_density=list) No Duplication information  The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
Re: [PR] WIP [hive]
sonarcloud[bot] commented on PR #4380: URL: https://github.com/apache/hive/pull/4380#issuecomment-1749455756 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4380) [](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4380=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=CODE_SMELL) [23 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4380=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4380=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4380=duplicated_lines_density=list) No Duplication information  The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
[PR] WIP [hive]
ayushtkn opened a new pull request, #4380: URL: https://github.com/apache/hive/pull/4380 Try jenkins draft PR: Do Not review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
