[jira] [Work logged] (HIVE-2420) partition pruner expr is not populated due to some bug in ppd
[
https://issues.apache.org/jira/browse/HIVE-2420?focusedWorklogId=590159=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-590159
]
ASF GitHub Bot logged work on HIVE-2420:
Author: ASF GitHub Bot
Created on: 28/Apr/21 05:52
Start Date: 28/Apr/21 05:52
Worklog Time Spent: 10m
Work Description: Dawn2111 commented on a change in pull request #2065:
URL: https://github.com/apache/hive/pull/2065#discussion_r621834393
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/KillMoveTriggerActionHandler.java
##
@@ -47,8 +47,10 @@ public void applyAction(final Map
queriesViolated) {
break;
case MOVE_TO_POOL:
String destPoolName = entry.getValue().getAction().getPoolName();
- Future moveFuture = wm.applyMoveSessionAsync(wmTezSession,
destPoolName);
- moveFutures.put(wmTezSession, moveFuture);
+ if (!wmTezSession.isDelayedMove()) {
+Future moveFuture =
wm.applyMoveSessionAsync(wmTezSession, destPoolName);
Review comment:
Dont think we need to - any query being completed/killed in the
destination pool will create a return/kill event. This in turn will wake up
the master thread which will retry the delayed move in the same iteration of
the master thread loop. So the existing delayed moves will be processed earlier
than any subsequent move events.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 590159)
Time Spent: 50m (was: 40m)
> partition pruner expr is not populated due to some bug in ppd
> -
>
> Key: HIVE-2420
> URL: https://issues.apache.org/jira/browse/HIVE-2420
> Project: Hive
> Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: He Yongqiang
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-2420.reproduce.diff
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-2420) partition pruner expr is not populated due to some bug in ppd
[
https://issues.apache.org/jira/browse/HIVE-2420?focusedWorklogId=590158=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-590158
]
ASF GitHub Bot logged work on HIVE-2420:
Author: ASF GitHub Bot
Created on: 28/Apr/21 05:51
Start Date: 28/Apr/21 05:51
Worklog Time Spent: 10m
Work Description: Dawn2111 commented on a change in pull request #2065:
URL: https://github.com/apache/hive/pull/2065#discussion_r621834393
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/KillMoveTriggerActionHandler.java
##
@@ -47,8 +47,10 @@ public void applyAction(final Map
queriesViolated) {
break;
case MOVE_TO_POOL:
String destPoolName = entry.getValue().getAction().getPoolName();
- Future moveFuture = wm.applyMoveSessionAsync(wmTezSession,
destPoolName);
- moveFutures.put(wmTezSession, moveFuture);
+ if (!wmTezSession.isDelayedMove()) {
+Future moveFuture =
wm.applyMoveSessionAsync(wmTezSession, destPoolName);
Review comment:
I dont think we need to because any query being completed/killed in the
destination pool will create a return/kill event. This in turn will wake up
the master thread which will retry the delayed move in the same iteration of
the master thread loop. So the existing delayed moves will be processed earlier
than any subsequent move events.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 590158)
Time Spent: 40m (was: 0.5h)
> partition pruner expr is not populated due to some bug in ppd
> -
>
> Key: HIVE-2420
> URL: https://issues.apache.org/jira/browse/HIVE-2420
> Project: Hive
> Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: He Yongqiang
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-2420.reproduce.diff
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-2420) partition pruner expr is not populated due to some bug in ppd
[
https://issues.apache.org/jira/browse/HIVE-2420?focusedWorklogId=590145=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-590145
]
ASF GitHub Bot logged work on HIVE-2420:
Author: ASF GitHub Bot
Created on: 28/Apr/21 04:32
Start Date: 28/Apr/21 04:32
Worklog Time Spent: 10m
Work Description: Dawn2111 commented on a change in pull request #2065:
URL: https://github.com/apache/hive/pull/2065#discussion_r621709390
##
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java
##
@@ -790,45 +842,72 @@ private void dumpPoolState(PoolState ps, List
set) {
}
}
- private void handleMoveSessionOnMasterThread(final MoveSession moveSession,
-final WmThreadSyncWork syncWork,
-final HashSet poolsToRedistribute,
-final Map toReuse,
-final Map recordMoveEvents) {
+ private static enum MoveSessionResult {
+OK, // Normal case - the session was moved.
+KILLED, // Killed because destination pool was full and delayed move is
false.
+CONVERTED_TO_DELAYED_MOVE, // the move session was added to the pool's
delayed moves as the dest. pool was full
+// and delayed move is true.
+ERROR
+ }
+
+ private MoveSessionResult handleMoveSessionOnMasterThread(final MoveSession
moveSession,
+ final WmThreadSyncWork syncWork,
+ final HashSet poolsToRedistribute,
+ final Map toReuse,
+ final Map recordMoveEvents,
+ final boolean convertToDelayedMove) {
String destPoolName = moveSession.destPool;
-LOG.info("Handling move session event: {}", moveSession);
+LOG.info("Handling move session event: {}, Convert to Delayed Move: {}",
moveSession, convertToDelayedMove);
if (validMove(moveSession.srcSession, destPoolName)) {
+ String srcPoolName = moveSession.srcSession.getPoolName();
+ PoolState srcPool = pools.get(srcPoolName);
+ boolean capacityAvailableInDest = capacityAvailable(destPoolName);
+ // If delayed move is set to true and if destination pool doesn't have
enough capacity, don't kill the query.
+ // Let the query run in source pool. Add the session to the source
pool's delayed move sessions.
+ if (convertToDelayedMove && !capacityAvailableInDest) {
+srcPool.delayedMoveSessions.add(moveSession);
+moveSession.srcSession.setDelayedMove(true);
Review comment:
When a pool is updated or destroyed as a consequence of disabling WLM ,
all the sessions in the pool are removed and we remove the delayed move
sessions at that time as well - PoolState.extractAllSessionsToKill().
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 590145)
Time Spent: 0.5h (was: 20m)
> partition pruner expr is not populated due to some bug in ppd
> -
>
> Key: HIVE-2420
> URL: https://issues.apache.org/jira/browse/HIVE-2420
> Project: Hive
> Issue Type: Bug
>Reporter: He Yongqiang
>Assignee: He Yongqiang
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-2420.reproduce.diff
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24909) Skip the repl events from getting logged in notification log
[
https://issues.apache.org/jira/browse/HIVE-24909?focusedWorklogId=590117=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-590117
]
ASF GitHub Bot logged work on HIVE-24909:
-
Author: ASF GitHub Bot
Created on: 28/Apr/21 02:15
Start Date: 28/Apr/21 02:15
Worklog Time Spent: 10m
Work Description: pkumarsinha commented on a change in pull request #2101:
URL: https://github.com/apache/hive/pull/2101#discussion_r621757736
##
File path: ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DbTxnManager.java
##
@@ -495,6 +501,7 @@ public void replCommitTxn(CommitTxnRequest rqst) throws
LockException {
// For transaction started internally by repl load command, heartbeat
needs to be stopped.
clearLocksAndHB();
}
+ rqst.setTxn_type(TxnType.REPL_CREATED);
Review comment:
rqst object is getting created somewhere else, why not to set the type
right there?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 590117)
Time Spent: 6h 50m (was: 6h 40m)
> Skip the repl events from getting logged in notification log
>
>
> Key: HIVE-24909
> URL: https://issues.apache.org/jira/browse/HIVE-24909
> Project: Hive
> Issue Type: Bug
>Reporter: Haymant Mangla
>Assignee: Haymant Mangla
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> Currently REPL dump events are logged and replicated as a part of replication
> policy. Whenever one replication cycle completed, we always have one
> transaction left open on the target corresponding to repl dump operation.
> This will never be caught up without manually dealing with the transaction on
> target cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24909) Skip the repl events from getting logged in notification log
[
https://issues.apache.org/jira/browse/HIVE-24909?focusedWorklogId=590110=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-590110
]
ASF GitHub Bot logged work on HIVE-24909:
-
Author: ASF GitHub Bot
Created on: 28/Apr/21 02:04
Start Date: 28/Apr/21 02:04
Worklog Time Spent: 10m
Work Description: hmangla98 commented on a change in pull request #2101:
URL: https://github.com/apache/hive/pull/2101#discussion_r621754207
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenariosAcidTables.java
##
@@ -1645,15 +1644,20 @@ public void testHdfsMaxDirItemsLimitDuringIncremental()
throws Throwable {
WarehouseInstance.Tuple incrementalDump2 = primary.run("use " +
primaryDbName)
.dump(primaryDbName, dumpClause);
-eventCount = Integer.parseInt(incrementalDump2.lastReplicationId)
-- Integer.parseInt(incrementalDump1.lastReplicationId);
-assertTrue(eventCount > 5);
+eventCount = getNoOfEventsDumped(incrementalDump2.dumpLocation);
+assertTrue(eventCount > 5 && eventCount < 1000);
Review comment:
This test is to assert that no of events dumped cannot be greater than
(DFS_MAX_DIR_ITEMS_CONFIG - RESERVED_DIR_ITEMS_COUNT). For this particular run
of repl dump (line no. 1645), This value is 1000 which was initially 5 for the
previous run. So, no of events should be greater than 5 and less than 1000.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 590110)
Time Spent: 6h 40m (was: 6.5h)
> Skip the repl events from getting logged in notification log
>
>
> Key: HIVE-24909
> URL: https://issues.apache.org/jira/browse/HIVE-24909
> Project: Hive
> Issue Type: Bug
>Reporter: Haymant Mangla
>Assignee: Haymant Mangla
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 40m
> Remaining Estimate: 0h
>
> Currently REPL dump events are logged and replicated as a part of replication
> policy. Whenever one replication cycle completed, we always have one
> transaction left open on the target corresponding to repl dump operation.
> This will never be caught up without manually dealing with the transaction on
> target cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24909) Skip the repl events from getting logged in notification log
[
https://issues.apache.org/jira/browse/HIVE-24909?focusedWorklogId=590107=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-590107
]
ASF GitHub Bot logged work on HIVE-24909:
-
Author: ASF GitHub Bot
Created on: 28/Apr/21 01:59
Start Date: 28/Apr/21 01:59
Worklog Time Spent: 10m
Work Description: pkumarsinha commented on a change in pull request #2101:
URL: https://github.com/apache/hive/pull/2101#discussion_r621752489
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenariosAcidTables.java
##
@@ -1645,15 +1644,20 @@ public void testHdfsMaxDirItemsLimitDuringIncremental()
throws Throwable {
WarehouseInstance.Tuple incrementalDump2 = primary.run("use " +
primaryDbName)
.dump(primaryDbName, dumpClause);
-eventCount = Integer.parseInt(incrementalDump2.lastReplicationId)
-- Integer.parseInt(incrementalDump1.lastReplicationId);
-assertTrue(eventCount > 5);
+eventCount = getNoOfEventsDumped(incrementalDump2.dumpLocation);
+assertTrue(eventCount > 5 && eventCount < 1000);
Review comment:
What is the significance of 5 and 1000 here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 590107)
Time Spent: 6.5h (was: 6h 20m)
> Skip the repl events from getting logged in notification log
>
>
> Key: HIVE-24909
> URL: https://issues.apache.org/jira/browse/HIVE-24909
> Project: Hive
> Issue Type: Bug
>Reporter: Haymant Mangla
>Assignee: Haymant Mangla
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> Currently REPL dump events are logged and replicated as a part of replication
> policy. Whenever one replication cycle completed, we always have one
> transaction left open on the target corresponding to repl dump operation.
> This will never be caught up without manually dealing with the transaction on
> target cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24909) Skip the repl events from getting logged in notification log
[
https://issues.apache.org/jira/browse/HIVE-24909?focusedWorklogId=590106=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-590106
]
ASF GitHub Bot logged work on HIVE-24909:
-
Author: ASF GitHub Bot
Created on: 28/Apr/21 01:57
Start Date: 28/Apr/21 01:57
Worklog Time Spent: 10m
Work Description: hmangla98 commented on a change in pull request #2101:
URL: https://github.com/apache/hive/pull/2101#discussion_r621751668
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenariosExclusiveReplica.java
##
@@ -74,6 +75,28 @@ public void tearDown() throws Throwable {
super.tearDown();
}
+ @Test
+ public void testReplOperationsNotCapturedInNotificationLog() throws
Throwable {
+List withClauseOptions =
getStagingLocationConfig(replica.repldDir, true);
+withClauseOptions.add("'" + HiveConf.ConfVars.HIVE_IN_TEST_REPL.varname +
"'='" + false + "'");
+//Perform empty bootstrap dump and load
+primary.dump(primaryDbName, withClauseOptions);
+replica.load(replicatedDbName, primaryDbName, withClauseOptions);
+//Perform empty incremental dump and load so that all db level properties
are altered.
+primary.dump(primaryDbName, withClauseOptions);
+replica.load(replicatedDbName, primaryDbName, withClauseOptions);
+
+long lastEventId = primary.getCurrentNotificationEventId().getEventId();
+primary.dump(primaryDbName, withClauseOptions);
+long currentEventId = primary.getCurrentNotificationEventId().getEventId();
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 590106)
Time Spent: 6h 20m (was: 6h 10m)
> Skip the repl events from getting logged in notification log
>
>
> Key: HIVE-24909
> URL: https://issues.apache.org/jira/browse/HIVE-24909
> Project: Hive
> Issue Type: Bug
>Reporter: Haymant Mangla
>Assignee: Haymant Mangla
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 20m
> Remaining Estimate: 0h
>
> Currently REPL dump events are logged and replicated as a part of replication
> policy. Whenever one replication cycle completed, we always have one
> transaction left open on the target corresponding to repl dump operation.
> This will never be caught up without manually dealing with the transaction on
> target cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24909) Skip the repl events from getting logged in notification log
[
https://issues.apache.org/jira/browse/HIVE-24909?focusedWorklogId=590105=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-590105
]
ASF GitHub Bot logged work on HIVE-24909:
-
Author: ASF GitHub Bot
Created on: 28/Apr/21 01:56
Start Date: 28/Apr/21 01:56
Worklog Time Spent: 10m
Work Description: pkumarsinha commented on a change in pull request #2101:
URL: https://github.com/apache/hive/pull/2101#discussion_r621751133
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenariosExclusiveReplica.java
##
@@ -74,6 +75,28 @@ public void tearDown() throws Throwable {
super.tearDown();
}
+ @Test
+ public void testReplOperationsNotCapturedInNotificationLog() throws
Throwable {
+List withClauseOptions =
getStagingLocationConfig(replica.repldDir, true);
+withClauseOptions.add("'" + HiveConf.ConfVars.HIVE_IN_TEST_REPL.varname +
"'='" + false + "'");
+//Perform empty bootstrap dump and load
+primary.dump(primaryDbName, withClauseOptions);
+replica.load(replicatedDbName, primaryDbName, withClauseOptions);
+//Perform empty incremental dump and load so that all db level properties
are altered.
+primary.dump(primaryDbName, withClauseOptions);
+replica.load(replicatedDbName, primaryDbName, withClauseOptions);
+
+long lastEventId = primary.getCurrentNotificationEventId().getEventId();
+primary.dump(primaryDbName, withClauseOptions);
+long currentEventId = primary.getCurrentNotificationEventId().getEventId();
Review comment:
Probe that there is no event dir created?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 590105)
Time Spent: 6h 10m (was: 6h)
> Skip the repl events from getting logged in notification log
>
>
> Key: HIVE-24909
> URL: https://issues.apache.org/jira/browse/HIVE-24909
> Project: Hive
> Issue Type: Bug
>Reporter: Haymant Mangla
>Assignee: Haymant Mangla
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> Currently REPL dump events are logged and replicated as a part of replication
> policy. Whenever one replication cycle completed, we always have one
> transaction left open on the target corresponding to repl dump operation.
> This will never be caught up without manually dealing with the transaction on
> target cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-2420) partition pruner expr is not populated due to some bug in ppd
[
https://issues.apache.org/jira/browse/HIVE-2420?focusedWorklogId=590059=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-590059
]
ASF GitHub Bot logged work on HIVE-2420:
Author: ASF GitHub Bot
Created on: 28/Apr/21 00:07
Start Date: 28/Apr/21 00:07
Worklog Time Spent: 10m
Work Description: Dawn2111 commented on a change in pull request #2065:
URL: https://github.com/apache/hive/pull/2065#discussion_r621674045
##
File path:
ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestWorkloadManager.java
##
@@ -1110,6 +1110,94 @@ public void testMoveSessionsMultiPool() throws Exception
{
assertFalse(allSessionProviders.get("A").getSessions().contains(sessionA1));
}
+ @Test(timeout=1)
+ public void testDelayedMoveSessions() throws Exception {
+final HiveConf conf = createConfForDelayedMove();
+MockQam qam = new MockQam();
+WMFullResourcePlan plan = new WMFullResourcePlan(plan(),
Lists.newArrayList(
+pool("A", 2, 0.6f), pool("B", 1, 0.4f)));
+plan.setMappings(Lists.newArrayList(mapping("A", "A"), mapping("B", "B")));
+final WorkloadManager wm = new WorkloadManagerForTest("test", conf, qam,
plan);
+wm.start();
+
+WmTezSession sessionA1 = (WmTezSession) wm.getSession(null,
mappingInput("A"), conf);
+
+// [A: 1, B: 0]
+Map allSessionProviders =
wm.getAllSessionTriggerProviders();
+assertEquals(1, allSessionProviders.get("A").getSessions().size());
+assertEquals(0, allSessionProviders.get("B").getSessions().size());
+assertTrue(allSessionProviders.get("A").getSessions().contains(sessionA1));
+
assertFalse(allSessionProviders.get("B").getSessions().contains(sessionA1));
+assertEquals(0.6f, sessionA1.getClusterFraction(), EPSILON);
+assertEquals("A", sessionA1.getPoolName());
+
+// If dest pool has capacity, move immediately
+// [A: 0, B: 1]
+Future future = wm.applyMoveSessionAsync(sessionA1, "B");
+assertNotNull(future.get());
+assertTrue(future.get());
+wm.addTestEvent().get();
+allSessionProviders = wm.getAllSessionTriggerProviders();
+assertEquals(0, allSessionProviders.get("A").getSessions().size());
+assertEquals(1, allSessionProviders.get("B").getSessions().size());
+
assertFalse(allSessionProviders.get("A").getSessions().contains(sessionA1));
+assertTrue(allSessionProviders.get("B").getSessions().contains(sessionA1));
+assertEquals(0.4f, sessionA1.getClusterFraction(), EPSILON);
+assertEquals("B", sessionA1.getPoolName());
+
+WmTezSession sessionA2 = (WmTezSession) wm.getSession(null,
mappingInput("A"), conf);
+// [A: 1, B: 1]
+allSessionProviders = wm.getAllSessionTriggerProviders();
+assertEquals(1, allSessionProviders.get("A").getSessions().size());
+assertEquals(1, allSessionProviders.get("B").getSessions().size());
+assertTrue(allSessionProviders.get("A").getSessions().contains(sessionA2));
+assertTrue(allSessionProviders.get("B").getSessions().contains(sessionA1));
+assertEquals(0.6f, sessionA2.getClusterFraction(), EPSILON);
+assertEquals(0.4f, sessionA1.getClusterFraction(), EPSILON);
+assertEquals("A", sessionA2.getPoolName());
+assertEquals("B", sessionA1.getPoolName());
+
+// Dest pool is maxed out. Keep running in source pool
+// [A: 1, B: 1]
+future = wm.applyMoveSessionAsync(sessionA2, "B");
+assertNotNull(future.get());
+assertFalse(future.get());
+wm.addTestEvent().get();
+allSessionProviders = wm.getAllSessionTriggerProviders();
+assertEquals(1, allSessionProviders.get("A").getSessions().size());
+assertEquals(1, allSessionProviders.get("B").getSessions().size());
+assertTrue(allSessionProviders.get("A").getSessions().contains(sessionA2));
+assertTrue(allSessionProviders.get("B").getSessions().contains(sessionA1));
+assertEquals(0.6f, sessionA2.getClusterFraction(), EPSILON);
+assertEquals(0.4f, sessionA1.getClusterFraction(), EPSILON);
+assertEquals("A", sessionA2.getPoolName());
+assertEquals("B", sessionA1.getPoolName());
+
+// A has queued requests. The new requests should get accepted. The
delayed move should be killed
+WmTezSession sessionA3 = (WmTezSession) wm.getSession(null,
mappingInput("A"), conf);
+WmTezSession sessionA4 = (WmTezSession) wm.getSession(null,
mappingInput("A"), conf);
+
+while(sessionA2.isOpen()) {
+ Thread.sleep(100);
+}
+assertNull(sessionA2.getPoolName());
+assertEquals("Destination pool B is full. Killing query.",
sessionA2.getReasonForKill());
+
+// [A: 2, B: 1]
+allSessionProviders = wm.getAllSessionTriggerProviders();
+assertEquals(2, allSessionProviders.get("A").getSessions().size());
+assertEquals(1, allSessionProviders.get("B").getSessions().size());
+
[jira] [Commented] (HIVE-25060) Hive Compactor doesn´t launch cleaner
[ https://issues.apache.org/jira/browse/HIVE-25060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17333441#comment-17333441 ] Peter Vary commented on HIVE-25060: --- I am not sure when someone can pick up this issue, but whenever it happens the full stack trace will be needed for the message "Max block location exceeded for split", and if the Cleaner is not started there also should be some exception in the logs, which could be useful too. > Hive Compactor doesn´t launch cleaner > - > > Key: HIVE-25060 > URL: https://issues.apache.org/jira/browse/HIVE-25060 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.1 > Environment: Hive 3.1.0 > Hadoop 3.1.1 >Reporter: Fran Gonzalez >Priority: Major > > Hello, > there are problems with Hive Compactor. We can see in hivemetastore.log this > message "Max block location exceeded for split" and it´s appearing more and > more times. > After that, the "compactor.Cleaner" is not launched. > We observed that after a Hive Metastore restart, the "compactor.Cleaner" has > not been launched nevermore, but logs doesn´t display any message about it. > Could be a degradation of the Hive Compactor when delta files are growing in > the partitions? > Regards. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25062) Iceberg: Fix date partition transform insert issue
[
https://issues.apache.org/jira/browse/HIVE-25062?focusedWorklogId=589917=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589917
]
ASF GitHub Bot logged work on HIVE-25062:
-
Author: ASF GitHub Bot
Created on: 27/Apr/21 18:02
Start Date: 27/Apr/21 18:02
Worklog Time Spent: 10m
Work Description: pvary opened a new pull request #2228:
URL: https://github.com/apache/hive/pull/2228
### What changes were proposed in this pull request?
Wrap the Record before calculating partitions
### Why are the changes needed?
Timestamp/Date needs values converted to long/int respectively to calculate
partitions
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Added new unit tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 589917)
Remaining Estimate: 0h
Time Spent: 10m
> Iceberg: Fix date partition transform insert issue
> --
>
> Key: HIVE-25062
> URL: https://issues.apache.org/jira/browse/HIVE-25062
> Project: Hive
> Issue Type: Bug
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{Repro steps:}}
> {code:java}
> CREATE EXTERNAL TABLE iceberg_hive_part (id int, part_field date)STORED BY
> 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler'
> TBLPROPERTIES (
> 'iceberg.mr.table.partition.spec'='{"spec-id":0,"fields":[{"name":"part_year","transform":"year","source-id":1,"field-id":1001}]}'
> ,'write.format.default'='PARQUET'){code}
> {code:java}
> INSERT INTO iceberg_hive_part values(1, cast('2021-04-20' as date))
> {code}
> throws:
> {code:java}
> (Not an instance of java.lang.Integer: 2021-04-20){code}
> Add unit tests covering partition transform reads/writes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25062) Iceberg: Fix date partition transform insert issue
[
https://issues.apache.org/jira/browse/HIVE-25062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-25062:
--
Labels: pull-request-available (was: )
> Iceberg: Fix date partition transform insert issue
> --
>
> Key: HIVE-25062
> URL: https://issues.apache.org/jira/browse/HIVE-25062
> Project: Hive
> Issue Type: Bug
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{Repro steps:}}
> {code:java}
> CREATE EXTERNAL TABLE iceberg_hive_part (id int, part_field date)STORED BY
> 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler'
> TBLPROPERTIES (
> 'iceberg.mr.table.partition.spec'='{"spec-id":0,"fields":[{"name":"part_year","transform":"year","source-id":1,"field-id":1001}]}'
> ,'write.format.default'='PARQUET'){code}
> {code:java}
> INSERT INTO iceberg_hive_part values(1, cast('2021-04-20' as date))
> {code}
> throws:
> {code:java}
> (Not an instance of java.lang.Integer: 2021-04-20){code}
> Add unit tests covering partition transform reads/writes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25064) Create separate shader maven module for Iceberg libraries
[ https://issues.apache.org/jira/browse/HIVE-25064?focusedWorklogId=589889=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589889 ] ASF GitHub Bot logged work on HIVE-25064: - Author: ASF GitHub Bot Created on: 27/Apr/21 17:24 Start Date: 27/Apr/21 17:24 Worklog Time Spent: 10m Work Description: pvary commented on pull request #2226: URL: https://github.com/apache/hive/pull/2226#issuecomment-827778624 If this solves the compilation problem in the IntelliJ as well, then we can remove the `iceberg` flag introduced by #2188. Or shall we revert it in a different PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 589889) Time Spent: 50m (was: 40m) > Create separate shader maven module for Iceberg libraries > - > > Key: HIVE-25064 > URL: https://issues.apache.org/jira/browse/HIVE-25064 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25064) Create separate shader maven module for Iceberg libraries
[
https://issues.apache.org/jira/browse/HIVE-25064?focusedWorklogId=589884=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589884
]
ASF GitHub Bot logged work on HIVE-25064:
-
Author: ASF GitHub Bot
Created on: 27/Apr/21 17:16
Start Date: 27/Apr/21 17:16
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #2226:
URL: https://github.com/apache/hive/pull/2226#discussion_r621428910
##
File path: iceberg/iceberg-shading/pom.xml
##
@@ -0,0 +1,148 @@
+
+
+http://maven.apache.org/POM/4.0.0;
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+
+org.apache.hive
+hive-iceberg
+4.0.0-SNAPSHOT
+../pom.xml
+
+4.0.0
+
+hive-iceberg-shading
+jar
+Hive Iceberg Shading
+
+This module collects all required Iceberg modules and their
dependencies, and shades them.
+Consuming modules only need to mark this module as their dependency to
get any required Iceberg libs.
+This module has no sources of its own, only the built jar file
contains the gathered and shaded libs.
+For Intellij this module is recommended to be ignored, to avoid the
dependent modules considering this as a
+module dependency, they should see this as a library dependency
instead.
+
+
+
+../..
+..
+
org.apache.hive.iceberg
+
+
+
+
+org.apache.iceberg
+patched-iceberg-core
+
patched-${iceberg-api.version}-${project.parent.version}
Review comment:
Could we move this to the parent pom?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 589884)
Time Spent: 40m (was: 0.5h)
> Create separate shader maven module for Iceberg libraries
> -
>
> Key: HIVE-25064
> URL: https://issues.apache.org/jira/browse/HIVE-25064
> Project: Hive
> Issue Type: Improvement
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25064) Create separate shader maven module for Iceberg libraries
[ https://issues.apache.org/jira/browse/HIVE-25064?focusedWorklogId=589882=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589882 ] ASF GitHub Bot logged work on HIVE-25064: - Author: ASF GitHub Bot Created on: 27/Apr/21 17:15 Start Date: 27/Apr/21 17:15 Worklog Time Spent: 10m Work Description: pvary commented on a change in pull request #2226: URL: https://github.com/apache/hive/pull/2226#discussion_r621428241 ## File path: iceberg/iceberg-handler/pom.xml ## @@ -120,95 +93,6 @@ Review comment: Do we need the `` here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 589882) Time Spent: 0.5h (was: 20m) > Create separate shader maven module for Iceberg libraries > - > > Key: HIVE-25064 > URL: https://issues.apache.org/jira/browse/HIVE-25064 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25064) Create separate shader maven module for Iceberg libraries
[
https://issues.apache.org/jira/browse/HIVE-25064?focusedWorklogId=589881=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589881
]
ASF GitHub Bot logged work on HIVE-25064:
-
Author: ASF GitHub Bot
Created on: 27/Apr/21 17:14
Start Date: 27/Apr/21 17:14
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #2226:
URL: https://github.com/apache/hive/pull/2226#discussion_r621427177
##
File path: iceberg/iceberg-catalog/pom.xml
##
@@ -21,9 +21,9 @@
-org.apache.iceberg
-iceberg-core
-true
+org.apache.hive
+hive-iceberg-shading
+${project.parent.version}
Review comment:
Could we move this to the parent pom?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 589881)
Time Spent: 20m (was: 10m)
> Create separate shader maven module for Iceberg libraries
> -
>
> Key: HIVE-25064
> URL: https://issues.apache.org/jira/browse/HIVE-25064
> Project: Hive
> Issue Type: Improvement
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25063) Enforce hive.default.nulls.last when enforce bucketing
[
https://issues.apache.org/jira/browse/HIVE-25063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-25063:
--
Labels: pull-request-available (was: )
> Enforce hive.default.nulls.last when enforce bucketing
> --
>
> Key: HIVE-25063
> URL: https://issues.apache.org/jira/browse/HIVE-25063
> Project: Hive
> Issue Type: Bug
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When creating ReduceSink operator for bucketing the sort key null sort order
> is hardcoded:
> {code}
> for (int sortOrder : sortOrders) {
> order.append(DirectionUtils.codeToSign(sortOrder));
> nullOrder.append(sortOrder == DirectionUtils.ASCENDING_CODE ? 'a' :
> 'z');
> }
> {code}
> It should depend on both the setting hive.default.nulls.last and the order
> direction.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25063) Enforce hive.default.nulls.last when enforce bucketing
[
https://issues.apache.org/jira/browse/HIVE-25063?focusedWorklogId=589839=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589839
]
ASF GitHub Bot logged work on HIVE-25063:
-
Author: ASF GitHub Bot
Created on: 27/Apr/21 16:04
Start Date: 27/Apr/21 16:04
Worklog Time Spent: 10m
Work Description: kasakrisz opened a new pull request #2227:
URL: https://github.com/apache/hive/pull/2227
### What changes were proposed in this pull request?
Setup null sort order based on the value of the setting
`hive.default.nulls.last` and the order direction when creating ReduceSink
operator for bucketing.
### Why are the changes needed?
Null sort order was setup based on the order direction only which may lead
to unexpected behavior.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Run existing tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 589839)
Remaining Estimate: 0h
Time Spent: 10m
> Enforce hive.default.nulls.last when enforce bucketing
> --
>
> Key: HIVE-25063
> URL: https://issues.apache.org/jira/browse/HIVE-25063
> Project: Hive
> Issue Type: Bug
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When creating ReduceSink operator for bucketing the sort key null sort order
> is hardcoded:
> {code}
> for (int sortOrder : sortOrders) {
> order.append(DirectionUtils.codeToSign(sortOrder));
> nullOrder.append(sortOrder == DirectionUtils.ASCENDING_CODE ? 'a' :
> 'z');
> }
> {code}
> It should depend on both the setting hive.default.nulls.last and the order
> direction.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25064) Create separate shader maven module for Iceberg libraries
[ https://issues.apache.org/jira/browse/HIVE-25064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25064: -- Labels: pull-request-available (was: ) > Create separate shader maven module for Iceberg libraries > - > > Key: HIVE-25064 > URL: https://issues.apache.org/jira/browse/HIVE-25064 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25064) Create separate shader maven module for Iceberg libraries
[ https://issues.apache.org/jira/browse/HIVE-25064?focusedWorklogId=589832=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589832 ] ASF GitHub Bot logged work on HIVE-25064: - Author: ASF GitHub Bot Created on: 27/Apr/21 15:47 Start Date: 27/Apr/21 15:47 Worklog Time Spent: 10m Work Description: szlta opened a new pull request #2226: URL: https://github.com/apache/hive/pull/2226 - create separate shader maven module for Iceberg libraries - fix transitive dependency issue (e.g. caffeine version 2.8.4 is bundled in the shaded jar instead of 2.7.0) - allow easier debugging of iceberg-handler and iceberg-catalog modules - allow referencing two separate versions of libraries that both Hive and Iceberg use, such as Parquet,Avro,Orc.. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 589832) Remaining Estimate: 0h Time Spent: 10m > Create separate shader maven module for Iceberg libraries > - > > Key: HIVE-25064 > URL: https://issues.apache.org/jira/browse/HIVE-25064 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25064) Create separate shader maven module for Iceberg libraries
[ https://issues.apache.org/jira/browse/HIVE-25064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita reassigned HIVE-25064: - > Create separate shader maven module for Iceberg libraries > - > > Key: HIVE-25064 > URL: https://issues.apache.org/jira/browse/HIVE-25064 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25063) Enforce hive.default.nulls.last when enforce bucketing
[
https://issues.apache.org/jira/browse/HIVE-25063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa reassigned HIVE-25063:
-
> Enforce hive.default.nulls.last when enforce bucketing
> --
>
> Key: HIVE-25063
> URL: https://issues.apache.org/jira/browse/HIVE-25063
> Project: Hive
> Issue Type: Bug
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
>
> When creating ReduceSink operator for bucketing the sort key null sort order
> is hardcoded:
> {code}
> for (int sortOrder : sortOrders) {
> order.append(DirectionUtils.codeToSign(sortOrder));
> nullOrder.append(sortOrder == DirectionUtils.ASCENDING_CODE ? 'a' :
> 'z');
> }
> {code}
> It should depend on both the setting hive.default.nulls.last and the order
> direction.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25057) Implement rollback for hive to iceberg migration
[
https://issues.apache.org/jira/browse/HIVE-25057?focusedWorklogId=589804=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589804
]
ASF GitHub Bot logged work on HIVE-25057:
-
Author: ASF GitHub Bot
Created on: 27/Apr/21 15:18
Start Date: 27/Apr/21 15:18
Worklog Time Spent: 10m
Work Description: lcspinter commented on a change in pull request #2219:
URL: https://github.com/apache/hive/pull/2219#discussion_r621314290
##
File path:
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergMetaHook.java
##
@@ -254,6 +263,35 @@ public void
commitAlterTable(org.apache.hadoop.hive.metastore.api.Table hmsTable
}
}
+ @Override
+ public void rollbackAlterTable(org.apache.hadoop.hive.metastore.api.Table
hmsTable, EnvironmentContext context)
+ throws MetaException {
+if
(Boolean.valueOf(context.getProperties().getOrDefault(MIGRATE_HIVE_TO_ICEBERG,
"false"))) {
+ LOG.debug("Initiating rollback for table {} at location {}",
+ hmsTable.getTableName(), hmsTable.getSd().getLocation());
+ context.getProperties().put(INITIALIZE_ROLLBACK_MIGRATION, "true");
+ this.catalogProperties = getCatalogProperties(hmsTable);
+ try {
+this.icebergTable = Catalogs.loadTable(conf, catalogProperties);
+ } catch (NoSuchTableException nte) {
+// iceberg table was not yet created, no need to delete the metadata
dir separately
+return;
+ }
+
+ // we want to keep the data files but get rid of the metadata directory
+ hmsTable.getParameters().put(InputFormatConfig.EXTERNAL_TABLE_PURGE,
"FALSE");
Review comment:
I'm confused too :). This shouldn't be there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 589804)
Time Spent: 6h 50m (was: 6h 40m)
> Implement rollback for hive to iceberg migration
>
>
> Key: HIVE-25057
> URL: https://issues.apache.org/jira/browse/HIVE-25057
> Project: Hive
> Issue Type: Improvement
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> This is a follow-up Jira of HIVE-25008.
> In case of an error during hive to iceberg migration, the original hive table
> must be restored.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25057) Implement rollback for hive to iceberg migration
[
https://issues.apache.org/jira/browse/HIVE-25057?focusedWorklogId=589760=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589760
]
ASF GitHub Bot logged work on HIVE-25057:
-
Author: ASF GitHub Bot
Created on: 27/Apr/21 14:01
Start Date: 27/Apr/21 14:01
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #2219:
URL: https://github.com/apache/hive/pull/2219#discussion_r621238461
##
File path:
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergMetaHook.java
##
@@ -254,6 +263,35 @@ public void
commitAlterTable(org.apache.hadoop.hive.metastore.api.Table hmsTable
}
}
+ @Override
+ public void rollbackAlterTable(org.apache.hadoop.hive.metastore.api.Table
hmsTable, EnvironmentContext context)
+ throws MetaException {
+if
(Boolean.valueOf(context.getProperties().getOrDefault(MIGRATE_HIVE_TO_ICEBERG,
"false"))) {
+ LOG.debug("Initiating rollback for table {} at location {}",
+ hmsTable.getTableName(), hmsTable.getSd().getLocation());
+ context.getProperties().put(INITIALIZE_ROLLBACK_MIGRATION, "true");
+ this.catalogProperties = getCatalogProperties(hmsTable);
+ try {
+this.icebergTable = Catalogs.loadTable(conf, catalogProperties);
+ } catch (NoSuchTableException nte) {
+// iceberg table was not yet created, no need to delete the metadata
dir separately
+return;
+ }
+
+ // we want to keep the data files but get rid of the metadata directory
+ hmsTable.getParameters().put(InputFormatConfig.EXTERNAL_TABLE_PURGE,
"FALSE");
Review comment:
Maybe I'm confused, but I thought this was to be removed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 589760)
Time Spent: 6h 40m (was: 6.5h)
> Implement rollback for hive to iceberg migration
>
>
> Key: HIVE-25057
> URL: https://issues.apache.org/jira/browse/HIVE-25057
> Project: Hive
> Issue Type: Improvement
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 40m
> Remaining Estimate: 0h
>
> This is a follow-up Jira of HIVE-25008.
> In case of an error during hive to iceberg migration, the original hive table
> must be restored.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25057) Implement rollback for hive to iceberg migration
[
https://issues.apache.org/jira/browse/HIVE-25057?focusedWorklogId=589757=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589757
]
ASF GitHub Bot logged work on HIVE-25057:
-
Author: ASF GitHub Bot
Created on: 27/Apr/21 13:58
Start Date: 27/Apr/21 13:58
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #2219:
URL: https://github.com/apache/hive/pull/2219#discussion_r621235567
##
File path:
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java
##
@@ -1158,32 +1172,39 @@ private void validateBasicStats(Table icebergTable,
String dbName, String tableN
}
private void validateMigration(String tableName) throws TException,
InterruptedException {
-List originalResult = shell.executeStatement("SELECT * FROM " +
tableName);
+List originalResult = shell.executeStatement("SELECT * FROM " +
tableName + " ORDER BY a");
shell.executeStatement("ALTER TABLE " + tableName + " SET TBLPROPERTIES " +
"('storage_handler'='org.apache.iceberg.mr.hive.HiveIcebergStorageHandler')");
-List alterResult = shell.executeStatement("SELECT * FROM " +
tableName);
+List alterResult = shell.executeStatement("SELECT * FROM " +
tableName + " ORDER BY a");
Assert.assertEquals(originalResult.size(), alterResult.size());
for (int i = 0; i < originalResult.size(); i++) {
- Arrays.equals(originalResult.get(i), alterResult.get(i));
+ Assert.assertTrue(Arrays.equals(originalResult.get(i),
alterResult.get(i)));
}
validateSd(tableName, "iceberg");
}
private void validateMigrationRollback(String tableName) throws TException,
InterruptedException {
-List originalResult = shell.executeStatement("SELECT * FROM " +
tableName);
+List originalResult = shell.executeStatement("SELECT * FROM " +
tableName + " ORDER BY a");
try (MockedStatic mockedTableUtil =
Mockito.mockStatic(HiveTableUtil.class)) {
mockedTableUtil.when(() ->
HiveTableUtil.importFiles(ArgumentMatchers.anyString(),
ArgumentMatchers.anyString(),
ArgumentMatchers.any(PartitionSpecProxy.class),
ArgumentMatchers.anyList(),
ArgumentMatchers.any(Properties.class),
ArgumentMatchers.any(Configuration.class)))
.thenThrow(new MetaException());
- shell.executeStatement("ALTER TABLE " + tableName + " SET TBLPROPERTIES
" +
-
"('storage_handler'='org.apache.iceberg.mr.hive.HiveIcebergStorageHandler')");
- List alterResult = shell.executeStatement("SELECT * FROM " +
tableName);
- Assert.assertEquals(originalResult.size(), alterResult.size());
- for (int i = 0; i < originalResult.size(); i++) {
-Arrays.equals(originalResult.get(i), alterResult.get(i));
+ try {
+shell.executeStatement("ALTER TABLE " + tableName + " SET
TBLPROPERTIES " +
+
"('storage_handler'='org.apache.iceberg.mr.hive.HiveIcebergStorageHandler')");
+ } catch (IllegalArgumentException e) {
+Assert.assertTrue(e.getMessage().contains("Error occurred during hive
table migration to iceberg."));
+shell.executeStatement("MSCK REPAIR TABLE " + tableName);
+List alterResult = shell.executeStatement("SELECT * FROM " +
tableName + " ORDER BY a");
+Assert.assertEquals(originalResult.size(), alterResult.size());
+for (int i = 0; i < originalResult.size(); i++) {
+ Assert.assertTrue(Arrays.equals(originalResult.get(i),
alterResult.get(i)));
+}
+validateSd(tableName, fileFormat.name());
Review comment:
nit: I think this sd validation check might belong logically before the
msck repair command. In case the sd wasn't reverted correctly, we'd get a
failure during the above select which could be harder to figure out why
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 589757)
Time Spent: 6.5h (was: 6h 20m)
> Implement rollback for hive to iceberg migration
>
>
> Key: HIVE-25057
> URL: https://issues.apache.org/jira/browse/HIVE-25057
> Project: Hive
> Issue Type: Improvement
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> This is a follow-up Jira of HIVE-25008.
> In case of an error during hive to iceberg migration, the original hive table
> must be restored.
--
This message was sent by
[jira] [Work logged] (HIVE-25057) Implement rollback for hive to iceberg migration
[ https://issues.apache.org/jira/browse/HIVE-25057?focusedWorklogId=589753=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589753 ] ASF GitHub Bot logged work on HIVE-25057: - Author: ASF GitHub Bot Created on: 27/Apr/21 13:52 Start Date: 27/Apr/21 13:52 Worklog Time Spent: 10m Work Description: marton-bod commented on a change in pull request #2219: URL: https://github.com/apache/hive/pull/2219#discussion_r621229746 ## File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/table/AbstractAlterTableOperation.java ## @@ -146,22 +147,23 @@ private void finalizeAlterTableWithWriteIdOp(Table table, Table oldTable, List Implement rollback for hive to iceberg migration > > > Key: HIVE-25057 > URL: https://issues.apache.org/jira/browse/HIVE-25057 > Project: Hive > Issue Type: Improvement >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > Labels: pull-request-available > Time Spent: 6h 20m > Remaining Estimate: 0h > > This is a follow-up Jira of HIVE-25008. > In case of an error during hive to iceberg migration, the original hive table > must be restored. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25057) Implement rollback for hive to iceberg migration
[ https://issues.apache.org/jira/browse/HIVE-25057?focusedWorklogId=589752=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589752 ] ASF GitHub Bot logged work on HIVE-25057: - Author: ASF GitHub Bot Created on: 27/Apr/21 13:51 Start Date: 27/Apr/21 13:51 Worklog Time Spent: 10m Work Description: marton-bod commented on a change in pull request #2219: URL: https://github.com/apache/hive/pull/2219#discussion_r621229448 ## File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/table/AbstractAlterTableOperation.java ## @@ -28,6 +28,7 @@ import org.apache.hadoop.hive.metastore.HiveMetaHook; import org.apache.hadoop.hive.metastore.Msck; import org.apache.hadoop.hive.metastore.MsckInfo; +import org.apache.hadoop.hive.metastore.PartitionIterable; Review comment: is this needed somewhere? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 589752) Time Spent: 6h 10m (was: 6h) > Implement rollback for hive to iceberg migration > > > Key: HIVE-25057 > URL: https://issues.apache.org/jira/browse/HIVE-25057 > Project: Hive > Issue Type: Improvement >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > Labels: pull-request-available > Time Spent: 6h 10m > Remaining Estimate: 0h > > This is a follow-up Jira of HIVE-25008. > In case of an error during hive to iceberg migration, the original hive table > must be restored. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25057) Implement rollback for hive to iceberg migration
[ https://issues.apache.org/jira/browse/HIVE-25057?focusedWorklogId=589740=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589740 ] ASF GitHub Bot logged work on HIVE-25057: - Author: ASF GitHub Bot Created on: 27/Apr/21 13:37 Start Date: 27/Apr/21 13:37 Worklog Time Spent: 10m Work Description: lcspinter commented on a change in pull request #2219: URL: https://github.com/apache/hive/pull/2219#discussion_r621215726 ## File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/table/AbstractAlterTableOperation.java ## @@ -138,8 +141,32 @@ private void finalizeAlterTableWithWriteIdOp(Table table, Table oldTable, List Implement rollback for hive to iceberg migration > > > Key: HIVE-25057 > URL: https://issues.apache.org/jira/browse/HIVE-25057 > Project: Hive > Issue Type: Improvement >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > Labels: pull-request-available > Time Spent: 6h > Remaining Estimate: 0h > > This is a follow-up Jira of HIVE-25008. > In case of an error during hive to iceberg migration, the original hive table > must be restored. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-25061) PTF: Improve BoundaryCache
[ https://issues.apache.org/jira/browse/HIVE-25061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17333210#comment-17333210 ] László Bodor commented on HIVE-25061: - the first version contains a simple implementation of floorEntry and ceilingEntry on a HashMap > PTF: Improve BoundaryCache > -- > > Key: HIVE-25061 > URL: https://issues.apache.org/jira/browse/HIVE-25061 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Attachments: Screen Shot 2021-04-27 at 1.02.37 PM.png > > Time Spent: 10m > Remaining Estimate: 0h > > First, I need to check whether TreeMap is really needed for our case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25062) Iceberg: Fix date partition transform insert issue
[
https://issues.apache.org/jira/browse/HIVE-25062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary reassigned HIVE-25062:
-
> Iceberg: Fix date partition transform insert issue
> --
>
> Key: HIVE-25062
> URL: https://issues.apache.org/jira/browse/HIVE-25062
> Project: Hive
> Issue Type: Bug
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
>
> {{Repro steps:}}
> {code:java}
> CREATE EXTERNAL TABLE iceberg_hive_part (id int, part_field date)STORED BY
> 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler'
> TBLPROPERTIES (
> 'iceberg.mr.table.partition.spec'='{"spec-id":0,"fields":[{"name":"part_year","transform":"year","source-id":1,"field-id":1001}]}'
> ,'write.format.default'='PARQUET'){code}
> {code:java}
> INSERT INTO iceberg_hive_part values(1, cast('2021-04-20' as date))
> {code}
> throws:
> {code:java}
> (Not an instance of java.lang.Integer: 2021-04-20){code}
> Add unit tests covering partition transform reads/writes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-24962) Enable partition pruning for Iceberg tables
[ https://issues.apache.org/jira/browse/HIVE-24962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary resolved HIVE-24962. --- Fix Version/s: 4.0.0 Resolution: Fixed Pushed to master. Thanks for the review [~lpinter] and [~mbod]! > Enable partition pruning for Iceberg tables > --- > > Key: HIVE-24962 > URL: https://issues.apache.org/jira/browse/HIVE-24962 > Project: Hive > Issue Type: Improvement >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 5h 20m > Remaining Estimate: 0h > > We should enable partition pruning above iceberg tables -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25061) PTF: Improve BoundaryCache
[ https://issues.apache.org/jira/browse/HIVE-25061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25061: -- Labels: pull-request-available (was: ) > PTF: Improve BoundaryCache > -- > > Key: HIVE-25061 > URL: https://issues.apache.org/jira/browse/HIVE-25061 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Attachments: Screen Shot 2021-04-27 at 1.02.37 PM.png > > Time Spent: 10m > Remaining Estimate: 0h > > First, I need to check whether TreeMap is really needed for our case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25061) PTF: Improve BoundaryCache
[ https://issues.apache.org/jira/browse/HIVE-25061?focusedWorklogId=589707=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589707 ] ASF GitHub Bot logged work on HIVE-25061: - Author: ASF GitHub Bot Created on: 27/Apr/21 12:46 Start Date: 27/Apr/21 12:46 Worklog Time Spent: 10m Work Description: abstractdog opened a new pull request #2225: URL: https://github.com/apache/hive/pull/2225 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 589707) Remaining Estimate: 0h Time Spent: 10m > PTF: Improve BoundaryCache > -- > > Key: HIVE-25061 > URL: https://issues.apache.org/jira/browse/HIVE-25061 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: Screen Shot 2021-04-27 at 1.02.37 PM.png > > Time Spent: 10m > Remaining Estimate: 0h > > First, I need to check whether TreeMap is really needed for our case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24962) Enable partition pruning for Iceberg tables
[ https://issues.apache.org/jira/browse/HIVE-24962?focusedWorklogId=589706=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589706 ] ASF GitHub Bot logged work on HIVE-24962: - Author: ASF GitHub Bot Created on: 27/Apr/21 12:45 Start Date: 27/Apr/21 12:45 Worklog Time Spent: 10m Work Description: pvary merged pull request #2137: URL: https://github.com/apache/hive/pull/2137 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 589706) Time Spent: 5h 20m (was: 5h 10m) > Enable partition pruning for Iceberg tables > --- > > Key: HIVE-24962 > URL: https://issues.apache.org/jira/browse/HIVE-24962 > Project: Hive > Issue Type: Improvement >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Labels: pull-request-available > Time Spent: 5h 20m > Remaining Estimate: 0h > > We should enable partition pruning above iceberg tables -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-24537) Optimise locking in LlapTaskSchedulerService
[ https://issues.apache.org/jira/browse/HIVE-24537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-24537 started by Panagiotis Garefalakis. - > Optimise locking in LlapTaskSchedulerService > > > Key: HIVE-24537 > URL: https://issues.apache.org/jira/browse/HIVE-24537 > Project: Hive > Issue Type: Sub-task > Components: llap >Reporter: Rajesh Balamohan >Assignee: Panagiotis Garefalakis >Priority: Major > Attachments: Screenshot 2020-12-15 at 11.41.49 AM.png > > > 1. Read lock should suffice for "notifyStarted()". > 2. Locking in "allocateTask()" can be optimised. > 3. Optimize preemptTasks() & preemptTasksFromMap(). This would help in > reducing the codepath with writeLock. Currently, it iterates through all > tasks. > > !Screenshot 2020-12-15 at 11.41.49 AM.png|width=847,height=446! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24537) Optimise locking in LlapTaskSchedulerService
[ https://issues.apache.org/jira/browse/HIVE-24537?focusedWorklogId=589700=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589700 ] ASF GitHub Bot logged work on HIVE-24537: - Author: ASF GitHub Bot Created on: 27/Apr/21 12:25 Start Date: 27/Apr/21 12:25 Worklog Time Spent: 10m Work Description: pgaref opened a new pull request #2224: URL: https://github.com/apache/hive/pull/2224 Change-Id: I91c004a06b35d60d11551c5fa04db7e6ba6f4550 ### What changes were proposed in this pull request? Optimise locking in LlapTaskSchedulerService ### Why are the changes needed? Perf optimization ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? TestLlapTaskSchedulerService -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 589700) Remaining Estimate: 0h Time Spent: 10m > Optimise locking in LlapTaskSchedulerService > > > Key: HIVE-24537 > URL: https://issues.apache.org/jira/browse/HIVE-24537 > Project: Hive > Issue Type: Sub-task > Components: llap >Reporter: Rajesh Balamohan >Assignee: Panagiotis Garefalakis >Priority: Major > Attachments: Screenshot 2020-12-15 at 11.41.49 AM.png > > Time Spent: 10m > Remaining Estimate: 0h > > 1. Read lock should suffice for "notifyStarted()". > 2. Locking in "allocateTask()" can be optimised. > 3. Optimize preemptTasks() & preemptTasksFromMap(). This would help in > reducing the codepath with writeLock. Currently, it iterates through all > tasks. > > !Screenshot 2020-12-15 at 11.41.49 AM.png|width=847,height=446! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24537) Optimise locking in LlapTaskSchedulerService
[ https://issues.apache.org/jira/browse/HIVE-24537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-24537: -- Labels: pull-request-available (was: ) > Optimise locking in LlapTaskSchedulerService > > > Key: HIVE-24537 > URL: https://issues.apache.org/jira/browse/HIVE-24537 > Project: Hive > Issue Type: Sub-task > Components: llap >Reporter: Rajesh Balamohan >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: Screenshot 2020-12-15 at 11.41.49 AM.png > > Time Spent: 10m > Remaining Estimate: 0h > > 1. Read lock should suffice for "notifyStarted()". > 2. Locking in "allocateTask()" can be optimised. > 3. Optimize preemptTasks() & preemptTasksFromMap(). This would help in > reducing the codepath with writeLock. Currently, it iterates through all > tasks. > > !Screenshot 2020-12-15 at 11.41.49 AM.png|width=847,height=446! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25061) PTF: Improve BoundaryCache
[ https://issues.apache.org/jira/browse/HIVE-25061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-25061: Attachment: Screen Shot 2021-04-27 at 1.02.37 PM.png > PTF: Improve BoundaryCache > -- > > Key: HIVE-25061 > URL: https://issues.apache.org/jira/browse/HIVE-25061 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: Screen Shot 2021-04-27 at 1.02.37 PM.png > > > First, I need to check whether TreeMap is really needed for our case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25061) PTF: Improve BoundaryCache
[ https://issues.apache.org/jira/browse/HIVE-25061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-25061: Description: First, I need to check whether TreeMap is really needed for our case. > PTF: Improve BoundaryCache > -- > > Key: HIVE-25061 > URL: https://issues.apache.org/jira/browse/HIVE-25061 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: Screen Shot 2021-04-27 at 1.02.37 PM.png > > > First, I need to check whether TreeMap is really needed for our case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25061) Improve BoundaryCache
[ https://issues.apache.org/jira/browse/HIVE-25061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned HIVE-25061: --- Assignee: László Bodor > Improve BoundaryCache > - > > Key: HIVE-25061 > URL: https://issues.apache.org/jira/browse/HIVE-25061 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25061) PTF: Improve BoundaryCache
[ https://issues.apache.org/jira/browse/HIVE-25061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-25061: Summary: PTF: Improve BoundaryCache (was: Improve BoundaryCache) > PTF: Improve BoundaryCache > -- > > Key: HIVE-25061 > URL: https://issues.apache.org/jira/browse/HIVE-25061 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-25007) Implement insert overwrite for Iceberg tables
[ https://issues.apache.org/jira/browse/HIVE-25007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod resolved HIVE-25007. --- Resolution: Fixed > Implement insert overwrite for Iceberg tables > - > > Key: HIVE-25007 > URL: https://issues.apache.org/jira/browse/HIVE-25007 > Project: Hive > Issue Type: New Feature >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25007) Implement insert overwrite for Iceberg tables
[ https://issues.apache.org/jira/browse/HIVE-25007?focusedWorklogId=589650=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589650 ] ASF GitHub Bot logged work on HIVE-25007: - Author: ASF GitHub Bot Created on: 27/Apr/21 10:10 Start Date: 27/Apr/21 10:10 Worklog Time Spent: 10m Work Description: lcspinter merged pull request #2178: URL: https://github.com/apache/hive/pull/2178 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 589650) Time Spent: 1h 40m (was: 1.5h) > Implement insert overwrite for Iceberg tables > - > > Key: HIVE-25007 > URL: https://issues.apache.org/jira/browse/HIVE-25007 > Project: Hive > Issue Type: New Feature >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-14679) csv2/tsv2 output format disables quoting by default and it's difficult to enable
[
https://issues.apache.org/jira/browse/HIVE-14679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17333091#comment-17333091
]
Robert Zhang commented on HIVE-14679:
-
Reading the comments the issue seems to have been resolved. But the status is
still "open", why?
> csv2/tsv2 output format disables quoting by default and it's difficult to
> enable
>
>
> Key: HIVE-14679
> URL: https://issues.apache.org/jira/browse/HIVE-14679
> Project: Hive
> Issue Type: Bug
>Reporter: Brock Noland
>Assignee: Jianguo Tian
>Priority: Major
> Attachments: HIVE-14769.1.patch, HIVE-14769.2 .patch
>
>
> Over in HIVE-9788 we made quoting optional for csv2/tsv2.
> However I see the following issues:
> * JIRA doc doesn't mention it's disabled by default, this should be there an
> in the output of beeline help.
> * The JIRA says the property is {{--disableQuotingForSV}} but it's actually a
> system property. We should not use a system property as it's non-standard so
> extremely hard for users to set. For example I must do: {{env
> HADOOP_CLIENT_OPTS="-Ddisable.quoting.for.sv=false" beeline ...}}
> * The arg {{--disableQuotingForSV}} should be documented in beeline help.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22271) Create index on the TBL_COL_PRIVS table for the columns COLUMN_NAME, PRINCIPAL_NAME, PRINCIPAL_TYPE and TBL_ID
[
https://issues.apache.org/jira/browse/HIVE-22271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17333068#comment-17333068
]
zhaojk commented on HIVE-22271:
---
you can turn this configuration off, set hive.privileges.synchronizer false in
hive-site.xml
> Create index on the TBL_COL_PRIVS table for the columns COLUMN_NAME,
> PRINCIPAL_NAME, PRINCIPAL_TYPE and TBL_ID
> --
>
> Key: HIVE-22271
> URL: https://issues.apache.org/jira/browse/HIVE-22271
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Marta Kuczora
>Assignee: wenjun ma
>Priority: Major
>
> In one of the escalations for HDP-3.1.0 we found that the table privilege
> checks could be very slow and these checks could be speed up by defining an
> INDEX on the TBL_COL_PRIVS table for the following columns:
> COLUMN_NAME,PRINCIPAL_NAME,PRINCIPAL_TYPE,TBL_ID
> In the MYSQL slow query log, we found that the following query is executed
> slowly:
> {noformat}
> SELECT DISTINCT
> 'org.apache.hadoop.hive.metastore.model.MTableColumnPrivilege' AS
> `NUCLEUS_TYPE`,`A0`.`AUTHORIZER`,`A0`.`COLUMN_NAME`,`A0`.`CREATE_TIME`,`A0`.`GRANT_OPTION`,`A0`.`GRANTOR`,`A0`.`GRANTOR_TYPE`,`A0`.`PRINCIPAL_NAME`,`A0`.`PRINCIPAL_TYPE`,`A0`.`TBL_COL_PRIV`,`A0`.`TBL_COLUMN_GRANT_ID`
> FROM `TBL_COL_PRIVS` `A0` LEFT OUTER JOIN `TBLS` `B0` ON `A0`.`TBL_ID` =
> `B0`.`TBL_ID` LEFT OUTER JOIN `DBS` `C0` ON `B0`.`DB_ID` = `C0`.`DB_ID` WHERE
> `A0`.`PRINCIPAL_NAME` = 'xxx' AND `A0`.`PRINCIPAL_TYPE` = 'GROUP' AND
> `B0`.`TBL_NAME` = '' AND `C0`.`NAME` = 'xxx' AND `C0`.`CTLG_NAME` = 'xxx'
> AND `A0`.`COLUMN_NAME` = 'xxx'
> {noformat}
> When checked the explain plan of the this query, it could be seen that the
> index defined on the TBL_COL_PRIVS table is not used. In the slow query, the
> COLUMN_NAME, PRINCIPAL_NAME, PRINCIPAL_TYPE and TBL_ID columns were used, and
> after creating an index on these columns only, we saw significant performance
> improvement.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25059) Alter event is converted to rename during replication
[ https://issues.apache.org/jira/browse/HIVE-25059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25059: -- Labels: pull-request-available (was: ) > Alter event is converted to rename during replication > - > > Key: HIVE-25059 > URL: https://issues.apache.org/jira/browse/HIVE-25059 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > In case the database/table name have different cases, while creating an alter > event it considers change of name and creates a RENAME event rather than ALTER -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25059) Alter event is converted to rename during replication
[ https://issues.apache.org/jira/browse/HIVE-25059?focusedWorklogId=589626=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589626 ] ASF GitHub Bot logged work on HIVE-25059: - Author: ASF GitHub Bot Created on: 27/Apr/21 09:19 Start Date: 27/Apr/21 09:19 Worklog Time Spent: 10m Work Description: ayushtkn opened a new pull request #2223: URL: https://github.com/apache/hive/pull/2223 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 589626) Remaining Estimate: 0h Time Spent: 10m > Alter event is converted to rename during replication > - > > Key: HIVE-25059 > URL: https://issues.apache.org/jira/browse/HIVE-25059 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > In case the database/table name have different cases, while creating an alter > event it considers change of name and creates a RENAME event rather than ALTER -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25059) Alter event is converted to rename during replication
[ https://issues.apache.org/jira/browse/HIVE-25059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HIVE-25059: --- > Alter event is converted to rename during replication > - > > Key: HIVE-25059 > URL: https://issues.apache.org/jira/browse/HIVE-25059 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > > In case the database/table name have different cases, while creating an alter > event it considers change of name and creates a RENAME event rather than ALTER -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25058) PTF: TimestampValueBoundaryScanner can be optimised during range computation pt2 - isDistanceGreater
[ https://issues.apache.org/jira/browse/HIVE-25058?focusedWorklogId=589615=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589615 ] ASF GitHub Bot logged work on HIVE-25058: - Author: ASF GitHub Bot Created on: 27/Apr/21 08:19 Start Date: 27/Apr/21 08:19 Worklog Time Spent: 10m Work Description: abstractdog commented on pull request #: URL: https://github.com/apache/hive/pull/#issuecomment-827415122 cc: @rbalamohan: we missed isDistanceGreater last time, please find flamegraph in jira I'll include tests later -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 589615) Time Spent: 20m (was: 10m) > PTF: TimestampValueBoundaryScanner can be optimised during range computation > pt2 - isDistanceGreater > > > Key: HIVE-25058 > URL: https://issues.apache.org/jira/browse/HIVE-25058 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Attachments: Screen Shot 2021-04-27 at 10.15.46 AM.png > > Time Spent: 20m > Remaining Estimate: 0h > > HIVE-24746 optimized isEqual, but we can do the same optimization for > isDistanceGreater. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25058) PTF: TimestampValueBoundaryScanner can be optimised during range computation pt2 - isDistanceGreater
[ https://issues.apache.org/jira/browse/HIVE-25058?focusedWorklogId=589614=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589614 ] ASF GitHub Bot logged work on HIVE-25058: - Author: ASF GitHub Bot Created on: 27/Apr/21 08:18 Start Date: 27/Apr/21 08:18 Worklog Time Spent: 10m Work Description: abstractdog opened a new pull request #: URL: https://github.com/apache/hive/pull/ ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 589614) Remaining Estimate: 0h Time Spent: 10m > PTF: TimestampValueBoundaryScanner can be optimised during range computation > pt2 - isDistanceGreater > > > Key: HIVE-25058 > URL: https://issues.apache.org/jira/browse/HIVE-25058 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: Screen Shot 2021-04-27 at 10.15.46 AM.png > > Time Spent: 10m > Remaining Estimate: 0h > > HIVE-24746 optimized isEqual, but we can do the same optimization for > isDistanceGreater. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25058) PTF: TimestampValueBoundaryScanner can be optimised during range computation pt2 - isDistanceGreater
[ https://issues.apache.org/jira/browse/HIVE-25058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-25058: -- Labels: pull-request-available (was: ) > PTF: TimestampValueBoundaryScanner can be optimised during range computation > pt2 - isDistanceGreater > > > Key: HIVE-25058 > URL: https://issues.apache.org/jira/browse/HIVE-25058 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Attachments: Screen Shot 2021-04-27 at 10.15.46 AM.png > > Time Spent: 10m > Remaining Estimate: 0h > > HIVE-24746 optimized isEqual, but we can do the same optimization for > isDistanceGreater. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25058) PTF: TimestampValueBoundaryScanner can be optimised during range computation pt2
[ https://issues.apache.org/jira/browse/HIVE-25058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-25058: Attachment: Screen Shot 2021-04-27 at 10.15.46 AM.png > PTF: TimestampValueBoundaryScanner can be optimised during range computation > pt2 > > > Key: HIVE-25058 > URL: https://issues.apache.org/jira/browse/HIVE-25058 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: Screen Shot 2021-04-27 at 10.15.46 AM.png > > > HIVE-24746 optimized isEqual, but we can do the same optimization for > isDistanceGreater. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25058) PTF: TimestampValueBoundaryScanner can be optimised during range computation pt2 - isDistanceGreater
[ https://issues.apache.org/jira/browse/HIVE-25058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-25058: Summary: PTF: TimestampValueBoundaryScanner can be optimised during range computation pt2 - isDistanceGreater (was: PTF: TimestampValueBoundaryScanner can be optimised during range computation pt2) > PTF: TimestampValueBoundaryScanner can be optimised during range computation > pt2 - isDistanceGreater > > > Key: HIVE-25058 > URL: https://issues.apache.org/jira/browse/HIVE-25058 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: Screen Shot 2021-04-27 at 10.15.46 AM.png > > > HIVE-24746 optimized isEqual, but we can do the same optimization for > isDistanceGreater. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25058) PTF: TimestampValueBoundaryScanner can be optimised during range computation pt2
[ https://issues.apache.org/jira/browse/HIVE-25058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-25058: Attachment: (was: flamegraph_1619432614_container_e02_1619108909155_0205_01_06_ctr-e163-1618833847257-10852-01-09.hwx.site.svg) > PTF: TimestampValueBoundaryScanner can be optimised during range computation > pt2 > > > Key: HIVE-25058 > URL: https://issues.apache.org/jira/browse/HIVE-25058 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > > HIVE-24746 optimized isEqual, but we can do the same optimization for > isDistanceGreater. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25058) PTF: TimestampValueBoundaryScanner can be optimised during range computation pt2
[ https://issues.apache.org/jira/browse/HIVE-25058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-25058: Description: HIVE-24746 optimized isEqual, but we can do the same optimization for isDistanceGreater. > PTF: TimestampValueBoundaryScanner can be optimised during range computation > pt2 > > > Key: HIVE-25058 > URL: https://issues.apache.org/jira/browse/HIVE-25058 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: > flamegraph_1619432614_container_e02_1619108909155_0205_01_06_ctr-e163-1618833847257-10852-01-09.hwx.site.svg > > > HIVE-24746 optimized isEqual, but we can do the same optimization for > isDistanceGreater. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-25058) PTF: TimestampValueBoundaryScanner can be optimised during range computation pt2
[ https://issues.apache.org/jira/browse/HIVE-25058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-25058: Attachment: flamegraph_1619432614_container_e02_1619108909155_0205_01_06_ctr-e163-1618833847257-10852-01-09.hwx.site.svg > PTF: TimestampValueBoundaryScanner can be optimised during range computation > pt2 > > > Key: HIVE-25058 > URL: https://issues.apache.org/jira/browse/HIVE-25058 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: > flamegraph_1619432614_container_e02_1619108909155_0205_01_06_ctr-e163-1618833847257-10852-01-09.hwx.site.svg > > > HIVE-24746 optimized isEqual, but we can do the same optimization for > isDistanceGreater. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-25058) PTF: TimestampValueBoundaryScanner can be optimised during range computation pt2
[ https://issues.apache.org/jira/browse/HIVE-25058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned HIVE-25058: --- Assignee: László Bodor > PTF: TimestampValueBoundaryScanner can be optimised during range computation > pt2 > > > Key: HIVE-25058 > URL: https://issues.apache.org/jira/browse/HIVE-25058 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25055) Improve the exception handling in HMSHandler
[ https://issues.apache.org/jira/browse/HIVE-25055?focusedWorklogId=589588=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589588 ] ASF GitHub Bot logged work on HIVE-25055: - Author: ASF GitHub Bot Created on: 27/Apr/21 06:50 Start Date: 27/Apr/21 06:50 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on pull request #2218: URL: https://github.com/apache/hive/pull/2218#issuecomment-827358824 @vihangk1 could you please take a look when you have a sec? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 589588) Time Spent: 1h 20m (was: 1h 10m) > Improve the exception handling in HMSHandler > > > Key: HIVE-25055 > URL: https://issues.apache.org/jira/browse/HIVE-25055 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25056) cast ('000-00-00 00:00:00' as timestamp/datetime) results in wrong conversion
[
https://issues.apache.org/jira/browse/HIVE-25056?focusedWorklogId=589574=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589574
]
ASF GitHub Bot logged work on HIVE-25056:
-
Author: ASF GitHub Bot
Created on: 27/Apr/21 06:23
Start Date: 27/Apr/21 06:23
Worklog Time Spent: 10m
Work Description: anuragshekhar2020 opened a new pull request #2221:
URL: https://github.com/apache/hive/pull/2221
…s in wrong conversion
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 589574)
Remaining Estimate: 0h
Time Spent: 10m
> cast ('000-00-00 00:00:00' as timestamp/datetime) results in wrong conversion
> --
>
> Key: HIVE-25056
> URL: https://issues.apache.org/jira/browse/HIVE-25056
> Project: Hive
> Issue Type: Bug
>Reporter: Anurag Shekhar
>Assignee: Anurag Shekhar
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> select cast ('-00-00' as date) , cast ('000-00-00 00:00:00' as timestamp)
> +--+---+
> |_c0|_c1|
> +--+---+
> |0002-11-30|0002-11-30 00:00:00.0|
> +--+---+
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-25056) cast ('000-00-00 00:00:00' as timestamp/datetime) results in wrong conversion
[
https://issues.apache.org/jira/browse/HIVE-25056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-25056:
--
Labels: pull-request-available (was: )
> cast ('000-00-00 00:00:00' as timestamp/datetime) results in wrong conversion
> --
>
> Key: HIVE-25056
> URL: https://issues.apache.org/jira/browse/HIVE-25056
> Project: Hive
> Issue Type: Bug
>Reporter: Anurag Shekhar
>Assignee: Anurag Shekhar
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> select cast ('-00-00' as date) , cast ('000-00-00 00:00:00' as timestamp)
> +--+---+
> |_c0|_c1|
> +--+---+
> |0002-11-30|0002-11-30 00:00:00.0|
> +--+---+
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
