Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/17308 )
Change subject: IMPALA-10502: Handle CREATE/DROP events correctly ...................................................................... Patch Set 3: (31 comments) http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java: http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1526 PS3, Line 1526: .removePartitionsIfNotAddedLater(eventId_, dbName_, tblName_, droppedPartitions_, > line too long (95 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java: http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@257 PS3, Line 257: String.format(CatalogOpExecutor.HMS_RPC_ERROR_FORMAT_STR, "getNextNotification")); > line too long (92 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@737 PS3, Line 737: CatalogOpExecutor catalogOpExecutor, long startSyncFromId, long eventPollingInterval) > line too long (91 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@744 PS3, Line 744: new MetastoreEventsProcessor(catalogOpExecutor, startSyncFromId, eventPollingInterval); > line too long (95 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1304 PS3, Line 1304: private Table addHdfsPartitions(MetaStoreClient msClient, Table tbl, List<Partition> partitions) > line too long (98 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1704 PS3, Line 1704: CreateDatabaseEvent.CREATE_DATABASE_EVENT_TYPE.equals(notificationEvent.getEventType()) > line too long (105 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1715 PS3, Line 1715: // Due to HIVE-24899 we cannot rely on the database object present in the event > line too long (91 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1918 PS3, Line 1918: eventIdToPartVals.computeIfAbsent(eventId, l -> new ArrayList<>()).add(partVals); > line too long (91 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2544 PS3, Line 2544: events = MetastoreEventsProcessor.getNextMetastoreEvents(catalog_, eventId, new NotificationFilter() { > line too long (110 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2547 PS3, Line 2547: return DropTableEvent.DROP_TABLE_EVENT_TYPE.equals(notificationEvent.getEventType()) && finalMsTbl > line too long (110 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3119 PS3, Line 3119: Pair<Long, org.apache.hadoop.hive.metastore.api.Table> eventTblPair = getTableFromEvents( > line too long (99 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3188 PS3, Line 3188: CreateTableEvent.CREATE_TABLE_EVENT_TYPE.equals(notificationEvent.getEventType()) > line too long (97 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3385 PS3, Line 3385: return CreateTableEvent.CREATE_TABLE_EVENT_TYPE.equals(notificationEvent.getEventType()) > line too long (104 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3386 PS3, Line 3386: && newTable.getDbName().equalsIgnoreCase(notificationEvent.getDbName()) > line too long (91 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3387 PS3, Line 3387: && newTable.getTableName().equalsIgnoreCase(notificationEvent.getTableName()); > line too long (98 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3780 PS3, Line 3780: "EventId: {} Not adding partitions since the database {} does not exist anymore.", > line too long (92 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3786 PS3, Line 3786: "EventId: {} Not adding partitions since the table {}.{} does not exist anymore.", > line too long (92 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3832 PS3, Line 3832: private List<Partition> filterPartitionsToAddFromEvent(long eventId, HdfsTable hdfsTable, > line too long (91 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3914 PS3, Line 3914: "EventId: {} Not removing partitions since the database {} does not exist anymore.", > line too long (94 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3920 PS3, Line 3920: "EventId: {} Not dropping partitions since the table {}.{} does not exist anymore", > line too long (93 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3973 PS3, Line 3973: "Event does not contain partition value for key %s. Event contains %s", > line too long (91 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@4013 PS3, Line 4013: "Not dropping partition {} of table {} since it's create event id {} is higher than eventid {}", > line too long (106 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@4039 PS3, Line 4039: "Not reloading the partition of table {} since it was removed later in catalog", > line too long (92 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@4114 PS3, Line 4114: && msTbl.getTableName().equalsIgnoreCase(notificationEvent.getTableName()); > line too long (95 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@4354 PS3, Line 4354: for (Map.Entry<Long, List<List<String>>> eventToPartitionNames : droppedPartitionsFromEvent > line too long (95 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@4445 PS3, Line 4445: events = MetastoreEventsProcessor.getNextMetastoreEvents(catalog_, eventId, new NotificationFilter() { > line too long (108 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@4456 PS3, Line 4456: org.apache.hadoop.hive.metastore.api.Table>> renamedTable = getRenamedTableFromEvents( > line too long (94 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/test/java/org/apache/impala/catalog/events/SynchronousHMSEventProcessorForTests.java File fe/src/test/java/org/apache/impala/catalog/events/SynchronousHMSEventProcessorForTests.java: http://gerrit.cloudera.org:8080/#/c/17308/3/fe/src/test/java/org/apache/impala/catalog/events/SynchronousHMSEventProcessorForTests.java@37 PS3, Line 37: CatalogOpExecutor catalogOpExecutor, long startSyncFromId, long pollingFrequencyInSec) > line too long (92 > 90) Done http://gerrit.cloudera.org:8080/#/c/17308/3/tests/custom_cluster/test_event_processing.py File tests/custom_cluster/test_event_processing.py: http://gerrit.cloudera.org:8080/#/c/17308/3/tests/custom_cluster/test_event_processing.py@30 PS3, Line 30: from time import sleep > flake8: F401 'time.sleep' imported but unused Done http://gerrit.cloudera.org:8080/#/c/17308/3/tests/custom_cluster/test_event_processing.py@224 PS3, Line 224: > flake8: E251 unexpected spaces around keyword / parameter equals Done http://gerrit.cloudera.org:8080/#/c/17308/3/tests/custom_cluster/test_event_processing.py@224 PS3, Line 224: > flake8: E251 unexpected spaces around keyword / parameter equals Done -- To view, visit http://gerrit.cloudera.org:8080/17308 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia2c5e96b48abac015240f20295b3ec3b1d71f24a Gerrit-Change-Number: 17308 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sourabh Goyal <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]> Gerrit-Comment-Date: Tue, 20 Apr 2021 18:00:37 +0000 Gerrit-HasComments: Yes
