Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24259 )
Change subject: IMPALA-14930: Auto comments on unused imports in Java ...................................................................... IMPALA-14930: Auto comments on unused imports in Java This patch improves critique-gerrit-review.py to add comments on unused imports in Java changes. Different to the existing Python unused import check that depends on flake8 (F401), the script simply uses string parsing to extract imported class/method names and check if they are used in non-comment codes. Note that wildcard imports like "import java.util.*" are not allowed unless for test codes or explicitly added in the allow list. E.g. we allow "import static org.junit.Assert.*" in test codes for simplicity. Another difference to the Python check is the whole Java file is checked instead of just checking the changed codes. This makes it possible to find unused imports that are caused by removed codes. This patch also removes the existing unused imports in Java files so they won't trigger Gerrit comments on unrelated patches. Expands "import java.util.*" in Graph.java to what it actually needs. Tests: - Used the logic in critique-gerrit-review.py to remove all unused imports in Java files. - Compiled Impala locally after removing the unused imports. - Verified the script in some commits that have unused imports. Assisted-by: Composer 2 Change-Id: I5a260c05e02e96aaf1331bc31ed92b0e368403bd Reviewed-on: http://gerrit.cloudera.org:8080/24259 Reviewed-by: Michael Smith <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M bin/jenkins/critique-gerrit-review.py M fe/src/compat-apache-hive-2/java/org/apache/hadoop/hive/metastore/api/TableValidWriteIds.java M fe/src/compat-apache-hive-2/java/org/apache/impala/catalog/Hive2MetastoreShimBase.java M fe/src/compat-apache-hive-2/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/compat-apache-hive-2/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/HiveEsriGeospatialBuiltins.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableDropPartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableExecuteRepairMetadataStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableSetTblProperties.java M fe/src/main/java/org/apache/impala/analysis/AlterTableUnSetTblProperties.java M fe/src/main/java/org/apache/impala/analysis/AuthorizationStmt.java M fe/src/main/java/org/apache/impala/analysis/CompoundVerticalBarExpr.java M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateDropRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/DeleteStmt.java M fe/src/main/java/org/apache/impala/analysis/ExprSubstitutionMap.java M fe/src/main/java/org/apache/impala/analysis/GroupByClause.java M fe/src/main/java/org/apache/impala/analysis/IcebergDeleteImpl.java M fe/src/main/java/org/apache/impala/analysis/IcebergPartitionExpr.java M fe/src/main/java/org/apache/impala/analysis/IcebergPartitionExpressionRewriter.java M fe/src/main/java/org/apache/impala/analysis/LiteralExpr.java M fe/src/main/java/org/apache/impala/analysis/ModifyImpl.java M fe/src/main/java/org/apache/impala/analysis/NullLiteral.java M fe/src/main/java/org/apache/impala/analysis/OptimizeStmt.java M fe/src/main/java/org/apache/impala/analysis/PartitionKeyValue.java M fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java M fe/src/main/java/org/apache/impala/analysis/ShowCurrentGroupsStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowFilesStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowMetadataTablesStmt.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/TableDef.java M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java M fe/src/main/java/org/apache/impala/analysis/TrimFromExpr.java M fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java M fe/src/main/java/org/apache/impala/authorization/Authorizable.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/TableMask.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerUtil.java M fe/src/main/java/org/apache/impala/catalog/BinaryCompatibility.java M fe/src/main/java/org/apache/impala/catalog/CatalogHmsAPIHelper.java M fe/src/main/java/org/apache/impala/catalog/CtasTargetTable.java M fe/src/main/java/org/apache/impala/catalog/DataSourceTable.java M fe/src/main/java/org/apache/impala/catalog/Db.java M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java M fe/src/main/java/org/apache/impala/catalog/FeDb.java M fe/src/main/java/org/apache/impala/catalog/FeHBaseTable.java M fe/src/main/java/org/apache/impala/catalog/HdfsStorageDescriptor.java M fe/src/main/java/org/apache/impala/catalog/IcebergDeleteTable.java M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/PrunablePartition.java M fe/src/main/java/org/apache/impala/catalog/ScalarFunction.java M fe/src/main/java/org/apache/impala/catalog/VirtualTable.java M fe/src/main/java/org/apache/impala/catalog/events/DeleteEventLog.java M fe/src/main/java/org/apache/impala/catalog/events/ExternalEventsProcessor.java M fe/src/main/java/org/apache/impala/catalog/events/InFlightEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/main/java/org/apache/impala/catalog/events/NoOpEventProcessor.java M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCatalogUtil.java M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergMetadataTable.java M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java M fe/src/main/java/org/apache/impala/catalog/local/IcebergMetaProvider.java M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java M fe/src/main/java/org/apache/impala/catalog/metastore/CatalogHmsClientUtils.java M fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServer.java M fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java M fe/src/main/java/org/apache/impala/catalog/paimon/FeShowFileStmtSupport.java M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java M fe/src/main/java/org/apache/impala/common/TransactionKeepalive.java M fe/src/main/java/org/apache/impala/extdatasource/ExternalDataSourceExecutor.java M fe/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java M fe/src/main/java/org/apache/impala/hive/executor/BinaryToBinaryHiveLegacyFunctionExtractor.java M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java M fe/src/main/java/org/apache/impala/hooks/QueryEventHookManager.java M fe/src/main/java/org/apache/impala/planner/AnalyticPlanner.java M fe/src/main/java/org/apache/impala/planner/DataSourceScanNode.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/IcebergMergeSink.java M fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java M fe/src/main/java/org/apache/impala/planner/TrivialQueryChecker.java M fe/src/main/java/org/apache/impala/planner/TupleCacheInfo.java M fe/src/main/java/org/apache/impala/planner/TupleCacheNode.java M fe/src/main/java/org/apache/impala/planner/UnpivotNode.java M fe/src/main/java/org/apache/impala/rewrite/AddEnvIntersectsRule.java M fe/src/main/java/org/apache/impala/rewrite/ExtractCompoundVerticalBarExprRule.java M fe/src/main/java/org/apache/impala/rewrite/NormalizeGeospatialRelationsRule.java M fe/src/main/java/org/apache/impala/rewrite/PointEnvIntersectsRule.java M fe/src/main/java/org/apache/impala/service/Canceller.java M fe/src/main/java/org/apache/impala/service/CompilerFactory.java M fe/src/main/java/org/apache/impala/service/CompilerFactoryImpl.java M fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/main/java/org/apache/impala/util/FunctionUtils.java M fe/src/main/java/org/apache/impala/util/Graph.java M fe/src/main/java/org/apache/impala/util/HdfsCachingUtil.java M fe/src/main/java/org/apache/impala/util/IcebergUtil.java M fe/src/main/java/org/apache/impala/util/MigrateTableUtil.java M fe/src/main/java/org/apache/impala/util/StringUtils.java M fe/src/main/java/org/apache/impala/util/paimon/PaimonArrowFormatNativeWriter.java M fe/src/main/java/org/apache/impala/util/paimon/PaimonArrowFormatWriter.java M fe/src/main/java/org/apache/impala/util/paimon/PaimonArrowUtils.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeUpsertStmtTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzerTest.java M fe/src/test/java/org/apache/impala/analysis/AuditingKuduTest.java M fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java M fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java M fe/src/test/java/org/apache/impala/analysis/ExprRewriterTest.java M fe/src/test/java/org/apache/impala/analysis/StmtMetadataLoaderTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M fe/src/test/java/org/apache/impala/authorization/CatalogServiceTestCatalogWithRanger.java M fe/src/test/java/org/apache/impala/catalog/AlterDatabaseTest.java M fe/src/test/java/org/apache/impala/catalog/HdfsPartitionTest.java M fe/src/test/java/org/apache/impala/catalog/MetastoreApiTestUtils.java M fe/src/test/java/org/apache/impala/catalog/PartialCatalogInfoTest.java M fe/src/test/java/org/apache/impala/catalog/PartialCatalogInfoWriteIdTest.java M fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorStressTest.java M fe/src/test/java/org/apache/impala/catalog/local/CatalogdMetaProviderTest.java M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java M fe/src/test/java/org/apache/impala/catalog/metastore/AbstractCatalogMetastoreTest.java M fe/src/test/java/org/apache/impala/catalog/metastore/CatalogHmsFileMetadataTest.java M fe/src/test/java/org/apache/impala/common/AnalysisSessionFixture.java M fe/src/test/java/org/apache/impala/common/FrontendFixture.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java M fe/src/test/java/org/apache/impala/customcluster/JwtHttpTest.java M fe/src/test/java/org/apache/impala/customcluster/SpnegoAuthTest.java M fe/src/test/java/org/apache/impala/customcluster/THttpClientWithHeaders.java M fe/src/test/java/org/apache/impala/customservice/KuduHMSIntegrationTest.java M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdfWithJavaReturnTypes.java M fe/src/test/java/org/apache/impala/planner/ClusterSizeTest.java M fe/src/test/java/org/apache/impala/planner/ParquetBloomFilterTblPropParserTest.java M fe/src/test/java/org/apache/impala/planner/TpcdsCpuCostPlannerTest.java M fe/src/test/java/org/apache/impala/planner/TpcdsTupleCachePlannerTest.java M fe/src/test/java/org/apache/impala/planner/TupleCacheInfoTest.java M fe/src/test/java/org/apache/impala/planner/TupleCacheTest.java M fe/src/test/java/org/apache/impala/service/JdbcTest.java M fe/src/test/java/org/apache/impala/testutil/CatalogServiceTestCatalog.java M fe/src/test/java/org/apache/impala/testutil/StatsJsonParser.java M fe/src/test/java/org/apache/impala/testutil/TestUtils.java M fe/src/test/java/org/apache/impala/util/AcidUtilsTest.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/coercenodes/CoerceNodes.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/coercenodes/CoerceOperandShuttle.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/AnalyzedCastExpr.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/AnalyzedInPredicate.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/AnalyzedNullLiteral.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionResolver.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/ImplicitTypeChecker.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/RexLiteralConverter.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/CommonOperatorFunctions.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaAdjustScaleFunction.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaAggOperator.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaAvgAggFunction.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaCastFunction.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaConcatOrOperator.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaCustomOperatorTable.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaDecodeFunction.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaGroupingIdFunction.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaMinMaxAggFunction.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaOperator.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ConvertToImpalaRelRules.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaHdfsScanRel.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaJoinRel.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaPlanRel.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaProjectRel.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/phys/ImpalaHdfsScanNode.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/phys/ImpalaNestedLoopJoinNode.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/util/ExprConjunctsConverter.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/util/PrunedPartitionHelper.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/CombineValuesNodesRule.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/ExtractLiteralAgg.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/ImpalaFilterSimplifyRule.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/ImpalaJoinProjectTransposeRule.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/ImpalaMinusToDistinctRule.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/ImpalaRexExecutor.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/RewriteRexOverRule.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/CalciteTable.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/FilterSelectivityEstimator.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/ImpalaCost.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/ImpalaRelMdNonCumulativeCost.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/ImpalaRelMdRowCount.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteAnalysisDriver.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteAnalysisResult.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteMetadataHandler.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteParsedStatement.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalcitePhysPlanCreator.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteQueryParser.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteRelNodeConverter.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/ImpalaSqlValidatorImpl.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/UnsupportedChecker.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaSqlIntervalQualifier.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeCoercionImpl.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeConverter.java M java/calcite-planner/src/test/java/org/apache/impala/CompatibilityTest.java M java/calcite-planner/src/test/java/org/apache/impala/calcite/planner/TestCalciteStats.java M java/calcite-planner/src/test/java/org/apache/impala/calcite/planner/TestLoptJoinRule.java M java/calcite-planner/src/test/java/org/apache/impala/calcite/planner/TestReduceExprShuttle.java M java/calcite-planner/src/test/java/org/apache/impala/calcite/planner/TpcdsCpuCostPlannerTest.java M java/datagenerator/src/main/java/org/apache/impala/datagenerator/JsonToParquetConverter.java M java/datagenerator/src/main/java/org/apache/impala/datagenerator/RandomNestedDataGenerator.java M java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgConstCheckUdf.java M java/test-hive-udfs/src/main/java/org/apache/impala/GenericImportsNearbyClassesUdf.java M java/test-hive-udfs/src/main/java/org/apache/impala/GenericReplaceStringUdf.java M java/test-hive-udfs/src/main/java/org/apache/impala/IncompatibleUdfTest.java M java/test-hive-udfs/src/main/java/org/apache/impala/JavaUdfTest.java M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfException.java M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdfWithJavaReturnTypes.java M java/test-hive-udfs/src/main/java/org/apache/impala/UtilForUdf.java M java/test-hive-udfs/src/main/java/org/apache/impala/UtilForUdfConstructor.java M java/test-hive-udfs/src/main/java/org/apache/impala/UtilForUdfInitialize.java 219 files changed, 119 insertions(+), 577 deletions(-) Approvals: Michael Smith: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/24259 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I5a260c05e02e96aaf1331bc31ed92b0e368403bd Gerrit-Change-Number: 24259 Gerrit-PatchSet: 6 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Peter Rozsa <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]>
