Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/17054 )
Change subject: IMPALA-10497: Fix flakiness in test_no_fd_caching_on_cached_data. ...................................................................... Patch Set 2: (5 comments) http://gerrit.cloudera.org:8080/#/c/17054/1/tests/custom_cluster/test_hdfs_fd_caching.py File tests/custom_cluster/test_hdfs_fd_caching.py: http://gerrit.cloudera.org:8080/#/c/17054/1/tests/custom_cluster/test_hdfs_fd_caching.py@179 PS1, Line 179: le handles cached > One thing to watch out for here is if you inherit from TestHdfsFdCaching, y Done http://gerrit.cloudera.org:8080/#/c/17054/1/tests/custom_cluster/test_hdfs_fd_caching.py@184 PS1, Line 184: # handle ca > I believe that the number of rows should not impact the behavior. The diffe Done http://gerrit.cloudera.org:8080/#/c/17054/1/tests/custom_cluster/test_hdfs_fd_caching.py@190 PS1, Line 190: > I will remove this so the cluster size back to default. I change my mind, I think we should keep cluster_size=1 here. On start up, impalad delete the old cache file by listing file matching CACHE_FILE_PREFIX and delete them one by one. https://github.com/apache/impala/blob/b42c649/be/src/runtime/io/data-cache.cc#L415 So if cluster_size > 1, multiple impalad might race to delete the same old cache file and crash the losing ones. Also, setting cluster_size=1 does not change the behavior being tested by this test, so I think it should be okay. http://gerrit.cloudera.org:8080/#/c/17054/1/tests/custom_cluster/test_hdfs_fd_caching.py@207 PS1, Line 207: : def cached_handles(self): : return self.get_ag > This makes a lot of sense to me as the explanation. There are times when da Done http://gerrit.cloudera.org:8080/#/c/17054/1/tests/custom_cluster/test_hdfs_fd_caching.py@224 PS1, Line 224: > I'm fine with either value here, but I also don't see this changing the beh Done -- To view, visit http://gerrit.cloudera.org:8080/17054 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I774f9dfea7dcc107c3c7f2b76db3aaf4b2dd7952 Gerrit-Change-Number: 17054 Gerrit-PatchSet: 2 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Bikramjeet Vig <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Comment-Date: Thu, 11 Feb 2021 03:09:44 +0000 Gerrit-HasComments: Yes
