This is an automated email from the ASF dual-hosted git repository.
vinoth pushed a change to branch rfc-15
in repository https://gitbox.apache.org/repos/asf/hudi.git.
from 8bdbdf4 [HUDI-842] Reader and Writer for Metadata.
new 389be1b [RFC-15] Add metrics to track the time for each file system
call.
new e8f1698 [RFC-15] Added a distributed metrics registry for spark which
can be used to collect metrics from executors.
new ce1c983 [RFC-15] Using distributed metrics registry in metadata code.
new 7bea565 [MINOR] Reduced number of fs calls to check for table
existence. Removed dead code.
new a2dcacf [HUDI-1346] Choose a new instant time when performing
autoClean.
new b16586d [HUDI-1321] Created HoodieMetadataConfig to specify
configuration for the metadata table.
new 6ade850 [HUDI-1317] Fix initialization when Async jobs are scheduled.
new 48d2f80 [RFC-15] Fixing checkstyle errors.
new 86fdd65 [HUDI-1305] Added an API to shutdown and remove the metrics
reporter. (#2132)
new 9e8025d [RFC-15] Fixing metrics printing to console which generates a
lot of text.
new 5e13b19 [RFC-15] Fixing checkstyle errors.
new 34872bb [RFC-15] Reverting change to AbstractHoodieLogRecordScanner
as they lead to hung tests.
new 4b7a4b3 [RFC-15] Fixing getFileSystemView() which requires
getCompletedCommitsTimeline().
new d305948 [RFC-15] Fixing code review comments
The 14 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../apache/hudi/cli/commands/MetadataCommand.java | 23 +-
.../apache/hudi/client/AbstractHoodieClient.java | 27 +-
.../hudi/client/AbstractHoodieWriteClient.java | 5 +
.../apache/hudi/client/AsyncCleanerService.java | 7 +-
.../org/apache/hudi/client/HoodieWriteClient.java | 48 ++--
.../apache/hudi/config/HoodieMetadataConfig.java | 151 +++++++++++
.../apache/hudi/config/HoodieMetricsConfig.java | 8 +
.../org/apache/hudi/config/HoodieWriteConfig.java | 85 +++----
.../apache/hudi/metadata/HoodieMetadataWriter.java | 122 ++++++---
.../hudi/metrics/ConsoleMetricsReporter.java | 3 +
.../apache/hudi/metrics/DistributedRegistry.java | 107 ++++++++
.../main/java/org/apache/hudi/metrics/Metrics.java | 30 ++-
.../java/org/apache/hudi/table/HoodieTable.java | 9 +-
.../apache/hudi/metadata/TestHoodieMetadata.java | 99 ++++---
.../hudi/metrics/TestHoodieConsoleMetrics.java | 6 +
.../java/org/apache/hudi/common/fs/FSUtils.java | 17 --
.../hudi/common/fs/HoodieWrapperFileSystem.java | 283 +++++++++++++--------
.../common/fs/SizeAwareFSDataOutputStream.java | 17 +-
.../hudi/common/fs/TimedFSDataInputStream.java | 79 ++++++
.../metrics/{Registry.java => LocalRegistry.java} | 62 +----
.../org/apache/hudi/common/metrics/Registry.java | 108 ++++----
.../table/log/AbstractHoodieLogRecordScanner.java | 14 +-
.../hudi/common/table/log/HoodieLogFileReader.java | 5 +-
.../hudi/exception/TableNotFoundException.java | 14 +-
.../apache/hudi/metadata/HoodieMetadataReader.java | 105 ++++----
25 files changed, 971 insertions(+), 463 deletions(-)
create mode 100644
hudi-client/src/main/java/org/apache/hudi/config/HoodieMetadataConfig.java
create mode 100644
hudi-client/src/main/java/org/apache/hudi/metrics/DistributedRegistry.java
create mode 100644
hudi-common/src/main/java/org/apache/hudi/common/fs/TimedFSDataInputStream.java
copy hudi-common/src/main/java/org/apache/hudi/common/metrics/{Registry.java
=> LocalRegistry.java} (58%)
