walkinggo opened a new pull request, #23820:
URL: https://github.com/apache/pulsar/pull/23820
<!--
### Contribution Checklist
- PR title format should be *[type][component] summary*. For details, see
*[Guideline - Pulsar PR Naming
Convention](https://pulsar.apache.org/contribute/develop-semantic-title/)*.
- Fill out the template below to describe the changes contributed by the
pull request. That will give reviewers the context they need to do the review.
- Each pull request should address only one issue, not mix up code from
multiple issues.
- Each commit in the pull request has a meaningful commit message
- Once all items of the checklist are addressed, remove the above text and
this checklist, leaving only the filled out template below.
-->
<!-- Either this PR fixes an issue, -->
see #23811
<!-- or this PR is one task of an issue -->
Main Issue: see #23705
<!-- If the PR belongs to a PIP, please add the PIP link here -->
<!-- Details of when a PIP is required and how the PIP process work, please
see: https://github.com/apache/pulsar/blob/master/pip/README.md -->
### Motivation
In the realm of asynchronous processing, precise timing and performance
metrics are essential for effective monitoring and optimization. Apache Pulsar
Functions, as a distributed compute platform, relies heavily on asynchronous
operations to process and transform data streams. However, the current
implementation lacks a robust mechanism for accurately capturing and reporting
the execution times of these asynchronous functions.
The primary motivation for this Pull Request is to address the limitations
of the existing asynchronous function execution time tracking system. By
centralizing the start time recording and improving the consistency of
execution time data, we aim to provide users with a more reliable and
comprehensive view of function performance.
<!-- Explain here the context, and why you're making that change. What is
the problem you're trying to solve. -->
### Modifications
This Pull Request introduces several modifications to the Apache Pulsar
Functions project aimed at improving the accuracy and reliability of
asynchronous function execution time statistics. The primary goal is to enhance
monitoring and analysis capabilities for function performance. Here’s a
breakdown of the key changes:
1. Removal of processTimeStart Method: The processTimeStart method in
ComponentStatsManager has been removed. This method was previously used to
record the start time of asynchronous function execution. The start time is now
recorded in the JavaExecutionResult object, providing a more centralized and
consistent approach.
2. Modification of JavaInstanceRunnable: The run method in
JavaInstanceRunnable has been updated to reflect the removal of
processTimeStart. The stats.processTimeStart() call has been deleted, and the
stats.processTimeEnd() method now accepts the start time as a parameter to
calculate the total execution time.
3. Update to FunctionStatsManager: The processTimeEnd method in
FunctionStatsManager has been modified to remove the processTimeStart member
variable and accept the start time as a parameter. This allows for accurate
calculation of the function’s execution time.
4. Changes to SinkStatsManager and SourceStatsManager: The processTimeEnd
methods in SinkStatsManager and SourceStatsManager have been updated to accept
the start time as a parameter. However, since these classes do not record
processing time, the method bodies remain empty.
5. Addition of startTime in JavaExecutionResult: The JavaExecutionResult
class now includes a startTime member variable to store the start time of
asynchronous function execution. This allows for accurate calculation of
execution time within the handleResult method of JavaInstanceRunnable.
6. Modification of AsyncFuncRequest: The AsyncFuncRequest class in
JavaInstance now includes a result member variable of type JavaExecutionResult.
This change ensures that the processAsyncResultsInInputOrder method uses the
existing JavaExecutionResult object instead of creating a new one, maintaining
consistency and avoiding duplication.
7. Use of Same ExecutionResult in Non-asyncPreserveInputOrder Mode: In
scenarios where asyncPreserveInputOrderForOutputMessages is disabled, the same
executionResult object is now used to avoid unnecessary object creation and
potential issues with result assignment.
8. Fix for Result and Exception Handling: Two patches address potential bugs
related to result and exception handling in JavaInstance. The
processAsyncResultsInInputOrder method now ensures that the result and
userException fields of JavaExecutionResult are properly set, improving the
reliability of the execution result.
9. Addition of Test Case: A new test case, testAsyncFunctionTime, has been
added to verify the accuracy of asynchronous function execution time recording
and calculation. This test ensures that the start time recorded in
JavaExecutionResult is within an acceptable range of the actual start time.
<!-- Describe the modifications you've done. -->
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] `doc` <!-- Your PR contains doc changes. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update
later -->
- [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
### Matching PR in forked repository
PR in forked repository: <!-- ENTER URL HERE -->
<!--
After opening this PR, the build in apache/pulsar will fail and instructions
will
be provided for opening a PR in the PR author's forked repository.
apache/pulsar pull requests should be first tested in your own fork since
the
apache/pulsar CI based on GitHub Actions has constrained resources and quota.
GitHub Actions provides separate quota for pull requests that are executed
in
a forked repository.
The tests will be run in the forked repository until all PR review comments
have
been handled, the tests pass and the PR is approved by a reviewer.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
