Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21469
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21469
**[Test build #91509 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91509/testReport)**
for PR 21469 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21469
**[Test build #91509 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91509/testReport)**
for PR 21469 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21469
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91503/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21469
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21469
**[Test build #91503 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91503/testReport)**
for PR 21469 at commit
Github user arunmahadevan commented on the issue:
https://github.com/apache/spark/pull/21469
Nice, LGTM.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21469
@arunmahadevan
Added custom metrics in state store to streaming query status as well. You
can see `providerLoadedMapSize` is added to `stateOperators.customMetrics` in
below output.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21469
**[Test build #91503 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91503/testReport)**
for PR 21469 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21469
@jose-torres is it good to go?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21469
**[Test build #91486 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91486/testReport)**
for PR 21469 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21469
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91486/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21469
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user arunmahadevan commented on the issue:
https://github.com/apache/spark/pull/21469
>I didn't add the metric to StateOperatorProgress cause this behavior is
specific to HDFSBackedStateStoreProvider
May be this can be reported as a custom metrics and keep it optional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21469
**[Test build #91486 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91486/testReport)**
for PR 21469 at commit
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21469
Also added custom metric for the count of versions stored in loadedMaps.
This is a new screenshot:
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21469
Looks like the size is added only once for same identity on
SizeEstimator.estimate(), so SizeEstimator.estimate() is working correctly in
this case. There might be other valid cases, but not
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21469
@arunmahadevan
I didn't add the metric to StateOperatorProgress cause this behavior is
specific to HDFSBackedStateStoreProvider (though this is only one
implementation available in Apache
Github user arunmahadevan commented on the issue:
https://github.com/apache/spark/pull/21469
@HeartSaVioR , may be then this should be reported in the "memoryUsedBytes"
in the StateOperatorProgress (value reported in StreamingQueryProgress) because
currently the usage reported does
101 - 119 of 119 matches
Mail list logo
