This is an automated email from the ASF dual-hosted git repository.
zihanli58 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/gobblin.git
The following commit(s) were added to refs/heads/master by this push:
new dd9f9c454 [GOBBLIN-1845] Changes parallelstream to stream in
DatasetsFinderFilteringDecorator to avoid classloader issues in spark (#3706)
dd9f9c454 is described below
commit dd9f9c454832036a4bbab9c1997f5f9da9cb0891
Author: vikram bohra <[email protected]>
AuthorDate: Mon Jun 19 18:56:31 2023 -0700
[GOBBLIN-1845] Changes parallelstream to stream in
DatasetsFinderFilteringDecorator to avoid classloader issues in spark (#3706)
I think this is a quick operation so that we don't have much performance
concern. Will merge it for now to unblock one urgent fix, we can open another
PR if we want to improve the performance.
---
.../data/management/dataset/DatasetsFinderFilteringDecorator.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/gobblin-data-management/src/main/java/org/apache/gobblin/data/management/dataset/DatasetsFinderFilteringDecorator.java
b/gobblin-data-management/src/main/java/org/apache/gobblin/data/management/dataset/DatasetsFinderFilteringDecorator.java
index 46dbdc800..ab6743f6b 100644
---
a/gobblin-data-management/src/main/java/org/apache/gobblin/data/management/dataset/DatasetsFinderFilteringDecorator.java
+++
b/gobblin-data-management/src/main/java/org/apache/gobblin/data/management/dataset/DatasetsFinderFilteringDecorator.java
@@ -75,7 +75,7 @@ public class DatasetsFinderFilteringDecorator<T extends
Dataset> implements Data
log.info("Found {} datasets", datasets.size());
List<T> allowedDatasets = Collections.emptyList();
try {
- allowedDatasets = datasets.parallelStream()
+ allowedDatasets = datasets.stream()
.filter(dataset -> allowDatasetPredicates.stream()
.map(CheckedExceptionPredicate::wrapToTunneled)
.allMatch(p -> p.test(dataset)))
