[GitHub] [dolphinscheduler] Radeity opened a new pull request, #12930: [Bug][Worker] Fix duplicated key bug when have out parameters of same name

[GitBox]

Radeity opened a new pull request, #12930:
URL: https://github.com/apache/dolphinscheduler/pull/12930

   <!--Thanks very much for contributing to Apache DolphinScheduler. Please 
review 
https://dolphinscheduler.apache.org/en-us/community/development/pull-request.html
 before opening a pull request.-->
   
   ## Purpose of the pull request
   
   -  This PR closes https://github.com/apache/dolphinscheduler/issues/12927
   
   ## Brief change log
   
   Workflow as below:
   
   <img width="645" alt="image" 
src="https://user-images.githubusercontent.com/45198818/202390469-fd3e00c7-aff9-456a-b6ed-0d0194124e5e.png";>
   
   Define out parameter in n1:
   
   <img width="644" alt="image" 
src="https://user-images.githubusercontent.com/45198818/202390339-ad9d4223-91f4-4cd5-a0dd-afa7a11354a2.png";>
   
   
   Define out parameter in n2:
   
   <img width="641" alt="image" 
src="https://user-images.githubusercontent.com/45198818/202390384-4f81090a-4c5a-43be-84cc-37afda4352f4.png";>
   
   Same parameter name will cause error when executing the following code in 
`TaskFilesTransferUtils.java` because of the same key in map ( `varPools` have 
two property which have the same prop `output`, one direct is `IN` and the 
other is `OUT`).
   
   ```java
   Map<String, Property> varPoolsMap = 
varPools.stream().collect(Collectors.toMap(Property::getProp, x -> x));
   ```
   
   Error logs are as follows:
   ```shell
   [INFO] 2022-11-17 14:50:29.508 +0800 TaskLogLogger-class 
org.apache.dolphinscheduler.server.worker.utils.TaskFilesTransferUtils:[75] - 
[varvarvarvarvar] output : 1  
   [INFO] 2022-11-17 14:50:29.508 +0800 TaskLogLogger-class 
org.apache.dolphinscheduler.server.worker.utils.TaskFilesTransferUtils:[75] - 
[varvarvarvarvar] output : 200  
   [ERROR] 2022-11-17 14:50:29.509 +0800 TaskLogLogger-class 
org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[185]
 - Task execute failed, due to meet an exception  
   java.lang.IllegalStateException: Duplicate key Property{prop='output', 
direct=IN, type=VARCHAR, value='1'}  
    at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133) 
 
    at java.util.HashMap.merge(HashMap.java:1254)  
    at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320)  
    at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) 
   at 
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)  
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)  
    at 
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)  
    at 
java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
   ```
   <br>
   `List.addAll` will not remove duplicated, so i change the func to 
`addPropertyToValPool`
   
   ## Verify this pull request
   
   <!--*(Please pick either of the following options)*-->
   
   This pull request is code cleanup without any test coverage.
   
   *(or)*
   
   This pull request is already covered by existing tests, such as *(please 
describe tests)*.
   
   (or)
   
   This change added tests and can be verified as follows:
   
   <!--*(example:)*
   - *Added dolphinscheduler-dao tests for end-to-end.*
   - *Added CronUtilsTest to verify the change.*
   - *Manually verified the change by testing locally.* -->
   
   (or)
   
   If your pull request contain incompatible change, you should also add it to 
`docs/docs/en/guide/upgrede/incompatible.md`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org