> The JDK has many implementations of concurrent lists and maps. You don't need to write anything. The code to deal with thread pool vs. the alternative approach would be different, yes, but you make it sound like you'd have to implement some really complicated data structure when that is definitely not the case.
I am not saying that it is complicated, I am saying that, since we can avoid thread synchronization with the other pros which I have described and also the cons like more dependencies introduced, which side we are more interested > Please don't go down this path. I'm not mixing topic. I made suggestions that you seemed to not be willing to address, or misunderstood. I am addressing your suggestions, by asking you what's your concrete ideas about your suggestion, e.g. how to handle A,B,C cases, It looks like what you expect is I accept it without any asking > I didn't blame anyone. I'm trying to get you to address my suggestion and trying to understand some of your replies, which really were a bit puzzling to me. > If the non-REST API doesn't have such API, then great, you addressed my suggestion. I don't think it's worth it to rewrite the whole YARN handling code to use the REST path here. That wasn't hard, was it. Looks like non-REST API also contains this https://hadoop.apache. org/docs/r2.7.0/api/src-html/org/apache/hadoop/yarn/client/ api/YarnClient.html#line.225 my concern which was skipped in your last email (again) is that, how many app states we want to fetch through this API. What I can see is we cannot filter applications since application state can change between two polls, any thoughts? On Wed, Aug 16, 2017 at 10:36 AM, Marcelo Vanzin <[email protected]> wrote: > On Wed, Aug 16, 2017 at 10:31 AM, Nan Zhu <[email protected]> wrote: > >> In the first case your thread pool is the "shared data structure", in > the > > second case this map of handles is the "shared data structure", so I > don't > > understand why you think there is any difference. > > > > I do not understand why there is no difference > > > > In your words, when the thread pool is the "shared data structure", we do > > not need to involve any synchronization when applying CRUD to the > handles. > > If you shared "some map or list" between servlet threads and monitoring > > thread, you have to handle synchronization of this "map or list". > > The JDK has many implementations of concurrent lists and maps. You > don't need to write anything. The code to deal with thread pool vs. > the alternative approach would be different, yes, but you make it > sound like you'd have to implement some really complicated data > structure when that is definitely not the case. > > > You mixed two topics again > > Please don't go down this path. I'm not mixing topic. I made > suggestions that you seemed to not be willing to address, or > misunderstood. > > > Topic 2 - How we discuss: I am surprised that you blamed my way to > discuss > > I didn't blame anyone. I'm trying to get you to address my suggestion > and trying to understand some of your replies, which really were a bit > puzzling to me. > > > OK, I found it, > > https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/ > ResourceManagerRest.html#Cluster_Applications_API, > > which is a restful API, > > > > two concerns > > > > 1. change Livy's current impl is based on YarnClient not restful API, are > > we going to change it? > > If the non-REST API doesn't have such API, then great, you addressed > my suggestion. I don't think it's worth it to rewrite the whole YARN > handling code to use the REST path here. > > That wasn't hard, was it. > > -- > Marcelo >
