Re: Hi, Why close this issue #6067 silently without any response for months?

2020-08-10 Thread Jason Joo
Aha

Feel sorry to you two guys.
Not stake holder.

I also have several such cases but try to think of the other hand, different 
people have different minds.
Some can be merged together while some can't. And open sources in china have a 
common trouble which is the inactive communities. Maybe due to IM and other 
communicating ways weaken the activeness of community, maybe due to the willing 
of developers in china is inactive in open source, maybe due to some others.

But hope that we can just move on and keep sharing.


regards,

Jason

> On Aug 10, 2020, at 16:37, 一直以来 <279377...@qq.com> wrote:
> 
> many many like the question to end close it !!
> 
> 
> alibaba dubbo group is very very poor!
> 
> 
> direct close!
> 
> 
> 
> 
> -- 原始邮件 --
> 发件人:  
> "dev" 
> >;
> 发送时间: 2020年8月10日(星期一) 下午4:35
> 收件人: "dev"mailto:dev@dubbo.apache.org>>;
> 
> 主题: Re: Hi, Why close this issue #6067 silently without any response for 
> months?
> 
> 
> 
> It is very disappointing.
> 
> The issue was open for months, and nobody saw it.
> 
> Months later, the issue was silently closed without any response.
> 
> Jun Liu  
> > Please feel free to reopen if necessary.
> >
> > Jun
> >
> > > On Aug 7, 2020, at 3:48 PM, Lonre Wang  > wrote:
> > >
> > > Hi, @chickenlj
> > >
> > > Why close this issue #6067 silently without any response for months?
> > >
> > > 😂
> >
> >



Re: Adaptive thread isolation between services about Dubbo

2020-07-21 Thread Jason Joo
Hi, 陆

The scenario you described in last post is more like a scene in which a 
protector may participate in. General implementation includes hystrix[1], 
sentinel[2], etc. . They can introduce abilities like flow control, traffic 
shaping, circuit breaking to various systems including Dubbo, servlet, gateway 
and so on. You can take a look and have a try with any of them to see whether 
it can match your concern.

For your information :)

[1] Hystrix https://github.com/Netflix/Hystrix
[2] Sentinel https://github.com/alibaba/sentinel/

best regards,

Jason

> On Jul 21, 2020, at 22:25, 陆 錞斐  wrote:
> 
> Hello everyone, I'm trying to make a little contribution to Dubbo by adding a 
> new feature. In this feature, threads can be allocated dynamically according 
> to service invocations, so as to achieve thread isolation between services. 
> In this way, we can set the minimum number of threads per service to prevent 
> thread pool resources from being exhausted by some services and thus 
> affecting other services.
> 
> Now the problem is how to set up an evaluation model according to the service 
> call situation, such as QPS, RT, success, exception, reject, and so on, so as 
> to reasonably allocate threads to each service.
> 
> I am lack of experience in this area, I hope you can give some suggestions, 
> either algorithm or book is OK, thank you!



Re: Maximize the consumer throughput and CPU usage

2019-11-29 Thread Jason Joo
Tien,

Glad to hear that.
One more thing 10k problem for single node is quite different from that in a 
cluster. And there will be a big jitter in service when doing deployment or 
there was a failure. 
So It's better to split consumers into small nodes and adjust / split providers 
into groups if it's necessary. 
Keep lightweight and simple.

best regards,

Jason

> On Nov 28, 2019, at 18:16, Tien Dat PHAN  wrote:
> 
> Dear Jason,
> 
> Thanks a lot for your help so far.
> We just found out the issue.
> Our laptop at the consumer side has 2 cores 4 threads. While the Provider is 
> running on VM of 48 cores. So we thought that from consumer side, running 
> with 4 threads, or maximum 8 would be enough to consume the maximum capacity 
> of our laptop. 
> But we were wrong. When we increase the consuming process from 8 threads to 
> 16, 32 and 48 threads, we increase the performance from 40K TPS to 70K TPS, 
> 98K TPS and 100K TPS, respectively. So we guess that, at the consuming side, 
> the performance is maximized if the parallelism reaching the maximum 
> parallelism at the provider side.
> 
> This put us to a question that if the consumer sends requests to a cluster of 
> providers, the number of threads might need to be increased to a very high 
> number.
> This still need to be benched to make sure.
> 
> That is our information so far. Hope this one can be helpful also.
> Thank you also for your help.
> 
> Best regards
> Dat
> 
> On 2019/11/27 16:24:18, Jason Joo  wrote: 
>> Tien,
>> 
>> Or maybe you can use single consumer but set the target busy service's 
>> "connections" in configuration to "2" or bigger value.
>> 
>> If you integrated dubbo using XML:
>> http://dubbo.apache.org/en-us/docs/user/references/xml/dubbo-service.html 
>> <http://dubbo.apache.org/en-us/docs/user/references/xml/dubbo-service.html>
>> 
>> Or if you integrated dubbo using annotations:
>> @Reference(connections = 2)
>> 
>> And so on. Just try this for another try.
>> 
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Nov 27, 2019, at 22:55, Tien Dat PHAN  wrote:
>>> 
>>> Dear Jason,
>>> 
>>> You were right. The fact that when we tried to run two processes of 
>>> consumer, we increased the total throughput by almost 2X. This means that 
>>> the infrastructure and OS (the laptop and Mac OS in this case) is capable 
>>> of providing higher throughput.
>>> 
>>> So the suspect comes back to the configuration of the consumer process.
>>> 
>>> We have tried the Arthas tool that you suggested. A very useful tool in 
>>> general. However, it still does not show any interesting lead that can 
>>> explain the problem.
>>> 
>>> We are stuck for now :(
>>> 
>>> Best 
>>> Dat
>>> 
>>> On 2019/11/27 10:38:32, Jason Joo  wrote: 
>>>> Tien,
>>>> 
>>>> Performance issue is quite a complicated problem and the open files count 
>>>> you just mentioned in last mail is not the only factor.
>>>> According to the persistent connection in dubbo design I don't think is 
>>>> the cause due to your issue and maybe the value you saw in your work 
>>>> machine is not the same on your servers in production. And if you hit the 
>>>> limit some error logs will occur in your log files.
>>>> 
>>>> You mentioned that only one core can reach into a high load work status I 
>>>> think you should probably check the IO workers count in the configuration. 
>>>> And an efficient debugging tool which could work here is Arthas[1] you can 
>>>> dig into the JVM process and check which thread is spinning to 100% cpu 
>>>> time.
>>>> 
>>>> Certainly for further performance optimization you should also adjust the 
>>>> parameters of JVM after you SOLVING this.
>>>> 
>>>> 
>>>> [1] https://github.com/alibaba/arthas
>>>> 
>>>> best regards,
>>>> 
>>>> Jason
>>>> 
>>>>> On Nov 27, 2019, at 18:25, Tien Dat PHAN  wrote:
>>>>> 
>>>>> Dear all,
>>>>> 
>>>>> We just found out that the ulimit on the consumer side is relatively low 
>>>>> (ulimit -n returns 256, we are testing it on MAC OS). We suspect this one 
>>>>> may be the root cause of the low throughput that we have observed. Do any 
>>>>> of you have experience with having performance deg

Re: Maximize the consumer throughput and CPU usage

2019-11-27 Thread Jason Joo
Tien,

Or maybe you can use single consumer but set the target busy service's 
"connections" in configuration to "2" or bigger value.

If you integrated dubbo using XML:
http://dubbo.apache.org/en-us/docs/user/references/xml/dubbo-service.html 
<http://dubbo.apache.org/en-us/docs/user/references/xml/dubbo-service.html>

Or if you integrated dubbo using annotations:
@Reference(connections = 2)

And so on. Just try this for another try.


best regards,

Jason

> On Nov 27, 2019, at 22:55, Tien Dat PHAN  wrote:
> 
> Dear Jason,
> 
> You were right. The fact that when we tried to run two processes of consumer, 
> we increased the total throughput by almost 2X. This means that the 
> infrastructure and OS (the laptop and Mac OS in this case) is capable of 
> providing higher throughput.
> 
> So the suspect comes back to the configuration of the consumer process.
> 
> We have tried the Arthas tool that you suggested. A very useful tool in 
> general. However, it still does not show any interesting lead that can 
> explain the problem.
> 
> We are stuck for now :(
> 
> Best 
> Dat
> 
> On 2019/11/27 10:38:32, Jason Joo  wrote: 
>> Tien,
>> 
>> Performance issue is quite a complicated problem and the open files count 
>> you just mentioned in last mail is not the only factor.
>> According to the persistent connection in dubbo design I don't think is the 
>> cause due to your issue and maybe the value you saw in your work machine is 
>> not the same on your servers in production. And if you hit the limit some 
>> error logs will occur in your log files.
>> 
>> You mentioned that only one core can reach into a high load work status I 
>> think you should probably check the IO workers count in the configuration. 
>> And an efficient debugging tool which could work here is Arthas[1] you can 
>> dig into the JVM process and check which thread is spinning to 100% cpu time.
>> 
>> Certainly for further performance optimization you should also adjust the 
>> parameters of JVM after you SOLVING this.
>> 
>> 
>> [1] https://github.com/alibaba/arthas
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Nov 27, 2019, at 18:25, Tien Dat PHAN  wrote:
>>> 
>>> Dear all,
>>> 
>>> We just found out that the ulimit on the consumer side is relatively low 
>>> (ulimit -n returns 256, we are testing it on MAC OS). We suspect this one 
>>> may be the root cause of the low throughput that we have observed. Do any 
>>> of you have experience with having performance degradation due to system 
>>> limit? 
>>> 
>>> Best regards
>>> Tien Dat
>>> 
>>> On 2019/11/19 10:56:27, Tien Dat PHAN  wrote: 
>>>> Dear experts,
>>>> 
>>>> Could any of you shed some light on this topic?
>>>> 
>>>> Best regards
>>>> Tien Dat PHAN
>>>> 
>>>> On 2019/11/14 15:07:09, Tien Dat PHAN  wrote: 
>>>>> Dear experts,
>>>>> 
>>>>> We are having a powerful server (48 cores, four disks and 512 GB RAM) 
>>>>> providing the services.
>>>>> From my laptop (2 cores - 4 threads), we start to consume the public 
>>>>> services.
>>>>> In order to bench the throughput of our public service provider, we just 
>>>>> simply consume a very basic method (an Object receiving method, sending 
>>>>> object from consumer and provider receives it, no storage or cache is 
>>>>> invoked after that). The Object size is only 1K.
>>>>> 
>>>>> We use Dubbo protocol for our service provider.
>>>>> We noticed that: the consumer does not seem to consume to the second core 
>>>>> of my laptop (it only consumes one full core). This results in a 
>>>>> throughput of 4000 requests per second.
>>>>> To prove that our laptop can reach higher throughput, we run the second 
>>>>> process of consuming our public service. With two processes running 
>>>>> simultaneously, we can reach the throughput of 7500 request per second, 
>>>>> and almost double the CPU usage, in total.
>>>>> Our consumer implementation is multiple threads. So we don't think the 
>>>>> implementation is the issue. We suspect that some configuration is needed 
>>>>> to improve the performance and concurrency. For the above results, we 
>>>>> just use the default configuration of Dubbo.
>>>>> 
>>>>> Do you have any best practice example for improving the performance and 
>>>>> concurrency of consumer? If yes, could you please enlighten us?
>>>>> 
>>>>> Thank you guys in advance.
>>>>> 
>>>>> Best regards
>>>>> Tien Dat PHAN
>>>>> 
>>>>> 
>>>>> 
>>>> 
>> 
>> 



Re: Maximize the consumer throughput and CPU usage

2019-11-27 Thread Jason Joo
Tien,

Performance issue is quite a complicated problem and the open files count you 
just mentioned in last mail is not the only factor.
According to the persistent connection in dubbo design I don't think is the 
cause due to your issue and maybe the value you saw in your work machine is not 
the same on your servers in production. And if you hit the limit some error 
logs will occur in your log files.

You mentioned that only one core can reach into a high load work status I think 
you should probably check the IO workers count in the configuration. And an 
efficient debugging tool which could work here is Arthas[1] you can dig into 
the JVM process and check which thread is spinning to 100% cpu time.

Certainly for further performance optimization you should also adjust the 
parameters of JVM after you SOLVING this.


[1] https://github.com/alibaba/arthas

best regards,

Jason

> On Nov 27, 2019, at 18:25, Tien Dat PHAN  wrote:
> 
> Dear all,
> 
> We just found out that the ulimit on the consumer side is relatively low 
> (ulimit -n returns 256, we are testing it on MAC OS). We suspect this one may 
> be the root cause of the low throughput that we have observed. Do any of you 
> have experience with having performance degradation due to system limit? 
> 
> Best regards
> Tien Dat
> 
> On 2019/11/19 10:56:27, Tien Dat PHAN  wrote: 
>> Dear experts,
>> 
>> Could any of you shed some light on this topic?
>> 
>> Best regards
>> Tien Dat PHAN
>> 
>> On 2019/11/14 15:07:09, Tien Dat PHAN  wrote: 
>>> Dear experts,
>>> 
>>> We are having a powerful server (48 cores, four disks and 512 GB RAM) 
>>> providing the services.
>>> From my laptop (2 cores - 4 threads), we start to consume the public 
>>> services.
>>> In order to bench the throughput of our public service provider, we just 
>>> simply consume a very basic method (an Object receiving method, sending 
>>> object from consumer and provider receives it, no storage or cache is 
>>> invoked after that). The Object size is only 1K.
>>> 
>>> We use Dubbo protocol for our service provider.
>>> We noticed that: the consumer does not seem to consume to the second core 
>>> of my laptop (it only consumes one full core). This results in a throughput 
>>> of 4000 requests per second.
>>> To prove that our laptop can reach higher throughput, we run the second 
>>> process of consuming our public service. With two processes running 
>>> simultaneously, we can reach the throughput of 7500 request per second, and 
>>> almost double the CPU usage, in total.
>>> Our consumer implementation is multiple threads. So we don't think the 
>>> implementation is the issue. We suspect that some configuration is needed 
>>> to improve the performance and concurrency. For the above results, we just 
>>> use the default configuration of Dubbo.
>>> 
>>> Do you have any best practice example for improving the performance and 
>>> concurrency of consumer? If yes, could you please enlighten us?
>>> 
>>> Thank you guys in advance.
>>> 
>>> Best regards
>>> Tien Dat PHAN
>>> 
>>> 
>>> 
>> 



Re: Dubbo connection random port

2019-11-06 Thread Jason Joo
Tien,

Generally it's a server socket. So all connections to provider will use the 
specific (listening) port on PROVIDER side but it may be a random one on 
consumer side.
This is the rule from TCP. Connections are recognized by 4-element on both 
side: src:srcPort-dst:dstPort.

best regards,

Jason

> On Nov 6, 2019, at 23:07, Tien Dat PHAN  wrote:
> 
> Dear experts, 
> 
> As we know, the protocol is open on a specific port if one set the 
> "dubbo.protocol.port" parameter to a given port. 
> In addition, we just wanted to know how Dubbo provides the connection from 
> API consumer to the providers. Does it require extra ports to be open? If 
> YES, how these ports are selected?
> 
> Best 
> Tien Dat PHAN



Re: Duplicate class on dubbo-common

2019-10-25 Thread Jason Joo
Hi, tien

I think I have understood what you're talking about.
The issue occurred because one of your third-party dependencies DECLAREs that 
it requires `dubbo-serialization-kryo`. right?

The best solution is that you can just add `` lines in the section 
of that third-party dependency for `dubbo-serialization-kryo` and everything 
will go fine.

Still `dubbo` is combined of many smaller dubbo modules including 
`dubbo-serialization` as shown below:


==
jason-mbp:wav hblzxsj$ unzip -Z dubbo-serialization-kryo-2.7.4.jar  |awk 
'{print $9}' |grep class$
org/apache/dubbo/common/serialize/kryo/KryoObjectInput.class
org/apache/dubbo/common/serialize/kryo/utils/AbstractKryoFactory.class
org/apache/dubbo/common/serialize/kryo/KryoSerialization.class
org/apache/dubbo/common/serialize/kryo/CompatibleKryo.class
org/apache/dubbo/common/serialize/kryo/utils/PooledKryoFactory.class
org/apache/dubbo/common/serialize/kryo/KryoObjectOutput.class
org/apache/dubbo/common/serialize/kryo/utils/ThreadLocalKryoFactory$1.class
org/apache/dubbo/common/serialize/kryo/utils/PrototypeKryoFactory.class
org/apache/dubbo/common/serialize/kryo/utils/ReflectionUtils.class
org/apache/dubbo/common/serialize/kryo/utils/KryoUtils.class
org/apache/dubbo/common/serialize/kryo/utils/ThreadLocalKryoFactory.class
===

So when you don't have a `dubbo` dependency in your project you need 
`dubbo-serialization-kryo` but if so just ignore it.


best regards,

Jason

> On Oct 25, 2019, at 23:11, Tien Dat PHAN  wrote:
> 
> Hi Jason,
> 
> I really appreciate your attention on this issue.
> 
> As we have described, the duplicate class error is popped up when we declare 
> both dependency on dubbo and dubbo-serialization-kryo (as we need third-party 
> dependencies required by dubbo-serialization-kryo):
>
>org.apache.dubbo
>dubbo
>${dubbo.version}
>
>
>com.apache.dubbo
>dubbo-common
>
>
>
>
>org.apache.dubbo
>dubbo-serialization-kryo
>${dubbo.version}
>
>
>com.apache.dubbo
>dubbo-common
>
>
>
> 
> Of course, we can still avoid the "Duplicate class" ERROR by declaring 
> dependency only on dubbo, plus the kryo library that dubbo-serialization-kryo 
> uses:
>
>org.apache.dubbo
>dubbo
>${dubbo.version}
>
>
>com.esotericsoftware
>kryo
>4.0.1
>
> 
> If you really need a project for quicker reproduction, I can make a github 
> project. The content will most likely just these poms.
> 
> Best 
> Tien Dat
> 
> On 2019/10/25 13:13:29, Jason Joo  wrote: 
>> Hi, tien,
>> 
>> Err, I don't think the error message of duplicated class is a normal thing, 
>> could you upload a sample to reproduce it?
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Oct 25, 2019, at 19:15, Tien Dat PHAN  wrote:
>>> 
>>> Hi Jason,
>>> 
>>> Thanks for asking. Everything is working if we add this directly.
>>> And we understand your concern about having conflicting dependencies with 
>>> our application dependencies.
>>> And this is at the end a tradeoff between memory footprint optimization vs 
>>> software project management efficiency.
>>> 
>>> Our application does not need specifically com.esotericsoftware.kryo. This 
>>> is needed only by org.apache.dubbo:dubbo-serialization-kryo. And we really 
>>> don't like the fact that we need to repeat the dependency declaration on 
>>> this library, which is already defined by 
>>> org.apache.dubbo:dubbo-serialization-kryo, but got ignored. Moreover, if we 
>>> upgrade Dubbo version, we have to go check the new version of 
>>> com.esotericsoftware.kryo used by the new Dubbo, and update it accordingly.
>>> 
>>> At the end, I think we will accept the ERROR message on "Duplicate class". 
>>> As their is no win-win solution for this case.
>>> 
>>> Best regards 
>>> Dat
>>> 
>>> On 2019/10/25 10:47:32, Jason Joo  wrote: 
>>>> Tien,
>>>> 
>>>> It makes sense because it (and others like this) is an optional extension 
>>>> / support / function. We should decide to introduce the extra dependencies 
>>>> it requires when we enable it in configuration compared to the default. 
>>

Re: Duplicate class on dubbo-common

2019-10-25 Thread Jason Joo
Hi, tien,

Err, I don't think the error message of duplicated class is a normal thing, 
could you upload a sample to reproduce it?

best regards,

Jason

> On Oct 25, 2019, at 19:15, Tien Dat PHAN  wrote:
> 
> Hi Jason,
> 
> Thanks for asking. Everything is working if we add this directly.
> And we understand your concern about having conflicting dependencies with our 
> application dependencies.
> And this is at the end a tradeoff between memory footprint optimization vs 
> software project management efficiency.
> 
> Our application does not need specifically com.esotericsoftware.kryo. This is 
> needed only by org.apache.dubbo:dubbo-serialization-kryo. And we really don't 
> like the fact that we need to repeat the dependency declaration on this 
> library, which is already defined by 
> org.apache.dubbo:dubbo-serialization-kryo, but got ignored. Moreover, if we 
> upgrade Dubbo version, we have to go check the new version of 
> com.esotericsoftware.kryo used by the new Dubbo, and update it accordingly.
> 
> At the end, I think we will accept the ERROR message on "Duplicate class". As 
> their is no win-win solution for this case.
> 
> Best regards 
> Dat
> 
> On 2019/10/25 10:47:32, Jason Joo  wrote: 
>> Tien,
>> 
>> It makes sense because it (and others like this) is an optional extension / 
>> support / function. We should decide to introduce the extra dependencies it 
>> requires when we enable it in configuration compared to the default. 
>> Another similar thing is that which driver you should introduce among 
>> zookeeper / nacos / redis / others depends which implementation of registry 
>> you want to use and actually config.
>> 
>> In most cases the version may not matter so much but you'd better to pick it 
>> carefully to match just or newer to the version dubbo uses by default.
>> 
>> So is everything going well now? After introducing the necessary 
>> dependencies?
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Oct 25, 2019, at 18:19, Tien Dat PHAN  wrote:
>>> 
>>> Hi Jason,
>>> 
>>> The org.apache.dubbo:dubbo-serialization-kryo, for example, needs 
>>> com.esotericsoftware.kryo.Kryo for Kryo serialization.
>>> 
>>> org.apache.dubbo:dubbo-serialization-kryo itself has it declared in pom.xml
>>> 
>>> But you do "unzip -Z dubbo-2.7.3.jar |awk '{print $9}' |grep kryo", you 
>>> don't see it included.
>>> And we have to declare it directly in our application pom.xml
>>> We believe that this is not relevant, since we have to update the version 
>>> of com.esotericsoftware.kryo according to 
>>> org.apache.dubbo:dubbo-serialization-kryo.
>>> 
>>> Best 
>>> Dat
>>> 
>>> On 2019/10/25 08:17:18, Jason Joo  wrote: 
>>>> Hi, tien
>>>> 
>>>> `dubbo` is a "fat" jar which includes all the classes you may possibly 
>>>> use. 
>>>> 
>>>> You can confirm it using "unzip" as shown below:
>>>> 
>>>> 
>>>> bogon:2.7.3 hblzxsj$ unzip -Z dubbo-2.7.3.jar |awk '{print $9}' |grep kryo
>>>> org/apache/dubbo/common/serialize/kryo/
>>>> org/apache/dubbo/common/serialize/kryo/KryoObjectInput.class
>>>> org/apache/dubbo/common/serialize/kryo/utils/
>>>> org/apache/dubbo/common/serialize/kryo/utils/AbstractKryoFactory.class
>>>> org/apache/dubbo/common/serialize/kryo/KryoSerialization.class
>>>> org/apache/dubbo/common/serialize/kryo/utils/PooledKryoFactory.class
>>>> META-INF/maven/org.apache.dubbo/dubbo-serialization-kryo/
>>>> META-INF/maven/org.apache.dubbo/dubbo-serialization-kryo/pom.properties
>>>> org/apache/dubbo/common/serialize/kryo/CompatibleKryo.class
>>>> org/apache/dubbo/common/serialize/kryo/utils/ThreadLocalKryoFactory$1.class
>>>> org/apache/dubbo/common/serialize/kryo/KryoObjectOutput.class
>>>> org/apache/dubbo/common/serialize/kryo/utils/PrototypeKryoFactory.class
>>>> org/apache/dubbo/common/serialize/kryo/utils/ReflectionUtils.class
>>>> org/apache/dubbo/common/serialize/kryo/utils/KryoUtils.class
>>>> org/apache/dubbo/common/serialize/kryo/utils/ThreadLocalKryoFactory.class
>>>> META-INF/maven/org.apache.dubbo/dubbo-serialization-kryo/pom.xml
>>>> 
>>>> 
>>>> 
>>>> best regards,
>>>> 
>>>> Jason
>>>> 
>>>>> On Oct 25, 2019, at 16:06, Tien Dat PHAN  wrote:
>>>

Re: Duplicate class on dubbo-common

2019-10-25 Thread Jason Joo
Tien,

It makes sense because it (and others like this) is an optional extension / 
support / function. We should decide to introduce the extra dependencies it 
requires when we enable it in configuration compared to the default. 
Another similar thing is that which driver you should introduce among zookeeper 
/ nacos / redis / others depends which implementation of registry you want to 
use and actually config.

In most cases the version may not matter so much but you'd better to pick it 
carefully to match just or newer to the version dubbo uses by default.

So is everything going well now? After introducing the necessary dependencies?

best regards,

Jason

> On Oct 25, 2019, at 18:19, Tien Dat PHAN  wrote:
> 
> Hi Jason,
> 
> The org.apache.dubbo:dubbo-serialization-kryo, for example, needs 
> com.esotericsoftware.kryo.Kryo for Kryo serialization.
> 
> org.apache.dubbo:dubbo-serialization-kryo itself has it declared in pom.xml
> 
> But you do "unzip -Z dubbo-2.7.3.jar |awk '{print $9}' |grep kryo", you don't 
> see it included.
> And we have to declare it directly in our application pom.xml
> We believe that this is not relevant, since we have to update the version of 
> com.esotericsoftware.kryo according to 
> org.apache.dubbo:dubbo-serialization-kryo.
> 
> Best 
> Dat
> 
> On 2019/10/25 08:17:18, Jason Joo  wrote: 
>> Hi, tien
>> 
>> `dubbo` is a "fat" jar which includes all the classes you may possibly use. 
>> 
>> You can confirm it using "unzip" as shown below:
>> 
>> 
>> bogon:2.7.3 hblzxsj$ unzip -Z dubbo-2.7.3.jar |awk '{print $9}' |grep kryo
>> org/apache/dubbo/common/serialize/kryo/
>> org/apache/dubbo/common/serialize/kryo/KryoObjectInput.class
>> org/apache/dubbo/common/serialize/kryo/utils/
>> org/apache/dubbo/common/serialize/kryo/utils/AbstractKryoFactory.class
>> org/apache/dubbo/common/serialize/kryo/KryoSerialization.class
>> org/apache/dubbo/common/serialize/kryo/utils/PooledKryoFactory.class
>> META-INF/maven/org.apache.dubbo/dubbo-serialization-kryo/
>> META-INF/maven/org.apache.dubbo/dubbo-serialization-kryo/pom.properties
>> org/apache/dubbo/common/serialize/kryo/CompatibleKryo.class
>> org/apache/dubbo/common/serialize/kryo/utils/ThreadLocalKryoFactory$1.class
>> org/apache/dubbo/common/serialize/kryo/KryoObjectOutput.class
>> org/apache/dubbo/common/serialize/kryo/utils/PrototypeKryoFactory.class
>> org/apache/dubbo/common/serialize/kryo/utils/ReflectionUtils.class
>> org/apache/dubbo/common/serialize/kryo/utils/KryoUtils.class
>> org/apache/dubbo/common/serialize/kryo/utils/ThreadLocalKryoFactory.class
>> META-INF/maven/org.apache.dubbo/dubbo-serialization-kryo/pom.xml
>> 
>> 
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Oct 25, 2019, at 16:06, Tien Dat PHAN  wrote:
>>> 
>>> Hi Jason,
>>> 
>>> We found that is not enough.
>>> As you have pointed out the dependency on  
>>> `http://repo1.maven.org/maven2/org/apache/dubbo/`
>>> All dependencies wrapped by org.apache.dubbo:dubbo are marked as optional.
>>> This means on our application pom.xml, we have to declare the libraries 
>>> that we need directly, as they are not brought directly from 
>>> org.apache.dubbo:dubbo dependencies (check 
>>> `https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html`)
>>> 
>>> Best
>>> Dat
>>> 
>>> On 2019/10/25 01:48:29, Jason Joo  wrote: 
>>>> Hi, tien
>>>> 
>>>> There is a problem in your `dependencies` section.
>>>> `dubbo` is a single package including all codes related dubbo core like 
>>>> `netty-all` while `dubbo-serialization-kryo` is already included in 
>>>> `dubbo`.
>>>> 
>>>> If you want to do some customizing like removing some useless support in 
>>>> `dubbo` you could use another way to introduce the dependencies like: 
>>>> dubbo-common, dubbo-cluster and etc. .
>>>> 
>>>> Certainly it will be so complicated because there are so many packages to 
>>>> decide which are required to be introduced and you can see the full list 
>>>> in `http://repo1.maven.org/maven2/org/apache/dubbo/` 
>>>> <http://repo1.maven.org/maven2/org/apache/dubbo/%60>.
>>>> 
>>>> So generally we just introduce `dubbo` instead. That's already enough.
>>>> 
>>>> best regards,
>>>> 
>>>> Jason
>>>> 
>>>>> On Oct 25, 2019, at 00:31, Tien Dat PHAN  wrote:
>>>>> 
>>>>> the dubbo-common library as the suggestion with the help o
>>>> 
>>>> 
>> 
>> 



Re: Duplicate class on dubbo-common

2019-10-25 Thread Jason Joo
Hi, tien

`dubbo` is a "fat" jar which includes all the classes you may possibly use. 

You can confirm it using "unzip" as shown below:


bogon:2.7.3 hblzxsj$ unzip -Z dubbo-2.7.3.jar |awk '{print $9}' |grep kryo
org/apache/dubbo/common/serialize/kryo/
org/apache/dubbo/common/serialize/kryo/KryoObjectInput.class
org/apache/dubbo/common/serialize/kryo/utils/
org/apache/dubbo/common/serialize/kryo/utils/AbstractKryoFactory.class
org/apache/dubbo/common/serialize/kryo/KryoSerialization.class
org/apache/dubbo/common/serialize/kryo/utils/PooledKryoFactory.class
META-INF/maven/org.apache.dubbo/dubbo-serialization-kryo/
META-INF/maven/org.apache.dubbo/dubbo-serialization-kryo/pom.properties
org/apache/dubbo/common/serialize/kryo/CompatibleKryo.class
org/apache/dubbo/common/serialize/kryo/utils/ThreadLocalKryoFactory$1.class
org/apache/dubbo/common/serialize/kryo/KryoObjectOutput.class
org/apache/dubbo/common/serialize/kryo/utils/PrototypeKryoFactory.class
org/apache/dubbo/common/serialize/kryo/utils/ReflectionUtils.class
org/apache/dubbo/common/serialize/kryo/utils/KryoUtils.class
org/apache/dubbo/common/serialize/kryo/utils/ThreadLocalKryoFactory.class
META-INF/maven/org.apache.dubbo/dubbo-serialization-kryo/pom.xml



best regards,

Jason

> On Oct 25, 2019, at 16:06, Tien Dat PHAN  wrote:
> 
> Hi Jason,
> 
> We found that is not enough.
> As you have pointed out the dependency on  
> `http://repo1.maven.org/maven2/org/apache/dubbo/`
> All dependencies wrapped by org.apache.dubbo:dubbo are marked as optional.
> This means on our application pom.xml, we have to declare the libraries that 
> we need directly, as they are not brought directly from 
> org.apache.dubbo:dubbo dependencies (check 
> `https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html`)
> 
> Best
> Dat
> 
> On 2019/10/25 01:48:29, Jason Joo  wrote: 
>> Hi, tien
>> 
>> There is a problem in your `dependencies` section.
>> `dubbo` is a single package including all codes related dubbo core like 
>> `netty-all` while `dubbo-serialization-kryo` is already included in `dubbo`.
>> 
>> If you want to do some customizing like removing some useless support in 
>> `dubbo` you could use another way to introduce the dependencies like: 
>> dubbo-common, dubbo-cluster and etc. .
>> 
>> Certainly it will be so complicated because there are so many packages to 
>> decide which are required to be introduced and you can see the full list in 
>> `http://repo1.maven.org/maven2/org/apache/dubbo/` 
>> <http://repo1.maven.org/maven2/org/apache/dubbo/%60>.
>> 
>> So generally we just introduce `dubbo` instead. That's already enough.
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Oct 25, 2019, at 00:31, Tien Dat PHAN  wrote:
>>> 
>>> the dubbo-common library as the suggestion with the help o
>> 
>> 



Re: Duplicate class on dubbo-common

2019-10-24 Thread Jason Joo
Hi, tien

There is a problem in your `dependencies` section.
`dubbo` is a single package including all codes related dubbo core like 
`netty-all` while `dubbo-serialization-kryo` is already included in `dubbo`.

If you want to do some customizing like removing some useless support in 
`dubbo` you could use another way to introduce the dependencies like: 
dubbo-common, dubbo-cluster and etc. .

Certainly it will be so complicated because there are so many packages to 
decide which are required to be introduced and you can see the full list in 
`http://repo1.maven.org/maven2/org/apache/dubbo/` 
.

So generally we just introduce `dubbo` instead. That's already enough.

best regards,

Jason

> On Oct 25, 2019, at 00:31, Tien Dat PHAN  wrote:
> 
> the dubbo-common library as the suggestion with the help o



Re: How can I configure to start the client first and then start the server

2019-10-23 Thread Jason Joo
Tien,

It depends on your integrating. Generally the caching logic is existed in the 
integrating support codes.
So maybe you should point out the method you use DUBBO in. 
XML or Annotation or else. Then we can figure out the best practice is.

best regards,

Jason

> On Oct 23, 2019, at 17:46, Tien Dat PHAN  wrote:
> 
> Hi Jason,
> 
> Does this mean even I re-invoke the ReferenceConfig.get() the second time, 
> the reference may not need to be generated, as it is already cached somewhere 
> in Dubbo client holder?
> 
> Best
> Dat 
> 
> On 2019/10/23 09:43:10, Jason Joo  wrote: 
>> Hi, tien
>> 
>> Oh that's a plain example to illustrate it simply and clearly but in actual 
>> practice we use xml / annotation or other form of integrating.
>> 
>> If you use annotations in spring with `@EnableDubbo` for example you can 
>> refer the implementation of `ReferenceAnnotationBeanPostProcessor ` to see 
>> how they would be cached.
>> 
>> And in this kind of use case a reference should not be necessary to be GC 
>> once created because it will be reused repeatedly. Can it solve your problem?
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Oct 23, 2019, at 17:26, Tien Dat PHAN  wrote:
>>> 
>>> Hi Jason,
>>> 
>>> According to this: 
>>> http://dubbo.apache.org/en-us/docs/user/demos/reference-config-cache.html
>>> 
>>> If we use ReferenceConfig.get(), the usage of it remains at our concern.
>>> Does invoking it, every time we use the API, is sub-optimal?
>>> And how the reference is GCed?
>>> 
>>> Best 
>>> Tien Dat
>>> 
>>> On 2019/10/23 09:03:06, Jason Joo  wrote: 
>>>> Hi, tien
>>>> 
>>>> I just curious on why you use `ReferenceConfigCache` to get the reference? 
>>>> Why not using `@Reference` everywhere instead?
>>>> 
>>>> best regards,
>>>> 
>>>> Jason
>>>> 
>>>>> On Oct 23, 2019, at 16:55, Tien Dat PHAN  wrote:
>>>>> 
>>>>> Dear
>>>>> 
>>>>> Related to the topic of this thread, I have a question:
>>>>> When starting the consumer, we create a reference holder that is 
>>>>> initialized to hold the API reference. We use the ReferenceConfigCache to 
>>>>> keep the reference cache.
>>>>> We only invoke the ReferenceConfigCache.get(ReferenceConfig) at 
>>>>> the very beginning of the holder.
>>>>> This seems to result in bad outcome if the 
>>>>> ReferenceConfigCache.get(ReferenceConfig) is invoked when the 
>>>>> service provider is not ready (not yet exported).
>>>>> 
>>>>> Our questions are:
>>>>> 1. If this is a bad practice, should we invoke the 
>>>>> ReferenceConfigCache.get(ReferenceConfig) every time we want to 
>>>>> use it?
>>>>> 2. If the answer to the 1. question is YES. Then our concern is: Does 
>>>>> creating new API reference everytime we use it lead to sub-performance? 
>>>>> And how the system can GC the connection created everytime we invoke 
>>>>> ReferenceConfigCache.get(ReferenceConfig)? Or we, at the consumer 
>>>>> side, have to handle the connection closeness?
>>>>> 
>>>>> Best regards
>>>>> Tien Dat
>>>>> 
>>>>> On 2019/10/14 09:21:14, 郑泽超  wrote: 
>>>>>> 
>>>>>> Hello,
>>>>>> You can set check=false in your reference config
>>>>>> 
>>>>>>> 在 2019年10月14日,下午4:32,一陣風& <316272...@qq.com> 写道:
>>>>>>> 
>>>>>>> Start the client first and then start the server to report an error:
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Failed to invoke the method getBalance in the service 
>>>>>>> com.github.autfish.dubbo.service.BalanceService. No provider available 
>>>>>>> for the service com.github.autfish.dubbo.service.BalanceService from 
>>>>>>> registry localhost:9090 on the consumer 192.168.8.249 using the dubbo 
>>>>>>> version 2.7.1. Please check if the providers have been started and 
>>>>>>> registered.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> How should I configure in yml file
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>> 
>> 



Re: How can I configure to start the client first and then start the server

2019-10-23 Thread Jason Joo
Hi, tien

Oh that's a plain example to illustrate it simply and clearly but in actual 
practice we use xml / annotation or other form of integrating.

If you use annotations in spring with `@EnableDubbo` for example you can refer 
the implementation of `ReferenceAnnotationBeanPostProcessor ` to see how they 
would be cached.

And in this kind of use case a reference should not be necessary to be GC once 
created because it will be reused repeatedly. Can it solve your problem?

best regards,

Jason

> On Oct 23, 2019, at 17:26, Tien Dat PHAN  wrote:
> 
> Hi Jason,
> 
> According to this: 
> http://dubbo.apache.org/en-us/docs/user/demos/reference-config-cache.html
> 
> If we use ReferenceConfig.get(), the usage of it remains at our concern.
> Does invoking it, every time we use the API, is sub-optimal?
> And how the reference is GCed?
> 
> Best 
> Tien Dat
> 
> On 2019/10/23 09:03:06, Jason Joo  wrote: 
>> Hi, tien
>> 
>> I just curious on why you use `ReferenceConfigCache` to get the reference? 
>> Why not using `@Reference` everywhere instead?
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Oct 23, 2019, at 16:55, Tien Dat PHAN  wrote:
>>> 
>>> Dear
>>> 
>>> Related to the topic of this thread, I have a question:
>>> When starting the consumer, we create a reference holder that is 
>>> initialized to hold the API reference. We use the ReferenceConfigCache to 
>>> keep the reference cache.
>>> We only invoke the ReferenceConfigCache.get(ReferenceConfig) at the 
>>> very beginning of the holder.
>>> This seems to result in bad outcome if the 
>>> ReferenceConfigCache.get(ReferenceConfig) is invoked when the 
>>> service provider is not ready (not yet exported).
>>> 
>>> Our questions are:
>>> 1. If this is a bad practice, should we invoke the 
>>> ReferenceConfigCache.get(ReferenceConfig) every time we want to use 
>>> it?
>>> 2. If the answer to the 1. question is YES. Then our concern is: Does 
>>> creating new API reference everytime we use it lead to sub-performance? And 
>>> how the system can GC the connection created everytime we invoke 
>>> ReferenceConfigCache.get(ReferenceConfig)? Or we, at the consumer 
>>> side, have to handle the connection closeness?
>>> 
>>> Best regards
>>> Tien Dat
>>> 
>>> On 2019/10/14 09:21:14, 郑泽超  wrote: 
>>>> 
>>>> Hello,
>>>> You can set check=false in your reference config
>>>> 
>>>>> 在 2019年10月14日,下午4:32,一陣風& <316272...@qq.com> 写道:
>>>>> 
>>>>> Start the client first and then start the server to report an error:
>>>>> 
>>>>> 
>>>>> 
>>>>> Failed to invoke the method getBalance in the service 
>>>>> com.github.autfish.dubbo.service.BalanceService. No provider available 
>>>>> for the service com.github.autfish.dubbo.service.BalanceService from 
>>>>> registry localhost:9090 on the consumer 192.168.8.249 using the dubbo 
>>>>> version 2.7.1. Please check if the providers have been started and 
>>>>> registered.
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> How should I configure in yml file
>>>> 
>>>> 
>> 
>> 



Re: How can I configure to start the client first and then start the server

2019-10-23 Thread Jason Joo
Hi, tien

I just curious on why you use `ReferenceConfigCache` to get the reference? Why 
not using `@Reference` everywhere instead?

best regards,

Jason

> On Oct 23, 2019, at 16:55, Tien Dat PHAN  wrote:
> 
> Dear
> 
> Related to the topic of this thread, I have a question:
> When starting the consumer, we create a reference holder that is initialized 
> to hold the API reference. We use the ReferenceConfigCache to keep the 
> reference cache.
> We only invoke the ReferenceConfigCache.get(ReferenceConfig) at the 
> very beginning of the holder.
> This seems to result in bad outcome if the 
> ReferenceConfigCache.get(ReferenceConfig) is invoked when the service 
> provider is not ready (not yet exported).
> 
> Our questions are:
> 1. If this is a bad practice, should we invoke the 
> ReferenceConfigCache.get(ReferenceConfig) every time we want to use 
> it?
> 2. If the answer to the 1. question is YES. Then our concern is: Does 
> creating new API reference everytime we use it lead to sub-performance? And 
> how the system can GC the connection created everytime we invoke 
> ReferenceConfigCache.get(ReferenceConfig)? Or we, at the consumer 
> side, have to handle the connection closeness?
> 
> Best regards
> Tien Dat
> 
> On 2019/10/14 09:21:14, 郑泽超  wrote: 
>> 
>> Hello,
>> You can set check=false in your reference config
>> 
>>> 在 2019年10月14日,下午4:32,一陣風& <316272...@qq.com> 写道:
>>> 
>>> Start the client first and then start the server to report an error:
>>> 
>>> 
>>> 
>>> Failed to invoke the method getBalance in the service 
>>> com.github.autfish.dubbo.service.BalanceService. No provider available for 
>>> the service com.github.autfish.dubbo.service.BalanceService from registry 
>>> localhost:9090 on the consumer 192.168.8.249 using the dubbo version 2.7.1. 
>>> Please check if the providers have been started and registered.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> How should I configure in yml file
>> 
>> 



Re: [VOTE] Accept RitterHou/python-dubbo donation

2019-10-23 Thread Jason Joo
Hi, Ian

Generally in practice it's good to create a new repository with a new name (eg. 
log4j, log4j2, while they haven't so much inner connections). My proposal is to 
name it with something like `dubbo-python2` or others.

Just for reference.


best regards,

Jason

> On Oct 23, 2019, at 16:00, Ian Luo  wrote:
> 
> Hi Folks,
> 
> I think I have finished all steps to move this project into apache group
> except for the last one step. Before I begins, I'd like to discuss with you
> that if it is appropriate to replace the existing dubbo-python [1] entirely
> with this new implementation, which is much better, from my own opinion.
> BTW, in case of you have not realized, the existing dubbo-python [1] is
> also donated from Qianmi too.
> 
> Regards,
> -Ian.
> 
> 
> 1. https://github.com/apache/dubbo-python
> 
> On Thu, Oct 17, 2019 at 5:17 PM Ian Luo  wrote:
> 
>> Hi,
>> 
>> After waiting more than 72 hours, the vote has passed with: 5 +2
>> binding vote and no -1 binding vote.
>> 
>> +1 binding: Yuhang Xiu, Minxuan Zhuang, Xin Wang, Mercy Ma, Ian Luo
>> +1 non-binding: Yuhao Bi, Zhechao Zheng
>> 
>> 
>> Regards,
>> -Ian.
>> 
>> On Tue, Sep 24, 2019 at 2:34 PM 郑泽超  wrote:
>> 
>>> +1 accept.
>>> It is a good news for pythoner~
>>> 
>> 



Re: [VOTE]: Release Apache Dubbo 2.7.4 [RC2]

2019-10-21 Thread Jason Joo
Hi, victory


Thank you for your following.
And the issue is #5186 (https://github.com/apache/dubbo/issues/5186 
<https://github.com/apache/dubbo/issues/5186>)
We will not need a new issue I think.

And I filed two issues one of which is a resubmitted for quite an old one to 
mark it out that we should 
reach a conclusion to do or not to do. (Which was submitted by @Ianluo 
originally last year)

best regards,

Jason

> On Oct 21, 2019, at 16:15, victory  wrote:
> 
> I will check this problem, and I will give you reply.
> If this problem is  servere and urgent,  I will try to fix it in a fix
> version on this week.
> 
> Jason Joo  于2019年10月21日周一 下午3:26写道:
> 
>> And I'm ok to file a NEW issue but will anyone to discuss and follow
>> together?
>> 
>> It seems that most guys are busy currently. Maybe for the event of 11.11?
>> This issue would affect quite a large amount of upgraded users.
>> 
>> An active issue needs our following up in time so we can do it better.
>> 
>> And I also spend quite a lot of time in debugging and analyzing related
>> codes to implement a multiple registries configuration. This is also
>> undocumented and I feel a little buggy after trying all possible ways.
>> Thank god finally we got one correct way to achieve that.
>> And is this also necessary to file an issue for discussing if anyone liked
>> involving in?
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Oct 21, 2019, at 15:04, bailei  wrote:
>>> 
>>> IMO, it is quite serve.
>>> 
>>> Step 1: Upgrade dubbo to 2.7.4 in dubbo-samples-api.
>>> Step 2: Start `org.apache.dubbo.samples.provider.Application`. Will
>> report an NPE problem.
>>> 
>>>> 在 2019年10月21日,下午2:30,victory  写道:
>>>> 
>>>> I have released dubbo-2.7.4 two days ago.
>>>> If you have some problem, please file an issue on
>>>> https://github.com/apache/dubbo/issues
>>>> If your problem is very severe,  we will try to fix in next release as
>> soon
>>>> as possible.
>>>> 
>>>> Jason Joo  于2019年10月21日周一 上午8:49写道:
>>>> 
>>>>> -1.
>>>>> 
>>>>> So will it be included in 2.7.4 or 2.7.5?
>>>>> 
>>>>> One of our project using annotations came to introduce multiple
>>>>> registries. But we found that there was a bug on subscribing same
>> interface
>>>>> from different registries before 2.7.3(inclusive) util #5033(
>>>>> https://github.com/apache/dubbo/pull/5033) had been merged.
>>>>> 
>>>>> But after updating to branch of 2.7.4 we continue to suffer the NPE
>> issue
>>>>> (Location AbstractInterfaceConfig.useRegistryForConfigIfNecessary
>> which is
>>>>> mentioned by @bailei).
>>>>> 
>>>>> We will make a temporary build first in our private repository and I
>>>>> suggest it is necessary being included in 2.7.4. Btw, `sonymoon` has
>>>>> already given a fix in his repo (
>>>>> 
>> https://github.com/sonymoon/dubbo/commit/7b73f52349d7a37572bb724b28ec62e08d3a29fd
>>>>> ).
>>>>> 
>>>>> And more pls consider that how to solve the scanning problem mentioned
>> in
>>>>> issue #4895 (https://github.com/apache/dubbo/pull/4895) because it is
>>>>> long time no response. It is not as urgent as the NPE issue.
>>>>> 
>>>>> And I will close #4199 now and I think we'd better to describe it in
>>>>> document for those which have mixed versions deployed in environment
>> and
>>>>> also have implemented MonitorService. (The simple solution is just
>> copying
>>>>> the legacy interface into project and public them together).
>>>>> 
>>>>> best regards,
>>>>> 
>>>>> Jason
>>>>> 
>>>>>> On Oct 20, 2019, at 15:00, bailei  wrote:
>>>>>> 
>>>>>> -1.
>>>>>> I don’t know if I am qualified from voting.
>>>>>> Is 2.7.4 released?
>>>>>> I think there is a issue need to be checked (
>>>>> https://github.com/apache/dubbo/issues/5186 <
>>>>> https://github.com/apache/dubbo/issues/5186>). Fix me if I was wrong.
>>>>>> 
>>>>>> 
>>>>>>> 在 2019年10月18日,上午9:46,victory  写道:
>>>>>>> 
>>>>>>> We’ve received 3 +1 binding vote

Re: [VOTE]: Release Apache Dubbo 2.7.4 [RC2]

2019-10-21 Thread Jason Joo
And I'm ok to file a NEW issue but will anyone to discuss and follow together? 

It seems that most guys are busy currently. Maybe for the event of 11.11? This 
issue would affect quite a large amount of upgraded users.

An active issue needs our following up in time so we can do it better.

And I also spend quite a lot of time in debugging and analyzing related codes 
to implement a multiple registries configuration. This is also undocumented and 
I feel a little buggy after trying all possible ways. Thank god finally we got 
one correct way to achieve that.
And is this also necessary to file an issue for discussing if anyone liked 
involving in?

best regards,

Jason

> On Oct 21, 2019, at 15:04, bailei  wrote:
> 
> IMO, it is quite serve.
> 
> Step 1: Upgrade dubbo to 2.7.4 in dubbo-samples-api.
> Step 2: Start `org.apache.dubbo.samples.provider.Application`. Will report an 
> NPE problem.
> 
>> 在 2019年10月21日,下午2:30,victory  写道:
>> 
>> I have released dubbo-2.7.4 two days ago.
>> If you have some problem, please file an issue on
>> https://github.com/apache/dubbo/issues
>> If your problem is very severe,  we will try to fix in next release as soon
>> as possible.
>> 
>> Jason Joo  于2019年10月21日周一 上午8:49写道:
>> 
>>> -1.
>>> 
>>> So will it be included in 2.7.4 or 2.7.5?
>>> 
>>> One of our project using annotations came to introduce multiple
>>> registries. But we found that there was a bug on subscribing same interface
>>> from different registries before 2.7.3(inclusive) util #5033(
>>> https://github.com/apache/dubbo/pull/5033) had been merged.
>>> 
>>> But after updating to branch of 2.7.4 we continue to suffer the NPE issue
>>> (Location AbstractInterfaceConfig.useRegistryForConfigIfNecessary which is
>>> mentioned by @bailei).
>>> 
>>> We will make a temporary build first in our private repository and I
>>> suggest it is necessary being included in 2.7.4. Btw, `sonymoon` has
>>> already given a fix in his repo (
>>> https://github.com/sonymoon/dubbo/commit/7b73f52349d7a37572bb724b28ec62e08d3a29fd
>>> ).
>>> 
>>> And more pls consider that how to solve the scanning problem mentioned in
>>> issue #4895 (https://github.com/apache/dubbo/pull/4895) because it is
>>> long time no response. It is not as urgent as the NPE issue.
>>> 
>>> And I will close #4199 now and I think we'd better to describe it in
>>> document for those which have mixed versions deployed in environment and
>>> also have implemented MonitorService. (The simple solution is just copying
>>> the legacy interface into project and public them together).
>>> 
>>> best regards,
>>> 
>>> Jason
>>> 
>>>> On Oct 20, 2019, at 15:00, bailei  wrote:
>>>> 
>>>> -1.
>>>> I don’t know if I am qualified from voting.
>>>> Is 2.7.4 released?
>>>> I think there is a issue need to be checked (
>>> https://github.com/apache/dubbo/issues/5186 <
>>> https://github.com/apache/dubbo/issues/5186>). Fix me if I was wrong.
>>>> 
>>>> 
>>>>> 在 2019年10月18日,上午9:46,victory  写道:
>>>>> 
>>>>> We’ve received 3 +1 binding votes and 2 +1 non-binding vote: +1 binding,
>>>>> Mercy Ma +1 binding, Justin Mclean +1 binding, Jun Liu +1 non-binding,
>>> Lonre
>>>>> Wang
>>>>> +1 non-binding ,  郑泽超
>>>>> 
>>>>> I will release dubbo-2.7.4 today. Best regards, The Apache Dubbo
>>>>> (Incubating) Team
>>>>> 
>>>>> Justin Mclean  于2019年10月16日周三 上午10:58写道:
>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> +1 (binding)
>>>>>> 
>>>>>> I checked:
>>>>>> - signatures and hashes fine
>>>>>> - LICENSE and NOTICE OK
>>>>>> - no unexpected binary files
>>>>>> - source files have ASF headers
>>>>>> - can compile from source
>>>>>> 
>>>>>> RE the software grant normally when a copyright is removed and the
>>> headers
>>>>>> replaced you need to put something in NOTICE saying this was done. If
>>>>>> original headers are left in there then this should be mentioned in
>>>>>> LICENSE. I suggest you fix this in the next release
>>>>>> 
>>>>>> Thanks,
>>>>>> Justin
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> 
>>>>> Best Regard!
>>>>> cvitory
>>>> 
>>> 
>>> 
>> 
>> -- 
>> 
>> Best Regard!
>> cvitory



Re: [VOTE]: Release Apache Dubbo 2.7.4 [RC2]

2019-10-20 Thread Jason Joo
-1.

So will it be included in 2.7.4 or 2.7.5?

One of our project using annotations came to introduce multiple registries. But 
we found that there was a bug on subscribing same interface from different 
registries before 2.7.3(inclusive) util 
#5033(https://github.com/apache/dubbo/pull/5033) had been merged.

But after updating to branch of 2.7.4 we continue to suffer the NPE issue 
(Location AbstractInterfaceConfig.useRegistryForConfigIfNecessary which is 
mentioned by @bailei).  

We will make a temporary build first in our private repository and I suggest it 
is necessary being included in 2.7.4. Btw, `sonymoon` has already given a fix 
in his repo 
(https://github.com/sonymoon/dubbo/commit/7b73f52349d7a37572bb724b28ec62e08d3a29fd).

And more pls consider that how to solve the scanning problem mentioned in issue 
#4895 (https://github.com/apache/dubbo/pull/4895) because it is long time no 
response. It is not as urgent as the NPE issue.

And I will close #4199 now and I think we'd better to describe it in document 
for those which have mixed versions deployed in environment and also have 
implemented MonitorService. (The simple solution is just copying the legacy 
interface into project and public them together).

best regards,

Jason

> On Oct 20, 2019, at 15:00, bailei  wrote:
> 
> -1. 
> I don’t know if I am qualified from voting. 
> Is 2.7.4 released?
> I think there is a issue need to be checked 
> (https://github.com/apache/dubbo/issues/5186 
> ). Fix me if I was wrong.
> 
> 
>> 在 2019年10月18日,上午9:46,victory  写道:
>> 
>> We’ve received 3 +1 binding votes and 2 +1 non-binding vote: +1 binding,
>> Mercy Ma +1 binding, Justin Mclean +1 binding, Jun Liu +1 non-binding, Lonre
>> Wang
>> +1 non-binding ,  郑泽超
>> 
>> I will release dubbo-2.7.4 today. Best regards, The Apache Dubbo
>> (Incubating) Team
>> 
>> Justin Mclean  于2019年10月16日周三 上午10:58写道:
>> 
>>> Hi,
>>> 
>>> +1 (binding)
>>> 
>>> I checked:
>>> - signatures and hashes fine
>>> - LICENSE and NOTICE OK
>>> - no unexpected binary files
>>> - source files have ASF headers
>>> - can compile from source
>>> 
>>> RE the software grant normally when a copyright is removed and the headers
>>> replaced you need to put something in NOTICE saying this was done. If
>>> original headers are left in there then this should be mentioned in
>>> LICENSE. I suggest you fix this in the next release
>>> 
>>> Thanks,
>>> Justin
>> 
>> 
>> 
>> -- 
>> 
>> Best Regard!
>> cvitory
> 



Re: Customize the target content to be hashed with Consistent hash load balance

2019-09-25 Thread Jason Joo
Hi, tien

It's such a matter that can hardly say right or wrong. `hashCode` also has its 
own purpose but not for load balancing. I agree with you that hashCode() is a 
better practice to toString() if we took default implementation away.

Maybe we can make it a configurable item in method level, class level or also 
global level like:

algorithm = "hash" / "string" / etc...

or make it an annotation like: (more aggressive, bad practice)

@ConsistentHash
public String some() {}

Maybe the first way is more acceptable.

best regards,

Jason

> On Sep 25, 2019, at 18:05, Tien Dat PHAN  wrote:
> 
> Dear Jason,
> 
> Thanks for your answer.
> 
> The default 'hashCode()' is inconsistent between JVMs. That is why we suggest 
> to override it with a customized hashCode generating method, that, of course, 
> has to make sure the consistency across multi-JVMs.
> 
> On the other hand, 'toString()' can be use as a replacement. But this mean we 
> just break the original purpose of 'toString()' method.
> 
> At the end, it is clear that using 'hashCode()' will not guarantee the 
> consistency between multiple JVMs for default JAVA objects. But with 
> customized JAVA objects, with of course proper 'hashCode' implementation, 
> using 'hashCode' would be the most relevant approach for Consistent hash 
> algorithm.
> 
> So do you think it would be reasonable to enable this in the future as an 
> additional option? The default option is still using 'toString' method.
> 
> Best
> Tien Dat
> 
> On 2019/09/25 01:14:39, Jason Joo  wrote: 
>> Hi, tien
>> 
>> `hashCode` maybe consistent in single node but will be inconsistent between 
>> JVM processes.(under default implementation)
>> 
>> And you also have the ability to override the `toString()` method.
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Sep 24, 2019, at 22:00, Tien Dat PHAN  wrote:
>>> 
>>> Hi Jason,
>>> 
>>> I just had some time looking back at this thread. 
>>> I just realized that it may be useful if we apply the consistent hash 
>>> algorithm on simply the hashCode() of the first parameters. I think it 
>>> would be better compare to use the toString() method.
>>> 
>>> With that, users can override the hashCode() method of their own objects. 
>>> 
>>> What do you think?
>>> 
>>> Best regards
>>> Tien Dat PHAN
>>> 
>>> On 2019/06/12 11:56:25, Jason Joo  wrote: 
>>>> Hi, Dat
>>>> 
>>>> Stand at the point of DUBBO maybe it is too complex to support a grammar 
>>>> as EL expressions like 'param[0].url' due to invokers are so dynamic in 
>>>> same invocation group.
>>>> 
>>>> So I think taking an obvious and special partition/hashing parameter is 
>>>> acceptable for me instead of taking it as a kind of duplicated information.
>>>> 
>>>> Again if you have better ideas it could be welcome in community.
>>>> 
>>>> 
>>>> best regards,
>>>> 
>>>> Jason
>>>> 
>>>>> On Jun 12, 2019, at 19:34, tphan@gmail.com wrote:
>>>>> 
>>>>> Hi Jason,
>>>>> 
>>>>> Thanks for sharing your opinion :) 
>>>>> This is actually what we are currently doing, for now, as we do not find 
>>>>> a better solution for this yet.
>>>>> 
>>>>> Best
>>>>> Dat
>>>>> 
>>>>> On 2019/06/12 08:28:32, Jason Joo  wrote: 
>>>>>> Hi, Tien
>>>>>> 
>>>>>> Agree to you that it will break the principle of design if overriding 
>>>>>> toString().
>>>>>> Currently consist hash of load balance only support specifying which 
>>>>>> argument(s) can take charge in it but not support specify certain 
>>>>>> property of a parameter to do so.
>>>>>> 
>>>>>> For better understanding maybe add a parameter like
>>>>>> 
>>>>>>> MyResponse getResponse(String partitionKey, MyRequestParams params);
>>>>>> 
>>>>>> 
>>>>>> may be more easy.
>>>>>> 
>>>>>> It's only my opinion.
>>>>>> 
>>>>>> 
>>>>>> best regards,
>>>>>> 
>>>>>> Jason
>>>>>> 
>>>>>>&g

Re: Customize the target content to be hashed with Consistent hash load balance

2019-09-24 Thread Jason Joo
Hi, tien

`hashCode` maybe consistent in single node but will be inconsistent between JVM 
processes.(under default implementation)

And you also have the ability to override the `toString()` method.

best regards,

Jason

> On Sep 24, 2019, at 22:00, Tien Dat PHAN  wrote:
> 
> Hi Jason,
> 
> I just had some time looking back at this thread. 
> I just realized that it may be useful if we apply the consistent hash 
> algorithm on simply the hashCode() of the first parameters. I think it would 
> be better compare to use the toString() method.
> 
> With that, users can override the hashCode() method of their own objects. 
> 
> What do you think?
> 
> Best regards
> Tien Dat PHAN
> 
> On 2019/06/12 11:56:25, Jason Joo  wrote: 
>> Hi, Dat
>> 
>> Stand at the point of DUBBO maybe it is too complex to support a grammar as 
>> EL expressions like 'param[0].url' due to invokers are so dynamic in same 
>> invocation group.
>> 
>> So I think taking an obvious and special partition/hashing parameter is 
>> acceptable for me instead of taking it as a kind of duplicated information.
>> 
>> Again if you have better ideas it could be welcome in community.
>> 
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Jun 12, 2019, at 19:34, tphan@gmail.com wrote:
>>> 
>>> Hi Jason,
>>> 
>>> Thanks for sharing your opinion :) 
>>> This is actually what we are currently doing, for now, as we do not find a 
>>> better solution for this yet.
>>> 
>>> Best
>>> Dat
>>> 
>>> On 2019/06/12 08:28:32, Jason Joo  wrote: 
>>>> Hi, Tien
>>>> 
>>>> Agree to you that it will break the principle of design if overriding 
>>>> toString().
>>>> Currently consist hash of load balance only support specifying which 
>>>> argument(s) can take charge in it but not support specify certain property 
>>>> of a parameter to do so.
>>>> 
>>>> For better understanding maybe add a parameter like
>>>> 
>>>>> MyResponse getResponse(String partitionKey, MyRequestParams params);
>>>> 
>>>> 
>>>> may be more easy.
>>>> 
>>>> It's only my opinion.
>>>> 
>>>> 
>>>> best regards,
>>>> 
>>>> Jason
>>>> 
>>>>> On Jun 12, 2019, at 15:31, tphan@gmail.com wrote:
>>>>> 
>>>>> Dear Jason,
>>>>> 
>>>>> I am aware of that. 
>>>>> However, as the MyRequestParams has multiple fields, we would like to 
>>>>> avoid implementing the toString method that return a string containing 
>>>>> only the URI field.
>>>>> This would break the purpose of the toString method.
>>>>> 
>>>>> Do you know any other approaches/work around for this?
>>>>> 
>>>>> Best
>>>>> Dat
>>>>> 
>>>>> On 2019/06/12 00:56:42, Jason Joo  wrote: 
>>>>>> Hi, Tien
>>>>>> 
>>>>>> Consistent hash is relying mainly the object's "toString()" protocol. So 
>>>>>> pls make sure your objects have a unique and stable toString() method 
>>>>>> and it will be ok.
>>>>>> 
>>>>>> best regards,
>>>>>> 
>>>>>> Jason
>>>>>> 
>>>>>>> On Jun 11, 2019, at 16:58, tphan@gmail.com wrote:
>>>>>>> 
>>>>>>> Dear experts,
>>>>>>> 
>>>>>>> We are using Dubbo to export our services. For data load balancing, we 
>>>>>>> would like to use the consistent hash load balancer.
>>>>>>> For the default usage of this load balancer, we can define either one 
>>>>>>> or several parameters of the consumed method to be used as the hashing 
>>>>>>> data.
>>>>>>> We wonder if it is possible to target the hashing on the field of an 
>>>>>>> object passed as the param.
>>>>>>> For instance, the method interface is as follows:
>>>>>>> 
>>>>>>> MyResponse getResponse(MyRequestParams params);
>>>>>>> 
>>>>>>> The object params has a field named URI, which we would like to apply 
>>>>>>> the hash algorithm upon.
>>>>>>> Of course, we can modify our method to:
>>>>>>> 
>>>>>>> MyResponse getResponse(String uri, MyRequestParams params);
>>>>>>> 
>>>>>>> But this really is contradictory with the design goal of MyRequestParam 
>>>>>>> object (which is defined to wrap all our request params into one object)
>>>>>>> 
>>>>>>> Best regards
>>>>>>> Tien Dat PHAN
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>> 
>> 



Re: Gracefully unexporting/passivating the exported services

2019-09-24 Thread Jason Joo
Hi, tien

So far as I know there wasn't.
In my opinion it should not be complicated to implement similar function in 
such scenario. 
So the road is under your feet, haha

best regards,

Jason

> On Sep 24, 2019, at 15:25, Tien Dat PHAN  wrote:
> 
> Dear Jason,
> 
> Thank you for your interesting material.
> This would answer the need of our use case.
> Just a small question is: Does Dubbo provide a JAVA API for such QoS features?
> 
> Best
> Tien Dat
> 
> On 2019/09/24 00:50:58, Jason Joo  wrote: 
>> Hi Tien
>> 
>> You can refer to QOS feature of DUBBO.
>> 
>> QoS's Parameters: 
>> http://dubbo.apache.org/en-us/docs/user/references/qos.html 
>> <http://dubbo.apache.org/en-us/docs/user/references/qos.html>
>> Usage: https://dubbo.apache.org/en-us/blog/introduction-to-dubbo-qos.html 
>> <https://dubbo.apache.org/en-us/blog/introduction-to-dubbo-qos.html>
>> 
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Sep 23, 2019, at 16:50, Tien Dat PHAN  wrote:
>>> 
>>> Dear experts,
>>> 
>>> We are using Dubbo for exporting our services to outside clients.
>>> During the execution, we may have to switch some nodes into maintenance 
>>> mode (for maintenance purposes of course), which means those nodes are not 
>>> serving requests until they are switched back to normal mode.
>>> 
>>> Our question is: what is the recommended procedure to gracefully put an 
>>> exported service into maintenance mode, without causing a perturbation on 
>>> the client side, which may still maintain an open connection with the 
>>> exported service?
>>> 
>>> Best regards
>>> Tien Dat PHAN
>> 
>> 



Re: Gracefully unexporting/passivating the exported services

2019-09-23 Thread Jason Joo
Hi Tien

You can refer to QOS feature of DUBBO.

QoS's Parameters: http://dubbo.apache.org/en-us/docs/user/references/qos.html 

Usage: https://dubbo.apache.org/en-us/blog/introduction-to-dubbo-qos.html 



best regards,

Jason

> On Sep 23, 2019, at 16:50, Tien Dat PHAN  wrote:
> 
> Dear experts,
> 
> We are using Dubbo for exporting our services to outside clients.
> During the execution, we may have to switch some nodes into maintenance mode 
> (for maintenance purposes of course), which means those nodes are not serving 
> requests until they are switched back to normal mode.
> 
> Our question is: what is the recommended procedure to gracefully put an 
> exported service into maintenance mode, without causing a perturbation on the 
> client side, which may still maintain an open connection with the exported 
> service?
> 
> Best regards
> Tien Dat PHAN



Re: 2.7.4 release plan

2019-09-09 Thread Jason Joo
Hi, victory

There's one old issue (https://github.com/apache/dubbo/issues/4693) which 
hasn't got any further response or discussion for long time but closed five 
days ago.
I want to figure out should we should treat it not an issue? Because there was 
one related PR (https://github.com/apache/dubbo/pull/4895) which contains 
unfinished discussion.

best regards,

Jason

> On Sep 9, 2019, at 15:06, victory  wrote:
> 
> [



Re: Using FileInputStream with dubbo protocol

2019-07-23 Thread Jason Joo
Hi Tien

My suggestion is that we should not slack on it when we use tradition 
REQEUST-RESPONSE model.

1. Stream Object couldn't be transferred
DUBBO use an embedded version of hessian which is the project 
"dubbo-hessian-lite". If you use a unserializable parameter in your interface 
then you will get a null value.

2. Several Serializations integrated into Dubbo
When we use model of REQUEST-RESPONSE we must consider how to generate the 
complete request for a correct response. So how to give stream-like (not 
stream-style) requests the ability of keeping working when response is 
generating is a big task to solve.

3. Make Everything in the AIR is difficult
I know your purpose is to let things in the air to simplify the job or improve 
the performance. But even load balancer or container like tomcat will create 
temporarily file on disk (There is a threshold of file size) to hold it before 
proxying the request. So if you can make things in the air it depends on your 
scenario and consideration of that.

My suggestion is:
It may has no general solution. You should design it according your actual 
cases. For files small enough considering byte[]; If they are large consider to 
find a sharing place (Distributed storage / NFS), or split them into pieces to 
consume them.

best regards,

Jason

> On Jul 23, 2019, at 14:56, tphan@gmail.com wrote:
> 
> Hi Jason,
> 
> I just learnt that Hessian protocol with Hessian2 support InputStream 
> serialization.
> Have you ever played with this protocol for large file transfer?
> 
> Best
> Tien Dat 
> 
> On 2019/07/19 17:58:56, Jason Joo  wrote: 
>> Hi, Tien
>> 
>> Questions on the road map it's better to turn to @lan luo. He will get you 
>> know that I think. And I should say that stream type is quite different to 
>> streaming style api.
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Jul 19, 2019, at 15:25, tphan@gmail.com wrote:
>>> 
>>> Hi Jason,
>>> 
>>> Thanks for your advice.
>>> These solutions appear to be the most relevant for our usecase, for now.
>>> 
>>> We just wonder also if it will be addressed in upcoming version of Dubbo, 
>>> to support InputStream in the public APIs? Since we believe that 
>>> InputStream itself is a complete and efficient interface for transferring 
>>> large series of data, and we don't think that one can make it equivalent, 
>>> let alone makeing it better by re-implementing the functionality of the 
>>> InputStream (splitting the large files and transfer piece by piece)
>>> 
>>> As far as we know, Dubbo version 3.x will come with reactive stream 
>>> supported.
>>> So whether the InputStream can be also supported in this version?
>>> 
>>> Best
>>> Tien Dat
>>> 
>>> On 2019/07/19 01:07:51, Jason Joo  wrote: 
>>>> Hi, Tien
>>>> 
>>>> You can only use serializable types as parameters in API definition and 
>>>> maybe you cannot do that.
>>>> 
>>>> You can choose an alternative design like:
>>>> 1. use byte[] (When the size of file is small) (If it's large or 
>>>> undetermined splitting it into pieces will also work)
>>>> 2. use a place to store temporarily and then use the address (like url) to 
>>>> identify it.
>>>> 
>>>> best regards,
>>>> 
>>>> Jason
>>>> 
>>>>> On Jul 18, 2019, at 19:18, tphan@gmail.com wrote:
>>>>> 
>>>>> Dear experts
>>>>> 
>>>>> We are newbie with Dubbo.
>>>>> What we are trying to do is providing a public API that supports 
>>>>> uploading the content of a file.
>>>>> As a result, the API method has a parameter as FileInputStream.
>>>>> 
>>>>> However, the dubbo protocol does not support Streaming.
>>>>> 
>>>>> Do you know how we can implement our API with Dubbo?
>>>>> 
>>>>> Best
>>>>> Tien Dat
>>>> 
>>>> 
>> 
>> 



Re: Using FileInputStream with dubbo protocol

2019-07-19 Thread Jason Joo
Hi, Tien

Questions on the road map it's better to turn to @lan luo. He will get you know 
that I think. And I should say that stream type is quite different to streaming 
style api.

best regards,

Jason

> On Jul 19, 2019, at 15:25, tphan@gmail.com wrote:
> 
> Hi Jason,
> 
> Thanks for your advice.
> These solutions appear to be the most relevant for our usecase, for now.
> 
> We just wonder also if it will be addressed in upcoming version of Dubbo, to 
> support InputStream in the public APIs? Since we believe that InputStream 
> itself is a complete and efficient interface for transferring large series of 
> data, and we don't think that one can make it equivalent, let alone makeing 
> it better by re-implementing the functionality of the InputStream (splitting 
> the large files and transfer piece by piece)
> 
> As far as we know, Dubbo version 3.x will come with reactive stream supported.
> So whether the InputStream can be also supported in this version?
> 
> Best
> Tien Dat
> 
> On 2019/07/19 01:07:51, Jason Joo  wrote: 
>> Hi, Tien
>> 
>> You can only use serializable types as parameters in API definition and 
>> maybe you cannot do that.
>> 
>> You can choose an alternative design like:
>> 1. use byte[] (When the size of file is small) (If it's large or 
>> undetermined splitting it into pieces will also work)
>> 2. use a place to store temporarily and then use the address (like url) to 
>> identify it.
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Jul 18, 2019, at 19:18, tphan@gmail.com wrote:
>>> 
>>> Dear experts
>>> 
>>> We are newbie with Dubbo.
>>> What we are trying to do is providing a public API that supports uploading 
>>> the content of a file.
>>> As a result, the API method has a parameter as FileInputStream.
>>> 
>>> However, the dubbo protocol does not support Streaming.
>>> 
>>> Do you know how we can implement our API with Dubbo?
>>> 
>>> Best
>>> Tien Dat
>> 
>> 



Re: Using FileInputStream with dubbo protocol

2019-07-18 Thread Jason Joo
Hi, Tien

You can only use serializable types as parameters in API definition and maybe 
you cannot do that.

You can choose an alternative design like:
1. use byte[] (When the size of file is small) (If it's large or undetermined 
splitting it into pieces will also work)
2. use a place to store temporarily and then use the address (like url) to 
identify it.

best regards,

Jason

> On Jul 18, 2019, at 19:18, tphan@gmail.com wrote:
> 
> Dear experts
> 
> We are newbie with Dubbo.
> What we are trying to do is providing a public API that supports uploading 
> the content of a file.
> As a result, the API method has a parameter as FileInputStream.
> 
> However, the dubbo protocol does not support Streaming.
> 
> Do you know how we can implement our API with Dubbo?
> 
> Best
> Tien Dat



Re: Customizing the Zookeeper Znode name for Dubbo

2019-07-12 Thread Jason Joo
Hi, Tien

Good to know it. Because we use several bases together we didn't notice it.

Thank you

best regards,

Jason

> On Jul 12, 2019, at 18:45, tphan@gmail.com wrote:
> 
> FYI, that this metadata Znode is configured by the MetadataReportConfig.
> Just needed to set the right MetadataReportConfig (similarly to 
> RegistryConfig, we just need to set the right group directory)
> 
> Best
> 
> On 2019/07/12 10:03:56, tphan@gmail.com  wrote: 
>> Dear Jason,
>> 
>> I have just noticed an extra /dubbo znode is still created.
>> This Znode contains the metadata directory.
>> Is this out of the group name setting?
>> 
>> Since it would be better to collocated both the public APIs states and the 
>> metadata within the same parent Znode
>> 
>> Best
>> Tien Dat
>> 
>> On 2019/07/11 10:54:59, Jason Joo  wrote: 
>>> Hi Tien
>>> 
>>> You can try dubbo.registry.group
>>> 
>>> But because of multiple storage implementation maybe it's hard to add sub 
>>> folders. eg.
>>> 
>>> dubbo.registry.group=dubbo_alternative
>>> 
>>> best regards,
>>> 
>>> Jason
>>> 
>>>> On Jul 11, 2019, at 16:50, tphan@gmail.com wrote:
>>>> 
>>>> Dear experts,
>>>> 
>>>> We just wonder if it is possible to reallocated the Znode of Dubbo in 
>>>> Zookeeper into a different location?
>>>> 
>>>> By default, Dubbo states in Zookeeper is kept in Znode located at /dubbo, 
>>>> we would like to customize its location, for instance at /publicApi/dubbo.
>>>> 
>>>> Best
>>>> Tien Dat
>>> 
>>> 
>> 



Re: Customizing the Zookeeper Znode name for Dubbo

2019-07-11 Thread Jason Joo
Hi Tien

You can try dubbo.registry.group

But because of multiple storage implementation maybe it's hard to add sub 
folders. eg.

dubbo.registry.group=dubbo_alternative

best regards,

Jason

> On Jul 11, 2019, at 16:50, tphan@gmail.com wrote:
> 
> Dear experts,
> 
> We just wonder if it is possible to reallocated the Znode of Dubbo in 
> Zookeeper into a different location?
> 
> By default, Dubbo states in Zookeeper is kept in Znode located at /dubbo, we 
> would like to customize its location, for instance at /publicApi/dubbo.
> 
> Best
> Tien Dat



Re: No such extension org.apache.dubbo.rpc.Protocol by name dubbo

2019-07-10 Thread Jason Joo
Hi Brooks,

If you use zookeeper as configuration center you should add it to your 
dubbo.properties:

dubbo.registry.protocol=zookeeper



best regards,

Jason

> On Jul 10, 2019, at 23:30, bro...@chtrak.com wrote:
> 
> java.lang.IllegalStateException: No such extension 
> org.apache.dubbo.rpc.Protocol by name dubbo
> at 
> org.apache.dubbo.common.extension.ExtensionLoader.findException(ExtensionLoader.java:516)
>  ~[dubbo-common-2.7.1.jar:2.7.1]
> at 
> org.apache.dubbo.common.extension.ExtensionLoader.createExtension(ExtensionLoader.java:523)
>  ~[dubbo-common-2.7.1.jar:2.7.1]
> at 
> org.apache.dubbo.common.extension.ExtensionLoader.getExtension(ExtensionLoader.java:347)
>  ~[dubbo-common-2.7.1.jar:2.7.1]
> at 
> org.apache.dubbo.config.ServiceConfig.findConfigedPorts(ServiceConfig.java:696)
>  ~[dubbo-config-api-2.7.1.jar:2.7.1]
> at 
> org.apache.dubbo.config.ServiceConfig.doExportUrlsFor1Protocol(ServiceConfig.java:517)
>  ~[dubbo-config-api-2.7.1.jar:2.7.1]
> at org.apache.dubbo.config.ServiceConfig.doExportUrls(ServiceConfig.java:417) 
> ~[dubbo-config-api-2.7.1.jar:2.7.1]
> at org.apache.dubbo.config.ServiceConfig.doExport(ServiceConfig.java:375) 
> ~[dubbo-config-api-2.7.1.jar:2.7.1]
> at org.apache.dubbo.config.ServiceConfig.export(ServiceConfig.java:337) 
> ~[dubbo-config-api-2.7.1.jar:2.7.1]
> at org.apache.dubbo.config.spring.ServiceBean.export(ServiceBean.java:319) 
> ~[dubbo-config-spring-2.7.1.jar:2.7.1]
> at 
> org.apache.dubbo.config.spring.ServiceBean.onApplicationEvent(ServiceBean.java:113)
>  ~[dubbo-config-spring-2.7.1.jar:2.7.1]
> at 
> org.apache.dubbo.config.spring.ServiceBean.onApplicationEvent(ServiceBean.java:59)
>  ~[dubbo-config-spring-2.7.1.jar:2.7.1]
> at 
> org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
>  ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
> at 
> org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
>  ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
> at 
> org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
>  ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
> at 
> org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:402)
>  ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
> at 
> org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:359)
>  ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
> at 
> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:896)
>  ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
> at 
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:552)
>  ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
> at 
> org.springframework.boot.SpringApplication.refresh(SpringApplication.java:742)
>  ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
> at 
> org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:389)
>  ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
> at org.springframework.boot.SpringApplication.run(SpringApplication.java:311) 
> ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
> at 
> org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:139)
>  [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
> at com.fahuo123.fh.FhApplication.main(FhApplication.java:24) [classes/:na]
> 2019-07-10 23:10:49.406 INFO 4716 --- [ main] 
> .b.c.e.AwaitingNonWebApplicationListener : [Dubbo] Current Spring Boot 
> Application is about to shutdown...
> 2019-07-10 23:10:49.415 INFO 4716 --- [ main] 
> f.a.ReferenceAnnotationBeanPostProcessor : class 
> org.apache.dubbo.config.spring.beans.factory.annotation.ReferenceAnnotationBeanPostProcessor
>  was destroying!
> Process finished with exit code 1



Re: 一个关于dubbo 无故超时调用的问题,无奈联系到了您

2019-06-25 Thread Jason Joo
图看不见。

这种情况下,需要进一步提供consumer端的详细监控数据和指标,如jvm内存使用,进程cpu,请求压力,各接口的性能指标等,问题多半在consumer或实际的业务特性(例如死锁)。

best regards,

Jason

> On Jun 25, 2019, at 14:05, Huxing Zhang  wrote:
> 
> Forwarding to dev@ list.
> For further questions, please ask on the dev@dubbo.apache.org 
>  mailing list.
> 
> On Fri, Jun 21, 2019 at 12:18 PM snow  > wrote:
> 很抱歉,打扰到了你,我们是一个上海的创业团队,用了dubbo的微服务,但是近期却出现了一些很不明朗的问题。无奈之举,找了很多人帮忙,困扰了良久。
> 
> 下面是问题描述,如果有可能,希望 麻烦大佬能帮忙查看,十分感谢!!
> 
> 1.项目服务器配置:
> 
> 四台:一台12核96G, 三台8核32G
> 
> 2.应用服务信息:
> 
> 项目拆分为七个服务,每台服务器上都会部署这七个服务
> 
> 其中一个服务(简称api服务)作为用户请求入口,通过dubbo对另外六个服务进行调用
> 
> 注册中心:单机zookeeper
> 
> 3.问题:
> 
> 项目常态下接口平均响应时间90ms左右, 运行一段时间后(目前是约一天左右),  会出现部分机器上的api服务调用dubbo服务耗时增长, 
> 直至无法在超时时间内响应(超时时间配置的是30秒), 引起崩溃。(注:不是四台机器上的api调用dubbo服务都会超时, 其中1~2台, 
> 每次崩溃的机器也不一样)
> 
> 崩溃时服务器cpu, 内存, 带宽出入均正常。
> 
> 数据库cpu, 内存, 连接数正常,没有慢查询。
> 
> dubbo监控图显示provider耗时正常, consumer耗时很高
> 
> -- 崩溃前后平均响应时间走势图
> 
>  
> 
> -- 崩溃前后请求量走势图
> 
>  
> 
> -- 崩溃前后部分dubbo接口监控走势图
> 
>  
> 
> -- 异常摘要截图
> 
>  
> 
>  
> 
> 
> 
> -- 
> Best Regards!
> Huxing



Re: [DISCUSS] Dubbo release versioning

2019-06-17 Thread Jason Joo
+1

It seems like the beginning which is Jun's opinion, and for addition optimizing 
the version naming strategy.

Users won't care about whether we go with 2.7.1, 2.7.2 or 2.7.x, 2.8.x, 2.9.x, 
2.10.x so i think both of them are fine for me. Just pick one and go on.

best regards,

Jason

> On Jun 18, 2019, at 11:45, Shunyu Lei  wrote:
> 
> I m leaning towards option 3,Because most framework versions have this
> semantics, it is better for users.
> 
> Huxing Zhang  于2019年6月18日周二 上午9:44写道:
> 
>> Hi All,
>> 
>> I'd like to bring a discussion about the release versioning of Dubbo.
>> 
>> The problem is the that the currently release versioning won't lead to
>> a stable state eventually. In each 2.7.x release, a lot of new
>> features were added, as well as bugfix, however, newly added features
>> will introduce additional bugs, which make 2.7.x not to be in a stable
>> state.
>> 
>> The solution will be cut a branch that only contains bugfix, and do
>> not adding any new features.
>> This will cause us to consider what versioning scheme should be
>> considered for Dubbo.
>> 
>> Option 1: use 4 digit versioning.
>> 
>> For example, cut a branch called 2.7.2.x, that only contains bugfix,
>> next version should be 2.7.2.1, and 2.7.2.2, and so on.
>> New features are added to 2.7.3 or 2.7.4, and so on.
>> 
>> Option 2: follow semantic versioning
>> 
>> For example, 2.7.x only contains bugfix, next bugfix version should be
>> 2.7.3, 2.7.4, and so on, on some day the branch should be stable and
>> marked production ready.
>> New features are added to 2.8.x, to avoid possible conflict with
>> dubbox (they made serval releases to these versions), I suggest to
>> skip 2.8.x and go to 2.9.x.
>> 
>> Option 3: new features go to 3.x
>> 
>> Same to option 2, 2.7.x only contains bugfix, new features still go to 3.x
>> 
>> I am leaning towards option 2 because it follow the semantic versioning
>> most.
>> I also did an investigation of other famous project like spring-boot,
>> kubernates, and grpc, they all follow semantic versioning.
>> 
>> Regarding version ended with RC or Milestone, based on my
>> investigation above, in most cases, they only add RC or Milestone
>> suffix when there is at least a minor version upgrade (which means new
>> features are added).
>> 
>> For example, in spring-boot, the version is:
>> 
>> - 2.0.8.RELEASE
>> - 2.0.9.RELEASE
>> - 2.1.0.M1
>> - 2.1.0.M2
>> - ...
>> - 2.1.0.RC1
>> - 2.1.0.RELEASE
>> - 2.1.1.RELEASE
>> 
>> Normally there won't be a RC or Milestone for a bugfix release.
>> Kubernetes has its own release versioning specification as well [2].
>> 
>> So I suggest for newly added feature, if we think it is unstable, we
>> should release serveral RC or milestone first. e.g. in the 2.9.x
>> branch, do some relasese called 2.9.x-M1  or 2.9.x-RC1.
>> 
>> What do you think?
>> 
>> 
>> [1] https://semver.org/
>> [2]
>> https://github.com/kubernetes/community/blob/master/contributors/design-proposals/release/versioning.md#kubernetes-release-versioning
>> --
>> Best Regards!
>> Huxing
>> 



Re: [DISCUSS] Observation of performance degradation of 2.7.2 under JDK 1.8.

2019-06-13 Thread Jason Joo
Hi, Huxing

In my view generally one release is followed after Alpha, Beta, RC is a quite 
normal workflow. And it keeps the stability, steady and seriousness to the 
final release. Especially for common users they can not recognize where 2.7.3 
is a release for RC purpose in name "2.7.3" but things could be clear if name 
it "2.7.3-RC1" or "2.7.3-BETA".

Sure it's a personal opinion.

best regards,

Jason

> On Jun 14, 2019, at 12:50, Huxing Zhang  wrote:
> 
> Hi,
> 
> On Fri, Jun 14, 2019 at 11:42 AM Jason Joo  <mailto:hblz...@163.com>> wrote:
>> 
>> Hi, Huxing
>> 
>> Something you may misunderstand that the versions in descriptions from Jun 
>> should like:
>> 
>> 2.7.3-RC1
>> 2.7.3-RC2
>> 2.7.3-RC3
>> 2.7.3
> 
> This naming looks weird to me. If 2.7.3-RC1 is not production ready,
> what does 2.7.2 mean? They are inconsistent.
> 
> What I am proposing is:
> 
> 2.7.3 (Not production ready)
> 2.7.4 (Not production ready)
> ...
> 2.7.N (mark as production ready)
> 
> 
> 
>> 
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Jun 14, 2019, at 11:26, Huxing Zhang >> <mailto:hux...@apache.org>> wrote:
>>> 
>>> Hi,
>>> 
>>> On Fri, Jun 14, 2019 at 10:01 AM Jun Liu >> <mailto:liu...@apache.org> <mailto:liu...@apache.org 
>>> <mailto:liu...@apache.org>>> wrote:
>>>> 
>>>>> I am +1 on marking current release NOT production ready.
>>>> 
>>>> Do you think several rounds of beta or RC is necessary before every formal 
>>>> release?
>>> 
>>> Given that 2.7.0-2.7.2 has already been published. I think it is weird
>>> that the next version is 2.7.3-RC1.
>>> Instead of that I think keep bumping the version to 2.7.3 is ok, as
>>> long as we let the community know that it is not production ready
>>> until the community formally announce it.
>>> 
>>>> 
>>>> Other people have different opinions?
>>>> 
>>>> Jun
>>>> 
>>>>> On Jun 12, 2019, at 2:46 PM, Huxing Zhang  wrote:
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> 
>>>>> On Wed, Jun 12, 2019 at 10:02 AM Jun Liu >>>> <mailto:liu...@apache.org>> wrote:
>>>>>> 
>>>>>> Hi, All
>>>>>> 
>>>>>> Recently, Jeff and I and some other volunteers from the community are 
>>>>>> trying to improve the performance of Dubbo. When doing benchmark, we 
>>>>>> found that the usage of CompletableFuture in the 2.7.2 has a significant 
>>>>>> performance degradation (both QPS and RT) when running under the JDK 1.8 
>>>>>> version (but performs ok under JDK 11), check this issue[1] for more 
>>>>>> details. Thinking of some other problems found recently, the service 
>>>>>> registration discovery problem in 2.7.1[2], the configuration model 
>>>>>> unification problem[3], etc. I think we need to reconsider the evolution 
>>>>>> plan and stability guarantee of 2.7.
>>>>>> 
>>>>>> From my point of view, version 2.7 is releasing in a relatively fast 
>>>>>> pace, with each version containing lots of features and refactoring 
>>>>>> changes, I think this is a good sign for the community. But this also 
>>>>>> brings us new problems, especially when we don't have enough 
>>>>>> infrastructures and time to test each version, it is very difficult to 
>>>>>> ensure the functional stability and well performance of each version. 
>>>>>> Considering our roadmap in the near future, this situation seems to be 
>>>>>> even worse. According to our draft roadmap released in last meetup in 
>>>>>> Beijing, we will release the native cloud service discovery model in 
>>>>>> version 2.7.3 or 2.7.4, which is almost a complete refactoring of 
>>>>>> Dubbo's current service discovery functionality, I doubt the both the 
>>>>>> API and feature stability is hard to guarantee without several releases.
>>>>>> 
>>>>>> So my main concern is the stability of the 2.7.x version. Maybe the 
>>>>>> released or the following several releases should be marked as 
>>>>>> non-production available from the community level officially, or 
>>>>>> consider add beta, RC, etc. tags to some version numbers if necessary. 
>>>>>> What do others think?
>>>>>> 
>>>>>> What do others think?
>>>>> 
>>>>> I am +1 on marking current release NOT production ready.
>>>>> It is necessary for users to try out the new features and provide
>>>>> feedback, I think after several iterations, the 2.7.x will enter into
>>>>> stability eventually.
>>>>> 
>>>>>> 
>>>>>> 1. https://github.com/apache/dubbo/issues/4279 
>>>>>> <https://github.com/apache/dubbo/issues/4279>
>>>>>> 2. https://github.com/apache/dubbo/issues/4213 
>>>>>> <https://github.com/apache/dubbo/issues/4213>
>>>>>> 3. https://github.com/apache/dubbo-website/pull/388 
>>>>>> <https://github.com/apache/dubbo-website/pull/388>
>>>>>> 
>>>>>> Jun
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Best Regards!
>>>>> Huxing
>>>> 
>>> 
>>> 
>>> --
>>> Best Regards!
>>> Huxing
>> 
> 
> 
> -- 
> Best Regards!
> Huxing



Re: [DISCUSS] Observation of performance degradation of 2.7.2 under JDK 1.8.

2019-06-13 Thread Jason Joo
Hi, Huxing

Something you may misunderstand that the versions in descriptions from Jun 
should like:

2.7.3-RC1
2.7.3-RC2
2.7.3-RC3
2.7.3


best regards,

Jason

> On Jun 14, 2019, at 11:26, Huxing Zhang  wrote:
> 
> Hi,
> 
> On Fri, Jun 14, 2019 at 10:01 AM Jun Liu  > wrote:
>> 
>>> I am +1 on marking current release NOT production ready.
>> 
>> Do you think several rounds of beta or RC is necessary before every formal 
>> release?
> 
> Given that 2.7.0-2.7.2 has already been published. I think it is weird
> that the next version is 2.7.3-RC1.
> Instead of that I think keep bumping the version to 2.7.3 is ok, as
> long as we let the community know that it is not production ready
> until the community formally announce it.
> 
>> 
>> Other people have different opinions?
>> 
>> Jun
>> 
>>> On Jun 12, 2019, at 2:46 PM, Huxing Zhang  wrote:
>>> 
>>> Hi,
>>> 
>>> 
>>> On Wed, Jun 12, 2019 at 10:02 AM Jun Liu >> > wrote:
 
 Hi, All
 
 Recently, Jeff and I and some other volunteers from the community are 
 trying to improve the performance of Dubbo. When doing benchmark, we found 
 that the usage of CompletableFuture in the 2.7.2 has a significant 
 performance degradation (both QPS and RT) when running under the JDK 1.8 
 version (but performs ok under JDK 11), check this issue[1] for more 
 details. Thinking of some other problems found recently, the service 
 registration discovery problem in 2.7.1[2], the configuration model 
 unification problem[3], etc. I think we need to reconsider the evolution 
 plan and stability guarantee of 2.7.
 
 From my point of view, version 2.7 is releasing in a relatively fast pace, 
 with each version containing lots of features and refactoring changes, I 
 think this is a good sign for the community. But this also brings us new 
 problems, especially when we don't have enough infrastructures and time to 
 test each version, it is very difficult to ensure the functional stability 
 and well performance of each version. Considering our roadmap in the near 
 future, this situation seems to be even worse. According to our draft 
 roadmap released in last meetup in Beijing, we will release the native 
 cloud service discovery model in version 2.7.3 or 2.7.4, which is almost a 
 complete refactoring of Dubbo's current service discovery functionality, I 
 doubt the both the API and feature stability is hard to guarantee without 
 several releases.
 
 So my main concern is the stability of the 2.7.x version. Maybe the 
 released or the following several releases should be marked as 
 non-production available from the community level officially, or consider 
 add beta, RC, etc. tags to some version numbers if necessary. What do 
 others think?
 
 What do others think?
>>> 
>>> I am +1 on marking current release NOT production ready.
>>> It is necessary for users to try out the new features and provide
>>> feedback, I think after several iterations, the 2.7.x will enter into
>>> stability eventually.
>>> 
 
 1. https://github.com/apache/dubbo/issues/4279 
 
 2. https://github.com/apache/dubbo/issues/4213 
 
 3. https://github.com/apache/dubbo-website/pull/388 
 
 
 Jun
>>> 
>>> 
>>> 
>>> --
>>> Best Regards!
>>> Huxing
>> 
> 
> 
> -- 
> Best Regards!
> Huxing



Re: Customize the target content to be hashed with Consistent hash load balance

2019-06-12 Thread Jason Joo
Hi, Dat

Stand at the point of DUBBO maybe it is too complex to support a grammar as EL 
expressions like 'param[0].url' due to invokers are so dynamic in same 
invocation group.

So I think taking an obvious and special partition/hashing parameter is 
acceptable for me instead of taking it as a kind of duplicated information.

Again if you have better ideas it could be welcome in community.


best regards,

Jason

> On Jun 12, 2019, at 19:34, tphan@gmail.com wrote:
> 
> Hi Jason,
> 
> Thanks for sharing your opinion :) 
> This is actually what we are currently doing, for now, as we do not find a 
> better solution for this yet.
> 
> Best
> Dat
> 
> On 2019/06/12 08:28:32, Jason Joo  wrote: 
>> Hi, Tien
>> 
>> Agree to you that it will break the principle of design if overriding 
>> toString().
>> Currently consist hash of load balance only support specifying which 
>> argument(s) can take charge in it but not support specify certain property 
>> of a parameter to do so.
>> 
>> For better understanding maybe add a parameter like
>> 
>>> MyResponse getResponse(String partitionKey, MyRequestParams params);
>> 
>> 
>> may be more easy.
>> 
>> It's only my opinion.
>> 
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Jun 12, 2019, at 15:31, tphan@gmail.com wrote:
>>> 
>>> Dear Jason,
>>> 
>>> I am aware of that. 
>>> However, as the MyRequestParams has multiple fields, we would like to avoid 
>>> implementing the toString method that return a string containing only the 
>>> URI field.
>>> This would break the purpose of the toString method.
>>> 
>>> Do you know any other approaches/work around for this?
>>> 
>>> Best
>>> Dat
>>> 
>>> On 2019/06/12 00:56:42, Jason Joo  wrote: 
>>>> Hi, Tien
>>>> 
>>>> Consistent hash is relying mainly the object's "toString()" protocol. So 
>>>> pls make sure your objects have a unique and stable toString() method and 
>>>> it will be ok.
>>>> 
>>>> best regards,
>>>> 
>>>> Jason
>>>> 
>>>>> On Jun 11, 2019, at 16:58, tphan@gmail.com wrote:
>>>>> 
>>>>> Dear experts,
>>>>> 
>>>>> We are using Dubbo to export our services. For data load balancing, we 
>>>>> would like to use the consistent hash load balancer.
>>>>> For the default usage of this load balancer, we can define either one or 
>>>>> several parameters of the consumed method to be used as the hashing data.
>>>>> We wonder if it is possible to target the hashing on the field of an 
>>>>> object passed as the param.
>>>>> For instance, the method interface is as follows:
>>>>> 
>>>>> MyResponse getResponse(MyRequestParams params);
>>>>> 
>>>>> The object params has a field named URI, which we would like to apply the 
>>>>> hash algorithm upon.
>>>>> Of course, we can modify our method to:
>>>>> 
>>>>> MyResponse getResponse(String uri, MyRequestParams params);
>>>>> 
>>>>> But this really is contradictory with the design goal of MyRequestParam 
>>>>> object (which is defined to wrap all our request params into one object)
>>>>> 
>>>>> Best regards
>>>>> Tien Dat PHAN
>>>> 
>>>> 
>> 
>> 



Re: Customize the target content to be hashed with Consistent hash load balance

2019-06-12 Thread Jason Joo
Hi, Tien

Agree to you that it will break the principle of design if overriding 
toString().
Currently consist hash of load balance only support specifying which 
argument(s) can take charge in it but not support specify certain property of a 
parameter to do so.

For better understanding maybe add a parameter like

> MyResponse getResponse(String partitionKey, MyRequestParams params);


may be more easy.

It's only my opinion.


best regards,

Jason

> On Jun 12, 2019, at 15:31, tphan@gmail.com wrote:
> 
> Dear Jason,
> 
> I am aware of that. 
> However, as the MyRequestParams has multiple fields, we would like to avoid 
> implementing the toString method that return a string containing only the URI 
> field.
> This would break the purpose of the toString method.
> 
> Do you know any other approaches/work around for this?
> 
> Best
> Dat
> 
> On 2019/06/12 00:56:42, Jason Joo  wrote: 
>> Hi, Tien
>> 
>> Consistent hash is relying mainly the object's "toString()" protocol. So pls 
>> make sure your objects have a unique and stable toString() method and it 
>> will be ok.
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Jun 11, 2019, at 16:58, tphan@gmail.com wrote:
>>> 
>>> Dear experts,
>>> 
>>> We are using Dubbo to export our services. For data load balancing, we 
>>> would like to use the consistent hash load balancer.
>>> For the default usage of this load balancer, we can define either one or 
>>> several parameters of the consumed method to be used as the hashing data.
>>> We wonder if it is possible to target the hashing on the field of an object 
>>> passed as the param.
>>> For instance, the method interface is as follows:
>>> 
>>> MyResponse getResponse(MyRequestParams params);
>>> 
>>> The object params has a field named URI, which we would like to apply the 
>>> hash algorithm upon.
>>> Of course, we can modify our method to:
>>> 
>>> MyResponse getResponse(String uri, MyRequestParams params);
>>> 
>>> But this really is contradictory with the design goal of MyRequestParam 
>>> object (which is defined to wrap all our request params into one object)
>>> 
>>> Best regards
>>> Tien Dat PHAN
>> 
>> 



Re: [DISCUSS] Observation of performance degradation of 2.7.2 under JDK 1.8.

2019-06-11 Thread Jason Joo
Hi, jun

> Take milestones with big features included as "RC" versions.

Totally agree with you and that's my previous suggestion that we should release 
several RC into repository enabling limited production tests.
But in my opinion we shouldn't worry too much on the issues happening 
currently. Because as some discussions in some issues we indeed need a 
performance test before releasing, merging PRs and i think the problems will be 
disappeared after that.

So is it in process? Anyone know it?

best regards,

Jason

> On Jun 12, 2019, at 10:01, Jun Liu  wrote:
> 
> 



Re: Customize the target content to be hashed with Consistent hash load balance

2019-06-11 Thread Jason Joo
Hi, Tien

Consistent hash is relying mainly the object's "toString()" protocol. So pls 
make sure your objects have a unique and stable toString() method and it will 
be ok.

best regards,

Jason

> On Jun 11, 2019, at 16:58, tphan@gmail.com wrote:
> 
> Dear experts,
> 
> We are using Dubbo to export our services. For data load balancing, we would 
> like to use the consistent hash load balancer.
> For the default usage of this load balancer, we can define either one or 
> several parameters of the consumed method to be used as the hashing data.
> We wonder if it is possible to target the hashing on the field of an object 
> passed as the param.
> For instance, the method interface is as follows:
> 
> MyResponse getResponse(MyRequestParams params);
> 
> The object params has a field named URI, which we would like to apply the 
> hash algorithm upon.
> Of course, we can modify our method to:
> 
> MyResponse getResponse(String uri, MyRequestParams params);
> 
> But this really is contradictory with the design goal of MyRequestParam 
> object (which is defined to wrap all our request params into one object)
> 
> Best regards
> Tien Dat PHAN



Re: How about add warning notes to 2.7.1 release notes to warn users of one known critical problem?

2019-06-02 Thread Jason Joo
Hi, lan

Addressing issue also due to some changes recently and was fixed just after 
when 2.7.1 was released.

Now i have checked our internal branch based on older master but newer than 
2.7.1 that has no issue currently.

best regards,

Jason

> On Jun 3, 2019, at 14:21, Ian Luo  wrote:
> 
> Jason
> 
> 
>> And the reason generating the internal branch is that the address
>> fetching logic is not correct in release 2.7.1
> 
> 
> Have this fixed in 2.7.2 too?
> 
> Maybe we could introduce several RC versions before release so we have some
>> time to take it to production environments mixed together.
> 
> 
> This is really a good idea, to give users a chance to evaluate if it's
> stable enough to deploy the new version into production environment. We
> will consider it.
> 
> Thanks,
> -Ian.
> 
> 
> 
> On Mon, Jun 3, 2019 at 2:05 PM Jason Joo  wrote:
> 
>> Hi, lan
>> 
>> I have checked that day and found our internal branch was base on master
>> newer than the release.
>> And the reason generating the internal branch is that the address fetching
>> logic is not correct in release 2.7.1 (Surely the static registration is
>> another problem). So we escape from this issue fortunately.
>> 
>> And again i think we should pay more attention no either unit tests and
>> backward compatibility carefully with kinds of issue before:
>> 
>> Attachments support issue(compatibility issue) on 2.5.8
>> qos.enable issue 2.6.x
>> Maybe we could introduce several RC versions before release so we have
>> some time to take it to production environments mixed together.
>> 
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Jun 3, 2019, at 13:45, Ian Luo  wrote:
>>> 
>>> Jason,
>>> 
>>> Sorry for introducing this severe regression. Pls. hold until 2.7.2
>>> releases which is expected to be public available in this week. We are
>> now
>>> considering to init a regression test project. Again, sorry about it.
>>> 
>>> -Ian.
>>> 
>>> On Fri, May 31, 2019 at 11:50 AM Jason Joo  wrote:
>>> 
>>>> hi, jun
>>>> 
>>>> Long time not focus on dubbo but this news surprises me. I just upgraded
>>>> some new spring-boot projects to use a new internal branch based on
>> 2.7.1.
>>>> 
>>>> I think it maybe better to describe how to trigger it in the notes?
>>>> 
>>>> 
>>>> 
>>>> best regards,
>>>> 
>>>> Jason
>>>> 
>>>>> On May 31, 2019, at 11:39, Jun Liu  wrote:
>>>>> 
>>>>> Hi, All
>>>>> 
>>>>> Recently, we keep receiving issues from the community reporting
>>>> duplicate URL addresses in Registry, mostly zookeeper, after upgrading
>> to
>>>> 2.7.1. Those URLs are all pushed to the Consumer and because only one of
>>>> the duplicated URLs is valid, it causes the consumer to fail when the
>>>> consumer chooses the invalid one during load balance.
>>>>> 
>>>>> So I am thinking of add the following warning notes on top of 2.7.1
>>>> release notes[1] to warn users of the possible risks before our release
>> of
>>>> 2.7.2.
>>>>> 
>>>>> ——
>>>>> Warning: According to reports from the community, be aware of possible
>>>> Service Discovery problems in certain circumstances. Check #4213 for
>>>> details.
>>>>> 
>>>> 
>> 风险提示:升级到2.7.1版本后注册中心(多数是zookeeper)会出现重复URL地址数据无法删除,导致消费方拿到的是失效地址,从而导致调用失败的问题,2.7.2版本里面会修复此问题,预计6月初发布。详细原因请参考#4213。
>>>>> 
>>>>> Jun
>>>> 
>>>> 
>> 
>> 



Re: How about add warning notes to 2.7.1 release notes to warn users of one known critical problem?

2019-06-02 Thread Jason Joo
Hi, lan

I have checked that day and found our internal branch was base on master newer 
than the release.
And the reason generating the internal branch is that the address fetching 
logic is not correct in release 2.7.1 (Surely the static registration is 
another problem). So we escape from this issue fortunately.

And again i think we should pay more attention no either unit tests and 
backward compatibility carefully with kinds of issue before:

Attachments support issue(compatibility issue) on 2.5.8
qos.enable issue 2.6.x
Maybe we could introduce several RC versions before release so we have some 
time to take it to production environments mixed together.


best regards,

Jason

> On Jun 3, 2019, at 13:45, Ian Luo  wrote:
> 
> Jason,
> 
> Sorry for introducing this severe regression. Pls. hold until 2.7.2
> releases which is expected to be public available in this week. We are now
> considering to init a regression test project. Again, sorry about it.
> 
> -Ian.
> 
> On Fri, May 31, 2019 at 11:50 AM Jason Joo  wrote:
> 
>> hi, jun
>> 
>> Long time not focus on dubbo but this news surprises me. I just upgraded
>> some new spring-boot projects to use a new internal branch based on 2.7.1.
>> 
>> I think it maybe better to describe how to trigger it in the notes?
>> 
>> 
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On May 31, 2019, at 11:39, Jun Liu  wrote:
>>> 
>>> Hi, All
>>> 
>>> Recently, we keep receiving issues from the community reporting
>> duplicate URL addresses in Registry, mostly zookeeper, after upgrading to
>> 2.7.1. Those URLs are all pushed to the Consumer and because only one of
>> the duplicated URLs is valid, it causes the consumer to fail when the
>> consumer chooses the invalid one during load balance.
>>> 
>>> So I am thinking of add the following warning notes on top of 2.7.1
>> release notes[1] to warn users of the possible risks before our release of
>> 2.7.2.
>>> 
>>> ——
>>> Warning: According to reports from the community, be aware of possible
>> Service Discovery problems in certain circumstances. Check #4213 for
>> details.
>>> 
>> 风险提示:升级到2.7.1版本后注册中心(多数是zookeeper)会出现重复URL地址数据无法删除,导致消费方拿到的是失效地址,从而导致调用失败的问题,2.7.2版本里面会修复此问题,预计6月初发布。详细原因请参考#4213。
>>> 
>>> Jun
>> 
>> 



Re: How about add warning notes to 2.7.1 release notes to warn users of one known critical probletm?

2019-05-30 Thread Jason Joo
hi, jun

Thanks for your information and i will check it.

I also will upgrade our branch after releasing 2.7.2.

Keep going and come on


best regards,

Jason

> On May 31, 2019, at 12:53, Jun Liu  wrote:
> 
>> Long time not focus on dubbo but this news surprises me. I just upgraded 
>> some new spring-boot projects to use a new internal branch based on 2.7.1.
>> 
>> I think it maybe better to describe how to trigger it in the notes?
> 
> Sorry for possible problems that may affect you, this problem will only occur 
> on certain circumstances, you can check this issue[1] for more details. The 
> community is working on 2.7.2 to fix it.
> 
> 1. https://github.com/apache/dubbo/issues/4213
> 
> Jun
> 
>> On May 31, 2019, at 11:50 AM, Jason Joo  wrote:
>> 
>> hi, jun
>> 
>> Long time not focus on dubbo but this news surprises me. I just upgraded 
>> some new spring-boot projects to use a new internal branch based on 2.7.1.
>> 
>> I think it maybe better to describe how to trigger it in the notes?
>> 
>> 
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On May 31, 2019, at 11:39, Jun Liu  wrote:
>>> 
>>> Hi, All
>>> 
>>> Recently, we keep receiving issues from the community reporting duplicate 
>>> URL addresses in Registry, mostly zookeeper, after upgrading to 2.7.1. 
>>> Those URLs are all pushed to the Consumer and because only one of the 
>>> duplicated URLs is valid, it causes the consumer to fail when the consumer 
>>> chooses the invalid one during load balance.
>>> 
>>> So I am thinking of add the following warning notes on top of 2.7.1 release 
>>> notes[1] to warn users of the possible risks before our release of 2.7.2.
>>> 
>>> ——
>>> Warning: According to reports from the community, be aware of possible 
>>> Service Discovery problems in certain circumstances. Check #4213 for 
>>> details.
>>> 风险提示:升级到2.7.1版本后注册中心(多数是zookeeper)会出现重复URL地址数据无法删除,导致消费方拿到的是失效地址,从而导致调用失败的问题,2.7.2版本里面会修复此问题,预计6月初发布。详细原因请参考#4213。
>>> 
>>> Jun
>> 



Re: How about add warning notes to 2.7.1 release notes to warn users of one known critical problem?

2019-05-30 Thread Jason Joo
hi, jun

Long time not focus on dubbo but this news surprises me. I just upgraded some 
new spring-boot projects to use a new internal branch based on 2.7.1.

I think it maybe better to describe how to trigger it in the notes?



best regards,

Jason

> On May 31, 2019, at 11:39, Jun Liu  wrote:
> 
> Hi, All
> 
> Recently, we keep receiving issues from the community reporting duplicate URL 
> addresses in Registry, mostly zookeeper, after upgrading to 2.7.1. Those URLs 
> are all pushed to the Consumer and because only one of the duplicated URLs is 
> valid, it causes the consumer to fail when the consumer chooses the invalid 
> one during load balance.
> 
> So I am thinking of add the following warning notes on top of 2.7.1 release 
> notes[1] to warn users of the possible risks before our release of 2.7.2.
> 
> ——
> Warning: According to reports from the community, be aware of possible 
> Service Discovery problems in certain circumstances. Check #4213 for details.
> 风险提示:升级到2.7.1版本后注册中心(多数是zookeeper)会出现重复URL地址数据无法删除,导致消费方拿到的是失效地址,从而导致调用失败的问题,2.7.2版本里面会修复此问题,预计6月初发布。详细原因请参考#4213。
> 
> Jun



Re: ConsistentHash LoadBalance中的解释:缺省只对第一个参数进行Hash运算是什么意思?

2019-02-22 Thread Jason Joo
Hi, Kitito

I get your thought. 
And the "IP" in our opinion is it a kind of addition (ip + parameters) or can 
be used alone (only ip)?

Though IP + parameters may leads to unbalanced load between providers but i 
think it maybe still make sense according to the concept of ConsistentHash, 
while it will not if supporting only ip_hash.

Hashed by IP maybe not a need so common i think.

best regards,

Jason

> On Feb 22, 2019, at 15:19, 徐靖峰  wrote:
> 
> Hi, Jason
> 
> I know that the default logic of ConsistentHash LoadBalance is hashed by 
> parameters, and dubbo provide the configuration
> 
>   to config the hash 
> params
> 
>  to config virtual nodes
> 
> My opinion is Dubbo can extend the hash key, not only the request method 
> params. 
> 
> IP consistent hash is a such common requirement while Dubbo’s existing 
> extension don’t include it, there is no need to let the users extend by 
> themselves,
> 
> A little change for Consistent Hash LoadBalance will support such function.
> 
> Kitito
> 
>> 在 2019年2月22日,上午11:39,Jason Joo  写道:
>> 
>> hi, moe
>> 
>> I think what you mention is like "ip_hash" in NGINX or some other.
>> 
>> ConsistentHashLB is hashed by parameters because a same logic will just act 
>> the same under the same parameters (which we call it reenterability). Adding 
>> other parameters will break it.
>> 
>> While DUBBO provides the ability extending load balancers which makes it 
>> possible to implement kinds of balancers like ip_hash, app_version_hash or 
>> others.
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Feb 22, 2019, at 11:00, 徐靖峰  wrote:
>>> 
>>> Hi all
>>> 
>>> I don’t think that ConsistentHashLoadBalancer should only load balance by 
>>> the method param, maybe more configuration can provide to the user to 
>>> achieve more type of dynamic hash parameters, such as hash by the 
>>> requestIp, more request param.
>>> 
>>> 
>> 
> 
> 



Re: ConsistentHash LoadBalance中的解释:缺省只对第一个参数进行Hash运算是什么意思?

2019-02-21 Thread Jason Joo
hi, moe

I think what you mention is like "ip_hash" in NGINX or some other.

ConsistentHashLB is hashed by parameters because a same logic will just act the 
same under the same parameters (which we call it reenterability). Adding other 
parameters will break it.

While DUBBO provides the ability extending load balancers which makes it 
possible to implement kinds of balancers like ip_hash, app_version_hash or 
others.

best regards,

Jason

> On Feb 22, 2019, at 11:00, 徐靖峰  wrote:
> 
> Hi all
> 
> I don’t think that ConsistentHashLoadBalancer should only load balance by the 
> method param, maybe more configuration can provide to the user to achieve 
> more type of dynamic hash parameters, such as hash by the requestIp, more 
> request param.
> 
> 



Re: ConsistentHash LoadBalance中的解释:缺省只对第一个参数进行Hash运算是什么意思?

2019-02-13 Thread Jason Joo
+1

Methods with no arguments or little possible values of arguments should not be 
applied `ConsistentHashLoadBalance`. Or requests can not be smoothly 
distributed when nodes change.

best regards,

Jason

> On Feb 13, 2019, at 22:30, 田 小波  wrote:
> 
> 数



Re: [Discuss] About Clean Code Plan

2019-01-28 Thread Jason Joo
Hi, lan

Agree with you.

It's great for shorter, cleaner, more efficient logic optimizing. So I suggest 
that modifications including grammar sugar purposed for shorter, cleaner or 
more efficient are great but not just rewriting with another form. I surely 
will not object it, too.

But pay attention to some features or new styles that will generate more 
objects.

I think it would be more simple and easy we organize these as a kind of issue. 
Focused and small PRs are easy to review after all.

best regards,

Jason

> On Jan 29, 2019, at 14:38, Ian Luo  wrote:
> 
> Jason,
> 
> Totally agree with you on this point. We should focus less on grammar sugar
> but focus more on make the code clean, for say: divide long method into
> shorter methods, better naming, or better doc, or even better design.
> 
> But on the other hand, if someone from the community shows enthusiasm for
> updating code with new sugar, I will not object to the effort as long as it
> doesn't hurt readability, after all it is low hanging fruit :)
> 
> What I propose is we should gather all possible ideas and group them in one
> project on GitHub [1], like I said in my previous email. It is critical to
> make the code clean as much as possible in my opinion. It deserves one
> dedicated project to track all ideas. By doing this, the community can also
> understand the current focuses are.
> 
> So my question is, should we init a project focusing on clean code, while
> the purpose doesn't limit to grammar sugar?
> 
> Regards,
> -Ian.
> 
> 1. https://github.com/apache/incubator-dubbo/projects
> 
> On Tue, Jan 29, 2019 at 11:00 AM Jason Joo  wrote:
> 
>> Hi, Huxing & LiZhen
>> 
>> Things like diamond is something called grammar SUGAR.
>> Tasted good for developers but the same on byte code.
>> 
>> So in my opinion we should not take it as a kind of "policy correct".
>> we can make the changes in the code of recent PRs but not for all the
>> code. Think about there may be more sugars in future JVM.
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Jan 29, 2019, at 10:42, yuhang xiu  wrote:
>>> 
>>> Your pr has modified more than 30,000 lines of code, and most of them are
>>> modifications of the entire file.
>>> What is the meaning of your pr? Why do I need to keep it open?
>>> 
>>> If you need to discuss, issues and mailing lists are good choices, why do
>>> you need to discuss pr?
>>> 
>>> Who can review this pr[1] and tell me what this pr is going to do?
>>> 
>>> [1] https://github.com/apache/incubator-dubbo/pull/3371
>>> 
>>> LiZhenNet  于2019年1月29日周二 上午10:29写道:
>>> 
>>>> Yes, we can create a project to manage it, and we can talk about it.
>> like:
>>>> Should we used Diamond Operator replace  explicit type? Use Lambda?
>>>> Let's discuss the two pull requests above instead of closed it
>> immediately.
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> Huxing Zhang  于2019年1月29日周二 上午9:58写道:
>>>> 
>>>>> Hi,
>>>>> 
>>>>> Nice suggestion!
>>>>> 
>>>>> Shall we can create a project to manage it?
>>>>> 
>>>>> On Fri, Jan 25, 2019 at 11:25 AM LiZhenNet 
>>>> wrote:
>>>>>> 
>>>>>> Now ,Dubbo has 167 contributors , 3125 commits. There are a lot of
>>>> unused
>>>>>> methods ,unused logic , nonstandard naming  in the code. And there are
>>>>> some
>>>>>> code that can use the new features of java to make the code more
>>>> concise
>>>>>> and understandable. like Lambda , Diamond Operator and so on.These are
>>>>> easy
>>>>>> to find and fixed because of the tips of the idea .We should do it
>>>> before
>>>>>> completing the incubation.
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Best Regards!
>>>>> Huxing
>>>>> 
>>>> 
>> 
>> 



Re: [Discuss] About Clean Code Plan

2019-01-28 Thread Jason Joo
Hi, Huxing & LiZhen

Things like diamond is something called grammar SUGAR.
Tasted good for developers but the same on byte code.

So in my opinion we should not take it as a kind of "policy correct".
we can make the changes in the code of recent PRs but not for all the code. 
Think about there may be more sugars in future JVM.

best regards,

Jason

> On Jan 29, 2019, at 10:42, yuhang xiu  wrote:
> 
> Your pr has modified more than 30,000 lines of code, and most of them are
> modifications of the entire file.
> What is the meaning of your pr? Why do I need to keep it open?
> 
> If you need to discuss, issues and mailing lists are good choices, why do
> you need to discuss pr?
> 
> Who can review this pr[1] and tell me what this pr is going to do?
> 
> [1] https://github.com/apache/incubator-dubbo/pull/3371
> 
> LiZhenNet  于2019年1月29日周二 上午10:29写道:
> 
>> Yes, we can create a project to manage it, and we can talk about it. like:
>> Should we used Diamond Operator replace  explicit type? Use Lambda?
>> Let's discuss the two pull requests above instead of closed it immediately.
>> 
>> 
>> 
>> 
>> 
>> Huxing Zhang  于2019年1月29日周二 上午9:58写道:
>> 
>>> Hi,
>>> 
>>> Nice suggestion!
>>> 
>>> Shall we can create a project to manage it?
>>> 
>>> On Fri, Jan 25, 2019 at 11:25 AM LiZhenNet 
>> wrote:
 
 Now ,Dubbo has 167 contributors , 3125 commits. There are a lot of
>> unused
 methods ,unused logic , nonstandard naming  in the code. And there are
>>> some
 code that can use the new features of java to make the code more
>> concise
 and understandable. like Lambda , Diamond Operator and so on.These are
>>> easy
 to find and fixed because of the tips of the idea .We should do it
>> before
 completing the incubation.
>>> 
>>> 
>>> 
>>> --
>>> Best Regards!
>>> Huxing
>>> 
>> 



Re: [Discuss] The Timer Design in Dubbo from 2.6 to 2.7

2019-01-23 Thread Jason Joo
hi, moe

This is the main differences:

ScheduledThreadPoolExecutor

O(log N) for adding new task
O(1) per each timer tick (but tick per each task, so N overall)
O(log N) cancelling the task
lock per each tick/task
HashedWheelTimer

O(1) adding new task
O(m) per each timer tick (m ~ N/C where C > 512 approx), so ~C ticks overall
O(m) for cancelling a task
lock per bucket of tasks (on each tick)


best regards,

Jason

> On Jan 24, 2019, at 10:09, 徐靖峰  wrote:
> 
> Hi, Jason,a nice opinion.
> 
> I have noticed that this issue[1] in Github encourage  the change from 
> Schedule to HashedWheelTimer, but didn’t give a vert detailed reason.
> 
> Is HashedWheelTimer really an unquestionable choice?
> 
> [1] https://github.com/apache/incubator-dubbo/issues/1371 
> <https://github.com/apache/incubator-dubbo/issues/1371>
> 
>> 在 2019年1月24日,上午9:53,Jason Joo  写道:
>> 
>> Maybe because the main scenario dubbo faces is persistent connection. 
>> For persistent connections the most frequent thing is ticking.
>> 
>> ScheduledThreadPoolExecutor is O(1)*connections compared to O(C) in 
>> HashedWheelTimer when ticking.
>> Considering connections is far less then C ScheduledThreadPoolExecutor maybe 
>> more efficient in my opinion.
>> 
>> best regards,
>> 
>> Jason
> 



Re: [Discuss] The Timer Design in Dubbo from 2.6 to 2.7

2019-01-23 Thread Jason Joo
Maybe because the main scenario dubbo faces is persistent connection. 
For persistent connections the most frequent thing is ticking.

ScheduledThreadPoolExecutor is O(1)*connections compared to O(C) in 
HashedWheelTimer when ticking.
Considering connections is far less then C ScheduledThreadPoolExecutor maybe 
more efficient in my opinion.

best regards,

Jason

> On Jan 24, 2019, at 07:05, 华 钟明  wrote:
> 
> Thank you for your question, your question has been placed in the issue, 
> https://github.com/apache/incubator-dubbo/issues/3324
> please continue to pay attention
> Best Regards!
> Zhongming Hua
> 
> 
> 在 2019/1/24 上午12:31,“kirito@foxmail.com” 写入:
> 
>Hi Folks:
> 
>最近在研究 Dubbo 2.7 代码时,我发现了一个有意思的改动,Dubbo 2.6 中大量使用了 
> ScheduledThreadPoolExecutor,例如实现定时发送心跳,开启定时器用于判断异步请求超时等场景,在 Dubbo 2.7 之中替换为了由 
> Netty 实现的 HashedWheelTimer。
> 
>我对这一改动比较好奇,于是对定时器的实现做了一些调研,探讨目的是想为 2.7 选用一个最合适的定时器实现。
> 
>JDK 的 Timer 存在设计问题,不予讨论,我想主要对比 HashedWheelTimer 和 
> ScheduledThreadPoolExecutor 的区别。
> 
>Netty 在 3.x 中有大量使用 HashedWheelTimer,但是在 4.1 中,我们可以发现,Netty 保留了 
> HashedWheelTimer,但在其源码中并未使用它,而是选择了 ScheduledThreadPoolExecutor,在 Stack 
> Overflow 
> 中也有人提出这样的疑问:https://stackoverflow.com/questions/867621/efficient-timer-algorithm。
> 
>Netty 提供的 HashedWheelTimer 正如其 java doc 中所言:A  Timer optimized for 
> approximated I/O timeout scheduling. 
> 是适用于固定场景的,如果有深入研究过时间轮算法,会发现,其并没有实现多级时间轮,时间轮算法可以参考这篇论文:http://www.cs.columbia.edu/~nahum/w6998/papers/sosp87-timing-wheels.pdf
>  
> 
>我并没有找到 2.7 有关于这个改动的邮件,不太清楚去除 ScheduledThreadPoolExecutor, 使用 
> HashedWheelTimer 的原因,不知道我的检查是否有所遗漏。
> 
>下面是四个讨论点,索引一下,方便大家讨论:
> 
>1. 2.7 使用 Netty 的 HashedWheelTimer 是为了解决什么现有问题?
>2. 使用 ScheduledThreadPoolExecutor 可以更好的控制多线程执行定时任务(虽然现在的 HashWheelTimer 
> 也可以做到),尽量保持了和 2.6 的风格一致。
>3. Netty4.1 没有使用 HashedWheelTimer,我没有搜索到相关的原因,但我觉得 Netty 
> 这款优秀的框架必然考虑到了什么,希望得到大家的讨论。由于 Netty 的这一点改动,我更倾向于在 Dubbo 中也使用 
> ScheduledThreadPoolExecutor 
>4. 在心跳这样的场景,我们可以把定时器交给 netty 的 eventLoop 去做 
> Schedule,这样并不是耦合,可以做成可配置的项来选择性的重载,可以保证 Timer 线程和请求线程的线程安全(这一点是参考了 
> IdleStateHandler 的设计)。在 Dubbo 2.7 中,Timer reput 的逻辑耦合在了抽象类中,这样导致了 Timer 
> 并不具备很高的重用性。
>5. HashedWheelTimer 和 ScheduledThreadPoolExecutor 分别适用的场景
> 
>期待大家的讨论
> 
>Recently, when I was researching Dubbo 2.7 code, I found an interesting 
> change. Dubbo 2.6 used the ScheduledThreadPoolExecutor extensively. For 
> example, Dubbo implemented a timed heartbeat and started the timer to judge 
> the asynchronous request timeout. It is replaced in Dubbo 2.7 using 
> HashedWheelTimer implemented by Netty.
> 
>I was curious about this change, so I did some research on the 
> implementation of the timer. The purpose of the discussion was to choose the 
> most suitable timer for 2.7.
> 
>JDK's Timer has design issues that is no use to be discussed. I want to 
> compare the differences between HashedWheelTimer and 
> ScheduledThreadPoolExecutor.
> 
>Netty has a lot of use of HashedWheelTimer in 3.x, but in 4.1, we can find 
> that Netty retains HashedWheelTimer, but it is not used in its source code, 
> but chose ScheduledThreadPoolExecutor, which was also proposed in Stack 
> Overflow. Question: 
> https://stackoverflow.com/questions/867621/efficient-timer-algorithm.
> 
>The HashedWheelTimer provided by Netty is as described in its java doc: A 
> Timer optimized for approximated I/O timeout scheduling. It is suitable for 
> fixed scenes. If you have studied the time wheel algorithm in depth, you will 
> find that it does not implement multi-level time wheel. The time wheel 
> algorithm can refer to this paper: 
> http://www.cs.columbia.edu/~nahum/w6998/papers/sosp87-timing-wheels.pdf
> 
>I didn't find the 2.7 email about this change. I don't know why I removed 
> the ScheduledThreadPoolExecutor and used HashedWheelTimer. I don't know if my 
> check is missing.
> 
>Here are four discussion points, indexing, for your convenience:
> 
>1. 2.7 Is Netty's HashedWheelTimer used to solve any existing problems?
>2. Use ScheduledThreadPoolExecutor to better control multi-threaded 
> execution timing tasks (although HashWheelTimer can do it now), and try to 
> keep the style consistent with 2.6.
>3. Netty4.1 didn't use HashedWheelTimer, I didn't search for related 
> reasons, but I think Netty's excellent framework must take into account what 
> I want, and I hope to get everyone's discussion. Due to this change in Netty, 
> I prefer to use ScheduledThreadPoolExecutor in Dubbo as well.
>4. In a scenario like heartbeat, we can pass the timer to netty's 
> eventLoop to do the Schedule. This is not a coupling. It can be made into a 
> configurable item for selective overloading. It can guarantee the thread of 
> the Timer thread and the request thread. Security (this is a reference to the 
> design of the IdleStateHandler). In Dubbo 2.7, the logic of Timer reput is 
> coupled in an abstract class, which results in a Timer that is not highly 
>

Re: Telnet optimization suggest

2018-12-17 Thread Jason Joo
And is there other situations to deal with? group/version/etc...

For now there is only different implementations in same application and looks 
good to deal with.

best regards,

Jason

> On Dec 18, 2018, at 10:33, yeling  wrote:
> 
> Ok,I've created an issue.
> https://github.com/apache/incubator-dubbo/issues/2988
> 
> Ian Luo  于2018年12月17日周一 下午11:41写道:
> 
>> Zhen,
>> 
>> 
>>> 1. When my interface implements multiple services, and all of them are
>>> exposed, using the Dubbo parameter group, at this time, under the telnet
>>> command, I can not choose the service I want to invoke.
>> 
>> 
>> I am afraid I fully understand the problem. Could you pls. file an issue on
>> GitHub together with a sample service to reproduce the problem?
>> 
>> Thanks,
>> -Ian.
>> 
>> On Mon, Dec 17, 2018 at 7:52 PM 李振  wrote:
>> 
 
 I will try to fix it。
>>> 
>> 



Re: dubbo使用中的疑问

2018-11-09 Thread Jason Joo
啊,抱歉,那可能是记错了method的属性集。

之前老版本的xsd貌似会有多余的自动提示,只测试过多个service同时设置connections时的情况(为了解决100倍以上的小尖峰问题)。

其实关于单连接模型,这个是有什么约束的吗?是否有可能设置为数量N(default=1)的长连接池,全局均可受惠。

best regards,

Jason

> On Nov 9, 2018, at 11:19, yuhang xiu  wrote:
> 
>> 
>>> 另外由于method/service均可设置该属性,多个属性会叠加这个问题
>> 我对着保持怀疑态度,晚点我会仔细check一下
> 
> 
> 我也对这个存疑,目前来看method的connections属性对整体的连接数影响我还没有发现。测试了一下发现method设置的connections不会起作用。
> 
> Zonghai Shang  于2018年11月9日周五 上午11:16写道:
> 
>> Hi,
>> 
>>> 另外由于method/service均可设置该属性,多个属性会叠加这个问题
>> 
>> 我对着保持怀疑态度,晚点我会仔细check一下
>> 
>> -yiji
>> 
>> Zonghai Shang  于2018年11月9日周五 上午10:46写道:
>> 
>>> Hi,
>>> 
>>> 
>>> 
>> dubbo默认值会复用建立的1个tcp连接,nettyServerBootstrapAcceptor会监听到,会用其中一个childgroup和这个channel做绑定,其他eventloop只会空轮训,绑定的channel线程会读写数据,没有理解哪里有问题。
>>> 
>>>> 最终每个consumer是N * n,对于每个provider则是N * m
>>> 
>>> 
>> 我这里针对他的问题,提供建议,让他增大连接数验证,netty收到同一个client多个channel连接一定会均衡绑定eventloop(默认策略)。
>>> 
>>> -yiji
>>> 
>>> Ian Luo  于2018年11月9日周五 上午10:37写道:
>>> 
>>>> +1, total agree with Jason's suggestion.
>>>> 
>>>> On Fri, Nov 9, 2018 at 5:16 AM Jason Joo  wrote:
>>>> 
>>>>> Hi,
>>>>> 
>>>>> 
>>>> 
>> 设置connection属性要慎重,建议从1起,因为对于connection=N,每个设置了该属性的项目会在连provider时每个provider除已连接的共用连接外均新建N个连接,也就是最终每个consumer是N
>>>>> * n,对于每个provider则是N * m
>>>>> 
>>>>> 其中,
>>>>> n - provider数
>>>>> m - 消费数
>>>>> 
>>>>> 另外由于method/service均可设置该属性,多个属性会叠加这个问题
>>>>> 
>>>>> best regards,
>>>>> 
>>>>> Jason
>>>>> 
>>>>>> On Nov 8, 2018, at 11:54, Zonghai Shang  wrote:
>>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> 麻烦图片重新发下,这边看不到图片。
>>>>>> 
>>>>>> 1. dubbo默认多个请求会复用同一个tcp连接,所以只有一个NettyServerWork去处理client
>>>>>> io事件,netty模型会把chanel绑定到eventloop,一旦绑定永远不会改变。
>>>>>> 
>>>>>> 2. 你可以增加client端连接数,比如connections=3 或者4,5,然后再观察。
>>>>>> 
>>>>>> 备注: netty服务端默认会把tcp连接公平分发给children eventloop。
>>>>>> 
>>>>>> -yiji
>>>>>> 
>>>>>> 噬音 <402264...@qq.com> 于2018年11月7日周三 下午7:29写道:
>>>>>> 
>>>>>>> 你们好
>>>>>>>  我在使用过程中有个疑问,请在百忙中抽空解答一下:
>>>>>>> 
>>>>> 
>>>> 
>> 官网说明iothreads默认是cpu个数+1,从线程栈中看到有1个NettyClientWoker、3个NettyServerWork和1个NettyServerBoss线程;在服务运行过程中,看到NettyClientWoker和其中一个NettyServerWork线程是有CPU负载的,按照Netty框架,通过Boss线程去select
>>>>>>> 
>>>>> 
>>>> 
>> work线程,然后work去run,应该三个NettyServerWork线程cpu使用率应该是相近的;为什么只有一个NettyServerWork
>>>>>>> cpu负载高,其它的两个线程NettyServerWork几乎看不到cpu负载(线程状态都是RUNNABLE的)?
>>>>>>> 
>>>>>>> PS:使用的是4核cpu的服务器,dubbo版本号2.6.2,dubbo使用默认配置
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>> 



Re: dubbo使用中的疑问

2018-11-08 Thread Jason Joo
Hi,

设置connection属性要慎重,建议从1起,因为对于connection=N,每个设置了该属性的项目会在连provider时每个provider除已连接的共用连接外均新建N个连接,也就是最终每个consumer是N
 * n,对于每个provider则是N * m

其中,
n - provider数
m - 消费数

另外由于method/service均可设置该属性,多个属性会叠加这个问题

best regards,

Jason

> On Nov 8, 2018, at 11:54, Zonghai Shang  wrote:
> 
> Hi,
> 
> 麻烦图片重新发下,这边看不到图片。
> 
> 1. dubbo默认多个请求会复用同一个tcp连接,所以只有一个NettyServerWork去处理client
> io事件,netty模型会把chanel绑定到eventloop,一旦绑定永远不会改变。
> 
> 2. 你可以增加client端连接数,比如connections=3 或者4,5,然后再观察。
> 
> 备注: netty服务端默认会把tcp连接公平分发给children eventloop。
> 
> -yiji
> 
> 噬音 <402264...@qq.com> 于2018年11月7日周三 下午7:29写道:
> 
>> 你们好
>>   我在使用过程中有个疑问,请在百忙中抽空解答一下:
>> 官网说明iothreads默认是cpu个数+1,从线程栈中看到有1个NettyClientWoker、3个NettyServerWork和1个NettyServerBoss线程;在服务运行过程中,看到NettyClientWoker和其中一个NettyServerWork线程是有CPU负载的,按照Netty框架,通过Boss线程去select
>> work线程,然后work去run,应该三个NettyServerWork线程cpu使用率应该是相近的;为什么只有一个NettyServerWork
>> cpu负载高,其它的两个线程NettyServerWork几乎看不到cpu负载(线程状态都是RUNNABLE的)?
>> 
>> PS:使用的是4核cpu的服务器,dubbo版本号2.6.2,dubbo使用默认配置
>> 
>> 
>> 
>> 



Re: pull request #2647: [Dubbo-2578]Smooth Round Robin selection

2018-10-25 Thread Jason Joo
hi, Luo

Implementation of 2.5.3 is indeed fast but it's not correct. Here is example 
running report:

Weights array: 
100, 100, 200, 200, 300, 300, 400, 400, 500, 500

2.5.3:
2, 2, 45000, 45000, 78333, 78333, 128333, 128333, 228334, 228334

before patch:
33400, 33400, 66700, 66700, 10, 10, 133300, 133300, 166600, 166600

after patch:
33399, 33399, 66701, 66701, 10, 10, 133300, 133300, 166600, 166600

smooth one:
3, 3, 7, 7, 10, 10, 13, 13, 17, 17


And the smooth version is a strict implementation to see how many invocation 
should really happen.

best regards,

Jason

> On Oct 26, 2018, at 11:28, Ian Luo  wrote:
> 
> one



Re: pull request #2647: [Dubbo-2578]Smooth Round Robin selection

2018-10-25 Thread Jason Joo
hi, luo

Changes has been applied last night, both in 2.6.x/master

best regards,

Jason

> On Oct 26, 2018, at 10:14, Ian Luo  wrote:
> 
> Jason,
> 
> The problem is, if we use Map instead of ConcurrentMap, then
> weightedRoundRobin could be two different objects when two threads
> executes at the same time if they happen to use the same URL.
> 
> String identifyString = invoker.getUrl().toIdentityString();
> WeightedRoundRobin weightedRoundRobin = map.get(identifyString);
> int weight = getWeight(invoker, invocation);
> if (weight < 0) {
>weight = 0;
> }
> if (weightedRoundRobin == null) {
>weightedRoundRobin = new WeightedRoundRobin();
>weightedRoundRobin.setWeight(weight);
>map.putIfAbsent(identifyString, weightedRoundRobin);
>weightedRoundRobin = map.get(identifyString);
> }
> 
> 
> Thanks,
> -Ian.
> On Thu, Oct 25, 2018 at 8:27 PM Jason Joo  wrote:
>> 
>> hi, Luo
>> 
>> Due to a loop on MAP object each time selecting the performance will reduce 
>> by 9.3% in single thread test.
>> 
>> Does it worth it?
>> 
>> Under 5, 10, 20 threads the difference is not stable: 8~13% loss
>> 
>> What do you think?
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Oct 25, 2018, at 19:42, Ian Luo  wrote:
>>> 
>>> Jason,
>>> 
>>> what I suggest is to simply declare methodWeightMap from
>>> ConcurrentMap> to
>>> ConcurrentMap>, then
>>> you will have 'putIfAbsent', what do you think?
>>> 
>>> Thanks,
>>> -Ian.
>>> 
>>> On Thu, Oct 25, 2018 at 3:47 PM Jason Joo  wrote:
>>>> 
>>>> hi, luo
>>>> 
>>>> Just replace putIfAbsent with put for compatibility and tested in racing 
>>>> condition.
>>>> 
>>>> best regards,
>>>> 
>>>> Jason
>>>> 
>>>>> On Oct 25, 2018, at 11:33, Ian Luo  wrote:
>>>>> 
>>>>> great.
>>>>> On Thu, Oct 25, 2018 at 11:33 AM Jason Joo  wrote:
>>>>>> 
>>>>>> hi, Luo
>>>>>> 
>>>>>> I am switching branch and ide hangs.I am figuring it
>>>>>> 
>>>>>> best regards,
>>>>>> 
>>>>>> Jason
>>>>>> 
>>>>>>> On Oct 25, 2018, at 11:28, Ian Luo  wrote:
>>>>>>> 
>>>>>>> Jason,
>>>>>>> 
>>>>>>> Last comment on 'putIfAbsent', pls. take a look.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> -Ian.
>>>>>>> 
>>>>>>> On Thu, Oct 25, 2018 at 11:02 AM Ian Luo  wrote:
>>>>>>>> 
>>>>>>>> Jason,
>>>>>>>> 
>>>>>>>> More comments, pls. take a look :)
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> -Ian.
>>>>>>>> On Thu, Oct 25, 2018 at 10:50 AM Jason Joo  wrote:
>>>>>>>>> 
>>>>>>>>> hi, Luo
>>>>>>>>> 
>>>>>>>>> Thanks for your great suggestions and PR has been updated.
>>>>>>>>> 
>>>>>>>>> We need more work testing and reviewing to prevent poor performance 
>>>>>>>>> and so on mentioned in #2578.
>>>>>>>>> 
>>>>>>>>> best regards,
>>>>>>>>> 
>>>>>>>>> Jason
>>>>>>>>> 
>>>>>>>>>> On Oct 25, 2018, at 10:35, Ian Luo  wrote:
>>>>>>>>>> 
>>>>>>>>>> Jason,
>>>>>>>>>> 
>>>>>>>>>> I take a look at the updated pull request, and give a few 
>>>>>>>>>> suggestions.
>>>>>>>>>> I think it's on the right direction, pls. take a look.
>>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> -Ian.
>>>>>>>>>> On Thu, Oct 25, 2018 at 10:19 AM Ian Luo  wrote:
>>>>>>>>>>> 
>>>>>>>>>>> The data seems convincing. What's meaning of 'before patch' and 
>>>>>>>>>>> 'after patc

Re: pull request #2647: [Dubbo-2578]Smooth Round Robin selection

2018-10-25 Thread Jason Joo
hi, Luo

Due to a loop on MAP object each time selecting the performance will reduce by 
9.3% in single thread test.

Does it worth it?

Under 5, 10, 20 threads the difference is not stable: 8~13% loss

What do you think?

best regards,

Jason

> On Oct 25, 2018, at 19:42, Ian Luo  wrote:
> 
> Jason,
> 
> what I suggest is to simply declare methodWeightMap from
> ConcurrentMap> to
> ConcurrentMap>, then
> you will have 'putIfAbsent', what do you think?
> 
> Thanks,
> -Ian.
> 
> On Thu, Oct 25, 2018 at 3:47 PM Jason Joo  wrote:
>> 
>> hi, luo
>> 
>> Just replace putIfAbsent with put for compatibility and tested in racing 
>> condition.
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Oct 25, 2018, at 11:33, Ian Luo  wrote:
>>> 
>>> great.
>>> On Thu, Oct 25, 2018 at 11:33 AM Jason Joo  wrote:
>>>> 
>>>> hi, Luo
>>>> 
>>>> I am switching branch and ide hangs.I am figuring it
>>>> 
>>>> best regards,
>>>> 
>>>> Jason
>>>> 
>>>>> On Oct 25, 2018, at 11:28, Ian Luo  wrote:
>>>>> 
>>>>> Jason,
>>>>> 
>>>>> Last comment on 'putIfAbsent', pls. take a look.
>>>>> 
>>>>> Thanks,
>>>>> -Ian.
>>>>> 
>>>>> On Thu, Oct 25, 2018 at 11:02 AM Ian Luo  wrote:
>>>>>> 
>>>>>> Jason,
>>>>>> 
>>>>>> More comments, pls. take a look :)
>>>>>> 
>>>>>> Thanks,
>>>>>> -Ian.
>>>>>> On Thu, Oct 25, 2018 at 10:50 AM Jason Joo  wrote:
>>>>>>> 
>>>>>>> hi, Luo
>>>>>>> 
>>>>>>> Thanks for your great suggestions and PR has been updated.
>>>>>>> 
>>>>>>> We need more work testing and reviewing to prevent poor performance and 
>>>>>>> so on mentioned in #2578.
>>>>>>> 
>>>>>>> best regards,
>>>>>>> 
>>>>>>> Jason
>>>>>>> 
>>>>>>>> On Oct 25, 2018, at 10:35, Ian Luo  wrote:
>>>>>>>> 
>>>>>>>> Jason,
>>>>>>>> 
>>>>>>>> I take a look at the updated pull request, and give a few suggestions.
>>>>>>>> I think it's on the right direction, pls. take a look.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> -Ian.
>>>>>>>> On Thu, Oct 25, 2018 at 10:19 AM Ian Luo  wrote:
>>>>>>>>> 
>>>>>>>>> The data seems convincing. What's meaning of 'before patch' and 
>>>>>>>>> 'after patch'?
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> -Ian.
>>>>>>>>> On Thu, Oct 25, 2018 at 1:17 AM Jason Joo  wrote:
>>>>>>>>>> 
>>>>>>>>>> hi, king
>>>>>>>>>> 
>>>>>>>>>> I think we also care about if it would cause abusing in practices of 
>>>>>>>>>> DUBBO.
>>>>>>>>>> The smooth version is at least twenty times faster compared with 
>>>>>>>>>> implementation of 2.6.x, and more smooth and make sense compared 
>>>>>>>>>> with implementation of 2.5.3. Actually it maybe about up to three 
>>>>>>>>>> times cost compared with recently correction.
>>>>>>>>>> 
>>>>>>>>>> typically cost(1000, 000 loops, simplified code, in milliseconds, 
>>>>>>>>>> single thread):
>>>>>>>>>> ---
>>>>>>>>>> before patch:  8536
>>>>>>>>>> after patch: 140
>>>>>>>>>> @2.5.3:   141
>>>>>>>>>> smooth:   414
>>>>>>>>>> 
>>>>>>>>>> but things changed in multi-thread condition(10 threads, using 
>>>>>>>>>> strict condition inner thread to begin loops):
>>>>>>>>>> -

Re: pull request #2647: [Dubbo-2578]Smooth Round Robin selection

2018-10-25 Thread Jason Joo
hi, luo

Just replace putIfAbsent with put for compatibility and tested in racing 
condition.

best regards,

Jason

> On Oct 25, 2018, at 11:33, Ian Luo  wrote:
> 
> great.
> On Thu, Oct 25, 2018 at 11:33 AM Jason Joo  wrote:
>> 
>> hi, Luo
>> 
>> I am switching branch and ide hangs.I am figuring it
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Oct 25, 2018, at 11:28, Ian Luo  wrote:
>>> 
>>> Jason,
>>> 
>>> Last comment on 'putIfAbsent', pls. take a look.
>>> 
>>> Thanks,
>>> -Ian.
>>> 
>>> On Thu, Oct 25, 2018 at 11:02 AM Ian Luo  wrote:
>>>> 
>>>> Jason,
>>>> 
>>>> More comments, pls. take a look :)
>>>> 
>>>> Thanks,
>>>> -Ian.
>>>> On Thu, Oct 25, 2018 at 10:50 AM Jason Joo  wrote:
>>>>> 
>>>>> hi, Luo
>>>>> 
>>>>> Thanks for your great suggestions and PR has been updated.
>>>>> 
>>>>> We need more work testing and reviewing to prevent poor performance and 
>>>>> so on mentioned in #2578.
>>>>> 
>>>>> best regards,
>>>>> 
>>>>> Jason
>>>>> 
>>>>>> On Oct 25, 2018, at 10:35, Ian Luo  wrote:
>>>>>> 
>>>>>> Jason,
>>>>>> 
>>>>>> I take a look at the updated pull request, and give a few suggestions.
>>>>>> I think it's on the right direction, pls. take a look.
>>>>>> 
>>>>>> Thanks,
>>>>>> -Ian.
>>>>>> On Thu, Oct 25, 2018 at 10:19 AM Ian Luo  wrote:
>>>>>>> 
>>>>>>> The data seems convincing. What's meaning of 'before patch' and 'after 
>>>>>>> patch'?
>>>>>>> 
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> -Ian.
>>>>>>> On Thu, Oct 25, 2018 at 1:17 AM Jason Joo  wrote:
>>>>>>>> 
>>>>>>>> hi, king
>>>>>>>> 
>>>>>>>> I think we also care about if it would cause abusing in practices of 
>>>>>>>> DUBBO.
>>>>>>>> The smooth version is at least twenty times faster compared with 
>>>>>>>> implementation of 2.6.x, and more smooth and make sense compared with 
>>>>>>>> implementation of 2.5.3. Actually it maybe about up to three times 
>>>>>>>> cost compared with recently correction.
>>>>>>>> 
>>>>>>>> typically cost(1000, 000 loops, simplified code, in milliseconds, 
>>>>>>>> single thread):
>>>>>>>> ---
>>>>>>>> before patch:  8536
>>>>>>>> after patch: 140
>>>>>>>> @2.5.3:   141
>>>>>>>> smooth:   414
>>>>>>>> 
>>>>>>>> but things changed in multi-thread condition(10 threads, using strict 
>>>>>>>> condition inner thread to begin loops):
>>>>>>>> ---
>>>>>>>> before patch: 25002   23939 23435
>>>>>>>> after patch:2141 2034   2033
>>>>>>>> @2.5.3: 1241 1229   1193
>>>>>>>> smooth: 2081 2231   2376
>>>>>>>> 
>>>>>>>> 
>>>>>>>> So they are more or less in multi threaded environment due to some 
>>>>>>>> local object trade off.  Test code is in attachment.
>>>>>>>> 
>>>>>>>> Certainly it's a little geek in this case and i will be ok about our 
>>>>>>>> future conclusion.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> best regards,
>>>>>>>> 
>>>>>>>> Jason
>>>>>>>> 
>>>>>>>> On Oct 24, 2018, at 23:17, KimmKing  wrote:
>>>>>>>> 
>>>>>>>> Thanks for a good pr first.
>>>>>>>> But I think old-way is better useful.
>>>>>>>> When we

Re: pull request #2647: [Dubbo-2578]Smooth Round Robin selection

2018-10-24 Thread Jason Joo
hi, Luo

I am switching branch and ide hangs.I am figuring it

best regards,

Jason

> On Oct 25, 2018, at 11:28, Ian Luo  wrote:
> 
> Jason,
> 
> Last comment on 'putIfAbsent', pls. take a look.
> 
> Thanks,
> -Ian.
> 
> On Thu, Oct 25, 2018 at 11:02 AM Ian Luo  wrote:
>> 
>> Jason,
>> 
>> More comments, pls. take a look :)
>> 
>> Thanks,
>> -Ian.
>> On Thu, Oct 25, 2018 at 10:50 AM Jason Joo  wrote:
>>> 
>>> hi, Luo
>>> 
>>> Thanks for your great suggestions and PR has been updated.
>>> 
>>> We need more work testing and reviewing to prevent poor performance and so 
>>> on mentioned in #2578.
>>> 
>>> best regards,
>>> 
>>> Jason
>>> 
>>>> On Oct 25, 2018, at 10:35, Ian Luo  wrote:
>>>> 
>>>> Jason,
>>>> 
>>>> I take a look at the updated pull request, and give a few suggestions.
>>>> I think it's on the right direction, pls. take a look.
>>>> 
>>>> Thanks,
>>>> -Ian.
>>>> On Thu, Oct 25, 2018 at 10:19 AM Ian Luo  wrote:
>>>>> 
>>>>> The data seems convincing. What's meaning of 'before patch' and 'after 
>>>>> patch'?
>>>>> 
>>>>> 
>>>>> Thanks,
>>>>> -Ian.
>>>>> On Thu, Oct 25, 2018 at 1:17 AM Jason Joo  wrote:
>>>>>> 
>>>>>> hi, king
>>>>>> 
>>>>>> I think we also care about if it would cause abusing in practices of 
>>>>>> DUBBO.
>>>>>> The smooth version is at least twenty times faster compared with 
>>>>>> implementation of 2.6.x, and more smooth and make sense compared with 
>>>>>> implementation of 2.5.3. Actually it maybe about up to three times cost 
>>>>>> compared with recently correction.
>>>>>> 
>>>>>> typically cost(1000, 000 loops, simplified code, in milliseconds, single 
>>>>>> thread):
>>>>>> ---
>>>>>> before patch:  8536
>>>>>> after patch: 140
>>>>>> @2.5.3:   141
>>>>>> smooth:   414
>>>>>> 
>>>>>> but things changed in multi-thread condition(10 threads, using strict 
>>>>>> condition inner thread to begin loops):
>>>>>> ---
>>>>>> before patch: 25002   23939 23435
>>>>>> after patch:2141 2034   2033
>>>>>> @2.5.3: 1241 1229   1193
>>>>>> smooth: 2081 2231   2376
>>>>>> 
>>>>>> 
>>>>>> So they are more or less in multi threaded environment due to some local 
>>>>>> object trade off.  Test code is in attachment.
>>>>>> 
>>>>>> Certainly it's a little geek in this case and i will be ok about our 
>>>>>> future conclusion.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> best regards,
>>>>>> 
>>>>>> Jason
>>>>>> 
>>>>>> On Oct 24, 2018, at 23:17, KimmKing  wrote:
>>>>>> 
>>>>>> Thanks for a good pr first.
>>>>>> But I think old-way is better useful.
>>>>>> When we need a balance, it means so much invoking occurs, so the most 
>>>>>> important point is:
>>>>>> 
>>>>>> * Lightweight and Performance
>>>>>> Then accuracy and smoothness are not so much prior.
>>>>>> And actually nobody would have more than 100 providers for a single 
>>>>>> service, so the number of invokers is small,
>>>>>> we could not care about its change, check stage is not a key point.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> At 2018-10-24 17:48:31, "Jason Joo"  wrote:
>>>>>> 
>>>>>> hi, xiu
>>>>>> 
>>>>>> After some deeper research on the algorithm i found it seemed like 
>>>>>> thread-safe.
>>>>>> 
>>>>>> I updated my PR and unit test on it so we can examine if it acts as 
>>>>>> expected.
>>>>>

Re: pull request #2647: [Dubbo-2578]Smooth Round Robin selection

2018-10-24 Thread Jason Joo
hi, Luo

Thanks for your great suggestions and PR has been updated.

We need more work testing and reviewing to prevent poor performance and so on 
mentioned in #2578.

best regards,

Jason

> On Oct 25, 2018, at 10:35, Ian Luo  wrote:
> 
> Jason,
> 
> I take a look at the updated pull request, and give a few suggestions.
> I think it's on the right direction, pls. take a look.
> 
> Thanks,
> -Ian.
> On Thu, Oct 25, 2018 at 10:19 AM Ian Luo  wrote:
>> 
>> The data seems convincing. What's meaning of 'before patch' and 'after 
>> patch'?
>> 
>> 
>> Thanks,
>> -Ian.
>> On Thu, Oct 25, 2018 at 1:17 AM Jason Joo  wrote:
>>> 
>>> hi, king
>>> 
>>> I think we also care about if it would cause abusing in practices of DUBBO.
>>> The smooth version is at least twenty times faster compared with 
>>> implementation of 2.6.x, and more smooth and make sense compared with 
>>> implementation of 2.5.3. Actually it maybe about up to three times cost 
>>> compared with recently correction.
>>> 
>>> typically cost(1000, 000 loops, simplified code, in milliseconds, single 
>>> thread):
>>> ---
>>> before patch:  8536
>>> after patch: 140
>>> @2.5.3:   141
>>> smooth:   414
>>> 
>>> but things changed in multi-thread condition(10 threads, using strict 
>>> condition inner thread to begin loops):
>>> ---
>>> before patch: 25002   23939 23435
>>> after patch:2141 2034   2033
>>> @2.5.3: 1241 1229   1193
>>> smooth: 2081 2231   2376
>>> 
>>> 
>>> So they are more or less in multi threaded environment due to some local 
>>> object trade off.  Test code is in attachment.
>>> 
>>> Certainly it's a little geek in this case and i will be ok about our future 
>>> conclusion.
>>> 
>>> 
>>> 
>>> best regards,
>>> 
>>> Jason
>>> 
>>> On Oct 24, 2018, at 23:17, KimmKing  wrote:
>>> 
>>> Thanks for a good pr first.
>>> But I think old-way is better useful.
>>> When we need a balance, it means so much invoking occurs, so the most 
>>> important point is:
>>> 
>>> * Lightweight and Performance
>>> Then accuracy and smoothness are not so much prior.
>>> And actually nobody would have more than 100 providers for a single 
>>> service, so the number of invokers is small,
>>> we could not care about its change, check stage is not a key point.
>>> 
>>> 
>>> 
>>> At 2018-10-24 17:48:31, "Jason Joo"  wrote:
>>> 
>>> hi, xiu
>>> 
>>> After some deeper research on the algorithm i found it seemed like 
>>> thread-safe.
>>> 
>>> I updated my PR and unit test on it so we can examine if it acts as 
>>> expected.
>>> 
>>> So the recycle logic I change it into an optimistic lock.
>>> 
>>> NGINX is a single-process model which lead me to a wrong path.
>>> 
>>> 
>>> 
>>> So it's lock-free and with memory recycler now.
>>> 
>>> Let's check if it works correctly.
>>> 
>>> 
>>> best regards,
>>> 
>>> Jason
>>> 
>>> On Oct 24, 2018, at 16:38, yuhang xiu  wrote:
>>> 
>>> Hi, jason
>>> 
>>> If we can have a good balance on synchronize, thread and memory, I support
>>> the way of smooth.
>>> We need more discussion and solutions on this issue. Looking forward to
>>> your solutions. :)
>>> 
>>> Thx
>>> 
>>> Jason Joo  于2018年10月24日周三 下午4:32写道:
>>> 
>>> hi, xiu
>>> 
>>> Good proposal.
>>> 
>>> The old not-very-smooth RoundRobin implementation can work fine in most
>>> conditions.
>>> Actually it depends the number of weight.
>>> That means 500:500:500 will be quite different from 1:1:1 and maybe we can
>>> simplify the weights when selecting.
>>> 
>>> 
>>> And i have another thought about the KEY.
>>> Now the key is defined by Method (which can make method's invocation RR).
>>> If we change it into by Service (which the method will not be RR if its
>>> invocations just match a round) the memory issue may looked better.
>>> 
>>> 
>

Re: pull request #2647: [Dubbo-2578]Smooth Round Robin selection

2018-10-24 Thread Jason Joo
hi, luo

The patch is in PR #2586 which improves the performance of round robin in 
versions 2.6.x.

Link is: https://github.com/apache/incubator-dubbo/pull/2586
Issue: https://github.com/apache/incubator-dubbo/issues/2578

best regards,

Jason

> On Oct 25, 2018, at 10:19, Ian Luo  wrote:
> 
> The data seems convincing. What's meaning of 'before patch' and 'after patch'?
> 
> 
> Thanks,
> -Ian.
> On Thu, Oct 25, 2018 at 1:17 AM Jason Joo  wrote:
>> 
>> hi, king
>> 
>> I think we also care about if it would cause abusing in practices of DUBBO.
>> The smooth version is at least twenty times faster compared with 
>> implementation of 2.6.x, and more smooth and make sense compared with 
>> implementation of 2.5.3. Actually it maybe about up to three times cost 
>> compared with recently correction.
>> 
>> typically cost(1000, 000 loops, simplified code, in milliseconds, single 
>> thread):
>> ---
>> before patch:  8536
>> after patch: 140
>> @2.5.3:   141
>> smooth:   414
>> 
>> but things changed in multi-thread condition(10 threads, using strict 
>> condition inner thread to begin loops):
>> ---
>> before patch: 25002   23939 23435
>> after patch:2141 2034   2033
>> @2.5.3: 1241 1229   1193
>> smooth: 2081 2231   2376
>> 
>> 
>> So they are more or less in multi threaded environment due to some local 
>> object trade off.  Test code is in attachment.
>> 
>> Certainly it's a little geek in this case and i will be ok about our future 
>> conclusion.
>> 
>> 
>> 
>> best regards,
>> 
>> Jason
>> 
>> On Oct 24, 2018, at 23:17, KimmKing  wrote:
>> 
>> Thanks for a good pr first.
>> But I think old-way is better useful.
>> When we need a balance, it means so much invoking occurs, so the most 
>> important point is:
>> 
>> * Lightweight and Performance
>> Then accuracy and smoothness are not so much prior.
>> And actually nobody would have more than 100 providers for a single service, 
>> so the number of invokers is small,
>> we could not care about its change, check stage is not a key point.
>> 
>> 
>> 
>> At 2018-10-24 17:48:31, "Jason Joo"  wrote:
>> 
>> hi, xiu
>> 
>> After some deeper research on the algorithm i found it seemed like 
>> thread-safe.
>> 
>> I updated my PR and unit test on it so we can examine if it acts as expected.
>> 
>> So the recycle logic I change it into an optimistic lock.
>> 
>> NGINX is a single-process model which lead me to a wrong path.
>> 
>> 
>> 
>> So it's lock-free and with memory recycler now.
>> 
>> Let's check if it works correctly.
>> 
>> 
>> best regards,
>> 
>> Jason
>> 
>> On Oct 24, 2018, at 16:38, yuhang xiu  wrote:
>> 
>> Hi, jason
>> 
>> If we can have a good balance on synchronize, thread and memory, I support
>> the way of smooth.
>> We need more discussion and solutions on this issue. Looking forward to
>> your solutions. :)
>> 
>> Thx
>> 
>> Jason Joo  于2018年10月24日周三 下午4:32写道:
>> 
>> hi, xiu
>> 
>> Good proposal.
>> 
>> The old not-very-smooth RoundRobin implementation can work fine in most
>> conditions.
>> Actually it depends the number of weight.
>> That means 500:500:500 will be quite different from 1:1:1 and maybe we can
>> simplify the weights when selecting.
>> 
>> 
>> And i have another thought about the KEY.
>> Now the key is defined by Method (which can make method's invocation RR).
>> If we change it into by Service (which the method will not be RR if its
>> invocations just match a round) the memory issue may looked better.
>> 
>> 
>> So what do other guys think about?
>> 
>> 
>> best regards,
>> 
>> Jason
>> 
>> On Oct 24, 2018, at 16:09, yuhang xiu  wrote:
>> 
>> 你好,
>> 
>> 我个人认为我们应该在smooth和性能之间做一个取舍。我个人倾向于使用非smooth的方式,原因:
>> 1.使用smooth需要增加线程开销或者增加内存开销,或者用锁来增加了性能开销,就像 @jason
>> 说的那样。而本身负载均衡是一个很轻量的组件,我们是否有必要把他做的这么复杂?
>> 
>> 2.我承认smooth的方式更好,但是如果我们把一个provider的权重设置为特定的值,那么说明他能承受相应的请求量,不管是N个请求同时进行还是分开进行。如果这个provider承受不住同时进行N个请求,我们需要做的应该是把他的权重调低。
>> 
>> 
>> 
>> 综上,我个人认为,当前的smooth实现方式不是很理想,我们没法在锁、线程和内存上都达到一个非常优秀的平衡。所以我认为可以不使用smooth的方式。
>> 
>> 
>> 谢谢。
>> 
>

Re: pull request #2647: [Dubbo-2578]Smooth Round Robin selection

2018-10-24 Thread Jason Joo
hi, kingI think we also care about if it would cause abusing in practices of DUBBO.The smooth version is at least twenty times faster compared with implementation of 2.6.x, and more smooth and make sense compared with implementation of 2.5.3. Actually it maybe about up to three times cost compared with recently correction.typically cost(1000, 000 loops, simplified code, in milliseconds, single thread):---before patch:  8536after patch:     140@2.5.3:           141smooth:           414but things changed in multi-thread condition(10 threads, using strict condition inner thread to begin loops):---before patch:     25002       23939         23435after patch:        2141         2034           2033@2.5.3:             1241         1229           1193smooth:             2081         2231           2376So they are more or less in multi threaded environment due to some local object trade off.  Test code is in attachment.Certainly it's a little geek in this case and i will be ok about our future conclusion.

DemoDubbo.java
Description: Binary data
best regards,Jason

On Oct 24, 2018, at 23:17, KimmKing <kimmk...@apache.org> wrote:Thanks for a good pr first.But I think old-way is better useful.When we need a balance, it means so much invoking occurs, so the most important point is:* Lightweight and PerformanceThen accuracy and smoothness are not so much prior.And actually nobody would have more than 100 providers for a single service, so the number of invokers is small,we could not care about its change, check stage is not a key point.At 2018-10-24 17:48:31, "Jason Joo" <hblz...@163.com> wrote:hi, xiuAfter some deeper research on the algorithm i found it seemed like thread-safe.I updated my PR and unit test on it so we can examine if it acts as expected.So the recycle logic I change it into an optimistic lock.NGINX is a single-process model which lead me to a wrong path.So it's lock-free and with memory recycler now.Let's check if it works correctly.best regards,JasonOn Oct 24, 2018, at 16:38, yuhang xiu <carry...@gmail.com> wrote:Hi, jasonIf we can have a good balance on synchronize, thread and memory, I supportthe way of smooth.We need more discussion and solutions on this issue. Looking forward toyour solutions. :)ThxJason Joo <hblz...@163.com> 于2018年10月24日周三 下午4:32写道:hi, xiuGood proposal.The old not-very-smooth RoundRobin implementation can work fine in mostconditions.Actually it depends the number of weight.That means 500:500:500 will be quite different from 1:1:1 and maybe we cansimplify the weights when selecting.And i have another thought about the KEY.Now the key is defined by Method (which can make method's invocation RR).If we change it into by Service (which the method will not be RR if itsinvocations just match a round) the memory issue may looked better.So what do other guys think about?best regards,JasonOn Oct 24, 2018, at 16:09, yuhang xiu <carry...@gmail.com> wrote:你好,我个人认为我们应该在smooth和性能之间做一个取舍。我个人倾向于使用非smooth的方式,原因:1.使用smooth需要增加线程开销或者增加内存开销,或者用锁来增加了性能开销,就像 @jason说的那样。而本身负载均衡是一个很轻量的组件,我们是否有必要把他做的这么复杂?2.我承认smooth的方式更好,但是如果我们把一个provider的权重设置为特定的值,那么说明他能承受相应的请求量,不管是N个请求同时进行还是分开进行。如果这个provider承受不住同时进行N个请求,我们需要做的应该是把他的权重调低。综上,我个人认为,当前的smooth实现方式不是很理想,我们没法在锁、线程和内存上都达到一个非常优秀的平衡。所以我认为可以不使用smooth的方式。谢谢。Jason Joo <hblz...@gmail.com> 于2018年10月24日周三 下午3:57写道:hi, LuoGlad to receive your response. I posted a reply before including someminds about the lock:* Separated recycle thread (more threads)or* Lock free (more memory in special conditions)It's truly a struggling job to make a decision and we can do ittogether.For the lock-free implementation it will waste or even fail if theproviders are fully dynamic deployed after running enough time(OOM?).best regards,JasonOn Oct 24, 2018, at 14:50, Ian Luo <ian@gmail.com> wrote:Hi Jason,I reviewed your pull request 2647 ( [Dubbo-2578]Smooth Round Robinselection) [1] today. It is an awesome implementation and I like it.There's one particular issue you may need to consider further is Ithinkwe should avoid synchronized block. I have drop review comments on thepullrequest. Would you mind to consider this further. After that, I thinkthischange is ready to go.Thanks,-Ian.1. https://github.com/apache/incubator-dubbo/pull/2647 <https://github.com/apache/incubator-dubbo/pull/2647>

Re: pull request #2647: [Dubbo-2578]Smooth Round Robin selection

2018-10-24 Thread Jason Joo
hi, xiu

After some deeper research on the algorithm i found it seemed like thread-safe.

I updated my PR and unit test on it so we can examine if it acts as expected.

So the recycle logic I change it into an optimistic lock.

NGINX is a single-process model which lead me to a wrong path.



So it's lock-free and with memory recycler now.

Let's check if it works correctly.


best regards,

Jason

> On Oct 24, 2018, at 16:38, yuhang xiu  wrote:
> 
> Hi, jason
> 
> If we can have a good balance on synchronize, thread and memory, I support
> the way of smooth.
> We need more discussion and solutions on this issue. Looking forward to
> your solutions. :)
> 
> Thx
> 
> Jason Joo  于2018年10月24日周三 下午4:32写道:
> 
>> hi, xiu
>> 
>> Good proposal.
>> 
>> The old not-very-smooth RoundRobin implementation can work fine in most
>> conditions.
>> Actually it depends the number of weight.
>> That means 500:500:500 will be quite different from 1:1:1 and maybe we can
>> simplify the weights when selecting.
>> 
>> 
>> And i have another thought about the KEY.
>> Now the key is defined by Method (which can make method's invocation RR).
>> If we change it into by Service (which the method will not be RR if its
>> invocations just match a round) the memory issue may looked better.
>> 
>> 
>> So what do other guys think about?
>> 
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Oct 24, 2018, at 16:09, yuhang xiu  wrote:
>>> 
>>> 你好,
>>> 
>>> 我个人认为我们应该在smooth和性能之间做一个取舍。我个人倾向于使用非smooth的方式,原因:
>>> 1.使用smooth需要增加线程开销或者增加内存开销,或者用锁来增加了性能开销,就像 @jason
>>> 说的那样。而本身负载均衡是一个很轻量的组件,我们是否有必要把他做的这么复杂?
>>> 
>> 2.我承认smooth的方式更好,但是如果我们把一个provider的权重设置为特定的值,那么说明他能承受相应的请求量,不管是N个请求同时进行还是分开进行。如果这个provider承受不住同时进行N个请求,我们需要做的应该是把他的权重调低。
>>> 
>>> 
>> 综上,我个人认为,当前的smooth实现方式不是很理想,我们没法在锁、线程和内存上都达到一个非常优秀的平衡。所以我认为可以不使用smooth的方式。
>>> 
>>> 谢谢。
>>> 
>>> Jason Joo  于2018年10月24日周三 下午3:57写道:
>>> 
>>>> hi, Luo
>>>> 
>>>> Glad to receive your response. I posted a reply before including some
>>>> minds about the lock:
>>>> 
>>>> * Separated recycle thread (more threads)
>>>> 
>>>> or
>>>> 
>>>> * Lock free (more memory in special conditions)
>>>> 
>>>> It's truly a struggling job to make a decision and we can do it
>> together.
>>>> For the lock-free implementation it will waste or even fail if the
>>>> providers are fully dynamic deployed after running enough time(OOM?).
>>>> 
>>>> best regards,
>>>> 
>>>> Jason
>>>> 
>>>>> On Oct 24, 2018, at 14:50, Ian Luo  wrote:
>>>>> 
>>>>> Hi Jason,
>>>>> 
>>>>> I reviewed your pull request 2647 ( [Dubbo-2578]Smooth Round Robin
>>>> selection) [1] today. It is an awesome implementation and I like it.
>>>>> 
>>>>> There's one particular issue you may need to consider further is I
>> think
>>>> we should avoid synchronized block. I have drop review comments on the
>> pull
>>>> request. Would you mind to consider this further. After that, I think
>> this
>>>> change is ready to go.
>>>>> 
>>>>> Thanks,
>>>>> -Ian.
>>>>> 
>>>>> 1. https://github.com/apache/incubator-dubbo/pull/2647 <
>>>> https://github.com/apache/incubator-dubbo/pull/2647>
>>>> 
>> 
>> 



Re: pull request #2647: [Dubbo-2578]Smooth Round Robin selection

2018-10-24 Thread Jason Joo
hi, xiu

Good proposal.

The old not-very-smooth RoundRobin implementation can work fine in most 
conditions. 
Actually it depends the number of weight. 
That means 500:500:500 will be quite different from 1:1:1 and maybe we can 
simplify the weights when selecting.


And i have another thought about the KEY. 
Now the key is defined by Method (which can make method's invocation RR). If we 
change it into by Service (which the method will not be RR if its invocations 
just match a round) the memory issue may looked better.


So what do other guys think about?


best regards,

Jason

> On Oct 24, 2018, at 16:09, yuhang xiu  wrote:
> 
> 你好,
> 
> 我个人认为我们应该在smooth和性能之间做一个取舍。我个人倾向于使用非smooth的方式,原因:
> 1.使用smooth需要增加线程开销或者增加内存开销,或者用锁来增加了性能开销,就像 @jason
> 说的那样。而本身负载均衡是一个很轻量的组件,我们是否有必要把他做的这么复杂?
> 2.我承认smooth的方式更好,但是如果我们把一个provider的权重设置为特定的值,那么说明他能承受相应的请求量,不管是N个请求同时进行还是分开进行。如果这个provider承受不住同时进行N个请求,我们需要做的应该是把他的权重调低。
> 
> 综上,我个人认为,当前的smooth实现方式不是很理想,我们没法在锁、线程和内存上都达到一个非常优秀的平衡。所以我认为可以不使用smooth的方式。
> 
> 谢谢。
> 
> Jason Joo  于2018年10月24日周三 下午3:57写道:
> 
>> hi, Luo
>> 
>> Glad to receive your response. I posted a reply before including some
>> minds about the lock:
>> 
>> * Separated recycle thread (more threads)
>> 
>> or
>> 
>> * Lock free (more memory in special conditions)
>> 
>> It's truly a struggling job to make a decision and we can do it together.
>> For the lock-free implementation it will waste or even fail if the
>> providers are fully dynamic deployed after running enough time(OOM?).
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Oct 24, 2018, at 14:50, Ian Luo  wrote:
>>> 
>>> Hi Jason,
>>> 
>>> I reviewed your pull request 2647 ( [Dubbo-2578]Smooth Round Robin
>> selection) [1] today. It is an awesome implementation and I like it.
>>> 
>>> There's one particular issue you may need to consider further is I think
>> we should avoid synchronized block. I have drop review comments on the pull
>> request. Would you mind to consider this further. After that, I think this
>> change is ready to go.
>>> 
>>> Thanks,
>>> -Ian.
>>> 
>>> 1. https://github.com/apache/incubator-dubbo/pull/2647 <
>> https://github.com/apache/incubator-dubbo/pull/2647>
>> 



Re: pull request #2647: [Dubbo-2578]Smooth Round Robin selection

2018-10-24 Thread Jason Joo
hi, Luo

Glad to receive your response. I posted a reply before including some minds 
about the lock: 

* Separated recycle thread (more threads)

or

* Lock free (more memory in special conditions)

It's truly a struggling job to make a decision and we can do it together.
For the lock-free implementation it will waste or even fail if the providers 
are fully dynamic deployed after running enough time(OOM?).

best regards,

Jason

> On Oct 24, 2018, at 14:50, Ian Luo  wrote:
> 
> Hi Jason,
> 
> I reviewed your pull request 2647 ( [Dubbo-2578]Smooth Round Robin selection) 
> [1] today. It is an awesome implementation and I like it. 
> 
> There's one particular issue you may need to consider further is I think we 
> should avoid synchronized block. I have drop review comments on the pull 
> request. Would you mind to consider this further. After that, I think this 
> change is ready to go.
> 
> Thanks,
> -Ian.
> 
> 1. https://github.com/apache/incubator-dubbo/pull/2647 
> 


Re: A preformance issue And Optimizing RoundRobinLoadBalance

2018-10-21 Thread Jason Joo
hi, kimmking

Maybe github has some problem today i cannot submit a reply in issue.

Here's my reply on locking:

---

There are two things should be pointed out:

1. check in a separated thread.
This method take one more thread in dubbo framework and should also add a lock 
here(or create another map when really recycle happen? It's lock free then.).

2. just ignore it
It's good in most cases but will cause increasing memory when the providers are 
frequently changed. So this balancer will not be operate free.

And the lock is distributed between methods level and only used for some 
frequently called methods. 

Synchronized lock is an auto-upgrade lock from thin lock in object header (when 
no concurrent) to monitor lock(mutex). So if the code block can be executed 
fast enough we may get a thin lock mostly.

@kimmking So what do you think about? Lock or lock-free, or a separate thread 
to do that?

best regards,

Jason

> On Oct 19, 2018, at 03:56, KimmKing  wrote:
> 
> hi, shuaijunlan
>> wrote:
>>>>> When calling the service seven times, the responding order is [A ,A, A,  
>>>>> A, A, B, C].
> 
> 
> Not at all, you can try it 7 times
> 
> 
> 
> 
> 
> 
> At 2018-10-16 14:37:33, "Ian Luo"  wrote:
>> I got your point, Jason, Would you mind to update your explanation on
>> https://github.com/apache/incubator-dubbo/issues/2578? It is convenient for
>> other people to revisit this issue later :)
>> 
>> Thanks,
>> -Ian.
>> 
>> On Tue, Oct 16, 2018 at 2:25 PM Jason Joo  wrote:
>> 
>>> Hi, lan
>>> 
>>> I gave a sample from Nginx® Code in JAVA and gained better performance
>>> compared to previous one(in that issue reply list).
>>> 
>>> Junlan's point is not to shuffle the addresses list and to select a
>>> different node every time as possible, eg:
>>> 
>>> A:B:C = 1:2:3
>>> [A, B, C, B, C, C,  A, B, C, B, C, C,A, B, C, B, C, C,
>>>   A, B, C, B, C, C, ..]
>>> 
>>> especially the factor number is large as: [90 : 100 : 111]
>>> 
>>> best regards,
>>> 
>>> Jason
>>> 
>>>> On Oct 16, 2018, at 14:12, Ian Luo  wrote:
>>>> 
>>>> Hi Junlan,
>>>> 
>>>> Are you suggesting we should shuffle the result address array?
>>>> 
>>>> Thanks,
>>>> -Ian.
>>>> 
>>>> On Fri, Oct 12, 2018 at 4:24 PM Junlan Shuai 
>>> wrote:
>>>> 
>>>>> Hi,
>>>>> The RoundRobinLoadBalance Algorithm is not smoothing.
>>>>> For example, assuming that the weight of Server A is five, the weight of
>>>>> Server
>>>>> B is one, the weight of Server C is one.
>>>>> When calling the service seven times, the responding order is [A ,A, A,
>>> A,
>>>>> A, B, C].
>>>>> By this responding order, we can find that the first five callings
>>> point to
>>>>> Server A, it will cause Server A to be overloaded in a short period of
>>>>> time.
>>>>> Ideally, the call order would be [A ,A, B, A, C, A, A].
>>>>> So, I think there may be a better way to do this.
>>>>> 
>>>>> KimmKing  于2018年9月30日周日 下午4:09写道:
>>>>> 
>>>>>> Hi, community.
>>>>>> 
>>>>>> 
>>>>>> Dubbo user manzhizhen find a preformance bug in
>>>>>> dubbo-cluster/RoundRobinLoadBalance and discuss in Github Is
>>>>>> https://github.com/apache/incubator-dubbo/issues/2578
>>>>>> 
>>>>>> 
>>>>>> Here I describe it shortly:
>>>>>> 1. The new version RoundRobinLoadBalance algorithm use an iterator
>>> from 0
>>>>>> to maxWeight:
>>>>>> for (int i = 0; i < maxWeight; i++) { // this is the reason
>>>>>>for (Map.Entry each :
>>>>>> invokerToWeightMap.entrySet()) {
>>>>>>   
>>>>>>}
>>>>>> }
>>>>>> 2.If maxWeight is a number  much more than 100, then this select method
>>>>>> will be executed so mush slowly.
>>>>>> 3. When call it 1M times: maxWeight is 5000, then it need 60+ seconds;
>>>>>> maxWeight is 5, then it need 10+ minutes.
>>>>>> 4. And in a deep compare testing, RandomLoadBalance &
>>>>>> RoundRobinLoadBalance (in 2.5.3, but implement is wrong) need 100-200ms
>>>>>> while calling 1M times.
>>>>>> 5. So we should Optimize RoundRobinLoadBalance to a new
>>> high-performance
>>>>>> algorithm implement, abandoning iterator from 0 to maxWeight.
>>>>>> 6. Then gudegg submit a PR to Optimize RoundRobinLoadBalance correctly,
>>>>>> the balance results by new PR is absolutely correct and only elapse
>>>>>> 100-300ms in my test like step 3.
>>>>>> 
>>>>>> 
>>>>>> Think it above over and over, I suggest we should merge the PR
>>> recently.
>>>>>> Any questions, buddies?
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Kimm King(kimmk...@apache.org/kimmk...@163.com)
>>>>>> Apache Dubbo Committer/Alibaba Fastjson Developer/Aliyun MVP
>>>>>> 
>>>>>> 
>>>>> 
>>>>> --
>>>>> Junlan
>>>>> Best regards!
>>>>> 
>>> 
>>> 



Re: A preformance issue And Optimizing RoundRobinLoadBalance

2018-10-16 Thread Jason Joo
Hi lan,

I try to submit a new PR to see whether it is useful (And restructure the code 
of its unit test).

https://github.com/apache/incubator-dubbo/pull/2647 
<https://github.com/apache/incubator-dubbo/pull/2647>


best regards,

Jason

> On Oct 16, 2018, at 16:18, Ian Luo  wrote:
> 
> Jason, if you've had a pull request ready, or you are willing to work on
> it, why not :)
> 
> Thanks,
> -Ian.
> 
> On Tue, Oct 16, 2018 at 3:16 PM Jason Joo  wrote:
> 
>> Hi lan,
>> 
>> I will update it.
>> 
>> Should I submit a PR as proposal?
>> The implementation is just a little "longer" then before and we can't have
>> a real performance result in unit test because the mock's poor performance.
>> 
>> best regards,
>> 
>> Jason
>> 
>>> On Oct 16, 2018, at 14:37, Ian Luo  wrote:
>>> 
>>> other people to revisit this issue later :)
>> 
>> 



Re: A preformance issue And Optimizing RoundRobinLoadBalance

2018-10-16 Thread Jason Joo
Hi lan,

I will update it.

Should I submit a PR as proposal?
The implementation is just a little "longer" then before and we can't have a 
real performance result in unit test because the mock's poor performance.

best regards,

Jason

> On Oct 16, 2018, at 14:37, Ian Luo  wrote:
> 
> other people to revisit this issue later :)



Re: A preformance issue And Optimizing RoundRobinLoadBalance

2018-10-15 Thread Jason Joo
Hi, lan

I gave a sample from Nginx® Code in JAVA and gained better performance compared 
to previous one(in that issue reply list).

Junlan's point is not to shuffle the addresses list and to select a different 
node every time as possible, eg:

A:B:C = 1:2:3
[A, B, C, B, C, C,  A, B, C, B, C, C,A, B, C, B, C, C,   A, 
B, C, B, C, C, ..]

especially the factor number is large as: [90 : 100 : 111]

best regards,

Jason

> On Oct 16, 2018, at 14:12, Ian Luo  wrote:
> 
> Hi Junlan,
> 
> Are you suggesting we should shuffle the result address array?
> 
> Thanks,
> -Ian.
> 
> On Fri, Oct 12, 2018 at 4:24 PM Junlan Shuai  wrote:
> 
>> Hi,
>> The RoundRobinLoadBalance Algorithm is not smoothing.
>> For example, assuming that the weight of Server A is five, the weight of
>> Server
>> B is one, the weight of Server C is one.
>> When calling the service seven times, the responding order is [A ,A, A, A,
>> A, B, C].
>> By this responding order, we can find that the first five callings point to
>> Server A, it will cause Server A to be overloaded in a short period of
>> time.
>> Ideally, the call order would be [A ,A, B, A, C, A, A].
>> So, I think there may be a better way to do this.
>> 
>> KimmKing  于2018年9月30日周日 下午4:09写道:
>> 
>>> Hi, community.
>>> 
>>> 
>>> Dubbo user manzhizhen find a preformance bug in
>>> dubbo-cluster/RoundRobinLoadBalance and discuss in Github Is
>>> https://github.com/apache/incubator-dubbo/issues/2578
>>> 
>>> 
>>> Here I describe it shortly:
>>> 1. The new version RoundRobinLoadBalance algorithm use an iterator from 0
>>> to maxWeight:
>>> for (int i = 0; i < maxWeight; i++) { // this is the reason
>>> for (Map.Entry each :
>>> invokerToWeightMap.entrySet()) {
>>>
>>> }
>>> }
>>> 2.If maxWeight is a number  much more than 100, then this select method
>>> will be executed so mush slowly.
>>> 3. When call it 1M times: maxWeight is 5000, then it need 60+ seconds;
>>> maxWeight is 5, then it need 10+ minutes.
>>> 4. And in a deep compare testing, RandomLoadBalance &
>>> RoundRobinLoadBalance (in 2.5.3, but implement is wrong) need 100-200ms
>>> while calling 1M times.
>>> 5. So we should Optimize RoundRobinLoadBalance to a new high-performance
>>> algorithm implement, abandoning iterator from 0 to maxWeight.
>>> 6. Then gudegg submit a PR to Optimize RoundRobinLoadBalance correctly,
>>> the balance results by new PR is absolutely correct and only elapse
>>> 100-300ms in my test like step 3.
>>> 
>>> 
>>> Think it above over and over, I suggest we should merge the PR recently.
>>> Any questions, buddies?
>>> 
>>> 
>>> 
>>> 
>>> --
>>> Kimm King(kimmk...@apache.org/kimmk...@163.com)
>>> Apache Dubbo Committer/Alibaba Fastjson Developer/Aliyun MVP
>>> 
>>> 
>> 
>> --
>> Junlan
>> Best regards!
>> 



Re: about grpc support

2018-09-12 Thread Jason Joo
hi, dp

According to principle of gRPC which is a solution crossing front-end(mobile) 
to back-end(server) using HTTP/2, i think it's suitable to be integrated in 
Controller layer(some like dubbo-consumer) but not service 
layer(dubbo-provider).

Nothing blocks us to use gRPC in consumers, right?


gRPC是一个跨端整体调用解决方案,这个个人理解一般应用在业务逻辑层,也就是一般对应dubbo的消费端,而不是服务端,所以两者完全可以融合使用,如下:

mobile <---gRPC> controller <-dubbo-> service <> ...

不知道这个跟你想的是否一致?


best regards,

Jason

> On Sep 12, 2018, at 11:16, 莫里亚提 <1249499...@qq.com> wrote:
> 
> hello:
> 
> 
>  想了解下dubbo grpc协议支持feature的进展。我的理解,要支持grpc streaming调用的特性的话,需要使用grpc stub 
> layer,但是在export, refer, invoker这些grpc组件对接
> 
> 
> 的设计上有些疑惑。有人正在做相应的事情吗?
> 
> 
> 
> 
> Best Regards,
> dp



Re: The chats must be public

2018-09-07 Thread Jason Joo
+1

Maybe we all have so much work to do, and also need time to write whole 
thoughts down.

IM will destroy it and can't go deeper and multi-thread.

best regards,

Jason

> On Sep 7, 2018, at 23:14, Huxing Zhang  wrote:
> 
> Hi,
> 
> On Fri, Sep 7, 2018 at 11:33 AM Jerrick Zhu  > wrote:
>> 
>> hi, community
>> 
>> Now, we have some chat groups, which are used to discuss about dubbo, such
>> as:
>> 
>> * Dingtalk group, nearly 30 people, committers and contributors
>> * WeChat group, contains committers, contributors, and many users
>> 
>> I think we need to public these chat groups, so that all people can join
>> us.
>> 
>> Maybe Gitter? or Slack ?
>> 
>> Or maybe we can drop the chat, just use the mailing list and github issue
>> for discuss.
> 
> I am +1 on using mailing list without any instant messages chat
> groups. My reasons:
> 
> 1. My past experience tells me that once people start to chat in chat
> groups, they never come back to mailing list, even if you ask them to
> do so.
> 2. Chat groups may be efficient, in terms of you can get an answer
> quickly, but actually in most cases it produces too much useless
> information, because it is cheap. Instead, when you are writing email
> you will have more detailed thoughts. Sometimes when I wrote down my
> initial thoughts, it turns out the initial thought is wrong.
> 3. In the long term goal, I think Dubbo community should think
> globally, if there are contributors are from a different time zone,
> then asynchronized communication will be necessary.
> 
> What I suggest:
> 
> * Stop answering question in chat groups, instead, encourage them to
> ask on mailing list or file an Github issue. I believe if the core
> contributors leave, the questions will shift.
> * Think before you ask. Do you really need an answer so urgently? If
> not, ask asynchronously.
> * If you feel difficult to ask in English, please use Chinese followed
> by English.
> 
>> 
>> Any other suggestions?
>> 
>> Sincerely.
>> 
>> jerrick
> 
> 
> 
> -- 
> Best Regards!
> Huxing



Re: The chats must be public

2018-09-06 Thread Jason Joo
hi, jerrick

in my opinion,
mailing list is better for development of incubator.

best regards,

Jason

> On Sep 7, 2018, at 11:32, Jerrick Zhu  wrote:
> 
> hi, community
> 
> Now, we have some chat groups, which are used to discuss about dubbo, such
> as:
> 
> * Dingtalk group, nearly 30 people, committers and contributors
> * WeChat group, contains committers, contributors, and many users
> 
> I think we need to public these chat groups, so that all people can join
> us.
> 
> Maybe Gitter? or Slack ?
> 
> Or maybe we can drop the chat, just use the mailing list and github issue
> for discuss.
> 
> Any other suggestions?
> 
> Sincerely.
> 
> jerrick



Re: [VOTE]: Release Apache Dubbo (Incubating) 2.6.3 [RC3]

2018-08-20 Thread Jason Joo
hi,


I suggest the next iteration of development should be on another branch such as 
"2.6.4" or "2.6.x".
We indeed should keep the pre-release branch clean (I just do some local work 
changing versions for our own nexus repo before).

best regards,

Jason

> On Aug 20, 2018, at 11:02, Jun Liu  wrote:
> 
>> The 2.6.3 tag does not agree with the source release. This is a
>> significant issue and enough for me to vote against the release. A diff
>> shows most (all?) of the pom.xml have a version of "2.6.4-SNAPSHOT" in
>> the tag but "2.6.3" in the source release.
>> 
>> I dug into this a little. At first I thought the tag / commit in the
>> vote was wrong. It is. But is isn't just that. If I go back to
>> 
>> a8be0eaaddab198ed03b0150d4db03e2b22f023f
>> 
>> things are better but:
>> a) there are still differences
>> b) the tag includes multiple commits after this point
> 
> Thanks for your feedback, Mark, I have double checked the version mismatch 
> problem and it does exist.
> 
> Despite that all files except for the pom version are the same, I agree it’s 
> still an issue that needs revoting. It will confuse developers when trying to 
> rebuild from the tag. The root problem is more about a package or upload 
> problem than a technical problem. We used maven-release-plugin to prepare for 
> the release, which has changed the pom version to ‘2.6.4-SNAPSHOT’ since RC1, 
> I remember to change to 2.6.3 before tagging in RC2 but forget to do that in 
> this round. 
> 
> I think we should build a release script to automate the whole release 
> process to avoid this kind of problems.
> 
> Best regards,
> Jun
> 
>> On 18 Aug 2018, at 05:56, Mark Thomas  wrote:
>> 
>> On 12/08/18 08:12, Jun Liu wrote:
>> 
>> 
>> 
>>> Please vote accordingly:
>>> [ ] +1 approve 
>>> [ ] +0 no opinion 
>>> [X] -1 disapprove with the reason (binding)
>> 
>> The sha512 hashes are missing the '*' marker that indicates they are
>> hashes for binary files rather than text files. Trivial issue. Can be
>> addressed in the next release.
>> 
>> I'd expect the files to be named "apache-dubbo..." not "dubbo...". Nice
>> to have (not all Apache projects use this naming convention). Something
>> to consider for the next release.
>> 
>> Consider including mvnw and mvnw.cmd in the source release so it is
>> simpler to get started with the build from a source release.
>> 
>> The 2.6.3 tag does not agree with the source release. This is a
>> significant issue and enough for me to vote against the release. A diff
>> shows most (all?) of the pom.xml have a version of "2.6.4-SNAPSHOT" in
>> the tag but "2.6.3" in the source release.
>> 
>> I dug into this a little. At first I thought the tag / commit in the
>> vote was wrong. It is. But is isn't just that. If I go back to
>> 
>> a8be0eaaddab198ed03b0150d4db03e2b22f023f
>> 
>> things are better but:
>> a) there are still differences
>> b) the tag includes multiple commits after this point
>> 
>> 
>> Mark
> 



Re: 2.6.3 compatible problem with 2.5.[9, 10]

2018-08-02 Thread Jason Joo
add the table as image:



best regards,

Jason

> On Aug 3, 2018, at 10:38, Jason Joo  wrote:
> 
> hey, everyone
> 
> Today i found a big compatible problem in our staging env.
> 
> Issue has been submitted as 
> https://github.com/apache/incubator-dubbo/issues/2174 
> <https://github.com/apache/incubator-dubbo/issues/2174>
> 
> And we can check it quickly in below table:
> 
> dubbo protocolfail
> 2.5.8 2.5.8   no
> 2.5.9 2.0.0(spec) 
> 2.0.1(impl)(why 2.0.1?)   yes
> 2.5.102.0.0(spec) 
> 2.0.1(impl)   yes
> 2.6.1 2.0.0   no
> 2.6.2 2.0.0   no
> 2.6.2 2.0.1   no
> 
> 
> Maybe the best way to solve this is to patch 2.6.3(just released) and change 
> it to "2.0.2"
> 
> Any opinion?
> 
> It's better to document the change log of protocol versions.
> 
> 
> best regards,
> 
> Jason
> 



2.6.3 compatible problem with 2.5.[9, 10]

2018-08-02 Thread Jason Joo
hey, everyone

Today i found a big compatible problem in our staging env.

Issue has been submitted as 
https://github.com/apache/incubator-dubbo/issues/2174 


And we can check it quickly in below table:

dubbo   protocolfail
2.5.8   2.5.8   no
2.5.9   2.0.0(spec) 
2.0.1(impl)(why 2.0.1?) yes
2.5.10  2.0.0(spec) 
2.0.1(impl) yes
2.6.1   2.0.0   no
2.6.2   2.0.0   no
2.6.2   2.0.1   no


Maybe the best way to solve this is to patch 2.6.3(just released) and change it 
to "2.0.2"

Any opinion?

It's better to document the change log of protocol versions.


best regards,

Jason