Hi Terry,
Something came to my mind that the configuration item
`max.connections.size.per.query`(Default value is 1) can impact the elapsed
time of metadata loading, can you configure it higher?
Juan Pan (Trista)
Senior DBA & PPMC of Apache ShardingSphere(Incubating)
E-mail: [email protected]
On 03/9/2020 14:38??Juan Pan<[email protected]> wrote??
Hi Terry,
Thanks for your update.
I want to make sure whether your sharding configuration contains the item,
i.e., `defaultDataSourceName`?
Our master branch has optimized the loading process of metaData for tables
positioning in `defaultDataSource`. Based on that, if your team did not
configure `defaultDataSourceName`, no significant improvement is expected.
However, we are still working on this issue to find a better way. So you can
watch ISSUE[1] and wait for our update.
Cheer,
Trista
[1] https://github.com/apache/incubator-shardingsphere/issues/4604
Juan Pan (Trista)
Senior DBA & PPMC of Apache ShardingSphere(Incubating)
E-mail: [email protected]
On 03/8/2020 20:07??Terry Zhang<[email protected]> wrote??
Dear team,
Let me update our test result for the version of branch 'master'.
The result is as following:
Disable "check.table.metadata.enabled" : 78.9s elapsed, please check
below.
"2020-03-06 19:09:18.951 [main] INFO ShardingSphere-MetaData - Meta data
loading finished, cost 78982 milliseconds."
However, when we enable "check.table.metadata.enabled", an exception was
thrown, it promoted that a bean is not initialised successfully as we are using
Spring Cloud, we will look into it when we are free later.
Based on above, we don't think the version under branch 'master' is making a
significant improvement on this.
Thanks.
Regards.
Terry Zhang
------------------ ???????? ------------------
??????: "KimmKing"<[email protected]>;
????????: 2020??3??8??(??????) ????1:57
??????: "dev"<[email protected]>;
????: Re:Re:Re:Too slow to launch the service once ShardingJDBC is adopted
Good job.
And I will follow this issue.
Once your team complete the test, we will consider a further improvement of
this issue.
?? 2020-03-06 13:21:51??"Terry Zhang" <[email protected]> ??????
>SURE, I will have team test it ASAP.&nbsp; Thanks.
>
>Best Regards
>
>Terry Zhang
>
>
>------------------ ???????? ------------------
>??????:&nbsp;"Juan Pan" <[email protected]&gt;;
>????????:&nbsp;2020??3??6??(??????) 11:44
>??????:&nbsp;"[email protected]"
<[email protected]&gt;;
>????:&nbsp;"dev" <[email protected]&gt;;
>????:&nbsp;Re:Too slow to launch the service once ShardingJDBC is
adopted
>
>
>
>Hi Terry,
>
>
>Appreciated your detailed report.
>Currently, the option `check.table.metadata.enabled` is the best way to
speed up launching. However much feedback from community
>made us consider to optimize it ASAP. Could i invite you to join us to make
it better, even just an assisted test for this new enhancement is a help.
>I will update progress on ISSSUE[1], which you can refer to for learning
more.
>
>
>Cheers,
>Trista
>
>
>
>
>[1] https://github.com/apache/incubator-shardingsphere/issues/4604
>
>
>&nbsp;Juan Pan (Trista)
>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
>Senior DBA &amp; PPMC of Apache ShardingSphere(Incubating)
>E-mail: [email protected]
>
>
>
>
>On 03/5/2020 21:04??Terry Zhang<[email protected]&gt; wrote??
>Dear ShardingSphere team,&amp;nbsp;
>
>
>We are using ShardingJDBC module in our project and running well for in
prod environment. However, we are encountering a serious issue of that once
ShardingJDBC is integrated within our project it can take us 1+ minutes to
launch the service in our dev/test/prod environment. You know it is so slow
resulting in our bad work efficiency in both dev/test and prod environment as
we have to wait for its launch every time.
>
>
>Due to demonstration above, we try to resolve this issue by ourselves, and
move forward to this project's official website to find the resolution.
Finally, we found the config item "check.table.metadata.enabled" via the web
page&amp;nbsp;
>https://shardingsphere.apache.org/document/legacy/3.x/document/en/manual/sharding-jdbc/configuration/config-yaml/.
Unfortunately, we gain a negative result after our trial.
>
>
>Below is our test result:
>---------------------------
>ShardingJDBC: 4.0.0 RC3
>JDK: 1.8
>Database: MySQL 5.7.25&amp;nbsp;
>Tables:&amp;nbsp; totally 465 tables, including 132 sharding tables
>Enable "check.table.metadata.enabled" : about 139s elapsed.
>Disable "check.table.metadata.enabled" : about 80s elapsed.
>---------------------------
>I attached the test log for your reference at the bottom of this email.
>
>
>In summary, although we set the prop to false, it still consume about 80s,
that is too low for us. I can??t confirm whether the prop is corrected, if no,
please advise which one is corrected or is there other solution? Your advice is
appreciated. Looking forward to your reply. Thanks.&amp;nbsp;
>
>
>Rgds, Terry Zhang
>
>
>------------TEST LOG ATTACHED--------
>
>
>With prop "check.table.metadata.enabled = true"
>
>
>2020-03-05 17:10:17.390 [main] INFO&amp;nbsp;
o.a.s.core.util.ConfigurationLogger - Properties
>check.table.metadata.enabled: true
>
>
>2020-03-05 17:12:09.394 [main] INFO&amp;nbsp;
o.s.c.s.PostProcessorRegistrationDelegate$BeanPostProcessorChecker - Bean
'shardingDataSource' of type
[org.apache.shardingsphere.shardingjdbc.jdbc.core.datasource.ShardingDataSource]
is not eligible for getting processed by all BeanPostProcessors (for example:
not eligible for auto-proxying)
>2020-03-05 17:12:09.402 [main] INFO&amp;nbsp;
o.s.c.s.PostProcessorRegistrationDelegate$BeanPostProcessorChecker - Bean
'org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration' of
type
[org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration$$EnhancerBySpringCGLIB$$f1ce2b62]
is not eligible for getting processed by all BeanPostProcessors (for example:
not eligible for auto-proxying)
>2020-03-05 17:12:09.439 [main] INFO&amp;nbsp;
o.s.c.s.PostProcessorRegistrationDelegate$BeanPostProcessorChecker - Bean
'spring.datasource-org.springframework.boot.autoconfigure.jdbc.DataSourceProperties'
of type [org.springframework.boot.autoconfigure.jdbc.DataSourceProperties] is
not eligible for getting processed by all BeanPostProcessors (for example: not
eligible for auto-proxying)
>2020-03-05 17:12:09.457 [main] INFO&amp;nbsp;
o.s.c.s.PostProcessorRegistrationDelegate$BeanPostProcessorChecker - Bean
'dataSourceInitializer' of type
[org.springframework.boot.autoconfigure.jdbc.DataSourceInitializer] is not
eligible for getting processed by all BeanPostProcessors (for example: not
eligible for auto-proxying)
>2020-03-05 17:12:09.467 [main] INFO&amp;nbsp;
o.s.c.s.PostProcessorRegistrationDelegate$BeanPostProcessorChecker - Bean
'org.springframework.boot.autoconfigure.transaction.TransactionAutoConfiguration'
of type
[org.springframework.boot.autoconfigure.transaction.TransactionAutoConfiguration$$EnhancerBySpringCGLIB$$a44e830c]
is not eligible for getting processed by all BeanPostProcessors (for example:
not eligible for auto-proxying)
>
>
>
>
>With prop "check.table.metadata.enabled = false"
>
>
>2020-03-05 17:12:51.295 [main] INFO&amp;nbsp;
o.a.s.core.util.ConfigurationLogger - Properties
>check.table.metadata.enabled: false
>2020-03-05 17:14:12.015 [main] INFO&amp;nbsp;
o.s.c.s.PostProcessorRegistrationDelegate$BeanPostProcessorChecker - Bean
'shardingDataSource' of type
[org.apache.shardingsphere.shardingjdbc.jdbc.core.datasource.ShardingDataSource]
is not eligible for getting processed by all BeanPostProcessors (for example:
not eligible for auto-proxying)
>2020-03-05 17:14:12.022 [main] INFO&amp;nbsp;
o.s.c.s.PostProcessorRegistrationDelegate$BeanPostProcessorChecker - Bean
'org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration' of
type
[org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration$$EnhancerBySpringCGLIB$$37874c71]
is not eligible for getting processed by all BeanPostProcessors (for example:
not eligible for auto-proxying)
>2020-03-05 17:14:12.058 [main] INFO&amp;nbsp;
o.s.c.s.PostProcessorRegistrationDelegate$BeanPostProcessorChecker - Bean
'spring.datasource-org.springframework.boot.autoconfigure.jdbc.DataSourceProperties'
of type [org.springframework.boot.autoconfigure.jdbc.DataSourceProperties] is
not eligible for getting processed by all BeanPostProcessors (for example: not
eligible for auto-proxying)
