[jira] [Commented] (OFBIZ-10514) Refactoring ContactMechWorker.get[Entity]ContactMechValueMaps
[
https://issues.apache.org/jira/browse/OFBIZ-10514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16577654#comment-16577654
]
Nicolas Malin commented on OFBIZ-10514:
---
I created a new patch with differents improvement induced by this issue
[^OFBIZ-10514.patch]
* Convert the getFacilityContactMechValueMaps like Party
* Convert getOrderContactMechValueMaps and getWorkEffortContactMechValueMaps
on one generic function
* Homogenise the way to use ContactMech and related entity on one view
ContactMechDetail and use it to PartyAndContactMech view
* Add view FacilityAndContactMech, OrderAndContactMech and
WorkEffortAndContactMech like PartyAndContactMech
* Add gunit test to control the
ContactMechWorker.get[Entity]ContactMechValueMaps
> Refactoring ContactMechWorker.get[Entity]ContactMechValueMaps
> -
>
> Key: OFBIZ-10514
> URL: https://issues.apache.org/jira/browse/OFBIZ-10514
> Project: OFBiz
> Issue Type: Improvement
> Components: party
>Reporter: Nicolas Malin
>Assignee: Nicolas Malin
>Priority: Major
> Attachments: OFBIZ-10514.patch, OFBIZ-10514.patch
>
>
> ContactMechWorker.get[Entity]ContactMechValueMaps are old historic functions
> that resolve all contact mech context related to an Entity (Party, Facility,
> Order, WorkEffort).
> The problem that they create too many db call during their execution that
> decrease OFBiz performance when the contactMech history grow.
> On customer site I realized a new adaptation (oriented and not generic at
> all) and control the performance issue with a unit test :
> {code:java}
> public void testContactMechSearch() throws Exception {
> UtilTimer timer = new UtilTimer("test contactmech", true, true);
> timer.startTimer();
> List> cmwPostalAdress =
> ContactMechWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "POSTAL_ADDRESS");
> timer.timerString("PA cmw " + cmwPostalAdress.size());
> List> cmwTelecomNumber =
> ContactMechWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "TELECOM_NUMBER");
> timer.timerString("TN cmw " + cmwTelecomNumber.size());
> List> cmwEmail =
> ContactMechWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "EMAIL_ADDRESS");
> timer.timerString("E cmw " + cmwEmail.size());
> List> epwPostalAdress =
> CustomPartyWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "POSTAL_ADDRESS", null);
> timer.timerString("PA cpw " + epwPostalAdress.size());
> List> epwTelecomNumber =
> CustomPartyWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "TELECOM_NUMBER", null);
> timer.timerString("TN cpw " + epwTelecomNumber.size());
> List> epwEmail =
> CustomPartyWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "EMAIL_ADDRESS", null);
> timer.timerString("E cpw " + epwEmail.size());
> }
> {code}
> The partyId "10375" is linked with more tha 600 contactMechs on my local site
> db. And the test excecution give :
> {code:java}
> With ContactMechWorker
> [java] [[PA cmw 2- total:,since last(Begin):33.707]] - 'test
> contactmech'
> [java] [[TN cmw 2- total:,since last(PA cmw 2):33.837]] - 'test
> contactmech'
> [java] [[E cmw 1- total:1..,since last(TN cmw 2):33.115]] - 'test
> contactmech'
> With CustomPartyWorker
> [java] [[PA epw 2- total:...,since last(E cmw 1):0.53]] - 'test
> contactmech'
> [java] [[TN epw 2- total:...,since last(PA epw 2):0.588]] - 'test
> contactmech'
> [java] [[E epw 1- total:...,since last(TN epw 2):0.631]] - 'test
> contactmech'
> {code}
> total ~100s for ContactMechWorker and ~1.5s for CustomPartyWorker
> Difference come that CustomPartyWorker works on a view entity gathering the
> data and populating GenericValue with makeValue (using the data within the
> view) instead of creating new getRelated db access.
> I will implement this idea on generic context to improve these functions
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OFBIZ-10514) Refactoring ContactMechWorker.get[Entity]ContactMechValueMaps
[
https://issues.apache.org/jira/browse/OFBIZ-10514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16577459#comment-16577459
]
Nicolas Malin commented on OFBIZ-10514:
---
I used EntityUtil.filterByDate after the query to ensure that the cache will be
use because if you create a cache result with a now on parameters, you never
reuse your cache.
But after check how the EntityQuery works with filterByDate, It use the same
logic
{code}
private List query(EntityFindOptions efo) throws
GenericEntityException {
EntityFindOptions findOptions = null;
if (efo == null) {
findOptions = makeEntityFindOptions();
} else {
findOptions = efo;
}
List result = null;
if (dynamicViewEntity == null) {
result = delegator.findList(entityName,
makeWhereCondition(useCache), fieldsToSelect, orderBy, findOptions, useCache);
} else {
try (EntityListIterator it = queryIterator()) {
result = it.getCompleteList();
}
}
if (filterByDate && useCache) {
return EntityUtil.filterByCondition(result,
this.makeDateCondition());
}
return result;
{code}
So if you use cache and you filter by date, you run the filter after the query
:)
Nice, I will update I patch
> Refactoring ContactMechWorker.get[Entity]ContactMechValueMaps
> -
>
> Key: OFBIZ-10514
> URL: https://issues.apache.org/jira/browse/OFBIZ-10514
> Project: OFBiz
> Issue Type: Improvement
> Components: party
>Reporter: Nicolas Malin
>Assignee: Nicolas Malin
>Priority: Major
> Attachments: OFBIZ-10514.patch
>
>
> ContactMechWorker.get[Entity]ContactMechValueMaps are old historic functions
> that resolve all contact mech context related to an Entity (Party, Facility,
> Order, WorkEffort).
> The problem that they create too many db call during their execution that
> decrease OFBiz performance when the contactMech history grow.
> On customer site I realized a new adaptation (oriented and not generic at
> all) and control the performance issue with a unit test :
> {code:java}
> public void testContactMechSearch() throws Exception {
> UtilTimer timer = new UtilTimer("test contactmech", true, true);
> timer.startTimer();
> List> cmwPostalAdress =
> ContactMechWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "POSTAL_ADDRESS");
> timer.timerString("PA cmw " + cmwPostalAdress.size());
> List> cmwTelecomNumber =
> ContactMechWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "TELECOM_NUMBER");
> timer.timerString("TN cmw " + cmwTelecomNumber.size());
> List> cmwEmail =
> ContactMechWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "EMAIL_ADDRESS");
> timer.timerString("E cmw " + cmwEmail.size());
> List> epwPostalAdress =
> CustomPartyWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "POSTAL_ADDRESS", null);
> timer.timerString("PA cpw " + epwPostalAdress.size());
> List> epwTelecomNumber =
> CustomPartyWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "TELECOM_NUMBER", null);
> timer.timerString("TN cpw " + epwTelecomNumber.size());
> List> epwEmail =
> CustomPartyWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "EMAIL_ADDRESS", null);
> timer.timerString("E cpw " + epwEmail.size());
> }
> {code}
> The partyId "10375" is linked with more tha 600 contactMechs on my local site
> db. And the test excecution give :
> {code:java}
> With ContactMechWorker
> [java] [[PA cmw 2- total:,since last(Begin):33.707]] - 'test
> contactmech'
> [java] [[TN cmw 2- total:,since last(PA cmw 2):33.837]] - 'test
> contactmech'
> [java] [[E cmw 1- total:1..,since last(TN cmw 2):33.115]] - 'test
> contactmech'
> With CustomPartyWorker
> [java] [[PA epw 2- total:...,since last(E cmw 1):0.53]] - 'test
> contactmech'
> [java] [[TN epw 2- total:...,since last(PA epw 2):0.588]] - 'test
> contactmech'
> [java] [[E epw 1- total:...,since last(TN epw 2):0.631]] - 'test
> contactmech'
> {code}
> total ~100s for ContactMechWorker and ~1.5s for CustomPartyWorker
> Difference come that CustomPartyWorker works on a view entity gathering the
> data and populating GenericValue with makeValue (using the data within the
> view) instead of creating new getRelated db access.
> I will implement this idea on generic context to improve these functions
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OFBIZ-10514) Refactoring ContactMechWorker.get[Entity]ContactMechValueMaps
[
https://issues.apache.org/jira/browse/OFBIZ-10514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16577323#comment-16577323
]
Mathieu Lirzin commented on OFBIZ-10514:
Hello [~soledad],
The performance difference is impressive, well done. I am quite interested by
the details of the issue and of your implementation, so here is a first
question.
In {{getPartyContactMechValueMaps}} is the use of {{EntityUtil.filterByDate}}:
{code:java}
allPartyContactMechs =
EntityQuery.use(delegator).from("PartyAndContactMech").where(EntityCondition.makeCondition(conditionList)).cache().queryList();
if (date != null) {
allPartyContactMechs = EntityUtil.filterByDate(allPartyContactMechs, date);
}
{code}
equivalent to using {{filteringByDate}} directly on the entity query?
{code:java}
allPartyContactMechs = EntityQuery.use(delegator)
.from("PartyAndContactMech")
.where(conditions)
.cache()
.filterByDate(date)
.queryList();
{code}
> Refactoring ContactMechWorker.get[Entity]ContactMechValueMaps
> -
>
> Key: OFBIZ-10514
> URL: https://issues.apache.org/jira/browse/OFBIZ-10514
> Project: OFBiz
> Issue Type: Improvement
> Components: party
>Reporter: Nicolas Malin
>Assignee: Nicolas Malin
>Priority: Major
> Attachments: OFBIZ-10514.patch
>
>
> ContactMechWorker.get[Entity]ContactMechValueMaps are old historic functions
> that resolve all contact mech context related to an Entity (Party, Facility,
> Order, WorkEffort).
> The problem that they create too many db call during their execution that
> decrease OFBiz performance when the contactMech history grow.
> On customer site I realized a new adaptation (oriented and not generic at
> all) and control the performance issue with a unit test :
> {code:java}
> public void testContactMechSearch() throws Exception {
> UtilTimer timer = new UtilTimer("test contactmech", true, true);
> timer.startTimer();
> List> cmwPostalAdress =
> ContactMechWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "POSTAL_ADDRESS");
> timer.timerString("PA cmw " + cmwPostalAdress.size());
> List> cmwTelecomNumber =
> ContactMechWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "TELECOM_NUMBER");
> timer.timerString("TN cmw " + cmwTelecomNumber.size());
> List> cmwEmail =
> ContactMechWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "EMAIL_ADDRESS");
> timer.timerString("E cmw " + cmwEmail.size());
> List> epwPostalAdress =
> CustomPartyWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "POSTAL_ADDRESS", null);
> timer.timerString("PA cpw " + epwPostalAdress.size());
> List> epwTelecomNumber =
> CustomPartyWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "TELECOM_NUMBER", null);
> timer.timerString("TN cpw " + epwTelecomNumber.size());
> List> epwEmail =
> CustomPartyWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "EMAIL_ADDRESS", null);
> timer.timerString("E cpw " + epwEmail.size());
> }
> {code}
> The partyId "10375" is linked with more tha 600 contactMechs on my local site
> db. And the test excecution give :
> {code:java}
> With ContactMechWorker
> [java] [[PA cmw 2- total:,since last(Begin):33.707]] - 'test
> contactmech'
> [java] [[TN cmw 2- total:,since last(PA cmw 2):33.837]] - 'test
> contactmech'
> [java] [[E cmw 1- total:1..,since last(TN cmw 2):33.115]] - 'test
> contactmech'
> With CustomPartyWorker
> [java] [[PA epw 2- total:...,since last(E cmw 1):0.53]] - 'test
> contactmech'
> [java] [[TN epw 2- total:...,since last(PA epw 2):0.588]] - 'test
> contactmech'
> [java] [[E epw 1- total:...,since last(TN epw 2):0.631]] - 'test
> contactmech'
> {code}
> total ~100s for ContactMechWorker and ~1.5s for CustomPartyWorker
> Difference come that CustomPartyWorker works on a view entity gathering the
> data and populating GenericValue with makeValue (using the data within the
> view) instead of creating new getRelated db access.
> I will implement this idea on generic context to improve these functions
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OFBIZ-10514) Refactoring ContactMechWorker.get[Entity]ContactMechValueMaps
[
https://issues.apache.org/jira/browse/OFBIZ-10514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16576859#comment-16576859
]
Nicolas Malin commented on OFBIZ-10514:
---
Optimisation example on ContactMechWorker.getPartyContactMechValueMaps
[^OFBIZ-10514.patch]
I will continue to analyse if this introduce no regression and I continue on
others entities
> Refactoring ContactMechWorker.get[Entity]ContactMechValueMaps
> -
>
> Key: OFBIZ-10514
> URL: https://issues.apache.org/jira/browse/OFBIZ-10514
> Project: OFBiz
> Issue Type: Improvement
> Components: party
>Reporter: Nicolas Malin
>Assignee: Nicolas Malin
>Priority: Major
> Attachments: OFBIZ-10514.patch
>
>
> ContactMechWorker.get[Entity]ContactMechValueMaps are old historic functions
> that resolve all contact mech context related to an Entity (Party, Facility,
> Order, WorkEffort).
> The problem that they create too many db call during their execution that
> decrease OFBiz performance when the contactMech history grow.
> On customer site I realized a new adaptation (oriented and not generic at
> all) and control the performance issue with a unit test :
> {code:java}
> public void testContactMechSearch() throws Exception {
> UtilTimer timer = new UtilTimer("test contactmech", true, true);
> timer.startTimer();
> List> cmwPostalAdress =
> ContactMechWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "POSTAL_ADDRESS");
> timer.timerString("PA cmw " + cmwPostalAdress.size());
> List> cmwTelecomNumber =
> ContactMechWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "TELECOM_NUMBER");
> timer.timerString("TN cmw " + cmwTelecomNumber.size());
> List> cmwEmail =
> ContactMechWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "EMAIL_ADDRESS");
> timer.timerString("E cmw " + cmwEmail.size());
> List> epwPostalAdress =
> CustomPartyWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "POSTAL_ADDRESS", null);
> timer.timerString("PA cpw " + epwPostalAdress.size());
> List> epwTelecomNumber =
> CustomPartyWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "TELECOM_NUMBER", null);
> timer.timerString("TN cpw " + epwTelecomNumber.size());
> List> epwEmail =
> CustomPartyWorker.getPartyContactMechValueMaps(delegator, "10375", false,
> "EMAIL_ADDRESS", null);
> timer.timerString("E cpw " + epwEmail.size());
> }
> {code}
> The partyId "10375" is linked with more tha 600 contactMechs on my local site
> db. And the test excecution give :
> {code:java}
> With ContactMechWorker
> [java] [[PA cmw 2- total:,since last(Begin):33.707]] - 'test
> contactmech'
> [java] [[TN cmw 2- total:,since last(PA cmw 2):33.837]] - 'test
> contactmech'
> [java] [[E cmw 1- total:1..,since last(TN cmw 2):33.115]] - 'test
> contactmech'
> With CustomPartyWorker
> [java] [[PA epw 2- total:...,since last(E cmw 1):0.53]] - 'test
> contactmech'
> [java] [[TN epw 2- total:...,since last(PA epw 2):0.588]] - 'test
> contactmech'
> [java] [[E epw 1- total:...,since last(TN epw 2):0.631]] - 'test
> contactmech'
> {code}
> total ~100s for ContactMechWorker and ~1.5s for CustomPartyWorker
> Difference come that CustomPartyWorker works on a view entity gathering the
> data and populating GenericValue with makeValue (using the data within the
> view) instead of creating new getRelated db access.
> I will implement this idea on generic context to improve these functions
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
