Change in vdsm[master]: api: Speed up schema loading

2016-10-07 Thread automation 
gerrit-hooks has posted comments on this change.

Change subject: api: Speed up schema loading
..


Patch Set 2:

* #1208141::Update tracker: OK
* #1381899::Update tracker: OK
* #65230::Update tracker: OK
* Set MODIFIED::bug 1381899#1381899::IGNORE, skipping for branch 'master'

-- 
To view, visit https://gerrit.ovirt.org/65230
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer 
Gerrit-Reviewer: Dan Kenigsberg 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer 
Gerrit-Reviewer: Piotr Kliczewski 
Gerrit-Reviewer: Simone Tiraboschi 
Gerrit-Reviewer: gerrit-hooks 
Gerrit-HasComments: No
___
vdsm-patches mailing list -- vdsm-patches@lists.fedorahosted.org
To unsubscribe send an email to vdsm-patches-le...@lists.fedorahosted.org

Change in vdsm[master]: api: Speed up schema loading

2016-10-07 Thread danken 
Dan Kenigsberg has submitted this change and it was merged.

Change subject: api: Speed up schema loading
..


api: Speed up schema loading

Use yaml.CLoader for 13x speedup in vdsm schema loading.

Here is a comparison with other formats:

$ python bench.py

  yaml: 1.053720 seconds per load
 cyaml: 0.086472 seconds per load
  json: 0.049521 seconds per load
pickle: 0.001516 seconds per load

$ python3 bench.py

  yaml: 1.340607 seconds per load
 cyaml: 0.100010 seconds per load
  json: 0.026627 seconds per load
pickle: 0.002153 seconds per load

$ grep 'model name' /proc/cpuinfo | head -n1
model name  : Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz

Clearly the best format is pickle, 700X times faster on python 2 and
638X times faster on python 3. However loading the schema in 100
milliseconds seems to be good enough for now.

The benchmark script:
https://bugzilla.redhat.com/attachment.cgi?id=1208141

Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Bug-Url: https://bugzilla.redhat.com/1381899
Signed-off-by: Nir Soffer 
Reviewed-on: https://gerrit.ovirt.org/65230
Reviewed-by: Simone Tiraboschi 
Continuous-Integration: Jenkins CI
Reviewed-by: Piotr Kliczewski 
Tested-by: Simone Tiraboschi 
---
M lib/api/vdsmapi.py
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Piotr Kliczewski: Looks good to me, approved
  Nir Soffer: Verified
  Simone Tiraboschi: Verified; Looks good to me, but someone else must approve
  Jenkins CI: Passed CI tests



-- 
To view, visit https://gerrit.ovirt.org/65230
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer 
Gerrit-Reviewer: Dan Kenigsberg 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer 
Gerrit-Reviewer: Piotr Kliczewski 
Gerrit-Reviewer: Simone Tiraboschi 
Gerrit-Reviewer: gerrit-hooks 
___
vdsm-patches mailing list -- vdsm-patches@lists.fedorahosted.org
To unsubscribe send an email to vdsm-patches-le...@lists.fedorahosted.org

Change in vdsm[master]: api: Speed up schema loading

2016-10-07 Thread stirabos 
Simone Tiraboschi has posted comments on this change.

Change subject: api: Speed up schema loading
..


Patch Set 1: Verified+1

Verified with ovirt-hosted-engine-ha and ovirt-hosted-engine-setup

-- 
To view, visit https://gerrit.ovirt.org/65230
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer 
Gerrit-Reviewer: Adam Litke 
Gerrit-Reviewer: Dan Kenigsberg 
Gerrit-Reviewer: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer 
Gerrit-Reviewer: Piotr Kliczewski 
Gerrit-Reviewer: Simone Tiraboschi 
Gerrit-Reviewer: gerrit-hooks 
Gerrit-HasComments: No
___
vdsm-patches mailing list -- vdsm-patches@lists.fedorahosted.org
To unsubscribe send an email to vdsm-patches-le...@lists.fedorahosted.org

Change in vdsm[master]: api: Speed up schema loading

2016-10-07 Thread nsoffer 
Nir Soffer has posted comments on this change.

Change subject: api: Speed up schema loading
..


Patch Set 1: Verified+1

Verified by the tests.

-- 
To view, visit https://gerrit.ovirt.org/65230
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer 
Gerrit-Reviewer: Adam Litke 
Gerrit-Reviewer: Dan Kenigsberg 
Gerrit-Reviewer: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer 
Gerrit-Reviewer: Piotr Kliczewski 
Gerrit-Reviewer: Simone Tiraboschi 
Gerrit-Reviewer: gerrit-hooks 
Gerrit-HasComments: No
___
vdsm-patches mailing list -- vdsm-patches@lists.fedorahosted.org
To unsubscribe send an email to vdsm-patches-le...@lists.fedorahosted.org

Change in vdsm[master]: api: Speed up schema loading

2016-10-07 Thread piotr . kliczewski 
Piotr Kliczewski has posted comments on this change.

Change subject: api: Speed up schema loading
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://gerrit.ovirt.org/65230
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer 
Gerrit-Reviewer: Adam Litke 
Gerrit-Reviewer: Dan Kenigsberg 
Gerrit-Reviewer: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Piotr Kliczewski 
Gerrit-Reviewer: Simone Tiraboschi 
Gerrit-Reviewer: gerrit-hooks 
Gerrit-HasComments: No
___
vdsm-patches mailing list -- vdsm-patches@lists.fedorahosted.org
To unsubscribe send an email to vdsm-patches-le...@lists.fedorahosted.org

Change in vdsm[master]: api: Speed up schema loading

2016-10-07 Thread stirabos 
Simone Tiraboschi has posted comments on this change.

Change subject: api: Speed up schema loading
..


Patch Set 1: Code-Review+1

-- 
To view, visit https://gerrit.ovirt.org/65230
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer 
Gerrit-Reviewer: Adam Litke 
Gerrit-Reviewer: Dan Kenigsberg 
Gerrit-Reviewer: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Piotr Kliczewski 
Gerrit-Reviewer: Simone Tiraboschi 
Gerrit-Reviewer: gerrit-hooks 
Gerrit-HasComments: No
___
vdsm-patches mailing list -- vdsm-patches@lists.fedorahosted.org
To unsubscribe send an email to vdsm-patches-le...@lists.fedorahosted.org

Change in vdsm[master]: api: Speed up schema loading

2016-10-07 Thread automation 
gerrit-hooks has posted comments on this change.

Change subject: api: Speed up schema loading
..


Patch Set 1:

* #1208141::Update tracker: OK
* #1381899::Update tracker: OK
* Check Bug-Url::OK
* Check Public Bug::#1381899::OK, public bug
* Check Product::IGNORE, not relevant for branch: master
* Check TM::SKIP, not in a monitored branch (ovirt-3.6 ovirt-4.0)
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 
'ovirt-4.0'])

-- 
To view, visit https://gerrit.ovirt.org/65230
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: gerrit-hooks 
Gerrit-HasComments: No
___
vdsm-patches mailing list -- vdsm-patches@lists.fedorahosted.org
To unsubscribe send an email to vdsm-patches-le...@lists.fedorahosted.org

Change in vdsm[master]: api: Speed up schema loading

2016-10-07 Thread nsoffer 
Nir Soffer has uploaded a new change for review.

Change subject: api: Speed up schema loading
..

api: Speed up schema loading

Use yaml.CLoader for 13x speedup in vdsm schema loading.

Here is a comparison with other formats:

$ python bench.py

  yaml: 1.053720 seconds per load
 cyaml: 0.086472 seconds per load
  json: 0.049521 seconds per load
pickle: 0.001516 seconds per load

$ python3 bench.py

  yaml: 1.340607 seconds per load
 cyaml: 0.100010 seconds per load
  json: 0.026627 seconds per load
pickle: 0.002153 seconds per load

$ grep 'model name' /proc/cpuinfo | head -n1
model name  : Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz

Clearly the best format is pickle, 700X times faster on python 2 and
638X times faster on python 3. However loading the schema in 100
milliseconds seems to be good enough for now.

The benchmark script:
https://bugzilla.redhat.com/attachment.cgi?id=1208141

Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Bug-Url: https://bugzilla.redhat.com/1381899
Signed-off-by: Nir Soffer 
---
M lib/api/vdsmapi.py
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/30/65230/1

diff --git a/lib/api/vdsmapi.py b/lib/api/vdsmapi.py
index 4f79efb..ced56dd 100755
--- a/lib/api/vdsmapi.py
+++ b/lib/api/vdsmapi.py
@@ -125,7 +125,7 @@
 try:
 for path in paths:
 with open(path) as f:
-loaded_schema = yaml.load(f)
+loaded_schema = yaml.load(f, Loader=yaml.CLoader)
 
 types = loaded_schema.pop('types')
 self._types.update(types)


-- 
To view, visit https://gerrit.ovirt.org/65230
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer 
___
vdsm-patches mailing list -- vdsm-patches@lists.fedorahosted.org
To unsubscribe send an email to vdsm-patches-le...@lists.fedorahosted.org