[
https://issues.apache.org/jira/browse/AMBARI-11529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14564707#comment-14564707
]
Hudson commented on AMBARI-11529:
---------------------------------
FAILURE: Integrated in Ambari-trunk-Commit #2747 (See
[https://builds.apache.org/job/Ambari-trunk-Commit/2747/])
AMBARI-11529. Work with json module takes too long (aonishuk) (aonishuk:
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=0224b675cbac6134f0f1bf25fb7a9d0de091e098)
* ambari-agent/src/main/python/ambari_agent/LiveStatus.py
* ambari-common/src/main/python/ambari_simplejson/README.txt
* ambari-server/src/main/python/upgradeHelper.py
*
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/alerts/alert_nodemanagers_summary.py
* ambari-agent/src/test/python/ambari_agent/TestSecurity.py
* ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py
* ambari-common/src/main/python/ambari_simplejson/decoder.py
*
ambari-common/src/main/python/resource_management/libraries/functions/curl_krb_request.py
* ambari-common/src/main/python/resource_management/libraries/script/script.py
*
ambari-common/src/main/python/resource_management/libraries/functions/security_commons.py
* ambari-common/src/main/python/ambari_simplejson/__init__.py
* ambari-agent/src/main/python/ambari_agent/alerts/metric_alert.py
* ambari-server/conf/unix/install-helper.sh
* ambari-agent/src/main/python/ambari_agent/security.py
*
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py
*
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_checkpoint_time.py
*
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py
*
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/alerts/alert_nodemanager_health.py
* ambari-agent/src/main/python/ambari_agent/ActionQueue.py
* ambari-agent/src/main/python/ambari_agent/AmbariConfig.py
* ambari-agent/conf/unix/install-helper.sh
*
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/storm_upgrade.py
* NOTICE.txt
* ambari-agent/pom.xml
* ambari-agent/src/test/python/ambari_agent/TestController.py
* ambari-common/src/main/python/resource_management/libraries/functions/jmx.py
*
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
* ambari-server/pom.xml
* ambari-agent/src/main/python/ambari_agent/PythonExecutor.py
*
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_ha_namenode_health.py
*
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py
* ambari-agent/src/main/python/ambari_agent/AlertSchedulerHandler.py
* ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
* LICENSE.txt
*
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
*
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
* ambari-common/src/main/python/resource_management/core/logger.py
*
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_webhcat_server.py
*
ambari-common/src/main/python/resource_management/libraries/functions/ranger_functions.py
*
ambari-common/src/main/python/resource_management/libraries/functions/ranger_functions_v2.py
*
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py
* ambari-agent/src/main/python/ambari_agent/Heartbeat.py
*
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py
*
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
* ambari-agent/src/test/python/ambari_agent/TestClusterConfigurationCache.py
*
ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume.py
*
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
* ambari-agent/src/test/python/resource_management/TestSecurityCommons.py
* ambari-server/src/main/resources/custom_actions/scripts/install_packages.py
*
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py
*
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/files/validateYarnComponentStatusWindows.py
* ambari-agent/src/main/python/ambari_agent/Controller.py
* ambari-common/src/main/python/resource_management/core/shell.py
* ambari-server/src/main/python/ambari_server/setupSecurity.py
*
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
*
ambari-common/src/main/python/resource_management/libraries/functions/flume_agent_helper.py
* ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
* ambari-common/src/main/python/ambari_simplejson/scanner.py
* ambari-server/src/main/python/ambari_server/serverUpgrade.py
* ambari-agent/src/main/python/ambari_agent/CommandStatusDict.py
* ambari-common/src/main/python/ambari_simplejson/encoder.py
* ambari-common/src/main/python/ambari_simplejson/_speedups.so
* ambari-agent/src/main/python/ambari_agent/ClusterConfiguration.py
* ambari-server/src/main/resources/custom_actions/scripts/ru_execute_tasks.py
*
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/params_linux.py
*
ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
> Work with json module takes too long
> ------------------------------------
>
> Key: AMBARI-11529
> URL: https://issues.apache.org/jira/browse/AMBARI-11529
> Project: Ambari
> Issue Type: Bug
> Reporter: Andrew Onischuk
> Assignee: Andrew Onischuk
> Fix For: 2.1.0
>
>
> Due to profiling below, from 420 seconds of deploy json.load and json.loads
> take 40 seconds (which even doesn't include working with json in ambari-agent
> itself, only script):
>
>
> 204 0.007 0.000 20.182 0.099 __init__.py:243(load)
> 351 0.002 0.000 17.894 0.051 __init__.py:270(loads)
>
> This is because json module for Python 2.6 is extremety slow. There are
> benchmark for it on here:
> <http://stackoverflow.com/questions/706101/python-json-decoding-performance>
> due to it simplejson is at least ~40 times faster for deserializing.
> Also another thing to do in scope of this jira is to replace inspect module
> usage with os._getframe() which doesn't use realpath function that
> intensively, taking away 10 more seconds.
>
>
> 30049/21438 1.712 0.000 10.873 0.001 posixpath.py:354(realpath)
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
