[ 
https://issues.apache.org/jira/browse/YARN-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591294#comment-14591294
 ] 

Ray Chiang commented on YARN-3069:
----------------------------------

Most of the manual verification were in the following categories:
- Hardcoded value
- Not using DEFAULT_FOO for FOO member variable naming convention
- No default value at all
- Variable is used indirectly

Manual verification specifics:

CLIENT_FAILOVER_MAX_ATTEMPTS
- Hardcoded default to -1 in RMProxy

CLIENT_FAILOVER_SLEEPTIME_BASE_MS
CLIENT_FAILOVER_SLEEPTIME_MAX_MS
- Defaults to RESOURCEMANAGER_CONNECT_RETRY_INTERVAL_MS or 
  DEFAULT_RESOURCEMANAGER_CONNECT_RETRY_INTERVAL_MS

DEBUG_NM_DELETE_DELAY_SEC
- Hardcoded default to 0 in DeletionService

FS_NODE_LABELS_STORE_ROOT_DIR
- Defaults to FileSystemNodeLabelsStore#getDefaultFSNodeLabelsRootDir() return 
value

FS_RM_STATE_STORE_URI
- No default value anywhere

IS_MINI_YARN_CLUSTER
- Hardcoded to false in Client, MRApps, ResourceManager

NM_AUX_SERVICES
- No default value anywhere.  Maybe whatever 
Configuration#getStringCollection() returns.

NM_BIND_HOST
- No default value anywhere

NM_CONTAINER_EXECUTOR
- Hardcoded to DefaultContainerExecutor.class in NodeManager

NM_CONTAINER_LOCALIZER_JAVA_OPTS_KEY
- Defaults to YarnConfiguration.NM_CONTAINER_LOCALIZER_JAVA_OPTS_DEFAULT in 
ContainerLocalizer

NM_CONTAINER_MON_PROCESS_TREE
NM_CONTAINER_MON_RESOURCE_CALCULATOR
- Hardcoded to null in ContainersMonitorImpl

NM_DISK_HEALTH_CHECK_ENABLE
- Hardcoded to true in LocalDirsHanderService

NM_DOCKER_CONTAINER_EXECUTOR_EXEC_NAME
- Defaults to unconventional name 
YarnConfiguration.NM_DEFAULT_DOCKER_CONTAINER_EXECUTOR_EXEC_NAME

NM_DOCKER_CONTAINER_EXECUTOR_IMAGE_NAME
- No default value anywhere

NM_HEALTH_CHECK_SCRIPT_OPTS
- Defaults to empty String array in NodeManager

NM_HEALTH_CHECK_SCRIPT_PATH
- No default value anywhere

NM_KEYTAB
- Defaults to YarnConfiguration.NM_PRINCIPAL

NM_LINUX_CONTAINER_CGROUPS_HIERARCHY
- Hardcoded to "/hadoop-yarn" in CGroupsHandlerImpl and 
CgroupsLCEResourcesHandler

NM_LINUX_CONTAINER_CGROUPS_MOUNT
- Hardcoded to false in CGroupsHandlerImpl and CgroupsLCEResourcesHandler

NM_LINUX_CONTAINER_CGROUPS_MOUNT_PATH
- Hardcoded to null in CGroupsHandlerImpl and CgroupsLCEResourcesHandler

NM_LINUX_CONTAINER_EXECUTOR_PATH
- Defaults to internal variable defaultPath (which looks to be based off 
HADOOP_YARN_HOME environment)

NM_LINUX_CONTAINER_GROUP
- Not used anywhere

NM_LINUX_CONTAINER_RESOURCES_HANDLER
- Hardcoded to DefaultLCEResourcesHandler.class in LinuxContainerExecutor

NM_LOG_DELETION_THREADS_COUNT
- Defaults to unconventional name 
YarnConfiguration.DEFAULT_NM_LOG_DELETE_THREAD_COUNT

NM_NONSECURE_MODE_LOCAL_USER_KEY
- Defaults to unconventional name 
YarnConfiguration.DEFAULT_NM_NONSECURE_MODE_LOCAL_USER

NM_NONSECURE_MODE_USER_PATTERN_KEY
- Defaults to unconventional name 
YarnConfiguration.DEFAULT_NM_NONSECURE_MODE_USER_PATTERN

NM_PRINCIPAL
- Is the default value for YarnConfiguration.NM_KEYTAB

NM_RECOVERY_DIR
- No default value anywhere

NM_SYSTEM_RESERVED_PMEM_MB
- Hardcoded to -1 in NodeManagerHardwareUtils

NM_WEBAPP_SPNEGO_KEYTAB_FILE_KEY
NM_WEBAPP_SPNEGO_USER_NAME_KEY
- No default value anywhere

NM_WINDOWS_SECURE_CONTAINER_GROUP
- No default value anywhere

PROXY_KEYTAB
PROXY_PRINCIPAL
- No default value anywhere

RECOVERY_ENABLED
- Defaults to YarnConfiguration.DEFAULT_NM_NONSECURE_MODE_USER_PATTERN in 
ResourceManager

RM_BIND_HOST
- No default value anywhere

RM_CLUSTER_ID
- No default value anywhere

RM_DELEGATION_KEY_UPDATE_INTERVAL_KEY
- Defaults to YarnConfiguration.RM_DELEGATION_KEY_UPDATE_INTERVAL_DEFAULT in 
RMSecretManagerService

RM_DELEGATION_TOKEN_MAX_LIFETIME_KEY
- Defaults to YarnConfiguration.RM_DELEGATION_TOKEN_MAX_LIFETIME_DEFAULT in 
RMSecretManagerService

RM_DELEGATION_TOKEN_RENEW_INTERVAL_KEY
- Defaults to YarnConfiguration.RM_DELEGATION_TOKEN_RENEW_INTERVAL_DEFAULT in 
RMSecretManagerService

RM_HA_ID
- Defaults to values from RM_HA_IDS

RM_HA_IDS
- No default value, but gets validation in HAUtil#verifyAndSetRMHAIdsList()

RM_HOSTNAME
- Defaults to internal variable RMId in HAUtils

RM_KEYTAB
- Defaults to YarnConfiguration.RM_PRINCIPAL

RM_LEVELDB_STORE_PATH
- No default value anywhere

RM_PRINCIPAL
- Default value for RM_KEYTAB

RM_PROXY_USER_PRIVILEGES_ENABLED
- Defaults to YarnConfiguration.DEFAULT_RM_PROXY_USER_PRIVILEGES_ENABLED.  
Needs final keyword added.

RM_RESERVATION_SYSTEM_CLASS
- Defaults to AbstractReservationSystem#getDefaultReservationSystem(scheduler)

RM_RESERVATION_SYSTEM_PLAN_FOLLOWER
- Defaults to AbstractReservationSystem.getDefaultPlanFollower()

RM_SCHEDULER_INCLUDE_PORT_IN_NODE_NAME
- Unconventional default 
YarnConfiguration.DEFAULT_RM_SCHEDULER_USE_PORT_FOR_NODE_NAME

RM_SCHEDULER_MONITOR_POLICIES
- Defaults to an SchedulingEditPolicy.class as an Interface

RM_STORE
- Hardcoded to MemoryRMStateStore.class in RMStateStoreFactory

RM_WEBAPP_SPNEGO_KEYTAB_FILE_KEY
RM_WEBAPP_SPNEGO_USER_NAME_KEY
- No default value anywhere

RM_ZK_ADDRESS
- No default value anywhere

RM_ZK_AUTH
- No default value anywhere

RM_ZK_NUM_RETRIES
- Unconventional default YarnConfiguration.DEFAULT_ZK_RM_NUM_RETRIES

YARN_APPLICATION_CLASSPATH
- Defaults to String array 
YarnConfiguration.DEFAULT_YARN_CROSS_PLATFORM_APPLICATION_CLASSPATH

YARN_AUTHORIZATION_PROVIDER
- Hardcoded to ConfiguredYarnAuthorizer.class in YarnAuthorizationProvider

YARN_HTTP_POLICY_KEY
- Unconventional default of YarnConfiguration.YARN_HTTP_POLICY_DEFAULT

YARN_INTERMEDIATE_DATA_ENCRYPTION
- Non-primitive, non-String default in 
YarnConfiguration.YARN_INTERMEDIATE_DATA_ENCRYPTION

YARN_LOG_SERVER_URL
- No default value anywhere

YARN_SECURITY_SERVICE_AUTHORIZATION_APPLICATIONHISTORY_PROTOCOL
- Hardcoded to ApplicationMasterProtocolPB.class in RMPolicyProvider

YARN_TRACKING_URL_GENERATOR
- Hardcoded to TrackingUriPlugin.class in WebAppProxyServlet

ZK_RM_STATE_STORE_ROOT_NODE_ACL
- Passed into HAUtil.getConfValueForRMInstance


> Document missing properties in yarn-default.xml
> -----------------------------------------------
>
>                 Key: YARN-3069
>                 URL: https://issues.apache.org/jira/browse/YARN-3069
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: documentation
>            Reporter: Ray Chiang
>            Assignee: Ray Chiang
>              Labels: BB2015-05-TBR, supportability
>         Attachments: YARN-3069.001.patch, YARN-3069.002.patch, 
> YARN-3069.003.patch, YARN-3069.004.patch, YARN-3069.005.patch, 
> YARN-3069.006.patch, YARN-3069.007.patch, YARN-3069.008.patch, 
> YARN-3069.009.patch, YARN-3069.010.patch, YARN-3069.011.patch
>
>
> The following properties are currently not defined in yarn-default.xml.  
> These properties should either be
>   A) documented in yarn-default.xml OR
>   B)  listed as an exception (with comments, e.g. for internal use) in the 
> TestYarnConfigurationFields unit test
> Any comments for any of the properties below are welcome.
>   org.apache.hadoop.yarn.server.sharedcachemanager.RemoteAppChecker
>   org.apache.hadoop.yarn.server.sharedcachemanager.store.InMemorySCMStore
>   security.applicationhistory.protocol.acl
>   yarn.app.container.log.backups
>   yarn.app.container.log.dir
>   yarn.app.container.log.filesize
>   yarn.client.app-submission.poll-interval
>   yarn.client.application-client-protocol.poll-timeout-ms
>   yarn.is.minicluster
>   yarn.log.server.url
>   yarn.minicluster.control-resource-monitoring
>   yarn.minicluster.fixed.ports
>   yarn.minicluster.use-rpc
>   yarn.node-labels.fs-store.retry-policy-spec
>   yarn.node-labels.fs-store.root-dir
>   yarn.node-labels.manager-class
>   yarn.nodemanager.container-executor.os.sched.priority.adjustment
>   yarn.nodemanager.container-monitor.process-tree.class
>   yarn.nodemanager.disk-health-checker.enable
>   yarn.nodemanager.docker-container-executor.image-name
>   yarn.nodemanager.linux-container-executor.cgroups.delete-timeout-ms
>   yarn.nodemanager.linux-container-executor.group
>   yarn.nodemanager.log.deletion-threads-count
>   yarn.nodemanager.user-home-dir
>   yarn.nodemanager.webapp.https.address
>   yarn.nodemanager.webapp.spnego-keytab-file
>   yarn.nodemanager.webapp.spnego-principal
>   yarn.nodemanager.windows-secure-container-executor.group
>   yarn.resourcemanager.configuration.file-system-based-store
>   yarn.resourcemanager.delegation-token-renewer.thread-count
>   yarn.resourcemanager.delegation.key.update-interval
>   yarn.resourcemanager.delegation.token.max-lifetime
>   yarn.resourcemanager.delegation.token.renew-interval
>   yarn.resourcemanager.history-writer.multi-threaded-dispatcher.pool-size
>   yarn.resourcemanager.metrics.runtime.buckets
>   yarn.resourcemanager.nm-tokens.master-key-rolling-interval-secs
>   yarn.resourcemanager.reservation-system.class
>   yarn.resourcemanager.reservation-system.enable
>   yarn.resourcemanager.reservation-system.plan.follower
>   yarn.resourcemanager.reservation-system.planfollower.time-step
>   yarn.resourcemanager.rm.container-allocation.expiry-interval-ms
>   yarn.resourcemanager.webapp.spnego-keytab-file
>   yarn.resourcemanager.webapp.spnego-principal
>   yarn.scheduler.include-port-in-node-name
>   yarn.timeline-service.delegation.key.update-interval
>   yarn.timeline-service.delegation.token.max-lifetime
>   yarn.timeline-service.delegation.token.renew-interval
>   yarn.timeline-service.generic-application-history.enabled
>   
> yarn.timeline-service.generic-application-history.fs-history-store.compression-type
>   yarn.timeline-service.generic-application-history.fs-history-store.uri
>   yarn.timeline-service.generic-application-history.store-class
>   yarn.timeline-service.http-cross-origin.enabled
>   yarn.tracking.url.generator



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to