Repository: lens Updated Branches: refs/heads/master 0cce226f7 -> 078555c1a
http://git-wip-us.apache.org/repos/asf/lens/blob/078555c1/src/site/apt/admin/config.apt ---------------------------------------------------------------------- diff --git a/src/site/apt/admin/config.apt b/src/site/apt/admin/config.apt index 21b23f9..f89caab 100644 --- a/src/site/apt/admin/config.apt +++ b/src/site/apt/admin/config.apt @@ -44,245 +44,249 @@ Lens server configuration *--+--+---+--+ |10|hive.server2.logging.operation.enabled|false|Disable the operation logging, since there are no execution operation happening in lens session. Keeping it true might conflict with Hive Server if both are running on the same machine causing disappearance of the operation log directory.| *--+--+---+--+ -|11|lens.query.current.time.millis|0|Query current time in millis. This is used to resolve 'now'. If value is set to zero, 'now' is resolved to current value| +|11|lens.cube.metastore.completeness.checker.class|org.apache.lens.server.api.metastore.DefaultChecker|The class that implements the DataCompletenessChecker Interface. This will take effect if the flag "lens.cube.metastore.enable.datacompleteness.check" is set.| *--+--+---+--+ -|12|lens.server.alarm.service.impl|org.apache.lens.server.scheduler.AlarmService|Implementation class for alarm service| +|12|lens.cube.metastore.enable.datacompleteness.check|false|This property is to enable Data Completeness Checks while resolving partitions.| *--+--+---+--+ -|13|lens.server.appevent.ws.listener.impl|org.apache.lens.server.LensApplicationListener|Implementation class for Lens Application Listener| +|13|lens.query.current.time.millis|0|Query current time in millis. This is used to resolve 'now'. If value is set to zero, 'now' is resolved to current value| *--+--+---+--+ -|14|lens.server.base.url|http://0.0.0.0:9999/lensapi/|The base url for the lens server| +|14|lens.server.alarm.service.impl|org.apache.lens.server.scheduler.AlarmService|Implementation class for alarm service| *--+--+---+--+ -|15|lens.server.consistentState.ws.filter.impl|org.apache.lens.server.ConsistentStateFilter|Implementation class for ConsistentState Filter| +|15|lens.server.appevent.ws.listener.impl|org.apache.lens.server.LensApplicationListener|Implementation class for Lens Application Listener| *--+--+---+--+ -|16|lens.server.database.resource.dir|/tmp/lens/resources|Top level directory to store database specific static resources which are added to each database. To add a static resource for a DB, create a directory with the same name as that of the DB under this directory, and place the resources inside the DB directory. Any subdirectories under the DB directory and the resources under them will be ignored| +|16|lens.server.base.url|http://0.0.0.0:9999/lensapi/|The base url for the lens server| *--+--+---+--+ -|17|lens.server.db.driver.name|org.hsqldb.jdbcDriver|Database driver for database where Finished queries have to be stored. Also used for database based user config loaders| +|17|lens.server.consistentState.ws.filter.impl|org.apache.lens.server.ConsistentStateFilter|Implementation class for ConsistentState Filter| *--+--+---+--+ -|18|lens.server.db.jdbc.pass| |JDBC Password for Finished queries table. Also used for database based user config loaders| +|18|lens.server.database.resource.dir|/tmp/lens/resources|Top level directory to store database specific static resources which are added to each database. To add a static resource for a DB, create a directory with the same name as that of the DB under this directory, and place the resources inside the DB directory. Any subdirectories under the DB directory and the resources under them will be ignored| *--+--+---+--+ -|19|lens.server.db.jdbc.url|jdbc:hsqldb:/tmp/lensserver/queries.db|JDBC URL where the database for storing finished queries is located. Also used for database based user config loaders| +|19|lens.server.db.driver.name|org.hsqldb.jdbcDriver|Database driver for database where Finished queries have to be stored. Also used for database based user config loaders| *--+--+---+--+ -|20|lens.server.db.jdbc.user|SA|JDBC User for Finished queries table. Also used for database based user config loaders| +|20|lens.server.db.jdbc.pass| |JDBC Password for Finished queries table. Also used for database based user config loaders| *--+--+---+--+ -|21|lens.server.db.validation.query|select 1 from INFORMATION_SCHEMA.SYSTEM_USERS|BasicDatasource has a config parameter called validationQuery. This query is run before passing the Datasource to the application to verify if connection is valid.\ | +|21|lens.server.db.jdbc.url|jdbc:hsqldb:/tmp/lensserver/queries.db|JDBC URL where the database for storing finished queries is located. Also used for database based user config loaders| +*--+--+---+--+ +|22|lens.server.db.jdbc.user|SA|JDBC User for Finished queries table. Also used for database based user config loaders| +*--+--+---+--+ +|23|lens.server.db.validation.query|select 1 from INFORMATION_SCHEMA.SYSTEM_USERS|BasicDatasource has a config parameter called validationQuery. This query is run before passing the Datasource to the application to verify if connection is valid.\ | | | | |{{{http://commons.apache.org/proper/commons-dbcp/configuration.html}dbcp configuration}} \ | | | | |{{{http://stackoverflow.com/questions/10684244/dbcp-validationquery-for-different-databases}validationquery for different databases}} \ | | | | | | *--+--+---+--+ -|22|lens.server.domain|company.com|Server domain. This will be used in authentication and Sending emails to users. For the server, the username will be without domain and domain will come through conf.| +|24|lens.server.domain|company.com|Server domain. This will be used in authentication and Sending emails to users. For the server, the username will be without domain and domain will come through conf.| *--+--+---+--+ -|23|lens.server.driver.selector.class|org.apache.lens.server.api.driver.MinQueryCostSelector|Class for selecting best driver given the query context| +|25|lens.server.driver.selector.class|org.apache.lens.server.api.driver.MinQueryCostSelector|Class for selecting best driver given the query context| *--+--+---+--+ -|24|lens.server.drivers|hive:org.apache.lens.driver.hive.HiveDriver|Drivers enabled for this lens server instance| +|26|lens.server.drivers|hive:org.apache.lens.driver.hive.HiveDriver|Drivers enabled for this lens server instance| *--+--+---+--+ -|25|lens.server.duplicate.query.allowed|false|If this is false and same query is submitted by a user in the same session and with the same configuration while earlier query is not completed then lens server will return the handle of the previous query| +|27|lens.server.duplicate.query.allowed|false|If this is false and same query is submitted by a user in the same session and with the same configuration while earlier query is not completed then lens server will return the handle of the previous query| *--+--+---+--+ -|26|lens.server.enable.console.metrics|false|Enable metrics to be reported on console| +|28|lens.server.enable.console.metrics|false|Enable metrics to be reported on console| *--+--+---+--+ -|27|lens.server.enable.csv.metrics|false|Enable metrics to be reported as csv files in a directory| +|29|lens.server.enable.csv.metrics|false|Enable metrics to be reported as csv files in a directory| *--+--+---+--+ -|28|lens.server.enable.ganglia.metrics|false|Enable metrics to be reported on ganglia| +|30|lens.server.enable.ganglia.metrics|false|Enable metrics to be reported on ganglia| *--+--+---+--+ -|29|lens.server.enable.graphite.metrics|false|Enable metrics to be reported on graphite| +|31|lens.server.enable.graphite.metrics|false|Enable metrics to be reported on graphite| *--+--+---+--+ -|30|lens.server.enable.resource.method.metering|false|Whether to Enable metering for all resource methods.| +|32|lens.server.enable.resource.method.metering|false|Whether to Enable metering for all resource methods.| *--+--+---+--+ -|31|lens.server.estimate.pool.keepalive.millis|60000|Thread keep alive time in milliseconds for the estimate thread pool. If there are no estimate requests for this period,then cached threads are released from the pool.| +|33|lens.server.estimate.pool.keepalive.millis|60000|Thread keep alive time in milliseconds for the estimate thread pool. If there are no estimate requests for this period,then cached threads are released from the pool.| *--+--+---+--+ -|32|lens.server.estimate.pool.max.threads|100|Maximum number of threads in the estimate thread pool| +|34|lens.server.estimate.pool.max.threads|100|Maximum number of threads in the estimate thread pool| *--+--+---+--+ -|33|lens.server.estimate.pool.min.threads|3|Minimum number of threads in the estimate thread pool| +|35|lens.server.estimate.pool.min.threads|3|Minimum number of threads in the estimate thread pool| *--+--+---+--+ -|34|lens.server.estimate.timeout.millis|300000|Timeout for parallel query estimate calls in milliseconds. A driver needs to comeback with a query estimate within this timeout. If the timeout is reached, only the drivers that have provided an estimate would be considered for query selection. If the timeout is reached and none of the drivers have provided an estimate then estimate calls fails with an exception.| +|36|lens.server.estimate.timeout.millis|300000|Timeout for parallel query estimate calls in milliseconds. A driver needs to comeback with a query estimate within this timeout. If the timeout is reached, only the drivers that have provided an estimate would be considered for query selection. If the timeout is reached and none of the drivers have provided an estimate then estimate calls fails with an exception.| *--+--+---+--+ -|35|lens.server.event.service.thread.pool.size| |The size of thread pool for notifying events. The no value is specified, it uses the available processors as the number.| +|37|lens.server.event.service.thread.pool.size| |The size of thread pool for notifying events. The no value is specified, it uses the available processors as the number.| *--+--+---+--+ -|36|lens.server.grizzly.core.pool.size|20|Core pool size for lens grizzly server| +|38|lens.server.grizzly.core.pool.size|20|Core pool size for lens grizzly server| *--+--+---+--+ -|37|lens.server.grizzly.max.pool.size|40|Max pool size for lens grizzly server| +|39|lens.server.grizzly.max.pool.size|40|Max pool size for lens grizzly server| *--+--+---+--+ -|38|lens.server.hdfs.output.retention|1 day|hdfs output retention period. Default 1 day| +|40|lens.server.hdfs.output.retention|1 day|hdfs output retention period. Default 1 day| *--+--+---+--+ -|39|lens.server.http.notification.conn.timeout.millis|5000|This is the connection timeout for all HTTP Notifications sent by lens server. Default connection timeout is 5 secs| +|41|lens.server.http.notification.conn.timeout.millis|5000|This is the connection timeout for all HTTP Notifications sent by lens server. Default connection timeout is 5 secs| *--+--+---+--+ -|40|lens.server.http.notification.read.timeout.millis|10000|This is the read timeout for all HTTP Notifications sent by lens server. Default read timeout is 10 secs| +|42|lens.server.http.notification.read.timeout.millis|10000|This is the read timeout for all HTTP Notifications sent by lens server. Default read timeout is 10 secs| *--+--+---+--+ -|41|lens.server.index.ws.resource.impl|org.apache.lens.server.IndexResource|Implementation class for Index Resource| +|43|lens.server.index.ws.resource.impl|org.apache.lens.server.IndexResource|Implementation class for Index Resource| *--+--+---+--+ -|42|lens.server.inmemory.resultset.ttl.secs|300|This property defines the TTL(time to live) in seconds for all result sets of type InMemoryResultSet beyond which they are eligible for purging irrespective of whether the result set has been read or not. The default value is 300 seconds (5 minutes).| +|44|lens.server.inmemory.resultset.ttl.secs|300|This property defines the TTL(time to live) in seconds for all result sets of type InMemoryResultSet beyond which they are eligible for purging irrespective of whether the result set has been read or not. The default value is 300 seconds (5 minutes).| *--+--+---+--+ -|43|lens.server.launcher.pool.keepalive.millis|60000|Thread keep alive time in milliseconds for the query launcher thread pool. If there are no query launches for this period,then cached threads will be released from the pool.| +|45|lens.server.launcher.pool.keepalive.millis|60000|Thread keep alive time in milliseconds for the query launcher thread pool. If there are no query launches for this period,then cached threads will be released from the pool.| *--+--+---+--+ -|44|lens.server.launcher.pool.max.threads|100|Maximum number of threads in the query launcher thread pool. Keeping the default to hundred, we may never grow till there, it would go to max for concurrrent queries allowed on all drivers together. This value should be greater than the max concurrent queries allowed on all drivers.| +|46|lens.server.launcher.pool.max.threads|100|Maximum number of threads in the query launcher thread pool. Keeping the default to hundred, we may never grow till there, it would go to max for concurrrent queries allowed on all drivers together. This value should be greater than the max concurrent queries allowed on all drivers.| *--+--+---+--+ -|45|lens.server.launcher.pool.min.threads|3|Minimum number of threads in the query launcher thread pool| +|47|lens.server.launcher.pool.min.threads|3|Minimum number of threads in the query launcher thread pool| *--+--+---+--+ -|46|lens.server.log.ws.resource.impl|org.apache.lens.server.LogResource|Implementation class for Log Resource| +|48|lens.server.log.ws.resource.impl|org.apache.lens.server.LogResource|Implementation class for Log Resource| *--+--+---+--+ -|47|lens.server.mail.from.address|b...@company.com|The from field in the notifier mail to the submitter.| +|49|lens.server.mail.from.address|b...@company.com|The from field in the notifier mail to the submitter.| *--+--+---+--+ -|48|lens.server.mail.host|mail-host.company.com|SMTP Host for sending mail| +|50|lens.server.mail.host|mail-host.company.com|SMTP Host for sending mail| *--+--+---+--+ -|49|lens.server.mail.port|25|SMTP Port| +|51|lens.server.mail.port|25|SMTP Port| *--+--+---+--+ -|50|lens.server.mail.smtp.connectiontimeout|15000|Socket connection timeout value in milliseconds. This timeout is implemented by java.net.Socket. Default is 15 seconds.| +|52|lens.server.mail.smtp.connectiontimeout|15000|Socket connection timeout value in milliseconds. This timeout is implemented by java.net.Socket. Default is 15 seconds.| *--+--+---+--+ -|51|lens.server.mail.smtp.timeout|30000|Socket read timeout value in milliseconds. This timeout is implemented by java.net.Socket. Default is 30 seconds.| +|53|lens.server.mail.smtp.timeout|30000|Socket read timeout value in milliseconds. This timeout is implemented by java.net.Socket. Default is 30 seconds.| *--+--+---+--+ -|52|lens.server.max.sessions.per.user|10|Number of sessions can be allowed for each user. User has to close one of the active sessions to open a new session once limit is reached. Otherwise Server throws an exception by saying that opened session limit has been already reached for user.| +|54|lens.server.max.sessions.per.user|10|Number of sessions can be allowed for each user. User has to close one of the active sessions to open a new session once limit is reached. Otherwise Server throws an exception by saying that opened session limit has been already reached for user.| *--+--+---+--+ -|53|lens.server.metastore.service.impl|org.apache.lens.server.metastore.CubeMetastoreServiceImpl|Implementation class for metastore service| +|55|lens.server.metastore.service.impl|org.apache.lens.server.metastore.CubeMetastoreServiceImpl|Implementation class for metastore service| *--+--+---+--+ -|54|lens.server.metastore.ws.resource.impl|org.apache.lens.server.metastore.MetastoreResource|Implementation class for Metastore Resource| +|56|lens.server.metastore.ws.resource.impl|org.apache.lens.server.metastore.MetastoreResource|Implementation class for Metastore Resource| *--+--+---+--+ -|55|lens.server.metrics.csv.directory.path|metrics/|Path of the directory in which to report metrics as separate csv files.| +|57|lens.server.metrics.csv.directory.path|metrics/|Path of the directory in which to report metrics as separate csv files.| *--+--+---+--+ -|56|lens.server.metrics.ganglia.host| |The ganglia host name| +|58|lens.server.metrics.ganglia.host| |The ganglia host name| *--+--+---+--+ -|57|lens.server.metrics.ganglia.port| |The ganglia port| +|59|lens.server.metrics.ganglia.port| |The ganglia port| *--+--+---+--+ -|58|lens.server.metrics.graphite.host| |The graphite host name| +|60|lens.server.metrics.graphite.host| |The graphite host name| *--+--+---+--+ -|59|lens.server.metrics.graphite.port| |The graphite port| +|61|lens.server.metrics.graphite.port| |The graphite port| *--+--+---+--+ -|60|lens.server.metrics.reporting.period|10|The reporting period for metrics. The value is in seconds| +|62|lens.server.metrics.reporting.period|10|The reporting period for metrics. The value is in seconds| *--+--+---+--+ -|61|lens.server.mode|OPEN|The mode in which server should run. Allowed values are OPEN, READ_ONLY, METASTORE_READONLY, METASTORE_NODROP. OPEN mode will allow all requests. READ_ONLY mode will allow all requests on session resouce and only GET requests on all other resources. METASTORE_READONLY will allow GET on metastore and all other requests in other services. METASTORE_NODROP will not allow DELETE on metastore, will allow all other requests.| +|63|lens.server.mode|OPEN|The mode in which server should run. Allowed values are OPEN, READ_ONLY, METASTORE_READONLY, METASTORE_NODROP. OPEN mode will allow all requests. READ_ONLY mode will allow all requests on session resouce and only GET requests on all other resources. METASTORE_READONLY will allow GET on metastore and all other requests in other services. METASTORE_NODROP will not allow DELETE on metastore, will allow all other requests.| *--+--+---+--+ -|62|lens.server.moxyjson.ws.feature.impl|org.glassfish.jersey.moxy.json.MoxyJsonFeature|Enable Moxy json feature| +|64|lens.server.moxyjson.ws.feature.impl|org.glassfish.jersey.moxy.json.MoxyJsonFeature|Enable Moxy json feature| *--+--+---+--+ -|63|lens.server.moxyjsonconfigresovler.ws.feature.impl|org.apache.lens.api.util.MoxyJsonConfigurationContextResolver|Moxy json configuration resolver| +|65|lens.server.moxyjsonconfigresovler.ws.feature.impl|org.apache.lens.api.util.MoxyJsonConfigurationContextResolver|Moxy json configuration resolver| *--+--+---+--+ -|64|lens.server.multipart.ws.feature.impl|org.glassfish.jersey.media.multipart.MultiPartFeature|Implementation class for query scheduler resource| +|66|lens.server.multipart.ws.feature.impl|org.glassfish.jersey.media.multipart.MultiPartFeature|Implementation class for query scheduler resource| *--+--+---+--+ -|65|lens.server.persist.location|file:///tmp/lensserver|The directory in which lens server will persist its state when it is going down. The location be on any Hadoop compatible file system. Server will read from the location when it is restarted and recovery is enabled. So, Server should have both read and write permissions to the location| +|67|lens.server.persist.location|file:///tmp/lensserver|The directory in which lens server will persist its state when it is going down. The location be on any Hadoop compatible file system. Server will read from the location when it is restarted and recovery is enabled. So, Server should have both read and write permissions to the location| *--+--+---+--+ -|66|lens.server.query.acceptors| |Query Acceptors configured. Query acceptors are consulted first, before anything happens for the given query. They can either return null or return a messaging indicating why the given query shouldn't be accepted. These can be used to filter out queries at the earliest.| +|68|lens.server.query.acceptors| |Query Acceptors configured. Query acceptors are consulted first, before anything happens for the given query. They can either return null or return a messaging indicating why the given query shouldn't be accepted. These can be used to filter out queries at the earliest.| *--+--+---+--+ -|67|lens.server.query.expiry.check.interval.millis|60000|The interval(milliseconds) with which query expiry will run periodically. Default is 1 minute. The value needs to be much lower than lens.query.timeout.millis. If the final deployment values of query timeout can be smaller, then reduce this value to be much lower.| +|69|lens.server.query.expiry.check.interval.millis|60000|The interval(milliseconds) with which query expiry will run periodically. Default is 1 minute. The value needs to be much lower than lens.query.timeout.millis. If the final deployment values of query timeout can be smaller, then reduce this value to be much lower.| *--+--+---+--+ -|68|lens.server.query.launching.constraint.factories|org.apache.lens.server.query.constraint.TotalQueryCostCeilingConstraintFactory|Factories used to instantiate constraints enforced on queries by lens. Every Factory should be an implementation of org.apache.lens.server.api.common.ConfigBasedObjectCreationFactory and create an implementation of org.apache.lens.server.api.query.constraint.QueryLaunchingConstraint. A query will be launched only if all constraints pass.| +|70|lens.server.query.launching.constraint.factories|org.apache.lens.server.query.constraint.TotalQueryCostCeilingConstraintFactory|Factories used to instantiate constraints enforced on queries by lens. Every Factory should be an implementation of org.apache.lens.server.api.common.ConfigBasedObjectCreationFactory and create an implementation of org.apache.lens.server.api.query.constraint.QueryLaunchingConstraint. A query will be launched only if all constraints pass.| *--+--+---+--+ -|69|lens.server.query.phase1.rewriters| |Query phase 1 rewriters. This is to convert user query to cube query. The resulting cube query will be passed for validation and rewriting to hql query.\ | +|71|lens.server.query.phase1.rewriters| |Query phase 1 rewriters. This is to convert user query to cube query. The resulting cube query will be passed for validation and rewriting to hql query.\ | | | | |Use cases will be to use extra intelligence to convert user query to optimized cube query. \ | | | | |Or define shortcuts for certain frequently used queries :) | *--+--+---+--+ -|70|lens.server.query.resultset.retention|1 day|Lens query resultset retention period. Default 1 day| +|72|lens.server.query.resultset.retention|1 day|Lens query resultset retention period. Default 1 day| *--+--+---+--+ -|71|lens.server.query.service.impl|org.apache.lens.server.query.QueryExecutionServiceImpl|Implementation class for query execution service| +|73|lens.server.query.service.impl|org.apache.lens.server.query.QueryExecutionServiceImpl|Implementation class for query execution service| *--+--+---+--+ -|72|lens.server.query.state.logger.enabled|true|Disable or enable the query state logger with this config. The location for the logger can be specified in logback xml for the class org.apache.lens.server.query.QueryExecutionServiceImpl.QueryStatusLogger| +|74|lens.server.query.state.logger.enabled|true|Disable or enable the query state logger with this config. The location for the logger can be specified in logback xml for the class org.apache.lens.server.query.QueryExecutionServiceImpl.QueryStatusLogger| *--+--+---+--+ -|73|lens.server.query.ws.resource.impl|org.apache.lens.server.query.QueryServiceResource|Implementation class for Query Resource| +|75|lens.server.query.ws.resource.impl|org.apache.lens.server.query.QueryServiceResource|Implementation class for Query Resource| *--+--+---+--+ -|74|lens.server.querypurger.sleep.interval|10000|The interval(milliseconds) with which purger to run periodically. Default 10 sec.| +|76|lens.server.querypurger.sleep.interval|10000|The interval(milliseconds) with which purger to run periodically. Default 10 sec.| *--+--+---+--+ -|75|lens.server.quota.service.impl|org.apache.lens.server.quota.QuotaServiceImpl|Implementation class for quota service| +|77|lens.server.quota.service.impl|org.apache.lens.server.quota.QuotaServiceImpl|Implementation class for quota service| *--+--+---+--+ -|76|lens.server.quota.ws.resource.impl|org.apache.lens.server.quota.QuotaResource|Implementation class for Quota Resource| +|78|lens.server.quota.ws.resource.impl|org.apache.lens.server.quota.QuotaResource|Implementation class for Quota Resource| *--+--+---+--+ -|77|lens.server.requestlogger.ws.filter.impl|org.apache.lens.server.LensRequestLoggingFilter|Implementation class for Request logging Filter| +|79|lens.server.requestlogger.ws.filter.impl|org.apache.lens.server.LensRequestLoggingFilter|Implementation class for Request logging Filter| *--+--+---+--+ -|78|lens.server.resultset.purge.enabled|false|Whether to purge the query results| +|80|lens.server.resultset.purge.enabled|false|Whether to purge the query results| *--+--+---+--+ -|79|lens.server.resultsetpurger.sleep.interval.secs|3600|Periodicity for Query result purger runs. Default 1 hour.| +|81|lens.server.resultsetpurger.sleep.interval.secs|3600|Periodicity for Query result purger runs. Default 1 hour.| *--+--+---+--+ -|80|lens.server.savedquery.jdbc.dialectclass|org.apache.lens.server.query.save.SavedQueryDao$HSQLDialect|Dialect of the target DB, Default is HSQL. Override with the target DB used.| +|82|lens.server.savedquery.jdbc.dialectclass|org.apache.lens.server.query.save.SavedQueryDao$HSQLDialect|Dialect of the target DB, Default is HSQL. Override with the target DB used.| *--+--+---+--+ -|81|lens.server.savedquery.list.default.count|20|Key denoting the default fetch value of saved query list api.| +|83|lens.server.savedquery.list.default.count|20|Key denoting the default fetch value of saved query list api.| *--+--+---+--+ -|82|lens.server.savedquery.list.default.offset|0|Key denoting the default start value of saved query list api.| +|84|lens.server.savedquery.list.default.offset|0|Key denoting the default start value of saved query list api.| *--+--+---+--+ -|83|lens.server.savedquery.service.impl|org.apache.lens.server.query.save.SavedQueryServiceImpl|Implementation class for saved query service| +|85|lens.server.savedquery.service.impl|org.apache.lens.server.query.save.SavedQueryServiceImpl|Implementation class for saved query service| *--+--+---+--+ -|84|lens.server.savedquery.ws.resource.impl|org.apache.lens.server.query.save.SavedQueryResource|Implementation class for Saved query Resource| +|86|lens.server.savedquery.ws.resource.impl|org.apache.lens.server.query.save.SavedQueryResource|Implementation class for Saved query Resource| *--+--+---+--+ -|85|lens.server.scheduler.instance.waiting.thread.interval.millis|300000|Thread interval for checking the waiting instances in milliseconds| +|87|lens.server.scheduler.instance.waiting.thread.interval.millis|300000|Thread interval for checking the waiting instances in milliseconds| *--+--+---+--+ -|86|lens.server.scheduler.max.job.per.user|-1|Maximum number of jobs that can be scheduled by a single user. If the number is less than zero, then there is no restriction on the number of jobs scheduled.| +|88|lens.server.scheduler.max.job.per.user|-1|Maximum number of jobs that can be scheduled by a single user. If the number is less than zero, then there is no restriction on the number of jobs scheduled.| *--+--+---+--+ -|87|lens.server.scheduler.service.impl|org.apache.lens.server.scheduler.SchedulerServiceImpl|Implementation class for query scheduler service| +|89|lens.server.scheduler.service.impl|org.apache.lens.server.scheduler.SchedulerServiceImpl|Implementation class for query scheduler service| *--+--+---+--+ -|88|lens.server.scheduler.store.class|org.apache.lens.server.scheduler.SchedulerDAO$SchedulerHsqlDBStore|A subclass of SchedulerDBStore class used for storing scheduler related information.| +|90|lens.server.scheduler.store.class|org.apache.lens.server.scheduler.SchedulerDAO$SchedulerHsqlDBStore|A subclass of SchedulerDBStore class used for storing scheduler related information.| *--+--+---+--+ -|89|lens.server.scheduler.ws.resource.impl|org.apache.lens.server.scheduler.ScheduleResource|Implementation class for query scheduler resource| +|91|lens.server.scheduler.ws.resource.impl|org.apache.lens.server.scheduler.ScheduleResource|Implementation class for query scheduler resource| *--+--+---+--+ -|90|lens.server.scheduling.queue.poll.interval.millisec|2000|The interval at which submission thread will poll scheduling queue to fetch the next query for submission. If value is less than equal to 0, then it would mean that thread will continuosly poll without sleeping. The interval has to be given in milliseconds.| +|92|lens.server.scheduling.queue.poll.interval.millisec|2000|The interval at which submission thread will poll scheduling queue to fetch the next query for submission. If value is less than equal to 0, then it would mean that thread will continuosly poll without sleeping. The interval has to be given in milliseconds.| *--+--+---+--+ -|91|lens.server.serverMode.ws.filter.impl|org.apache.lens.server.ServerModeFilter|Implementation class for ServerMode Filter| +|93|lens.server.serverMode.ws.filter.impl|org.apache.lens.server.ServerModeFilter|Implementation class for ServerMode Filter| *--+--+---+--+ -|92|lens.server.service.provider.factory|org.apache.lens.server.ServiceProviderFactoryImpl|Service provider factory implementation class. This parameter is used to lookup the factory implementation class name that would provide an instance of ServiceProvider. Users should instantiate the class to obtain its instance. Example -- Class spfClass = conf.getClass("lens.server.service.provider.factory", null, ServiceProviderFactory.class); ServiceProviderFactory spf = spfClass.newInstance(); ServiceProvider serviceProvider = spf.getServiceProvider(); -- This is not supposed to be overridden by users.| +|94|lens.server.service.provider.factory|org.apache.lens.server.ServiceProviderFactoryImpl|Service provider factory implementation class. This parameter is used to lookup the factory implementation class name that would provide an instance of ServiceProvider. Users should instantiate the class to obtain its instance. Example -- Class spfClass = conf.getClass("lens.server.service.provider.factory", null, ServiceProviderFactory.class); ServiceProviderFactory spf = spfClass.newInstance(); ServiceProvider serviceProvider = spf.getServiceProvider(); -- This is not supposed to be overridden by users.| *--+--+---+--+ -|93|lens.server.servicenames|session,alarm,query,savedquery,metastore,scheduler,quota|These services would be started in the specified order when lens-server starts up| +|95|lens.server.servicenames|session,alarm,query,savedquery,metastore,scheduler,quota|These services would be started in the specified order when lens-server starts up| *--+--+---+--+ -|94|lens.server.session.expiry.service.interval.secs|3600|Interval at which lens session expiry service runs| +|96|lens.server.session.expiry.service.interval.secs|3600|Interval at which lens session expiry service runs| *--+--+---+--+ -|95|lens.server.session.service.impl|org.apache.lens.server.session.HiveSessionService|Implementation class for session service| +|97|lens.server.session.service.impl|org.apache.lens.server.session.HiveSessionService|Implementation class for session service| *--+--+---+--+ -|96|lens.server.session.timeout.seconds|86400|Lens session timeout in seconds.If there is no activity on the session for this period then the session will be closed.Default timeout is one day.| +|98|lens.server.session.timeout.seconds|86400|Lens session timeout in seconds.If there is no activity on the session for this period then the session will be closed.Default timeout is one day.| *--+--+---+--+ -|97|lens.server.session.ws.resource.impl|org.apache.lens.server.session.SessionResource|Implementation class for Session Resource| +|99|lens.server.session.ws.resource.impl|org.apache.lens.server.session.SessionResource|Implementation class for Session Resource| *--+--+---+--+ -|98|lens.server.state.persist.out.stream.buffer.size|1048576|Output Stream Buffer Size used in writing lens server state to file system. Size is in bytes.| +|100|lens.server.state.persist.out.stream.buffer.size|1048576|Output Stream Buffer Size used in writing lens server state to file system. Size is in bytes.| *--+--+---+--+ -|99|lens.server.state.persistence.enabled|true|If flag is enabled, state of all the services will be persisted periodically to a location specified by lens.server.persist.location and on server restart all the services will be started from last saved state.| +|101|lens.server.state.persistence.enabled|true|If flag is enabled, state of all the services will be persisted periodically to a location specified by lens.server.persist.location and on server restart all the services will be started from last saved state.| *--+--+---+--+ -|100|lens.server.state.persistence.interval.millis|300000|Lens server state persistence time interval in milliseconds| +|102|lens.server.state.persistence.interval.millis|300000|Lens server state persistence time interval in milliseconds| *--+--+---+--+ -|101|lens.server.statistics.db|lensstats|Database to which statistics tables are created and partitions are added.| +|103|lens.server.statistics.db|lensstats|Database to which statistics tables are created and partitions are added.| *--+--+---+--+ -|102|lens.server.statistics.log.rollover.interval|3600000|Default rate which log statistics store scans for rollups in milliseconds.| +|104|lens.server.statistics.log.rollover.interval|3600000|Default rate which log statistics store scans for rollups in milliseconds.| *--+--+---+--+ -|103|lens.server.statistics.store.class|org.apache.lens.server.stats.store.log.LogStatisticsStore|Default implementation of class used to persist Lens Statistics.| +|105|lens.server.statistics.store.class|org.apache.lens.server.stats.store.log.LogStatisticsStore|Default implementation of class used to persist Lens Statistics.| *--+--+---+--+ -|104|lens.server.statistics.warehouse.dir|file:///tmp/lens/statistics/warehouse|Default top level location where stats are moved by the log statistics store.| +|106|lens.server.statistics.warehouse.dir|file:///tmp/lens/statistics/warehouse|Default top level location where stats are moved by the log statistics store.| *--+--+---+--+ -|105|lens.server.status.update.exponential.wait.millis|30000|Number of millis that would grow exponentially for next update, incase of transient failures.| +|107|lens.server.status.update.exponential.wait.millis|30000|Number of millis that would grow exponentially for next update, incase of transient failures.| *--+--+---+--+ -|106|lens.server.status.update.maximum.delay.secs|1800|The maximum delay in seconds for next status update to happen after any transient failure. This will be used a maximum delay sothat exponential wait times not to grow to bigger value.| +|108|lens.server.status.update.maximum.delay.secs|1800|The maximum delay in seconds for next status update to happen after any transient failure. This will be used a maximum delay sothat exponential wait times not to grow to bigger value.| *--+--+---+--+ -|107|lens.server.status.update.num.retries|10|The number of retries a status update will tried with exponentital back off, in case of transient issues, upon which query will be marked FAILED.| +|109|lens.server.status.update.num.retries|10|The number of retries a status update will tried with exponentital back off, in case of transient issues, upon which query will be marked FAILED.| *--+--+---+--+ -|108|lens.server.total.query.cost.ceiling.per.user|-1.0|A query submitted by user will be launched only if total query cost of all current launched queries of user is less than or equal to total query cost ceiling defined by this property. This configuration value is only useful when TotalQueryCostCeilingConstraint is enabled by using org.apache.lens.server.query.constraint.TotalQueryCostCeilingConstraintFactory as one of the factories in lens.server.query.constraint.factories property. Default is -1.0 which means that there is no limit on the total query cost of launched queries submitted by a user.| +|110|lens.server.total.query.cost.ceiling.per.user|-1.0|A query submitted by user will be launched only if total query cost of all current launched queries of user is less than or equal to total query cost ceiling defined by this property. This configuration value is only useful when TotalQueryCostCeilingConstraint is enabled by using org.apache.lens.server.query.constraint.TotalQueryCostCeilingConstraintFactory as one of the factories in lens.server.query.constraint.factories property. Default is -1.0 which means that there is no limit on the total query cost of launched queries submitted by a user.| *--+--+---+--+ -|109|lens.server.user.resolver.custom.class|full.package.name.Classname|Required for CUSTOM user resolver. In case the provided implementations are not sufficient for user config resolver, a custom classname can be provided. Class should extend org.apache.lens.server.user.UserConfigLoader| +|111|lens.server.user.resolver.custom.class|full.package.name.Classname|Required for CUSTOM user resolver. In case the provided implementations are not sufficient for user config resolver, a custom classname can be provided. Class should extend org.apache.lens.server.user.UserConfigLoader| *--+--+---+--+ -|110|lens.server.user.resolver.db.keys|lens.session.cluster.user,mapred.job.queue.name|Required for DATABASE and LDAP_BACKED_DATABASE user resolvers. For database based user config loaders, the conf keys that will be loaded from database.| +|112|lens.server.user.resolver.db.keys|lens.session.cluster.user,mapred.job.queue.name|Required for DATABASE and LDAP_BACKED_DATABASE user resolvers. For database based user config loaders, the conf keys that will be loaded from database.| *--+--+---+--+ -|111|lens.server.user.resolver.db.query|select clusteruser,queue from user_config_table where username=?|Required for DATABASE and LDAP_BACKED_DATABASE user resolvers. For database based user config loader, this query will be run with single argument = logged in user and the result columns will be assigned to lens.server.user.resolver.db.keys in order. For ldap backed database resolver, the argument to this query will be the intermediate values obtained from ldap.| +|113|lens.server.user.resolver.db.query|select clusteruser,queue from user_config_table where username=?|Required for DATABASE and LDAP_BACKED_DATABASE user resolvers. For database based user config loader, this query will be run with single argument = logged in user and the result columns will be assigned to lens.server.user.resolver.db.keys in order. For ldap backed database resolver, the argument to this query will be the intermediate values obtained from ldap.| *--+--+---+--+ -|112|lens.server.user.resolver.fixed.value| |Required for FIXED user resolver. when lens.server.user.resolver.type=FIXED, This will be the value cluster user will resolve to.| +|114|lens.server.user.resolver.fixed.value| |Required for FIXED user resolver. when lens.server.user.resolver.type=FIXED, This will be the value cluster user will resolve to.| *--+--+---+--+ -|113|lens.server.user.resolver.ldap.bind.dn| |Required for LDAP_BACKED_DATABASE user resolvers. ldap dn for admin binding example: CN=company-it-admin,ou=service-account,ou=company-service-account,dc=dc1,dc=com...| +|115|lens.server.user.resolver.ldap.bind.dn| |Required for LDAP_BACKED_DATABASE user resolvers. ldap dn for admin binding example: CN=company-it-admin,ou=service-account,ou=company-service-account,dc=dc1,dc=com...| *--+--+---+--+ -|114|lens.server.user.resolver.ldap.bind.password| |Required for LDAP_BACKED_DATABASE user resolvers. ldap password for admin binding above| +|116|lens.server.user.resolver.ldap.bind.password| |Required for LDAP_BACKED_DATABASE user resolvers. ldap password for admin binding above| *--+--+---+--+ -|115|lens.server.user.resolver.ldap.fields|department|Required for LDAP_BACKED_DATABASE user resolvers. list of fields to be obtained from ldap. These will be cached by the intermediate db.| +|117|lens.server.user.resolver.ldap.fields|department|Required for LDAP_BACKED_DATABASE user resolvers. list of fields to be obtained from ldap. These will be cached by the intermediate db.| *--+--+---+--+ -|116|lens.server.user.resolver.ldap.intermediate.db.delete.sql|delete from user_department where username=?|Required for LDAP_BACKED_DATABASE user resolvers. query to delete intermediate values from database backing ldap as cache. one argument: logged in user.| +|118|lens.server.user.resolver.ldap.intermediate.db.delete.sql|delete from user_department where username=?|Required for LDAP_BACKED_DATABASE user resolvers. query to delete intermediate values from database backing ldap as cache. one argument: logged in user.| *--+--+---+--+ -|117|lens.server.user.resolver.ldap.intermediate.db.insert.sql|insert into user_department (username, department, expiry) values (?, ?, ?)|Required for LDAP_BACKED_DATABASE user resolvers. query to insert intermediate values from database backing ldap as cache. arguments: first logged in user, then all intermediate values, then current time + expiration time| +|119|lens.server.user.resolver.ldap.intermediate.db.insert.sql|insert into user_department (username, department, expiry) values (?, ?, ?)|Required for LDAP_BACKED_DATABASE user resolvers. query to insert intermediate values from database backing ldap as cache. arguments: first logged in user, then all intermediate values, then current time + expiration time| *--+--+---+--+ -|118|lens.server.user.resolver.ldap.intermediate.db.query|select department from user_department where username=? and expiry>?|Required for LDAP_BACKED_DATABASE user resolvers. query to obtain intermediate values from database backing ldap as cache. two arguments: logged in user and current time.| +|120|lens.server.user.resolver.ldap.intermediate.db.query|select department from user_department where username=? and expiry>?|Required for LDAP_BACKED_DATABASE user resolvers. query to obtain intermediate values from database backing ldap as cache. two arguments: logged in user and current time.| *--+--+---+--+ -|119|lens.server.user.resolver.ldap.search.base| |Required for LDAP_BACKED_DATABASE user resolvers. for searching intermediate values for a user, the search keys. example: cn=users,dc=dc1,dc=dc2...| +|121|lens.server.user.resolver.ldap.search.base| |Required for LDAP_BACKED_DATABASE user resolvers. for searching intermediate values for a user, the search keys. example: cn=users,dc=dc1,dc=dc2...| *--+--+---+--+ -|120|lens.server.user.resolver.ldap.search.filter|(&(objectClass=user)(sAMAccountName=%s))|Required for LDAP_BACKED_DATABASE user resolvers. filter pattern for ldap search| +|122|lens.server.user.resolver.ldap.search.filter|(&(objectClass=user)(sAMAccountName=%s))|Required for LDAP_BACKED_DATABASE user resolvers. filter pattern for ldap search| *--+--+---+--+ -|121|lens.server.user.resolver.ldap.url| |Required for LDAP_BACKED_DATABASE user resolvers. ldap url to connect to.| +|123|lens.server.user.resolver.ldap.url| |Required for LDAP_BACKED_DATABASE user resolvers. ldap url to connect to.| *--+--+---+--+ -|122|lens.server.user.resolver.propertybased.filename|/path/to/propertyfile|Required for PROPERTYBASED user resolver. when lens.server.user.resolver.type is PROPERTYBASED, then this file will be read and parsed to determine cluster user. Each line should contain username followed by DOT followed by property full name followed by equal-to sign and followed by value. example schema of the file is: user1.lens.server.cluster.user=clusteruser1 user1.mapred.job.queue.name=queue1 *.lens.server.cluster.user=defaultclusteruser *.mapred.job.queue.name=default| +|124|lens.server.user.resolver.propertybased.filename|/path/to/propertyfile|Required for PROPERTYBASED user resolver. when lens.server.user.resolver.type is PROPERTYBASED, then this file will be read and parsed to determine cluster user. Each line should contain username followed by DOT followed by property full name followed by equal-to sign and followed by value. example schema of the file is: user1.lens.server.cluster.user=clusteruser1 user1.mapred.job.queue.name=queue1 *.lens.server.cluster.user=defaultclusteruser *.mapred.job.queue.name=default| *--+--+---+--+ -|123|lens.server.user.resolver.type|FIXED|Type of user config resolver. allowed values are FIXED, PROPERTYBASED, DATABASE, LDAP_BACKED_DATABASE, CUSTOM.| +|125|lens.server.user.resolver.type|FIXED|Type of user config resolver. allowed values are FIXED, PROPERTYBASED, DATABASE, LDAP_BACKED_DATABASE, CUSTOM.| *--+--+---+--+ -|124|lens.server.waiting.queries.selection.policy.factories|org.apache.lens.server.query.collect.UserSpecificWaitingQueriesSelectionPolicyFactory|Factories used to instantiate waiting queries selection policies. Every factory should be an implementation of org.apache.lens.server.api.common.ConfigBasedObjectCreationFactory and create an implementation of org.apache.lens.server.api.query.collect.WaitingQueriesSelectionPolicy.| +|126|lens.server.waiting.queries.selection.policy.factories|org.apache.lens.server.query.collect.UserSpecificWaitingQueriesSelectionPolicyFactory|Factories used to instantiate waiting queries selection policies. Every factory should be an implementation of org.apache.lens.server.api.common.ConfigBasedObjectCreationFactory and create an implementation of org.apache.lens.server.api.query.collect.WaitingQueriesSelectionPolicy.| *--+--+---+--+ -|125|lens.server.ws.featurenames|multipart,moxyjson,moxyjsonconfigresovler|These JAX-RS Feature(s) would be started in the specified order when lens-server starts up| +|127|lens.server.ws.featurenames|multipart,moxyjson,moxyjsonconfigresovler|These JAX-RS Feature(s) would be started in the specified order when lens-server starts up| *--+--+---+--+ -|126|lens.server.ws.filternames|requestlogger,consistentState,serverMode|These JAX-RS filters would be started in the specified order when lens-server starts up| +|128|lens.server.ws.filternames|requestlogger,consistentState,serverMode|These JAX-RS filters would be started in the specified order when lens-server starts up| *--+--+---+--+ -|127|lens.server.ws.listenernames|appevent|These listeners would be called in the specified order when lens-server starts up| +|129|lens.server.ws.listenernames|appevent|These listeners would be called in the specified order when lens-server starts up| *--+--+---+--+ -|128|lens.server.ws.resourcenames|session,metastore,query,savedquery,quota,scheduler,index,log|These JAX-RS resources would be started in the specified order when lens-server starts up| +|130|lens.server.ws.resourcenames|session,metastore,query,savedquery,quota,scheduler,index,log|These JAX-RS resources would be started in the specified order when lens-server starts up| *--+--+---+--+ The configuration parameters and their default values http://git-wip-us.apache.org/repos/asf/lens/blob/078555c1/src/site/apt/admin/session-config.apt ---------------------------------------------------------------------- diff --git a/src/site/apt/admin/session-config.apt b/src/site/apt/admin/session-config.apt index 663a605..74c4482 100644 --- a/src/site/apt/admin/session-config.apt +++ b/src/site/apt/admin/session-config.apt @@ -38,80 +38,82 @@ Lens session configuration *--+--+---+--+ |7|hive.metastore.uris| |The hive metastore server URI that the lens server is talking to| *--+--+---+--+ -|8|lens.cube.query.disable.aggregate.resolver|false|Tells whether to disable automatic resolution of aggregations for measures in a cube. To enable automatic resolution, this value should be false.| +|8|lens.cube.query.completeness.threshold|100|The query will fail if data completeness is less than the set threshold given that the flag "lens.cube.query.fail.if.data.partial" is set as true| *--+--+---+--+ -|9|lens.cube.query.disable.auto.join|false|Tells whether to disable automatic resolution of join conditions between tables involved. To enable automatic resolution, this value should be false.| +|9|lens.cube.query.disable.aggregate.resolver|false|Tells whether to disable automatic resolution of aggregations for measures in a cube. To enable automatic resolution, this value should be false.| *--+--+---+--+ -|10|lens.cube.query.fail.if.data.partial|true|Whether to fail the query of data is partial| +|10|lens.cube.query.disable.auto.join|false|Tells whether to disable automatic resolution of join conditions between tables involved. To enable automatic resolution, this value should be false.| *--+--+---+--+ -|11|lens.cube.query.promote.select.togroupby|true|Tells whether to promote select expressions which is not inside any aggregate, to be promoted to groupby clauses, if they are already not part of groupby clauses. To enable automatic promotion, this value should be true.| +|11|lens.cube.query.fail.if.data.partial|true|Whether to fail the query of data is partial| *--+--+---+--+ -|12|lens.query.add.insert.overwrite|true|Prefix query with insert overwrite clause if the query is persistent. User can disable if user gave the clause himself.| +|12|lens.cube.query.promote.select.togroupby|true|Tells whether to promote select expressions which is not inside any aggregate, to be promoted to groupby clauses, if they are already not part of groupby clauses. To enable automatic promotion, this value should be true.| *--+--+---+--+ -|13|lens.query.cancel.on.timeout|true|Specifies whether to attempt cancellation of a query whose execution takes longer than the timeout value specified while submitting the query for execution. The default value is true.| +|13|lens.query.add.insert.overwrite|true|Prefix query with insert overwrite clause if the query is persistent. User can disable if user gave the clause himself.| *--+--+---+--+ -|14|lens.query.enable.mail.notify|false|When a query ends, whether to notify the submitter by mail or not.| +|14|lens.query.cancel.on.timeout|true|Specifies whether to attempt cancellation of a query whose execution takes longer than the timeout value specified while submitting the query for execution. The default value is true.| *--+--+---+--+ -|15|lens.query.enable.metrics.per.query|false|Generates gauge metrics for each query to measure time taken with unique id appended for each query. Should be enabled only for performance measurements. Should not be enabled in day to day production environment.| +|15|lens.query.enable.mail.notify|false|When a query ends, whether to notify the submitter by mail or not.| *--+--+---+--+ -|16|lens.query.enable.persistent.resultset|false|Whether to enable persistent resultset for queries. When enabled, server will fetch results from driver, custom format them if any and store in a configured location. The file name of query output is queryhandle-id, with configured extensions| +|16|lens.query.enable.metrics.per.query|false|Generates gauge metrics for each query to measure time taken with unique id appended for each query. Should be enabled only for performance measurements. Should not be enabled in day to day production environment.| *--+--+---+--+ -|17|lens.query.enable.persistent.resultset.indriver|true|Whether the result should be persisted by driver. Currently only HiveDriver persists the results in a HDFS location.| +|17|lens.query.enable.persistent.resultset|false|Whether to enable persistent resultset for queries. When enabled, server will fetch results from driver, custom format them if any and store in a configured location. The file name of query output is queryhandle-id, with configured extensions| *--+--+---+--+ -|18|lens.query.hdfs.output.path|hdfsout|The directory under the parent result directory, in which HiveDriver will persist the results, if persisting by driver is enabled. This directory should exist and should have world writable permissions sothat all users will be able put query outputs here.| +|18|lens.query.enable.persistent.resultset.indriver|true|Whether the result should be persisted by driver. Currently only HiveDriver persists the results in a HDFS location.| *--+--+---+--+ -|19|lens.query.http.notification.mediatype|application/json|This is the media type for Query Http notifications. Accepted types are "application/json" and "application/xml". The default value is "application/json"| +|19|lens.query.hdfs.output.path|hdfsout|The directory under the parent result directory, in which HiveDriver will persist the results, if persisting by driver is enabled. This directory should exist and should have world writable permissions sothat all users will be able put query outputs here.| *--+--+---+--+ -|20|lens.query.http.notification.type.FINISHED|false|Setting this property to true will enable query FINISHED notifications which includes SUCCESSFUL, FAILED and CANCELLED queries. The notification will have eventtype = "FINISHED", eventtime = long event time and query = org.apache.lens.api.query.LensQuery instance. The mediatype for eventtype and eventtime will be TEXT/PLAIN and the mediatype for query will be based on property lens.query.http.notification.mediatype. Default value of this property is false.| +|20|lens.query.http.notification.mediatype|application/json|This is the media type for Query Http notifications. Accepted types are "application/json" and "application/xml". The default value is "application/json"| *--+--+---+--+ -|21|lens.query.http.notification.urls| |These are the http end points for Query http notifications. Users can specify more than one comma separated end points for a query. Url parameter values that include special characters should be encoded. Please note that if this property is not set, no http notification will be sent out by lens server for the query.| +|21|lens.query.http.notification.type.FINISHED|false|Setting this property to true will enable query FINISHED notifications which includes SUCCESSFUL, FAILED and CANCELLED queries. The notification will have eventtype = "FINISHED", eventtime = long event time and query = org.apache.lens.api.query.LensQuery instance. The mediatype for eventtype and eventtime will be TEXT/PLAIN and the mediatype for query will be based on property lens.query.http.notification.mediatype. Default value of this property is false.| *--+--+---+--+ -|22|lens.query.output.charset.encoding|UTF-8|The charset encoding for formatting query result. It supports all the encodings supported by java.io.OutputStreamWriter.| +|22|lens.query.http.notification.urls| |These are the http end points for Query http notifications. Users can specify more than one comma separated end points for a query. Url parameter values that include special characters should be encoded. Please note that if this property is not set, no http notification will be sent out by lens server for the query.| *--+--+---+--+ -|23|lens.query.output.compression.codec|org.apache.hadoop.io.compress.GzipCodec|The codec used to compress the query output, if compression is enabled| +|23|lens.query.output.charset.encoding|UTF-8|The charset encoding for formatting query result. It supports all the encodings supported by java.io.OutputStreamWriter.| *--+--+---+--+ -|24|lens.query.output.enable.compression|false|Whether to compress the query result output| +|24|lens.query.output.compression.codec|org.apache.hadoop.io.compress.GzipCodec|The codec used to compress the query output, if compression is enabled| *--+--+---+--+ -|25|lens.query.output.file.extn|.csv|The extension name for the persisted query output file. If file is compressed, the extension from compression codec will be appended to this extension.| +|25|lens.query.output.enable.compression|false|Whether to compress the query result output| *--+--+---+--+ -|26|lens.query.output.footer| |The value of custom footer that should be written, if any. This footer will be added in formatting driver persisted results.| +|26|lens.query.output.file.extn|.csv|The extension name for the persisted query output file. If file is compressed, the extension from compression codec will be appended to this extension.| *--+--+---+--+ -|27|lens.query.output.formatter| |The query result output formatter for the query. If no value is specified, then org.apache.lens.lib.query.FileSerdeFormatter will be used to format in-memory result sets, org.apache.lens.lib.query.FilePersistentFormatter will be used to format driver persisted result sets.| +|27|lens.query.output.footer| |The value of custom footer that should be written, if any. This footer will be added in formatting driver persisted results.| *--+--+---+--+ -|28|lens.query.output.header| |The value of custom header that should be written, if any. If no value column names will be used as header.| +|28|lens.query.output.formatter| |The query result output formatter for the query. If no value is specified, then org.apache.lens.lib.query.FileSerdeFormatter will be used to format in-memory result sets, org.apache.lens.lib.query.FilePersistentFormatter will be used to format driver persisted result sets.| *--+--+---+--+ -|29|lens.query.output.write.footer|false|Whether to write footer as part of query result. When enabled, total number of rows will be written as part of header.| +|29|lens.query.output.header| |The value of custom header that should be written, if any. If no value column names will be used as header.| *--+--+---+--+ -|30|lens.query.output.write.header|false|Whether to write header as part of query result formatting. When enabled the user given header will be added in case of driver persisted results, and column names chosen will be added as header for in-memory results.| +|30|lens.query.output.write.footer|false|Whether to write footer as part of query result. When enabled, total number of rows will be written as part of header.| *--+--+---+--+ -|31|lens.query.prefetch.inmemory.resultset|true|When set to true, specified number of rows of result set will be pre-fetched if the result set is of type InMemoryResultSet and query execution is not asynchronous i.e. query should be launched with operation as EXECUTE_WITH_TIMEOUT. Suggested usage of this property: It can be used by client to stream as well as persist results in server for queries that finish fast and produce results with fewer rows (should be less than number of rows pre-fetched). Note that the results are streamed to the client early, without waiting for persistence to finish. Default value of this property is true.| +|31|lens.query.output.write.header|false|Whether to write header as part of query result formatting. When enabled the user given header will be added in case of driver persisted results, and column names chosen will be added as header for in-memory results.| *--+--+---+--+ -|32|lens.query.prefetch.inmemory.resultset.rows|100|Specifies the number of rows to pre-fetch when lens.query.prefetch.inmemory.resultset is set to true. Default value is 100 rows.| +|32|lens.query.prefetch.inmemory.resultset|true|When set to true, specified number of rows of result set will be pre-fetched if the result set is of type InMemoryResultSet and query execution is not asynchronous i.e. query should be launched with operation as EXECUTE_WITH_TIMEOUT. Suggested usage of this property: It can be used by client to stream as well as persist results in server for queries that finish fast and produce results with fewer rows (should be less than number of rows pre-fetched). Note that the results are streamed to the client early, without waiting for persistence to finish. Default value of this property is true.| *--+--+---+--+ -|33|lens.query.result.email.cc| |When query ends, the result/failure reason will be sent to the user via email. The mail would be cc'ed to the addresses provided in this field.| +|33|lens.query.prefetch.inmemory.resultset.rows|100|Specifies the number of rows to pre-fetch when lens.query.prefetch.inmemory.resultset is set to true. Default value is 100 rows.| *--+--+---+--+ -|34|lens.query.result.fs.read.url| |Http read URL for FileSystem on which result is present, if available. For example webhdfs as http read url should http://host:port/webhdfs/v1. Currently we support only webhdfs url as the http url for HDFS file system| +|34|lens.query.result.email.cc| |When query ends, the result/failure reason will be sent to the user via email. The mail would be cc'ed to the addresses provided in this field.| *--+--+---+--+ -|35|lens.query.result.output.dir.format| |The format of the output if result is persisted in hdfs. The format should be expressed in HQL.| +|35|lens.query.result.fs.read.url| |Http read URL for FileSystem on which result is present, if available. For example webhdfs as http read url should http://host:port/webhdfs/v1. Currently we support only webhdfs url as the http url for HDFS file system| *--+--+---+--+ -|36|lens.query.result.output.serde|org.apache.lens.lib.query.CSVSerde|The default serde class name that should be used by org.apache.lens.lib.query.FileSerdeFormatter for formatting the output| +|36|lens.query.result.output.dir.format| |The format of the output if result is persisted in hdfs. The format should be expressed in HQL.| *--+--+---+--+ -|37|lens.query.result.parent.dir|file:///tmp/lensreports|The directory for storing persisted result of query. This directory should exist and should have writable permissions by lens server| +|37|lens.query.result.output.serde|org.apache.lens.lib.query.CSVSerde|The default serde class name that should be used by org.apache.lens.lib.query.FileSerdeFormatter for formatting the output| *--+--+---+--+ -|38|lens.query.result.size.format.threshold|10737418240|The maximum allowed size of the query result. If exceeds, no server side formatting would be done.| +|38|lens.query.result.parent.dir|file:///tmp/lensreports|The directory for storing persisted result of query. This directory should exist and should have writable permissions by lens server| *--+--+---+--+ -|39|lens.query.result.split.multiple|false|Whether to split the result into multiple files. If enabled, each file will be restricted to max rows configured. All the files will be available as zip.| +|39|lens.query.result.size.format.threshold|10737418240|The maximum allowed size of the query result. If exceeds, no server side formatting would be done.| *--+--+---+--+ -|40|lens.query.result.split.multiple.maxrows|100000|The maximum number of rows allowed in each file, when splitting the result into multiple files is enabled.| +|40|lens.query.result.split.multiple|false|Whether to split the result into multiple files. If enabled, each file will be restricted to max rows configured. All the files will be available as zip.| *--+--+---+--+ -|41|lens.query.timeout.millis|86400000|The runtime(millis) of the query after which query will be timedout and cancelled. Default is 1 day.| +|41|lens.query.result.split.multiple.maxrows|100000|The maximum number of rows allowed in each file, when splitting the result into multiple files is enabled.| *--+--+---+--+ -|42|lens.session.aux.jars| |List of comma separated jar paths, which will added to the session| +|42|lens.query.timeout.millis|86400000|The runtime(millis) of the query after which query will be timedout and cancelled. Default is 1 day.| *--+--+---+--+ -|43|lens.session.cluster.user| |Session level config which will determine which cluster user will access hdfs| +|43|lens.session.aux.jars| |List of comma separated jar paths, which will added to the session| *--+--+---+--+ -|44|lens.session.loggedin.user| |The username used to log in to lens. e.g. LDAP user| +|44|lens.session.cluster.user| |Session level config which will determine which cluster user will access hdfs| *--+--+---+--+ -|45|lens.session.metastore.exclude.cubetables.from.nativetables|true|Exclude cube related tables when fetching native tables| +|45|lens.session.loggedin.user| |The username used to log in to lens. e.g. LDAP user| +*--+--+---+--+ +|46|lens.session.metastore.exclude.cubetables.from.nativetables|true|Exclude cube related tables when fetching native tables| *--+--+---+--+ The configuration parameters and their default values http://git-wip-us.apache.org/repos/asf/lens/blob/078555c1/src/site/apt/user/olap-query-conf.apt ---------------------------------------------------------------------- diff --git a/src/site/apt/user/olap-query-conf.apt b/src/site/apt/user/olap-query-conf.apt index ce35ec4..90d635b 100644 --- a/src/site/apt/user/olap-query-conf.apt +++ b/src/site/apt/user/olap-query-conf.apt @@ -28,56 +28,60 @@ OLAP query configuration *--+--+---+--+ |2|lens.cube.query.bridge.table.field.array.filter|array_contains|The array filter function to be used for filter on fields from bridge tables. Would be used only when flattening is enabled and not flattening early. The value can be passed for each query. During typical deployments it would be passed for each driver, as the function could be different for each driver. Signature of the filter getting passed here should be very similar to array_contains. i.e. array_contains(array, value) to return true if array contains value.| *--+--+---+--+ -|3|lens.cube.query.disable.aggregate.resolver|true|Tells whether to disable automatic resolution of aggregations for measures in a cube. To enable automatic resolution, this value should be false.| +|3|lens.cube.query.completeness.check.partition.column| |The Supported Partition Column for the Data Completeness check| *--+--+---+--+ -|4|lens.cube.query.disable.auto.join|true|Tells whether to disable automatic resolution of join conditions between tables involved. To enable automatic resolution, this value should be false.| +|4|lens.cube.query.completeness.threshold|100|The query will fail if data completeness is less than the set threshold given that the flag "lens.cube.query.fail.if.data.partial" is set as true| *--+--+---+--+ -|5|lens.cube.query.driver.supported.storages| |List of comma separated storage names that supported by a driver. If no value is specified, all storages are valid| +|5|lens.cube.query.disable.aggregate.resolver|true|Tells whether to disable automatic resolution of aggregations for measures in a cube. To enable automatic resolution, this value should be false.| *--+--+---+--+ -|6|lens.cube.query.enable.attrfields.add.distinct|true|When the query has only attribute fields projected from cube and the flag is set to true, distinct clause will be added for the projection so that no duplicate values will be projected. If flag is set to false, projection wont be changed, result might include duplicate values.| +|6|lens.cube.query.disable.auto.join|true|Tells whether to disable automatic resolution of join conditions between tables involved. To enable automatic resolution, this value should be false.| *--+--+---+--+ -|7|lens.cube.query.enable.flattening.bridge.tables|false|Flag specifies if fields selected have to be flattened or not, if they are coming from tables with many to many relationship in join. If false, field selection will be simple join and selecting the field. If true, the fields from bridge tables will be aggregated grouped by join key.| +|7|lens.cube.query.driver.supported.storages| |List of comma separated storage names that supported by a driver. If no value is specified, all storages are valid| *--+--+---+--+ -|8|lens.cube.query.enable.storages.union|false|Sometimes One storage table doesn't contain all required partitions, and the query needs to be answered from two storage tables. Enabling this (make value = <true>) allows rewrite of such queries. If it's <false>, then such queries will fail in rewrite phase. The feature should only be enabled when all the aggregate functions used in the query (explicitly or implicitly picked from default aggregates of used measures) are transitive. Transitive aggregate functions are those that follow the following property:\ | -| | | | \ | -| | | |<<<f(a, b, c, d) = f(f(a, b), f(c, d)) for all possible values of a,b,c,d.>>> \ | -| | | | \ | -| | | |e.g. SUM, MAX, MIN etc are transitive aggregate functions, while AVG, COUNT etc are not. | +|8|lens.cube.query.enable.attrfields.add.distinct|true|When the query has only attribute fields projected from cube and the flag is set to true, distinct clause will be added for the projection so that no duplicate values will be projected. If flag is set to false, projection wont be changed, result might include duplicate values.| *--+--+---+--+ -|9|lens.cube.query.fail.if.data.partial|false|Whether to fail the query of data is partial| +|9|lens.cube.query.enable.flattening.bridge.tables|false|Flag specifies if fields selected have to be flattened or not, if they are coming from tables with many to many relationship in join. If false, field selection will be simple join and selecting the field. If true, the fields from bridge tables will be aggregated grouped by join key.| *--+--+---+--+ -|10|lens.cube.query.flatten.bridge.tables.early|false|Flag specifies if fields selected have to be flattened earlier to applying any expression on top of them, For ex: sub_str(bridge_field) is the expression, if the property is set to true, sub_str will be applied on aggregated field. If set to false, aggregate will applied on top of the expression i.e. sub_str.| +|10|lens.cube.query.enable.storages.union|false|Sometimes One storage table doesn't contain all required partitions, and the query needs to be answered from two storage tables. Enabling this (make value = <true>) allows rewrite of such queries. If it's <false>, then such queries will fail in rewrite phase. The feature should only be enabled when all the aggregate functions used in the query (explicitly or implicitly picked from default aggregates of used measures) are transitive. Transitive aggregate functions are those that follow the following property:\ | +| | | | \ | +| | | |<<<f(a, b, c, d) = f(f(a, b), f(c, d)) for all possible values of a,b,c,d.>>> \ | +| | | | \ | +| | | |e.g. SUM, MAX, MIN etc are transitive aggregate functions, while AVG, COUNT etc are not. | *--+--+---+--+ -|11|lens.cube.query.join.type|INNER|Tells what the join type is, in-case of automatic resolution of joins is enabled| +|11|lens.cube.query.fail.if.data.partial|false|Whether to fail the query of data is partial| *--+--+---+--+ -|12|lens.cube.query.lookahead.ptparts.forinterval.${interval}|1|The value of number of lookahead process time partitions for interval specified. Interval can be any Update period.| +|12|lens.cube.query.flatten.bridge.tables.early|false|Flag specifies if fields selected have to be flattened earlier to applying any expression on top of them, For ex: sub_str(bridge_field) is the expression, if the property is set to true, sub_str will be applied on aggregated field. If set to false, aggregate will applied on top of the expression i.e. sub_str.| *--+--+---+--+ -|13|lens.cube.query.max.interval| |Maximum value of the update period that the query timed dimensions can take values of. For example, if query involves month ranges, user can say query maximum interval is daily, then no monthly partitions will be picked.| +|13|lens.cube.query.join.type|INNER|Tells what the join type is, in-case of automatic resolution of joins is enabled| *--+--+---+--+ -|14|lens.cube.query.nonexisting.partitions| |The list of comma separated non existing partitions, if query can run with partial data. The value will be set by the cube query rewriter| +|14|lens.cube.query.lookahead.ptparts.forinterval.${interval}|1|The value of number of lookahead process time partitions for interval specified. Interval can be any Update period.| *--+--+---+--+ -|15|lens.cube.query.partition.where.clause.format| |The simple date format of how the queried partition should be put in where clause. If nothing is specified, it will use the format from org.apache.lens.cube.metadata.UpdatePeriod for each type of partition| +|15|lens.cube.query.max.interval| |Maximum value of the update period that the query timed dimensions can take values of. For example, if query involves month ranges, user can say query maximum interval is daily, then no monthly partitions will be picked.| *--+--+---+--+ -|16|lens.cube.query.pick.lightest.fact.first|false|If set to true, lightest fact will be resolved first than resolving storages. Otherwise, storages will be resolved to check all partitions exist and then pick lightest fact among candidates| +|16|lens.cube.query.nonexisting.partitions| |The list of comma separated non existing partitions, if query can run with partial data. The value will be set by the cube query rewriter| *--+--+---+--+ -|17|lens.cube.query.process.time.partition.column| |The column name which is a process time column. If process time column is specified, query rewriter will look ahead the partitions of other timed dimensions inside this column.| +|17|lens.cube.query.partition.where.clause.format| |The simple date format of how the queried partition should be put in where clause. If nothing is specified, it will use the format from org.apache.lens.cube.metadata.UpdatePeriod for each type of partition| *--+--+---+--+ -|18|lens.cube.query.promote.groupby.toselect|false|Tells whether to promote group by clauses to be promoted to select expressions if they are already not projected. To enable automatic promotion, this value should be true.| +|18|lens.cube.query.pick.lightest.fact.first|false|If set to true, lightest fact will be resolved first than resolving storages. Otherwise, storages will be resolved to check all partitions exist and then pick lightest fact among candidates| *--+--+---+--+ -|19|lens.cube.query.promote.select.togroupby|false|Tells whether to promote select expressions which is not inside any aggregate, to be promoted to groupby clauses, if they are already not part of groupby clauses. To enable automatic promotion, this value should be true.| +|19|lens.cube.query.process.time.partition.column| |The column name which is a process time column. If process time column is specified, query rewriter will look ahead the partitions of other timed dimensions inside this column.| *--+--+---+--+ -|20|lens.cube.query.replace.timedim|true|Tells whether timedim attribute queried in the time range should be replaced with its corresponding partition column name.| +|20|lens.cube.query.promote.groupby.toselect|false|Tells whether to promote group by clauses to be promoted to select expressions if they are already not projected. To enable automatic promotion, this value should be true.| *--+--+---+--+ -|21|lens.cube.query.rewrite.dim.filter.to.fact.filter|false|Flag specifies if dimension filter has to be rewritten as fact filter. for eg. where dim.name in ('x', 'y') will become where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))| +|21|lens.cube.query.promote.select.togroupby|false|Tells whether to promote select expressions which is not inside any aggregate, to be promoted to groupby clauses, if they are already not part of groupby clauses. To enable automatic promotion, this value should be true.| *--+--+---+--+ -|22|lens.cube.query.time.range.writer.class|org.apache.lens.cube.parse.ORTimeRangeWriter|The timerange writer class which specifies how the resolved partitions in timeranges should be written in final query. Available writers are org.apache.lens.cube.parse.ORTimeRangeWriter and org.apache.lens.cube.parse.BetweenTimeRangeWriter| +|22|lens.cube.query.replace.timedim|true|Tells whether timedim attribute queried in the time range should be replaced with its corresponding partition column name.| *--+--+---+--+ -|23|lens.cube.query.valid.${cubename}.facttables| |List of comma separated fact tables that are valid for cube. If no value is specified, all fact tables are valid| +|23|lens.cube.query.rewrite.dim.filter.to.fact.filter|false|Flag specifies if dimension filter has to be rewritten as fact filter. for eg. where dim.name in ('x', 'y') will become where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))| *--+--+---+--+ -|24|lens.cube.query.valid.dim.storgaetables| |List of comma separated dimension storage tables that are valid. If no value is specified, all tables are valid| +|24|lens.cube.query.time.range.writer.class|org.apache.lens.cube.parse.ORTimeRangeWriter|The timerange writer class which specifies how the resolved partitions in timeranges should be written in final query. Available writers are org.apache.lens.cube.parse.ORTimeRangeWriter and org.apache.lens.cube.parse.BetweenTimeRangeWriter| *--+--+---+--+ -|25|lens.cube.query.valid.fact.${facttable}.storage.${storagename}.updateperiods| |List of comma separated update periods that are valid for a fact on a storage. If no value is specified, all update periods are valid| +|25|lens.cube.query.valid.${cubename}.facttables| |List of comma separated fact tables that are valid for cube. If no value is specified, all fact tables are valid| *--+--+---+--+ -|26|lens.cube.query.valid.fact.${facttable}.storagetables| |List of comma separated storage tables that are valid for a fact. If no value is specified, all storage tables are valid| +|26|lens.cube.query.valid.dim.storgaetables| |List of comma separated dimension storage tables that are valid. If no value is specified, all tables are valid| +*--+--+---+--+ +|27|lens.cube.query.valid.fact.${facttable}.storage.${storagename}.updateperiods| |List of comma separated update periods that are valid for a fact on a storage. If no value is specified, all update periods are valid| +*--+--+---+--+ +|28|lens.cube.query.valid.fact.${facttable}.storagetables| |List of comma separated storage tables that are valid for a fact. If no value is specified, all storage tables are valid| *--+--+---+--+ The configuration parameters and their default values
