http://git-wip-us.apache.org/repos/asf/drill-site/blob/ee640eab/docs/configuring-drill-memory/index.html ---------------------------------------------------------------------- diff --git a/docs/configuring-drill-memory/index.html b/docs/configuring-drill-memory/index.html index 973b145..3ad3402 100644 --- a/docs/configuring-drill-memory/index.html +++ b/docs/configuring-drill-memory/index.html @@ -9,26 +9,21 @@ <title>Configuring Drill Memory - Apache Drill</title> -<link href="/css/syntax.css" rel="stylesheet" type="text/css"> -<link href="/css/style.css" rel="stylesheet" type="text/css"> -<link href="/css/arrows.css" rel="stylesheet" type="text/css"> -<link href="/css/breadcrumbs.css" rel="stylesheet" type="text/css"> -<link href="/css/code.css" rel="stylesheet" type="text/css"> -<link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/font-awesome/4.3.0/css/font-awesome.min.css"> -<link href="/css/responsive.css" rel="stylesheet" type="text/css"> - -<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"> -<link rel="icon" href="/favicon.ico" type="image/x-icon"> - -<script language="javascript" type="text/javascript" src="/js/lib/jquery-1.11.1.min.js"></script> -<script language="javascript" type="text/javascript" src="/js/lib/jquery.easing.1.3.js"></script> +<link href="//maxcdn.bootstrapcdn.com/font-awesome/4.3.0/css/font-awesome.min.css" rel="stylesheet" type="text/css"/> +<link href="/css/site.css" rel="stylesheet" type="text/css"/> + +<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"/> +<link rel="icon" href="/favicon.ico" type="image/x-icon"/> + +<script src="//ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js" language="javascript" type="text/javascript"></script> +<script src="//cdnjs.cloudflare.com/ajax/libs/jquery-easing/1.3/jquery.easing.min.js" language="javascript" type="text/javascript"></script> <script language="javascript" type="text/javascript" src="/js/modernizr.custom.js"></script> <script language="javascript" type="text/javascript" src="/js/script.js"></script> <script language="javascript" type="text/javascript" src="/js/drill.js"></script> - </head> + <body onResize="resized();"> <div class="page-wrap"> <div class="bui"></div> @@ -63,6 +58,12 @@ <li><a href="/docs/query-data/">Query Data</a></li> + <li><a href="/docs/performance-tuning/">Performance Tuning</a></li> + + <li><a href="/docs/log-and-debug/">Log and Debug</a></li> + + <li><a href="/docs/query-audit-logging/">Query Audit Logging</a></li> + <li><a href="/docs/sql-reference/">SQL Reference</a></li> <li><a href="/docs/data-sources-and-file-formats/">Data Sources and File Formats</a></li> @@ -81,6 +82,8 @@ <li><a href="/docs/project-bylaws/">Project Bylaws</a></li> + <li><a href="/docs/log-and-debug/">Log and Debug</a></li> + </ul> </li> <li class='nav'> @@ -110,7 +113,9 @@ </ul> </div> - + <link href="/css/content.css" rel="stylesheet" type="text/css"> + + @@ -149,14 +154,7 @@ - <li class="toctree-l2"><a href="javascript: void(0);">Architectural Highlights</a></li> - <ul style="display: none"> - - <li class="toctree-l3"><a class="reference internal" href="/docs/flexibility/">Flexibility</a></li> - - <li class="toctree-l3"><a class="reference internal" href="/docs/performance/">Performance</a></li> - - </ul> + <li class="toctree-l2"><a class="reference internal" href="/docs/performance/">Performance</a></li> </ul> @@ -203,6 +201,10 @@ <li class="toctree-l2"><a class="reference internal" href="/docs/analyzing-highly-dynamic-datasets/">Analyzing Highly Dynamic Datasets</a></li> + + <li class="toctree-l2"><a class="reference internal" href="/docs/analyzing-social-media/">Analyzing Social Media</a></li> + + </ul> @@ -300,7 +302,7 @@ - <li class="toctree-l2"><a class="reference internal" href="/docs/partition-pruning/">Partition Pruning</a></li> + <li class="toctree-l2"><a class="reference internal" href="/docs/configuring-the-drill-shell/">Configuring the Drill Shell</a></li> </ul> @@ -357,7 +359,7 @@ - <li class="toctree-l2"><a class="reference internal" href="/docs/using-jdbc/">Using JDBC</a></li> + <li class="toctree-l2"><a class="reference internal" href="/docs/using-jdbc-with-squirrel-on-windows/">Using JDBC with SQuirreL on Windows</a></li> @@ -409,6 +411,10 @@ + <li class="toctree-l2"><a class="reference internal" href="/docs/configuring-tibco-spotfire-server-with-drill/">Configuring Tibco Spotfire Server with Drill</a></li> + + + <li class="toctree-l2"><a class="reference internal" href="/docs/using-apache-drill-with-tableau-9-desktop/">Using Apache Drill with Tableau 9 Desktop</a></li> @@ -487,6 +493,32 @@ + <li class="toctree-l1"><a href="javascript: void(0);">Performance Tuning</a></li> + <ul style="display: none"> + + + <li class="toctree-l2"><a class="reference internal" href="/docs/performance-tuning-introduction/">Performance Tuning Introduction</a></li> + + + </ul> + + + + <li class="toctree-l1"><a class="reference internal" href="/docs/log-and-debug/">Log and Debug</a></li> + + + + <li class="toctree-l1"><a href="javascript: void(0);">Query Audit Logging</a></li> + <ul style="display: none"> + + + <li class="toctree-l2"><a class="reference internal" href="/docs/getting-query-information/">Getting Query Information</a></li> + + + </ul> + + + <li class="toctree-l1"><a href="javascript: void(0);">SQL Reference</a></li> <ul style="display: none"> @@ -563,27 +595,47 @@ <li class="toctree-l3"><a class="reference internal" href="/docs/supported-sql-commands/">Supported SQL Commands</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/alter-session-command/">ALTER SESSION Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/alter-session/">ALTER SESSION</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/alter-system/">ALTER SYSTEM</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/create-table-as-ctas/">CREATE TABLE AS (CTAS)</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/create-view/">CREATE VIEW</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/drop-view/">DROP VIEW</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/describe/">DESCRIBE</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/explain/">EXPLAIN</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/select/">SELECT</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/select-from/">SELECT FROM</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/select-group-by/">SELECT GROUP BY</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/alter-system-command/">ALTER SYSTEM Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-having/">SELECT HAVING</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/create-table-as-ctas-command/">CREATE TABLE AS (CTAS) Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-limit/">SELECT LIMIT</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/create-view-command/">CREATE VIEW Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-offset/">SELECT OFFSET</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/describe-command/">DESCRIBE Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-order-by/">SELECT ORDER BY</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/explain-commands/">EXPLAIN Commands</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-union/">SELECT UNION</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/select-statements/">SELECT Statements</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-where/">SELECT WHERE</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/show-databases-and-show-schemas-command/">SHOW DATABASES and SHOW SCHEMAS Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-with/">SELECT WITH</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/show-files-command/">SHOW FILES Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/show-databases-and-show-schemas/">SHOW DATABASES and SHOW SCHEMAS</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/show-tables-command/">SHOW TABLES Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/show-files/">SHOW FILES</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/use-command/">USE Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/show-tables/">SHOW TABLES</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/use/">USE</a></li> </ul> @@ -782,6 +834,10 @@ <li class="toctree-l2"><a class="reference internal" href="/docs/what-is-apache-drill/">What is Apache Drill</a></li> + + <li class="toctree-l2"><a class="reference internal" href="/docs/partition-pruning/">Partition Pruning</a></li> + + </ul> @@ -800,6 +856,29 @@ <li class="toctree-l1"><a class="reference internal" href="/docs/project-bylaws/">Project Bylaws</a></li> + + <li class="toctree-l1"><a href="javascript: void(0);">Log and Debug</a></li> + <ul style="display: none"> + + + <li class="toctree-l2"><a class="reference internal" href="/docs/log-and-debug-introduction/">Log and Debug Introduction</a></li> + + + + <li class="toctree-l2"><a class="reference internal" href="/docs/error-messages/">Error Messages</a></li> + + + + <li class="toctree-l2"><a class="reference internal" href="/docs/modify-logback-xml/">Modify logback.xml</a></li> + + + + <li class="toctree-l2"><a class="reference internal" href="/docs/review-the-java-stack-trace/">Review the Java Stack Trace</a></li> + + + </ul> + + </ul> </div> @@ -807,7 +886,7 @@ </aside> - <nav class="breadcrumbs"> +<nav class="breadcrumbs"> <li><a href="/docs/">Docs</a></li> @@ -816,23 +895,23 @@ <li>Configuring Drill Memory</li> </nav> - <div class="main-content-wrapper"> - <div class="main-content"> +<div class="main-content-wrapper"> + <div class="main-content"> - - <a class="edit-link" href="https://github.com/apache/drill/blob/gh-pages/_docs/configure-drill/020-configuring-drill-memory.md"; target="_blank"><i class="fa fa-pencil-square-o"></i></a> - + + <a class="edit-link" href="https://github.com/apache/drill/blob/gh-pages/_docs/configure-drill/020-configuring-drill-memory.md"; target="_blank"><i class="fa fa-pencil-square-o"></i></a> + - <div class="int_title"> - <h1>Configuring Drill Memory</h1> + <div class="int_title left"> + <h1>Configuring Drill Memory</h1> - </div> + </div> - <link href="/css/docpage.css" rel="stylesheet" type="text/css"> + <link href="/css/docpage.css" rel="stylesheet" type="text/css"> - <div class="int_text" align="left"> - - <p>This section describes how to configure the amount of direct memory allocated to a Drillbit for query processing in any Drill cluster, multitenant or not. The default memory for a Drillbit is 8G, but Drill prefers 16G or more depending on the workload. The total amount of direct memory that a Drillbit allocates to query operations cannot exceed the limit set.</p> + <div class="int_text" align="left"> + + <p>You can configure the amount of direct memory allocated to a Drillbit for query processing in any Drill cluster, multitenant or not. The default memory for a Drillbit is 8G, but Drill prefers 16G or more depending on the workload. The total amount of direct memory that a Drillbit allocates to query operations cannot exceed the limit set.</p> <p>Drill uses Java direct memory and performs well when executing operations in memory instead of storing the operations on disk. Drill does not @@ -858,7 +937,7 @@ Drillbit startup script, <code>drill-env.sh</code>, located in <code><drill_i <h2 id="about-the-drillbit-startup-script">About the Drillbit startup script</h2> -<p>The drill-env.sh file contains the following options:</p> +<p>The <code>drill-env.sh</code> file contains the following options:</p> <div class="highlight"><pre><code class="language-text" data-lang="text">DRILL_MAX_DIRECT_MEMORY="8G" DRILL_MAX_HEAP="4G" @@ -871,17 +950,17 @@ export DRILL_JAVA_OPTS="-Xms1G -Xmx$DRILL_MAX_HEAP -XX:MaxDirectMemorySize= <li>Xms specifies the initial memory allocation pool.</li> </ul> + - - <div class="doc-nav"> + <div class="doc-nav"> <span class="previous-toc"><a href="/docs/configure-drill-introduction/">â Configure Drill Introduction</a></span><span class="next-toc"><a href="/docs/configuring-a-multitenant-cluster/">Configuring a Multitenant Cluster â</a></span> </div> - - </div> + </div> </div> +</div> </div> <p class="push"></p>
http://git-wip-us.apache.org/repos/asf/drill-site/blob/ee640eab/docs/configuring-multitenant-resources/index.html ---------------------------------------------------------------------- diff --git a/docs/configuring-multitenant-resources/index.html b/docs/configuring-multitenant-resources/index.html index 7c15255..c62e7a8 100644 --- a/docs/configuring-multitenant-resources/index.html +++ b/docs/configuring-multitenant-resources/index.html @@ -9,26 +9,21 @@ <title>Configuring Multitenant Resources - Apache Drill</title> -<link href="/css/syntax.css" rel="stylesheet" type="text/css"> -<link href="/css/style.css" rel="stylesheet" type="text/css"> -<link href="/css/arrows.css" rel="stylesheet" type="text/css"> -<link href="/css/breadcrumbs.css" rel="stylesheet" type="text/css"> -<link href="/css/code.css" rel="stylesheet" type="text/css"> -<link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/font-awesome/4.3.0/css/font-awesome.min.css"> -<link href="/css/responsive.css" rel="stylesheet" type="text/css"> - -<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"> -<link rel="icon" href="/favicon.ico" type="image/x-icon"> - -<script language="javascript" type="text/javascript" src="/js/lib/jquery-1.11.1.min.js"></script> -<script language="javascript" type="text/javascript" src="/js/lib/jquery.easing.1.3.js"></script> +<link href="//maxcdn.bootstrapcdn.com/font-awesome/4.3.0/css/font-awesome.min.css" rel="stylesheet" type="text/css"/> +<link href="/css/site.css" rel="stylesheet" type="text/css"/> + +<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"/> +<link rel="icon" href="/favicon.ico" type="image/x-icon"/> + +<script src="//ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js" language="javascript" type="text/javascript"></script> +<script src="//cdnjs.cloudflare.com/ajax/libs/jquery-easing/1.3/jquery.easing.min.js" language="javascript" type="text/javascript"></script> <script language="javascript" type="text/javascript" src="/js/modernizr.custom.js"></script> <script language="javascript" type="text/javascript" src="/js/script.js"></script> <script language="javascript" type="text/javascript" src="/js/drill.js"></script> - </head> + <body onResize="resized();"> <div class="page-wrap"> <div class="bui"></div> @@ -63,6 +58,12 @@ <li><a href="/docs/query-data/">Query Data</a></li> + <li><a href="/docs/performance-tuning/">Performance Tuning</a></li> + + <li><a href="/docs/log-and-debug/">Log and Debug</a></li> + + <li><a href="/docs/query-audit-logging/">Query Audit Logging</a></li> + <li><a href="/docs/sql-reference/">SQL Reference</a></li> <li><a href="/docs/data-sources-and-file-formats/">Data Sources and File Formats</a></li> @@ -81,6 +82,8 @@ <li><a href="/docs/project-bylaws/">Project Bylaws</a></li> + <li><a href="/docs/log-and-debug/">Log and Debug</a></li> + </ul> </li> <li class='nav'> @@ -110,7 +113,9 @@ </ul> </div> - + <link href="/css/content.css" rel="stylesheet" type="text/css"> + + @@ -149,14 +154,7 @@ - <li class="toctree-l2"><a href="javascript: void(0);">Architectural Highlights</a></li> - <ul style="display: none"> - - <li class="toctree-l3"><a class="reference internal" href="/docs/flexibility/">Flexibility</a></li> - - <li class="toctree-l3"><a class="reference internal" href="/docs/performance/">Performance</a></li> - - </ul> + <li class="toctree-l2"><a class="reference internal" href="/docs/performance/">Performance</a></li> </ul> @@ -203,6 +201,10 @@ <li class="toctree-l2"><a class="reference internal" href="/docs/analyzing-highly-dynamic-datasets/">Analyzing Highly Dynamic Datasets</a></li> + + <li class="toctree-l2"><a class="reference internal" href="/docs/analyzing-social-media/">Analyzing Social Media</a></li> + + </ul> @@ -300,7 +302,7 @@ - <li class="toctree-l2"><a class="reference internal" href="/docs/partition-pruning/">Partition Pruning</a></li> + <li class="toctree-l2"><a class="reference internal" href="/docs/configuring-the-drill-shell/">Configuring the Drill Shell</a></li> </ul> @@ -357,7 +359,7 @@ - <li class="toctree-l2"><a class="reference internal" href="/docs/using-jdbc/">Using JDBC</a></li> + <li class="toctree-l2"><a class="reference internal" href="/docs/using-jdbc-with-squirrel-on-windows/">Using JDBC with SQuirreL on Windows</a></li> @@ -409,6 +411,10 @@ + <li class="toctree-l2"><a class="reference internal" href="/docs/configuring-tibco-spotfire-server-with-drill/">Configuring Tibco Spotfire Server with Drill</a></li> + + + <li class="toctree-l2"><a class="reference internal" href="/docs/using-apache-drill-with-tableau-9-desktop/">Using Apache Drill with Tableau 9 Desktop</a></li> @@ -487,6 +493,32 @@ + <li class="toctree-l1"><a href="javascript: void(0);">Performance Tuning</a></li> + <ul style="display: none"> + + + <li class="toctree-l2"><a class="reference internal" href="/docs/performance-tuning-introduction/">Performance Tuning Introduction</a></li> + + + </ul> + + + + <li class="toctree-l1"><a class="reference internal" href="/docs/log-and-debug/">Log and Debug</a></li> + + + + <li class="toctree-l1"><a href="javascript: void(0);">Query Audit Logging</a></li> + <ul style="display: none"> + + + <li class="toctree-l2"><a class="reference internal" href="/docs/getting-query-information/">Getting Query Information</a></li> + + + </ul> + + + <li class="toctree-l1"><a href="javascript: void(0);">SQL Reference</a></li> <ul style="display: none"> @@ -563,27 +595,47 @@ <li class="toctree-l3"><a class="reference internal" href="/docs/supported-sql-commands/">Supported SQL Commands</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/alter-session-command/">ALTER SESSION Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/alter-session/">ALTER SESSION</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/alter-system/">ALTER SYSTEM</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/create-table-as-ctas/">CREATE TABLE AS (CTAS)</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/create-view/">CREATE VIEW</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/drop-view/">DROP VIEW</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/describe/">DESCRIBE</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/explain/">EXPLAIN</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/select/">SELECT</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/select-from/">SELECT FROM</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/select-group-by/">SELECT GROUP BY</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/select-having/">SELECT HAVING</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/alter-system-command/">ALTER SYSTEM Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-limit/">SELECT LIMIT</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/create-table-as-ctas-command/">CREATE TABLE AS (CTAS) Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-offset/">SELECT OFFSET</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/create-view-command/">CREATE VIEW Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-order-by/">SELECT ORDER BY</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/describe-command/">DESCRIBE Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-union/">SELECT UNION</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/explain-commands/">EXPLAIN Commands</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-where/">SELECT WHERE</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/select-statements/">SELECT Statements</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-with/">SELECT WITH</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/show-databases-and-show-schemas-command/">SHOW DATABASES and SHOW SCHEMAS Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/show-databases-and-show-schemas/">SHOW DATABASES and SHOW SCHEMAS</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/show-files-command/">SHOW FILES Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/show-files/">SHOW FILES</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/show-tables-command/">SHOW TABLES Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/show-tables/">SHOW TABLES</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/use-command/">USE Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/use/">USE</a></li> </ul> @@ -782,6 +834,10 @@ <li class="toctree-l2"><a class="reference internal" href="/docs/what-is-apache-drill/">What is Apache Drill</a></li> + + <li class="toctree-l2"><a class="reference internal" href="/docs/partition-pruning/">Partition Pruning</a></li> + + </ul> @@ -800,6 +856,29 @@ <li class="toctree-l1"><a class="reference internal" href="/docs/project-bylaws/">Project Bylaws</a></li> + + <li class="toctree-l1"><a href="javascript: void(0);">Log and Debug</a></li> + <ul style="display: none"> + + + <li class="toctree-l2"><a class="reference internal" href="/docs/log-and-debug-introduction/">Log and Debug Introduction</a></li> + + + + <li class="toctree-l2"><a class="reference internal" href="/docs/error-messages/">Error Messages</a></li> + + + + <li class="toctree-l2"><a class="reference internal" href="/docs/modify-logback-xml/">Modify logback.xml</a></li> + + + + <li class="toctree-l2"><a class="reference internal" href="/docs/review-the-java-stack-trace/">Review the Java Stack Trace</a></li> + + + </ul> + + </ul> </div> @@ -807,7 +886,7 @@ </aside> - <nav class="breadcrumbs"> +<nav class="breadcrumbs"> <li><a href="/docs/">Docs</a></li> @@ -818,23 +897,23 @@ <li>Configuring Multitenant Resources</li> </nav> - <div class="main-content-wrapper"> - <div class="main-content"> +<div class="main-content-wrapper"> + <div class="main-content"> - - <a class="edit-link" href="https://github.com/apache/drill/blob/gh-pages/_docs/configure-drill/050-configuring-multitenant-resources.md"; target="_blank"><i class="fa fa-pencil-square-o"></i></a> - + + <a class="edit-link" href="https://github.com/apache/drill/blob/gh-pages/_docs/configure-drill/050-configuring-multitenant-resources.md"; target="_blank"><i class="fa fa-pencil-square-o"></i></a> + - <div class="int_title"> - <h1>Configuring Multitenant Resources</h1> + <div class="int_title left"> + <h1>Configuring Multitenant Resources</h1> - </div> + </div> - <link href="/css/docpage.css" rel="stylesheet" type="text/css"> + <link href="/css/docpage.css" rel="stylesheet" type="text/css"> - <div class="int_text" align="left"> - - <p>Drill operations are memory and CPU-intensive. Currently, Drill resources are managed outside of any cluster management service. In a multitenant or any other type of cluster, YARN-enabled or not, you configure memory and memory usage limits for Drill by modifying the <code>drill-env.sh</code> file as described in <a href="/docs/configuring-drill-memory">"Configuring Drill Memory"</a>.</p> + <div class="int_text" align="left"> + + <p>Drill operations are memory and CPU-intensive. Currently, Drill resources are managed outside of any cluster management service. In a multitenant or any other type of cluster, YARN-enabled or not, you configure memory and memory usage limits for Drill by modifying the <code>drill-env.sh</code> file as described in <a href="/docs/configuring-drill-memory">"Configuring Drill Memory"</a>.</p> <p>Configure a multitenant cluster manager to account for resources required for Drill. Configuring <code>drill-env.sh</code> allocates resources for Drill to use during query execution. It might be necessary to configure the cluster manager from committing the resources to other processes.</p> @@ -873,17 +952,21 @@ There is one instance per node. </li> <p>Currently, you do not manage CPU resources within Drill. <a href="http://en.wikipedia.org/wiki/Cgroups";>Use Linux <code>cgroups</code></a> to manage the CPU resources.</p> +<h2 id="how-to-manage-disk-resources">How to Manage Disk Resources</h2> + +<p>The <code>planner.add_producer_consumer</code> system option enables or disables a secondary reading thread that works out of band of the rest of the scanning fragment to prefetch data from disk. If you interact with a certain type of storage medium that is slow or does not prefetch much data, this option tells Drill to add a producer consumer reading thread to the operation. Drill can then assign one thread that focuses on a single reading fragment. If Drill is using memory, you can disable this option to get better performance. If Drill is using disk space, you should enable this option and set a reasonable queue size for the <code>planner.producer_consumer_queue_size</code> option. For more information about these options, see the section, <a href="/docs/performance-tuning-introduction/">"Performance Tuning"</a>.</p> + + - - <div class="doc-nav"> + <div class="doc-nav"> <span class="previous-toc"><a href="/docs/configuring-a-multitenant-cluster-introduction/">â Configuring a Multitenant Cluster Introduction</a></span><span class="next-toc"><a href="/docs/configuring-resources-for-a-shared-drillbit/">Configuring Resources for a Shared Drillbit â</a></span> </div> - - </div> + </div> </div> +</div> </div> <p class="push"></p> http://git-wip-us.apache.org/repos/asf/drill-site/blob/ee640eab/docs/configuring-resources-for-a-shared-drillbit/index.html ---------------------------------------------------------------------- diff --git a/docs/configuring-resources-for-a-shared-drillbit/index.html b/docs/configuring-resources-for-a-shared-drillbit/index.html index 094e6b4..bc5fd58 100644 --- a/docs/configuring-resources-for-a-shared-drillbit/index.html +++ b/docs/configuring-resources-for-a-shared-drillbit/index.html @@ -9,26 +9,21 @@ <title>Configuring Resources for a Shared Drillbit - Apache Drill</title> -<link href="/css/syntax.css" rel="stylesheet" type="text/css"> -<link href="/css/style.css" rel="stylesheet" type="text/css"> -<link href="/css/arrows.css" rel="stylesheet" type="text/css"> -<link href="/css/breadcrumbs.css" rel="stylesheet" type="text/css"> -<link href="/css/code.css" rel="stylesheet" type="text/css"> -<link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/font-awesome/4.3.0/css/font-awesome.min.css"> -<link href="/css/responsive.css" rel="stylesheet" type="text/css"> - -<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"> -<link rel="icon" href="/favicon.ico" type="image/x-icon"> - -<script language="javascript" type="text/javascript" src="/js/lib/jquery-1.11.1.min.js"></script> -<script language="javascript" type="text/javascript" src="/js/lib/jquery.easing.1.3.js"></script> +<link href="//maxcdn.bootstrapcdn.com/font-awesome/4.3.0/css/font-awesome.min.css" rel="stylesheet" type="text/css"/> +<link href="/css/site.css" rel="stylesheet" type="text/css"/> + +<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"/> +<link rel="icon" href="/favicon.ico" type="image/x-icon"/> + +<script src="//ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js" language="javascript" type="text/javascript"></script> +<script src="//cdnjs.cloudflare.com/ajax/libs/jquery-easing/1.3/jquery.easing.min.js" language="javascript" type="text/javascript"></script> <script language="javascript" type="text/javascript" src="/js/modernizr.custom.js"></script> <script language="javascript" type="text/javascript" src="/js/script.js"></script> <script language="javascript" type="text/javascript" src="/js/drill.js"></script> - </head> + <body onResize="resized();"> <div class="page-wrap"> <div class="bui"></div> @@ -63,6 +58,12 @@ <li><a href="/docs/query-data/">Query Data</a></li> + <li><a href="/docs/performance-tuning/">Performance Tuning</a></li> + + <li><a href="/docs/log-and-debug/">Log and Debug</a></li> + + <li><a href="/docs/query-audit-logging/">Query Audit Logging</a></li> + <li><a href="/docs/sql-reference/">SQL Reference</a></li> <li><a href="/docs/data-sources-and-file-formats/">Data Sources and File Formats</a></li> @@ -81,6 +82,8 @@ <li><a href="/docs/project-bylaws/">Project Bylaws</a></li> + <li><a href="/docs/log-and-debug/">Log and Debug</a></li> + </ul> </li> <li class='nav'> @@ -110,7 +113,9 @@ </ul> </div> - + <link href="/css/content.css" rel="stylesheet" type="text/css"> + + @@ -149,14 +154,7 @@ - <li class="toctree-l2"><a href="javascript: void(0);">Architectural Highlights</a></li> - <ul style="display: none"> - - <li class="toctree-l3"><a class="reference internal" href="/docs/flexibility/">Flexibility</a></li> - - <li class="toctree-l3"><a class="reference internal" href="/docs/performance/">Performance</a></li> - - </ul> + <li class="toctree-l2"><a class="reference internal" href="/docs/performance/">Performance</a></li> </ul> @@ -203,6 +201,10 @@ <li class="toctree-l2"><a class="reference internal" href="/docs/analyzing-highly-dynamic-datasets/">Analyzing Highly Dynamic Datasets</a></li> + + <li class="toctree-l2"><a class="reference internal" href="/docs/analyzing-social-media/">Analyzing Social Media</a></li> + + </ul> @@ -300,7 +302,7 @@ - <li class="toctree-l2"><a class="reference internal" href="/docs/partition-pruning/">Partition Pruning</a></li> + <li class="toctree-l2"><a class="reference internal" href="/docs/configuring-the-drill-shell/">Configuring the Drill Shell</a></li> </ul> @@ -357,7 +359,7 @@ - <li class="toctree-l2"><a class="reference internal" href="/docs/using-jdbc/">Using JDBC</a></li> + <li class="toctree-l2"><a class="reference internal" href="/docs/using-jdbc-with-squirrel-on-windows/">Using JDBC with SQuirreL on Windows</a></li> @@ -409,6 +411,10 @@ + <li class="toctree-l2"><a class="reference internal" href="/docs/configuring-tibco-spotfire-server-with-drill/">Configuring Tibco Spotfire Server with Drill</a></li> + + + <li class="toctree-l2"><a class="reference internal" href="/docs/using-apache-drill-with-tableau-9-desktop/">Using Apache Drill with Tableau 9 Desktop</a></li> @@ -487,6 +493,32 @@ + <li class="toctree-l1"><a href="javascript: void(0);">Performance Tuning</a></li> + <ul style="display: none"> + + + <li class="toctree-l2"><a class="reference internal" href="/docs/performance-tuning-introduction/">Performance Tuning Introduction</a></li> + + + </ul> + + + + <li class="toctree-l1"><a class="reference internal" href="/docs/log-and-debug/">Log and Debug</a></li> + + + + <li class="toctree-l1"><a href="javascript: void(0);">Query Audit Logging</a></li> + <ul style="display: none"> + + + <li class="toctree-l2"><a class="reference internal" href="/docs/getting-query-information/">Getting Query Information</a></li> + + + </ul> + + + <li class="toctree-l1"><a href="javascript: void(0);">SQL Reference</a></li> <ul style="display: none"> @@ -563,27 +595,47 @@ <li class="toctree-l3"><a class="reference internal" href="/docs/supported-sql-commands/">Supported SQL Commands</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/alter-session-command/">ALTER SESSION Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/alter-session/">ALTER SESSION</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/alter-system/">ALTER SYSTEM</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/create-table-as-ctas/">CREATE TABLE AS (CTAS)</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/create-view/">CREATE VIEW</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/drop-view/">DROP VIEW</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/describe/">DESCRIBE</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/explain/">EXPLAIN</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/alter-system-command/">ALTER SYSTEM Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select/">SELECT</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/create-table-as-ctas-command/">CREATE TABLE AS (CTAS) Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-from/">SELECT FROM</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/create-view-command/">CREATE VIEW Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-group-by/">SELECT GROUP BY</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/describe-command/">DESCRIBE Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-having/">SELECT HAVING</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/explain-commands/">EXPLAIN Commands</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-limit/">SELECT LIMIT</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/select-statements/">SELECT Statements</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-offset/">SELECT OFFSET</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/show-databases-and-show-schemas-command/">SHOW DATABASES and SHOW SCHEMAS Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-order-by/">SELECT ORDER BY</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/show-files-command/">SHOW FILES Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-union/">SELECT UNION</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/show-tables-command/">SHOW TABLES Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-where/">SELECT WHERE</a></li> - <li class="toctree-l3"><a class="reference internal" href="/docs/use-command/">USE Command</a></li> + <li class="toctree-l3"><a class="reference internal" href="/docs/select-with/">SELECT WITH</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/show-databases-and-show-schemas/">SHOW DATABASES and SHOW SCHEMAS</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/show-files/">SHOW FILES</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/show-tables/">SHOW TABLES</a></li> + + <li class="toctree-l3"><a class="reference internal" href="/docs/use/">USE</a></li> </ul> @@ -782,6 +834,10 @@ <li class="toctree-l2"><a class="reference internal" href="/docs/what-is-apache-drill/">What is Apache Drill</a></li> + + <li class="toctree-l2"><a class="reference internal" href="/docs/partition-pruning/">Partition Pruning</a></li> + + </ul> @@ -800,6 +856,29 @@ <li class="toctree-l1"><a class="reference internal" href="/docs/project-bylaws/">Project Bylaws</a></li> + + <li class="toctree-l1"><a href="javascript: void(0);">Log and Debug</a></li> + <ul style="display: none"> + + + <li class="toctree-l2"><a class="reference internal" href="/docs/log-and-debug-introduction/">Log and Debug Introduction</a></li> + + + + <li class="toctree-l2"><a class="reference internal" href="/docs/error-messages/">Error Messages</a></li> + + + + <li class="toctree-l2"><a class="reference internal" href="/docs/modify-logback-xml/">Modify logback.xml</a></li> + + + + <li class="toctree-l2"><a class="reference internal" href="/docs/review-the-java-stack-trace/">Review the Java Stack Trace</a></li> + + + </ul> + + </ul> </div> @@ -807,7 +886,7 @@ </aside> - <nav class="breadcrumbs"> +<nav class="breadcrumbs"> <li><a href="/docs/">Docs</a></li> @@ -818,23 +897,23 @@ <li>Configuring Resources for a Shared Drillbit</li> </nav> - <div class="main-content-wrapper"> - <div class="main-content"> +<div class="main-content-wrapper"> + <div class="main-content"> - - <a class="edit-link" href="https://github.com/apache/drill/blob/gh-pages/_docs/configure-drill/060-configuring-a-shared-drillbit.md"; target="_blank"><i class="fa fa-pencil-square-o"></i></a> - + + <a class="edit-link" href="https://github.com/apache/drill/blob/gh-pages/_docs/configure-drill/060-configuring-a-shared-drillbit.md"; target="_blank"><i class="fa fa-pencil-square-o"></i></a> + - <div class="int_title"> - <h1>Configuring Resources for a Shared Drillbit</h1> + <div class="int_title left"> + <h1>Configuring Resources for a Shared Drillbit</h1> - </div> + </div> - <link href="/css/docpage.css" rel="stylesheet" type="text/css"> + <link href="/css/docpage.css" rel="stylesheet" type="text/css"> - <div class="int_text" align="left"> - - <p>To manage a cluster in which multiple users share a Drillbit, you configure Drill queuing and parallelization in addition to memory, as described in the previous section, <a href="/docs/configuring-drill-memory/">"Configuring Drill Memory"</a>.</p> + <div class="int_text" align="left"> + + <p>To manage a cluster in which multiple users share a Drillbit, you configure Drill queuing and parallelization in addition to memory, as described in the previous section, <a href="/docs/configuring-drill-memory/">"Configuring Drill Memory"</a>.</p> <h2 id="configuring-query-queuing">Configuring Query Queuing</h2> @@ -845,22 +924,7 @@ <li>exec.queue.small<br></li> </ul> -<h3 id="example-configuration">Example Configuration</h3> - -<p>For example, you configure the queue reserved for large queries for a 5-query maximum. You configure the queue reserved for small queries for 20 queries. Users start to run queries, and Drill receives the following query requests in this order:</p> - -<ul> -<li>Query A (blue): 1 billion records, Drill estimates 10 million rows will be processed<br></li> -<li>Query B (red): 2 billion records, Drill estimates 20 million rows will be processed<br></li> -<li>Query C: 1 billion records<br></li> -<li>Query D: 100 records</li> -</ul> - -<p>The exec.queue.threshold default is 30 million, which is the estimated rows to be processed by the query. Queries A and B are queued in the large queue. The estimated rows to be processed reaches the 30 million threshold, filling the queue to capacity. The query C request arrives and goes on the wait list, and then query D arrives. Query D is queued immediately in the small queue because of its small size, as shown in the following diagram: </p> - -<p><img src="/docs/img/queuing.png" alt="drill queuing"></p> - -<p>The Drill queuing configuration in this example tends to give many users running small queries a rapid response. Users running a large query might experience some delay until an earlier-received large query returns, freeing space in the large queue to process queries that are waiting.</p> +<p>For more information, see the section, <a href="/docs/performance-tuning-introduction/">"Performance Tuning"</a>.</p> <h2 id="configuring-parallelization">Configuring Parallelization</h2> @@ -877,7 +941,23 @@ The maximum degree of distribution of a query across cores and cluster nodes.</l Same as max per node but applies to the query as executed by the entire cluster.</li> </ul> -<p>Configure the <code>planner.width.max.per.node</code> to achieve fine grained, absolute control over parallelization. </p> +<h3 id="planner.width.max_per_node">planner.width.max_per_node</h3> + +<p>Configure the <code>planner.width.max.per.node</code> to achieve fine grained, absolute control over parallelization. In this context <em>width</em> refers to fanout or distribution potential: the ability to run a query in parallel across the cores on a node and the nodes on a cluster. A physical plan consists of intermediate operations, known as query "fragments," that run concurrently, yielding opportunities for parallelism above and below each exchange operator in the plan. An exchange operator represents a breakpoint in the execution flow where processing can be distributed. For example, a single-process scan of a file may flow into an exchange operator, followed by a multi-process aggregation fragment.</p> + +<p>The maximum width per node defines the maximum degree of parallelism for any fragment of a query, but the setting applies at the level of a single node in the cluster. The <em>default</em> maximum degree of parallelism per node is calculated as follows, with the theoretical maximum automatically scaled back (and rounded down) so that only 70% of the actual available capacity is taken into account: number of active drillbits (typically one per node) * number of cores per node * 0.7</p> + +<p>For example, on a single-node test system with 2 cores and hyper-threading enabled: 1 * 4 * 0.7 = 3</p> + +<p>When you modify the default setting, you can supply any meaningful number. The system does not automatically scale down your setting.</p> + +<h3 id="planner.width.max_per_query">planner.width.max_per_query</h3> + +<p>The max_per_query value also sets the maximum degree of parallelism for any given stage of a query, but the setting applies to the query as executed by the whole cluster (multiple nodes). In effect, the actual maximum width per query is the <em>minimum of two values</em>: min((number of nodes * width.max_per_node), width.max_per_query)</p> + +<p>For example, on a 4-node cluster where <code>width.max_per_node</code> is set to 6 and <code>width.max_per_query</code> is set to 30: min((4 * 6), 30) = 24</p> + +<p>In this case, the effective maximum width per query is 24, not 30.</p> <!-- ??For example, setting the `planner.width.max.per.query` to 60 will not accelerate Drill operations because overlapping does not occur when executing 60 queries at the same time.?? @@ -893,17 +973,17 @@ A parallelizer in the Foreman transforms the physical plan into multiple phases. <p>Tenants can share data on a cluster using Drill views and <a href="/docs/configuring-user-impersonation">impersonation</a>. </p> + - - <div class="doc-nav"> + <div class="doc-nav"> <span class="previous-toc"><a href="/docs/configuring-multitenant-resources/">â Configuring Multitenant Resources</a></span><span class="next-toc"><a href="/docs/configuring-user-impersonation/">Configuring User Impersonation â</a></span> </div> - - </div> + </div> </div> +</div> </div> <p class="push"></p>
