[
https://issues.apache.org/jira/browse/DRILL-5735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16512193#comment-16512193
]
ASF GitHub Bot commented on DRILL-5735:
---------------------------------------
arina-ielchiieva commented on a change in pull request #1279: DRILL-5735: Allow
search/sort in the Options webUI
URL: https://github.com/apache/drill/pull/1279#discussion_r195345439
##########
File path: exec/java-exec/src/main/resources/options.describe.properties
##########
@@ -0,0 +1,106 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+### Descriptions of all available options
+drill.exec.default_temporary_workspace=Available as of Drill 1.10. Sets the
workspace for temporary tables. The workspace must be writable, file-based, and
point to a location that already exists.
+drill.exec.http.jetty.server.acceptors=Available as of Drill 1.13. HTTP
connector option that limits the number of worker threads dedicated to
accepting connections. Limiting the number of acceptors also limits the number
threads needed.
+drill.exec.http.jetty.server.selectors=Available as of Drill1.13. HTTP
connector option that limits the number of worker threads dedicated to sending
and receiving data. Limiting the number of selectors also limits the number
threads needed.
+drill.exec.memory.operator.output_batch_size=Available as of Drill 1.13.
Limits the amount of memory that the Flatten, Merge Join, and External Sort
operators allocate to outgoing batches.
+drill.exec.storage.implicit.filename.column.label=Available as of Drill 1.10.
Sets the implicit column name for the filename column.
+drill.exec.storage.implicit.filepath.column.label=Available as of Drill 1.10.
Sets the implicit column name for the filepath column.
+drill.exec.storage.implicit.fqn.column.label=Available as of Drill 1.10. Sets
the implicit column name for the fqn column.
+drill.exec.storage.implicit.suffix.column.label=Available as of Drill 1.10.
Sets the implicit column name for the suffix column.
+drill.exec.functions.cast_empty_string_to_null=In a text file, treat empty
fields as NULL values instead of empty string.
+drill.exe.spill.fs=Introduced in Drill 1.11. The default file system on the
local machine into which the Sort, Hash Aggregate, and Hash Join operators
spill data.
+drill.exec.spill.directories=Introduced in Drill 1.11. The list of directories
into which the Sort, Hash Aggregate, and Hash Join operators spill data. The
list must be an array with directories separated by a comma, for example
["/fs1/drill/spill" , "/fs2/drill/spill" , "/fs3/drill/spill"].
+drill.exec.storage.file.partition.column.label=The column label for directory
levels in results of queries of files in a directory. Accepts a string input.
+exec.enable_union_type=Enable support for Avro union type.
+exec.errors.verbose=Toggles verbose output of executable error messages
+exec.java_compiler=Switches between DEFAULT, JDK, and JANINO mode for the
current session. Uses Janino by default for generated source code of less than
exec.java_compiler_janino_maxsize; otherwise, switches to the JDK compiler.
+exec.java_compiler_debug=Toggles the output of debug-level compiler error
messages in runtime generated code.
+exec.java.compiler.exp_in_method_size=Introduced in Drill 1.8. For queries
with complex or multiple expressions in the query logic, this option limits the
number of expressions allowed in each method to prevent Drill from generating
code that exceeds the Java limit of 64K bytes. If a method approaches the 64K
limit, the Java compiler returns a message stating that the code is too large
to compile. If queries return such a message, reduce the value of this option
at the session level. The default value for this option is 50. The value is the
count of expressions allowed in a method. Expressions are added to a method
until they hit the Java 64K limit, when a new inner method is created and
called from the existing method. Note: This logic has not been implemented for
all operators. If a query uses operators for which the logic is not
implemented, reducing the setting for this option may not resolve the error.
Setting this option at the system level impacts all queries and can degrade
query performance.
+exec.java_compiler_janino_maxsize=See the exec.java_compiler option comment.
Accepts inputs of type LONG.
+exec.max_hash_table_size=Ending size in buckets for hash tables. Range: 0 -
1073741824.
+exec.min_hash_table_size=Starting size in bucketsfor hash tables. Increase
according to available memory to improve performance. Increasing for very large
aggregations or joins when you have large amounts of memory for Drill to use.
Range: 0 - 1073741824.
+exec.queue.enable=Changes the state of query queues. False allows unlimited
concurrent queries.
+exec.queue.large=Sets the number of large queries that can run concurrently in
the cluster. Range: 0-1000
+exec.queue.small=Sets the number of small queries that can run concurrently in
the cluster. Range: 0-1001
+exec.queue.threshold=Sets the cost threshold, which depends on the complexity
of the queries in queue, for determining whether query is large or small.
Complex queries have higher thresholds. Range: 0-9223372036854775807
+exec.queue.timeout_millis=Indicates how long a query can wait in queue before
the query fails. Range: 0-9223372036854775807
+exec.schedule.assignment.old=Used to prevent query failure when no work units
are assigned to a minor fragment, particularly when the number of files is much
larger than the number of leaf fragments.
+exec.storage.enable_new_text_reader=Enables the text reader that complies with
the RFC 4180 standard for text/csv files.
+new_view_default_permissions=Sets view permissions using an octal code in the
Unix tradition.
+planner.add_producer_consumer=Increase prefetching of data from disk. Disable
for in-memory reads.
+planner.affinity_factor=Factor by which a node with endpoint affinity is
favored while creating assignment. Accepts inputs of type DOUBLE.
+planner.broadcast_factor=A heuristic parameter for influencing the broadcast
of records as part of a query.
+planner.broadcast_threshold=The maximum number of records allowed to be
broadcast as part of a query. After one million records, Drill reshuffles data
rather than doing a broadcast to one side of the join. Range: 0-2147483647
+planner.disable_exchanges=Toggles the state of hashing to a random exchange.
+planner.enable_broadcast_join=Changes the state of aggregation and join
operators. The broadcast join can be used for hash join, merge join and nested
loop join. Use to join a large (fact) table to relatively smaller (dimension)
tables. Do not disable.
+planner.enable_constant_folding=If one side of a filter condition is a
constant expression, constant folding evaluates the expression in the planning
phase and replaces the expression with the constant value. For example, Drill
can rewrite WHERE age + 5 < 42 as WHERE age < 37.
+planner.enable_decimal_data_type=False disables the DECIMAL data type,
including casting to DECIMAL and reading DECIMAL types from Parquet and Hive.
+planner.enable_demux_exchange=Toggles the state of hashing to a demulitplexed
exchange.
+planner.enable_hash_single_key=Each hash key is associated with a single value.
+planner.enable_hashagg=Enable hash aggregation; otherwise, Drill does a
sort-based aggregation. Writes to disk. Enable is recommended.
+planner.enable_hashjoin=Enable the memory hungry hash join. Drill assumes that
a query will have adequate memory to complete and tries to use the fastest
operations possible to complete the planned inner, left, right, or full outer
joins using a hash table. Does not write to disk. Disabling hash join allows
Drill to manage arbitrarily large data in a small memory footprint.
+planner.enable_hashjoin_swap=Enables consideration of multiple join order
sequences during the planning phase. Might negatively affect the performance of
some queries due to inaccuracy of estimated row count especially after a
filter, join, or aggregation.
+planner.enable_hep_join_opt=Enables the heuristic planner for joins.
+planner.enable_mergejoin=Sort-based operation. A merge join is used for inner
join, left and right outer joins. Inputs to the merge join must be sorted. It
reads the sorted input streams from both sides and finds matching rows. Writes
to disk.
+planner.enable_multiphase_agg=Each minor fragment does a local aggregation in
phase 1, distributes on a hash basis using GROUP-BY keys partially aggregated
results to other fragments, and all the fragments perform a total aggregation
using this data.
+planner.enable_mux_exchange=Toggles the state of hashing to a multiplexed
exchange.
+planner.enable_nestedloopjoin=Sort-based operation. Writes to disk.
+planner.enable_nljoin_for_scalar_only=Supports nested loop join planning where
the right input is scalar in order to enable NOT-IN, Inequality, Cartesian, and
uncorrelated EXISTS planning.
+planner.enable_streamagg=Sort-based operation. Writes to disk.
+planner.filter.max_selectivity_estimate_factor=Available as of Drill 1.8. Sets
the maximum filter selectivity estimate. The selectivity can vary between 0 and
1. For more details, see planner.filter.min_selectivity_estimate_factor.
+planner.filter.min_selectivity_estimate_factor=Available as of Drill 1.8. Sets
the minimum filter selectivity estimate to increase the parallelization of the
major fragment performing a join. This option is useful for deeply nested
queries with complicated predicates and serves as a workaround when statistics
are insufficient or unavailable. The selectivity can vary between 0 and 1. The
value of this option caps the estimated SELECTIVITY. The estimated ROWCOUNT is
derived by multiplying the estimated SELECTIVITY by the estimated ROWCOUNT of
the upstream operator. The estimated ROWCOUNT displays when you use the EXPLAIN
PLAN INCLUDING ALL ATTRIBUTES FOR command. This option does not control the
estimated ROWCOUNT of downstream operators (post FILTER). However, estimated
ROWCOUNTs may change because the operator ROWCOUNTs depend on their downstream
operators. The FILTER operator relies on the input of its immediate upstream
operator, for example SCAN, AGGREGATE. If two filters are present in a plan,
each filter may have a different estimated ROWCOUNT based on the immediate
upstream operator\'s estimated ROWCOUNT.
+planner.identifier_max_length=A minimum length is needed because option names
are identifiers themselves.
+planner.join.hash_join_swap_margin_factor=The number of join order sequences
to consider during the planning phase.
+planner.join.row_count_estimate_factor=The factor for adjusting the estimated
row count when considering multiple join order sequences during the planning
phase.
+planner.memory.average_field_width=Used in estimating memory requirements.
+planner.memory.enable_memory_estimation=Toggles the state of memory estimation
and re-planning of the query. When enabled, Drill conservatively estimates
memory requirements and typically excludes these operators from the plan and
negatively impacts performance.
+planner.memory.hash_agg_table_factor=A heuristic value for influencing the
size of the hash aggregation table.
+planner.memory.hash_join_table_factor=A heuristic value for influencing the
size of the hash aggregation table.
+planner.memory.max_query_memory_per_node=Sets the maximum amount of direct
memory allocated to the Sort and Hash Aggregate operators during each query on
a node. This memory is split between operators. If a query plan contains
multiple Sort and/or Hash Aggregate operators, the memory is divided between
them. The default limit should be increased for queries on large data sets.
+planner.memory.non_blocking_operators_memory=Extra query memory per node for
non-blocking operators. This option is currently used only for memory
estimation. Range: 0-2048 MB
+planner.memory_limit=Defines the maximum amount of direct memory allocated to
a query for planning. When multiple queries run concurrently, each query is
allocated the amount of memory set by this parameter.Increase the value of this
parameter and rerun the query if partition pruning failed due to insufficient
memory.
+planner.memory.percent_per_query=Sets the memory as a percentage of the total
direct memory.
+planner.nestedloopjoin_factor=A heuristic value for influencing the nested
loop join.
+planner.partitioner_sender_max_threads=Upper limit of threads for outbound
queuing.
+planner.partitioner_sender_set_threads=Overwrites the number of threads used
to send out batches of records. Set to -1 to disable. Typically not changed.
+planner.partitioner_sender_threads_factor=A heuristic param to use to
influence final number of threads. The higher the value the fewer the number of
threads.
+planner.producer_consumer_queue_size=How much data to prefetch from disk in
record batches out-of-band of query execution. The larger the queue size, the
greater the amount of memory that the queue and overall query execution
consumes.
+planner.slice_target=The number of records manipulated within a fragment
before Drill parallelizes operations.
+planner.width.max_per_node=Maximum number of threads that can run in parallel
for a query on a node. A slice is an individual thread. This number indicates
the maximum number of slices per query for the query’s major fragment on a node.
+planner.width.max_per_query=Same as max per node but applies to the query as
executed by the entire cluster. For example, this value might be the number of
active Drillbits, or a higher number to return results faster.
+security.admin.user_groups=Unsupported as of 1.4. A comma-separated list of
administrator groups for Web Console security.
+security.admin.users=Unsupported as of 1.4. A comma-separated list of user
names who you want to give administrator privileges.
+store.format=Output format for data written to tables with the CREATE TABLE AS
(CTAS) command. Allowed values are parquet, json, psv, csv, or tsv.
+store.hive.optimize_scan_with_native_readers=Optimize reads of Parquet-backed
external tables from Hive by using Drill native readers instead of the Hive
Serde interface. (Drill 1.2 and later)
+store.json.all_text_mode=Drill reads all data from the JSON files as VARCHAR.
Prevents schema change errors.
+store.json.extended_types=Turns on special JSON structures that Drill
serializes for storing more type information than the four basic JSON types.
+store.json.read_numbers_as_double=Reads numbers with or without a decimal
point as DOUBLE. Prevents schema change errors.
+store.mongo.all_text_mode=Similar to store.json.all_text_mode for MongoDB.
+store.mongo.read_numbers_as_double=Similar to
store.json.read_numbers_as_double.
+store.parquet.block-size=Sets the size of a Parquet row group to the number of
bytes less than or equal to the block size of MFS, HDFS, or the file system.
+store.parquet.compression=Compression type for storing Parquet output. Allowed
values: snappy, gzip, none
+store.parquet.enable_dictionary_encoding=For internal use. Do not change.
+store.parquet.dictionary.page-size=
Review comment:
description?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> UI options grouping and filtering & Metrics hints
> -------------------------------------------------
>
> Key: DRILL-5735
> URL: https://issues.apache.org/jira/browse/DRILL-5735
> Project: Apache Drill
> Issue Type: Improvement
> Components: Web Server
> Affects Versions: 1.9.0, 1.10.0, 1.11.0
> Reporter: Muhammad Gelbana
> Assignee: Kunal Khatua
> Priority: Major
> Fix For: 1.14.0
>
>
> I'm thinking of some UI improvements that could make all the difference for
> users trying to optimize low-performing queries.
> h2. Options
> h3. Grouping
> We can organize the options to be grouped by their scope of effect, this will
> help users easily locate the options they may need to tune.
> h3. Filtering
> Since the options are a lot, we can add a filtering mechanism (i.e. string
> search or group\scope filtering) so the user can filter out the options he's
> not interested in. To provide more benefit than the grouping idea mentioned
> above, filtering may include keywords also and not just the option name,
> since the user may not be aware of the name of the option he's looking for.
> h2. Metrics
> I'm referring here to the metrics page and the query execution plan page that
> displays the overview section and major\minor fragments metrics. We can show
> hints for each metric such as:
> # What does it represent in more details.
> # What option\scope-of-options to tune (increase ? decrease ?) to improve the
> performance reported by this metric.
> # May be even provide a small dialog to quickly allow the modification of the
> related option(s) to that metric
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
