[CARBONDATA-2818] Upgrade presto integration version to 0.210 Upgrade presto integration version to 0.210
This closes #2733 Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/8427771f Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/8427771f Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/8427771f Branch: refs/heads/branch-1.5 Commit: 8427771fc22071099186d3310500d5fcd6c419a6 Parents: 629d625 Author: chenliang613 <[email protected]> Authored: Wed Sep 19 08:18:28 2018 +0800 Committer: Raghunandan S <[email protected]> Committed: Fri Sep 28 11:39:48 2018 +0530 ---------------------------------------------------------------------- docs/quick-start-guide.md | 29 ++++++++++++-------- .../Presto_Cluster_Setup_For_Carbondata.md | 24 ++++++++-------- integration/presto/README.md | 12 +++++--- integration/presto/pom.xml | 4 +-- 4 files changed, 40 insertions(+), 29 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/carbondata/blob/8427771f/docs/quick-start-guide.md ---------------------------------------------------------------------- diff --git a/docs/quick-start-guide.md b/docs/quick-start-guide.md index 37c398c..0fdf055 100644 --- a/docs/quick-start-guide.md +++ b/docs/quick-start-guide.md @@ -300,24 +300,24 @@ Once the table is created,it can be queried from Presto.** ### Installing Presto - 1. Download the 0.187 version of Presto using: - `wget https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.187/presto-server-0.187.tar.gz` + 1. Download the 0.210 version of Presto using: + `wget https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.210/presto-server-0.210.tar.gz` - 2. Extract Presto tar file: `tar zxvf presto-server-0.187.tar.gz`. + 2. Extract Presto tar file: `tar zxvf presto-server-0.210.tar.gz`. 3. Download the Presto CLI for the coordinator and name it presto. ``` - wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.187/presto-cli-0.187-executable.jar + wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.210/presto-cli-0.210-executable.jar - mv presto-cli-0.187-executable.jar presto + mv presto-cli-0.210-executable.jar presto chmod +x presto ``` ### Create Configuration Files - 1. Create `etc` folder in presto-server-0.187 directory. + 1. Create `etc` folder in presto-server-0.210 directory. 2. Create `config.properties`, `jvm.config`, `log.properties`, and `node.properties` files. 3. Install uuid to generate a node.id. @@ -363,10 +363,15 @@ Once the table is created,it can be queried from Presto.** coordinator=true node-scheduler.include-coordinator=false http-server.http.port=8086 - query.max-memory=50GB - query.max-memory-per-node=2GB + query.max-memory=5GB + query.max-total-memory-per-node=5GB + query.max-memory-per-node=3GB + memory.heap-headroom-per-node=1GB discovery-server.enabled=true - discovery.uri=<coordinator_ip>:8086 + discovery.uri=http://localhost:8086 + task.max-worker-threads=4 + optimizer.dictionary-aggregation=true + optimizer.optimize-hash-generation = false ``` The options `node-scheduler.include-coordinator=false` and `coordinator=true` indicate that the node is the coordinator and tells the coordinator not to do any of the computation work itself and to use the workers. @@ -383,7 +388,7 @@ Then, `query.max-memory=<30GB * number of nodes>`. ``` coordinator=false http-server.http.port=8086 - query.max-memory=50GB + query.max-memory=5GB query.max-memory-per-node=2GB discovery.uri=<coordinator_ip>:8086 ``` @@ -405,12 +410,12 @@ Then, `query.max-memory=<30GB * number of nodes>`. ### Start Presto Server on all nodes ``` -./presto-server-0.187/bin/launcher start +./presto-server-0.210/bin/launcher start ``` To run it as a background process. ``` -./presto-server-0.187/bin/launcher run +./presto-server-0.210/bin/launcher run ``` To run it in foreground. http://git-wip-us.apache.org/repos/asf/carbondata/blob/8427771f/integration/presto/Presto_Cluster_Setup_For_Carbondata.md ---------------------------------------------------------------------- diff --git a/integration/presto/Presto_Cluster_Setup_For_Carbondata.md b/integration/presto/Presto_Cluster_Setup_For_Carbondata.md index 082b8fe..975e1fa 100644 --- a/integration/presto/Presto_Cluster_Setup_For_Carbondata.md +++ b/integration/presto/Presto_Cluster_Setup_For_Carbondata.md @@ -2,24 +2,24 @@ ## Installing Presto - 1. Download the 0.187 version of Presto using: - `wget https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.187/presto-server-0.187.tar.gz` + 1. Download the 0.210 version of Presto using: + `wget https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.210/presto-server-0.210.tar.gz` - 2. Extract Presto tar file: `tar zxvf presto-server-0.187.tar.gz`. + 2. Extract Presto tar file: `tar zxvf presto-server-0.210.tar.gz`. 3. Download the Presto CLI for the coordinator and name it presto. ``` - wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.187/presto-cli-0.187-executable.jar + wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.210/presto-cli-0.210-executable.jar - mv presto-cli-0.187-executable.jar presto + mv presto-cli-0.210-executable.jar presto chmod +x presto ``` ## Create Configuration Files - 1. Create `etc` folder in presto-server-0.187 directory. + 1. Create `etc` folder in presto-server-0.210 directory. 2. Create `config.properties`, `jvm.config`, `log.properties`, and `node.properties` files. 3. Install uuid to generate a node.id. @@ -65,8 +65,10 @@ coordinator=true node-scheduler.include-coordinator=false http-server.http.port=8086 - query.max-memory=50GB - query.max-memory-per-node=2GB + query.max-memory=5GB + query.max-total-memory-per-node=5GB + query.max-memory-per-node=3GB + memory.heap-headroom-per-node=1GB discovery-server.enabled=true discovery.uri=<coordinator_ip>:8086 ``` @@ -85,7 +87,7 @@ Then, `query.max-memory=<30GB * number of nodes>`. ``` coordinator=false http-server.http.port=8086 - query.max-memory=50GB + query.max-memory=5GB query.max-memory-per-node=2GB discovery.uri=<coordinator_ip>:8086 ``` @@ -107,12 +109,12 @@ Then, `query.max-memory=<30GB * number of nodes>`. ## Start Presto Server on all nodes ``` -./presto-server-0.187/bin/launcher start +./presto-server-0.210/bin/launcher start ``` To run it as a background process. ``` -./presto-server-0.187/bin/launcher run +./presto-server-0.210/bin/launcher run ``` To run it in foreground. http://git-wip-us.apache.org/repos/asf/carbondata/blob/8427771f/integration/presto/README.md ---------------------------------------------------------------------- diff --git a/integration/presto/README.md b/integration/presto/README.md index 5a44f5c..ceab0fe 100644 --- a/integration/presto/README.md +++ b/integration/presto/README.md @@ -18,7 +18,7 @@ Please follow the below steps to query carbondata in presto ### Config presto server -* Download presto server (0.187 is suggested and supported) : https://repo1.maven.org/maven2/com/facebook/presto/presto-server/ +* Download presto server (0.210 is suggested and supported) : https://repo1.maven.org/maven2/com/facebook/presto/presto-server/ * Finish presto configuration following https://prestodb.io/docs/current/installation/deployment.html. A configuration example: ``` @@ -27,10 +27,14 @@ Please follow the below steps to query carbondata in presto node-scheduler.include-coordinator=true http-server.http.port=8086 query.max-memory=5GB - query.max-memory-per-node=1GB + query.max-total-memory-per-node=5GB + query.max-memory-per-node=3GB + memory.heap-headroom-per-node=1GB discovery-server.enabled=true discovery.uri=http://localhost:8086 - reorder-joins=true + task.max-worker-threads=4 + optimizer.dictionary-aggregation=true + optimizer.optimize-hash-generation = false jvm.config: @@ -131,7 +135,7 @@ Load data statement in Spark can be used to create carbondata tables. And then y carbondata files. ### Query carbondata in CLI of presto -* Download presto cli client following: https://prestodb.io/docs/current/installation/cli.html +* Download presto cli client of version 0.210 : https://repo1.maven.org/maven2/com/facebook/presto/presto-cli * Start CLI: http://git-wip-us.apache.org/repos/asf/carbondata/blob/8427771f/integration/presto/pom.xml ---------------------------------------------------------------------- diff --git a/integration/presto/pom.xml b/integration/presto/pom.xml index 9a9bc55..72847e2 100644 --- a/integration/presto/pom.xml +++ b/integration/presto/pom.xml @@ -31,7 +31,7 @@ <packaging>presto-plugin</packaging> <properties> - <presto.version>0.208</presto.version> + <presto.version>0.210</presto.version> <dev.path>${basedir}/../../dev</dev.path> <jacoco.append>true</jacoco.append> </properties> @@ -455,7 +455,7 @@ <groupId>org.antlr</groupId> <artifactId>antlr4-runtime</artifactId> <scope>test</scope> - <version>4.5.3</version> + <version>4.7.1</version> </dependency> <dependency> <groupId>com.google.code.findbugs</groupId>
