[bigtop] branch cnb updated: BIGTOP-3242: Add instructions on configuring kubectl for local k8s cluster
This is an automated email from the ASF dual-hosted git repository. ywkim pushed a commit to branch cnb in repository https://gitbox.apache.org/repos/asf/bigtop.git The following commit(s) were added to refs/heads/cnb by this push: new 1511c90 BIGTOP-3242: Add instructions on configuring kubectl for local k8s cluster 1511c90 is described below commit 1511c907d039895d7d1ed25fc9e7b47a261dc727 Author: Youngwoo Kim AuthorDate: Mon Oct 14 12:15:44 2019 +0900 BIGTOP-3242: Add instructions on configuring kubectl for local k8s cluster --- README.md | 16 1 file changed, 16 insertions(+) diff --git a/README.md b/README.md index 0db5be6..5952ea0 100755 --- a/README.md +++ b/README.md @@ -35,6 +35,22 @@ $ cd dl/kubespray-2.11.0/ && cp ../../kubespray/vagrant/Vagrantfile . $ vagrant up ``` +## Configuring ```kubectl``` for local cluster +``` +$ vagrant ssh k8s-1 + +k8s-1$ mkdir -p ~/.kube +k8s-1$ sudo cp /etc/kubernetes/admin.conf .kube/config +k8s-1$ sudo chown -R vagrant:vagrant .kube +k8s-1$ kubectl cluster-info + +Kubernetes master is running at https://172.17.8.101:6443 +coredns is running at https://172.17.8.101:6443/api/v1/namespaces/kube-system/services/coredns:dns/proxy +kubernetes-dashboard is running at https://172.17.8.101:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy + +To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. +``` + # Cloud Native Bigtop This is the content for the talk given by jay vyas and sid mani @ apachecon 2019 in Las Vegas, you can watch it here https://www.youtube.com/watch?v=LUCE63q !
[bigtop] branch cnb updated: BIGTOP-3238: Local muti-node cluster via vagrant and kubespray
This is an automated email from the ASF dual-hosted git repository.
ywkim pushed a commit to branch cnb
in repository https://gitbox.apache.org/repos/asf/bigtop.git
The following commit(s) were added to refs/heads/cnb by this push:
new dccbc42 BIGTOP-3238: Local muti-node cluster via vagrant and kubespray
dccbc42 is described below
commit dccbc42270058fe29af5223bf8e6b642320be469
Author: Youngwoo Kim
AuthorDate: Mon Oct 14 10:29:58 2019 +0900
BIGTOP-3238: Local muti-node cluster via vagrant and kubespray
---
README.md | 38 +++
bigtop.bom| 12 +++
build.gradle | 2 +-
kubespray/vagrant/Vagrantfile | 227 ++
4 files changed, 278 insertions(+), 1 deletion(-)
diff --git a/README.md b/README.md
index 5bec287..0db5be6 100755
--- a/README.md
+++ b/README.md
@@ -1,3 +1,41 @@
+[](https://travis-ci.org/apache/bigtop)
+
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+
+[Apache Bigtop](http://bigtop.apache.org/)
+==
+
+TBD
+
+# Get Started with Deployment and Smoke Testing of Cloud Native BigTop
+
+Prerequisites:
+- Vagrant
+- Java
+
+## Set up 3-Node Kubernetes cluster via Kubespray on local machine
+```
+$ cd $BIGTOP_HOME
+$ ./gradlew kubespray-clean kubespray-download && cd dl/ && tar xvfz
kubespray-2.11.0.tar.gz
+$ cd dl/kubespray-2.11.0/ && cp ../../kubespray/vagrant/Vagrantfile .
+$ vagrant up
+```
+
+# Cloud Native Bigtop
This is the content for the talk given by jay vyas and sid mani @ apachecon
2019 in Las Vegas, you can watch it here
https://www.youtube.com/watch?v=LUCE63q !
# TLDR, heres how you create an analytics distro on K8s...
diff --git a/bigtop.bom b/bigtop.bom
index bafca95..99ab5c8 100644
--- a/bigtop.bom
+++ b/bigtop.bom
@@ -122,6 +122,18 @@ bigtop {
]
components {
+'kubespray' {
+ name= "kubespray"
+ pkg = "kubespray"
+ relNotes = ""
+ website = "https://github.com/kubernetes-sigs/kubespray;
+ version { base = '2.11.0'; pkg = base; release = 1 }
+ tarball { destination = "$name-${version.base}.tar.gz"
+source = "v${version.base}.tar.gz" }
+ url { site = "https://github.com/kubernetes-sigs/kubespray/archive/;
+archive = site }
+}
+
'zookeeper' {
name= 'zookeeper'
pkg = name
diff --git a/build.gradle b/build.gradle
index 8dc9095..a9065fd 100644
--- a/build.gradle
+++ b/build.gradle
@@ -23,7 +23,7 @@ buildscript {
}
plugins {
- id "de.undercouch.download" version "3.2.0"
+ id "de.undercouch.download" version "4.0.0"
id "org.nosphere.apache.rat" version "0.2.0"
}
diff --git a/kubespray/vagrant/Vagrantfile b/kubespray/vagrant/Vagrantfile
new file mode 100644
index 000..9702059
--- /dev/null
+++ b/kubespray/vagrant/Vagrantfile
@@ -0,0 +1,227 @@
+# -*- mode: ruby -*-
+# # vi: set ft=ruby :
+
+# For help on using kubespray with vagrant, check out docs/vagrant.md
+
+require 'fileutils'
+
+Vagrant.require_version ">= 2.0.0"
+
+CONFIG = File.join(File.dirname(__FILE__), "vagrant/config.rb")
+
+COREOS_URL_TEMPLATE =
"https://storage.googleapis.com/%s.release.core-os.net/amd64-usr/current/coreos_production_vagrant.json;
+
+# Uniq disk UUID for libvirt
+DISK_UUID = Time.now.utc.to_i
+
+SUPPORTED_OS = {
+ "coreos-stable" => {box: "coreos-stable", user: "core", box_url:
COREOS_URL_TEMPLATE % ["stable"]},
+ "coreos-alpha"=> {box: "coreos-alpha", user: "core", box_url:
COREOS_URL_TEMPLATE % ["alpha"]},
+ "coreos-beta" => {box: "coreos-beta",user: "core", box_url:
COREOS_URL_TEMPLATE % ["beta"]},
+ "ubuntu1604" => {box: "generic/ubuntu1604", user: "vagrant"},
+ "ubuntu1804" => {box: "generic/ubuntu1804", user: "vagrant"},
+ "centos" => {box: "centos/7", user: "vagrant"},
+ "centos-bento"=> {box: "bento/centos-7.6", user: "vagrant"},
+ "fedora" => {box: "fedora/28-cloud-base",user:
"vagrant"},
+ "opensuse"=> {box: "opensuse/openSUSE-15.0-x86_64", user:
"vagrant"},
+
[bigtop] branch cnb updated: BIGTOP-3230: Import the groundwork for CNB
This is an automated email from the ASF dual-hosted git repository.
ywkim pushed a commit to branch cnb
in repository https://gitbox.apache.org/repos/asf/bigtop.git
The following commit(s) were added to refs/heads/cnb by this push:
new e76ae87 BIGTOP-3230: Import the groundwork for CNB
e76ae87 is described below
commit e76ae872de0c51f6ed405d34dcff7db1a7411339
Author: Youngwoo Kim
AuthorDate: Mon Oct 7 12:09:43 2019 +0900
BIGTOP-3230: Import the groundwork for CNB
---
POSTINSTALL_EXAMPLE.md | 26 ++
README.md | 354 +++--
README.md => README.md.BAK | 0
README_STORAGE.md | 14 +
data/a.txt | 9 +
data/b.txt | 15 +
get_helm.sh| 245 ++
hbase-helm/.helmignore | 21 ++
hbase-helm/Chart.yaml | 13 +
hbase-helm/README.md | 1 +
hbase-helm/templates/NOTES.txt | 26 ++
hbase-helm/templates/_helpers.yaml | 14 +
hbase-helm/templates/hadoop-configmap.yaml | 296 +
hbase-helm/templates/hbase-configmap.yaml | 67
hbase-helm/templates/hbase-master-pdb.yaml | 17 +
hbase-helm/templates/hbase-master-statefulset.yaml | 59
hbase-helm/templates/hbase-master-svc.yaml | 26 ++
hbase-helm/templates/hbase-rs-pdb.yaml | 17 +
hbase-helm/templates/hbase-rs-statefulset.yaml | 59
hbase-helm/templates/hbase-rs-svc.yaml | 22 ++
hbase-helm/templates/hdfs-dn-pdb.yaml | 17 +
hbase-helm/templates/hdfs-dn-pvc.yaml | 25 ++
hbase-helm/templates/hdfs-dn-statefulset.yaml | 82 +
hbase-helm/templates/hdfs-dn-svc.yaml | 23 ++
hbase-helm/templates/hdfs-nn-pdb.yaml | 18 ++
hbase-helm/templates/hdfs-nn-pvc.yaml | 25 ++
hbase-helm/templates/hdfs-nn-statefulset.yaml | 82 +
hbase-helm/templates/hdfs-nn-svc.yaml | 23 ++
hbase-helm/tools/calc_resources.sh | 75 +
hbase-helm/values.yaml | 55
kafka/README.md| 25 ++
kafka/kafka.yaml | 7 +
minio/object-store.yaml| 63
minio/operator.yaml| 99 ++
ms-spark/README.md | 21 ++
ms-spark/core-site.xml | 49 +++
ms-spark/log4j.properties | 12 +
ms-spark/spark-defaults.conf | 1 +
ms-spark/spark-deployment.yaml | 94 ++
ms-spark/spark-env.sh | 1 +
nifi/README.md | 21 ++
nifi/volume.yaml | 31 ++
presto3-minio/README.md| 32 ++
presto3-minio/coordinator/Dockerfile | 6 +
presto3-minio/coordinator/README.md| 8 +
presto3-minio/coordinator/config.properties| 8 +
presto3-minio/coordinator/json | 2 +
presto3-minio/coordinator/minio.properties | 14 +
presto3-minio/presto-deployment-master.yaml| 26 ++
presto3-minio/presto-deployment-worker.yaml| 26 ++
presto3-minio/presto-deployment.yaml | 23 ++
presto3-minio/service.yaml | 9 +
presto3-minio/test.sh | 19 ++
presto3-minio/worker/Dockerfile| 5 +
presto3-minio/worker/README.md | 6 +
presto3-minio/worker/config.properties | 7 +
presto3-minio/worker/json | 2 +
presto3-minio/worker/minio.properties | 14 +
storage-provisioner/README.md | 28 ++
59 files changed, 2151 insertions(+), 234 deletions(-)
diff --git a/POSTINSTALL_EXAMPLE.md b/POSTINSTALL_EXAMPLE.md
new file mode 100755
index 000..67b7823
--- /dev/null
+++ b/POSTINSTALL_EXAMPLE.md
@@ -0,0 +1,26 @@
+
+
+# Example of what needs to be done...
+
+Minio can be accessed via port 9000 on the following DNS name from within your
cluster:
+minio.bigdata.svc.cluster.local
+
+To access Minio from localhost, run the below commands:
+
+ 1. export POD_NAME=$(kubectl get pods --namespace bigdata -l "release=minio"
-o jsonpath="{.items[0].metadata.name}")
+
+ 2. kubectl port-forward $POD_NAME 9000 --namespace bigdata
+
+Read more about port forwarding here:
http://kubernetes.io/docs/user-guide/kubectl/kubectl_port-forward/
+
+You can now access Minio server on http://localhost:9000. Follow the below
steps to connect to Minio server with
