[GitHub] incubator-trafodion pull request #810: [TRAFODION-2293] remove minor version...

[mashengchen]

GitHub user mashengchen opened a pull request:

https://github.com/apache/incubator-trafodion/pull/810

[TRAFODION-2293] remove minor version for the hbase-trx-apache_*.jar



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mashengchen/incubator-trafodion trafodion-2293

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-trafodion/pull/810.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #810


commit 8c38f482b8085f6f9676a0d6d93834e65e17795a
Author: mashengchen 
Date:   2016-11-02T02:58:21Z

Update install_local_hadoop




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-trafodion pull request #807: [TRAFODION-1839] Trafodion Installer ...

[svarnau]

Github user svarnau commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/807#discussion_r86036495
  
--- Diff: install/python-installer/traf_dep.py ---
@@ -0,0 +1,119 @@
+#!/usr/bin/env python
+
+# @@@ START COPYRIGHT @@@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# @@@ END COPYRIGHT @@@
+
+### this script should be run on all nodes with sudo user ###
+
+import re
+import os
+import sys
+import json
+import platform
+from common import run_cmd, cmd_output, err
+
+# not used
+EPEL_REPO = """
--- End diff --

This should not need to be hard-coded. Doing a yum install of 
"epel-release" package will get you the epel repo file.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-trafodion pull request #807: [TRAFODION-1839] Trafodion Installer ...

[svarnau]

Github user svarnau commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/807#discussion_r86034218
  
--- Diff: install/python-installer/copy_files.py ---
@@ -0,0 +1,64 @@
+#!/usr/bin/env python
+
+# @@@ START COPYRIGHT @@@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# @@@ END COPYRIGHT @@@
+
+### this script should be run on local node ###
+
+import sys
+import json
+from threading import Thread
+from common import Remote, run_cmd, err
+
+def run(pwd):
+""" gen ssh key on local and copy to all nodes
+copy traf package file from local to all nodes
+"""
+dbcfgs = json.loads(dbcfgs_json)
+hosts = dbcfgs['node_list'].split(',')
+traf_package = dbcfgs['traf_package']
+
+key_file = '/tmp/id_rsa'
+run_cmd('sudo rm -rf %s*' % key_file)
+run_cmd('sudo echo -e "y" | ssh-keygen -t rsa -N "" -f %s' % key_file)
+
--- End diff --

Use -q option to prevent ssh-keygen from prompting.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-trafodion pull request #807: [TRAFODION-1839] Trafodion Installer ...

[svarnau]

Github user svarnau commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/807#discussion_r86037754
  
--- Diff: install/python-installer/traf_package.py ---
@@ -0,0 +1,47 @@
+#!/usr/bin/env python
+
+# @@@ START COPYRIGHT @@@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# @@@ END COPYRIGHT @@@
+
+## This script should be run on all nodes with trafodion user ##
+
+import sys
+import json
+from common import run_cmd, err
+
+def run():
+dbcfgs = json.loads(dbcfgs_json)
+
+TRAF_DIR = '%s-%s' % (dbcfgs['traf_basename'], dbcfgs['traf_version'])
+
+# untar traf package
+TRAF_PACKAGE_FILE = '/tmp/' + dbcfgs['traf_package'].split('/')[-1]
+run_cmd('mkdir -p %s' % TRAF_DIR)
+run_cmd('tar xf %s -C %s' % (TRAF_PACKAGE_FILE, TRAF_DIR))
+
+print 'Trafodion package uncompressed successfully!'
--- End diff --

I think "extracted" would make more sense to me than "uncompressed". But 
opinions may vary.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-trafodion pull request #807: [TRAFODION-1839] Trafodion Installer ...

[svarnau]

Github user svarnau commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/807#discussion_r86020847
  
--- Diff: install/python-installer/README.md ---
@@ -0,0 +1,21 @@
+# Apache Trafodion Python Installer
+
+## Prerequisite:
+
+- CDH/HDP is installed on Trafodion nodes with web UI enabled, or Apache 
Hadoop, HBase is installed on the same directory on all nodes
+- /etc/hosts contains hostname info for all Trafodion nodes on **installer 
node**
+- python version 2.6/2.7, and python library `httplib2`, `prettytable`
+- Trafodion server package file is stored on **installer node**
+- Passwordless SSH login, two ways:
--- End diff --

Need to clarify that one of these two options is needed, not both.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-trafodion pull request #807: [TRAFODION-1839] Trafodion Installer ...

[svarnau]

Github user svarnau commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/807#discussion_r86025475
  
--- Diff: install/python-installer/common.py ---
@@ -0,0 +1,479 @@
+#!/usr/bin/env python
+
+# @@@ START COPYRIGHT @@@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# @@@ END COPYRIGHT @@@
+
+### The common functions ###
+
+import os
+import pty
+import sys
+import json
+import re
+import time
+import base64
+import subprocess
+import logging
+try:
+import xml.etree.cElementTree as ET
+except ImportError:
+import xml.etree.ElementTree as ET
+from ConfigParser import ConfigParser
+from collections import defaultdict
+
+__VERSION__ = 'v1.0.0'
--- End diff --

Hard-coding a version number here is not a good idea. If we need one, it 
should be generated at time the installer is packaged.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-trafodion pull request #807: [TRAFODION-1839] Trafodion Installer ...

[svarnau]

Github user svarnau commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/807#discussion_r86027515
  
--- Diff: install/python-installer/common.py ---
@@ -0,0 +1,479 @@
+#!/usr/bin/env python
+
+# @@@ START COPYRIGHT @@@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# @@@ END COPYRIGHT @@@
+
+### The common functions ###
+
+import os
+import pty
+import sys
+import json
+import re
+import time
+import base64
+import subprocess
+import logging
+try:
+import xml.etree.cElementTree as ET
+except ImportError:
+import xml.etree.ElementTree as ET
+from ConfigParser import ConfigParser
+from collections import defaultdict
+
+__VERSION__ = 'v1.0.0'
+INSTALLER_LOC = sys.path[0]
+
+USER_PROMPT_FILE = INSTALLER_LOC + '/prompt.json'
+SCRCFG_FILE = INSTALLER_LOC + '/script.json'
+VERSION_FILE = INSTALLER_LOC + '/version.json'
+MODCFG_FILE = INSTALLER_LOC + '/mod_cfgs.json'
+
+DBCFG_FILE = INSTALLER_LOC + '/db_config'
+DBCFG_TMP_FILE = INSTALLER_LOC + '/.db_config_temp'
+
+TMP_DIR = '/tmp/.install'
+MARK = '[ERR]'
--- End diff --

.install is a very generic name. It may possibly be used for other things 
and hence not secured properly, etc.
Maybe something like /tmp/trafodion_install_temp would be better.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-trafodion pull request #807: [TRAFODION-1839] Trafodion Installer ...

[svarnau]

Github user svarnau commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/807#discussion_r86029193
  
--- Diff: install/python-installer/common.py ---
@@ -0,0 +1,479 @@
+#!/usr/bin/env python
+
+# @@@ START COPYRIGHT @@@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# @@@ END COPYRIGHT @@@
+
+### The common functions ###
+
+import os
+import pty
+import sys
+import json
+import re
+import time
+import base64
+import subprocess
+import logging
+try:
+import xml.etree.cElementTree as ET
+except ImportError:
+import xml.etree.ElementTree as ET
+from ConfigParser import ConfigParser
+from collections import defaultdict
+
+__VERSION__ = 'v1.0.0'
+INSTALLER_LOC = sys.path[0]
+
+USER_PROMPT_FILE = INSTALLER_LOC + '/prompt.json'
+SCRCFG_FILE = INSTALLER_LOC + '/script.json'
+VERSION_FILE = INSTALLER_LOC + '/version.json'
+MODCFG_FILE = INSTALLER_LOC + '/mod_cfgs.json'
+
+DBCFG_FILE = INSTALLER_LOC + '/db_config'
+DBCFG_TMP_FILE = INSTALLER_LOC + '/.db_config_temp'
+
+TMP_DIR = '/tmp/.install'
+MARK = '[ERR]'
+
+def version():
+print 'Installer version: %s' % __VERSION__
+exit(0)
+
+def ok(msg):
+print '\n\33[32m***[OK]: %s \33[0m' % msg
+
+def info(msg):
+print '\n\33[33m***[INFO]: %s \33[0m' % msg
+
+def err_m(msg):
+""" used by main script """
+sys.stderr.write('\n\33[31m***[ERROR]: %s \33[0m\n' % msg)
+sys.exit(1)
+
+def err(msg):
+""" used by sub script """
+sys.stderr.write(MARK + msg)
+sys.exit(1)
+
+def get_logger(log_file):
+
+log_dir = os.path.dirname(log_file)
+if not os.path.exists(log_dir): os.mkdir(log_dir)
+
+logger = logging.getLogger()
+logger.setLevel(logging.INFO)
+
+formatter = logging.Formatter('[%(asctime)s %(levelname)s]: 
%(message)s')
+
+fh = logging.FileHandler(log_file)
+fh.setFormatter(formatter)
+
+logger.addHandler(fh)
+
+return logger
+
+def run_cmd(cmd):
+""" check command return value and return stdout """
+p = subprocess.Popen(cmd, stdin=subprocess.PIPE, 
stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
+stdout, stderr = p.communicate()
+if p.returncode != 0:
+err('Failed to run command %s: %s' % (cmd, stderr))
+return stdout.strip()
+
+def run_cmd_as_user(user, cmd):
+return run_cmd('sudo su - %s -c \'%s\'' % (user, cmd))
+
--- End diff --

Probably want to use -n (non-interactive) option to sudo, so we never get 
password prompt, etc.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-trafodion pull request #808: Major reorganization of the Client In...

[gtapper]

GitHub user gtapper opened a pull request:

https://github.com/apache/incubator-trafodion/pull/808

Major reorganization of the Client Installation Guide.

Corresponding changes in the odb User's Guide to keep installation in one 
place.

Formatting changes in some guides. Fixed DROP TABLE syntax mistakes.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gtapper/incubator-trafodion TRAFODION-2323

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-trafodion/pull/808.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #808


commit da748b4d887f20026d0afc1d137fc88a72cf717b
Author: Gunnar Tapper 
Date:   2016-11-01T20:37:58Z

Major reorganization of the Client Installation Guide.

Corresponding changes in the odb User's Guide to keep installation in one 
place.

Formatting changes in some guides.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-trafodion pull request #806: TRAFODION-2330 Using trafci, a select...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-trafodion/pull/806


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-trafodion pull request #806: TRAFODION-2330 Using trafci, a select...

[sureshsubbiah]

Github user sureshsubbiah commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/806#discussion_r85935717
  
--- Diff: core/sql/arkcmp/CmpContext.cpp ---
@@ -1157,4 +1157,16 @@ void CmpContext::resetLogmxEventSqlText()
delete sqlTextBuf_ ;
sqlTextBuf_ = NULL ;
 }
+
+void CmpContext::clearAllCaches()
+{
+   qcache_->makeEmpty();
--- End diff --

The constructor for CmpContext seems to initialize histogramCache_ the same 
as qcache_. Is it possible qcache_ can be NULL at some point? It may be safer 
to guard it, this is a really minor nit to be handled later if necessary.

There is one more cache, optPCodeCache_ . Now this cache may not cause any 
priv violations I suppose. So a comment later here may be all we need, that 
PCode cache is not addressed by this method.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-trafodion pull request #805: [TRAFODION-1077] AQR info missing fro...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-trafodion/pull/805


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-trafodion pull request #807: [TRAFODION-1839] Trafodion Installer ...

[mkby]

GitHub user mkby opened a pull request:

https://github.com/apache/incubator-trafodion/pull/807

[TRAFODION-1839] Trafodion Installer Evolution



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mkby/incubator-trafodion master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-trafodion/pull/807.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #807


commit 904b53df5c75f0dd5b0bd84e483c7d060023a7d6
Author: Eason 
Date:   2016-11-01T08:53:27Z

[TRAFODION-1839] Trafodion Installer Evolution




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---