[jira] [Created] (TRAFODION-2874) LOB: Add syntax to return filename of a LOB data file for external LOBs.

2018-01-02 Thread Sandhya Sundaresan (JIRA) 
Sandhya Sundaresan created TRAFODION-2874:
-

 Summary: LOB: Add syntax to return filename  of a LOB data file 
for external LOBs.
 Key: TRAFODION-2874
 URL: https://issues.apache.org/jira/browse/TRAFODION-2874
 Project: Apache Trafodion
  Issue Type: Bug
  Components: sql-exe
Affects Versions: 2.3
Reporter: Sandhya Sundaresan
Assignee: Sandhya Sundaresan


For external LOBs,  Trafodion does not save the LOB data in it's internal 
trafodion namespace. It saves only the LOB  handle information and the actual 
LOB data remains outside in users namespace in HDFS.So inserts are very 
eifficient. During extract from an external LOB today, we extract the LOB data 
from the external file and return it to the user. Need to extend the Extract 
syntax to also be able to return the external LOB data filename alone . 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (TRAFODION-2873) LOB:Cleanup usage of LOBLoad which si deprecated and LobGlobals

2018-01-02 Thread Sandhya Sundaresan (JIRA) 
Sandhya Sundaresan created TRAFODION-2873:
-

 Summary: LOB:Cleanup usage of LOBLoad which si deprecated and 
LobGlobals
 Key: TRAFODION-2873
 URL: https://issues.apache.org/jira/browse/TRAFODION-2873
 Project: Apache Trafodion
  Issue Type: Bug
  Components: sql-exe
Affects Versions: 2.3
Reporter: Sandhya Sundaresan
Assignee: Sandhya Sundaresan


The LOBGlobals structure contains information relevant to LOBLoad which was an 
operator initially designed to operate at the disk level. It is no longer 
needed/relevant so cleaning up that code and simplifying the LOBGlobals 
sturcture as well to keep only the relevant data members. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (TRAFODION-2866) Add HBase Configuration Changes for *Trafodion Provisioning Guide*

2018-01-02 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TRAFODION-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309043#comment-16309043
 ] 

ASF GitHub Bot commented on TRAFODION-2866:
---

Github user liuyu000 commented on a diff in the pull request:

https://github.com/apache/trafodion/pull/1361#discussion_r159353975
  
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/requirements.adoc 
---
@@ -331,13 +331,38 @@ default. Depending on the size of a user table, we 
have experienced timeout fail
 from this setting. The underlying issue is the length of the execution of 
the coprocessor within HBase.
  +
 NOTE: HBase uses the smaller of `hbase.rpc.timeout` and 
`hbase.client.scanner.timeout.period` to calculate the scanner timeout. 
-| hbase.snapshot.master.timeoutMillis and hbase.snapshot.region.timeout | 
10 minutes | HBase's default setting is 6 milliseconds. 
+| hbase.snapshot.master.timeoutMillis 
+
+and 
+
+hbase.snapshot.region.timeout | 10 minutes | HBase's default setting is 
6 milliseconds. 
 If you experience timeout issues with HBase snapshots when you use the 
{project-name} Bulk Loader or other statements, 
 you can set the value for these two HBase properties to 10 minutes 
(600,000 milliseconds).
-| hbase.hregion.max.filesize | 107374182400 bytes | HBase's default 
setting is 10737418240 (10 GB). We have increased the setting to 
-107374182400 (100 GB), which reduces the number of HStoreFiles per table 
and appears to reduce disruptions to active transactions from 
+| hbase.hregion.max.filesize | 107374182400 bytes | HBase's default 
setting is 10737418240 bytes (10 GB). You can increased the setting to 
+107374182400 bytes (100 GB), which reduces the number of HStoreFiles per 
table and appears to reduce disruptions to active transactions from 
 region splitting.
-| hbase.hstore.blockingStoreFiles | 10 | 
http://gbif.blogspot.com/2012/07/optimizing-writes-in-hbase.html
+| hbase.hregion.memstore.block.multiplier | 7
+
+When you have enough memory, you can increase this value to 7 so that more 
data can be temporarily accepted before flushing to disk instead of blocking 
writes.
+|This property blocks any further writes from clients to memstores if the 
memstores exceed the value of `multiplier * flush size`.
+
+Default value: 2
+| hbase.hregion.memstore.flush.size | 536870912 bytes | HBase uses 
memstore to buffer data before writing it to disk. Once the data in memstore 
has outgrown this size, it is flushed as an HFile to disk.
+
+Default value: 134217728 bytes (128M)
+| hbase.hstore.blockingStoreFiles | 200 | 
http://gbif.blogspot.com/2012/07/optimizing-writes-in-hbase.html
+
+This property blocks any further writes from memstores to regions after 
HFile number hitting this limit until compactions are completed.
--- End diff --

Thanks Ming, your comment has been incorporated. :)


> Add HBase Configuration Changes for *Trafodion Provisioning Guide*
> --
>
> Key: TRAFODION-2866
> URL: https://issues.apache.org/jira/browse/TRAFODION-2866
> Project: Apache Trafodion
>  Issue Type: Documentation
>Reporter: Liu Yu
>Assignee: Liu Yu
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (TRAFODION-2664) Instance will be down when the zookeeper on name node has been down

2018-01-02 Thread Gonzalo E Correa (JIRA) 

[ 
https://issues.apache.org/jira/browse/TRAFODION-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16308899#comment-16308899
 ] 

Gonzalo E Correa commented on TRAFODION-2664:
-

This issue was fixed and should be marked resolved as of Release 2.2.

> Instance will be down when the zookeeper on name node has been down
> ---
>
> Key: TRAFODION-2664
> URL: https://issues.apache.org/jira/browse/TRAFODION-2664
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: foundation
>Affects Versions: 2.2-incubating
> Environment: Test Environment:
> CDH5.4.8: 10.10.23.19:7180, total 6 nodes.
> HDFS-HA and DCS-HA: enabled
> OS: Centos6.8, physic machine.
> SW Build: R2.2.3 (EsgynDB_Enterprise Release 2.2.3 (Build release [sbroeder], 
> branch 1ce8d39-xdc_nari, date 11Jun17)
>Reporter: Jarek
>Assignee: Gonzalo E Correa
>Priority: Critical
>  Labels: build
> Fix For: 2.2-incubating
>
>
> Description: Instance will be down when the zookeeper on name node has been 
> down
> Test Steps:
> Step 1. Start OE and 4 long queries with trafci on the first node 
> esggy-clu-n010
> Step 2. Wait several minutes and stop zookeeper on name node of node 
> esggy-clu-n010  in Cloudera Manager page.
> Step 3. With trafci, run a basic query and 4 long queries again.
> In the above Step 3, we will see the whole instance as down after a while. 
> For this test scenario, I tried it several times, always found instance as 
> down.
> Timestamp:
> Test Start Time: 20170616132939
> Test End  Time: 20170616134350
> Stop zookeeper on name node of node esggy-clu-n010: 20170616133344
> Check logs:
> 1) Each node displays the following error:
> 2017-06-16 13:33:46,276, ERROR, MON, Node Number: 0,, PIN: 5017 , Process 
> Name: $MONITOR,,, TID: 5429, Message ID: 101371801, 
> [CZClient::IsZNodeExpired], zoo_exists() for 
> /trafodion/instance/cluster/esggy-clu-n010.esgyn.cn failed with error 
> ZCONNECTIONLOSS
> 2) Zookeeper displays:
> ls /trafodion/instance/cluster
> []
> So, It seems zclient has been lost on each node.
> Location of logs:
> esggy-clu-n010: 
> /data4/jarek/ha.interactive/trafodion_and_cluster_logs/cluster_logs.20170616134816.tar.gz
>  and trafodion_logs.20170616134816.tar.gz
> By the way, because the size of the logs is out of the limited value, so i 
> cannot upload it as the attachment in this JIRA ID.
> How many zookeeper quorum servers in the cluster? total 3.
>   
> dcs.zookeeper.quorum
> 
> esggy-clu-n010.esgyn.cn,esggy-clu-n011.esgyn.cn,esggy-clu-n012.esgyn.cn
>   
> How to access the cluster?
> 1. Login 10.10.10.8 from US machine: trafodion/traf123
> 2. Login 10.10.23.19 from 10.10.10.8: trafodion/traf123



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (TRAFODION-2746) Monitor exhibits memory corruption in large cluster configuration > 30 nodes

2018-01-02 Thread Gonzalo E Correa (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TRAFODION-2746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gonzalo E Correa resolved TRAFODION-2746.
-
Resolution: Fixed

> Monitor exhibits memory corruption in large cluster configuration > 30 nodes
> 
>
> Key: TRAFODION-2746
> URL: https://issues.apache.org/jira/browse/TRAFODION-2746
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: foundation
>Affects Versions: 2.3
>Reporter: Gonzalo E Correa
>Assignee: Gonzalo E Correa
> Fix For: 2.3
>
>
> Found the following problems in the monitor when trying to bring up 120 nodes:
> 1.A segmentation violation occurred during the Integration phase, when 
> the new monitor is establishing the socket communication paths between itself 
> and the existing monitors.
> 2.A second segmentation violation was due to a buffer overwrite during 
> the Joining (revive) phase.
> 3.One of the monitor would remain in the Joining state and never come out 
> of it.
> 4.Stderr buffer overwrite in CRedirectStderr::handleOutput()



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (TRAFODION-2651) The monitor to monitor process communication cannot handle a network reset

2018-01-02 Thread Gonzalo E Correa (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TRAFODION-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gonzalo E Correa resolved TRAFODION-2651.
-
Resolution: Fixed

> The monitor to monitor process communication cannot handle a network reset 
> ---
>
> Key: TRAFODION-2651
> URL: https://issues.apache.org/jira/browse/TRAFODION-2651
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: foundation
>Affects Versions: 2.2-incubating
>Reporter: Gonzalo E Correa
>Assignee: Gonzalo E Correa
> Fix For: 2.3
>
>
> The monitor to monitor socket communication does not have reconnect logic to 
> handle a network reset or transient network errors.
> Analysis:
> • During a ~20 second network reset window, no errors are detected by 
> open sockets
> o Open sockets are dead, but there is no indication from the TCP/IP stack 
> that socket is in an error condition
> • Once the network is restored, a CONNECTIONLOSS is reported by the 
> Zookeeper Client Library.
> o However, reconnect logic reestablishes connection with quorum.
> • At EPOLL expiration time, EPOLL logic report “Not heard from peer=n” 
> and treats peer as Node Down.
> o The node down logic deletes corresponding znode, 
> CZClient::WatchNodeDelete()
> o All monitor processes continually check for expired znodes for each 
> node in the cluster, including their own znode
>  An expired znode is handled as a down node



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (TRAFODION-2854) Load encounter Operating system error 201

2018-01-02 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TRAFODION-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16308804#comment-16308804
 ] 

ASF GitHub Bot commented on TRAFODION-2854:
---

GitHub user selvaganesang opened a pull request:

https://github.com/apache/trafodion/pull/1364

[TRAFODION-2854] Load encounter Operating system error 201

When load returns an error during insert, the row is reconstructed
to log in the same format as the source. During the reconstruction process
a wrong tuple format was used.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/selvaganesang/incubator-trafodion 
trafodion-2854

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/trafodion/pull/1364.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1364


commit daaadefec5e20e2f39f7da72348978d5611cf5dd
Author: selvaganesang 
Date:   2018-01-02T21:31:24Z

[TRAFODION-2854] Load encounter Operating system error 201

When load returns an error during insert, the row is reconstructed
to log in the same format as the source. During the reconstruction process
a wrong tuple format was used.




> Load encounter Operating system error 201
> -
>
> Key: TRAFODION-2854
> URL: https://issues.apache.org/jira/browse/TRAFODION-2854
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: sql-exe
>Reporter: Selvaganesan Govindarajan
>Assignee: Selvaganesan Govindarajan
> Fix For: 2.3
>
>
> Load data from hive data encounter 201 error, but using uspert using load the 
> error is different
> >>load with log error rows to '/bulkload/logs' into 
> >>TRAFODION.ODS_SC.DM_FUNCTION_LOCATION select * from 
> >>hive.hive.DM_FUNCTION_LOCATION;
> Task: LOAD Status: Started Object: TRAFODION.ODS_SC.DM_FUNCTION_LOCATION
> Task: CLEANUP Status: Started Time: 2017-12-15 10:04:17.366
> Task: CLEANUP Status: Ended Time: 2017-12-15 10:04:17.385
> Task: CLEANUP Status: Ended Elapsed Time: 00:00:00.019
>Logging Location: 
> /bulkload/logs/ERR_TRAFODION.ODS_SC.DM_FUNCTION_LOCATION_20171215_020417
> Task: LOADING DATA Status: Started Time: 2017-12-15 10:04:17.385
> *** ERROR[2034] $Z000RN4:502: Operating system error 201 while communicating 
> with server process $Z0211QE:526.
> *** ERROR[2034] $Z000RN4:502: Operating system error 201 while communicating 
> with server process $Z0211QE:526.
> *** ERROR[2034] $Z000RN4:502: Operating system error 201 while communicating 
> with server process $Z0211QE:526.
> SQL>upsert using load into TRAFODION.ODS_SC.DM_FUNCTION_LOCATION select * 
> from hive.hive.DM_FUNCTION_LOCATION;
> *** ERROR[8411] A numeric overflow occurred during an arithmetic computation 
> or data conversion. Conversion of Source Type:CHAR(REC_BYTE_F_ASCII,15 
> BYTES,ISO88591) Source Value:214040391595900 to Target Type:DECIMAL 
> SIGNED(REC_DECIMAL_LSE). [2017-12-15 13:46:59]
> Additional InformationCore file and stack trace are under 
> 10.10.22.152:/opt/GuangXi_20171218
> Account: root/linux
> Step to analyze core:
> [root@esggy-del-n002 GuangXi_20171218]# gdb
> (gdb) source zgdb-GuangXi_20171218-
> (gdb) file /opt/esgynDB223/export/bin64/tdm_arkesp
> (gdb) core core.44989
> (gdb) bt
> #0 0x7f440e0ea1d7 in raise () from /opt/GuangXi_20171218/lib64/libc.so.6
> #1 0x7f440e0eb8c8 in abort () from /opt/GuangXi_20171218/lib64/libc.so.6
> #2 0x7f4410039f85 in os::abort(bool) () from 
> /opt/GuangXi_20171218/opt/jdk1.8.0_112/jre/lib/amd64/server/libjvm.so
> #3 0x7f44101dc383 in VMError::report_and_die() () from 
> /opt/GuangXi_20171218/opt/jdk1.8.0_112/jre/lib/amd64/server/libjvm.so
> 004 0x7f441003f48f in JVM_handle_linux_signal () from 
> /opt/GuangXi_20171218/opt/jdk1.8.0_112/jre/lib/amd64/server/libjvm.so
> 005 0x7f44100359d3 in signalHandler(int, siginfo*, void*) ()
>from /opt/GuangXi_20171218/opt/jdk1.8.0_112/jre/lib/amd64/server/libjvm.so
> 006 
> 007 0x7f440e1ffde0 in __memcpy_ssse3_back () from 
> /opt/GuangXi_20171218/lib64/libc.so.6
> 008 0x7f441308aeaf in str_cpy_all (length=-7167, src= variable: Cannot access memory at address 0xbc>, 
> tgt=0x7fff64f1ed60 "\200") at ../common/str.h:265
> 009 getLength (data= address 0xbc>, this=0x80) at ../exp/exp_attrs.h:415
> 010 ExHbaseAccessBulkLoadPrepSQTcb::createLoggingRow (this= out>, tuppIndex=, tuppRow=0x7f43fe67e3d0 "", 
> targetRow=0x7f43fe68e401 "", targetRowLen=@0x7fff64f1eea0: 0) at 
> ../executor/ExHbaseIUD.cpp:1754
> 011 0x7f441308ed1c in ExHbaseAccessBulkLoadPrepSQTcb::work 
> (this=0x7f43fe662f48) at

[jira] [Created] (TRAFODION-2872) Adjust notification file to reflect 2018

2018-01-02 Thread Pierre Smits (JIRA) 
Pierre Smits created TRAFODION-2872:
---

 Summary: Adjust notification file to reflect 2018
 Key: TRAFODION-2872
 URL: https://issues.apache.org/jira/browse/TRAFODION-2872
 Project: Apache Trafodion
  Issue Type: Sub-task
Reporter: Pierre Smits






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (TRAFODION-2868) bug of hbase option TTL

2018-01-02 Thread Suresh Subbiah (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TRAFODION-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suresh Subbiah reassigned TRAFODION-2868:
-

Assignee: Suresh Subbiah

> bug of hbase option TTL 
> 
>
> Key: TRAFODION-2868
> URL: https://issues.apache.org/jira/browse/TRAFODION-2868
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: sql-cmp
>Affects Versions: any
>Reporter: liyingshuai
>Assignee: Suresh Subbiah
>
> create a table with TTL option, then an error occurred:
> CREATE TABLE t_hbaseoption22(c1 INT NOT NULL, c2 varchar(5)) SALT USING 2 
> PARTITIONS ON (c1) store BY (c1) HBASE_OPTIONS (TTL='-1');
> *** ERROR[8448] Unable to access Hbase interface. Call to 
> CmpSeabaseDDL::generateHbaseOptionsArray() returned error 
> HBASE_CREATE_OPTIONS_ERROR(710). Cause: TTL. [2017-12-25 17:45:01]
> 
> as described in SQL REFERENCE MANUAL that the accepted value of TTL can 
> be '-1' (forever) and  'positive-integer'
>  
> i found the following source code, if the value is -1 then throw an error 
> and TTL in hbase source code -1 means forever:
>  
> CmpSeabaseDDLcommon.cpp
> // 代码中设置不能等于-1
> else if ((hbaseOption->key() == "TIME_TO_LIVE") || (hbaseOption->key() == 
> "TTL"))
>{
> if ((str_atoi(hbaseOption->val().data(), 
> hbaseOption->val().length()) == -1) || 
> (!hbaseCreateOptionsArray[HBASE_TTL].empty()))
> isError = TRUE;
> hbaseCreateOptionsArray[HBASE_TTL] = hbaseOption->val();



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (TRAFODION-2871) Trafodion Security Team and ML

2018-01-02 Thread Pierre Smits (JIRA) 
Pierre Smits created TRAFODION-2871:
---

 Summary: Trafodion Security Team and ML
 Key: TRAFODION-2871
 URL: https://issues.apache.org/jira/browse/TRAFODION-2871
 Project: Apache Trafodion
  Issue Type: Improvement
  Components: documentation, website
Reporter: Pierre Smits


Security threats are everywhere. We should have the constructs in place to 
address these properly, meaning:

* have a Trafodion security ml
* etc.





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (TRAFODION-2861) Remove incubating reference(s) from code base

2018-01-02 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TRAFODION-2861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307959#comment-16307959
 ] 

ASF GitHub Bot commented on TRAFODION-2861:
---

GitHub user PierreSmits opened a pull request:

https://github.com/apache/trafodion/pull/1363

TRAFODION-2861

Removal of incubator references

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/PierreSmits/trafodion master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/trafodion/pull/1363.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1363


commit 40760c78ff34a7b81ec488006eac032fe637adee
Author: Pierre Smits 
Date:   2018-01-02T12:27:26Z

TRAFODION-2861
Removal of incubator references




> Remove incubating reference(s) from code base
> -
>
> Key: TRAFODION-2861
> URL: https://issues.apache.org/jira/browse/TRAFODION-2861
> Project: Apache Trafodion
>  Issue Type: Sub-task
>  Components: documentation, website
>Reporter: Pierre Smits
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (TRAFODION-2870) DCS can not be normal visit.

2018-01-02 Thread surgingcode (JIRA) 
surgingcode created TRAFODION-2870:
--

 Summary: DCS can not be normal visit.
 Key: TRAFODION-2870
 URL: https://issues.apache.org/jira/browse/TRAFODION-2870
 Project: Apache Trafodion
  Issue Type: Bug
  Components: client-jdbc-t4
Affects Versions: 2.3
 Environment: vmware+centos 6.5
Reporter: surgingcode
 Fix For: 2.3


After deploying a standalone version of trafodion.
Java normal use jdbcT4.jar connect to the database and found that the client 
can not connect to the DCS service.
But the server trafci can be used normally.

so,I found that because the host default name is localhost.

Not feeling reasonable, stand-alone version does not modify the host name, but 
also commonly used things.

This problem occurs, suggesting that the certificate can not be downloaded 
normally, even if the server name should not be configured localhost, it should 
not prompt network error, because the core of this problem is the client and 
server can not be properly resolved to the correct server address (internet 
Address / intranet address / loop address).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)