[jira] [Commented] (TRAFODION-1988) Better java exception handling in the java layer of Trafodion
[
https://issues.apache.org/jira/browse/TRAFODION-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15363523#comment-15363523
]
ASF GitHub Bot commented on TRAFODION-1988:
---
Github user selvaganesang commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/559#discussion_r69658915
--- Diff:
core/sqf/src/seatrans/tm/hbasetmlib2/src/main/java/org/trafodion/dtm/HBaseTxClient.java
---
@@ -452,7 +438,15 @@ public short abortTransaction(final long
transactionID) throws Exception {
Object commitDDLLock = new Object();
synchronized(commitDDLLock)
{
-commitDDLLock.wait();
+ boolean loopBack = false;
+ do {
+try {
+commitDDLLock.wait();
--- End diff --
Yep. Will fix it.
> Better java exception handling in the java layer of Trafodion
> -
>
> Key: TRAFODION-1988
> URL: https://issues.apache.org/jira/browse/TRAFODION-1988
> Project: Apache Trafodion
> Issue Type: Improvement
> Components: dtm, sql-exe
>Affects Versions: 2.1-incubating
>Reporter: Selvaganesan Govindarajan
>Assignee: Selvaganesan Govindarajan
>
> Java exceptions are not handled in consistent manner in Trafodion. The SQL
> interface layer in Trafodion is capable of displaying the entire java stack
> trace to the client application when an exception is raised in java portion
> of the Trafodion/Hbase/Hdfs stack. However, there are portions of the code in
> Trafodion where the exceptions are not handled in a consistent manner. This
> JIRA attempts to fix this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TRAFODION-2091) hdfsFS filesystem usage causies abnormal behavior in Trafodion
[
https://issues.apache.org/jira/browse/TRAFODION-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15363504#comment-15363504
]
ASF GitHub Bot commented on TRAFODION-2091:
---
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/574#discussion_r69658107
--- Diff: core/sql/src/main/java/org/trafodion/sql/HBulkLoadClient.java ---
@@ -475,10 +475,12 @@ public boolean release( ) throws IOException {
writer.close();
writer = null;
}
-if (fileSys !=null)
+// sss This is one place that is unconditionally closing the
--- End diff --
WIll change this.
> hdfsFS filesystem usage causies abnormal behavior in Trafodion
> ---
>
> Key: TRAFODION-2091
> URL: https://issues.apache.org/jira/browse/TRAFODION-2091
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-general
>Affects Versions: 2.1-incubating
>Reporter: Sandhya Sundaresan
>Assignee: Sandhya Sundaresan
>
> Several components use the libhdfs API to perform operations on hdfs files in
> Trafodion. Some places use the JNI API as well to access hdfs.
> The inconsistency observed was ehen one component cached and kept the hdfsFS
> hande for it's use another part of the code would perform a "close" or a
> dsconnect and prematurely close the hdfsFS handle which is currently being
> used. The reason is that the hdfs layer seems to cache the hdfsFS handle
> underneathThe solution now is to keep a list of hdfsFS handles at the global
> context level. All disconnect calls and JNI close calls are remmoved. The
> hdfHS handles will get cleaned up either at context destructor time or at
> endSession time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TRAFODION-2094) ODB TRUNCATE option takes a long time
[ https://issues.apache.org/jira/browse/TRAFODION-2094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15363316#comment-15363316 ] ASF GitHub Bot commented on TRAFODION-2094: --- GitHub user sureshsubbiah opened a pull request: https://github.com/apache/incubator-trafodion/pull/576 [TRAFODION-2094] Truncate option slow in ODB Mispaced paranthesis causes us to take the generic database codepath, with dbt = 0, instead of dbt = 8 for Trafodion. All compiler warnings for ODB are now fixed or suppressed. You can merge this pull request into a Git repository by running: $ git pull https://github.com/sureshsubbiah/incubator-trafodion traf-2094 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafodion/pull/576.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #576 commit 7166c64db17e3b5f457b55dc1c491a01b070dcbc Author: Suresh Subbiah Date: 2016-07-05T21:42:26Z [TRAFODION-2094] Truncate option slow in ODB Misplaced paranthesis caused ODB to not recognize Trafodion database. We were going through the generic codepath. Also made some changes to remove/suppress compiler warnings. > ODB TRUNCATE option takes a long time > - > > Key: TRAFODION-2094 > URL: https://issues.apache.org/jira/browse/TRAFODION-2094 > Project: Apache Trafodion > Issue Type: Bug > Components: db-utility-odb >Reporter: Suresh Subbiah >Assignee: Suresh Subbiah > > Truncate option of ODB uses DELETE command. This works too slowly for large > tables and may run into transaction issues. > Will be changed to use PURGEDATA instead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1182) LP Bug: 1444684 - Drop schema fails and objects become un-droppable with invalid state
[ https://issues.apache.org/jira/browse/TRAFODION-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandhya Sundaresan resolved TRAFODION-1182. --- Resolution: Fixed The DDL transaction checkin should address this issue. > LP Bug: 1444684 - Drop schema fails and objects become un-droppable with > invalid state > -- > > Key: TRAFODION-1182 > URL: https://issues.apache.org/jira/browse/TRAFODION-1182 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-exe >Reporter: Weishiun Tsai >Assignee: Prashanth Vasudev >Priority: Critical > Fix For: 2.1-incubating > > > We still see random situations that an SQL object would get into an invalid > state and become un-droppable. When it happens, using the cleanup utility to > clean them up becomes the only option. This is a rare occasion that we were > able to capture a short script to reproduce it. As shown here. The sequence > of statements causes drop schema at the end to return TMF 97 error. After > that, any attempt to drop an object in the schema would return the 4254 > ‘invalid state’ error. The user then has to rely on the cleanup utility to > clean them up manually. > SQL>drop schema mytest cascade; > *** ERROR[8606] Transaction subsystem TMF returned error 97 on a commit > transaction. [2015-04-15 20:10:00] > SQL>set schema mytest; > --- SQL operation complete. > SQL>get tables; > Tables in Schema TRAFODION.MYTEST > = > NIZ000 > NIZ001 > SB_HISTOGRAMS > SB_HISTOGRAM_INTERVALS > --- SQL operation complete. > SQL>drop table niz000; > *** ERROR[4254] Object TRAFODION.MYTEST.NIZ000 has invalid state and cannot > be accessed. Use cleanup command to drop it. [2015-04-15 20:10:35] > SQL>drop table niz001; > *** ERROR[4254] Object TRAFODION.MYTEST.NIZ001 has invalid state and cannot > be accessed. Use cleanup command to drop it. [2015-04-15 20:10:40] > SQL>drop table sb_histograms; > *** ERROR[4254] Object TRAFODION.MYTEST.SB_HISTOGRAMS has invalid state and > cannot be accessed. Use cleanup command to drop it. [2015-04-15 20:10:43] > SQL>drop table sb_histogram_intervals; > *** ERROR[4254] Object TRAFODION.MYTEST.SB_HISTOGRAM_INTERVALS has invalid > state and cannot be accessed. Use cleanup command to drop it. [2015-04-15 > 20:10:51] > This is seen on the v0415 build installed on a workstation. It’s fairly > reproducible with this build. Unfortunately, this script can only reproduce > this problem from trafci. Running it from sqlci cannot reproduce it. It’s > not clear why, but it’s possible that mxosrvr handles things differently. > - > Here are the steps to start trafci from your workstation instance: > Copy and paste the script bellow to an obey file, say mytest.sql > cd $MY_SQROOT/trafci/bin > trafci.sh -h localhost: -u dontcare -p dontcare > SQL> obey mytest.sql; > - > Here is the entire script to reproduce it: > log mytest.log clear; > create schema mytest; > set schema mytest; > create table niz000 > ( > seqno integer not nullnot droppable, > smin1 smallintsigned default null, > smin2 smallintunsigneddefault null, > inte1 integer signed default null, > inte2 integer unsigneddefault null, > lint1 largeintdefault null, > lint2 largeintdefault null, > nume1 numeric(7) unsigneddefault null, > nume2 numeric(9,3)unsigneddefault null, > nume3 numeric(18) signed default null, > nume4 numeric(18,15) signed default null, > deci1 decimal(3) unsigneddefault null, > deci2 decimal(18,0) signed default null, > deci3 decimal(18,9) signed default null, > pict1 pic s9(18) compdefault null, > pict2 pic sv9(2) compdefault null, > pict3 pic s9(13)v9(5) default null, > pict4 pic 9(3)v9(6) default null, > flot1 float (12) default null, > flot2 float (52) default null, > real1 realdefault null, > real2 realdefault null, > dblp1 double precisiondefault null, > dblp2 double precisiondefault null, > primary key (seqno) > ); > create index smin_idx on niz000(smin1); > create index inte_idx on niz000(inte1); > create index lint_idx on niz000(lint1); > create index nume_idx on niz000(nume1); > create index deci_idx on niz000(deci1); > create index
[jira] [Resolved] (TRAFODION-1239) LP Bug: 1455670 - insert using params to varchar column of > 32k colsize fails
[ https://issues.apache.org/jira/browse/TRAFODION-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandhya Sundaresan resolved TRAFODION-1239. --- Resolution: Fixed > LP Bug: 1455670 - insert using params to varchar column of > 32k colsize fails > -- > > Key: TRAFODION-1239 > URL: https://issues.apache.org/jira/browse/TRAFODION-1239 > Project: Apache Trafodion > Issue Type: Bug > Components: client-jdbc-t4 >Reporter: Aruna Sadashiva >Assignee: Arvind Narain >Priority: Blocker > Fix For: 2.1-incubating > > > Using bound params, insert into varchar column of colsize >32k fails with > string overflow error. > To reproduce using Trafci : > SQL>CREATE TABLE TAB3 ( A INT, B VARCHAR(4)); > --- SQL operation complete. > SQL>prepare s from insert into tab3 values(?a, ?b); > --- SQL command prepared. > SQL>set param ?a 100; > SQL>set param ?b 'aaaZZ'; > SQL>execute s; > *** ERROR[8402] A string overflow occurred during the evaluation of a > character expression. [2015-05-15 21:27:01] > Steps from sqlci: > >>create table a3(a int, b varchar(32000)); > --- SQL operation complete. > >>prepare s1 from insert into a3 values(?a,?b); > --- SQL command prepared. > >>set param ?a 1; > >>set param ?b 'aaaZZ'; > >>execute s1; > --- 1 row(s) inserted. > >>create table a4(a int, b varchar(4)); > --- SQL operation complete. > >>prepare s2 from insert into a4 values(?a,?b); > --- SQL command prepared. > >>set param ?a 1; > >>set param ?b 'bbbZZ'; > >>execute s2; > *** ERROR[8402] A string overflow occurred during the evaluation of a > character expression. > --- 0 row(s) inserted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-2069) 'sqcheck' script should not report on hosts in TRAF_EXCLUDE_LIST
[ https://issues.apache.org/jira/browse/TRAFODION-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gonzalo E Correa resolved TRAFODION-2069. - Resolution: Fixed > 'sqcheck' script should not report on hosts in TRAF_EXCLUDE_LIST > > > Key: TRAFODION-2069 > URL: https://issues.apache.org/jira/browse/TRAFODION-2069 > Project: Apache Trafodion > Issue Type: Bug > Components: foundation >Affects Versions: 2.1-incubating >Reporter: Gonzalo E Correa >Assignee: Gonzalo E Correa > Fix For: 2.1-incubating > > > The 'sqcheck' script does not evaluate the contents of the TRAF_EXCLUDE_LIST > environment variable when the elasticity feature is used. Script logic > calculating configured process type counts needs to account for non-existent > nodes to calculate number of configured processes accurately. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-2008) Support for external file LOBs in Trafodion
[ https://issues.apache.org/jira/browse/TRAFODION-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandhya Sundaresan resolved TRAFODION-2008. --- Resolution: Fixed > Support for external file LOBs in Trafodion > --- > > Key: TRAFODION-2008 > URL: https://issues.apache.org/jira/browse/TRAFODION-2008 > Project: Apache Trafodion > Issue Type: New Feature > Components: sql-exe >Reporter: Sandhya Sundaresan >Assignee: Sandhya Sundaresan > Fix For: 2.1-incubating > > > Add support for a new lobtype and lob function (externaltolob) that will > allow files to reside in HDFS outside o Trafodion. The file is not imported > nor part of Trafodion but the file lcoator will be a part of the Trafodion > column. > Once a column is declared with the "external" attribute, it cannot contain an > internal LOB type . All data that's part of that column will have to reside > outside Trafodion. Similarly if a column is declared as a regular blob/clb, > then an external file cannot be linked to that column using the externaltolob > function. > Also external file LOBs can be inserted or updated. They cannot be > "appended". So each a particular LOB value in a column can be pointing to > just one external file . It cannot point ot multiple files so append is not > allowed. > If an external LOB is updated or modified by the user outside Trafodion,t he > user will need to do an "update" to update the information in the Trafodion > LOB column. If not, the information in the Trafodion table will be stale. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-2094) ODB TRUNCATE option takes a long time
Suresh Subbiah created TRAFODION-2094: - Summary: ODB TRUNCATE option takes a long time Key: TRAFODION-2094 URL: https://issues.apache.org/jira/browse/TRAFODION-2094 Project: Apache Trafodion Issue Type: Bug Components: db-utility-odb Reporter: Suresh Subbiah Assignee: Suresh Subbiah Truncate option of ODB uses DELETE command. This works too slowly for large tables and may run into transaction issues. Will be changed to use PURGEDATA instead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1609) Change swstatus to give a simple "up" or "down" answer, instead of the current, confusing display
[ https://issues.apache.org/jira/browse/TRAFODION-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Jin resolved TRAFODION-1609. - Resolution: Fixed Fix Version/s: 2.1-incubating > Change swstatus to give a simple "up" or "down" answer, instead of the > current, confusing display > - > > Key: TRAFODION-1609 > URL: https://issues.apache.org/jira/browse/TRAFODION-1609 > Project: Apache Trafodion > Issue Type: Improvement > Components: sql-general >Reporter: Hans Zeller >Assignee: Eason Zhang >Priority: Minor > Fix For: 2.1-incubating > > > The "swstatus" script used in cases where we did install_local_hadoop counts > the MySQL processes and then simply invokes jps. > It would be better if the script would check specifically for HMaster, > NameNode, DataNode and maybe others, like NodeManager and ResourceManager. If > any are missing, it should print out which ones are missing, otherwise it > should just say something like "processes are up". > It should also return an exit code of 0 if everything is up and 1 if some > processes are missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1609) Change swstatus to give a simple "up" or "down" answer, instead of the current, confusing display
[ https://issues.apache.org/jira/browse/TRAFODION-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Jin updated TRAFODION-1609: Assignee: Eason Zhang (was: Jian Jin) > Change swstatus to give a simple "up" or "down" answer, instead of the > current, confusing display > - > > Key: TRAFODION-1609 > URL: https://issues.apache.org/jira/browse/TRAFODION-1609 > Project: Apache Trafodion > Issue Type: Improvement > Components: sql-general >Reporter: Hans Zeller >Assignee: Eason Zhang >Priority: Minor > > The "swstatus" script used in cases where we did install_local_hadoop counts > the MySQL processes and then simply invokes jps. > It would be better if the script would check specifically for HMaster, > NameNode, DataNode and maybe others, like NodeManager and ResourceManager. If > any are missing, it should print out which ones are missing, otherwise it > should just say something like "processes are up". > It should also return an exit code of 0 if everything is up and 1 if some > processes are missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1676) support better runtime error message when a SQL function meet fital error
[ https://issues.apache.org/jira/browse/TRAFODION-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362591#comment-15362591 ] liu ming commented on TRAFODION-1676: - This PR (564) enhance error messages for these SQL Errors: EXE_CONVERTTIMESTAMP_ERROR = 8405, EXE_JULIANTIMESTAMP_ERROR = 8407, EXE_CONVERT_STRING_ERROR = 8413 EXE_CONVERT_DATETIME_ERROR = 8415, By displaying the offending source data content, for CONVERT_STRING_ERROR, because it always contains multi-byte character, so display the data in HEX mode. > support better runtime error message when a SQL function meet fital error > - > > Key: TRAFODION-1676 > URL: https://issues.apache.org/jira/browse/TRAFODION-1676 > Project: Apache Trafodion > Issue Type: Improvement > Components: sql-exe >Reporter: liu ming >Assignee: liu ming >Priority: Minor > > A sql contains some SQL function, for example UPPER(), when the input data of > UPPER() contains invalid data, UPPER() will fail. In this case, the whole > query abort, and the error message cannot tell which row has invalid data, so > it is very hard to fix the problem. > It will be good to have row id or the real value of the error row in the > error message to help trouble shooting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-2034) Cannot output bad records in the loaded file even if specified file for storing bad records
[ https://issues.apache.org/jira/browse/TRAFODION-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Jin resolved TRAFODION-2034. - Resolution: Fixed Fix Version/s: 2.1-incubating > Cannot output bad records in the loaded file even if specified file for > storing bad records > --- > > Key: TRAFODION-2034 > URL: https://issues.apache.org/jira/browse/TRAFODION-2034 > Project: Apache Trafodion > Issue Type: Bug > Components: db-utility-odb > Environment: centos6.7 HBase 1.0.0-cdh5.4.8 >Reporter: zhangliang >Assignee: zhangliang >Priority: Minor > Fix For: 2.1-incubating > > > This issue is from mantis 330. > Load file where is bad record in it, with option bad=output_data/bad_records > to hold the bad records, but the result is the file output_data/bad_records > generated but it is empty lines. > [root@suse-1 odb_test]# ./odb64luo -u trafodion -p traf123 -d traf -l > src=output_data/ext_person3.csv:pre=@scripts/ddl_person3.sql:tgt=trafodion.odb_test.person3:max=1000:rows=5000:parallel=5:loadcmd=UL:fs=\|:sq=\":bad=output_data/bad_records > odb [2016-04-25 19:17:05]: starting ODBC connection(s)... 0 1 2 3 4 5 > [0.0.0]Executing: 'drop table TRAFODION.odb_test.person3;' > [0.0.0]--- command executed in 7.279s (prep 0.001s, exec 7.278s, fetch > 0.000s/0.000s) > [0.0.1]Executing: 'CREATE TABLE TRAFODION.odb_test."PERSON3" ( > PID BIGINT SIGNED NOT NULL > ,FNAME CHAR(20) NOT NULL > ,LNAME CHAR(20) NOT NULL > ,COUNTRY VARCHAR(40) NOT NULL > ,CITY VARCHAR(40) NOT NULL > ,BDATE DATE NOT NULL > ,SEX CHAR(1) NOT NULL > ,EMAIL VARCHAR(40) NOT NULL > ,SALARY NUMERIC(9,2) NOT NULL > ,EMPL VARCHAR(40) NOT NULL > ,NOTES VARCHAR(80) > ,LOADTS TIMESTAMP(0) > ,PRIMARY KEY (PID) > );' > [0.0.1]--- command executed in 1.388s (prep 0.002s, exec 1.386s, fetch > 0.000s/0.000s) > Connected to Trafodion > [1] odb [Oloadbuff(9438)] - Error loading row 5 (State: 23000, Native 0) > [Trafodion ODBC Driver] GENERAL ERROR. Null Value in a non nullable column. > Row: 5 Column: 1 > [1] 999 records inserted [commit] > [0] odb version 1.1.0 Load statistics: > [0] Target table: TRAFODION.ODB_TEST.PERSON3 > [0] Source: output_data/ext_person3.csv > [0] Pre-loading time: 10.920 s (00:00:10.920) > [0] Loading time: 0.127 s(00:00:00.127) > [0] Total records read: 1,000 > [0] Total records inserted: 999 > [0] Total number of columns: 12 > [0] Total bytes read: 180,517 > [0] Average input row size: 180.5 B > [0] ODBC row size: 341 B (data) + 96 B (len ind) > [0] Rowset size: 1,000 > [0] Rowset buffer size: 426.76 KiB > [0] Load throughput (real data): 1,388.080 KiB/s > [0] Load throughput (ODBC): 2,619.487 KiB/s > [0] Reader Total/Wait Cycles: 1/0 > odb [2016-04-25 19:17:16]: exiting. Session Elapsed time 11.060 seconds > (00:00:11.060) > SQL>showddl person3; > > CREATE TABLE TRAFODION.ODB_TEST.PERSON3 > ( > PID LARGEINT NO DEFAULT NOT NULL NOT DROPPABLE > SERIALIZED > , FNAME CHAR(20) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , LNAME CHAR(20) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , COUNTRY VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , CITY VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , BDATE DATE NO DEFAULT NOT NULL NOT DROPPABLE NOT > SERIALIZED > , SEX CHAR(1) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , EMAIL VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , SALARY NUMERIC(9, 2) NO DEFAULT NOT NULL NOT > DROPPABLE SERIALIZED > , EMPL VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , NOTES VARCHAR(80) CHARACTER SET ISO88591 COLLATE > DEFAULT DEFAULT NULL SERIALIZED > , LOADTS TIMESTAMP(0) DEFAULT NULL NOT SERIALIZED > , PRIMARY KEY (PID ASC) > ) > ; > --- SQL operation complete. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (TRAFODION-1609) Change swstatus to give a simple "up" or "down" answer, instead of the current, confusing display
[ https://issues.apache.org/jira/browse/TRAFODION-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TRAFODION-1609 started by Jian Jin. --- > Change swstatus to give a simple "up" or "down" answer, instead of the > current, confusing display > - > > Key: TRAFODION-1609 > URL: https://issues.apache.org/jira/browse/TRAFODION-1609 > Project: Apache Trafodion > Issue Type: Improvement > Components: sql-general >Reporter: Hans Zeller >Assignee: Jian Jin >Priority: Minor > > The "swstatus" script used in cases where we did install_local_hadoop counts > the MySQL processes and then simply invokes jps. > It would be better if the script would check specifically for HMaster, > NameNode, DataNode and maybe others, like NodeManager and ResourceManager. If > any are missing, it should print out which ones are missing, otherwise it > should just say something like "processes are up". > It should also return an exit code of 0 if everything is up and 1 if some > processes are missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-2093) Licensing issues found during review of release 2.0.1
[ https://issues.apache.org/jira/browse/TRAFODION-2093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362572#comment-15362572 ] Roberta Marton commented on TRAFODION-2093: --- Link to email: http://mail-archives.apache.org/mod_mbox/incubator-general/201607.mbox/ajax/%3C2CF860BA-8473-4F9F-A570-72931EE7FB0A%40classsoftware.com%3E > Licensing issues found during review of release 2.0.1 > - > > Key: TRAFODION-2093 > URL: https://issues.apache.org/jira/browse/TRAFODION-2093 > Project: Apache Trafodion > Issue Type: Bug > Components: Build Infrastructure >Reporter: Roberta Marton > > The following issues were reported during the review our our 2.0.1 release > package. They should be fixed for the next release: > "I’m also a little confused by “trafodion_license” in the installer as it > states: > “Project Trafodion includes third party copyrighted software components, > which may be licensed under third party and or open source license terms as > set forth in the LICENSE.TXT file accompanying the Project Trafodion > distribution or as indicated in the applicable source code file.” > But LICENSE (not LICENSE.TXT) has no 3rd party software. Is the > trafodion_license file even needed? > With the addition of the new 3 clause BSD licenses the number “2” is the > second license point has been dropped, you might want to fix that." -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-2092) Names of installer tar file in release package incorrect
[ https://issues.apache.org/jira/browse/TRAFODION-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362571#comment-15362571 ] Roberta Marton commented on TRAFODION-2092: --- link to email: http://mail-archives.apache.org/mod_mbox/incubator-general/201607.mbox/ajax/%3C2CF860BA-8473-4F9F-A570-72931EE7FB0A%40classsoftware.com%3E > Names of installer tar file in release package incorrect > > > Key: TRAFODION-2092 > URL: https://issues.apache.org/jira/browse/TRAFODION-2092 > Project: Apache Trafodion > Issue Type: Bug > Components: Build Infrastructure >Reporter: Roberta Marton > > A comment received from the IPMC during our release process: > Is there any reason for the installer to not include apache-trafodion and > incubating? Please fix for next release. > In addition, we should make all the names for our released tar file follow > the same format. There seems to be 3 different formats. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-2092) Names of installer tar file in release package incorrect
Roberta Marton created TRAFODION-2092: - Summary: Names of installer tar file in release package incorrect Key: TRAFODION-2092 URL: https://issues.apache.org/jira/browse/TRAFODION-2092 Project: Apache Trafodion Issue Type: Bug Components: Build Infrastructure Reporter: Roberta Marton A comment received from the IPMC during our release process: Is there any reason for the installer to not include apache-trafodion and incubating? Please fix for next release. In addition, we should make all the names for our released tar file follow the same format. There seems to be 3 different formats. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-2093) Licensing issues found during review of release 2.0.1
Roberta Marton created TRAFODION-2093: - Summary: Licensing issues found during review of release 2.0.1 Key: TRAFODION-2093 URL: https://issues.apache.org/jira/browse/TRAFODION-2093 Project: Apache Trafodion Issue Type: Bug Components: Build Infrastructure Reporter: Roberta Marton The following issues were reported during the review our our 2.0.1 release package. They should be fixed for the next release: "I’m also a little confused by “trafodion_license” in the installer as it states: “Project Trafodion includes third party copyrighted software components, which may be licensed under third party and or open source license terms as set forth in the LICENSE.TXT file accompanying the Project Trafodion distribution or as indicated in the applicable source code file.” But LICENSE (not LICENSE.TXT) has no 3rd party software. Is the trafodion_license file even needed? With the addition of the new 3 clause BSD licenses the number “2” is the second license point has been dropped, you might want to fix that." -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-2091) hdfsFS filesystem usage causies abnormal behavior in Trafodion
[ https://issues.apache.org/jira/browse/TRAFODION-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362524#comment-15362524 ] ASF GitHub Bot commented on TRAFODION-2091: --- Github user sureshsubbiah commented on a diff in the pull request: https://github.com/apache/incubator-trafodion/pull/574#discussion_r69567976 --- Diff: core/sql/cli/Context.cpp --- @@ -298,6 +299,8 @@ ContextCli::ContextCli(CliGlobals *cliGlobals) seqGen_ = new(exCollHeap()) SequenceValueGenerator(exCollHeap()); + hdfsHandleList_ = new(exCollHeap()) HashQueue(exCollHeap(), 50); // The hfsHandleList_ represents a list of distict hdfs Handles with unique hdfs port numbers and server names. Assume not more than 50 hdfsServers could be connected in the Trafodion setup. These will get initialized the first time access is made to a particular hdfs server. This list gets cleaned up when the thread exits. --- End diff -- Can the literal 50 be a constant in a subsequent checkin? > hdfsFS filesystem usage causies abnormal behavior in Trafodion > --- > > Key: TRAFODION-2091 > URL: https://issues.apache.org/jira/browse/TRAFODION-2091 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-general >Affects Versions: 2.1-incubating >Reporter: Sandhya Sundaresan >Assignee: Sandhya Sundaresan > > Several components use the libhdfs API to perform operations on hdfs files in > Trafodion. Some places use the JNI API as well to access hdfs. > The inconsistency observed was ehen one component cached and kept the hdfsFS > hande for it's use another part of the code would perform a "close" or a > dsconnect and prematurely close the hdfsFS handle which is currently being > used. The reason is that the hdfs layer seems to cache the hdfsFS handle > underneathThe solution now is to keep a list of hdfsFS handles at the global > context level. All disconnect calls and JNI close calls are remmoved. The > hdfHS handles will get cleaned up either at context destructor time or at > endSession time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-2091) hdfsFS filesystem usage causies abnormal behavior in Trafodion
[ https://issues.apache.org/jira/browse/TRAFODION-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362521#comment-15362521 ] ASF GitHub Bot commented on TRAFODION-2091: --- Github user sureshsubbiah commented on a diff in the pull request: https://github.com/apache/incubator-trafodion/pull/574#discussion_r69567539 --- Diff: core/sql/regress/hive/TEST005 --- @@ -314,7 +314,7 @@ select a from hive.hive.thive; select * from hive.hive.thive; -- truncate of hive data -cqd query_cache '0'; +--cqd query_cache '0'; --- End diff -- Glad that the test is being run with qcache ON now. > hdfsFS filesystem usage causies abnormal behavior in Trafodion > --- > > Key: TRAFODION-2091 > URL: https://issues.apache.org/jira/browse/TRAFODION-2091 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-general >Affects Versions: 2.1-incubating >Reporter: Sandhya Sundaresan >Assignee: Sandhya Sundaresan > > Several components use the libhdfs API to perform operations on hdfs files in > Trafodion. Some places use the JNI API as well to access hdfs. > The inconsistency observed was ehen one component cached and kept the hdfsFS > hande for it's use another part of the code would perform a "close" or a > dsconnect and prematurely close the hdfsFS handle which is currently being > used. The reason is that the hdfs layer seems to cache the hdfsFS handle > underneathThe solution now is to keep a list of hdfsFS handles at the global > context level. All disconnect calls and JNI close calls are remmoved. The > hdfHS handles will get cleaned up either at context destructor time or at > endSession time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-2091) hdfsFS filesystem usage causies abnormal behavior in Trafodion
[
https://issues.apache.org/jira/browse/TRAFODION-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362519#comment-15362519
]
ASF GitHub Bot commented on TRAFODION-2091:
---
Github user sureshsubbiah commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/574#discussion_r69567472
--- Diff: core/sql/src/main/java/org/trafodion/sql/HBulkLoadClient.java ---
@@ -475,10 +475,12 @@ public boolean release( ) throws IOException {
writer.close();
writer = null;
}
-if (fileSys !=null)
+// sss This is one place that is unconditionally closing the
--- End diff --
Should we remove "sss". It is a minor nit, and I know it is an oversight.
Maybe this can be done in a subsequent checkin?
> hdfsFS filesystem usage causies abnormal behavior in Trafodion
> ---
>
> Key: TRAFODION-2091
> URL: https://issues.apache.org/jira/browse/TRAFODION-2091
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-general
>Affects Versions: 2.1-incubating
>Reporter: Sandhya Sundaresan
>Assignee: Sandhya Sundaresan
>
> Several components use the libhdfs API to perform operations on hdfs files in
> Trafodion. Some places use the JNI API as well to access hdfs.
> The inconsistency observed was ehen one component cached and kept the hdfsFS
> hande for it's use another part of the code would perform a "close" or a
> dsconnect and prematurely close the hdfsFS handle which is currently being
> used. The reason is that the hdfs layer seems to cache the hdfsFS handle
> underneathThe solution now is to keep a list of hdfsFS handles at the global
> context level. All disconnect calls and JNI close calls are remmoved. The
> hdfHS handles will get cleaned up either at context destructor time or at
> endSession time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TRAFODION-2034) Cannot output bad records in the loaded file even if specified file for storing bad records
[ https://issues.apache.org/jira/browse/TRAFODION-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362511#comment-15362511 ] ASF GitHub Bot commented on TRAFODION-2034: --- Github user asfgit closed the pull request at: https://github.com/apache/incubator-trafodion/pull/570 > Cannot output bad records in the loaded file even if specified file for > storing bad records > --- > > Key: TRAFODION-2034 > URL: https://issues.apache.org/jira/browse/TRAFODION-2034 > Project: Apache Trafodion > Issue Type: Bug > Components: db-utility-odb > Environment: centos6.7 HBase 1.0.0-cdh5.4.8 >Reporter: zhangliang >Assignee: zhangliang >Priority: Minor > > This issue is from mantis 330. > Load file where is bad record in it, with option bad=output_data/bad_records > to hold the bad records, but the result is the file output_data/bad_records > generated but it is empty lines. > [root@suse-1 odb_test]# ./odb64luo -u trafodion -p traf123 -d traf -l > src=output_data/ext_person3.csv:pre=@scripts/ddl_person3.sql:tgt=trafodion.odb_test.person3:max=1000:rows=5000:parallel=5:loadcmd=UL:fs=\|:sq=\":bad=output_data/bad_records > odb [2016-04-25 19:17:05]: starting ODBC connection(s)... 0 1 2 3 4 5 > [0.0.0]Executing: 'drop table TRAFODION.odb_test.person3;' > [0.0.0]--- command executed in 7.279s (prep 0.001s, exec 7.278s, fetch > 0.000s/0.000s) > [0.0.1]Executing: 'CREATE TABLE TRAFODION.odb_test."PERSON3" ( > PID BIGINT SIGNED NOT NULL > ,FNAME CHAR(20) NOT NULL > ,LNAME CHAR(20) NOT NULL > ,COUNTRY VARCHAR(40) NOT NULL > ,CITY VARCHAR(40) NOT NULL > ,BDATE DATE NOT NULL > ,SEX CHAR(1) NOT NULL > ,EMAIL VARCHAR(40) NOT NULL > ,SALARY NUMERIC(9,2) NOT NULL > ,EMPL VARCHAR(40) NOT NULL > ,NOTES VARCHAR(80) > ,LOADTS TIMESTAMP(0) > ,PRIMARY KEY (PID) > );' > [0.0.1]--- command executed in 1.388s (prep 0.002s, exec 1.386s, fetch > 0.000s/0.000s) > Connected to Trafodion > [1] odb [Oloadbuff(9438)] - Error loading row 5 (State: 23000, Native 0) > [Trafodion ODBC Driver] GENERAL ERROR. Null Value in a non nullable column. > Row: 5 Column: 1 > [1] 999 records inserted [commit] > [0] odb version 1.1.0 Load statistics: > [0] Target table: TRAFODION.ODB_TEST.PERSON3 > [0] Source: output_data/ext_person3.csv > [0] Pre-loading time: 10.920 s (00:00:10.920) > [0] Loading time: 0.127 s(00:00:00.127) > [0] Total records read: 1,000 > [0] Total records inserted: 999 > [0] Total number of columns: 12 > [0] Total bytes read: 180,517 > [0] Average input row size: 180.5 B > [0] ODBC row size: 341 B (data) + 96 B (len ind) > [0] Rowset size: 1,000 > [0] Rowset buffer size: 426.76 KiB > [0] Load throughput (real data): 1,388.080 KiB/s > [0] Load throughput (ODBC): 2,619.487 KiB/s > [0] Reader Total/Wait Cycles: 1/0 > odb [2016-04-25 19:17:16]: exiting. Session Elapsed time 11.060 seconds > (00:00:11.060) > SQL>showddl person3; > > CREATE TABLE TRAFODION.ODB_TEST.PERSON3 > ( > PID LARGEINT NO DEFAULT NOT NULL NOT DROPPABLE > SERIALIZED > , FNAME CHAR(20) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , LNAME CHAR(20) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , COUNTRY VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , CITY VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , BDATE DATE NO DEFAULT NOT NULL NOT DROPPABLE NOT > SERIALIZED > , SEX CHAR(1) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , EMAIL VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , SALARY NUMERIC(9, 2) NO DEFAULT NOT NULL NOT > DROPPABLE SERIALIZED > , EMPL VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , NOTES VARCHAR(80) CHARACTER SET ISO88591 COLLATE > DEFAULT DEFAULT NULL SERIALIZED > , LOADTS TIMESTAMP(0) DEFAULT NULL NOT SERIALIZED > , PRIMARY KEY (PID ASC) > ) > ; > --- SQL operation complete. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1609) Change swstatus to give a simple "up" or "down" answer, instead of the current, confusing display
[ https://issues.apache.org/jira/browse/TRAFODION-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362502#comment-15362502 ] ASF GitHub Bot commented on TRAFODION-1609: --- Github user asfgit closed the pull request at: https://github.com/apache/incubator-trafodion/pull/562 > Change swstatus to give a simple "up" or "down" answer, instead of the > current, confusing display > - > > Key: TRAFODION-1609 > URL: https://issues.apache.org/jira/browse/TRAFODION-1609 > Project: Apache Trafodion > Issue Type: Improvement > Components: sql-general >Reporter: Hans Zeller >Assignee: Eason Zhang >Priority: Minor > > The "swstatus" script used in cases where we did install_local_hadoop counts > the MySQL processes and then simply invokes jps. > It would be better if the script would check specifically for HMaster, > NameNode, DataNode and maybe others, like NodeManager and ResourceManager. If > any are missing, it should print out which ones are missing, otherwise it > should just say something like "processes are up". > It should also return an exit code of 0 if everything is up and 1 if some > processes are missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1676) support better runtime error message when a SQL function meet fital error
[ https://issues.apache.org/jira/browse/TRAFODION-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362491#comment-15362491 ] ASF GitHub Bot commented on TRAFODION-1676: --- Github user asfgit closed the pull request at: https://github.com/apache/incubator-trafodion/pull/564 > support better runtime error message when a SQL function meet fital error > - > > Key: TRAFODION-1676 > URL: https://issues.apache.org/jira/browse/TRAFODION-1676 > Project: Apache Trafodion > Issue Type: Improvement > Components: sql-exe >Reporter: liu ming >Assignee: liu ming >Priority: Minor > > A sql contains some SQL function, for example UPPER(), when the input data of > UPPER() contains invalid data, UPPER() will fail. In this case, the whole > query abort, and the error message cannot tell which row has invalid data, so > it is very hard to fix the problem. > It will be good to have row id or the real value of the error row in the > error message to help trouble shooting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-2034) Cannot output bad records in the loaded file even if specified file for storing bad records
[ https://issues.apache.org/jira/browse/TRAFODION-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Jin updated TRAFODION-2034: Assignee: zhangliang (was: Jian Jin) > Cannot output bad records in the loaded file even if specified file for > storing bad records > --- > > Key: TRAFODION-2034 > URL: https://issues.apache.org/jira/browse/TRAFODION-2034 > Project: Apache Trafodion > Issue Type: Bug > Components: db-utility-odb > Environment: centos6.7 HBase 1.0.0-cdh5.4.8 >Reporter: zhangliang >Assignee: zhangliang >Priority: Minor > > This issue is from mantis 330. > Load file where is bad record in it, with option bad=output_data/bad_records > to hold the bad records, but the result is the file output_data/bad_records > generated but it is empty lines. > [root@suse-1 odb_test]# ./odb64luo -u trafodion -p traf123 -d traf -l > src=output_data/ext_person3.csv:pre=@scripts/ddl_person3.sql:tgt=trafodion.odb_test.person3:max=1000:rows=5000:parallel=5:loadcmd=UL:fs=\|:sq=\":bad=output_data/bad_records > odb [2016-04-25 19:17:05]: starting ODBC connection(s)... 0 1 2 3 4 5 > [0.0.0]Executing: 'drop table TRAFODION.odb_test.person3;' > [0.0.0]--- command executed in 7.279s (prep 0.001s, exec 7.278s, fetch > 0.000s/0.000s) > [0.0.1]Executing: 'CREATE TABLE TRAFODION.odb_test."PERSON3" ( > PID BIGINT SIGNED NOT NULL > ,FNAME CHAR(20) NOT NULL > ,LNAME CHAR(20) NOT NULL > ,COUNTRY VARCHAR(40) NOT NULL > ,CITY VARCHAR(40) NOT NULL > ,BDATE DATE NOT NULL > ,SEX CHAR(1) NOT NULL > ,EMAIL VARCHAR(40) NOT NULL > ,SALARY NUMERIC(9,2) NOT NULL > ,EMPL VARCHAR(40) NOT NULL > ,NOTES VARCHAR(80) > ,LOADTS TIMESTAMP(0) > ,PRIMARY KEY (PID) > );' > [0.0.1]--- command executed in 1.388s (prep 0.002s, exec 1.386s, fetch > 0.000s/0.000s) > Connected to Trafodion > [1] odb [Oloadbuff(9438)] - Error loading row 5 (State: 23000, Native 0) > [Trafodion ODBC Driver] GENERAL ERROR. Null Value in a non nullable column. > Row: 5 Column: 1 > [1] 999 records inserted [commit] > [0] odb version 1.1.0 Load statistics: > [0] Target table: TRAFODION.ODB_TEST.PERSON3 > [0] Source: output_data/ext_person3.csv > [0] Pre-loading time: 10.920 s (00:00:10.920) > [0] Loading time: 0.127 s(00:00:00.127) > [0] Total records read: 1,000 > [0] Total records inserted: 999 > [0] Total number of columns: 12 > [0] Total bytes read: 180,517 > [0] Average input row size: 180.5 B > [0] ODBC row size: 341 B (data) + 96 B (len ind) > [0] Rowset size: 1,000 > [0] Rowset buffer size: 426.76 KiB > [0] Load throughput (real data): 1,388.080 KiB/s > [0] Load throughput (ODBC): 2,619.487 KiB/s > [0] Reader Total/Wait Cycles: 1/0 > odb [2016-04-25 19:17:16]: exiting. Session Elapsed time 11.060 seconds > (00:00:11.060) > SQL>showddl person3; > > CREATE TABLE TRAFODION.ODB_TEST.PERSON3 > ( > PID LARGEINT NO DEFAULT NOT NULL NOT DROPPABLE > SERIALIZED > , FNAME CHAR(20) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , LNAME CHAR(20) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , COUNTRY VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , CITY VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , BDATE DATE NO DEFAULT NOT NULL NOT DROPPABLE NOT > SERIALIZED > , SEX CHAR(1) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , EMAIL VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , SALARY NUMERIC(9, 2) NO DEFAULT NOT NULL NOT > DROPPABLE SERIALIZED > , EMPL VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , NOTES VARCHAR(80) CHARACTER SET ISO88591 COLLATE > DEFAULT DEFAULT NULL SERIALIZED > , LOADTS TIMESTAMP(0) DEFAULT NULL NOT SERIALIZED > , PRIMARY KEY (PID ASC) > ) > ; > --- SQL operation complete. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (TRAFODION-2034) Cannot output bad records in the loaded file even if specified file for storing bad records
[ https://issues.apache.org/jira/browse/TRAFODION-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TRAFODION-2034 started by Jian Jin. --- > Cannot output bad records in the loaded file even if specified file for > storing bad records > --- > > Key: TRAFODION-2034 > URL: https://issues.apache.org/jira/browse/TRAFODION-2034 > Project: Apache Trafodion > Issue Type: Bug > Components: db-utility-odb > Environment: centos6.7 HBase 1.0.0-cdh5.4.8 >Reporter: zhangliang >Assignee: Jian Jin >Priority: Minor > > This issue is from mantis 330. > Load file where is bad record in it, with option bad=output_data/bad_records > to hold the bad records, but the result is the file output_data/bad_records > generated but it is empty lines. > [root@suse-1 odb_test]# ./odb64luo -u trafodion -p traf123 -d traf -l > src=output_data/ext_person3.csv:pre=@scripts/ddl_person3.sql:tgt=trafodion.odb_test.person3:max=1000:rows=5000:parallel=5:loadcmd=UL:fs=\|:sq=\":bad=output_data/bad_records > odb [2016-04-25 19:17:05]: starting ODBC connection(s)... 0 1 2 3 4 5 > [0.0.0]Executing: 'drop table TRAFODION.odb_test.person3;' > [0.0.0]--- command executed in 7.279s (prep 0.001s, exec 7.278s, fetch > 0.000s/0.000s) > [0.0.1]Executing: 'CREATE TABLE TRAFODION.odb_test."PERSON3" ( > PID BIGINT SIGNED NOT NULL > ,FNAME CHAR(20) NOT NULL > ,LNAME CHAR(20) NOT NULL > ,COUNTRY VARCHAR(40) NOT NULL > ,CITY VARCHAR(40) NOT NULL > ,BDATE DATE NOT NULL > ,SEX CHAR(1) NOT NULL > ,EMAIL VARCHAR(40) NOT NULL > ,SALARY NUMERIC(9,2) NOT NULL > ,EMPL VARCHAR(40) NOT NULL > ,NOTES VARCHAR(80) > ,LOADTS TIMESTAMP(0) > ,PRIMARY KEY (PID) > );' > [0.0.1]--- command executed in 1.388s (prep 0.002s, exec 1.386s, fetch > 0.000s/0.000s) > Connected to Trafodion > [1] odb [Oloadbuff(9438)] - Error loading row 5 (State: 23000, Native 0) > [Trafodion ODBC Driver] GENERAL ERROR. Null Value in a non nullable column. > Row: 5 Column: 1 > [1] 999 records inserted [commit] > [0] odb version 1.1.0 Load statistics: > [0] Target table: TRAFODION.ODB_TEST.PERSON3 > [0] Source: output_data/ext_person3.csv > [0] Pre-loading time: 10.920 s (00:00:10.920) > [0] Loading time: 0.127 s(00:00:00.127) > [0] Total records read: 1,000 > [0] Total records inserted: 999 > [0] Total number of columns: 12 > [0] Total bytes read: 180,517 > [0] Average input row size: 180.5 B > [0] ODBC row size: 341 B (data) + 96 B (len ind) > [0] Rowset size: 1,000 > [0] Rowset buffer size: 426.76 KiB > [0] Load throughput (real data): 1,388.080 KiB/s > [0] Load throughput (ODBC): 2,619.487 KiB/s > [0] Reader Total/Wait Cycles: 1/0 > odb [2016-04-25 19:17:16]: exiting. Session Elapsed time 11.060 seconds > (00:00:11.060) > SQL>showddl person3; > > CREATE TABLE TRAFODION.ODB_TEST.PERSON3 > ( > PID LARGEINT NO DEFAULT NOT NULL NOT DROPPABLE > SERIALIZED > , FNAME CHAR(20) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , LNAME CHAR(20) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , COUNTRY VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , CITY VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , BDATE DATE NO DEFAULT NOT NULL NOT DROPPABLE NOT > SERIALIZED > , SEX CHAR(1) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , EMAIL VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , SALARY NUMERIC(9, 2) NO DEFAULT NOT NULL NOT > DROPPABLE SERIALIZED > , EMPL VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , NOTES VARCHAR(80) CHARACTER SET ISO88591 COLLATE > DEFAULT DEFAULT NULL SERIALIZED > , LOADTS TIMESTAMP(0) DEFAULT NULL NOT SERIALIZED > , PRIMARY KEY (PID ASC) > ) > ; > --- SQL operation complete. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-2034) Cannot output bad records in the loaded file even if specified file for storing bad records
[ https://issues.apache.org/jira/browse/TRAFODION-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Jin updated TRAFODION-2034: Assignee: zhangliang > Cannot output bad records in the loaded file even if specified file for > storing bad records > --- > > Key: TRAFODION-2034 > URL: https://issues.apache.org/jira/browse/TRAFODION-2034 > Project: Apache Trafodion > Issue Type: Bug > Components: db-utility-odb > Environment: centos6.7 HBase 1.0.0-cdh5.4.8 >Reporter: zhangliang >Assignee: zhangliang >Priority: Minor > > This issue is from mantis 330. > Load file where is bad record in it, with option bad=output_data/bad_records > to hold the bad records, but the result is the file output_data/bad_records > generated but it is empty lines. > [root@suse-1 odb_test]# ./odb64luo -u trafodion -p traf123 -d traf -l > src=output_data/ext_person3.csv:pre=@scripts/ddl_person3.sql:tgt=trafodion.odb_test.person3:max=1000:rows=5000:parallel=5:loadcmd=UL:fs=\|:sq=\":bad=output_data/bad_records > odb [2016-04-25 19:17:05]: starting ODBC connection(s)... 0 1 2 3 4 5 > [0.0.0]Executing: 'drop table TRAFODION.odb_test.person3;' > [0.0.0]--- command executed in 7.279s (prep 0.001s, exec 7.278s, fetch > 0.000s/0.000s) > [0.0.1]Executing: 'CREATE TABLE TRAFODION.odb_test."PERSON3" ( > PID BIGINT SIGNED NOT NULL > ,FNAME CHAR(20) NOT NULL > ,LNAME CHAR(20) NOT NULL > ,COUNTRY VARCHAR(40) NOT NULL > ,CITY VARCHAR(40) NOT NULL > ,BDATE DATE NOT NULL > ,SEX CHAR(1) NOT NULL > ,EMAIL VARCHAR(40) NOT NULL > ,SALARY NUMERIC(9,2) NOT NULL > ,EMPL VARCHAR(40) NOT NULL > ,NOTES VARCHAR(80) > ,LOADTS TIMESTAMP(0) > ,PRIMARY KEY (PID) > );' > [0.0.1]--- command executed in 1.388s (prep 0.002s, exec 1.386s, fetch > 0.000s/0.000s) > Connected to Trafodion > [1] odb [Oloadbuff(9438)] - Error loading row 5 (State: 23000, Native 0) > [Trafodion ODBC Driver] GENERAL ERROR. Null Value in a non nullable column. > Row: 5 Column: 1 > [1] 999 records inserted [commit] > [0] odb version 1.1.0 Load statistics: > [0] Target table: TRAFODION.ODB_TEST.PERSON3 > [0] Source: output_data/ext_person3.csv > [0] Pre-loading time: 10.920 s (00:00:10.920) > [0] Loading time: 0.127 s(00:00:00.127) > [0] Total records read: 1,000 > [0] Total records inserted: 999 > [0] Total number of columns: 12 > [0] Total bytes read: 180,517 > [0] Average input row size: 180.5 B > [0] ODBC row size: 341 B (data) + 96 B (len ind) > [0] Rowset size: 1,000 > [0] Rowset buffer size: 426.76 KiB > [0] Load throughput (real data): 1,388.080 KiB/s > [0] Load throughput (ODBC): 2,619.487 KiB/s > [0] Reader Total/Wait Cycles: 1/0 > odb [2016-04-25 19:17:16]: exiting. Session Elapsed time 11.060 seconds > (00:00:11.060) > SQL>showddl person3; > > CREATE TABLE TRAFODION.ODB_TEST.PERSON3 > ( > PID LARGEINT NO DEFAULT NOT NULL NOT DROPPABLE > SERIALIZED > , FNAME CHAR(20) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , LNAME CHAR(20) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , COUNTRY VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , CITY VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , BDATE DATE NO DEFAULT NOT NULL NOT DROPPABLE NOT > SERIALIZED > , SEX CHAR(1) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , EMAIL VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , SALARY NUMERIC(9, 2) NO DEFAULT NOT NULL NOT > DROPPABLE SERIALIZED > , EMPL VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , NOTES VARCHAR(80) CHARACTER SET ISO88591 COLLATE > DEFAULT DEFAULT NULL SERIALIZED > , LOADTS TIMESTAMP(0) DEFAULT NULL NOT SERIALIZED > , PRIMARY KEY (PID ASC) > ) > ; > --- SQL operation complete. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
