[jira] [Commented] (TRAFODION-343) LP Bug: 1325716 - TIME(1) default CURRENT_TIME reported wrong values
[ https://issues.apache.org/jira/browse/TRAFODION-343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14951922#comment-14951922 ] liu ming commented on TRAFODION-343: Verified at latest code build, the problem is already fixed. Can be closed. > LP Bug: 1325716 - TIME(1) default CURRENT_TIME reported wrong values > > > Key: TRAFODION-343 > URL: https://issues.apache.org/jira/browse/TRAFODION-343 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmp >Reporter: Apache Trafodion >Assignee: liu ming >Priority: Critical > Fix For: 1.0 (pre-incubation) > > > expected default CURRENT_TIME ti01 returned -34.07.06. > not as expected 19:55:28.6 > SQL>create table a05table ( > int1 integer no default not null > , vch2 CHAR VARYING(8) > , dt00 DATE > , tidf TIME default CURRENT_TIME > , tsdf TIMESTAMP(0) default CURRENT_TIMESTAMP > , dt01 DATE default CURRENT_DATE > , ti01 TIME(1) default CURRENT_TIME > , ts01 TIMESTAMP(1) default CURRENT_TIMESTAMP > , primary key (int1 asc) not droppable > ) > store by primary key > ; > --- SQL operation complete. > SQL>insert into a05table (int1, vch2) values (1,'row_1'); > --- 1 row(s) inserted. > SQL>insert into a05table (int1, tsdf ) values > (13, timestamp '1997-01-31:13:13:13'); > --- 1 row(s) inserted. > SQL>select * from a05table order by int1; > INT1VCH2 DT00 TIDF TSDFDT01 TI01 > TS01 > --- -- --- -- > -- - > 1 row_1NULL 19:55:27 2014-06-02 19:55:27 2014-06-02 > -34:07:06. 2014-06-02 19:55:27.6 > 13 NULL NULL 19:55:28 1997-01-31 13:13:13 2014-06-02 > 19:55:28.6 2014-06-02 19:55:28.6 > --- 2 row(s) selected. > SQL>select int1, dt01, ti01, ts01 from a05table order by int1; > INT1DT01 TI01 TS01 > --- -- -- - > 1 2014-06-02 -34:07:06. 2014-06-02 19:55:27.6 > 13 2014-06-02 19:55:28.6 2014-06-02 19:55:28.6 > --- 2 row(s) selected. > SQL>insert into a05table (int1, dt01) values > (20, date '2005-03-31'); > --- 1 row(s) inserted. > SQL>select int1, dt01, ti01, ts01 from a05table order by int1; > INT1DT01 TI01 TS01 > --- -- -- - > 1 2014-06-02 -34:07:06. 2014-06-02 19:55:27.6 > 13 2014-06-02 19:55:28.6 2014-06-02 19:55:28.6 > 20 2005-03-31 -34:07:06. 2014-06-02 19:55:28.6 > --- 3 row(s) selected. > SQL>exit; > -- test script > log a09log clear; > drop schema debug_addt105 cascade; > create schema debug_addt105; > set schema debug_addt105; > drop table a05table; > -- table start with 2 fixed length field > create table a05table ( > int1 integer no default not null > , vch2 CHAR VARYING(8) > , dt00 DATE > , tidf TIME default CURRENT_TIME > , tsdf TIMESTAMP(0) default CURRENT_TIMESTAMP > , dt01 DATE default CURRENT_DATE > , ti01 TIME(1) default CURRENT_TIME > , ts01 TIMESTAMP(1) default CURRENT_TIMESTAMP > , primary key (int1 asc) not droppable > ) > store by primary key > ; > insert into a05table (int1, vch2) values (1,'row_1'); > insert into a05table (int1, tsdf ) values > (13, timestamp '1997-01-31:13:13:13'); > select * from a05table order by int1; > select int1, dt01, ti01, ts01 from a05table order by int1; > insert into a05table (int1, dt01) values > (20, date '2005-03-31'); > select int1, dt01, ti01, ts01 from a05table order by int1; > exit; > > GIT 0530 build. Also failed on 0601 build. > Assigned to LaunchPad User James Capps -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1523) Trafodion expand new node fail due to upgrade installation fail
liu ming created TRAFODION-1523: --- Summary: Trafodion expand new node fail due to upgrade installation fail Key: TRAFODION-1523 URL: https://issues.apache.org/jira/browse/TRAFODION-1523 Project: Apache Trafodion Issue Type: Bug Components: installer Reporter: liu ming When we have a 4 nodes cluster up and running for a while, our test engineers use 'trafodion' user's VNC to do some testing. So the HOME directory of user ‘trafodion’ have many new things created. Like two directories: .cache , .mozilla . When do expand, the installer will work in UPGRADE mode. The installer will regenerate the ssh key and copy the whole HOME directory to workdir and then use PDCP to copy the workdir/HOME to all the nodes. Since there are direcoties like ‘.cache’, ‘.mozilla’, it failed. We noticed the script traf_add_user will remove the .pulse before do PDCP. We think it also needs to remove ‘.cache’ and ‘.mozilla’. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1578) Proposal for SPJ management
[ https://issues.apache.org/jira/browse/TRAFODION-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14987369#comment-14987369 ] liu ming commented on TRAFODION-1578: - I like this idea very much. It allows user to deploy SPJ via a simple trafci command, since itself is a SPJ, so it is also very easy to integrated into a GUI manager. Need to discuss a little more: 1. Put DLL/jar as LOB or as HDFS file? HDFS file seems very good to me. 2. Location of DLL/jar at server side. $MY_SQROOT/export/lib/udr/cache is a good place, and as Kevin proposed, there are benefit to put DLL/jar files in directories per user. So we can have finer control, user A cannot call user B’s SPJ for example. So maybe we can upload the jar to $MY_SQROOT/export/lib/udr/cache/$USER If the SPJ/UDR need extra DLL/jar files, we can simply put into $MY_SQROOT/export/lib along with the SPJ jar, $MY_SQROOT/export/lib is already in CLASSPATH and LD_LIBRARY_PATH. Or we can ask the developer of UDR/SPJ to pack all required DLL/jar into a single file. > Proposal for SPJ management > --- > > Key: TRAFODION-1578 > URL: https://issues.apache.org/jira/browse/TRAFODION-1578 > Project: Apache Trafodion > Issue Type: Improvement > Components: connectivity-dcs >Reporter: Kevin Xu > > JAR upload process: > 1. Initialize JAR upload procedure by default > 2. JAR upload by Trafci(add library LIB_NAME JAR_LOCAL_PATH). Upload and > create library will be done here. And also, you can only upload the JARs by > UPLOAD command on Trafci that it will not create a lib. >Tips: Before put the JAR into HDFS check MD5 first, if the file exists, > only add a record in metadata table in case users upload the same JAR many > times on platform. > 3. On server-side, the JAR will store in HDFS. At the same time JAR > metadata(path in HDFS, MD5 of the file, and others) stores in store procedure > metadata table. > 4. create procedure is the same as now. > JAR perform process: > 1. Send a CALL by Trafci/JDBC/ODBC/ADO.NET. > 2. DCSMaster assign a DCSServer for the CALL. > 3. DCSServer start a JVM for the user. User can modify JVM options, program > properties and JAVA classpath. At the same time, a monitor class will be > starting in the JVM witch will register a node on Zookeeper for this JVM as > well as metadata info( process id, server info and so on) and the node will > be removed while JVM exiting. It allows customer to specify JVM idle time in > case of some realtime senarior like Kafka consumer. > 4. Useful commands on Trafci: list all JVMs in user; kill one of them that no > long in use; Restart JVMs with latest JARs and so on. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1643) instantiation of ScanInfo changed after hbase-0.98.8
[ https://issues.apache.org/jira/browse/TRAFODION-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1643: --- Assignee: liu ming > instantiation of ScanInfo changed after hbase-0.98.8 > > > Key: TRAFODION-1643 > URL: https://issues.apache.org/jira/browse/TRAFODION-1643 > Project: Apache Trafodion > Issue Type: Bug >Reporter: mashengchen >Assignee: liu ming > > after hbase-0.98.8(include 0.98.8), the instantiation process of > ScanInfo(class) is different from before, which need KeepDeletedCells(enum) > as parameter instead of boolean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1643) instantiation of ScanInfo changed after hbase-0.98.8
[ https://issues.apache.org/jira/browse/TRAFODION-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1643: Assignee: Eason Zhang (was: liu ming) > instantiation of ScanInfo changed after hbase-0.98.8 > > > Key: TRAFODION-1643 > URL: https://issues.apache.org/jira/browse/TRAFODION-1643 > Project: Apache Trafodion > Issue Type: Bug >Reporter: mashengchen >Assignee: Eason Zhang > > after hbase-0.98.8(include 0.98.8), the instantiation process of > ScanInfo(class) is different from before, which need KeepDeletedCells(enum) > as parameter instead of boolean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1643) instantiation of ScanInfo changed after hbase-0.98.8
[ https://issues.apache.org/jira/browse/TRAFODION-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1643: --- Assignee: liu ming (was: Eason Zhang) > instantiation of ScanInfo changed after hbase-0.98.8 > > > Key: TRAFODION-1643 > URL: https://issues.apache.org/jira/browse/TRAFODION-1643 > Project: Apache Trafodion > Issue Type: Bug >Reporter: mashengchen >Assignee: liu ming > > after hbase-0.98.8(include 0.98.8), the instantiation process of > ScanInfo(class) is different from before, which need KeepDeletedCells(enum) > as parameter instead of boolean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1643) instantiation of ScanInfo changed after hbase-0.98.8
[ https://issues.apache.org/jira/browse/TRAFODION-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1643: Assignee: Eason Zhang (was: liu ming) > instantiation of ScanInfo changed after hbase-0.98.8 > > > Key: TRAFODION-1643 > URL: https://issues.apache.org/jira/browse/TRAFODION-1643 > Project: Apache Trafodion > Issue Type: Bug >Reporter: mashengchen >Assignee: Eason Zhang > > after hbase-0.98.8(include 0.98.8), the instantiation process of > ScanInfo(class) is different from before, which need KeepDeletedCells(enum) > as parameter instead of boolean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1643) instantiation of ScanInfo changed after hbase-0.98.8
[ https://issues.apache.org/jira/browse/TRAFODION-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1643: Assignee: (was: Eason Zhang) > instantiation of ScanInfo changed after hbase-0.98.8 > > > Key: TRAFODION-1643 > URL: https://issues.apache.org/jira/browse/TRAFODION-1643 > Project: Apache Trafodion > Issue Type: Bug >Reporter: mashengchen > > after hbase-0.98.8(include 0.98.8), the instantiation process of > ScanInfo(class) is different from before, which need KeepDeletedCells(enum) > as parameter instead of boolean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1643) instantiation of ScanInfo changed after hbase-0.98.8
[ https://issues.apache.org/jira/browse/TRAFODION-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1643: Reporter: Michel Alessandrini (was: mashengchen) > instantiation of ScanInfo changed after hbase-0.98.8 > > > Key: TRAFODION-1643 > URL: https://issues.apache.org/jira/browse/TRAFODION-1643 > Project: Apache Trafodion > Issue Type: Bug >Reporter: Michel Alessandrini > > after hbase-0.98.8(include 0.98.8), the instantiation process of > ScanInfo(class) is different from before, which need KeepDeletedCells(enum) > as parameter instead of boolean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1643) instantiation of ScanInfo changed after hbase-0.98.8
[ https://issues.apache.org/jira/browse/TRAFODION-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1643: Reporter: liu ming (was: Michel Alessandrini) > instantiation of ScanInfo changed after hbase-0.98.8 > > > Key: TRAFODION-1643 > URL: https://issues.apache.org/jira/browse/TRAFODION-1643 > Project: Apache Trafodion > Issue Type: Bug >Reporter: liu ming > > after hbase-0.98.8(include 0.98.8), the instantiation process of > ScanInfo(class) is different from before, which need KeepDeletedCells(enum) > as parameter instead of boolean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1643) instantiation of ScanInfo changed after hbase-0.98.8
[ https://issues.apache.org/jira/browse/TRAFODION-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025906#comment-15025906 ] liu ming commented on TRAFODION-1643: - Reporter is Ma,ShengChen. Liu Ming make some mistake to do assignment. Need to correct the owner asap > instantiation of ScanInfo changed after hbase-0.98.8 > > > Key: TRAFODION-1643 > URL: https://issues.apache.org/jira/browse/TRAFODION-1643 > Project: Apache Trafodion > Issue Type: Bug >Reporter: liu ming > > after hbase-0.98.8(include 0.98.8), the instantiation process of > ScanInfo(class) is different from before, which need KeepDeletedCells(enum) > as parameter instead of boolean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1558) Tableau returns to tables or data
[ https://issues.apache.org/jira/browse/TRAFODION-1558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1558: Assignee: Weiqing Xu > Tableau returns to tables or data > - > > Key: TRAFODION-1558 > URL: https://issues.apache.org/jira/browse/TRAFODION-1558 > Project: Apache Trafodion > Issue Type: Bug > Components: client-odbc-windows >Affects Versions: 1.2-incubating > Environment: Windows 10 with latest Tableau version. >Reporter: Gunnar Tapper >Assignee: Weiqing Xu >Priority: Critical > Fix For: 1.2-incubating > > > Used driver connection with TCP:/. Connection succeeded but no > schemas or tables are listed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1557) Datasource cannot connect using Tableau
[ https://issues.apache.org/jira/browse/TRAFODION-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1557: Assignee: Weiqing Xu > Datasource cannot connect using Tableau > --- > > Key: TRAFODION-1557 > URL: https://issues.apache.org/jira/browse/TRAFODION-1557 > Project: Apache Trafodion > Issue Type: Bug > Components: client-odbc-windows >Affects Versions: 1.2-incubating > Environment: Windows 10 with latest Tableau version. >Reporter: Gunnar Tapper >Assignee: Weiqing Xu >Priority: Critical > Fix For: 1.2-incubating > > > Configured a data source in ODBC manager. Verified via Test Connection. Tried > to use data source in Tableau. Does not work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1657) support 'create view' invovling native hive table
liu ming created TRAFODION-1657: --- Summary: support 'create view' invovling native hive table Key: TRAFODION-1657 URL: https://issues.apache.org/jira/browse/TRAFODION-1657 Project: Apache Trafodion Issue Type: New Feature Components: foundation Reporter: liu ming it will be helpful to support the creation of view which select data from native hive table. The difficulty is native hive/hbase table do not have metadata in Trafodion, like object ID. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1676) support better runtime error message when a SQL function meet fital error
liu ming created TRAFODION-1676: --- Summary: support better runtime error message when a SQL function meet fital error Key: TRAFODION-1676 URL: https://issues.apache.org/jira/browse/TRAFODION-1676 Project: Apache Trafodion Issue Type: Improvement Components: sql-exe Reporter: liu ming Priority: Minor A sql contains some SQL function, for example UPPER(), when the input data of UPPER() contains invalid data, UPPER() will fail. In this case, the whole query abort, and the error message cannot tell which row has invalid data, so it is very hard to fix the problem. It will be good to have row id or the real value of the error row in the error message to help trouble shooting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1643) instantiation of ScanInfo changed after hbase-0.98.8
[ https://issues.apache.org/jira/browse/TRAFODION-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1643: Assignee: mashengchen > instantiation of ScanInfo changed after hbase-0.98.8 > > > Key: TRAFODION-1643 > URL: https://issues.apache.org/jira/browse/TRAFODION-1643 > Project: Apache Trafodion > Issue Type: Bug >Reporter: liu ming >Assignee: mashengchen > > after hbase-0.98.8(include 0.98.8), the instantiation process of > ScanInfo(class) is different from before, which need KeepDeletedCells(enum) > as parameter instead of boolean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1643) instantiation of ScanInfo changed after hbase-0.98.8
[ https://issues.apache.org/jira/browse/TRAFODION-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1643: Reporter: mashengchen (was: liu ming) > instantiation of ScanInfo changed after hbase-0.98.8 > > > Key: TRAFODION-1643 > URL: https://issues.apache.org/jira/browse/TRAFODION-1643 > Project: Apache Trafodion > Issue Type: Bug >Reporter: mashengchen >Assignee: mashengchen > > after hbase-0.98.8(include 0.98.8), the instantiation process of > ScanInfo(class) is different from before, which need KeepDeletedCells(enum) > as parameter instead of boolean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1720) add GBK support in SQL translate function
liu ming created TRAFODION-1720: --- Summary: add GBK support in SQL translate function Key: TRAFODION-1720 URL: https://issues.apache.org/jira/browse/TRAFODION-1720 Project: Apache Trafodion Issue Type: Improvement Components: sql-exe Reporter: liu ming Trafodion support TRANSLATE function now and support character set conversion among ISO88591, SJIS, UCS2 and UTF8. It will be useful to support GBK and later BIG5. GBK is widely used in China. Without support, all source data need to be converted before data loading. With built-in support, it will decrease the overall data loading time. External interface Use the current SQL Function TRANSLATE, introduce new translation, keep the original syntax. TRANSLATE(character-value-expression USING translation-name) Translation-nameSource charset Target charset Comments == == = GBKTOUTF8 GBK2312 UTF8 When error occur during translation, SQL Error UTF8TOGBK UTF8GB2312 Internal Parser change Let class Translate support new translation-name. CodeGen change Translate::codeGen Executor change convDoIt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1720) add GBK support in SQL translate function
[ https://issues.apache.org/jira/browse/TRAFODION-1720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1720: --- Assignee: liu ming > add GBK support in SQL translate function > - > > Key: TRAFODION-1720 > URL: https://issues.apache.org/jira/browse/TRAFODION-1720 > Project: Apache Trafodion > Issue Type: Improvement > Components: sql-exe >Reporter: liu ming >Assignee: liu ming > > Trafodion support TRANSLATE function now and support character set conversion > among ISO88591, SJIS, UCS2 and UTF8. It will be useful to support GBK and > later BIG5. > GBK is widely used in China. Without support, all source data need to be > converted before data loading. With built-in support, it will decrease the > overall data loading time. > External interface > Use the current SQL Function TRANSLATE, introduce new translation, keep the > original syntax. > > TRANSLATE(character-value-expression USING translation-name) > Translation-name Source charset Target charset Comments > == == = > GBKTOUTF8 GBK2312 UTF8When error occur during > translation, SQL Error > UTF8TOGBK UTF8GB2312 > Internal > Parser change > Let class Translate support new translation-name. > CodeGen change > Translate::codeGen > Executor change > convDoIt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1729) change the coprocessor deployment method
liu ming created TRAFODION-1729: --- Summary: change the coprocessor deployment method Key: TRAFODION-1729 URL: https://issues.apache.org/jira/browse/TRAFODION-1729 Project: Apache Trafodion Issue Type: Improvement Components: dtm Reporter: liu ming I have a proposal to change our current HBase coprocessor configuration method. There are three ways to add a coprocessor to a HBase table: 1. Via editing hbase-site.xml, which will load coprocessor for ALL tables (Trafodion is using this method now) 2. Via HBase shell command 3. Via HTableDescriptor.addCoprocessor() java API Trafodion now is using the first method. I proposed to use method 3, I finished a prototype and test seems works well. Here are the reasons I propose for this change: At present, the Trafodion installer needs to modify the hbase-site.xml and then restart HBase instance for the configuration to take effect. This step not only complicate the installer but also let user think Trafodion is intrusive into underlying HBase system. It will be ideal if we can avoid this step. Another problem: in CDH, there is a concept called ‘region server group’ or something, so the settings will have to carefully handled by installer to apply to all groups. As we saw recently in WebRoot deployment, Trafodion failed due to this reason. All these are very error prone and complicate the Trafodion installer. Once CDH or HDP changed something, Trafodion may fail again. So I spent time to investigate why we need to restart HBase in order to install Trafodion. As I understand, there are 3 major reasons 1. To add hbase-trx coprocessors 2. To overload HRegion with TransactionalRegion 3. Various configuration settings, need to check one by one. The first configuration can be avoided by applying my proposed change. The second one, I look through the TransactionalRegion.java, and find out the only reason (now) is to overload the getScanner() method to be public so can be invoked by the coprocessor. And there are only 1 or 2 places that API is invoked in Trafodion code. I checked with Kevin and he proposed by using ‘java reflection’ we can also avoid this. All other configuration items to some extent look like ‘best to have’, but not ‘must to have’. And I also find two config items seems never been used: hbase.bulkload.staging.dir /hbase-staging (Suresh can confirm, but I search in all code, seems this is never used) hbase.regionserver.region.transactional.tlog true (Narendra can confirm, this is NEVER used, maybe a legacy config item?) Yes, by now, there are still some other config items seems cannot be avoided, but I hope we can find some way to remove them in the future. I am not trying to solve all issues right now, just want to start the effort to remove unnecessary hbase reconfiguration. For this example, Coprocessors can be added to a table at run time, no need to edit the hbase-site.xml and restart hbase. This is only the first step to try to remove the deep impact to the current HBase config and restart HBase. So I asked for your opinions about this change. If you think this is necessary, I will continue to file a JIRA and fix it. I strongly recommend to get rid of the step of ‘modify hbase-site.xml and restart your hbase’ for Trafodion installation, it should be an option , to tune the system to best suit Trafodion, but should not be a forced step. To be note: Apache Phoenix is also a SQL on HBase, its installation will change nothing of underlying HBase, very lightweight, no ‘intrude into’ the existing HBase system. Trafodion is considered to be heavy and intrusive in this manner, and I feel maybe we can change this. Should I start this discussion in the dev mail list? P.S. a list of changed config items. My proposal will remove the last one, hope we can get rid of all of them: hbase.master.distributed.log.splitting false hbase.snapshot.master.timeoutMillis 60 hbase_regionserver_lease_period 60 hbase.hregion.impl org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion hbase.regionserver.region.split.policy org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy hbase.snapshot.enabled true hbase.bulkload.staging.dir/hbase-staging hbase.regionserver.region.transactional.tlog true hbase.snapshot.region.timeout60 hbase_coprocessor_region_classes org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver,org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint,org.apache.hadoop.hbase.coprocessor.AggregateImplementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1729) change the coprocessor deployment method
[ https://issues.apache.org/jira/browse/TRAFODION-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1729: Assignee: mashengchen > change the coprocessor deployment method > > > Key: TRAFODION-1729 > URL: https://issues.apache.org/jira/browse/TRAFODION-1729 > Project: Apache Trafodion > Issue Type: Improvement > Components: dtm >Reporter: liu ming >Assignee: mashengchen > > I have a proposal to change our current HBase coprocessor configuration > method. > There are three ways to add a coprocessor to a HBase table: > 1. Via editing hbase-site.xml, which will load coprocessor for ALL > tables (Trafodion is using this method now) > 2. Via HBase shell command > 3. Via HTableDescriptor.addCoprocessor() java API > Trafodion now is using the first method. I proposed to use method 3, I > finished a prototype and test seems works well. > > Here are the reasons I propose for this change: > At present, the Trafodion installer needs to modify the hbase-site.xml and > then restart HBase instance for the configuration to take effect. This step > not only complicate the installer but also let user think Trafodion is > intrusive into underlying HBase system. It will be ideal if we can avoid this > step. Another problem: in CDH, there is a concept called ‘region server > group’ or something, so the settings will have to carefully handled by > installer to apply to all groups. As we saw recently in WebRoot deployment, > Trafodion failed due to this reason. All these are very error prone and > complicate the Trafodion installer. Once CDH or HDP changed something, > Trafodion may fail again. > > So I spent time to investigate why we need to restart HBase in order to > install Trafodion. > As I understand, there are 3 major reasons > 1. To add hbase-trx coprocessors > 2. To overload HRegion with TransactionalRegion > 3. Various configuration settings, need to check one by one. > The first configuration can be avoided by applying my proposed change. > The second one, I look through the TransactionalRegion.java, and find out the > only reason (now) is to overload the getScanner() method to be public so can > be invoked by the coprocessor. And there are only 1 or 2 places that API is > invoked in Trafodion code. I checked with Kevin and he proposed by using > ‘java reflection’ we can also avoid this. > All other configuration items to some extent look like ‘best to have’, but > not ‘must to have’. And I also find two config items seems never been used: > hbase.bulkload.staging.dir /hbase-staging (Suresh can confirm, > but I search in all code, seems this is never used) > hbase.regionserver.region.transactional.tlog true (Narendra can > confirm, this is NEVER used, maybe a legacy config item?) > Yes, by now, there are still some other config items seems cannot be avoided, > but I hope we can find some way to remove them in the future. I am not trying > to solve all issues right now, just want to start the effort to remove > unnecessary hbase reconfiguration. > For this example, Coprocessors can be added to a table at run time, no need > to edit the hbase-site.xml and restart hbase. This is only the first step to > try to remove the deep impact to the current HBase config and restart HBase. > > So I asked for your opinions about this change. If you think this is > necessary, I will continue to file a JIRA and fix it. > > I strongly recommend to get rid of the step of ‘modify hbase-site.xml and > restart your hbase’ for Trafodion installation, it should be an option , to > tune the system to best suit Trafodion, but should not be a forced step. To > be note: Apache Phoenix is also a SQL on HBase, its installation will change > nothing of underlying HBase, very lightweight, no ‘intrude into’ the existing > HBase system. Trafodion is considered to be heavy and intrusive in this > manner, and I feel maybe we can change this. > > Should I start this discussion in the dev mail list? > > P.S. a list of changed config items. My proposal will remove the last one, > hope we can get rid of all of them: > hbase.master.distributed.log.splitting false > hbase.snapshot.master.timeoutMillis 60 > hbase_regionserver_lease_period 60 > hbase.hregion.impl > org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion > hbase.regionserver.region.split.policy > org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy > hbase.snapshot.enabled true > hbase.bulkload.staging.dir/hbase-staging > hbase.regionserver.region.transactional.tlog true > hbase.snapshot.region.timeout60 > hbase_coprocessor_region_classes >
[jira] [Created] (TRAFODION-1730) try to remove the requirement of using hbase.hregion.impl in hbase-site.xml
liu ming created TRAFODION-1730: --- Summary: try to remove the requirement of using hbase.hregion.impl in hbase-site.xml Key: TRAFODION-1730 URL: https://issues.apache.org/jira/browse/TRAFODION-1730 Project: Apache Trafodion Issue Type: Improvement Reporter: liu ming Assignee: mashengchen look through the TransactionalRegion.java, and find out the only reason (now) is to overload the getScanner() method to be public so can be invoked by the coprocessor. And there are only 1 or 2 places that API is invoked in Trafodion code. Should be able to user 'java reflection' to overcome this requirement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1729) change the coprocessor deployment method
[ https://issues.apache.org/jira/browse/TRAFODION-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15075963#comment-15075963 ] liu ming commented on TRAFODION-1729: - thanks Stack very much for your comments. Let me try to clarify items one by one. 1. Overloading To implement transaction semantics, Trafodion save the updated rows in RS heap, call it writeOrdering list, all Put and Delete are saved in this list and not go into HBase Region until commit, so other transaction cannot see the update. So, before commit, when it does a scan, Trafodion needs to filter out those 'deleted' rows for current transaction, and plus those new put rows which still in memory not in the Region yet. To archieve this, in the coprocessor, Trafodion needs to invoke a protected HRegion method: protected RegionScanner getScanner(Scan scan, List additionalScanners, boolean copyCellsFromSharedMem) throws IOException To combine two scanners' result as final scan result. The first scanner will equiped with a filter to filter out 'deleted' rows which matching the delete objects in writeOrdering list. But new inserted rows or updated rows are Put objects in writeOrdering as well, so Trafodion need another additonalScanners to get those rows in the writeOrdering list and combine the result. But that getScanner() method is protected, so Trafodion overload HRegion and make it public, in order to invoke it. Currently, Trafodion overload the HRegion just to make that method as public, so can be invoked within the coprocessor. Our proposal is to use Java's reflection technique to invoke this 'proctected' method without overload. It is used only once during a scanner construction, so we feel it will not impact the performance as well. This is the idea of this proposal. 2. Phoenix I am wrong with Phoenix, I was playing with it long ago and as I remember, I just download a tar file and untar it and no change to my HBase system and get Phoenix working, so I said that, but probably my memeory is wrong, but the point is , we wish to avoid too much changes into HBase and restart it in order to use Trafodion. And I don't know if it is possible to make that getScanner() method to public in future HBase release? If that can be made, it will help Trafodion development a lot. thanks. > change the coprocessor deployment method > > > Key: TRAFODION-1729 > URL: https://issues.apache.org/jira/browse/TRAFODION-1729 > Project: Apache Trafodion > Issue Type: Improvement > Components: dtm >Reporter: liu ming >Assignee: mashengchen > > I have a proposal to change our current HBase coprocessor configuration > method. > There are three ways to add a coprocessor to a HBase table: > 1. Via editing hbase-site.xml, which will load coprocessor for ALL > tables (Trafodion is using this method now) > 2. Via HBase shell command > 3. Via HTableDescriptor.addCoprocessor() java API > Trafodion now is using the first method. I proposed to use method 3, I > finished a prototype and test seems works well. > > Here are the reasons I propose for this change: > At present, the Trafodion installer needs to modify the hbase-site.xml and > then restart HBase instance for the configuration to take effect. This step > not only complicate the installer but also let user think Trafodion is > intrusive into underlying HBase system. It will be ideal if we can avoid this > step. Another problem: in CDH, there is a concept called ‘region server > group’ or something, so the settings will have to carefully handled by > installer to apply to all groups. As we saw recently in WebRoot deployment, > Trafodion failed due to this reason. All these are very error prone and > complicate the Trafodion installer. Once CDH or HDP changed something, > Trafodion may fail again. > > So I spent time to investigate why we need to restart HBase in order to > install Trafodion. > As I understand, there are 3 major reasons > 1. To add hbase-trx coprocessors > 2. To overload HRegion with TransactionalRegion > 3. Various configuration settings, need to check one by one. > The first configuration can be avoided by applying my proposed change. > The second one, I look through the TransactionalRegion.java, and find out the > only reason (now) is to overload the getScanner() method to be public so can > be invoked by the coprocessor. And there are only 1 or 2 places that API is > invoked in Trafodion code. I checked with Kevin and he proposed by using > ‘java reflection’ we can also avoid this. > All other configuration items to some extent look like ‘best to have’, but > not ‘must to have’. And I also find two config items seems never been use
[jira] [Commented] (TRAFODION-1730) try to remove the requirement of using hbase.hregion.impl in hbase-site.xml
[ https://issues.apache.org/jira/browse/TRAFODION-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15075997#comment-15075997 ] liu ming commented on TRAFODION-1730: - We are working on the HBase 1.0 change for Apache Trafodion now. Sorry that I don't quite understand what means T8 codebase?.My colleague is now working on the HBase 1.0 changes for Trafodion, and he already pass initial regression tests, I did not notice there is change for protobuf. Protobuf in Trafodion is only used as coprocessor required, as per my understanding HBase 1.0 is backward compatible with HBase 0.98 in this area. There are also effort to clean up KeyValue and other changes going on. I am not aware of what means 'T8 folks' here again. > try to remove the requirement of using hbase.hregion.impl in hbase-site.xml > --- > > Key: TRAFODION-1730 > URL: https://issues.apache.org/jira/browse/TRAFODION-1730 > Project: Apache Trafodion > Issue Type: Improvement >Reporter: liu ming >Assignee: mashengchen > > look through the TransactionalRegion.java, and find out the only reason (now) > is to overload the getScanner() method to be public so can be invoked by the > coprocessor. And there are only 1 or 2 places that API is invoked in > Trafodion code. > Should be able to user 'java reflection' to overcome this requirement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1729) change the coprocessor deployment method
[ https://issues.apache.org/jira/browse/TRAFODION-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15080609#comment-15080609 ] liu ming commented on TRAFODION-1729: - Hi, Stack, Yes, the patch is exactly what Trafodion need, if that can be merged into HBase upstream, it will help T8 folks a lot! Thank you for considering this. So you will do that change in HBase? thanks, Ming > change the coprocessor deployment method > > > Key: TRAFODION-1729 > URL: https://issues.apache.org/jira/browse/TRAFODION-1729 > Project: Apache Trafodion > Issue Type: Improvement > Components: dtm >Reporter: liu ming >Assignee: mashengchen > > I have a proposal to change our current HBase coprocessor configuration > method. > There are three ways to add a coprocessor to a HBase table: > 1. Via editing hbase-site.xml, which will load coprocessor for ALL > tables (Trafodion is using this method now) > 2. Via HBase shell command > 3. Via HTableDescriptor.addCoprocessor() java API > Trafodion now is using the first method. I proposed to use method 3, I > finished a prototype and test seems works well. > > Here are the reasons I propose for this change: > At present, the Trafodion installer needs to modify the hbase-site.xml and > then restart HBase instance for the configuration to take effect. This step > not only complicate the installer but also let user think Trafodion is > intrusive into underlying HBase system. It will be ideal if we can avoid this > step. Another problem: in CDH, there is a concept called ‘region server > group’ or something, so the settings will have to carefully handled by > installer to apply to all groups. As we saw recently in WebRoot deployment, > Trafodion failed due to this reason. All these are very error prone and > complicate the Trafodion installer. Once CDH or HDP changed something, > Trafodion may fail again. > > So I spent time to investigate why we need to restart HBase in order to > install Trafodion. > As I understand, there are 3 major reasons > 1. To add hbase-trx coprocessors > 2. To overload HRegion with TransactionalRegion > 3. Various configuration settings, need to check one by one. > The first configuration can be avoided by applying my proposed change. > The second one, I look through the TransactionalRegion.java, and find out the > only reason (now) is to overload the getScanner() method to be public so can > be invoked by the coprocessor. And there are only 1 or 2 places that API is > invoked in Trafodion code. I checked with Kevin and he proposed by using > ‘java reflection’ we can also avoid this. > All other configuration items to some extent look like ‘best to have’, but > not ‘must to have’. And I also find two config items seems never been used: > hbase.bulkload.staging.dir /hbase-staging (Suresh can confirm, > but I search in all code, seems this is never used) > hbase.regionserver.region.transactional.tlog true (Narendra can > confirm, this is NEVER used, maybe a legacy config item?) > Yes, by now, there are still some other config items seems cannot be avoided, > but I hope we can find some way to remove them in the future. I am not trying > to solve all issues right now, just want to start the effort to remove > unnecessary hbase reconfiguration. > For this example, Coprocessors can be added to a table at run time, no need > to edit the hbase-site.xml and restart hbase. This is only the first step to > try to remove the deep impact to the current HBase config and restart HBase. > > So I asked for your opinions about this change. If you think this is > necessary, I will continue to file a JIRA and fix it. > > I strongly recommend to get rid of the step of ‘modify hbase-site.xml and > restart your hbase’ for Trafodion installation, it should be an option , to > tune the system to best suit Trafodion, but should not be a forced step. To > be note: Apache Phoenix is also a SQL on HBase, its installation will change > nothing of underlying HBase, very lightweight, no ‘intrude into’ the existing > HBase system. Trafodion is considered to be heavy and intrusive in this > manner, and I feel maybe we can change this. > > Should I start this discussion in the dev mail list? > > P.S. a list of changed config items. My proposal will remove the last one, > hope we can get rid of all of them: > hbase.master.distributed.log.splitting false > hbase.snapshot.master.timeoutMillis 60 > hbase_regionserver_lease_period 60 > hbase.hregion.impl > org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion > hbase.regionserver.region.split.policy > org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy
[jira] [Commented] (TRAFODION-1729) change the coprocessor deployment method
[ https://issues.apache.org/jira/browse/TRAFODION-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15084500#comment-15084500 ] liu ming commented on TRAFODION-1729: - Thanks Stack, I need to learn how to submit a patch to HBase community. And thanks Dave, here is the list of coprocessors: org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint org.apache.hadoop.hbase.coprocessor.AggregateImplementation in the future, there is one more: org.apache.hadoop.hbase.coprocessor.transactional.SsccRegionEndpoint The overhead of adding coprocessor is only involved when a Region is first open. And I assume there is no difference between we use addCoprocessor() java API to add, or we modify hbase-site.xml to add. Just two different ways to add coprocessor in HBase. hbase-site.xml is a global setting. In fact, it will let ALL hbase tables to load Trafodion coprocessors, even for those native hbase tables, which not created by Trafodion. With this change, we can control that only Trafodion tables will be equipped with these coprocessors. Yet another reason to do this change :-) > change the coprocessor deployment method > > > Key: TRAFODION-1729 > URL: https://issues.apache.org/jira/browse/TRAFODION-1729 > Project: Apache Trafodion > Issue Type: Improvement > Components: dtm >Reporter: liu ming >Assignee: mashengchen > > I have a proposal to change our current HBase coprocessor configuration > method. > There are three ways to add a coprocessor to a HBase table: > 1. Via editing hbase-site.xml, which will load coprocessor for ALL > tables (Trafodion is using this method now) > 2. Via HBase shell command > 3. Via HTableDescriptor.addCoprocessor() java API > Trafodion now is using the first method. I proposed to use method 3, I > finished a prototype and test seems works well. > > Here are the reasons I propose for this change: > At present, the Trafodion installer needs to modify the hbase-site.xml and > then restart HBase instance for the configuration to take effect. This step > not only complicate the installer but also let user think Trafodion is > intrusive into underlying HBase system. It will be ideal if we can avoid this > step. Another problem: in CDH, there is a concept called ‘region server > group’ or something, so the settings will have to carefully handled by > installer to apply to all groups. As we saw recently in WebRoot deployment, > Trafodion failed due to this reason. All these are very error prone and > complicate the Trafodion installer. Once CDH or HDP changed something, > Trafodion may fail again. > > So I spent time to investigate why we need to restart HBase in order to > install Trafodion. > As I understand, there are 3 major reasons > 1. To add hbase-trx coprocessors > 2. To overload HRegion with TransactionalRegion > 3. Various configuration settings, need to check one by one. > The first configuration can be avoided by applying my proposed change. > The second one, I look through the TransactionalRegion.java, and find out the > only reason (now) is to overload the getScanner() method to be public so can > be invoked by the coprocessor. And there are only 1 or 2 places that API is > invoked in Trafodion code. I checked with Kevin and he proposed by using > ‘java reflection’ we can also avoid this. > All other configuration items to some extent look like ‘best to have’, but > not ‘must to have’. And I also find two config items seems never been used: > hbase.bulkload.staging.dir /hbase-staging (Suresh can confirm, > but I search in all code, seems this is never used) > hbase.regionserver.region.transactional.tlog true (Narendra can > confirm, this is NEVER used, maybe a legacy config item?) > Yes, by now, there are still some other config items seems cannot be avoided, > but I hope we can find some way to remove them in the future. I am not trying > to solve all issues right now, just want to start the effort to remove > unnecessary hbase reconfiguration. > For this example, Coprocessors can be added to a table at run time, no need > to edit the hbase-site.xml and restart hbase. This is only the first step to > try to remove the deep impact to the current HBase config and restart HBase. > > So I asked for your opinions about this change. If you think this is > necessary, I will continue to file a JIRA and fix it. > > I strongly recommend to get rid of the step of ‘modify hbase-site.xml and > restart your hbase’ for Trafodion installation, it should be an option , to > tune the system to best suit Trafodion, but should not be a forced step. To > be note: Apache Phoenix is also a SQL on HBase, its in
[jira] [Commented] (TRAFODION-1729) change the coprocessor deployment method
[ https://issues.apache.org/jira/browse/TRAFODION-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086429#comment-15086429 ] liu ming commented on TRAFODION-1729: - Thanks Sean, After this change, and 1730, there are still these items seems not able to avoid HBase restart: hbase.snapshot.master.timeoutMillis 60 hbase_regionserver_lease_period 60 hbase.regionserver.region.split.policy org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy hbase.snapshot.enabled true hbase.master.distributed.log.splitting false My hope is : hbase.snapsot.enable is by default true, and in most case it is. So we don't need to change it. I think with new improvement, we can allow distributed log splitting in HBase. In fact, I am not quite sure if Trafodion real change this or not, and why we need this. For others, I still need to investigate, but seems they are no longer 'forced' changes (some timeouts), so we can ask user to change it in Installer, but not a forced step, an optional step. thanks, Ming > change the coprocessor deployment method > > > Key: TRAFODION-1729 > URL: https://issues.apache.org/jira/browse/TRAFODION-1729 > Project: Apache Trafodion > Issue Type: Improvement > Components: dtm >Reporter: liu ming >Assignee: mashengchen > > I have a proposal to change our current HBase coprocessor configuration > method. > There are three ways to add a coprocessor to a HBase table: > 1. Via editing hbase-site.xml, which will load coprocessor for ALL > tables (Trafodion is using this method now) > 2. Via HBase shell command > 3. Via HTableDescriptor.addCoprocessor() java API > Trafodion now is using the first method. I proposed to use method 3, I > finished a prototype and test seems works well. > > Here are the reasons I propose for this change: > At present, the Trafodion installer needs to modify the hbase-site.xml and > then restart HBase instance for the configuration to take effect. This step > not only complicate the installer but also let user think Trafodion is > intrusive into underlying HBase system. It will be ideal if we can avoid this > step. Another problem: in CDH, there is a concept called ‘region server > group’ or something, so the settings will have to carefully handled by > installer to apply to all groups. As we saw recently in WebRoot deployment, > Trafodion failed due to this reason. All these are very error prone and > complicate the Trafodion installer. Once CDH or HDP changed something, > Trafodion may fail again. > > So I spent time to investigate why we need to restart HBase in order to > install Trafodion. > As I understand, there are 3 major reasons > 1. To add hbase-trx coprocessors > 2. To overload HRegion with TransactionalRegion > 3. Various configuration settings, need to check one by one. > The first configuration can be avoided by applying my proposed change. > The second one, I look through the TransactionalRegion.java, and find out the > only reason (now) is to overload the getScanner() method to be public so can > be invoked by the coprocessor. And there are only 1 or 2 places that API is > invoked in Trafodion code. I checked with Kevin and he proposed by using > ‘java reflection’ we can also avoid this. > All other configuration items to some extent look like ‘best to have’, but > not ‘must to have’. And I also find two config items seems never been used: > hbase.bulkload.staging.dir /hbase-staging (Suresh can confirm, > but I search in all code, seems this is never used) > hbase.regionserver.region.transactional.tlog true (Narendra can > confirm, this is NEVER used, maybe a legacy config item?) > Yes, by now, there are still some other config items seems cannot be avoided, > but I hope we can find some way to remove them in the future. I am not trying > to solve all issues right now, just want to start the effort to remove > unnecessary hbase reconfiguration. > For this example, Coprocessors can be added to a table at run time, no need > to edit the hbase-site.xml and restart hbase. This is only the first step to > try to remove the deep impact to the current HBase config and restart HBase. > > So I asked for your opinions about this change. If you think this is > necessary, I will continue to file a JIRA and fix it. > > I strongly recommend to get rid of the step of ‘modify hbase-site.xml and > restart your hbase’ for Trafodion installation, it should be an option , to > tune the system to best suit Trafodion, but should not be a forced step. To > be note: Apache Phoenix is also a SQL on HBase, its installation will change > nothing of underlying HBase, very lightweight, no ‘intrude into’ the existing > HBase system. Trafodion is conside
[jira] [Commented] (TRAFODION-1729) change the coprocessor deployment method
[ https://issues.apache.org/jira/browse/TRAFODION-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086433#comment-15086433 ] liu ming commented on TRAFODION-1729: - Hi, Dave, As per my understanding, Trafodion cannot support ACID on native HBase tables access. If this is a requirement, yes, this is a problem. However, I feel it is still possible, we can alter the HBase table to add coprocessor when Trafodion need to transactionally access it. I will do an prototype. But I think we first need to confirm: Is now Trafodion support ACID on native HBase tables? thanks, Ming > change the coprocessor deployment method > > > Key: TRAFODION-1729 > URL: https://issues.apache.org/jira/browse/TRAFODION-1729 > Project: Apache Trafodion > Issue Type: Improvement > Components: dtm >Reporter: liu ming >Assignee: mashengchen > > I have a proposal to change our current HBase coprocessor configuration > method. > There are three ways to add a coprocessor to a HBase table: > 1. Via editing hbase-site.xml, which will load coprocessor for ALL > tables (Trafodion is using this method now) > 2. Via HBase shell command > 3. Via HTableDescriptor.addCoprocessor() java API > Trafodion now is using the first method. I proposed to use method 3, I > finished a prototype and test seems works well. > > Here are the reasons I propose for this change: > At present, the Trafodion installer needs to modify the hbase-site.xml and > then restart HBase instance for the configuration to take effect. This step > not only complicate the installer but also let user think Trafodion is > intrusive into underlying HBase system. It will be ideal if we can avoid this > step. Another problem: in CDH, there is a concept called ‘region server > group’ or something, so the settings will have to carefully handled by > installer to apply to all groups. As we saw recently in WebRoot deployment, > Trafodion failed due to this reason. All these are very error prone and > complicate the Trafodion installer. Once CDH or HDP changed something, > Trafodion may fail again. > > So I spent time to investigate why we need to restart HBase in order to > install Trafodion. > As I understand, there are 3 major reasons > 1. To add hbase-trx coprocessors > 2. To overload HRegion with TransactionalRegion > 3. Various configuration settings, need to check one by one. > The first configuration can be avoided by applying my proposed change. > The second one, I look through the TransactionalRegion.java, and find out the > only reason (now) is to overload the getScanner() method to be public so can > be invoked by the coprocessor. And there are only 1 or 2 places that API is > invoked in Trafodion code. I checked with Kevin and he proposed by using > ‘java reflection’ we can also avoid this. > All other configuration items to some extent look like ‘best to have’, but > not ‘must to have’. And I also find two config items seems never been used: > hbase.bulkload.staging.dir /hbase-staging (Suresh can confirm, > but I search in all code, seems this is never used) > hbase.regionserver.region.transactional.tlog true (Narendra can > confirm, this is NEVER used, maybe a legacy config item?) > Yes, by now, there are still some other config items seems cannot be avoided, > but I hope we can find some way to remove them in the future. I am not trying > to solve all issues right now, just want to start the effort to remove > unnecessary hbase reconfiguration. > For this example, Coprocessors can be added to a table at run time, no need > to edit the hbase-site.xml and restart hbase. This is only the first step to > try to remove the deep impact to the current HBase config and restart HBase. > > So I asked for your opinions about this change. If you think this is > necessary, I will continue to file a JIRA and fix it. > > I strongly recommend to get rid of the step of ‘modify hbase-site.xml and > restart your hbase’ for Trafodion installation, it should be an option , to > tune the system to best suit Trafodion, but should not be a forced step. To > be note: Apache Phoenix is also a SQL on HBase, its installation will change > nothing of underlying HBase, very lightweight, no ‘intrude into’ the existing > HBase system. Trafodion is considered to be heavy and intrusive in this > manner, and I feel maybe we can change this. > > Should I start this discussion in the dev mail list? > > P.S. a list of changed config items. My proposal will remove the last one, > hope we can get rid of all of them: > hbase.master.distributed.log.splitting false > hbase.snapshot.master.timeoutMillis 60 > hbase_regionserver_lease_period 60 > hbase.hregion.impl
[jira] [Commented] (TRAFODION-1729) change the coprocessor deployment method
[ https://issues.apache.org/jira/browse/TRAFODION-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086565#comment-15086565 ] liu ming commented on TRAFODION-1729: - Thanks Narendra, Then, this may be a very big issue. I will try to do some testing. The solution in my mind is before access the native HBase table, call method to alter the target table, to add trafodion coprocessors at runtime, when close the table, remove trafodion coprocessors. But I don't know if HBase support this or not, need some investigation. Ming > change the coprocessor deployment method > > > Key: TRAFODION-1729 > URL: https://issues.apache.org/jira/browse/TRAFODION-1729 > Project: Apache Trafodion > Issue Type: Improvement > Components: dtm >Reporter: liu ming >Assignee: mashengchen > > I have a proposal to change our current HBase coprocessor configuration > method. > There are three ways to add a coprocessor to a HBase table: > 1. Via editing hbase-site.xml, which will load coprocessor for ALL > tables (Trafodion is using this method now) > 2. Via HBase shell command > 3. Via HTableDescriptor.addCoprocessor() java API > Trafodion now is using the first method. I proposed to use method 3, I > finished a prototype and test seems works well. > > Here are the reasons I propose for this change: > At present, the Trafodion installer needs to modify the hbase-site.xml and > then restart HBase instance for the configuration to take effect. This step > not only complicate the installer but also let user think Trafodion is > intrusive into underlying HBase system. It will be ideal if we can avoid this > step. Another problem: in CDH, there is a concept called ‘region server > group’ or something, so the settings will have to carefully handled by > installer to apply to all groups. As we saw recently in WebRoot deployment, > Trafodion failed due to this reason. All these are very error prone and > complicate the Trafodion installer. Once CDH or HDP changed something, > Trafodion may fail again. > > So I spent time to investigate why we need to restart HBase in order to > install Trafodion. > As I understand, there are 3 major reasons > 1. To add hbase-trx coprocessors > 2. To overload HRegion with TransactionalRegion > 3. Various configuration settings, need to check one by one. > The first configuration can be avoided by applying my proposed change. > The second one, I look through the TransactionalRegion.java, and find out the > only reason (now) is to overload the getScanner() method to be public so can > be invoked by the coprocessor. And there are only 1 or 2 places that API is > invoked in Trafodion code. I checked with Kevin and he proposed by using > ‘java reflection’ we can also avoid this. > All other configuration items to some extent look like ‘best to have’, but > not ‘must to have’. And I also find two config items seems never been used: > hbase.bulkload.staging.dir /hbase-staging (Suresh can confirm, > but I search in all code, seems this is never used) > hbase.regionserver.region.transactional.tlog true (Narendra can > confirm, this is NEVER used, maybe a legacy config item?) > Yes, by now, there are still some other config items seems cannot be avoided, > but I hope we can find some way to remove them in the future. I am not trying > to solve all issues right now, just want to start the effort to remove > unnecessary hbase reconfiguration. > For this example, Coprocessors can be added to a table at run time, no need > to edit the hbase-site.xml and restart hbase. This is only the first step to > try to remove the deep impact to the current HBase config and restart HBase. > > So I asked for your opinions about this change. If you think this is > necessary, I will continue to file a JIRA and fix it. > > I strongly recommend to get rid of the step of ‘modify hbase-site.xml and > restart your hbase’ for Trafodion installation, it should be an option , to > tune the system to best suit Trafodion, but should not be a forced step. To > be note: Apache Phoenix is also a SQL on HBase, its installation will change > nothing of underlying HBase, very lightweight, no ‘intrude into’ the existing > HBase system. Trafodion is considered to be heavy and intrusive in this > manner, and I feel maybe we can change this. > > Should I start this discussion in the dev mail list? > > P.S. a list of changed config items. My proposal will remove the last one, > hope we can get rid of all of them: > hbase.master.distributed.log.splitting false > hbase.snapshot.master.timeoutMillis 60 > hbase_regionserver_lease_period 60 > hbase.hregion.impl > org.apache.hadoop.hbase.regionserver.
[jira] [Commented] (TRAFODION-1729) change the coprocessor deployment method
[ https://issues.apache.org/jira/browse/TRAFODION-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15087053#comment-15087053 ] liu ming commented on TRAFODION-1729: - Hi, Stack, As per my understanding, some configuration items are client-side . (I don't know if there is such a concept in HBase .. ). For example: hbase.rpc.timeout I think a client app can write code like this : Configuration conf = HBaseConfiguration.create(); conf.set(“hbase.rpc.timeout”,”6”); to change this setting , without HBase master or Region Server to restart for that new timeout to take effect. But settings like 'hbase.snapshot.master.timeoutMillis' is a server-side config, and need a restart. In fact, this is one of the confusing concept for me. Both client and HBase server use hbase-site.xml , and I don't find an article talking about differences. Some can be changed in app side, some not. BTW, I am studying how to submit a patch to HBase, it will take some time. And I file a JIRA already: https://issues.apache.org/jira/browse/HBASE-15076 And get a comment, it seems I need to study HBase internals more, it is not a single line of change. thanks, Ming > change the coprocessor deployment method > > > Key: TRAFODION-1729 > URL: https://issues.apache.org/jira/browse/TRAFODION-1729 > Project: Apache Trafodion > Issue Type: Improvement > Components: dtm >Reporter: liu ming >Assignee: mashengchen > > I have a proposal to change our current HBase coprocessor configuration > method. > There are three ways to add a coprocessor to a HBase table: > 1. Via editing hbase-site.xml, which will load coprocessor for ALL > tables (Trafodion is using this method now) > 2. Via HBase shell command > 3. Via HTableDescriptor.addCoprocessor() java API > Trafodion now is using the first method. I proposed to use method 3, I > finished a prototype and test seems works well. > > Here are the reasons I propose for this change: > At present, the Trafodion installer needs to modify the hbase-site.xml and > then restart HBase instance for the configuration to take effect. This step > not only complicate the installer but also let user think Trafodion is > intrusive into underlying HBase system. It will be ideal if we can avoid this > step. Another problem: in CDH, there is a concept called ‘region server > group’ or something, so the settings will have to carefully handled by > installer to apply to all groups. As we saw recently in WebRoot deployment, > Trafodion failed due to this reason. All these are very error prone and > complicate the Trafodion installer. Once CDH or HDP changed something, > Trafodion may fail again. > > So I spent time to investigate why we need to restart HBase in order to > install Trafodion. > As I understand, there are 3 major reasons > 1. To add hbase-trx coprocessors > 2. To overload HRegion with TransactionalRegion > 3. Various configuration settings, need to check one by one. > The first configuration can be avoided by applying my proposed change. > The second one, I look through the TransactionalRegion.java, and find out the > only reason (now) is to overload the getScanner() method to be public so can > be invoked by the coprocessor. And there are only 1 or 2 places that API is > invoked in Trafodion code. I checked with Kevin and he proposed by using > ‘java reflection’ we can also avoid this. > All other configuration items to some extent look like ‘best to have’, but > not ‘must to have’. And I also find two config items seems never been used: > hbase.bulkload.staging.dir /hbase-staging (Suresh can confirm, > but I search in all code, seems this is never used) > hbase.regionserver.region.transactional.tlog true (Narendra can > confirm, this is NEVER used, maybe a legacy config item?) > Yes, by now, there are still some other config items seems cannot be avoided, > but I hope we can find some way to remove them in the future. I am not trying > to solve all issues right now, just want to start the effort to remove > unnecessary hbase reconfiguration. > For this example, Coprocessors can be added to a table at run time, no need > to edit the hbase-site.xml and restart hbase. This is only the first step to > try to remove the deep impact to the current HBase config and restart HBase. > > So I asked for your opinions about this change. If you think this is > necessary, I will continue to file a JIRA and fix it. > > I strongly recommend to get rid of the step of ‘modify hbase-site.xml and > restart your hbase’ for Trafodion installation, it should be an option , to > tune the system to best suit Trafodion, but should not be a forced step. To > be note: Apache Phoenix is also a SQL on HBase, its install
[jira] [Created] (TRAFODION-1745) show more related info when TRANSLATE run into SQL Error
liu ming created TRAFODION-1745: --- Summary: show more related info when TRANSLATE run into SQL Error Key: TRAFODION-1745 URL: https://issues.apache.org/jira/browse/TRAFODION-1745 Project: Apache Trafodion Issue Type: Improvement Reporter: liu ming The current error message for a TRANSLATE error can be : 8690 "An invalid character value encountered in TRANSLATE function." Need to show more helping info for easier debug, and user friendly. to include a short section of offending source data (may be in hex form), source charset, and target charset. Checking other SQL errors related if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1730) try to remove the requirement of using hbase.hregion.impl in hbase-site.xml
[ https://issues.apache.org/jira/browse/TRAFODION-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088270#comment-15088270 ] liu ming commented on TRAFODION-1730: - HBase 1.2+ will support the public getScanner(scan, additionalScanner) method. Trafodion needs to port to HBase 1.2+ , then can remove this dependency from Installer. > try to remove the requirement of using hbase.hregion.impl in hbase-site.xml > --- > > Key: TRAFODION-1730 > URL: https://issues.apache.org/jira/browse/TRAFODION-1730 > Project: Apache Trafodion > Issue Type: Improvement >Reporter: liu ming >Assignee: mashengchen > > look through the TransactionalRegion.java, and find out the only reason (now) > is to overload the getScanner() method to be public so can be invoked by the > coprocessor. And there are only 1 or 2 places that API is invoked in > Trafodion code. > Should be able to user 'java reflection' to overcome this requirement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1720) add GBK support in SQL translate function
[ https://issues.apache.org/jira/browse/TRAFODION-1720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090544#comment-15090544 ] liu ming commented on TRAFODION-1720: - Following the discussion in dev list, the current solution is as follow: add a new CQD: HIVE_FILE_CHARSET, default it is 'UTF8'. If the raw data in Hive table is GBK, then set this CQD to 'GBK' before doing the bulkloading. In the bulkloader, use TRANSLATE function 'GBKTOUTF8' to convert data from GBK into UTF8. In this way, the data will be converted and go into Trafodion table in UTF8 encoding. Since Trafodion does not support GBK totally, so it is not proper to change the HIVE_DEFAULT_CHARSET to 'GBK' at this point. In the future, when Trafodion can support GBK as column charset type, we can get rid of HIVE_FILE_CHARSET and use HIVE_DEFAULT_CHARSET only. In the code, Trafodion will check the HIVE_FILE_CHARSET vs. HIVE_DEFAULT_CHARSET , if they are same, then the TRANSLATE must match the source charset and the target charset, otherwise, ignore the source charset type, which defined by HIVE_DEFAULT_CHARSET. In other words, HIVE_FILE_CHARSET will overwrite HIVE_DEFAULT_CHARSET in a TRANSALTE GBKTOUTF8 function. With this change, the HIVE_DEFAULT_CHARSET is keep as before, no change to other behavior in the Trafodion database. Just affect the TRANSLATE function 'GBKTOUTF8', this is a solution to allow user to load raw GBK data into Trafodion without do conversion themselves. > add GBK support in SQL translate function > - > > Key: TRAFODION-1720 > URL: https://issues.apache.org/jira/browse/TRAFODION-1720 > Project: Apache Trafodion > Issue Type: Improvement > Components: sql-exe >Reporter: liu ming >Assignee: liu ming > > Trafodion support TRANSLATE function now and support character set conversion > among ISO88591, SJIS, UCS2 and UTF8. It will be useful to support GBK and > later BIG5. > GBK is widely used in China. Without support, all source data need to be > converted before data loading. With built-in support, it will decrease the > overall data loading time. > External interface > Use the current SQL Function TRANSLATE, introduce new translation, keep the > original syntax. > > TRANSLATE(character-value-expression USING translation-name) > Translation-name Source charset Target charset Comments > == == = > GBKTOUTF8 GBK2312 UTF8When error occur during > translation, SQL Error > UTF8TOGBK UTF8GB2312 > Internal > Parser change > Let class Translate support new translation-name. > CodeGen change > Translate::codeGen > Executor change > convDoIt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1750) remove build dependency on qpid
liu ming created TRAFODION-1750: --- Summary: remove build dependency on qpid Key: TRAFODION-1750 URL: https://issues.apache.org/jira/browse/TRAFODION-1750 Project: Apache Trafodion Issue Type: Improvement Reporter: liu ming Assignee: liu ming Trafodion build now has a dependency on qpid, but in fact, qpid is not used inside Trafodion. It is just a legacy code dependency, some source code refer to qpid but never used. So we can remove those unused functions/source-code to remove the dependency for qpid, and simplify the preparation of a development environment. Now one must install qpid-cpp-client-devel package in special way for CentOS newer than 6.3, which is too old : yum --enablerepo=C6.3-updates install qpid-cpp-client-devel This is one of the effort to make Trafodion development easier, we should also try to clean up other legacy dependency as much as possible, this will not only make build easier so as installation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (TRAFODION-1720) add GBK support in SQL translate function
[ https://issues.apache.org/jira/browse/TRAFODION-1720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TRAFODION-1720 started by liu ming. --- > add GBK support in SQL translate function > - > > Key: TRAFODION-1720 > URL: https://issues.apache.org/jira/browse/TRAFODION-1720 > Project: Apache Trafodion > Issue Type: Improvement > Components: sql-exe >Reporter: liu ming >Assignee: liu ming > > Trafodion support TRANSLATE function now and support character set conversion > among ISO88591, SJIS, UCS2 and UTF8. It will be useful to support GBK and > later BIG5. > GBK is widely used in China. Without support, all source data need to be > converted before data loading. With built-in support, it will decrease the > overall data loading time. > External interface > Use the current SQL Function TRANSLATE, introduce new translation, keep the > original syntax. > > TRANSLATE(character-value-expression USING translation-name) > Translation-name Source charset Target charset Comments > == == = > GBKTOUTF8 GBK2312 UTF8When error occur during > translation, SQL Error > UTF8TOGBK UTF8GB2312 > Internal > Parser change > Let class Translate support new translation-name. > CodeGen change > Translate::codeGen > Executor change > convDoIt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1750) remove build dependency on qpid
[ https://issues.apache.org/jira/browse/TRAFODION-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1750: Assignee: taian.wei (was: liu ming) > remove build dependency on qpid > --- > > Key: TRAFODION-1750 > URL: https://issues.apache.org/jira/browse/TRAFODION-1750 > Project: Apache Trafodion > Issue Type: Improvement >Reporter: liu ming >Assignee: taian.wei > > Trafodion build now has a dependency on qpid, but in fact, qpid is not used > inside Trafodion. It is just a legacy code dependency, some source code refer > to qpid but never used. > So we can remove those unused functions/source-code to remove the dependency > for qpid, and simplify the preparation of a development environment. > Now one must install qpid-cpp-client-devel package in special way for CentOS > newer than 6.3, which is too old : > yum --enablerepo=C6.3-updates install qpid-cpp-client-devel > This is one of the effort to make Trafodion development easier, we should > also try to clean up other legacy dependency as much as possible, this will > not only make build easier so as installation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1751) create table crash
liu ming created TRAFODION-1751: --- Summary: create table crash Key: TRAFODION-1751 URL: https://issues.apache.org/jira/browse/TRAFODION-1751 Project: Apache Trafodion Issue Type: Bug Reporter: liu ming Priority: Critical The following create table DDL will crash Trafodion instance. Tested in two different systems and can reproduce: in sqlci CREATE TABLE FactJCSales ( ReportDateID int NOT NULL, FinanceDateID int NULL, ProvinceCenterID int NULL, CityCenterID int NULL, OrgKey int NULL, GameKey int NULL, DrawKey int NULL, TerminalKey int NULL, BranchID int NULL, ShopKey int NULL, LogonID int NULL, AccountID int NULL, TechSystemID int NULL, ChannelTypeCode int NULL, SaleLotteryCnt int NULL, SaleStakeCnt int NULL, SaleAmount decimal (18, 4) NULL, SaleLotteryCntSettled int NULL, SaleStakeCntSettled int NULL, SaleAmountSettled decimal (18, 4) NULL, CancelLotteryCnt int NULL, CancelStakeCnt int NULL, CancelAmount decimal (18, 4) NULL, ProvCancelLotteryCnt int NULL, ProvCancelStakeCnt int NULL, ProvCancelAmount decimal (18, 4) NULL, BranchCancelLotteryCnt int NULL, BranchCancelStakeCnt int NULL, BranchCancelAmount decimal (18, 4) NULL ) store by (ReprotDateID) SALT USING 80 PARTITIONS HBASE_OPTIONS ( DATA_BLOCK_ENCODING = 'FAST_DIFF', COMPRESSION = 'SNAPPY', MEMSTORE_FLUSH_SIZE = '1073741824' ) ; *** EXECUTOR ASSERTION FAILURE *** Time: Mon Jan 11 08:57:50 2016 *** Process: 6860 *** File: ../common/Collections.cpp *** Line: 874 *** Message: List index exceeds # of entries Aborted -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1751) create table crash
[ https://issues.apache.org/jira/browse/TRAFODION-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1751: Priority: Major (was: Critical) > create table crash > -- > > Key: TRAFODION-1751 > URL: https://issues.apache.org/jira/browse/TRAFODION-1751 > Project: Apache Trafodion > Issue Type: Bug >Reporter: liu ming > > The following create table DDL will crash Trafodion instance. > Tested in two different systems and can reproduce: > in sqlci > CREATE TABLE FactJCSales > ( > ReportDateID int NOT NULL, > FinanceDateID int NULL, > ProvinceCenterID int NULL, > CityCenterID int NULL, > OrgKey int NULL, > GameKey int NULL, > DrawKey int NULL, > TerminalKey int NULL, > BranchID int NULL, > ShopKey int NULL, > LogonID int NULL, > AccountID int NULL, > TechSystemID int NULL, > ChannelTypeCode int NULL, > SaleLotteryCnt int NULL, > SaleStakeCnt int NULL, > SaleAmount decimal (18, 4) NULL, > SaleLotteryCntSettled int NULL, > SaleStakeCntSettled int NULL, > SaleAmountSettled decimal (18, 4) NULL, > CancelLotteryCnt int NULL, > CancelStakeCnt int NULL, > CancelAmount decimal (18, 4) NULL, > ProvCancelLotteryCnt int NULL, > ProvCancelStakeCnt int NULL, > ProvCancelAmount decimal (18, 4) NULL, > BranchCancelLotteryCnt int NULL, > BranchCancelStakeCnt int NULL, > BranchCancelAmount decimal (18, 4) NULL > ) > store by (ReprotDateID) > SALT USING 80 PARTITIONS > HBASE_OPTIONS > ( > DATA_BLOCK_ENCODING = 'FAST_DIFF', > COMPRESSION = 'SNAPPY', > MEMSTORE_FLUSH_SIZE = '1073741824' > ) > ; > *** EXECUTOR ASSERTION FAILURE > *** Time: Mon Jan 11 08:57:50 2016 > *** Process: 6860 > *** File: ../common/Collections.cpp > *** Line: 874 > *** Message: List index exceeds # of entries > Aborted -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1751) create table crash
[ https://issues.apache.org/jira/browse/TRAFODION-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093480#comment-15093480 ] liu ming commented on TRAFODION-1751: - thanks Anoop, I lower the priority, since it is not a crash of Trafodion instance, and it is a typo in the DDL. Just need to enhance the error message, and not abort in this case. So no longer a critical issue. > create table crash > -- > > Key: TRAFODION-1751 > URL: https://issues.apache.org/jira/browse/TRAFODION-1751 > Project: Apache Trafodion > Issue Type: Bug >Reporter: liu ming > > The following create table DDL will crash Trafodion instance. > Tested in two different systems and can reproduce: > in sqlci > CREATE TABLE FactJCSales > ( > ReportDateID int NOT NULL, > FinanceDateID int NULL, > ProvinceCenterID int NULL, > CityCenterID int NULL, > OrgKey int NULL, > GameKey int NULL, > DrawKey int NULL, > TerminalKey int NULL, > BranchID int NULL, > ShopKey int NULL, > LogonID int NULL, > AccountID int NULL, > TechSystemID int NULL, > ChannelTypeCode int NULL, > SaleLotteryCnt int NULL, > SaleStakeCnt int NULL, > SaleAmount decimal (18, 4) NULL, > SaleLotteryCntSettled int NULL, > SaleStakeCntSettled int NULL, > SaleAmountSettled decimal (18, 4) NULL, > CancelLotteryCnt int NULL, > CancelStakeCnt int NULL, > CancelAmount decimal (18, 4) NULL, > ProvCancelLotteryCnt int NULL, > ProvCancelStakeCnt int NULL, > ProvCancelAmount decimal (18, 4) NULL, > BranchCancelLotteryCnt int NULL, > BranchCancelStakeCnt int NULL, > BranchCancelAmount decimal (18, 4) NULL > ) > store by (ReprotDateID) > SALT USING 80 PARTITIONS > HBASE_OPTIONS > ( > DATA_BLOCK_ENCODING = 'FAST_DIFF', > COMPRESSION = 'SNAPPY', > MEMSTORE_FLUSH_SIZE = '1073741824' > ) > ; > *** EXECUTOR ASSERTION FAILURE > *** Time: Mon Jan 11 08:57:50 2016 > *** Process: 6860 > *** File: ../common/Collections.cpp > *** Line: 874 > *** Message: List index exceeds # of entries > Aborted -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1745) show more related info when TRANSLATE run into SQL Error
[ https://issues.apache.org/jira/browse/TRAFODION-1745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1745: --- Assignee: liu ming > show more related info when TRANSLATE run into SQL Error > > > Key: TRAFODION-1745 > URL: https://issues.apache.org/jira/browse/TRAFODION-1745 > Project: Apache Trafodion > Issue Type: Improvement >Reporter: liu ming >Assignee: liu ming > > The current error message for a TRANSLATE error can be : > 8690 "An invalid character value encountered in TRANSLATE function." > Need to show more helping info for easier debug, and user friendly. > to include a short section of offending source data (may be in hex form), > source charset, and target charset. > Checking other SQL errors related if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1751) create table crash
[ https://issues.apache.org/jira/browse/TRAFODION-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1751: --- Assignee: liu ming > create table crash > -- > > Key: TRAFODION-1751 > URL: https://issues.apache.org/jira/browse/TRAFODION-1751 > Project: Apache Trafodion > Issue Type: Bug >Reporter: liu ming >Assignee: liu ming > > The following create table DDL will crash Trafodion instance. > Tested in two different systems and can reproduce: > in sqlci > CREATE TABLE FactJCSales > ( > ReportDateID int NOT NULL, > FinanceDateID int NULL, > ProvinceCenterID int NULL, > CityCenterID int NULL, > OrgKey int NULL, > GameKey int NULL, > DrawKey int NULL, > TerminalKey int NULL, > BranchID int NULL, > ShopKey int NULL, > LogonID int NULL, > AccountID int NULL, > TechSystemID int NULL, > ChannelTypeCode int NULL, > SaleLotteryCnt int NULL, > SaleStakeCnt int NULL, > SaleAmount decimal (18, 4) NULL, > SaleLotteryCntSettled int NULL, > SaleStakeCntSettled int NULL, > SaleAmountSettled decimal (18, 4) NULL, > CancelLotteryCnt int NULL, > CancelStakeCnt int NULL, > CancelAmount decimal (18, 4) NULL, > ProvCancelLotteryCnt int NULL, > ProvCancelStakeCnt int NULL, > ProvCancelAmount decimal (18, 4) NULL, > BranchCancelLotteryCnt int NULL, > BranchCancelStakeCnt int NULL, > BranchCancelAmount decimal (18, 4) NULL > ) > store by (ReprotDateID) > SALT USING 80 PARTITIONS > HBASE_OPTIONS > ( > DATA_BLOCK_ENCODING = 'FAST_DIFF', > COMPRESSION = 'SNAPPY', > MEMSTORE_FLUSH_SIZE = '1073741824' > ) > ; > *** EXECUTOR ASSERTION FAILURE > *** Time: Mon Jan 11 08:57:50 2016 > *** Process: 6860 > *** File: ../common/Collections.cpp > *** Line: 874 > *** Message: List index exceeds # of entries > Aborted -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1676) support better runtime error message when a SQL function meet fital error
[ https://issues.apache.org/jira/browse/TRAFODION-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1676: --- Assignee: liu ming > support better runtime error message when a SQL function meet fital error > - > > Key: TRAFODION-1676 > URL: https://issues.apache.org/jira/browse/TRAFODION-1676 > Project: Apache Trafodion > Issue Type: Improvement > Components: sql-exe >Reporter: liu ming >Assignee: liu ming >Priority: Minor > > A sql contains some SQL function, for example UPPER(), when the input data of > UPPER() contains invalid data, UPPER() will fail. In this case, the whole > query abort, and the error message cannot tell which row has invalid data, so > it is very hard to fix the problem. > It will be good to have row id or the real value of the error row in the > error message to help trouble shooting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1720) add GBK support in SQL translate function
[ https://issues.apache.org/jira/browse/TRAFODION-1720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming resolved TRAFODION-1720. - Resolution: Fixed merged into github > add GBK support in SQL translate function > - > > Key: TRAFODION-1720 > URL: https://issues.apache.org/jira/browse/TRAFODION-1720 > Project: Apache Trafodion > Issue Type: Improvement > Components: sql-exe >Reporter: liu ming >Assignee: liu ming > > Trafodion support TRANSLATE function now and support character set conversion > among ISO88591, SJIS, UCS2 and UTF8. It will be useful to support GBK and > later BIG5. > GBK is widely used in China. Without support, all source data need to be > converted before data loading. With built-in support, it will decrease the > overall data loading time. > External interface > Use the current SQL Function TRANSLATE, introduce new translation, keep the > original syntax. > > TRANSLATE(character-value-expression USING translation-name) > Translation-name Source charset Target charset Comments > == == = > GBKTOUTF8 GBK2312 UTF8When error occur during > translation, SQL Error > UTF8TOGBK UTF8GB2312 > Internal > Parser change > Let class Translate support new translation-name. > CodeGen change > Translate::codeGen > Executor change > convDoIt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (TRAFODION-1745) show more related info when TRANSLATE run into SQL Error
[ https://issues.apache.org/jira/browse/TRAFODION-1745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TRAFODION-1745 started by liu ming. --- > show more related info when TRANSLATE run into SQL Error > > > Key: TRAFODION-1745 > URL: https://issues.apache.org/jira/browse/TRAFODION-1745 > Project: Apache Trafodion > Issue Type: Improvement >Reporter: liu ming >Assignee: liu ming > > The current error message for a TRANSLATE error can be : > 8690 "An invalid character value encountered in TRANSLATE function." > Need to show more helping info for easier debug, and user friendly. > to include a short section of offending source data (may be in hex form), > source charset, and target charset. > Checking other SQL errors related if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1750) remove build dependency on qpid
[ https://issues.apache.org/jira/browse/TRAFODION-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1750: --- Assignee: liu ming (was: taian.wei) > remove build dependency on qpid > --- > > Key: TRAFODION-1750 > URL: https://issues.apache.org/jira/browse/TRAFODION-1750 > Project: Apache Trafodion > Issue Type: Improvement >Reporter: liu ming >Assignee: liu ming > > Trafodion build now has a dependency on qpid, but in fact, qpid is not used > inside Trafodion. It is just a legacy code dependency, some source code refer > to qpid but never used. > So we can remove those unused functions/source-code to remove the dependency > for qpid, and simplify the preparation of a development environment. > Now one must install qpid-cpp-client-devel package in special way for CentOS > newer than 6.3, which is too old : > yum --enablerepo=C6.3-updates install qpid-cpp-client-devel > This is one of the effort to make Trafodion development easier, we should > also try to clean up other legacy dependency as much as possible, this will > not only make build easier so as installation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (TRAFODION-1750) remove build dependency on qpid
[ https://issues.apache.org/jira/browse/TRAFODION-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TRAFODION-1750 started by liu ming. --- > remove build dependency on qpid > --- > > Key: TRAFODION-1750 > URL: https://issues.apache.org/jira/browse/TRAFODION-1750 > Project: Apache Trafodion > Issue Type: Improvement >Reporter: liu ming >Assignee: liu ming > > Trafodion build now has a dependency on qpid, but in fact, qpid is not used > inside Trafodion. It is just a legacy code dependency, some source code refer > to qpid but never used. > So we can remove those unused functions/source-code to remove the dependency > for qpid, and simplify the preparation of a development environment. > Now one must install qpid-cpp-client-devel package in special way for CentOS > newer than 6.3, which is too old : > yum --enablerepo=C6.3-updates install qpid-cpp-client-devel > This is one of the effort to make Trafodion development easier, we should > also try to clean up other legacy dependency as much as possible, this will > not only make build easier so as installation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1531) Hitting WrongRegionException during trickle load testing
[ https://issues.apache.org/jira/browse/TRAFODION-1531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1531: --- Assignee: liu ming > Hitting WrongRegionException during trickle load testing > > > Key: TRAFODION-1531 > URL: https://issues.apache.org/jira/browse/TRAFODION-1531 > Project: Apache Trafodion > Issue Type: Bug > Components: dtm >Affects Versions: 1.0 (pre-incubation) >Reporter: Oliver Bucaojit >Assignee: liu ming > > Selva and Dennis have been testing trickle loading and an error is causing > the logs to be filled and the WrongRegionException to be logged in the HBase > server logs. > Here is a stack trace that is being logged: > 2015-10-14 16:46:02,664 WARN org.apache.hadoop.hbase.regionserver.HRegion: > Failed getting lock in batch put, > row=\x00\x00\x00\x18\x80\x00\x00\x00\x00\x00\x0B\xA0\x80\x00\x00\x01 > org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row out > of range for row lock on HRegion > .. > .. > >at > org.apache.hadoop.hbase.regionserver.HRegion.doBatchMutate(HRegion.java:3380) >at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:2546) >at > org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint.commit(TrxRegionEndpoint.java:3754) >at > org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint.commit(TrxRegionEndpoint.java:4615) >at > org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint.commit(TrxRegionEndpoint.java:663) > > The root cause is still unknown, a few ideas we have would be: > 1) The list of puts method is being used and is batching row(s) to the > incorrect region > 2) There is an issue with the endkey byte-1 method and the row is being sent > to the wrong region from the client > One change that will help debug the issue and keep the database consistent is > to add a checkRow() method on the server-side put and delete to make sure the > row key belongs in the region. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1566) Ungraceful failure when transaction size limit reached
[ https://issues.apache.org/jira/browse/TRAFODION-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1566: --- Assignee: liu ming > Ungraceful failure when transaction size limit reached > -- > > Key: TRAFODION-1566 > URL: https://issues.apache.org/jira/browse/TRAFODION-1566 > Project: Apache Trafodion > Issue Type: Bug > Components: dtm, sql-exe >Affects Versions: 1.3-incubating >Reporter: David Wayne Birdsall >Assignee: liu ming >Priority: Minor > > When exceeding transaction size limits, DELETE fails with a puzzling error > message. > The following script produces the problem on a workstation (using > install_local_hadoop, so the HMaster process is handling all four regions). > Best results if the setup section is done in a separate sqlci from the > deleteTest section (so you get current statistics): > ?section setup > create schema DeleteFailure; > set schema DeleteFailure; > -- create a table saltx with 458752 (=7*65536) rows, and another table > -- salty, that is a copy of saltx > CREATE TABLE saltx > ( > AINT NO DEFAULT NOT NULL NOT DROPPABLE > SERIALIZED > , BINT NO DEFAULT NOT NULL NOT DROPPABLE > SERIALIZED > , CVARCHAR(20) CHARACTER SET ISO88591 > COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , PRIMARY KEY (A ASC, B ASC) > ) > SALT USING 4 PARTITIONS > ; > insert into saltx values (1,1,'hi there!'), > (2,1,'bye there!'),(3,1,'Happy Tuesday!'),(4,1,'Huckleberry Pie'); > insert into saltx select a+4,b,c from saltx; > insert into saltx select a+8,b,c from saltx; > insert into saltx select a+16,b,c from saltx; > insert into saltx select a+32,b,c from saltx; > insert into saltx select a+64,b,c from saltx; > insert into saltx select a+128,b,c from saltx; > insert into saltx select a+256,b,c from saltx; > insert into saltx select a+512,b,c from saltx; > insert into saltx select a+1024,b,c from saltx; > upsert into saltx select a+2048,b,c from saltx; > upsert into saltx select a+4096,b,c from saltx; > upsert into saltx select a+8192,b,c from saltx; > upsert into saltx select a+16384,b,c from saltx; > upsert into saltx select a+32768,b,c from saltx; > upsert using load into saltx select a,b+1,c from saltx; > upsert using load into saltx select a,b+2,c from saltx where b = 1; > upsert using load into saltx select a,b+3,c from saltx where b = 1; > upsert using load into saltx select a,b+4,c from saltx where b = 1; > upsert using load into saltx select a,b+5,c from saltx where b = 1; > upsert using load into saltx select a,b+6,c from saltx where b = 1; > update statistics for table saltx on every column; > create table salty like saltx; > upsert using load into salty select * from saltx; > update statistics for table salty on every column; > ?section deleteTest > set schema DeleteFailure; > set param ?b '4'; -- change it to '5' and the delete will succeed > prepare xx from delete from saltx where b > ?b; > explain options 'f' xx; > execute xx; -- fails with ungracious error message > Here is a log showing the deleteTest section failing: > [birdsall@dev02 IUDCosting]$ sqlci > Apache Trafodion Conversational Interface 1.3.0 > Copyright (c) 2015 Apache Software Foundation > >>obey deleteFailure.sql(deleteTest); > >>?section deleteTest > >> > >>set schema DeleteFailure; > --- SQL operation complete. > >> > >>set param ?b '4'; > >> -- change it to '5' and the delete will succeed > >> > >>prepare xx from delete from saltx where b > ?b; > --- SQL command prepared. > >> > >>explain options 'f' xx; > LC RC OP OPERATOR OPT DESCRIPTION CARD > - > 4.5rootx 1.52E+005 > 3.4esp_exchange1:4(hash2)1.52E+005 > 123tuple_flow1.52E+005 > ..2trafodion_vsbb_deletSALTX 1.00E+000 > ..1trafodion_scan SALTX 1.52E+005 > --- SQL operation complete. > >> > >>execute xx; > *** ERROR[8448] Unable to access Hbase interface. Call to > ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-706). Cause: > java.util.concurrent.ExecutionException: java.io.IOException: PerformScan > error on coprocessor call, scannerID: 14 java.io.IOException: performScan > encountered Exception txID: 70081 Exception: > org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: > TrxRegionEndpoint coprocessor: getScanner - scanner id 14, Expected > nextCallSeq: 8, But the nextCallSeq r
[jira] [Commented] (TRAFODION-1750) remove build dependency on qpid
[ https://issues.apache.org/jira/browse/TRAFODION-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15133471#comment-15133471 ] liu ming commented on TRAFODION-1750: - Hi, Steve, Yes, I verified by 'yum remove qpid*' in my development machine, and do a git clean to remove everything and rebuild, it works for me. I will update the wiki after some more tests. thanks, Ming > remove build dependency on qpid > --- > > Key: TRAFODION-1750 > URL: https://issues.apache.org/jira/browse/TRAFODION-1750 > Project: Apache Trafodion > Issue Type: Improvement >Reporter: liu ming >Assignee: liu ming > > Trafodion build now has a dependency on qpid, but in fact, qpid is not used > inside Trafodion. It is just a legacy code dependency, some source code refer > to qpid but never used. > So we can remove those unused functions/source-code to remove the dependency > for qpid, and simplify the preparation of a development environment. > Now one must install qpid-cpp-client-devel package in special way for CentOS > newer than 6.3, which is too old : > yum --enablerepo=C6.3-updates install qpid-cpp-client-devel > This is one of the effort to make Trafodion development easier, we should > also try to clean up other legacy dependency as much as possible, this will > not only make build easier so as installation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1812) cleanup schema causes transaction to hang in aborting state
[ https://issues.apache.org/jira/browse/TRAFODION-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1812: Assignee: Roberta Marton (was: liu ming) > cleanup schema causes transaction to hang in aborting state > --- > > Key: TRAFODION-1812 > URL: https://issues.apache.org/jira/browse/TRAFODION-1812 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmu >Reporter: Roberta Marton >Assignee: Roberta Marton > > If a cleanup schema command is executed and the schema contains histogram > tables, the operation "succeeds" (but not really). One of the transactions > performing the cleanup commands is left in the aborting state, for example: > (1,0,13)0,12159 281474976710669 ABORTING PT2 > willRollback tmAborted readOnly > Test case: > create schema sch1; > set schema sch1; > create table t1(a int not null primary key); > create table t2 (a int not null primary key, > b int not null unique, > c int not null references t1(a), > d blob, > e char(2) check (e > 'a'), > f largeint generated by default as identity not > null); > create index t2 on t2(b); > insert into t1 values (1), (2), (3), (4), (5), (6), (7), (8); > update statistics for table t1 on every column; > get tables; > cleanup schema sch1; > get tables; > get schemas; > After recovering from the transaction issue (restarting Trafodion), the > following objects still exist in hbase: > TRAFODION.SCH1.SB_HISTOGRAM_INTERVALS > > TRAFODION.SCH1.T1 > > TRAFODION.SCH1.T2 > > TRAFODION.SCH1.T2_651147319_7148 > > TRAFODION.SCH1.T2_945247319_7148 > After removing objects from hbase, restarted sqlci: > get schemas - schema sch1 does not exist. > create schema sch1 - succeeded > create table t1 - succeeded > create table t2 - failed *** ERROR[1390] Object TRAFODION.SEABASE.T2 already > exists in Trafodion. > Table t2 is accessible - so metadata entries were not removed. > Looking a bit into the cleanup code - part of removing an object, tries to > delete rows from the histograms table. So if we are removing a histogram > table, probably should not be deleting from it at the same time. May also > want to wait until all the other tables in the schema are removed before > delete histogram tables (or delete the histogram tables first). > Prototyped a change that: > As part of cleanup schema, cleaned up all the non histogram tables first then > cleaned up the histogram tables. > As part of dropping a histogram table, did not call dropSeabaseStats. > This seemed to fix the problem. > May want to handle failures from delete histogram rows better. Also, if > someone tries to cleanup one histogram table, perhaps all should be dropped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1813) add commons-httpclient depenedency into dcs/pom.xml
[ https://issues.apache.org/jira/browse/TRAFODION-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1813: --- Assignee: liu ming > add commons-httpclient depenedency into dcs/pom.xml > --- > > Key: TRAFODION-1813 > URL: https://issues.apache.org/jira/browse/TRAFODION-1813 > Project: Apache Trafodion > Issue Type: Bug >Reporter: liu ming >Assignee: liu ming >Priority: Minor > > Not sure why the dependency of httpclient is missing from pom.xml under dcs > folder, it doesn't affect build in most cases, but I cannot make a build on a > fresh machine, so suggest to explicit add this dependency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1813) add commons-httpclient depenedency into dcs/pom.xml
liu ming created TRAFODION-1813: --- Summary: add commons-httpclient depenedency into dcs/pom.xml Key: TRAFODION-1813 URL: https://issues.apache.org/jira/browse/TRAFODION-1813 Project: Apache Trafodion Issue Type: Bug Reporter: liu ming Priority: Minor Not sure why the dependency of httpclient is missing from pom.xml under dcs folder, it doesn't affect build in most cases, but I cannot make a build on a fresh machine, so suggest to explicit add this dependency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1812) cleanup schema causes transaction to hang in aborting state
[ https://issues.apache.org/jira/browse/TRAFODION-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1812: --- Assignee: liu ming (was: Roberta Marton) > cleanup schema causes transaction to hang in aborting state > --- > > Key: TRAFODION-1812 > URL: https://issues.apache.org/jira/browse/TRAFODION-1812 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmu >Reporter: Roberta Marton >Assignee: liu ming > > If a cleanup schema command is executed and the schema contains histogram > tables, the operation "succeeds" (but not really). One of the transactions > performing the cleanup commands is left in the aborting state, for example: > (1,0,13)0,12159 281474976710669 ABORTING PT2 > willRollback tmAborted readOnly > Test case: > create schema sch1; > set schema sch1; > create table t1(a int not null primary key); > create table t2 (a int not null primary key, > b int not null unique, > c int not null references t1(a), > d blob, > e char(2) check (e > 'a'), > f largeint generated by default as identity not > null); > create index t2 on t2(b); > insert into t1 values (1), (2), (3), (4), (5), (6), (7), (8); > update statistics for table t1 on every column; > get tables; > cleanup schema sch1; > get tables; > get schemas; > After recovering from the transaction issue (restarting Trafodion), the > following objects still exist in hbase: > TRAFODION.SCH1.SB_HISTOGRAM_INTERVALS > > TRAFODION.SCH1.T1 > > TRAFODION.SCH1.T2 > > TRAFODION.SCH1.T2_651147319_7148 > > TRAFODION.SCH1.T2_945247319_7148 > After removing objects from hbase, restarted sqlci: > get schemas - schema sch1 does not exist. > create schema sch1 - succeeded > create table t1 - succeeded > create table t2 - failed *** ERROR[1390] Object TRAFODION.SEABASE.T2 already > exists in Trafodion. > Table t2 is accessible - so metadata entries were not removed. > Looking a bit into the cleanup code - part of removing an object, tries to > delete rows from the histograms table. So if we are removing a histogram > table, probably should not be deleting from it at the same time. May also > want to wait until all the other tables in the schema are removed before > delete histogram tables (or delete the histogram tables first). > Prototyped a change that: > As part of cleanup schema, cleaned up all the non histogram tables first then > cleaned up the histogram tables. > As part of dropping a histogram table, did not call dropSeabaseStats. > This seemed to fix the problem. > May want to handle failures from delete histogram rows better. Also, if > someone tries to cleanup one histogram table, perhaps all should be dropped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1750) remove build dependency on qpid
[ https://issues.apache.org/jira/browse/TRAFODION-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming resolved TRAFODION-1750. - Resolution: Fixed > remove build dependency on qpid > --- > > Key: TRAFODION-1750 > URL: https://issues.apache.org/jira/browse/TRAFODION-1750 > Project: Apache Trafodion > Issue Type: Improvement >Reporter: liu ming >Assignee: liu ming > > Trafodion build now has a dependency on qpid, but in fact, qpid is not used > inside Trafodion. It is just a legacy code dependency, some source code refer > to qpid but never used. > So we can remove those unused functions/source-code to remove the dependency > for qpid, and simplify the preparation of a development environment. > Now one must install qpid-cpp-client-devel package in special way for CentOS > newer than 6.3, which is too old : > yum --enablerepo=C6.3-updates install qpid-cpp-client-devel > This is one of the effort to make Trafodion development easier, we should > also try to clean up other legacy dependency as much as possible, this will > not only make build easier so as installation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (TRAFODION-1750) remove build dependency on qpid
[ https://issues.apache.org/jira/browse/TRAFODION-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming closed TRAFODION-1750. --- > remove build dependency on qpid > --- > > Key: TRAFODION-1750 > URL: https://issues.apache.org/jira/browse/TRAFODION-1750 > Project: Apache Trafodion > Issue Type: Improvement >Reporter: liu ming >Assignee: liu ming > > Trafodion build now has a dependency on qpid, but in fact, qpid is not used > inside Trafodion. It is just a legacy code dependency, some source code refer > to qpid but never used. > So we can remove those unused functions/source-code to remove the dependency > for qpid, and simplify the preparation of a development environment. > Now one must install qpid-cpp-client-devel package in special way for CentOS > newer than 6.3, which is too old : > yum --enablerepo=C6.3-updates install qpid-cpp-client-devel > This is one of the effort to make Trafodion development easier, we should > also try to clean up other legacy dependency as much as possible, this will > not only make build easier so as installation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1729) change the coprocessor deployment method
[ https://issues.apache.org/jira/browse/TRAFODION-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming resolved TRAFODION-1729. - Resolution: Fixed Fix Version/s: 2.0-incubating > change the coprocessor deployment method > > > Key: TRAFODION-1729 > URL: https://issues.apache.org/jira/browse/TRAFODION-1729 > Project: Apache Trafodion > Issue Type: Improvement > Components: dtm >Reporter: liu ming >Assignee: mashengchen > Fix For: 2.0-incubating > > > I have a proposal to change our current HBase coprocessor configuration > method. > There are three ways to add a coprocessor to a HBase table: > 1. Via editing hbase-site.xml, which will load coprocessor for ALL > tables (Trafodion is using this method now) > 2. Via HBase shell command > 3. Via HTableDescriptor.addCoprocessor() java API > Trafodion now is using the first method. I proposed to use method 3, I > finished a prototype and test seems works well. > > Here are the reasons I propose for this change: > At present, the Trafodion installer needs to modify the hbase-site.xml and > then restart HBase instance for the configuration to take effect. This step > not only complicate the installer but also let user think Trafodion is > intrusive into underlying HBase system. It will be ideal if we can avoid this > step. Another problem: in CDH, there is a concept called ‘region server > group’ or something, so the settings will have to carefully handled by > installer to apply to all groups. As we saw recently in WebRoot deployment, > Trafodion failed due to this reason. All these are very error prone and > complicate the Trafodion installer. Once CDH or HDP changed something, > Trafodion may fail again. > > So I spent time to investigate why we need to restart HBase in order to > install Trafodion. > As I understand, there are 3 major reasons > 1. To add hbase-trx coprocessors > 2. To overload HRegion with TransactionalRegion > 3. Various configuration settings, need to check one by one. > The first configuration can be avoided by applying my proposed change. > The second one, I look through the TransactionalRegion.java, and find out the > only reason (now) is to overload the getScanner() method to be public so can > be invoked by the coprocessor. And there are only 1 or 2 places that API is > invoked in Trafodion code. I checked with Kevin and he proposed by using > ‘java reflection’ we can also avoid this. > All other configuration items to some extent look like ‘best to have’, but > not ‘must to have’. And I also find two config items seems never been used: > hbase.bulkload.staging.dir /hbase-staging (Suresh can confirm, > but I search in all code, seems this is never used) > hbase.regionserver.region.transactional.tlog true (Narendra can > confirm, this is NEVER used, maybe a legacy config item?) > Yes, by now, there are still some other config items seems cannot be avoided, > but I hope we can find some way to remove them in the future. I am not trying > to solve all issues right now, just want to start the effort to remove > unnecessary hbase reconfiguration. > For this example, Coprocessors can be added to a table at run time, no need > to edit the hbase-site.xml and restart hbase. This is only the first step to > try to remove the deep impact to the current HBase config and restart HBase. > > So I asked for your opinions about this change. If you think this is > necessary, I will continue to file a JIRA and fix it. > > I strongly recommend to get rid of the step of ‘modify hbase-site.xml and > restart your hbase’ for Trafodion installation, it should be an option , to > tune the system to best suit Trafodion, but should not be a forced step. To > be note: Apache Phoenix is also a SQL on HBase, its installation will change > nothing of underlying HBase, very lightweight, no ‘intrude into’ the existing > HBase system. Trafodion is considered to be heavy and intrusive in this > manner, and I feel maybe we can change this. > > Should I start this discussion in the dev mail list? > > P.S. a list of changed config items. My proposal will remove the last one, > hope we can get rid of all of them: > hbase.master.distributed.log.splitting false > hbase.snapshot.master.timeoutMillis 60 > hbase_regionserver_lease_period 60 > hbase.hregion.impl > org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion > hbase.regionserver.region.split.policy > org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy > hbase.snapshot.enabled true > hbase.bulkload.staging.dir/hbase-staging > hbase.regionserver.region.transactional.tlog true > hbase.snapsh
[jira] [Closed] (TRAFODION-1813) add commons-httpclient depenedency into dcs/pom.xml
[ https://issues.apache.org/jira/browse/TRAFODION-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming closed TRAFODION-1813. --- Resolution: Not A Problem not an issue. Close. > add commons-httpclient depenedency into dcs/pom.xml > --- > > Key: TRAFODION-1813 > URL: https://issues.apache.org/jira/browse/TRAFODION-1813 > Project: Apache Trafodion > Issue Type: Bug >Reporter: liu ming >Assignee: liu ming >Priority: Minor > > Not sure why the dependency of httpclient is missing from pom.xml under dcs > folder, it doesn't affect build in most cases, but I cannot make a build on a > fresh machine, so suggest to explicit add this dependency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1834) ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in sqconfig are fully qualified
[ https://issues.apache.org/jira/browse/TRAFODION-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1834: --- Assignee: liu ming > ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in > sqconfig are fully qualified > -- > > Key: TRAFODION-1834 > URL: https://issues.apache.org/jira/browse/TRAFODION-1834 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmp >Affects Versions: 1.3-incubating >Reporter: Atanu Mishra >Assignee: liu ming > > Paraphrasing Hans, who looked into this briefly: > Looking at the code, I can see that we remove anything from the first dot in > the node name of the region. Then we search for the unqualified node name in > the list of node names of the Trafodion cluster. > Both of these clusters use FQDNs in their configuration, though, therefore we > won't find the node names. > To Reproduce: > Do an EXPLAIN of a parallel query with and without TRAF_ALLOW_ESP_COLOCATION > set, and the node map will not show a co-located plan when the CQD is on - if > the system uses FQDNs in its sqconfig file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1834) ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in sqconfig are fully qualified
[ https://issues.apache.org/jira/browse/TRAFODION-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153850#comment-15153850 ] liu ming commented on TRAFODION-1834: - The issue is slightly different. The FQDN is not a problem. The binder will build the initial node_map for a given table, at that time, the API invoked is called createNodeMapForHbase(), in that function, it will truncate the FQDN into hostname. And the NAClusterInfo class which collecting the hostname to nodeId mapping will also do the truncation. So it is a match. But there is a new problem that this jira will fix: In a system, there is a cluster with 10 nodes, HBase RS installed on 8 of them. Table bltest have 100M rows, 250G in one-replica size, split into 100 regions, evenly spread over 8 nodes. do a very simple test: select [last 1]* from bltest; Without this CQD set, it launched 10 ESPs over 10 nodes, and the ESP node_map printed as: esp_2_node_map . (\NSK:-1:-1:-1:-1:-1:-1:-1:-1:-1:-1) which means randomly locate each ESP With this CQD, each time, the node_map is different, such as: esp_2_node_map . (\NSK:0:0:0:0:0:0:0:0:0:0) esp_2_node_map . (\NSK:7:7:7:7:7:7:7:7:7:7) you can notice, the 10 ESP will be put into same node when CQD is ‘on’, and the node number is random. This bug is in optimizer’s NodeMap::getPopularNodeNumber() function, it tries to find out a most popular node. In above case, 10 ESPs try to read 100 regions, so for the first ESP, it needs to read 10 regions, 0~9, but region 0 and region 1 may be in different RS node, so this function is trying to find a RS which serves the most regions from Region 0 to Region 9, and locate first ESP there. But this function use an uninitialized array to do the job. It malloc a new buffer and use as counter array, but GCC is not always clear the newly alloc buffer, so you never know what init value in that array, so if node[0] init with a very big number, it will always win, and we saw: esp_2_node_map . (\NSK:0:0:0:0:0:0:0:0:0:0) a fix is to init the array. And in this change, we remove the bias in getPopularNodeNumber() to lowest node ID, but make it random. > ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in > sqconfig are fully qualified > -- > > Key: TRAFODION-1834 > URL: https://issues.apache.org/jira/browse/TRAFODION-1834 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmp >Affects Versions: 1.3-incubating >Reporter: Atanu Mishra >Assignee: liu ming > > Paraphrasing Hans, who looked into this briefly: > Looking at the code, I can see that we remove anything from the first dot in > the node name of the region. Then we search for the unqualified node name in > the list of node names of the Trafodion cluster. > Both of these clusters use FQDNs in their configuration, though, therefore we > won't find the node names. > To Reproduce: > Do an EXPLAIN of a parallel query with and without TRAF_ALLOW_ESP_COLOCATION > set, and the node map will not show a co-located plan when the CQD is on - if > the system uses FQDNs in its sqconfig file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1841) DROP TABLE just after CREATE TABLE fails
[ https://issues.apache.org/jira/browse/TRAFODION-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15155879#comment-15155879 ] liu ming commented on TRAFODION-1841: - seems not reproducible, I tested on my two dev system with latest code, not see same issue. Recommend to build from scratch and run install_local_hadoop from scratch. > DROP TABLE just after CREATE TABLE fails > > > Key: TRAFODION-1841 > URL: https://issues.apache.org/jira/browse/TRAFODION-1841 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmp >Affects Versions: 2.0-incubating > Environment: This happened on a workstation. Don't know if it happens > everywhere. >Reporter: David Wayne Birdsall > > If one creates a table in sqlci and then drops it, the drop fails. An example: > [birdsall@edev05 Traf1838]$ sqlci > Apache Trafodion Conversational Interface 2.0.0 > Copyright (c) 2015 Apache Software Foundation > >>create table t2 (a int); > --- SQL operation complete. > >>drop table t2; > *** ERROR[8448] Unable to access Hbase interface. Call to > ExpHbaseInterface::deleteRow returned error HBASE_ACCESS_ERROR(-706). Cause: > java.io.IOException: Coprocessor result is null, retries exhausted > org.apache.hadoop.hbase.client.transactional.TransactionalTable.delete(TransactionalTable.java:300) > org.apache.hadoop.hbase.client.transactional.RMInterface.delete(RMInterface.java:325) > org.trafodion.sql.HTableClient.deleteRow(HTableClient.java:1377) > org.trafodion.sql.HBaseClient.deleteRow(HBaseClient.java:1680) > . > *** ERROR[8839] Transaction was aborted. > (The failure then repeats several times, possibly because of a retry loop in > the code.) > I am wondering if this is related at all to JIRA TRAFODION-1729. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1858) RI predicate generating string should not contain _SALT_ column
liu ming created TRAFODION-1858:
---
Summary: RI predicate generating string should not contain _SALT_
column
Key: TRAFODION-1858
URL: https://issues.apache.org/jira/browse/TRAFODION-1858
Project: Apache Trafodion
Issue Type: Bug
Reporter: liu ming
When table created with SALT, it cannot be used in a foreign key reference.
To reproduce:
>>CREATE TABLE a ( id int not null, PRIMARY KEY (id))SALT USING 9 PARTITIONS;
--- SQL operation complete.
>>CREATE TABLE b ( id int not null, val int,
+> PRIMARY KEY (id),
+> CONSTRAINT FK FOREIGN KEY (val) REFERENCES a (id))
+>SALT USING 9 PARTITIONS;
--- SQL operation complete.
>>INSERT INTO a values(1);
--- 1 row(s) inserted.
>>INSERT INTO b values(1,1);
*** ERROR[15001] A syntax error occurred at or before:
("NEW@".VAL)=(TRAFODION.SEABASE.A."_SALT_",TRAFODION.SEABASE.A.ID);
^ (43 characters from start of SQL
statement)
*** ERROR[8822] The statement was not prepared.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1858) RI predicate generating string should not contain _SALT_ column
[
https://issues.apache.org/jira/browse/TRAFODION-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liu ming reassigned TRAFODION-1858:
---
Assignee: liu ming
> RI predicate generating string should not contain _SALT_ column
> ---
>
> Key: TRAFODION-1858
> URL: https://issues.apache.org/jira/browse/TRAFODION-1858
> Project: Apache Trafodion
> Issue Type: Bug
>Reporter: liu ming
>Assignee: liu ming
>
> When table created with SALT, it cannot be used in a foreign key reference.
> To reproduce:
> >>CREATE TABLE a ( id int not null, PRIMARY KEY (id))SALT USING 9 PARTITIONS;
> --- SQL operation complete.
> >>CREATE TABLE b ( id int not null, val int,
> +> PRIMARY KEY (id),
> +> CONSTRAINT FK FOREIGN KEY (val) REFERENCES a (id))
> +>SALT USING 9 PARTITIONS;
> --- SQL operation complete.
> >>INSERT INTO a values(1);
> --- 1 row(s) inserted.
> >>INSERT INTO b values(1,1);
> *** ERROR[15001] A syntax error occurred at or before:
> ("NEW@".VAL)=(TRAFODION.SEABASE.A."_SALT_",TRAFODION.SEABASE.A.ID);
> ^ (43 characters from start of SQL
> statement)
> *** ERROR[8822] The statement was not prepared.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TRAFODION-1858) RI predicate generating string should not contain _SALT_ column
[
https://issues.apache.org/jira/browse/TRAFODION-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15170648#comment-15170648
]
liu ming commented on TRAFODION-1858:
-
in core/sql/optimizer/BindRI.cpp
RefConstraint::getPredicateText()
one should check the column name, if it is "_SALT_", ignore it.
I assume there are other 'hidden' column as well. Need to get a list of them.
> RI predicate generating string should not contain _SALT_ column
> ---
>
> Key: TRAFODION-1858
> URL: https://issues.apache.org/jira/browse/TRAFODION-1858
> Project: Apache Trafodion
> Issue Type: Bug
>Reporter: liu ming
>Assignee: liu ming
>
> When table created with SALT, it cannot be used in a foreign key reference.
> To reproduce:
> >>CREATE TABLE a ( id int not null, PRIMARY KEY (id))SALT USING 9 PARTITIONS;
> --- SQL operation complete.
> >>CREATE TABLE b ( id int not null, val int,
> +> PRIMARY KEY (id),
> +> CONSTRAINT FK FOREIGN KEY (val) REFERENCES a (id))
> +>SALT USING 9 PARTITIONS;
> --- SQL operation complete.
> >>INSERT INTO a values(1);
> --- 1 row(s) inserted.
> >>INSERT INTO b values(1,1);
> *** ERROR[15001] A syntax error occurred at or before:
> ("NEW@".VAL)=(TRAFODION.SEABASE.A."_SALT_",TRAFODION.SEABASE.A.ID);
> ^ (43 characters from start of SQL
> statement)
> *** ERROR[8822] The statement was not prepared.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Work started] (TRAFODION-1858) RI predicate generating string should not contain _SALT_ column
[
https://issues.apache.org/jira/browse/TRAFODION-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on TRAFODION-1858 started by liu ming.
---
> RI predicate generating string should not contain _SALT_ column
> ---
>
> Key: TRAFODION-1858
> URL: https://issues.apache.org/jira/browse/TRAFODION-1858
> Project: Apache Trafodion
> Issue Type: Bug
>Reporter: liu ming
>Assignee: liu ming
>
> When table created with SALT, it cannot be used in a foreign key reference.
> To reproduce:
> >>CREATE TABLE a ( id int not null, PRIMARY KEY (id))SALT USING 9 PARTITIONS;
> --- SQL operation complete.
> >>CREATE TABLE b ( id int not null, val int,
> +> PRIMARY KEY (id),
> +> CONSTRAINT FK FOREIGN KEY (val) REFERENCES a (id))
> +>SALT USING 9 PARTITIONS;
> --- SQL operation complete.
> >>INSERT INTO a values(1);
> --- 1 row(s) inserted.
> >>INSERT INTO b values(1,1);
> *** ERROR[15001] A syntax error occurred at or before:
> ("NEW@".VAL)=(TRAFODION.SEABASE.A."_SALT_",TRAFODION.SEABASE.A.ID);
> ^ (43 characters from start of SQL
> statement)
> *** ERROR[8822] The statement was not prepared.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (TRAFODION-1861) support Trafodion running on CentOS 7
liu ming created TRAFODION-1861: --- Summary: support Trafodion running on CentOS 7 Key: TRAFODION-1861 URL: https://issues.apache.org/jira/browse/TRAFODION-1861 Project: Apache Trafodion Issue Type: Umbrella Reporter: liu ming -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1865) uninitialized buffer cause Monitor abort in CentOS 7
[
https://issues.apache.org/jira/browse/TRAFODION-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liu ming reassigned TRAFODION-1865:
---
Assignee: liu ming
> uninitialized buffer cause Monitor abort in CentOS 7
>
>
> Key: TRAFODION-1865
> URL: https://issues.apache.org/jira/browse/TRAFODION-1865
> Project: Apache Trafodion
> Issue Type: Bug
>Reporter: liu ming
>Assignee: liu ming
>
> sqstart failed on CentOS.
> The Monitor process abort in CProcessContainer::CProcessContainer() due to
> failed to create semaphore.
> //create & initialize existing semaphore
> char sem_name[MAX_PROCESS_PATH];
> snprintf(sem_name,sizeof(sem_name), "/monitor.sem.%s", getenv("USER"));
> Mutex = sem_open(sem_name,O_CREAT,0644,0);
> if(Mutex == SEM_FAILED)
> {
> char buf[MON_STRING_BUF_SIZE];
> snprintf(buf, sizeof(buf), "[%s], Can't create semaphore %s!\n",
> method_name, sem_name);
> mon_log_write(MON_PROCESSCONT_PROCESSCONT_3, SQ_LOG_ERR, buf);
> sem_unlink(sem_name);
> abort();
> }
> sem_name is not initialized and snprintf will not add \0 after copy required
> bytes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (TRAFODION-1865) uninitialized buffer cause Monitor abort in CentOS 7
liu ming created TRAFODION-1865:
---
Summary: uninitialized buffer cause Monitor abort in CentOS 7
Key: TRAFODION-1865
URL: https://issues.apache.org/jira/browse/TRAFODION-1865
Project: Apache Trafodion
Issue Type: Bug
Reporter: liu ming
sqstart failed on CentOS.
The Monitor process abort in CProcessContainer::CProcessContainer() due to
failed to create semaphore.
//create & initialize existing semaphore
char sem_name[MAX_PROCESS_PATH];
snprintf(sem_name,sizeof(sem_name), "/monitor.sem.%s", getenv("USER"));
Mutex = sem_open(sem_name,O_CREAT,0644,0);
if(Mutex == SEM_FAILED)
{
char buf[MON_STRING_BUF_SIZE];
snprintf(buf, sizeof(buf), "[%s], Can't create semaphore %s!\n",
method_name, sem_name);
mon_log_write(MON_PROCESSCONT_PROCESSCONT_3, SQ_LOG_ERR, buf);
sem_unlink(sem_name);
abort();
}
sem_name is not initialized and snprintf will not add \0 after copy required
bytes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Work started] (TRAFODION-1865) uninitialized buffer cause Monitor abort in CentOS 7
[
https://issues.apache.org/jira/browse/TRAFODION-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on TRAFODION-1865 started by liu ming.
---
> uninitialized buffer cause Monitor abort in CentOS 7
>
>
> Key: TRAFODION-1865
> URL: https://issues.apache.org/jira/browse/TRAFODION-1865
> Project: Apache Trafodion
> Issue Type: Bug
>Reporter: liu ming
>Assignee: liu ming
>
> sqstart failed on CentOS.
> The Monitor process abort in CProcessContainer::CProcessContainer() due to
> failed to create semaphore.
> //create & initialize existing semaphore
> char sem_name[MAX_PROCESS_PATH];
> snprintf(sem_name,sizeof(sem_name), "/monitor.sem.%s", getenv("USER"));
> Mutex = sem_open(sem_name,O_CREAT,0644,0);
> if(Mutex == SEM_FAILED)
> {
> char buf[MON_STRING_BUF_SIZE];
> snprintf(buf, sizeof(buf), "[%s], Can't create semaphore %s!\n",
> method_name, sem_name);
> mon_log_write(MON_PROCESSCONT_PROCESSCONT_3, SQ_LOG_ERR, buf);
> sem_unlink(sem_name);
> abort();
> }
> sem_name is not initialized and snprintf will not add \0 after copy required
> bytes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (TRAFODION-1865) uninitialized buffer cause Monitor abort in CentOS 7
[
https://issues.apache.org/jira/browse/TRAFODION-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liu ming updated TRAFODION-1865:
Component/s: foundation
> uninitialized buffer cause Monitor abort in CentOS 7
>
>
> Key: TRAFODION-1865
> URL: https://issues.apache.org/jira/browse/TRAFODION-1865
> Project: Apache Trafodion
> Issue Type: Bug
> Components: foundation
>Reporter: liu ming
>Assignee: liu ming
>
> sqstart failed on CentOS.
> The Monitor process abort in CProcessContainer::CProcessContainer() due to
> failed to create semaphore.
> //create & initialize existing semaphore
> char sem_name[MAX_PROCESS_PATH];
> snprintf(sem_name,sizeof(sem_name), "/monitor.sem.%s", getenv("USER"));
> Mutex = sem_open(sem_name,O_CREAT,0644,0);
> if(Mutex == SEM_FAILED)
> {
> char buf[MON_STRING_BUF_SIZE];
> snprintf(buf, sizeof(buf), "[%s], Can't create semaphore %s!\n",
> method_name, sem_name);
> mon_log_write(MON_PROCESSCONT_PROCESSCONT_3, SQ_LOG_ERR, buf);
> sem_unlink(sem_name);
> abort();
> }
> sem_name is not initialized and snprintf will not add \0 after copy required
> bytes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (TRAFODION-1869) 'initialized trafodion' failed on CentOS 7.2
liu ming created TRAFODION-1869: --- Summary: 'initialized trafodion' failed on CentOS 7.2 Key: TRAFODION-1869 URL: https://issues.apache.org/jira/browse/TRAFODION-1869 Project: Apache Trafodion Issue Type: Bug Reporter: liu ming On CentOS 7.2, install Trafodion then run 'initialize trafodion' >>initialize trafodion; *** ERROR[2034] $Z000HQ0:48: Operating system error 201 while communicating with server process $Z000I2H:52. *** ERROR[2013] Server process tdm_arkcmp could not be created on \\NSK - Operating system error 201. *** ERROR[2002] Internal error: cannot create compiler. *** ERROR[2005] Internal error: from compilation, no errors in diagnostics yet for statement: control query shape hold; *** ERROR[8822] The statement was not prepared. --- SQL operation failed with errors. related core is from tdm_arkcmp Core was generated by `tdm_arkcmp SQMON1.1 0 0 022137 $Z000I2H 192.168.0.10:56579 4 0'. Program terminated with signal 11, Segmentation fault. #0 0x in ?? () Missing separate debuginfos, use: debuginfo-install trafodion-2.0.1-1.x86_64 trafodion-2.0.1-devel.x86_64 (gdb) bt #0 0x in ?? () #1 0x7f44a8552fc6 in SB_Thread::Sthr::time () at /home/centos/traf-plus/core/sqf/export/include/seabed/int/thread.inl:115 #2 0x7f44adebeef4 in fs_int_fs_file_awaitiox (pp_filenum=0x1603034, ppp_buf=0x7ffe5d71b208, pp_xfercount=0x7ffe5d71b1f4, pp_tag=0x7ffe5d71b1f8, pv_timeout=6, pp_segid=0x7ffe5d71b206, pv_int=false, pv_ts=false) at fsi.cpp:1301 #3 0x7f44adeb8a80 in BAWAITIOX (pp_filenum=0x1603034, ppp_buf=0x7ffe5d71b5f8, pp_xfercount=0x7ffe5d71b60c, pp_tag=0x7ffe5d71b5f0, pv_timeout=6, pp_segid=0x0) at fs.cpp:563 #4 0x7f44b5d629e7 in GuaReceiveControlConnection::wait (this=0x1603020, timeout=6, eventConsumed=, ipcAwaitiox=0x0) at ../common/IpcGuardian.cpp:2692 #5 0x7f44b5d643d8 in GuaReceiveControlConnection::waitForMaster (this=0x1603020) at ../common/IpcGuardian.cpp:3603 #6 0x00406366 in main () (gdb) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1860) sqgen failed to execute on CentOS7 system.
[ https://issues.apache.org/jira/browse/TRAFODION-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1860: Assignee: Eason Zhang > sqgen failed to execute on CentOS7 system. > -- > > Key: TRAFODION-1860 > URL: https://issues.apache.org/jira/browse/TRAFODION-1860 > Project: Apache Trafodion > Issue Type: Bug > Components: foundation >Reporter: Eason Zhang >Assignee: Eason Zhang > > missing ctime.pl file on CentOS7. > [trafodion@vmcentos7 ~]$ sqgen > Workstation environment - Not a clustered environment > Can't locate ctime.pl in @INC (@INC contains: > /home/trafodion/trafodion-2.0.1/export/lib /usr/local/lib64/perl5 > /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl > /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at > ./gensq.pl line 24. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1865) uninitialized buffer cause Monitor abort in CentOS 7
[
https://issues.apache.org/jira/browse/TRAFODION-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15179225#comment-15179225
]
liu ming commented on TRAFODION-1865:
-
the problem is not caused by 'uninitialized buffer, the title is not proper.
Will continue to work on this.
> uninitialized buffer cause Monitor abort in CentOS 7
>
>
> Key: TRAFODION-1865
> URL: https://issues.apache.org/jira/browse/TRAFODION-1865
> Project: Apache Trafodion
> Issue Type: Bug
> Components: foundation
>Reporter: liu ming
>Assignee: liu ming
>
> sqstart failed on CentOS.
> The Monitor process abort in CProcessContainer::CProcessContainer() due to
> failed to create semaphore.
> //create & initialize existing semaphore
> char sem_name[MAX_PROCESS_PATH];
> snprintf(sem_name,sizeof(sem_name), "/monitor.sem.%s", getenv("USER"));
> Mutex = sem_open(sem_name,O_CREAT,0644,0);
> if(Mutex == SEM_FAILED)
> {
> char buf[MON_STRING_BUF_SIZE];
> snprintf(buf, sizeof(buf), "[%s], Can't create semaphore %s!\n",
> method_name, sem_name);
> mon_log_write(MON_PROCESSCONT_PROCESSCONT_3, SQ_LOG_ERR, buf);
> sem_unlink(sem_name);
> abort();
> }
> sem_name is not initialized and snprintf will not add \0 after copy required
> bytes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (TRAFODION-1874) find a new way to check CLUSTER environment instead of checking pdsh
liu ming created TRAFODION-1874: --- Summary: find a new way to check CLUSTER environment instead of checking pdsh Key: TRAFODION-1874 URL: https://issues.apache.org/jira/browse/TRAFODION-1874 Project: Apache Trafodion Issue Type: Improvement Reporter: liu ming Some Trafodion script need to tell if it is running in a dev workstation or a real cluster, and behave differently. Current, some script tell this difference by checking the rpm package pdsh, if it is installed, then the script think it is running on a cluster. PDSH is a public package, and can be installed in a workstation as well. So it is better to find a new testing logic for this purpose. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1854) Trafodion cannot start on nodes with uppercase hostname.
[ https://issues.apache.org/jira/browse/TRAFODION-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1854: --- Assignee: liu ming > Trafodion cannot start on nodes with uppercase hostname. > > > Key: TRAFODION-1854 > URL: https://issues.apache.org/jira/browse/TRAFODION-1854 > Project: Apache Trafodion > Issue Type: Bug > Components: foundation >Reporter: Eason Zhang >Assignee: liu ming > > set the node's hostname to uppercase, in this case it is set to 'H1 H2 H3' > Trafodion’s .bashrc is setting correctly: > > # These env vars define all nodes in the cluster > export NODE_LIST=" H1 H2 H3" > export MY_NODES=" -w H1 -w H2 -w H3" > > /etc/hosts are also set as ‘H1 H2 H3’. > sqconfig is also set as uppercase hostname. > when starting trafodion instance, it will report below error in sqmon.log: > Processing cluster.conf on local host H1 > [SHELL] Shell/shell Version 1.0.1 EsgynDB_Enterprise Release 2.0.0 (Build > release [EsgynDB-2.0.0-0-g2ba9dde_Bld402], date 20151121_0002) > > [SHELL] % > ! Start the monitor processes across the cluster > startup > [SHELL] %startup > [SHELL] Cannot start monitor from node 'H1' since it is not member of the > cluster configuration or 'hostname' string does not match configuration > string. > [SHELL] Configuration node names: > [SHELL]'h1' > [SHELL]'h2' > [SHELL]'h3' > [SHELL] Failed to start environment! > > [SHELL] % > exit -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1854) Trafodion cannot start on nodes with uppercase hostname.
[ https://issues.apache.org/jira/browse/TRAFODION-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15182198#comment-15182198 ] liu ming commented on TRAFODION-1854: - Monitor's CClusterConfig class treat all character as lower case when reading the cluster configuration file. Monitor will not use hostname in normal operation, so convert the hostname into lowercase in shell.cxx will solve the issue. > Trafodion cannot start on nodes with uppercase hostname. > > > Key: TRAFODION-1854 > URL: https://issues.apache.org/jira/browse/TRAFODION-1854 > Project: Apache Trafodion > Issue Type: Bug > Components: foundation >Reporter: Eason Zhang >Assignee: liu ming > > set the node's hostname to uppercase, in this case it is set to 'H1 H2 H3' > Trafodion’s .bashrc is setting correctly: > > # These env vars define all nodes in the cluster > export NODE_LIST=" H1 H2 H3" > export MY_NODES=" -w H1 -w H2 -w H3" > > /etc/hosts are also set as ‘H1 H2 H3’. > sqconfig is also set as uppercase hostname. > when starting trafodion instance, it will report below error in sqmon.log: > Processing cluster.conf on local host H1 > [SHELL] Shell/shell Version 1.0.1 EsgynDB_Enterprise Release 2.0.0 (Build > release [EsgynDB-2.0.0-0-g2ba9dde_Bld402], date 20151121_0002) > > [SHELL] % > ! Start the monitor processes across the cluster > startup > [SHELL] %startup > [SHELL] Cannot start monitor from node 'H1' since it is not member of the > cluster configuration or 'hostname' string does not match configuration > string. > [SHELL] Configuration node names: > [SHELL]'h1' > [SHELL]'h2' > [SHELL]'h3' > [SHELL] Failed to start environment! > > [SHELL] % > exit -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (TRAFODION-1854) Trafodion cannot start on nodes with uppercase hostname.
[ https://issues.apache.org/jira/browse/TRAFODION-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TRAFODION-1854 started by liu ming. --- > Trafodion cannot start on nodes with uppercase hostname. > > > Key: TRAFODION-1854 > URL: https://issues.apache.org/jira/browse/TRAFODION-1854 > Project: Apache Trafodion > Issue Type: Bug > Components: foundation >Reporter: Eason Zhang >Assignee: liu ming > > set the node's hostname to uppercase, in this case it is set to 'H1 H2 H3' > Trafodion’s .bashrc is setting correctly: > > # These env vars define all nodes in the cluster > export NODE_LIST=" H1 H2 H3" > export MY_NODES=" -w H1 -w H2 -w H3" > > /etc/hosts are also set as ‘H1 H2 H3’. > sqconfig is also set as uppercase hostname. > when starting trafodion instance, it will report below error in sqmon.log: > Processing cluster.conf on local host H1 > [SHELL] Shell/shell Version 1.0.1 EsgynDB_Enterprise Release 2.0.0 (Build > release [EsgynDB-2.0.0-0-g2ba9dde_Bld402], date 20151121_0002) > > [SHELL] % > ! Start the monitor processes across the cluster > startup > [SHELL] %startup > [SHELL] Cannot start monitor from node 'H1' since it is not member of the > cluster configuration or 'hostname' string does not match configuration > string. > [SHELL] Configuration node names: > [SHELL]'h1' > [SHELL]'h2' > [SHELL]'h3' > [SHELL] Failed to start environment! > > [SHELL] % > exit -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1869) 'initialized trafodion' failed on CentOS 7.2
[ https://issues.apache.org/jira/browse/TRAFODION-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1869: --- Assignee: liu ming > 'initialized trafodion' failed on CentOS 7.2 > > > Key: TRAFODION-1869 > URL: https://issues.apache.org/jira/browse/TRAFODION-1869 > Project: Apache Trafodion > Issue Type: Bug >Reporter: liu ming >Assignee: liu ming > > On CentOS 7.2, install Trafodion then run 'initialize trafodion' > >>initialize trafodion; > *** ERROR[2034] $Z000HQ0:48: Operating system error 201 while communicating > with server process $Z000I2H:52. > *** ERROR[2013] Server process tdm_arkcmp could not be created on \\NSK - > Operating system error 201. > *** ERROR[2002] Internal error: cannot create compiler. > *** ERROR[2005] Internal error: from compilation, no errors in diagnostics > yet for statement: control query shape hold; > *** ERROR[8822] The statement was not prepared. > --- SQL operation failed with errors. > related core is from tdm_arkcmp > Core was generated by `tdm_arkcmp SQMON1.1 0 0 022137 $Z000I2H > 192.168.0.10:56579 4 0'. > Program terminated with signal 11, Segmentation fault. > #0 0x in ?? () > Missing separate debuginfos, use: debuginfo-install trafodion-2.0.1-1.x86_64 > trafodion-2.0.1-devel.x86_64 > (gdb) bt > #0 0x in ?? () > #1 0x7f44a8552fc6 in SB_Thread::Sthr::time () at > /home/centos/traf-plus/core/sqf/export/include/seabed/int/thread.inl:115 > #2 0x7f44adebeef4 in fs_int_fs_file_awaitiox (pp_filenum=0x1603034, > ppp_buf=0x7ffe5d71b208, pp_xfercount=0x7ffe5d71b1f4, > pp_tag=0x7ffe5d71b1f8, pv_timeout=6, pp_segid=0x7ffe5d71b206, > pv_int=false, pv_ts=false) at fsi.cpp:1301 > #3 0x7f44adeb8a80 in BAWAITIOX (pp_filenum=0x1603034, > ppp_buf=0x7ffe5d71b5f8, pp_xfercount=0x7ffe5d71b60c, pp_tag=0x7ffe5d71b5f0, > pv_timeout=6, pp_segid=0x0) at fs.cpp:563 > #4 0x7f44b5d629e7 in GuaReceiveControlConnection::wait (this=0x1603020, > timeout=6, eventConsumed=, ipcAwaitiox=0x0) > at ../common/IpcGuardian.cpp:2692 > #5 0x7f44b5d643d8 in GuaReceiveControlConnection::waitForMaster > (this=0x1603020) at ../common/IpcGuardian.cpp:3603 > #6 0x00406366 in main () > (gdb) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (TRAFODION-1869) 'initialized trafodion' failed on CentOS 7.2
[ https://issues.apache.org/jira/browse/TRAFODION-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TRAFODION-1869 started by liu ming. --- > 'initialized trafodion' failed on CentOS 7.2 > > > Key: TRAFODION-1869 > URL: https://issues.apache.org/jira/browse/TRAFODION-1869 > Project: Apache Trafodion > Issue Type: Bug >Reporter: liu ming >Assignee: liu ming > > On CentOS 7.2, install Trafodion then run 'initialize trafodion' > >>initialize trafodion; > *** ERROR[2034] $Z000HQ0:48: Operating system error 201 while communicating > with server process $Z000I2H:52. > *** ERROR[2013] Server process tdm_arkcmp could not be created on \\NSK - > Operating system error 201. > *** ERROR[2002] Internal error: cannot create compiler. > *** ERROR[2005] Internal error: from compilation, no errors in diagnostics > yet for statement: control query shape hold; > *** ERROR[8822] The statement was not prepared. > --- SQL operation failed with errors. > related core is from tdm_arkcmp > Core was generated by `tdm_arkcmp SQMON1.1 0 0 022137 $Z000I2H > 192.168.0.10:56579 4 0'. > Program terminated with signal 11, Segmentation fault. > #0 0x in ?? () > Missing separate debuginfos, use: debuginfo-install trafodion-2.0.1-1.x86_64 > trafodion-2.0.1-devel.x86_64 > (gdb) bt > #0 0x in ?? () > #1 0x7f44a8552fc6 in SB_Thread::Sthr::time () at > /home/centos/traf-plus/core/sqf/export/include/seabed/int/thread.inl:115 > #2 0x7f44adebeef4 in fs_int_fs_file_awaitiox (pp_filenum=0x1603034, > ppp_buf=0x7ffe5d71b208, pp_xfercount=0x7ffe5d71b1f4, > pp_tag=0x7ffe5d71b1f8, pv_timeout=6, pp_segid=0x7ffe5d71b206, > pv_int=false, pv_ts=false) at fsi.cpp:1301 > #3 0x7f44adeb8a80 in BAWAITIOX (pp_filenum=0x1603034, > ppp_buf=0x7ffe5d71b5f8, pp_xfercount=0x7ffe5d71b60c, pp_tag=0x7ffe5d71b5f0, > pv_timeout=6, pp_segid=0x0) at fs.cpp:563 > #4 0x7f44b5d629e7 in GuaReceiveControlConnection::wait (this=0x1603020, > timeout=6, eventConsumed=, ipcAwaitiox=0x0) > at ../common/IpcGuardian.cpp:2692 > #5 0x7f44b5d643d8 in GuaReceiveControlConnection::waitForMaster > (this=0x1603020) at ../common/IpcGuardian.cpp:3603 > #6 0x00406366 in main () > (gdb) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1868) Compatibility with gcc 4.8
[
https://issues.apache.org/jira/browse/TRAFODION-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15182385#comment-15182385
]
liu ming commented on TRAFODION-1868:
-
there are multiple issues to build using gcc 4.8.
1. Werror
gcc 4.8 is more strict than gcc 4.4. There are several programming issues
that not reported as warning by 4.4
- declare and init a var , but never use it anymore
- assign 0 to a pointer var
- etc.
2. syntax checking
gcc 4.8 will report error for some code like
for ( int idx = 0; idx < 100; idx++)
{
ANewClass idx = new ANewClass();
...
need to fix these issues.
3. boost conflict with c++11
4. must use -Xlinker --copy-dt-needed-entries, otherwise linker will not find
all required so automatically.
These are some issues I met, issues related to item 1 are too many. I temply
remove -Werror to overcome it. and make a build. But we need to fix all issues
related to item 1.
There are so many changes to many src code. So I suggest to do this
increamentally. It is easier to test, and easier for others to do code review.
> Compatibility with gcc 4.8
> --
>
> Key: TRAFODION-1868
> URL: https://issues.apache.org/jira/browse/TRAFODION-1868
> Project: Apache Trafodion
> Issue Type: Bug
>Reporter: Steve Varnau
>Priority: Minor
>
> Current code will not compile with gcc 4.8. That is the default version on
> RH/CentOS 7. This is needed to move to a development environment on CentOS 7.
> There is a 4.4 version of gcc available on 7. The RPM package is
> compat-gcc-44, I have not tried building on 7.x with older compiler, but will
> likely work.
> This is not urgent at this time, but we should move toward enabling a newer
> dev environment. But this needs to be coordinated with with versions of OS
> are supported for runtime use of trafodion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (TRAFODION-1861) support Trafodion running on CentOS 7
[ https://issues.apache.org/jira/browse/TRAFODION-1861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1861: Assignee: Eason Zhang > support Trafodion running on CentOS 7 > - > > Key: TRAFODION-1861 > URL: https://issues.apache.org/jira/browse/TRAFODION-1861 > Project: Apache Trafodion > Issue Type: Umbrella >Reporter: liu ming >Assignee: Eason Zhang > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1881) Follow up TRAFODION-1858 , need a better fix
liu ming created TRAFODION-1881: --- Summary: Follow up TRAFODION-1858 , need a better fix Key: TRAFODION-1881 URL: https://issues.apache.org/jira/browse/TRAFODION-1881 Project: Apache Trafodion Issue Type: Bug Reporter: liu ming TRAFODION-1858 is now fixed using a temp method. A better and complete solution is required. But to solve it entirely, it requires a more throughout refactor of many related parts of code. It is a real customer blocking issue, so we fix it with the temp solution and using this JIRA to follow up. The solution in TRAFODION-1858 is to add a new method to check the string name of a given column to tell if it is SALT column, or DIVSION column. The checking method is hardcoding the naming conversion, which may change and break the function. However, to use NAColumn's built-in helper checking methods, it needs the SQL compiler to generate correct constraint info from begining and change all the intermediate objects along the path. It may also need to change the METATABLE which contains information about constraints. So we use this JIRA to track, and need a longer time to find a good solution to solve TRAFODION-1858. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1858) RI predicate generating string should not contain _SALT_ column
[
https://issues.apache.org/jira/browse/TRAFODION-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184182#comment-15184182
]
liu ming commented on TRAFODION-1858:
-
TRAFODION-1880 is created for same issue and replace this JIRA.
This JIRA will be closed with a temp ,simple solution.
TRAFODION-1880 will track the effort to find a complete total solution to this
kind of issue.
> RI predicate generating string should not contain _SALT_ column
> ---
>
> Key: TRAFODION-1858
> URL: https://issues.apache.org/jira/browse/TRAFODION-1858
> Project: Apache Trafodion
> Issue Type: Bug
>Reporter: liu ming
>Assignee: liu ming
>
> When table created with SALT, it cannot be used in a foreign key reference.
> To reproduce:
> >>CREATE TABLE a ( id int not null, PRIMARY KEY (id))SALT USING 9 PARTITIONS;
> --- SQL operation complete.
> >>CREATE TABLE b ( id int not null, val int,
> +> PRIMARY KEY (id),
> +> CONSTRAINT FK FOREIGN KEY (val) REFERENCES a (id))
> +>SALT USING 9 PARTITIONS;
> --- SQL operation complete.
> >>INSERT INTO a values(1);
> --- 1 row(s) inserted.
> >>INSERT INTO b values(1,1);
> *** ERROR[15001] A syntax error occurred at or before:
> ("NEW@".VAL)=(TRAFODION.SEABASE.A."_SALT_",TRAFODION.SEABASE.A.ID);
> ^ (43 characters from start of SQL
> statement)
> *** ERROR[8822] The statement was not prepared.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1858) RI predicate generating string should not contain _SALT_ column
[
https://issues.apache.org/jira/browse/TRAFODION-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liu ming resolved TRAFODION-1858.
-
Resolution: Fixed
Fix Version/s: 2.0-incubating
> RI predicate generating string should not contain _SALT_ column
> ---
>
> Key: TRAFODION-1858
> URL: https://issues.apache.org/jira/browse/TRAFODION-1858
> Project: Apache Trafodion
> Issue Type: Bug
>Reporter: liu ming
>Assignee: liu ming
> Fix For: 2.0-incubating
>
>
> When table created with SALT, it cannot be used in a foreign key reference.
> To reproduce:
> >>CREATE TABLE a ( id int not null, PRIMARY KEY (id))SALT USING 9 PARTITIONS;
> --- SQL operation complete.
> >>CREATE TABLE b ( id int not null, val int,
> +> PRIMARY KEY (id),
> +> CONSTRAINT FK FOREIGN KEY (val) REFERENCES a (id))
> +>SALT USING 9 PARTITIONS;
> --- SQL operation complete.
> >>INSERT INTO a values(1);
> --- 1 row(s) inserted.
> >>INSERT INTO b values(1,1);
> *** ERROR[15001] A syntax error occurred at or before:
> ("NEW@".VAL)=(TRAFODION.SEABASE.A."_SALT_",TRAFODION.SEABASE.A.ID);
> ^ (43 characters from start of SQL
> statement)
> *** ERROR[8822] The statement was not prepared.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (TRAFODION-1858) RI predicate generating string should not contain _SALT_ column
[
https://issues.apache.org/jira/browse/TRAFODION-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liu ming closed TRAFODION-1858.
---
There is TRAFODION-1880 which continue on this issue. This is a temp solution.
> RI predicate generating string should not contain _SALT_ column
> ---
>
> Key: TRAFODION-1858
> URL: https://issues.apache.org/jira/browse/TRAFODION-1858
> Project: Apache Trafodion
> Issue Type: Bug
>Reporter: liu ming
>Assignee: liu ming
> Fix For: 2.0-incubating
>
>
> When table created with SALT, it cannot be used in a foreign key reference.
> To reproduce:
> >>CREATE TABLE a ( id int not null, PRIMARY KEY (id))SALT USING 9 PARTITIONS;
> --- SQL operation complete.
> >>CREATE TABLE b ( id int not null, val int,
> +> PRIMARY KEY (id),
> +> CONSTRAINT FK FOREIGN KEY (val) REFERENCES a (id))
> +>SALT USING 9 PARTITIONS;
> --- SQL operation complete.
> >>INSERT INTO a values(1);
> --- 1 row(s) inserted.
> >>INSERT INTO b values(1,1);
> *** ERROR[15001] A syntax error occurred at or before:
> ("NEW@".VAL)=(TRAFODION.SEABASE.A."_SALT_",TRAFODION.SEABASE.A.ID);
> ^ (43 characters from start of SQL
> statement)
> *** ERROR[8822] The statement was not prepared.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (TRAFODION-1875) 'sqnodeipcrm' script failed to parse node name during sqstart
[ https://issues.apache.org/jira/browse/TRAFODION-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1875: Assignee: Eason Zhang > 'sqnodeipcrm' script failed to parse node name during sqstart > - > > Key: TRAFODION-1875 > URL: https://issues.apache.org/jira/browse/TRAFODION-1875 > Project: Apache Trafodion > Issue Type: Bug >Reporter: Eason Zhang >Assignee: Eason Zhang > > If there's 'dot' in trafodion directory name, the script 'sqnodeipcrm' will > fail to parse the correct node name during sqstart: > /home/trafodion/trafodion-1.3.0/sql/scripts/sqnodeipcrm: line 28: let: > lv_virtual_node=0/tmp/monitor: division by 0 (error token is "/monitor") -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-1878) Create a docker environment
[ https://issues.apache.org/jira/browse/TRAFODION-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-1878: Assignee: Nitin Lamba > Create a docker environment > --- > > Key: TRAFODION-1878 > URL: https://issues.apache.org/jira/browse/TRAFODION-1878 > Project: Apache Trafodion > Issue Type: New Feature > Components: dev-environment >Reporter: Nitin Lamba >Assignee: Nitin Lamba > > Create a portable development environment using docker. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1834) ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in sqconfig are fully qualified
[ https://issues.apache.org/jira/browse/TRAFODION-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming resolved TRAFODION-1834. - Resolution: Fixed Fix Version/s: 2.0-incubating > ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in > sqconfig are fully qualified > -- > > Key: TRAFODION-1834 > URL: https://issues.apache.org/jira/browse/TRAFODION-1834 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmp >Affects Versions: 1.3-incubating >Reporter: Atanu Mishra >Assignee: liu ming > Fix For: 2.0-incubating > > > Paraphrasing Hans, who looked into this briefly: > Looking at the code, I can see that we remove anything from the first dot in > the node name of the region. Then we search for the unqualified node name in > the list of node names of the Trafodion cluster. > Both of these clusters use FQDNs in their configuration, though, therefore we > won't find the node names. > To Reproduce: > Do an EXPLAIN of a parallel query with and without TRAF_ALLOW_ESP_COLOCATION > set, and the node map will not show a co-located plan when the CQD is on - if > the system uses FQDNs in its sqconfig file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work stopped] (TRAFODION-1834) ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in sqconfig are fully qualified
[ https://issues.apache.org/jira/browse/TRAFODION-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TRAFODION-1834 stopped by liu ming. --- > ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in > sqconfig are fully qualified > -- > > Key: TRAFODION-1834 > URL: https://issues.apache.org/jira/browse/TRAFODION-1834 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmp >Affects Versions: 1.3-incubating >Reporter: Atanu Mishra >Assignee: liu ming > Fix For: 2.0-incubating > > > Paraphrasing Hans, who looked into this briefly: > Looking at the code, I can see that we remove anything from the first dot in > the node name of the region. Then we search for the unqualified node name in > the list of node names of the Trafodion cluster. > Both of these clusters use FQDNs in their configuration, though, therefore we > won't find the node names. > To Reproduce: > Do an EXPLAIN of a parallel query with and without TRAF_ALLOW_ESP_COLOCATION > set, and the node map will not show a co-located plan when the CQD is on - if > the system uses FQDNs in its sqconfig file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (TRAFODION-1834) ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in sqconfig are fully qualified
[ https://issues.apache.org/jira/browse/TRAFODION-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TRAFODION-1834 started by liu ming. --- > ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in > sqconfig are fully qualified > -- > > Key: TRAFODION-1834 > URL: https://issues.apache.org/jira/browse/TRAFODION-1834 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmp >Affects Versions: 1.3-incubating >Reporter: Atanu Mishra >Assignee: liu ming > Fix For: 2.0-incubating > > > Paraphrasing Hans, who looked into this briefly: > Looking at the code, I can see that we remove anything from the first dot in > the node name of the region. Then we search for the unqualified node name in > the list of node names of the Trafodion cluster. > Both of these clusters use FQDNs in their configuration, though, therefore we > won't find the node names. > To Reproduce: > Do an EXPLAIN of a parallel query with and without TRAF_ALLOW_ESP_COLOCATION > set, and the node map will not show a co-located plan when the CQD is on - if > the system uses FQDNs in its sqconfig file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1854) Trafodion cannot start on nodes with uppercase hostname.
[ https://issues.apache.org/jira/browse/TRAFODION-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming resolved TRAFODION-1854. - Resolution: Fixed Fix Version/s: 2.0-incubating > Trafodion cannot start on nodes with uppercase hostname. > > > Key: TRAFODION-1854 > URL: https://issues.apache.org/jira/browse/TRAFODION-1854 > Project: Apache Trafodion > Issue Type: Bug > Components: foundation >Reporter: Eason Zhang >Assignee: liu ming > Fix For: 2.0-incubating > > > set the node's hostname to uppercase, in this case it is set to 'H1 H2 H3' > Trafodion’s .bashrc is setting correctly: > > # These env vars define all nodes in the cluster > export NODE_LIST=" H1 H2 H3" > export MY_NODES=" -w H1 -w H2 -w H3" > > /etc/hosts are also set as ‘H1 H2 H3’. > sqconfig is also set as uppercase hostname. > when starting trafodion instance, it will report below error in sqmon.log: > Processing cluster.conf on local host H1 > [SHELL] Shell/shell Version 1.0.1 EsgynDB_Enterprise Release 2.0.0 (Build > release [EsgynDB-2.0.0-0-g2ba9dde_Bld402], date 20151121_0002) > > [SHELL] % > ! Start the monitor processes across the cluster > startup > [SHELL] %startup > [SHELL] Cannot start monitor from node 'H1' since it is not member of the > cluster configuration or 'hostname' string does not match configuration > string. > [SHELL] Configuration node names: > [SHELL]'h1' > [SHELL]'h2' > [SHELL]'h3' > [SHELL] Failed to start environment! > > [SHELL] % > exit -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1868) Compatibility with gcc 4.8
[ https://issues.apache.org/jira/browse/TRAFODION-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1868: --- Assignee: liu ming > Compatibility with gcc 4.8 > -- > > Key: TRAFODION-1868 > URL: https://issues.apache.org/jira/browse/TRAFODION-1868 > Project: Apache Trafodion > Issue Type: Bug >Reporter: Steve Varnau >Assignee: liu ming >Priority: Minor > > Current code will not compile with gcc 4.8. That is the default version on > RH/CentOS 7. This is needed to move to a development environment on CentOS 7. > There is a 4.4 version of gcc available on 7. The RPM package is > compat-gcc-44, I have not tried building on 7.x with older compiler, but will > likely work. > This is not urgent at this time, but we should move toward enabling a newer > dev environment. But this needs to be coordinated with with versions of OS > are supported for runtime use of trafodion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (TRAFODION-1868) Compatibility with gcc 4.8
[ https://issues.apache.org/jira/browse/TRAFODION-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TRAFODION-1868 started by liu ming. --- > Compatibility with gcc 4.8 > -- > > Key: TRAFODION-1868 > URL: https://issues.apache.org/jira/browse/TRAFODION-1868 > Project: Apache Trafodion > Issue Type: Bug >Reporter: Steve Varnau >Assignee: liu ming >Priority: Minor > > Current code will not compile with gcc 4.8. That is the default version on > RH/CentOS 7. This is needed to move to a development environment on CentOS 7. > There is a 4.4 version of gcc available on 7. The RPM package is > compat-gcc-44, I have not tried building on 7.x with older compiler, but will > likely work. > This is not urgent at this time, but we should move toward enabling a newer > dev environment. But this needs to be coordinated with with versions of OS > are supported for runtime use of trafodion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1865) uninitialized buffer cause Monitor abort in CentOS 7
[
https://issues.apache.org/jira/browse/TRAFODION-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15188594#comment-15188594
]
liu ming commented on TRAFODION-1865:
-
The root cause is incompatibility of glibc from CentOS 6 to CentOS 7.
It has no relationship with uninitialized buffer.
The right solution is to build trafodion binary using gcc 4.8 under centOS 7 to
run it on CentOS 7.
So close this jira now.
> uninitialized buffer cause Monitor abort in CentOS 7
>
>
> Key: TRAFODION-1865
> URL: https://issues.apache.org/jira/browse/TRAFODION-1865
> Project: Apache Trafodion
> Issue Type: Bug
> Components: foundation
>Reporter: liu ming
>Assignee: liu ming
>
> sqstart failed on CentOS.
> The Monitor process abort in CProcessContainer::CProcessContainer() due to
> failed to create semaphore.
> //create & initialize existing semaphore
> char sem_name[MAX_PROCESS_PATH];
> snprintf(sem_name,sizeof(sem_name), "/monitor.sem.%s", getenv("USER"));
> Mutex = sem_open(sem_name,O_CREAT,0644,0);
> if(Mutex == SEM_FAILED)
> {
> char buf[MON_STRING_BUF_SIZE];
> snprintf(buf, sizeof(buf), "[%s], Can't create semaphore %s!\n",
> method_name, sem_name);
> mon_log_write(MON_PROCESSCONT_PROCESSCONT_3, SQ_LOG_ERR, buf);
> sem_unlink(sem_name);
> abort();
> }
> sem_name is not initialized and snprintf will not add \0 after copy required
> bytes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1865) uninitialized buffer cause Monitor abort in CentOS 7
[
https://issues.apache.org/jira/browse/TRAFODION-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liu ming resolved TRAFODION-1865.
-
Resolution: Duplicate
Fix Version/s: 2.0-incubating
jira TRAFODION-1868 takes care of this issue instead.
> uninitialized buffer cause Monitor abort in CentOS 7
>
>
> Key: TRAFODION-1865
> URL: https://issues.apache.org/jira/browse/TRAFODION-1865
> Project: Apache Trafodion
> Issue Type: Bug
> Components: foundation
>Reporter: liu ming
>Assignee: liu ming
> Fix For: 2.0-incubating
>
>
> sqstart failed on CentOS.
> The Monitor process abort in CProcessContainer::CProcessContainer() due to
> failed to create semaphore.
> //create & initialize existing semaphore
> char sem_name[MAX_PROCESS_PATH];
> snprintf(sem_name,sizeof(sem_name), "/monitor.sem.%s", getenv("USER"));
> Mutex = sem_open(sem_name,O_CREAT,0644,0);
> if(Mutex == SEM_FAILED)
> {
> char buf[MON_STRING_BUF_SIZE];
> snprintf(buf, sizeof(buf), "[%s], Can't create semaphore %s!\n",
> method_name, sem_name);
> mon_log_write(MON_PROCESSCONT_PROCESSCONT_3, SQ_LOG_ERR, buf);
> sem_unlink(sem_name);
> abort();
> }
> sem_name is not initialized and snprintf will not add \0 after copy required
> bytes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TRAFODION-1869) 'initialized trafodion' failed on CentOS 7.2
[ https://issues.apache.org/jira/browse/TRAFODION-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15188596#comment-15188596 ] liu ming commented on TRAFODION-1869: - improper link order of ms.o and mpitmsg.o to build libsbms CentOS 7.2 dynamic linker will invoke constructor in the order of the link order. Will fix it soon. > 'initialized trafodion' failed on CentOS 7.2 > > > Key: TRAFODION-1869 > URL: https://issues.apache.org/jira/browse/TRAFODION-1869 > Project: Apache Trafodion > Issue Type: Bug >Reporter: liu ming >Assignee: liu ming > > On CentOS 7.2, install Trafodion then run 'initialize trafodion' > >>initialize trafodion; > *** ERROR[2034] $Z000HQ0:48: Operating system error 201 while communicating > with server process $Z000I2H:52. > *** ERROR[2013] Server process tdm_arkcmp could not be created on \\NSK - > Operating system error 201. > *** ERROR[2002] Internal error: cannot create compiler. > *** ERROR[2005] Internal error: from compilation, no errors in diagnostics > yet for statement: control query shape hold; > *** ERROR[8822] The statement was not prepared. > --- SQL operation failed with errors. > related core is from tdm_arkcmp > Core was generated by `tdm_arkcmp SQMON1.1 0 0 022137 $Z000I2H > 192.168.0.10:56579 4 0'. > Program terminated with signal 11, Segmentation fault. > #0 0x in ?? () > Missing separate debuginfos, use: debuginfo-install trafodion-2.0.1-1.x86_64 > trafodion-2.0.1-devel.x86_64 > (gdb) bt > #0 0x in ?? () > #1 0x7f44a8552fc6 in SB_Thread::Sthr::time () at > /home/centos/traf-plus/core/sqf/export/include/seabed/int/thread.inl:115 > #2 0x7f44adebeef4 in fs_int_fs_file_awaitiox (pp_filenum=0x1603034, > ppp_buf=0x7ffe5d71b208, pp_xfercount=0x7ffe5d71b1f4, > pp_tag=0x7ffe5d71b1f8, pv_timeout=6, pp_segid=0x7ffe5d71b206, > pv_int=false, pv_ts=false) at fsi.cpp:1301 > #3 0x7f44adeb8a80 in BAWAITIOX (pp_filenum=0x1603034, > ppp_buf=0x7ffe5d71b5f8, pp_xfercount=0x7ffe5d71b60c, pp_tag=0x7ffe5d71b5f0, > pv_timeout=6, pp_segid=0x0) at fs.cpp:563 > #4 0x7f44b5d629e7 in GuaReceiveControlConnection::wait (this=0x1603020, > timeout=6, eventConsumed=, ipcAwaitiox=0x0) > at ../common/IpcGuardian.cpp:2692 > #5 0x7f44b5d643d8 in GuaReceiveControlConnection::waitForMaster > (this=0x1603020) at ../common/IpcGuardian.cpp:3603 > #6 0x00406366 in main () > (gdb) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (TRAFODION-1854) Trafodion cannot start on nodes with uppercase hostname.
[ https://issues.apache.org/jira/browse/TRAFODION-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reopened TRAFODION-1854: - The original fix not correct, problem still there, need rework > Trafodion cannot start on nodes with uppercase hostname. > > > Key: TRAFODION-1854 > URL: https://issues.apache.org/jira/browse/TRAFODION-1854 > Project: Apache Trafodion > Issue Type: Bug > Components: foundation >Reporter: Eason Zhang >Assignee: liu ming > Fix For: 2.0-incubating > > > set the node's hostname to uppercase, in this case it is set to 'H1 H2 H3' > Trafodion’s .bashrc is setting correctly: > > # These env vars define all nodes in the cluster > export NODE_LIST=" H1 H2 H3" > export MY_NODES=" -w H1 -w H2 -w H3" > > /etc/hosts are also set as ‘H1 H2 H3’. > sqconfig is also set as uppercase hostname. > when starting trafodion instance, it will report below error in sqmon.log: > Processing cluster.conf on local host H1 > [SHELL] Shell/shell Version 1.0.1 EsgynDB_Enterprise Release 2.0.0 (Build > release [EsgynDB-2.0.0-0-g2ba9dde_Bld402], date 20151121_0002) > > [SHELL] % > ! Start the monitor processes across the cluster > startup > [SHELL] %startup > [SHELL] Cannot start monitor from node 'H1' since it is not member of the > cluster configuration or 'hostname' string does not match configuration > string. > [SHELL] Configuration node names: > [SHELL]'h1' > [SHELL]'h2' > [SHELL]'h3' > [SHELL] Failed to start environment! > > [SHELL] % > exit -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (TRAFODION-1854) Trafodion cannot start on nodes with uppercase hostname.
[ https://issues.apache.org/jira/browse/TRAFODION-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TRAFODION-1854 started by liu ming. --- > Trafodion cannot start on nodes with uppercase hostname. > > > Key: TRAFODION-1854 > URL: https://issues.apache.org/jira/browse/TRAFODION-1854 > Project: Apache Trafodion > Issue Type: Bug > Components: foundation >Reporter: Eason Zhang >Assignee: liu ming > Fix For: 2.0-incubating > > > set the node's hostname to uppercase, in this case it is set to 'H1 H2 H3' > Trafodion’s .bashrc is setting correctly: > > # These env vars define all nodes in the cluster > export NODE_LIST=" H1 H2 H3" > export MY_NODES=" -w H1 -w H2 -w H3" > > /etc/hosts are also set as ‘H1 H2 H3’. > sqconfig is also set as uppercase hostname. > when starting trafodion instance, it will report below error in sqmon.log: > Processing cluster.conf on local host H1 > [SHELL] Shell/shell Version 1.0.1 EsgynDB_Enterprise Release 2.0.0 (Build > release [EsgynDB-2.0.0-0-g2ba9dde_Bld402], date 20151121_0002) > > [SHELL] % > ! Start the monitor processes across the cluster > startup > [SHELL] %startup > [SHELL] Cannot start monitor from node 'H1' since it is not member of the > cluster configuration or 'hostname' string does not match configuration > string. > [SHELL] Configuration node names: > [SHELL]'h1' > [SHELL]'h2' > [SHELL]'h3' > [SHELL] Failed to start environment! > > [SHELL] % > exit -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1890) disable SSCC entirely in current Trafodion
liu ming created TRAFODION-1890: --- Summary: disable SSCC entirely in current Trafodion Key: TRAFODION-1890 URL: https://issues.apache.org/jira/browse/TRAFODION-1890 Project: Apache Trafodion Issue Type: Bug Reporter: liu ming Assignee: liu ming Priority: Minor I have difficulties to start HBase using 'install_local_hadoop' each time I have a new workspace. Most of the time, I simply remove the sscc coprocessor from hbase-site.xml and hbase can start well. Since SSCC has not been maintained for a long time, and need more effort to test it again. So I think we should disable it totally and remove it from hbase-site.xml -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (TRAFODION-1890) disable SSCC entirely in current Trafodion
[ https://issues.apache.org/jira/browse/TRAFODION-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TRAFODION-1890 started by liu ming. --- > disable SSCC entirely in current Trafodion > -- > > Key: TRAFODION-1890 > URL: https://issues.apache.org/jira/browse/TRAFODION-1890 > Project: Apache Trafodion > Issue Type: Bug >Reporter: liu ming >Assignee: liu ming >Priority: Minor > > I have difficulties to start HBase using 'install_local_hadoop' each time I > have a new workspace. > Most of the time, I simply remove the sscc coprocessor from hbase-site.xml > and hbase can start well. > Since SSCC has not been maintained for a long time, and need more effort to > test it again. So I think we should disable it totally and remove it from > hbase-site.xml -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1901) allow more flexible column-definition in create table ddl
liu ming created TRAFODION-1901: --- Summary: allow more flexible column-definition in create table ddl Key: TRAFODION-1901 URL: https://issues.apache.org/jira/browse/TRAFODION-1901 Project: Apache Trafodion Issue Type: Improvement Reporter: liu ming Current Trafodion DDL syntax for CREATE TABLE is very strict to ANSI standard about column-definition. column-definition is: column data-type [DEFAULT default | NO DEFAULT] [[CONSTRAINT constraint-name] column-constraint] So constraint like 'NOT NULL' must follow the DEFAULT descriptor. In many other databases, this is allowed. So if Trafodion make this more flexible, it will help database migration. Here is a test case: CREATE TABLE TABLETEST1 (MARK_ID SMALLINTNOT NULL, BEGIN_TIME DATENOT NULL DEFAULT date'2008-01-01', END_TIMEDATENOT NULL DEFAULT date'2018-01-01', ACTIVE_FLAG SMALLINT, MARK_NAME VARCHAR(20), DESC_TXTVARCHAR(80), primary key(MARK_ID, BEGIN_TIME, END_TIME) ); *** ERROR[15001] A syntax error occurred at or before: CREATE TABLE TABLETEST (MARK_ID SMALLINTNO T NULL, BEGIN_TIME DATENOT NULL DEFAULT date'2008-01-01', END ^ (134 characters from start of SQL statement) *** ERROR[8822] The statement was not prepared. Trafodion support of above DDL will greatly simplify the database migration from other databases into Trafodion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
