[2/2] trafodion git commit: Merge PR 1681 [TRAFODION-3174] Add Syntax for *CREATE TABLE Statement* and *ALTER TABLE Statement* and Fix Typos in *Trafodion SQL Reference Manual*
Merge PR 1681 [TRAFODION-3174] Add Syntax for *CREATE TABLE Statement* and *ALTER TABLE Statement* and Fix Typos in *Trafodion SQL Reference Manual* Project: http://git-wip-us.apache.org/repos/asf/trafodion/repo Commit: http://git-wip-us.apache.org/repos/asf/trafodion/commit/843e089c Tree: http://git-wip-us.apache.org/repos/asf/trafodion/tree/843e089c Diff: http://git-wip-us.apache.org/repos/asf/trafodion/diff/843e089c Branch: refs/heads/master Commit: 843e089c85a7536d8407e87dd63bc7d472bed76a Parents: 0d30493 1ceff37 Author: liu.yu Authored: Tue Aug 7 13:31:37 2018 +0800 Committer: liu.yu Committed: Tue Aug 7 13:31:37 2018 +0800 -- .../src/asciidoc/_chapters/sql_clauses.adoc | 44 --- .../_chapters/sql_language_elements.adoc| 125 ++- .../src/asciidoc/_chapters/sql_statements.adoc | 19 ++- 3 files changed, 109 insertions(+), 79 deletions(-) --
[1/2] trafodion git commit: [TRAFODION-3174] Add Syntax for *CREATE TABLE Statement* and *ALTER TABLE Statement* and Fix Typos in *Trafodion SQL Reference Manual*
Repository: trafodion
Updated Branches:
refs/heads/master 0d30493de -> 843e089c8
[TRAFODION-3174] Add Syntax for *CREATE TABLE Statement* and *ALTER TABLE
Statement* and Fix Typos in *Trafodion SQL Reference Manual*
Project: http://git-wip-us.apache.org/repos/asf/trafodion/repo
Commit: http://git-wip-us.apache.org/repos/asf/trafodion/commit/1ceff371
Tree: http://git-wip-us.apache.org/repos/asf/trafodion/tree/1ceff371
Diff: http://git-wip-us.apache.org/repos/asf/trafodion/diff/1ceff371
Branch: refs/heads/master
Commit: 1ceff3712b8677bf7e53e04c517fc639ee6fe991
Parents: 9c59d78
Author: liu.yu
Authored: Mon Aug 6 19:48:01 2018 +0800
Committer: liu.yu
Committed: Mon Aug 6 19:48:01 2018 +0800
--
.../src/asciidoc/_chapters/sql_clauses.adoc | 44 ---
.../_chapters/sql_language_elements.adoc| 125 ++-
.../src/asciidoc/_chapters/sql_statements.adoc | 19 ++-
3 files changed, 109 insertions(+), 79 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/trafodion/blob/1ceff371/docs/sql_reference/src/asciidoc/_chapters/sql_clauses.adoc
--
diff --git a/docs/sql_reference/src/asciidoc/_chapters/sql_clauses.adoc
b/docs/sql_reference/src/asciidoc/_chapters/sql_clauses.adoc
index 450dd9f..2f2b5ad 100644
--- a/docs/sql_reference/src/asciidoc/_chapters/sql_clauses.adoc
+++ b/docs/sql_reference/src/asciidoc/_chapters/sql_clauses.adoc
@@ -62,8 +62,8 @@ default is:
specifies the column has no default value. You cannot specify NO DEFAULT
in an ALTER TABLE statement. See <>.
-[[syntax_for_default_clause]]
-=== Syntax for Default Clause
+[[syntax_description_for_default]]
+=== Syntax Description for Default
* `DEFAULT _literal_`
+
@@ -157,8 +157,12 @@ separating the digits of integer output with colons.
(FORMAT 'format-string') |
(DATE, FORMAT 'format-string')
+```
+
++
+```
+format-string is:
-format-string for Date Formats is:
-MM-DD
MM/DD/
YY/MM/DD
@@ -173,13 +177,19 @@ format-string for Date Formats is:
+
```
(FORMAT 'format-string')
+```
++
+```
+format-string is:
-format-string for other formats is:
XXX
99:99:99:99
-99:99:99:99
```
+[[syntax_description_of_format]]
+=== Syntax Description of Format
+
* `-MM-DD`
+
specifies that the FORMAT clause output format is _year-month-day_.
@@ -227,23 +237,26 @@ must be a numeric value.
specifies that the FORMAT clause output format is a timestamp. The input
must be a numeric value.
+[[considerations_for_format]]
+=== Considerations for FORMAT
+
[[considerations_for_date_formats]]
-=== Considerations for Date Formats
+ Considerations for Date Formats
-The expression preceding the (FORMAT â_format-string_') clause must be
+* The expression preceding the (`FORMAT _format-string_`) clause must be
a DATE value.
-The expression preceding the (DATE, FORMAT _'format-string_') clause
+* The expression preceding the (DATE, `FORMAT _format-string_`) clause
must be a quoted string in the USA, EUROPEAN, or DEFAULT date format.
[[considerations_for_other_formats]]
Considerations for Other Formats
-For XXX, the expression preceding the (FORMAT _'format-string_')
+* For XXX, the expression preceding the (FORMAT `_format-string_`)
clause must be a numeric value or a string value.
-For 99:99:99:99 and -99:99:99:99, the expression preceding the (FORMAT
-_'format-string_') clause must be a numeric value.
+* For 99:99:99:99 and -99:99:99:99, the expression preceding the (FORMAT
+`_format-string_`) clause must be a numeric value.
[[examples_of_format]]
=== Examples of FORMAT
@@ -344,6 +357,9 @@ rows-size is:
[ELSE number-rows ROWS] END
```
+[[syntax_description_of_sample]]
+=== Syntax Description of SAMPLE
+
* `RANDOM _percent-size_`
+
directs {project-name} SQL to choose rows randomly (each row having an
@@ -364,7 +380,7 @@ exist in the intermediate result table. Each row is picked
with a
probability of _r_%, where _r_ is the sample size in PERCENT.
Therefore, the actual size of the resulting sample is approximately _r_% of
_N_.
The number of rows picked follows a binomial distribution with
-mean equal to _r_ *c_N_/100.
+mean equal to _r_*_N_/100.
+
If you specify a sample size greater than 100 PERCENT, {project-name} SQL
returns all the rows in the result table plus duplicate rows. The
@@ -780,9 +796,7 @@ LASERPRINTER, X1 W 42000.00
== SEQUENCE BY Clause
The SEQUENCE BY clause of the SELECT statement specifies the order in
-which to sort the rows
-
-of the intermediate result table for calculating sequence functions.
+which to sort the rows of the intermediate result table for calculating
sequence functions.
This option is used for processing time-sequenced rows in data mining
applications. See <>.
@@ -1055,7 +1069,7 @@
[1/2] trafodion git commit: TRAFODION-3165 HDFS storage option not work well if table use namespace
Repository: trafodion
Updated Branches:
refs/heads/master c8ffae3b4 -> 0d30493de
TRAFODION-3165 HDFS storage option not work well if table use namespace
Project: http://git-wip-us.apache.org/repos/asf/trafodion/repo
Commit: http://git-wip-us.apache.org/repos/asf/trafodion/commit/4ac689a6
Tree: http://git-wip-us.apache.org/repos/asf/trafodion/tree/4ac689a6
Diff: http://git-wip-us.apache.org/repos/asf/trafodion/diff/4ac689a6
Branch: refs/heads/master
Commit: 4ac689a67a80e3627e20a3146565513bd1e5a4ba
Parents: 20724db
Author: Liu Ming
Authored: Thu Aug 2 14:51:28 2018 -0400
Committer: Liu Ming
Committed: Thu Aug 2 14:51:28 2018 -0400
--
.../hadoop/hbase/client/transactional/TransactionManager.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/trafodion/blob/4ac689a6/core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/client/transactional/TransactionManager.java
--
diff --git
a/core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/client/transactional/TransactionManager.java
b/core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/client/transactional/TransactionManager.java
index 1eceb71..a2e14e3 100644
---
a/core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/client/transactional/TransactionManager.java
+++
b/core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/client/transactional/TransactionManager.java
@@ -3285,7 +3285,7 @@ public class TransactionManager {
String namespacestr="";
String fullPath = hbaseRoot + "/data/" ;
String fullPath2 = hbaseRoot + "/data/default/";
- if(fs.exists(new Path(fullPath2)))
+ if(fs.exists(new Path(fullPath2)) && parts.length == 1) // no
namespace in the path
fullPath = fullPath2;
if(parts.length >1) //have namespace
[2/2] trafodion git commit: merge [TRAFODION-3165]
merge [TRAFODION-3165] Project: http://git-wip-us.apache.org/repos/asf/trafodion/repo Commit: http://git-wip-us.apache.org/repos/asf/trafodion/commit/0d30493d Tree: http://git-wip-us.apache.org/repos/asf/trafodion/tree/0d30493d Diff: http://git-wip-us.apache.org/repos/asf/trafodion/diff/0d30493d Branch: refs/heads/master Commit: 0d30493de9bcbf1a7bb48cfaddd1e4723b8aca55 Parents: c8ffae3 4ac689a Author: Liu Ming Authored: Mon Aug 6 19:27:09 2018 + Committer: Liu Ming Committed: Mon Aug 6 19:27:09 2018 + -- .../hadoop/hbase/client/transactional/TransactionManager.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --
[2/3] trafodion git commit: [TRAFODION-3171] Refactor Hive sequence file reading to use the new implementation
[TRAFODION-3171] Refactor Hive sequence file reading to use the new
implementation
Fix to resolve the issue highlighted in the review comment
Project: http://git-wip-us.apache.org/repos/asf/trafodion/repo
Commit: http://git-wip-us.apache.org/repos/asf/trafodion/commit/abbfe856
Tree: http://git-wip-us.apache.org/repos/asf/trafodion/tree/abbfe856
Diff: http://git-wip-us.apache.org/repos/asf/trafodion/diff/abbfe856
Branch: refs/heads/master
Commit: abbfe85683c299be35b79f64b90d90f7c253dc51
Parents: 6165e6b
Author: selvaganesang
Authored: Fri Aug 3 17:27:27 2018 +
Committer: selvaganesang
Committed: Fri Aug 3 17:27:27 2018 +
--
core/sql/src/main/java/org/trafodion/sql/HDFSClient.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/trafodion/blob/abbfe856/core/sql/src/main/java/org/trafodion/sql/HDFSClient.java
--
diff --git a/core/sql/src/main/java/org/trafodion/sql/HDFSClient.java
b/core/sql/src/main/java/org/trafodion/sql/HDFSClient.java
index 8a14b61..c04c256 100644
--- a/core/sql/src/main/java/org/trafodion/sql/HDFSClient.java
+++ b/core/sql/src/main/java/org/trafodion/sql/HDFSClient.java
@@ -280,7 +280,7 @@ public class HDFSClient
}
byteArray = ((Text)value_).getBytes();
readLen = ((Text)value_).getLength();
- if (readLen <= lenRemain) {
+ if ((readLen+1) <= lenRemain) {
buf_.put(byteArray, 0, readLen);
buf_.put(recDelimiter_);
[1/3] trafodion git commit: [TRAFODION-3171] Refactor Hive sequence file reading to use the new implementation
Repository: trafodion
Updated Branches:
refs/heads/master 9c59d7803 -> c8ffae3b4
[TRAFODION-3171] Refactor Hive sequence file reading to use the new
implementation
Project: http://git-wip-us.apache.org/repos/asf/trafodion/repo
Commit: http://git-wip-us.apache.org/repos/asf/trafodion/commit/6165e6b1
Tree: http://git-wip-us.apache.org/repos/asf/trafodion/tree/6165e6b1
Diff: http://git-wip-us.apache.org/repos/asf/trafodion/diff/6165e6b1
Branch: refs/heads/master
Commit: 6165e6b1017521abed062c55931d4555b886387d
Parents: 95eaa31
Author: selvaganesang
Authored: Wed Aug 1 20:08:46 2018 +
Committer: selvaganesang
Committed: Wed Aug 1 20:08:46 2018 +
--
core/sql/executor/ExHdfsScan.cpp| 3 +-
core/sql/executor/HdfsClient_JNI.cpp| 12 +-
core/sql/executor/HdfsClient_JNI.h | 6 +-
.../main/java/org/trafodion/sql/HDFSClient.java | 116 ---
.../main/java/org/trafodion/sql/HdfsScan.java | 24 ++--
5 files changed, 129 insertions(+), 32 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/trafodion/blob/6165e6b1/core/sql/executor/ExHdfsScan.cpp
--
diff --git a/core/sql/executor/ExHdfsScan.cpp b/core/sql/executor/ExHdfsScan.cpp
index b09cffd..d4cf717 100644
--- a/core/sql/executor/ExHdfsScan.cpp
+++ b/core/sql/executor/ExHdfsScan.cpp
@@ -130,8 +130,6 @@ ExHdfsScanTcb::ExHdfsScanTcb(
Space * space = (glob ? glob->getSpace() : 0);
CollHeap * heap = (glob ? glob->getDefaultHeap() : 0);
useLibhdfsScan_ = hdfsScanTdb.getUseLibhdfsScan();
- if (isSequenceFile())
- useLibhdfsScan_ = TRUE;
lobGlob_ = NULL;
hdfsScanBufMaxSize_ = hdfsScanTdb.hdfsBufSize_;
headRoom_ = (Int32)hdfsScanTdb.rangeTailIOSize_;
@@ -571,6 +569,7 @@ ExWorkProcRetcode ExHdfsScanTcb::work()
hdfsScan_ = HdfsScan::newInstance((NAHeap *)getHeap(),
hdfsScanBuf_, hdfsScanBufMaxSize_,
hdfsScanTdb().hdfsIoByteArraySizeInKB_,
_, beginRangeNum_,
numRanges_, hdfsScanTdb().rangeTailIOSize_,
+isSequenceFile(), hdfsScanTdb().recordDelimiter_,
hdfsStats_, hdfsScanRetCode);
if (hdfsScanRetCode != HDFS_SCAN_OK) {
setupError(EXE_ERROR_HDFS_SCAN, hdfsScanRetCode,
"SETUP_HDFS_SCAN",
http://git-wip-us.apache.org/repos/asf/trafodion/blob/6165e6b1/core/sql/executor/HdfsClient_JNI.cpp
--
diff --git a/core/sql/executor/HdfsClient_JNI.cpp
b/core/sql/executor/HdfsClient_JNI.cpp
index 65c83cd..5ae2805 100644
--- a/core/sql/executor/HdfsClient_JNI.cpp
+++ b/core/sql/executor/HdfsClient_JNI.cpp
@@ -85,7 +85,7 @@ HDFS_Scan_RetCode HdfsScan::init()
JavaMethods_[JM_CTOR ].jm_name = "";
JavaMethods_[JM_CTOR ].jm_signature = "()V";
JavaMethods_[JM_SET_SCAN_RANGES].jm_name = "setScanRanges";
-JavaMethods_[JM_SET_SCAN_RANGES].jm_signature =
"(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;I[Ljava/lang/String;[J[J[I[S)V";
+JavaMethods_[JM_SET_SCAN_RANGES].jm_signature =
"(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;I[Ljava/lang/String;[J[J[I[SZB)V";
JavaMethods_[JM_TRAF_HDFS_READ].jm_name = "trafHdfsRead";
JavaMethods_[JM_TRAF_HDFS_READ].jm_signature = "()[I";
JavaMethods_[JM_STOP].jm_name = "stop";
@@ -109,7 +109,7 @@ char* HdfsScan::getErrorText(HDFS_Scan_RetCode errEnum)
/
HDFS_Scan_RetCode HdfsScan::setScanRanges(ExHdfsScanTcb::HDFS_SCAN_BUF
*hdfsScanBuf, int scanBufSize, int hdfsIoByteArraySizeInKB,
- HdfsFileInfoArray *hdfsFileInfoArray, Int32 beginRangeNum, Int32
numRanges, int rangeTailIOSize)
+ HdfsFileInfoArray *hdfsFileInfoArray, Int32 beginRangeNum, Int32
numRanges, int rangeTailIOSize, NABoolean sequenceFile, char recDelimiter)
{
QRLogger::log(CAT_SQL_HDFS, LL_DEBUG, "HdfsScan::setScanRanges() called.");
@@ -216,12 +216,13 @@ HDFS_Scan_RetCode
HdfsScan::setScanRanges(ExHdfsScanTcb::HDFS_SCAN_BUF *hdfsScan
ex_assert(compressionMethod >= 0 && compressionMethod <
ComCompressionInfo::SUPPORTED_COMPRESSIONS, "Illegal CompressionMethod Value");
jenv_->SetShortArrayRegion(j_compress, rangeCount, 1,
);
}
-
+ jboolean j_sequenceFile = sequenceFile;
+ jbyte j_recDelimiter = (BYTE)recDelimiter;
if (hdfsStats_ != NULL)
hdfsStats_->getHdfsTimer().start();
tsRecentJMFromJNI = JavaMethods_[JM_SET_SCAN_RANGES].jm_full_name;
jenv_->CallVoidMethod(javaObj_, JavaMethods_[JM_SET_SCAN_RANGES].methodID,
j_buf1, j_buf2, j_hdfsIoByteArraySizeInKB,
- j_filenames, j_offsets, j_lens, j_rangenums, j_compress);
+
[3/3] trafodion git commit: Merge PR 1674 [TRAFODION-3171] Refactor Hive sequence file reading to use the new implemenation
Merge PR 1674 [TRAFODION-3171] Refactor Hive sequence file reading to use the new implemenation Project: http://git-wip-us.apache.org/repos/asf/trafodion/repo Commit: http://git-wip-us.apache.org/repos/asf/trafodion/commit/c8ffae3b Tree: http://git-wip-us.apache.org/repos/asf/trafodion/tree/c8ffae3b Diff: http://git-wip-us.apache.org/repos/asf/trafodion/diff/c8ffae3b Branch: refs/heads/master Commit: c8ffae3b450dcf2e40e33d02a3bea2d38e1a64a3 Parents: 9c59d78 abbfe85 Author: selvaganesang Authored: Mon Aug 6 18:26:53 2018 + Committer: selvaganesang Committed: Mon Aug 6 18:26:53 2018 + -- core/sql/executor/ExHdfsScan.cpp| 3 +- core/sql/executor/HdfsClient_JNI.cpp| 12 +- core/sql/executor/HdfsClient_JNI.h | 6 +- .../main/java/org/trafodion/sql/HDFSClient.java | 116 --- .../main/java/org/trafodion/sql/HdfsScan.java | 24 ++-- 5 files changed, 129 insertions(+), 32 deletions(-) --
