date:20160927

[GitHub] incubator-hawq pull request #944: HAWQ-991. Add try catch for parsing the ya...

2016-09-27 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/944


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (HAWQ-1080) make unittest-check fails on local dev environment

2016-09-27 Thread Goden Yao (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Goden Yao updated HAWQ-1080:

Assignee: Oleksandr Diachenko  (was: Lei Chang)

> make unittest-check fails on local dev environment
> --
>
> Key: HAWQ-1080
> URL: https://issues.apache.org/jira/browse/HAWQ-1080
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: PXF
>Reporter: Oleksandr Diachenko
>Assignee: Oleksandr Diachenko
> Fix For: 2.0.1.0-incubating
>
>
> {code}
> $ make unittest-check
> {code}
> ...
> {code}
> emanager  -c -o pxfheaders_test.o pxfheaders_test.c
> pxfheaders_test.c:124:1: error: conflicting types for 'expect_churl_headers'
> expect_churl_headers(const char *key, const char *value)
> ^
> pxfheaders_test.c:40:2: note: previous implicit declaration is here
> expect_churl_headers("X-GP-SEGMENT-ID", mock_extvar->GP_SEGMENT_ID);
> ^
> pxfheaders_test.c:139:1: error: conflicting types for 
> 'expect_churl_headers_alignment'
> expect_churl_headers_alignment()
> ^
> pxfheaders_test.c:43:2: note: previous implicit declaration is here
> expect_churl_headers_alignment();
> ^
> pxfheaders_test.c:159:1: error: conflicting types for 'store_gucs'
> store_gucs()
> ^
> pxfheaders_test.c:152:2: note: previous implicit declaration is here
> store_gucs();
> ^
> pxfheaders_test.c:166:1: error: conflicting types for 'setup_gphd_uri'
> setup_gphd_uri()
> ^
> pxfheaders_test.c:153:2: note: previous implicit declaration is here
> setup_gphd_uri();
> ^
> pxfheaders_test.c:176:1: error: conflicting types for 'setup_input_data'
> setup_input_data()
> ^
> pxfheaders_test.c:154:2: note: previous implicit declaration is here
> setup_input_data();
> ^
> pxfheaders_test.c:184:1: error: conflicting types for 'setup_external_vars'
> setup_external_vars()
> ^
> pxfheaders_test.c:155:2: note: previous implicit declaration is here
> setup_external_vars();
> ^
> pxfheaders_test.c:193:6: error: conflicting types for 'expect_external_vars'
> void expect_external_vars()
>  ^
> pxfheaders_test.c:38:2: note: previous implicit declaration is here
> expect_external_vars();
> ^
> pxfheaders_test.c:220:6: error: conflicting types for 'restore_gucs'
> void restore_gucs()
>  ^
> pxfheaders_test.c:217:2: note: previous implicit declaration is here
> restore_gucs();
> ^
> 8 errors generated.
> make[3]: *** [pxfheaders_test.o] Error 1
> make[2]: *** [unittest-check] Error 2
> make[1]: *** [unittest-check] Error 2
> make: *** [unittest-check] Error 2
> {code}
> ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-1080) make unittest-check fails on local dev environment

2016-09-27 Thread Goden Yao (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Goden Yao updated HAWQ-1080:

Fix Version/s: 2.0.1.0-incubating

> make unittest-check fails on local dev environment
> --
>
> Key: HAWQ-1080
> URL: https://issues.apache.org/jira/browse/HAWQ-1080
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: PXF
>Reporter: Oleksandr Diachenko
>Assignee: Lei Chang
> Fix For: 2.0.1.0-incubating
>
>
> {code}
> $ make unittest-check
> {code}
> ...
> {code}
> emanager  -c -o pxfheaders_test.o pxfheaders_test.c
> pxfheaders_test.c:124:1: error: conflicting types for 'expect_churl_headers'
> expect_churl_headers(const char *key, const char *value)
> ^
> pxfheaders_test.c:40:2: note: previous implicit declaration is here
> expect_churl_headers("X-GP-SEGMENT-ID", mock_extvar->GP_SEGMENT_ID);
> ^
> pxfheaders_test.c:139:1: error: conflicting types for 
> 'expect_churl_headers_alignment'
> expect_churl_headers_alignment()
> ^
> pxfheaders_test.c:43:2: note: previous implicit declaration is here
> expect_churl_headers_alignment();
> ^
> pxfheaders_test.c:159:1: error: conflicting types for 'store_gucs'
> store_gucs()
> ^
> pxfheaders_test.c:152:2: note: previous implicit declaration is here
> store_gucs();
> ^
> pxfheaders_test.c:166:1: error: conflicting types for 'setup_gphd_uri'
> setup_gphd_uri()
> ^
> pxfheaders_test.c:153:2: note: previous implicit declaration is here
> setup_gphd_uri();
> ^
> pxfheaders_test.c:176:1: error: conflicting types for 'setup_input_data'
> setup_input_data()
> ^
> pxfheaders_test.c:154:2: note: previous implicit declaration is here
> setup_input_data();
> ^
> pxfheaders_test.c:184:1: error: conflicting types for 'setup_external_vars'
> setup_external_vars()
> ^
> pxfheaders_test.c:155:2: note: previous implicit declaration is here
> setup_external_vars();
> ^
> pxfheaders_test.c:193:6: error: conflicting types for 'expect_external_vars'
> void expect_external_vars()
>  ^
> pxfheaders_test.c:38:2: note: previous implicit declaration is here
> expect_external_vars();
> ^
> pxfheaders_test.c:220:6: error: conflicting types for 'restore_gucs'
> void restore_gucs()
>  ^
> pxfheaders_test.c:217:2: note: previous implicit declaration is here
> restore_gucs();
> ^
> 8 errors generated.
> make[3]: *** [pxfheaders_test.o] Error 1
> make[2]: *** [unittest-check] Error 2
> make[1]: *** [unittest-check] Error 2
> make: *** [unittest-check] Error 2
> {code}
> ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HAWQ-1061) Improve hawq register for already bugs found

2016-09-27 Thread hongwu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hongwu resolved HAWQ-1061.
--
Resolution: Fixed

> Improve hawq register for already bugs found
> 
>
> Key: HAWQ-1061
> URL: https://issues.apache.org/jira/browse/HAWQ-1061
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Command Line Tools
>Affects Versions: 2.0.1.0-incubating
>Reporter: Lili Ma
>Assignee: hongwu
> Fix For: 2.0.1.0-incubating
>
>
> Fix the bugs found by the verification process:
> - Data loss when file locations include directories
> - Hash table registry consistent issue, which need to be check before 
> register. Need check distributed policy and bucket number besides files 
> number.
> - Register multiple files need to be rollback when there is some error before 
> done. It’s better to check source/target files exists and permissions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #943: HAWQ-1061. Check folder and eof not conflict unde...

2016-09-27 Thread xunzhang

Github user xunzhang commented on the issue:

https://github.com/apache/incubator-hawq/pull/943
  
Merged into master, thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #943: HAWQ-1061. Check folder and eof not confli...

2016-09-27 Thread xunzhang

Github user xunzhang closed the pull request at:

https://github.com/apache/incubator-hawq/pull/943


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #944: HAWQ-991. Add try catch for parsing the ya...

2016-09-27 Thread zhangh43

GitHub user zhangh43 opened a pull request:

https://github.com/apache/incubator-hawq/pull/944

HAWQ-991. Add try catch for parsing the yaml file.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zhangh43/incubator-hawq hawq9913

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/944.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #944


commit 4ddbc249e524c5b4b40ac9cddf6e7562dad74a92
Author: hzhang2 
Date:   2016-09-28T02:27:06Z

HAWQ-991. Add try catch for parsing the yaml file.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (HAWQ-1077) Table insert hangs due to stack overwrite which is caused by a bug in ao snappy code

2016-09-27 Thread Paul Guo (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Guo resolved HAWQ-1077.

Resolution: Fixed

> Table insert hangs due to stack overwrite which is caused by a bug in ao 
> snappy code
> 
>
> Key: HAWQ-1077
> URL: https://issues.apache.org/jira/browse/HAWQ-1077
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Paul Guo
> Fix For: 2.0.1.0-incubating
>
>
> It hang when try to insert large data into append-only table with 
> compression. To be specific, the QE process spins at compression phase. Per 
> RCA, there is stack overwritten in append-only table ith snappy compression.
> The call stack is as following:
> {noformat}
> #0  0x008607c4 in gp_trycompress_new (sourceData=0x2eec5b0 
> "\340\352\356\002", sourceLen=0,
> compressedBuffer=0xa2b3b4 "\311\303UH\211\345H\211}\370H\213E\370\307@<", 
> compressedBufferWithOverrrunLen=38253,
> maxCompressedLen=32767, compressedLen=0xf8c800a29636, compressLevel=0, 
> compressor=0x5d9cdf ,
> compressionState=0x2eee690) at gp_compress.c:56
> #1  0x00a2a1c6 in AppendOnlyStorageWrite_CompressAppend 
> (storageWrite=0x2eec558,
> sourceData=0x33efa68 "ata value for text data typelarge data value for 
> text data typelarge data value for text data typelarge data value for text 
> data typelarge data value for text data typelarge data value for text data 
> t"..., sourceLen=0,
> executorBlockKind=2, itemCount=0, compressedLen=0x7fffa7804f74, 
> bufferLen=0x7fffa7804f70) at cdbappendonlystoragewrite.c:1255
> #2  0x00a2ae1b in AppendOnlyStorageWrite_Content 
> (storageWrite=0x2eec558, content=0x33e7a70 "\242\241:\200\351\003",
> contentLen=3842464, executorBlockKind=2, rowCount=1) at 
> cdbappendonlystoragewrite.c:1868
> #3  0x0056b805 in appendonly_insert (aoInsertDesc=0x2eec460, 
> instup=0x33e7a70, tupleOid=0x7fffa7805094,
> aoTupleId=0x7fffa7805080) at appendonlyam.c:2030
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-1077) Table insert hangs due to stack overwrite which is caused by a bug in ao snappy code

2016-09-27 Thread Paul Guo (JIRA)


 [ 
https://issues.apache.org/jira/browse/HAWQ-1077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Guo closed HAWQ-1077.
--

> Table insert hangs due to stack overwrite which is caused by a bug in ao 
> snappy code
> 
>
> Key: HAWQ-1077
> URL: https://issues.apache.org/jira/browse/HAWQ-1077
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Paul Guo
> Fix For: 2.0.1.0-incubating
>
>
> It hang when try to insert large data into append-only table with 
> compression. To be specific, the QE process spins at compression phase. Per 
> RCA, there is stack overwritten in append-only table ith snappy compression.
> The call stack is as following:
> {noformat}
> #0  0x008607c4 in gp_trycompress_new (sourceData=0x2eec5b0 
> "\340\352\356\002", sourceLen=0,
> compressedBuffer=0xa2b3b4 "\311\303UH\211\345H\211}\370H\213E\370\307@<", 
> compressedBufferWithOverrrunLen=38253,
> maxCompressedLen=32767, compressedLen=0xf8c800a29636, compressLevel=0, 
> compressor=0x5d9cdf ,
> compressionState=0x2eee690) at gp_compress.c:56
> #1  0x00a2a1c6 in AppendOnlyStorageWrite_CompressAppend 
> (storageWrite=0x2eec558,
> sourceData=0x33efa68 "ata value for text data typelarge data value for 
> text data typelarge data value for text data typelarge data value for text 
> data typelarge data value for text data typelarge data value for text data 
> t"..., sourceLen=0,
> executorBlockKind=2, itemCount=0, compressedLen=0x7fffa7804f74, 
> bufferLen=0x7fffa7804f70) at cdbappendonlystoragewrite.c:1255
> #2  0x00a2ae1b in AppendOnlyStorageWrite_Content 
> (storageWrite=0x2eec558, content=0x33e7a70 "\242\241:\200\351\003",
> contentLen=3842464, executorBlockKind=2, rowCount=1) at 
> cdbappendonlystoragewrite.c:1868
> #3  0x0056b805 in appendonly_insert (aoInsertDesc=0x2eec460, 
> instup=0x33e7a70, tupleOid=0x7fffa7805094,
> aoTupleId=0x7fffa7805080) at appendonlyam.c:2030
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #943: HAWQ-1061. Check folder and eof not conflict unde...

2016-09-27 Thread zhangh43

Github user zhangh43 commented on the issue:

https://github.com/apache/incubator-hawq/pull/943
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #937: HAWQ-1077. Query hangs due to stack overwr...

2016-09-27 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/937


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (HAWQ-1080) make unittest-check fails on local dev environment

2016-09-27 Thread Oleksandr Diachenko (JIRA)

Oleksandr Diachenko created HAWQ-1080:
-

 Summary: make unittest-check fails on local dev environment
 Key: HAWQ-1080
 URL: https://issues.apache.org/jira/browse/HAWQ-1080
 Project: Apache HAWQ
  Issue Type: Bug
  Components: PXF
Reporter: Oleksandr Diachenko
Assignee: Lei Chang


{code}
$ make unittest-check
{code}
...
{code}
emanager  -c -o pxfheaders_test.o pxfheaders_test.c
pxfheaders_test.c:124:1: error: conflicting types for 'expect_churl_headers'
expect_churl_headers(const char *key, const char *value)
^
pxfheaders_test.c:40:2: note: previous implicit declaration is here
expect_churl_headers("X-GP-SEGMENT-ID", mock_extvar->GP_SEGMENT_ID);
^
pxfheaders_test.c:139:1: error: conflicting types for 
'expect_churl_headers_alignment'
expect_churl_headers_alignment()
^
pxfheaders_test.c:43:2: note: previous implicit declaration is here
expect_churl_headers_alignment();
^
pxfheaders_test.c:159:1: error: conflicting types for 'store_gucs'
store_gucs()
^
pxfheaders_test.c:152:2: note: previous implicit declaration is here
store_gucs();
^
pxfheaders_test.c:166:1: error: conflicting types for 'setup_gphd_uri'
setup_gphd_uri()
^
pxfheaders_test.c:153:2: note: previous implicit declaration is here
setup_gphd_uri();
^
pxfheaders_test.c:176:1: error: conflicting types for 'setup_input_data'
setup_input_data()
^
pxfheaders_test.c:154:2: note: previous implicit declaration is here
setup_input_data();
^
pxfheaders_test.c:184:1: error: conflicting types for 'setup_external_vars'
setup_external_vars()
^
pxfheaders_test.c:155:2: note: previous implicit declaration is here
setup_external_vars();
^
pxfheaders_test.c:193:6: error: conflicting types for 'expect_external_vars'
void expect_external_vars()
 ^
pxfheaders_test.c:38:2: note: previous implicit declaration is here
expect_external_vars();
^
pxfheaders_test.c:220:6: error: conflicting types for 'restore_gucs'
void restore_gucs()
 ^
pxfheaders_test.c:217:2: note: previous implicit declaration is here
restore_gucs();
^
8 errors generated.
make[3]: *** [pxfheaders_test.o] Error 1
make[2]: *** [unittest-check] Error 2
make[1]: *** [unittest-check] Error 2
make: *** [unittest-check] Error 2
{code}
...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HAWQ-1071) add PXF HiveText and HiveRC profile examples to the documentation

2016-09-27 Thread Oleksandr Diachenko (JIRA)


[ 
https://issues.apache.org/jira/browse/HAWQ-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527463#comment-15527463
 ] 

Oleksandr Diachenko edited comment on HAWQ-1071 at 9/27/16 9:41 PM:


Data types tested and supported by HiveText profile:
||Hive datatype||HAWQ datatype||
|text|string|
|integer/int4|int|
|double precision/float8|double|

Data types tested and supported by HiveRC profile:
||Hive datatype||HAWQ datatype||
|text|string|
|integer/int4|int|
|double precision/float8|double|
|timestamp without time zone|timestamp| 
|real/float4|float|
|bigint/int8|bigint|
|bool|boolean|
|smallint/int2|smallint|
|bytea|binary|



was (Author: odiachenko):
Data types tested and supported by HiveText profile:
||Hive datatype||HAWQ datatype||
|text|string|
|integer/int4|int|
|double precision/float8|double|

> add PXF HiveText and HiveRC profile examples to the documentation
> -
>
> Key: HAWQ-1071
> URL: https://issues.apache.org/jira/browse/HAWQ-1071
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lisa Owen
>Assignee: David Yozie
>Priority: Minor
> Fix For: 2.0.1.0-incubating
>
>
> the current PXF Hive documentation includes an example for only the Hive 
> profile.  add examples for HiveText and HiveRC profiles.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1071) add PXF HiveText and HiveRC profile examples to the documentation

2016-09-27 Thread Oleksandr Diachenko (JIRA)


[ 
https://issues.apache.org/jira/browse/HAWQ-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527463#comment-15527463
 ] 

Oleksandr Diachenko commented on HAWQ-1071:
---

Data types tested and supported by HiveText profile:
||Hive datatype||HAWQ datatype||
|text|string|
|integer/int4|int|
|double precision/float8|double|

> add PXF HiveText and HiveRC profile examples to the documentation
> -
>
> Key: HAWQ-1071
> URL: https://issues.apache.org/jira/browse/HAWQ-1071
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lisa Owen
>Assignee: David Yozie
>Priority: Minor
> Fix For: 2.0.1.0-incubating
>
>
> the current PXF Hive documentation includes an example for only the Hive 
> profile.  add examples for HiveText and HiveRC profiles.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1054) Real/float4 rounding issues for HiveORC profile

2016-09-27 Thread Oleksandr Diachenko (JIRA)


[ 
https://issues.apache.org/jira/browse/HAWQ-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527423#comment-15527423
 ] 

Oleksandr Diachenko commented on HAWQ-1054:
---

It seems to be postgresql behaviour - 
https://www.postgresql.org/message-id/200107241630.f6ogud128...@hub.org

> Real/float4 rounding issues for HiveORC profile
> ---
>
> Key: HAWQ-1054
> URL: https://issues.apache.org/jira/browse/HAWQ-1054
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: PXF
>Reporter: Oleksandr Diachenko
>Assignee: Oleksandr Diachenko
>Priority: Critical
> Fix For: 2.0.1.0-incubating
>
>
> Looks like real values are being incorrectly rounded:
> {code}
>  SELECT t1, r FROM pxf_hive_orc_types WHERE r = 7.7;
>  t1 | r 
> +---
> (0 rows)
> SELECT t1, r FROM pxf_hive_orc_types WHERE r > 7.6;
>   t1  |  r   
> --+--
>  row1 |  7.7
>  row2 |  8.7
>  row3 |  9.7
>  row4 | 10.7
>  row5 | 11.7
>  row6 | 12.7
>  row7 |  7.7
>  row8 |  7.7
>  row9 |  7.7
>  row10|  7.7
>  row11|  7.7
>  row12_text_null  |  7.7
>  row13_int_null   |  7.7
>  row14_double_null|  7.7
>  row15_decimal_null   |  7.7
>  row16_timestamp_null |  7.7
>  row18_bigint_null|  7.7
>  row19_bool_null  |  7.7
>  row20_tinyint_null   |  7.7
>  row21_smallint_null  |  7.7
>  row22_date_null  |  7.7
>  row23_varchar_null   |  7.7
>  row24_char_null  |  7.7
>  row25_binary_null|  7.7
> (24 rows)
> {code}
> The same query works fine in Hive:
> {code}
> hive> select f from hive_orc_all_types where f = 7.7;
> OK
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> Time taken: 0.032 seconds, Fetched: 19 row(s)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HAWQ-1054) Real/float4 rounding issues for HiveORC profile

2016-09-27 Thread Oleksandr Diachenko (JIRA)


[ 
https://issues.apache.org/jira/browse/HAWQ-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527330#comment-15527330
 ] 

Oleksandr Diachenko edited comment on HAWQ-1054 at 9/27/16 8:32 PM:


[~GodenYao] right, not very intuitive, that's why I was confused.
{code}
desc hive_orc_all_types;
OK
s1  string  
s2  string  
n1  int 
d1  double  
dc1 decimal(38,18)  
tm  timestamp   
f   float   
bg  bigint  
b   boolean 
tn  tinyint 
sml smallint
dt  date
vc1 varchar(5)  
c1  char(3) 
bin binary  
Time taken: 0.917 seconds, Fetched: 15 row(s)
{code}

{code}
\d pxf_hive_orc_types
External table "public.pxf_hive_orc_types"
 Column |Type | Modifiers 
+-+---
 t1 | text| 
 t2 | text| 
 num1   | integer | 
 dub1   | double precision| 
 dec1   | numeric | 
 tm | timestamp without time zone | 
 r  | real| 
 bg | bigint  | 
 b  | boolean | 
 tn | smallint| 
 sml| smallint| 
 dt | date| 
 vc1| character varying(5)| 
 c1 | character(3)| 
 bin| bytea   | 
Type: readable
Encoding: UTF8
Format type: custom
Format options: formatter 'pxfwritable_import' 
External location: 
pxf://localhost:51200/hive_orc_all_types?PROFILE=HiveORC=^A
{code}


Table definitions are correct, float column mapped to real.


was (Author: odiachenko):
[~GodenYao] right, not very intuitive, that's why I was confused.
{code}
desc hive_orc_all_types;
OK
s1  string  
s2  string  
n1  int 
d1  double  
dc1 decimal(38,18)  
tm  timestamp   
f   float   
bg  bigint  
b   boolean 
tn  tinyint 
sml smallint
dt  date
vc1 varchar(5)  
c1  char(3) 
bin binary  
Time taken: 0.917 seconds, Fetched: 15 row(s)
{code}

{code}
\d pxf_hive_orc_types
External table "public.pxf_hive_orc_types"
 Column |Type | Modifiers 
+-+---
 t1 | text| 
 t2 | text| 
 num1   | integer | 
 dub1   | double precision| 
 dec1   | numeric | 
 tm | timestamp without time zone | 
 r  | real| 
 bg | bigint  | 
 b  | boolean | 
 tn | smallint| 
 sml| smallint| 
 dt | date| 
 vc1| character varying(5)| 
 c1 | character(3)| 
 bin| bytea   | 
Type: readable
Encoding: UTF8
Format type: custom
Format options: formatter 'pxfwritable_import' 
External location: 
pxf://localhost:51200/hive_orc_all_types?PROFILE=HiveORC=^A
{code}

> Real/float4 rounding issues for HiveORC profile
> ---
>
> Key: HAWQ-1054
> URL: https://issues.apache.org/jira/browse/HAWQ-1054
> Project: Apache HAWQ
>  Issue Type: Bug
>

[jira] [Commented] (HAWQ-1054) Real/float4 rounding issues for HiveORC profile

2016-09-27 Thread Oleksandr Diachenko (JIRA)


[ 
https://issues.apache.org/jira/browse/HAWQ-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527330#comment-15527330
 ] 

Oleksandr Diachenko commented on HAWQ-1054:
---

[~GodenYao] right, not very intuitive, that's why I was confused.
{code}
desc hive_orc_all_types;
OK
s1  string  
s2  string  
n1  int 
d1  double  
dc1 decimal(38,18)  
tm  timestamp   
f   float   
bg  bigint  
b   boolean 
tn  tinyint 
sml smallint
dt  date
vc1 varchar(5)  
c1  char(3) 
bin binary  
Time taken: 0.917 seconds, Fetched: 15 row(s)
{code}

{code}
\d pxf_hive_orc_types
External table "public.pxf_hive_orc_types"
 Column |Type | Modifiers 
+-+---
 t1 | text| 
 t2 | text| 
 num1   | integer | 
 dub1   | double precision| 
 dec1   | numeric | 
 tm | timestamp without time zone | 
 r  | real| 
 bg | bigint  | 
 b  | boolean | 
 tn | smallint| 
 sml| smallint| 
 dt | date| 
 vc1| character varying(5)| 
 c1 | character(3)| 
 bin| bytea   | 
Type: readable
Encoding: UTF8
Format type: custom
Format options: formatter 'pxfwritable_import' 
External location: 
pxf://localhost:51200/hive_orc_all_types?PROFILE=HiveORC=^A
{code}

> Real/float4 rounding issues for HiveORC profile
> ---
>
> Key: HAWQ-1054
> URL: https://issues.apache.org/jira/browse/HAWQ-1054
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: PXF
>Reporter: Oleksandr Diachenko
>Assignee: Oleksandr Diachenko
>Priority: Critical
> Fix For: 2.0.1.0-incubating
>
>
> Looks like real values are being incorrectly rounded:
> {code}
>  SELECT t1, r FROM pxf_hive_orc_types WHERE r = 7.7;
>  t1 | r 
> +---
> (0 rows)
> SELECT t1, r FROM pxf_hive_orc_types WHERE r > 7.6;
>   t1  |  r   
> --+--
>  row1 |  7.7
>  row2 |  8.7
>  row3 |  9.7
>  row4 | 10.7
>  row5 | 11.7
>  row6 | 12.7
>  row7 |  7.7
>  row8 |  7.7
>  row9 |  7.7
>  row10|  7.7
>  row11|  7.7
>  row12_text_null  |  7.7
>  row13_int_null   |  7.7
>  row14_double_null|  7.7
>  row15_decimal_null   |  7.7
>  row16_timestamp_null |  7.7
>  row18_bigint_null|  7.7
>  row19_bool_null  |  7.7
>  row20_tinyint_null   |  7.7
>  row21_smallint_null  |  7.7
>  row22_date_null  |  7.7
>  row23_varchar_null   |  7.7
>  row24_char_null  |  7.7
>  row25_binary_null|  7.7
> (24 rows)
> {code}
> The same query works fine in Hive:
> {code}
> hive> select f from hive_orc_all_types where f = 7.7;
> OK
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> Time taken: 0.032 seconds, Fetched: 19 row(s)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1054) Real/float4 rounding issues for HiveORC profile

2016-09-27 Thread Goden Yao (JIRA)


[ 
https://issues.apache.org/jira/browse/HAWQ-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527236#comment-15527236
 ] 

Goden Yao commented on HAWQ-1054:
-

this is a little bit counter intuitive.
What's table definition in pxf external table vs. Hive table? Any difference?

> Real/float4 rounding issues for HiveORC profile
> ---
>
> Key: HAWQ-1054
> URL: https://issues.apache.org/jira/browse/HAWQ-1054
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: PXF
>Reporter: Oleksandr Diachenko
>Assignee: Oleksandr Diachenko
>Priority: Critical
> Fix For: 2.0.1.0-incubating
>
>
> Looks like real values are being incorrectly rounded:
> {code}
>  SELECT t1, r FROM pxf_hive_orc_types WHERE r = 7.7;
>  t1 | r 
> +---
> (0 rows)
> SELECT t1, r FROM pxf_hive_orc_types WHERE r > 7.6;
>   t1  |  r   
> --+--
>  row1 |  7.7
>  row2 |  8.7
>  row3 |  9.7
>  row4 | 10.7
>  row5 | 11.7
>  row6 | 12.7
>  row7 |  7.7
>  row8 |  7.7
>  row9 |  7.7
>  row10|  7.7
>  row11|  7.7
>  row12_text_null  |  7.7
>  row13_int_null   |  7.7
>  row14_double_null|  7.7
>  row15_decimal_null   |  7.7
>  row16_timestamp_null |  7.7
>  row18_bigint_null|  7.7
>  row19_bool_null  |  7.7
>  row20_tinyint_null   |  7.7
>  row21_smallint_null  |  7.7
>  row22_date_null  |  7.7
>  row23_varchar_null   |  7.7
>  row24_char_null  |  7.7
>  row25_binary_null|  7.7
> (24 rows)
> {code}
> The same query works fine in Hive:
> {code}
> hive> select f from hive_orc_all_types where f = 7.7;
> OK
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> Time taken: 0.032 seconds, Fetched: 19 row(s)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1054) Real/float4 rounding issues for HiveORC profile

2016-09-27 Thread Oleksandr Diachenko (JIRA)


[ 
https://issues.apache.org/jira/browse/HAWQ-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527171#comment-15527171
 ] 

Oleksandr Diachenko commented on HAWQ-1054:
---

Issue caused by incorrect type of literal:
{code}
# SELECT t1, r, pg_typeof(7.7) FROM pxf_hive_orc_types LIMIT 1;
  t1  |  r  | pg_typeof 
--+-+---
 row1 | 7.7 | numeric
(1 row)

{code}

> Real/float4 rounding issues for HiveORC profile
> ---
>
> Key: HAWQ-1054
> URL: https://issues.apache.org/jira/browse/HAWQ-1054
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: PXF
>Reporter: Oleksandr Diachenko
>Assignee: Oleksandr Diachenko
>Priority: Critical
> Fix For: 2.0.1.0-incubating
>
>
> Looks like real values are being incorrectly rounded:
> {code}
>  SELECT t1, r FROM pxf_hive_orc_types WHERE r = 7.7;
>  t1 | r 
> +---
> (0 rows)
> SELECT t1, r FROM pxf_hive_orc_types WHERE r > 7.6;
>   t1  |  r   
> --+--
>  row1 |  7.7
>  row2 |  8.7
>  row3 |  9.7
>  row4 | 10.7
>  row5 | 11.7
>  row6 | 12.7
>  row7 |  7.7
>  row8 |  7.7
>  row9 |  7.7
>  row10|  7.7
>  row11|  7.7
>  row12_text_null  |  7.7
>  row13_int_null   |  7.7
>  row14_double_null|  7.7
>  row15_decimal_null   |  7.7
>  row16_timestamp_null |  7.7
>  row18_bigint_null|  7.7
>  row19_bool_null  |  7.7
>  row20_tinyint_null   |  7.7
>  row21_smallint_null  |  7.7
>  row22_date_null  |  7.7
>  row23_varchar_null   |  7.7
>  row24_char_null  |  7.7
>  row25_binary_null|  7.7
> (24 rows)
> {code}
> The same query works fine in Hive:
> {code}
> hive> select f from hive_orc_all_types where f = 7.7;
> OK
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> 7.7
> Time taken: 0.032 seconds, Fetched: 19 row(s)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #941: HAWQ-1079. Fixed issue with symlink.

2016-09-27 Thread mithmatt

Github user mithmatt commented on the issue:

https://github.com/apache/incubator-hawq/pull/941
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #941: HAWQ-1079. Fixed issue with symlink.

2016-09-27 Thread ljainpivotalio

Github user ljainpivotalio commented on the issue:

https://github.com/apache/incubator-hawq/pull/941
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #939: HAWQ-991. Fix bug of yaml configuration fi...

2016-09-27 Thread kdunn-pivotal

Github user kdunn-pivotal commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/939#discussion_r80729846
  
--- Diff: tools/bin/hawqregister ---
@@ -729,6 +728,23 @@ class HawqRegister(object):
 for k, eof in enumerate(eofs[1:]):
 query += ',(%d, %d, %d, %d, %d)' % (self.firstsegno + 
k + 1, eof, -1, -1, -1)
--- End diff --

Yes, upgrade to 3.x is a big deal - I'm definitely *not* proposing that. 
Rather, we should minimize the technical debt by making net-new code (like this 
tool) better aligned with 3.x conventions, to avoid revisiting it later on.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #943: HAWQ-1061. Check folder and eof not confli...

2016-09-27 Thread xunzhang

GitHub user xunzhang opened a pull request:

https://github.com/apache/incubator-hawq/pull/943

HAWQ-1061. Check folder and eof not conflict under usage1.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xunzhang/incubator-hawq HAWQ-1061

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/943.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #943


commit 05e4636652a0cf0d3df0e28b2b0a12f503cdc8f0
Author: xunzhang 
Date:   2016-09-27T08:34:54Z

HAWQ-1061. Check folder and eof not conflict under usage1.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #943: HAWQ-1061. Check folder and eof not conflict unde...

2016-09-27 Thread xunzhang

Github user xunzhang commented on the issue:

https://github.com/apache/incubator-hawq/pull/943
  
cc @ictmalili @linwen 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #942: HAWQ-991. Bugfix for hawq register under -...

2016-09-27 Thread xunzhang

Github user xunzhang closed the pull request at:

https://github.com/apache/incubator-hawq/pull/942


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #942: HAWQ-991. Bugfix for hawq register under --force ...

2016-09-27 Thread xunzhang

Github user xunzhang commented on the issue:

https://github.com/apache/incubator-hawq/pull/942
  
Merged into master, thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #942: HAWQ-991. Bugfix for hawq register under --force ...

2016-09-27 Thread liming01

Github user liming01 commented on the issue:

https://github.com/apache/incubator-hawq/pull/942
  
+1.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #942: HAWQ-991. Bugfix for hawq register under --force ...

2016-09-27 Thread linwen

Github user linwen commented on the issue:

https://github.com/apache/incubator-hawq/pull/942
  
+1 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #942: HAWQ-991. Bugfix for hawq register under --force ...

2016-09-27 Thread xunzhang

Github user xunzhang commented on the issue:

https://github.com/apache/incubator-hawq/pull/942
  
cc @linwen @liming01 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #942: HAWQ-991. Bugfix for hawq register under -...

2016-09-27 Thread xunzhang

GitHub user xunzhang opened a pull request:

https://github.com/apache/incubator-hawq/pull/942

HAWQ-991. Bugfix for hawq register under --force mode.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xunzhang/incubator-hawq HAWQ-991

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/942.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #942


commit 60b26eddf5c496387b6af32b0ce2024f0e8fe95c
Author: xunzhang 
Date:   2016-09-27T07:34:36Z

HAWQ-991. Bugfix for hawq register under --force mode.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #944: HAWQ-991. Add try catch for parsing the ya...

[jira] [Updated] (HAWQ-1080) make unittest-check fails on local dev environment

[jira] [Updated] (HAWQ-1080) make unittest-check fails on local dev environment

[jira] [Resolved] (HAWQ-1061) Improve hawq register for already bugs found

[GitHub] incubator-hawq issue #943: HAWQ-1061. Check folder and eof not conflict unde...

[GitHub] incubator-hawq pull request #943: HAWQ-1061. Check folder and eof not confli...

[GitHub] incubator-hawq pull request #944: HAWQ-991. Add try catch for parsing the ya...

[jira] [Resolved] (HAWQ-1077) Table insert hangs due to stack overwrite which is caused by a bug in ao snappy code

[jira] [Closed] (HAWQ-1077) Table insert hangs due to stack overwrite which is caused by a bug in ao snappy code

[GitHub] incubator-hawq issue #943: HAWQ-1061. Check folder and eof not conflict unde...

[GitHub] incubator-hawq pull request #937: HAWQ-1077. Query hangs due to stack overwr...

[jira] [Created] (HAWQ-1080) make unittest-check fails on local dev environment

[jira] [Comment Edited] (HAWQ-1071) add PXF HiveText and HiveRC profile examples to the documentation

[jira] [Commented] (HAWQ-1071) add PXF HiveText and HiveRC profile examples to the documentation

[jira] [Commented] (HAWQ-1054) Real/float4 rounding issues for HiveORC profile

[jira] [Comment Edited] (HAWQ-1054) Real/float4 rounding issues for HiveORC profile

[jira] [Commented] (HAWQ-1054) Real/float4 rounding issues for HiveORC profile

[jira] [Commented] (HAWQ-1054) Real/float4 rounding issues for HiveORC profile

[jira] [Commented] (HAWQ-1054) Real/float4 rounding issues for HiveORC profile

[GitHub] incubator-hawq issue #941: HAWQ-1079. Fixed issue with symlink.

[GitHub] incubator-hawq issue #941: HAWQ-1079. Fixed issue with symlink.

[GitHub] incubator-hawq pull request #939: HAWQ-991. Fix bug of yaml configuration fi...

[GitHub] incubator-hawq pull request #943: HAWQ-1061. Check folder and eof not confli...

[GitHub] incubator-hawq issue #943: HAWQ-1061. Check folder and eof not conflict unde...

[GitHub] incubator-hawq pull request #942: HAWQ-991. Bugfix for hawq register under -...

[GitHub] incubator-hawq issue #942: HAWQ-991. Bugfix for hawq register under --force ...

[GitHub] incubator-hawq issue #942: HAWQ-991. Bugfix for hawq register under --force ...

[GitHub] incubator-hawq issue #942: HAWQ-991. Bugfix for hawq register under --force ...

[GitHub] incubator-hawq issue #942: HAWQ-991. Bugfix for hawq register under --force ...

[GitHub] incubator-hawq pull request #942: HAWQ-991. Bugfix for hawq register under -...

30 matches

Site Navigation

Mail list logo

Footer information